From ee46cb49e8aa8f9a1bde8f0df3c2ec8cb59d70b0 Mon Sep 17 00:00:00 2001 From: Matthew Meszaros Date: Fri, 18 Sep 2026 22:06:09 -0700 Subject: [PATCH 1/2] feat: route new Gmail and Google Workspace mailboxes through a guided three-step app-password connect over smtp.gmail.com and imap.gmail.com instead of Google sign-in, behind BOX_GOOGLE_OAUTH_CONNECT (off by default) and announced to clients as gmail_oauth_connect on /auth/config, refusing a new gmail OAuth start with 403 mailbox_gmail_oauth_disabled in both the direct and Warmbly Cloud broker paths while leaving mailboxes already connected that way sending, syncing and re-authorizable --- docs/content/docs/api/endpoints.mdx | 2 + docs/content/docs/api/error-codes.mdx | 17 + docs/content/docs/api/reference/mailboxes.mdx | 2 +- .../docs/development/configuration.mdx | 3 +- .../docs/development/deployment-guide.mdx | 5 +- docs/content/docs/guides/mailboxes.mdx | 29 +- internal/api/handler/auth_config.go | 6 + internal/app/email/onboarding.go | 4 + .../app/email/onboarding_gmail_gate_test.go | 53 +++ internal/app/instanceconfig/entries.go | 6 + internal/app/poollink/oauth.go | 5 + internal/config/inbox.go | 12 + internal/errx/common.go | 6 + site/src/pages/sending.astro | 6 +- .../app/emails/GmailAppPasswordPanel.tsx | 392 ++++++++++++++++++ .../app/emails/GoogleOAuthNotReadyDialog.tsx | 245 ----------- .../components/app/modals/AddEmailModal.tsx | 158 +++---- web/src/lib/api/hooks/auth/useAuthConfig.ts | 3 + web/src/lib/api/models/auth/AuthConfig.ts | 5 + 19 files changed, 595 insertions(+), 364 deletions(-) create mode 100644 internal/app/email/onboarding_gmail_gate_test.go create mode 100644 web/src/components/app/emails/GmailAppPasswordPanel.tsx delete mode 100644 web/src/components/app/emails/GoogleOAuthNotReadyDialog.tsx diff --git a/docs/content/docs/api/endpoints.mdx b/docs/content/docs/api/endpoints.mdx index 0bfb54f72..2a2ed29c5 100644 --- a/docs/content/docs/api/endpoints.mdx +++ b/docs/content/docs/api/endpoints.mdx @@ -399,6 +399,8 @@ These never accept an API key. They depend on a human-bound session: billing flo Alongside them, `api_url` is this API's own public base (for a copyable example that names the right server), and `brand` is who the deployment says it is: `name`, and `website_url`, `website_label`, `terms_url`, `privacy_url` and `support_email`, each omitted when unset. On a self-hosted instance that configured no `EMAIL_BRAND_*` only `name` is present, and a client should render no link at all rather than substituting one of its own. See [configuration](/development/configuration/). +`GET /auth/config` also carries `gmail_oauth_connect` (boolean): whether a new Gmail mailbox may be connected with Google sign-in. It is `false` unless the deployment sets `BOX_GOOGLE_OAUTH_CONNECT=true`, and a client should then offer the app-password route (`POST /emails/onboarding/smtp-imap` against Gmail's servers) rather than start an OAuth round trip that returns `mailbox_gmail_oauth_disabled`. Mailboxes already connected with Google sign-in are unaffected either way. + `GET /auth/config` also carries `billing_enabled` (boolean). It is `false` when the deployment runs with `BILLING_PROVIDER=none`, which is the self-host default: every feature is unlocked server-side, so the dashboard shows the workspace as self-hosted instead of on a free trial and hides the billing and referral pages. `self_hosted` alone does not imply this, because a self-hosted install may still run Stripe. - `POST /auth/setup` (first-run claim: exchanges the one-time token printed at boot for the owner account. Refused once any account exists) - `GET /auth/providers`, `POST /auth/apple`, `POST /auth/google` (native-app social sign-in) diff --git a/docs/content/docs/api/error-codes.mdx b/docs/content/docs/api/error-codes.mdx index 08bc2b998..d8f85330b 100644 --- a/docs/content/docs/api/error-codes.mdx +++ b/docs/content/docs/api/error-codes.mdx @@ -167,6 +167,23 @@ Returned when authenticated but lacking necessary permissions. - Verify IP restrictions if configured - Request additional permissions if needed +#### `mailbox_gmail_oauth_disabled` + +A `403` whose `code` is `mailbox_gmail_oauth_disabled` comes from `POST /emails/onboarding/oauth/start` with `provider: "gmail"`. The deployment routes new Gmail mailboxes through an app password over IMAP and SMTP instead of Google sign-in, which is the default; `GET /auth/config` announces it as `gmail_oauth_connect: false`. Nothing about the caller's permissions is wrong. Re-authorizing an existing Gmail mailbox (`POST /emails/onboarding/oauth/reauth/:id`) is never refused this way. + +```json +{ + "error": "Forbidden", + "message": "New Gmail mailboxes connect with an app password over IMAP and SMTP on this deployment, not with Google sign-in.", + "code": "mailbox_gmail_oauth_disabled", + "request_id": "4bbbd1b2-8f86-47dd-8a7f-9476501ad20e" +} +``` + +**How to fix:** +- Connect the mailbox through `POST /emails/onboarding/smtp-imap` with `smtp.gmail.com:465` and `imap.gmail.com:993`, both TLS, and a Google app password. See [Gmail and Google Workspace](/guides/mailboxes/#gmail-and-google-workspace) +- A self-hosted instance with its own Google app can set `BOX_GOOGLE_OAUTH_CONNECT=true` to allow Google sign-in for new mailboxes + #### Registration and invitation refusals Signup and invitation refusals carry their own `code`, so a client can branch on the specific condition instead of matching on text. They describe the deployment's policy and never say anything about whether a given address exists. diff --git a/docs/content/docs/api/reference/mailboxes.mdx b/docs/content/docs/api/reference/mailboxes.mdx index 82c130442..ccce97c42 100644 --- a/docs/content/docs/api/reference/mailboxes.mdx +++ b/docs/content/docs/api/reference/mailboxes.mdx @@ -722,7 +722,7 @@ Begins an OAuth round trip for a Gmail or Outlook mailbox and returns the provid | Field | Type | Required | Description | |-------|------|----------|-------------| -| `provider` | string | yes | `gmail` or `outlook`. | +| `provider` | string | yes | `gmail` or `outlook`. `gmail` is refused with `403` `mailbox_gmail_oauth_disabled` unless `GET /auth/config` reports `gmail_oauth_connect: true`; new Gmail mailboxes go through [Connect SMTP/IMAP](#connect-smtpimap) with an app password instead. Re-authorizing an existing Gmail mailbox is not gated. | ```json { diff --git a/docs/content/docs/development/configuration.mdx b/docs/content/docs/development/configuration.mdx index 8b0a22075..8bb574a6c 100644 --- a/docs/content/docs/development/configuration.mdx +++ b/docs/content/docs/development/configuration.mdx @@ -432,7 +432,8 @@ Needed on the backend **and** every worker: the backend starts the OAuth flow, a | Variable | What it does | Default | |---|---|---| -| `BOX_GOOGLE_CLIENT_ID`, `BOX_GOOGLE_CLIENT_SECRET` | Connect Gmail and Google Workspace mailboxes. Redirect URI is your API base plus `/addresses/google/callback` | unset | +| `BOX_GOOGLE_CLIENT_ID`, `BOX_GOOGLE_CLIENT_SECRET` | The Google app used by Gmail mailboxes connected with Google sign-in: the ones already connected that way, and new ones once `BOX_GOOGLE_OAUTH_CONNECT` is on. Redirect URI is your API base plus `/addresses/google/callback` | unset | +| `BOX_GOOGLE_OAUTH_CONNECT` | `true` lets a new Gmail mailbox connect with Google sign-in. Off, the connect dialog walks through an app password over IMAP and SMTP instead, which needs no Google app; mailboxes already on Google sign-in keep working and can be re-authorized either way. Announced to clients as `gmail_oauth_connect` on `GET /auth/config` | `false` | | `BOX_OUTLOOK_CLIENT_ID`, `BOX_OUTLOOK_CLIENT_SECRET` | Connect Outlook and Microsoft 365 mailboxes. Redirect URI is your API base plus `/addresses/outlook/callback` | unset | Plain SMTP and IMAP mailboxes need none of this. If a worker is missing these values, the mailbox connects fine and then silently stops about an hour later, when its first access token expires. diff --git a/docs/content/docs/development/deployment-guide.mdx b/docs/content/docs/development/deployment-guide.mdx index c95a74fde..e7a5b465f 100644 --- a/docs/content/docs/development/deployment-guide.mdx +++ b/docs/content/docs/development/deployment-guide.mdx @@ -449,7 +449,7 @@ There are three ways to attach a sending mailbox. Only the OAuth ones need setup | Mailbox type | Setup needed | Variables | |--------------|--------------|-----------| | Any SMTP + IMAP provider | None | none | -| Gmail / Google Workspace | A Google Cloud OAuth client | `BOX_GOOGLE_CLIENT_ID`, `BOX_GOOGLE_CLIENT_SECRET` | +| Gmail / Google Workspace | None with an app password (the default). A Google Cloud OAuth client to offer Google sign-in | `BOX_GOOGLE_OAUTH_CONNECT`, `BOX_GOOGLE_CLIENT_ID`, `BOX_GOOGLE_CLIENT_SECRET` | | Outlook / Microsoft 365 | An Entra ID app registration | `BOX_OUTLOOK_CLIENT_ID`, `BOX_OUTLOOK_CLIENT_SECRET` | @@ -462,6 +462,8 @@ Nothing to configure. Add the mailbox in the dashboard with its host, port, user ### Gmail and Google Workspace +Nothing to configure by default: the connect dialog walks users through a Google app password over `smtp.gmail.com` and `imap.gmail.com`, see [the mailbox guide](/guides/mailboxes/#gmail-and-google-workspace). Set up a Google OAuth client only if you want to offer Google sign-in for new Gmail mailboxes, which also needs `BOX_GOOGLE_OAUTH_CONNECT=true`; without it the client is used solely to keep mailboxes that are already on Google sign-in refreshing. + @@ -490,6 +492,7 @@ Put the client id and secret in your root `.env`: ```bash BOX_GOOGLE_CLIENT_ID=1234567890-abc123.apps.googleusercontent.com BOX_GOOGLE_CLIENT_SECRET=GOCSPX-your-secret-here +BOX_GOOGLE_OAUTH_CONNECT=true ``` Then `make up` to recreate the containers with the new values. diff --git a/docs/content/docs/guides/mailboxes.mdx b/docs/content/docs/guides/mailboxes.mdx index ecf5b6ab2..73b331670 100644 --- a/docs/content/docs/guides/mailboxes.mdx +++ b/docs/content/docs/guides/mailboxes.mdx @@ -11,12 +11,12 @@ Open **Accounts** and choose **Add account**. | Provider | Method | Notes | |----------|--------|-------| -| Gmail / Google Workspace | OAuth (`gmail`) | Not recommended yet, see below. Connect Gmail over IMAP + SMTP with an app password for now | +| Gmail / Google Workspace | App password over IMAP + SMTP (`smtp_imap`) | The connect dialog walks you through it in three steps, see [Gmail and Google Workspace](#gmail-and-google-workspace) below. Google sign-in for new mailboxes is coming soon | | Outlook / Microsoft 365 | OAuth (`outlook`) | Recommended. No password stored. Runs on Microsoft Graph | | Any other server | IMAP + SMTP (`smtp_imap`) | Custom domains, self-hosted, or providers without OAuth. Any IMAP server works, including Outlook.com, Microsoft 365 over IMAP, Yahoo, Fastmail, Zoho, cPanel and self-hosted Dovecot | - - Warmbly's Google app is still going through review for the Gmail access it needs, so a mailbox connected with Google sign-in can fail to send or lose its authorization without warning. Google also caps how many accounts an app in review may hold, so a mailbox connected now may be disconnected later if there is no capacity left, and you would have to connect it again. The connect dialog marks the Gmail row in red and explains this. Connect Gmail and Google Workspace mailboxes over IMAP and SMTP with an app password instead; see [Gmail over IMAP and SMTP](#gmail-over-imap-and-smtp) below. Nothing has to change once Google sign-in is ready. + + For now, new Gmail and Google Workspace mailboxes connect with an app password; the connect dialog says so and walks you through it. A mailbox you already connected with Google sign-in is not affected: it keeps sending and syncing as before, and if Google ever invalidates its token the **Re-authorize** button in its drawer still works. Nothing has to be reconnected. A self-hosted instance with its own Google app can turn sign-in back on for new mailboxes with `BOX_GOOGLE_OAUTH_CONNECT=true`, see [configuration](/development/configuration/#mailbox-connections). **OAuth** sends you to your provider's consent screen and returns a token instead of a password. Both OAuth providers use the provider's native API, never IMAP or SMTP, so consent asks to send mail and to read and organize your mailbox. Google additionally asks to read your mail settings, which is what lets Warmbly offer the addresses Google has verified you to send as and import the signature you already wrote there. It needs no app passwords or server settings. Note that Google revokes Gmail tokens when the account's password changes, so a password change there means [re-authorizing the mailbox](#reconnecting-an-account) once. @@ -52,23 +52,24 @@ With two-factor authentication on, generate an app password in your provider's s Warmbly signs in with whichever method your server offers, preferring CRAM-MD5, then LOGIN, then PLAIN. Servers that accept only one of these, which includes Microsoft 365 relays and most appliance relays, work without any setting to change. -### Gmail over IMAP and SMTP +### Gmail and Google Workspace -Until Google sign-in is recommended, this is how to connect a Gmail or Google Workspace mailbox: +Choose **Add account**, then **Gmail / Google Workspace**. The dialog walks through three steps, and the only things you type are the address and the app password; the server settings are Gmail's own and are filled in for you. -1. Turn on 2-Step Verification on the Google account, under [Google Account, Security](https://myaccount.google.com/security). Google only offers app passwords once it is on, and only alongside a second-step method other than a security key. Add an authenticator app if security keys are the only method on the account. -2. Create an app password at [myaccount.google.com/apppasswords](https://myaccount.google.com/apppasswords), name it Warmbly, and copy the 16 characters. Google shows them once. -3. There is nothing to turn on for IMAP. Google removed the IMAP setting in January 2025 and personal Gmail accounts have it on permanently. On Google Workspace it is the administrator's call: they can switch IMAP off for the organization under **Apps**, **Google Workspace**, **Gmail**, **End user access**, and can block app passwords as well. -4. In Warmbly choose **Add account**, then **Other (SMTP / IMAP)**, and fill in: +1. **Turn on 2-Step Verification** on the Google account, under [Google Account, Security](https://myaccount.google.com/security). Google only offers app passwords once it is on, and only alongside a second-step method other than a security key (a phone prompt, an authenticator app or a text message). Already on? Continue. +2. **Create an app password** at [myaccount.google.com/apppasswords](https://myaccount.google.com/apppasswords): name it Warmbly, press **Create**, and copy the 16 characters. Google shows them once; the spaces between the groups do not matter, the dialog drops them. +3. **Connect**: enter your name, the full address (including `@gmail.com` or your Workspace domain) and the app password. Warmbly checks it against Google before saving anything. + +The settings the dialog uses, in case you connect the same mailbox through **Other (SMTP / IMAP)** or the [CSV import](#connecting-many-mailboxes-at-once): ```text SMTP host: smtp.gmail.com port: 465 security: SSL / TLS IMAP host: imap.gmail.com port: 993 security: SSL / TLS ``` -The username is the full address, including `@gmail.com` or your Workspace domain. The password is the app password, never the account password. Deleting the app password in the Google account disconnects the mailbox, which is the same revocation an OAuth token gives you. +The username is the full address on both. The password is the app password, never the account password. There is nothing to turn on for IMAP: Google removed that setting in January 2025 and personal Gmail accounts have it on permanently. Deleting the app password in the Google account disconnects the mailbox, which is the same revocation an OAuth token gives you. -Some accounts cannot have an app password at all: enrolling in Google's [Advanced Protection Program](https://landing.google.com/advancedprotection/) revokes them and hides the page, and a Workspace administrator can block them for the organization. Ask the administrator to allow app passwords, or connect the mailbox with Google sign-in and accept the limits in the warning above. +On Google Workspace the administrator decides. They can switch IMAP off for the organization under **Apps**, **Google Workspace**, **Gmail**, **End user access**, and can block app passwords under **Security**, **Less secure apps**. Some accounts cannot have an app password at all: enrolling in Google's [Advanced Protection Program](https://landing.google.com/advancedprotection/) revokes them and hides the page. Ask the administrator to allow app passwords for the mailboxes you send from. Credentials and both connections are validated when you add the account, so wrong settings fail immediately rather than silently at send time. Tokens and credentials are sealed with envelope encryption before they touch storage. @@ -116,7 +117,7 @@ There is no daily cap on how many mailboxes you connect: a Business workspace ca ## Connecting many mailboxes at once -Pick **Bulk import from CSV** in the connect dialog to connect any number of SMTP and IMAP mailboxes from one file. Gmail and Outlook mailboxes sign in one at a time, because each needs its own consent. +Pick **Bulk import from CSV** in the connect dialog to connect any number of SMTP and IMAP mailboxes from one file. Gmail mailboxes qualify, with the [settings above](#gmail-and-google-workspace) and an app password per row. Outlook mailboxes sign in one at a time, because each needs its own consent. **The file.** One mailbox per row. `email`, `smtp_host` and `imap_host` are required, plus a password; everything else has a default. @@ -144,7 +145,7 @@ The same endpoint is available to the API as `POST /emails/onboarding/smtp-imap/ When the provider stops accepting a mailbox's stored credential (a password change, a revoked grant, an expired app password), the mailbox is taken out of sending and syncing and its drawer shows the reason under **Needs attention**, with the fix right on the error: -- **Gmail and Outlook**: a **Re-authorize** button re-runs the provider consent in a popup, preselecting the mailbox's own address. The consent must be for that same address; signing in with a different account is refused instead of quietly connecting the wrong mailbox. +- **Gmail (connected with Google sign-in) and Outlook**: a **Re-authorize** button re-runs the provider consent in a popup, preselecting the mailbox's own address. This works for an existing Gmail mailbox even while Google sign-in is paused for new ones. The consent must be for that same address; signing in with a different account is refused instead of quietly connecting the wrong mailbox. - **SMTP / IMAP**: an **Update credentials** button opens a form for the new password (or new host and port). The replacement is validated against your server before it is saved, the same as at connect time. A successful reconnect stores the new credential, clears the authentication error, reactivates the mailbox on its existing worker, and it resumes syncing from where it stopped. Nothing else changes: settings, history, warmup progress, and campaign membership all stay. @@ -206,7 +207,7 @@ A signature carrying a `