1651 Commits

Author SHA1 Message Date
Matthew Meszaros 93e8451738 feat: organization data export and import for moving a workspace between instances (#132)
* feat: add the org_export_jobs and org_import_jobs tables plus the models behind them, so a whole organization can be written to a portable archive and read back on another instance, keeping the option columns typed (a text[] of data groups, an include_secrets boolean, a conflict_strategy check constraint) rather than a settings blob because the option set is small and fixed, and reserving jsonb only for the genuinely free-form parts that are read back for display alone (the source archive's manifest, per-table row counts, the import warning list), with partial indexes on the in-flight and expiring rows so the maintenance sweep stays cheap however much transfer history accumulates, an OrgDataGroup catalog that names the twelve slices of a workspace and carries the dependencies between them, and an org_archive audit entity so an export or import rides the existing audit spine into every teammate's dashboard

* feat: add the schema-generic repository behind workspace archives, which reads and writes tables by name rather than through typed structs because that is the only way an archive stays correct as the schema grows, moving rows as jsonb in both directions via to_jsonb on the way out and jsonb_populate_recordset on the way in so Postgres performs every type conversion and no hand-written Go column mapping can drift from arrays, jsonb, tsvector, inet or enums, lifting the pool's 60s statement_timeout inside the export transaction because a full inbox read legitimately runs longer than that, introspecting generated, identity and not-null columns plus primary keys and foreign keys from the catalog rather than trusting a compiled list, and treating identifier safety as structural: table names come from the compiled registry and column names are always intersected against the destination catalog before reaching a query, so nothing out of an uploaded archive is ever interpolated

* feat: add the workspace archive registry and on-disk format, covering all 110 organization-owned relations with their scope SQL, dependency order and per-table policy, plus 10 explicitly excluded ones each carrying the reason it must never travel (the KMS-wrapped org data key, in-flight OAuth handshakes, the websocket outbox, live sessions, a pending deletion that would otherwise schedule the destination workspace for destruction), naming the two key domains separately because Warmbly seals mailbox credentials under the instance CREDENTIALS_ENCRYPTION_KEY and everything else under the per-organization DEK and confusing them produces mailboxes that authenticate against nothing, defining the archive as a plain zip of newline-delimited JSON so an operator can unzip it and read the data in a text editor and so the manifest can be written last yet still be read first, and sealing archive secrets under an argon2id passphrase key with parameters deliberately heavier than the login hash since it is derived once per archive and guards every credential in the workspace against offline grinding

* feat: implement the workspace export and import engines, streaming rows straight through untouched for the tables that have neither secrets nor blobs so a million-row inbox export stays cheap and only decoding the rows that must change, opening every sealed value against whichever key domain wrote it and re-sealing it under the archive passphrase on the way out then against the destination's own keys on the way in, blanking a credential rather than sinking the whole export when one mailbox cannot be read and clearing the guard flag alongside it so no row is left claiming ciphertext it no longer holds, applying an import inside a single transaction because a half-applied workspace is far worse than a long-running one, rewriting the organization id and matching members to destination accounts by email with unresolvable people blanked where the column is nullable and redirected to the importer where it is not, and running transfers in the accepting process rather than through a queue for the one reason that matters: the passphrase is then never written down anywhere

* feat: make the per-organization DEK cache nil-safe in internal/app/cipher so a process built without Redis falls through to KMS on every call instead of dereferencing a nil cache handle, which is what lets warmblyctl run the workspace export and import commands at all: it deliberately attaches Redis as optional because the whole point of that CLI is working while the rest of the instance is down, and the decrypted-key cache was always an optimisation rather than a requirement

* feat: add the hourly workspace-archive maintenance job that deletes finished archives past their seven-day retention window, since each one is a complete copy of a workspace sitting in object storage and must not accumulate, and closes out any export or import whose process died mid-run, which is the necessary counterpart to executing transfers in the accepting process so the passphrase is never persisted: without this sweep a restart would leave a job reporting running forever

* feat: expose workspace export and import over the JWT-only organization routes and wire the service into the backend, gating every endpoint on workspace ownership through the existing requireOrgOwner check rather than a permission bit because an export with credentials is the single most sensitive artifact this product can produce and an import rewrites the workspace wholesale, so both belong at the same level as deleting it, spooling uploads to a temporary file since a zip needs random access and a length that a multi-gigabyte archive cannot supply from memory, handing that file's ownership to the background import so it outlives the request and is closed exactly when the job ends, streaming downloads with the archive's sha256 in a response header, and constructing the service with both key domains plus object storage so an archive can be opened, re-keyed and stored

* feat: add warmblyctl org list, export and import so a self-hoster can move a workspace from the box without a browser, running the same engine in-process against Postgres and adding no HTTP surface to a CLI whose entire trust model is container or host access, resolving --org from whichever handle the operator has (id, slug, or the owner's email), streaming the archive to a file or to stdout so it can be piped straight into ssh with progress still readable on stderr, prompting for the credential passphrase twice through the existing password prompt so the terminal and pipe rules stay identical across every command, and defaulting the import path to a preflight report that names what already exists here and which members have no account before anything is written, with --dry-run to stop there

* feat: add the dashboard API layer for workspace archives, fetching the data-group catalog from the server rather than restating it in the client so a new group appears the moment the backend knows about it, mirroring the server's group-dependency closure in expandGroups so the toggles a user sees always match what the archive actually gets, polling only while a transfer is in flight and dropping to no interval the moment none are active since a running job has no realtime event of its own, and downloading a finished archive as a blob through the authenticated client because the endpoint is bearer-authenticated and a plain anchor href cannot carry the token

* feat: build the Settings and Data dashboard page for exporting and importing a workspace, following the settings section conventions and the in-app confirm rather than window.confirm, defaulting the export to every data group because a migration that quietly leaves data behind is worse than one that takes a while, marking the heavy groups so nobody exports a decade of inbox history unaware, requiring the credential passphrase twice behind a confirm that states plainly what the file will contain, and making the import a two-step flow where a preflight reads the archive and reports its origin, row counts, unsealable credentials, existing rows and unknown members before a single byte is written, so confirming is never a leap of faith

* feat: register the Data settings section in the dashboard rail, route and realtime spine, placing it under Advanced beside the danger zone and gating it to the workspace owner so the nav matches what the endpoints actually allow, and mapping the new org_archive audit entity to the export and import query keys in useRealtimeEvents so an archive starting or landing refreshes the page for every teammate through the existing audit spine rather than a bespoke event

* feat: document workspace export and import as a customer guide registered under Account and team, covering what each of the twelve data groups contains and which four dominate archive size, why credentials need a passphrase to travel at all and what happens to mailboxes when they do not, how members are matched to destination accounts by email and what becomes of anyone without one, the difference between keeping existing rows and replacing them, and a table of what deliberately does not import with the reason for each, because billing, plan overrides, worker placement, sync checkpoints and warmup pool membership belong to an instance rather than to a workspace

* feat: document org list, export and import in the warmblyctl reference and point the deployment guide at them as the supported route between a self-hosted install and the hosted service in either direction, adding every flag with what it does, the two extra environment variables those commands read and the difference between them (a missing KMS provider stops the command because sealed values cannot be opened, while a missing CREDENTIALS_ENCRYPTION_KEY is only a warning that mailbox credentials will not move), the behaviour when Redis is down, and the warning that an archive carrying credentials is the most sensitive file this product produces

* feat: record in AGENTS.md that a migration adding an organization-scoped table is not finished until that table is registered in internal/app/orgtransfer/spec.go, either in Tables with its group and scope or in ExcludedTables with the reason it must not travel, because data left out of the registry is silently absent from every archive and nobody discovers it until a customer's migration lands on the other side missing a feature's data, and spelling out the four things that are easy to get wrong when adding one: dependency order, the group boundary that needs a Requires entry only when a NOT NULL foreign key crosses it, which of the two key domains seals a ciphertext column, and which columns name something only the source instance knows
2026-08-18 07:53:39 -07:00
Matthew Meszaros c39c29ab6b feat: rebuild the new-campaign wizard with animated step transitions, a numbered stepper, the shared Toggle instead of a broken hand-rolled switch, per-step validation that explains itself and a discard guard, register PopoverMenu's click-outside in the capture phase so dropdowns inside dialogs close on click-away, add a Campaigns back link and clickable breadcrumb crumbs, add a From contacts leads picker with category filter and select-all-matching backed by the bulk add_campaigns path whose SQL now scopes campaigns by organization instead of the caller, and stop self-hosted no-billing deployments presenting as a free trial or plan-metered by exposing billing_enabled on GET /auth/config, showing a Self-hosted badge, hiding Billing and Refer & earn, and reporting AI credits as unlimited with the header gauge and cost copy hidden 2026-08-18 07:48:58 -07:00
Matthew Meszaros 7521575cef feat: correct the documented order of the four blocks warmblyctl status prints, in both the new warmblyctl reference and the first-run sample output, after running the command against a live instance showed printStatus emits Instance, Platform admins and How to get in before runStatus calls printChecks, so the checks come last and not third as both pages claimed, which also means the first-run sample had its Checks and How to get in blocks transposed, and an operator scrolling for the recovery commands was being told to look past a findings list that is actually printed after them (#129) 2026-08-16 08:47:46 +02:00
Matthew Meszaros 619f4fd9a1 feat: add a warmblyctl reference page at docs/content/docs/development/warmblyctl.mdx documenting all nine commands with every flag, because the README and the recovery sections only ever showed 'warmblyctl user create --email ... --admin' without saying where the password comes from, leaving self-hosters with an account they could not sign in to and no page that answered it, covering that user create prompts for the password twice on a terminal and refuses on a non-TTY unless --password-stdin is passed, that docker compose exec allocates the TTY those prompts need unless -T is given and piped input needs -T precisely because it removes it, that the password rule is the dashboard's own 8 to 128 characters, and that signing in afterwards needs nothing else on a stock self-host since AUTH_LOGIN_CODE defaults to off and REQUIRE_EMAIL_VERIFICATION to false when self-hosted, captcha stays off without TURNSTILE_SECRET, and --admin opens the panel on ADMIN_URL rather than APP_URL, plus the status JSON contract and exit codes, the four environment variables each command reads, the per-command behaviour when Redis is down, the admin role masks read from AdminRolePermissions, and the make wrappers, registering it in the Development meta.json between accounts-and-access and configuration, linking it from the four pages that already print these commands, correcting the super-admin mask in the first-run sample output from 4294967295 to the 4194303 that AllAdminPermissions actually is since it is (1 << 22) - 1 and not the full uint32 range, and shortening the README self-hosting section by folding the three-bullet gotcha list into a four-row table that also names make doctor and dropping the duplicated make dev claim warning already stated above it, keeping every fact (#128) 2026-08-16 08:39:57 +02:00
Matthew Meszaros bd8545a1c5 feat: stop a freshly connected mailbox being silently excluded from every campaign send, by making an unset mailbox timezone representable as the empty string the campaign scheduler already checks for, since email_accounts.timezone defaulted to 'UTC' while campaigns.timezone defaults to 'Europe/London' and nothing in the OAuth or SMTP onboarding paths ever set either, so a brand new mailbox looked deliberately placed in UTC, was compared against the differing campaign zone and dropped by the hardcoded 8am-8pm business-hours gate whenever the current UTC hour fell outside it, emptying the candidate pool and failing the campaign start, adding a migration that changes the column default and converts existing 'UTC' rows because until now no API field, dashboard control or onboarding path could set that column at all so every such row is the old default rather than a choice, adding the missing Timezone field to UpdateEmail with IANA validation so the setting the sending-behaviour UI already tells people to change is finally reachable and an unloadable zone is rejected instead of being silently coerced to UTC by the scheduler, and replacing the misleading 'no active email accounts found for campaign's email tags' response for a pool that exists but is entirely gated out with a distinct message naming the real cause, via an ErrNoEligibleMailbox that wraps ErrNoEmailAccounts so the three callers that pause a campaign on it are unaffected (#126) (#125) (#127) 2026-08-16 07:58:52 +02:00
Matthew Meszaros f0846eb034 feat: stop a freshly connected mailbox being silently excluded from every campaign send, by making an unset mailbox timezone representable as the empty string the campaign scheduler already checks for, since email_accounts.timezone defaulted to 'UTC' while campaigns.timezone defaults to 'Europe/London' and nothing in the OAuth or SMTP onboarding paths ever set either, so a brand new mailbox looked deliberately placed in UTC, was compared against the differing campaign zone and dropped by the hardcoded 8am-8pm business-hours gate whenever the current UTC hour fell outside it, emptying the candidate pool and failing the campaign start, adding a migration that changes the column default and converts existing 'UTC' rows because until now no API field, dashboard control or onboarding path could set that column at all so every such row is the old default rather than a choice, adding the missing Timezone field to UpdateEmail with IANA validation so the setting the sending-behaviour UI already tells people to change is finally reachable and an unloadable zone is rejected instead of being silently coerced to UTC by the scheduler, and replacing the misleading 'no active email accounts found for campaign's email tags' response for a pool that exists but is entirely gated out with a distinct message naming the real cause, via an ErrNoEligibleMailbox that wraps ErrNoEmailAccounts so the three callers that pause a campaign on it are unaffected (#126) (#125) 2026-08-16 07:54:45 +02:00
Matthew Meszaros fe9a21a79f feat: stop a freshly connected mailbox being silently excluded from every campaign send, by making an unset mailbox timezone representable as the empty string the campaign scheduler already checks for, since email_accounts.timezone defaulted to 'UTC' while campaigns.timezone defaults to 'Europe/London' and nothing in the OAuth or SMTP onboarding paths ever set either, so a brand new mailbox looked deliberately placed in UTC, was compared against the differing campaign zone and dropped by the hardcoded 8am-8pm business-hours gate whenever the current UTC hour fell outside it, emptying the candidate pool and failing the campaign start, adding a migration that changes the column default and converts existing 'UTC' rows because until now no API field, dashboard control or onboarding path could set that column at all so every such row is the old default rather than a choice, adding the missing Timezone field to UpdateEmail with IANA validation so the setting the sending-behaviour UI already tells people to change is finally reachable and an unloadable zone is rejected instead of being silently coerced to UTC by the scheduler, and replacing the misleading 'no active email accounts found for campaign's email tags' response for a pool that exists but is entirely gated out with a distinct message naming the real cause, via an ErrNoEligibleMailbox that wraps ErrNoEmailAccounts so the three callers that pause a campaign on it are unaffected (#126) 2026-08-16 07:46:37 +02:00
Matthew Meszaros ee61c19faf feat: create /data/blobs in the backend, consumer and worker images owned by the non-root user they run as, because Docker seeds a fresh named volume from the image and the path did not exist there, so it created the mount point root:root while every service runs as uid 1000 and the very first attempt to store an outbound body failed with 'mkdir /data/blobs/emails: permission denied' and the task went to the dead-letter queue, which is invisible until a real send is attempted since the stack is otherwise healthy, and documenting the one-time chown that existing installs still need because Docker only applies image ownership when it first creates the volume (#124) 2026-08-16 07:46:22 +02:00
Matthew Meszaros 248a32dbec feat: fix the tracking dedupe insert that failed on every single open and click event, where the ON CONFLICT target named the expression COALESCE(url_hash, '') while the table's primary key is the three plain columns (task_id, event_type, url_hash) and no matching expression index exists, so Postgres rejected every insert with SQLSTATE 42P10 regardless of whether the row was actually a duplicate and tracking_events_processed stayed permanently empty, and NULLIF turned an open event's empty url_hash into NULL against a NOT NULL DEFAULT empty-string column which would have been refused even had the conflict target been right, leaving the consumer-level dedupe with nothing to check so a JetStream redelivery (MaxDeliver is 10, so not hypothetical) or an event slipping past the Rust service's in-memory hourly cache after a tracking restart could double-count an open or a click, and dropping the equally redundant COALESCE from the IsProcessed lookup so it can use the primary key (#123) 2026-08-16 07:46:19 +02:00
Matthew Meszaros ec4cd3160b feat: thread Unibox dashboard replies into the conversation they answer, by carrying the composer's thread_id all the way from email_tasks to the provider (EmailMessage had no ThreadID field at all, so the column was read from the database and silently dropped in user_email_task, and Gmail only appends to an existing thread when threadId is set on the outbound message since a matching Subject and In-Reply-To do not do it), populating the models.SendEmail.Parent field that already existed with an avro tag and that the worker already read but nothing ever set, replacing the worker's gate that required InReplyTo to be non-empty before it would look at Parent (a dashboard reply never sets that header, so a perfectly valid ThreadID was discarded and the provider opened a new conversation) with a parentReference helper shared by the Gmail and Graph send paths that resolves the two genuinely independent handles separately, and backfilling the RFC In-Reply-To header server-side in UniboxReply from the newest Message-ID in the thread via a new org-scoped LatestMessageIDInThread query, because a provider thread id is meaningless outside the mailbox that issued it and the recipient's mail client can only thread on References and In-Reply-To (#122) 2026-08-16 07:46:16 +02:00
Matthew Meszaros 7f1f46ea02 feat: resume a reloaded Gmail mailbox from its saved history checkpoint instead of the never-written email_accounts.last_id column, which no code in the repository ever assigns so it is NULL forever and every worker restart, deploy, crash or docker compose up handed the mailbox a zero cursor that re-bootstrapped straight to Gmail's current historyId and silently skipped everything that arrived since the last successful sync, unrecoverably because the history API only walks forward from the id it is given, by threading the EmailHistoryIDRepository the consumer already writes to into emailService as the Google counterpart of the existing WireGraphDelta cursor plumbing for Outlook, reading it in buildAddWorkerEmail through a lastHistoryFor helper that prefers the live checkpoint, still falls back to last_id for rows carrying a value from before the checkpoint table existed, and returns zero only when genuinely nothing is known so a first-time mailbox bootstraps a fresh baseline, plus tests covering the precedence and every fallback including an unwired repository (#121) 2026-08-16 07:46:13 +02:00
Matthew Meszaros 2c56dc9075 feat: make the Gmail history checkpoint persist and advance, by setting UserID and EmailID on the JobEventHistoryIDUpdate that NewHistoryID publishes (email_history_ids is keyed (user_id, email_id) with a foreign key to users, so the zero UUIDs it was sending made every checkpoint write fail on email_history_ids_user_id_fkey and no Gmail mailbox ever got a row, which is why replies, opens via label changes and every other inbound signal never reached Unibox), bootstrapping a mailbox with no baseline from Users.GetProfile instead of calling history.list with startHistoryId=0 which Gmail rejects with 'Requested entity was not found' so a freshly connected mailbox could never establish one, advancing the in-memory GoogleData.LastHistoryID after a successful walk since it was only ever written at construction and a stale cursor re-walks the window just processed while a zero one re-bootstraps past everything that arrived in between, and no longer discarding a MailError when a partial history was processed in the same tick, matching the field assignment the already-correct but uncalled ImapGoogleSync has had all along (#120) 2026-08-16 07:46:11 +02:00
Matthew Meszaros 4e14df16e2 feat: send every Gmail message as raw RFC 5322 instead of the structured gmail.MessagePart payload, which Gmail's users.messages.send rejects outright with "'raw' RFC822 payload message string or uploading message via /upload/* URL required" because the structured Payload tree is the read representation returned by messages.get and is not accepted on send, so every non-attachment Gmail send failed with a 400 that was retried and then dead-lettered under a misleading SERVER_UNREACHABLE label while the attachment path already built raw correctly, building the narrowest correct MIME structure per message rather than routing everything through the multipart/mixed attachment builder (bare text/plain for warmup and text-only campaigns, multipart/alternative once there is an HTML body, multipart/mixed only when files are attached, because a needlessly nested tree is a structural difference cold outreach does not need), RFC 2047-encoding the Subject and building the From header through net/mail.Address now that header encoding is ours rather than the API's, so a non-ASCII display name is no longer emitted as bare 8-bit bytes and a name containing a comma no longer splits the header into two recipients, and adding tests that parse the built message back with net/mail and mime/multipart to assert the structure instead of matching strings (#119) 2026-08-16 07:46:08 +02:00
Matthew Meszaros 16b672e6f6 feat: wire OnTokenRefresh on the Gmail worker client so every send and sync stops panicking, since goog.Client was constructed with all four message and label callbacks but no token callback while goog.Init unconditionally wrapped the token source in stoken, whose Token() calls that callback on every single request from inside the oauth2 transport's RoundTrip, making the nil func value a guaranteed nil-pointer dereference on the first Gmail API call any mailbox made (the Outlook path immediately below it set the same field correctly, so no Microsoft mailbox was affected), additionally guarding both goog.Init and msgraph.Init so the stoken wrapper is only installed when there is somewhere to persist a refreshed token to, hardening stoken.Token itself against a nil callback because it runs inside RoundTrip where a panic takes down the caller's request rather than surfacing as an error, and adding a regression test that panics without the guard and passes with it (#118) 2026-08-16 07:46:00 +02:00
Matthew Meszaros cdf200191c feat: accept the API's origin in the mailbox OAuth callback listener so connecting Gmail or Outlook completes on a split-domain deployment, where the bridge page is served by the backend (deliberately, so the registered redirect_uri survives front-end changes) and therefore arrives with event.origin equal to API_URL while the dashboard only ever compared it against APP_URL, silently discarding every callback and leaving the connect modal on 'Waiting for authorization' forever even though the provider exchange had already succeeded, normalising both configured bases through URL.origin so a trailing slash no longer breaks the comparison either, and separately deriving the bridge's postMessage target origin from APP_URL when APP_ORIGIN is unset instead of falling back to a wildcard that posts the authorization code to whatever origin the opener happens to have, since compose never set APP_ORIGIN despite the configuration table claiming it was derived, with the app_origin_wildcard health check and both docs pages updated to match the narrower condition that now triggers it (#117) 2026-08-16 07:45:57 +02:00
Matthew Meszaros 8d2968eaa8 feat: build the Gmail and Outlook mailbox-connect redirect_uri from API_PUBLIC_URL instead of API_HOST, which is the listener's bind address and stays 0.0.0.0:8080 in a container, so every self-hosted OAuth connect sent Google and Microsoft a redirect_uri that is not even an absolute URI and was rejected with invalid_request before the account picker appeared, adding an oauthPublicBaseURL helper next to the existing oidcRedirectURL that reads the same already-documented variable and falls back to a browsable http://localhost:PORT derived from the bind address (mapping the 0.0.0.0, :: and bare-port wildcard forms to localhost, and passing through a value that is already a URL) so a stock local install emits exactly the http://localhost:8080/addresses/google/callback the guide tells operators to register, plus a test pinning the resolution order and the root-registered callback path, and a deployment-guide note naming API_PUBLIC_URL as the variable that has to match the registered URI (#116) 2026-08-16 07:45:54 +02:00
Matthew Meszaros 1a7cdc955f feat: seal Gmail and Outlook OAuth access and refresh tokens at rest in email_accounts_oauth instead of storing the provider's raw tokens, which the connect UI already promised were encrypted and which the read path could never open because it unconditionally hex-decodes, adding sealCredential/openCredential helpers that fail closed when CREDENTIALS_ENCRYPTION_KEY is unset, encrypting on both write paths (NewOauthAccount at connect time and RefreshBoxToken on every worker token refresh, which would otherwise revert a sealed row to plaintext on first refresh), migrating pre-existing plaintext rows lazily on first read because the key lives in the application and no SQL-only migration can reach it, restoring the missing return on the OAuth insert failure that let a failed token write commit an account row with no credentials, keeping token parameters out of Sentry error reports, and correcting the two docs tables that scoped the key to SMTP and IMAP only (#115) 2026-08-16 07:45:52 +02:00
Matthew Meszaros 734cb5fe08 feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114) 2026-08-16 05:58:11 +02:00
Matthew Meszaros 0ae4db2c41 feat: make self-hosted auth work without a mail relay by rewriting the platform SMTP transport with real AUTH and TLS (it did neither, so SMTP_USERNAME/SMTP_PASSWORD were dead and every documented relay was unreachable), adding MAIL_TRANSPORT=smtp|log|ses with a log transport that prints codes so a fresh install can sign in with no relay, demoting the emailed login code to AUTH_LOGIN_CODE=always|new_device|off (off on self-host, per NIST SP 800-63B and OWASP ASVS), claiming the first owner through a single-use setup link or WARMBLY_BOOTSTRAP_* instead of register-then-psql, deriving every emailed URL from APP_URL rather than a hardcoded app.warmbly.com that leaked live reset tokens to the vendor, fixing the confirm hooks that read path params against paramless routes and broke login, register and reset confirmation in the dashboard everywhere, adding generic OIDC with PKCE, one-time state, verified nonce and (issuer,subject) identity binding, enforcing 2FA on the social paths that skipped it, adding a per-IP limiter and trusted-proxy handling to the unthrottled auth group, refusing boot on the published default secrets, and dropping mailpit from the default stack (#99) 2026-08-14 14:57:09 +02:00
Matthew Meszaros 8f465fdb1c feat: give each mailbox a human sending persona (randomized daily and hourly caps, send spacing, work start/end, lunch break and working weekdays, rolled once per local day in the mailbox's own timezone and applied across the campaign, warmup and smart-send schedulers), add campaign auto-pause guardrails that stop a campaign when its bounce, complaint or reply rate leaves the configured band, make mailbox rotation actually rotate for tag-resolved and all-mailbox campaigns, stop every scheduler from ever returning a slot in the past, and correct the mailbox min-gap field that stored seconds while labelling them minutes 2026-08-13 16:51:29 +02:00
Matthew Meszaros 8bd2c2b57a feat: make self-hosting work end to end and rewrite the guide around what was tested (#97) 2026-08-13 09:47:46 +02:00
Matthew Meszaros a7518a8558 docs: refresh the documentation site, fix inaccurate claims and contact addresses, add SEO primitives (#90)
* feat: rewrite the self-hosting docs against repo ground truth: turn the deployment guide into a full self-host guide (quick start with first-admin bootstrap via make grant-admin, .env secrets with exact key formats, PUBLIC_HOST derivation and HTTPS reverse-proxy vars, provider switches with build-tag caveats, mailbox OAuth, remote worker enrollment via SSH or wmenroll tokens, real CI image tags, upgrades and backups), rewrite the events page around the real NATS/Kafka bus topics and {type,body} envelopes, fix Kafka-era and make-target claims in architecture/local-development/deploy README, add API_PUBLIC_URL and drop the dead LOG_DISCORD_WEBHOOK_URL in env.example, and remove the docker-compose.kafka.yml comment pointing at a file that does not exist

* feat: make the self-hosting docs visual and skimmable by adding a Mermaid MDX component (client-rendered, theme-aware) to the docs site, condensing the self-host guide around a control-plane topology diagram, a worker enrollment sequence diagram, a dashboard screenshot, and symptom/check troubleshooting + optional-subsystem tables, and adding an execution-plane flowchart to the architecture page

* feat: stop the docs root flashing a 'Continue to the Warmbly docs' link before redirecting by navigating with an inline location.replace that runs during HTML parse, and demoting the visible link and meta refresh to no-JS fallbacks inside noscript

* feat: cut docs bulk and duplication by deleting three orphaned API pages that were stale forks of the reference section and were unreachable from the sidebar (porting their unique social sign-in, promo-code, and referral endpoints into api/reference/account-org.mdx as compact tables), condensing the deliverability and warmup guides to roughly half their length around tables instead of prose, replacing prose em dashes across the guides and MCP pages, and adding the required trailing slashes to internal links in 24 files

* feat: condense the sequences guide by about 40 percent, folding the switch-step deciders and branch conditions into tables and cutting restated prose while keeping every rule about threading, instant branches, reply matching, and stop on reply

* feat: condense the automations, unibox, advisor, and expressions guides by roughly 40 percent each, folding trigger lists, action catalogs, sending controls, and advisor checks into tables, adding a trigger-condition-action flow diagram to automations, and cutting restated prose while preserving every threshold, permission boundary, and rule

* feat: condense the mailboxes, campaigns, analytics, and team-roles guides by roughly 45 percent each, replacing prose walks through providers, rotation modes, lead statuses, counting rules, A/B confidence, and the permission matrix with compact tables and collapsing the four-way role grid into one capability table plus a one-line mapping

* feat: condense the AI-steps, security, and contacts-CRM guides by roughly 40 percent, turning sign-in methods, AI step modes, switch deciders, credit and failure behavior, import field mappings, and deal views into tables while keeping every safety boundary and dedupe rule

* feat: condense the meetings, notifications, AI-credits, and AI-assistant guides by roughly 40 percent, merging notification categories and their defaults into one table, collapsing credit costs, spend controls, and plan allowances into tables, and tightening the assistant page around its approval and permission boundaries

* feat: condense the integrations, collaboration, zapier, and make guides by roughly 35 percent, grouping the thirty-row Zapier and Make action lists into eight labelled areas, folding CRM default field mappings and presence indicators into tables, and promoting the destructive-action and unattended-delete warnings into callouts

* fix: correct three factual errors in the development docs: NOTIFICATION_EMAIL_DAILY_CAP=0 means uncapped rather than disabled (overEmailBudget returns false at limit<=0, so documenting it as a kill switch inverted the behavior), and the worker-SSH and warmup-pool migration citations in architecture.mdx pointed at pre-squash filenames that no longer exist or now belong to unrelated migrations, so both now cite the tables in 000001_baseline.up.sql

* feat: add the missing docs SEO primitives: a build-time sitemap.xml covering all 64 pages, a robots.txt that points at it and keeps the llms.mdx and og mirrors out of the index as duplicate content, and per-page canonical plus richer OpenGraph URL/title/description metadata

* fix: use the single real team@warmbly.com address everywhere a human is told to write in, replacing the invented hello/sales/legal/support inboxes across the marketing site, the transactional email footer, and the admin outreach composer default Reply-To (which pointed replies at a mailbox that does not exist), and collapse the contact page's two-inbox framing into one inbox with one published response time
v0.2.0
2026-08-05 10:37:27 +02:00
Matthew Meszaros 50f50e680d feat: make the inbound mail pipeline work end to end by never publishing the eventbus partition key as Nats-Msg-Id (JetStream deduped every event after the first per mailbox), fetching IMAP message bodies after the outer FETCH closes instead of nesting one inside it (which deadlocked sync on the first message), wrapping NEW_EMAIL in JobEventNewEmail across all three providers so the consumer stops nil-derefing, coalescing nil arrays before the NOT NULL unibox columns, sealing validation credentials on a copy so stored SMTP/IMAP passwords are not double encrypted, routing the email task type to the user email handler, and returning false instead of closing a nil conn in VerifySMTP (#88) 2026-07-31 09:41:36 +02:00
Matthew Meszaros 5e6287c920 feat: add the Advisor, continuous sending checks surfaced on the row they are about (#86)
* feat: index advisor findings by subject and parent entity so a list page fetches its whole surface once and every row resolves its own advice from the shared cache instead of firing a request per row

* feat: rebuild the advisor fix drawer as a three-screen resolution flow (why it fired with the measured evidence, the exact before and after, then an animated outcome with undo) with a progress rail and direction-aware transitions, and deep-link manual fixes to the screen where they are made

* feat: add AdvisorRowFlag, the inline per-row advisor indicator that renders on the mailbox or campaign the problem is about and opens that row's findings in an anchored panel instead of making the reader join a card list against a table

* feat: add AdvisorSummaryBar, a one-line collapsible page summary that replaces the stack of advisor cards above a list, counts the distinct rows implicated rather than the findings, and forces itself open only for critical or workspace-level advice no row flag can carry

* feat: put advisor advice on the mailbox row it is about in the accounts list, replace the card stack above the table with the collapsible summary bar, and support ?mailbox=<id> so a finding can deep-link straight to the mailbox detail instead of the top of the list

* feat: flag advisor findings on the campaign row in the campaigns list, including step-level copy problems which index onto their parent campaign since a step has no row of its own, and add the collapsible summary bar above the list

* feat: move the deliverability and contacts pages onto the collapsible advisor summary bar so their findings stop pushing the numbers they describe below the fold

* feat: add an ordered Steps field to advisor findings, persisted as text[] and always refreshed from the current build, and write real how-to steps for the deliverability checks that have no one-click fix (bounce rate, spam placement, tracking domain, and per-record SPF/DKIM/DMARC instructions)

* feat: write ordered how-to steps for the manual advisor findings where the remedy alone leaves someone stuck (broken template syntax, missing first-name fallback, unsubscribed contacts still enrolled, a campaign with no resolvable sender, and a mailbox that lost warmup pool standing) and correct the personalization detail that named a merge syntax this product does not use

* feat: show a mailbox's advisor findings at the top of its detail drawer, which is where both the row flag and the ?mailbox deep link now land

* feat: open the resolution flow from findings that have no one-click fix too, since the ordered how-to lives there and a card with no Fix button previously left the steps unreachable

* docs: document the per-row advisor flags, the collapsible page summary, the three-screen resolution flow, and the ordered manual steps for findings with no one-click fix

* feat: align the advisor summary bar to the px-5 page gutter used by SectionBar and the list rows on all four surfaces, instead of sitting flush against the edge while the table it describes is indented

* fix: stop the resolution drawer collapsing to zero height between screens by switching the step transition to popLayout with a layout-animated container, so the dialog resizes into the next screen instead of snapping shut and reopening

* feat: wire the advisor repository, narrator, service, tool registration, and background runner into the backend boot path so findings evaluate on a schedule and the assistant can read them

* docs: register the advisor guide in the sidebar, add its endpoint scope table to the API reference, and document the sandbox advisor showcase

* fix: darken the advisor nav badge to solid orange-600 on white instead of a pale amber-100 chip that read as a disabled control beside the sidebar's saturated indicators, and drop the critical badge to rose-600 so the two stay in the same weight class

* fix: use orange-500 for the advisor nav badge, matching the high-severity dot on the row it points at, rather than the darker orange-600

* feat: add an Auto safety class to advisor actions and mark the seven fixes autopilot may apply unattended (the cap cuts, the send-gap widen, the campaign limit matches, and the unsubscribe header), with a test pinning the boundary so nothing that halts sending or generates new outbound mail can drift into it

* feat: add advisor autopilot, which applies the auto-safe fixes unattended as the member who switched it on, resolving their live permissions each run so it fails closed when they leave the org, bounded to 10 changes per evaluation and audited per fix like any hand-made change

* feat: add the advisor agent fix, a bounded per-finding agent run that resolves the problems a settings change cannot (broken template syntax, bulk-reading copy, shared-inbox lists) as the calling member inside a tool allowlist scoped to the finding's category, metered per iteration and marked applied only when it actually called a write tool

* feat: surface autopilot and the agent fix in the dashboard, adding the workspace toggle that names exactly which changes it may make, an Auto chip on the findings it is allowed to take, and an agent-fix path in the resolution drawer that reports the tools it actually called rather than only its own account of them

* docs: document the agent fix and autopilot, naming the exact set of changes autopilot may make, that it acts as the member who enabled it and stops when they leave, and why the agent-fix endpoint is JWT only

* fix: gate the agent fix per detector instead of per category, so a missing DMARC record no longer offers a Fix-with-agent button it can never satisfy and then reports failure; findings whose fix lives in DNS or a provider console now show their manual steps, and the client is told which is which via agent_fixable

* feat: soften the advisor surfaces to translucent washes, replacing the filled nav badge with a tinted pill that carries its colour in the text, frosting the row panel and the resolution drawer, and turning the severity chips and cards into layers the page shows through

* docs: correct the agent-fix scope to name the findings it cannot resolve, and why a DNS record shows steps instead of a button

* feat: ship the actual DNS records for the findings that live outside the platform, with the provider's SPF include resolved, the DMARC record scoped to the sending domain and starting at p=none, the DKIM host plus the console that generates its value, and a tracking CNAME pointing at this install's own tracking host

* feat: render advisor snippets as labelled copy-button rows so a DNS record is one click per field rather than a text-selection exercise, with no copy affordance on a value the server could not supply

* docs: document the pasteable DNS records and the guarantee that every check offers a fix, an agent, or ordered steps

* fix: bump golang.org/x/text to 0.39.0 to clear CVE-2026-56852, a HIGH-severity infinite loop in norm.Iter that Trivy started failing the security scan on
2026-07-30 17:15:09 +02:00
Matthew Meszaros b342629534 Merge pull request #85 from Turtlesfr/fix/duplicate-migration-000077 2026-07-26 19:44:20 +02:00
Matthew Meszaros 764500391c fix: clear the trivy security scan by bumping react-router-dom to 7.18.1 in web and admin, postcss to 8.5.23 across web/admin/docs (docs via a pnpm-workspace override since next pins 8.4.31), next and eslint-config-next to 16.2.11, and ignoring the RSC-only react-router CSRF advisory 2026-07-26 19:36:23 +02:00
Alexandre a8dcc8e49b fix: renumber migration 000077_ai_variables_resolved to 000078 to resolve the duplicate version with 000077_warmup_generation_scale that makes golang-migrate fail on backend boot 2026-07-26 11:05:59 -04:00
Matthew Meszaros 7efd4c09df Merge pull request #84 from warmbly/release-frontend-assets
Ship the dashboard and admin as production release images
v0.1.0
2026-07-22 18:43:25 +02:00
Matthew Meszaros 53a05fa7dd docs: document the web and admin release images and their WARMBLY_* runtime env in the deployment guide 2026-07-22 18:40:06 +02:00
Matthew Meszaros 084dd0a3d9 ci: build the web and admin images on PRs that touch them and run the web typecheck so broken frontend Dockerfiles and type errors are caught before a release 2026-07-22 18:40:06 +02:00
Matthew Meszaros 79efbd4b5b feat: publish the web and admin images to GHCR in the release workflow and serve those production builds from the prod docker-compose instead of the Vite dev servers 2026-07-22 18:40:06 +02:00
Matthew Meszaros 664774e1e3 feat: add production nginx Dockerfiles for the web dashboard and admin panel that build the SPA once and render /config.js from WARMBLY_* env at container start 2026-07-22 18:40:06 +02:00
Matthew Meszaros 611ecba1c2 feat: add the same runtime config shim to the admin panel so its built image reads api url, dashboard url, env label, and turnstile key from container env 2026-07-22 18:40:06 +02:00
Matthew Meszaros 9a67008ea2 feat: add a runtime config shim to the dashboard so a single built image reads its API url, app url, tracking domain, and turnstile key from container env via /config.js 2026-07-22 18:40:06 +02:00
Matthew Meszaros 666ea44256 fix: type the conditional token build/parse helpers with a uid-less subset so the web typecheck passes on the transient uid attribute 2026-07-22 18:40:06 +02:00
Matthew Meszaros 443dcbf4b5 Merge pull request #83 from warmbly/ai-content-blocks
AI content blocks and a much more capable dashboard assistant
2026-07-22 17:59:36 +02:00
Matthew Meszaros cca3bd5ba0 fix: keep the marketing site on sharp 0.34.5 since 0.35 breaks its Cloudflare build, bump svgo to 4.0.2, and suppress the remaining build-only sharp HIGH in the documented .trivyignore 2026-07-22 17:53:13 +02:00
Matthew Meszaros 00e12fab8f fix: upgrade grpc to 1.82.1 (plus its otel/oauth2/genproto bumps) to clear the HIGH GHSA-hrxh-6v49-42gf xDS RBAC and HTTP/2 advisories 2026-07-22 17:53:13 +02:00
Matthew Meszaros 66e9856ea8 fix: bump sharp to 0.35 in docs and site and linkify-it to 5.0.2 in web via pnpm overrides to clear the HIGH-severity Trivy CVEs blocking the security scan 2026-07-22 17:33:52 +02:00
Matthew Meszaros 55dfa56c33 docs: wrap the company merge token in backticks in the AI variables guide so MDX does not parse it as a JS expression and the docs build succeeds 2026-07-22 17:24:07 +02:00
Matthew Meszaros a7d06cbfbe docs: italicize the support and enterprise note callout to match the repo aside style 2026-07-22 17:17:14 +02:00
Matthew Meszaros f2f06b10ed docs: add a Runs on Docker Compose badge to the self-hosting section and restyle the support and enterprise section as a blue-bordered note callout 2026-07-22 17:14:22 +02:00
Matthew Meszaros 225a9c7472 docs: add a support and enterprise section to the README with managed-infrastructure and enterprise-support offers plus WarmblyHQ X follow buttons 2026-07-22 17:05:38 +02:00
Matthew Meszaros 8032167837 docs: document the expanded agent tool surface in the MCP reference and rewrite the AI assistant guide for real sending with approval and full workspace parity 2026-07-22 17:05:38 +02:00
Matthew Meszaros fe127c9d8d feat: add automation agent tools to list, read, update, enable or disable, and delete automations while preserving the node graph 2026-07-22 17:05:37 +02:00
Matthew Meszaros 67c4dbfdfb feat: add API key and webhook management agent tools gated on manage-api-keys and manage-settings 2026-07-22 17:05:37 +02:00
Matthew Meszaros 1efc722458 feat: add JWT-only team, org-settings, voice-profile, and read-only billing agent tools that API keys and MCP cannot reach 2026-07-22 17:05:37 +02:00
Matthew Meszaros 49507b0244 feat: add mailbox management agent tools for reading, updating sending limits and warmup, tracking domains, warmup appeals, and disconnecting a mailbox 2026-07-22 17:05:37 +02:00
Matthew Meszaros 64ccd1b273 feat: add send_reply and compose_email agent tools as RiskSend with a recipient-suppression gate, so the assistant can send with per-action approval and never over MCP 2026-07-22 17:05:37 +02:00
Matthew Meszaros ae99322f29 feat: add unified-inbox agent tools for mark-seen, thread labels, snooze, and scheduled-send review and cancel 2026-07-22 17:05:24 +02:00