150 Commits

Author SHA1 Message Date
Matthew Meszaros 93e8451738 feat: organization data export and import for moving a workspace between instances (#132)
* feat: add the org_export_jobs and org_import_jobs tables plus the models behind them, so a whole organization can be written to a portable archive and read back on another instance, keeping the option columns typed (a text[] of data groups, an include_secrets boolean, a conflict_strategy check constraint) rather than a settings blob because the option set is small and fixed, and reserving jsonb only for the genuinely free-form parts that are read back for display alone (the source archive's manifest, per-table row counts, the import warning list), with partial indexes on the in-flight and expiring rows so the maintenance sweep stays cheap however much transfer history accumulates, an OrgDataGroup catalog that names the twelve slices of a workspace and carries the dependencies between them, and an org_archive audit entity so an export or import rides the existing audit spine into every teammate's dashboard

* feat: add the schema-generic repository behind workspace archives, which reads and writes tables by name rather than through typed structs because that is the only way an archive stays correct as the schema grows, moving rows as jsonb in both directions via to_jsonb on the way out and jsonb_populate_recordset on the way in so Postgres performs every type conversion and no hand-written Go column mapping can drift from arrays, jsonb, tsvector, inet or enums, lifting the pool's 60s statement_timeout inside the export transaction because a full inbox read legitimately runs longer than that, introspecting generated, identity and not-null columns plus primary keys and foreign keys from the catalog rather than trusting a compiled list, and treating identifier safety as structural: table names come from the compiled registry and column names are always intersected against the destination catalog before reaching a query, so nothing out of an uploaded archive is ever interpolated

* feat: add the workspace archive registry and on-disk format, covering all 110 organization-owned relations with their scope SQL, dependency order and per-table policy, plus 10 explicitly excluded ones each carrying the reason it must never travel (the KMS-wrapped org data key, in-flight OAuth handshakes, the websocket outbox, live sessions, a pending deletion that would otherwise schedule the destination workspace for destruction), naming the two key domains separately because Warmbly seals mailbox credentials under the instance CREDENTIALS_ENCRYPTION_KEY and everything else under the per-organization DEK and confusing them produces mailboxes that authenticate against nothing, defining the archive as a plain zip of newline-delimited JSON so an operator can unzip it and read the data in a text editor and so the manifest can be written last yet still be read first, and sealing archive secrets under an argon2id passphrase key with parameters deliberately heavier than the login hash since it is derived once per archive and guards every credential in the workspace against offline grinding

* feat: implement the workspace export and import engines, streaming rows straight through untouched for the tables that have neither secrets nor blobs so a million-row inbox export stays cheap and only decoding the rows that must change, opening every sealed value against whichever key domain wrote it and re-sealing it under the archive passphrase on the way out then against the destination's own keys on the way in, blanking a credential rather than sinking the whole export when one mailbox cannot be read and clearing the guard flag alongside it so no row is left claiming ciphertext it no longer holds, applying an import inside a single transaction because a half-applied workspace is far worse than a long-running one, rewriting the organization id and matching members to destination accounts by email with unresolvable people blanked where the column is nullable and redirected to the importer where it is not, and running transfers in the accepting process rather than through a queue for the one reason that matters: the passphrase is then never written down anywhere

* feat: make the per-organization DEK cache nil-safe in internal/app/cipher so a process built without Redis falls through to KMS on every call instead of dereferencing a nil cache handle, which is what lets warmblyctl run the workspace export and import commands at all: it deliberately attaches Redis as optional because the whole point of that CLI is working while the rest of the instance is down, and the decrypted-key cache was always an optimisation rather than a requirement

* feat: add the hourly workspace-archive maintenance job that deletes finished archives past their seven-day retention window, since each one is a complete copy of a workspace sitting in object storage and must not accumulate, and closes out any export or import whose process died mid-run, which is the necessary counterpart to executing transfers in the accepting process so the passphrase is never persisted: without this sweep a restart would leave a job reporting running forever

* feat: expose workspace export and import over the JWT-only organization routes and wire the service into the backend, gating every endpoint on workspace ownership through the existing requireOrgOwner check rather than a permission bit because an export with credentials is the single most sensitive artifact this product can produce and an import rewrites the workspace wholesale, so both belong at the same level as deleting it, spooling uploads to a temporary file since a zip needs random access and a length that a multi-gigabyte archive cannot supply from memory, handing that file's ownership to the background import so it outlives the request and is closed exactly when the job ends, streaming downloads with the archive's sha256 in a response header, and constructing the service with both key domains plus object storage so an archive can be opened, re-keyed and stored

* feat: add warmblyctl org list, export and import so a self-hoster can move a workspace from the box without a browser, running the same engine in-process against Postgres and adding no HTTP surface to a CLI whose entire trust model is container or host access, resolving --org from whichever handle the operator has (id, slug, or the owner's email), streaming the archive to a file or to stdout so it can be piped straight into ssh with progress still readable on stderr, prompting for the credential passphrase twice through the existing password prompt so the terminal and pipe rules stay identical across every command, and defaulting the import path to a preflight report that names what already exists here and which members have no account before anything is written, with --dry-run to stop there

* feat: add the dashboard API layer for workspace archives, fetching the data-group catalog from the server rather than restating it in the client so a new group appears the moment the backend knows about it, mirroring the server's group-dependency closure in expandGroups so the toggles a user sees always match what the archive actually gets, polling only while a transfer is in flight and dropping to no interval the moment none are active since a running job has no realtime event of its own, and downloading a finished archive as a blob through the authenticated client because the endpoint is bearer-authenticated and a plain anchor href cannot carry the token

* feat: build the Settings and Data dashboard page for exporting and importing a workspace, following the settings section conventions and the in-app confirm rather than window.confirm, defaulting the export to every data group because a migration that quietly leaves data behind is worse than one that takes a while, marking the heavy groups so nobody exports a decade of inbox history unaware, requiring the credential passphrase twice behind a confirm that states plainly what the file will contain, and making the import a two-step flow where a preflight reads the archive and reports its origin, row counts, unsealable credentials, existing rows and unknown members before a single byte is written, so confirming is never a leap of faith

* feat: register the Data settings section in the dashboard rail, route and realtime spine, placing it under Advanced beside the danger zone and gating it to the workspace owner so the nav matches what the endpoints actually allow, and mapping the new org_archive audit entity to the export and import query keys in useRealtimeEvents so an archive starting or landing refreshes the page for every teammate through the existing audit spine rather than a bespoke event

* feat: document workspace export and import as a customer guide registered under Account and team, covering what each of the twelve data groups contains and which four dominate archive size, why credentials need a passphrase to travel at all and what happens to mailboxes when they do not, how members are matched to destination accounts by email and what becomes of anyone without one, the difference between keeping existing rows and replacing them, and a table of what deliberately does not import with the reason for each, because billing, plan overrides, worker placement, sync checkpoints and warmup pool membership belong to an instance rather than to a workspace

* feat: document org list, export and import in the warmblyctl reference and point the deployment guide at them as the supported route between a self-hosted install and the hosted service in either direction, adding every flag with what it does, the two extra environment variables those commands read and the difference between them (a missing KMS provider stops the command because sealed values cannot be opened, while a missing CREDENTIALS_ENCRYPTION_KEY is only a warning that mailbox credentials will not move), the behaviour when Redis is down, and the warning that an archive carrying credentials is the most sensitive file this product produces

* feat: record in AGENTS.md that a migration adding an organization-scoped table is not finished until that table is registered in internal/app/orgtransfer/spec.go, either in Tables with its group and scope or in ExcludedTables with the reason it must not travel, because data left out of the registry is silently absent from every archive and nobody discovers it until a customer's migration lands on the other side missing a feature's data, and spelling out the four things that are easy to get wrong when adding one: dependency order, the group boundary that needs a Requires entry only when a NOT NULL foreign key crosses it, which of the two key domains seals a ciphertext column, and which columns name something only the source instance knows
2026-08-18 07:53:39 -07:00
Matthew Meszaros 7f1f46ea02 feat: resume a reloaded Gmail mailbox from its saved history checkpoint instead of the never-written email_accounts.last_id column, which no code in the repository ever assigns so it is NULL forever and every worker restart, deploy, crash or docker compose up handed the mailbox a zero cursor that re-bootstrapped straight to Gmail's current historyId and silently skipped everything that arrived since the last successful sync, unrecoverably because the history API only walks forward from the id it is given, by threading the EmailHistoryIDRepository the consumer already writes to into emailService as the Google counterpart of the existing WireGraphDelta cursor plumbing for Outlook, reading it in buildAddWorkerEmail through a lastHistoryFor helper that prefers the live checkpoint, still falls back to last_id for rows carrying a value from before the checkpoint table existed, and returns zero only when genuinely nothing is known so a first-time mailbox bootstraps a fresh baseline, plus tests covering the precedence and every fallback including an unwired repository (#121) 2026-08-16 07:46:13 +02:00
Matthew Meszaros 8d2968eaa8 feat: build the Gmail and Outlook mailbox-connect redirect_uri from API_PUBLIC_URL instead of API_HOST, which is the listener's bind address and stays 0.0.0.0:8080 in a container, so every self-hosted OAuth connect sent Google and Microsoft a redirect_uri that is not even an absolute URI and was rejected with invalid_request before the account picker appeared, adding an oauthPublicBaseURL helper next to the existing oidcRedirectURL that reads the same already-documented variable and falls back to a browsable http://localhost:PORT derived from the bind address (mapping the 0.0.0.0, :: and bare-port wildcard forms to localhost, and passing through a value that is already a URL) so a stock local install emits exactly the http://localhost:8080/addresses/google/callback the guide tells operators to register, plus a test pinning the resolution order and the root-registered callback path, and a deployment-guide note naming API_PUBLIC_URL as the variable that has to match the registered URI (#116) 2026-08-16 07:45:54 +02:00
Matthew Meszaros 734cb5fe08 feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114) 2026-08-16 05:58:11 +02:00
Matthew Meszaros 0ae4db2c41 feat: make self-hosted auth work without a mail relay by rewriting the platform SMTP transport with real AUTH and TLS (it did neither, so SMTP_USERNAME/SMTP_PASSWORD were dead and every documented relay was unreachable), adding MAIL_TRANSPORT=smtp|log|ses with a log transport that prints codes so a fresh install can sign in with no relay, demoting the emailed login code to AUTH_LOGIN_CODE=always|new_device|off (off on self-host, per NIST SP 800-63B and OWASP ASVS), claiming the first owner through a single-use setup link or WARMBLY_BOOTSTRAP_* instead of register-then-psql, deriving every emailed URL from APP_URL rather than a hardcoded app.warmbly.com that leaked live reset tokens to the vendor, fixing the confirm hooks that read path params against paramless routes and broke login, register and reset confirmation in the dashboard everywhere, adding generic OIDC with PKCE, one-time state, verified nonce and (issuer,subject) identity binding, enforcing 2FA on the social paths that skipped it, adding a per-IP limiter and trusted-proxy handling to the unthrottled auth group, refusing boot on the published default secrets, and dropping mailpit from the default stack (#99) 2026-08-14 14:57:09 +02:00
Matthew Meszaros 8f465fdb1c feat: give each mailbox a human sending persona (randomized daily and hourly caps, send spacing, work start/end, lunch break and working weekdays, rolled once per local day in the mailbox's own timezone and applied across the campaign, warmup and smart-send schedulers), add campaign auto-pause guardrails that stop a campaign when its bounce, complaint or reply rate leaves the configured band, make mailbox rotation actually rotate for tag-resolved and all-mailbox campaigns, stop every scheduler from ever returning a slot in the past, and correct the mailbox min-gap field that stored seconds while labelling them minutes 2026-08-13 16:51:29 +02:00
Matthew Meszaros 8bd2c2b57a feat: make self-hosting work end to end and rewrite the guide around what was tested (#97) 2026-08-13 09:47:46 +02:00
Matthew Meszaros 5e6287c920 feat: add the Advisor, continuous sending checks surfaced on the row they are about (#86)
* feat: index advisor findings by subject and parent entity so a list page fetches its whole surface once and every row resolves its own advice from the shared cache instead of firing a request per row

* feat: rebuild the advisor fix drawer as a three-screen resolution flow (why it fired with the measured evidence, the exact before and after, then an animated outcome with undo) with a progress rail and direction-aware transitions, and deep-link manual fixes to the screen where they are made

* feat: add AdvisorRowFlag, the inline per-row advisor indicator that renders on the mailbox or campaign the problem is about and opens that row's findings in an anchored panel instead of making the reader join a card list against a table

* feat: add AdvisorSummaryBar, a one-line collapsible page summary that replaces the stack of advisor cards above a list, counts the distinct rows implicated rather than the findings, and forces itself open only for critical or workspace-level advice no row flag can carry

* feat: put advisor advice on the mailbox row it is about in the accounts list, replace the card stack above the table with the collapsible summary bar, and support ?mailbox=<id> so a finding can deep-link straight to the mailbox detail instead of the top of the list

* feat: flag advisor findings on the campaign row in the campaigns list, including step-level copy problems which index onto their parent campaign since a step has no row of its own, and add the collapsible summary bar above the list

* feat: move the deliverability and contacts pages onto the collapsible advisor summary bar so their findings stop pushing the numbers they describe below the fold

* feat: add an ordered Steps field to advisor findings, persisted as text[] and always refreshed from the current build, and write real how-to steps for the deliverability checks that have no one-click fix (bounce rate, spam placement, tracking domain, and per-record SPF/DKIM/DMARC instructions)

* feat: write ordered how-to steps for the manual advisor findings where the remedy alone leaves someone stuck (broken template syntax, missing first-name fallback, unsubscribed contacts still enrolled, a campaign with no resolvable sender, and a mailbox that lost warmup pool standing) and correct the personalization detail that named a merge syntax this product does not use

* feat: show a mailbox's advisor findings at the top of its detail drawer, which is where both the row flag and the ?mailbox deep link now land

* feat: open the resolution flow from findings that have no one-click fix too, since the ordered how-to lives there and a card with no Fix button previously left the steps unreachable

* docs: document the per-row advisor flags, the collapsible page summary, the three-screen resolution flow, and the ordered manual steps for findings with no one-click fix

* feat: align the advisor summary bar to the px-5 page gutter used by SectionBar and the list rows on all four surfaces, instead of sitting flush against the edge while the table it describes is indented

* fix: stop the resolution drawer collapsing to zero height between screens by switching the step transition to popLayout with a layout-animated container, so the dialog resizes into the next screen instead of snapping shut and reopening

* feat: wire the advisor repository, narrator, service, tool registration, and background runner into the backend boot path so findings evaluate on a schedule and the assistant can read them

* docs: register the advisor guide in the sidebar, add its endpoint scope table to the API reference, and document the sandbox advisor showcase

* fix: darken the advisor nav badge to solid orange-600 on white instead of a pale amber-100 chip that read as a disabled control beside the sidebar's saturated indicators, and drop the critical badge to rose-600 so the two stay in the same weight class

* fix: use orange-500 for the advisor nav badge, matching the high-severity dot on the row it points at, rather than the darker orange-600

* feat: add an Auto safety class to advisor actions and mark the seven fixes autopilot may apply unattended (the cap cuts, the send-gap widen, the campaign limit matches, and the unsubscribe header), with a test pinning the boundary so nothing that halts sending or generates new outbound mail can drift into it

* feat: add advisor autopilot, which applies the auto-safe fixes unattended as the member who switched it on, resolving their live permissions each run so it fails closed when they leave the org, bounded to 10 changes per evaluation and audited per fix like any hand-made change

* feat: add the advisor agent fix, a bounded per-finding agent run that resolves the problems a settings change cannot (broken template syntax, bulk-reading copy, shared-inbox lists) as the calling member inside a tool allowlist scoped to the finding's category, metered per iteration and marked applied only when it actually called a write tool

* feat: surface autopilot and the agent fix in the dashboard, adding the workspace toggle that names exactly which changes it may make, an Auto chip on the findings it is allowed to take, and an agent-fix path in the resolution drawer that reports the tools it actually called rather than only its own account of them

* docs: document the agent fix and autopilot, naming the exact set of changes autopilot may make, that it acts as the member who enabled it and stops when they leave, and why the agent-fix endpoint is JWT only

* fix: gate the agent fix per detector instead of per category, so a missing DMARC record no longer offers a Fix-with-agent button it can never satisfy and then reports failure; findings whose fix lives in DNS or a provider console now show their manual steps, and the client is told which is which via agent_fixable

* feat: soften the advisor surfaces to translucent washes, replacing the filled nav badge with a tinted pill that carries its colour in the text, frosting the row panel and the resolution drawer, and turning the severity chips and cards into layers the page shows through

* docs: correct the agent-fix scope to name the findings it cannot resolve, and why a DNS record shows steps instead of a button

* feat: ship the actual DNS records for the findings that live outside the platform, with the provider's SPF include resolved, the DMARC record scoped to the sending domain and starting at p=none, the DKIM host plus the console that generates its value, and a tracking CNAME pointing at this install's own tracking host

* feat: render advisor snippets as labelled copy-button rows so a DNS record is one click per field rather than a text-selection exercise, with no copy affordance on a value the server could not supply

* docs: document the pasteable DNS records and the guarantee that every check offers a fix, an agent, or ordered steps

* fix: bump golang.org/x/text to 0.39.0 to clear CVE-2026-56852, a HIGH-severity infinite loop in norm.Iter that Trivy started failing the security scan on
2026-07-30 17:15:09 +02:00
Matthew Meszaros 2da507c673 feat: wire the new agent tool service dependencies into the tool registry and hoist advancedService construction so the compose and reply suppression gate is available 2026-07-22 17:05:24 +02:00
Matthew Meszaros bb040828b1 feat: rework automation AI steps to mirror campaign step types with one agentic AI step plus an AI switch, per-case switch handles on the canvas, a Stop node, and a step-type add menu 2026-07-22 10:58:35 +02:00
Matthew Meszaros 93b64e2afa feat: add shared run time listing and arg resolving for ai agents 2026-07-21 17:35:11 +02:00
Matthew Meszaros 8055c2f05f Merge remote-tracking branch 'origin/main' into self-host/local-stack
# Conflicts:
#	cmd/backend/main.go
2026-07-20 11:53:21 +02:00
Matthew Meszaros 543d97f6f3 feat: wire the no-cloud providers into the consumer and worker 2026-07-20 09:56:29 +02:00
Matthew Meszaros ed51b081e6 feat: wire the no-cloud providers and billing-off into the backend 2026-07-20 09:56:29 +02:00
Matthew Meszaros a534ef1908 feat: notification email channel becomes digest-first: email-channel notifications queue as pending rows (migration 000076 adds group_key/email_state/email_due_at/email_attempts) with a due time from a new per-user email_digest cadence (instant/smart 15m/hourly/daily, security sign-ins always immediate), a 30s flush loop with SKIP LOCKED claims bundles a user's pending rows into one digest email and coalesces org-shared group_key events into a single email with every recipient in To, reading a notification in-app cancels its pending email, and NotifyOrg targets only members holding a permission (Slack fires once per group) with new producers: dead-worker downtime to manage_emails members (SetNX-deduped per incident), trial expiry to manage_billing members via new billing_alert category (replacing the direct owner email), and invitation accepts to manage_team members via new team_activity category 2026-07-20 07:16:25 +02:00
Matthew Meszaros f7a1027367 feat: add workspace-shared assistant history: a manage_settings toggle on the organization (assistant_shared_history, off by default) makes every conversation visible to and continuable by every member with the use-AI permission — the service resolves a session to its owner's rows so open/continue/approve/delete all work cross-member while credits still charge the acting member, the org-wide listing joins owner names for attribution in the history rail, the workspace settings page gains the AI assistant section with the toggle and an explicit exposure warning, and the assistant guide's access and privacy sections describe both modes 2026-07-19 13:22:56 +02:00
Matthew Meszaros cf58c6d1d2 feat: add the compose backend: POST /unibox/compose sends a brand-new outbound email with org-wide recipient suppression enforced and optional auto mailbox selection, GET /unibox/compose/candidates scores every active mailbox for a recipient (conversation affinity from unibox history plus queued email tasks, remaining daily budget from daily_email_counts, domain-auth health) with human-readable reasons, a recommended pick, contact resolution, and suppression state 2026-07-19 07:27:23 +02:00
Matthew Meszaros c03cf37716 feat: seed the dev org as a mid-flight workspace: new internal/seed dev_org.go + dev_history.go give dev@warmbly.com 4 warmed premium-pool mailboxes, folders/tags/categories with real bindings (email_tags, campaign_folders, contact_categories, unibox thread labels), ~30 contacts with suppression states, an active 3-step campaign with 24 leads on a deterministic funnel and NOW()-relative sends including today, 14d campaign/warmup/daily stats rollups, a 13-message unified inbox that always resolves a dev-org sending mailbox, CRM pipeline with deals/tasks/notes/activity, reply templates, notifications, AI credit ledger history, and idempotent campaign logs 2026-07-18 12:00:24 +02:00
Matthew Meszaros 163d3057de feat: add web-search and extended-thinking capabilities to the campaign switch AI decider — a per-step web lookup about the contact's company (query derived from contact fields only, results fenced as untrusted, +1 credit charged only when results land, wired through a new tasks SetAISearch) and a thinking toggle that routes to the stronger model tier with a 2048-token budget priced through usage metering, with editor toggles under Capabilities and sequences/ai-credits docs 2026-07-18 08:42:08 +02:00
Matthew Meszaros 053ca6a71d feat: usage-based AI credit system — every AI call reserves its flat minimum then settles the real token cost per model (light 1500/standard 400 tokens per credit, drain-to-zero overage settle wired into writing assistant, reply drafts, dashboard agent, inbox agent, research, automation nodes, and campaign switches), org spend controls in a new org_ai_settings table (day/week/month hard limits enforced in Consume, low-balance alert threshold, auto top-up config), a credit-watch monitor hook that fires BILLING_CREDITS_LOW realtime alerts once per day and buys the configured pack off-session via a new Stripe AutoTopUpCredits (idempotent on the PaymentIntent, bounded per month, Redis-locked), GET /subscription/credits/usage + GET/PATCH settings endpoints, and an AI usage & spend controls billing card (spend vs limits, 30-day chart, per-feature and per-model breakdowns, limits/reminder/auto-top-up form) with a realtime low-credit toast 2026-07-18 08:39:41 +02:00
Matthew Meszaros 273dd7ce33 feat: merge main (deliverability auth-state sweep, watch-band cold throttle, research roadmap) plus the 000067 email-auth-state migration renumber into the campaign AI step branch 2026-07-17 18:29:06 +02:00
Matthew Meszaros d18c5ebe4a feat: execute campaign AI steps in the scheduler path — one metered completion per contact (SetAI-wired provider + credits with per-step idempotency and refund-on-failure) that labels progress, fills contact custom fields which feed the chosen actions' templates, runs AI-picked pre-configured actions with per-decision campaign logging and tag/label choice resolution, grounds decisions in campaign history and the newest inbound reply via LatestInboundFromContact, and hands instant chains back to the scheduler at AI nodes 2026-07-17 16:26:23 +00:00
Matthew Meszaros cd5e1e9cf7 Merge pull request #72 from warmbly/research/warmup-abuse-cold-effectiveness
feat: deliverability research + observe-only SPF/DKIM/DMARC auth state and watch-band cold throttle
2026-07-17 05:29:58 +02:00
Matthew Meszaros d0e8bc1c97 feat: add one-command MCP OAuth connect on api.warmbly.com/v1/mcp — RFC 7591 dynamic client registration for public PKCE clients (dcr.go), RFC 9728 protected-resource metadata + WWW-Authenticate challenge via MCPAuthMiddleware, /v1/mcp now accepts an API key or OAuth token, public-client auth with no secret and mandatory PKCE reusing the existing OAuth 2.1 server (nullable-org clients, migration 000066), executable-redirect-scheme hardening on the open register endpoint, plus mcp/oauth/authentication/endpoints docs 2026-07-16 08:57:19 +02:00
Matthew Meszaros 29e3bb3a64 feat: drop the legacy OPENAI_*/ANTHROPIC_API_KEY/AI_LOCAL_MODEL env fallbacks (no prod deployments exist): AI_PROVIDER/AI_API_KEY/AI_MODEL/AI_BASE_URL/AI_FREE are the only AI vars, empty provider means openai, the anthropic preset and writing client key off AI_API_KEY, and the warmup Batch generator runs only when the selected provider is OpenAI 2026-07-16 06:39:06 +02:00
Matthew Meszaros 917451b105 feat: select the AI provider via AI_PROVIDER + AI_* vars in the backend and consumer (legacy OPENAI_*/ANTHROPIC_API_KEY still honored as fallbacks), logging a clear message when the provider is misconfigured 2026-07-16 06:14:19 +02:00
Matthew Meszaros d94bd70f88 feat: wire AI_LOCAL_MODEL into the backend and consumer provider config and pass the agent voice preamble into the dashboard agent service 2026-07-15 18:51:51 +02:00
Matthew Meszaros 78ff73e45c feat: ground inbox-agent reply drafts in the counterpart contact's CRM record - the inbox agent now folds the replying contact's name, company, and known custom fields into the draft prompt (mirroring the manual unibox reply draft) via an optional ContactReader wired from repository.ContactRepository in both the backend and consumer, best-effort so a lookup miss just drops the grounding, so suggested replies reference what Warmbly already knows about the person 2026-07-14 07:29:56 +02:00
Matthew Meszaros 56de65380b feat: inbox agent that drafts a suggested unibox reply on inbound human replies for human approve/edit/discard - paid + per-org opt-in (organizations.inbox_agent_enabled) feature where the consumer's reply hook, on a non-automated reply, detaches a goroutine (panic-contained, never blocks ingest) that checks entitlement, dedupes via ai_thread_drafts partial unique indexes (one pending per thread, unique source_message_id), pre-checks balance, grounds a reply in the thread history + org voice + skills via generation.Provider.Complete, reserves a draft row then charges 5 credits (idempotency inbox_agent:<draft.ID>, row unwound on a fresh context if the charge fails so no unpaid draft lingers), and emits an org-scoped AI_DRAFT_READY event gated on access_unibox; the agent never sends - only a human POST to /unibox/agent-drafts/:id/approve sends through the normal reply path, claiming pending->approved before send with an approved->pending revert on send failure, alongside list + discard endpoints; plus the unibox awaiting_agent_draft badge + agent_drafts search scope, an AgentDraftCard in the thread view with inline-editable approve-and-send/discard, the workspace settings opt-in toggle, CanUseInboxAgent paid gate, migration 000065, and docs (inbox-agent guide, endpoints, realtime) 2026-07-14 06:13:27 +02:00
Matthew Meszaros a067bf5922 feat: AI action nodes in automations - ai_classify/ai_extract/ai_generate run one generation.Provider completion per node over the event data and merge the result back as a variable (ai_class, output_keys[], ai_text) so downstream conditions branch on it, charging 1 credit per node with a run_id+node_id idempotency key, refund on provider failure, out-of-credits fails only that node while the run continues, and a per-automation ai_credit_failures counter (migration 000064) auto-pauses a flow after 20 consecutive credit misses; classify/extract run deterministic (temperature 0), dry-run executes AI nodes for real and is charged but never touches the auto-pause lifecycle, provider + credit ledger wired into both backend and consumer integration services, plus the reply-classifier Layer 3 ported off its bespoke OpenAI client onto the platform provider via an injected ModelClassifyFunc (platform-paid, deterministic), a CompletionRequest.Temperature primitive threaded through the OpenAI and Anthropic providers, the flow builder palette/config panels/validation for the three AI steps, and a new AI-steps-in-automations guide 2026-07-14 05:30:55 +02:00
Matthew Meszaros 2aec6745fa feat: connect external MCP servers whose tools the AI assistant can use - ai_mcp_servers table with bearer tokens sealed by the org DEK cipher (never returned) and SSRF-validated https urls, a dependency-free streamable-HTTP JSON-RPC client (initialize/tools/list/tools/call over safehttp dial-time IP blocking with SSE+JSON handling and body caps), an mcp service that discovers tools on connect and contributes only enabled servers' tools to the dashboard agent as namespaced mcp_<server>_<tool> defs that are always write-class and never auto-allowed, a registry DynamicToolSource hook so per-org tools join the agent's tool set and resume executes them through ToolDefs, /ai/connections CRUD gated on manage_settings with an mcp_server audit entity and spine, a Connections settings page to add servers and review/enable discovered tools, and docs 2026-07-13 20:05:24 +02:00
Matthew Meszaros a2f50b6572 feat: org AI skills (playbooks) that every AI feature loads and follows - ai_skills table (org-scoped, unique name per org, 32KB content cap) with a skills service exposing CRUD plus an enabled-skills preamble injected into the dashboard agent, contact research, and reply-draft prompts and a load_skill read-tool that returns a playbook's full content by name, /ai/skills CRUD gated on manage_settings (JWT) or the AI_AGENT scope (API) with an ai_skill audit entity and spine entry, an AI skills settings page (list rows opening a right-side drawer with name, one-line description, enable toggle, and a markdown body), and docs with two example playbooks 2026-07-13 19:46:25 +02:00
Matthew Meszaros f48d51bf16 feat: AI contact research agent that gathers cited web findings per contact with sync and background-batch runs - contact_research_runs table plus a strict save_research schema (every signal and public_artifact must carry a url, confidence high/medium/low, signals<=5 hooks<=3, reject-and-reprompt once then fail), a text/template runtime prompt injecting org voice, contact record, objective, and 5-search/6-fetch budgets, a research service that drives search_web/fetch_url through the provider loop and charges 2 credits on save (billable even for nothing_found, refund-free since charged only on save, balance and abuse-cap pre-checked so a capped org never does free work), POST/GET /contacts/:id/research plus a 500-cap /contacts/research/batch draining through a bounded pool of 4 workers over FOR-UPDATE-SKIP-LOCKED claims (no new Kafka), an org-scoped AI_RESEARCH_PROGRESS realtime event gated to view_contacts, APIPermAIResearch bit 23, a ContactEdit Research tab and a ContactsTable bulk Research action, and docs; also org-scopes every credit and research idempotency key so a client-supplied Idempotency-Key can never replay across tenants 2026-07-13 19:34:11 +02:00
Matthew Meszaros 7f425c1624 feat: dashboard-wide AI assistant with streamed tool-use runs, per-action approvals, and per-iteration credits - agent_sessions/agent_messages/ai_tool_policies migration, aiagent service runs the M2 registry through the provider loop over a resumable jsonb transcript, streams text/tool-step/approval/done events over SSE, gates write tools behind approve/deny/always-allow (send always per-action) and charges 1 credit per iteration (budget 20, refund on provider failure, out-of-credits 402 insufficient_credits, cap 429 usage_cap_exceeded) with a resume-safe idempotency namespace, provider PreIteration budget hook, /ai/sessions endpoints (cursor list + two SSE runs) gated by membership with tools bound to the member's org-permission bits, APIPermAIAgent bit 22, ai_session audit entity + spine, and a right-side resizable panel (Cmd+I / sparkle button) with streamed text, collapsible tool steps, inline approval and draft-artifact deep-link cards, stop, new chat, and a credits/budget meter; tenancy enforced at the SQL layer and docs for the assistant, endpoints, permissions, and error codes 2026-07-13 18:46:43 +02:00
Matthew Meszaros 6ad2f13a39 feat: shared AI tool registry (internal/app/aitools) that runs 16 permission-gated tools as the invoking user over existing service-layer calls - Registry+Invocation enforce org-permission bits (JWT) or API-key bits per tool, ToolDefs binds handlers into the provider agent loop and Call powers direct MCP invocation; tools cover contacts (search/get/update/add_tag/remove_tag), CRM (create_task/create_deal), campaigns (list/stats/create_campaign_draft), unibox (list_threads/get_thread/draft_reply, gated on CanUseUnibox, draft never sends), create_automation_draft (disabled), and web search_web + fetch_url (webhook SSRF validator + safehttp dial-block + 2MB cap + html-to-text + 15min Redis cache + rune-safe truncation); every write audits via AuditService.LogAction so the spine refreshes teammates, registry built once in main and exposed on the handler for M3/M8 2026-07-13 18:15:58 +02:00
Matthew Meszaros e07c33d3cd feat: provider-agnostic LLM agent-loop layer (OpenAI-first, Anthropic self-host connector) with RunAgent tool-use loop, approval/resume contract, pluggable web search, and extracted voice-rules builder - generation.Provider.RunAgent executes Go tool handlers over a resumable transcript with a read-auto/write-approve gate, openAIProvider drives it via a lean HTTP client retargetable with OPENAI_BASE_URL for any OpenAI-compatible endpoint and also serves the writing assistant, anthropicProvider mirrors it with tool_use/tool_result multi-turn plus hosted web_search, NewProvider prefers OPENAI_API_KEY, SearchClient supports Serper/SearXNG with a clean not-configured error, and BuildVoiceRules extracts the humanizer prompt for reuse; refreshed stale model ids to sonnet-4-6 and haiku-4-5 2026-07-13 17:52:09 +02:00
Matthew Meszaros 5d6ed2a068 feat: AI credit economy with monthly plan grants, non-expiring purchased top-ups, and Stripe billing - two-pool credit_ledger (monthly allowance reset each cycle on invoice.paid, purchased pool via checkout mode=payment fulfilled webhook-only), consume drains monthly-then-purchased, 3 fixed top-up packs, /subscription/credits balance+checkout+paginated transactions gated on manage_billing, one-time trial grant, billing credits card with balance ring and pack buttons, credit_purchase/credit_grant audit entities and spine, wired plan.monthly_credits through pg_plan reads/writes, retry-safe webhook idempotency, and an AI credits guide 2026-07-13 17:40:20 +02:00
Matthew Meszaros b9a99f91a9 Merge pull request #67 from warmbly/feature/ios-realtime-steps-preview
iOS realtime parity, richer sequence preview, and mobile push notifications
2026-07-13 16:15:18 +02:00
Matthew Meszaros d6b7885e8b feat: mobile push notifications end to end - APNs provider-token client, device_tokens table with session-scoped register/delete endpoints, a push channel in notification preferences (web + iOS toggles), and Redis-backed immediate-then-digest batching (first event pushes now, bursts summarize when the 5h window closes) wired in backend and consumer, with iOS registration/badge sync and docs for the channel, endpoints, and APNS_* deploy env 2026-07-13 16:11:15 +02:00
Matthew Meszaros 262d25779c feat: realtime admin dashboard + diagnosability - new Elixir admin:platform channel (JWT-only, gated on users.admin_permissions) mirrors every platform event, the admin app connects through a /getaway-tokenized Phoenix socket client with a throttled react-query invalidation spine, a Live Events firehose page streams the platform with filters/pause/JSON expand, a System Status page runs parallel infra probes (postgres/redis/kafka/schema-registry/realtime/tracking via /admin/system/status), and the worker logs viewer gains follow mode, line-count selection, and copy 2026-07-12 09:00:10 +02:00
Matthew Meszaros 48ec53d781 feat: add the sandbox seeder and simulator (cmd/sandbox, internal/sandbox) - seeds the Sunrise Labs showcase org (paid plan, six live mailboxes with sealed mailpit/dovecot credentials, active tracked campaigns, pre-verified contacts, warmup pool membership) and repairs every fixture smtp_imap account and .test contact; the simulator plays the internet by polling mailpit, appending mail into dovecot inboxes for the real IMAP sync to pick up, hitting tracking pixels and click tickets, and replying as hash-stable contact personas with correct In-Reply-To threading 2026-07-11 17:31:42 +02:00
Matthew Meszaros 6bc2073379 feat: honor CODEC_PROVIDER=json across backend and consumer (previously worker-only) and round-trip untyped event bodies into their typed payloads in both dispatchers - the worker command/result envelopes carry any-typed bodies that Avro cannot serialize, so worker messaging only works on the JSON codec; also wires the credentials encrypter into both mains 2026-07-11 17:31:02 +02:00
Matthew Meszaros a20ba926cb feat: add native Apple/Google sign-in to the backend - POST /auth/apple and /auth/google exchange provider-signed ID tokens for sessions, GET /auth/providers exposes configured providers for app discovery, with a JWKS-verifying idtoken package, auth config, stable error codes, and API/security docs 2026-07-07 05:56:35 +02:00
Matthew Meszaros cb13c65d5d feat: load active mailboxes onto their assigned workers via a backend reconciler and immediately on onboarding, decrypting each account's credentials into the AddWorkerEmail payload, fixing gmail/outlook/smtp accounts that were assigned but never loaded 2026-07-04 08:07:49 +02:00
Matthew Meszaros 52c5f224fa feat: rebuild the provider oauth2 config locally on the worker so delegated mailbox tokens refresh, and ship BOX_ google/outlook client credentials to provisioned workers through the orchestrator env 2026-07-04 08:07:49 +02:00
Matthew Meszaros eb0a5654fd feat: persist the opaque per-folder Graph delta cursor in a new email_delta_links table via a repository and a GRAPH_DELTA_UPDATE consumer handler wired into the consumer service 2026-07-04 08:07:49 +02:00
Matthew Meszaros ffe823cfa8 feat: persist and surface per-mailbox SPF/DKIM/DMARC authentication state via an hourly observe-only consumer sweep, threaded through the mailbox list/detail API and documented in the deliverability guide and mailboxes API reference 2026-07-01 02:32:23 +00:00
Matthew Meszaros f49816a7b1 feat: construct and bidirectionally wire the referral service into the backend bootstrap 2026-06-28 05:09:50 +00:00
Matthew Meszaros c5dfa5e4e7 feat: remove the HTTP-request action from campaign steps and automations in favor of signed webhooks, keep fire_event for custom payloads, and drop the now-unused outbound quota plumbing 2026-06-15 08:11:35 +02:00
Matthew Meszaros 660d717456 feat: register the webhook and oauth-app-webhook routes and wire the delivery worker, audit-to-webhook bridge, and app domain resolver into the backend 2026-06-15 08:11:20 +02:00