starred/warmbly
1
0
Fork 0
mirror of https://github.com/warmbly/warmbly.git synced 2026-09-11 00:05:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,190 Commits 8 Branches 12 Tags
83ba4b38b465bc0ea1ac97e57771f7bd2fa54536
Commit Graph
889 Commits
Author SHA1 Message Date
joao-crm 83ba4b38b4 fix: re-check the addresses the built-in probe left unknown as soon as a workspace connects a paid verifier, and withdraw the connection's degraded state once that verifier answers again, since a verdict reached before the connection existed otherwise waited out the 30-day shelf life while the credits sat unused and the card kept reporting whatever the provider said on the very first pass 2026-09-08 15:48:12 +00:00
Matthew Meszaros 25484f70ab Merge remote-tracking branch 'origin/main' into feat/issue-357-loopback-mail-relays 2026-09-07 09:05:11 -07:00
Matthew Meszaros 8b757985f3 feat: address the review on the folder identity change by retiring a deleted folder's backfill floor with it, since a name is reusable and an inherited floor silently skips the next folder's history, by claiming a rename only when a UIDVALIDITY has exactly one missing folder and one new one, because two missing folders and one arrival cannot say which was renamed and guessing moves the wrong folder's mail, by deduplicating names before the folder cap rather than after so a name listed twice cannot spend a real folder's slot, by deleting on a legacy UIDVALIDITY-only event only when that number still names exactly one folder, and by moving the folder row and its mail in one transaction so a refused rename cannot leave the messages in a folder nothing renamed 2026-09-07 08:49:23 -07:00
Matthew Meszaros d48d57464b feat: hand the SMTP auth negotiation the normalized host, because net/smtp records the name NewClient was given as the server name and PlainAuth refuses to authenticate when its own host does not match it, so a bracketed IPv6 literal failed on an address it was correct about, and stop the bulk CSV calling an unencrypted row invalid while the deployment config is still loading, where a self-hosted instance would have accepted it 2026-09-07 08:44:32 -07:00
Matthew Meszaros 63070fb833 feat: address the review on the loopback mailbox mode by building every mail address with net.JoinHostPort, so an IPv6 literal keeps the brackets a host:port string needs and "::1" on 1143 stops dialling a host called "::1:1143" with no port, by moving the self-hosted half of the rule into the dialers as well as the connect form, because an organization archive exported from a self-hosted instance carries its mailboxes and an import must not hand a hosted worker one that dials its own loopback in the clear, by refusing an unencrypted CSV row on a hosted instance where the API would only reject it a moment later, and by saying in the docs that the port never selects the mode and that the whole 127.0.0.0/8 range counts 2026-09-07 08:27:54 -07:00
Matthew Meszaros 769a05aa90 feat: identify an IMAP folder by its name rather than by its UIDVALIDITY, which RFC 3501 never promised was unique across folders, so a mailbox on a server that stamps that number with the folder's creation time no longer loses the entire sync of every folder in a tree created in the same second, with the folder row keyed on (email_id, mailbox), each stored message stamped with its folder's name alongside the UIDVALIDITY generation its uid belongs to, a rename followed as a move that carries the mail and the cursor instead of orphaning both, and a changed UIDVALIDITY treated as what it is, the cursor going void 2026-09-07 08:21:17 -07:00
Matthew Meszaros 766bd3ae6a feat: let a self-hosted instance connect a mail server on its own machine by adding a third mailbox security mode, "none", accepted only for a loopback literal and only where the worker shares a host with the relay, so Proton Bridge on 127.0.0.1:1143/1025 and a local Dovecot or Mailpit can be connected at all, with the rule enforced in onboarding and reauth validation, again by the worker against the peer it actually dialled rather than the name it was given, and hidden from the connect form on the hosted product where the worker is never the customer's machine 2026-09-07 08:07:40 -07:00
Matthew Meszaros a5802b53cf feat: address the review on the unsubscribe-link change by closing the tag scanner over quoted attributes in both the send path and the editor, so a > inside an attribute can no longer split a tag and rewrite the href that follows it into a dead link, and by scanning only the copy a plain-text step actually ships, resolving the campaign's inherited opt-out mode before warning, checking the subject too, and reporting an unreadable step list as a failed check rather than an empty scan 2026-09-07 06:34:45 -07:00
Matthew Meszaros b25897fe67 feat: render a hand-placed {{.UnsubscribeLink}} as a real anchor labelled with the workspace's unsubscribe link text instead of shipping the bare signed API URL in the body, let the composer turn a text selection into that link so the wording stays the author's, and warn in the step editor and at preflight when a plain-text-only campaign carries a body opt-out link that has nowhere to hide its address 2026-09-07 06:34:45 -07:00
Matthew Meszaros f826cf0803 feat: state at the acquisition filter itself that it selects on the presence of a record rather than of a UTM tag, so the query, the search model and the admin panel's labels and Channel column all carry the same definition of a direct signup 2026-09-07 06:02:10 -07:00
Matthew Meszaros 26c88d2426 feat: address the CodeRabbit review on the PostHog PR by dropping a referrer that is not a hostname instead of storing its query string or fragment, redacting email-shaped values out of every acquisition field before they reach the database or an analytics property, counting an invited signup which returned before the count was taken, hanging subscription_started off the persisted trial-to-paid transition so a redelivered webhook cannot report a second start, capturing the validated provider in both OAuth mailbox paths including Warmbly Cloud, making the two acquisition toggles mutually exclusive and naming them after what they actually select, and replacing the unsupported CNIL consent-exemption claim with what the guidance says and an explicit note that qualifying is a deployment-specific assessment this document does not make 2026-09-07 05:45:11 -07:00
Matthew Meszaros 8843e23138 feat: cover the acquisition record with live repository tests against a real schema, so the LEFT JOIN, the four columns added to the admin projection and the three channel filters are exercised rather than assumed, including that a second signup link cannot rewrite where a workspace came from and that the join does not fan out a row 2026-09-07 05:08:35 -07:00
Matthew Meszaros 7bd812d568 feat: warn once rather than silently when the analytics host is unreachable or rejects the key, so a wrong POSTHOG_KEY does not look like a quiet week, and cover the client with tests that assert the cookieless ingestion contract against a stub capture host: the sentinel distinct id, the mode flag, the three hash inputs, and that no event property names a person 2026-09-07 05:06:13 -07:00
Matthew Meszaros b204737a05 Merge the Sentry branch (with main) into the PostHog branch 2026-09-07 05:01:38 -07:00
Matthew Meszaros 1239f1d191 Merge main into the Sentry branch, routing the new smtp reporting through the errs wrapper instead of the SDK import main added 2026-09-07 05:00:43 -07:00
Matthew Meszaros 8f604921ab Merge branch 'main' into fix/issue-343-behaviour 2026-09-07 04:49:48 -07:00
Matthew Meszaros 04dc83da38 Merge the Sentry review fixes into the PostHog branch, keeping the new JavaScript escaping for the PostHog runtime-config values too 2026-09-07 04:48:30 -07:00
Matthew Meszaros f043bfc761 feat: address the CodeRabbit review on the Sentry PR by escaping every runtime-config value the web and admin entrypoints write into a JavaScript string literal so a quote or newline in a DSN cannot inject or discard the whole config, reporting only the count and types of a failed query's parameters instead of their values now that the db scope actually applies, cloning the fallback hub in errs.Hub so scope cannot leak between requests, flushing the tracking service before every fatal exit since process::exit skips the guard's drop, treating a blank WARMBLY_RELEASE or APP_ENV as unset in realtime, stamping the environment and a service tag into the forms page, and correcting the docs claim that an unset DSN means the SDK is never initialised 2026-09-07 04:47:51 -07:00
Matthew Meszaros 372df39eaa feat: decide the hosted form URL scheme from the host rather than the port, so an install that terminates TLS on a non-default port (forms.example.com:8443) keeps https in its share links, embeds and base_url instead of being downgraded to http by the port check, with a private-network host now treated as the LAN install it is, one scheme helper shared by FormsBaseURL and FormURLOn so the builder's base_url and a form's share_url can never disagree, and a table test covering every install shape 2026-09-07 04:45:21 -07:00
Matthew Meszaros 4582896f54 merge 2026-09-07 04:43:45 -07:00
Matthew Meszaros f1b6345337 feat: address the CodeRabbit review on the mailbox error lifecycle PR by bounding the transient-error resolution to failures raised before the sync pass ran, because JetStream is configured with MaxDeliver and no MaxAckPending so a redelivered older success could otherwise clear a newer outage and leave the mailbox looking healthy until the next distinct one, naming the archive folder in the drawer's folder note alongside the others the cap always keeps, and giving the duplicate-folder note a singular verb 2026-09-07 04:36:50 -07:00
Matthew Meszaros 87a7e3e1ed feat: make a hosted form creatable again by binding an unset embed allowlist and field list as empty arrays instead of NULL in the forms repository writes, which is what made every New form fail with a not-null violation on forms.allowed_domains since the feature shipped (issue #343), and while proving the flow end to end keep the port on the shared forms host so share links and embeds resolve on a ported install, build the embed snippet on the form's own origin so an organization on a verified custom forms domain does not embed from the shared one, and apply the same empty-array fix to webhook endpoints created without event_types, which answered the raw Postgres error instead of the documented subscribe-to-everything 2026-09-07 04:35:56 -07:00
Matthew Meszaros 6bfba363d3 feat: address the CodeRabbit review on the SMTP compatibility PR by carrying the server's own refusal in a rejected-recipient error so an address that no longer exists is distinguishable from one a policy blocked, classifying our own refusal to authenticate over an unencrypted link as the configuration problem it is rather than retrying it four times as an outage against a server that is answering fine, and fixing the fake server's canned refusal at construction so the session goroutine and the test no longer race on it under go test -race 2026-09-07 04:30:04 -07:00
Matthew Meszaros 6f230a3e17 feat: renumber the organization_acquisition migration from 000132 to 000133 after main landed its own 000132 for unibox mailbox uid_next, since two branches that each took the next number are green alone and collide once both merge 2026-09-07 04:27:02 -07:00
Matthew Meszaros d00960ad59 Merge feat/sentry-everywhere (with main) into the PostHog branch 2026-09-07 04:26:17 -07:00
Matthew Meszaros be16f77f5d Merge main into the Sentry branch 2026-09-07 04:25:41 -07:00
Matthew Meszaros b61a3588b6 Merge feat/sentry-everywhere into the PostHog branch to keep the stack current 2026-09-07 04:24:31 -07:00
Matthew Meszaros b1a1941574 feat: flush the SDK before exiting on a captured boot failure via a new errs.CaptureFatal, because the previous capture-then-log.Fatal pattern in the backend and consumer mains killed the background sender before it had sent anything, making a failure to boot the one error that never reached Sentry 2026-09-07 04:24:30 -07:00
Matthew Meszaros 1af5ac1ea6 feat: let a fixed mailbox problem stop showing as a permanent error (issues #362, #363): resolve a mailbox's connection errors on the first sync pass that reaches the server again, because nothing but a credential reconnect ever resolved an error row, so a five-minute outage left a red needs-attention on the mailbox for good and held its health at warning; carry what the folder listing had to skip as sync state shown in the drawer's Sync card instead of raising an error row nobody can withdraw, so the note disappears once the mailbox is back under the folder limit or the duplicate folder id is renamed; and keep only a fingerprint per message in the IMAP flag scan rather than the whole flag set and Message-ID, which held 74 MB per mailbox at the window and folder limits against 14 MB for the digests, on a worker whose base capacity is 16 mailboxes 2026-09-07 04:24:09 -07:00
Matthew Meszaros 211650af8d feat: send through every kind of SMTP server (issues #359-#361): negotiate the sign-in method from what the server advertises, preferring CRAM-MD5 then LOGIN then PLAIN, because sending AUTH PLAIN blind was refused by every server that offers only LOGIN, which is Microsoft 365 relays and most appliance relays, and that refusal was reported to the mailbox's owner as a wrong password and deactivated the account; put a deadline on the whole SMTP conversation so a peer that stops answering without closing the connection can no longer park a send goroutine forever, which only the dial was protected against; classify a refusal by its reply code so a permanent 5xx on the sender, the recipient or the message is reported as the rejection it is and not retried four times as though the server were offline, while a 4xx still retries; announce the sender's own domain in EHLO rather than net/smtp's localhost, which relays read as a spam signal; and share one AUTH LOGIN implementation with the notification mailer instead of keeping two copies of the code that handles credentials 2026-09-07 04:19:42 -07:00
Matthew Meszaros 8e9f67f46b feat: add cookieless PostHog analytics for the hosted marketing site and dashboard with server-side signup and subscription events, a first-party acquisition record written once at signup on a new organization_acquisition table registered in the org-transfer spec, an acquisition column and channel filter in the admin org list, and never a single request from a self-host because every key is unset by default 2026-09-07 04:18:33 -07:00
Matthew Meszaros b32a28778c Merge pull request #355 from warmbly/feat/agent-tools-forms-segments-suppressions 
feat: expose forms, segments and the suppression list to agents
 2026-09-07 04:06:57 -07:00
Matthew Meszaros 51dedc90ee feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service 2026-09-07 03:51:06 -07:00
Matthew Meszaros 875cea477f feat: give a duplicate IMAP folder id its own error code and guidance instead of reusing the folder-limit one, because the two problems have different fixes: getting under the folder cap versus renaming the folder the mail server gave a duplicate UIDVALIDITY, and the drawer was telling a user to delete folders when neither the cap nor deleting anything was involved 2026-09-07 03:42:57 -07:00
Matthew Meszaros 63d6637d0c feat: expose forms, segments and the suppression list to agents by adding twenty-three tools to the shared registry, so an MCP client, a Hermes-style REST agent, the dashboard assistant and warmblyctl can all build an audience, publish a lead-capture form, link one to a campaign and manage who is unreachable, each gated on the same contact and campaign permissions its HTTP route already uses 2026-09-07 03:39:42 -07:00
Matthew Meszaros 5a49a39980 feat: state that the IMAP folder ranking applies whether or not the listing exceeds the cap, because the sync pass walks folders in that order and can run out of budget partway, so an outreach reply in the inbox should land before a mailing list in a user folder whatever order the server listed them in, and correct the under-cap test whose comment claimed the opposite of what it asserted 2026-09-07 03:36:34 -07:00
Matthew Meszaros 883178718e feat: address the CodeRabbit review on the IMAP compatibility PR by guarding the NIL hierarchy delimiter so a server that reports none is not given a NUL separator that hides the leaf fallback and loses the Sent folder, serializing the idle connection's deadline state under a mutex so a warmup action finishing cannot clear the deadline out from under a sync fetch still in flight on the same session, and threading the delimiter the server itself reports through the folder classification so a folder under a separator other than a dot or a slash is recognized instead of filing as inbox 2026-09-07 03:27:50 -07:00
Matthew Meszaros 90f5e8c625 feat: follow only one folder per UIDVALIDITY on IMAP, because the folder row is keyed on that id while RFC 3501 only promises UIDs are stable within a folder, so a server that derives it from the creation time gives a folder tree made in one second a single shared id and the two folders would advance each other's cursor and delete each other's row; the inbox and special folders win the collision, the dropped folder is reported in the mailbox drawer and named in the worker log, and the folder classification, ranking and localized-name matching gain direct tests 2026-09-07 03:20:46 -07:00
Matthew Meszaros 0df1a57589 feat: renumber the unibox_mailboxes uid_next migration from 000131 to 000132 after the campaign continuous lead sources migration took 000131 on main, so golang-migrate does not refuse to build its source driver once both are merged 2026-09-07 03:15:06 -07:00
Matthew Meszaros 1a87720d5a merge 2026-09-07 03:14:42 -07:00
Matthew Meszaros f2c35cf872 feat: sync every kind of IMAP mailbox (issues #345-#349): connect servers without CONDSTORE by following UIDNEXT per folder and mirroring read state with a periodic flag scan, so Outlook.com, Microsoft 365 over IMAP and Yahoo work instead of failing at load; fall back to a STATUS per folder when the server has no LIST-STATUS, which silently made an account look empty; bound the wait between IMAP responses so a peer that vanishes without a FIN cannot park a command forever; keep the inbox and special folders when a mailbox has more folders than the cap (raised to 100) and relay the overflow as a warning instead of failing the mailbox silently; recognize localized folder names and the server's own hierarchy delimiter so a Sent folder called Gesendete Elemente is not filed as inbox; and back off on a widening interval while a mail server is unreachable so one outage is one warning rather than one a minute 2026-09-07 03:12:22 -07:00
Matthew Meszaros fb7cf397db Merge remote-tracking branch 'origin/main' into fix/campaign-restart-keep-running 2026-09-07 03:05:06 -07:00
Matthew Meszaros 889b96c38e feat: address the CodeRabbit review on the campaign restart PR by turning on Keep running for a form's campaign only after every field is validated and failing the form save on any error other than a foreign campaign, running the automation's campaign flip before the automation is saved and failing the save visibly when the campaign is not the workspace's or the flip does not land, and clarifying in the error-code table and the campaigns API reference that no_leads covers a campaign that never had a lead while one whose leads all finished starts and waits 2026-09-07 02:55:12 -07:00
Matthew Meszaros 5d60fe9a21 feat: fix campaign restart dead end (issue #340): resolve preflight tracking senders through the same explicit/tags/all pool the scheduler uses so a connected mailbox is never reported missing, fetch the launch dialog's step count from the steps endpoint instead of a field the campaign API never returns, turn on Keep running for new leads when a form or an automation feeds a campaign (migration 000131 backfills existing ones) and when a member starts a campaign whose every lead has finished so it goes active and waits for leads instead of answering 400, return waiting_for_leads from the start endpoint for the dialog's success screen, and document the no_leads and no_remaining_leads codes and the new behaviour in the campaigns, forms, automations and API docs 2026-09-07 02:26:50 -07:00
SUMAN JANA 61348a55f7 fix(worker): reuse only an authenticated IMAP session; drain LIST on the folder cap 
Address review:

- ensureConnected kept any client not in Logout, so a session whose Login
  failed (NotAuthenticated) was reused instead of re-dialed. Reuse only
  Authenticated or Selected, and close the half-open session when auth or
  the CONDSTORE check fails.
- Folders returned from inside the LIST loop on the folder cap without
  closing the command, leaving unread results to stall the next command.
 2026-09-07 07:25:21 +00:00
SUMAN JANA 5eaf4481d5 Merge branch 'fix/imap-dead-session-reconnect' of https://github.com/rocker1166/warmbly into fix/imap-dead-session-reconnect 2026-09-07 07:15:49 +00:00
SUMAN JANA b9fa144d1c fix(worker): guard the IMAP client across reconnects; Gmail-only name fallback 
Address review:

- The client field was read by commands while ensureConnected could swap
  it. A lifecycle RWMutex now holds the write lock through dial, auth and
  assignment, and every command holds the read lock for its duration.
  Lock order is mu before lifecycle; sentMailbox resolves under mu before
  AppendToSent takes the read lock.
- The virtual-folder name fallback applied to any server, so a plain IMAP
  account with a real folder called "Important" or "Starred" would have
  been dropped from sync. It now applies only inside Gmail's own
  "[Gmail]/" and "[Google Mail]/" namespace; regression cases added.
 2026-09-07 07:15:20 +00:00
Suman Jana c347d34d5a Merge branch 'main' into fix/imap-dead-session-reconnect 2026-09-07 12:38:52 +05:30
SUMAN JANA 04d73acafd fix(worker): reconnect dropped IMAP sessions and sync nested Gmail folders 
The worker opened one IMAP session per mailbox at load and never re-dialed.
When the server dropped it (Gmail does after a while), handleError mapped
net.ErrClosed to nil, so every pass ran as a clean "no folders" pass:
nothing logged, no error record, last_synced_at kept moving, no new mail
for days, and sent copies failed with "use of closed network connection".

- imap.Client: ensureConnected re-dials a client parked in Logout state;
  every entry point (Folders, AppendToSent, warmup actions) runs through it.
- handleError: transport errors return a retryable server-unreachable
  error instead of nil, so the sync loop logs and retries.
- Folders: LIST "*" instead of "%", which stopped at the top level and
  never reached [Gmail]/Sent Mail (or Dovecot's INBOX.*); request
  SPECIAL-USE, without which Gmail reports no \Sent/\Trash/\Junk/\All.
- Sync: skip Gmail's virtual label views (All Mail, Starred, Important)
  so known mail is not re-filed as archive under a different UID; retire
  a cursor an earlier build baselined for them. "Bin" is trash.
 2026-09-07 06:44:54 +00:00
Matthew Meszaros 54b5cba9a5 feat: address the CodeRabbit review on the continuous campaigns PR by clearing idle_since in the same update that turns the continuous setting off so the dashboard stops reading a campaign as waiting for leads, adding idle_since to the documented campaign response, qualifying the start endpoint's nothing-left-to-send 400 for continuous campaigns, and noting in the automations and forms guides that a finished campaign restarts through the launch checks and a refusal is written to its activity log 2026-09-06 02:58:33 -07:00