Matthew Meszaros
e668a2a36b
feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa
2026-09-19 08:18:35 +02:00
Matthew Meszaros
49acd51b64
feat: stop one recurring fault burying error tracking by reporting it once per five minutes with the count it stands for, keep a cache outage from answering every signed-in request with a 500 and from taking realtime down by treating an unreachable Redis as a miss and the websocket handshake nonce nothing reads as best-effort, answer a 5xx with a sentence the reader can act on while the call site's own words go to the log against the same request id, prefer the API's own message over the HTTP class in the admin and dashboard clients, and name the fix on a schema registry refusal, an SES sandbox rejection and a mailbox check that could not be run
2026-09-19 07:39:40 +02:00
Matthew Meszaros
31c1f101c2
feat: give EmailSentEvent and WarmupEmailSentEvent their own derived Avro schemas so the email-events and warmup-events analytics streams stop failing at serialize on every send and finally register a subject ( #546 )
2026-09-16 04:20:15 -07:00
Matthew Meszaros
0a1ed6f04e
feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing ( #440 )
2026-09-15 01:48:45 -07:00
Matthew Meszaros
619eb2729a
fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider ( #522 )
2026-09-14 21:28:07 -07:00
Matthew Meszaros
1ca3bd19b3
feat: carry a unibox read or unread change out to the mailbox itself through a new MESSAGE_SEEN worker command, so a conversation read in Warmbly stops showing bold in Gmail, Outlook and IMAP, relaying the state the row holds rather than the one the request asked for, only for messages that actually changed, dispatched detached from the request and never retried ( #515 )
2026-09-14 21:20:42 -07:00
Matthew Meszaros
92e298b6ec
fix: clear every open issue in error tracking by fixing the bugs behind them rather than the reports: a document-level mouseleave handing RippleProvider the document itself, whose classList is undefined; the admin panel posting /getaway without the /v1 its baseURL omits, so its realtime socket 404d on every page; Gmail throttles classified from the 403 status alone and told to re-authorize instead of back off; consumer flag and folder events retrying forever on a message the unibox never stored; a lost token-refresh race answered 500 instead of the documented 401; a nil email_accounts slice crashing the admin user page; Turnstile mounted with an empty sitekey; conditional passkey autofill run after the user navigated away; a boot log filed as an issue; and one publish failure per message on a topic the broker refuses ( #519 )
2026-09-14 21:06:14 -07:00
Matthew Meszaros
8d790ede6c
feat: send from any address Google has verified a Gmail mailbox to send as and import the signature its owner already wrote in Gmail, reading both through gmail.settings.basic at connect and on demand via GET/POST /emails/:id/identity, validating the choice against the provider's own list in the service and again inside the UPDATE, clearing it when the provider stops verifying it, and never applying it to warmup ( #514 )
2026-09-14 10:13:36 -07:00
Matthew Meszaros
6684dafe3a
feat: classify opens and clicks that arrive from a known mail-filtering network as automated, so Microsoft 365 Defender's delivery-time pixel fetches and Safe Links URL detonations stop counting as engagement (issue #410 ), with a shipped scanner catalogue, TRACKING_SCANNER_* overrides and optional ASN matching from a trusted edge header
2026-09-10 03:38:50 -07:00
Matthew Meszaros
51dedc90ee
feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service
2026-09-07 03:51:06 -07:00
Matthew Meszaros
f844c14804
feat: carry the mailbox display name in the emsg body blob of every send so a renamed sender reaches the worker on its next message instead of waiting for an inactive-active toggle, read it into the From header of the Gmail, Graph and SMTP clients with the name cached at load time as the fallback, and say so in the mailboxes guide
2026-09-04 02:58:36 -07:00
Matthew Meszaros
31dabea0a4
feat: rebuild the click-without-open fix on top of the per-link click attribution from #298 : a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated ( fixes #294 )
2026-09-03 03:49:44 -07:00
Matthew Meszaros
99be92f159
feat: attribute every campaign click to the exact link with a per-link click log (email_link_clicks) behind the contact activity timeline, campaign live feed, recent activity and the email_clicked webhook, add per-campaign automatic UTM tagging (utm_tracking with source, medium and campaign overrides, utm_content from the link text) applied at send time to the stored ticket destination, and classify opens and clicks as machine when they arrive within ten seconds of dispatch or when several links of one email are followed within five seconds, so scanner clicks are logged but never count as engagement, fire automations or send webhooks
2026-09-03 01:05:34 -07:00
Matthew Meszaros
6b614ad4bb
feat: never hand a send to a worker that is not heartbeating (registry row plus the Redis heartbeat key, which the worker now sets at boot instead of 90 seconds later), because a command queued for a dead worker is neither executed nor answered; reload a worker's mailboxes the moment its boot heartbeat arrives instead of waiting for the reconciler's republish window, make the publisher fail instead of silently succeeding when no bus or object storage is configured, and log a send that could not reach a worker to the campaign feed as a red, retrying entry
2026-08-23 10:18:50 -07:00
Matthew Meszaros
ec4cd3160b
feat: thread Unibox dashboard replies into the conversation they answer, by carrying the composer's thread_id all the way from email_tasks to the provider (EmailMessage had no ThreadID field at all, so the column was read from the database and silently dropped in user_email_task, and Gmail only appends to an existing thread when threadId is set on the outbound message since a matching Subject and In-Reply-To do not do it), populating the models.SendEmail.Parent field that already existed with an avro tag and that the worker already read but nothing ever set, replacing the worker's gate that required InReplyTo to be non-empty before it would look at Parent (a dashboard reply never sets that header, so a perfectly valid ThreadID was discarded and the provider opened a new conversation) with a parentReference helper shared by the Gmail and Graph send paths that resolves the two genuinely independent handles separately, and backfilling the RFC In-Reply-To header server-side in UniboxReply from the newest Message-ID in the thread via a new org-scoped LatestMessageIDInThread query, because a provider thread id is meaningless outside the mailbox that issued it and the recipient's mail client can only thread on References and In-Reply-To ( #122 )
2026-08-16 07:46:16 +02:00
Matthew Meszaros
fb853ea23d
refactor: publish mailbox validation over the bus and drop the raw kafka producer
2026-07-20 09:56:16 +02:00
Matthew Meszaros
ac3c11bf9a
feat: seal outbound email content with the organization DEK
...
The send pipeline (publisher subject/body encryption, the S3 emsg blob,
and the worker-side decrypt) now keys off models.SendEmail.OrgID
instead of UserID. EmailMessage loses its UserID field entirely:
emailSender.Send derives the cipher identity from the email account's
OrganizationID and refuses to send for an account without one.
This also fixes two latent bugs. Emails sent through user_email_task
never set UserID, so they were silently encrypted under the zero-UUID
platform key. campaign_task's discarded Encrypt() pair is replaced with
an explicit DEK warm that fails fast when KMS is unavailable.
2026-06-10 17:16:42 +02:00
Matthew Meszaros
b701e4efab
feat: send campaign email attachments
...
Carry attachment references inside the stored email body blob, resolve bytes on workers, and encode attachments for Gmail API and SMTP sends without changing the Kafka send-email contract.
2026-06-05 06:03:21 +02:00
Matthew Meszaros
35fc12b624
worker+events: route through EventBus + Codec, drop direct Kafka coupling
...
WorkerService now holds eventbus.EventBus + codec.Codec instead of
*kafka.Producer / *kafka.Consumer. Receive() satisfies the
eventbus.Handler signature; Produce() goes through Codec.Serialize +
Bus.Publish.
events.Publisher likewise switches to (bus, codec) and stops
serializing via *kafka.Avrov2 directly.
The kafka package and its Avrov2/Producer/Consumer types remain for
the few non-worker call sites (tracking consumer, validate_credentials)
that haven't been migrated yet; bundled with KafkaBus.Producer() so
existing Avro framing on Kafka is preserved.
Worker boot wiring is split into cmd/* entry-point commits.
2026-05-27 14:43:03 +00:00
Matthew Meszaros
d227038ca0
ci: drop unused/unconvert/gosimple + shadow/nilness, run gofmt
...
Disable the linters that fire on legacy code without flagging real
bugs: `unused` (orphan repos kept for future feature flags),
`unconvert` (defensive type conversions), `gosimple` (style
suggestions in code we don't want to touch).
govet: disable `shadow` (idiomatic `err :=` re-decls in transaction
patterns) and `nilness` (legitimate defensive nil checks that look
tautological to the analyzer).
Ran `gofmt -w internal/ cmd/` — every Go file now passes
gofmt -l with no output.
Kept: govet, staticcheck, ineffassign, typecheck, bodyclose, noctx,
sqlclosecheck, gofmt, goimports, misspell — the real-bug checks.
2026-05-23 16:54:12 +00:00
Matthew Meszaros
dd1fc3d41d
feat: generate List-Unsubscribe URL for campaign emails when unsubscribe header is enabled
2026-04-09 15:17:39 +00:00
Matthew Meszaros
0799020dac
feat: add metrics, warmup content variety, tz-aware scheduling, org budget, admin stubs, and bug fixes
2026-04-09 14:33:45 +00:00
Matthew Meszaros
308f72b874
feat: add CORS origin config, encrypt email bodies in S3, replace printfs with structured logging
2026-04-03 06:24:20 +00:00
Máté Mészáros (Laptop)
41624a6f79
Analytics & Tracking
2026-01-29 05:59:04 +01:00
Máté Mészáros (Laptop)
81d5970ea8
Realtime, Task Handler, Worker & Consumer Setup
2026-01-26 04:42:19 +01:00
Matthew Meszaros