starred/warmbly
1
0
Fork 0
mirror of https://github.com/warmbly/warmbly.git synced 2026-09-06 00:01:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,935 Commits 1 Branch 8 Tags
55307b0ad9f4f9cbeef4aedabb793dcee69e2bd4
Commit Graph
43 Commits
Author SHA1 Message Date
Matthew Meszaros 27630eec0a feat: add website visitor tracking for issue #255 with migration 000106 (website_tracking_settings, website_visitors, website_page_hits, all registered in the orgtransfer spec), a consent-gated dependency-free tracking.js served by the Rust tracking service with a rate-limited, size-capped, prefetch-filtered POST /p ingest that forwards to a new backend internal page-hits endpoint for server-side user-agent and GeoIP enrichment, contact identification only through the click ticket the redirect appends to registered hosts, a per-workspace retention job, page_hit events with an expandable detail view in the contact Activity timeline, a Settings > Website tracking page for the snippet and consent, location and retention configuration, realtime PAGE_HIT fanout, and a website tracking guide plus endpoint, export and configuration docs 2026-08-29 03:25:50 -07:00
Matthew Meszaros 42ddb78e60 feat: make the realtime service survive a client that spams channel joins, and stop both clients from being that client: ws_join is now spent per phx_join in a shared RealtimeWeb.ChannelGuard that every channel runs BEFORE its Auth lookup (it was only ever spent on the socket handshake, so an established socket could issue unlimited joins and each one cost a Postgres query), handshakes move to their own ws_connect bucket so a reconnect storm cannot eat the budget a client needs to rejoin with, the retry_after_ms hint now points at the next window instead of shrinking as overage grew, every channel's join rejection carries the numeric code the API reference already promised (plus 4005 for a malformed topic), BulkChannel only pushes an event to the user who owns the operation instead of to anyone who guesses the id, and the web and admin clients treat a rate_limited join as transient (wait out the hint, keep the socket) while backing off per-channel rejoins on a decaying schedule rather than reconnecting on the 120ms floor forever; adds an ExUnit suite for realtime and a vitest suite for admin, both wired into CI 2026-08-26 21:14:34 -07:00
Matthew Meszaros 8bd2c2b57a feat: make self-hosting work end to end and rewrite the guide around what was tested (#97) 2026-08-13 09:47:46 +02:00
Matthew Meszaros bebc09289d build: cross compile the go images and fix realtime and the healthchecks 2026-07-21 17:34:59 +02:00
Matthew Meszaros 4962765a79 fix: format the org_channel AI_DRAFT access_unibox clause to satisfy mix format --check-formatted (Elixir CI) 2026-07-15 06:47:07 +02:00
Matthew Meszaros 56de65380b feat: inbox agent that drafts a suggested unibox reply on inbound human replies for human approve/edit/discard - paid + per-org opt-in (organizations.inbox_agent_enabled) feature where the consumer's reply hook, on a non-automated reply, detaches a goroutine (panic-contained, never blocks ingest) that checks entitlement, dedupes via ai_thread_drafts partial unique indexes (one pending per thread, unique source_message_id), pre-checks balance, grounds a reply in the thread history + org voice + skills via generation.Provider.Complete, reserves a draft row then charges 5 credits (idempotency inbox_agent:<draft.ID>, row unwound on a fresh context if the charge fails so no unpaid draft lingers), and emits an org-scoped AI_DRAFT_READY event gated on access_unibox; the agent never sends - only a human POST to /unibox/agent-drafts/:id/approve sends through the normal reply path, claiming pending->approved before send with an approved->pending revert on send failure, alongside list + discard endpoints; plus the unibox awaiting_agent_draft badge + agent_drafts search scope, an AgentDraftCard in the thread view with inline-editable approve-and-send/discard, the workspace settings opt-in toggle, CanUseInboxAgent paid gate, migration 000065, and docs (inbox-agent guide, endpoints, realtime) 2026-07-14 06:13:27 +02:00
Matthew Meszaros f48d51bf16 feat: AI contact research agent that gathers cited web findings per contact with sync and background-batch runs - contact_research_runs table plus a strict save_research schema (every signal and public_artifact must carry a url, confidence high/medium/low, signals<=5 hooks<=3, reject-and-reprompt once then fail), a text/template runtime prompt injecting org voice, contact record, objective, and 5-search/6-fetch budgets, a research service that drives search_web/fetch_url through the provider loop and charges 2 credits on save (billable even for nothing_found, refund-free since charged only on save, balance and abuse-cap pre-checked so a capped org never does free work), POST/GET /contacts/:id/research plus a 500-cap /contacts/research/batch draining through a bounded pool of 4 workers over FOR-UPDATE-SKIP-LOCKED claims (no new Kafka), an org-scoped AI_RESEARCH_PROGRESS realtime event gated to view_contacts, APIPermAIResearch bit 23, a ContactEdit Research tab and a ContactsTable bulk Research action, and docs; also org-scopes every credit and research idempotency key so a client-supplied Idempotency-Key can never replay across tenants 2026-07-13 19:34:11 +02:00
Matthew Meszaros 262d25779c feat: realtime admin dashboard + diagnosability - new Elixir admin:platform channel (JWT-only, gated on users.admin_permissions) mirrors every platform event, the admin app connects through a /getaway-tokenized Phoenix socket client with a throttled react-query invalidation spine, a Live Events firehose page streams the platform with filters/pause/JSON expand, a System Status page runs parallel infra probes (postgres/redis/kafka/schema-registry/realtime/tracking via /admin/system/status), and the worker logs viewer gains follow mode, line-count selection, and copy 2026-07-12 09:00:10 +02:00
Matthew Meszaros 15088a6f64 feat: pin cowboy >= 2.16 in the realtime service so CR/LF response header values are rejected before the wire (EEF-CVE-2026-43966/43969 has no patched cowlib release) 2026-07-07 05:56:47 +02:00
Matthew Meszaros e190571696 feat: harden the live channel with an is_map payload guard, byte-bounded chat/id/resource strings, a per-surface permission gate that strips cursor-chat text, and a websocket max_frame_size cap 2026-07-02 20:33:44 +02:00
Matthew Meszaros 88cabf5cac feat: cap the live:select ids payload before traversal so an oversized selection list is never fully walked by the realtime channel 2026-07-02 14:31:36 +02:00
Matthew Meszaros 02b25724f4 feat: extend the ephemeral live channel with a sanitized chat field on live:cursor frames and a new live:select kind that broadcasts a capped node-id list for canvas selections 2026-07-02 07:40:06 +02:00
Matthew Meszaros 3ffd4416a2 feat: add a generic live:patch channel kind that fans out a small sanitized scalar map for ephemeral collaborative-state hints 2026-06-30 10:34:14 +02:00
Matthew Meszaros 248c886290 feat: add an ephemeral live:cursor/live:node channel path to the realtime org channel that bypasses the Redis ws limiters and the sequenced resume buffer via an in-process token bucket and broadcast_from fan-out 2026-06-30 10:06:19 +02:00
Matthew Meszaros f9d5c85a32 fix(security): move realtime sentry onto a Finch HTTP client and drop the hackney dependency to clear CVE-2026-47071 2026-06-28 06:18:33 +00:00
Matthew Meszaros b83aa51921 feat: make CUSTOM_EVENT fire-event signals an explicit allow on the org channel and return structured {code, reason} errors on a failed channel join 2026-06-14 11:02:12 +02:00
Matthew Meszaros 799e5f10ef feat: surface WebSocket connection-rejection reasons to the client as a structured {code, reason, retry_after_ms} payload (Discord-style 4003/4004/4007/4009/4010) instead of a bare 403, and fix the rate-limit 3-tuple match 2026-06-14 11:02:12 +02:00
Matthew Meszaros 34e579fc0f feat: authenticate the realtime WebSocket with OAuth access tokens (wmat_ with the realtime scope), validated against oauth_access_grants alongside wmbly_ API keys and JWTs, and map not-a-member/forbidden rejections to close code 4010 2026-06-14 11:02:12 +02:00
Matthew Meszaros f2fa0d756c feat: guarantee in-order gateway delivery via a per-org sequencer pool — serialize each org's events (partitioned by org id) so seq order always equals broadcast order even under concurrent ingest, closing the resume race where a higher seq could arrive first and a disconnect would skip the lower one 2026-06-13 11:31:07 +02:00
Matthew Meszaros 4552ab4e31 feat: make the org realtime channel a resumable gateway — assign a monotonic per-org sequence + buffer each event in a capped Redis stream (Realtime.EventLog) before broadcast, advertise the current seq in the HELLO and stamp it on every event, and replay the missed gap (re-filtered) on reconnect via resume.last_seq or signal a full resync when the buffer no longer covers the client 2026-06-13 11:23:07 +02:00
Matthew Meszaros c5ced50868 fix: apply mix format to the realtime service (supervisor child-list indentation in application.ex + org_channel.ex) so the Elixir CI format check passes 2026-06-12 17:45:32 +02:00
Matthew Meszaros 7b34b96b37 feat: keep the per-user realtime connection cap at 10 but reap dead connections via a periodic liveness sweep and process pids, so the count reflects real presence and stale sockets never hold a slot 2026-06-12 07:46:13 +02:00
Matthew Meszaros 3cbfc06bc4 feat: add WebSocket intents (selective event-family subscription) and a HELLO-style org join reply advertising heartbeat cadence, and correct realtime docs on connection rejections and at-most-once delivery 2026-06-12 07:17:09 +02:00
Matthew Meszaros 4f1ac8c273 feat: enforce org team-presence privacy in the realtime OrgChannel — gate Presence.track on show_online, strip viewing/editing detail when show_activity is off, and re-track/untrack live on PRESENCE_POLICY_UPDATED 2026-06-12 05:59:01 +02:00
Matthew Meszaros ba3ab98e1e feat: bridge realtime events over Redis pub/sub when Google Pub/Sub is unconfigured (RedisBus publisher in backend/consumer plus Realtime.Redis.EventSubscriber and a shared EventBroadcaster) so dashboard live updates and presence-driven collaboration actually fire in local dev 2026-06-12 04:49:58 +02:00
Matthew Meszaros 70064e9e1d fix: define OrgChannel.handle_out/3 so presence_diff broadcasts are pushed to clients instead of crashing the channel — the missing callback was terminating the org channel and dropping every websocket connection in a reconnect loop 2026-06-11 13:47:11 +02:00
Matthew Meszaros 80d080b982 feat: fix realtime UUID param encoding crashing org-channel joins, remove the permission matrix from Roles & access, compact the role dropdown, and drop the redundant Integrations tab from settings nav 2026-06-11 11:16:13 +02:00
Matthew Meszaros 3473d09bcc feat: fix review findings in the roles redesign — GetMembers role_id column (members endpoint 500), TransferOwnership role_id hygiene, accept-time role re-resolution, race-free in-use delete guard covering invitations, assignment anti-escalation with self-role-change block, canManage-gated members UI, colored RolePills, fresh currentOrganization on refetch, dev JWT_SECRET wiring for make realtime, docs corrections 2026-06-11 10:45:21 +02:00
Matthew Meszaros 371fe0c0a3 feat: add Phoenix.Presence org collaboration layer (online members, viewing/editing/replying activity) with permission-gated org event fanout in the realtime service 2026-06-11 07:51:34 +02:00
Matthew Meszaros 5b735bb528 feat: replace ascii architecture diagrams in the realtime readme and resources docs with prose and tables 2026-06-10 18:27:42 +02:00
Matthew Meszaros da82993846 feat: add campaign workflow backend support 
Add sequence action-node storage and execution, template conditional rendering, lead progress state, profile updates, webhook fan-out throttling, and supporting repository fixes.
 2026-06-06 07:49:52 +02:00
Matthew Meszaros 60773b3d8e feat: make dashboard realtime 2026-05-30 04:17:10 +00:00
Matthew Meszaros 99d2bff0cb feat: unify dev-mode hot-reload for go, rust, elixir under make dev 2026-05-25 03:21:42 +00:00
Matthew Meszaros ce870a15b3 ci: fix Elixir formatting / Elixir version / missing pnpm lockfile 
- realtime/: `mix format` applied; long Logger calls reformatted across
  config/runtime/application/connections/user_channel/user_socket/
  endpoint. CI's "Check formatting" step now passes.
- CI Elixir bumped from 1.16 → 1.18 (with OTP 27) to match mix.exs's
  `~> 1.18` requirement. Phoenix 1.8.7 + plug 1.19 also expect this.
- web/: generate + commit pnpm-lock.yaml so actions/setup-node@v4's
  pnpm cache step + `pnpm install --frozen-lockfile` can resolve.
 2026-05-23 16:17:27 +00:00
Matthew Meszaros d8bb10bf8f ci: fix Bad-credentials + bump deps to clear Trivy CVEs 
Workflow:
- Add explicit `permissions: contents: read, pull-requests: read`
  so dorny/paths-filter can list PR files via the GitHub API. Without
  it the "Detect Changes" job dies with "Bad credentials" on PRs and
  every downstream language CI gets skipped.

Go:
- google.golang.org/grpc v1.78.0 → v1.79.3 (CVE-2026-33186 — HTTP/2
  path validation authorization bypass).

Elixir (realtime):
- cowboy 2.14.2 → 2.15.0 (CVE-2026-8466)
- cowlib 2.16.0 → 2.16.1 (CVE-2026-43970, CVE-2026-7790)
- phoenix 1.8.3 → 1.8.7 (CVE-2026-32689 — long-poll memory blow-up)
- plug 1.19.1 → 1.19.2 (CVE-2026-8468 — multipart header overflow)
- plug_cowboy 2.7.5 → 2.8.1 (CVE-2026-32688 — unauth DoS)
- postgrex 0.22.0 → 0.22.2 (CVE-2026-32687 — channel-name SQLi)

Rust (tracking):
- aws-lc-rs 1.15.4 → 1.17.0 (pulls aws-lc-sys to 0.41.0 — fixes
  GHSA-394x-vwmw-crm3, GHSA-65p9-r9h6-22vj, GHSA-9f94-5g5w-gf6r,
  GHSA-hfpc-8r3f-gw53, GHSA-vw5v-4f2q-w9xf)
- openssl 0.10.75 → 0.10.80 (CVE-2026-41676/8/81/898, -42327)
- rustls-webpki 0.103.9 → 0.103.13

- Add .trivyignore for GHSA-82j2-j2ch-gfr8 on the old rustls-webpki
  0.101.7 path that aws-smithy-http-client / hyper-rustls 0.24 still
  pulls in. AWS SDK hasn't migrated to rustls 0.23+ yet; the CRL
  parsing path the advisory covers isn't reachable from our usage
  (SSM + Secrets Manager at startup over the public CA chain).
 2026-05-23 16:13:39 +00:00
Matthew Meszaros 695e2b5a33 fix: contacts crash, campaigns panic, websocket — 4 distinct bugs found while triaging the page-blank symptom 
1) Contacts crash "c is null":
   contactRepository.Search declared `var contacts []models.Contact` so
   an empty result set returned a nil slice, which Go marshals as JSON
   null. The frontend's flatMap((p) => p.data) over null yields [null],
   and the page then accesses c.subscribed → throws. Initialize as
   make([]models.Contact, 0, limit+1) so the wire format is always [].
   Also defensive on the client: useSearchContacts + useCampaigns now
   coerce p.data ?? [] and drop nulls before returning.

2) Campaigns panic on any non-empty result:
   campaignRepository.Search allocated `make([]models.Campaign, 0, limit+1)`
   (length 0) then did `campaigns[i] = campaign`. That's an
   index-out-of-range on the first iteration. Switched to `append`.
   Anyone with at least one campaign would see a 500 / blank screen.

3) Websocket "Token expired":
   SocketTTL was 60s. The frontend reconnect backoff caps at 30s, so
   after a rejected handshake the next attempt could fire 30-60s
   later. Combined with rare back-pressure on /getaway the token was
   already past exp by the time the realtime saw it. Bumped to 10 min
   — short enough to keep the token low-impact, long enough to outlast
   the backoff schedule.

4) Websocket "Connection limit exceeded":
   Realtime.Connections only untracked on channel terminate, never on
   socket disconnect. Sockets that connected and disconnected without
   joining a channel leaked. Each reconnect loop bumped the counter
   until the per-user limit (10) was hit, after which every legitimate
   connect was rejected even after fixing #3.
   Fix: GenServer Process.monitor's the socket pid on track, and
   `:DOWN` handler calls do_untrack with the right (user_id, ip).

5) Phoenix protocol mismatch:
   Frontend appended vsn=2.0.0 to the WS URL, but sendRaw + joinChannel
   send the V1 object format. Realtime's Phoenix.Socket.V2.JSONSerializer
   crashed with a badmatch on the first phx_join, killing the socket
   right after connect. Switched to vsn=1.0.0 to match what the client
   actually emits.
 2026-05-23 05:32:28 +00:00
Matthew Meszaros c12fd93def Add local Sentry logging 2026-02-14 05:49:49 +01:00
Matthew Meszaros 141bc54974 Add sample auth UI theme 2026-02-10 19:30:47 +01:00
Máté Mészáros (Laptop) ed35ab2dbc Realtime Updates 2026-01-30 15:32:58 +01:00
Máté Mészáros (Laptop) 6adb4cdd5a Organization, Subscription, Inqueries, limits. 2026-01-27 05:55:48 +01:00
Máté Mészáros (Laptop) 5ac159d2f8 Realtime, api keys & more 2026-01-26 16:04:42 +01:00
Máté Mészáros (Laptop) 7ae36cc689 Convert realtime submodule to regular directory 2026-01-26 04:50:21 +01:00
Máté Mészáros (Laptop) 81d5970ea8 Realtime, Task Handler, Worker & Consumer Setup 2026-01-26 04:42:19 +01:00