Files
warmbly/internal/app/advisor/advisor.go
T
Matthew Meszaros 5e6287c920 feat: add the Advisor, continuous sending checks surfaced on the row they are about (#86)
* feat: index advisor findings by subject and parent entity so a list page fetches its whole surface once and every row resolves its own advice from the shared cache instead of firing a request per row

* feat: rebuild the advisor fix drawer as a three-screen resolution flow (why it fired with the measured evidence, the exact before and after, then an animated outcome with undo) with a progress rail and direction-aware transitions, and deep-link manual fixes to the screen where they are made

* feat: add AdvisorRowFlag, the inline per-row advisor indicator that renders on the mailbox or campaign the problem is about and opens that row's findings in an anchored panel instead of making the reader join a card list against a table

* feat: add AdvisorSummaryBar, a one-line collapsible page summary that replaces the stack of advisor cards above a list, counts the distinct rows implicated rather than the findings, and forces itself open only for critical or workspace-level advice no row flag can carry

* feat: put advisor advice on the mailbox row it is about in the accounts list, replace the card stack above the table with the collapsible summary bar, and support ?mailbox=<id> so a finding can deep-link straight to the mailbox detail instead of the top of the list

* feat: flag advisor findings on the campaign row in the campaigns list, including step-level copy problems which index onto their parent campaign since a step has no row of its own, and add the collapsible summary bar above the list

* feat: move the deliverability and contacts pages onto the collapsible advisor summary bar so their findings stop pushing the numbers they describe below the fold

* feat: add an ordered Steps field to advisor findings, persisted as text[] and always refreshed from the current build, and write real how-to steps for the deliverability checks that have no one-click fix (bounce rate, spam placement, tracking domain, and per-record SPF/DKIM/DMARC instructions)

* feat: write ordered how-to steps for the manual advisor findings where the remedy alone leaves someone stuck (broken template syntax, missing first-name fallback, unsubscribed contacts still enrolled, a campaign with no resolvable sender, and a mailbox that lost warmup pool standing) and correct the personalization detail that named a merge syntax this product does not use

* feat: show a mailbox's advisor findings at the top of its detail drawer, which is where both the row flag and the ?mailbox deep link now land

* feat: open the resolution flow from findings that have no one-click fix too, since the ordered how-to lives there and a card with no Fix button previously left the steps unreachable

* docs: document the per-row advisor flags, the collapsible page summary, the three-screen resolution flow, and the ordered manual steps for findings with no one-click fix

* feat: align the advisor summary bar to the px-5 page gutter used by SectionBar and the list rows on all four surfaces, instead of sitting flush against the edge while the table it describes is indented

* fix: stop the resolution drawer collapsing to zero height between screens by switching the step transition to popLayout with a layout-animated container, so the dialog resizes into the next screen instead of snapping shut and reopening

* feat: wire the advisor repository, narrator, service, tool registration, and background runner into the backend boot path so findings evaluate on a schedule and the assistant can read them

* docs: register the advisor guide in the sidebar, add its endpoint scope table to the API reference, and document the sandbox advisor showcase

* fix: darken the advisor nav badge to solid orange-600 on white instead of a pale amber-100 chip that read as a disabled control beside the sidebar's saturated indicators, and drop the critical badge to rose-600 so the two stay in the same weight class

* fix: use orange-500 for the advisor nav badge, matching the high-severity dot on the row it points at, rather than the darker orange-600

* feat: add an Auto safety class to advisor actions and mark the seven fixes autopilot may apply unattended (the cap cuts, the send-gap widen, the campaign limit matches, and the unsubscribe header), with a test pinning the boundary so nothing that halts sending or generates new outbound mail can drift into it

* feat: add advisor autopilot, which applies the auto-safe fixes unattended as the member who switched it on, resolving their live permissions each run so it fails closed when they leave the org, bounded to 10 changes per evaluation and audited per fix like any hand-made change

* feat: add the advisor agent fix, a bounded per-finding agent run that resolves the problems a settings change cannot (broken template syntax, bulk-reading copy, shared-inbox lists) as the calling member inside a tool allowlist scoped to the finding's category, metered per iteration and marked applied only when it actually called a write tool

* feat: surface autopilot and the agent fix in the dashboard, adding the workspace toggle that names exactly which changes it may make, an Auto chip on the findings it is allowed to take, and an agent-fix path in the resolution drawer that reports the tools it actually called rather than only its own account of them

* docs: document the agent fix and autopilot, naming the exact set of changes autopilot may make, that it acts as the member who enabled it and stops when they leave, and why the agent-fix endpoint is JWT only

* fix: gate the agent fix per detector instead of per category, so a missing DMARC record no longer offers a Fix-with-agent button it can never satisfy and then reports failure; findings whose fix lives in DNS or a provider console now show their manual steps, and the client is told which is which via agent_fixable

* feat: soften the advisor surfaces to translucent washes, replacing the filled nav badge with a tinted pill that carries its colour in the text, frosting the row panel and the resolution drawer, and turning the severity chips and cards into layers the page shows through

* docs: correct the agent-fix scope to name the findings it cannot resolve, and why a DNS record shows steps instead of a button

* feat: ship the actual DNS records for the findings that live outside the platform, with the provider's SPF include resolved, the DMARC record scoped to the sending domain and starting at p=none, the DKIM host plus the console that generates its value, and a tracking CNAME pointing at this install's own tracking host

* feat: render advisor snippets as labelled copy-button rows so a DNS record is one click per field rather than a text-selection exercise, with no copy affordance on a value the server could not supply

* docs: document the pasteable DNS records and the guarantee that every check offers a fix, an agent, or ordered steps

* fix: bump golang.org/x/text to 0.39.0 to clear CVE-2026-56852, a HIGH-severity infinite loop in norm.Iter that Trivy started failing the security scan on
2026-07-30 17:15:09 +02:00

301 lines
9.7 KiB
Go

// Package advisor continuously evaluates an organization's sending posture and
// turns what it finds into specific, fixable advice.
//
// The split is deliberate:
//
// - Detection is pure Go over a single snapshot (see repository.LoadSnapshot).
// Detectors are total functions with no I/O, no model calls, and explicit
// sample floors, so the same data always produces the same findings and the
// Advisor keeps working with AI switched off entirely.
// - Narration is the only AI step. It rewrites a finding's title/detail/remedy
// from the evidence the detector already computed, in the org's voice, and
// is cached per (detector, evidence shape). It can never invent a finding,
// change a severity, or alter an action.
// - Remediation runs through the shared aitools registry as the invoking user,
// after they confirm a rendered before/after preview.
//
// Every threshold here traces to the sending-safety policy in CLAUDE.md or to
// the provider guidance it cites (Google's 0.1%/0.3% complaint bands, SES's 5%
// /10% bounce bands). When they disagree, the stricter one wins: a shared
// warmup pool has to act before the mailbox providers do.
package advisor
import (
"encoding/json"
"fmt"
"sort"
"github.com/google/uuid"
"github.com/warmbly/warmbly/internal/models"
"github.com/warmbly/warmbly/internal/repository"
)
// Finding is one detector's output before it is persisted. The Title/Detail/
// Remedy written here are the deterministic fallback: complete, specific, and
// good enough to ship on their own. Narration only makes them warmer.
type Finding struct {
Key string
Category models.AdvisorCategory
Severity models.AdvisorSeverity
Surface models.AdvisorSurface
EntityType string
EntityID *uuid.UUID
EntityLabel string
// ParentType / ParentID let a finding show up on the page someone would
// actually look at: a step's copy problem is listed on its campaign.
ParentType string
ParentID *uuid.UUID
// Impact ranks findings of equal severity (0-100). Detectors set it from
// the size of the thing at stake — volume at risk, contacts affected.
Impact int
Title string
// GroupTitle is how this finding names itself when several of its kind are
// shown together, with a {count} placeholder. Set it on any detector that
// can plausibly fire on many entities at once: without it, one
// misconfiguration repeated across a fleet becomes a wall of identical
// cards that people learn to scroll past.
GroupTitle string
Detail string
Remedy string
// Steps is the ordered how-to for a finding the platform cannot fix itself.
// Set it wherever there is no Action: "open the mailbox and work through the
// errors" is a summary, not an instruction, and the person reading it is
// usually the one who does not already know the answer.
Steps []string
// Snippets are the exact values to paste, for fixes that live outside the
// platform. A DNS step without the record is a step that ends in a search.
Snippets []models.AdvisorSnippet
// Evidence is the numbers the detector fired on. It is shown in the card,
// hashed for narration caching, and is the ONLY input the narrator gets, so
// it must be self-contained and already rounded into bands.
Evidence map[string]any
Action *models.AdvisorAction
}
// Fingerprint is the finding's stable identity across runs.
func (f Finding) Fingerprint() string {
entity := "org"
if f.EntityID != nil {
entity = f.EntityType + ":" + f.EntityID.String()
}
return f.Key + "|" + entity
}
// toModel converts a detector finding into its persistable form.
func (f Finding) toModel(orgID uuid.UUID) *models.AdvisorFinding {
evidence, err := json.Marshal(f.Evidence)
if err != nil || len(f.Evidence) == 0 {
evidence = []byte(`{}`)
}
return &models.AdvisorFinding{
OrganizationID: orgID,
Fingerprint: f.Fingerprint(),
DetectorKey: f.Key,
Category: f.Category,
Severity: f.Severity,
Surface: f.Surface,
EntityType: f.EntityType,
EntityID: f.EntityID,
EntityLabel: f.EntityLabel,
ParentType: f.ParentType,
ParentID: f.ParentID,
Impact: f.Impact,
Title: f.Title,
GroupTitle: f.GroupTitle,
Detail: f.Detail,
Remedy: f.Remedy,
Steps: f.Steps,
Snippets: f.Snippets,
Evidence: json.RawMessage(evidence),
Action: f.Action,
}
}
// Detector is one named check.
type Detector struct {
Key string
Category models.AdvisorCategory
// About is the one-line statement of what this detector looks for and why
// it matters. It grounds the narrator so the copy explains the real rule
// rather than paraphrasing the numbers back at the user.
About string
Run func(s *repository.AdvisorSnapshot) []Finding
}
// AllDetectors returns every detector in evaluation order. Order does not
// affect correctness (findings are sorted by severity for display) but keeping
// the categories grouped makes the run log readable.
func AllDetectors() []Detector {
var all []Detector
all = append(all, deliverabilityDetectors()...)
all = append(all, mailboxDetectors()...)
all = append(all, warmupDetectors()...)
all = append(all, campaignDetectors()...)
all = append(all, copyDetectors()...)
all = append(all, listDetectors()...)
return all
}
// DetectorAbout indexes the detector descriptions by key, for the narrator.
func DetectorAbout() map[string]string {
out := map[string]string{}
for _, d := range AllDetectors() {
out[d.Key] = d.About
}
return out
}
// Detect runs every detector that the org's settings leave enabled and returns
// the findings sorted most-urgent-first.
func Detect(s *repository.AdvisorSnapshot, settings *models.AdvisorSettings) []Finding {
muted := map[string]bool{}
for _, k := range settings.MutedDetectors {
muted[k] = true
}
mutedCat := map[string]bool{}
for _, c := range settings.MutedCategories {
mutedCat[c] = true
}
out := []Finding{}
for _, d := range AllDetectors() {
if muted[d.Key] || mutedCat[string(d.Category)] {
continue
}
for _, f := range d.Run(s) {
if !f.Severity.AtLeast(settings.MinSeverity) {
continue
}
// Detectors declare their own category only when it differs from
// their registration, which no detector currently does; defaulting
// here keeps a new detector from silently landing in "".
if f.Category == "" {
f.Category = d.Category
}
if f.Key == "" {
f.Key = d.Key
}
out = append(out, f)
}
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Severity.Rank() != out[j].Severity.Rank() {
return out[i].Severity.Rank() > out[j].Severity.Rank()
}
return out[i].Impact > out[j].Impact
})
return out
}
// --- shared helpers -------------------------------------------------------
// rate returns n/total as a percentage, or 0 when there is nothing to divide by.
func rate(n, total int) float64 {
if total <= 0 {
return 0
}
return float64(n) / float64(total) * 100
}
// band rounds a rate to a coarse bucket for the evidence hash, so ordinary
// drift (4.1% -> 4.3% bounce) does not invalidate cached narration while a real
// move (4% -> 9%) does.
func band(v float64) float64 {
switch {
case v >= 10:
return float64(int(v/5) * 5)
case v >= 1:
return float64(int(v))
default:
return float64(int(v*20)) / 20
}
}
// clampImpact keeps a detector's impact weight in range.
func clampImpact(v int) int {
if v < 0 {
return 0
}
if v > 100 {
return 100
}
return v
}
// pct formats a rate for the fallback copy.
func pct(v float64) string {
if v < 1 {
return fmt.Sprintf("%.2f%%", v)
}
return fmt.Sprintf("%.1f%%", v)
}
// ref returns a pointer to a UUID, for Finding.EntityID.
func ref(id uuid.UUID) *uuid.UUID { return &id }
// mailboxAction builds an update_mailbox one-click fix with its preview.
func mailboxAction(id uuid.UUID, label string, args map[string]any, preview ...models.AdvisorPreviewChange) *models.AdvisorAction {
args["email_account_id"] = id.String()
return toolAction("update_mailbox", label, args, preview...)
}
// campaignAction builds an update_campaign one-click fix with its preview.
func campaignAction(id uuid.UUID, label string, args map[string]any, preview ...models.AdvisorPreviewChange) *models.AdvisorAction {
args["campaign_id"] = id.String()
return toolAction("update_campaign", label, args, preview...)
}
func toolAction(tool, label string, args map[string]any, preview ...models.AdvisorPreviewChange) *models.AdvisorAction {
raw, err := json.Marshal(args)
if err != nil {
return nil
}
return &models.AdvisorAction{
Tool: tool,
Args: json.RawMessage(raw),
Label: label,
Preview: preview,
}
}
// withUndo attaches the reverting tool call to an action, so an applied fix can
// be rolled back from the card without hunting for the original value.
func withUndo(a *models.AdvisorAction, args map[string]any) *models.AdvisorAction {
if a == nil {
return nil
}
raw, err := json.Marshal(args)
if err != nil {
return a
}
a.Undo = &models.AdvisorUndo{Tool: a.Tool, Args: json.RawMessage(raw)}
return a
}
// auto marks a fix as safe for autopilot to apply on its own.
//
// The bar is deliberately narrow. It must be a bounded settings change, it must
// move in the direction that reduces risk, and it must already carry an Undo.
// Pausing a mailbox qualifies on risk and reverts cleanly, but it stops a
// customer's sending without warning, so it is not on this list: autopilot is
// for the changes nobody would argue with.
func auto(a *models.AdvisorAction) *models.AdvisorAction {
if a == nil || a.Undo == nil {
return a
}
a.Auto = true
return a
}
// change is shorthand for one preview line.
func change(field, from, to string) models.AdvisorPreviewChange {
return models.AdvisorPreviewChange{Field: field, From: from, To: to}
}