hugocasa
cbcdcbd28b
Three recurring gaps, all cases where a pointer existed but nothing triggered on it. Component reuse. The svelte-frontend skill documented three components with props, which reads as the whole catalog; the barrel exports 23 and common/ has 34 subdirectories against those 23. So "never use raw HTML elements" was an instruction agents could not follow. Added a mandatory discovery step: read the barrel, grep the tree, and treat the documented three as examples. Brand guidelines. frontend/brand-guidelines.md is 34k characters referenced by bare path, which nothing opens speculatively. Added a table mapping what you are building to the section that governs it, entered with grep rather than a full read. Product telemetry. feature_usage has 14 registered actions across three features, and an unregistered (feature, kind) pair is dropped by valid_feature_usage_event with a bare continue — no error, still a 204 — so frontend-only instrumentation silently records nothing. New docs/feature-telemetry.md carries the criteria for when to instrument, the four-step recipe including the allowlist and the InstanceSettings disclosure, and the privacy rules. Raised in the plan for user-facing work, not as a separate question, and not at all for bugfixes or refactors. Also: validation now ends at exercising the change on the running instance, with standing permission to spin up whatever that takes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4.5 KiB
Feature usage telemetry
feature_usage is the product-telemetry accumulator: day-bucketed counters that roll into the
anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick"
without any identifying data leaving the instance.
It currently carries 14 registered actions across three features (ai_session, ai_chat,
flow_editor). Nearly all of the product is uninstrumented, so new user-facing work is the
opportunity to change that.
When to instrument
Raise it in the plan, with the concrete vocabulary written out, and let the user keep or drop it in one line. Don't stop and ask as a standalone question.
Propose it when a new user-facing affordance leaves a real question open:
- a new panel, mode, tab, toggle, or entry point — is it discovered and used at all?
- competing UX paths, or a new default — which one wins?
- an opt-in or beta gate — what is the take rate?
- a multi-step flow — where do people stop?
Stay silent for bugfixes, refactors, internal plumbing, and anything whose useful signal would need per-item identifiers (paths, names, prompts, code) — those cannot be logged at all, see Privacy rules. If the answer wouldn't change a decision, instrumenting is overkill; say nothing.
Designing the vocabulary
| Field | Meaning | Limits |
|---|---|---|
feature |
Product area: ai_chat, flow_editor |
≤50 chars |
kind |
The action within it: message, panel_placement. (feature, kind) is the allowlisted pair |
≤50 chars |
key |
A facet of the action — mode, tab kind, tool name, provider:model. Aggregation groups by (feature, kind, key), so this is what splits one counter into comparable buckets |
≤100 chars, identifier-shaped, optional |
entity_id |
An opaque random id (e.g. a session id) when you need per-entity distributions rather than a flat count | ≤50 chars, identifier-shaped, optional |
value |
Increment, default 1 | clamped to 1…1,000,000 |
Identifier-shaped means ASCII alphanumerics plus _ - : . / — no spaces. Anything else is
rejected.
Supplying entity_id is what unlocks the distribution stats: the payload reports entity_count,
total_value, median_value, p90_value, and inactive_3d_entity_count per
(feature, kind, key). Omit it for a plain "how many times did this happen" counter. Keep the key
vocabulary closed and small — enumerate the values in a TS union next to the call site, the way
flowEditorTelemetry.ts does, so the whole set is reviewable in one place.
The recipe
Four steps. Skipping step 1 or 3 fails quietly.
1. Register the pair in FEATURE_USAGE_KINDS
(backend/windmill-api-workspaces/src/workspaces.rs). An unregistered (feature, kind) is
dropped by valid_feature_usage_event with a bare continue — no error, no log, still a 204 to
the browser. Frontend-only instrumentation records nothing and looks like it worked.
2. Log from the frontend:
import { logFeatureUsage } from '$lib/utils/featureUsage'
logFeatureUsage('flow_editor', 'panel_placement', { key: 'force_detach' })
Fire-and-forget. Events sum locally per (workspace, feature, kind, key, entityId) and flush
every 30s, on visibilitychange → hidden, and on pagehide; 50 events per request, and a failed
batch is dropped rather than retried.
3. Update the disclosure. InstanceSettings.svelte lists what a non-minimal payload contains
(two places — the copy appears twice). A new counter that isn't named there means the instance
under-discloses what it sends. This has already drifted once.
4. Verify a row lands. The silent-drop path means "no error" proves nothing:
SELECT feature, kind, key, entity_id, day, value FROM feature_usage ORDER BY updated_at DESC LIMIT 10;
Privacy rules
Only aggregated counts ever leave the instance, and only when telemetry is enabled and minimal
mode is off. Never put a path, prompt, script body, workspace name, email, or any user identifier
into key or entity_id. Entity ids must be opaque random ids, never anything that maps back to
a user or a resource. If the signal you want can only be expressed with identifying data, it
cannot be collected — drop it.
Counters aggregate over the last 30 days; rows are pruned after 60.
Backend-only features
Ingestion is frontend-only: log_feature_usage is an HTTP route the browser posts to, and there
is no Rust-side helper. A feature with no UI cannot be instrumented today without adding one.
Scope the default to user-facing work, and say so rather than implying backend coverage exists.