Files
windmill/docs/feature-telemetry.md
T
ef5fa21603 feat: add a share dialog and a first-deploy dialog (#11950)
* feat: harmonize flow, script and app settings drawers

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: name the on-behalf-of setting by the access it grants

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: add a permissions tab and confirm who apps run on behalf of

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: sandbox draft-only app deploys and keep app settings out of exports

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep drafted run-as users and custom url removals

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm the run-as user on every deploy that uses one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: let new raw apps take the folder default and follow synced settings

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: add a share dialog next to deploy and a first-deploy dialog

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: start AI-session raw apps without a run-as user

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mock the deployed app read in chat deploy tests and type the list height

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: point at the ee branch that registers the sharing counters

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: share the deployed path, log first deploys once and report the kept app access

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: stop drafting an app's access mode and custom url

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep a deployed app's access mode on an mcp deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: ask who a first deploy runs as in the first-deploy dialog's second step

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: drop the first-deploy footer rule and test the run-as question

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: let a flow or script deploy turn off run on behalf of

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: apply who can open an app on change, as on main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mute the error handler from the editor menu and keep it on deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: rename the Permissions entries and drawer to Share

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm deploys that move an item to another folder

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change only the access mode when it applies on click

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the mute flag from the deployed row at its own path

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read a raw app's access mode after bundling it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep group members off the list of who a move takes access from

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change an app's access mode under the deploy row lock

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: defer a mode change to deploy on an app run as another user

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: hold the access toggle while its change is in flight

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: bind the app policy down to the settings drawer

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep app draft labels in the mcp session tools

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* revert: leave the deploy panel's access toggle to the settings drawer PR

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change only the access mode on click and let deploys keep the deployed one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send Share's mode change alone and keep it out of MCP deploys

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the caller's draft on a mode-only app update

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: leave the access mode out of every update deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: explain in Share what an item's runs can reach

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: describe a viewer-mode app as using its viewer's access

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm Share's app access changes with a toast

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: refuse a builder's mode-only change on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: disable Share's access mode for a builder on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: check the reason for the unsandboxed mode-only refusal

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: defer a builder's access-mode change on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: state the stored access mode in the chat's raw app deploy note

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: state the stored access mode in the MCP raw app deploy note

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: lock the access mode when the server would refuse the change

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: address settings and run-as review findings

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: say to deploy first when Share's access mode is locked

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: hold the run-as confirmation until the folder default has loaded

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the run-as dialog cancelable while the folder default loads

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: harmonize flow and script settings layout

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: name who can change a locked access mode without taking the app over

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the priority input open while it is edited

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm run-as on flow and script deploy, mute from editor menu

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep settings rows open while edited and name the deploying user for non-admins

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: reread the deployed run-as user after a deploy that keeps the editor open

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: say in Share that admins and deployers can change a locked access mode

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: tell a builder who keeps the run-as user to pick themselves before deploying

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: write a setting before its row opens and show its real state while edited

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: slide settings rows open to their full height

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: open Share from the menu of Home's favorite and recent cards

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send the popover's deployment message when Deploy is clicked

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: remount a settings row's fields when its toggle turns it on

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send the popover's deployment message from Deploy & Stay here

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: count Share opened from a Home card under its own telemetry key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: clear the deployment message after a successful deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: show the stored access mode in Share after a refused change

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep app access mode display in sync with the stored mode and badge app settings tabs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: clear the app Deploy popover's message after a successful deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the deployed row before muting from the menu, title Deploy only when icon-only

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: give the app settings badges the chip label and row key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mark the app labels and access rows so their badges reach them

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: show the app settings as three sections instead of tabs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop unused publishAccessMode and app sharing badges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: update ee-repo-ref to d16494953659e00dd169fb8f73266f22f937b862

This commit updates the EE repository reference after PR #911 was merged in windmill-ee-private.

Previous ee-repo-ref: 07f0090acb4a1f206c37f34411819b8c86690388

New ee-repo-ref: d16494953659e00dd169fb8f73266f22f937b862

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-10-29 19:28:41 +01:00

117 lines
6.0 KiB
Markdown

# Feature usage telemetry
`feature_usage` is the product-telemetry accumulator: day-bucketed counters that roll into the
anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick"
without any identifying data leaving the instance.
It currently carries 93 registered actions across thirty-six features (`ai_session`, `ai_chat`,
`ai_fix`, `ai_agent`, `ai_decision`, `ai_agent_eval`, `app_sandbox`, `datatable`, `db_manager`, `flow_editor`,
`flow_run`, `flow_step`, `home`, `header_nav`, `run_form`, `sharing`, `debugger`, `trigger`,
`command_script`, `mcp`, `mcp_workspace`, `hub_script`, `usage_meter`, `sso_groups_claim`,
`cloud_trial_offer`, `job_token_scopes`, `worker_draining`, `worker_group`,
`worker_resource_stats`, `dev_workspace_isolation`, `slack_agent`, `variable_expiration`,
`anthropic_federation`, `deployment_request`, `fork_roles`, `git_sync_review`). Nearly all of the
product is uninstrumented, so new user-facing work is the opportunity to change that.
## When to instrument
Raise it **in the plan**, with the concrete vocabulary written out, and let the user keep or drop
it in one line. Don't stop and ask as a standalone question.
Propose it when a new user-facing affordance leaves a real question open:
- a new panel, mode, tab, toggle, or entry point — is it discovered and used at all?
- competing UX paths, or a new default — which one wins?
- an opt-in or beta gate — what is the take rate?
- a multi-step flow — where do people stop?
Stay silent for bugfixes, refactors, internal plumbing, and anything whose useful signal would
need per-item identifiers (paths, names, prompts, code) — those cannot be logged at all, see
[Privacy rules](#privacy-rules). If the answer wouldn't change a decision, instrumenting is
overkill; say nothing.
## Designing the vocabulary
| Field | Meaning | Limits |
|---|---|---|
| `feature` | Product area: `ai_chat`, `flow_editor` | ≤50 chars |
| `kind` | The action within it: `message`, `panel_placement`. `(feature, kind)` is the allowlisted pair | ≤50 chars |
| `key` | A facet of the action — mode, tab kind, tool name, `provider:model`. Aggregation groups by `(feature, kind, key)`, so this is what splits one counter into comparable buckets | ≤100 chars, identifier-shaped, optional |
| `entity_id` | An **opaque random** id (e.g. a session id) when you need per-entity distributions rather than a flat count | ≤50 chars, identifier-shaped, optional |
| `value` | Increment, default 1 | clamped to 1…1,000,000 |
Identifier-shaped means ASCII alphanumerics plus `_ - : . /` — no spaces. Anything else is
rejected.
Supplying `entity_id` is what unlocks the distribution stats: the payload reports `entity_count`,
`total_value`, `median_value`, `p90_value`, and `inactive_3d_entity_count` per
`(feature, kind, key)`. Omit it for a plain "how many times did this happen" counter. Keep the key
vocabulary closed and small — enumerate the values in a TS union next to the call site, the way
`flowEditorTelemetry.ts` does, so the whole set is reviewable in one place.
## The recipe
Four steps. Skipping step 1 fails quietly.
**1. Register the pair** in `FEATURE_USAGE_KINDS`
(`backend/windmill-common/src/feature_usage_ee.rs`, tracked in `windmill-ee-private`). An
unregistered `(feature, kind)` is dropped by `is_recordable_event` with a bare `continue` — no
error, no log, still a 204 to the browser. Frontend-only instrumentation records **nothing** and
looks like it worked.
**2. Log from the frontend:**
```ts
import { logFeatureUsage } from '$lib/utils/featureUsage'
logFeatureUsage('flow_editor', 'panel_placement', { key: 'force_detach' })
```
Fire-and-forget. Events sum locally per `(workspace, feature, kind, key, entityId)` and flush
every 30s, on `visibilitychange` → hidden, and on `pagehide`; 50 events per request, and a failed
batch is dropped rather than retried.
**3. Check the disclosure.** `InstanceSettings.svelte` describes a non-minimal payload by
category, not counter by counter (two places — the copy appears twice); keep it short. A plain
counter needs no edit. Only a key that sends a new kind of value beyond a count — a name or an
identifier, like the AI model and hub names it already lists — must be added there.
**4. Verify a row lands.** The silent-drop path means "no error" proves nothing:
```sql
SELECT feature, kind, key, entity_id, day, value FROM feature_usage ORDER BY updated_at DESC LIMIT 10;
```
Collection sits behind the `private` feature, so a public build records nothing from either the
HTTP route or the Rust helper. Run the backend with `--features enterprise,private` or this query
stays empty however correct the instrumentation is.
## Privacy rules
Only aggregated counts ever leave the instance, and only when telemetry is enabled and minimal
mode is off. Never put a path, prompt, script body, workspace name, email, or any user identifier
into `key` or `entity_id`. Entity ids must be opaque random ids, never anything that maps back to
a user or a resource. If the signal you want can only be expressed with identifying data, it
cannot be collected — drop it.
Counters aggregate over the last 30 days; rows are pruned after 60.
## Logging from the backend
A feature with no UI is instrumented the same way, from Rust:
```rust
windmill_common::feature_usage::log_feature_usage("trigger", "fired", kind.as_str());
```
Same registry, same key rules, and the same silent drop when the pair is unregistered. `feature`
and `kind` are `&'static str` so a call site cannot pass a computed pair. The call increments an
in-memory counter and returns; the monitor loop flushes the accumulator, so it is cheap enough for
hot paths — but only cheap per call, not free: a key with unbounded cardinality would grow the map
until it hits the per-action cap and starts dropping new keys.
There is no `entity_id` and no explicit `value` on this path: it counts occurrences.
`feature_usage_ee` holds the registry and the writer; the public build gets the inert
`feature_usage_oss`, since a CE instance never sends a stats payload.