Files
windmill/docs/feature-telemetry.md
T
ef5fa21603 feat: add a share dialog and a first-deploy dialog (#11950)
* feat: harmonize flow, script and app settings drawers

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: name the on-behalf-of setting by the access it grants

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: add a permissions tab and confirm who apps run on behalf of

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: sandbox draft-only app deploys and keep app settings out of exports

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep drafted run-as users and custom url removals

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm the run-as user on every deploy that uses one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: let new raw apps take the folder default and follow synced settings

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: add a share dialog next to deploy and a first-deploy dialog

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: start AI-session raw apps without a run-as user

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mock the deployed app read in chat deploy tests and type the list height

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: point at the ee branch that registers the sharing counters

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: share the deployed path, log first deploys once and report the kept app access

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: stop drafting an app's access mode and custom url

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep a deployed app's access mode on an mcp deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: ask who a first deploy runs as in the first-deploy dialog's second step

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: drop the first-deploy footer rule and test the run-as question

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: let a flow or script deploy turn off run on behalf of

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: apply who can open an app on change, as on main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mute the error handler from the editor menu and keep it on deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: rename the Permissions entries and drawer to Share

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm deploys that move an item to another folder

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change only the access mode when it applies on click

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the mute flag from the deployed row at its own path

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read a raw app's access mode after bundling it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep group members off the list of who a move takes access from

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change an app's access mode under the deploy row lock

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: defer a mode change to deploy on an app run as another user

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: hold the access toggle while its change is in flight

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: bind the app policy down to the settings drawer

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep app draft labels in the mcp session tools

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* revert: leave the deploy panel's access toggle to the settings drawer PR

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: change only the access mode on click and let deploys keep the deployed one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send Share's mode change alone and keep it out of MCP deploys

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the caller's draft on a mode-only app update

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: leave the access mode out of every update deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: explain in Share what an item's runs can reach

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: describe a viewer-mode app as using its viewer's access

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm Share's app access changes with a toast

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: refuse a builder's mode-only change on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: disable Share's access mode for a builder on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: check the reason for the unsandboxed mode-only refusal

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: defer a builder's access-mode change on an unsandboxed app

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: state the stored access mode in the chat's raw app deploy note

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: state the stored access mode in the MCP raw app deploy note

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: lock the access mode when the server would refuse the change

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: address settings and run-as review findings

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: say to deploy first when Share's access mode is locked

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: hold the run-as confirmation until the folder default has loaded

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the run-as dialog cancelable while the folder default loads

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: harmonize flow and script settings layout

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: name who can change a locked access mode without taking the app over

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the priority input open while it is edited

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: confirm run-as on flow and script deploy, mute from editor menu

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep settings rows open while edited and name the deploying user for non-admins

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: reread the deployed run-as user after a deploy that keeps the editor open

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: say in Share that admins and deployers can change a locked access mode

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: tell a builder who keeps the run-as user to pick themselves before deploying

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: write a setting before its row opens and show its real state while edited

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: slide settings rows open to their full height

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: open Share from the menu of Home's favorite and recent cards

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send the popover's deployment message when Deploy is clicked

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: remount a settings row's fields when its toggle turns it on

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: send the popover's deployment message from Deploy & Stay here

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: count Share opened from a Home card under its own telemetry key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: clear the deployment message after a successful deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: show the stored access mode in Share after a refused change

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep app access mode display in sync with the stored mode and badge app settings tabs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: clear the app Deploy popover's message after a successful deploy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the deployed row before muting from the menu, title Deploy only when icon-only

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: give the app settings badges the chip label and row key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: mark the app labels and access rows so their badges reach them

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: show the app settings as three sections instead of tabs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop unused publishAccessMode and app sharing badges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: update ee-repo-ref to d16494953659e00dd169fb8f73266f22f937b862

This commit updates the EE repository reference after PR #911 was merged in windmill-ee-private.

Previous ee-repo-ref: 07f0090acb4a1f206c37f34411819b8c86690388

New ee-repo-ref: d16494953659e00dd169fb8f73266f22f937b862

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-10-29 19:28:41 +01:00

6.0 KiB

Feature usage telemetry

feature_usage is the product-telemetry accumulator: day-bucketed counters that roll into the anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick" without any identifying data leaving the instance.

It currently carries 93 registered actions across thirty-six features (ai_session, ai_chat, ai_fix, ai_agent, ai_decision, ai_agent_eval, app_sandbox, datatable, db_manager, flow_editor, flow_run, flow_step, home, header_nav, run_form, sharing, debugger, trigger, command_script, mcp, mcp_workspace, hub_script, usage_meter, sso_groups_claim, cloud_trial_offer, job_token_scopes, worker_draining, worker_group, worker_resource_stats, dev_workspace_isolation, slack_agent, variable_expiration, anthropic_federation, deployment_request, fork_roles, git_sync_review). Nearly all of the product is uninstrumented, so new user-facing work is the opportunity to change that.

When to instrument

Raise it in the plan, with the concrete vocabulary written out, and let the user keep or drop it in one line. Don't stop and ask as a standalone question.

Propose it when a new user-facing affordance leaves a real question open:

  • a new panel, mode, tab, toggle, or entry point — is it discovered and used at all?
  • competing UX paths, or a new default — which one wins?
  • an opt-in or beta gate — what is the take rate?
  • a multi-step flow — where do people stop?

Stay silent for bugfixes, refactors, internal plumbing, and anything whose useful signal would need per-item identifiers (paths, names, prompts, code) — those cannot be logged at all, see Privacy rules. If the answer wouldn't change a decision, instrumenting is overkill; say nothing.

Designing the vocabulary

Field Meaning Limits
feature Product area: ai_chat, flow_editor ≤50 chars
kind The action within it: message, panel_placement. (feature, kind) is the allowlisted pair ≤50 chars
key A facet of the action — mode, tab kind, tool name, provider:model. Aggregation groups by (feature, kind, key), so this is what splits one counter into comparable buckets ≤100 chars, identifier-shaped, optional
entity_id An opaque random id (e.g. a session id) when you need per-entity distributions rather than a flat count ≤50 chars, identifier-shaped, optional
value Increment, default 1 clamped to 1…1,000,000

Identifier-shaped means ASCII alphanumerics plus _ - : . / — no spaces. Anything else is rejected.

Supplying entity_id is what unlocks the distribution stats: the payload reports entity_count, total_value, median_value, p90_value, and inactive_3d_entity_count per (feature, kind, key). Omit it for a plain "how many times did this happen" counter. Keep the key vocabulary closed and small — enumerate the values in a TS union next to the call site, the way flowEditorTelemetry.ts does, so the whole set is reviewable in one place.

The recipe

Four steps. Skipping step 1 fails quietly.

1. Register the pair in FEATURE_USAGE_KINDS (backend/windmill-common/src/feature_usage_ee.rs, tracked in windmill-ee-private). An unregistered (feature, kind) is dropped by is_recordable_event with a bare continue — no error, no log, still a 204 to the browser. Frontend-only instrumentation records nothing and looks like it worked.

2. Log from the frontend:

import { logFeatureUsage } from '$lib/utils/featureUsage'

logFeatureUsage('flow_editor', 'panel_placement', { key: 'force_detach' })

Fire-and-forget. Events sum locally per (workspace, feature, kind, key, entityId) and flush every 30s, on visibilitychange → hidden, and on pagehide; 50 events per request, and a failed batch is dropped rather than retried.

3. Check the disclosure. InstanceSettings.svelte describes a non-minimal payload by category, not counter by counter (two places — the copy appears twice); keep it short. A plain counter needs no edit. Only a key that sends a new kind of value beyond a count — a name or an identifier, like the AI model and hub names it already lists — must be added there.

4. Verify a row lands. The silent-drop path means "no error" proves nothing:

SELECT feature, kind, key, entity_id, day, value FROM feature_usage ORDER BY updated_at DESC LIMIT 10;

Collection sits behind the private feature, so a public build records nothing from either the HTTP route or the Rust helper. Run the backend with --features enterprise,private or this query stays empty however correct the instrumentation is.

Privacy rules

Only aggregated counts ever leave the instance, and only when telemetry is enabled and minimal mode is off. Never put a path, prompt, script body, workspace name, email, or any user identifier into key or entity_id. Entity ids must be opaque random ids, never anything that maps back to a user or a resource. If the signal you want can only be expressed with identifying data, it cannot be collected — drop it.

Counters aggregate over the last 30 days; rows are pruned after 60.

Logging from the backend

A feature with no UI is instrumented the same way, from Rust:

windmill_common::feature_usage::log_feature_usage("trigger", "fired", kind.as_str());

Same registry, same key rules, and the same silent drop when the pair is unregistered. feature and kind are &'static str so a call site cannot pass a computed pair. The call increments an in-memory counter and returns; the monitor loop flushes the accumulator, so it is cheap enough for hot paths — but only cheap per call, not free: a key with unbounded cardinality would grow the map until it hits the per-action cap and starts dropping new keys.

There is no entity_id and no explicit value on this path: it counts occurrences.

feature_usage_ee holds the registry and the writer; the public build gets the inert feature_usage_oss, since a CE instance never sends a stats payload.