mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-11-17 08:20:12 +00:00
* feat: harmonize flow, script and app settings drawers Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor: name the on-behalf-of setting by the access it grants Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: add a permissions tab and confirm who apps run on behalf of Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: sandbox draft-only app deploys and keep app settings out of exports Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep drafted run-as users and custom url removals Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: confirm the run-as user on every deploy that uses one Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: let new raw apps take the folder default and follow synced settings Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: add a share dialog next to deploy and a first-deploy dialog Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: start AI-session raw apps without a run-as user Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: mock the deployed app read in chat deploy tests and type the list height Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * chore: point at the ee branch that registers the sharing counters Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: share the deployed path, log first deploys once and report the kept app access Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor: stop drafting an app's access mode and custom url Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep a deployed app's access mode on an mcp deploy Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: ask who a first deploy runs as in the first-deploy dialog's second step Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: drop the first-deploy footer rule and test the run-as question Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: let a flow or script deploy turn off run on behalf of Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: apply who can open an app on change, as on main Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: mute the error handler from the editor menu and keep it on deploy Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor: rename the Permissions entries and drawer to Share Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: confirm deploys that move an item to another folder Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: change only the access mode when it applies on click Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: read the mute flag from the deployed row at its own path Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: read a raw app's access mode after bundling it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep group members off the list of who a move takes access from Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: change an app's access mode under the deploy row lock Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: defer a mode change to deploy on an app run as another user Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: hold the access toggle while its change is in flight Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: bind the app policy down to the settings drawer Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep app draft labels in the mcp session tools Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * revert: leave the deploy panel's access toggle to the settings drawer PR Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: change only the access mode on click and let deploys keep the deployed one Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: send Share's mode change alone and keep it out of MCP deploys Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep the caller's draft on a mode-only app update Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: leave the access mode out of every update deploy Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: explain in Share what an item's runs can reach Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: describe a viewer-mode app as using its viewer's access Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: confirm Share's app access changes with a toast Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: refuse a builder's mode-only change on an unsandboxed app Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: disable Share's access mode for a builder on an unsandboxed app Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test: check the reason for the unsandboxed mode-only refusal Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: defer a builder's access-mode change on an unsandboxed app Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: state the stored access mode in the chat's raw app deploy note Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: state the stored access mode in the MCP raw app deploy note Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: lock the access mode when the server would refuse the change Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: address settings and run-as review findings Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: say to deploy first when Share's access mode is locked Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: hold the run-as confirmation until the folder default has loaded Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep the run-as dialog cancelable while the folder default loads Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor: harmonize flow and script settings layout Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: name who can change a locked access mode without taking the app over Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep the priority input open while it is edited Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: confirm run-as on flow and script deploy, mute from editor menu Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep settings rows open while edited and name the deploying user for non-admins Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: reread the deployed run-as user after a deploy that keeps the editor open Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: say in Share that admins and deployers can change a locked access mode Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: tell a builder who keeps the run-as user to pick themselves before deploying Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: write a setting before its row opens and show its real state while edited Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: slide settings rows open to their full height Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: open Share from the menu of Home's favorite and recent cards Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: send the popover's deployment message when Deploy is clicked Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: remount a settings row's fields when its toggle turns it on Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: send the popover's deployment message from Deploy & Stay here Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: count Share opened from a Home card under its own telemetry key Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: clear the deployment message after a successful deploy Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: show the stored access mode in Share after a refused change Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep app access mode display in sync with the stored mode and badge app settings tabs Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: clear the app Deploy popover's message after a successful deploy Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: read the deployed row before muting from the menu, title Deploy only when icon-only Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: give the app settings badges the chip label and row key Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: mark the app labels and access rows so their badges reach them Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: show the app settings as three sections instead of tabs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: drop unused publishAccessMode and app sharing badges Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: update ee-repo-ref to d16494953659e00dd169fb8f73266f22f937b862 This commit updates the EE repository reference after PR #911 was merged in windmill-ee-private. Previous ee-repo-ref: 07f0090acb4a1f206c37f34411819b8c86690388 New ee-repo-ref: d16494953659e00dd169fb8f73266f22f937b862 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
117 lines
6.0 KiB
Markdown
117 lines
6.0 KiB
Markdown
# Feature usage telemetry
|
|
|
|
`feature_usage` is the product-telemetry accumulator: day-bucketed counters that roll into the
|
|
anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick"
|
|
without any identifying data leaving the instance.
|
|
|
|
It currently carries 93 registered actions across thirty-six features (`ai_session`, `ai_chat`,
|
|
`ai_fix`, `ai_agent`, `ai_decision`, `ai_agent_eval`, `app_sandbox`, `datatable`, `db_manager`, `flow_editor`,
|
|
`flow_run`, `flow_step`, `home`, `header_nav`, `run_form`, `sharing`, `debugger`, `trigger`,
|
|
`command_script`, `mcp`, `mcp_workspace`, `hub_script`, `usage_meter`, `sso_groups_claim`,
|
|
`cloud_trial_offer`, `job_token_scopes`, `worker_draining`, `worker_group`,
|
|
`worker_resource_stats`, `dev_workspace_isolation`, `slack_agent`, `variable_expiration`,
|
|
`anthropic_federation`, `deployment_request`, `fork_roles`, `git_sync_review`). Nearly all of the
|
|
product is uninstrumented, so new user-facing work is the opportunity to change that.
|
|
|
|
## When to instrument
|
|
|
|
Raise it **in the plan**, with the concrete vocabulary written out, and let the user keep or drop
|
|
it in one line. Don't stop and ask as a standalone question.
|
|
|
|
Propose it when a new user-facing affordance leaves a real question open:
|
|
|
|
- a new panel, mode, tab, toggle, or entry point — is it discovered and used at all?
|
|
- competing UX paths, or a new default — which one wins?
|
|
- an opt-in or beta gate — what is the take rate?
|
|
- a multi-step flow — where do people stop?
|
|
|
|
Stay silent for bugfixes, refactors, internal plumbing, and anything whose useful signal would
|
|
need per-item identifiers (paths, names, prompts, code) — those cannot be logged at all, see
|
|
[Privacy rules](#privacy-rules). If the answer wouldn't change a decision, instrumenting is
|
|
overkill; say nothing.
|
|
|
|
## Designing the vocabulary
|
|
|
|
| Field | Meaning | Limits |
|
|
|---|---|---|
|
|
| `feature` | Product area: `ai_chat`, `flow_editor` | ≤50 chars |
|
|
| `kind` | The action within it: `message`, `panel_placement`. `(feature, kind)` is the allowlisted pair | ≤50 chars |
|
|
| `key` | A facet of the action — mode, tab kind, tool name, `provider:model`. Aggregation groups by `(feature, kind, key)`, so this is what splits one counter into comparable buckets | ≤100 chars, identifier-shaped, optional |
|
|
| `entity_id` | An **opaque random** id (e.g. a session id) when you need per-entity distributions rather than a flat count | ≤50 chars, identifier-shaped, optional |
|
|
| `value` | Increment, default 1 | clamped to 1…1,000,000 |
|
|
|
|
Identifier-shaped means ASCII alphanumerics plus `_ - : . /` — no spaces. Anything else is
|
|
rejected.
|
|
|
|
Supplying `entity_id` is what unlocks the distribution stats: the payload reports `entity_count`,
|
|
`total_value`, `median_value`, `p90_value`, and `inactive_3d_entity_count` per
|
|
`(feature, kind, key)`. Omit it for a plain "how many times did this happen" counter. Keep the key
|
|
vocabulary closed and small — enumerate the values in a TS union next to the call site, the way
|
|
`flowEditorTelemetry.ts` does, so the whole set is reviewable in one place.
|
|
|
|
## The recipe
|
|
|
|
Four steps. Skipping step 1 fails quietly.
|
|
|
|
**1. Register the pair** in `FEATURE_USAGE_KINDS`
|
|
(`backend/windmill-common/src/feature_usage_ee.rs`, tracked in `windmill-ee-private`). An
|
|
unregistered `(feature, kind)` is dropped by `is_recordable_event` with a bare `continue` — no
|
|
error, no log, still a 204 to the browser. Frontend-only instrumentation records **nothing** and
|
|
looks like it worked.
|
|
|
|
**2. Log from the frontend:**
|
|
|
|
```ts
|
|
import { logFeatureUsage } from '$lib/utils/featureUsage'
|
|
|
|
logFeatureUsage('flow_editor', 'panel_placement', { key: 'force_detach' })
|
|
```
|
|
|
|
Fire-and-forget. Events sum locally per `(workspace, feature, kind, key, entityId)` and flush
|
|
every 30s, on `visibilitychange` → hidden, and on `pagehide`; 50 events per request, and a failed
|
|
batch is dropped rather than retried.
|
|
|
|
**3. Check the disclosure.** `InstanceSettings.svelte` describes a non-minimal payload by
|
|
category, not counter by counter (two places — the copy appears twice); keep it short. A plain
|
|
counter needs no edit. Only a key that sends a new kind of value beyond a count — a name or an
|
|
identifier, like the AI model and hub names it already lists — must be added there.
|
|
|
|
**4. Verify a row lands.** The silent-drop path means "no error" proves nothing:
|
|
|
|
```sql
|
|
SELECT feature, kind, key, entity_id, day, value FROM feature_usage ORDER BY updated_at DESC LIMIT 10;
|
|
```
|
|
|
|
Collection sits behind the `private` feature, so a public build records nothing from either the
|
|
HTTP route or the Rust helper. Run the backend with `--features enterprise,private` or this query
|
|
stays empty however correct the instrumentation is.
|
|
|
|
## Privacy rules
|
|
|
|
Only aggregated counts ever leave the instance, and only when telemetry is enabled and minimal
|
|
mode is off. Never put a path, prompt, script body, workspace name, email, or any user identifier
|
|
into `key` or `entity_id`. Entity ids must be opaque random ids, never anything that maps back to
|
|
a user or a resource. If the signal you want can only be expressed with identifying data, it
|
|
cannot be collected — drop it.
|
|
|
|
Counters aggregate over the last 30 days; rows are pruned after 60.
|
|
|
|
## Logging from the backend
|
|
|
|
A feature with no UI is instrumented the same way, from Rust:
|
|
|
|
```rust
|
|
windmill_common::feature_usage::log_feature_usage("trigger", "fired", kind.as_str());
|
|
```
|
|
|
|
Same registry, same key rules, and the same silent drop when the pair is unregistered. `feature`
|
|
and `kind` are `&'static str` so a call site cannot pass a computed pair. The call increments an
|
|
in-memory counter and returns; the monitor loop flushes the accumulator, so it is cheap enough for
|
|
hot paths — but only cheap per call, not free: a key with unbounded cardinality would grow the map
|
|
until it hits the per-action cap and starts dropping new keys.
|
|
|
|
There is no `entity_id` and no explicit `value` on this path: it counts occurrences.
|
|
|
|
`feature_usage_ee` holds the registry and the writer; the public build gets the inert
|
|
`feature_usage_oss`, since a CE instance never sends a stats payload.
|