mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-21 08:02:38 +00:00
3dfbf64d9db6dabecc6412d2ed0e35a287fb48e8
2090
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Diego ImbertandClaude Opus 5
|
22b3e69c07 |
feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection, `custom_instance_user`, for everyone who could reach it at all. There was no way to say this job reads, that one writes, this one never sees the salaries table. A data table role is now a real Postgres login on the cluster, defined once for the instance by a superadmin and named exactly as they named it. A script that declares `-- role analytics` connects as `analytics`, and Postgres decides what it may touch — grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`. A data table with no `permissions` block behaves exactly as before. Everything that opens a connection on someone's behalf goes through one chokepoint, `get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when there is none. The role logs in as itself — never `SET ROLE`, which a script could `RESET ROLE` its way out of. A fork's data table entry becomes a pointer at the workspace that governs it rather than a copy of it. The settings clone used to hand a fork a byte-identical entry naming the parent's database, which a fork admin could edit to grant themselves `admin` there; a pointer has nothing local to edit, and its tenants are evaluated as a member of the governing workspace, by email. `permissions` is stripped from the workspace export and ignored on import: tenants name principals of one workspace, and a settings push is not where an access decision should be made. Operations that see the whole database whatever the roles grant stay with the governing workspace's admins: editing the roles, a migration that declares none, and opening a replication stream for a Postgres trigger or capture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR |
||
  Ruben Fiszelandrubenfiszel
|
c3e11bd223 |
chore(main): release 1.813.0 (#11141)
* chore(main): release 1.813.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
9fe493311d |
chore(main): release 1.812.0 (#11111)
* chore(main): release 1.812.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
 
|
69e6efd875 |
fix(git-sync): run auto-pull as the admin who enabled it (#11121)
* fix(git-sync): run auto-pull as the admin who enabled it Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(git-sync): audit the admin grant fork pulls make Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: bump ee ref for the post-commit fork grant audit Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(git-sync): address review nits on the auto-pull stamp Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to ccada062c072d7b74894b63863728fd1ef9bdffd This commit updates the EE repository reference after PR #799 was merged in windmill-ee-private. Previous ee-repo-ref: 7cee30f0cf12721cba551cd754dc817444810470 New ee-repo-ref: ccada062c072d7b74894b63863728fd1ef9bdffd Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszelandwindmill-internal-app[bot]
|
e3e638f7f5 |
fix: skip instance group members that are not email addresses (#11128)
* fix: skip instance group members that are not email addresses * fix: keep provisioned members whose address only proper_email accepts * fix: judge instance group members by a mirror of the usr email constraint * fix: fold ascii only in the proper_email mirror, like the constraint * fix: let the database judge which instance group members usr will store * fix: cut a derived username to the column width so a long local part can be provisioned * chore: move the ee pin to the scim member doc fix * chore: update ee-repo-ref to 0780955effb657807d14f0eb503cba1d49cee007 This commit updates the EE repository reference after PR #801 was merged in windmill-ee-private. Previous ee-repo-ref: ee6452d489563204a98df883703f78d5e74cdd69 New ee-repo-ref: 0780955effb657807d14f0eb503cba1d49cee007 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
|
80eba80d6e |
feat(git-sync): gate GitHub PRs on Windmill CI test results (WIN-2051) (#10096)
* docs: add design doc for automatic git-to-windmill pull sync
* docs: add migration plan and implementation phases to git-sync pull design
* feat(git-sync): add auto_pull settings schema and pull enqueue primitive
Adds AutoPullSettings/AutoPullMode/AutoPullStatus on GitRepositorySettings
(workspace_settings.git_sync JSONB), the GIT_SYNC_PULL_SCRIPT_PATH constant,
and should_pull/effective_poll_interval_s helpers with unit tests. Exports the
EE enqueue_git_pull_job primitive. Foundation for repo→Windmill auto-pull.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): poll repos and auto-pull new commits into the workspace
Phase 1 of automatic repo → Windmill sync. A monitor task (EE-licensed,
single-replica via advisory lock) git ls-remotes each auto-pull-enabled
repository ~every minute and enqueues a pull when the tracked branch moves,
reusing the {workspace_id}:git_sync concurrency key so pulls serialize with
in-flight push commits.
- windmill-store: background (no-authed) resolver get_git_repo_head_for_autopull
that resolves the repo resource (incl. $var: refs) and ls-remotes; GitHub-App
repos are skipped here and will sync via webhooks (phase 2).
- monitor.rs: poll/reconcile/persist with optimistic sha advance and failure
status; targeted jsonb update so concurrent settings edits aren't clobbered.
- edit_git_sync_repository: preserve server-owned auto_pull state on UI save.
- openapi: AutoPullSettings/AutoPullMode/AutoPullStatus + auto_pull field.
- frontend: per-repo "Automatically deploy changes from Git" toggle with last
sync status; demote the GitHub Actions link to an advanced CI option.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): wire webhook lifecycle + receiver; share reconcile logic
OSS side of phase 2 auto-pull webhooks:
- edit_git_sync_repository creates/removes the repo webhook on save (EE-gated,
best-effort → falls back to polling).
- monitor poller now delegates to the shared windmill_git_sync reconcile/persist
helpers (also used by the webhook receiver), removing duplicated logic.
- export the shared reconcile/persist/failure helpers; bump EE ref.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref for phase 3 in-app PR creation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): show webhook vs polling status on the auto-pull toggle
When a repo has an active webhook (auto_pull.webhook_id set), the status line
reads "instant via webhook"; otherwise it reads the ~1-minute polling cadence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): post PR diff check on dry-run completion (phase 4)
Worker completion hook in process_completed_job: when a DeploymentCallback job
carrying the __git_sync_pr_check marker finishes, parse the dry-run SyncResponse
and patch the GitHub check run with the diff summary (success/neutral/failure).
Export enqueue_git_pull_dry_run; bump EE ref.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref (drop unused GHES webhook_secret)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* revert(git-sync): defer phase 4 PR diff checks (OSS side)
Remove the worker completion hook that posted the PR check run, drop the
enqueue_git_pull_dry_run re-export and the orphaned sqlx cache, bump EE ref.
Phases 1-3 (polling, webhooks, in-app PR creation) are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Revert "revert(git-sync): defer phase 4 PR diff checks (OSS side)"
This reverts commit
|
||
|
Ruben Fiszelandrubenfiszel
|
7b89e37322 |
chore(main): release 1.811.1 (#11107)
* chore(main): release 1.811.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
62d4632fad |
chore(main): release 1.811.0 (#11098)
* chore(main): release 1.811.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
4a293cf77a |
chore(main): release 1.810.0 (#11079)
* chore(main): release 1.810.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 5
|
b156778da2 |
fix: support gzip and zstd compression for OTLP export over gRPC (#11077)
Claude-Session: https://claude.ai/code/session_01RbLzVfFDZ9pjBGHSzCSrNZ Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
7915540f66 |
chore(main): release 1.809.0 (#11045)
* chore(main): release 1.809.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
133e21080a |
chore(main): release 1.808.0 (#11043)
* chore(main): release 1.808.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
22c1a106cf |
chore(main): release 1.807.0 (#11034)
* chore(main): release 1.807.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
63c40f046b |
chore(main): release 1.806.0 (#11012)
* chore(main): release 1.806.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 5
|
de98adf055 |
feat: let a worker group override the dependency cache object store (#11019)
* feat: let a worker group override the dependency cache object store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: address review findings on the worker-group cache override Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: close the remaining config read route and re-evaluate the override on plan change Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: serialize override reloads and keep a store a failed rebuild still serves Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: require enterprise for the cache override and lock its whole transition Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 5
|
0139467b01 |
feat: ingest dbt column lineage and real column schemas from the engine's parquet index (#10977)
* feat: column-level lineage for dbt from the engine's parquet index `manifest.json` carries no column-to-column edges, which is why decision 14 recorded column lineage as unavailable. The edges live in a different artifact: `dbt compile --static-analysis strict --write-index` writes `target/index/`, whose `dbt.column_lineage.parquet` holds them and whose `dbt.node_columns.parquet` holds every column of every node, typed and ordered rather than only the ones an author documented. Strict analysis rejects SQL the default accepts, so this is a separate compile with its own `--target-path`, opt-in per project via `column_lineage: true`, and best-effort throughout: a project it cannot analyze keeps exactly the graph it had, with the engine's own diagnostics in the job log. Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with the same composite FK to `script` and the same sweeps. The typed column list lands in `dbt_node.column_schema`, beside `columns` rather than merged into it, so `columns` stays what the author declared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: address review findings on the dbt column-lineage pass - The workspace fork copied every other dbt sidecar table and not this one, so a fork lost its column lineage silently and could not recover it: the cloned digest covers the column edges, so a dynamic run in the fork matched it and stored nothing. - The parquet was collected whole before the edge cap applied, which is exactly the input the cap exists for — a project whose `scan` lineage is quadratic in its widest model could take the worker process down. Decoded a row at a time with the bound enforced during the decode. - The pass swallowed every error from the runner, including the job poller's cancellation and deadline, so a run that blew its timeout inside an optional annotation could still publish a graph and report success. `run_captured` now carries the exit status in its value, so only a failed COMPILE is downgraded, and the pass may spend at most half the remaining wall clock so it cannot starve the build that follows it. - `scan` edges are stored but no longer served: they are most of a project's lineage, nothing renders them, and the graph endpoint is polled by the run page. They are also the first thing the storage cap gives up now, rather than evicting the direct edges the trace draws. - `column_schema` and the column edges take the same gate as the model's SQL. A column-level view is the shape of what the author wrote, one level finer than the `ref()` graph, which is ungated only because it draws relations the caller already sees. - `graph_digest` hashes the new section only when it has edges, so a project that never asked for the pass keeps the digest it has instead of re-snapshotting on every dynamic run until it is redeployed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: the editor buffer's column lineage, and three bounds that were wrong Round-2 review found four defects, all of them introduced by the round-1 fixes. - The `script_visible` gate on the column edges was copied from the node query without its `script_hash IS NULL` arm. `= NULL` is never true, so every version-less row was filtered out and an editor buffer's parse rendered its typed columns and none of their lineage — the one place the feature is meant to be used. Pinned by an assertion in `dbt_pinned_graph.rs`, which is where this class of bug already had a home. - The phase budget was handed to the poller, whose expiry is an `Err` indistinguishable from a cancellation or the job's own deadline, so a slow but valid analysis aborted the build it exists to annotate. The runner gets the full deadline again — those two must still fail the job — and the budget is a race around the whole pass, where expiring is this budget and nothing else. - The decode cap counted parquet ROWS, so `scan` and out-of-graph rows could spend it before a single drawn edge was read. It now counts what is kept, takes direct kinds in a first pass, and is handed the graph's own nodes so the budget cannot go on rows that could never be stored. - Hashing the new digest section conditionally did not preserve old digests, because an absent `column_schema` still serialized as `null` inside the nodes. It is skipped when absent instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: split the lineage pass by error contract, and read it in one query Round 3's findings were all consequences of round 1 and 2's fixes, clustered in the same two files, so this reshapes those two seams rather than patching again. The worker pass was one function being three things at once — a subprocess runner with job-lifecycle error semantics, a bounded decoder, and a best-effort degrader — which is why each fix to one perturbed another. It is now `compile_index`, which owns the JOB's semantics (only a cancellation or the job's deadline can `Err`; a non-zero exit, the output ceiling and the phase budget are outcomes), and `read_index`, which owns the ARTIFACT's and knows nothing about the job. The budget wraps the compile alone, so a decode can no longer outlive the timeout that reported the build would get the rest. The output ceiling likewise becomes a value rather than a job error, for the caller that can carry on without the tail of a compile's stdout. The column edges were read by a fourth hand-written copy of the `live`/`chosen` CTEs and the version/editor-buffer join conditions, and copying them is what dropped the `script_hash IS NULL` arm and hid every buffer parse's lineage. Both kinds of edge now come from ONE statement over a `UNION ALL`'d edge source, so those conditions exist once. The union is at the source rather than a join because column lineage can name a node pair `dbt_edge` has no row for: a model reading `{{ this }}` gets edges from itself to itself, and `parent_map` has no self-loop. The cap on the column half now sits after the scope filter, the visibility check and the graph joins — the scope moved into SQL via the existing `ScopePathFilter` — so a row the caller may not read can no longer spend it and leave an allowed project's trace short. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: serve dbt column lineage from its own endpoint The column edges rode on the folder-wide asset graph, which a run page polls, while the trace is drawn for one selected relation. That needed a cap, and a cap has to be applied after every filter that can drop a row. Keyed to the asset there is no cap: `assets/column_lineage` answers for one relation, and the caller's `scripts:read` scope and the project's visibility are decided once, for the script that owns it. Pinning to a run's snapshot or the editor's parse of its buffer costs the job-read gate, so that form is `jobs/dbt_column_lineage/{id}` — the same shape `jobs/dbt_graph/{id}` has. The worker's decode now bounds work and memory separately, and a compile stopped by the output ceiling reports as truncated rather than complete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve the owning dbt version the way the graph does The unpinned arm picked the newest live version at the path without narrowing to dbt, so a path since redeployed in another language answered with no lineage while the graph beside it still drew that project's stale nodes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate pinned column lineage on reading the project, and answer the component Four things round 5 found, three of them in code this branch rewrote: - The pinned arm resolved the version from the job and stopped there, so a share-link viewer entitled to a run got the project's column names and edges while the graph beside it still redacted `raw_code` and `column_schema`. Resolving WHICH version answers is not deciding whether the caller may read it; the version-less editor buffer keeps its exemption, having no `script` row to ask. - The answer was the whole owning project's edges. The canvas lays out the connected component of the selected relation's columns, so the rest was unrenderable weight; a recursive walk over both directions returns exactly what is drawn, and the project key travels with it so a `unique_id` two projects share cannot walk from one graph into the other. - The decode had no exit but the 4M-row backstop once its buckets were full, spending wall clock the build below does not get. - An unreadable index was reported as a missing one, sending the reader to look at their engine rather than at the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stitch the two column graphs, and walk the component in Rust Round 6's two findings, both regressions this branch introduced: - The decode returned `Continue` on the edge that FILLED the direct-edge budget, so a `scan`-only tail after it decoded to the 4M-row backstop with nowhere to put anything. The read now ends on that edge. - Seam 3 made the pipeline page choose between the dbt graph and the producer one. They share node ids — `// column total <- dbt://wh/analytics/orders.amount` mints the same `(dbt, path, column)` node dbt's own lineage does — so choosing ended a trace at the boundary in both directions. They are merged again, and a ducklake selection asks about the dbt relation its producers name so the chain continues past it. The dbt editor gets the same merge. Also: the component is walked in Rust rather than by a recursive CTE. A CTE has no index, so the recursive term rescanned the doubled edge set once per level — 1243ms against 59ms for the query alone on a 3000-model project, 11.7M rows in the plan. Same answers, same tests; end to end 1.48s to 0.73s there and 1.60s to 0.26s on a 1000-deep chain. The client stops re-asking for a component it already holds, which is most clicks within one project. The four doc sites that described a whole-project answer are rewritten around what it now is, rather than edited where they disagreed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: expand every dbt boundary a selection reaches, and only skip what was asked Round 7's findings, all in the frontend seam this branch added: - A ducklake selection seeded the dbt fetch from the FIRST boundary relation it found, so a table derived from two unconnected dbt relations expanded one and left the other a leaf — the same "stops at the boundary" symptom the round-6 fix removed, one hop further along. Every distinct boundary is fetched now and the components merged. - The component cache skipped a relation merely PRESENT in the graph in hand. A relation two projects describe has an owner row in each, and a component fetched for one carries it as an endpoint without the other's half, so that skipped the request that would have resolved the second owner. Only a relation actually asked about under this pin is skipped. - A comment still called the producer graph gated to ducklake selections after it was widened to dbt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: land dbt column lineage as storage and ingest only The API surface that draws a column trace moves to a follow-up PR, on `dbt-column-lineage-surface`. It kept generating findings — a client cache whose premise was wrong for a two-owner relation, then staleness and a lost retry from tightening it, and a seed walk that stopped at the first boundary — and the fix for the last of them is a transitive owner expansion, which has to re-apply the caller's gate to every newly discovered project. That is the same shape as the leak four reviewers caught in the pinned arm, and it wants its own review rather than being the fourth fix at the end of this one. What lands here stands on its own: the analysis pass, `dbt_column_edge`, `dbt_node.column_schema`, the engine gating and the error-contract split — plus the one user-visible half, the typed and ordered column list, which rides the asset graph the details pane already fetches and replaces a panel that could only show the columns an author had documented. Also fixes a real bug in the pass, found in review: it compiled without the build's `--full-refresh`. `is_incremental()` branches on that flag, so an incremental model reading `{{ this }}` compiles its self-join — and any `ref()` inside that branch — only when the flag is absent, and the pass was storing lineage for SQL a full-refresh run never executed. The flag now comes from one place shared with the build, and a run that overrides it gets its own graph rather than standing as the version's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: say why direct kinds get the budget without naming a view The bucketing comments explained the priority by what a trace draws, which is a forward reference now that the surface moved out. The reason stands on its own: `copy`/`mod` say the value travelled, `scan` says the column was read to produce the row and so reaches every output column of its model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: round-9 findings on the descoped PR - The `full_refresh` helper was inserted between `selection_is_overridden` and its doc comment, so thirteen lines about `select`/`exclude` echoes documented the wrong function and the one they were written for had none. Moved below it. - The parse path ran the analysis compile and the parquet decode BEFORE the guard that returns when there is no warehouse identity, paying for both and dropping the result. Moved after it. - Three sites still described a `/column_lineage` endpoint this branch no longer has, and two user-facing strings promised a column trace it no longer renders: the panel's hint and the descriptor template now say what the flag actually buys, which is the typed column schema. - Dropped test scaffolding the removed suite left behind: a `raw_orders` node and `dbt_edge` whose only assertion re-tested pre-existing graph behaviour, and a second editor-buffer node nothing asserts on. Documented rather than fixed: an incremental model has two shapes, and which one the index holds depends on whether the target existed when the pass ran. `is_incremental()` is false with no target as well as under `--full-refresh`, and dbt has no mode that emits both — so a version's graph describes the compile that produced it, and only a re-ingesting run describes its own run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep lineage_kind in the edge key, and one answer for --full-refresh - Both unique indexes omitted `lineage_kind`, so a column that is projected AND used as a predicate for the same output column — an ordinary shape — had its `copy` and `scan` edges collapse under `ON CONFLICT DO NOTHING`, while the digest counted both. The kind is part of the fact, so it is part of the key. Edited in the migration rather than added as a second one: it has not landed. - `full_refresh` was shared between the build and the analysis pass without the `command != "test"` condition that sat at the build's call site, so the two disagreed for exactly the runs that build nothing. The condition moved inside the function, which is the point of sharing it, and the command is threaded to the pass. - The "what a trace draws" rewrite missed the copy in `dbt_manifest.rs`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the unreachable full_refresh threading, test the uniqueness key `DBT_COMMANDS` is `["build", "retry", "show", "parse"]` and `default_command` returns `build` in every arm, so `command == "test"` cannot happen — the guard the last commit moved into `full_refresh` was already inert where it came from. Threading the command through five signatures to preserve it bought nothing, so it is gone; the build and the pass call one function of the descriptor and the invocation, which is what the sharing was for. The uniqueness-key fix now has a test: a column projected AND used as a predicate for the same output column stores both its `copy` and its `scan` row. Verified against the old key, where it returns 1 instead of 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: restore the dbt test --full-refresh guard I removed on a wrong premise The previous commit removed it after reading `DBT_COMMANDS` and concluding `"test"` was unreachable. That is only true of the command a CALLER can name: `run_dbt` is invoked with `"test"` directly for the `after_all` test phase, so an `after_all` project with `full_refresh: true` reached it — and dbt rejects `--full-refresh` on `test`, failing the phase. Both reviewers caught it. The guard is back inside the shared function, where the build and the pass get one answer, and its doc now records why reading the allowlist alone is misleading. The test covering the `test` case is restored with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: notice a job that ended during the decode, and name truncation as the cause - The parquet decode runs on a blocking thread with no poller watching it, so a cancellation or an expired deadline during it was invisible: `dbt_dep` went on to publish the graph and the job returned success. The job's state is checked once the decode returns, before the caller publishes anything, and an ended job `Err`s — which this module may always do for the job's own semantics. - A compile stopped by the output ceiling could leave no artifact, and the log then blamed the engine's capability, sending the reader to check their adapter rather than the ceiling. Truncation now names itself in the missing and unreadable branches too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read cancellation from the DB after the decode, not from a poller's field `ctx.canceled_by` is only ever written by a poller, and no poller runs during the blocking decode — which is the exact window the check was added for. So the guard caught only a cancellation already observed before it, and the comment beside it claimed more than it did. It now queries `v2_job_queue` directly, the same probe `worker_lockfiles` uses before it overwrites a flow. A failed probe answers "still running": this decides whether to discard work already done, so an unreachable database must not be the reason a healthy deploy loses its graph. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: reuse job_is_canceled rather than a second copy of it The probe added last round was `job_is_canceled` from the same file, retyped — same query, same `Connection::Http` behaviour. Reused instead. Its doc said a non-database connection was "a failed probe", which reads as an error path. It is not: it is the agent worker, and on one there is no database to ask, so only the deadline answers and a cancel issued during the decode is not observable. The retry path avoids that by refusing to run on an agent worker at all — which an optional annotation has no business doing — so the gap is recorded at both ends instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: close the agent-worker cancellation gap instead of documenting it The previous commit said a cancel issued during the decode is not observable on an agent worker. It is: `ping_job_status` returns `canceled_by` over both connection kinds, and is how the poller itself notices one there. So the check asks through the ping rather than querying `v2_job_queue` directly, and holds on an agent worker, where a direct query reaches no database at all. `job_is_canceled` goes back to private and its doc to what it said before — the retry that calls it still refuses to run on an agent worker for its own reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: decode the index under the job poller instead of checking after it Two findings with one cause: the decode was the only phase of this pass with no subprocess behind it, so nothing heartbeated while it ran. A large index left the worker silent for as long as it took, which the zombie sweep reads as a dead job and restarts — and the cancellation check bolted on afterwards could only ever report what had already happened, while dropping the ping's `already_completed`, so a force-cancelled deploy still published its graph. Running it under `run_future_with_polling_update_job_poller` answers all of it: the poller pings throughout, and ends the phase with an `Err` on cancellation, `AlreadyCompleted` or the phase timeout. The bespoke probe is gone with it. Verified on a live deploy: 32 edges and 4 typed schemas ingested through the polled decode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop a cancelled decode, and say what the read phase can now do Putting the decode under the poller heartbeats it and ends the phase when the job does, but dropping a `JoinHandle` detaches a blocking task rather than cancelling it — so a cancelled job left a thread decoding up to four million rows for a job that was over. The row loop reads an abandonment flag that a drop guard on the awaiting future sets, so the decode stops at its next row. That same change made the read phase able to `Err`, and three places still said it could not — decision 14 in as many words. The distinction that holds is narrower: nothing the ARTIFACT does or fails to do can fail a job, so absent, unreadable and partial are all values; the JOB can still end the phase the read runs in. Stated that way in the module doc, the `Artifact` doc, `MAX_INDEX_ROWS` and the decision. Verified on a live deploy: 32 edges and 4 typed schemas. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: share AbortOnDrop, and stop citing a hazard that is now handled `Abandon` was `ansible_executor`'s `AbortOnDrop` retyped — same struct, same reason, same `spawn_blocking` shape. Moved to `common` and used from both. The paragraph explaining why the phase budget wraps the compile alone gave as its reason "a decode still running on a blocking thread", which is exactly what the abandonment flag now prevents. The reason that survives is the one that was always the point: the budget exists to leave the build its share of the clock, and only the compile can spend that share unboundedly. The decode's end is the job's, through the poller it runs under. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: put both doc comments back on the items they describe Moving AbortOnDrop orphaned a doc at each end: it landed between `raw_to_string`'s doc and `raw_to_string`, and the doc of the struct it replaced stayed behind to prefix `fetch_repo_archive`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: name the binding the row loop actually reads `Abandoned` was neither the type nor the binding; the flag is `abandoned`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
a9d42b489f |
chore(main): release 1.805.0 (#10995)
* chore(main): release 1.805.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
c37f59e22a |
chore(main): release 1.804.0 (#10963)
* chore(main): release 1.804.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
|
8aab5034a6 |
feat: guest JWT entry for embedded apps (#10954)
* feat: guest JWT entry for embedded apps (jwt_guest_) A second way in for a guest, alongside the signed-in guest session: a JWT the embedding customer's backend mints and signs, verified per request against a per-workspace key (a PEM public key or a JWKS URL), resolving to the same seatless guest identity confined to the one app its app_path claim names. Bearer prefix jwt_guest_, stateless (no token row). See PR #10954. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: surface guest JWT as the embed method in the app deploy drawer The deploy drawer explained the secret-URL embed but not the guest JWT path, so the primary way to embed an app for a customer's own authenticated users was undiscoverable. For a guest-mode app with guests enabled, show how to mint a `jwt_guest_` token and append `guest.<jwt>` to the app URL, with a copyable iframe template pre-filled with this app's workspace_id and app_path, and a note that new guest emails are refused past the instance's free allowance (the live count is shown just above). Also log a guest JWT allowance refusal at warn, not info: the caller gets a bare 401 (the reason must not leak to an unauthenticated caller), so the log is the admin's signal that the instance hit its guest cap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: correct the guest JWT minting instructions in the embed block The block said "sign it with the workspace's guest JWT key", but that setting holds the public verification key. Clarify the keypair relationship (configure the public key or a JWKS URL in the workspace; sign with the matching private key), name the accepted algorithms (RS/PS/ES; HS* refused), and keep the required claims, so an embedder knows how to actually mint the token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: fall back to the instance JWT issuer for guest verification (off on cloud) A workspace with no guest key of its own now verifies guest JWTs against the instance issuer (JWT_EXT_JWKS_URL, already used by jwt_ext_), so an operator running one issuer configures it once. Verification and the guest grant are CE; granting a full login from that issuer stays EE (jwt_ext_, unchanged). Disabled under CLOUD_HOSTED, where one instance issuer must not be trusted to mint guests in every tenant's workspace — there the per-workspace key is the only source, which also stays the override everywhere. The workspace settings note (hidden on cloud) explains the fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: embed instructions cover both the workspace key and instance issuer The embed block said to set the workspace's guest JWT key; now it says Windmill verifies against the workspace key or, off cloud, the instance issuer (JWT_EXT_JWKS_URL) when no workspace key is set. The instance clause is hidden under isCloudHosted(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: show the guest JWT embed block only when Embed is toggled It belongs with the iframe snippet, not the plain-URL view, so gate it on embedMode alongside the guest-mode / guests-enabled checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: trust the instance issuer in the guest fallback; refresh stale docs P1 (CI review): the fallback wrapped JWT_EXT_JWKS_URL as a workspace JwksUrl, so it hit validate_guest_jwks_url and was refused for http/private issuers unless ALLOW_PRIVATE_GUEST_JWKS_URLS was also set — a self-hosted internal issuer that works for jwt_ext_ failed for guests, though the UI says setting the env var is enough. fetch_jwks now fetches the instance issuer without the https/private restriction (matching the jwt_ext_ loader; it stays operator-trusted), while a workspace-admin URL is validated and pinned as before. All the size/key/URL bounds still apply to both. P2 (CI review): refresh the stale docs that said a missing workspace key always refuses a guest JWT — the module, bearer, key-source, and EditGuestJwtKey field docs now describe the workspace key with the off-cloud instance-issuer fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: fetch the trusted instance issuer like the jwt_ext_ loader P1 (CI review): the instance-issuer fetch skipped SSRF validation but still disabled redirects and default cert validation, so an instance issuer that works for jwt_ext_ through a redirect or an operator-approved self-signed cert failed the guest fallback. Fetch it with HTTP_CLIENT_PERMISSIVE (follows redirects, honors ACCEPT_INVALID_CERTS) — the same behavior jwt_ext_ has — while a workspace-admin URL stays validated, DNS-pinned and redirect-free. The body size cap still bounds both. P2 (CI review): the WorkspaceSettings field doc still said None/None means no JWT guests; it now names the off-cloud instance-issuer fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: schema summary + OpenAPI cover the guest JWT columns and fallback P2 (CI review): summarized_schema.txt was missing guest_activity.jwt_entry and the two workspace_settings guest-JWT key columns (required by docs/validation.md after a schema change). The edit_guest_jwt_key OpenAPI description now notes that clearing the workspace key falls back to the instance issuer (JWT_EXT_JWKS_URL) off cloud rather than necessarily stopping guest JWTs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: keep JWKS single-flight locks in a self-cleaning map, not a bounded cache P1 (CI review): JWKS_FETCH_LOCKS was a 200-entry quick_cache. Past 200 cold URLs it can evict a lock whose fetch is still in flight; the next request for that URL then mints a fresh lock and starts a second fetch, so cycling configured workspaces defeats single-flight and can storm the issuers. Replace it with a plain map guarded by a JwksFetchLock RAII handle that removes each entry once its last holder drops, so the map only ever holds the fetches in flight and never evicts an in-flight lock. Add a unit test pinning the shared-lock and self-cleaning invariants. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: update ee-repo-ref to c2270eb5fe2d9f0968253e6b460c33186363f4e7 This commit updates the EE repository reference after PR #773 was merged in windmill-ee-private. Previous ee-repo-ref: 5a1d9dee34159512c0823fddcd3d096490edbcce New ee-repo-ref: c2270eb5fe2d9f0968253e6b460c33186363f4e7 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
|
fce635d3c4 |
feat: guest app execution mode, a role that takes no seat (#10929)
* feat: guest app execution mode, a fourth role that takes no seat Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: make the guest grant a server-minted label, not a declarable scope Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref to the guest session companion branch Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: close the relabel hole, guest embed tokens, read-path switch, custom-path entry Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest tokens are not rescopable and guest embed tokens keep the sentinel Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest-derived tokens share one constraint set; gate sign-in on guest discovery Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the label alone governs a guest; refuse guests with accounts; unserialize discovery Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest discovery fails closed; SAML aborts if the guest cookie write fails Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * refactor: enforce the guest switch once at the auth door; sign-in for a guest of another app Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest app-mode decided once at the on-behalf resolver; clear a stale guest session before offering another app's sign-in Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest may use anonymous apps; await the stale-session logout; trim comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest's path confinement waits for the app's mode, so anonymous apps stay open to it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest target survives http (Lax cookie), rides SAML RelayState; tell account holders on arrival * fix: a guest uses an anonymous app as itself; S3 uploads confined by app mode * fix: a guest upload needs an app policy; a missing app does not skip the confinement * fix: guests are gated on the Enterprise plan server-side; pin ee-repo-ref * fix: the guest plan gate fails closed on non-enterprise builds; settings report the effective switch * fix: guest controls read the plan, not the key; gate the guest tests on the features they need * docs: tighten the guest session invariant comments * feat: 100 free guests per 30 days, then a quarter seat each on Enterprise and a hard cap elsewhere; superadmin guest list; refusals reach the page Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the cap is exact, an account ends a guest session at the door, popups close, and guest mode survives the CLI round trip Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * feat: a superadmin switch over guests for the whole instance; the pre-existing-user flag keeps its meaning Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: drop the dead guest-access helper, name the instance setting once, guests tab states, CE save order Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest app path is refused at the mint if it could widen the scope; the instance toggle waits for its reload Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guests stop at the launched-by-me job grant; canonical app paths at the mint and discovery; the toggle ends on the stored value Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: only the scope grammar's own characters bar an app path from guests, refused at deploy as well as at the mint Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the deploy-time guest path guard checks the destination of a rename and refuses a leading slash Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a workspace rename keeps the guest switch; the rename guard reads the deployed mode under the row lock Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest_activity follows a workspace rename and goes with a workspace delete Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref to the state-bound guest target Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref; the guest cookie is never cleared by a callback Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * docs: the workspace-scoped guest_activity delete moves an instance-wide count; assert the mint records the guest Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * test: the seeded allowance is a day old, so only the mint can write today's guest_activity row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: update ee-repo-ref to 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07 This commit updates the EE repository reference after PR #769 was merged in windmill-ee-private. Previous ee-repo-ref: 32841072aa396bff91d30bd91854fa348cb3c439 New ee-repo-ref: 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
38fc0d3a12 |
chore(main): release 1.803.0 (#10952)
* chore(main): release 1.803.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
hugocasaandClaude Opus 5
|
17ba521c35 |
fix: record supplied script lock hashes so importers can skip relocking (#10915)
* fix: record supplied script lock hashes so importers can skip relocking
Creating a script with a caller-supplied lock — a CLI push, a git-sync deploy,
any create carrying a lockfile — stored the lock on `script` but never wrote the
matching `lock_hash(workspace_id, path, hash_script(lock))` row. Only
worker-generated locks did.
`try_skip_relock` treats a missing hash for an imported script as changed, so no
importer of such a script could ever satisfy the skip predicate: every deploy of
it relocked every importer, forever.
The create transaction now records the hash for any lock it accepts, including
the empty one a codebase or a language with no lock generation carries — the
worker writes `hash_script("")` there, and a path going from a real lock to an
empty one has to stop matching what its importers recorded. Only a lock left to
a dependency job is skipped, because that job writes it.
A workspace clone now carries `lock_hash` too, without which every
dependency-map snapshot the clone later recorded held NULL and nothing in it
could ever skip. `dependency_map.imported_lockfile_hash` is deliberately not
copied: it records what an importer resolved against when it was last locked,
the clone runs READ COMMITTED, and a relock landing in the source between the
scripts being cloned and that statement would attach a hash the cloned
importer's lock was never resolved against — a hash older than the cloned
scripts costs one relock, a newer one skips a relock that was needed.
Lock generation is untouched, as is everything a relock does once it runs. The
only behavior that moves is which relocks are skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* fix: narrow to the create-path lock hash
Drop the workspace-clone copy of lock_hash. It sits outside the reported
bug, and its double join over `script` can emit a path twice where two
versions are live, which the unique key on (workspace_id, path) then
rejects, failing the whole fork.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* fix: restore the workspace-clone lock hash copy, guarded against fanout
A path can hold two live versions, and both joins match on path alone, so
the select can emit it four times against a primary key that admits one.
Every such row carries the single hash the path has, so ON CONFLICT DO
NOTHING settles it rather than aborting the fork.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* fix: hash a clone's own locks rather than copying the source's rows
A source row is only as current as the last write to it, and a supplied
lock deployed before this was recorded leaves one naming a lock the path
no longer holds. Copying that into a fork hands an importer a hash it
never resolved against; hashing what the clone holds cannot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* test: pin the lock hash written on a no-op push
Removing that write leaves the assertion with no row, which is the state
a script deployed before this shipped would stay in.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* refactor: share one lock hash writer between the create and clone paths
Both wrote the same upsert with different SQL. The existing writers fold
theirs into the statement that writes the lock itself, which is what keeps
the two consistent; these two have nothing to fold it into, so they take a
shared one instead. The clone walks its pages by path rather than listing
them first, dropping a query with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* fix: stream a clone's locks rather than reading them in pages
script.lock is unbounded, so a page of them is bounded only by how many
it holds. Hashing each as it arrives keeps one in memory at a time and
lets the clone site collapse to a single call.
Also states on both writers that they check no access to the workspace
they write, which their callers are the ones to have established.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
* fix: make the lock hash writer safe to repeat and free when unchanged
A path given twice in one call would have Postgres reject the whole
statement, so the last hash for each wins. And recording a hash a path
already has cut a row version for nothing on every unchanged sync, which
is the mode the no-op push runs in.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Ruben Fiszelandrubenfiszel
|
74c1813f98 |
chore(main): release 1.801.0 (#10921)
* chore(main): release 1.801.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
870f67121d |
chore(main): release 1.800.1 (#10910)
* chore(main): release 1.800.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
412eb90c0d |
chore(main): release 1.800.0 (#10888)
* chore(main): release 1.800.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandwindmill-internal-app[bot]
|
2fb790338d |
upgrade argon2 to 0.6 and migrate the password hashing API (#10902)
* fix: upgrade argon2 to 0.6 and migrate the password hashing API * test: pin that an unparseable stored hash reads as a failed login * chore: update ee-repo-ref to 58738c39ac41d57917bbd9400318704763d997f7 This commit updates the EE repository reference after PR #759 was merged in windmill-ee-private. Previous ee-repo-ref: 02a89fc4d27e49a494112fa91a8812e3ee4fb8a6 New ee-repo-ref: 58738c39ac41d57917bbd9400318704763d997f7 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 5
|
419d3adb6c |
chore: bump tantivy to 0.27 and pin argon2 to 0.5 (#10890)
* chore: bump tantivy to 0.27 and pin argon2 to 0.5 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * chore: pin tantivy to the merged fork main head --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
7c1a785f75 |
feat: serve service log retrieval from a columnar parquet store (#10886)
* feat: always write service log files as json so they index structured Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * feat: serve service log retrieval from a columnar parquet store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * feat: shrink the service log index to the per-host count it still serves Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * fix: reclaim the superseded service log index on upgrade Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * fix: address review findings in the service log store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * chore: update ee-repo-ref to ad9e899dfd2ee4e3d18ecf06d016f821968c5a83 This commit updates the EE repository reference after PR #751 was merged in windmill-ee-private. Previous ee-repo-ref: 6ad4064f9d58d83612b42b4ec870384994d64bcb New ee-repo-ref: ad9e899dfd2ee4e3d18ecf06d016f821968c5a83 Automated by sync-ee-ref workflow. * fix: address review nits on the service log store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
7a0c81d722 |
chore(main): release 1.799.0 (#10874)
* chore(main): release 1.799.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
90b40fffc3 |
chore(main): release 1.798.1 (#10870)
* chore(main): release 1.798.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
2302e58c24 |
chore(main): release 1.798.0 (#10868)
* chore(main): release 1.798.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
52ca19e9ae |
chore(main): release 1.797.0 (#10848)
* chore(main): release 1.797.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
0f3d884c6f |
chore(main): release 1.796.0 (#10810)
* chore(main): release 1.796.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 5
|
8dbd12ecc1 |
fix: patch sqlx so a cancelled BEGIN cannot poison a pooled connection (#10823)
* fix: patch sqlx so a cancelled BEGIN cannot poison a pooled connection Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qzqmh52NU8fB9RBQNNkJGt * test: drop the migration run and fixed sleep from the sqlx patch guard Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qzqmh52NU8fB9RBQNNkJGt * test: ignore the sqlx patch guard by default and point at it from where sqlx is changed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qzqmh52NU8fB9RBQNNkJGt --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
74af4ed939 |
chore(main): release 1.795.0 (#10807)
* chore(main): release 1.795.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
1a506b8f22 |
chore(main): release 1.794.1 (#10801)
* chore(main): release 1.794.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
55b6279058 |
chore(main): release 1.794.0 (#10782)
* chore(main): release 1.794.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
|
8e508ea01a |
feat: support application default credentials for gcp pub/sub triggers (#10778)
* feat: support application default credentials for gcp pub/sub triggers Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address review findings on gcp application default credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address review nits on gcp application default credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: key the gcp credential-mode permission off the loaded mode Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: gate enabling an ADC gcp trigger on workspace admin Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: lock the gcp trigger row while authorizing a mode change Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: skip admin-only gcp listing when the caller cannot use those credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 54bf630681000c8ed87a7067e357118e015123b1 This commit updates the EE repository reference after PR #738 was merged in windmill-ee-private. Previous ee-repo-ref: 91d0e228a0ad226625278b400c64f96a61404a10 New ee-repo-ref: 54bf630681000c8ed87a7067e357118e015123b1 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszelandrubenfiszel
|
2439a610be |
chore(main): release 1.793.0 (#10764)
* chore(main): release 1.793.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
|
5099f405d4 |
feat: make the Git Repo Viewer work with GitHub App repositories (#10765)
* fix: resolve the head commit of GitHub App repos in the git repo viewer `get_git_commit_hash` ran `git ls-remote` against the raw resource URL. A GitHub-App-backed repository stores a tokenless URL, so the probe failed with "could not read Username" and the viewer never got past its first step. Resolve the head over the GitHub REST API with a server-side installation token instead, reusing the lookup the auto-pull poller already uses for app repos. Non-app repositories keep the ls-remote path. Also picks up the EE-side allowlist fix that lets the clone hub script request an installation token. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 63c67e2a2db198af26a0334f5be14af7d9987eb1 This commit updates the EE repository reference after PR #732 was merged in windmill-ee-private. Previous ee-repo-ref: 2a260961fa0a9bb5631c17e2f718cb8efb4f9aa2 New ee-repo-ref: 63c67e2a2db198af26a0334f5be14af7d9987eb1 Automated by sync-ee-ref workflow. * fix: honour the app-repo head lookup's not-app-backed result `get_app_repo_head_for_autopull` documents `Ok(None)` as "this repo is not app-backed, use the ls-remote path", which is what the other two callers do. Fall through to `ls-remote` on `None` instead of turning it into a 500, and drop the handler's own `is_github_app` read now that the callee's answer is honoured. Also bumps ee-repo-ref to pick up route-safe ref handling in that lookup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: serve GitHub App repositories as an archive instead of a token The viewer's clone script asked the server for an installation token and put it in the clone URL. That token is installation-wide and carries the App's full permissions, so minting one requires a workspace admin, and the viewer was therefore admin-only for app-backed repositories. The server now streams a tarball of the commit instead, authorized by read access to the git_repository resource, so no GitHub credential reaches the job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: run delegate_to_git_repo playbooks from GitHub App repositories An Ansible job's runnable_path is the user's own script, which no entry in the git-sync script allowlist can match, so `delegate_to_git_repo` could never obtain a token for an app-backed repo. It also gave up entirely on agent workers, whose connection has no database to mint one from. A playbook run only reads a working tree: the clone is followed by one rev-parse for a log line, and nothing after that touches git. So take the same archive route the viewer uses, extracting the commit's tarball into the job's repository directory. No GitHub credential reaches the worker, and agent workers work because the route is HTTP. Archive entries are joined onto the target by hand so a crafted archive cannot write outside the job directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: drop the now-immutable secret_url binding Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: point the repo viewer at the archive-based clone script hub/28905 reads app-backed repositories through the server's archive route instead of minting an installation token, which the backend in this release no longer grants it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: stream repository archives to disk rather than into memory The archive download went through `AuthedClient::get`, whose client caps a request at 20 seconds and whose response was then buffered whole. A repository is arbitrarily large, so that cut off slow downloads and put every job on the worker at risk of running the process out of memory. Add `get_streaming`, the read counterpart to the streaming upload path, and write the response out chunk by chunk. Extraction now creates each entry's parent directory: a tar carries directory entries only by convention, and the traversal guard now has tests, one of which caught the missing parent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: require admin to read an app-backed repository A `git_repository` resource names the repository rather than holding a credential for it, so read access to one authorizes nothing: anyone who can write a resource path can point one at any repository the GitHub App installation reaches, then read their own resource. The head lookup now requires admin for app-backed repos, matching the archive route and the repository picker, which already limits itself to workspaces where the caller is an admin. Repos that aren't app-backed are untouched and stay open to any reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: describe the repo viewer's hub script as it stands The file read as a patch waiting to be applied, against a hub version two releases stale. Describe what the published script does, including the archive route app-backed repositories now take. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: run the archive fetch under the job poller, off the job directory Three defects in the delegate path's fetch: The download and extraction ran outside the job poller that the git clone paths go through, so a cancelled or timed-out run kept streaming and extracting an arbitrarily large repository while holding the worker. There is no wall-clock bound on the download itself, by design, which is exactly why it needs the poller. The archive was written to a fixed name inside the job directory, where `create_file_resources` has already laid down the run's own files at paths the playbook chooses. A run naming a file `repo_archive.tar.gz` had it truncated and then deleted. It goes to a per-job temp path now. Link entries were unpacked with their target unchecked. `Entry::unpack` writes the link verbatim, so a link out of the tree plus a later entry descending through it writes wherever it points. Targets now face the same containment check as entry paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: keep repo symlinks, refuse only writes that go through them The link check rejected any target containing `..`, which is ordinary in a repository — `docs/x -> ../README.md` resolves inside the tree, and a git checkout keeps it. Rejecting it failed the whole extraction for repositories the clone path handles, and app-backed repos have no clone path to fall back to. Targets are preserved as git preserves them. What would let one escape is a later entry written at or underneath the link, so that is what is refused. Extraction also polls an abort flag now: a `spawn_blocking` task outlives the join handle its caller drops, so a cancelled job left it unpacking in the background. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: refuse hard links in a repository archive Leaving link targets verbatim is right for symlinks — git checks them out that way, and an escape needs a second entry descending through the link, which is refused. A hard link is not like that: unpacking one creates it against a target resolved there and then, so an escaping target is useful on its own. No git tree can express a hard link, so an archive carrying one did not come from a repository. Refuse it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 21f79bbbd39ae89665d1a89738630978616aa309 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 37695a769b25d16b34107eedc1076793a8b388c8 This commit updates the EE repository reference after PR #737 was merged in windmill-ee-private. Previous ee-repo-ref: 21f79bbbd39ae89665d1a89738630978616aa309 New ee-repo-ref: 37695a769b25d16b34107eedc1076793a8b388c8 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
a7637aca31 |
chore(main): release 1.792.2 (#10753)
* chore(main): release 1.792.2 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
9f517d5a40 |
chore(main): release 1.792.1 (#10750)
* chore(main): release 1.792.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
8efede55d6 |
chore(main): release 1.792.0 (#10745)
* chore(main): release 1.792.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
ce71756c89 |
chore(main): release 1.791.0 (#10718)
* chore(main): release 1.791.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
010d67e07f |
chore(main): release 1.790.1 (#10712)
* chore(main): release 1.790.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
944ad1083a |
chore(main): release 1.790.0 (#10699)
* chore(main): release 1.790.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
hugocasaandClaude Opus 5
|
98bacab907 |
refactor: combine the per-minute counters onto one shared helper (#10687)
* refactor: combine the per-minute counters onto one shared helper Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep dashmap in windmill-store for the azure devops token cache Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: name the sweep counter for what it counts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
80a18ec284 |
chore(main): release 1.789.0 (#10670)
* chore(main): release 1.789.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
b39860235c |
chore(main): release 1.788.0 (#10664)
* chore(main): release 1.788.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
2ac3e64fe2 |
chore(main): release 1.787.0 (#10657)
* chore(main): release 1.787.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |