mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-13 08:02:47 +00:00
* fix(s3_proxy): enforce CE 50MB upload cap on multipart uploads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): replace CE 50MB upload cap with 10GiB workspace storage quota Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): gate CE quota OSS stubs to not(enterprise) to match callers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): delta-aware CE storage quota + guard usage-load retry loop Account for the overwritten object's size in the quota check so valid same-size overwrites near quota are not rejected (Codex review), and stop the storage-usage $effect from re-firing on persistent API errors (Pi review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): count chunked PUTs; revert overreaching volume quota copy Volumes write to workspace storage via a separate worker-side path with its own 50MB-per-file cap that this PR does not change, so revert the drawer copy that claimed they count toward the 10GiB quota (Codex review). Bump ee-repo-ref for the chunked-PUT accounting fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): add SQLx cache for CE usage-bump/quota queries; exclude volumes Regenerate the missing offline SQLx cache for the not(enterprise) bump and remaining-quota queries so the private CE offline build compiles, and bump ee-repo-ref for the volumes/-prefix exclusion from the counted quota (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): always HEAD for CE upload delta so overwrites don't inflate usage Bump ee-repo-ref for the fast-path overwrite-accounting fix (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): reserve volumes/ prefix on CE write surfaces to close quota bypass Reject direct writes to the reserved volume prefix on the app-upload surface and add the OSS stub; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): refuse new multipart work when CE workspace is at quota Bump ee-repo-ref for the multipart-initiate/part quota gate (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): reserve in-flight multipart bytes against CE storage quota Add workspace_multipart_inflight table + grants, SQLx cache for the reservation queries, and bump ee-repo-ref. Bounds abandoned multipart uploads that the list-based recount can't see (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): clear multipart reservation only after a successful complete Add exclude-upload arg to the OSS quota stub/caller and the SQLx cache for the updated remaining-quota query; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): per-part multipart reservation; commit only on part success Per-part workspace_multipart_inflight schema (upload_id, part_id) so retries replace rather than double-count; SQLx cache for the reworked queries; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(s3): HEAD the multipart overwrite target once per upload, not per part SQLx cache for the stored-credit lookup; bump ee-repo-ref. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to bea5a8b5120d6d69cab1ad4611ebe463559bd200 This commit updates the EE repository reference after PR #640 was merged in windmill-ee-private. Previous ee-repo-ref: 6e6ff86f1939cf74736b7d435bf6851416437523 New ee-repo-ref: bea5a8b5120d6d69cab1ad4611ebe463559bd200 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
33 lines
1.8 KiB
SQL
33 lines
1.8 KiB
SQL
-- Reservation for the parts of in-flight (initiated but not yet completed)
|
|
-- multipart uploads to workspace object storage. Uncommitted parts occupy
|
|
-- object-store capacity but are invisible to the list-based storage recount
|
|
-- until completion, so CE folds this reservation into the remaining quota to
|
|
-- bound abandoned uploads. One row per uploaded part so a re-uploaded part
|
|
-- (same part_id) replaces rather than double-counts; a part is recorded only
|
|
-- after its upstream upload succeeds. Rows are removed on successful complete
|
|
-- and lazily expired after a TTL (abort/abandon rely on the TTL, which matches
|
|
-- when the object store reaps the uncommitted parts).
|
|
-- part_id - S3 part number or Azure block id (string)
|
|
-- part_bytes - size of that part
|
|
-- target_existing_size - size of the object the upload will overwrite (0 if new),
|
|
-- credited so an overwrite only reserves the net growth
|
|
CREATE TABLE workspace_multipart_inflight (
|
|
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace (id) ON DELETE CASCADE,
|
|
upload_id VARCHAR(512) NOT NULL,
|
|
part_id VARCHAR(256) NOT NULL,
|
|
storage VARCHAR(255) NOT NULL,
|
|
part_bytes BIGINT NOT NULL DEFAULT 0,
|
|
target_existing_size BIGINT NOT NULL DEFAULT 0,
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
|
PRIMARY KEY (workspace_id, upload_id, part_id)
|
|
);
|
|
|
|
CREATE INDEX idx_workspace_multipart_inflight_created_at
|
|
ON workspace_multipart_inflight (created_at);
|
|
|
|
-- Tables created after the one-time GRANT ALL in 20250205131523 need explicit
|
|
-- grants: ALTER DEFAULT PRIVILEGES only covers objects created by the role that
|
|
-- set them (same gap as workspace_storage_usage, notify_event, script_trigger).
|
|
GRANT ALL ON workspace_multipart_inflight TO windmill_user;
|
|
GRANT ALL ON workspace_multipart_inflight TO windmill_admin;
|