mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-19 00:02:03 +00:00
5ad2de91a2
* feat(pipelines): ingestion (EL) templates + docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): review nits — draft collision guard, template-mode selection reset, invariant test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): lead the insert menu with ingestion templates Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(pipelines): ingestion story as docs-only — drop editor template UI The insert-menu template section mixed two selection grammars in one popover and confused more than it helped. The three E2E-verified example pipelines now live verbatim in docs/pipeline-ingestion.md; the Python bare-string S3 key fix in pipelineTemplates.ts stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sdk): bare string S3 keys in py/ts clients + asset parsers A plain string passed where an S3Object is expected is now a bare key in the default storage — previously the py client silently degraded it to s3="" (auto-generated key) and both asset parsers canonicalized it without the leading slash, splitting lineage. parseS3Object moves to s3Types.ts so it is unit-testable without the generated services. The pipeline template fix from the earlier commit is superseded (bare strings are the supported spelling again); docs examples flipped to bare keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(sdk): enforce s3:// URIs for string S3Object params Bare strings now raise/throw with a hint pointing at the s3:///<key> spelling instead of being treated as keys (previous commit) or silently degrading to an empty key (original behavior). One string spelling everywhere: SDK calls, // on annotations, and DuckDB SQL all use s3:///<key>. TS regains the s3://-template-literal type; the asset parsers record no asset for a bare string (the call can only error); templates emit the URI form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(pipelines): move ingestion (EL) guide to windmilldocs, keep design constraints User-facing how-to (engine choice, cursor recipes, schema drift, worked examples) moves to windmilldocs core_concepts/63_pipelines (windmilldocs#1462); the repo keeps only the design constraints future feature work must not break, as a section of ducklake-materialization.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate system prompts after parse_s3_object docstring change Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): reject empty-key s3 URIs; align asset parsers with the runtime rule Addresses CI review: s3:/// and s3://bucket/ now raise (an empty key would fall back to the auto-generated-key path the strict contract exists to prevent); the asset parsers' string branch applies the same valid-URI-with-non-empty-key rule so no R/W edge is recorded for a call that can only error (the generic URI-literal scan still records ambiguous access-None assets, by design); comments rephrased as current constraints per AGENTS.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
53 lines
1.7 KiB
TypeScript
53 lines
1.7 KiB
TypeScript
import { describe, expect, test } from "bun:test";
|
|
import { parseS3Object } from "../s3Types";
|
|
|
|
describe("parseS3Object", () => {
|
|
test("bare string throws with the s3:/// hint", () => {
|
|
// A bare key is rejected rather than silently uploading under an
|
|
// auto-generated name; the error points at the s3:/// spelling.
|
|
expect(() => parseS3Object("dir/file.json" as any)).toThrow(
|
|
/s3:\/\/\/dir\/file\.json/
|
|
);
|
|
});
|
|
|
|
test("triple-slash URI targets the default storage", () => {
|
|
expect(parseS3Object("s3:///dir/file.json")).toEqual({
|
|
storage: undefined,
|
|
s3: "dir/file.json",
|
|
});
|
|
});
|
|
|
|
test("full URI splits storage and key", () => {
|
|
expect(parseS3Object("s3://bucket/dir/f")).toEqual({
|
|
storage: "bucket",
|
|
s3: "dir/f",
|
|
});
|
|
});
|
|
|
|
test("malformed s3:// URI throws", () => {
|
|
// `s3://x` has no key part — fail loudly instead of silently misplacing
|
|
// the object.
|
|
expect(() => parseS3Object("s3://broken" as any)).toThrow(
|
|
/Invalid s3 object/
|
|
);
|
|
});
|
|
|
|
test("empty-key URIs throw", () => {
|
|
// An empty key is never a valid target: it would fall back to an
|
|
// auto-generated key, which is requested by omitting the object.
|
|
expect(() => parseS3Object("s3:///" as any)).toThrow(/Invalid s3 object/);
|
|
expect(() => parseS3Object("s3://bucket/")).toThrow(/Invalid s3 object/);
|
|
});
|
|
|
|
test("empty string throws (omit the object for an auto-generated key)", () => {
|
|
expect(() => parseS3Object("" as any)).toThrow(/Invalid s3 object/);
|
|
});
|
|
|
|
test("record form passes through", () => {
|
|
expect(parseS3Object({ s3: "x", storage: "b" })).toEqual({
|
|
s3: "x",
|
|
storage: "b",
|
|
});
|
|
});
|
|
});
|