4 Commits

Author	SHA1	Message	Date
Ruben Fiszel	d131d754e1	feat: ducklake time-travel UX (snapshot history + AT VERSION reads) (#9709) ... * feat: ducklake time-travel UX (snapshot history + AT VERSION reads) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: catalog-qualify ducklake time-travel FROM hints (lake. prefix) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: render ducklake snapshot_time (microseconds since epoch) correctly Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: merge ducklake History + Query into one master-detail tab Snapshot list (left) selects the version previewed in the read-only grid (right); newest auto-selected. Copy-clause moved to the preview's SQL line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope ducklake snapshot history to the table's versions Catalog-wide snapshots predate a table's creation; previewing AT a version before the table existed errored ("Table ... does not exist at version N"). The DUCKLAKE_SNAPSHOTS marker now takes the table and lists only snapshots from its first creation onward. Also: narrower snapshot-list pane on large screens (target a fixed width, not a fixed fraction). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: load ducklake preview columns at the pinned version + reset on asset switch Addresses CI review (codex/pi P1, cubic P2): - Historical previews loaded current-schema columns, so an AT(VERSION) read enumerating a column added in a later snapshot failed. Now DESCRIBE-loads the column set at the pinned version; the read is gated on columns matching the current version to avoid a stale-colDefs race on version switch. - selectedVersion no longer sticks across assets: the panel is keyed on path (remounts per asset) and effectiveVersion falls back to newest when the pick isn't in the current list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: match History tab UI (master-detail, full-FROM copy) after merge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: handle catalog-only ducklake asset paths (no table segment) parseDbInputFromAssetSyntax threw on a catalog-only path like 'ducklake://main' (undefined.split('.')) — a real graph node (e.g. a consumer of the whole catalog). It now returns a table-less input instead of throwing, and DucklakeAssetPanel renders only the partition grid (no per-table history/time-travel) for table-less nodes. Adds parser unit tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: escape ducklake catalog name in client-built time-travel DESCRIBE fetchDucklakeColumnsAtVersion interpolated the catalog name into an ATTACH string literal without escaping; double single-quotes (mirrors backend escape_sql_literal) so a quote-containing catalog name can't break out. Also fixed the v1.x docs checklist line to match the shipped full-FROM copy affordance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-24 23:11:47 +02:00
Ruben Fiszel	3371265382	fix(frontend): ignore hash/assets in script diffs and drafts (WIN-2071) (#9664) ... * fix(frontend): ignore hash/assets in script diffs and drafts The script editor's draft value is seeded from the full `getScriptByPath` DB row (since #9351), so it carries `hash` (the deployed version's identity) and `assets` (re-derived from the script content by the editor). Neither is editable draft content, yet both were persisted into the draft row and surfaced as spurious changes in the workspace/fork compare diff view. - Add `hash`, `assets`, and the read-time-computed `inherited_labels` to `CLEANED_VALUE_KEYS` so the shared diff/unsaved-change strip ignores them everywhere (DiffDrawer + WorkspaceItemDiffViewer). - Strip `hash` and `assets` from script drafts at the single persistence chokepoint (`UserDraftDbSyncer.save`) so every path — reactive autosave, Ctrl/Cmd+S flush, the pagehide keepalive — sends the same trimmed payload. On reload the deployed row re-supplies them. Fixes WIN-2071 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(frontend): clarify draft sanitizer vs diff-strip relationship The two field lists are intentionally not equal — only the hash/assets overlap must stay consistent. Reword the comment so a future maintainer doesn't add keys to one expecting parity with the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-19 11:21:24 +00:00
Guilhem	c213801b5a	fix(frontend): strip server-managed fields from value diffs (#9599) ... * fix(frontend): strip server-managed fields from value diffs The script editor's "Deployed <> Current" diff rendered spurious metadata changes (created_at, created_by, extra_perms, lock_error_logs, ...) the user never touched. Since #9351 the edit loader fetches the full Script row via getScriptByPath (instead of the trimmed NewScript-shaped getScriptByPathWithDraft), so the editing object carries these DB-managed fields. The deployed side is trimmed in syncWithDeployed, so the two sides no longer match. Normalize both sides at the shared chokepoint: cleanValueProperties now also strips created_at, created_by, extra_perms, workspace_id, parent_hashes, lock and lock_error_logs. These are never user-editable, so this also fixes the draft<>current diff and unsaved-change detection, and benefits the flow/app diff viewers that share the helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(frontend): cover cleanValueProperties; keep lock in diffs Add unit tests for cleanValueProperties asserting the server-managed bookkeeping keys are stripped while user-editable keys survive. Keep `lock` out of the stripped set: it was part of the value comparison before the full-DB-row loader (#9351) and version-to-version diff viewers (WorkspaceItemDiffViewer) legitimately surface lockfile changes. Only the fields that the full Script row newly introduced as diff noise are stripped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): keep extra_perms in shared diff, strip script noise at source Address review: stripping `extra_perms` in the shared `cleanValueProperties` hid folder sharing-permission changes in workspace/fork diffs (the backend's compare_two_folders treats folder extra_perms as a real change). Remove it from the global skip-list so folder diffs surface it again. The script-editor noise it (and `lock`) would otherwise cause is now stripped at the source instead: `ScriptBuilder.openDiffDrawer` nulls `lock`/`extra_perms` on the current side to match the existing deployed-side strip in `syncWithDeployed`. Also strip the draft-overlay bookkeeping fields the full DB row carries (`draft_saved_at`, `draft_created_at`, `is_draft`, `other_drafts_users`) — they leaked into the current side and showed as spurious metadata diffs. Verified in browser: no-edit diff shows "No changes detected"; a summary edit surfaces only that change, with no lock/extra_perms/draft metadata noise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 08:58:22 +02:00
Diego Imbert	3affbb3321	feat: type-checked data tables v0 (#7381) ... * data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit 6e1588d59e. * datatables work (without leaking pg credentials) * println * separate sqlUtils.ts * nit * Separate custom instance db Select and Wizard components * nit * nit wording * add tags to custom instance dbs * error when trying to use ducklake as datatable or opposite * show status in dropdown * data table instance setup works * sqk function for ducklake * factorize logic * fix temp reactivity * Data table assetexplore * Migrate S3 permissions to modal * Revert "Migrate S3 permissions to modal" This reverts commit 0631d03cb0. * nit query -> fetch * Custom instance setup new look * run_language_executor separate fn * run_inline param * nit wording * Better typed client * Data tables display as assets in frontend * asset db icon * nit * cleaner errors * nit * Fix sed calls in mac * run_inline_script_preview in python client * basic python datatable client * datatable and datalake parser in python * ducklake client python * nit fix * Fix migration producing NULL instead of {} when no custom databases * merge conflict fail * python ducklake client arg fix * parse or infer sql types in ts client * ts asset parser, detect datatable & ducklake R/W * fix sql repl for other read ops than select * export type SqlTemplateFunction * rename list_custom_instance_pg_databases * typecheck datatable and ducklake name in Typescript * Fix typecheck datatable and ducklake in TS * declare module overriding instead of extending * infer_sql_type in python client * SqlQuery object in python * fix merge conflicts * update const_format * CI fix * factor out to var_identifiers * sqlx prepare * unnecessary security (admin is required) * clearer comment * ee repo ref * nit snake case * claude step 1: detect var declarations * move detect_sql_access_type to common mod * claude step 2: detect when saved vars are queried * Revert "claude step 2: detect when saved vars are queried" This reverts commit 1e1f930568. * Revert "claude step 1: detect var declarations" This reverts commit f866f4819d. * remove ducklake/datatable and default * detect data table assigns in var_identifiers * Python parser successfully infers R/W/RW from ducklake / datatable * still register ducklake/datatable if not used as unknown R/W * Go to settings button in Assets Dropdown on not found * nit * sqlx prepare fail * manual fix, somehow sqlx prepare won't do it * fix frontend ci * ee repo ref * ducklake_user doesnt exist in unit tests * nit fix * ui nit * nit * nit missing clone * fork ducklakes and datatables * fix surface hover bug * stupid mistake * better deeply reactive mutable derived * Ducklake picker * Editor bar data tables * DuckDB supports datatables * datatable in duckdb asset parser * duckdb asset parser var_identifiers * Revert "duckdb asset parser var_identifiers" This reverts commit 88068b1a77. * sqlx prepare * Box pin in test_workflow_as_code to fix stack overflow * stash * sql asset parser parses most s3 literals * nit * Detect attach + handle returning RW * detect assets used with dot notation * detect implicit access with USE dl; syntax * Add assets as unknown if var was never used * Support default ducklake/datatable main in parser * ignore asset parsing errors in frontend (avoid flow layout shift) * super weird duplication (merge conflict ?) * nits * fix duckdb parser detecting too much as asset when RW ctx is unknown * fix transparent assets btn * missing arg * nit styling * asset parser specific table parsing * fix resource specific table parsing * More concise asset display in flows + better icons * fix assets page filtering out resources with added table * Fix frontend to support specific table assets * Open DB Manager to specific table * Specific table parser in Python and TS + unit tests * Fix UPDATE setting access to None * fix flow edge rendering on top of output picker * python parser fix var override bug * add ts test * fix compilation * sqlx prepare * update parsers version * fix missing schema key onDelete * Grant permission to create schemas in custom instance databases * Update pg query to return empty schemas * Create schema * Select nits * support schemas in sql parser * ts parser handle schema with sql parser result * detect .schema() syntax * detect schema syntax in python * support .schema() in ts and py SDK * open db manager to specific schema * support reassignment in ts parser * nit better unitest * : syntax in ts * datatable:schema syntax in python * fix client py * nit select dropdown darkmode * object | null fetchOne * ts client nits * parse_sql_client_name fn * getImportWmillTsStatement refactor in EditorBar * text to json() in python client * update parser versions * pkg lock * Sql query details in TS asset parser * code transformation with type parameter in Editor * Custom Language Worker, code substition works ! * Error marker mapping works * hover info is correct * completions work correctly * other overrides * type inference kinda works * Position mapping tests * refactor prepare_queries * Refactor PgDatabase to share common code * Pgdatabase in prepare_queries * TokioPgConnection refactor * refactor prepare_queries * type parameter to sql function * Fix deadlock * nit fix * Fix worker async call freezing because of svelte Proxy * Force worker to recompute when we set queries * nit refactor * nits console logs * wait that ts worker initialize * monaco change file version * update diagnostics * Refactor for errors * Show SQL errors in Monaco * improve sdk * cleaning refactor + MapResource + usePreparedAssetSqlQueries * Fixes * Fix error position mapping * cache in typescript worker * fix insert no values * don't inject type if already present * Support schema in prepare queries * update parsers * ChangeOnDeepInequality * inferAsset ScriptEditor usage refactor * sql query typecheck work in flow editor * Assets and SQL Query check in Raw App Inline Editor * pkg lock * Fix DatatableSqlTemplateFunction nit * prepare query schema nit * duplicate diagnostics * nit getScriptVersion mock * Reprepare queries when switching workspaces * nit fix * nit fix * fetch_one_scalar and execute in python client * limit pg_connections * -- prepare flag in postgres * skip serializing * fix destructuring undefined * Prepare queries in workers instead of backend * nit * Execute search_path instructions normally * nit fix * Fix SET search_path issue in prepare * only support preparing single-statement queries for now * update parsers * safety * better remove_comments * Fix getQueryStmtCountHeuristic * getQueryStmtCountHeuristic tests * comment out failing tests * Fix getQueryStmtCountHeuristic impl * only datatable	2025-12-20 08:51:07 +00:00