Files
windmill/frontend/src/lib/components/DBManagerDrawer.svelte
T
3d4f4c6c38 feat: Fork datatables (#8339)
* export_datatable_schema

* Propose to fork the datatable on ws fork

* dump datatable

* Dockerfile

* Fix import_datatable_dump

* datatable schema fork works!

* Option to copy both schema and data

* Datatable fork behavior

* nit ui

* use psql instead

* remove fork_datatable route

* feat: add fork_pg_database and export_pg_schema routes with DB Manager UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: pluralize "schema" to "schemas" in DB Manager export/import UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add import mode select (schema only vs schema + data) to DB Manager import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Select schema or schema+data when important database

* fix: prepend $res: prefix to resource paths in DB Manager import/export

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: dynamic import button label based on selected mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nits

* feat: add warning alert when schema+data import mode is selected

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit hide on cloud hosted

* refactor: remove fork_behavior from datatable settings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split CreateWorkspace into layout wrapper and CreateWorkspaceInner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: instantiate CreateWorkspaceInner in globalForkModal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit icons

* Data table fork UI

* feat: pass per-datatable fork behaviors from UI to backend during workspace fork

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix fork overwriting all datatables

* UI nits

* custom instance db refactor

* custom instance db wizard btn for all in dropdown

* nit

* Delete custom instance database button

* Disable forking for resource datatables

* Big import buttons when db empty

* Revert "Disable forking for resource datatables"

This reverts commit 9561cc8fd4.

* feat: add non_diffable flag to resource table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add resource-type datatable fork with CREATE DATABASE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: tag forked datatables with nonDiffable and forkedFrom

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: diff datatable and ducklake settings individually on workspace merge

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: skip non_diffable resources and datatables in workspace diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: default datatable fork behavior to keep_original

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make grant permissions non-fatal in instance datatable fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make datatable and ducklake diffs visible in workspace comparison

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove datatable fork logic from workspace fork route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct ahead/behind logic for datatable and ducklake diffs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: correct ahead/behind logic for datatable and ducklake diffs"

This reverts commit 6b50884dc6.

* revert: remove datatable and ducklake settings diffing logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable clone UI with step-by-step confirmation modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract datatable fork UI into ForkDatatableSection component

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: run datatable cloning before workspace fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit disable fork admins

* nit fix switching workspace prematurely

* fix: use source workspace for forkPgDatabase calls during fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: update forked workspace datatable settings after fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add forked_from field to DataTable and set it for instance forks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit onFinish

* fix: add forked_from to DataTableSettings OpenAPI schema

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: track datatable table DDL changes in workspace_diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "feat: track datatable table DDL changes in workspace_diff"

This reverts commit 7526dd68b9.

* feat: add get_datatable_full_schema endpoint and snapshot schema on fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix duplicate migration key

* fix: set forked_from on datatable config for both instance and resource types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* feat: drop forked databases on workspace deletion with confirmation UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases from delete_workspace

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cast pg char columns to text in FK schema query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show dbname instead of resource type in fork deletion modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nit

* refactor: extract drop_custom_instance_database into windmill-common

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable schema diff section to merge UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* UI

* feat: add review drawer with YAML diff and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use Monaco DiffEditor for YAML diff in review drawer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* Revert "refactor: use Monaco DiffEditor for YAML diff in review drawer"

This reverts commit a86008ba4c.

* Revert "feat: add review drawer with YAML diff and SQL migration runner"

This reverts commit 0a0deb5ddb.

* feat: add review drawer with DiffEditor and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nits

* fix: show diff between forked_from schema and changed side

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: re-fetch target live schema after migration for correct baseline

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert

* nit auto next

* feat: add confirmation modal before deploying migration to parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle missing columns/foreignKeys in schema conversion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* refactor: use temp file on disk for pg_dump instead of in-memory string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Don't replace postgres dbname

* fix: add validation to drop_custom_instance_database and use source db for CREATE/DROP

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: type DataTable.forked_from as DataTableForkedFrom struct

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify fork_pg_database to take source + target_dbname

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* dead code

* feat: enforce schema_and_data admin-only and extract create_custom_instance_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: rename fork_pg_database to import_pg_database with source/target/override params

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* refactor: remove original_dbname/original_resource from forked_from, resolve from parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: resolve forked dbname from fork workspace when dropping resource databases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* fix: always clean up global_settings even if database doesn't exist

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: check datatable resource_type from config instead of URL prefix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: wrap PG default value expressions in braces to prevent CAST quoting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: wrap PG default value expressions in braces to prevent CAST quoting"

This reverts commit 77f5a2c4e8.

* refactor: reuse columnDefToTableEditorValuesColumn for default value handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: store raw API schema in forked_from to avoid double transformation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: store raw API schema in forked_from to avoid double transformation"

This reverts commit e326197a20.

* Revert "refactor: reuse columnDefToTableEditorValuesColumn for default value handling"

This reverts commit bd8f071d9f.

* fix: validate dbname with strict regex to prevent SQL injection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix default value

* always validate dbname

* refactor: move get_datatable_full_schema structs and logic to query_builders.rs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: split import_pg_database into create_pg_database + import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases into its own route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: require admin when using $res: resource paths in import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use UserDB for $res: resource access and restrict dbname creation

- resolve_pg_source_checked uses UserDB (row-level security) for $res: paths
- transform_json_unchecked is now pub(crate) to prevent misuse
- Non-superadmins can only create databases with wm_fork_ prefix
- datatable:// remains accessible to everyone

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: refuse to drop forked databases unless name starts with wm_fork_

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove resolve_pg_source, use resolve_pg_source_checked everywhere

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix permissions

* sqlx prepare

* compilation nits

* sqlx prepare

* sqlx prepare

* wrong route syntax

* fix: allow workspace owner to edit datatable config for fork setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: allow workspace owner to edit datatable config for fork setup"

This reverts commit ab683e637b.

* refactor: move datatable fork setup into create_workspace_fork backend

Instead of updating datatable settings from the frontend after fork
creation (which required admin/owner access), pass forked_datatables
info to create_workspace_fork and handle it atomically in the same
transaction. Removes applyPostForkDatatableUpdates from frontend.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: snapshot schema in backend during fork instead of frontend

The schema snapshot is now taken by the backend in apply_forked_datatable
via snapshot_datatable_schema, which connects to the parent workspace's
datatable and runs pg_get_full_schema. This removes the need for the
frontend to call getDatatableFullSchema and pass the schema through.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use get_resource_value_interpolated_internal for $res: to resolve $var: references

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* sqlx prepare

* fix: add permission check to drop_forked_datatable_databases, validate dbnames, restrict temp file perms

- drop_forked_datatable_databases: same permission as delete_workspace
  (fork owner or super admin)
- validate_dbname on target_dbname_override and ForkedDatatableInfo.new_dbname
- Enforce wm_fork_ prefix on forked datatable new_dbname
- DumpFile: set /tmp/windmill/ to 0700 and create files with 0600

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit CLI

* Rename to ws_specific

* sqlx prepare

* nit always validate dbname

* fix: include foreign keys in CREATE TABLE migration for added tables

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: detect nextval defaults and use SERIAL/BIGSERIAL types in CREATE TABLE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update frontend/src/lib/components/DBManagerDrawer.svelte

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* fix: sort foreign keys by constraint name for deterministic schema output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx prepare

* rename migration to update timestamp

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 21:03:06 +00:00

297 lines
8.3 KiB
Svelte

<script lang="ts">
import { superadmin, userStore, workspaceStore } from '$lib/stores'
import { WorkspaceService } from '$lib/gen'
import Button from './common/button/Button.svelte'
import Drawer from './common/drawer/Drawer.svelte'
import DrawerContent from './common/drawer/DrawerContent.svelte'
import Select from './select/Select.svelte'
import {
ArrowLeft,
Copy,
Download,
Expand,
LoaderCircle,
Minimize,
RefreshCcw,
Upload
} from 'lucide-svelte'
import DBManagerContent from './DBManagerContent.svelte'
import { resource } from 'runed'
import { untrack } from 'svelte'
import type { DbManagerUriState } from './dbManagerDrawerModel.svelte'
import ResourcePicker from './ResourcePicker.svelte'
import Alert from './common/alert/Alert.svelte'
import { sendUserToast } from '$lib/toast'
import { isCloudHosted } from '$lib/cloud'
interface Props {
uriState: DbManagerUriState
/** Z-index offset for the drawer, useful when opening from within modals */
offset?: number
}
let { uriState, offset = 0 }: Props = $props()
let open = $derived(uriState.open)
// Load available datatables when drawer opens with datatable input
const datatables = resource<string[]>([], async () => {
if (!$workspaceStore) return []
try {
return (await WorkspaceService.listDataTables({ workspace: $workspaceStore })).map(
(d) => d.name
)
} catch (e) {
console.error('Failed to load datatables:', e)
return []
}
})
const datatableItems = $derived(
datatables.current.map((dt) => ({
value: dt,
label: dt
}))
)
// Refetch datatables when switching to a datatable input
$effect(() => {
if (uriState.isDatatableInput) {
untrack(() => datatables.refetch())
}
})
function handleClose() {
uriState.closeDrawer()
dbManagerContent?.clearReplResult()
}
let windowWidth = $state(window.innerWidth)
let expand = $state(false)
$effect(() => {
if (!open) {
expand = false
uriState.closeDrawer()
}
})
let dbManagerContent: DBManagerContent | undefined = $state()
let hasReplResult = $state(false)
// Export/Import state
let exportDrawerOpen = $state(false)
let exportResult = $state('')
let importDrawerOpen = $state(false)
let importLoading = $state(false)
let importSource = $state<string | undefined>(undefined)
let importBehavior = $state<'schema_only' | 'schema_and_data'>('schema_only')
let isPostgresqlInput = $derived(
uriState.isDatatableInput ||
(uriState.input?.type === 'database' && uriState.input.resourceType === 'postgresql')
)
let enableImportExport = $derived(isPostgresqlInput)
function toSourceIdentifier(raw: string): string {
if (raw.startsWith('datatable://') || raw.startsWith('$res:')) return raw
return `$res:${raw}`
}
function currentSourceIdentifier(): string | undefined {
const input = uriState.effectiveInput
if (!input || input.type !== 'database') return undefined
return toSourceIdentifier(input.resourcePath)
}
async function handleExportSchema() {
const source = currentSourceIdentifier()
if (!source || !$workspaceStore) return
try {
exportResult = await WorkspaceService.exportPgSchema({
workspace: $workspaceStore,
requestBody: { source }
})
exportDrawerOpen = true
} catch (e) {
sendUserToast(`Failed to export schema: ${e}`, true)
}
}
async function handleImportDatabase() {
if (!importSource || !$workspaceStore) return
const target = currentSourceIdentifier()
if (!target) return
importLoading = true
try {
await WorkspaceService.importPgDatabase({
workspace: $workspaceStore,
requestBody: {
source: toSourceIdentifier(importSource),
target,
fork_behavior: importBehavior
}
})
sendUserToast('Database import completed successfully')
importDrawerOpen = false
importSource = undefined
dbManagerContent?.refresh()
} catch (e) {
sendUserToast(`Failed to import database: ${e}`, true)
} finally {
importLoading = false
}
}
</script>
<svelte:window bind:innerWidth={windowWidth} />
<Drawer
bind:open
size={expand ? `${windowWidth}px` : '1200px'}
preventEscape
{offset}
on:close={handleClose}
>
<DrawerContent
title={hasReplResult ? 'Query Result' : 'Database Manager'}
on:close={() => {
if (hasReplResult) {
dbManagerContent?.clearReplResult()
} else {
handleClose()
}
}}
CloseIcon={hasReplResult ? ArrowLeft : undefined}
noPadding
id="db-manager-drawer"
>
{#if uriState.effectiveInput && $workspaceStore}
{#key uriState.selectedDatatable}
<DBManagerContent
bind:this={dbManagerContent}
input={uriState.effectiveInput}
bind:hasReplResult
bind:selectedSchemaKey={uriState.selectedSchema}
bind:selectedTableKey={uriState.selectedTable}
onImport={enableImportExport
? (mode) => ((importDrawerOpen = true), (importBehavior = mode))
: undefined}
>
{#snippet dbSelector()}
{#if uriState.isDatatableInput}
{#if datatables.loading}
<div class="flex items-center gap-2 text-tertiary ml-2">
<LoaderCircle size={14} class="animate-spin" />
<span class="text-sm">Loading...</span>
</div>
{:else if datatables.current.length >= 1}
<Select
transformInputSelectedText={(s) => `Datatable: ${s}`}
items={datatableItems}
bind:value={uriState.selectedDatatable}
placeholder="Select data table"
size="md"
/>
{/if}
{/if}
{/snippet}
</DBManagerContent>
{/key}
{/if}
{#snippet actions()}
{#if enableImportExport}
<Button startIcon={{ icon: Download }} onClick={handleExportSchema}>Export</Button>
<Button startIcon={{ icon: Upload }} onClick={() => (importDrawerOpen = true)}>
Import
</Button>
{/if}
<Button
loading={dbManagerContent?.isLoading() ?? false}
on:click={() => {
dbManagerContent?.refresh()
dbManagerContent?.dbManager()?.dbTable()?.refresh()
}}
startIcon={{ icon: RefreshCcw }}
size="xs"
color="light"
>
Refresh
</Button>
<Button
on:click={() => (expand = !expand)}
startIcon={{ icon: expand ? Minimize : Expand }}
size="xs"
color="light"
/>
{/snippet}
</DrawerContent>
</Drawer>
<Drawer bind:open={exportDrawerOpen} size="800px" offset={offset + 1}>
<DrawerContent title="Export Schemas" on:close={() => (exportDrawerOpen = false)}>
{#if exportResult}
<div class="flex flex-col gap-2 h-full relative">
<pre class="overflow-auto text-xs bg-surface-secondary p-4 rounded flex-1">{exportResult}</pre>
<Button
size="xs"
color="light"
startIcon={{ icon: Copy }}
wrapperClasses="absolute top-2 right-2"
btnClasses="bg-surface-tertiary"
on:click={() => {
navigator.clipboard.writeText(exportResult)
sendUserToast('Copied to clipboard')
}}
>
Copy
</Button>
</div>
{/if}
</DrawerContent>
</Drawer>
<Drawer bind:open={importDrawerOpen} size="600px" offset={offset + 1}>
<DrawerContent title="Import Database" on:close={() => (importDrawerOpen = false)}>
<div class="flex flex-col gap-4">
<Alert type="warning" title="Warning">
This will import the schemas from the selected source into the current database. Existing
tables with the same names may be affected.
</Alert>
<div class="flex flex-col gap-2">
<span class="text-sm font-medium">Source database</span>
<ResourcePicker datatableAsPgResource bind:value={importSource} resourceType="postgresql" />
</div>
<div class="flex flex-col gap-2">
<span class="text-sm font-medium">Import mode</span>
<Select
items={[
{ value: 'schema_only', label: 'Schema only' },
...(isCloudHosted() || (!$superadmin && !$userStore?.is_admin)
? []
: [{ value: 'schema_and_data', label: 'Schema and data' }])
]}
bind:value={importBehavior}
/>
</div>
{#if importBehavior === 'schema_and_data'}
<Alert type="warning" title="Heavy operation">
Importing schema and data will copy all rows from every table in the source database. This
may take a long time and use significant storage space depending on the size of the
source.
</Alert>
{/if}
<Button
disabled={!importSource}
loading={importLoading}
color="red"
on:click={handleImportDatabase}
>
Import {importBehavior === 'schema_and_data' ? 'schemas and data' : 'schemas'} into current database
</Button>
</div>
</DrawerContent>
</Drawer>