Files
windmill/frontend/src/lib/components/assets/AssetsDropdownButton.svelte
T
Diego Imbert 3d4f4c6c38 feat: Fork datatables (#8339)
* export_datatable_schema

* Propose to fork the datatable on ws fork

* dump datatable

* Dockerfile

* Fix import_datatable_dump

* datatable schema fork works!

* Option to copy both schema and data

* Datatable fork behavior

* nit ui

* use psql instead

* remove fork_datatable route

* feat: add fork_pg_database and export_pg_schema routes with DB Manager UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: pluralize "schema" to "schemas" in DB Manager export/import UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add import mode select (schema only vs schema + data) to DB Manager import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Select schema or schema+data when important database

* fix: prepend $res: prefix to resource paths in DB Manager import/export

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: dynamic import button label based on selected mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nits

* feat: add warning alert when schema+data import mode is selected

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit hide on cloud hosted

* refactor: remove fork_behavior from datatable settings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split CreateWorkspace into layout wrapper and CreateWorkspaceInner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: instantiate CreateWorkspaceInner in globalForkModal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit icons

* Data table fork UI

* feat: pass per-datatable fork behaviors from UI to backend during workspace fork

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix fork overwriting all datatables

* UI nits

* custom instance db refactor

* custom instance db wizard btn for all in dropdown

* nit

* Delete custom instance database button

* Disable forking for resource datatables

* Big import buttons when db empty

* Revert "Disable forking for resource datatables"

This reverts commit 9561cc8fd4.

* feat: add non_diffable flag to resource table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add resource-type datatable fork with CREATE DATABASE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: tag forked datatables with nonDiffable and forkedFrom

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: diff datatable and ducklake settings individually on workspace merge

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: skip non_diffable resources and datatables in workspace diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: default datatable fork behavior to keep_original

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make grant permissions non-fatal in instance datatable fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make datatable and ducklake diffs visible in workspace comparison

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove datatable fork logic from workspace fork route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct ahead/behind logic for datatable and ducklake diffs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: correct ahead/behind logic for datatable and ducklake diffs"

This reverts commit 6b50884dc6.

* revert: remove datatable and ducklake settings diffing logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable clone UI with step-by-step confirmation modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract datatable fork UI into ForkDatatableSection component

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: run datatable cloning before workspace fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit disable fork admins

* nit fix switching workspace prematurely

* fix: use source workspace for forkPgDatabase calls during fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: update forked workspace datatable settings after fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add forked_from field to DataTable and set it for instance forks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit onFinish

* fix: add forked_from to DataTableSettings OpenAPI schema

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: track datatable table DDL changes in workspace_diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "feat: track datatable table DDL changes in workspace_diff"

This reverts commit 7526dd68b9.

* feat: add get_datatable_full_schema endpoint and snapshot schema on fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix duplicate migration key

* fix: set forked_from on datatable config for both instance and resource types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* feat: drop forked databases on workspace deletion with confirmation UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases from delete_workspace

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cast pg char columns to text in FK schema query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show dbname instead of resource type in fork deletion modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nit

* refactor: extract drop_custom_instance_database into windmill-common

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable schema diff section to merge UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* UI

* feat: add review drawer with YAML diff and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use Monaco DiffEditor for YAML diff in review drawer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* Revert "refactor: use Monaco DiffEditor for YAML diff in review drawer"

This reverts commit a86008ba4c.

* Revert "feat: add review drawer with YAML diff and SQL migration runner"

This reverts commit 0a0deb5ddb.

* feat: add review drawer with DiffEditor and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nits

* fix: show diff between forked_from schema and changed side

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: re-fetch target live schema after migration for correct baseline

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert

* nit auto next

* feat: add confirmation modal before deploying migration to parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle missing columns/foreignKeys in schema conversion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* refactor: use temp file on disk for pg_dump instead of in-memory string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Don't replace postgres dbname

* fix: add validation to drop_custom_instance_database and use source db for CREATE/DROP

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: type DataTable.forked_from as DataTableForkedFrom struct

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify fork_pg_database to take source + target_dbname

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* dead code

* feat: enforce schema_and_data admin-only and extract create_custom_instance_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: rename fork_pg_database to import_pg_database with source/target/override params

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* refactor: remove original_dbname/original_resource from forked_from, resolve from parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: resolve forked dbname from fork workspace when dropping resource databases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* fix: always clean up global_settings even if database doesn't exist

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: check datatable resource_type from config instead of URL prefix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: wrap PG default value expressions in braces to prevent CAST quoting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: wrap PG default value expressions in braces to prevent CAST quoting"

This reverts commit 77f5a2c4e8.

* refactor: reuse columnDefToTableEditorValuesColumn for default value handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: store raw API schema in forked_from to avoid double transformation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: store raw API schema in forked_from to avoid double transformation"

This reverts commit e326197a20.

* Revert "refactor: reuse columnDefToTableEditorValuesColumn for default value handling"

This reverts commit bd8f071d9f.

* fix: validate dbname with strict regex to prevent SQL injection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix default value

* always validate dbname

* refactor: move get_datatable_full_schema structs and logic to query_builders.rs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: split import_pg_database into create_pg_database + import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases into its own route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: require admin when using $res: resource paths in import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use UserDB for $res: resource access and restrict dbname creation

- resolve_pg_source_checked uses UserDB (row-level security) for $res: paths
- transform_json_unchecked is now pub(crate) to prevent misuse
- Non-superadmins can only create databases with wm_fork_ prefix
- datatable:// remains accessible to everyone

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: refuse to drop forked databases unless name starts with wm_fork_

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove resolve_pg_source, use resolve_pg_source_checked everywhere

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix permissions

* sqlx prepare

* compilation nits

* sqlx prepare

* sqlx prepare

* wrong route syntax

* fix: allow workspace owner to edit datatable config for fork setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: allow workspace owner to edit datatable config for fork setup"

This reverts commit ab683e637b.

* refactor: move datatable fork setup into create_workspace_fork backend

Instead of updating datatable settings from the frontend after fork
creation (which required admin/owner access), pass forked_datatables
info to create_workspace_fork and handle it atomically in the same
transaction. Removes applyPostForkDatatableUpdates from frontend.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: snapshot schema in backend during fork instead of frontend

The schema snapshot is now taken by the backend in apply_forked_datatable
via snapshot_datatable_schema, which connects to the parent workspace's
datatable and runs pg_get_full_schema. This removes the need for the
frontend to call getDatatableFullSchema and pass the schema through.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use get_resource_value_interpolated_internal for $res: to resolve $var: references

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* sqlx prepare

* fix: add permission check to drop_forked_datatable_databases, validate dbnames, restrict temp file perms

- drop_forked_datatable_databases: same permission as delete_workspace
  (fork owner or super admin)
- validate_dbname on target_dbname_override and ForkedDatatableInfo.new_dbname
- Enforce wm_fork_ prefix on forked datatable new_dbname
- DumpFile: set /tmp/windmill/ to 0700 and create files with 0600

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit CLI

* Rename to ws_specific

* sqlx prepare

* nit always validate dbname

* fix: include foreign keys in CREATE TABLE migration for added tables

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: detect nextval defaults and use SERIAL/BIGSERIAL types in CREATE TABLE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update frontend/src/lib/components/DBManagerDrawer.svelte

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* fix: sort foreign keys by constraint name for deterministic schema output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx prepare

* rename migration to update timestamp

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 21:03:06 +00:00

225 lines
6.8 KiB
Svelte

<script lang="ts">
import { clone, pluralize } from '$lib/utils'
import { Pyramid } from 'lucide-svelte'
import { twMerge } from 'tailwind-merge'
import { Popover } from '../meltComponents'
import S3FilePicker from '../S3FilePicker.svelte'
import {
assetsEq,
formatAssetAccessType,
formatAssetKind,
getAccessType,
type Asset,
type AssetWithAltAccessType
} from './lib'
import { untrack } from 'svelte'
import { ResourceService, WorkspaceService } from '$lib/gen'
import { workspaceStore } from '$lib/stores'
import Tooltip from '../meltComponents/Tooltip.svelte'
import Tooltip2 from '../Tooltip.svelte'
import ResourceEditorDrawer from '../ResourceEditorDrawer.svelte'
import type { Placement } from '@floating-ui/core'
import AssetButtons from './AssetButtons.svelte'
import ToggleButtonGroup from '../common/toggleButton-v2/ToggleButtonGroup.svelte'
import ToggleButton from '../common/toggleButton-v2/ToggleButton.svelte'
import { resource } from 'runed'
let {
assets,
enableChangeAnimation = true,
size = 'xs',
noBtnText = false,
popoverPlacement = 'bottom-end',
disableLiTooltip = false,
onHoverLi,
liSubtitle
}: {
assets: AssetWithAltAccessType[]
enableChangeAnimation?: boolean
size?: 'xs' | '3xs'
noBtnText?: boolean
popoverPlacement?: Placement
disableLiTooltip?: boolean
onHoverLi?: (asset: Asset, eventType: 'enter' | 'leave') => void
liSubtitle?: (asset: Asset) => string
} = $props()
let prevAssets = $state<typeof assets>([])
let blueBgDiv: HTMLDivElement | undefined = $state()
let s3FilePicker: S3FilePicker | undefined = $state()
let resourceEditorDrawer: ResourceEditorDrawer | undefined = $state()
let isOpen = $state(false)
let resourceDataCache: Record<string, string | undefined> = $state({})
$effect(() => {
if (!enableChangeAnimation) {
if (blueBgDiv) {
blueBgDiv.classList.remove('animate-fade-out')
}
}
})
let datatables = resource([], () =>
WorkspaceService.listDataTables({ workspace: $workspaceStore ?? '' }).then((d) =>
d.map((d) => d.name)
)
)
let ducklakes = resource([], () =>
WorkspaceService.listDucklakes({ workspace: $workspaceStore ?? '' })
)
$effect(() => {
assets
untrack(() => {
if (assetsEq(assets, prevAssets)) return
prevAssets = clone(assets)
// Replay animation
if (blueBgDiv && enableChangeAnimation) {
blueBgDiv.classList.add('animate-fade-out')
blueBgDiv.style.animation = 'none'
blueBgDiv.offsetHeight /* trigger reflow */
blueBgDiv.style.animation = ''
}
for (const asset of assets) {
if (asset.kind == 'resource') {
let truncatedPath = asset.path.split('?table=')[0]
if (truncatedPath in resourceDataCache) continue
resourceDataCache[truncatedPath] = undefined // avoid fetching multiple times because of async
ResourceService.getResource({ path: truncatedPath, workspace: $workspaceStore! })
.then((r) => (resourceDataCache[truncatedPath] = r.resource_type))
.catch((err) => console.error("Couldn't fetch resource", truncatedPath, err))
}
}
})
})
</script>
<Popover
floatingConfig={{ strategy: 'absolute', placement: popoverPlacement }}
usePointerDownOutside
closeOnOtherPopoverOpen
bind:isOpen
escapeBehavior="ignore"
>
{#snippet trigger()}
<div
class={twMerge(
size === '3xs' ? 'h-[1.6rem]' : 'py-1.5',
'text-xs flex items-center gap-1.5 px-2 rounded-md relative',
'border',
'bg-surface hover:bg-surface-hover active:bg-surface',
'transition-all hover:text-primary backdrop-blur-md cursor-pointer'
)}
>
<div
bind:this={blueBgDiv}
class="absolute pointer-events-none bg-slate-300 dark:bg-[#576278] inset-0 rounded-md opacity-0"
></div>
<Pyramid size={size === '3xs' ? 13 : 16} class="z-10" />
<span
class={twMerge('z-10 font-normal', size === '3xs' ? 'text-3xs mt-[0.08rem]' : 'text-xs')}
>
{noBtnText ? assets.length : pluralize(assets.length, 'asset')}
</span>
</div>
{/snippet}
{#snippet content()}
<ul class="divide-y rounded-md">
{#each assets as asset}
{@const ducklakeNotFound =
asset.kind === 'ducklake' &&
ducklakes.current &&
!ducklakes.current.find((name) => name === asset.path.split('/')[0])}
{@const datatableNotFound =
asset.kind === 'datatable' &&
datatables.current &&
!datatables.current.find((name) => name === asset.path.split('/')[0])}
<li
class="text-sm px-3 h-12 flex gap-3 items-center"
onmouseenter={() => onHoverLi?.(asset, 'enter')}
onmouseleave={() => onHoverLi?.(asset, 'leave')}
>
<Popover
contentClasses="py-2 px-4 flex flex-col gap-2"
disablePopup={!!asset.access_type}
>
{#snippet trigger()}
<div
class={twMerge(
'text-xs font-normal border text-primary w-10 p-1 text-center rounded-md',
!asset.access_type ? 'text-orange-600 !border-orange-600' : '',
!asset.access_type ? 'hover:bg-surface active:opacity-80' : ''
)}
>
{formatAssetAccessType(getAccessType(asset)) ?? '?'}
</div>
{/snippet}
{#snippet content()}
{#if !asset.access_type}
<span class="text-sm text-primary leading-4">
Could not infer automatically <br />
<span class="text-xs">Please select manually </span>
</span>
<div class="flex items-center gap-2">
<ToggleButtonGroup
allowEmpty
bind:selected={asset.alt_access_type}
class="max-w-fit"
>
{#snippet children({ item })}
<ToggleButton value="r" label="Read" {item} />
<ToggleButton value="w" label="Write" {item} />
<ToggleButton value="rw" label="Read/Write" {item} />
{/snippet}
</ToggleButtonGroup>
<Tooltip2>
This is used to determine if the asset should be displayed as an input or an
output node in the flow editor
</Tooltip2>
</div>
{/if}
{/snippet}
</Popover>
<div class="flex flex-col flex-1">
<Tooltip class="select-none w-48 truncate" disablePopup={disableLiTooltip}>
{asset.path}
{#snippet text()}
{asset.path}
{/snippet}
</Tooltip>
<span class="text-xs text-primary select-none">
{liSubtitle?.(asset) ??
formatAssetKind({
...asset,
...(asset.kind === 'resource'
? {
metadata: {
resource_type: resourceDataCache[asset.path.split('?table=')[0]]
}
}
: {})
})}
</span>
</div>
<AssetButtons
onClick={() => (isOpen = false)}
{asset}
{resourceDataCache}
{resourceEditorDrawer}
{s3FilePicker}
{ducklakeNotFound}
{datatableNotFound}
/>
</li>
{/each}
</ul>
{/snippet}
</Popover>
<S3FilePicker bind:this={s3FilePicker} readOnlyMode />
<ResourceEditorDrawer bind:this={resourceEditorDrawer} />