starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-18 16:02:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: column-level asset tracking for ducklake and datatables (#7774)

Browse Source 
* Refactor 1

* claude tmp1

* fixes1

* support for insert and update

* Fix returning

* docs: add nuanced mutex selection guidance to Rust backend skill (#7737)

Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): revert findCodebase change that broke ../shared codebases (#7740)

* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.621.2 (#7735)

* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: workspace dedicated workers (#7741)

* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove uuid-ossp extension requirement for RDS compatibility

The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: indexer build error (#7744)

* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml

* refactor: remove seed parameter from AI chat completions (#7745)

* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)

* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode

- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: forward teams error to client (#7746)

* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* feat: add token usage tracking to AI agent output (#7738)

* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.622.0 (#7742)

* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix npm check

* fix: add schema compatibility layer for MCP clients like n8n (#7747)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* nit ui nextcloud triggers (#7749)

* feat: add PocketID OAuth provider support (#7318)

* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>

* fix: preserve script envs field during sync push

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit frontend fix

* chore(main): release 1.623.0 (#7748)

* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: resolve infinite effect loop in PocketIdSetting component (#7753)

* fix: prevent retention cleanup from deleting jobs of active flows (#7755)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.623.1 (#7754)

* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: default to quickjs on ce for flow eval (#7756)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: runtime assets (#7656)

* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove $schema field from Google AI output schema requests (#7765)

* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Fix app multiselect not refreshing result when creating element (#7766)

* Fix returning

* asset columns are saved and displayed in the assets page

* runtime assets column detectionz

* frontend nits

* update regex parsers

* UI nits

* Display asset columns in flow graph

* Column hint directly in asset node

* nit bg

* sqlx prepare

* ee repo ref

* chore: update ee-repo-ref to 66a68df97e8c65c498b28f302a365ab8687cad9e

This commit updates the EE repository reference after PR #402 was merged in windmill-ee-private.

Previous ee-repo-ref: 0a32bc104cbaec9664a4d7cb1565823722c875a1

New ee-repo-ref: 66a68df97e8c65c498b28f302a365ab8687cad9e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
This commit is contained in:
Diego Imbert
2026-02-04 17:27:53 +01:00
committed by GitHub
parent 5e62690d52
commit 0caa533fbd
28 changed files with 1068 additions and 312 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/.sqlx/query-a750630d79166f5b6d5be0e049c36135212666ce74dbf290d701b6402b800f13.json
Generated
+35
View File
@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_strip_nulls(jsonb_build_object(\n 'path', path,\n 'kind', kind,\n 'access_type', usage_access_type,\n 'columns', columns\n )) as \"list!: _\"\n FROM asset\n WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3\n ORDER BY path, kind",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "list!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
}
]
},
"nullable": [
null
]
},
"hash": "a750630d79166f5b6d5be0e049c36135212666ce74dbf290d701b6402b800f13"
}
backend/.sqlx/query-a9e29764b5b9d94269e2b8aa755c71b61774c8ff8ae218d7a8d6ed0ac0169366.json
Generated
+55
View File
@@ -0,0 +1,55 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, columns)\n VALUES ($1, $2, $3, $4, $5, $6, $7) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
{
"Custom": {
"name": "asset_access_type",
"kind": {
"Enum": [
"r",
"w",
"rw"
]
}
}
},
"Varchar",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
},
"Jsonb"
]
},
"nullable": []
},
"hash": "a9e29764b5b9d94269e2b8aa755c71b61774c8ff8ae218d7a8d6ed0ac0169366"
}
backend/ee-repo-ref.txt
+1 -1
View File
@@ -1 +1 @@
88e49a7c9746080a8a95e30828655d5783a616d6
88e49a7c9746080a8a95e30828655d5783a616d6
backend/migrations/20260203122047_asset_columns.down.sql
+2
View File
@@ -0,0 +1,2 @@
-- Remove columns field from asset table
ALTER TABLE asset DROP COLUMN columns;
backend/migrations/20260203122047_asset_columns.up.sql
+3
View File
@@ -0,0 +1,3 @@
-- Add columns field to asset table to store column-level access information
-- This is a JSONB map of column name to access type (r, w, or rw)
ALTER TABLE asset ADD COLUMN columns JSONB;
backend/parsers/windmill-parser-py/src/asset_parser.rs
+48 -21
View File
@@ -19,9 +19,12 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
// if a db = wmill.datatable() was never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
if asset_was_used(&assets_finder.assets, (kind, &path)) == false {
assets_finder
.assets
.push(ParseAssetsResult { kind, access_type: None, path });
assets_finder.assets.push(ParseAssetsResult {
kind,
path,
access_type: None,
columns: None,
});
}
}
@@ -48,8 +51,12 @@ impl Visitor for AssetsFinder {
match removed {
Some((kind, path, _)) => {
if !asset_was_used(&self.assets, (kind, &path)) {
self.assets
.push(ParseAssetsResult { kind, access_type: None, path });
self.assets.push(ParseAssetsResult {
kind,
path,
access_type: None,
columns: None,
});
}
}
None => {}
@@ -76,6 +83,7 @@ impl Visitor for AssetsFinder {
kind,
path: path.to_string(),
access_type: None,
columns: None,
});
}
}
@@ -97,6 +105,7 @@ impl Visitor for AssetsFinder {
kind,
path: path.to_string(),
access_type: None,
columns: None,
});
}
}
@@ -252,8 +261,12 @@ impl AssetsFinder {
let path = parse_asset_syntax(&value, false)
.map(|(_, p)| p)
.unwrap_or(&value);
self.assets
.push(ParseAssetsResult { kind, path: path.to_string(), access_type });
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
access_type,
columns: None,
});
}
_ => return Err(()),
};
@@ -281,7 +294,8 @@ def main():
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -299,7 +313,8 @@ def main():
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main".to_string(),
access_type: None
access_type: None,
columns: None,
},])
);
}
@@ -318,7 +333,8 @@ def main(x: int):
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -340,12 +356,14 @@ def main(x: int):
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/analytics".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: None,
},
])
);
@@ -372,17 +390,20 @@ def g():
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1/customers".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None,
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "another2".to_string(),
access_type: None
access_type: None,
columns: None,
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},
])
);
@@ -404,12 +425,14 @@ def g():
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1".to_string(),
access_type: None
access_type: None,
columns: None,
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
access_type: None,
columns: None,
},
])
);
@@ -429,7 +452,8 @@ def main(x: int):
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -448,7 +472,8 @@ def main():
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "lake1/analytics.metrics".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -468,7 +493,8 @@ def main(x: int):
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: None,
},])
);
}
@@ -486,7 +512,8 @@ def main():
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
access_type: None,
columns: None,
},])
);
}
backend/parsers/windmill-parser-sql/src/asset_parser.rs
+585 -70
View File
@@ -1,9 +1,9 @@
use std::collections::HashMap;
use std::collections::BTreeMap;
use sqlparser::{
ast::{
CopyTarget, Expr, ObjectName, TableFactor, TableObject, Value, ValueWithSpan, Visit,
Visitor,
CopyTarget, Expr, ObjectName, ObjectNamePart, SelectItem, TableFactor, TableObject, Value,
ValueWithSpan, Visit, Visitor,
},
dialect::DuckDbDialect,
parser::Parser,
@@ -24,9 +24,12 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
for (_, (kind, path)) in collector.var_identifiers {
if !asset_was_used(&collector.assets, (kind, &path)) {
collector
.assets
.push(ParseAssetsResult { kind, access_type: None, path: path });
collector.assets.push(ParseAssetsResult {
kind,
access_type: None,
path: path,
columns: None,
});
}
}
@@ -39,7 +42,7 @@ struct AssetCollector {
// e.g set to Read when we are inside a SELECT ... FROM ... statement
current_access_type_stack: Vec<AssetUsageAccessType>,
// e.g ATTACH 'ducklake://a' AS dl; => { "dl": (Ducklake, "a") }
var_identifiers: HashMap<String, (AssetKind, String)>,
var_identifiers: BTreeMap<String, (AssetKind, String)>,
// e.g USE dl;
currently_used_asset: Option<(AssetKind, String)>,
}
@@ -49,15 +52,19 @@ impl AssetCollector {
Self {
assets: Vec::new(),
current_access_type_stack: Vec::with_capacity(8),
var_identifiers: HashMap::new(),
var_identifiers: BTreeMap::new(),
currently_used_asset: None,
}
}
// Detect when we do 'a.b' and 'a' is associated with an asset in var_identifiers
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(&self, name: &ObjectName) -> Option<ParseAssetsResult> {
let access_type = self.current_access_type_stack.last().copied();
fn get_associated_asset_from_obj_name(
&self,
name: &ObjectName,
access_type: Option<AssetUsageAccessType>,
) -> Option<ParseAssetsResult> {
let access_type = access_type.or_else(|| self.current_access_type_stack.last().copied());
if let Some((kind, path)) = &self.currently_used_asset {
// We don't want to infer that any simple identifier refers to an asset if
// we are not in a known R/W context
@@ -81,7 +88,7 @@ impl AssetCollector {
.collect::<Option<Vec<String>>>()?
.join(".");
let path = format!("{}/{}", path, specific_table);
return Some(ParseAssetsResult { kind: *kind, access_type, path });
return Some(ParseAssetsResult { kind: *kind, access_type, path, columns: None });
}
}
@@ -101,7 +108,7 @@ impl AssetCollector {
} else {
path.clone()
};
Some(ParseAssetsResult { kind: *kind, access_type, path })
Some(ParseAssetsResult { kind: *kind, access_type, path, columns: None })
}
fn handle_string_literal(&mut self, s: &str) {
@@ -112,6 +119,7 @@ impl AssetCollector {
kind,
path: path.to_string(),
access_type: self.current_access_type_stack.last().copied(),
columns: None,
});
}
}
@@ -126,13 +134,6 @@ impl AssetCollector {
if let Some(str_lit) = get_str_lit_from_obj_name(name) {
self.handle_string_literal(str_lit);
}
// Writes to tables should be handled directly when visiting the statement
if self.current_access_type_stack.last() == Some(&R) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
}
fn handle_obj_name_post(&mut self, name: &ObjectName) {
@@ -146,20 +147,144 @@ impl AssetCollector {
}
}
fn handle_table_with_joins(&mut self, table_with_joins: &sqlparser::ast::TableWithJoins) {
if let TableFactor::Table { name, .. } = &table_with_joins.relation {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
fn handle_table_with_joins(
&mut self,
table_with_joins: &sqlparser::ast::TableWithJoins,
access_type: Option<AssetUsageAccessType>,
) {
if let TableFactor::Table { name, args, .. } = &table_with_joins.relation {
if args.is_some() && args.as_ref().map_or(0, |a| a.args.len()) > 0 {
return;
}
if let Some(asset) = self.get_associated_asset_from_obj_name(name, access_type) {
self.assets.push(asset);
}
}
for join in &table_with_joins.joins {
if let TableFactor::Table { name, .. } = &join.relation {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, access_type) {
self.assets.push(asset);
}
}
}
}
// Extract columns from SELECT items and create individual asset results for each column
// Only processes columns that reference known assets to avoid false positives
fn extract_column_assets(
&mut self,
projection: &[SelectItem],
from_tables: &[sqlparser::ast::TableWithJoins],
) {
// Check if this is a single-table SELECT (to avoid ambiguity)
let single_table = if from_tables.len() == 1 {
if let TableFactor::Table { name, args, .. } = &from_tables[0].relation {
if args.is_some() && args.as_ref().map_or(0, |a| a.args.len()) > 0 {
return; // Skip table functions
}
self.get_associated_asset_from_obj_name(name, Some(R))
} else {
None
}
} else {
None
};
// Build a map of table aliases/names to assets for multi-table queries
let mut table_to_asset: BTreeMap<String, ParseAssetsResult> = BTreeMap::new();
for table_with_joins in from_tables {
if let TableFactor::Table { name, alias, args, .. } = &table_with_joins.relation {
if args.is_some() && args.as_ref().map_or(0, |a| a.args.len()) > 0 {
continue; // Skip table functions
}
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(R)) {
// Use alias if present, otherwise use the table name
let table_key = if let Some(alias) = alias {
alias.name.value.clone()
} else {
// For qualified names like "dl.table1", use just the last part
name.0
.last()
.and_then(|id| id.as_ident())
.map(|id| id.value.clone())
.unwrap_or_default()
};
table_to_asset.insert(table_key, asset);
}
}
}
// Process each SELECT item
for item in projection {
match item {
SelectItem::UnnamedExpr(Expr::Identifier(ident))
| SelectItem::ExprWithAlias { expr: Expr::Identifier(ident), .. } => {
// Simple column: SELECT a
// Only add if we have a single table (unambiguous)
if let Some(asset) = &single_table {
let mut columns = BTreeMap::new();
columns.insert(ident.value.clone(), R);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(R),
columns: Some(columns),
});
}
}
SelectItem::UnnamedExpr(Expr::CompoundIdentifier(parts))
| SelectItem::ExprWithAlias { expr: Expr::CompoundIdentifier(parts), .. } => {
// Qualified column: SELECT table1.a or SELECT x.table1.a
if parts.len() >= 2 {
let column_name = parts.last().map(|id| id.value.clone());
if let Some(column_name) = column_name {
// Check if the prefix matches a known table
let table_prefix = parts.first().map(|id| id.value.clone());
if let Some(table_prefix) = table_prefix {
if let Some(asset) = table_to_asset.get(&table_prefix) {
// Found a matching table, add column asset
let mut columns = BTreeMap::new();
columns.insert(column_name.clone(), R);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(R),
columns: Some(columns),
});
} else if parts.len() >= 3 {
// Could be x.table1.column format or db.schema.table.column
// Convert Idents to ObjectNameParts
let obj_parts: Vec<ObjectNamePart> = parts[..parts.len() - 1]
.iter()
.cloned()
.map(|ident| ObjectNamePart::Identifier(ident))
.collect();
let obj_name = ObjectName(obj_parts);
if let Some(asset) =
self.get_associated_asset_from_obj_name(&obj_name, Some(R))
{
let mut columns = BTreeMap::new();
columns.insert(column_name.clone(), R);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(R),
columns: Some(columns),
});
}
}
}
}
}
}
_ => {
// Ignore wildcards, expressions, etc.
}
}
}
}
}
impl Visitor for AssetCollector {
@@ -218,51 +343,154 @@ impl Visitor for AssetCollector {
statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(_) => {
// don't forget pop() in post_visit_statement
self.current_access_type_stack.push(R);
sqlparser::ast::Statement::Query(q) => {
if let Some(select) = q.body.as_select() {
// First, handle table references (adds table-level assets)
for t in &select.from {
self.handle_table_with_joins(t, Some(R));
}
// Then, extract column-level assets
self.extract_column_assets(&select.projection, &select.from);
}
}
sqlparser::ast::Statement::Insert(insert) => {
let access_type = if insert.returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
match insert.table {
TableObject::TableName(ref name) => {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
if let Some(asset) =
self.get_associated_asset_from_obj_name(name, Some(access_type))
{
// Add table-level asset
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: asset.access_type,
columns: None,
});
// Extract column information for INSERT with explicit columns (Write access)
if !insert.columns.is_empty() {
for col in &insert.columns {
let columns = BTreeMap::from([(col.value.clone(), W)]);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(W),
columns: Some(columns),
});
}
}
// Extract column information from RETURNING clause (Read access)
if let Some(returning) = &insert.returning {
for item in returning {
match item {
SelectItem::UnnamedExpr(Expr::Identifier(ident))
| SelectItem::ExprWithAlias {
expr: Expr::Identifier(ident),
..
} => {
let mut col_map = BTreeMap::new();
col_map.insert(ident.value.clone(), R);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(R),
columns: Some(col_map),
});
}
_ => {
// Ignore wildcards and complex expressions
}
}
}
}
}
}
_ => {}
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::Update { returning, table, from, .. } => {
sqlparser::ast::Statement::Update { returning, table, from, assignments, .. } => {
if let Some(from_tables) = from {
let from_tables = match from_tables {
sqlparser::ast::UpdateTableFromKind::AfterSet(tables) => tables,
sqlparser::ast::UpdateTableFromKind::BeforeSet(tables) => tables,
};
self.current_access_type_stack.push(R);
for table_with_joins in from_tables {
self.handle_table_with_joins(table_with_joins);
self.handle_table_with_joins(table_with_joins, Some(R));
}
self.current_access_type_stack.pop();
}
let access_type = if returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
self.handle_table_with_joins(table, Some(access_type));
self.handle_table_with_joins(table);
// Extract column information from UPDATE SET clauses (Write access)
// Only process if it's a single table update
if let TableFactor::Table { name, .. } = &table.relation {
if let Some(asset) =
self.get_associated_asset_from_obj_name(name, Some(access_type))
{
// Process each assignment to extract column names
for assignment in assignments {
// assignment.target is an AssignmentTarget enum
// We only handle simple column names (ColumnName variant)
if let sqlparser::ast::AssignmentTarget::ColumnName(col_name) =
&assignment.target
{
// For simple column updates, this is typically a single ident
if col_name.0.len() == 1 {
if let Some(col_ident) =
col_name.0.first().and_then(|p| p.as_ident())
{
let mut col_map = BTreeMap::new();
col_map.insert(col_ident.value.clone(), W);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(W),
columns: Some(col_map),
});
}
}
}
}
self.current_access_type_stack.pop();
// Extract column information from RETURNING clause (Read access)
if let Some(returning_items) = returning {
for item in returning_items {
match item {
SelectItem::UnnamedExpr(Expr::Identifier(ident))
| SelectItem::ExprWithAlias {
expr: Expr::Identifier(ident),
..
} => {
let mut col_map = BTreeMap::new();
col_map.insert(ident.value.clone(), R);
self.assets.push(ParseAssetsResult {
kind: asset.kind,
path: asset.path.clone(),
access_type: Some(R),
columns: Some(col_map),
});
}
_ => {
// Ignore wildcards and complex expressions
}
}
}
}
}
}
}
sqlparser::ast::Statement::Delete(delete) => {
let access_type = if delete.returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
for name in &delete.tables {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
if let Some(asset) =
self.get_associated_asset_from_obj_name(name, Some(access_type))
{
self.assets.push(asset);
}
}
@@ -271,25 +499,22 @@ impl Visitor for AssetCollector {
sqlparser::ast::FromTable::WithoutKeyword(tables) => tables,
};
for table_with_joins in tables {
self.handle_table_with_joins(table_with_joins);
self.handle_table_with_joins(table_with_joins, Some(access_type));
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::CreateTable(create_table) => {
self.current_access_type_stack.push(W);
if let Some(asset) = self.get_associated_asset_from_obj_name(&create_table.name) {
if let Some(asset) =
self.get_associated_asset_from_obj_name(&create_table.name, Some(W))
{
self.assets.push(asset);
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::CreateView { name, .. } => {
self.current_access_type_stack.push(W);
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(W)) {
self.assets.push(asset);
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::Copy { target: CopyTarget::File { filename }, .. } => {
@@ -339,14 +564,8 @@ impl Visitor for AssetCollector {
fn post_visit_statement(
&mut self,
statement: &sqlparser::ast::Statement,
_statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(_) => {
self.current_access_type_stack.pop();
}
_ => {}
}
std::ops::ControlFlow::Continue(())
}
@@ -409,17 +628,20 @@ mod tests {
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/a.parquet".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None
},
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/c.parquet".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None
},
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "snd/b.parquet".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None
},
])
);
@@ -438,7 +660,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl".to_string(),
access_type: None
access_type: None,
columns: None
},])
);
}
@@ -455,7 +678,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/table1".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None
},])
);
}
@@ -473,7 +697,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "my_dt/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None
},])
);
}
@@ -504,7 +729,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None
},])
);
}
@@ -521,7 +747,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None
},])
);
}
@@ -543,7 +770,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/friends".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: None
},])
);
}
@@ -561,7 +789,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
access_type: None,
columns: None
},])
);
}
@@ -579,7 +808,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None
},])
);
}
@@ -597,7 +827,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: Some(BTreeMap::from([("id".to_string(), W)])),
},])
);
}
@@ -615,7 +846,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Resource,
path: "u/user/pg_resource/table1".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None
},])
);
}
@@ -632,7 +864,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: Some(BTreeMap::from([("id".to_string(), W)])),
},])
);
}
@@ -650,7 +883,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: Some(BTreeMap::from([("id".to_string(), W)])),
},])
);
}
@@ -669,8 +903,289 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: Some(BTreeMap::from([("id".to_string(), W)])),
},])
);
}
#[test]
fn test_sql_asset_parser_single_table_column_detection() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT a, b FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should have one asset with merged columns
assert_eq!(result.len(), 1);
assert_eq!(result[0].path, "my_dl/table1");
assert_eq!(result[0].access_type, Some(R));
// Check that both columns are present in the merged asset
let columns = result[0].columns.as_ref().expect("Should have columns");
assert_eq!(columns.len(), 2);
assert_eq!(columns.get("a"), Some(&R));
assert_eq!(columns.get("b"), Some(&R));
}
#[test]
fn test_sql_asset_parser_explicit_table_prefix_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT dl.table1.a, dl.table1.b FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
// Should detect columns with explicit table prefix
let result = s.unwrap();
// Check we have the table asset
// Check we have column assets
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("a"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("b"))
}));
}
#[test]
fn test_sql_asset_parser_multi_table_no_simple_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT a, b FROM dl.table1, dl.table2;
"#;
let s = parse_assets(input).map(|s| s.assets);
// Simple columns (a, b) should NOT be detected with multiple tables
// Only table-level assets should be present
let result = s.unwrap();
// Should have 2 table assets
assert_eq!(result.iter().filter(|a| a.columns.is_none()).count(), 2);
// Should have NO column assets (ambiguous which table they belong to)
assert_eq!(result.iter().filter(|a| a.columns.is_some()).count(), 0);
}
#[test]
fn test_sql_asset_parser_multi_table_with_qualified_columns() {
let input = r#"
ATTACH 'ducklake://my_dl1' AS dl1;
ATTACH 'ducklake://my_dl2' AS dl2;
SELECT table1.a, table2.b FROM dl1.table1, dl2.table2;
"#;
let s = parse_assets(input).map(|s| s.assets);
// Qualified columns should be detected even with multiple tables
let result = s.unwrap();
// Check we have column assets for both tables
assert!(result.iter().any(|a| {
a.path == "my_dl1/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("a"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl2/table2"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("b"))
}));
}
#[test]
fn test_sql_asset_parser_use_with_simple_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
SELECT a, b, c FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should detect columns since it's a single table
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("a"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("b"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("c"))
}));
}
#[test]
fn test_sql_asset_parser_wildcard_no_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Wildcard should NOT create column assets, only table asset
assert_eq!(result.len(), 1);
assert!(result[0].columns.is_none());
}
#[test]
fn test_sql_asset_parser_columns_with_alias() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT a AS column_a, b AS column_b FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should detect columns even when aliased
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("a"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("b"))
}));
}
#[test]
fn test_sql_asset_parser_columns_with_table_alias() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT t.a, t.b FROM dl.table1 AS t;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should detect columns using the table alias
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("a"))
}));
assert!(result.iter().any(|a| {
a.path == "my_dl/table1"
&& a.columns
.as_ref()
.map_or(false, |cols| cols.contains_key("b"))
}));
}
#[test]
fn test_sql_asset_parser_insert_with_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
INSERT INTO dl.table1 (name, age, email) VALUES ('John', 30, 'john@example.com');
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should have one asset with merged columns
assert_eq!(result.len(), 1);
assert_eq!(result[0].path, "my_dl/table1");
assert_eq!(result[0].access_type, Some(W));
// Check that all columns are present in the merged asset
let columns = result[0].columns.as_ref().expect("Should have columns");
assert_eq!(columns.len(), 3);
assert_eq!(columns.get("name"), Some(&W));
assert_eq!(columns.get("age"), Some(&W));
assert_eq!(columns.get("email"), Some(&W));
}
#[test]
fn test_sql_asset_parser_insert_without_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
INSERT INTO dl.table1 VALUES ('John', 30);
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should have one asset without column information
assert_eq!(result.len(), 1);
assert_eq!(result[0].path, "my_dl/table1");
assert_eq!(result[0].access_type, Some(W));
assert!(result[0].columns.is_none());
}
#[test]
fn test_sql_asset_parser_update_multiple_columns() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
UPDATE dl.table1 SET name = 'Jane', age = 25, active = true;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should have one asset with merged columns
assert_eq!(result.len(), 1);
assert_eq!(result[0].path, "my_dl/table1");
assert_eq!(result[0].access_type, Some(W));
// Check that all columns are present
let columns = result[0].columns.as_ref().expect("Should have columns");
assert_eq!(columns.len(), 3);
assert_eq!(columns.get("name"), Some(&W));
assert_eq!(columns.get("age"), Some(&W));
assert_eq!(columns.get("active"), Some(&W));
}
#[test]
fn test_sql_asset_parser_update_returning() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
UPDATE dl.table1 SET name = 'Jane', age = 26 RETURNING id, name;
"#;
let s = parse_assets(input).map(|s| s.assets);
let result = s.unwrap();
// Should have RW access type when RETURNING is used
assert_eq!(result.len(), 1);
assert_eq!(result[0].path, "my_dl/table1");
assert_eq!(result[0].access_type, Some(RW));
// Check that columns are present with correct access types
// name and age are written (W), id and name are read (R)
// name should be RW (both written and read)
let columns = result[0].columns.as_ref().expect("Should have columns");
assert_eq!(columns.len(), 3);
assert_eq!(columns.get("name"), Some(&RW)); // Written in SET, read in RETURNING
assert_eq!(columns.get("age"), Some(&W)); // Only written
assert_eq!(columns.get("id"), Some(&R)); // Only read
}
}
backend/parsers/windmill-parser-ts/src/asset_parser.rs
+45 -20
View File
@@ -117,6 +117,7 @@ impl Visit for AssetsFinder {
kind,
path: path.to_string(),
access_type: None,
columns: None,
});
}
}
@@ -177,8 +178,12 @@ impl Visit for AssetsFinder {
if asset_was_used(&self.assets, (kind, path)) {
continue;
}
self.assets
.push(ParseAssetsResult { kind, access_type: None, path: path.clone() });
self.assets.push(ParseAssetsResult {
kind,
access_type: None,
path: path.clone(),
columns: None,
});
}
// Restore state - identifiers declared in this block go out of scope
@@ -294,8 +299,12 @@ impl AssetsFinder {
let path = parse_asset_syntax(&value, false)
.map(|(_, p)| p)
.unwrap_or(&value);
self.assets
.push(ParseAssetsResult { kind, path: path.to_string(), access_type });
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
access_type,
columns: None,
});
}
_ => return Err(()),
}
@@ -321,7 +330,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -340,7 +350,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
access_type: None,
columns: None,
},])
);
}
@@ -360,7 +371,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -383,12 +395,14 @@ mod tests {
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/analytics".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: None,
},
])
);
@@ -418,17 +432,20 @@ mod tests {
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1/customers".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None,
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "another2".to_string(),
access_type: None
access_type: None,
columns: None,
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},
])
);
@@ -452,12 +469,14 @@ mod tests {
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1".to_string(),
access_type: None
access_type: None,
columns: None,
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
access_type: None,
columns: None,
},
])
);
@@ -478,7 +497,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/myschema.friends".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -499,7 +519,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
access_type: Some(RW),
columns: None,
},])
);
}
@@ -518,7 +539,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
access_type: None,
columns: None,
},])
);
}
@@ -539,7 +561,8 @@ mod tests {
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/users".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},])
);
}
@@ -562,12 +585,14 @@ mod tests {
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/private.users".to_string(),
access_type: Some(R)
access_type: Some(R),
columns: None,
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/test".to_string(),
access_type: Some(W)
access_type: Some(W),
columns: None,
},
])
);
backend/parsers/windmill-parser-yaml/src/asset_parser.rs
+3
View File
@@ -12,6 +12,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
kind: AssetKind::Resource,
path: delegate_to_git_repo_details.resource,
access_type: Some(AssetUsageAccessType::R),
columns: None,
})
}
@@ -21,6 +22,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
kind: AssetKind::Resource,
path: pinned_res,
access_type: Some(AssetUsageAccessType::R),
columns: None,
})
}
}
@@ -31,6 +33,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
kind: AssetKind::Resource,
path: resource,
access_type: Some(AssetUsageAccessType::R),
columns: None,
})
}
}
backend/parsers/windmill-parser/src/asset_parser.rs
+36 -1
View File
@@ -1,4 +1,5 @@
use serde::Serialize;
use std::collections::BTreeMap;
#[derive(Serialize, PartialEq, Clone, Copy, Debug)]
#[serde(rename_all(serialize = "lowercase"))]
@@ -19,12 +20,14 @@ pub enum AssetKind {
DataTable,
}
#[derive(Serialize, Debug, PartialEq)]
#[derive(Serialize, Debug, PartialEq, Clone)]
pub struct ParseAssetsResult {
pub kind: AssetKind,
pub path: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub access_type: Option<AssetUsageAccessType>, // None in case of ambiguity
#[serde(skip_serializing_if = "Option::is_none")]
pub columns: Option<BTreeMap<String, AssetUsageAccessType>>, // Map column name to access type, "*" represents wildcard
}
#[derive(Serialize, Debug, PartialEq)]
@@ -66,6 +69,8 @@ pub fn merge_assets(assets: Vec<ParseAssetsResult>) -> Vec<ParseAssetsResult> {
(Some(R), Some(R)) => Some(R),
(Some(W), Some(W)) => Some(W),
};
// merge columns: union the column sets and merge access types per column
existing.columns = merge_column_maps(existing.columns.take(), asset.columns);
} else {
arr.push(asset);
}
@@ -74,6 +79,36 @@ pub fn merge_assets(assets: Vec<ParseAssetsResult>) -> Vec<ParseAssetsResult> {
arr
}
fn merge_column_maps(
existing: Option<BTreeMap<String, AssetUsageAccessType>>,
new: Option<BTreeMap<String, AssetUsageAccessType>>,
) -> Option<BTreeMap<String, AssetUsageAccessType>> {
match (existing, new) {
(None, None) => None,
(Some(map), None) | (None, Some(map)) => Some(map),
(Some(mut existing_map), Some(new_map)) => {
for (col_name, new_access) in new_map {
existing_map
.entry(col_name)
.and_modify(|existing_access| {
*existing_access = merge_access_types(*existing_access, new_access);
})
.or_insert(new_access);
}
Some(existing_map)
}
}
}
fn merge_access_types(a: AssetUsageAccessType, b: AssetUsageAccessType) -> AssetUsageAccessType {
match (a, b) {
(R, W) | (W, R) => RW,
(RW, _) | (_, RW) => RW,
(R, R) => R,
(W, W) => W,
}
}
// Will return false if the user assigned an asset to a variable like:
// let sql = wmill.datatable('main')
// But never used it. In that case we don't know which table is being used,
backend/windmill-api/openapi.yaml
+5
View File
@@ -16526,6 +16526,11 @@ paths:
$ref: "#/components/schemas/AssetUsageKind"
access_type:
$ref: "#/components/schemas/AssetUsageAccessType"
columns:
type: object
description: The columns used (for tables)
additionalProperties:
$ref: "#/components/schemas/AssetUsageAccessType"
created_at:
type: string
format: date-time
backend/windmill-api/src/assets.rs
+5 -3
View File
@@ -158,6 +158,7 @@ async fn list_assets(
'path', asset.usage_path,
'kind', asset.usage_kind,
'access_type', asset.usage_access_type,
'columns', asset.columns,
'created_at', asset.created_at,
'metadata', (CASE
WHEN asset.usage_kind = 'job' THEN
@@ -266,11 +267,12 @@ async fn list_assets_by_usages(
for usage in body.usages {
let assets = sqlx::query_scalar!(
r#"SELECT
jsonb_build_object(
jsonb_strip_nulls(jsonb_build_object(
'path', path,
'kind', kind,
'access_type', usage_access_type
) as "list!: _"
'access_type', usage_access_type,
'columns', columns
)) as "list!: _"
FROM asset
WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3
ORDER BY path, kind"#,
backend/windmill-api/src/jobs.rs
+8
View File
@@ -6378,10 +6378,18 @@ fn register_potential_assets_on_inline_execution(
match assets {
Some(Ok(assets)) => {
for asset in assets {
let columns = asset.columns.as_ref().map(|cols| {
cols.iter()
.map(|(col_name, col_access_type)| {
(col_name.clone(), (*col_access_type).into())
})
.collect()
});
register_runtime_asset(InsertRuntimeAssetParams {
access_type: asset.access_type.map(|a| a.into()),
asset_kind: asset.kind.into(),
asset_path: asset.path,
columns,
job_id,
workspace_id: w_id.to_string(),
created_at: None,
backend/windmill-common/src/assets.rs
+37 -15
View File
@@ -1,5 +1,6 @@
use serde::{Deserialize, Serialize};
use sqlx::PgExecutor;
use std::collections::BTreeMap;
use crate::{error, scripts::ScriptHash};
@@ -37,23 +38,15 @@ pub enum AssetUsageAccessType {
RW,
}
pub struct Asset {
pub path: String,
pub kind: AssetKind,
}
pub struct AssetUsage {
pub path: String,
pub kind: AssetUsageKind,
pub access_type: AssetUsageAccessType,
}
#[derive(Serialize, Deserialize, Debug, Clone, Hash, sqlx::Type)]
#[derive(Serialize, Deserialize, Debug, Clone, Hash)]
pub struct AssetWithAltAccessType {
pub path: String,
pub kind: AssetKind,
pub access_type: Option<AssetUsageAccessType>,
pub alt_access_type: Option<AssetUsageAccessType>,
/// Map of column name to access type for column-level access tracking
#[serde(skip_serializing_if = "Option::is_none")]
pub columns: Option<BTreeMap<String, AssetUsageAccessType>>,
}
pub async fn insert_static_asset_usage<'e>(
@@ -63,15 +56,22 @@ pub async fn insert_static_asset_usage<'e>(
usage_path: &str,
usage_kind: AssetUsageKind,
) -> error::Result<()> {
// Convert columns BTreeMap to JSONB format
let columns_json = asset
.columns
.as_ref()
.map(|cols| serde_json::to_value(cols).unwrap_or(serde_json::Value::Null));
sqlx::query!(
r#"INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind)
VALUES ($1, $2, $3, $4, $5, $6) ON CONFLICT DO NOTHING"#,
r#"INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, columns)
VALUES ($1, $2, $3, $4, $5, $6, $7) ON CONFLICT DO NOTHING"#,
workspace_id,
asset.path,
asset.kind as AssetKind,
(asset.access_type.or(asset.alt_access_type)) as Option<AssetUsageAccessType>,
usage_path,
usage_kind as AssetUsageKind
usage_kind as AssetUsageKind,
columns_json as Option<serde_json::Value>
)
.execute(executor)
.await?;
@@ -125,6 +125,28 @@ pub fn merge_asset_usage_access_types(
}
}
pub fn merge_asset_columns(
a: &Option<BTreeMap<String, AssetUsageAccessType>>,
b: &Option<BTreeMap<String, AssetUsageAccessType>>,
) -> Option<BTreeMap<String, AssetUsageAccessType>> {
match (a, b) {
(None, None) => None,
(Some(cols), None) | (None, Some(cols)) => Some(cols.clone()),
(Some(cols_a), Some(cols_b)) => {
let mut merged = cols_a.clone();
for (col, access_b) in cols_b {
let access_a = merged.get(col);
let merged_access =
merge_asset_usage_access_types(access_a.cloned(), Some(*access_b));
if let Some(access) = merged_access {
merged.insert(col.clone(), access);
}
}
Some(merged)
}
}
}
impl From<windmill_parser::asset_parser::AssetKind> for AssetKind {
fn from(parser_kind: windmill_parser::asset_parser::AssetKind) -> Self {
match parser_kind {
backend/windmill-common/src/runtime_assets.rs
+13 -4
View File
@@ -1,13 +1,19 @@
use std::{collections::HashMap, sync::OnceLock};
use std::{
collections::{BTreeMap, HashMap},
sync::OnceLock,
};
use itertools::Itertools;
use serde_json::value::RawValue;
use sqlx::{Pool, Postgres, QueryBuilder};
use sqlx::{types::Json, Pool, Postgres, QueryBuilder};
use tokio::sync::mpsc;
use windmill_parser::asset_parser::parse_asset_syntax;
use crate::{
assets::{merge_asset_usage_access_types, AssetKind, AssetUsageAccessType, AssetUsageKind},
assets::{
merge_asset_columns, merge_asset_usage_access_types, AssetKind, AssetUsageAccessType,
AssetUsageKind,
},
error,
};
@@ -59,6 +65,7 @@ pub struct InsertRuntimeAssetParams {
pub job_id: uuid::Uuid,
pub access_type: Option<AssetUsageAccessType>,
pub created_at: Option<chrono::DateTime<chrono::Utc>>,
pub columns: Option<BTreeMap<String, AssetUsageAccessType>>,
}
async fn insert_runtime_assets(
@@ -66,13 +73,14 @@ async fn insert_runtime_assets(
assets: &[InsertRuntimeAssetParams],
) -> error::Result<()> {
for chunk in assets.chunks(1000) {
let mut query_builder = QueryBuilder::new("INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, created_at) ");
let mut query_builder = QueryBuilder::new("INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, columns, usage_kind, created_at) ");
query_builder.push_values(chunk, |mut b, asset| {
b.push_bind(&asset.workspace_id)
.push_bind(&asset.asset_path)
.push_bind(&asset.asset_kind)
.push_bind(&asset.access_type)
.push_bind(asset.job_id.to_string())
.push_bind(Json(&asset.columns))
.push_bind(&AssetUsageKind::Job)
.push_bind(&asset.created_at);
});
@@ -104,6 +112,7 @@ async fn prune_runtime_assets(
// Same job used the same asset multiple times
last_same_job.access_type =
merge_asset_usage_access_types(last_same_job.access_type, asset.access_type);
last_same_job.columns = merge_asset_columns(&last_same_job.columns, &asset.columns);
} else if v.len() < max_n {
v.push(asset);
}
backend/windmill-worker/src/worker.rs
+9 -2
View File
@@ -1533,7 +1533,11 @@ pub async fn run_worker(
let is_dedicated_worker: bool = {
let config = WORKER_CONFIG.read().await;
config.dedicated_worker.is_some() || config.dedicated_workers.as_ref().is_some_and(|dws| !dws.is_empty())
config.dedicated_worker.is_some()
|| config
.dedicated_workers
.as_ref()
.is_some_and(|dws| !dws.is_empty())
};
#[cfg(feature = "benchmark")]
@@ -2045,7 +2049,9 @@ pub async fn run_worker(
dedicated_workers.get(&key)
})
} else {
job.runnable_path.as_ref().and_then(|path| dedicated_workers.get(path))
job.runnable_path
.as_ref()
.and_then(|path| dedicated_workers.get(path))
};
if let Some(dedicated_worker_tx) = dedicated_worker_tx {
let dedicated_job = DedicatedWorkerJob {
@@ -2773,6 +2779,7 @@ async fn detect_and_store_runtime_assets_from_job_args(
asset_kind: asset.kind,
access_type: None,
created_at: None,
columns: None,
};
register_runtime_asset(asset);
}
frontend/package-lock.json
Generated
+53 -141
View File
@@ -78,11 +78,11 @@
"windmill-parser-wasm-java": "1.510.1",
"windmill-parser-wasm-nu": "1.510.1",
"windmill-parser-wasm-php": "1.574.1",
"windmill-parser-wasm-py": "1.601.1",
"windmill-parser-wasm-regex": "1.593.0",
"windmill-parser-wasm-py": "1.623.1",
"windmill-parser-wasm-regex": "1.623.1",
"windmill-parser-wasm-ruby": "1.526.1",
"windmill-parser-wasm-rust": "1.558.1",
"windmill-parser-wasm-ts": "1.593.0",
"windmill-parser-wasm-ts": "1.623.1",
"windmill-parser-wasm-yaml": "1.593.0",
"windmill-sql-datatype-parser-wasm": "1.512.0",
"windmill-utils-internal": "^1.3.2",
@@ -258,7 +258,6 @@
"integrity": "sha512-cjQ7ZlQ0Mv3b47hABuTevyTuYN4i+loJKGeV9flcCgIK37cCXRh+L1bd3iBHlynerhQ7BhCkn2BPbQUL+rGqFg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/helper-validator-identifier": "^7.27.1",
"js-tokens": "^4.0.0",
@@ -274,7 +273,6 @@
"integrity": "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=6.9.0"
}
@@ -820,7 +818,6 @@
}
],
"license": "MIT",
"peer": true,
"engines": {
"node": "^14 || ^16 || >=18"
},
@@ -1330,6 +1327,7 @@
"integrity": "sha512-Jer+M7DgIwT5IHfTayb4Iw/fkkxWNmC/mqn/nMh9JrbPbkxmyabfLQnhJ+JDn5HK77f84j34lubO3iqFtYAfMg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@floating-ui/core": "^1.3.1",
"@floating-ui/dom": "^1.4.5",
@@ -2105,6 +2103,7 @@
"integrity": "sha512-Vp3zX/qlwerQmHMP6x0Ry1oY7eKKRcOWGc2P59srOp4zcqyn+etJyQpELgOi4+ZSUgteX8Y387NuwruLgGXLUQ==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@standard-schema/spec": "^1.0.0",
"@sveltejs/acorn-typescript": "^1.0.5",
@@ -2182,6 +2181,7 @@
"integrity": "sha512-YZs/OSKOQAQCnJvM/P+F1URotNnYNeU3P2s4oIpzm1uFaqUEqRxUB0g5ejMjEb5Gjb9/PiBI5Ktrq4rUUF8UVQ==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@sveltejs/vite-plugin-svelte-inspector": "^5.0.0",
"debug": "^4.4.1",
@@ -2708,8 +2708,7 @@
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.5.tgz",
"integrity": "sha512-hov8bUuiLiyFPGyFPE1lwWhmzYbirOXQNNo40+y3zow8aFVTeyn3VWL0VFFfdNddA8S4Vf0Tc062rzyNr7Paag==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/@types/ms": {
"version": "2.1.0",
@@ -2722,8 +2721,7 @@
"resolved": "https://registry.npmjs.org/@types/normalize-package-data/-/normalize-package-data-2.4.4.tgz",
"integrity": "sha512-37i+OaWTh9qeK4LSHPsyRC7NahnGotNuZvjLSgcPzblpHB3rrCJxAOgI5gCdKm7coonsaX1Of0ILiTcnZjbfxA==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/@types/semver": {
"version": "7.7.1",
@@ -2793,6 +2791,7 @@
"integrity": "sha512-VlJEV0fOQ7BExOsHYAGrgbEiZoi8D+Bl2+f6V2RrXerRSylnp+ZBHmPvaIa8cz0Ajx7WO7Z5RqfgYg7ED1nRhA==",
"dev": true,
"license": "BSD-2-Clause",
"peer": true,
"dependencies": {
"@typescript-eslint/scope-manager": "5.62.0",
"@typescript-eslint/types": "5.62.0",
@@ -2961,7 +2960,6 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@vitest/mocker": "4.0.15",
"@vitest/utils": "4.0.15",
@@ -2986,7 +2984,6 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@vitest/browser": "4.0.15",
"@vitest/mocker": "4.0.15",
@@ -3012,7 +3009,6 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@vitest/spy": "4.0.15",
"estree-walker": "^3.0.3",
@@ -3041,7 +3037,6 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@vitest/spy": "4.0.15",
"estree-walker": "^3.0.3",
@@ -3273,6 +3268,7 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
"integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
"license": "MIT",
"peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -3326,6 +3322,7 @@
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz",
"integrity": "sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g==",
"license": "MIT",
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.3",
"fast-uri": "^3.0.1",
@@ -3471,7 +3468,6 @@
"integrity": "sha512-3CYzex9M9FGQjCGMGyi6/31c8GJbgb0qGyrx5HWxPd0aCwh4cB2YjMb2Xf9UuoogrMrlO9cTqnB5rI5GHZTcUA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -3499,7 +3495,6 @@
"integrity": "sha512-Z7tMw1ytTXt5jqMcOP+OQteU1VuNK9Y02uuJtKQ1Sv69jXQKKg5cibLwGJow8yzZP+eAc18EmLGPal0bp36rvQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
}
@@ -3592,8 +3587,7 @@
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-2.0.0.tgz",
"integrity": "sha512-1ugUSr8BHXRnK23KfuYS+gVMC3LB8QGH9W1iGtDPsNWoQbgtXSExkBu2aDR4epiGWZOjZsj6lDl/N/AqqTC3UA==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/base64-js": {
"version": "1.5.1",
@@ -3720,6 +3714,7 @@
}
],
"license": "MIT",
"peer": true,
"dependencies": {
"baseline-browser-mapping": "^2.8.9",
"caniuse-lite": "^1.0.30001746",
@@ -3917,7 +3912,6 @@
"integrity": "sha512-Rjs1H+A9R+Ig+4E/9oyB66UC5Mj9Xq3N//vcLf2WzgdTi/3gUu3Z9KoqmlrEG4VuuLK8wJHofxzdQXz/knhiYg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"camelcase": "^6.3.0",
"map-obj": "^4.1.0",
@@ -3937,7 +3931,6 @@
"integrity": "sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -3951,7 +3944,6 @@
"integrity": "sha512-WuyALRjWPDGtt/wzJiadO5AXY+8hZ80hVpe6MyivgraREW751X3SbhRvG3eLKOYN+8VEvqLcf3wdnt44Z4S4SA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -3965,7 +3957,6 @@
"integrity": "sha512-yGSza74xk0UG8k+pLh5oeoYirvIiWo5t0/o3zHHAO2tRDiZcxWP7fywNlXhqb6/r6sWvwi+RsyQMWhVLe4BVuA==",
"dev": true,
"license": "(MIT OR CC0-1.0)",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -4074,6 +4065,7 @@
"resolved": "https://registry.npmjs.org/chart.js/-/chart.js-4.5.1.tgz",
"integrity": "sha512-GIjfiT9dbmHRiYi6Nl2yFCq7kkwdkp1W/lp2J99rX0yo9tgJGn3lKQATztIjb5tVtevcBtIdICNWqlq5+E8/Pw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@kurkle/color": "^0.3.0"
},
@@ -4314,7 +4306,6 @@
"integrity": "sha512-kcZ6+W5QzcJ3P1Mt+83OUv/oHFqZHIx8DuxG6eZ5RGMERoLqp4BuGjhHLYGK+Kf5XVkQvqBSmAy/nGWN3qDgEA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"import-fresh": "^3.3.0",
"js-yaml": "^4.1.0",
@@ -4379,7 +4370,6 @@
"integrity": "sha512-IQOkD3hbR5KrN93MtcYuad6YPuTSUhntLHDuLEbFWE+ff2/XSZNdZG+LcbbIW5AXKg/WFIfYItIzVoHngHXZzA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12 || >=16"
}
@@ -4645,6 +4635,7 @@
"resolved": "https://registry.npmjs.org/d3-selection/-/d3-selection-3.0.0.tgz",
"integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==",
"license": "ISC",
"peer": true,
"engines": {
"node": ">=12"
}
@@ -4698,6 +4689,7 @@
"resolved": "https://registry.npmjs.org/date-fns/-/date-fns-2.30.0.tgz",
"integrity": "sha512-fnULvOpxnC5/Vg3NCiWelDsLiUc9bRwAPs/+LfTLNvetFCtCTN+yQz15C/fs4AwX1R9K5GLtLfn8QW+dWisaAw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/runtime": "^7.21.0"
},
@@ -4732,7 +4724,6 @@
"integrity": "sha512-VfxadyCECXgQlkoEAjeghAr5gY3Hf+IKjKb+X8tGVDtveCjN+USwprd2q3QXBR9T1+x2DG0XZF5/w+7HAtSaXA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -4746,7 +4737,6 @@
"integrity": "sha512-WiPxgEirIV0/eIOMcnFBA3/IJZAZqKnwAwWyvvdi4lsr1WCN22nhdf/3db3DoZcUjTV2SqfzIwNyp6y2xs3nmg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"decamelize": "^1.1.0",
"map-obj": "^1.0.0"
@@ -4764,7 +4754,6 @@
"integrity": "sha512-z2S+W9X73hAUUki+N+9Za2lBlun89zigOyGrsax+KUQ6wKW4ZoWpEYBkGhQjwAjjDCkWxhY0VKEhk8wzY7F5cA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -4775,7 +4764,6 @@
"integrity": "sha512-7N/q3lyZ+LVCp7PzuxrJr4KMbBE2hW7BT7YNia330OFxIf4d3r5zVpicP2650l7CPN6RM9zOJRl3NGpqSiw3Eg==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -5261,7 +5249,6 @@
"integrity": "sha512-sqQamAnR14VgCr1A618A3sGrygcpK+HEbenA/HiEAkkUwcZIIB/tgWqHFxWgOyDh4nB4JCRimh79dR5Ywc9MDQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"is-arrayish": "^0.2.1"
}
@@ -5349,6 +5336,7 @@
"deprecated": "This version is no longer supported. Please see https://eslint.org/version-support for other options.",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@eslint-community/eslint-utils": "^4.2.0",
"@eslint-community/regexpp": "^4.6.1",
@@ -5895,7 +5883,6 @@
"integrity": "sha512-eRnCtTTtGZFpQCwhJiUOuxPQWRXVKYDn0b2PeHfXL6/Zi53SLAzAHfVhVWK2AryC/WH05kGfxhFIPvTF0SXQzg==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">= 4.9.1"
}
@@ -6381,7 +6368,6 @@
"integrity": "sha512-NGbfmJBp9x8IxyJSd1P+otYK8vonoJactOogrVfFRIAEY1ukil8RSKDz2Yo7wh1oihl51l/r6W4epkeKJHqL8A==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"global-prefix": "^3.0.0"
},
@@ -6395,7 +6381,6 @@
"integrity": "sha512-awConJSVCHVGND6x3tmMaKcQvwXLhjdkmomy2W+Goaui8YPgYgXJZewhg3fWC+DlfqqQuWg8AwqjGTD2nAPVWg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"ini": "^1.3.5",
"kind-of": "^6.0.2",
@@ -6411,7 +6396,6 @@
"integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==",
"dev": true,
"license": "ISC",
"peer": true,
"dependencies": {
"isexe": "^2.0.0"
},
@@ -6461,8 +6445,7 @@
"resolved": "https://registry.npmjs.org/globjoin/-/globjoin-0.1.4.tgz",
"integrity": "sha512-xYfnw62CKG8nLkZBfWbhWwDw02CHty86jfPcc2cr3ZfeuK9ysoVPPEUxf21bAD/rWAgk52SuBrLJlefNy8mvFg==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/gopd": {
"version": "1.2.0",
@@ -6539,7 +6522,6 @@
"integrity": "sha512-VIZB+ibDhx7ObhAe7OVtoEbuP4h/MuOTHJ+J8h/eBXotJYl0fBgR72xDFCKgIh22OJZIOVNxBMWuhAr10r8HdA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=6"
}
@@ -6761,7 +6743,6 @@
"integrity": "sha512-kyCuEOWjJqZuDbRHzL8V93NzQhwIB71oFWSyzVo+KPZI+pnQPPxucdkrOZvkLRnrf5URsQM+IJ09Dw29cRALIA==",
"dev": true,
"license": "ISC",
"peer": true,
"dependencies": {
"lru-cache": "^6.0.0"
},
@@ -6775,7 +6756,6 @@
"integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==",
"dev": true,
"license": "ISC",
"peer": true,
"dependencies": {
"yallist": "^4.0.0"
},
@@ -6788,8 +6768,7 @@
"resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz",
"integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==",
"dev": true,
"license": "ISC",
"peer": true
"license": "ISC"
},
"node_modules/html-tags": {
"version": "3.3.1",
@@ -6797,7 +6776,6 @@
"integrity": "sha512-ztqyC3kLto0e9WbNp0aeP+M3kTt+nbaIveGmUxAtZa+8iFgKLUOD4YKM5j+f3QD89bra7UeumolZHKuOXnTmeQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
},
@@ -6881,7 +6859,6 @@
"integrity": "sha512-rKtvo6a868b5Hu3heneU+L4yEQ4jYKLtjpnPeUdK7h0yzXGmyBTypknlkCvHFBqfX9YlorEiMM6Dnq/5atfHkw==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
}
@@ -6902,7 +6879,6 @@
"integrity": "sha512-m6FAo/spmsW2Ab2fU35JTYwtOKa2yAwXSwgjSv1TJzh4Mh7mC3lzAOVLBprb72XsTrgkEIsl7YrFNAiDiRhIGg==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -6974,8 +6950,7 @@
"resolved": "https://registry.npmjs.org/is-arrayish/-/is-arrayish-0.2.1.tgz",
"integrity": "sha512-zz06S8t0ozoDXMG+ube26zeCTNXcKIPJZJi8hBrF4idCLms4CG9QtK7qBl1boi5ODzFpjswb5JPmHCbMpjaYzg==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/is-binary-path": {
"version": "2.1.0",
@@ -7080,7 +7055,6 @@
"integrity": "sha512-yvkRyxmFKEOQ4pNXCmJG5AEQNlXJS5LaONXo5/cLdTZdWvsZ1ioJEonLGAosKlMWE8lwUy/bJzMjcw8az73+Fg==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -7091,7 +7065,6 @@
"integrity": "sha512-VRSzKkbMm5jMDoKLbltAkFQ5Qr7VDiTFGXxYFXXowVj387GeGNOCsOH6Msy00SGZ3Fp84b1Naa1psqgcCIEP5Q==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -7190,8 +7163,7 @@
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/js-yaml": {
"version": "4.1.0",
@@ -7227,8 +7199,7 @@
"resolved": "https://registry.npmjs.org/json-parse-even-better-errors/-/json-parse-even-better-errors-2.3.1.tgz",
"integrity": "sha512-xyFwyhro/JEof6Ghe2iz2NcXoj2sloNsWr/XsERDK/oiPCfaNhl5ONfp+jQdAZRQQ0IJWNzH9zIZF7li91kh2w==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/json-refs": {
"version": "3.0.15",
@@ -7347,7 +7318,6 @@
"integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -7926,8 +7896,7 @@
"resolved": "https://registry.npmjs.org/lodash.truncate/-/lodash.truncate-4.4.2.tgz",
"integrity": "sha512-jttmRe7bRse52OsWIMDLaXxWqRAmtIUccAQ3garviCqJjafXOfNMO0yMfNpdD6zbGaTU0P5Nz7e7gAT6cKmJRw==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/lodash.uniq": {
"version": "4.5.0",
@@ -7995,7 +7964,6 @@
"integrity": "sha512-hdN1wVrZbb29eBGiGjJbeP8JbKjq1urkHJ/LIP/NY48MZ1QVXUsQBV1G1zvYFHn1XE06cwjBsOI2K3Ulnj1YXQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
},
@@ -8046,7 +8014,6 @@
"integrity": "sha512-APMBEanjybaPzUrfqU0IMU5I0AswKMH7k8OTLs0vvV4KZpExkTkY87nR/zpbuTPj+gARop7aGUbl11pnDfW6xg==",
"dev": true,
"license": "MIT",
"peer": true,
"funding": {
"type": "github",
"url": "https://github.com/sponsors/wooorm"
@@ -8287,7 +8254,6 @@
"integrity": "sha512-/d+PQ4GKmGvM9Bee/DPa8z3mXs/pkvJE2KEThngVNOqtmljC6K7NMPxtc2JeZYTmpWb9k/TmxjeL18ez3h7vCw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@types/minimist": "^1.2.2",
"camelcase-keys": "^7.0.0",
@@ -8315,7 +8281,6 @@
"integrity": "sha512-yGSza74xk0UG8k+pLh5oeoYirvIiWo5t0/o3zHHAO2tRDiZcxWP7fywNlXhqb6/r6sWvwi+RsyQMWhVLe4BVuA==",
"dev": true,
"license": "(MIT OR CC0-1.0)",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -9009,7 +8974,6 @@
"integrity": "sha512-Q4r8ghd80yhO/0j1O3B2BjweX3fiHg9cdOwjJd2J76Q135c+NDxGCqdYKQ1SKBuFfgWbAUzBfvYjPUEeNgqN1A==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"arrify": "^1.0.1",
"is-plain-obj": "^1.1.0",
@@ -9088,6 +9052,7 @@
"resolved": "https://registry.npmjs.org/@codingame/monaco-vscode-editor-api/-/monaco-vscode-editor-api-25.0.0.tgz",
"integrity": "sha512-uiY06RTWFo2WZdh6OybkLlDhuG+8LlkjUDpr9/wW55uucqHo4X8fx4XKEtD98cscC+6FKQkbG2yyUiOJ/npHOw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@codingame/monaco-vscode-api": "25.0.0"
}
@@ -9324,7 +9289,6 @@
"integrity": "sha512-p2W1sgqij3zMMyRC067Dg16bfzVH+w7hyegmpIvZ4JNjqtGOVAIvLmjBx3yP7YTe9vKJgkoNOPjwQGogDoMXFA==",
"dev": true,
"license": "BSD-2-Clause",
"peer": true,
"dependencies": {
"hosted-git-info": "^4.0.1",
"is-core-module": "^2.5.0",
@@ -9670,7 +9634,6 @@
"integrity": "sha512-ayCKvm/phCGxOkYRSCM82iDwct8/EonSEgCSxWxD7ve6jHggsFl4fZVQBPRNgQoKiuV/odhFrGzQXZwbifC8Rg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.0.0",
"error-ex": "^1.3.1",
@@ -9882,7 +9845,6 @@
"dev": true,
"license": "ISC",
"optional": true,
"peer": true,
"dependencies": {
"pngjs": "^7.0.0"
},
@@ -9972,7 +9934,6 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"engines": {
"node": ">=14.19.0"
}
@@ -9997,6 +9958,7 @@
}
],
"license": "MIT",
"peer": true,
"dependencies": {
"nanoid": "^3.3.11",
"picocolors": "^1.1.1",
@@ -10185,6 +10147,7 @@
}
],
"license": "MIT",
"peer": true,
"dependencies": {
"lilconfig": "^3.0.0",
"yaml": "^2.3.4"
@@ -10574,8 +10537,7 @@
"resolved": "https://registry.npmjs.org/postcss-resolve-nested-selector/-/postcss-resolve-nested-selector-0.1.6.tgz",
"integrity": "sha512-0sglIs9Wmkzbr8lQwEyIzlDOOC9bGmfVKcJTaxv3vMmd3uo4o4DerC3En0bnmgceeql9BfC8hRkp7cg0fjdVqw==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/postcss-safe-parser": {
"version": "6.0.0",
@@ -10751,6 +10713,7 @@
"integrity": "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ==",
"dev": true,
"license": "MIT",
"peer": true,
"bin": {
"prettier": "bin/prettier.cjs"
},
@@ -11055,7 +11018,6 @@
"integrity": "sha512-X1Fu3dPuk/8ZLsMhEj5f4wFAF0DWoK7qhGJvgaijocXxBmSToKfbFtqbxMO7bVjNA1dmE5huAzjXj/ey86iw9Q==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@types/normalize-package-data": "^2.4.0",
"normalize-package-data": "^3.0.2",
@@ -11075,7 +11037,6 @@
"integrity": "sha512-snVCqPczksT0HS2EC+SxUndvSzn6LRCwpfSvLrIfR5BKDQQZMaI6jPRC9dYvYFDRAuFEAnkwww8kBBNE/3VvzQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"find-up": "^5.0.0",
"read-pkg": "^6.0.0",
@@ -11094,7 +11055,6 @@
"integrity": "sha512-yGSza74xk0UG8k+pLh5oeoYirvIiWo5t0/o3zHHAO2tRDiZcxWP7fywNlXhqb6/r6sWvwi+RsyQMWhVLe4BVuA==",
"dev": true,
"license": "(MIT OR CC0-1.0)",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -11108,7 +11068,6 @@
"integrity": "sha512-yGSza74xk0UG8k+pLh5oeoYirvIiWo5t0/o3zHHAO2tRDiZcxWP7fywNlXhqb6/r6sWvwi+RsyQMWhVLe4BVuA==",
"dev": true,
"license": "(MIT OR CC0-1.0)",
"peer": true,
"engines": {
"node": ">=10"
},
@@ -11151,7 +11110,6 @@
"integrity": "sha512-tYkDkVVtYkSVhuQ4zBgfvciymHaeuel+zFKXShfDnFP5SyVEP7qo70Rf1jTOTCx3vGNAbnEi/xFkcfQVMIBWag==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"indent-string": "^5.0.0",
"strip-indent": "^4.0.0"
@@ -11788,7 +11746,6 @@
"integrity": "sha512-qMCMfhY040cVHT43K9BFygqYbUPFZKHOg7K73mtTWJRb8pyP3fzf4Ixd5SzdEJQ6MRUg/WBnOLxghZtKKurENQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"ansi-styles": "^4.0.0",
"astral-regex": "^2.0.0",
@@ -11865,7 +11822,6 @@
"integrity": "sha512-kN9dJbvnySHULIluDHy32WHRUu3Og7B9sbY7tsFLctQkIqnMh3hErYgdMjTYuqmcXX+lK5T1lnUt3G7zNswmZA==",
"dev": true,
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"spdx-expression-parse": "^3.0.0",
"spdx-license-ids": "^3.0.0"
@@ -11876,8 +11832,7 @@
"resolved": "https://registry.npmjs.org/spdx-exceptions/-/spdx-exceptions-2.5.0.tgz",
"integrity": "sha512-PiU42r+xO4UbUS1buo3LPJkjlO7430Xn5SVAhdpzzsPHsjbYVflnnFdATgabnLude+Cqu25p6N+g2lw/PFsa4w==",
"dev": true,
"license": "CC-BY-3.0",
"peer": true
"license": "CC-BY-3.0"
},
"node_modules/spdx-expression-parse": {
"version": "3.0.1",
@@ -11885,7 +11840,6 @@
"integrity": "sha512-cbqHunsQWnJNE6KhVSMsMeH5H/L9EpymbzqTQ3uLwNCLZ1Q481oWaofqH7nO6V07xlXwY6PhQdQ2IedWx/ZK4Q==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"spdx-exceptions": "^2.1.0",
"spdx-license-ids": "^3.0.0"
@@ -11896,8 +11850,7 @@
"resolved": "https://registry.npmjs.org/spdx-license-ids/-/spdx-license-ids-3.0.22.tgz",
"integrity": "sha512-4PRT4nh1EImPbt2jASOKHX7PB7I+e4IWNLvkKFDxNhJlfjbYlleYQh285Z/3mPTHSAK/AvdMmw5BNNuYH8ShgQ==",
"dev": true,
"license": "CC0-1.0",
"peer": true
"license": "CC0-1.0"
},
"node_modules/sprintf-js": {
"version": "1.0.3",
@@ -11991,7 +11944,6 @@
"integrity": "sha512-SlyRoSkdh1dYP0PzclLE7r0M9sgbFKKMFXpFRUMNuKhQSbC6VQIGzq3E0qsfvGJaUFJPGv6Ws1NZ/haTAjfbMA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -12017,8 +11969,7 @@
"resolved": "https://registry.npmjs.org/style-search/-/style-search-0.1.0.tgz",
"integrity": "sha512-Dj1Okke1C3uKKwQcetra4jSuk0DqbzbYtXipzFlFMZtowbF1x7BKJwB9AayVMyFARvU8EDrZdcax4At/452cAg==",
"dev": true,
"license": "ISC",
"peer": true
"license": "ISC"
},
"node_modules/style-to-object": {
"version": "0.4.4",
@@ -12067,7 +12018,6 @@
"integrity": "sha512-78O4c6IswZ9TzpcIiQJIN49K3qNoXTM8zEJzhaTE/xRTCZswaovSEVIa/uwbOltZrk16X4jAxjaOhzz/hTm1Kw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@csstools/css-parser-algorithms": "^2.3.1",
"@csstools/css-tokenizer": "^2.2.0",
@@ -12150,7 +12100,6 @@
}
],
"license": "MIT-0",
"peer": true,
"engines": {
"node": "^14 || ^16 || >=18"
},
@@ -12164,7 +12113,6 @@
"integrity": "sha512-TfW7/1iI4Cy7Y8L6iqNdZQVvdXn0f8B4QcIXmkIbtTIe/Okm/nSlHb4IwGzRVOd3WfSieCgvf5cMzEfySAIl0g==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"flat-cache": "^3.2.0"
},
@@ -12177,8 +12125,7 @@
"resolved": "https://registry.npmjs.org/known-css-properties/-/known-css-properties-0.29.0.tgz",
"integrity": "sha512-Ne7wqW7/9Cz54PDt4I3tcV+hAyat8ypyOGzYRJQfdxnnjeWsTxt1cy8pjvvKeI5kfXuyvULyeeAvwvvtAX3ayQ==",
"dev": true,
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/stylelint/node_modules/postcss-selector-parser": {
"version": "6.1.2",
@@ -12201,7 +12148,6 @@
"integrity": "sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
}
@@ -12336,7 +12282,6 @@
"integrity": "sha512-zFObLMyZeEwzAoKCyu1B91U79K2t7ApXuQfo8OuxwXLDgcKxuwM+YvcbIhm6QWqz7mHUH1TVytR1PwVVjEuMig==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"has-flag": "^4.0.0",
"supports-color": "^7.0.0"
@@ -12366,6 +12311,7 @@
"resolved": "https://registry.npmjs.org/svelte/-/svelte-5.39.12.tgz",
"integrity": "sha512-CEzwxFuEycokU8K8CE/OuwVbmei+ivu2HvBGYIdASfMa1hCRSNr4RRkzNSvbAvu6h+BOig2CsZTAEY+WKvwZpA==",
"license": "MIT",
"peer": true,
"dependencies": {
"@jridgewell/remapping": "^2.3.4",
"@jridgewell/sourcemap-codec": "^1.5.0",
@@ -12461,21 +12407,6 @@
}
}
},
"node_modules/svelte-check/node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/svelte-eslint-parser": {
"version": "0.43.0",
"resolved": "https://registry.npmjs.org/svelte-eslint-parser/-/svelte-eslint-parser-0.43.0.tgz",
@@ -12663,8 +12594,7 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/svg-tags/-/svg-tags-1.0.0.tgz",
"integrity": "sha512-ovssysQTa+luh7A5Weu3Rta6FJlFBBbInjOh722LIt6klpU2/HtdUbszju/G4devcvk8PGt7FCLv5wftu3THUA==",
"dev": true,
"peer": true
"dev": true
},
"node_modules/svgo": {
"version": "3.3.2",
@@ -12715,7 +12645,6 @@
"integrity": "sha512-9kY+CygyYM6j02t5YFHbNz2FN5QmYGv9zAjVp4lCDjlCw7amdckXlEt/bjMhUIfj4ThGRE4gCUH5+yGnNuPo5A==",
"dev": true,
"license": "BSD-3-Clause",
"peer": true,
"dependencies": {
"ajv": "^8.0.1",
"lodash.truncate": "^4.4.2",
@@ -12743,6 +12672,7 @@
"integrity": "sha512-6A2rnmW5xZMdw11LYjhcI5846rt9pbLSabY5XPxo+XWdxwZaFEn47Go4NzFiHu9sNNmr/kXivP1vStfvMaK1GQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@alloc/quick-lru": "^5.2.0",
"arg": "^5.0.2",
@@ -12985,6 +12915,7 @@
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"devOptional": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -13047,7 +12978,6 @@
"integrity": "sha512-jRKj0n0jXWo6kh62nA5TEh3+4igKDXLvzBJcPpiizP7oOolUrYIxmVBG9TOtHYFHoddUk6YvAkGeGoSVTXfQXQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -13146,6 +13076,7 @@
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -13360,7 +13291,6 @@
"integrity": "sha512-DpKm2Ui/xN7/HQKCtpZxoRWBhZ9Z0kqtygG8XCgNQ8ZlDnxuQmWhj566j8fN4Cu3/JmbhsDo7fcAJq4s9h27Ew==",
"dev": true,
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"spdx-correct": "^3.0.0",
"spdx-expression-parse": "^3.0.0"
@@ -13415,6 +13345,7 @@
"integrity": "sha512-5hI5NCJwKBGtzWtdKB3c2fOEpI77Iaa0z4mSzZPU1cJ/OqrGbFafm90edVCd7T9Snz+Sh09TMAv4EQqyVLzuEg==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@oxc-project/runtime": "0.101.0",
"fdir": "^6.5.0",
@@ -13527,6 +13458,7 @@
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"devOptional": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -13540,6 +13472,7 @@
"integrity": "sha512-n1RxDp8UJm6N0IbJLQo+yzLZ2sQCDyl1o0LeugbPWf8+8Fttp29GghsQBjYJVmWq3gBFfe9Hs1spR44vovn2wA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@vitest/expect": "4.0.15",
"@vitest/mocker": "4.0.15",
@@ -14168,6 +14101,7 @@
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -14191,6 +14125,7 @@
"integrity": "sha512-dZwN5L1VlUBewiP6H9s2+B3e3Jg96D0vzN+Ry73sOefebhYr9f94wwkMNN/9ouoU8pV1BqA1d1zGk8928cx0rg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"esbuild": "^0.27.0",
"fdir": "^6.5.0",
@@ -14517,14 +14452,14 @@
"integrity": "sha512-COyid6B1RYs+bpzUCInsA4HY/WZkpDLfkQ90+AqU/TVTpzYSbAC2JCbIwy0cRElBvlhI4bQ+9Wg6hSQKMpEkpA=="
},
"node_modules/windmill-parser-wasm-py": {
"version": "1.601.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-py/-/windmill-parser-wasm-py-1.601.1.tgz",
"integrity": "sha512-xcNZE/8B29yfl6UuQDPSXMD+83/W2Hzt2uhn+WrNvy0+qzk6nLh/vJGrf2srLBngYX1TxhUI5Jgseg0PK9yvNw=="
"version": "1.623.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-py/-/windmill-parser-wasm-py-1.623.1.tgz",
"integrity": "sha512-lFBlZg6hvhHzsU5oPJq0478UyMTZ9UKVd8Hc8ggmxPIHZaJBeJ+56NR75hmGwg0VJcRff8ed+zEm1PgmjVhD+w=="
},
"node_modules/windmill-parser-wasm-regex": {
"version": "1.593.0",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-regex/-/windmill-parser-wasm-regex-1.593.0.tgz",
"integrity": "sha512-m8BvTGJc2710YODmKKDXiASfssoJ/YFJGfYRhRnQvltENvaRee2NZuf4XoUkOVJESDdCSEX6U6fVkFvF9rXp2Q=="
"version": "1.623.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-regex/-/windmill-parser-wasm-regex-1.623.1.tgz",
"integrity": "sha512-rW3pl4ysIXmVAmwxSTKhR2sUfScXKfTqxrUebDg0ZNcUzY97S8mNRdQGZfaQ1NWhWPRQNXfL/z/yaUOQh5ConQ=="
},
"node_modules/windmill-parser-wasm-ruby": {
"version": "1.526.1",
@@ -14537,9 +14472,9 @@
"integrity": "sha512-21S7lm1KF8zO1187rbq14hzPHII2RdM2+D44MoAh1F6VoaScj+Puq0z5B1O/hwn/95R/a9jBlL2D8jbkXtlD1A=="
},
"node_modules/windmill-parser-wasm-ts": {
"version": "1.593.0",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-ts/-/windmill-parser-wasm-ts-1.593.0.tgz",
"integrity": "sha512-NFY9gaEIpJOwGZJeGYDS3+/16QiPYdxFgmb1bKhXyEAdWjYMQ+otrTezf+K09lmvPzB+len37GlNCU72OgyQ6A=="
"version": "1.623.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-ts/-/windmill-parser-wasm-ts-1.623.1.tgz",
"integrity": "sha512-FBwi/zXxjhZcCvi04oFdNivazru1ynIqSbafHSArfaaBWesBO3nye9UO/WXUlWZm5a7BExbU+3R/eVJrGaornw=="
},
"node_modules/windmill-parser-wasm-yaml": {
"version": "1.593.0",
@@ -14690,7 +14625,6 @@
"integrity": "sha512-+QU2zd6OTD8XWIJCbffaiQeH9U73qIqafo1x6V1snCWYGJf6cVE0cDR4D8xRzcEnfI21IFrUPzPGtcPf8AC+Rw==",
"dev": true,
"license": "ISC",
"peer": true,
"dependencies": {
"imurmurhash": "^0.1.4",
"signal-exit": "^4.0.1"
@@ -14699,29 +14633,6 @@
"node": "^14.17.0 || ^16.13.0 || >=18.0.0"
}
},
"node_modules/ws": {
"version": "8.19.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.19.0.tgz",
"integrity": "sha512-blAT2mjOEIi0ZzruJfIhb3nps74PRWTCz1IjglWEEpQl5XS/UNama6u2/rjFkDDouqr4L67ry+1aGIALViWjDg==",
"license": "MIT",
"optional": true,
"peer": true,
"engines": {
"node": ">=10.0.0"
},
"peerDependencies": {
"bufferutil": "^4.0.1",
"utf-8-validate": ">=5.0.2"
},
"peerDependenciesMeta": {
"bufferutil": {
"optional": true
},
"utf-8-validate": {
"optional": true
}
}
},
"node_modules/xml-utils": {
"version": "1.10.2",
"resolved": "https://registry.npmjs.org/xml-utils/-/xml-utils-1.10.2.tgz",
@@ -14909,7 +14820,6 @@
"integrity": "sha512-y11nGElTIV+CT3Zv9t7VKl+Q3hTQoT9a1Qzezhhl6Rp21gJ/IVTW7Z3y9EWXhuUBC2Shnf+DX0antecpAwSP8w==",
"dev": true,
"license": "ISC",
"peer": true,
"engines": {
"node": ">=10"
}
@@ -14929,6 +14839,7 @@
"resolved": "https://registry.npmjs.org/yjs/-/yjs-13.6.27.tgz",
"integrity": "sha512-OIDwaflOaq4wC6YlPBy2L6ceKeKuF7DeTxx+jPzv1FHn9tCZ0ZwSRnUBxD05E3yed46fv/FWJbvR+Ud7x0L7zw==",
"license": "MIT",
"peer": true,
"dependencies": {
"lib0": "^0.2.99"
},
@@ -14964,6 +14875,7 @@
"resolved": "https://registry.npmjs.org/zod/-/zod-4.1.12.tgz",
"integrity": "sha512-JInaHOamG8pt5+Ey8kGmdcAcg3OL9reK8ltczgHTAwNhMys/6ThXHityHxVV2p3fkw/c+MAvBHFVYHFZDmjMCQ==",
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
frontend/package.json
+3 -3
View File
@@ -148,11 +148,11 @@
"windmill-parser-wasm-java": "1.510.1",
"windmill-parser-wasm-nu": "1.510.1",
"windmill-parser-wasm-php": "1.574.1",
"windmill-parser-wasm-py": "1.601.1",
"windmill-parser-wasm-regex": "1.593.0",
"windmill-parser-wasm-py": "1.623.1",
"windmill-parser-wasm-regex": "1.623.1",
"windmill-parser-wasm-ruby": "1.526.1",
"windmill-parser-wasm-rust": "1.558.1",
"windmill-parser-wasm-ts": "1.593.0",
"windmill-parser-wasm-ts": "1.623.1",
"windmill-parser-wasm-yaml": "1.593.0",
"windmill-sql-datatype-parser-wasm": "1.512.0",
"windmill-utils-internal": "^1.3.2",
frontend/src/lib/components/TooltipInner.svelte
+7 -1
View File
@@ -2,13 +2,19 @@
import Markdown from 'svelte-exmarkdown'
import { ExternalLink } from 'lucide-svelte'
import { gfmPlugin } from 'svelte-exmarkdown/gfm'
import { twMerge } from 'tailwind-merge'
export let documentationLink: string | undefined = undefined
export let markdownTooltip: string | undefined = undefined
export let customBgClass: string | undefined = undefined
const plugins = [gfmPlugin()]
</script>
<div
class="shadow-lg max-w-sm break-words py-2 px-3 rounded-md text-xs font-normal text-primary bg-surface-secondary whitespace-normal text-left dark:border max-h-64 overflow-y-auto"
class={twMerge(
'shadow-lg max-w-sm break-words py-2 px-3 rounded-md text-xs font-normal text-primary whitespace-normal text-left dark:border max-h-64 overflow-y-auto',
customBgClass || 'bg-surface-secondary'
)}
>
{#if markdownTooltip}
<div class="prose-sm">
frontend/src/lib/components/assets/AssetColumnBadges.svelte
+44
View File
@@ -0,0 +1,44 @@
<script lang="ts">
import { type AssetUsageAccessType } from 'windmill-utils-internal/dist/gen/types.gen'
import { formatAssetAccessType } from './lib'
import Tooltip from '../meltComponents/Tooltip.svelte'
import { twMerge } from 'tailwind-merge'
type Props = {
columns: Record<string, AssetUsageAccessType> | undefined
disableTooltip?: boolean
badgeClasses?: string
disableWrap?: boolean
}
let { columns, disableTooltip, badgeClasses, disableWrap }: Props = $props()
let entries = $derived(columns && Object.entries(columns))
</script>
{#if entries?.length}
<div class={twMerge('flex gap-1', disableWrap ? '' : 'flex-wrap')}>
{#each entries as [columnName, accessType]}
{@const accessType2 = formatAssetAccessType(accessType)}
{#snippet badge()}
<div
class={twMerge(
'text-xs text-secondary border rounded-md px-1 bg-surface-tertiary dark:bg-surface-secondary',
badgeClasses
)}
>
{columnName}
</div>
{/snippet}
{#if disableTooltip}
{@render badge()}
{:else}
<Tooltip>
{@render badge()}
<svelte:fragment slot="text">
{accessType2} access to column "{columnName}"
</svelte:fragment>
</Tooltip>
{/if}
{/each}
</div>
{/if}
frontend/src/lib/components/assets/AssetsUsageDrawer.svelte
+16 -14
View File
@@ -6,8 +6,9 @@
import Tooltip from '../meltComponents/Tooltip.svelte'
import Tooltip2 from '../Tooltip.svelte'
import { twMerge } from 'tailwind-merge'
import { displayDate } from '$lib/utils'
import { capitalize, displayDate } from '$lib/utils'
import Alert from '../common/alert/Alert.svelte'
import AssetColumnBadges from './AssetColumnBadges.svelte'
let usagesDrawerData:
| {
@@ -64,14 +65,10 @@
</DrawerContent>
</Drawer>
{#snippet badge(text: string | undefined, tooltip?: string)}
{#snippet rightBadge(text: string | undefined, tooltip?: string)}
{#if text}
<Tooltip disablePopup={!tooltip}>
<div
class={twMerge(
'text-xs bg-surface font-normal border text-primary min-w-12 p-1 text-center rounded-md'
)}
>
<div class={twMerge('text-xs font-normal text-primary min-w-12 p-1 text-center rounded-md')}>
{text}
</div>
<svelte:fragment slot="text">
@@ -92,7 +89,7 @@
<a
href={getAssetUsagePageUri(u)}
aria-label={`${u.kind}/${u.path}`}
class="text-xs text-primary font-normal flex items-center py-3 px-4 gap-2 hover:bg-surface-hover cursor-pointer"
class="text-xs min-h-14 text-primary font-normal flex items-center py-2 px-4 gap-2 hover:bg-surface-hover cursor-pointer"
>
<RowIcon
kind={!u.metadata?.job_kind
@@ -110,14 +107,19 @@
} as const
)[u.metadata.job_kind] ?? 'script')}
/>
<div class="flex flex-col justify-center flex-1">
<span class="font-semibold text-emphasis">
{u.kind == 'job' ? (u.metadata?.runnable_path ?? 'Unknown job') : u.path}
<div class="flex flex-col justify-center flex-1 ml-2">
<span>
<span class="font-semibold text-emphasis">
{u.kind == 'job' ? (u.metadata?.runnable_path ?? 'Unknown job') : u.path}
</span>
</span>
<span class="text-2xs text-secondary">{u.kind == 'job' ? u.path : u.kind}</span>
<span class="text-2xs text-secondary">
{u.kind == 'job' ? u.path : capitalize(u.kind)}
</span>
<AssetColumnBadges columns={u.columns} badgeClasses="mt-0.5" />
</div>
{@render badge(displayDate(u.created_at), 'Asset detection time')}
{@render badge(accessType)}
{@render rightBadge(displayDate(u.created_at), 'Asset detection time')}
{@render rightBadge(accessType)}
</a>
</li>
{/each}
frontend/src/lib/components/assets/lib.ts
+1
View File
@@ -12,6 +12,7 @@ export type AssetKind = _AssetKind
export type AssetWithAccessType = Asset & { access_type?: AssetUsageAccessType }
export type AssetWithAltAccessType = AssetWithAccessType & {
alt_access_type?: AssetUsageAccessType
columns?: Record<string, AssetUsageAccessType>
}
export type AssetUsage = ListAssetsResponse['assets'][number]['usages'][number]
frontend/src/lib/components/graph/FlowGraphV2.svelte
+1 -1
View File
@@ -948,7 +948,7 @@
{height}
{width}
minZoom={0.2}
maxZoom={1.2}
maxZoom={1.6}
connectionLineType={ConnectionLineType.SmoothStep}
defaultEdgeOptions={{ type: 'smoothstep' }}
preventScrolling={scroll}
frontend/src/lib/components/graph/graphBuilder.svelte.ts
+2
View File
@@ -297,6 +297,7 @@ export type AssetN = {
type: 'asset'
data: {
asset: AssetWithAltAccessType
displayedAccessType: 'r' | 'w'
}
}
@@ -304,6 +305,7 @@ export type AssetsOverflowedN = {
type: 'assetsOverflowed'
data: {
overflowedAssets: AssetWithAltAccessType[]
displayedAccessType: 'r' | 'w'
}
}
frontend/src/lib/components/graph/renderers/nodes/AssetNode.svelte
+29 -10
View File
@@ -57,7 +57,7 @@
// All asset nodes displayed on top
const inputAssetNodes: (Node & AssetN)[] = displayedInputAssets.map((asset, i) => {
let inputAssetXGap = 12
let inputAssetWidth = 150
let inputAssetWidth = 165
const targetRowW =
MAX_ASSET_ROW_WIDTH -
@@ -73,7 +73,7 @@
return {
type: 'asset' as const,
parentId: node.id,
data: { asset },
data: { asset, displayedAccessType: 'r' },
id: `${node.id}-asset-in-${asset.kind}-${asset.path}`,
width: inputAssetWidth,
position: {
@@ -94,7 +94,7 @@
// All asset nodes displayed on the bottom
const outputAssetNodes: (Node & AssetN)[] = displayedOutputAssets.map((asset, i) => {
let outputAssetXGap = 12
let outputAssetWidth = 150
let outputAssetWidth = 165
const targetRowW =
MAX_ASSET_ROW_WIDTH -
@@ -110,7 +110,7 @@
return {
type: 'asset' as const,
parentId: node.id,
data: { asset },
data: { asset, displayedAccessType: 'w' },
id: `${node.id}-asset-out-${asset.kind}-${asset.path}`,
width: outputAssetWidth,
position: {
@@ -150,7 +150,7 @@
if (overflowedInputAssets.length)
allAssetNodes.push({
type: 'assetsOverflowed',
data: { overflowedAssets: overflowedInputAssets },
data: { overflowedAssets: overflowedInputAssets, displayedAccessType: 'r' },
id: `${node.id}-assets-overflowed-in`,
parentId: node.id,
width: ASSETS_OVERFLOWED_NODE_WIDTH,
@@ -169,7 +169,7 @@
if (overflowedOutputAssets.length)
allAssetNodes.push({
type: 'assetsOverflowed',
data: { overflowedAssets: overflowedOutputAssets },
data: { overflowedAssets: overflowedOutputAssets, displayedAccessType: 'w' },
id: `${node.id}-assets-overflowed-out`,
parentId: node.id,
width: ASSETS_OVERFLOWED_NODE_WIDTH,
@@ -237,6 +237,7 @@
import { userStore } from '$lib/stores'
import { deepEqual } from 'fast-equals'
import { slide } from 'svelte/transition'
import AssetColumnBadges from '$lib/components/assets/AssetColumnBadges.svelte'
interface Props {
data: AssetN['data']
@@ -254,15 +255,24 @@
})
const usageCount = $derived(flowGraphAssetsCtx?.val.computeAssetsCount?.(data.asset))
const colors = $derived(getNodeColorClasses(undefined, isSelected))
let assetColumns = $derived(
data.asset.columns &&
Object.fromEntries(
Object.entries(data.asset.columns).filter(
([_, accessType]) => accessType && accessType === data.displayedAccessType
)
)
)
</script>
<NodeWrapper wrapperClass="bg-surface-secondary rounded-md">
{#snippet children({ darkMode })}
<!-- svelte-ignore a11y_no_static_element_interactions -->
<Tooltip>
<Tooltip customBgClass="bg-surface-tertiary">
<div
class={twMerge(
'h-6 flex items-center gap-1.5 rounded-md drop-shadow-base overflow-clip transition-colors',
'h-6 flex items-center rounded-md drop-shadow-base overflow-clip transition-colors',
colors.outline,
colors.text,
colors.bg
@@ -274,11 +284,19 @@
>
<AssetGenericIcon
assetKind={data.asset.kind}
class="shrink-0 ml-1 {isSelected ? 'text-accent' : 'text-tertiary'}"
class="shrink-0 ml-1 mr-1.5 {isSelected ? 'text-accent' : 'text-tertiary'}"
size="16px"
/>
<span class="text-3xs truncate flex-1">
<span
class="text-3xs truncate flex-1 flex items-center gap-1 [mask-image:linear-gradient(to_right,black_85%,transparent)] mr-0.5"
>
{formatShortAssetPath(data.asset)}
<AssetColumnBadges
columns={assetColumns}
disableTooltip
disableWrap
badgeClasses="text-3xs transition-opacity opacity-50 {isSelected ? 'opacity-0' : ''}"
/>
</span>
{#if data.asset.kind === 'resource' && cachedResourceMetadata === undefined}
<Tooltip class={'pr-1 flex items-center justify-center'}>
@@ -319,6 +337,7 @@
<span class="text-hint text-xs">
{formatAssetKind({ ...data.asset, metadata: cachedResourceMetadata })}</span
>
<AssetColumnBadges columns={assetColumns} disableTooltip />
</svelte:fragment>
</Tooltip>
{/snippet}
frontend/src/lib/components/graph/renderers/nodes/AssetsOverflowedNode.svelte
+1 -1
View File
@@ -60,7 +60,7 @@
<ul>
{#each data.overflowedAssets as asset}
<li class="w-48">
<AssetNode data={{ asset }} />
<AssetNode data={{ asset, displayedAccessType: data.displayedAccessType }} />
</li>
{/each}
</ul>
frontend/src/lib/components/meltComponents/Tooltip.svelte
+2 -1
View File
@@ -16,6 +16,7 @@
export let openDelay: number = 300
export let closeDelay: number = 0
export let portal: string | undefined | null = 'body'
export let customBgClass: string | undefined = undefined
const {
elements: { trigger, content },
@@ -47,7 +48,7 @@
{#if $open && !disablePopup}
<div use:melt={$content} transition:fade={{ duration: 100 }} style="z-index: {zIndexes.tooltip}">
<TooltipInner {documentationLink} {markdownTooltip}>
<TooltipInner {documentationLink} {markdownTooltip} {customBgClass}>
<slot name="text" />
</TooltipInner>
</div>
frontend/src/lib/infer.ts
+19 -3
View File
@@ -1,4 +1,10 @@
import { ScriptService, type MainArgSignature, FlowService, type Script } from '$lib/gen'
import {
ScriptService,
type MainArgSignature,
FlowService,
type Script,
type AssetUsageAccessType
} from '$lib/gen'
import { get, writable } from 'svelte/store'
import type { Schema, SupportedLanguage } from './common.js'
import { emptySchema, sortObject } from './utils.js'
@@ -90,8 +96,18 @@ async function initWasmRuby() {
}
type InferAssetsResult =
| { status: 'ok'; assets: AssetWithAccessType[]; sql_queries?: InferAssetsSqlQueryDetails[] }
| { status: 'error'; error: string; assets?: undefined; sql_queries?: undefined }
| {
status: 'ok'
assets: AssetWithAccessType[]
sql_queries?: InferAssetsSqlQueryDetails[]
columns?: Record<string, AssetUsageAccessType>
}
| {
status: 'error'
error: string
assets?: undefined
sql_queries?: undefined
}
export type InferAssetsSqlQueryDetails = {
query_string: string // SQL query with $1 placeholders for interpolations