mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-22 08:02:19 +00:00
0c0c2f62a5
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit6e1588d59e. * datatables work (without leaking pg credentials) * println * separate sqlUtils.ts * nit * Separate custom instance db Select and Wizard components * nit * nit wording * add tags to custom instance dbs * error when trying to use ducklake as datatable or opposite * show status in dropdown * data table instance setup works * sqk function for ducklake * factorize logic * fix temp reactivity * Data table assetexplore * Migrate S3 permissions to modal * Revert "Migrate S3 permissions to modal" This reverts commit0631d03cb0. * nit query -> fetch * Custom instance setup new look * run_language_executor separate fn * run_inline param * nit wording * Better typed client * Data tables display as assets in frontend * asset db icon * nit * cleaner errors * nit * Fix sed calls in mac * run_inline_script_preview in python client * basic python datatable client * datatable and datalake parser in python * ducklake client python * nit fix * Fix migration producing NULL instead of {} when no custom databases * merge conflict fail * python ducklake client arg fix * parse or infer sql types in ts client * ts asset parser, detect datatable & ducklake R/W * fix sql repl for other read ops than select * export type SqlTemplateFunction * rename list_custom_instance_pg_databases * typecheck datatable and ducklake name in Typescript * Fix typecheck datatable and ducklake in TS * declare module overriding instead of extending * infer_sql_type in python client * SqlQuery object in python * fix merge conflicts * update const_format * CI fix * factor out to var_identifiers * sqlx prepare * unnecessary security (admin is required) * clearer comment * ee repo ref * nit snake case * claude step 1: detect var declarations * move detect_sql_access_type to common mod * claude step 2: detect when saved vars are queried * Revert "claude step 2: detect when saved vars are queried" This reverts commit1e1f930568. * Revert "claude step 1: detect var declarations" This reverts commitf866f4819d. * remove ducklake/datatable and default * detect data table assigns in var_identifiers * Python parser successfully infers R/W/RW from ducklake / datatable * still register ducklake/datatable if not used as unknown R/W * Go to settings button in Assets Dropdown on not found * nit * sqlx prepare fail * manual fix, somehow sqlx prepare won't do it * fix frontend ci * ee repo ref * ducklake_user doesnt exist in unit tests * nit fix * ui nit * nit * nit missing clone * fork ducklakes and datatables * fix surface hover bug * stupid mistake * better deeply reactive mutable derived * Ducklake picker * Editor bar data tables * DuckDB supports datatables * datatable in duckdb asset parser * duckdb asset parser var_identifiers * Revert "duckdb asset parser var_identifiers" This reverts commit88068b1a77. * sqlx prepare * Box pin in test_workflow_as_code to fix stack overflow * stash * sql asset parser parses most s3 literals * nit * Detect attach + handle returning RW * detect assets used with dot notation * detect implicit access with USE dl; syntax * Add assets as unknown if var was never used * Support default ducklake/datatable main in parser * ignore asset parsing errors in frontend (avoid flow layout shift) * super weird duplication (merge conflict ?) * nits * fix duckdb parser detecting too much as asset when RW ctx is unknown * fix transparent assets btn * missing arg * nit styling * asset parser specific table parsing * fix resource specific table parsing * More concise asset display in flows + better icons * fix assets page filtering out resources with added table * Fix frontend to support specific table assets * Open DB Manager to specific table * Specific table parser in Python and TS + unit tests * Fix UPDATE setting access to None * fix flow edge rendering on top of output picker * python parser fix var override bug * add ts test * fix compilation * sqlx prepare * update parsers version * fix missing schema key onDelete * Grant permission to create schemas in custom instance databases * Update pg query to return empty schemas * Create schema * Select nits * support schemas in sql parser * ts parser handle schema with sql parser result * detect .schema() syntax * detect schema syntax in python * support .schema() in ts and py SDK * open db manager to specific schema * support reassignment in ts parser * nit better unitest * : syntax in ts * datatable:schema syntax in python * fix client py * nit select dropdown darkmode * object | null fetchOne * ts client nits * parse_sql_client_name fn * getImportWmillTsStatement refactor in EditorBar * text to json() in python client * update parser versions * pkg lock * Sql query details in TS asset parser * code transformation with type parameter in Editor * Custom Language Worker, code substition works ! * Error marker mapping works * hover info is correct * completions work correctly * other overrides * type inference kinda works * Position mapping tests * refactor prepare_queries * Refactor PgDatabase to share common code * Pgdatabase in prepare_queries * TokioPgConnection refactor * refactor prepare_queries * type parameter to sql function * Fix deadlock * nit fix * Fix worker async call freezing because of svelte Proxy * Force worker to recompute when we set queries * nit refactor * nits console logs * wait that ts worker initialize * monaco change file version * update diagnostics * Refactor for errors * Show SQL errors in Monaco * improve sdk * cleaning refactor + MapResource + usePreparedAssetSqlQueries * Fixes * Fix error position mapping * cache in typescript worker * fix insert no values * don't inject type if already present * Support schema in prepare queries * update parsers * ChangeOnDeepInequality * inferAsset ScriptEditor usage refactor * sql query typecheck work in flow editor * Assets and SQL Query check in Raw App Inline Editor * pkg lock * Fix DatatableSqlTemplateFunction nit * prepare query schema nit * duplicate diagnostics * nit getScriptVersion mock * Reprepare queries when switching workspaces * nit fix * nit fix * fetch_one_scalar and execute in python client * limit pg_connections * -- prepare flag in postgres * skip serializing * fix destructuring undefined * Prepare queries in workers instead of backend * nit * Execute search_path instructions normally * nit fix * Fix SET search_path issue in prepare * only support preparing single-statement queries for now * update parsers * safety * better remove_comments * Fix getQueryStmtCountHeuristic * getQueryStmtCountHeuristic tests * comment out failing tests * Fix getQueryStmtCountHeuristic impl * only datatable
286 lines
9.2 KiB
TypeScript
286 lines
9.2 KiB
TypeScript
import { describe, it, expect } from 'vitest'
|
|
import { getQueryStmtCountHeuristic } from './utils'
|
|
|
|
describe('getQueryStmtCountHeuristic', () => {
|
|
describe('basic statements', () => {
|
|
it('should count a single statement without semicolon', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should count a single statement with semicolon', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users;')).toBe(1)
|
|
})
|
|
|
|
it('should count multiple statements', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users; SELECT * FROM posts;')).toBe(2)
|
|
})
|
|
|
|
it('should handle empty string as implicit statement', () => {
|
|
expect(getQueryStmtCountHeuristic('')).toBe(0)
|
|
})
|
|
|
|
it('should handle whitespace only as implicit statement', () => {
|
|
expect(getQueryStmtCountHeuristic(' \n\t ')).toBe(0)
|
|
})
|
|
|
|
it('should handle statement with trailing whitespace', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users; \n ')).toBe(1)
|
|
})
|
|
|
|
it('should count three statements', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic('SELECT 1; UPDATE users SET name = "test"; DELETE FROM logs;')
|
|
).toBe(3)
|
|
})
|
|
})
|
|
|
|
describe('single-quoted strings', () => {
|
|
it('should ignore semicolon in single-quoted string', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'hello;world' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle escaped single quote', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'it''s working' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle multiple escaped single quotes', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'it''s a ''test''' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle semicolon in escaped quote context', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'val''s;data' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle multiple statements with single-quoted strings', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic("SELECT 'hello;' FROM users; SELECT 'world;' FROM posts")
|
|
).toBe(2)
|
|
})
|
|
|
|
it('should handle adjacent escaped quotes', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '''' FROM users")).toBe(1)
|
|
})
|
|
})
|
|
|
|
describe('double-quoted strings', () => {
|
|
it('should ignore semicolon in double-quoted string', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "hello;world" FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle escaped double quote', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "it""s working" FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle multiple escaped double quotes', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "it""s a ""test""" FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle semicolon in escaped quote context', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "val""s;data" FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle adjacent escaped quotes', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT """" FROM users')).toBe(1)
|
|
})
|
|
})
|
|
|
|
describe('mixed quotes', () => {
|
|
it('should handle both single and double quotes in same query', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "col;1", \'val;2\' FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle single quote inside double quote', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "it\'s;ok" FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle double quote inside single quote', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT \'"quoted";value\' FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle complex nested scenarios', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic('SELECT "a\'b;c", \'d"e;f\', g FROM users; DELETE FROM logs')
|
|
).toBe(2)
|
|
})
|
|
})
|
|
|
|
describe('line comments', () => {
|
|
it('should ignore semicolon in line comment', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 -- comment;here\nFROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle multiple line comments', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 -- comment;1\n-- comment;2\nFROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle line comment at end of query', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users -- final;comment')).toBe(1)
|
|
})
|
|
|
|
it('should handle line comment with semicolon at end', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users; -- comment;here')).toBe(1)
|
|
})
|
|
|
|
it('should count statements after line comment', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1; -- comment\nSELECT 2')).toBe(2)
|
|
})
|
|
|
|
it('should handle line comment without newline at end (stays in comment state)', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 FROM users -- comment;here')).toBe(1)
|
|
})
|
|
|
|
it('should handle dashes inside string not as comment', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '--not;comment' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle single dash (not a comment)', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT a-b FROM users')).toBe(1)
|
|
})
|
|
})
|
|
|
|
describe('block comments', () => {
|
|
it('should ignore semicolon in block comment', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 /* comment;here */ FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle multiline block comment', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 /* comment;\nacross;\nlines */ FROM users')).toBe(
|
|
1
|
|
)
|
|
})
|
|
|
|
it('should handle multiple block comments', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic(
|
|
'SELECT /* c1;here */ 1, /* c2;here */ 2 FROM /* c3;here */ users'
|
|
)
|
|
).toBe(1)
|
|
})
|
|
|
|
it('should handle nested-looking block comments', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic('SELECT 1 /* outer /* inner;here */ still */ FROM users')
|
|
).toBe(1)
|
|
})
|
|
|
|
it('should handle block comment at end', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT * FROM users /* final;comment */')).toBe(1)
|
|
})
|
|
|
|
it('should count statements after block comment', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1; /* comment */ SELECT 2')).toBe(2)
|
|
})
|
|
|
|
it('should handle block comment markers in string', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '/* not;comment */' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle unclosed block comment (stays in comment state)', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1 /* unclosed;comment')).toBe(1)
|
|
})
|
|
|
|
it('should handle slash-star inside string not as comment', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '/*;not comment' FROM users")).toBe(1)
|
|
})
|
|
})
|
|
|
|
describe('mixed comments and strings', () => {
|
|
it('should handle comment after string', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'value;1' -- comment;2\nFROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle string after comment', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 1 -- comment\n, 'value;here' FROM users")).toBe(1)
|
|
})
|
|
|
|
it('should handle complex mix', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic(
|
|
'SELECT "col;1" /* comment;1 */ , \'val;2\' -- comment;2\nFROM users'
|
|
)
|
|
).toBe(1)
|
|
})
|
|
|
|
it('should handle quotes in comments', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 1 /* it's;working */ FROM users")).toBe(1)
|
|
})
|
|
it('should handle comment markers in string', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '--/*;*/' FROM users -- real;comment")).toBe(1)
|
|
})
|
|
})
|
|
|
|
describe('edge cases', () => {
|
|
it('should handle only semicolons', () => {
|
|
expect(getQueryStmtCountHeuristic(';;;')).toBe(3)
|
|
})
|
|
|
|
it('should handle semicolons with whitespace', () => {
|
|
expect(getQueryStmtCountHeuristic(' ; ; ; ')).toBe(3)
|
|
})
|
|
|
|
it('should handle query ending with multiple semicolons', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1;;')).toBe(2)
|
|
})
|
|
|
|
it('should handle unclosed single quote (stays in quote state)', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT 'unclosed")).toBe(1)
|
|
})
|
|
|
|
it('should handle unclosed double quote (stays in quote state)', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "unclosed')).toBe(1)
|
|
})
|
|
|
|
it('should handle unclosed line comment (no newline at end)', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT 1; -- comment;here')).toBe(1)
|
|
})
|
|
|
|
it('should handle all types together', () => {
|
|
expect(
|
|
getQueryStmtCountHeuristic(
|
|
`SELECT "col;1", 'val;2' FROM users WHERE x = 1; -- comment;here
|
|
/* block;comment */ UPDATE posts SET title = 'new;title';
|
|
DELETE FROM logs -- final;cleanup`
|
|
)
|
|
).toBe(3)
|
|
})
|
|
|
|
it('should handle empty statements', () => {
|
|
expect(getQueryStmtCountHeuristic(';')).toBe(1)
|
|
})
|
|
|
|
it('should handle consecutive escaped quotes at statement boundary', () => {
|
|
expect(getQueryStmtCountHeuristic("SELECT '''';")).toBe(1)
|
|
})
|
|
|
|
it('should handle alternating quote types', () => {
|
|
expect(getQueryStmtCountHeuristic('SELECT "a", \'b\', "c", \'d\' FROM users')).toBe(1)
|
|
})
|
|
|
|
it('should handle very long string with semicolons', () => {
|
|
const longString = ';'.repeat(1000)
|
|
expect(getQueryStmtCountHeuristic(`SELECT '${longString}' FROM users`)).toBe(1)
|
|
})
|
|
|
|
it('should handle realistic multiline query', () => {
|
|
const query = `
|
|
SELECT
|
|
id,
|
|
name,
|
|
"user;email" -- email column
|
|
FROM users
|
|
WHERE status = 'active;pending'
|
|
AND created_at > '2024-01-01';
|
|
|
|
/* Update user preferences */
|
|
UPDATE user_settings
|
|
SET theme = 'dark;mode'
|
|
WHERE user_id IN (SELECT id FROM users WHERE name LIKE '%test%');
|
|
|
|
-- Cleanup old data
|
|
DELETE FROM logs WHERE timestamp < NOW() - INTERVAL '30 days'
|
|
`
|
|
expect(getQueryStmtCountHeuristic(query)).toBe(3)
|
|
})
|
|
})
|
|
})
|