fix: support double-quoted filter identifiers (#3825)

## Summary

- tokenize predicates with the same GenericDialect lexical rules Lance
delegates to
- rewrite only SQL-standard double-quoted identifier tokens to Lance
backticks
- apply one predicate contract to query, count, update, delete, and both
merge conditions
- cover mixed-case identifiers, ordinary literals, comments, and every
filter-bearing table operation

## Root cause

Lance plans double-quoted tokens as string literals for compatibility.
As a result, `"PartyAbbrev" = 'D'` compared two literals and silently
evaluated to false instead of filtering the mixed-case column.

## Validation

- `cargo fmt --all -- --check`
- `cargo test --locked --quiet --features remote -p lancedb
expr::sql::tests`
- `cargo test --locked --quiet --features remote -p lancedb
test_double_quoted_predicates_across_table_operations`
- `cargo check --quiet --features remote --tests --examples`
- `cargo clippy --quiet --features remote --tests --examples`

Fixes #2057

<!-- lance-gatekeeper-fix:v1 agent=a44b6567cfd8890abb4f7395ff71971a
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
This commit is contained in:
lancedb-gatefixer[bot]
2026-08-27 06:17:28 +08:00
committed by GitHub
parent ae81d73563
commit 79f626b09e
14 changed files with 570 additions and 72 deletions
+2 -2
View File
@@ -105,7 +105,7 @@ def test_quickstart(tmp_path):
tbl.create_index(num_sub_vectors=1)
# --8<-- [end:create_index]
# --8<-- [start:delete_rows]
tbl.delete('item = "fizz"')
tbl.delete("item = 'fizz'")
# --8<-- [end:delete_rows]
# --8<-- [start:drop_table]
db.drop_table("my_table")
@@ -201,7 +201,7 @@ async def test_quickstart_async(tmp_path):
await tbl.create_index("vector")
# --8<-- [end:create_index_async]
# --8<-- [start:delete_rows_async]
await tbl.delete('item = "fizz"')
await tbl.delete("item = 'fizz'")
# --8<-- [end:delete_rows_async]
# --8<-- [start:drop_table_async]
await db.drop_table("my_table_async")
@@ -266,7 +266,7 @@ def test_table():
tbl.add(pydantic_model_items)
# --8<-- [end:add_table_from_pydantic]
# --8<-- [start:delete_row]
tbl.delete('item = "fizz"')
tbl.delete("item = 'fizz'")
# --8<-- [end:delete_row]
# --8<-- [start:delete_specific_row]
data = [
@@ -538,7 +538,7 @@ async def test_table_async():
await async_tbl.add(pydantic_model_items)
# --8<-- [end:add_table_async_from_pydantic]
# --8<-- [start:delete_row_async]
await async_tbl.delete('item = "fizz"')
await async_tbl.delete("item = 'fizz'")
# --8<-- [end:delete_row_async]
# --8<-- [start:delete_specific_row_async]
data = [