* feat: postgres copy-in support
* feat: add postgres copy-in framework
* feat(postgres): route COPY FROM STDIN flushes through the pending-rows batcher
Follow-up to #9302: begin_copy_in now uses the batching-aware
new_query_context, so copy-in batches are submitted to the
pending-rows batcher like SQL INSERTs when PostgreSQL batching is
enabled for the connection, and keep the direct write path when it
is not. The batcher acknowledges synchronously, so the
CommandComplete row count stays exact in both modes.
Adds test_postgres_copy_in_with_batching asserting the batcher
flush metric increments only when batching is enabled.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(postgres): open up the COPY FROM STDIN WITH clause for format-specific options
The WITH (...) option list is now open instead of restricted to
sqlparser's COPY grammar:
- COPY ... FROM STDIN statements are parsed textually (new
copy_in::parse module, tokenizer-based) so arbitrary KEY value
entries reach the codecs; sqlparser would reject options it does
not know before we ever see them. WITH is optional, matching
PostgreSQL.
- CopyOptionSet becomes an open ordered map of KEY -> String |
Number | Ident | Boolean | Flag with typed take_* accessors
(take_string, take_single_byte, take_flag, raw take_value) and
case-insensitive keys; duplicate options are rejected.
- Each format codec takes the options it understands and rejects the
leftovers with one message naming them ('COPY option(s) X are not
supported by format Y'), so a typo cannot be silently ignored.
The text format now also rejects HEADER/QUOTE/ESCAPE instead of
ignoring them, matching PostgreSQL.
A future format (e.g. influxdb line protocol with
WITH (FORMAT influxdb_line, precision 'ns')) only needs its codec
and a registry entry; no parser changes.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(postgres): reuse arrow-csv decoder for COPY FROM STDIN csv parsing
Replace the hand-rolled CSV record parser and per-value conversion
with arrow-csv's incremental Decoder — the same engine backing
COPY ... FROM 'file.csv' in common-datasource — so both COPY routes
accept identical data. The decoder handles record splitting (quoting,
escaped quotes, custom delimiters, CRLF, headers, chunk boundaries)
and typed conversion against the targeted columns' schema in one
pass; decoded batches become insert rows via the existing
api::helper::vectors_to_rows and datatypes Helper::try_into_vectors,
and the decode schema comes from the greptime Schema's arrow
projection.
The codec framework now has parsers emit complete insertable rows
(api Row) instead of raw byte fields: create_parser takes the
targeted columns and is fallible; text converts with the shared
text-representation helpers, csv with the arrow decoder.
Documented consequences of sharing the engine: JSON and Binary
columns and bare epoch-integer timestamps are rejected for csv, and
an empty field — quoted or not — is NULL; a custom NULL string is
rejected accordingly.
Signed-off-by: Ning Sun <sunning@greptime.com>
* docs(postgres): make the copy-in framework explicitly format-open
Reframe the codec framework docs so nothing assumes csv is the only
other text format: the shared pieces (Field, convert_text_field,
check_text_representable_type) are described as building blocks for
any format whose records carry values as text, while formats with
their own typed decoding (csv today) build rows themselves. State
explicitly that formats are not limited to PostgreSQL's own —
GreptimeDB-specific formats (e.g. a line protocol) register like any
other and define their own options through the open WITH clause.
No code changes beyond a test rename and a doc clarifier.
Signed-off-by: Ning Sun <sunning@greptime.com>
* test(postgres): driver-level coverage for CSV copy-in options
Add an end-to-end test through tokio-postgres exercising HEADER,
custom DELIMITER, quoted fields with embedded newlines, escaped
quotes and CRLF record separators, with read-back verification
through the real insert path. WITH is omitted, matching PostgreSQL.
Signed-off-by: Ning Sun <sunning@greptime.com>
* test(postgres): drop the raw-socket copy-in tests
The simple-protocol raw client duplicated what tokio-postgres and the
integration tests already cover; keep the driver-based tests only.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(postgres): address review: fix text-format O(n^2) drain, share type checks, hex::decode
- text feed: take the longest complete line prefix in one drain and
iterate lines over the slice, instead of draining per line (each
drain memmoved the whole buffer; ~1300 drains for a 64KB chunk of
short lines)
- csv check_column_type: delegate to the shared
check_text_representable_type and additionally reject Binary/Json
- decode_bytea: use the hex crate instead of a hand-rolled decoder
- semantic_type_of: use ColumnSchema::is_time_index() like the flow
adapter's column_schemas_to_proto, instead of index lookups
- parse: drop the unused expect_ident parameter; document the
identifier-case decision
- PostgresServer:🆕 document that query_handler and copy_in_handler
are the same frontend instance
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: remove unrelated files
* feat(postgres)!: drop the text codec; csv is the only COPY FROM STDIN format
Remove the hand-rolled PostgreSQL text-format codec (backslash
escapes, octal, \N nulls) that arrow-csv cannot express, keeping
arrow-csv-based csv as the only implemented format.
- COPY FROM STDIN without FORMAT now fails with guidance ('requires
an explicit format in this version; use WITH (FORMAT csv)') rather
than silently defaulting to the removed format
- FORMAT text/txt and FORMAT binary report 'not supported yet' via
registry stubs instead of unknown-format errors
- codec.rs shrinks to the trait framework: the text-representation
helpers (Field, convert_text_field, bytea hex, timestamp parsing)
are removed with the codec; the module docs note where to find
their shape should a future text-serializing format need them
- csv implements check_column_type directly (no shared default)
Breaking for clients relying on the PostgreSQL default text format;
explicit WITH (FORMAT csv) is required.
Signed-off-by: Ning Sun <sunning@greptime.com>
* perf(postgres): raise COPY FROM STDIN flush batch to 32768 rows and harden flush diagnostics
- COPY_IN_FLUSH_ROWS 8192 -> 32768: without the pending-rows batcher
each flush pays the full insert pipeline (catalog lookup, schema
check, partition split, region write round trip), so 4x larger
batches cut round trips 4x at linearly-bounded memory; the constant
also sizes the arrow decoder batch. Comment documents the trade-off
and points high-rate users at the pending-rows batcher, which turns
flushes into enqueues.
- rows buffer pre-allocates COPY_IN_FLUSH_ROWS.
- flush_copy_in warns on unexpected output variants instead of
silently counting 0 rows.
- batching wire test feeds 70000 rows to stay above the new threshold.
Verified against a real psql 18.6 client: \copy ... WITH (FORMAT csv)
sends FORMAT csv and succeeds; default \copy (text) gets the
'requires an explicit format' guidance; COPY TO STDOUT round-trips.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): reset copy-in state on successful CopyDone
on_copy_done left the CopyInState in place (inert but holding buffers)
until the next COPY overwrote it; take it like on_copy_fail does, so
buffers are freed eagerly and a stray post-done CopyData gets the
protocol-violation error instead of a finished parser.
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: bypass typo check
* chore: format change
* fix(postgres): multi-chunk COPY FROM STDIN corrupted at chunk boundaries
The csv feed loop flushed the arrow-csv decoder at every CopyData
chunk end — including partial batches at record boundaries — but
arrow-csv's contract permits flush() only after decode() returned 0
(a full batch of complete records). Flushing outside that point
corrupts subsequent decoding: fields mis-parse shifted by the size
of the record prefix buffered so far ('000001,host-001,...' parses
as '00001h,ost-001u,...'), either erroring (22P02/22P04) or, worse,
silently writing wrong values. Which chunks trigger it depends on
where record boundaries land, so it appears intermittent with
random-length data (psql 8KB / pgx ~64KB chunking both hit it).
Adopt the strict decoder protocol: feed decodes until input is
exhausted and flushes only when decode returns 0 (batch full);
trailing partial batches surface at the EOF signal in finish().
With batch_size == COPY_IN_FLUSH_ROWS the flush threshold is
unchanged; sub-threshold streams now emit their rows at CopyDone
instead of per chunk, which is semantically identical.
Adds test_csv_chunked_feeds_are_alignment_safe: the same payload
fed at chunk sizes 1..65531 (plus shifted starts) with every row's
content verified — reproduces the corruption on the old loop at
chunk sizes 7, 8, 13, 64 and 1000.
Verified with the reporter's Go reproducer (pgx, content-checked):
10/10 clean at -chunk 8192, and 4096/1000/97/65531 plus the
previously failing benchmark sizes 500/1000/2000/200/800 all round-
trip exactly.
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: bypass spell check
* chore: bypass spell check
* fix(postgres): address PR review — cross-catalog guard, identifier folding, tab delimiter
Three review findings (shuiyisong, PR #9297):
[P1] Enforce the catalog restriction on COPY targets. copy_in_table now
applies validate_catalog_and_schema when QueryOptions::
disallow_cross_catalog_query is set, mirroring SQL INSERT's
check_permission path, BEFORE begin_copy_in switches the query context
to the target catalog (which previously made the restriction
unenforceable downstream). Cross-catalog COPY is now rejected with
'Illegal access to catalog' like INSERT.
[P2] Canonicalize unquoted identifiers like the SQL parser
(ParserContext::canonicalize_identifier): unquoted table-name parts and
columns fold to lowercase, quoted ones keep their case. COPY Metrics
(Host, Ts) now addresses the table CREATE TABLE Metrics created, and
no longer silently misroutes into a coexisting quoted "Metrics".
[P2] Allow a tab as CSV delimiter: take_single_byte accepts a literal
tab while still rejecting newlines and other control characters, and
the option parser now also accepts E'...' escape-string constants
(DELIMITER E'\t'), processing PostgreSQL's backslash escapes.
Verified end-to-end with psql 18.6: unquoted/quoted mixed-case round-
trip into the right tables, cross-catalog rejection asserted in a new
frontend unit test (QueryOptions plugin + second catalog), and both
DELIMITER '<tab>' and DELIMITER E'\t' import tab-separated data.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): accept PG boolean spellings in COPY IN; drop copy state on failure
Two review findings (shuiyisong, PR #9297):
[P2] Boolean columns round-trip between the COPY directions. arrow-csv
only parses true/false while PostgreSQL COPY OUT emits t/f, so
boolean tables failed to re-import their own export (22P02). Boolean
columns now decode as Utf8 in the arrow schema and flush_batch
converts them with PostgreSQL's boolean input grammar (true/false,
t/f, yes/no, y/n, on/off, 1/0, case-insensitive, whitespace
trimmed). Round-trip covered end-to-end: COPY ... TO STDOUT's exact
bytes are fed back via COPY FROM STDIN and verified.
[P2] Drop the COPY state on every server-side failure. on_copy_data
and on_copy_done now take ownership of the state up front, so parse,
conversion or insert errors drop the buffered data instead of
retaining it on long-lived or pooled connections until the next COPY
or connection close (the reviewer measured ~17 MiB retained after a
failed 8 MiB row); on_copy_data success puts the state back. An
e2e test drives a failed COPY followed by a successful one on the
same connection.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): interpret offset-free COPY timestamps in the session timezone
Review finding (shuiyisong, PR #9297): the arrow decode schema carried
Timestamp fields with no timezone, so the CSV decoder parsed
offset-free timestamp strings as UTC while SQL INSERT parses them in
the session timezone — the same input string silently stored a
different instant depending on the ingestion method (8h apart in
Asia/Shanghai).
CopyInCodec::create_parser now receives the session timezone and the
csv decode schema attaches it to Timestamp fields, so naive strings
parse like INSERT; explicit offsets in the input (Z, +08:00) still
take precedence, and UTC sessions are unchanged.
Covered by test_csv_timestamp_session_timezone (naive vs +08:00 vs Z)
and the end-to-end test_postgres_copy_in_session_timezone_parity,
which SETs TIMEZONE TO 'Asia/Shanghai', inserts and COPY-loads the
same string, and asserts both CAST(ts AS BIGINT) readbacks agree on
1704038400000.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): don't unescape E-string COPY options twice
Review finding (shuiyisong, PR #9297): sqlparser's tokenizer already
decodes PostgreSQL escape-string constants before returning
Token::EscapedStringLiteral, so the second unescape_pg_string pass
altered the decoded value — DELIMITER E'\\t' (a literal backslash
followed by t, two characters) was turned into a tab and wrongly
accepted as a single-byte delimiter.
Use the EscapedStringLiteral payload as-is, like SingleQuotedString,
and delete unescape_pg_string. Regression coverage distinguishes the
three cases: E'\t' -> tab, E'\\' -> single backslash, and E'\\t'
-> rejected as a multi-character delimiter.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): COPY OUT timestamps carry session-tz offset; quoted empty strings survive COPY IN
Two review findings (shuiyisong, PR #9297):
[P2] Preserve timestamp instants across COPY OUT/IN in non-UTC
sessions. COPY OUT rendered naive UTC text while COPY IN (since the
session-tz fix) interprets offset-free strings as local time, so
re-importing an export shifted instants by the zone offset. COPY OUT
now renders timestamps with an explicit session-timezone offset
(2024-01-01 08:00:00.000000+08): the copy-out arrow schema carries the
session tz (arrow-pg converts to a tz-aware DateTime) and the pg field
type becomes TIMESTAMPTZ so the offset-bearing text format applies.
Embedded offsets win on re-import regardless of session, keeping the
OUT -> IN round trip exact; UTC sessions are unchanged.
[P2] Preserve quoted empty strings. The shared arrow-csv decoder
nulls every empty field, so a quoted empty string ("" — an empty
STRING in PostgreSQL) imported as NULL. A shadow quote-mask scanner
runs over the same bytes with the decoder's own splitting rules and
records per field whether it was quoted; flush_batch restores quoted
empties to empty strings for string columns, while unquoted empty
fields stay NULL. Masks track records across chunk boundaries and
align with decoder batches.
Covered end-to-end: test_postgres_copy_out_in_timestamp_round_trip
(Asia/Shanghai export asserted to carry +08, exact-epoch re-import
comparison) and test_postgres_copy_in_quoted_empty_string (NULL,
quoted empty and non-empty in one import), plus unit tests including
a chunk-boundary sweep for the quoted-empty mask alignment.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): keep COPY IN quote masks aligned after doubled quotes; allow quoted empties in NOT NULL columns
Two review findings (fengjiachun, PR #9297):
[P2] Doubled quotes desynchronized the shadow quote-mask scanner. In
QuoteInQuoted the doubled-quote branch skipped the second quote AND the
following byte, so a closing quote or delimiter directly after an
escaped quote was swallowed and later fields lost their masks --
import semantics depended on CopyData chunk boundaries. The branch now
consumes only the escaped quote and returns to the in-quoted state,
mirroring the decoder. Regression: "a""b","" ... verified across
chunk sizes 1-64.
[P2] A NOT NULL string column rejected quoted empty strings before
restoration could run: the decode schema preserved the target
nullability, and arrow-csv's batch validation rejects the temporary
NULL a quoted empty decodes to. String fields now decode as nullable
(safe: quoted-vs-unquoted is exactly the restoration's job), and the
target NOT NULL constraint is enforced in flush_batch after
restoration, so "" imports as an empty string while an unquoted empty
field fails with a not-null-constraint error. Regressions at unit and
e2e level: NOT NULL column accepting "" and rejecting unquoted NULL.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(postgres): align COPY IN quote-mask scanner with the CSV decoder on blank lines and CR terminators
Two review findings (fengjiachun, PR #9297):
[P2] Blank lines emitted a spurious empty mask. csv-core skips blank
lines, but the scanner queued one mask per blank record, so every
subsequent decoded row consumed the wrong mask (a leading blank line
turned a quoted empty into NULL and an unquoted empty into an empty
string, and tripped NOT NULL on the wrong row). A blank line before the
header also shifted the header expectation. end_record now returns no
mask for a completely blank record without consuming the header
expectation.
[P2] Standalone CR was not a record terminator. The decoder accepts CR,
CRLF and LF alike; the scanner only recognized CR before LF or at EOF,
so CR-separated input desynchronized masks and flush_batch failed with
'quote masks out of sync'. is_record_end now treats CR alone as a
terminator, consuming one byte (two for CRLF), while a CR at a chunk
boundary still defers via carry so CRLF split across CopyData chunks
stays one terminator.
Regressions: blank lines before/inside records and before a header
(NULL/empty preservation plus NOT NULL behavior), and mixed
CR/CRLF/LF terminators across chunk sizes 1-64 with a trailing
CR-only record.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(postgres)!: drop the quote-mask scanner; COPY IN CSV behaves exactly like file COPY
The shadow quote-mask scanner — kept to preserve PostgreSQL's
quoted-empty-string vs unquoted-NULL distinction — mirrored the
decoder's record-splitting grammar by hand and required a fix for every
rule it drifted on (doubled quotes, blank lines, CR terminators, NOT
NULL interplay). Per maintainers' decision, that complexity is not
worth the divergence: COPY FROM STDIN now behaves EXACTLY like the
file-based COPY ... FROM 'file.csv' — an empty field, quoted or not,
imports as NULL (arrow-csv's engine semantics, identical in both
routes).
Removed: QuoteMaskScanner and helpers, the always-nullable string
decode schema, the restoration + NOT NULL enforcement in flush_batch,
set_string_at, and the scanner-era unit tests. The decode schema again
preserves target nullability, so the decoder enforces NOT NULL itself,
same as the file route.
Kept (small, schema-level, no second grammar): boolean spellings
(t/f) and session-timezone timestamp decoding, both addressing
separate review findings.
E2E: test_postgres_copy_in_empty_field_is_null pins the parity —
quoted empty, unquoted empty and non-empty in one import.
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: remove empty file
* chore: lint issues
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
sqlx 0.8 pins the unmaintained webpki-roots 0.26 line for its rustls
trust store, which still contains the recently compromised e-Tugra CA
root. Upgrading sqlx to 0.9 switches its dependency to webpki-roots
1.x, and the lockfile now resolves to 1.0.10 where the distrusted
root has been removed from the compiled-in Mozilla trust store.
sqlx 0.9 requires dynamic SQL strings to be explicitly audited via
the new SqlSafeStr trait: only &'static str literals are accepted
as-is. All affected call sites (MySQL KV backend, MySQL election,
integration tests and fuzz targets) now wrap their internally
generated SQL templates in AssertSqlSafe; the wrapped SQL is built
from operator config or test-local identifiers only, and dynamic
data keeps flowing through bind parameters.
The mysql-rsa feature is enabled alongside mysql to preserve 0.8
authentication behavior: sqlx 0.9 gates non-TLS RSA password auth
(caching_sha2_password / sha256_password) behind it.
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: serialize struct to json in postgres
* fix: support view scalars and preserve null structs in scalar-to-value conversion
Address PR review:
- Utf8View/BinaryView ScalarValues now convert like their non-view forms
instead of failing row extraction for struct columns
- a null struct scalar converts to Value::Null so a null struct inside a
list stays null in the serialized JSON
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: return errors instead of panics for unsupported arrow field types
Struct-typed query results with arrow field types greptimedb cannot
represent (e.g. Decimal256) used to panic during schema conversion and
row extraction, dropping the client connection. They now surface as
query errors:
- ConcreteDataType::try_from builds struct types fallibly via the new
StructType::try_from_arrow_fields
- Value::try_from(ScalarValue::Struct) uses the same fallible path
- new try_value_from_array converts an arrow element to Value with
error propagation, used by the postgres struct encoding
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: postgres describe for more statements
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: cover more show statements
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: address review comments
- add missing `clippy::too_many_arguments` allow on
`query_from_information_schema_dataframe` (CI clippy failure)
- take `&ShowKind` in the information-schema dataframe helper so `kind`
is no longer cloned at every call site; only the WHERE arm (which needs
an owned expression for `sql_to_expr`) clones internally
- document why re-applying TQL explain formats never overwrites an
existing value (per-query context state)
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: trim comments to essentials
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(mysql): interpret prepared statement datetime params in session timezone
Binary DATETIME parameters of server-side prepared statements were
converted as if UTC, ignoring the session timezone set via SET time_zone.
Convert them with the session timezone and add an integration test
covering prepared inserts and predicates under Asia/Shanghai.
Signed-off-by: wy471x <wy471x@gmail.com>
* refactor: share naive datetime timezone policy via common-time
Address review feedback on the prepared-statement timezone fix:
- Expose Timestamp::from_naive_datetime in common-time so the DST policy
(gap -> error, ambiguous -> earlier instant) lives in one place, shared
by the text protocol (Timestamp::from_str) and the MySQL binary protocol.
- Route the MySQL prepared-statement datetime conversion through it.
- Match the target type before converting datetime params so
PreparedStmtTypeMismatch fails fast without wasted conversion.
- Use the short Timezone import form for consistency with the rest of servers.
Signed-off-by: wy471x <wy471x@gmail.com>
---------
Signed-off-by: wy471x <wy471x@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
* fix: record catalog and schema in slow queries
Add catalog and schema context to slow query records while appending the new columns after existing fields to preserve column order.
- `src/common/frontend/src/slow_query_event.rs`: extend `SlowQueryEvent` schema and rows with `catalog_name` and `schema_name`, and cover append-only ordering.
- `src/catalog/src/process_manager.rs`: carry catalog and schema through `SlowQueryTimer`.
- `src/frontend/src/instance.rs`: capture context for SQL, plan, and PromQL slow query timers.
- `tests-integration/tests/sql.rs`: assert MySQL and PostgreSQL slow query records include catalog and schema.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: address slow query review comment
Use `String::clone` when writing slow query catalog and schema values.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: keep slow query schema only
Remove the slow query `catalog_name` column and keep `schema_name` as a non-null tag dimension.
- `src/common/frontend/src/slow_query_event.rs`: expose only `schema_name` in `SlowQueryEvent` rows and mark it as a tag.
- `src/catalog/src/process_manager.rs`: stop carrying catalog context in `SlowQueryTimer`.
- `src/frontend/src/instance.rs`: pass only schema context to slow query timers.
- `tests-integration/tests/sql.rs`: assert slow query records include `schema_name` without `catalog_name`.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: schema name semantic should be field
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: typo
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(server): describe EXPLAIN statements so bind parameters work
`do_describe_inner` only planned `Insert`/`Query`/`Delete`, so
`EXPLAIN` and `EXPLAIN ANALYZE` fell through to the non-plan branch
and had no parameter-type inference. At Bind time the Postgres
handler then reported `unsupported_parameter_type` even though the
inner query would have worked on its own.
Recurse one level into `Statement::Explain` so that an EXPLAIN
wrapping a plannable statement goes through the same describe path.
Adds a tokio-postgres integration test that exercises
`EXPLAIN`/`EXPLAIN ANALYZE` over the extended query protocol.
Fixes#8029
Signed-off-by: BootstrapperSBL <yvanwww@gmail.com>
* refactor(server): extract plannable-inner check into closure
Reduce duplication between the direct match and the EXPLAIN inner match
by factoring out is_inner_plannable. Behaviour unchanged.
Signed-off-by: BootstrapperSBL <yvanwww@gmail.com>
---------
Signed-off-by: BootstrapperSBL <yvanwww@gmail.com>
* feat: add support for decimal parameter type, remove string replacement fallback
* chore: format
* fix: add support for using unsigned bigint in postgres
* chore: format toml
* refactor: cleanup duplicated code
* fix: rescale decimal
* feat: use arrow-pg for encode_row
* refactor: remove bytea and datetime module
* feat: port more encodings to arrow-pg
* feat: implement intervalstyle
* chore: format
* chore: remove error that is no longer used
* chore: use released arrow-pg
* Apply suggestions from code review
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
---------
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
* feat: add updated_on to tablemeta with a default of created_on
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: support the update_on on alter procedure
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: add updated_on into information_schema.tables
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* fix: make sqlness happy
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* test: add test case for tablemeta update
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* fix: fix failing test for ALTER TABLE
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: use created_on as default for updated_on when missing
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
---------
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat remove datetime type
* chore: fix unit test
* chore: add column test
* refactor: move create and alter validation to one place
* chore: minor refactor ut
* refactor: rename expr_factory to expr_helper
* chore: remove unnecessary args
* feat: list/array support for postgres output
* fix: implement time zone support for postgrsql
* feat: add a geohash function that returns array
* fix: typo
* fix: lint warnings
* test: add sqlness test
* refactor: check resolution range before convert value
* fix: test result for sqlness
* feat: upgrade pgwire apis
* feat: add json type and vector
* fix: allow to create and insert json data
* feat: udf to query json as string
* refactor: remove JsonbValue and JsonVector
* feat: show json value as strings
* chore: make ci happy
* test: adunit test and sqlness test
* refactor: use binary as grpc value of json
* fix: use non-preserve-order jsonb
* test: revert changed test
* refactor: change udf get_by_path to jq
* chore: make ci happy
* fix: distinguish binary and json in proto
* chore: delete udf for future pr
* refactor: remove Value(Json)
* chore: follow review comments
* test: some tests and checks
* test: fix unit tests
* chore: follow review comments
* chore: corresponding changes to proto
* fix: change grpc and pgsql server behavior alongside with sqlness/crud tests
* chore: follow review comments
* feat: udf of conversions between json and strings, used for grpc server
* refactor: rename to_string to json_to_string
* test: add more sqlness test for json
* chore: thanks for review :)
* Apply suggestions from code review
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com>
* test: add integration_test for datetime style
* feat: support various datestyle for postgres
* doc: rewrite the comment about merge_datestyle_value
* test: add more test to illustrate valid datestyle input
* feat: support set variable statement of session
* feat: support printing postgresql's bytea data type in its "hex" and "escape" format in ugly way
* refactor: add 'SessionConfigValue' type and unify the name
* doc: add license header
* refactor: confine coupling with 'sql::ast::Value' in SessionConfigValue
* refactor: move all bytea wrapper into bytea.rs
* fix: remove unused import in context.rs and postgres.rs
* refactor: rename 'set_configuration_parameter' to 'set_session_config'
rename 'set_configuration_parameter' in statement_.rs to 'set_session_config'
* refactor: use mod to organize options via macro
* refactor: re-model the session config value with static type
* test: add integration test
* refactor: move the encode bytea by format type logic into encoder
refactor: use Arc<DashMap> instead of DashMap in QueryContext
refactor: use Arc<DashMap> instead of DashMap in QueryContext
Avoid expensive clone
refactor: use unreachable!() instead of unimplemented!()
refactor: move the encode bytea by format type logic into encoder
test: add binary format integration test case
* test: add ut for byte related type
* doc: remove TODO of bytea_output
* refactor: simplify the implementation with simple struct instead of complex typing
* fix: typo of 'Available'
* fix compile
Signed-off-by: tison <wander4096@gmail.com>
---------
Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
Ning Sun
Yingwen
Weny Xu
discord9andNing Sun
wy471xandNing Sun
yimeng
Lei, HUANG
Yvan Wang
Ruihang Xia
Alan Tang
dennis zhuangandCopilot
zyy17
jeremyhi
Zhenchi
Yohan WalandWeny Xu
JohnsonLee
x³u³