mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-18 00:34:11 +00:00
* feat(postgres): Prometheus Remote Write protobuf ingest via COPY FROM STDIN COPY FROM STDIN WITH (FORMAT promwrite [, version 2] [, <options>]) ingests one Prometheus Remote Write request body over the postgres wire protocol — a GreptimeDB-specific, binary COPY format (wire format code 1). The payload is byte-for-byte the body a remote-write client POSTs to /v1/prometheus/write, compression included: buffered until CopyDone and completed through the exact ingestion core that endpoint runs. To share that core, the HTTP handlers' ingestion bodies are extracted into axum-free functions (ingest_remote_write_v1/v2 plus the context extension stamping), and the HTTP routes become thin wrappers — their behavior is unchanged, and the copy layer runs the identical decode, validation, conversion, batched/prepared write paths, per-batch write-progress accounting and context extensions, under the connection's regular postgres context. The WITH options carry what the HTTP endpoint takes as its content-type/-encoding and query parameters; session-provided values (database, user) are not repeated: - version 1|2 (default 1), matching the content-type's proto - zstd, matching Content-Encoding: zstd (snappy by default) - physical_table, matching the ?physical_table= parameter - pipeline_name/pipeline_version, matching the pipeline headers (v1 only; for v2 they are accepted and ignored, exactly like the HTTP route ignores them) Table-less like the other ingestion formats; one COPY statement is one bounded request (shared 64 MiB limit); the row count reports written rows (v1) or written samples plus histograms (v2). The flush enforces the frontend's shared request-memory quota (max_in_flight_write_bytes) — the same limiter the HTTP ingestion protocols use, so COPY cannot bypass the aggregate budget. Wiring: PostgresServer::with_prom_remote_write carries the handler, metric-engine flag, validation mode and shared memory limiter. e2e: v1 body chunked across CopyData boundaries; v2 samples with readback; physical_table option routing; garbage-body decode failure with connection reuse; a tiny-quota regression where a compressed body whose decompressed size exceeds the budget fails like the HTTP route. Signed-off-by: Ning Sun <sunning@greptime.com> * fix(servers): propagate COPY body reservations and test write accounting Signed-off-by: Ning Sun <sunning@greptime.com> * fix(postgres): share Prometheus logical batching with COPY ingestion Signed-off-by: Ning Sun <sunning@greptime.com> * refactor(postgres): narrow Prometheus COPY scope and share write outcomes Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com> Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>