* feat(frontend): run entity-graph derivation as the caller
The derivation contract requires the computed graph tables to run under
the outer query's identity. Capture the caller's QueryContext when the
computed table is resolved, thread it through EntityGraphProvider, and:
- authorize every contributing source table against the caller via the
new semantic_graph.query permission action, silently excluding denied
sources (entities, edges and source_tables never appear);
- execute the derivation plan under the caller's context so it inherits
permissions, cancellation and deadline instead of a fresh default.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(operator): derive the entity-graph window from the scan's time predicate
Implements the RFC window contract for the computed graph tables:
- table: add extract_time_range_strict, a strict variant of the lenient
time-range extraction that distinguishes an absent observed_at filter
from one that cannot be safely turned into a range;
- operator: replace GraphWindow with GraphQueryWindow, splitting the
queried observed_at range from the source-scan range widened to whole
60s buckets, so boundary buckets aggregate over their full extent;
- frontend: resolve the window from ScanRequest filters — no predicate
keeps the last-hour default, a missing upper bound means now, and a
missing lower bound or unextractable shape is an explicit error, never
a silent fallback.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(operator): system-defined declared-edge table for the entity graph
Reintroduces greptime_private.semantic_relationships_declared with a
canonical, system-owned definition:
- the CREATE TABLE expr (8-tag primary key, business validity columns,
RED fields, 30d TTL); attributes is now a json column so the future
union branch matches the computed table without a per-scan parse;
- created on first use on every write path: SQL INSERT creates it
before executing, and the gRPC row-insert auto-create substitutes the
canonical expr instead of deriving a schema from the request;
- user DDL (CREATE/ALTER/DROP/RENAME/TRUNCATE) and write-path
auto-ALTER are rejected via the new is_ddl_reserved_table guard,
while INSERT/DELETE stay allowed.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(operator): union declared edges into semantic_relationships
Adds the declared-edge branch to the relationship derivation
(build_relationships_plan replaces build_calls_plan):
- latest revision per edge key first (mito dedups on primary key plus
observed_at, so a re-asserted edge stores a new revision), then the
business-validity overlap against the queried window; valid_from
defaults to the declaration time and a NULL valid_until means the
edge holds while its row exists;
- the projected observed_at is synthesized inside the queried range
(Inexact pushdown re-applies the scan's filters above the computed
table, which would drop rows keyed by the physical revision time);
window_end/fresh_until of open-ended edges take the window's upper
bound so 'fresh_until >= now() - ...' queries see them;
- tag columns are cast out of dictionary encoding, and the union is
re-projected to the 16-column contract;
- the frontend feeds the branch only when the physical table exists,
the caller may read it, and its schema still matches the canonical
definition (mismatch is an explicit error, not a silent drop).
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: cover declared edges, window contract and caller authorization
- sqlness: system auto-create on first INSERT, latest-revision reads,
open-ended vs retired validity, explicit/lower-only/upper-only window
behavior, user-DDL rejection, rename-into rejection, DELETE cleanup;
- integration: a permission checker denying one trace table excludes it
from both semantic_relationships and semantic_entities.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: allow DROP/TRUNCATE on the declared-edge table and fix CI lints
The definition guard rejected every DDL, which left sqlness (and any
shared deployment) no way to remove the table the semantic_graph case
creates — its extra region then broke unrelated region/partition case
expectations. Narrow the guard to what actually protects the canonical
definition: user CREATE, ALTER, RENAME-into and repartition stay
rejected, while DROP and TRUNCATE are allowed — dropping loses nothing
structural, the next INSERT recreates the table canonically, and DROP
doubles as the recovery path if the canonical definition ever changes.
The sqlness case now verifies drop-then-recreate and cleans up after
itself.
Also: rustfmt for the catalog crate and two typo fixes.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: adapt canonical declared-table create to TriggerReason
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: address review on the declared-edge table lifecycle and revision reads
- gRPC first writes actually work now: the reserved table's creation
went through the generic create_table_inner, which the definition
guard itself rejects; both branches of create_or_alter_tables_on_demand
route it to create_declared_relationships_table instead, and being a
system action it also bypasses the auto_create_table config/hint;
- revision selection is as-of the queried window: revisions recorded
after the window's end, or whose validity starts after it, no longer
outrank (and hide) the revision that was in effect inside it;
- the canonical-schema check validates the whole definition the union
semantics lean on — time index, primary key, engine, append/merge
mode — not just column names and types;
- UNDROP TABLE of the reserved name is rejected like CREATE: it could
resurrect a pre-canonical shape, and the next INSERT recreates the
table anyway.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: trim over-commenting in the entity-graph code
Comments that restated adjacent code or narrated justification are cut;
the ones stating non-obvious contracts and gotchas stay.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject CREATE VIEW against DDL-reserved table names
A view named greptime_private.semantic_relationships_declared would
squat the reserved name: the first INSERT then skips the canonical
create (an object already exists) and graph reads fail on the schema
mismatch. CREATE VIEW now passes the same definition guard as CREATE
TABLE.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: debug-log authorization exclusions; declared-edge TTL to 90d
Sources the derivation contract silently excludes (per-table denial,
whole-scan denial, the declared-edge table) are invisible from outside;
a debug log at each names what was excluded and why.
The declared-edge table's default TTL becomes 90d, overridable at
creation time via GREPTIMEDB_DECLARED_RELATIONSHIPS_TTL (a proper
configuration option is a TODO).
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: rank declared-edge revisions by the visible edge identity
Ranking partitioned by the full primary key, but the projection drops
scope and generation_id: two assertions of the same visible edge under
different generations both ranked first and came out as duplicate,
indistinguishable rows. Rank by the exposed identity (endpoints,
rel_type, provenance) instead, with generation_id/scope as
deterministic tie-breakers for same-timestamp assertions.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: drop redundant declared-edge tests
The generations regression is already asserted by the revision and
as-of tests; the DDL shape test restated the declarative builder
against itself. Its one non-tautological check (attributes maps to the
json type) moves into the schema-matcher test.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject disjunctive graph windows and unmatchable future windows
- OR/IN over observed_at collapse disjoint ranges into their convex
hull; a declared edge's synthesized timestamp can land in a gap and
be dropped by the re-applied filter even though the edge is valid at
a requested instant. The strict extractor now rejects those shapes.
- A lower bound in the future inverts against the implicit up-to-now
upper bound; the declared branch then fabricated an edge observed at
the future bound. Such windows now derive nothing.
- The reserved-table gRPC create path classifies an instant-TTL table
like every sibling path.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(auth): support HTTP bearer-token authentication (#8718)
Adds an opt-in bearer-token (JWT / OAuth2) authentication path to the HTTP
layer, so clients can authenticate with `Authorization: Bearer <token>`
against any `/v1/` interface. Today such requests are rejected with
`UnsupportedAuthScheme("bearer")` -> 401 before any handler runs.
The token is treated as opaque by the server; validation and identity
derivation stay in the UserProvider, so JWT/JWKS/OIDC policy remains
pluggable and out of core.
Changes:
- `auth::UserProvider` gains `auth_token(token, catalog, schema) ->
Result<UserInfoRef>` with a default that rejects
(`Error::UnsupportedAuthMethod`), so password-only providers keep today's
behavior. A provider that supports token auth overrides it to validate the
token, resolve it to a user, and authorize the connection.
- `auth::Error::UnsupportedAuthMethod` for the default-reject case.
- `servers::http::authorize::inner_auth` extracts a bearer token
(`extract_bearer_token`) and, when present, authenticates via
`UserProvider::auth_token`; otherwise it falls through unchanged to the
username/password path (Basic / influxdb / splunk). Basic and bearer
coexist on the same server.
Backward compatible: the default impl preserves existing behavior, and
non-bearer requests take the exact same path as before.
Tests:
- `extract_bearer_token` recognizes `Bearer` (either header) and ignores
Basic/Token/Splunk/empty.
- `inner_auth` dispatches a bearer token to `auth_token` and populates the
QueryContext user on success; rejects on failure.
- A password-only provider (default `auth_token`) rejects bearer tokens.
Refs: #8718
* chore: fmt
* refactor(auth): address bearer-auth review feedback (#8719)
Address the review comments on the HTTP bearer-token authentication PR:
- Match the `Bearer` scheme case-insensitively (RFC 9110 §11.1) via
`eq_ignore_ascii_case`. `extract_bearer_token` previously only accepted
`Bearer`/`bearer`, so a valid `BEARER <token>` fell through to
`UnsupportedAuthScheme` and never reached the provider. The opaque token
itself is deliberately not lowercased.
- Return `Option<&str>` (borrowing the request headers) instead of
`Option<String>`, avoiding an allocation per bearer request.
- Rename `UserProvider::auth_token` -> `auth_bearer_token` for clarity.
- Route bearer-auth failures on Splunk HEC requests through `splunk_hec_err`
(FORBIDDEN, code 4) instead of the generic 401 `ErrorResponse`, so HEC
clients retain their `{"text":"Invalid token","code":4}` endpoint contract.
Adds test coverage for case-insensitive scheme parsing (token preserved
verbatim) and a regression test for the bearer/splunk routing path.
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(auth): warn when credential load disables Postgres SCRAM or drops a line
Static and watch user providers degraded silently in two ways:
- A single non-SCRAM verifier (mysql_native_password, or a legacy
pbkdf2_sha256 hash that predates SCRAM) disables Postgres SCRAM for
every user and falls back to cleartext, with no signal to the operator.
- A malformed credential line (commonly a plaintext password containing
'=', which splits into more than two parts) was dropped without a trace.
Emit a warning at each credential load for both cases so operators don't
unknowingly serve cleartext passwords over Postgres or lose a user. This
is logging only; authentication behavior is unchanged. The SCRAM check
never logs secrets, and the malformed-line warning logs the line number
and file, never the line content.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(auth): warn on credential file read error before truncating
A read error from lines() (I/O failure or invalid UTF-8) ends the
iterator via map_while, silently dropping every remaining credential.
Warn with the line number and file before truncating, matching the
malformed-line handling, so the drop is observable.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: support SCRAM auth for Postgres
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: add pg_scram_sha256 format to hash-password command
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: harden Postgres SCRAM auth
- Verify the client-final nonce matches the server-issued nonce, per RFC 5802
transcript validation, instead of only checking the channel-binding field.
- Replace the per-connection PBKDF2 over a random password for unknown users
with a deterministic mock verifier keyed by the username and a process-wide
secret. This avoids a CPU-exhaustion DoS on unknown usernames and removes a
username-enumeration oracle: the SCRAM server-first salt and iteration count
are now stable per username and indistinguishable from a real user, with no
PBKDF2 cost and random keys that never accept a proof.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* style: format PG_SCRAM_MOCK_SECRET declaration
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: precompute stable SCRAM verifier for plaintext users
Plaintext-backed credentials derived a Postgres SCRAM verifier on the fly
on every connection, using a fresh random salt and running PBKDF2 each
time. That made a known plaintext user distinguishable from stored-hash
and unknown (mock) users through both the unstable server-first salt and
the per-connection timing, enabling username enumeration.
Precompute the SCRAM verifier once at load time (stable salt, default
iteration count) and reuse it, matching the mock verifier handed to
unknown users. Document that non-default iteration counts remain
observable in the SCRAM handshake and weaken enumeration resistance.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: normalize passwords for Postgres SCRAM
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: docs
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: add password hash generation command
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject empty password in hash-password command
A blank plaintext password is rejected by the user provider before
verifier comparison, so a verifier generated from an empty password is
unusable. Fail fast on EOF or an empty line from --password-stdin (and on
an empty --password) instead of printing a dead verifier.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
* feat: add password verifier formats
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: harden password verifier parsing and auth config errors
- Reject pbkdf2_sha256 verifiers whose hash is not 32 bytes and bound the
salt length, preventing short-hash verifiers from matching on a prefix.
- Verify pbkdf2_sha256 with a stack-allocated buffer.
- Report only the length, not the bytes, when a mysql native password
verifier has an illegal length.
- Map empty frontend_auth credentials to an invalid-config error instead
of an internal error.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: update config.md
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: skip non-plain verifiers in get_one_user_pwd
Pick the first plain-text credential instead of failing when the first
user happens to hold a hashed verifier.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: format
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: remove unused get_one_user_pwd
Internal flownode-to-frontend communication no longer authenticates
(see #8244), so the plain-text credential export path is dead code.
Drop get_one_user_pwd, its now-orphan as_plain_text helper, and the
related tests.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: flow add static user/pwd auth
* fix: not print password
* chore: rm explict Any bound
* refactor: per review
* refactor: move away from plugin
* refactor: not use any
* chore: per revieww
* chore: complete a todo
* chore: fix after rebase
* chore: update sqlness results
* refactor: use rwlock for modifiable data in session and querycontext
* chore: format toml
* refactor: use mutable_inner structure for mutable fields
* refactor: remove arc wrapper
Lei, HUANG
shuiyisong
dennis zhuang
Ning Sun
Weny Xu
Ruihang Xia
LFC
discord9
localhost
tison
dimbtp
SteveLauC
Yingwen