mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 16:10:43 +00:00
Hengfei Yang
72f7bff378
## Summary - Page sessions by `session_id` before computing the expensive summary fields. - Aggregate only the selected page of sessions with exact `count(DISTINCT trace_id)`, removing trace ID materialization and Rust-side per-trace rollups. - Generate direct Agent predicates and evaluate them as aggregate membership conditions without a session-membership subquery. - Support efficient server pagination by fetching one extra session to determine `has_more`, without counting every distinct session on every page. ## Why The previous endpoint first materialized `array_agg(DISTINCT trace_id)` for every candidate session and then issued another query using those trace IDs. A single-pass full rollup was also slower because it aggregated every summary field for all matching sessions before applying TopK. This change keeps the inexpensive session page selection separate and bounds the expensive rollup to the requested page size. Page selection and display both use the latest `end_time` across the complete session; Agent predicates only decide whether a session is a member of the result set. The endpoint previously returned `total = hits.len()`. With a 20-row page, OTable therefore calculated one total page and permanently disabled Next even when more sessions existed. An exact `count(DISTINCT session_id)` query scanned the full data set and took about 1.18 seconds by itself on this benchmark. Instead, the page query now requests `size + 1`: while another page exists, `total` is a documented lower bound and the response includes `has_more=true` / `total_is_exact=false`. The final page returns the exact total. OTable displays lower bounds with `+` and hides Last-page navigation until the exact total is known. ## Performance Measured locally against `bench_traces` (167 files, about 99.9 million records), using five runs and ignoring the first two: | Query shape | Mean wall time | | --- | ---: | | Original trace-ID flow | 6.829 s | | Page session IDs + membership subquery | 3.219 s | | Page session IDs + Last activity | 0.850 s | The optimized query shape is about 87.6% faster than the original flow. The second-phase rollup returned 20 rows with about 74 KB of intermediate output and about 2.45 MB peak memory in the query plan. Fetching the extra pagination sentinel did not introduce a meaningful page-size-dependent regression in the final runtime sweep. ## Behavior note The Agent filter controls membership through `HAVING max(CASE WHEN <filter> THEN 1 ELSE 0 END) = 1`, while the final rollup includes all spans for each selected session. Both pagination and display use the complete session's `max(end_time)`, ordered by `session_last_activity DESC, session_id DESC` for stable offset pagination. The UI labels this value `Last activity`; session start and duration remain available from the rollup. Current clients send direct Agent predicates. For compatibility, the endpoint also recognizes the previous narrow same-stream `session_id IN (SELECT session_id ... GROUP BY session_id)` shape and safely extracts its inner WHERE predicate; unrelated subqueries are left unchanged. This assumes every relevant span has a `session_id`. ## Validation - `cargo build --features mimalloc --profile release` - `cargo test -p openobserve-api-search traces::session --lib` (18 passed) - `cargo test -p openobserve-api-search user::tests --lib` (13 passed in the earlier query-change validation) - `cargo fmt --all` - `git diff --check` - targeted Vitest: Sessions composable, SessionsList, and OTable (169 passed) - `vue-tsc --noEmit -p tsconfig.vitest.json --composite false` - targeted ESLint and Prettier checks - runtime pagination check against the supplied legacy-filter curl: - page 1: 20 hits, `total=21`, `has_more=true`, `total_is_exact=false` - page 2: 20 hits, `total=41`, `has_more=true`, `total_is_exact=false` - final page: 3 hits, `total=184443`, `has_more=false`, `total_is_exact=true` - pages 1 and 2 returned 40 unique sessions in descending `end_time` order across the page boundary
web
This template should help get you started developing with Vue 3 in Vite.
Recommended IDE Setup
VSCode + Volar (and disable Vetur) + TypeScript Vue Plugin (Volar).
Type Support for .vue Imports in TS
TypeScript cannot handle type information for .vue imports by default, so we replace the tsc CLI with vue-tsc for type checking. In editors, we need TypeScript Vue Plugin (Volar) to make the TypeScript language service aware of .vue types.
If the standalone TypeScript plugin doesn't feel fast enough to you, Volar has also implemented a Take Over Mode that is more performant. You can enable it by the following steps:
- Disable the built-in TypeScript Extension
- Run
Extensions: Show Built-in Extensionsfrom VSCode's command palette - Find
TypeScript and JavaScript Language Features, right click and selectDisable (Workspace)
- Run
- Reload the VSCode window by running
Developer: Reload Windowfrom the command palette.
Customize configuration
See Vite Configuration Reference.
Project Setup
npm install
Compile and Hot-Reload for Development
For stand-alone UI development environment, create .env file and which contains below environment variable
VITE_OPENOBSERVE_ENDPOINT=http://localhost:5080/
npm run dev
Type-Check, Compile and Minify for Production
npm run build
Run Unit Tests with Vitest
npm run test:unit
Run End-to-End Tests with Cypress
npm run test:e2e:dev
This runs the end-to-end tests against the Vite development server. It is much faster than the production build.
But it's still recommended to test the production build with test:e2e before deploying (e.g. in CI environments):
npm run build
npm run test:e2e
Lint with ESLint
npm run lint