5dd59cd495
fix: alerts folders UI improvements ( #6398 )
...
1. code refactoring for alert list
---------
Co-authored-by: omkark06 <omkar@zinclabs.io >
Co-authored-by: udev <uddhavdave1@gmail.com >
2025-03-27 07:46:17 +05:30
Omkar Kesarkhane
700b9e59c4
fix: fixed reset pagecount search on error in web-socket ( #6408 )
2025-03-26 14:13:14 -07:00
+2
8cd0683a0b
feat: ws v2 ( #6404 )
...
# WebSocket v2 Architecture
## NOTE: Open Issues:
- [ ] `session_drain` from server is experimental and not ready for
prod. Instead, the client maintains ongoing `trace_id`'s and when cookie
expires it will retry search for all the ongoing searches.
## Overview
Implements websocket v2 with the main goal to establish `Client-Router`
connections `per tab` and `Router-Querier` `persistent` connections.
## New Env (Default):
```sh
ZO_WEBSOCKET_CHECK_COOKIE_EXPIRY=true
ZO_WEBSOCKET_HEALTH_CHECK_INTERVAL_SECS=60
ZO_WEBSOCKET_IS_SESSION_DRAIN_ENABLED=false # This is experimental, not meant for prod
```
## High-Level Flow
1. Client establishes WebSocket connection with the router.
2. Router creates a session.
3. Router maintains persistent connections to all queriers.
4. Client requests are routed to the appropriate querier using
consistent hashing.
5. Responses are routed back using `trace_id` mappings.
6. Health checks maintain connection validity.
7. Failed connections trigger automatic cleanup and reconnection.
8. **Internal Communication Validation Using:**
- **8a.** Internal gRPC tokens
- **8b.** IP validation
- **8c.** WebSocket upgrade headers
## Connection Lifecycle
```mermaid
sequenceDiagram
participant Client
participant Router
participant ConnectionPool
participant QuerierConn
participant Querier
participant Cluster
Client->>Router: WS Connection Request
Router->>SessionManager: register_client(client_id)
Note over Router,SessionManager: Creates channels:<br/>response_tx/rx<br/>disconnect_tx/rx
par Handle Incoming Messages
Router->>Router: spawn handle_incoming task
and Handle Outgoing Messages
Router->>Router: spawn handle_outgoing task
end
Client->>Router: WS Client Event
Router->>SessionManager: get_querier_for_trace(trace_id)
alt No Existing Mapping
Router->>Cluster: get_node_from_consistent_hash(trace_id)
Router->>ConnectionPool: get_or_create_connection(querier)
alt No Existing Connection
ConnectionPool->>QuerierConn: create_connection
QuerierConn->>Querier: WS Connection Request
par Connection Tasks
QuerierConn->>QuerierConn: spawn listen_to_querier_response
QuerierConn->>QuerierConn: spawn health_check
end
Querier-->>QuerierConn: Connection Established
end
Router->>SessionManager: set_querier_for_trace(trace_id, querier)
end
Router->>QuerierConn: register_request(trace_id)
Router->>QuerierConn: send_message(event)
Querier-->>QuerierConn: Response
QuerierConn->>Router: Route Response via response_tx
Router->>Client: Forward Response
Note over Router,Client: On Disconnect/Error:<br/>1. Send disconnect signal<br/>2. Clean up sessions<br/>3. Drain if needed
```
## Connections
```mermaid
graph TB
subgraph "Client Layer"
UI1[UI Client 1]
UI2[UI Client 2]
UI3[UI Client 3]
end
subgraph "Router Layer"
R1[Router 1]
R2[Router 2]
subgraph "Router 1 Connection Pool"
CP1[Connection Pool]
QC11[Querier1 Conn]
QC12[Querier2 Conn]
QC13[Querier3 Conn]
CP1 --- QC11
CP1 --- QC12
CP1 --- QC13
end
subgraph "Router 2 Connection Pool"
CP2[Connection Pool]
QC21[Querier1 Conn]
QC22[Querier2 Conn]
QC23[Querier3 Conn]
CP2 --- QC21
CP2 --- QC22
CP2 --- QC23
end
R1 --- CP1
R2 --- CP2
end
subgraph "Querier Layer"
Q1[Querier 1]
Q2[Querier 2]
Q3[Querier 3]
end
UI1 -.->|WS| R1
UI2 -.->|WS| R1
UI3 -.->|WS| R2
QC11 ===|Persistent WS| Q1
QC12 ===|Persistent WS| Q2
QC13 ===|Persistent WS| Q3
QC21 ===|Persistent WS| Q1
QC22 ===|Persistent WS| Q2
QC23 ===|Persistent WS| Q3
classDef client fill:#f9f,stroke:#333,stroke-width:2px
classDef router fill:#bbf,stroke:#333,stroke-width:2px
classDef querier fill:#bfb,stroke:#333,stroke-width:2px
classDef pool fill:#ffd,stroke:#333,stroke-width:2px
class UI1,UI2,UI3 client
class R1,R2 router
class Q1,Q2,Q3 querier
class CP1,CP2 pool
```
## Components
```mermaid
classDiagram
class WsHandler {
+session_manager: Arc<SessionManager>
+connection_pool: Arc<QuerierConnectionPool>
+handle_connection()
+remove_querier_connection()
+_shutdown()
}
class SessionManager {
+sessions: RwAHashMap<ClientId, SessionInfo>
+mapped_queriers: RwAHashMap<QuerierName, HashSet<TraceId>>
+register_client()
+update_session_activity()
+remove_trace_id()
+unregister_client()
+set_querier_for_trace()
+get_querier_for_trace()
}
class QuerierConnectionPool {
+connections: RwAHashMap<QuerierName, Arc<QuerierConnection>>
+get_or_create_connection()
+remove_querier_connection()
+maintain_connections()
+get_active_connection()
}
class QuerierConnection {
+querier_name: QuerierName
+write: Arc<Mutex<Option<SplitSink>>>
+response_router: Arc<ResponseRouter>
+is_connected: Arc<AtomicBool>
+connect()
+disconnect()
+send_message()
+register_request()
+unregister_request()
}
class ResponseRouter {
+routes: RwAHashMap<TraceId, Sender<WsServerEvents>>
+route_response()
+flush()
+spawn_cleanup_task()
}
WsHandler --> SessionManager
WsHandler --> QuerierConnectionPool
QuerierConnectionPool --> QuerierConnection
QuerierConnection --> ResponseRouter
```
Related Issues: #6396
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com >
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com >
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com >
Co-authored-by: Ashish Kolhe <35160958+oasisk@users.noreply.github.com >
Co-authored-by: udev <uddhav@openobserve.ai >
Co-authored-by: omkark06 <omkar@zinclabs.io >
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech >
Co-authored-by: Omkar Kesarkhane <127078473+omkarK06@users.noreply.github.com >
Co-authored-by: udev <uddhavdave1@gmail.com >
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com >
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com >
2025-03-26 11:43:40 -07:00
ktx-vaidehi
64bfaa10e3
feat: dashboard background color option for metric chart ( #6397 )
2025-03-26 20:18:28 +05:30
Bhargav
dace1fef23
fix: UI changes on datasource page ( #6390 )
2025-03-25 22:08:29 +05:30
Sai Nikhil
b006800b37
fix: alert enable or disable is not updated in the UI when filter applied ( #6388 )
...
1. This PR fixes that alert enable is not updated in the UI when filter
is applied
2. This PR also fixes the alert enable disable in search across folders
3. This PR also removes the id from exported alert
2025-03-25 16:07:58 +05:30
Subhra264
5b7536d7fc
feat: add variables for trigger time in seconds, millis for alert template ( #6387 )
...
This pr adds two more variables to alert templates -
`alert_trigger_time_millis` (the alert trigger occurance time in epoch
milliseconds) and `alert_trigger_time_seconds` (the alert trigger
occurance time in epoch seconds). Note that the `alert_trigger_time` is
alert trigger occurance time in UTC epoch microseconds.
2025-03-25 15:03:34 +05:30
ktx-vaidehi and Kirtan Thakkar
b435ce798f
fix: passing variables of dashboard drilldown to same dashboard ( #6385 )
...
Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech >
2025-03-25 13:58:35 +05:30
Omkar Kesarkhane
f18f265e12
fix: Updated actions execution type label ( #6374 )
2025-03-25 09:00:38 +05:30
c3759a4231
fix: Migration for scheduled_jobs table to use alert id as module_key ( #6365 )
...
Co-authored-by: udev <uddhavdave1@gmail.com >
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai >
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-03-24 15:10:44 +05:30
Omkar Kesarkhane
b7af94f487
fix: Actions UI improvements ( #6372 )
...
1. Remove action_id from Histogram and Page-count search payload
2. Rename Action type `Service` to `Real Time`
3. Added validation for service account selection while creating action
2025-03-24 14:33:16 +05:30
Sai Nikhil
f7d3e5aee0
fix: added tooltip for all icons in icon bar ( #6371 )
...
1. added tooltip for every icons in icon bar
2025-03-24 12:32:27 +05:30
06920b203b
fix: previous histogram getting display even after there is no data ( #6346 )
...
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech >
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai >
2025-03-24 11:27:20 +05:30
ktx-vaidehi
986c01a4a2
fix: dashboard error message getting removed ( #6366 )
2025-03-23 11:47:52 +05:30
Sai Nikhil
c5be09fbf4
fix: changes for icon bar in logs page ( #6362 )
...
This PR includes changes
1. introduced icons for histogram , sql mode, quickmode and moved search
history , download logs , search scheduled jobs options into menu that
will be there at right end
2. this will make the UI to accommodate upto 1200px with datetime as
relative , 1350 when it is absolute
3. introducing icons saved space of ~200px in total for both sides
2025-03-23 10:16:34 +05:30
ktx-vaidehi
2af8b83699
fix: dashboard panel name and folder text changes ( #6364 )
2025-03-22 18:38:57 +05:30
Hengfei Yang
a09d626d42
fix: disable add stream & add field when user doesn't enable UDS ( #6361 )
2025-03-22 10:54:35 +08:00
Abhay Padamani
2c51f7fde3
fix: dashboard empty error issue ( #6348 )
...
- #3044
2025-03-21 17:01:57 +05:30
71b32480ab
feat: alerts folder implementation (UI) ( #6228 )
...
implemented alerts folders in the UI
1. search across folders
2. alert listing page moved condition and description column to the
expand so whenever user clicks on any row it will show respective
condition and description if there
3. provided fixed width for columns so that all columns are in view port
visibility expect name column as it can be varied depending upon name
given by user
4. search across all folders implemented in this PR only
Import alert add ons
1 additional things done to import alert as well like introducing the
folders in import page
---------
Co-authored-by: omkark06 <omkar@zinclabs.io >
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com >
2025-03-21 13:31:54 +05:30
ktx-vaidehi
4ad2865ddb
fix: dashboard variables dependency not working properly ( #6350 )
2025-03-20 18:19:51 +05:30
ktx-vaidehi
901ad1c5dc
fix: dashboard breakdown value not present chart not getting plot ( #6344 )
...
- #6310
2025-03-20 16:37:29 +05:30
Omkar Kesarkhane
ab0b3a64f8
fix: added errors when actions are not enabled while creating destination ( #6342 )
...
1. If actions are disabled and user tries to import action in
destination, Unknown Error was shown instead of 'Action type not
supported'
2025-03-20 13:47:41 +05:30
Omkar Kesarkhane and udev
28106fc564
fix: Actions UI issues ( #6325 )
...
1. Action Issues
2. #6288
---------
Co-authored-by: udev <uddhavdave1@gmail.com >
2025-03-20 09:39:54 +08:00
Abhay Padamani
0698b9f4e8
fix: with query fields validation issue ( #6334 )
...
- #6332
2025-03-19 18:10:32 +05:30
Sai Nikhil
4289ec0a68
fix: alert and pipeline frequency warning message change ( #6318 )
...
This PR fixes couple of issues
1. changed the warning message to minimum frequency should be -- minutes
when user tries to give less than minimum frequency interval in alert
and pipeline
2. other is made period in sync with frequency so whenever user toggles
the cron so in pipelines we are auto selecting the period according the
frequency so toggling not picking up the latest values of freq to period
2025-03-19 15:09:13 +05:30
Sai Nikhil
55f724ff7c
fix: pipeline query node validation issue ( #6303 )
...
this PR will introduce a validation check that if the query is
validating then we need to disable the validate and close button so that
user don't click on it multiple times and also changed text from
validate and close to validating while the query is validating.
2025-03-19 09:44:07 +05:30
Taiming Liu
097d906698
fix: _original in pipeline query ( #6299 )
...
fix : #6120
2 adjustments applied to pipeline queries comparing to normal queries,
when the stream has `store_original` set to true
- [x] stop automatically skipping `_original` for `select * ...` queries
- [x] stop automatically adding `_o2_id` when not selected
With these adjustments, users would need to explicitly select those two
fields when needed.
2025-03-18 12:40:28 -07:00
Omkar Kesarkhane and udev
bb4f30f21d
feat: Actions in Logs and Alert Destinations ( #6129 )
...
Co-authored-by: udev <uddhavdave1@gmail.com >
2025-03-18 14:42:30 +05:30
ktx-vaidehi
49e1a35cb1
feat: dashboard havingCondition for aggregation fields ( #6252 )
2025-03-18 12:25:02 +05:30
Abhay Padamani
20d759eba0
feat: download raw data as csv/json for dashboard panel ( #6175 )
...
- #6191
2025-03-18 10:54:33 +05:30
dependabot[bot]
53edd8f377
chore(deps): bump esbuild and vite in /web ( #6285 )
2025-03-18 08:45:08 +08:00
Bhargav
3ad2f823fc
feat: added updated documentation urls ( #5992 )
2025-03-17 22:14:14 +05:30
Sai Nikhil
37079bec30
fix: Alert list table changes ( #6287 )
...
this PR introduces some changes in alert list those are
1. added period , frequency columns in the table
2. moved condition , description to the expanded row so when user clicks
on row it will get expanded and will be visible
3. the alert name is now trimmed to 50 chars if it is greater than 50
and on hover it will be shown the full.
4. the owner it is greater than 20 then first 10 and last 4 chars will
be visible and remaining between and also added tooltip to view full
owner if is > 20 chars
2025-03-17 20:33:07 +05:30
ktx-vaidehi
4bff435c23
feat: dashboard variables support in html markdown ( #6183 )
2025-03-17 15:52:30 +05:30
ktx-vaidehi
e4dd5a44fe
feat: variables filters added support for more regex options ( #6186 )
2025-03-17 14:00:09 +05:30
Abhay Padamani
87f81355a9
feat: dashboard JSON schema editor ( #6226 )
...
- #6227
2025-03-17 12:52:39 +05:30
ktx-vaidehi
a30afe587f
fix: table chart when dashboard drilldown to logs page, having custom selected passing value of selected row ( #6020 )
2025-03-17 12:20:16 +05:30
Sai Nikhil
e2cfda47fb
fix: unflattened data expand in logs page fixed ( #6261 )
...
1. This PR fixes the issue in #5993
2025-03-14 14:19:39 +05:30
Sai Nikhil
ba1928ea81
fix: search history table revamp ( #6254 )
...
1. This PR fixes the #6223
2025-03-14 13:49:36 +05:30
Bhargav
64955ba5a8
fix: first page pagination offset fix ( #6260 )
2025-03-14 13:00:45 +05:30
Bhargav
0fc1e65fc7
fix: added debounce in stream list filter ( #6253 )
2025-03-13 22:57:32 +05:30
77a70ef4fa
fix: search type to lowercase ( #6197 )
...
Co-authored-by: Taiming Liu <liutaiming3@gmail.com >
Co-authored-by: Bhargav <BJP232004@GMAIL.COM >
2025-03-13 21:43:05 +05:30
ktx-vaidehi
2db5a8b041
feat: dashboard set show_symbol bydefault as true ( #6193 )
2025-03-13 19:57:55 +05:30
Sai Nikhil
fde7f66485
fix: splitter limit in management page ( #6246 )
...
1. this PR adds the limit for splitter in management page/ settings page
2025-03-13 15:07:43 +05:30
Sai Nikhil
b09347d58e
fix: dashboards folder search extra padding removed ( #6242 )
...
This PR removes the extra padding added around dashboard folder search
and also added some padding the y axis
2025-03-12 12:09:16 +05:30
Bhargav
97ca4db1eb
fix: cluster info in values api. ( #6241 )
...
issue #6225
2025-03-12 11:24:39 +05:30
Sai Nikhil
059fb9d268
fix: Dashboard folders improvements ( #6211 )
...
1. added search feature for folders
2. added navigation to respective folder when user clicks on any folder
name when using search across folders feature
2025-03-10 16:12:00 +05:30
Sai Nikhil
72aaa8d67a
fix: pipeline query node improvements ( #6188 )
...
1. validate and close text instead of save button
2. when user selects stream name query wipes out if any previous query
is there
3.add the click event on whole expand bar
4. after opening the sidebar clicking on the SQL editor moves the focus
away from the SQL editor
5. remove the pipeline view on the row and introduce a eye icon in
actions section
6. when user clicks on edit the schema should auto populate respective
to the query
2025-03-10 12:29:26 +05:30
Sai Nikhil
50a2c29492
feat: Import alert feature ( #5897 )
...
This PR implements the issue #5200
2025-03-10 10:39:52 +05:30
dependabot[bot]
5059719a1f
chore(deps): bump axios from 1.7.7 to 1.8.2 in /web ( #6216 )
2025-03-08 23:26:06 +08:00
Omkar Kesarkhane
ktx-vaidehi
Bhargav
Subhra264
dependabot[bot]