Patrick Sullivan and Subhradeep Chakraborty
edf8b7b069
feat: Migrate reports from meta into reports table ( #5946 )
...
Migrates reports out of the `meta` table into new `reports` and
`report_dashboards` tables.
- Creates a new `reports` table
- Creates a new `report_dashboards` junction/join table that establishes
a many-to-many relation between reports and dashboards.
- Transforms data from the `meta` table into the new tables
- Creates a new `"default"` report folder for any organization that has
a report.
- Creates a new `reports` record for each report in the `meta` table.
- Creates a new `report_dashboards` record to link each report to each
of the dashboards it references.
- Previously, in the `meta` table, reports referenced dashboards by
folder ID and dashboard ID that was stored in JSON. This meant that
moving a dashboard would cause the references to that dashboard in
different reports to break. Since the `report_dashboards` table links
reports and dashboards using foreign keys to their KSUIDs, moving a
dashboard can no longer invalidate this reference.
- When `dashboards` or `reports` records are deleted, those deletions
are cascaded to the `report_dashboards` table.
- Updates the existing reports service layer logic to use the new
`reports` table rather than the `meta` table.
- Adds logic so that when a dashboard is deleted, any report which
referenced it is deleted as well.
- ⚠️ This PR removes the reports cache. I'm not sure if this is actually
something that will be OK or if doing this would create problems.
- Rather than trying to update the existing reports cache logic to work
with the other changes introduced in this PR, I thought that it might be
easier to remove the cache and then add it back later in an updated form
which uses the KSUIDs as cache keys rather than the report names since
eventually we will want to make it possible to have reports with
duplicate names.
- If we do actually need the report cache, I can add it back to this PR
and make the necessary updates so that it continues functioning.
---------
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com >
2025-06-16 16:25:29 +05:30
Hengfei Yang
7f2857d705
ci: enable symbol for dev image ( #7177 )
2025-06-14 23:39:46 +08:00
Hengfei Yang
42541ad53a
fix: add memory circuit breaker for ingester ( #7171 )
...
### **User description**
This PR add circuit breaker for ingester to protect OOM. To enable this
feature we need to config:
```
ZO_MEMORY_CIRCUIT_BREAKER_ENABLED=true
ZO_MEMORY_CIRCUIT_BREAKER_RATIO=90
```
If the system memory usage over `90%` , we will return `503` with error
`MemoryCircuitBreakerError`.
Default is `false`, disabled.
___
### **PR Type**
Enhancement
___
### **Description**
- Add memory circuit breaker config and defaults
- Implement breaker check in ingester writer logic
- Integrate ingestion resource check across services
- Introduce new error variants for circuit breaker
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>config.rs</strong><dd><code>Define memory circuit breaker
configuration defaults</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62 ">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>loki.proto</strong><dd><code>Add Java proto generation
options</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-c6860d35cb793f55cbf91f6dc9e9366bf45a42c3b6385c6c07e813f22d18ccfb ">+10/-14</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Error
handling</strong></td><td><details><summary>3 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add IngestionError error
variant</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36 ">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>errors.rs</strong><dd><code>Introduce
MemoryCircuitBreakerError variant</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-bf45932466265b732f13d0cd9b9cffbaefbe37c5861ace9315449eeca9250713 ">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>syslog.rs</strong><dd><code>Handle circuit breaker error
logging</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa ">+2/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>2
files</summary><table>
<tr>
<td><strong>lib.rs</strong><dd><code>Export check_memory_circuit_breaker
function</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43 ">+4/-1</a>
</td>
</tr>
<tr>
<td><strong>hec.rs</strong><dd><code>Remove pre-ingest resource
check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-5434f454898e66d4a982801761a349ac7e6862836e15eefaf643b31d0d2933dd ">+1/-9</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>12
files</summary><table>
<tr>
<td><strong>writer.rs</strong><dd><code>Implement memory circuit breaker
logic</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f ">+16/-2</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Extend check_ingestion_allowed
signature and call breaker</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b ">+9/-2</a>
</td>
</tr>
<tr>
<td><strong>bulk.rs</strong><dd><code>Pass stream type to ingestion
check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48 ">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>ingest.rs</strong><dd><code>Include circuit breaker in log
ingest checks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f ">+3/-1</a>
</td>
</tr>
<tr>
<td><strong>otlp.rs</strong><dd><code>Add breaker check for OTLP
logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-183498d98b17ddf344cbab4da6b2d9fe89ac81112db51655c142a89023b46ef3 ">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>json.rs</strong><dd><code>Use unified ingestion check for
metrics</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a ">+10/-20</a>
</td>
</tr>
<tr>
<td><strong>otlp.rs</strong><dd><code>Add resource check for metrics
OTLP</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd ">+4/-23</a>
</td>
</tr>
<tr>
<td><strong>prom.rs</strong><dd><code>Invoke ingestion check in Prom
remote_write</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-c4e9e1d49a2bd0fefe67684b3768c4792be63cebcf727d71a48141f5b5957b8c ">+4/-18</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Use new error variant for circuit
breaker</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-124b08e44cb8740f8cb0dde3342ccd8c1b3b4e3c6a0edb12c151e060c54a0bd0 ">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>storage.rs</strong><dd><code>Always apply memory circuit
breaker check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094 ">+2/-3</a>
</td>
</tr>
<tr>
<td><strong>wal.rs</strong><dd><code>Enforce breaker and release files
on error</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-48aea52fd870678e577e03f29fc9769920b7f74761b1447242835dbf1b8ac8df ">+5/-6</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add breaker check for traces
ingestion</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30 ">+14/-58</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>loki.rs</strong><dd><code>Remove obsolete comment in
StreamAdapter</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-519e5b0c03a8916b62712d1c892dab46282acb6c3f9942c2001169a0e2ea2f3c ">+0/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search_stream.rs</strong><dd><code>Reformat long comment
lines for clarity</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4 ">+9/-6</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>write.rs</strong><dd><code>Adjust Writer initializer tuple
unpack</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-791f2d9a83e5cd9620a9649156f12988a8cf1846337d6953bfc07bea9a46b71a ">+1/-1</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
2025-06-14 16:25:07 +08:00
Loakesh Indiran and udev
c29bc97f0a
fix: histogram interval update subquery for HTTP2 ( #7170 ) ( #7174 )
...
### **User description**
fixes #7169 for HTTP2 streaming
cherry-picked from: https://github.com/openobserve/openobserve/pull/7170
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Replace manual AST walk with VisitorMut for histogram
- Inject missing interval arg into `histogram` calls
- Remove deprecated `visit_statements_mut` logic
- Reflow comments in search_stream.rs
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>websocket.rs</strong><dd><code>Use VisitorMut for
histogram interval update</code>
</dd></summary>
<hr>
src/common/utils/websocket.rs
<li>Introduce <code>HistogramIntervalVisitorMut</code> struct<br> <li>
Implement <code>VisitorMut</code> to append missing interval args<br>
<li> Update <code>update_histogram_interval_in_query</code> to use
visitor<br> <li> Remove old <code>visit_statements_mut</code> & helper
function
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7174/files#diff-6ff41099eded8eca4a0e579142d7af4429efb83350917780e0d3ae7dac13847a ">+39/-44</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Formatting</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>search_stream.rs</strong><dd><code>Reflow comments in
partitioned search</code>
</dd></summary>
<hr>
src/service/search/search_stream.rs
<li>Reflow and wrap multi-line comments<br> <li> Adjust comment
formatting around skip logic
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7174/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4 ">+9/-6</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
Co-authored-by: udev <uddhav@openobserve.ai >
2025-06-13 23:19:04 +05:30
udev
30a1c73b62
fix: temporarily change size ( #7167 ) ( #7168 )
2025-06-13 21:40:06 +08:00
Hengfei Yang
e543f1fdd1
perf: allow multiple compactor run delete files ( #7130 )
...
### **User description**
impl #7094
___
### **PR Type**
Enhancement
___
### **Description**
* Introduce `id` in deletion metadata for tracking.
* Use `FileKey` in batch_remove_deleted instead of strings.
* Add transactional locks and update `created_at` on deletion queries.
* Simplify compactor delete API and loop logic.
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>11
files</summary><table>
<tr>
<td><strong>stream.rs</strong><dd><code>Add `id` field to deletion
metadata structs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd ">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Switch batch_remove_deleted to use
`FileKey`</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-3df833bc826eb12e0819620d1fe316ef1ac352d293c11e57cc3df0da3c031e93 ">+4/-2</a>
</td>
</tr>
<tr>
<td><strong>mysql.rs</strong><dd><code>Implement id-based deletion with
MySQL locks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-06d02c738c6a4dc22ff022c16befef8d46aece6cfaa88bb4a5d3d5f92c2e50a1 ">+162/-10</a></td>
</tr>
<tr>
<td><strong>postgres.rs</strong><dd><code>Add advisory lock and id-based
deletion</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-bb24a30c9c3744eb31fb49e4271a9651d1f9a94a87455ac0074364b879af9651 ">+98/-21</a>
</td>
</tr>
<tr>
<td><strong>sqlite.rs</strong><dd><code>Use `FileKey` for SQLite batch
removals</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-85ff00530831fdaa4e13c05cb649f2e20479cce04a4b65ef78d131cdb89b6c71 ">+11/-4</a>
</td>
</tr>
<tr>
<td><strong>deleted.rs</strong><dd><code>Refactor delete API and batch
size handling</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-4900b3bce3c590ac846c7f12e82e1d1cf7c2de3de9fc385fdbe53deb8a8e50ab ">+15/-19</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Remove node hashing and offset for
deletion loop</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-dce43be4d49b0bf99b61099d5b348bc539433d6e9b1ce6b7025d65e907f00b95 ">+1/-12</a>
</td>
</tr>
<tr>
<td><strong>merge.rs</strong><dd><code>Include `id` in merge compaction
delete events</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-0ab016dbd5129c9c285c9c459bf78928147bce97dc92a03aa754da3da71168d7 ">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>retention.rs</strong><dd><code>Include `id` in retention
delete records</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc ">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>local.rs</strong><dd><code>Use `FileKey` in local cache
remove operations</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-9a5afae4b3561db4345d6fc0f36a255e931069e44ee0964443f389759605fa04 ">+3/-2</a>
</td>
</tr>
<tr>
<td><strong>file_list_dump.rs</strong><dd><code>Set `id` for dump
deletion mappings</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7130/files#diff-5ded270908494f33b48753b130a81371711a543920676ced3397ed26a5218b13 ">+1/-0</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
2025-06-12 10:39:16 +08:00
Huaijin
74cc408c91
fix: empty relation with optimizer rule ( #7136 )
2025-06-12 08:32:43 +08:00
Abhay Padamani
664a05e75e
fix: escape quote for variable values ( #7134 )
...
### **User description**
- Config to enable escape quotes.
- Needs to enable it if variable value contains single quote.
For example variable value is `EFhoAf&fugy=P'fG;LR;Hr=jTe3fr).p%"`
then will require to escape single quote from value.
- For custom typed variables, added some validations:
1. At least one option must there.
2. At lease one option should be selected as default value.
___
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Add `escapeSingleQuotes` flag to variable config
- Provide UI toggle with explanatory tooltip
- Update query loader to escape quotes conditionally
- Prevent removing last option and enforce default selection
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add escape_single_quotes field
to config</code>
</dd></summary>
<hr>
src/config/src/meta/dashboards/v5/mod.rs
<li>Introduce <code>escape_single_quotes</code> boolean field<br> <li>
Mark field as optional in <code>VariableList</code> struct
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7134/files#diff-da49585bbcb02bf0f737bb68739ebc248edd15c7ad9427c0e68f7728bb4a8c17 ">+2/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>AddSettingVariable.vue</strong><dd><code>Enhance
variable settings UI for quote escaping</code>
</dd></summary>
<hr>
web/src/components/dashboards/settings/AddSettingVariable.vue
<li>Add <code>escapeSingleQuotes</code> toggle with tooltip<br> <li>
Initialize default <code>escapeSingleQuotes</code> value to false<br>
<li> Disable remove button when only one option remains<br> <li>
Validate at least one default selection for custom variables
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7134/files#diff-e622f54c1f7629ee67d29ed2fc87008d07eb9246767a05435cc42c3c17ce459a ">+53/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>usePanelDataLoader.ts</strong><dd><code>Implement
conditional quote escaping in loader</code>
</dd></summary>
<hr>
web/src/composables/dashboard/usePanelDataLoader.ts
<li>Import <code>escapeSingleQuotes</code> utility<br> <li>
Conditionally escape single quotes in variable values<br> <li> Apply
escaping for both array and scalar values
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7134/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9 ">+14/-2</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Documentation</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>en.json</strong><dd><code>Add translation for
escapeSingleQuotes</code>
</dd></summary>
<hr>
web/src/locales/languages/en.json
- Add `"escapeSingleQuotes": "Escape Single Quotes"` entry
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7134/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824 ">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
2025-06-11 22:21:55 +05:30
udev and hengfei yang
59542d8dd7
fix: remote wal writer improvements ( #7108 )
...
### **PR Type**
Bug fix, Enhancement, Documentation
___
### **Description**
- Expose header size from `WalWriter::new` call
- Adapt ingester to new WAL signature
- Add pipeline WAL writer metrics
- Add task-bound processing warning comment
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>metrics.rs</strong><dd><code>Add pipeline WAL writer
metrics</code>
</dd></summary>
<hr>
src/config/src/metrics.rs
<li>Introduce pipeline WAL writer metrics<br> <li> Register new metrics
in <code>register_metrics</code><br> <li> Make
<code>create_const_labels</code> function public
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-950d37bdd77aa533539acd818a3fcdebd90ccb1f09acb5490b9c0f9b51cb2090 ">+50/-1</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>writer.rs</strong><dd><code>Adapt ingester writer to
WAL signature change</code>
</dd></summary>
<hr>
src/ingester/src/writer.rs
<li>Destructure <code>(WalWriter, usize)</code> tuple from
<code>WalWriter::new</code><br> <li> Use <code>.0</code> to extract the
WAL writer instance
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f ">+3/-2</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>writer.rs</strong><dd><code>Return header size from WAL
builder</code>
</dd></summary>
<hr>
src/wal/src/writer.rs
<li>Change <code>Writer::new</code> to return <code>(Self,
usize)</code><br> <li> Compute and return <code>header_len</code>
alongside writer<br> <li> Update return signature and path conversion
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-8821fda229e1219ff38091157f2065914187d34c29ccc8802c374af500f43c1f ">+15/-11</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Documentation</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>batch_execution.rs</strong><dd><code>Add processing
task warning</code>
</dd></summary>
<hr>
src/service/pipeline/batch_execution.rs
- Add warning comment about task-bound node processing
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69 ">+2/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
---------
Co-authored-by: hengfei yang <hengfei.yang@gmail.com >
2025-06-11 20:44:51 +05:30
Yashodhan Joshi and Yashodhan Joshi
ac5754b28b
fix: fix size reported in usage ( #7128 )
...
### **User description**
This fixes the size reported in usage stream. We used to calculate size
after flattening the data and adding _timestamp col etc, because of
which the size that is actually sent for ingestion and size reported in
usage stream differed. This fixes that issue.
Also we didn't cover all the cases when calculating size of string,
fixed some of them.
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
Enhance JSON size estimation with escape counting
Track raw JSON byte sizes per ingestion
Override usage reporting with actual bytes
Include HEC type in ingestion detection
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6
files</summary><table>
<tr>
<td><strong>usage.rs</strong><dd><code>Include HEC in ingestion usage
types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-b15a6adcb163dc27993c83528105a6726c44c86d8a84c5cf5d1d382c6ec8cc29 ">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>json.rs</strong><dd><code>Count escaped quotes and slashes
in JSON size</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-f638bd6d218ad53a671681c2fa8e420cfbf8369e6e93269c9559e9711338d52a ">+6/-1</a>
</td>
</tr>
<tr>
<td><strong>bulk.rs</strong><dd><code>Track raw JSON byte sizes per
ingestion stream</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48 ">+16/-1</a>
</td>
</tr>
<tr>
<td><strong>ingest.rs</strong><dd><code>Accumulate JSON sizes in generic
ingest module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f ">+18/-1</a>
</td>
</tr>
<tr>
<td><strong>otlp.rs</strong><dd><code>Track and accumulate OTLP JSON
byte sizes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-183498d98b17ddf344cbab4da6b2d9fe89ac81112db51655c142a89023b46ef3 ">+14/-1</a>
</td>
</tr>
<tr>
<td><strong>syslog.rs</strong><dd><code>Track and accumulate syslog JSON
byte sizes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa ">+16/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Override reported size using actual
byte counts</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7128/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2 ">+10/-1</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com >
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com >
2025-06-11 18:05:16 +05:30
udev
086ef89968
fix: handle offset pagination on leader ( #7132 )
...
### **PR Type**
Bug fix
___
### **Description**
- Support offset pagination with `hits_to_skip`
- Adjust cache flow logic on paginated queries
- Trim skipped hits in partitioned search
- Fix delta search truncation condition
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>search_stream.rs</strong><dd><code>Implement offset
pagination skip logic</code>
</dd></summary>
<hr>
src/service/search/search_stream.rs
<li>Added <code>hits_to_skip</code> calculation, reset
<code>req.query.from</code><br> <li> Updated cache usage for paginated
requests<br> <li> Passed <code>hits_to_skip</code> to partitioned
search, trimmed hits<br> <li> Corrected delta search size truncation
logic
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7132/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4 ">+46/-3</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
2025-06-11 16:07:52 +05:30
Hengfei Yang
b849f635fc
fix: call CryptoProvider::install_default() before this point ( #7125 )
2025-06-11 16:58:35 +08:00
Maheep Kumar
529e520df8
feat: TLS support for Syslog TCP Server ( #7072 )
2025-06-11 11:25:23 +08:00
Huaijin
f9968ac3b8
feat: optimizer dashboard select all ( #7114 )
2025-06-10 21:14:11 +08:00
Hengfei Yang and loaki07
a1445809b3
fix: stream name issue with pipeline ( #7113 )
...
### **User description**
fixed #7109
---------
Co-authored-by: loaki07 <rustyloaki@gmail.com >
2025-06-10 20:35:47 +08:00
Hengfei Yang and loaki07
5284df49dc
fix: improve streaming output ( #7106 )
...
### **User description**
This PR use `LRU` replace `VecDeque` to evict the oldest (no longer)
used streaming id
___
### **PR Type**
Enhancement, Tests
___
### **Description**
- Replace `VecDeque` with `LruCache` eviction
- Evict 1% oldest streaming entries on insert
- Refactor `StreamingIdCache` concurrency and eviction
- Update dependency, tests, bump version
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>streaming_aggs_exec.rs</strong><dd><code>Use LruCache
for streaming cache eviction</code>
</dd></summary>
<hr>
src/service/search/datafusion/distributed_plan/streaming_aggs_exec.rs
• Swapped out <code>VecDeque</code> for <code>LruCache</code> from
<code>hashlink</code> <br> • Evict oldest 1% entries via new
<code>gc</code> method <br> • Refactored <code>StreamingIdCache</code>
to use <code>RwLock<LruCache></code> <br> • Updated insert, exists,
remove, check_time logic and logs
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7106/files#diff-1c0c32a6e29645ada6b2d29147cca3ebc9251e144a998bdf43d74bb94b5dabe6 ">+44/-24</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>Cargo.toml</strong><dd><code>Bump version and add
hashlink dep</code>
</dd></summary>
<hr>
Cargo.toml
• Bumped package version to <code>0.15.0</code> <br> • Added
<code>hashlink</code> workspace dependency
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7106/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542 ">+3/-2</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
---------
Co-authored-by: loaki07 <rustyloaki@gmail.com >
2025-06-10 19:59:20 +08:00
Yashodhan Joshi and Yashodhan Joshi
1785b2b256
test: use multiple dbs ( #7084 )
...
This PR adds support for running unit tests with all three - sqlite,
postgres and mysql. This way we cover their code as well.
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com >
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com >
2025-06-10 10:32:00 +05:30
Hengfei Yang
b58dbda062
fix: change stream name label to o2_stream_name ( #7092 )
...
### **User description**
For `Loki` API we use `o2_stream_name` label to decide the stream name.
if we can't get this label value, we will use `default` as stream name.
___
### **PR Type**
Enhancement
___
### **Description**
- Refactor Loki stream label constant
- Update default config env values
- Increase file size limits defaults
- Reduce persistence and compaction intervals
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Update configuration
defaults and constants</code>
</dd></summary>
<hr>
src/config/src/config.rs
<li>Added <code>STREAM_NAME_LABEL</code> and
<code>DEFAULT_STREAM_NAME</code><br> <li> Enabled streaming, disabled
WAL fsync, disabled memory cache<br> <li> Increased default file size
limits<br> <li> Reduced persistence and compaction intervals
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7092/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62 ">+9/-7</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>loki.rs</strong><dd><code>Use constants for Loki stream
names</code>
</dd></summary>
<hr>
src/service/logs/loki.rs
<li>Import <code>STREAM_NAME_LABEL</code> and
<code>DEFAULT_STREAM_NAME</code><br> <li> Remove hardcoded
<code>stream_name</code> constant<br> <li> Use constants in stream name
function<br> <li> Update tests with <code>DEFAULT_STREAM_NAME</code>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7092/files#diff-a970c8c4785d4e4d878218615b17eec0cf4b4976121766b002c2b2f76371bfe6 ">+3/-5</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Miscellaneous</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>wal.rs</strong><dd><code>Remove redundant timeout
cast</code>
</dd></summary>
<hr>
src/service/promql/search/grpc/wal.rs
- Removed redundant `as u64` timeout cast
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7092/files#diff-f47892ffc5d3076d37a66f49f55db4e83cb6f42df517da68c001a42c04adac72 ">+1/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/ ">documentation</a>
for more information.</li></details>
2025-06-10 11:59:48 +08:00
Huaijin
bff7393ad8
fix: summary_percentile null value issue ( #7090 )
2025-06-10 11:35:17 +08:00
Jaydeep Maddipati
b3120ee7e1
feat: loki ingest api closes #7034 ( #7075 )
2025-06-10 11:02:43 +08:00
Subhra264 and Taiming Liu
0580c04870
fix: decode base64 auth_token cookie ( #7085 )
...
After sso login, we return the access_token and refresh_token as base64
encoded string through cookies. But when handling the `/dex_refresh`
endpoint, we don't decode the cookie, hence the dex_refresh does not
really work.
Co-authored-by: Taiming Liu <liutaiming3@gmail.com >
2025-06-09 06:34:32 -07:00
Subhra264
c4e1c6e363
fix: _around api rbac returns 403 error ( #7042 )
...
For rbac for `_around` api, the method needs to be converted into `GET`
method instead of `POST`.
2025-06-09 18:31:14 +05:30
Huaijin
d2178d82c2
feat: impl summary_percentile based on summary stream ( #7079 )
...
related to https://github.com/openobserve/o2-enterprise/issues/503
2025-06-09 19:53:17 +08:00
Ashish Kolhe
c70280bb9e
fix: use stats for partition api ( #7082 )
2025-06-09 15:01:57 +05:30
Ashish Kolhe
a13eef865b
fix: ai tool changes ( #7055 )
2025-06-06 20:49:43 +05:30
Yashodhan Joshi and Yashodhan Joshi
54a3fda236
feat: support hec format for ingestion ( #7043 )
...
closes #6996
This adds support for HEC data format in ingestion via `/org_id/_hec`
api path. See the issue for more details.
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com >
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com >
2025-06-06 20:00:20 +05:30
Loakesh Indiran
e8e5247b25
fix: disable streaming aggs for complex aggregates ( #7048 )
...
Disable streaming aggs for complex queries such as:
- [x] `has_union` only checked top-level queries, missing nested UNIONs
in subqueries
- [x] `has_subquery` only found expression subqueries (`WHERE col IN
(SELECT...)`) but missed table subqueries in FROM clauses (`FROM
(SELECT...) alias`)
- [x] window functions
- [x] CTE's (queries that have "WITH"): CTE's must complete the entire
query first before the main query can start
Distinct Queries with streaming aggs enabled can be suppirted
- [x] `has_distinct` needs to check conditions like `count(distinct
user_id)` as well
- [x] aggregate: distinct with group by
Other Fixes:
- [x] search partition api `streaming_output`
- [x] if search req has `track_total_hits` is `true` disable
`streaming_id`, `streaming_aggs`
---
## Summary:
| Complexity Factor | Why Can't Be Partitioned | Example |
|-------------------|-------------------|---------|
| **JOINs** | Need data from multiple tables | `FROM a JOIN b` |
| **Subqueries** | Dependencies between queries | `FROM (SELECT ...)` |
| **UNIONs** | Need all sub-results first | `SELECT ... UNION SELECT
...` |
| **Window Functions** | Need complete partitions | `SUM() OVER(...)` |
| **CTEs** | Must complete before main query | `WITH cte AS (...) SELECT
...` |
### Why we can support aggregate distinct with partitions?
1. The record batch for each partition are cached and distinct is
computed at the end.
---
Complex Aggregates:
```
---- utils::sql::tests::check_is_simple_aggregate_for_complex_queries_should_be_false stdout ----
Query [0]: Query with JOIN should not be simple - is_simple: false
Query [1]: Query with table subquery should not be simple - is_simple: false
Query [2]: Query with expression subquery should not be simple - is_simple: false
Query [3]: Query with UNION should not be simple - is_simple: false
Query [4]: Query with window functions should not be simple - is_simple: false
Query [5]: Query with multiple window functions should not be simple - is_simple: false
Query [6]: Query with subquery + union + window functions should not be simple - is_simple: false
Query [7]: Query with EXISTS subquery should not be simple - is_simple: false
Query [8]: Query with self-join should not be simple - is_simple: false
Query [9]: Query with nested UNION should not be simple - is_simple: false
Query [10]: Query with simple CTE should not be simple - is_simple: false
Query [11]: Query with multiple CTEs should not be simple - is_simple: false
Query [12]: Query with recursive CTE should not be simple - is_simple: false
Query [13]: Query with CTE containing window functions should not be simple - is_simple: false
```
---
Simple Aggregates:
```
---- utils::sql::tests::check_is_simple_aggregate_for_simple_queries_should_be_true stdout ----
Simple Query [0]: Simple COUNT query should be simple - is_simple: true
Simple Query [1]: Simple SUM with GROUP BY should be simple - is_simple: true
Simple Query [2]: Multiple aggregates with WHERE should be simple - is_simple: true
Simple Query [3]: Aggregate with GROUP BY and HAVING should be simple - is_simple: true
Simple Query [4]: Query with DISTINCT should be simple - is_simple: true
Simple Query [5]: Query with DISTINCT and aggregate should be simple - is_simple: true
Simple Query [6]: Query with DISTINCT on multiple columns should be simple - is_simple: true
Simple Query [7]: Query with COUNT(DISTINCT) should be simple - is_simple: true
Simple Query [8]: Query with DISTINCT and no group by should be simple - is_simple: true
```
---
### Window Function Example:
```sql
SELECT user_id, event_time,
ROW_NUMBER() OVER(PARTITION BY user_id ORDER BY event_time) as user_event_rank
FROM logs WHERE time BETWEEN '10:00' AND '11:00'
```
```text
Partition 1 (10:00-10:15):
user1: events at 10:05, 10:12
user2: events at 10:03, 10:08, 10:14
Partition 2 (10:15-10:30):
user1: events at 10:18, 10:25
user2: events at 10:20
```
**The Problem:**
- In Partition 1: `user1`'s first event gets `ROW_NUMBER = 1`
- In Partition 2: `user1` has more events, but they should be
`ROW_NUMBER = 3, 4`
- **Can't assign correct row numbers until ALL of user1's events are
seen!**
**Example Output (WRONG if streamed):**
```
-- After Partition 1 (WRONG!)
user1, 10:05, row_num=1
user1, 10:12, row_num=2
-- After Partition 2 (WRONG!)
user1, 10:18, row_num=1 ← Should be 3!
user1, 10:25, row_num=2 ← Should be 4!
```
**Correct Final Output:**
```
user1, 10:05, row_num=1
user1, 10:12, row_num=2
user1, 10:18, row_num=3
user1, 10:25, row_num=4
```
2025-06-06 19:32:29 +05:30
Taiming Liu and Subhra264
6b4b8f9099
fix: align_time allow frequency to be optional ( #7040 )
...
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com >
2025-06-06 06:28:15 -07:00
Hengfei Yang
94a0241e3f
ci: improve unit test coverage ( #7068 )
...
Now the coverage is:
✅ Functions coverage: 46.83%
✅ Lines coverage: 46.35%
✅ Regions coverage: 34.09%
2025-06-06 20:37:41 +08:00
udev and omkark06
2bcae056ac
fix: redirect short url ( #7026 ) ( #7033 )
...
with this POC we will let FE handle all short url redirections. We keep
the old routes intact for backwards compatibility
---------
---------
Co-authored-by: omkark06 <omkar@zinclabs.io >
2025-06-06 17:33:16 +05:30
Hengfei Yang
0310905787
fix: debug wal file lock issue ( #7066 )
...
There are some edge case `wal` file can't be released, then the `wal`
file can't be removed, it stays over 1 day.
2025-06-06 18:17:28 +08:00
udev and loaki07
3f7582e0db
fix: increase test case coverage ( #7054 )
...
Co-authored-by: loaki07 <rustyloaki@gmail.com >
2025-06-06 13:19:24 +05:30
Subhra264
340d18b5ea
fix: deserialize from member to admin ( #7047 )
...
`member` role should also be deserialized to the `admin` role.
2025-06-05 19:42:15 +08:00
Hengfei Yang
c657b6bbff
fix: parquet not found on ingester ( #7024 )
2025-06-04 17:55:19 +08:00
Huaijin and greptile-apps[bot]
c334ffbc87
fix: a corner case for join match one ( #7016 )
...
fix the below query
```sql
with abc as (select histogram(_timestamp, '1 minute') AS zo_sql_key, count(*) AS zo_sql_num from default GROUP BY zo_sql_key ORDER BY zo_sql_key), edf as (select histogram(_timestamp, '1 minute') AS zo_sql_key, sum(_timestamp) AS zo_sql_num from default GROUP BY zo_sql_key ORDER BY zo_sql_key) select abc.zo_sql_key from abc where zo_sql_key in (select zo_sql_key from edf)
```
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-04 11:51:48 +08:00
Taiming Liu
9eb21ceecd
fix: scheduled pipeline align query time instead of next_run_at ( #7007 )
2025-06-03 12:37:09 +08:00
Abhay Padamani
951f5f1d15
fix: trellis group by config save issue ( #7004 )
...
- #6997
2025-06-02 18:43:00 +05:30
udev
215ac530d6
fix: add check for non-zero size in search result truncation ( #7003 )
2025-06-02 18:02:20 +05:30
Hengfei Yang
578fb9a27b
fix: reset stream stats with limit ( #7001 )
2025-06-02 18:43:04 +08:00
Hengfei Yang
2cb55bb6b5
fix: reset stream stats with limit ( #6995 )
2025-06-02 16:01:43 +08:00
Sai Nikhil and oasisk
3c5ae41210
fix: add output format type for external destination(pipeline destination) ( #6993 )
...
1.This PR adds the output format to the external destination either json
/ ndjson , bydefault it is json
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com >
2025-06-02 11:59:49 +05:30
Hengfei Yang
4a60d6a0de
fix: reset stream stats with limit ( #6992 )
2025-06-02 13:05:59 +08:00
Taiming Liu and Subhra264
f18c098bdb
chore: porting pipeline fix to main ( #6986 )
...
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com >
2025-06-01 16:14:57 -07:00
Taiming Liu
5ca894aa2b
fix: pipeline next_run_at with delay ( #6872 )
2025-06-01 10:21:20 +08:00
udev and Hengfei Yang
ba6ba30d65
feat: add histogram metric for file access analysis ( #6982 )
...
This histogram metric should allow understanding the age of files
handled by querier.
Example queries
`histogram_quantile(0.95,
rate(zo_file_access_time_bucket{stream_type='logs'}[5m]))`
---------
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com >
2025-05-31 21:55:06 +08:00
Hengfei Yang
64e3d7d101
fix: update stream stats timeout ( #6984 )
2025-05-31 02:14:43 +08:00
Hengfei Yang
39a0deee33
perf: optimize stream stats calculate with limit ( #6981 )
...
This PR fixed the issue when compactor not working for a long time, it
will have millions files need to be update stream stats. it will take a
long time or time out.
2025-05-30 23:24:51 +08:00
Subhra264
0ab0ea1989
fix: convert frequency interval to seconds for reports to calculate next run at ( #6977 )
...
Fixes #6918
2025-05-30 21:21:04 +08:00
Subhra264 and sai nikhil kethe
197d4b6252
feat: return ai_enabled boolean in the config api ( #6971 )
...
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai >
2025-05-30 21:18:22 +08:00
Hengfei Yang
d9fa16fd7e
fix: file list dump with multiple account ( #6974 )
2025-05-30 18:49:37 +08:00
Patrick SullivanandSubhradeep Chakraborty
Hengfei Yang
Loakesh Indiranandudev
Huaijin
Abhay Padamani
Yashodhan JoshiandYashodhan Joshi
Maheep Kumar
Jaydeep Maddipati
Ashish Kolhe
Sai Nikhilandoasisk