mirror of https://github.com/openobserve/openobserve.git synced 2026-11-04 00:08:55 +00:00

Files

T

Hengfei Yang 42e0d0448e fix: skip caching files below record threshold (#13297 )

## Summary

- add `ZO_FILE_DOWNLOAD_MIN_RECORDS` with a default of 100 records
- skip undersized files before latest-file batch downloads and
background queue insertion
- apply the threshold consistently to Parquet and index files
- preserve caching when `records == 0`, which can represent missing
metadata from older gRPC senders

## Why

Downloading and caching very small files consumes queue capacity and
cache space without providing enough reuse benefit. Filtering them
before enqueueing avoids unnecessary download work.

## Impact

Files with a known positive record count below the configured threshold
are read from object storage without being added to the download cache.
A zero count is treated as unknown and remains cache-eligible; files at
or above the threshold keep the existing behavior.

## Validation

- `cargo fmt --all -- --check`
- focused minimum-record boundary test
- `cargo check -p search -p openobserve-core -p openobserve-api`
- `git diff --check`

Fixes #13288

2026-07-20 13:17:09 +00:00

src

fix: skip caching files below record threshold (#13297 )

2026-07-20 13:17:09 +00:00

build.rs

refactor: extract application core crate (#13168 )

2026-07-14 09:49:48 +00:00

Cargo.toml

refactor: extract shared common crate (#13252 )

2026-07-17 13:58:24 +00:00

README.md

refactor: extract shared common crate (#13252 )

2026-07-17 13:58:24 +00:00

README.md

openobserve-core

openobserve-core contains the application services and compatibility re-exports used by OpenObserve frontends and background jobs.

Its main modules are:

common: compatibility namespace backed by the common crate.
service: business services used by APIs and jobs.
cipher: enterprise-only encryption support.

This is an internal workspace crate and is not published independently. Transport-specific routing and background-job orchestration should remain in their respective crates instead of being added here.