Fix nightly fmt: merge crate imports in percentiles tests

Co-authored-by: Cursor <cursoragent@cursor.com>
Restore use_serde feature and simplify PercentilesCollector
2026-02-20 14:50:38 +00:00 · 2026-02-19 14:18:54 -05:00 · 2026-02-19 14:13:17 -05:00 · 2026-02-19 13:32:28 -05:00 · 2026-02-19 13:21:48 -05:00 · 2026-02-19 12:22:19 -05:00
2 changed files with 24 additions and 8 deletions
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -64,7 +64,7 @@ query-grammar = { version = "0.25.0", path = "./query-grammar", package = "tanti
 tantivy-bitpacker = { version = "0.9", path = "./bitpacker" }
 common = { version = "0.10", path = "./common/", package = "tantivy-common" }
 tokenizer-api = { version = "0.6", path = "./tokenizer-api", package = "tantivy-tokenizer-api" }
-sketches-ddsketch = { version = "0.3.0", features = ["use_serde"] }
+sketches-ddsketch = { git = "https://github.com/quickwit-oss/rust-sketches-ddsketch.git", rev = "555caf1", features = ["use_serde"] }
 datasketches = "0.2.0"
 futures-util = { version = "0.3.28", optional = true }
 futures-channel = { version = "0.3.28", optional = true }
--- a/src/aggregation/metric/percentiles.rs
+++ b/src/aggregation/metric/percentiles.rs
@@ -222,6 +222,12 @@ impl PercentilesCollector {
        self.sketch.add(val);
    }

+    /// Encode the underlying DDSketch to Java-compatible binary format
+    /// for cross-language serialization with Java consumers.
+    pub fn to_sketch_bytes(&self) -> Vec<u8> {
+        self.sketch.to_java_bytes()
+    }
+
    pub(crate) fn merge_fruits(&mut self, right: PercentilesCollector) -> crate::Result<()> {
        self.sketch.merge(&right.sketch).map_err(|err| {
            TantivyError::AggregationError(AggregationError::InternalError(format!(
@@ -325,7 +331,7 @@ mod tests {
    use crate::aggregation::AggregationCollector;
    use crate::query::AllQuery;
    use crate::schema::{Schema, FAST};
-    use crate::Index;
+    use crate::{assert_nearly_equals, Index};

    #[test]
    fn test_aggregation_percentiles_empty_index() -> crate::Result<()> {
@@ -608,12 +614,16 @@ mod tests {
        let res = exec_request_with_query(agg_req, &index, None)?;
        assert_eq!(res["range_with_stats"]["buckets"][0]["doc_count"], 3);

-        assert_eq!(
-            res["range_with_stats"]["buckets"][0]["percentiles"]["values"]["1.0"],
+        assert_nearly_equals!(
+            res["range_with_stats"]["buckets"][0]["percentiles"]["values"]["1.0"]
+                .as_f64()
+                .unwrap(),
            5.0028295751107414
        );
-        assert_eq!(
-            res["range_with_stats"]["buckets"][0]["percentiles"]["values"]["99.0"],
+        assert_nearly_equals!(
+            res["range_with_stats"]["buckets"][0]["percentiles"]["values"]["99.0"]
+                .as_f64()
+                .unwrap(),
            10.07469668951144
        );

@@ -659,8 +669,14 @@ mod tests {

        let res = exec_request_with_query(agg_req, &index, None)?;

-        assert_eq!(res["percentiles"]["values"]["1.0"], 5.0028295751107414);
-        assert_eq!(res["percentiles"]["values"]["99.0"], 10.07469668951144);
+        assert_nearly_equals!(
+            res["percentiles"]["values"]["1.0"].as_f64().unwrap(),
+            5.0028295751107414
+        );
+        assert_nearly_equals!(
+            res["percentiles"]["values"]["99.0"].as_f64().unwrap(),
+            10.07469668951144
+        );

        Ok(())
    }
Author	SHA1	Message	Date
cong.xie	18fedd9384	Fix nightly fmt: merge crate imports in percentiles tests Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-19 14:18:54 -05:00
cong.xie	2098fca47f	Restore use_serde feature and simplify PercentilesCollector Keep use_serde on sketches-ddsketch so DDSketch derives Serialize/Deserialize, removing the need for custom impls on PercentilesCollector. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-19 14:13:17 -05:00
cong.xie	1251b40c93	Drop use_serde feature; use Java binary encoding for PercentilesCollector Replace the derived Serialize/Deserialize on PercentilesCollector with custom impls that use DDSketch's Java-compatible binary encoding (encode_to_java_bytes / decode_from_java_bytes). This removes the need for the use_serde feature on sketches-ddsketch entirely. Also restore original float test values and use assert_nearly_equals! for all float comparisons in percentile tests, since DDSketch quantile estimates can have minor precision differences across platforms. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-19 13:32:28 -05:00
cong.xie	09a49b872c	Use assert_nearly_equals! for float comparisons in percentile test Address review feedback: replace assert_eq! with assert_nearly_equals! for float values that go through JSON serialization roundtrips, which can introduce minor precision differences. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-19 13:21:48 -05:00
cong.xie	b9ace002ce	Replace vendored sketches-ddsketch with git dependency Move the vendored sketches-ddsketch crate (with Java-compatible binary encoding) to its own repo at quickwit-oss/rust-sketches-ddsketch and reference it via git+rev in Cargo.toml. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-19 12:22:19 -05:00
cong.xie	2dc4e9ef78	fix: resolve remaining clippy errors in ddsketch - Replace approximate PI/E constants with non-famous value in test - Fix reversed empty range (2048..0) → (0..2048).rev() in store test Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 15:54:27 -05:00
cong.xie	aeea65f61d	refactor: rewrite encoding.rs with idiomatic Rust - Replace bare constants with FlagType and BinEncodingMode enums - Use const fn for flag byte construction instead of raw bit ops - Replace if-else chain with nested match in decode_from_java_bytes - Use split_first() in read_byte for idiomatic slice consumption - Use split_at in read_f64_le to avoid TryInto on edition 2018 - Use u64::from(next) instead of `next as u64` casts - Extract assert_golden, assert_quantiles_match, bytes_to_hex helpers to reduce duplication across golden byte tests - Fix edition-2018 assert! format string compatibility - Clean up is_valid_flag_byte with let-else and match Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 15:49:12 -05:00
cong.xie	4211d5a1ed	fix: resolve clippy warnings in vendored sketches-ddsketch - manual_range_contains: use !(0.0..=1.0).contains(&q) - identity_op: simplify (0 << 2) \| FLAG_TYPE to just FLAG_TYPE - manual_clamp: use .clamp(0, 8) instead of .max(0).min(8) - manual_repeat_n: use repeat_n() instead of repeat().take() - cast_abs_to_unsigned: use .unsigned_abs() instead of .abs() as usize Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 13:36:06 -05:00
cong.xie	d50c7a1daf	Add Java source links for cross-language alignment comments Reference the exact Java source files in DataDog/sketches-java for Config::new(), Config::key(), Config::value(), Config::from_gamma(), and Store::add_count() so readers can verify the alignment. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 13:25:12 -05:00
cong.xie	cf760fd5b6	fix: remove internal reference from code comment Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 12:59:25 -05:00
cong.xie	df04c7d8f1	fix: rustfmt nightly formatting for vendored sketches-ddsketch Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 12:53:01 -05:00
cong.xie	68626bf3a1	Vendor sketches-ddsketch with Java-compatible binary encoding Fork sketches-ddsketch as a workspace member to add native Java binary serialization (to_java_bytes/from_java_bytes) for DDSketch. This enables pomsky to return raw DDSketch bytes that event-query can deserialize via DDSketchWithExactSummaryStatistics.decode(). Key changes: - Vendor sketches-ddsketch crate with encoding.rs implementing VarEncoding, flag bytes, and INDEX_DELTAS_AND_COUNTS store format - Align Config::key() to floor-based indexing matching Java's LogarithmicMapping - Add PercentilesCollector::to_sketch_bytes() for pomsky integration - Cross-language golden byte tests verified byte-identical with Java output Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-18 11:36:21 -05:00