mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-13 07:22:32 +00:00
6 lines
13 KiB
HTML
6 lines
13 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="Memtable implementation for bulk load"><title>mito2::memtable::bulk - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Italic-81dc35de.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-MediumItalic-ccf7e434.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2"href="../../../static.files/${f}">`).join(""))</script><link rel="stylesheet" href="../../../static.files/normalize-9960930a.css"><link rel="stylesheet" href="../../../static.files/rustdoc-1ed95124.css"><meta name="rustdoc-vars" data-root-path="../../../" data-static-root-path="../../../static.files/" data-current-crate="mito2" data-themes="" data-resource-suffix="" data-rustdoc-version="1.98.1 (48a229cea 2026-09-01)" data-channel="1.98.1" data-search-js="search-9944d736.js" data-stringdex-js="stringdex-2da4960a.js" data-settings-js="settings-170eb4bf.js" ><script src="../../../static.files/storage-41dd4d93.js"></script><script defer src="../sidebar-items.js"></script><script defer src="../../../static.files/main-417ebf0a.js"></script><noscript><link rel="stylesheet" href="../../../static.files/noscript-f7c3ffd8.css"></noscript><link rel="alternate icon" type="image/png" href="../../../static.files/favicon-32x32-eab170b8.png"><link rel="icon" type="image/svg+xml" href="../../../static.files/favicon-044be391.svg"></head><body class="rustdoc mod"><a class="skip-main-content" href="#main-content">Skip to main content</a><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><rustdoc-topbar><h2><a href="#">Module bulk</a></h2></rustdoc-topbar><nav class="sidebar"><div class="sidebar-crate"><h2><a href="../../../mito2/index.html">mito2</a><span class="version">1.4.0-alpha.0</span></h2></div><div class="sidebar-elems"><section id="rustdoc-toc"><h2 class="location"><a href="#">Module bulk</a></h2><h3><a href="#modules">Module Items</a></h3><ul class="block"><li><a href="#modules" title="Modules">Modules</a></li><li><a href="#structs" title="Structs">Structs</a></li><li><a href="#enums" title="Enums">Enums</a></li><li><a href="#constants" title="Constants">Constants</a></li><li><a href="#statics" title="Statics">Statics</a></li><li><a href="#functions" title="Functions">Functions</a></li></ul></section><div id="rustdoc-modnav"><h2><a href="../index.html">In mito2::<wbr>memtable</a></h2></div></div></nav><div class="sidebar-resizer" title="Drag to resize sidebar"></div><main><div class="width-limiter"><section id="main-content" class="content" tabindex="-1"><div class="main-heading"><div class="rustdoc-breadcrumbs"><a href="../../index.html">mito2</a>::<wbr><a href="../index.html">memtable</a></div><h1>Module <span>bulk</span> <button id="copy-path" title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><span class="sub-heading"><a class="src" href="../../../src/mito2/memtable/bulk.rs.html#15-3222">Source</a> </span></div><details class="toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>Memtable implementation for bulk load</p>
|
||
</div></details><h2 id="modules" class="section-header">Modules<a href="#modules" class="anchor">§</a></h2><dl class="item-table"><dt><a class="mod" href="chunk_reader/index.html" title="mod mito2::memtable::bulk::chunk_reader">chunk_<wbr>reader</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>ChunkReader implementation for in-memory parquet bytes.</dd><dt><a class="mod" href="context/index.html" title="mod mito2::memtable::bulk::context">context</a></dt><dd>Context for iterating bulk memtable.</dd><dt><a class="mod" href="part/index.html" title="mod mito2::memtable::bulk::part">part</a></dt><dd>Bulk part encoder/decoder.</dd><dt><a class="mod" href="part_reader/index.html" title="mod mito2::memtable::bulk::part_reader">part_<wbr>reader</a></dt><dt><a class="mod" href="row_group_reader/index.html" title="mod mito2::memtable::bulk::row_group_reader">row_<wbr>group_<wbr>reader</a><span title="Restricted Visibility"> 🔒</span> </dt></dl><h2 id="structs" class="section-header">Structs<a href="#structs" class="anchor">§</a></h2><dl class="item-table"><dt><a class="struct" href="struct.BulkMemtable.html" title="struct mito2::memtable::bulk::BulkMemtable">Bulk<wbr>Memtable</a></dt><dd>Memtable that ingests and scans parts directly.</dd><dt><a class="struct" href="struct.BulkMemtableBuilder.html" title="struct mito2::memtable::bulk::BulkMemtableBuilder">Bulk<wbr>Memtable<wbr>Builder</a></dt><dd>Builder to build a <a href="struct.BulkMemtable.html" title="struct mito2::memtable::bulk::BulkMemtable">BulkMemtable</a>.</dd><dt><a class="struct" href="struct.BulkMemtableConfig.html" title="struct mito2::memtable::bulk::BulkMemtableConfig">Bulk<wbr>Memtable<wbr>Config</a></dt><dd>Configuration for bulk memtable.</dd><dt><a class="struct" href="struct.BulkPartWrapper.html" title="struct mito2::memtable::bulk::BulkPartWrapper">Bulk<wbr>Part<wbr>Wrapper</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="struct" href="struct.BulkParts.html" title="struct mito2::memtable::bulk::BulkParts">Bulk<wbr>Parts</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>All parts in a bulk memtable.</dd><dt><a class="struct" href="struct.BulkRangeIterBuilder.html" title="struct mito2::memtable::bulk::BulkRangeIterBuilder">Bulk<wbr>Range<wbr>Iter<wbr>Builder</a></dt><dd>Iterator builder for bulk range</dd><dt><a class="struct" href="struct.CollectedParts.html" title="struct mito2::memtable::bulk::CollectedParts">Collected<wbr>Parts</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Result of collecting parts to merge</dd><dt><a class="struct" href="struct.CompactDispatcher.html" title="struct mito2::memtable::bulk::CompactDispatcher">Compact<wbr>Dispatcher</a></dt><dd>Scheduler to run compact tasks in background.</dd><dt><a class="struct" href="struct.CompactTaskGuard.html" title="struct mito2::memtable::bulk::CompactTaskGuard">Compact<wbr>Task<wbr>Guard</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Keeps the compaction flag set while queued or running and clears it on drop,
|
||
including failures and cancellation while waiting for a dispatcher permit.</dd><dt><a class="struct" href="struct.EncodedBulkRangeIterBuilder.html" title="struct mito2::memtable::bulk::EncodedBulkRangeIterBuilder">Encoded<wbr>Bulk<wbr>Range<wbr>Iter<wbr>Builder</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Iterator builder for encoded bulk range</dd><dt><a class="struct" href="struct.MemCompactTask.html" title="struct mito2::memtable::bulk::MemCompactTask">MemCompact<wbr>Task</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>A memtable compact task to run in background.</dd><dt><a class="struct" href="struct.MemtableCompactor.html" title="struct mito2::memtable::bulk::MemtableCompactor">Memtable<wbr>Compactor</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="struct" href="struct.MergingFlagsGuard.html" title="struct mito2::memtable::bulk::MergingFlagsGuard">Merging<wbr>Flags<wbr>Guard</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>RAII guard for managing merging flags.
|
||
Automatically resets merging flags when dropped if the merge operation wasn’t successful.</dd><dt><a class="struct" href="struct.MultiBulkRangeIterBuilder.html" title="struct mito2::memtable::bulk::MultiBulkRangeIterBuilder">Multi<wbr>Bulk<wbr>Range<wbr>Iter<wbr>Builder</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Iterator builder for multi bulk range</dd></dl><h2 id="enums" class="section-header">Enums<a href="#enums" class="anchor">§</a></h2><dl class="item-table"><dt><a class="enum" href="enum.MergedPart.html" title="enum mito2::memtable::bulk::MergedPart">Merged<wbr>Part</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Result of merging parts - either a MultiBulkPart or an EncodedBulkPart</dd><dt><a class="enum" href="enum.PartToMerge.html" title="enum mito2::memtable::bulk::PartToMerge">Part<wbr>ToMerge</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Enum to wrap different types of parts for unified merging.</dd></dl><h2 id="constants" class="section-header">Constants<a href="#constants" class="anchor">§</a></h2><dl class="item-table"><dt><a class="constant" href="constant.DEFAULT_ENCODE_BYTES_THRESHOLD.html" title="constant mito2::memtable::bulk::DEFAULT_ENCODE_BYTES_THRESHOLD">DEFAULT_<wbr>ENCODE_<wbr>BYTES_<wbr>THRESHOLD</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Default bytes threshold for encoding.</dd><dt><a class="constant" href="constant.DEFAULT_MAX_MERGE_GROUPS.html" title="constant mito2::memtable::bulk::DEFAULT_MAX_MERGE_GROUPS">DEFAULT_<wbr>MAX_<wbr>MERGE_<wbr>GROUPS</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Default maximum number of groups for parallel merging.</dd><dt><a class="constant" href="constant.DEFAULT_MERGE_THRESHOLD.html" title="constant mito2::memtable::bulk::DEFAULT_MERGE_THRESHOLD">DEFAULT_<wbr>MERGE_<wbr>THRESHOLD</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Default merge threshold for triggering compaction.</dd><dt><a class="constant" href="constant.ENCODE_BYTES_THRESHOLD_DIVISOR.html" title="constant mito2::memtable::bulk::ENCODE_BYTES_THRESHOLD_DIVISOR">ENCODE_<wbr>BYTES_<wbr>THRESHOLD_<wbr>DIVISOR</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Divisor to derive the encode bytes threshold from the global write buffer size.</dd><dt><a class="constant" href="constant.MAX_ENCODE_BYTES_THRESHOLD.html" title="constant mito2::memtable::bulk::MAX_ENCODE_BYTES_THRESHOLD">MAX_<wbr>ENCODE_<wbr>BYTES_<wbr>THRESHOLD</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Maximum bytes threshold for encoding when adapting to the write buffer size.</dd></dl><h2 id="statics" class="section-header">Statics<a href="#statics" class="anchor">§</a></h2><dl class="item-table"><dt><a class="static" href="static.ENCODE_BYTES_THRESHOLD_OVERRIDE.html" title="static mito2::memtable::bulk::ENCODE_BYTES_THRESHOLD_OVERRIDE">ENCODE_<wbr>BYTES_<wbr>THRESHOLD_<wbr>OVERRIDE</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Optional bytes threshold override from <code>GREPTIME_BULK_ENCODE_BYTES_THRESHOLD</code>.</dd><dt><a class="static" href="static.ENCODE_ROW_THRESHOLD.html" title="static mito2::memtable::bulk::ENCODE_ROW_THRESHOLD">ENCODE_<wbr>ROW_<wbr>THRESHOLD</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Row threshold for encoding parts. Configurable via <code>GREPTIME_BULK_ENCODE_ROW_THRESHOLD</code>.
|
||
When estimated rows exceed this threshold, parts are encoded as EncodedBulkPart.</dd><dt><a class="static" href="static.MAX_MERGE_GROUPS.html" title="static mito2::memtable::bulk::MAX_MERGE_GROUPS">MAX_<wbr>MERGE_<wbr>GROUPS</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Maximum merge groups. Configurable via <code>GREPTIME_BULK_MAX_MERGE_GROUPS</code>.</dd><dt><a class="static" href="static.MERGE_THRESHOLD.html" title="static mito2::memtable::bulk::MERGE_THRESHOLD">MERGE_<wbr>THRESHOLD</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Threshold for triggering merge of parts. Configurable via <code>GREPTIME_BULK_MERGE_THRESHOLD</code>.</dd></dl><h2 id="functions" class="section-header">Functions<a href="#functions" class="anchor">§</a></h2><dl class="item-table"><dt><a class="fn" href="fn.adaptive_encode_bytes_threshold.html" title="fn mito2::memtable::bulk::adaptive_encode_bytes_threshold">adaptive_<wbr>encode_<wbr>bytes_<wbr>threshold</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Computes the encode bytes threshold adapted to the global write buffer size:
|
||
<code>max(64 MiB, min(global_write_buffer_size / 32, 512 MiB))</code>.</dd><dt><a class="fn" href="fn.default_encode_bytes_threshold.html" title="fn mito2::memtable::bulk::default_encode_bytes_threshold">default_<wbr>encode_<wbr>bytes_<wbr>threshold</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Returns the default bytes threshold for encoding parts.</dd><dt><a class="fn" href="fn.env_usize.html" title="fn mito2::memtable::bulk::env_usize">env_<wbr>usize</a><span title="Restricted Visibility"> 🔒</span> </dt><dd>Reads an environment variable as usize, returning default if not set or invalid.</dd></dl></section></div></main></body></html> |