mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-05-16 13:00:40 +00:00
18 lines
13 KiB
HTML
18 lines
13 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="API documentation for the Rust `pipeline` crate."><title>pipeline - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Italic-81dc35de.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-MediumItalic-ccf7e434.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2"href="../static.files/${f}">`).join(""))</script><link rel="stylesheet" href="../static.files/normalize-9960930a.css"><link rel="stylesheet" href="../static.files/rustdoc-e56847b5.css"><meta name="rustdoc-vars" data-root-path="../" data-static-root-path="../static.files/" data-current-crate="pipeline" data-themes="" data-resource-suffix="" data-rustdoc-version="1.92.0-nightly (fa3155a64 2025-09-30)" data-channel="nightly" data-search-js="search-e256b49e.js" data-stringdex-js="stringdex-828709d0.js" data-settings-js="settings-c38705f0.js" ><script src="../static.files/storage-e2aeef58.js"></script><script defer src="../crates.js"></script><script defer src="../static.files/main-ce535bd0.js"></script><noscript><link rel="stylesheet" href="../static.files/noscript-263c88ec.css"></noscript><link rel="alternate icon" type="image/png" href="../static.files/favicon-32x32-eab170b8.png"><link rel="icon" type="image/svg+xml" href="../static.files/favicon-044be391.svg"></head><body class="rustdoc mod crate"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><rustdoc-topbar><h2><a href="#">Crate pipeline</a></h2></rustdoc-topbar><nav class="sidebar"><div class="sidebar-crate"><h2><a href="../pipeline/index.html">pipeline</a><span class="version">1.0.0-rc.2</span></h2></div><div class="sidebar-elems"><ul class="block"><li><a id="all-types" href="all.html">All Items</a></li></ul><section id="rustdoc-toc"><h3><a href="#modules">Crate Items</a></h3><ul class="block"><li><a href="#modules" title="Modules">Modules</a></li><li><a href="#macros" title="Macros">Macros</a></li><li><a href="#structs" title="Structs">Structs</a></li><li><a href="#enums" title="Enums">Enums</a></li><li><a href="#constants" title="Constants">Constants</a></li><li><a href="#traits" title="Traits">Traits</a></li><li><a href="#functions" title="Functions">Functions</a></li><li><a href="#types" title="Type Aliases">Type Aliases</a></li></ul></section><div id="rustdoc-modnav"></div></div></nav><div class="sidebar-resizer" title="Drag to resize sidebar"></div><main><div class="width-limiter"><section id="main-content" class="content"><div class="main-heading"><h1>Crate <span>pipeline</span> <button id="copy-path" title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><span class="sub-heading"><a class="src" href="../src/pipeline/lib.rs.html#15-57">Source</a> </span></div><h2 id="modules" class="section-header">Modules<a href="#modules" class="anchor">§</a></h2><dl class="item-table"><dt><a class="mod" href="dispatcher/index.html" title="mod pipeline::dispatcher">dispatcher</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="mod" href="error/index.html" title="mod pipeline::error">error</a></dt><dt><a class="mod" href="etl/index.html" title="mod pipeline::etl">etl</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="mod" href="manager/index.html" title="mod pipeline::manager">manager</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="mod" href="metrics/index.html" title="mod pipeline::metrics">metrics</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="mod" href="pipeline_operator/index.html" title="mod pipeline::pipeline_operator">pipeline_<wbr>operator</a></dt><dt><a class="mod" href="table/index.html" title="mod pipeline::table">table</a></dt><dt><a class="mod" href="tablesuffix/index.html" title="mod pipeline::tablesuffix">tablesuffix</a><span title="Restricted Visibility"> 🔒</span> </dt><dt><a class="mod" href="util/index.html" title="mod pipeline::util">util</a></dt></dl><h2 id="macros" class="section-header">Macros<a href="#macros" class="anchor">§</a></h2><dl class="item-table"><dt><a class="macro" href="macro.setup_pipeline.html" title="macro pipeline::setup_pipeline">setup_<wbr>pipeline</a></dt><dd>This macro is test only, do not use it in production.
|
|
The schema_info cannot be used in auto-transform ts-infer mode for lacking the ts schema.</dd><dt><a class="macro" href="macro.unwrap_or_continue_if_err.html" title="macro pipeline::unwrap_or_continue_if_err">unwrap_<wbr>or_<wbr>continue_<wbr>if_<wbr>err</a></dt><dt><a class="macro" href="macro.yaml_map_get_str.html" title="macro pipeline::yaml_map_get_str">yaml_<wbr>map_<wbr>get_<wbr>str</a></dt><dd>Macro to extract a string value from a YAML map</dd></dl><h2 id="structs" class="section-header">Structs<a href="#structs" class="anchor">§</a></h2><dl class="item-table"><dt><a class="struct" href="struct.ContextOpt.html" title="struct pipeline::ContextOpt">Context<wbr>Opt</a></dt><dd>ContextOpt is a collection of options(including table options and pipeline options)
|
|
that should be extracted during the pipeline execution.</dd><dt><a class="struct" href="struct.ContextReq.html" title="struct pipeline::ContextReq">Context<wbr>Req</a></dt><dd>ContextReq is a collection of row insert requests with different options.
|
|
The default option is all empty.
|
|
Because options are set in query context, we have to split them into sequential calls
|
|
The key is a <a href="struct.ContextOpt.html" title="struct pipeline::ContextOpt"><code>ContextOpt</code></a> struct for strong type.
|
|
e.g:
|
|
{
|
|
“skip_wal=true,ttl=1d”: [RowInsertRequest],
|
|
“ttl=1d”: [RowInsertRequest],
|
|
}</dd><dt><a class="struct" href="struct.DispatchedTo.html" title="struct pipeline::DispatchedTo">Dispatched<wbr>To</a></dt><dd>Where the pipeline executed is dispatched to, with context information</dd><dt><a class="struct" href="struct.GreptimePipelineParams.html" title="struct pipeline::GreptimePipelineParams">Greptime<wbr>Pipeline<wbr>Params</a></dt><dd>Parameters that can be used to configure the greptime pipelines.</dd><dt><a class="struct" href="struct.GreptimeTransformer.html" title="struct pipeline::GreptimeTransformer">Greptime<wbr>Transformer</a></dt><dd>fields not in the columns will be discarded
|
|
to prevent automatic column creation in GreptimeDB</dd><dt><a class="struct" href="struct.Pipeline.html" title="struct pipeline::Pipeline">Pipeline</a></dt><dt><a class="struct" href="struct.PipelineContext.html" title="struct pipeline::PipelineContext">Pipeline<wbr>Context</a></dt><dt><a class="struct" href="struct.SchemaInfo.html" title="struct pipeline::SchemaInfo">Schema<wbr>Info</a></dt><dd>This is used to record the current state schema information and a sequential cache of field names.
|
|
As you traverse the user input JSON, this will change.
|
|
It will record a superset of all user input schemas.</dd><dt><a class="struct" href="struct.SelectInfo.html" title="struct pipeline::SelectInfo">Select<wbr>Info</a></dt><dd>SelectInfo is used to store the selected keys from OpenTelemetry record attrs
|
|
The key is used to uplift value from the attributes and serve as column name in the table</dd><dt><a class="struct" href="struct.TransformedOutput.html" title="struct pipeline::TransformedOutput">Transformed<wbr>Output</a></dt><dd>Output from a successful pipeline transformation.</dd></dl><h2 id="enums" class="section-header">Enums<a href="#enums" class="anchor">§</a></h2><dl class="item-table"><dt><a class="enum" href="enum.Content.html" title="enum pipeline::Content">Content</a></dt><dt><a class="enum" href="enum.IdentityTimeIndex.html" title="enum pipeline::IdentityTimeIndex">Identity<wbr>Time<wbr>Index</a></dt><dt><a class="enum" href="enum.PipelineDefinition.html" title="enum pipeline::PipelineDefinition">Pipeline<wbr>Definition</a></dt><dd>Enum for holding information of a pipeline, which is either pipeline itself,
|
|
or information that be used to retrieve a pipeline from <code>PipelineHandler</code></dd><dt><a class="enum" href="enum.PipelineExecOutput.html" title="enum pipeline::PipelineExecOutput">Pipeline<wbr>Exec<wbr>Output</a></dt><dd>The result of pipeline execution</dd><dt><a class="enum" href="enum.PipelineWay.html" title="enum pipeline::PipelineWay">Pipeline<wbr>Way</a></dt><dt><a class="enum" href="enum.TransformerMode.html" title="enum pipeline::TransformerMode">Transformer<wbr>Mode</a></dt></dl><h2 id="constants" class="section-header">Constants<a href="#constants" class="anchor">§</a></h2><dl class="item-table"><dt><a class="constant" href="constant.GREPTIME_INTERNAL_IDENTITY_PIPELINE_NAME.html" title="constant pipeline::GREPTIME_INTERNAL_IDENTITY_PIPELINE_NAME">GREPTIME_<wbr>INTERNAL_<wbr>IDENTITY_<wbr>PIPELINE_<wbr>NAME</a></dt><dt><a class="constant" href="constant.GREPTIME_INTERNAL_TRACE_PIPELINE_V1_NAME.html" title="constant pipeline::GREPTIME_INTERNAL_TRACE_PIPELINE_V1_NAME">GREPTIME_<wbr>INTERNAL_<wbr>TRACE_<wbr>PIPELINE_<wbr>V1_<wbr>NAME</a></dt></dl><h2 id="traits" class="section-header">Traits<a href="#traits" class="anchor">§</a></h2><dl class="item-table"><dt><a class="trait" href="trait.Processor.html" title="trait pipeline::Processor">Processor</a></dt><dd>Processor trait defines the interface for all processors.</dd></dl><h2 id="functions" class="section-header">Functions<a href="#functions" class="anchor">§</a></h2><dl class="item-table"><dt><a class="fn" href="fn.identity_pipeline.html" title="fn pipeline::identity_pipeline">identity_<wbr>pipeline</a></dt><dd>Identity pipeline for Greptime
|
|
This pipeline will convert the input JSON array to Greptime Rows
|
|
params table is used to set the semantic type of the row key column to Tag</dd><dt><a class="fn" href="fn.parse.html" title="fn pipeline::parse">parse</a></dt><dt><a class="fn" href="fn.truthy.html" title="fn pipeline::truthy">truthy</a></dt></dl><h2 id="types" class="section-header">Type Aliases<a href="#types" class="anchor">§</a></h2><dl class="item-table"><dt><a class="type" href="type.PipelineInfo.html" title="type pipeline::PipelineInfo">Pipeline<wbr>Info</a></dt><dd>Pipeline info. A tuple of timestamp and pipeline reference.</dd><dt><a class="type" href="type.PipelineRef.html" title="type pipeline::PipelineRef">Pipeline<wbr>Ref</a></dt><dt><a class="type" href="type.PipelineTableRef.html" title="type pipeline::PipelineTableRef">Pipeline<wbr>Table<wbr>Ref</a></dt><dt><a class="type" href="type.PipelineVersion.html" title="type pipeline::PipelineVersion">Pipeline<wbr>Version</a></dt><dd>Pipeline version. An optional timestamp with nanosecond precision.</dd></dl><script type="text/json" id="notable-traits-data">{"<Arc<S> as Service<Request>>::Future":"<h3>Notable traits for <code><a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></code></h3><pre><code><div class=\"where\">impl <a class=\"trait\" href=\"https://doc.rust-lang.org/nightly/std/io/trait.Read.html\" title=\"trait std::io::Read\">Read</a> for <a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></div><div class=\"where\">impl <a class=\"trait\" href=\"https://doc.rust-lang.org/nightly/std/io/trait.Write.html\" title=\"trait std::io::Write\">Write</a> for <a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></div>","<Arc<W> as MakeWriter<'a>>::Writer":"<h3>Notable traits for <code><a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></code></h3><pre><code><div class=\"where\">impl <a class=\"trait\" href=\"https://doc.rust-lang.org/nightly/std/io/trait.Read.html\" title=\"trait std::io::Read\">Read</a> for <a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></div><div class=\"where\">impl <a class=\"trait\" href=\"https://doc.rust-lang.org/nightly/std/io/trait.Write.html\" title=\"trait std::io::Write\">Write</a> for <a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/alloc/sync/struct.Arc.html\" title=\"struct alloc::sync::Arc\">Arc</a><<a class=\"struct\" href=\"https://doc.rust-lang.org/nightly/std/fs/struct.File.html\" title=\"struct std::fs::File\">File</a>></div>"}</script></section></div></main></body></html> |