chore: Merge branch 'main' into chore/bench-metrics

feat: bump opendal and switch prometheus layer to the upstream impl (#5179 )
* feat: bump opendal and switch prometheus layer to the upstream impl Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove unused files Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix tests Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove unused things Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove root dir on recovering cache Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * filter out non-files entry in test Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-12-25 15:40:02 +00:00 · 2024-12-19 16:07:43 +08:00 · 2024-12-19 03:42:05 +00:00 · 2024-12-19 03:29:34 +00:00 · 2024-12-18 15:15:55 +00:00 · 2024-12-18 12:41:24 +00:00
73 changed files with 420 additions and 2233 deletions
--- a/.github/pull_request_template.md
+++ b/.github/pull_request_template.md
@@ -4,8 +4,7 @@ I hereby agree to the terms of the [GreptimeDB CLA](https://github.com/GreptimeT

 ## What's changed and what's your intention?

-<!--    
- __!!! DO NOT LEAVE THIS BLOCK EMPTY !!!__
+__!!! DO NOT LEAVE THIS BLOCK EMPTY !!!__

 Please explain IN DETAIL what the changes are in this PR and why they are needed:

@@ -13,14 +12,9 @@ Please explain IN DETAIL what the changes are in this PR and why they are needed
 - How does this PR work? Need a brief introduction for the changed logic (optional)
 - Describe clearly one logical change and avoid lazy messages (optional)
 - Describe any limitations of the current code (optional)
- Describe if this PR will break **API or data compatibility**  (optional)
-->

-## PR Checklist
-Please convert it to a draft if some of the following conditions are not met.
+## Checklist

 - [ ] I have written the necessary rustdoc comments.
 - [ ] I have added the necessary unit tests and integration tests.
 - [ ] This PR requires documentation updates.
- [ ] API changes are backward compatible.
- [ ] Schema or data changes are backward compatible.
--- a/.github/workflows/dev-build.yml
+++ b/.github/workflows/dev-build.yml
@@ -29,7 +29,7 @@ on:
      linux_arm64_runner:
        type: choice
        description: The runner uses to build linux-arm64 artifacts
-        default: ec2-c6g.8xlarge-arm64
+        default: ec2-c6g.4xlarge-arm64
        options:
          - ec2-c6g.xlarge-arm64 # 4C8G
          - ec2-c6g.2xlarge-arm64 # 8C16G
--- a/.github/workflows/nightly-build.yml
+++ b/.github/workflows/nightly-build.yml
@@ -27,7 +27,7 @@ on:
      linux_arm64_runner:
        type: choice
        description: The runner uses to build linux-arm64 artifacts
-        default: ec2-c6g.8xlarge-arm64
+        default: ec2-c6g.4xlarge-arm64
        options:
          - ec2-c6g.xlarge-arm64 # 4C8G
          - ec2-c6g.2xlarge-arm64 # 8C16G
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -31,7 +31,7 @@ on:
      linux_arm64_runner:
        type: choice
        description: The runner uses to build linux-arm64 artifacts
-        default: ec2-c6g.8xlarge-arm64
+        default: ec2-c6g.4xlarge-arm64
        options:
          - ubuntu-2204-32-cores-arm
          - ec2-c6g.xlarge-arm64 # 4C8G
@@ -91,7 +91,7 @@ env:
  # The scheduled version is '${{ env.NEXT_RELEASE_VERSION }}-nightly-YYYYMMDD', like v0.2.0-nigthly-20230313;
  NIGHTLY_RELEASE_PREFIX: nightly
  # Note: The NEXT_RELEASE_VERSION should be modified manually by every formal release.
-  NEXT_RELEASE_VERSION: v0.11.0
+  NEXT_RELEASE_VERSION: v0.12.0

 # Permission reference: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
 permissions:
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -188,7 +188,7 @@ checksum = "d301b3b94cb4b2f23d7917810addbbaff90738e0ca2be692bd027e70d7e0330c"

 [[package]]
 name = "api"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-base",
 "common-decimal",
@@ -730,36 +730,6 @@ version = "1.1.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"

-[[package]]
-name = "attribute-derive"
-version = "0.10.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f1800e974930e9079c965b9ffbcb6667a40401063a26396c7b4f15edc92da690"
-dependencies = [
- "attribute-derive-macro",
- "derive-where",
- "manyhow",
- "proc-macro2",
- "quote",
- "syn 2.0.90",
-]
-
-[[package]]
-name = "attribute-derive-macro"
-version = "0.10.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5d908eb786ef94296bff86f90130b3b748b49401dc81fd2bb8b3dccd44cfacbd"
-dependencies = [
- "collection_literals",
- "interpolator",
- "manyhow",
- "proc-macro-utils",
- "proc-macro2",
- "quote",
- "quote-use",
- "syn 2.0.90",
-]
-
 [[package]]
 name = "atty"
 version = "0.2.14"
@@ -773,7 +743,7 @@ dependencies = [

 [[package]]
 name = "auth"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -896,18 +866,6 @@ dependencies = [
 "rand",
 ]

-[[package]]
-name = "backon"
-version = "0.4.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d67782c3f868daa71d3533538e98a8e13713231969def7536e8039606fc46bf0"
-dependencies = [
- "fastrand",
- "futures-core",
- "pin-project",
- "tokio",
-]
-
 [[package]]
 name = "backon"
 version = "1.2.0"
@@ -1326,7 +1284,7 @@ dependencies = [

 [[package]]
 name = "cache"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "catalog",
 "common-error",
@@ -1360,7 +1318,7 @@ checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5"

 [[package]]
 name = "catalog"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arrow",
@@ -1696,7 +1654,7 @@ checksum = "1462739cb27611015575c0c11df5df7601141071f07518d56fcc1be504cbec97"

 [[package]]
 name = "cli"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "auth",
@@ -1739,7 +1697,7 @@ dependencies = [
 "session",
 "snafu 0.8.5",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tempfile",
 "tokio",
@@ -1748,7 +1706,7 @@ dependencies = [

 [[package]]
 name = "client"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arc-swap",
@@ -1775,7 +1733,7 @@ dependencies = [
 "rand",
 "serde_json",
 "snafu 0.8.5",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "substrait 0.37.3",
 "tokio",
 "tokio-stream",
@@ -1816,7 +1774,7 @@ dependencies = [

 [[package]]
 name = "cmd"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "auth",
@@ -1876,7 +1834,7 @@ dependencies = [
 "similar-asserts",
 "snafu 0.8.5",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "temp-env",
 "tempfile",
@@ -1887,12 +1845,6 @@ dependencies = [
 "tracing-appender",
 ]

-[[package]]
-name = "collection_literals"
-version = "1.0.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "186dce98367766de751c42c4f03970fc60fc012296e706ccbb9d5df9b6c1e271"
-
 [[package]]
 name = "colorchoice"
 version = "1.0.2"
@@ -1928,7 +1880,7 @@ checksum = "55b672471b4e9f9e95499ea597ff64941a309b2cdbffcc46f2cc5e2d971fd335"

 [[package]]
 name = "common-base"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "anymap2",
 "async-trait",
@@ -1950,11 +1902,11 @@ dependencies = [

 [[package]]
 name = "common-catalog"
-version = "0.11.1"
+version = "0.12.0"

 [[package]]
 name = "common-config"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-base",
 "common-error",
@@ -1977,7 +1929,7 @@ dependencies = [

 [[package]]
 name = "common-datasource"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arrow",
 "arrow-schema",
@@ -2013,7 +1965,7 @@ dependencies = [

 [[package]]
 name = "common-decimal"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "bigdecimal 0.4.5",
 "common-error",
@@ -2026,7 +1978,7 @@ dependencies = [

 [[package]]
 name = "common-error"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "snafu 0.8.5",
 "strum 0.25.0",
@@ -2035,7 +1987,7 @@ dependencies = [

 [[package]]
 name = "common-frontend"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "common-error",
@@ -2045,7 +1997,7 @@ dependencies = [

 [[package]]
 name = "common-function"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "approx 0.5.1",
@@ -2089,7 +2041,7 @@ dependencies = [

 [[package]]
 name = "common-greptimedb-telemetry"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "common-runtime",
@@ -2106,7 +2058,7 @@ dependencies = [

 [[package]]
 name = "common-grpc"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arrow-flight",
@@ -2132,7 +2084,7 @@ dependencies = [

 [[package]]
 name = "common-grpc-expr"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "common-base",
@@ -2151,7 +2103,7 @@ dependencies = [

 [[package]]
 name = "common-macro"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arc-swap",
 "common-query",
@@ -2165,7 +2117,7 @@ dependencies = [

 [[package]]
 name = "common-mem-prof"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-error",
 "common-macro",
@@ -2178,7 +2130,7 @@ dependencies = [

 [[package]]
 name = "common-meta"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "anymap2",
 "api",
@@ -2235,7 +2187,7 @@ dependencies = [

 [[package]]
 name = "common-options"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-grpc",
 "humantime-serde",
@@ -2244,11 +2196,11 @@ dependencies = [

 [[package]]
 name = "common-plugins"
-version = "0.11.1"
+version = "0.12.0"

 [[package]]
 name = "common-pprof"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-error",
 "common-macro",
@@ -2260,11 +2212,11 @@ dependencies = [

 [[package]]
 name = "common-procedure"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-stream",
 "async-trait",
- "backon 1.2.0",
+ "backon",
 "common-base",
 "common-error",
 "common-macro",
@@ -2287,7 +2239,7 @@ dependencies = [

 [[package]]
 name = "common-procedure-test"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "common-procedure",
@@ -2295,7 +2247,7 @@ dependencies = [

 [[package]]
 name = "common-query"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -2321,7 +2273,7 @@ dependencies = [

 [[package]]
 name = "common-recordbatch"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arc-swap",
 "common-error",
@@ -2340,7 +2292,7 @@ dependencies = [

 [[package]]
 name = "common-runtime"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "clap 4.5.19",
@@ -2368,7 +2320,7 @@ dependencies = [

 [[package]]
 name = "common-telemetry"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "atty",
 "backtrace",
@@ -2396,7 +2348,7 @@ dependencies = [

 [[package]]
 name = "common-test-util"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "client",
 "common-query",
@@ -2408,7 +2360,7 @@ dependencies = [

 [[package]]
 name = "common-time"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arrow",
 "chrono",
@@ -2426,7 +2378,7 @@ dependencies = [

 [[package]]
 name = "common-version"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "build-data",
 "const_format",
@@ -2436,7 +2388,7 @@ dependencies = [

 [[package]]
 name = "common-wal"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "common-base",
 "common-error",
@@ -3235,7 +3187,7 @@ dependencies = [

 [[package]]
 name = "datanode"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arrow-flight",
@@ -3286,7 +3238,7 @@ dependencies = [
 "session",
 "snafu 0.8.5",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tokio",
 "toml 0.8.19",
@@ -3295,7 +3247,7 @@ dependencies = [

 [[package]]
 name = "datatypes"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arrow",
 "arrow-array",
@@ -3394,17 +3346,6 @@ dependencies = [
 "syn 2.0.90",
 ]

-[[package]]
-name = "derive-where"
-version = "1.2.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "62d671cc41a825ebabc75757b62d3d168c577f9149b2d49ece1dad1f72119d25"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn 2.0.90",
-]
-
 [[package]]
 name = "derive_arbitrary"
 version = "1.3.2"
@@ -3919,7 +3860,7 @@ dependencies = [

 [[package]]
 name = "file-engine"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -4035,7 +3976,7 @@ checksum = "8bf7cc16383c4b8d58b9905a8509f02926ce3058053c056376248d958c9df1e8"

 [[package]]
 name = "flow"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arrow",
@@ -4070,8 +4011,6 @@ dependencies = [
 "enum-as-inner",
 "enum_dispatch",
 "futures",
- "get-size-derive2",
- "get-size2",
 "greptime-proto",
 "hydroflow",
 "itertools 0.10.5",
@@ -4093,7 +4032,7 @@ dependencies = [
 "snafu 0.8.5",
 "store-api",
 "strum 0.25.0",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tokio",
 "tonic 0.11.0",
@@ -4131,7 +4070,7 @@ checksum = "6c2141d6d6c8512188a7891b4b01590a45f6dac67afb4f255c4124dbb86d4eaa"

 [[package]]
 name = "frontend"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arc-swap",
@@ -4476,23 +4415,6 @@ dependencies = [
 "libm",
 ]

-[[package]]
-name = "get-size-derive2"
-version = "0.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1fd26d3a97ea14d289c8b54180243ecfe465f3fa9c279a6336d7a003698fc39d"
-dependencies = [
- "attribute-derive",
- "quote",
- "syn 2.0.90",
-]
-
-[[package]]
-name = "get-size2"
-version = "0.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "159c430715e540d2198fa981d39cd45563ccc60900de187f5b152b33b1cb408e"
-
 [[package]]
 name = "gethostname"
 version = "0.2.3"
@@ -5280,7 +5202,7 @@ dependencies = [

 [[package]]
 name = "index"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "asynchronous-codec",
@@ -5424,12 +5346,6 @@ version = "4.0.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "0d762194228a2f1c11063e46e32e5acb96e66e906382b9eb5441f2e0504bbd5a"

-[[package]]
-name = "interpolator"
-version = "0.5.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "71dd52191aae121e8611f1e8dc3e324dd0dd1dee1e6dd91d10ee07a3cfb4d9d8"
-
 [[package]]
 name = "inventory"
 version = "0.3.15"
@@ -6129,7 +6045,7 @@ checksum = "a7a70ba024b9dc04c27ea2f0c0548feb474ec5c54bba33a7f72f873a39d07b24"

 [[package]]
 name = "log-query"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "chrono",
 "common-error",
@@ -6140,7 +6056,7 @@ dependencies = [

 [[package]]
 name = "log-store"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-stream",
 "async-trait",
@@ -6328,29 +6244,6 @@ dependencies = [
 "libc",
 ]

-[[package]]
-name = "manyhow"
-version = "0.11.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b33efb3ca6d3b07393750d4030418d594ab1139cee518f0dc88db70fec873587"
-dependencies = [
- "manyhow-macros",
- "proc-macro2",
- "quote",
- "syn 2.0.90",
-]
-
-[[package]]
-name = "manyhow-macros"
-version = "0.11.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "46fce34d199b78b6e6073abf984c9cf5fd3e9330145a93ee0738a7443e371495"
-dependencies = [
- "proc-macro-utils",
- "proc-macro2",
- "quote",
-]
-
 [[package]]
 name = "maplit"
 version = "1.0.2"
@@ -6484,7 +6377,7 @@ dependencies = [

 [[package]]
 name = "meta-client"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -6511,7 +6404,7 @@ dependencies = [

 [[package]]
 name = "meta-srv"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -6590,7 +6483,7 @@ dependencies = [

 [[package]]
 name = "metric-engine"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "aquamarine",
@@ -6684,7 +6577,7 @@ dependencies = [

 [[package]]
 name = "mito2"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "aquamarine",
@@ -7421,7 +7314,7 @@ dependencies = [

 [[package]]
 name = "object-store"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "anyhow",
 "bytes",
@@ -7481,13 +7374,13 @@ checksum = "b410bbe7e14ab526a0e86877eb47c6996a2bd7746f027ba551028c925390e4e9"

 [[package]]
 name = "opendal"
-version = "0.49.2"
+version = "0.50.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9b04d09b9822c2f75a1d2fc513a2c1279c70e91e7407936fffdf6a6976ec530a"
+checksum = "cb28bb6c64e116ceaf8dd4e87099d3cfea4a58e85e62b104fef74c91afba0f44"
 dependencies = [
 "anyhow",
 "async-trait",
- "backon 0.4.4",
+ "backon",
 "base64 0.22.1",
 "bytes",
 "chrono",
@@ -7500,6 +7393,7 @@ dependencies = [
 "md-5",
 "once_cell",
 "percent-encoding",
+ "prometheus",
 "quick-xml 0.36.2",
 "reqsign",
 "reqwest",
@@ -7674,7 +7568,7 @@ dependencies = [

 [[package]]
 name = "operator"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "ahash 0.8.11",
 "api",
@@ -7722,7 +7616,7 @@ dependencies = [
 "sql",
 "sqlparser 0.45.0 (git+https://github.com/GreptimeTeam/sqlparser-rs.git?rev=54a267ac89c09b11c0c88934690530807185d3e7)",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tokio",
 "tokio-util",
@@ -7972,7 +7866,7 @@ dependencies = [

 [[package]]
 name = "partition"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -8258,7 +8152,7 @@ checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"

 [[package]]
 name = "pipeline"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "ahash 0.8.11",
 "api",
@@ -8420,7 +8314,7 @@ dependencies = [

 [[package]]
 name = "plugins"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "auth",
 "clap 4.5.19",
@@ -8634,17 +8528,6 @@ dependencies = [
 "version_check",
 ]

-[[package]]
-name = "proc-macro-utils"
-version = "0.10.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "eeaf08a13de400bc215877b5bdc088f241b12eb42f0a548d3390dc1c56bb7071"
-dependencies = [
- "proc-macro2",
- "quote",
- "smallvec",
-]
-
 [[package]]
 name = "proc-macro2"
 version = "1.0.92"
@@ -8708,7 +8591,7 @@ dependencies = [

 [[package]]
 name = "promql"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "ahash 0.8.11",
 "async-trait",
@@ -8943,7 +8826,7 @@ dependencies = [

 [[package]]
 name = "puffin"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-compression 0.4.13",
 "async-trait",
@@ -9068,7 +8951,7 @@ dependencies = [

 [[package]]
 name = "query"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "ahash 0.8.11",
 "api",
@@ -9131,7 +9014,7 @@ dependencies = [
 "sqlparser 0.45.0 (git+https://github.com/GreptimeTeam/sqlparser-rs.git?rev=54a267ac89c09b11c0c88934690530807185d3e7)",
 "statrs",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tokio",
 "tokio-stream",
@@ -9224,28 +9107,6 @@ dependencies = [
 "proc-macro2",
 ]

-[[package]]
-name = "quote-use"
-version = "0.8.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9619db1197b497a36178cfc736dc96b271fe918875fbf1344c436a7e93d0321e"
-dependencies = [
- "quote",
- "quote-use-macros",
-]
-
-[[package]]
-name = "quote-use-macros"
-version = "0.8.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "82ebfb7faafadc06a7ab141a6f67bcfb24cb8beb158c6fe933f2f035afa99f35"
-dependencies = [
- "proc-macro-utils",
- "proc-macro2",
- "quote",
- "syn 2.0.90",
-]
-
 [[package]]
 name = "radium"
 version = "0.7.0"
@@ -9515,9 +9376,9 @@ dependencies = [

 [[package]]
 name = "reqsign"
-version = "0.16.0"
+version = "0.16.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "03dd4ba7c3901dd43e6b8c7446a760d45bc1ea4301002e1a6fa48f97c3a796fa"
+checksum = "eb0075a66c8bfbf4cc8b70dca166e722e1f55a3ea9250ecbb85f4d92a5f64149"
 dependencies = [
 "anyhow",
 "async-trait",
@@ -10615,7 +10476,7 @@ checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"

 [[package]]
 name = "script"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arc-swap",
@@ -10907,7 +10768,7 @@ dependencies = [

 [[package]]
 name = "servers"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "ahash 0.8.11",
 "api",
@@ -11018,7 +10879,7 @@ dependencies = [

 [[package]]
 name = "session"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arc-swap",
@@ -11372,7 +11233,7 @@ dependencies = [

 [[package]]
 name = "sql"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "chrono",
@@ -11436,7 +11297,7 @@ dependencies = [

 [[package]]
 name = "sqlness-runner"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "clap 4.5.19",
@@ -11654,7 +11515,7 @@ dependencies = [

 [[package]]
 name = "store-api"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "aquamarine",
@@ -11816,7 +11677,7 @@ dependencies = [

 [[package]]
 name = "substrait"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "async-trait",
 "bytes",
@@ -12015,7 +11876,7 @@ dependencies = [

 [[package]]
 name = "table"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "async-trait",
@@ -12292,7 +12153,7 @@ checksum = "3369f5ac52d5eb6ab48c6b4ffdc8efbcad6b89c765749064ba298f2c68a16a76"

 [[package]]
 name = "tests-fuzz"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "arbitrary",
 "async-trait",
@@ -12335,7 +12196,7 @@ dependencies = [

 [[package]]
 name = "tests-integration"
-version = "0.11.1"
+version = "0.12.0"
 dependencies = [
 "api",
 "arrow-flight",
@@ -12399,7 +12260,7 @@ dependencies = [
 "sql",
 "sqlx",
 "store-api",
- "substrait 0.11.1",
+ "substrait 0.12.0",
 "table",
 "tempfile",
 "time",
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -68,7 +68,7 @@ members = [
 resolver = "2"

 [workspace.package]
-version = "0.11.1"
+version = "0.12.0"
 edition = "2021"
 license = "Apache-2.0"

--- a/docker/dev-builder/ubuntu/Dockerfile
+++ b/docker/dev-builder/ubuntu/Dockerfile
@@ -15,8 +15,8 @@ RUN apt-get update && \
 RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    libssl-dev \
    tzdata \
+    protobuf-compiler \
    curl \
-    unzip \
    ca-certificates \
    git \
    build-essential \
@@ -24,20 +24,6 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    python3.10 \
    python3.10-dev

-ARG TARGETPLATFORM
-RUN echo "target platform: $TARGETPLATFORM"
-
-# Install protobuf, because the one in the apt is too old (v3.12).
-RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
-    curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v29.1/protoc-29.1-linux-aarch_64.zip && \
-    unzip protoc-29.1-linux-aarch_64.zip -d protoc3; \
-elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
-    curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v29.1/protoc-29.1-linux-x86_64.zip && \
-    unzip protoc-29.1-linux-x86_64.zip -d protoc3; \
-fi
-RUN mv protoc3/bin/* /usr/local/bin/
-RUN mv protoc3/include/* /usr/local/include/
-
 # https://github.com/GreptimeTeam/greptimedb/actions/runs/10935485852/job/30357457188#step:3:7106
 # `aws-lc-sys` require gcc >= 10.3.0 to work, hence alias to use gcc-10
 RUN apt-get remove -y gcc-9 g++-9 cpp-9 && \
@@ -63,7 +49,7 @@ RUN apt-get -y purge python3.8 && \
 # wildcard here. However, that requires the git's config files and the submodules all owned by the very same user.
 # It's troublesome to do this since the dev build runs in Docker, which is under user "root"; while outside the Docker,
 # it can be a different user that have prepared the submodules.
-RUN git config --global --add safe.directory '*'
+RUN git config --global --add safe.directory *

 # Install Python dependencies.
 COPY $DOCKER_BUILD_ROOT/docker/python/requirements.txt /etc/greptime/requirements.txt
--- a/src/catalog/src/error.rs
+++ b/src/catalog/src/error.rs
@@ -64,13 +64,6 @@ pub enum Error {
        source: BoxedError,
    },

-    #[snafu(display("Failed to list flow stats"))]
-    ListFlowStats {
-        #[snafu(implicit)]
-        location: Location,
-        source: BoxedError,
-    },
-
    #[snafu(display("Failed to list flows in catalog {catalog}"))]
    ListFlows {
        #[snafu(implicit)]
@@ -333,7 +326,6 @@ impl ErrorExt for Error {
            | Error::ListSchemas { source, .. }
            | Error::ListTables { source, .. }
            | Error::ListFlows { source, .. }
-            | Error::ListFlowStats { source, .. }
            | Error::ListProcedures { source, .. }
            | Error::ListRegionStats { source, .. }
            | Error::ConvertProtoData { source, .. } => source.status_code(),
--- a/src/catalog/src/information_extension.rs
+++ b/src/catalog/src/information_extension.rs
@@ -17,7 +17,6 @@ use common_error::ext::BoxedError;
 use common_meta::cluster::{ClusterInfo, NodeInfo};
 use common_meta::datanode::RegionStat;
 use common_meta::ddl::{ExecutorContext, ProcedureExecutor};
-use common_meta::key::flow::flow_state::FlowStat;
 use common_meta::rpc::procedure;
 use common_procedure::{ProcedureInfo, ProcedureState};
 use meta_client::MetaClientRef;
@@ -90,12 +89,4 @@ impl InformationExtension for DistributedInformationExtension {
            .map_err(BoxedError::new)
            .context(error::ListRegionStatsSnafu)
    }
-
-    async fn flow_stats(&self) -> std::result::Result<Option<FlowStat>, Self::Error> {
-        self.meta_client
-            .list_flow_stats()
-            .await
-            .map_err(BoxedError::new)
-            .context(crate::error::ListFlowStatsSnafu)
-    }
 }
--- a/src/catalog/src/system_schema/information_schema.rs
+++ b/src/catalog/src/system_schema/information_schema.rs
@@ -35,7 +35,6 @@ use common_catalog::consts::{self, DEFAULT_CATALOG_NAME, INFORMATION_SCHEMA_NAME
 use common_error::ext::ErrorExt;
 use common_meta::cluster::NodeInfo;
 use common_meta::datanode::RegionStat;
-use common_meta::key::flow::flow_state::FlowStat;
 use common_meta::key::flow::FlowMetadataManager;
 use common_procedure::ProcedureInfo;
 use common_recordbatch::SendableRecordBatchStream;
@@ -193,7 +192,6 @@ impl SystemSchemaProviderInner for InformationSchemaProvider {
            )) as _),
            FLOWS => Some(Arc::new(InformationSchemaFlows::new(
                self.catalog_name.clone(),
-                self.catalog_manager.clone(),
                self.flow_metadata_manager.clone(),
            )) as _),
            PROCEDURE_INFO => Some(
@@ -340,9 +338,6 @@ pub trait InformationExtension {

    /// Gets the region statistics.
    async fn region_stats(&self) -> std::result::Result<Vec<RegionStat>, Self::Error>;
-
-    /// Get the flow statistics. If no flownode is available, return `None`.
-    async fn flow_stats(&self) -> std::result::Result<Option<FlowStat>, Self::Error>;
 }

 pub struct NoopInformationExtension;
@@ -362,8 +357,4 @@ impl InformationExtension for NoopInformationExtension {
    async fn region_stats(&self) -> std::result::Result<Vec<RegionStat>, Self::Error> {
        Ok(vec![])
    }
-
-    async fn flow_stats(&self) -> std::result::Result<Option<FlowStat>, Self::Error> {
-        Ok(None)
-    }
 }
--- a/src/catalog/src/system_schema/information_schema/flows.rs
+++ b/src/catalog/src/system_schema/information_schema/flows.rs
@@ -12,12 +12,11 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-use std::sync::{Arc, Weak};
+use std::sync::Arc;

 use common_catalog::consts::INFORMATION_SCHEMA_FLOW_TABLE_ID;
 use common_error::ext::BoxedError;
 use common_meta::key::flow::flow_info::FlowInfoValue;
-use common_meta::key::flow::flow_state::FlowStat;
 use common_meta::key::flow::FlowMetadataManager;
 use common_meta::key::FlowId;
 use common_recordbatch::adapter::RecordBatchStreamAdapter;
@@ -29,9 +28,7 @@ use datatypes::prelude::ConcreteDataType as CDT;
 use datatypes::scalars::ScalarVectorBuilder;
 use datatypes::schema::{ColumnSchema, Schema, SchemaRef};
 use datatypes::value::Value;
-use datatypes::vectors::{
-    Int64VectorBuilder, StringVectorBuilder, UInt32VectorBuilder, UInt64VectorBuilder, VectorRef,
-};
+use datatypes::vectors::{Int64VectorBuilder, StringVectorBuilder, UInt32VectorBuilder, VectorRef};
 use futures::TryStreamExt;
 use snafu::{OptionExt, ResultExt};
 use store_api::storage::{ScanRequest, TableId};
@@ -41,8 +38,6 @@ use crate::error::{
 };
 use crate::information_schema::{Predicates, FLOWS};
 use crate::system_schema::information_schema::InformationTable;
-use crate::system_schema::utils;
-use crate::CatalogManager;

 const INIT_CAPACITY: usize = 42;

@@ -50,7 +45,6 @@ const INIT_CAPACITY: usize = 42;
 // pk is (flow_name, flow_id, table_catalog)
 pub const FLOW_NAME: &str = "flow_name";
 pub const FLOW_ID: &str = "flow_id";
-pub const STATE_SIZE: &str = "state_size";
 pub const TABLE_CATALOG: &str = "table_catalog";
 pub const FLOW_DEFINITION: &str = "flow_definition";
 pub const COMMENT: &str = "comment";
@@ -61,24 +55,20 @@ pub const FLOWNODE_IDS: &str = "flownode_ids";
 pub const OPTIONS: &str = "options";

 /// The `information_schema.flows` to provides information about flows in databases.
-///
 pub(super) struct InformationSchemaFlows {
    schema: SchemaRef,
    catalog_name: String,
-    catalog_manager: Weak<dyn CatalogManager>,
    flow_metadata_manager: Arc<FlowMetadataManager>,
 }

 impl InformationSchemaFlows {
    pub(super) fn new(
        catalog_name: String,
-        catalog_manager: Weak<dyn CatalogManager>,
        flow_metadata_manager: Arc<FlowMetadataManager>,
    ) -> Self {
        Self {
            schema: Self::schema(),
            catalog_name,
-            catalog_manager,
            flow_metadata_manager,
        }
    }
@@ -90,7 +80,6 @@ impl InformationSchemaFlows {
            vec![
                (FLOW_NAME, CDT::string_datatype(), false),
                (FLOW_ID, CDT::uint32_datatype(), false),
-                (STATE_SIZE, CDT::uint64_datatype(), true),
                (TABLE_CATALOG, CDT::string_datatype(), false),
                (FLOW_DEFINITION, CDT::string_datatype(), false),
                (COMMENT, CDT::string_datatype(), true),
@@ -110,7 +99,6 @@ impl InformationSchemaFlows {
        InformationSchemaFlowsBuilder::new(
            self.schema.clone(),
            self.catalog_name.clone(),
-            self.catalog_manager.clone(),
            &self.flow_metadata_manager,
        )
    }
@@ -156,12 +144,10 @@ impl InformationTable for InformationSchemaFlows {
 struct InformationSchemaFlowsBuilder {
    schema: SchemaRef,
    catalog_name: String,
-    catalog_manager: Weak<dyn CatalogManager>,
    flow_metadata_manager: Arc<FlowMetadataManager>,

    flow_names: StringVectorBuilder,
    flow_ids: UInt32VectorBuilder,
-    state_sizes: UInt64VectorBuilder,
    table_catalogs: StringVectorBuilder,
    raw_sqls: StringVectorBuilder,
    comments: StringVectorBuilder,
@@ -176,18 +162,15 @@ impl InformationSchemaFlowsBuilder {
    fn new(
        schema: SchemaRef,
        catalog_name: String,
-        catalog_manager: Weak<dyn CatalogManager>,
        flow_metadata_manager: &Arc<FlowMetadataManager>,
    ) -> Self {
        Self {
            schema,
            catalog_name,
-            catalog_manager,
            flow_metadata_manager: flow_metadata_manager.clone(),

            flow_names: StringVectorBuilder::with_capacity(INIT_CAPACITY),
            flow_ids: UInt32VectorBuilder::with_capacity(INIT_CAPACITY),
-            state_sizes: UInt64VectorBuilder::with_capacity(INIT_CAPACITY),
            table_catalogs: StringVectorBuilder::with_capacity(INIT_CAPACITY),
            raw_sqls: StringVectorBuilder::with_capacity(INIT_CAPACITY),
            comments: StringVectorBuilder::with_capacity(INIT_CAPACITY),
@@ -212,11 +195,6 @@ impl InformationSchemaFlowsBuilder {
            .flow_names(&catalog_name)
            .await;

-        let flow_stat = {
-            let information_extension = utils::information_extension(&self.catalog_manager)?;
-            information_extension.flow_stats().await?
-        };
-
        while let Some((flow_name, flow_id)) = stream
            .try_next()
            .await
@@ -235,7 +213,7 @@ impl InformationSchemaFlowsBuilder {
                    catalog_name: catalog_name.to_string(),
                    flow_name: flow_name.to_string(),
                })?;
-            self.add_flow(&predicates, flow_id.flow_id(), flow_info, &flow_stat)?;
+            self.add_flow(&predicates, flow_id.flow_id(), flow_info)?;
        }

        self.finish()
@@ -246,7 +224,6 @@ impl InformationSchemaFlowsBuilder {
        predicates: &Predicates,
        flow_id: FlowId,
        flow_info: FlowInfoValue,
-        flow_stat: &Option<FlowStat>,
    ) -> Result<()> {
        let row = [
            (FLOW_NAME, &Value::from(flow_info.flow_name().to_string())),
@@ -261,11 +238,6 @@ impl InformationSchemaFlowsBuilder {
        }
        self.flow_names.push(Some(flow_info.flow_name()));
        self.flow_ids.push(Some(flow_id));
-        self.state_sizes.push(
-            flow_stat
-                .as_ref()
-                .and_then(|state| state.state_size.get(&flow_id).map(|v| *v as u64)),
-        );
        self.table_catalogs.push(Some(flow_info.catalog_name()));
        self.raw_sqls.push(Some(flow_info.raw_sql()));
        self.comments.push(Some(flow_info.comment()));
@@ -298,7 +270,6 @@ impl InformationSchemaFlowsBuilder {
        let columns: Vec<VectorRef> = vec![
            Arc::new(self.flow_names.finish()),
            Arc::new(self.flow_ids.finish()),
-            Arc::new(self.state_sizes.finish()),
            Arc::new(self.table_catalogs.finish()),
            Arc::new(self.raw_sqls.finish()),
            Arc::new(self.comments.finish()),
--- a/src/cli/src/repl.rs
+++ b/src/cli/src/repl.rs
@@ -34,7 +34,7 @@ use common_query::Output;
 use common_recordbatch::RecordBatches;
 use common_telemetry::debug;
 use either::Either;
-use meta_client::client::{ClusterKvBackend, MetaClientBuilder};
+use meta_client::client::MetaClientBuilder;
 use query::datafusion::DatafusionQueryEngine;
 use query::parser::QueryLanguageParser;
 use query::query_engine::{DefaultSerializer, QueryEngineState};
--- a/src/cmd/src/standalone.rs
+++ b/src/cmd/src/standalone.rs
@@ -34,7 +34,6 @@ use common_meta::ddl::flow_meta::{FlowMetadataAllocator, FlowMetadataAllocatorRe
 use common_meta::ddl::table_meta::{TableMetadataAllocator, TableMetadataAllocatorRef};
 use common_meta::ddl::{DdlContext, NoopRegionFailureDetectorControl, ProcedureExecutorRef};
 use common_meta::ddl_manager::DdlManager;
-use common_meta::key::flow::flow_state::FlowStat;
 use common_meta::key::flow::{FlowMetadataManager, FlowMetadataManagerRef};
 use common_meta::key::{TableMetadataManager, TableMetadataManagerRef};
 use common_meta::kv_backend::KvBackendRef;
@@ -71,7 +70,7 @@ use servers::http::HttpOptions;
 use servers::tls::{TlsMode, TlsOption};
 use servers::Mode;
 use snafu::ResultExt;
-use tokio::sync::{broadcast, RwLock};
+use tokio::sync::broadcast;
 use tracing_appender::non_blocking::WorkerGuard;

 use crate::error::{
@@ -508,7 +507,7 @@ impl StartCommand {
            procedure_manager.clone(),
        ));
        let catalog_manager = KvBackendCatalogManager::new(
-            information_extension.clone(),
+            information_extension,
            kv_backend.clone(),
            layered_cache_registry.clone(),
            Some(procedure_manager.clone()),
@@ -533,14 +532,6 @@ impl StartCommand {
                .context(OtherSnafu)?,
        );

-        // set the ref to query for the local flow state
-        {
-            let flow_worker_manager = flownode.flow_worker_manager();
-            information_extension
-                .set_flow_worker_manager(flow_worker_manager.clone())
-                .await;
-        }
-
        let node_manager = Arc::new(StandaloneDatanodeManager {
            region_server: datanode.region_server(),
            flow_server: flownode.flow_worker_manager(),
@@ -678,7 +669,6 @@ pub struct StandaloneInformationExtension {
    region_server: RegionServer,
    procedure_manager: ProcedureManagerRef,
    start_time_ms: u64,
-    flow_worker_manager: RwLock<Option<Arc<FlowWorkerManager>>>,
 }

 impl StandaloneInformationExtension {
@@ -687,15 +677,8 @@ impl StandaloneInformationExtension {
            region_server,
            procedure_manager,
            start_time_ms: common_time::util::current_time_millis() as u64,
-            flow_worker_manager: RwLock::new(None),
        }
    }
-
-    /// Set the flow worker manager for the standalone instance.
-    pub async fn set_flow_worker_manager(&self, flow_worker_manager: Arc<FlowWorkerManager>) {
-        let mut guard = self.flow_worker_manager.write().await;
-        *guard = Some(flow_worker_manager);
-    }
 }

 #[async_trait::async_trait]
@@ -767,18 +750,6 @@ impl InformationExtension for StandaloneInformationExtension {
            .collect::<Vec<_>>();
        Ok(stats)
    }
-
-    async fn flow_stats(&self) -> std::result::Result<Option<FlowStat>, Self::Error> {
-        Ok(Some(
-            self.flow_worker_manager
-                .read()
-                .await
-                .as_ref()
-                .unwrap()
-                .gen_state_report()
-                .await,
-        ))
-    }
 }

 #[cfg(test)]
--- a/src/common/datasource/src/object_store/fs.rs
+++ b/src/common/datasource/src/object_store/fs.rs
@@ -27,7 +27,7 @@ pub fn build_fs_backend(root: &str) -> Result<ObjectStore> {
            DefaultLoggingInterceptor,
        ))
        .layer(object_store::layers::TracingLayer)
-        .layer(object_store::layers::PrometheusMetricsLayer::new(true))
+        .layer(object_store::layers::build_prometheus_metrics_layer(true))
        .finish();
    Ok(object_store)
 }
--- a/src/common/datasource/src/object_store/s3.rs
+++ b/src/common/datasource/src/object_store/s3.rs
@@ -89,7 +89,7 @@ pub fn build_s3_backend(
            DefaultLoggingInterceptor,
        ))
        .layer(object_store::layers::TracingLayer)
-        .layer(object_store::layers::PrometheusMetricsLayer::new(true))
+        .layer(object_store::layers::build_prometheus_metrics_layer(true))
        .finish())
 }

--- a/src/common/meta/src/key.rs
+++ b/src/common/meta/src/key.rs
@@ -137,7 +137,6 @@ use self::schema_name::{SchemaManager, SchemaNameKey, SchemaNameValue};
 use self::table_route::{TableRouteManager, TableRouteValue};
 use self::tombstone::TombstoneManager;
 use crate::error::{self, Result, SerdeJsonSnafu};
-use crate::key::flow::flow_state::FlowStateValue;
 use crate::key::node_address::NodeAddressValue;
 use crate::key::table_route::TableRouteKey;
 use crate::key::txn_helper::TxnOpGetResponseSet;
@@ -1263,8 +1262,7 @@ impl_metadata_value! {
    FlowRouteValue,
    TableFlowValue,
    NodeAddressValue,
-    SchemaNameValue,
-    FlowStateValue
+    SchemaNameValue
 }

 impl_optional_metadata_value! {
--- a/src/common/meta/src/key/flow.rs
+++ b/src/common/meta/src/key/flow.rs
@@ -15,7 +15,6 @@
 pub mod flow_info;
 pub(crate) mod flow_name;
 pub(crate) mod flow_route;
-pub mod flow_state;
 pub(crate) mod flownode_flow;
 pub(crate) mod table_flow;

@@ -36,7 +35,6 @@ use crate::ensure_values;
 use crate::error::{self, Result};
 use crate::key::flow::flow_info::FlowInfoManager;
 use crate::key::flow::flow_name::FlowNameManager;
-use crate::key::flow::flow_state::FlowStateManager;
 use crate::key::flow::flownode_flow::FlownodeFlowManager;
 pub use crate::key::flow::table_flow::{TableFlowManager, TableFlowManagerRef};
 use crate::key::txn_helper::TxnOpGetResponseSet;
@@ -104,8 +102,6 @@ pub struct FlowMetadataManager {
    flownode_flow_manager: FlownodeFlowManager,
    table_flow_manager: TableFlowManager,
    flow_name_manager: FlowNameManager,
-    /// only metasrv have access to itself's memory backend, so for other case it should be None
-    flow_state_manager: Option<FlowStateManager>,
    kv_backend: KvBackendRef,
 }

@@ -118,7 +114,6 @@ impl FlowMetadataManager {
            flow_name_manager: FlowNameManager::new(kv_backend.clone()),
            flownode_flow_manager: FlownodeFlowManager::new(kv_backend.clone()),
            table_flow_manager: TableFlowManager::new(kv_backend.clone()),
-            flow_state_manager: None,
            kv_backend,
        }
    }
@@ -128,10 +123,6 @@ impl FlowMetadataManager {
        &self.flow_name_manager
    }

-    pub fn flow_state_manager(&self) -> Option<&FlowStateManager> {
-        self.flow_state_manager.as_ref()
-    }
-
    /// Returns the [`FlowInfoManager`].
    pub fn flow_info_manager(&self) -> &FlowInfoManager {
        &self.flow_info_manager
--- a/src/common/meta/src/key/flow/flow_state.rs
+++ b/src/common/meta/src/key/flow/flow_state.rs
@@ -1,162 +0,0 @@
-// Copyright 2023 Greptime Team
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-use std::collections::BTreeMap;
-use std::sync::Arc;
-
-use serde::{Deserialize, Serialize};
-
-use crate::error::{self, Result};
-use crate::key::flow::FlowScoped;
-use crate::key::{FlowId, MetadataKey, MetadataValue};
-use crate::kv_backend::KvBackendRef;
-use crate::rpc::store::PutRequest;
-
-/// The entire FlowId to Flow Size's Map is stored directly in the value part of the key.
-const FLOW_STATE_KEY: &str = "state";
-
-/// The key of flow state.
-#[derive(Debug, Clone, Copy, PartialEq)]
-struct FlowStateKeyInner;
-
-impl FlowStateKeyInner {
-    pub fn new() -> Self {
-        Self
-    }
-}
-
-impl<'a> MetadataKey<'a, FlowStateKeyInner> for FlowStateKeyInner {
-    fn to_bytes(&self) -> Vec<u8> {
-        FLOW_STATE_KEY.as_bytes().to_vec()
-    }
-
-    fn from_bytes(bytes: &'a [u8]) -> Result<FlowStateKeyInner> {
-        let key = std::str::from_utf8(bytes).map_err(|e| {
-            error::InvalidMetadataSnafu {
-                err_msg: format!(
-                    "FlowInfoKeyInner '{}' is not a valid UTF8 string: {e}",
-                    String::from_utf8_lossy(bytes)
-                ),
-            }
-            .build()
-        })?;
-        if key != FLOW_STATE_KEY {
-            return Err(error::InvalidMetadataSnafu {
-                err_msg: format!("Invalid FlowStateKeyInner '{key}'"),
-            }
-            .build());
-        }
-        Ok(FlowStateKeyInner::new())
-    }
-}
-
-/// The key stores the state size of the flow.
-///
-/// The layout: `__flow/state`.
-pub struct FlowStateKey(FlowScoped<FlowStateKeyInner>);
-
-impl FlowStateKey {
-    /// Returns the [FlowStateKey].
-    pub fn new() -> FlowStateKey {
-        let inner = FlowStateKeyInner::new();
-        FlowStateKey(FlowScoped::new(inner))
-    }
-}
-
-impl Default for FlowStateKey {
-    fn default() -> Self {
-        Self::new()
-    }
-}
-
-impl<'a> MetadataKey<'a, FlowStateKey> for FlowStateKey {
-    fn to_bytes(&self) -> Vec<u8> {
-        self.0.to_bytes()
-    }
-
-    fn from_bytes(bytes: &'a [u8]) -> Result<FlowStateKey> {
-        Ok(FlowStateKey(FlowScoped::<FlowStateKeyInner>::from_bytes(
-            bytes,
-        )?))
-    }
-}
-
-/// The value of flow state size
-#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
-pub struct FlowStateValue {
-    /// For each key, the bytes of the state in memory
-    pub state_size: BTreeMap<FlowId, usize>,
-}
-
-impl FlowStateValue {
-    pub fn new(state_size: BTreeMap<FlowId, usize>) -> Self {
-        Self { state_size }
-    }
-}
-
-pub type FlowStateManagerRef = Arc<FlowStateManager>;
-
-/// The manager of [FlowStateKey]. Since state size changes frequently, we store it in memory.
-///
-/// This is only used in distributed mode. When meta-srv use heartbeat to update the flow stat report
-/// and frontned use get to get the latest flow stat report.
-pub struct FlowStateManager {
-    in_memory: KvBackendRef,
-}
-
-impl FlowStateManager {
-    pub fn new(in_memory: KvBackendRef) -> Self {
-        Self { in_memory }
-    }
-
-    pub async fn get(&self) -> Result<Option<FlowStateValue>> {
-        let key = FlowStateKey::new().to_bytes();
-        self.in_memory
-            .get(&key)
-            .await?
-            .map(|x| FlowStateValue::try_from_raw_value(&x.value))
-            .transpose()
-    }
-
-    pub async fn put(&self, value: FlowStateValue) -> Result<()> {
-        let key = FlowStateKey::new().to_bytes();
-        let value = value.try_as_raw_value()?;
-        let req = PutRequest::new().with_key(key).with_value(value);
-        self.in_memory.put(req).await?;
-        Ok(())
-    }
-}
-
-/// Flow's state report, send regularly through heartbeat message
-#[derive(Debug, Clone)]
-pub struct FlowStat {
-    /// For each key, the bytes of the state in memory
-    pub state_size: BTreeMap<u32, usize>,
-}
-
-impl From<FlowStateValue> for FlowStat {
-    fn from(value: FlowStateValue) -> Self {
-        Self {
-            state_size: value.state_size,
-        }
-    }
-}
-
-impl From<FlowStat> for FlowStateValue {
-    fn from(value: FlowStat) -> Self {
-        Self {
-            state_size: value.state_size,
-        }
-    }
-}
--- a/src/common/meta/src/kv_backend.rs
+++ b/src/common/meta/src/kv_backend.rs
@@ -36,7 +36,7 @@ pub mod postgres;
 pub mod test;
 pub mod txn;

-pub type KvBackendRef<E = Error> = Arc<dyn KvBackend<Error = E> + Send + Sync>;
+pub type KvBackendRef = Arc<dyn KvBackend<Error = Error> + Send + Sync>;

 #[async_trait]
 pub trait KvBackend: TxnService
@@ -161,9 +161,6 @@ where
    Self::Error: ErrorExt,
 {
    fn reset(&self);
-
-    /// Upcast as `KvBackendRef`. Since https://github.com/rust-lang/rust/issues/65991 is not yet stable.
-    fn as_kv_backend_ref(self: Arc<Self>) -> KvBackendRef<Self::Error>;
 }

-pub type ResettableKvBackendRef<E = Error> = Arc<dyn ResettableKvBackend<Error = E> + Send + Sync>;
+pub type ResettableKvBackendRef = Arc<dyn ResettableKvBackend<Error = Error> + Send + Sync>;
--- a/src/common/meta/src/kv_backend/memory.rs
+++ b/src/common/meta/src/kv_backend/memory.rs
@@ -16,13 +16,13 @@ use std::any::Any;
 use std::collections::BTreeMap;
 use std::fmt::{Display, Formatter};
 use std::marker::PhantomData;
-use std::sync::{Arc, RwLock};
+use std::sync::RwLock;

 use async_trait::async_trait;
 use common_error::ext::ErrorExt;
 use serde::Serializer;

-use super::{KvBackendRef, ResettableKvBackend};
+use super::ResettableKvBackend;
 use crate::kv_backend::txn::{Txn, TxnOp, TxnOpResponse, TxnRequest, TxnResponse};
 use crate::kv_backend::{KvBackend, TxnService};
 use crate::metrics::METRIC_META_TXN_REQUEST;
@@ -311,10 +311,6 @@ impl<T: ErrorExt + Send + Sync + 'static> ResettableKvBackend for MemoryKvBacken
    fn reset(&self) {
        self.clear();
    }
-
-    fn as_kv_backend_ref(self: Arc<Self>) -> KvBackendRef<T> {
-        self
-    }
 }

 #[cfg(test)]
--- a/src/common/procedure/src/local/runner.rs
+++ b/src/common/procedure/src/local/runner.rs
@@ -544,7 +544,7 @@ mod tests {
    use common_test_util::temp_dir::create_temp_dir;
    use futures_util::future::BoxFuture;
    use futures_util::FutureExt;
-    use object_store::ObjectStore;
+    use object_store::{EntryMode, ObjectStore};
    use tokio::sync::mpsc;

    use super::*;
@@ -578,7 +578,11 @@ mod tests {
    ) {
        let dir = proc_path!(procedure_store, "{procedure_id}/");
        let lister = object_store.list(&dir).await.unwrap();
-        let mut files_in_dir: Vec<_> = lister.into_iter().map(|de| de.name().to_string()).collect();
+        let mut files_in_dir: Vec<_> = lister
+            .into_iter()
+            .filter(|x| x.metadata().mode() == EntryMode::FILE)
+            .map(|de| de.name().to_string())
+            .collect();
        files_in_dir.sort_unstable();
        assert_eq!(files, files_in_dir);
    }
--- a/src/datanode/src/error.rs
+++ b/src/datanode/src/error.rs
@@ -193,6 +193,14 @@ pub enum Error {
        location: Location,
    },

+    #[snafu(display("Failed to build http client"))]
+    BuildHttpClient {
+        #[snafu(implicit)]
+        location: Location,
+        #[snafu(source)]
+        error: reqwest::Error,
+    },
+
    #[snafu(display("Missing required field: {}", name))]
    MissingRequiredField {
        name: String,
@@ -406,9 +414,10 @@ impl ErrorExt for Error {
            | MissingKvBackend { .. }
            | TomlFormat { .. } => StatusCode::InvalidArguments,

-            PayloadNotExist { .. } | Unexpected { .. } | WatchAsyncTaskChange { .. } => {
-                StatusCode::Unexpected
-            }
+            PayloadNotExist { .. }
+            | Unexpected { .. }
+            | WatchAsyncTaskChange { .. }
+            | BuildHttpClient { .. } => StatusCode::Unexpected,

            AsyncTaskExecute { source, .. } => source.status_code(),

--- a/src/datanode/src/store.rs
+++ b/src/datanode/src/store.rs
@@ -32,7 +32,7 @@ use object_store::{Access, Error, HttpClient, ObjectStore, ObjectStoreBuilder, O
 use snafu::prelude::*;

 use crate::config::{HttpClientConfig, ObjectStoreConfig, DEFAULT_OBJECT_STORE_CACHE_SIZE};
-use crate::error::{self, CreateDirSnafu, Result};
+use crate::error::{self, BuildHttpClientSnafu, CreateDirSnafu, Result};

 pub(crate) async fn new_raw_object_store(
    store: &ObjectStoreConfig,
@@ -236,7 +236,8 @@ pub(crate) fn build_http_client(config: &HttpClientConfig) -> Result<HttpClient>
        builder.timeout(config.timeout)
    };

-    HttpClient::build(http_builder).context(error::InitBackendSnafu)
+    let client = http_builder.build().context(BuildHttpClientSnafu)?;
+    Ok(HttpClient::with(client))
 }
 struct PrintDetailedError;

--- a/src/file-engine/src/manifest.rs
+++ b/src/file-engine/src/manifest.rs
@@ -46,7 +46,7 @@ impl FileRegionManifest {
    pub async fn store(&self, region_dir: &str, object_store: &ObjectStore) -> Result<()> {
        let path = &region_manifest_path(region_dir);
        let exist = object_store
-            .is_exist(path)
+            .exists(path)
            .await
            .context(CheckObjectSnafu { path })?;
        ensure!(!exist, ManifestExistsSnafu { path });
--- a/src/file-engine/src/region.rs
+++ b/src/file-engine/src/region.rs
@@ -130,7 +130,7 @@ mod tests {
        assert_eq!(region.metadata.primary_key, vec![1]);

        assert!(object_store
-            .is_exist("create_region_dir/manifest/_file_manifest")
+            .exists("create_region_dir/manifest/_file_manifest")
            .await
            .unwrap());

@@ -198,13 +198,13 @@ mod tests {
            .unwrap();

        assert!(object_store
-            .is_exist("drop_region_dir/manifest/_file_manifest")
+            .exists("drop_region_dir/manifest/_file_manifest")
            .await
            .unwrap());

        FileRegion::drop(&region, &object_store).await.unwrap();
        assert!(!object_store
-            .is_exist("drop_region_dir/manifest/_file_manifest")
+            .exists("drop_region_dir/manifest/_file_manifest")
            .await
            .unwrap());

--- a/src/flow/Cargo.toml
+++ b/src/flow/Cargo.toml
@@ -40,8 +40,6 @@ datatypes.workspace = true
 enum-as-inner = "0.6.0"
 enum_dispatch = "0.3"
 futures = "0.3"
-get-size-derive2 = "0.1.2"
-get-size2 = "0.1.2"
 greptime-proto.workspace = true
 # This fork of hydroflow is simply for keeping our dependency in our org, and pin the version
 # otherwise it is the same with upstream repo
--- a/src/flow/src/adapter.rs
+++ b/src/flow/src/adapter.rs
@@ -60,7 +60,6 @@ use crate::repr::{self, DiffRow, Row, BATCH_SIZE};

 mod flownode_impl;
 mod parse_expr;
-mod stat;
 #[cfg(test)]
 mod tests;
 mod util;
@@ -70,7 +69,6 @@ pub(crate) mod node_context;
 mod table_source;

 use crate::error::Error;
-use crate::utils::StateReportHandler;
 use crate::FrontendInvoker;

 // `GREPTIME_TIMESTAMP` is not used to distinguish when table is created automatically by flow
@@ -139,8 +137,6 @@ pub struct FlowWorkerManager {
    ///
    /// So that a series of event like `inserts -> flush` can be handled correctly
    flush_lock: RwLock<()>,
-    /// receive a oneshot sender to send state size report
-    state_report_handler: RwLock<Option<StateReportHandler>>,
 }

 /// Building FlownodeManager
@@ -174,15 +170,9 @@ impl FlowWorkerManager {
            tick_manager,
            node_id,
            flush_lock: RwLock::new(()),
-            state_report_handler: RwLock::new(None),
        }
    }

-    pub async fn with_state_report_handler(self, handler: StateReportHandler) -> Self {
-        *self.state_report_handler.write().await = Some(handler);
-        self
-    }
-
    /// Create a flownode manager with one worker
    pub fn new_with_worker<'s>(
        node_id: Option<u32>,
@@ -510,27 +500,6 @@ impl FlowWorkerManager {

 /// Flow Runtime related methods
 impl FlowWorkerManager {
-    /// Start state report handler, which will receive a sender from HeartbeatTask to send state size report back
-    ///
-    /// if heartbeat task is shutdown, this future will exit too
-    async fn start_state_report_handler(self: Arc<Self>) -> Option<JoinHandle<()>> {
-        let state_report_handler = self.state_report_handler.write().await.take();
-        if let Some(mut handler) = state_report_handler {
-            let zelf = self.clone();
-            let handler = common_runtime::spawn_global(async move {
-                while let Some(ret_handler) = handler.recv().await {
-                    let state_report = zelf.gen_state_report().await;
-                    ret_handler.send(state_report).unwrap_or_else(|err| {
-                        common_telemetry::error!(err; "Send state size report error");
-                    });
-                }
-            });
-            Some(handler)
-        } else {
-            None
-        }
-    }
-
    /// run in common_runtime background runtime
    pub fn run_background(
        self: Arc<Self>,
@@ -538,7 +507,6 @@ impl FlowWorkerManager {
    ) -> JoinHandle<()> {
        info!("Starting flownode manager's background task");
        common_runtime::spawn_global(async move {
-            let _state_report_handler = self.clone().start_state_report_handler().await;
            self.run(shutdown).await;
        })
    }
@@ -565,8 +533,6 @@ impl FlowWorkerManager {
        let default_interval = Duration::from_secs(1);
        let mut avg_spd = 0; // rows/sec
        let mut since_last_run = tokio::time::Instant::now();
-        let run_per_trace = 10;
-        let mut run_cnt = 0;
        loop {
            // TODO(discord9): only run when new inputs arrive or scheduled to
            let row_cnt = self.run_available(true).await.unwrap_or_else(|err| {
@@ -609,19 +575,10 @@ impl FlowWorkerManager {
            } else {
                (9 * avg_spd + cur_spd) / 10
            };
+            trace!("avg_spd={} r/s, cur_spd={} r/s", avg_spd, cur_spd);
            let new_wait = BATCH_SIZE * 1000 / avg_spd.max(1); //in ms
            let new_wait = Duration::from_millis(new_wait as u64).min(default_interval);
-
-            // print trace every `run_per_trace` times so that we can see if there is something wrong
-            // but also not get flooded with trace
-            if run_cnt >= run_per_trace {
-                trace!("avg_spd={} r/s, cur_spd={} r/s", avg_spd, cur_spd);
-                trace!("Wait for {} ms, row_cnt={}", new_wait.as_millis(), row_cnt);
-                run_cnt = 0;
-            } else {
-                run_cnt += 1;
-            }
-
+            trace!("Wait for {} ms, row_cnt={}", new_wait.as_millis(), row_cnt);
            METRIC_FLOW_RUN_INTERVAL_MS.set(new_wait.as_millis() as i64);
            since_last_run = tokio::time::Instant::now();
            tokio::time::sleep(new_wait).await;
@@ -681,18 +638,13 @@ impl FlowWorkerManager {
        &self,
        region_id: RegionId,
        rows: Vec<DiffRow>,
-        batch_datatypes: &[ConcreteDataType],
    ) -> Result<(), Error> {
        let rows_len = rows.len();
        let table_id = region_id.table_id();
        let _timer = METRIC_FLOW_INSERT_ELAPSED
            .with_label_values(&[table_id.to_string().as_str()])
            .start_timer();
-        self.node_context
-            .read()
-            .await
-            .send(table_id, rows, batch_datatypes)
-            .await?;
+        self.node_context.read().await.send(table_id, rows).await?;
        trace!(
            "Handling write request for table_id={} with {} rows",
            table_id,
--- a/src/flow/src/adapter/flownode_impl.rs
+++ b/src/flow/src/adapter/flownode_impl.rs
@@ -28,7 +28,6 @@ use itertools::Itertools;
 use snafu::{OptionExt, ResultExt};
 use store_api::storage::RegionId;

-use super::util::from_proto_to_data_type;
 use crate::adapter::{CreateFlowArgs, FlowWorkerManager};
 use crate::error::InternalSnafu;
 use crate::metrics::METRIC_FLOW_TASK_COUNT;
@@ -207,17 +206,9 @@ impl Flownode for FlowWorkerManager {
                })
                .map(|r| (r, now, 1))
                .collect_vec();
-            let batch_datatypes = insert_schema
-                .iter()
-                .map(from_proto_to_data_type)
-                .collect::<std::result::Result<Vec<_>, _>>()
-                .map_err(to_meta_err)?;
-            self.handle_write_request(region_id.into(), rows, &batch_datatypes)
+            self.handle_write_request(region_id.into(), rows)
                .await
-                .map_err(|err| {
-                    common_telemetry::error!(err;"Failed to handle write request");
-                    to_meta_err(err)
-                })?;
+                .map_err(to_meta_err)?;
        }
        Ok(Default::default())
    }
--- a/src/flow/src/adapter/node_context.rs
+++ b/src/flow/src/adapter/node_context.rs
@@ -19,7 +19,6 @@ use std::sync::atomic::{AtomicUsize, Ordering};
 use std::sync::Arc;

 use common_telemetry::trace;
-use datatypes::prelude::ConcreteDataType;
 use session::context::QueryContext;
 use snafu::{OptionExt, ResultExt};
 use table::metadata::TableId;
@@ -132,11 +131,7 @@ impl SourceSender {
    }

    /// return number of rows it actual send(including what's in the buffer)
-    pub async fn send_rows(
-        &self,
-        rows: Vec<DiffRow>,
-        batch_datatypes: &[ConcreteDataType],
-    ) -> Result<usize, Error> {
+    pub async fn send_rows(&self, rows: Vec<DiffRow>) -> Result<usize, Error> {
        METRIC_FLOW_INPUT_BUF_SIZE.add(rows.len() as _);
        while self.send_buf_row_cnt.load(Ordering::SeqCst) >= BATCH_SIZE * 4 {
            tokio::task::yield_now().await;
@@ -144,11 +139,8 @@ impl SourceSender {
        // row count metrics is approx so relaxed order is ok
        self.send_buf_row_cnt
            .fetch_add(rows.len(), Ordering::SeqCst);
-        let batch = Batch::try_from_rows_with_types(
-            rows.into_iter().map(|(row, _, _)| row).collect(),
-            batch_datatypes,
-        )
-        .context(EvalSnafu)?;
+        let batch = Batch::try_from_rows(rows.into_iter().map(|(row, _, _)| row).collect())
+            .context(EvalSnafu)?;
        common_telemetry::trace!("Send one batch to worker with {} rows", batch.row_count());
        self.send_buf_tx.send(batch).await.map_err(|e| {
            crate::error::InternalSnafu {
@@ -165,19 +157,14 @@ impl FlownodeContext {
    /// return number of rows it actual send(including what's in the buffer)
    ///
    /// TODO(discord9): make this concurrent
-    pub async fn send(
-        &self,
-        table_id: TableId,
-        rows: Vec<DiffRow>,
-        batch_datatypes: &[ConcreteDataType],
-    ) -> Result<usize, Error> {
+    pub async fn send(&self, table_id: TableId, rows: Vec<DiffRow>) -> Result<usize, Error> {
        let sender = self
            .source_sender
            .get(&table_id)
            .with_context(|| TableNotFoundSnafu {
                name: table_id.to_string(),
            })?;
-        sender.send_rows(rows, batch_datatypes).await
+        sender.send_rows(rows).await
    }

    /// flush all sender's buf
--- a/src/flow/src/adapter/stat.rs
+++ b/src/flow/src/adapter/stat.rs
@@ -1,40 +0,0 @@
-// Copyright 2023 Greptime Team
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-use std::collections::BTreeMap;
-
-use common_meta::key::flow::flow_state::FlowStat;
-
-use crate::FlowWorkerManager;
-
-impl FlowWorkerManager {
-    pub async fn gen_state_report(&self) -> FlowStat {
-        let mut full_report = BTreeMap::new();
-        for worker in self.worker_handles.iter() {
-            let worker = worker.lock().await;
-            match worker.get_state_size().await {
-                Ok(state_size) => {
-                    full_report.extend(state_size.into_iter().map(|(k, v)| (k as u32, v)))
-                }
-                Err(err) => {
-                    common_telemetry::error!(err; "Get flow stat size error");
-                }
-            }
-        }
-
-        FlowStat {
-            state_size: full_report,
-        }
-    }
-}
--- a/src/flow/src/adapter/util.rs
+++ b/src/flow/src/adapter/util.rs
@@ -16,27 +16,12 @@ use api::helper::ColumnDataTypeWrapper;
 use api::v1::column_def::options_from_column_schema;
 use api::v1::{ColumnDataType, ColumnDataTypeExtension, SemanticType};
 use common_error::ext::BoxedError;
-use datatypes::prelude::ConcreteDataType;
 use datatypes::schema::ColumnSchema;
 use itertools::Itertools;
 use snafu::ResultExt;

 use crate::error::{Error, ExternalSnafu};

-pub fn from_proto_to_data_type(
-    column_schema: &api::v1::ColumnSchema,
-) -> Result<ConcreteDataType, Error> {
-    let wrapper = ColumnDataTypeWrapper::try_new(
-        column_schema.datatype,
-        column_schema.datatype_extension.clone(),
-    )
-    .map_err(BoxedError::new)
-    .context(ExternalSnafu)?;
-    let cdt = ConcreteDataType::from(wrapper);
-
-    Ok(cdt)
-}
-
 /// convert `ColumnSchema` lists to it's corresponding proto type
 pub fn column_schemas_to_proto(
    column_schemas: Vec<ColumnSchema>,
--- a/src/flow/src/adapter/worker.rs
+++ b/src/flow/src/adapter/worker.rs
@@ -197,21 +197,6 @@ impl WorkerHandle {
            .fail()
        }
    }
-
-    pub async fn get_state_size(&self) -> Result<BTreeMap<FlowId, usize>, Error> {
-        let ret = self
-            .itc_client
-            .call_with_resp(Request::QueryStateSize)
-            .await?;
-        ret.into_query_state_size().map_err(|ret| {
-            InternalSnafu {
-                reason: format!(
-                    "Flow Node/Worker itc failed, expect Response::QueryStateSize, found {ret:?}"
-                ),
-            }
-            .build()
-        })
-    }
 }

 impl Drop for WorkerHandle {
@@ -376,13 +361,6 @@ impl<'s> Worker<'s> {
                Some(Response::ContainTask { result: ret })
            }
            Request::Shutdown => return Err(()),
-            Request::QueryStateSize => {
-                let mut ret = BTreeMap::new();
-                for (flow_id, task_state) in self.task_states.iter() {
-                    ret.insert(*flow_id, task_state.state.get_state_size());
-                }
-                Some(Response::QueryStateSize { result: ret })
-            }
        };
        Ok(ret)
    }
@@ -413,7 +391,6 @@ pub enum Request {
        flow_id: FlowId,
    },
    Shutdown,
-    QueryStateSize,
 }

 #[derive(Debug, EnumAsInner)]
@@ -429,10 +406,6 @@ enum Response {
        result: bool,
    },
    RunAvail,
-    QueryStateSize {
-        /// each flow tasks' state size
-        result: BTreeMap<FlowId, usize>,
-    },
 }

 fn create_inter_thread_call() -> (InterThreadCallClient, InterThreadCallServer) {
@@ -450,12 +423,10 @@ struct InterThreadCallClient {
 }

 impl InterThreadCallClient {
-    /// call without response
    fn call_no_resp(&self, req: Request) -> Result<(), Error> {
        self.arg_sender.send((req, None)).map_err(from_send_error)
    }

-    /// call with response
    async fn call_with_resp(&self, req: Request) -> Result<Response, Error> {
        let (tx, rx) = oneshot::channel();
        self.arg_sender
@@ -556,7 +527,6 @@ mod test {
        );
        tx.send(Batch::empty()).unwrap();
        handle.run_available(0, true).await.unwrap();
-        assert_eq!(handle.get_state_size().await.unwrap().len(), 1);
        assert_eq!(sink_rx.recv().await.unwrap(), Batch::empty());
        drop(handle);
        worker_thread_handle.join().unwrap();
--- a/src/flow/src/compute/render.rs
+++ b/src/flow/src/compute/render.rs
@@ -30,7 +30,7 @@ use crate::compute::types::{Collection, CollectionBundle, ErrCollector, Toff};
 use crate::error::{Error, InvalidQuerySnafu, NotImplementedSnafu};
 use crate::expr::{self, Batch, GlobalId, LocalId};
 use crate::plan::{Plan, TypedPlan};
-use crate::repr::{self, DiffRow, RelationType};
+use crate::repr::{self, DiffRow};

 mod map;
 mod reduce;
@@ -124,10 +124,10 @@ impl Context<'_, '_> {
    /// Like `render_plan` but in Batch Mode
    pub fn render_plan_batch(&mut self, plan: TypedPlan) -> Result<CollectionBundle<Batch>, Error> {
        match plan.plan {
-            Plan::Constant { rows } => Ok(self.render_constant_batch(rows, &plan.schema.typ)),
+            Plan::Constant { rows } => Ok(self.render_constant_batch(rows)),
            Plan::Get { id } => self.get_batch_by_id(id),
            Plan::Let { id, value, body } => self.eval_batch_let(id, value, body),
-            Plan::Mfp { input, mfp } => self.render_mfp_batch(input, mfp, &plan.schema.typ),
+            Plan::Mfp { input, mfp } => self.render_mfp_batch(input, mfp),
            Plan::Reduce {
                input,
                key_val_plan,
@@ -172,11 +172,7 @@ impl Context<'_, '_> {
    /// render Constant, take all rows that have a timestamp not greater than the current time
    /// This function is primarily used for testing
    /// Always assume input is sorted by timestamp
-    pub fn render_constant_batch(
-        &mut self,
-        rows: Vec<DiffRow>,
-        output_type: &RelationType,
-    ) -> CollectionBundle<Batch> {
+    pub fn render_constant_batch(&mut self, rows: Vec<DiffRow>) -> CollectionBundle<Batch> {
        let (send_port, recv_port) = self.df.make_edge::<_, Toff<Batch>>("constant_batch");
        let mut per_time: BTreeMap<repr::Timestamp, Vec<DiffRow>> = Default::default();
        for (key, group) in &rows.into_iter().group_by(|(_row, ts, _diff)| *ts) {
@@ -189,8 +185,6 @@ impl Context<'_, '_> {
        let scheduler_inner = scheduler.clone();
        let err_collector = self.err_collector.clone();

-        let output_type = output_type.clone();
-
        let subgraph_id =
            self.df
                .add_subgraph_source("ConstantBatch", send_port, move |_ctx, send_port| {
@@ -205,14 +199,7 @@ impl Context<'_, '_> {
                    not_great_than_now.into_iter().for_each(|(_ts, rows)| {
                        err_collector.run(|| {
                            let rows = rows.into_iter().map(|(row, _ts, _diff)| row).collect();
-                            let batch = Batch::try_from_rows_with_types(
-                                rows,
-                                &output_type
-                                    .column_types
-                                    .iter()
-                                    .map(|ty| ty.scalar_type().clone())
-                                    .collect_vec(),
-                            )?;
+                            let batch = Batch::try_from_rows(rows)?;
                            send_port.give(vec![batch]);
                            Ok(())
                        });
--- a/src/flow/src/compute/render/map.rs
+++ b/src/flow/src/compute/render/map.rs
@@ -25,7 +25,7 @@ use crate::compute::types::{Arranged, Collection, CollectionBundle, ErrCollector
 use crate::error::{Error, PlanSnafu};
 use crate::expr::{Batch, EvalError, MapFilterProject, MfpPlan, ScalarExpr};
 use crate::plan::TypedPlan;
-use crate::repr::{self, DiffRow, KeyValDiffRow, RelationType, Row};
+use crate::repr::{self, DiffRow, KeyValDiffRow, Row};
 use crate::utils::ArrangeHandler;

 impl Context<'_, '_> {
@@ -34,7 +34,6 @@ impl Context<'_, '_> {
        &mut self,
        input: Box<TypedPlan>,
        mfp: MapFilterProject,
-        _output_type: &RelationType,
    ) -> Result<CollectionBundle<Batch>, Error> {
        let input = self.render_plan_batch(*input)?;

--- a/src/flow/src/compute/render/reduce.rs
+++ b/src/flow/src/compute/render/reduce.rs
@@ -87,8 +87,6 @@ impl Context<'_, '_> {
        })?;
        let key_val_plan = key_val_plan.clone();

-        let output_type = output_type.clone();
-
        let now = self.compute_state.current_time_ref();

        let err_collector = self.err_collector.clone();
@@ -120,7 +118,6 @@ impl Context<'_, '_> {
                    src_data,
                    &key_val_plan,
                    &accum_plan,
-                    &output_type,
                    SubgraphArg {
                        now,
                        err_collector: &err_collector,
@@ -357,7 +354,6 @@ fn reduce_batch_subgraph(
    src_data: impl IntoIterator<Item = Batch>,
    key_val_plan: &KeyValPlan,
    accum_plan: &AccumulablePlan,
-    output_type: &RelationType,
    SubgraphArg {
        now,
        err_collector,
@@ -539,13 +535,17 @@ fn reduce_batch_subgraph(
    // this output part is not supposed to be resource intensive
    // (because for every batch there wouldn't usually be as many output row?),
    // so we can do some costly operation here
-    let output_types = output_type
-        .column_types
-        .iter()
-        .map(|t| t.scalar_type.clone())
-        .collect_vec();
+    let output_types = all_output_dict.first_entry().map(|entry| {
+        entry
+            .key()
+            .iter()
+            .chain(entry.get().iter())
+            .map(|v| v.data_type())
+            .collect::<Vec<ConcreteDataType>>()
+    });

-    err_collector.run(|| {
+    if let Some(output_types) = output_types {
+        err_collector.run(|| {
            let column_cnt = output_types.len();
            let row_cnt = all_output_dict.len();

@@ -585,6 +585,7 @@ fn reduce_batch_subgraph(

            Ok(())
        });
+    }
 }

 /// reduce subgraph, reduce the input data into a single row
@@ -1515,9 +1516,7 @@ mod test {
        let mut ctx = harness_test_ctx(&mut df, &mut state);

        let rows = vec![
-            (Row::new(vec![Value::Null]), -1, 1),
-            (Row::new(vec![1i64.into()]), 0, 1),
-            (Row::new(vec![Value::Null]), 1, 1),
+            (Row::new(vec![1i64.into()]), 1, 1),
            (Row::new(vec![2i64.into()]), 2, 1),
            (Row::new(vec![3i64.into()]), 3, 1),
            (Row::new(vec![1i64.into()]), 4, 1),
@@ -1559,15 +1558,13 @@ mod test {
                Box::new(input_plan.with_types(typ.into_unnamed())),
                &key_val_plan,
                &reduce_plan,
-                &RelationType::new(vec![ColumnType::new(CDT::int64_datatype(), true)]),
+                &RelationType::empty(),
            )
            .unwrap();

        {
            let now_inner = now.clone();
            let expected = BTreeMap::<i64, Vec<i64>>::from([
-                (-1, vec![]),
-                (0, vec![1i64]),
                (1, vec![1i64]),
                (2, vec![3i64]),
                (3, vec![6i64]),
@@ -1584,11 +1581,7 @@ mod test {

                    if let Some(expected) = expected.get(&now) {
                        let batch = expected.iter().map(|v| Value::from(*v)).collect_vec();
-                        let batch = Batch::try_from_rows_with_types(
-                            vec![batch.into()],
-                            &[CDT::int64_datatype()],
-                        )
-                        .unwrap();
+                        let batch = Batch::try_from_rows(vec![batch.into()]).unwrap();
                        assert_eq!(res.first(), Some(&batch));
                    }
                });
--- a/src/flow/src/compute/state.rs
+++ b/src/flow/src/compute/state.rs
@@ -16,7 +16,6 @@ use std::cell::RefCell;
 use std::collections::{BTreeMap, VecDeque};
 use std::rc::Rc;

-use get_size2::GetSize;
 use hydroflow::scheduled::graph::Hydroflow;
 use hydroflow::scheduled::SubgraphId;

@@ -110,10 +109,6 @@ impl DataflowState {
    pub fn expire_after(&self) -> Option<Timestamp> {
        self.expire_after
    }
-
-    pub fn get_state_size(&self) -> usize {
-        self.arrange_used.iter().map(|x| x.read().get_size()).sum()
-    }
 }

 #[derive(Debug, Clone)]
--- a/src/flow/src/expr.rs
+++ b/src/flow/src/expr.rs
@@ -24,7 +24,7 @@ mod scalar;
 mod signature;

 use arrow::compute::FilterBuilder;
-use datatypes::prelude::{ConcreteDataType, DataType};
+use datatypes::prelude::DataType;
 use datatypes::value::Value;
 use datatypes::vectors::{BooleanVector, Helper, VectorRef};
 pub(crate) use df_func::{DfScalarFunction, RawDfScalarFn};
@@ -85,18 +85,16 @@ impl Default for Batch {
 }

 impl Batch {
-    /// Get batch from rows, will try best to determine data type
-    pub fn try_from_rows_with_types(
-        rows: Vec<crate::repr::Row>,
-        batch_datatypes: &[ConcreteDataType],
-    ) -> Result<Self, EvalError> {
+    pub fn try_from_rows(rows: Vec<crate::repr::Row>) -> Result<Self, EvalError> {
        if rows.is_empty() {
            return Ok(Self::empty());
        }
        let len = rows.len();
-        let mut builder = batch_datatypes
+        let mut builder = rows
+            .first()
+            .unwrap()
            .iter()
-            .map(|ty| ty.create_mutable_vector(len))
+            .map(|v| v.data_type().create_mutable_vector(len))
            .collect_vec();
        for row in rows {
            ensure!(
@@ -223,25 +221,10 @@ impl Batch {
            return Ok(());
        }

-        let dts = {
-            let max_len = self.batch.len().max(other.batch.len());
-            let mut dts = Vec::with_capacity(max_len);
-            for i in 0..max_len {
-                if let Some(v) = self.batch().get(i)
-                    && !v.data_type().is_null()
-                {
-                    dts.push(v.data_type())
-                } else if let Some(v) = other.batch().get(i)
-                    && !v.data_type().is_null()
-                {
-                    dts.push(v.data_type())
-                } else {
-                    // both are null, so we will push null type
-                    dts.push(datatypes::prelude::ConcreteDataType::null_datatype())
-                }
-            }
-
-            dts
+        let dts = if self.batch.is_empty() {
+            other.batch.iter().map(|v| v.data_type()).collect_vec()
+        } else {
+            self.batch.iter().map(|v| v.data_type()).collect_vec()
        };

        let batch_builders = dts
--- a/src/flow/src/expr/linear.rs
+++ b/src/flow/src/expr/linear.rs
@@ -908,33 +908,20 @@ mod test {
            .unwrap()
            .unwrap();
        assert_eq!(ret, Row::pack(vec![Value::from(false), Value::from(true)]));
-        let ty = [
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-        ];
+
        // batch mode
-        let mut batch = Batch::try_from_rows_with_types(
-            vec![Row::from(vec![
-                Value::from(4),
-                Value::from(2),
-                Value::from(3),
-            ])],
-            &ty,
-        )
+        let mut batch = Batch::try_from_rows(vec![Row::from(vec![
+            Value::from(4),
+            Value::from(2),
+            Value::from(3),
+        ])])
        .unwrap();
        let ret = safe_mfp.eval_batch_into(&mut batch).unwrap();

        assert_eq!(
            ret,
-            Batch::try_from_rows_with_types(
-                vec![Row::from(vec![Value::from(false), Value::from(true)])],
-                &[
-                    ConcreteDataType::boolean_datatype(),
-                    ConcreteDataType::boolean_datatype(),
-                ],
-            )
-            .unwrap()
+            Batch::try_from_rows(vec![Row::from(vec![Value::from(false), Value::from(true)])])
+                .unwrap()
        );
    }

@@ -969,15 +956,7 @@ mod test {
            .unwrap();
        assert_eq!(ret, None);

-        let input_type = [
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::string_datatype(),
-        ];
-
-        let mut input1_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input1)], &input_type).unwrap();
+        let mut input1_batch = Batch::try_from_rows(vec![Row::new(input1)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input1_batch).unwrap();
        assert_eq!(
            ret_batch,
@@ -995,8 +974,7 @@ mod test {
            .unwrap();
        assert_eq!(ret, Some(Row::pack(vec![Value::from(11)])));

-        let mut input2_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input2)], &input_type).unwrap();
+        let mut input2_batch = Batch::try_from_rows(vec![Row::new(input2)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input2_batch).unwrap();
        assert_eq!(
            ret_batch,
@@ -1049,14 +1027,7 @@ mod test {
        let ret = safe_mfp.evaluate_into(&mut input1.clone(), &mut Row::empty());
        assert!(matches!(ret, Err(EvalError::InvalidArgument { .. })));

-        let input_type = [
-            ConcreteDataType::int64_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-        ];
-        let mut input1_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input1)], &input_type).unwrap();
+        let mut input1_batch = Batch::try_from_rows(vec![Row::new(input1)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input1_batch);
        assert!(matches!(ret_batch, Err(EvalError::InvalidArgument { .. })));

@@ -1066,13 +1037,7 @@ mod test {
            .unwrap();
        assert_eq!(ret, Some(Row::new(input2.clone())));

-        let input_type = [
-            ConcreteDataType::int64_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-        ];
-        let input2_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input2)], &input_type).unwrap();
+        let input2_batch = Batch::try_from_rows(vec![Row::new(input2)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input2_batch.clone()).unwrap();
        assert_eq!(ret_batch, input2_batch);

@@ -1082,8 +1047,7 @@ mod test {
            .unwrap();
        assert_eq!(ret, None);

-        let input3_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input3)], &input_type).unwrap();
+        let input3_batch = Batch::try_from_rows(vec![Row::new(input3)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input3_batch.clone()).unwrap();
        assert_eq!(
            ret_batch,
@@ -1119,13 +1083,7 @@ mod test {
        let ret = safe_mfp.evaluate_into(&mut input1.clone(), &mut Row::empty());
        assert_eq!(ret.unwrap(), Some(Row::new(vec![Value::from(false)])));

-        let input_type = [
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-            ConcreteDataType::int32_datatype(),
-        ];
-        let mut input1_batch =
-            Batch::try_from_rows_with_types(vec![Row::new(input1)], &input_type).unwrap();
+        let mut input1_batch = Batch::try_from_rows(vec![Row::new(input1)]).unwrap();
        let ret_batch = safe_mfp.eval_batch_into(&mut input1_batch).unwrap();

        assert_eq!(
--- a/src/flow/src/heartbeat.rs
+++ b/src/flow/src/heartbeat.rs
@@ -24,7 +24,6 @@ use common_meta::heartbeat::handler::{
 };
 use common_meta::heartbeat::mailbox::{HeartbeatMailbox, MailboxRef, OutgoingMessage};
 use common_meta::heartbeat::utils::outgoing_message_to_mailbox_message;
-use common_meta::key::flow::flow_state::FlowStat;
 use common_telemetry::{debug, error, info, warn};
 use greptime_proto::v1::meta::NodeInfo;
 use meta_client::client::{HeartbeatSender, HeartbeatStream, MetaClient};
@@ -35,27 +34,8 @@ use tokio::sync::mpsc;
 use tokio::time::Duration;

 use crate::error::ExternalSnafu;
-use crate::utils::SizeReportSender;
 use crate::{Error, FlownodeOptions};

-async fn query_flow_state(
-    query_stat_size: &Option<SizeReportSender>,
-    timeout: Duration,
-) -> Option<FlowStat> {
-    if let Some(report_requester) = query_stat_size.as_ref() {
-        let ret = report_requester.query(timeout).await;
-        match ret {
-            Ok(latest) => Some(latest),
-            Err(err) => {
-                error!(err; "Failed to get query stat size");
-                None
-            }
-        }
-    } else {
-        None
-    }
-}
-
 /// The flownode heartbeat task which sending `[HeartbeatRequest]` to Metasrv periodically in background.
 #[derive(Clone)]
 pub struct HeartbeatTask {
@@ -67,14 +47,9 @@ pub struct HeartbeatTask {
    resp_handler_executor: HeartbeatResponseHandlerExecutorRef,
    start_time_ms: u64,
    running: Arc<AtomicBool>,
-    query_stat_size: Option<SizeReportSender>,
 }

 impl HeartbeatTask {
-    pub fn with_query_stat_size(mut self, query_stat_size: SizeReportSender) -> Self {
-        self.query_stat_size = Some(query_stat_size);
-        self
-    }
    pub fn new(
        opts: &FlownodeOptions,
        meta_client: Arc<MetaClient>,
@@ -90,7 +65,6 @@ impl HeartbeatTask {
            resp_handler_executor,
            start_time_ms: common_time::util::current_time_millis() as u64,
            running: Arc::new(AtomicBool::new(false)),
-            query_stat_size: None,
        }
    }

@@ -138,7 +112,6 @@ impl HeartbeatTask {
        message: Option<OutgoingMessage>,
        peer: Option<Peer>,
        start_time_ms: u64,
-        latest_report: &Option<FlowStat>,
    ) -> Option<HeartbeatRequest> {
        let mailbox_message = match message.map(outgoing_message_to_mailbox_message) {
            Some(Ok(message)) => Some(message),
@@ -148,22 +121,11 @@ impl HeartbeatTask {
            }
            None => None,
        };
-        let flow_stat = latest_report
-            .as_ref()
-            .map(|report| {
-                report
-                    .state_size
-                    .iter()
-                    .map(|(k, v)| (*k, *v as u64))
-                    .collect()
-            })
-            .map(|f| api::v1::meta::FlowStat { flow_stat_size: f });

        Some(HeartbeatRequest {
            mailbox_message,
            peer,
            info: Self::build_node_info(start_time_ms),
-            flow_stat,
            ..Default::default()
        })
    }
@@ -189,27 +151,24 @@ impl HeartbeatTask {
            addr: self.peer_addr.clone(),
        });

-        let query_stat_size = self.query_stat_size.clone();
-
        common_runtime::spawn_hb(async move {
            // note that using interval will cause it to first immediately send
            // a heartbeat
            let mut interval = tokio::time::interval(report_interval);
            interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
-            let mut latest_report = None;

            loop {
                let req = tokio::select! {
                    message = outgoing_rx.recv() => {
                        if let Some(message) = message {
-                            Self::create_heartbeat_request(Some(message), self_peer.clone(), start_time_ms, &latest_report)
+                            Self::create_heartbeat_request(Some(message), self_peer.clone(), start_time_ms)
                        } else {
                            // Receives None that means Sender was dropped, we need to break the current loop
                            break
                        }
                    }
                    _ = interval.tick() => {
-                        Self::create_heartbeat_request(None, self_peer.clone(), start_time_ms, &latest_report)
+                        Self::create_heartbeat_request(None, self_peer.clone(), start_time_ms)
                    }
                };

@@ -221,10 +180,6 @@ impl HeartbeatTask {
                        debug!("Send a heartbeat request to metasrv, content: {:?}", req);
                    }
                }
-                // after sending heartbeat, try to get the latest report
-                // TODO(discord9): consider a better place to update the size report
-                // set the timeout to half of the report interval so that it wouldn't delay heartbeat if something went horribly wrong
-                latest_report = query_flow_state(&query_stat_size, report_interval / 2).await;
            }
        });
    }
--- a/src/flow/src/repr.rs
+++ b/src/flow/src/repr.rs
@@ -22,14 +22,12 @@ use api::v1::Row as ProtoRow;
 use datatypes::data_type::ConcreteDataType;
 use datatypes::types::cast;
 use datatypes::value::Value;
-use get_size2::GetSize;
 use itertools::Itertools;
 pub(crate) use relation::{ColumnType, Key, RelationDesc, RelationType};
 use serde::{Deserialize, Serialize};
 use snafu::ResultExt;

 use crate::expr::error::{CastValueSnafu, EvalError, InvalidArgumentSnafu};
-use crate::utils::get_value_heap_size;

 /// System-wide Record count difference type. Useful for capture data change
 ///
@@ -107,12 +105,6 @@ pub struct Row {
    pub inner: Vec<Value>,
 }

-impl GetSize for Row {
-    fn get_heap_size(&self) -> usize {
-        self.inner.iter().map(get_value_heap_size).sum()
-    }
-}
-
 impl Row {
    /// Create an empty row
    pub fn empty() -> Self {
--- a/src/flow/src/server.rs
+++ b/src/flow/src/server.rs
@@ -55,7 +55,6 @@ use crate::error::{
 };
 use crate::heartbeat::HeartbeatTask;
 use crate::transform::register_function_to_query_engine;
-use crate::utils::{SizeReportSender, StateReportHandler};
 use crate::{Error, FlowWorkerManager, FlownodeOptions};

 pub const FLOW_NODE_SERVER_NAME: &str = "FLOW_NODE_SERVER";
@@ -237,8 +236,6 @@ pub struct FlownodeBuilder {
    catalog_manager: CatalogManagerRef,
    flow_metadata_manager: FlowMetadataManagerRef,
    heartbeat_task: Option<HeartbeatTask>,
-    /// receive a oneshot sender to send state size report
-    state_report_handler: Option<StateReportHandler>,
 }

 impl FlownodeBuilder {
@@ -257,20 +254,17 @@ impl FlownodeBuilder {
            catalog_manager,
            flow_metadata_manager,
            heartbeat_task: None,
-            state_report_handler: None,
        }
    }

    pub fn with_heartbeat_task(self, heartbeat_task: HeartbeatTask) -> Self {
-        let (sender, receiver) = SizeReportSender::new();
        Self {
-            heartbeat_task: Some(heartbeat_task.with_query_stat_size(sender)),
-            state_report_handler: Some(receiver),
+            heartbeat_task: Some(heartbeat_task),
            ..self
        }
    }

-    pub async fn build(mut self) -> Result<FlownodeInstance, Error> {
+    pub async fn build(self) -> Result<FlownodeInstance, Error> {
        // TODO(discord9): does this query engine need those?
        let query_engine_factory = QueryEngineFactory::new_with_plugins(
            // query engine in flownode is only used for translate plan with resolved table source.
@@ -389,7 +383,7 @@ impl FlownodeBuilder {
    /// build [`FlowWorkerManager`], note this doesn't take ownership of `self`,
    /// nor does it actually start running the worker.
    async fn build_manager(
-        &mut self,
+        &self,
        query_engine: Arc<dyn QueryEngine>,
    ) -> Result<FlowWorkerManager, Error> {
        let table_meta = self.table_meta.clone();
@@ -408,15 +402,12 @@ impl FlownodeBuilder {
                info!("Flow Worker started in new thread");
                worker.run();
            });
-        let mut man = rx.await.map_err(|_e| {
+        let man = rx.await.map_err(|_e| {
            UnexpectedSnafu {
                reason: "sender is dropped, failed to create flow node manager",
            }
            .build()
        })?;
-        if let Some(handler) = self.state_report_handler.take() {
-            man = man.with_state_report_handler(handler).await;
-        }
        info!("Flow Node Manager started");
        Ok(man)
    }
--- a/src/flow/src/utils.rs
+++ b/src/flow/src/utils.rs
@@ -18,73 +18,16 @@ use std::collections::{BTreeMap, BTreeSet};
 use std::ops::Bound;
 use std::sync::Arc;

-use common_meta::key::flow::flow_state::FlowStat;
 use common_telemetry::trace;
-use datatypes::value::Value;
-use get_size2::GetSize;
 use smallvec::{smallvec, SmallVec};
-use tokio::sync::{mpsc, oneshot, RwLock};
-use tokio::time::Instant;
+use tokio::sync::RwLock;

-use crate::error::InternalSnafu;
 use crate::expr::{EvalError, ScalarExpr};
 use crate::repr::{value_to_internal_ts, DiffRow, Duration, KeyValDiffRow, Row, Timestamp};

 /// A batch of updates, arranged by key
 pub type Batch = BTreeMap<Row, SmallVec<[DiffRow; 2]>>;

-/// Get a estimate of heap size of a value
-pub fn get_value_heap_size(v: &Value) -> usize {
-    match v {
-        Value::Binary(bin) => bin.len(),
-        Value::String(s) => s.len(),
-        Value::List(list) => list.items().iter().map(get_value_heap_size).sum(),
-        _ => 0,
-    }
-}
-
-#[derive(Clone)]
-pub struct SizeReportSender {
-    inner: mpsc::Sender<oneshot::Sender<FlowStat>>,
-}
-
-impl SizeReportSender {
-    pub fn new() -> (Self, StateReportHandler) {
-        let (tx, rx) = mpsc::channel(1);
-        let zelf = Self { inner: tx };
-        (zelf, rx)
-    }
-
-    /// Query the size report, will timeout after one second if no response
-    pub async fn query(&self, timeout: std::time::Duration) -> crate::Result<FlowStat> {
-        let (tx, rx) = oneshot::channel();
-        self.inner.send(tx).await.map_err(|_| {
-            InternalSnafu {
-                reason: "failed to send size report request due to receiver dropped",
-            }
-            .build()
-        })?;
-        let timeout = tokio::time::timeout(timeout, rx);
-        timeout
-            .await
-            .map_err(|_elapsed| {
-                InternalSnafu {
-                    reason: "failed to receive size report after one second timeout",
-                }
-                .build()
-            })?
-            .map_err(|_| {
-                InternalSnafu {
-                    reason: "failed to receive size report due to sender dropped",
-                }
-                .build()
-            })
-    }
-}
-
-/// Handle the size report request, and send the report back
-pub type StateReportHandler = mpsc::Receiver<oneshot::Sender<FlowStat>>;
-
 /// A spine of batches, arranged by timestamp
 /// TODO(discord9): consider internally index by key, value, and timestamp for faster lookup
 pub type Spine = BTreeMap<Timestamp, Batch>;
@@ -106,24 +49,6 @@ pub struct KeyExpiryManager {
    event_timestamp_from_row: Option<ScalarExpr>,
 }

-impl GetSize for KeyExpiryManager {
-    fn get_heap_size(&self) -> usize {
-        let row_size = if let Some(row_size) = &self
-            .event_ts_to_key
-            .first_key_value()
-            .map(|(_, v)| v.first().get_heap_size())
-        {
-            *row_size
-        } else {
-            0
-        };
-        self.event_ts_to_key
-            .values()
-            .map(|v| v.len() * row_size + std::mem::size_of::<i64>())
-            .sum::<usize>()
-    }
-}
-
 impl KeyExpiryManager {
    pub fn new(
        key_expiration_duration: Option<Duration>,
@@ -229,7 +154,7 @@ impl KeyExpiryManager {
 ///
 /// Note the two way arrow between reduce operator and arrange, it's because reduce operator need to query existing state
 /// and also need to update existing state.
-#[derive(Debug, Clone, Eq, PartialEq, Ord, PartialOrd)]
+#[derive(Debug, Clone, Default, Eq, PartialEq, Ord, PartialOrd)]
 pub struct Arrangement {
    /// A name or identifier for the arrangement which can be used for debugging or logging purposes.
    /// This field is not critical to the functionality but aids in monitoring and management of arrangements.
@@ -271,61 +196,6 @@ pub struct Arrangement {

    /// The time that the last compaction happened, also known as the current time.
    last_compaction_time: Option<Timestamp>,
-
-    /// Estimated size of the arrangement in heap size.
-    estimated_size: usize,
-    last_size_update: Instant,
-    size_update_interval: tokio::time::Duration,
-}
-
-impl Arrangement {
-    fn compute_size(&self) -> usize {
-        self.spine
-            .values()
-            .map(|v| {
-                let per_entry_size = v
-                    .first_key_value()
-                    .map(|(k, v)| {
-                        k.get_heap_size()
-                            + v.len() * v.first().map(|r| r.get_heap_size()).unwrap_or(0)
-                    })
-                    .unwrap_or(0);
-                std::mem::size_of::<i64>() + v.len() * per_entry_size
-            })
-            .sum::<usize>()
-            + self.expire_state.get_heap_size()
-            + self.name.get_heap_size()
-    }
-
-    fn update_and_fetch_size(&mut self) -> usize {
-        if self.last_size_update.elapsed() > self.size_update_interval {
-            self.estimated_size = self.compute_size();
-            self.last_size_update = Instant::now();
-        }
-        self.estimated_size
-    }
-}
-
-impl GetSize for Arrangement {
-    fn get_heap_size(&self) -> usize {
-        self.estimated_size
-    }
-}
-
-impl Default for Arrangement {
-    fn default() -> Self {
-        Self {
-            spine: Default::default(),
-            full_arrangement: false,
-            is_written: false,
-            expire_state: None,
-            last_compaction_time: None,
-            name: Vec::new(),
-            estimated_size: 0,
-            last_size_update: Instant::now(),
-            size_update_interval: tokio::time::Duration::from_secs(3),
-        }
-    }
 }

 impl Arrangement {
@@ -337,9 +207,6 @@ impl Arrangement {
            expire_state: None,
            last_compaction_time: None,
            name,
-            estimated_size: 0,
-            last_size_update: Instant::now(),
-            size_update_interval: tokio::time::Duration::from_secs(3),
        }
    }

@@ -402,7 +269,6 @@ impl Arrangement {
            // without changing the order of updates within same tick
            key_updates.sort_by_key(|(_val, ts, _diff)| *ts);
        }
-        self.update_and_fetch_size();
        Ok(max_expired_by)
    }

@@ -524,7 +390,6 @@ impl Arrangement {

        // insert the compacted batch into spine with key being `now`
        self.spine.insert(now, compacting_batch);
-        self.update_and_fetch_size();
        Ok(max_expired_by)
    }

--- a/src/meta-client/src/client.rs
+++ b/src/meta-client/src/client.rs
@@ -25,7 +25,6 @@ use std::sync::Arc;

 use api::v1::meta::{ProcedureDetailResponse, Role};
 use cluster::Client as ClusterClient;
-pub use cluster::ClusterKvBackend;
 use common_error::ext::BoxedError;
 use common_grpc::channel_manager::{ChannelConfig, ChannelManager};
 use common_meta::cluster::{
@@ -34,8 +33,6 @@ use common_meta::cluster::{
 use common_meta::datanode::{DatanodeStatKey, DatanodeStatValue, RegionStat};
 use common_meta::ddl::{ExecutorContext, ProcedureExecutor};
 use common_meta::error::{self as meta_error, ExternalSnafu, Result as MetaResult};
-use common_meta::key::flow::flow_state::{FlowStat, FlowStateManager};
-use common_meta::kv_backend::KvBackendRef;
 use common_meta::range_stream::PaginationStream;
 use common_meta::rpc::ddl::{SubmitDdlTaskRequest, SubmitDdlTaskResponse};
 use common_meta::rpc::procedure::{
@@ -57,8 +54,7 @@ use store::Client as StoreClient;

 pub use self::heartbeat::{HeartbeatSender, HeartbeatStream};
 use crate::error::{
-    ConvertMetaRequestSnafu, ConvertMetaResponseSnafu, Error, GetFlowStatSnafu, NotStartedSnafu,
-    Result,
+    ConvertMetaRequestSnafu, ConvertMetaResponseSnafu, Error, NotStartedSnafu, Result,
 };

 pub type Id = (u64, u64);
@@ -351,15 +347,6 @@ fn decode_stats(kv: KeyValue) -> MetaResult<DatanodeStatValue> {
 }

 impl MetaClient {
-    pub async fn list_flow_stats(&self) -> Result<Option<FlowStat>> {
-        let cluster_backend = ClusterKvBackend::new(Arc::new(self.cluster_client()?));
-        let cluster_backend = Arc::new(cluster_backend) as KvBackendRef;
-        let flow_state_manager = FlowStateManager::new(cluster_backend);
-        let res = flow_state_manager.get().await.context(GetFlowStatSnafu)?;
-
-        Ok(res.map(|r| r.into()))
-    }
-
    pub fn new(id: Id) -> Self {
        Self {
            id,
--- a/src/meta-client/src/client/cluster.rs
+++ b/src/meta-client/src/client/cluster.rs
@@ -40,8 +40,8 @@ use tonic::Status;
 use crate::client::ask_leader::AskLeader;
 use crate::client::{util, Id};
 use crate::error::{
-    ConvertMetaResponseSnafu, CreateChannelSnafu, Error, IllegalGrpcClientStateSnafu,
-    ReadOnlyKvBackendSnafu, Result, RetryTimesExceededSnafu,
+    ConvertMetaResponseSnafu, CreateChannelSnafu, Error, IllegalGrpcClientStateSnafu, Result,
+    RetryTimesExceededSnafu,
 };

 #[derive(Clone, Debug)]
@@ -308,75 +308,3 @@ impl Inner {
        .map(|res| (res.leader, res.followers))
    }
 }
-
-/// A client for the cluster info. Read only and corresponding to
-/// `in_memory` kvbackend in the meta-srv.
-#[derive(Clone, Debug)]
-pub struct ClusterKvBackend {
-    inner: Arc<Client>,
-}
-
-impl ClusterKvBackend {
-    pub fn new(client: Arc<Client>) -> Self {
-        Self { inner: client }
-    }
-
-    fn unimpl(&self) -> common_meta::error::Error {
-        let ret: common_meta::error::Result<()> = ReadOnlyKvBackendSnafu {
-            name: self.name().to_string(),
-        }
-        .fail()
-        .map_err(BoxedError::new)
-        .context(common_meta::error::ExternalSnafu);
-        ret.unwrap_err()
-    }
-}
-
-impl TxnService for ClusterKvBackend {
-    type Error = common_meta::error::Error;
-}
-
-#[async_trait::async_trait]
-impl KvBackend for ClusterKvBackend {
-    fn name(&self) -> &str {
-        "ClusterKvBackend"
-    }
-
-    fn as_any(&self) -> &dyn Any {
-        self
-    }
-
-    async fn range(&self, req: RangeRequest) -> common_meta::error::Result<RangeResponse> {
-        self.inner
-            .range(req)
-            .await
-            .map_err(BoxedError::new)
-            .context(common_meta::error::ExternalSnafu)
-    }
-
-    async fn batch_get(&self, _: BatchGetRequest) -> common_meta::error::Result<BatchGetResponse> {
-        Err(self.unimpl())
-    }
-
-    async fn put(&self, _: PutRequest) -> common_meta::error::Result<PutResponse> {
-        Err(self.unimpl())
-    }
-
-    async fn batch_put(&self, _: BatchPutRequest) -> common_meta::error::Result<BatchPutResponse> {
-        Err(self.unimpl())
-    }
-
-    async fn delete_range(
-        &self,
-        _: DeleteRangeRequest,
-    ) -> common_meta::error::Result<DeleteRangeResponse> {
-        Err(self.unimpl())
-    }
-
-    async fn batch_delete(
-        &self,
-        _: BatchDeleteRequest,
-    ) -> common_meta::error::Result<BatchDeleteResponse> {
-        Err(self.unimpl())
-    }
-}
--- a/src/meta-client/src/error.rs
+++ b/src/meta-client/src/error.rs
@@ -99,22 +99,8 @@ pub enum Error {
        source: common_meta::error::Error,
    },

-    #[snafu(display("Failed to get flow stat"))]
-    GetFlowStat {
-        #[snafu(implicit)]
-        location: Location,
-        source: common_meta::error::Error,
-    },
-
    #[snafu(display("Retry exceeded max times({}), message: {}", times, msg))]
    RetryTimesExceeded { times: usize, msg: String },
-
-    #[snafu(display("Trying to write to a read-only kv backend: {}", name))]
-    ReadOnlyKvBackend {
-        name: String,
-        #[snafu(implicit)]
-        location: Location,
-    },
 }

 #[allow(dead_code)]
@@ -134,15 +120,13 @@ impl ErrorExt for Error {
            | Error::SendHeartbeat { .. }
            | Error::CreateHeartbeatStream { .. }
            | Error::CreateChannel { .. }
-            | Error::RetryTimesExceeded { .. }
-            | Error::ReadOnlyKvBackend { .. } => StatusCode::Internal,
+            | Error::RetryTimesExceeded { .. } => StatusCode::Internal,

            Error::MetaServer { code, .. } => *code,

            Error::InvalidResponseHeader { source, .. }
            | Error::ConvertMetaRequest { source, .. }
-            | Error::ConvertMetaResponse { source, .. }
-            | Error::GetFlowStat { source, .. } => source.status_code(),
+            | Error::ConvertMetaResponse { source, .. } => source.status_code(),
        }
    }
 }
--- a/src/meta-srv/src/error.rs
+++ b/src/meta-srv/src/error.rs
@@ -716,13 +716,6 @@ pub enum Error {
        #[snafu(implicit)]
        location: Location,
    },
-
-    #[snafu(display("Flow state handler error"))]
-    FlowStateHandler {
-        #[snafu(implicit)]
-        location: Location,
-        source: common_meta::error::Error,
-    },
 }

 impl Error {
@@ -768,8 +761,7 @@ impl ErrorExt for Error {
            | Error::Join { .. }
            | Error::PeerUnavailable { .. }
            | Error::ExceededDeadline { .. }
-            | Error::ChooseItems { .. }
-            | Error::FlowStateHandler { .. } => StatusCode::Internal,
+            | Error::ChooseItems { .. } => StatusCode::Internal,

            Error::Unsupported { .. } => StatusCode::Unsupported,

--- a/src/meta-srv/src/handler.rs
+++ b/src/meta-srv/src/handler.rs
@@ -51,7 +51,6 @@ use tokio::sync::mpsc::Sender;
 use tokio::sync::{oneshot, Notify, RwLock};

 use crate::error::{self, DeserializeFromJsonSnafu, Result, UnexpectedInstructionReplySnafu};
-use crate::handler::flow_state_handler::FlowStateHandler;
 use crate::metasrv::Context;
 use crate::metrics::{METRIC_META_HANDLER_EXECUTE, METRIC_META_HEARTBEAT_CONNECTION_NUM};
 use crate::pubsub::PublisherRef;
@@ -65,7 +64,6 @@ pub mod collect_stats_handler;
 pub mod extract_stat_handler;
 pub mod failure_handler;
 pub mod filter_inactive_region_stats;
-pub mod flow_state_handler;
 pub mod keep_lease_handler;
 pub mod mailbox_handler;
 pub mod on_leader_start_handler;
@@ -484,8 +482,6 @@ pub struct HeartbeatHandlerGroupBuilder {
    /// based on the number of received heartbeats. When the number of heartbeats
    /// reaches this factor, a flush operation is triggered.
    flush_stats_factor: Option<usize>,
-    /// A simple handler for flow internal state report
-    flow_state_handler: Option<FlowStateHandler>,

    /// The plugins.
    plugins: Option<Plugins>,
@@ -503,18 +499,12 @@ impl HeartbeatHandlerGroupBuilder {
            region_failure_handler: None,
            region_lease_handler: None,
            flush_stats_factor: None,
-            flow_state_handler: None,
            plugins: None,
            pushers,
            handlers: vec![],
        }
    }

-    pub fn with_flow_state_handler(mut self, handler: Option<FlowStateHandler>) -> Self {
-        self.flow_state_handler = handler;
-        self
-    }
-
    pub fn with_region_lease_handler(mut self, handler: Option<RegionLeaseHandler>) -> Self {
        self.region_lease_handler = handler;
        self
@@ -574,10 +564,6 @@ impl HeartbeatHandlerGroupBuilder {
        }
        self.add_handler_last(CollectStatsHandler::new(self.flush_stats_factor));

-        if let Some(flow_state_handler) = self.flow_state_handler.take() {
-            self.add_handler_last(flow_state_handler);
-        }
-
        self
    }

--- a/src/meta-srv/src/handler/flow_state_handler.rs
+++ b/src/meta-srv/src/handler/flow_state_handler.rs
@@ -1,58 +0,0 @@
-// Copyright 2023 Greptime Team
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-use api::v1::meta::{FlowStat, HeartbeatRequest, Role};
-use common_meta::key::flow::flow_state::{FlowStateManager, FlowStateValue};
-use snafu::ResultExt;
-
-use crate::error::{FlowStateHandlerSnafu, Result};
-use crate::handler::{HandleControl, HeartbeatAccumulator, HeartbeatHandler};
-use crate::metasrv::Context;
-
-pub struct FlowStateHandler {
-    flow_state_manager: FlowStateManager,
-}
-
-impl FlowStateHandler {
-    pub fn new(flow_state_manager: FlowStateManager) -> Self {
-        Self { flow_state_manager }
-    }
-}
-
-#[async_trait::async_trait]
-impl HeartbeatHandler for FlowStateHandler {
-    fn is_acceptable(&self, role: Role) -> bool {
-        role == Role::Flownode
-    }
-
-    async fn handle(
-        &self,
-        req: &HeartbeatRequest,
-        _ctx: &mut Context,
-        _acc: &mut HeartbeatAccumulator,
-    ) -> Result<HandleControl> {
-        if let Some(FlowStat { flow_stat_size }) = &req.flow_stat {
-            let state_size = flow_stat_size
-                .iter()
-                .map(|(k, v)| (*k, *v as usize))
-                .collect();
-            let value = FlowStateValue::new(state_size);
-            self.flow_state_manager
-                .put(value)
-                .await
-                .context(FlowStateHandlerSnafu)?;
-        }
-        Ok(HandleControl::Continue)
-    }
-}
--- a/src/meta-srv/src/metasrv/builder.rs
+++ b/src/meta-srv/src/metasrv/builder.rs
@@ -26,7 +26,6 @@ use common_meta::ddl::{
 };
 use common_meta::ddl_manager::DdlManager;
 use common_meta::distributed_time_constants;
-use common_meta::key::flow::flow_state::FlowStateManager;
 use common_meta::key::flow::FlowMetadataManager;
 use common_meta::key::maintenance::MaintenanceModeManager;
 use common_meta::key::TableMetadataManager;
@@ -48,7 +47,6 @@ use crate::error::{self, Result};
 use crate::flow_meta_alloc::FlowPeerAllocator;
 use crate::greptimedb_telemetry::get_greptimedb_telemetry_task;
 use crate::handler::failure_handler::RegionFailureHandler;
-use crate::handler::flow_state_handler::FlowStateHandler;
 use crate::handler::region_lease_handler::RegionLeaseHandler;
 use crate::handler::{HeartbeatHandlerGroupBuilder, HeartbeatMailbox, Pushers};
 use crate::lease::MetaPeerLookupService;
@@ -230,7 +228,6 @@ impl MetasrvBuilder {
                peer_allocator,
            ))
        });
-
        let flow_metadata_allocator = {
            // for now flownode just use round-robin selector
            let flow_selector = RoundRobinSelector::new(SelectTarget::Flownode);
@@ -251,9 +248,6 @@ impl MetasrvBuilder {
                peer_allocator,
            ))
        };
-        let flow_state_handler =
-            FlowStateHandler::new(FlowStateManager::new(in_memory.clone().as_kv_backend_ref()));
-
        let memory_region_keeper = Arc::new(MemoryRegionKeeper::default());
        let node_manager = node_manager.unwrap_or_else(|| {
            let datanode_client_channel_config = ChannelConfig::new()
@@ -356,7 +350,6 @@ impl MetasrvBuilder {
                    .with_region_failure_handler(region_failover_handler)
                    .with_region_lease_handler(Some(region_lease_handler))
                    .with_flush_stats_factor(Some(options.flush_stats_factor))
-                    .with_flow_state_handler(Some(flow_state_handler))
                    .add_default_handlers()
            }
        };
--- a/src/meta-srv/src/service/store/cached_kv.rs
+++ b/src/meta-srv/src/service/store/cached_kv.rs
@@ -386,10 +386,6 @@ impl ResettableKvBackend for LeaderCachedKvBackend {
    fn reset(&self) {
        self.cache.reset()
    }
-
-    fn as_kv_backend_ref(self: Arc<Self>) -> KvBackendRef<Self::Error> {
-        self
-    }
 }

 #[cfg(test)]
--- a/src/metric-engine/src/test_util.rs
+++ b/src/metric-engine/src/test_util.rs
@@ -313,12 +313,12 @@ mod test {
        let region_dir = "test_metric_region";
        // assert metadata region's dir
        let metadata_region_dir = join_dir(region_dir, METADATA_REGION_SUBDIR);
-        let exist = object_store.is_exist(&metadata_region_dir).await.unwrap();
+        let exist = object_store.exists(&metadata_region_dir).await.unwrap();
        assert!(exist);

        // assert data region's dir
        let data_region_dir = join_dir(region_dir, DATA_REGION_SUBDIR);
-        let exist = object_store.is_exist(&data_region_dir).await.unwrap();
+        let exist = object_store.exists(&data_region_dir).await.unwrap();
        assert!(exist);

        // check mito engine
--- a/src/mito2/src/cache/file_cache.rs
+++ b/src/mito2/src/cache/file_cache.rs
@@ -286,7 +286,7 @@ impl FileCache {
    }

    async fn get_reader(&self, file_path: &str) -> object_store::Result<Option<Reader>> {
-        if self.local_store.is_exist(file_path).await? {
+        if self.local_store.exists(file_path).await? {
            Ok(Some(self.local_store.reader(file_path).await?))
        } else {
            Ok(None)
@@ -480,7 +480,7 @@ mod tests {
        cache.memory_index.run_pending_tasks().await;

        // The file also not exists.
-        assert!(!local_store.is_exist(&file_path).await.unwrap());
+        assert!(!local_store.exists(&file_path).await.unwrap());
        assert_eq!(0, cache.memory_index.weighted_size());
    }

--- a/src/mito2/src/compaction.rs
+++ b/src/mito2/src/compaction.rs
@@ -271,11 +271,11 @@ impl CompactionScheduler {
            current_version.options.ttl,
            &schema_metadata_manager,
        )
-        .await
-        .unwrap_or_else(|e| {
-            warn!(e; "Failed to get ttl for region: {}", region_id);
-            TimeToLive::default()
-        });
+            .await
+            .unwrap_or_else(|e| {
+                warn!(e; "Failed to get ttl for region: {}", region_id);
+                TimeToLive::default()
+            });

        debug!(
            "Pick compaction strategy {:?} for region: {}, ttl: {:?}",
@@ -351,7 +351,7 @@ impl CompactionScheduler {
                                job_id: None,
                                reason: e.reason,
                            }
-                            .fail();
+                                .fail();
                        }

                        error!(e; "Failed to schedule remote compaction job for region {}, fallback to local compaction", region_id);
--- a/src/mito2/src/engine/create_test.rs
+++ b/src/mito2/src/engine/create_test.rs
@@ -192,12 +192,12 @@ async fn test_engine_create_with_custom_store() {
    assert!(object_store_manager
        .find("Gcs")
        .unwrap()
-        .is_exist(region_dir)
+        .exists(region_dir)
        .await
        .unwrap());
    assert!(!object_store_manager
        .default_object_store()
-        .is_exist(region_dir)
+        .exists(region_dir)
        .await
        .unwrap());
 }
--- a/src/mito2/src/engine/drop_test.rs
+++ b/src/mito2/src/engine/drop_test.rs
@@ -71,7 +71,7 @@ async fn test_engine_drop_region() {
    assert!(!env
        .get_object_store()
        .unwrap()
-        .is_exist(&join_path(&region_dir, DROPPING_MARKER_FILE))
+        .exists(&join_path(&region_dir, DROPPING_MARKER_FILE))
        .await
        .unwrap());

@@ -93,7 +93,7 @@ async fn test_engine_drop_region() {
    listener.wait().await;

    let object_store = env.get_object_store().unwrap();
-    assert!(!object_store.is_exist(&region_dir).await.unwrap());
+    assert!(!object_store.exists(&region_dir).await.unwrap());
 }

 #[tokio::test]
@@ -167,13 +167,13 @@ async fn test_engine_drop_region_for_custom_store() {
    assert!(object_store_manager
        .find("Gcs")
        .unwrap()
-        .is_exist(&custom_region_dir)
+        .exists(&custom_region_dir)
        .await
        .unwrap());
    assert!(object_store_manager
        .find("default")
        .unwrap()
-        .is_exist(&global_region_dir)
+        .exists(&global_region_dir)
        .await
        .unwrap());

@@ -190,13 +190,13 @@ async fn test_engine_drop_region_for_custom_store() {
    assert!(!object_store_manager
        .find("Gcs")
        .unwrap()
-        .is_exist(&custom_region_dir)
+        .exists(&custom_region_dir)
        .await
        .unwrap());
    assert!(object_store_manager
        .find("default")
        .unwrap()
-        .is_exist(&global_region_dir)
+        .exists(&global_region_dir)
        .await
        .unwrap());
 }
--- a/src/mito2/src/engine/open_test.rs
+++ b/src/mito2/src/engine/open_test.rs
@@ -228,13 +228,13 @@ async fn test_engine_region_open_with_custom_store() {
    let object_store_manager = env.get_object_store_manager().unwrap();
    assert!(!object_store_manager
        .default_object_store()
-        .is_exist(region.access_layer.region_dir())
+        .exists(region.access_layer.region_dir())
        .await
        .unwrap());
    assert!(object_store_manager
        .find("Gcs")
        .unwrap()
-        .is_exist(region.access_layer.region_dir())
+        .exists(region.access_layer.region_dir())
        .await
        .unwrap());
 }
--- a/src/mito2/src/manifest/tests/checkpoint.rs
+++ b/src/mito2/src/manifest/tests/checkpoint.rs
@@ -84,6 +84,7 @@ async fn manager_without_checkpoint() {

    // check files
    let mut expected = vec![
+        "/",
        "00000000000000000010.json",
        "00000000000000000009.json",
        "00000000000000000008.json",
@@ -130,6 +131,7 @@ async fn manager_with_checkpoint_distance_1() {

    // check files
    let mut expected = vec![
+        "/",
        "00000000000000000009.checkpoint",
        "00000000000000000010.checkpoint",
        "00000000000000000010.json",
--- a/src/mito2/src/sst/file_purger.rs
+++ b/src/mito2/src/sst/file_purger.rs
@@ -185,7 +185,7 @@ mod tests {

        scheduler.stop(true).await.unwrap();

-        assert!(!object_store.is_exist(&path).await.unwrap());
+        assert!(!object_store.exists(&path).await.unwrap());
    }

    #[tokio::test]
@@ -247,7 +247,7 @@ mod tests {

        scheduler.stop(true).await.unwrap();

-        assert!(!object_store.is_exist(&path).await.unwrap());
-        assert!(!object_store.is_exist(&index_path).await.unwrap());
+        assert!(!object_store.exists(&path).await.unwrap());
+        assert!(!object_store.exists(&index_path).await.unwrap());
    }
 }
--- a/src/mito2/src/worker.rs
+++ b/src/mito2/src/worker.rs
@@ -27,9 +27,10 @@ mod handle_truncate;
 mod handle_write;
 use std::collections::HashMap;
 use std::path::Path;
+use std::str::FromStr;
 use std::sync::atomic::{AtomicBool, Ordering};
 use std::sync::Arc;
-use std::time::Duration;
+use std::time::{Duration, Instant};

 use common_base::Plugins;
 use common_meta::key::SchemaMetadataManagerRef;
@@ -432,6 +433,12 @@ impl<S: LogStore> WorkerStarter<S> {
        let running = Arc::new(AtomicBool::new(true));
        let now = self.time_provider.current_time_millis();
        let id_string = self.id.to_string();
+
+        let slow_threshold = std::env::var("slow_threshold")
+            .ok()
+            .and_then(|v| u64::from_str(&v).ok())
+            .unwrap_or(1000);
+
        let mut worker_thread = RegionWorkerLoop {
            id: self.id,
            config: self.config.clone(),
@@ -471,6 +478,9 @@ impl<S: LogStore> WorkerStarter<S> {
            region_count: REGION_COUNT.with_label_values(&[&id_string]),
            region_edit_queues: RegionEditQueues::default(),
            schema_metadata_manager: self.schema_metadata_manager,
+            metrics: WorkerMetrics::default(),
+            stall_start: None,
+            slow_threshold: Duration::from_millis(slow_threshold),
        };
        let handle = common_runtime::spawn_global(async move {
            worker_thread.run().await;
@@ -624,6 +634,46 @@ impl StalledRequests {
    }
 }

+/// Local metrics of the worker.
+#[derive(Debug, Default)]
+struct WorkerMetrics {
+    /// Elapsed time of the select block.
+    select_cost: Duration,
+    /// Number of times waking up by flush.
+    num_flush_wake: usize,
+    /// Number of times waking up by periodical tasks.
+    num_periodical_wake: usize,
+    /// Number of requests to handle.
+    num_requests: usize,
+    /// Number of stalled requests to process.
+    num_stalled_request_processed: usize,
+    /// Number of stalled requests to add.
+    num_stalled_request_added: usize,
+    /// Number of write stall.
+    num_stall: usize,
+
+    /// Total time of handling stall requests.
+    handle_stall_cost: Duration,
+    /// Total time of handling requests.
+    handle_request_cost: Duration,
+    /// Total time of handling write requests.
+    handle_write_request_cost: Duration,
+    /// Total time of handling periodical tasks.
+    handle_periodical_task_cost: Duration,
+    /// Total time of handling requests after waking up.
+    handle_cost: Duration,
+
+    // Cost of handle write
+    /// Total time of flushing the worker.
+    flush_worker_cost: Duration,
+    /// Total time of writing WAL.
+    write_wal_cost: Duration,
+    /// Total time of writing memtables.
+    write_memtable_cost: Duration,
+    /// Total time of stall.
+    stall_cost: Duration,
+}
+
 /// Background worker loop to handle requests.
 struct RegionWorkerLoop<S> {
    /// Id of the worker.
@@ -682,6 +732,11 @@ struct RegionWorkerLoop<S> {
    region_edit_queues: RegionEditQueues,
    /// Database level metadata manager.
    schema_metadata_manager: SchemaMetadataManagerRef,
+    /// Metrics of the worker in one loop.
+    metrics: WorkerMetrics,
+    /// Last stall start time.
+    stall_start: Option<Instant>,
+    slow_threshold: Duration,
 }

 impl<S: LogStore> RegionWorkerLoop<S> {
@@ -697,6 +752,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {
        // Buffer to retrieve requests from receiver.
        let mut buffer = RequestBuffer::with_capacity(self.config.worker_request_batch_size);

+        let mut select_start = Instant::now();
        while self.running.load(Ordering::Relaxed) {
            // Clear the buffer before handling next batch of requests.
            buffer.clear();
@@ -714,6 +770,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {
                    }
                }
                recv_res = self.flush_receiver.changed() => {
+                    self.metrics.num_flush_wake += 1;
                    if recv_res.is_err() {
                        // The channel is disconnected.
                        break;
@@ -729,12 +786,16 @@ impl<S: LogStore> RegionWorkerLoop<S> {
                    }
                }
                _ = &mut sleep => {
+                    self.metrics.num_periodical_wake += 1;
                    // Timeout. Checks periodical tasks.
                    self.handle_periodical_tasks();
                    continue;
                }
            }

+            self.metrics.select_cost = select_start.elapsed();
+            let handle_start = Instant::now();
+
            if self.flush_receiver.has_changed().unwrap_or(false) {
                // Always checks whether we could process stalled requests to avoid a request
                // hangs too long.
@@ -751,12 +812,28 @@ impl<S: LogStore> RegionWorkerLoop<S> {
                    Err(_) => break,
                }
            }
+            self.metrics.num_requests = buffer.len();

            self.listener.on_recv_requests(buffer.len());

+            let start = Instant::now();
            self.handle_requests(&mut buffer).await;
+            self.metrics.handle_request_cost = start.elapsed();

+            let start = Instant::now();
            self.handle_periodical_tasks();
+            self.metrics.handle_periodical_task_cost = start.elapsed();
+
+            self.metrics.handle_cost = handle_start.elapsed();
+            if self.metrics.handle_cost + self.metrics.select_cost > self.slow_threshold {
+                info!(
+                    "Region worker too slow, id: {}, metrics: {:?}",
+                    self.id, self.metrics
+                );
+            }
+            // Clear the metrics.
+            self.metrics = WorkerMetrics::default();
+            select_start = Instant::now();
        }

        self.clean().await;
@@ -804,7 +881,9 @@ impl<S: LogStore> RegionWorkerLoop<S> {

        // Handles all write requests first. So we can alter regions without
        // considering existing write requests.
+        let start = Instant::now();
        self.handle_write_requests(write_requests, true).await;
+        self.metrics.handle_write_request_cost = start.elapsed();

        self.handle_ddl_requests(ddl_requests).await;
    }
--- a/src/mito2/src/worker/handle_compaction.rs
+++ b/src/mito2/src/worker/handle_compaction.rs
@@ -18,7 +18,7 @@ use store_api::region_request::RegionCompactRequest;
 use store_api::storage::RegionId;

 use crate::error::RegionNotFoundSnafu;
-use crate::metrics::COMPACTION_REQUEST_COUNT;
+use crate::metrics::{COMPACTION_REQUEST_COUNT};
 use crate::region::MitoRegionRef;
 use crate::request::{CompactionFailed, CompactionFinished, OnFailure, OptionOutputTx};
 use crate::worker::RegionWorkerLoop;
--- a/src/mito2/src/worker/handle_open.rs
+++ b/src/mito2/src/worker/handle_open.rs
@@ -51,7 +51,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {
        // Check if this region is pending drop. And clean the entire dir if so.
        if !self.dropping_regions.is_region_exists(region_id)
            && object_store
-                .is_exist(&join_path(&request.region_dir, DROPPING_MARKER_FILE))
+                .exists(&join_path(&request.region_dir, DROPPING_MARKER_FILE))
                .await
                .context(OpenDalSnafu)?
        {
--- a/src/mito2/src/worker/handle_write.rs
+++ b/src/mito2/src/worker/handle_write.rs
@@ -16,6 +16,7 @@

 use std::collections::{hash_map, HashMap};
 use std::sync::Arc;
+use std::time::Instant;

 use api::v1::OpType;
 use common_telemetry::debug;
@@ -43,7 +44,9 @@ impl<S: LogStore> RegionWorkerLoop<S> {
        }

        // Flush this worker if the engine needs to flush.
+        let start = Instant::now();
        self.maybe_flush_worker();
+        self.metrics.flush_worker_cost += start.elapsed();

        if self.should_reject_write() {
            // The memory pressure is still too high, reject write requests.
@@ -55,6 +58,9 @@ impl<S: LogStore> RegionWorkerLoop<S> {

        if self.write_buffer_manager.should_stall() && allow_stall {
            self.stalled_count.add(write_requests.len() as i64);
+            self.metrics.num_stalled_request_added += write_requests.len();
+            self.metrics.num_stall += 1;
+            self.stall_start = Some(Instant::now());
            self.stalled_requests.append(&mut write_requests);
            self.listener.on_write_stall();
            return;
@@ -70,6 +76,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {

        // Write to WAL.
        {
+            let start = Instant::now();
            let _timer = WRITE_STAGE_ELAPSED
                .with_label_values(&["write_wal"])
                .start_timer();
@@ -87,12 +94,14 @@ impl<S: LogStore> RegionWorkerLoop<S> {
                        let last_entry_id = response.last_entry_ids.get(region_id).unwrap();
                        region_ctx.set_next_entry_id(last_entry_id + 1);
                    }
+                    self.metrics.write_wal_cost += start.elapsed();
                }
                Err(e) => {
                    // Failed to write wal.
                    for mut region_ctx in region_ctxs.into_values() {
                        region_ctx.set_error(e.clone());
                    }
+                    self.metrics.write_wal_cost += start.elapsed();
                    return;
                }
            }
@@ -101,6 +110,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {
        let (mut put_rows, mut delete_rows) = (0, 0);
        // Write to memtables.
        {
+            let start = Instant::now();
            let _timer = WRITE_STAGE_ELAPSED
                .with_label_values(&["write_memtable"])
                .start_timer();
@@ -109,6 +119,7 @@ impl<S: LogStore> RegionWorkerLoop<S> {
                put_rows += region_ctx.put_num;
                delete_rows += region_ctx.delete_num;
            }
+            self.metrics.write_memtable_cost += start.elapsed();
        }
        WRITE_ROWS_TOTAL
            .with_label_values(&["put"])
@@ -120,13 +131,19 @@ impl<S: LogStore> RegionWorkerLoop<S> {

    /// Handles all stalled write requests.
    pub(crate) async fn handle_stalled_requests(&mut self) {
+        let handle_stall_start = Instant::now();
+        if let Some(start) = self.stall_start.take() {
+            self.metrics.stall_cost += start.elapsed();
+        }
        // Handle stalled requests.
        let stalled = std::mem::take(&mut self.stalled_requests);
        self.stalled_count.sub(stalled.requests.len() as i64);
        // We already stalled these requests, don't stall them again.
        for (_, (_, requests)) in stalled.requests {
+            self.metrics.num_stalled_request_processed += requests.len();
            self.handle_write_requests(requests, false).await;
        }
+        self.metrics.handle_stall_cost += handle_stall_start.elapsed();
    }

    /// Rejects all stalled requests.
@@ -149,7 +166,11 @@ impl<S: LogStore> RegionWorkerLoop<S> {
    /// Handles a specific region's stalled requests.
    pub(crate) async fn handle_region_stalled_requests(&mut self, region_id: &RegionId) {
        debug!("Handles stalled requests for region {}", region_id);
+        if let Some(start) = self.stall_start.take() {
+            self.metrics.stall_cost += start.elapsed();
+        }
        let requests = self.stalled_requests.remove(region_id);
+        self.metrics.num_stalled_request_processed += requests.len();
        self.stalled_count.sub(requests.len() as i64);
        self.handle_write_requests(requests, true).await;
    }
--- a/src/object-store/Cargo.toml
+++ b/src/object-store/Cargo.toml
@@ -17,8 +17,9 @@ futures.workspace = true
 lazy_static.workspace = true
 md5 = "0.7"
 moka = { workspace = true, features = ["future"] }
-opendal = { version = "0.49", features = [
+opendal = { version = "0.50", features = [
    "layers-tracing",
+    "layers-prometheus",
    "services-azblob",
    "services-fs",
    "services-gcs",
--- a/src/object-store/src/layers.rs
+++ b/src/object-store/src/layers.rs
@@ -13,8 +13,37 @@
 // limitations under the License.

 mod lru_cache;
-mod prometheus;

 pub use lru_cache::*;
 pub use opendal::layers::*;
-pub use prometheus::PrometheusMetricsLayer;
+pub use prometheus::build_prometheus_metrics_layer;
+
+mod prometheus {
+    use std::sync::{Mutex, OnceLock};
+
+    use opendal::layers::PrometheusLayer;
+
+    static PROMETHEUS_LAYER: OnceLock<Mutex<PrometheusLayer>> = OnceLock::new();
+
+    pub fn build_prometheus_metrics_layer(with_path_label: bool) -> PrometheusLayer {
+        PROMETHEUS_LAYER
+            .get_or_init(|| {
+                // This logical tries to extract parent path from the object storage operation
+                // the function also relies on assumption that the region path is built from
+                // pattern `<data|index>/catalog/schema/table_id/....`
+                //
+                // We'll get the data/catalog/schema from path.
+                let path_level = if with_path_label { 3 } else { 0 };
+
+                let layer = PrometheusLayer::builder()
+                    .path_label(path_level)
+                    .register_default()
+                    .unwrap();
+
+                Mutex::new(layer)
+            })
+            .lock()
+            .unwrap()
+            .clone()
+    }
+}
--- a/src/object-store/src/layers/lru_cache/read_cache.rs
+++ b/src/object-store/src/layers/lru_cache/read_cache.rs
@@ -156,9 +156,12 @@ impl<C: Access> ReadCache<C> {
            let size = entry.metadata().content_length();
            OBJECT_STORE_LRU_CACHE_ENTRIES.inc();
            OBJECT_STORE_LRU_CACHE_BYTES.add(size as i64);
-            self.mem_cache
-                .insert(read_key.to_string(), ReadResult::Success(size as u32))
-                .await;
+            // ignore root path
+            if entry.path() != "/" {
+                self.mem_cache
+                    .insert(read_key.to_string(), ReadResult::Success(size as u32))
+                    .await;
+            }
        }

        Ok(self.cache_stat().await)
--- a/src/object-store/src/layers/prometheus.rs
+++ b/src/object-store/src/layers/prometheus.rs
@@ -1,584 +0,0 @@
-// Copyright 2023 Greptime Team
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-//! code originally from <https://github.com/apache/incubator-opendal/blob/main/core/src/layers/prometheus.rs>, make a tiny change to avoid crash in multi thread env
-
-use std::fmt::{Debug, Formatter};
-
-use common_telemetry::debug;
-use lazy_static::lazy_static;
-use opendal::raw::*;
-use opendal::{Buffer, ErrorKind};
-use prometheus::{
-    exponential_buckets, histogram_opts, register_histogram_vec, register_int_counter_vec,
-    Histogram, HistogramTimer, HistogramVec, IntCounterVec,
-};
-
-use crate::util::extract_parent_path;
-
-type Result<T> = std::result::Result<T, opendal::Error>;
-
-lazy_static! {
-    static ref REQUESTS_TOTAL: IntCounterVec = register_int_counter_vec!(
-        "opendal_requests_total",
-        "Total times of all kinds of operation being called",
-        &["scheme", "operation", "path"],
-    )
-    .unwrap();
-    static ref REQUESTS_DURATION_SECONDS: HistogramVec = register_histogram_vec!(
-        histogram_opts!(
-            "opendal_requests_duration_seconds",
-            "Histogram of the time spent on specific operation",
-            exponential_buckets(0.01, 2.0, 16).unwrap()
-        ),
-        &["scheme", "operation", "path"]
-    )
-    .unwrap();
-    static ref BYTES_TOTAL: HistogramVec = register_histogram_vec!(
-        histogram_opts!(
-            "opendal_bytes_total",
-            "Total size of sync or async Read/Write",
-            exponential_buckets(0.01, 2.0, 16).unwrap()
-        ),
-        &["scheme", "operation", "path"]
-    )
-    .unwrap();
-}
-
-#[inline]
-fn increment_errors_total(op: Operation, kind: ErrorKind) {
-    debug!(
-        "Prometheus statistics metrics error, operation {} error {}",
-        op.into_static(),
-        kind.into_static()
-    );
-}
-
-/// Please refer to [prometheus](https://docs.rs/prometheus) for every operation.
-///
-/// # Prometheus Metrics
-///
-/// In this section, we will introduce three metrics that are currently being exported by opendal. These metrics are essential for understanding the behavior and performance of opendal.
-///
-///
-/// | Metric Name                       | Type      | Description                                          | Labels              |
-/// |-----------------------------------|-----------|------------------------------------------------------|---------------------|
-/// | opendal_requests_total            | Counter   | Total times of all kinds of operation being called   | scheme, operation   |
-/// | opendal_requests_duration_seconds | Histogram | Histogram of the time spent on specific operation    | scheme, operation   |
-/// | opendal_bytes_total               | Histogram | Total size of sync or async Read/Write               | scheme, operation   |
-///
-/// For a more detailed explanation of these metrics and how they are used, please refer to the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/).
-///
-/// # Histogram Configuration
-///
-/// The metric buckets for these histograms are automatically generated based on the `exponential_buckets(0.01, 2.0, 16)` configuration.
-#[derive(Default, Debug, Clone)]
-pub struct PrometheusMetricsLayer {
-    pub path_label: bool,
-}
-
-impl PrometheusMetricsLayer {
-    pub fn new(path_label: bool) -> Self {
-        Self { path_label }
-    }
-}
-
-impl<A: Access> Layer<A> for PrometheusMetricsLayer {
-    type LayeredAccess = PrometheusAccess<A>;
-
-    fn layer(&self, inner: A) -> Self::LayeredAccess {
-        let meta = inner.info();
-        let scheme = meta.scheme();
-
-        PrometheusAccess {
-            inner,
-            scheme: scheme.to_string(),
-            path_label: self.path_label,
-        }
-    }
-}
-
-#[derive(Clone)]
-pub struct PrometheusAccess<A: Access> {
-    inner: A,
-    scheme: String,
-    path_label: bool,
-}
-
-impl<A: Access> PrometheusAccess<A> {
-    fn get_path_label<'a>(&self, path: &'a str) -> &'a str {
-        if self.path_label {
-            extract_parent_path(path)
-        } else {
-            ""
-        }
-    }
-}
-
-impl<A: Access> Debug for PrometheusAccess<A> {
-    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
-        f.debug_struct("PrometheusAccessor")
-            .field("inner", &self.inner)
-            .finish_non_exhaustive()
-    }
-}
-
-impl<A: Access> LayeredAccess for PrometheusAccess<A> {
-    type Inner = A;
-    type Reader = PrometheusMetricWrapper<A::Reader>;
-    type BlockingReader = PrometheusMetricWrapper<A::BlockingReader>;
-    type Writer = PrometheusMetricWrapper<A::Writer>;
-    type BlockingWriter = PrometheusMetricWrapper<A::BlockingWriter>;
-    type Lister = A::Lister;
-    type BlockingLister = A::BlockingLister;
-
-    fn inner(&self) -> &Self::Inner {
-        &self.inner
-    }
-
-    async fn create_dir(&self, path: &str, args: OpCreateDir) -> Result<RpCreateDir> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::CreateDir.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::CreateDir.into_static(), path_label])
-            .start_timer();
-        let create_res = self.inner.create_dir(path, args).await;
-
-        timer.observe_duration();
-        create_res.inspect_err(|e| {
-            increment_errors_total(Operation::CreateDir, e.kind());
-        })
-    }
-
-    async fn read(&self, path: &str, args: OpRead) -> Result<(RpRead, Self::Reader)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Read.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Read.into_static(), path_label])
-            .start_timer();
-
-        let (rp, r) = self.inner.read(path, args).await.inspect_err(|e| {
-            increment_errors_total(Operation::Read, e.kind());
-        })?;
-
-        Ok((
-            rp,
-            PrometheusMetricWrapper::new(
-                r,
-                Operation::Read,
-                BYTES_TOTAL.with_label_values(&[
-                    &self.scheme,
-                    Operation::Read.into_static(),
-                    path_label,
-                ]),
-                timer,
-            ),
-        ))
-    }
-
-    async fn write(&self, path: &str, args: OpWrite) -> Result<(RpWrite, Self::Writer)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Write.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Write.into_static(), path_label])
-            .start_timer();
-
-        let (rp, r) = self.inner.write(path, args).await.inspect_err(|e| {
-            increment_errors_total(Operation::Write, e.kind());
-        })?;
-
-        Ok((
-            rp,
-            PrometheusMetricWrapper::new(
-                r,
-                Operation::Write,
-                BYTES_TOTAL.with_label_values(&[
-                    &self.scheme,
-                    Operation::Write.into_static(),
-                    path_label,
-                ]),
-                timer,
-            ),
-        ))
-    }
-
-    async fn stat(&self, path: &str, args: OpStat) -> Result<RpStat> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Stat.into_static(), path_label])
-            .inc();
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Stat.into_static(), path_label])
-            .start_timer();
-
-        let stat_res = self.inner.stat(path, args).await;
-        timer.observe_duration();
-        stat_res.inspect_err(|e| {
-            increment_errors_total(Operation::Stat, e.kind());
-        })
-    }
-
-    async fn delete(&self, path: &str, args: OpDelete) -> Result<RpDelete> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Delete.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Delete.into_static(), path_label])
-            .start_timer();
-
-        let delete_res = self.inner.delete(path, args).await;
-        timer.observe_duration();
-        delete_res.inspect_err(|e| {
-            increment_errors_total(Operation::Delete, e.kind());
-        })
-    }
-
-    async fn list(&self, path: &str, args: OpList) -> Result<(RpList, Self::Lister)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::List.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::List.into_static(), path_label])
-            .start_timer();
-
-        let list_res = self.inner.list(path, args).await;
-
-        timer.observe_duration();
-        list_res.inspect_err(|e| {
-            increment_errors_total(Operation::List, e.kind());
-        })
-    }
-
-    async fn batch(&self, args: OpBatch) -> Result<RpBatch> {
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Batch.into_static(), ""])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Batch.into_static(), ""])
-            .start_timer();
-        let result = self.inner.batch(args).await;
-
-        timer.observe_duration();
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::Batch, e.kind());
-        })
-    }
-
-    async fn presign(&self, path: &str, args: OpPresign) -> Result<RpPresign> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[&self.scheme, Operation::Presign.into_static(), path_label])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[&self.scheme, Operation::Presign.into_static(), path_label])
-            .start_timer();
-        let result = self.inner.presign(path, args).await;
-        timer.observe_duration();
-
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::Presign, e.kind());
-        })
-    }
-
-    fn blocking_create_dir(&self, path: &str, args: OpCreateDir) -> Result<RpCreateDir> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingCreateDir.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingCreateDir.into_static(),
-                path_label,
-            ])
-            .start_timer();
-        let result = self.inner.blocking_create_dir(path, args);
-
-        timer.observe_duration();
-
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::BlockingCreateDir, e.kind());
-        })
-    }
-
-    fn blocking_read(&self, path: &str, args: OpRead) -> Result<(RpRead, Self::BlockingReader)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingRead.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingRead.into_static(),
-                path_label,
-            ])
-            .start_timer();
-
-        self.inner
-            .blocking_read(path, args)
-            .map(|(rp, r)| {
-                (
-                    rp,
-                    PrometheusMetricWrapper::new(
-                        r,
-                        Operation::BlockingRead,
-                        BYTES_TOTAL.with_label_values(&[
-                            &self.scheme,
-                            Operation::BlockingRead.into_static(),
-                            path_label,
-                        ]),
-                        timer,
-                    ),
-                )
-            })
-            .inspect_err(|e| {
-                increment_errors_total(Operation::BlockingRead, e.kind());
-            })
-    }
-
-    fn blocking_write(&self, path: &str, args: OpWrite) -> Result<(RpWrite, Self::BlockingWriter)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingWrite.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingWrite.into_static(),
-                path_label,
-            ])
-            .start_timer();
-
-        self.inner
-            .blocking_write(path, args)
-            .map(|(rp, r)| {
-                (
-                    rp,
-                    PrometheusMetricWrapper::new(
-                        r,
-                        Operation::BlockingWrite,
-                        BYTES_TOTAL.with_label_values(&[
-                            &self.scheme,
-                            Operation::BlockingWrite.into_static(),
-                            path_label,
-                        ]),
-                        timer,
-                    ),
-                )
-            })
-            .inspect_err(|e| {
-                increment_errors_total(Operation::BlockingWrite, e.kind());
-            })
-    }
-
-    fn blocking_stat(&self, path: &str, args: OpStat) -> Result<RpStat> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingStat.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingStat.into_static(),
-                path_label,
-            ])
-            .start_timer();
-        let result = self.inner.blocking_stat(path, args);
-        timer.observe_duration();
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::BlockingStat, e.kind());
-        })
-    }
-
-    fn blocking_delete(&self, path: &str, args: OpDelete) -> Result<RpDelete> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingDelete.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingDelete.into_static(),
-                path_label,
-            ])
-            .start_timer();
-        let result = self.inner.blocking_delete(path, args);
-        timer.observe_duration();
-
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::BlockingDelete, e.kind());
-        })
-    }
-
-    fn blocking_list(&self, path: &str, args: OpList) -> Result<(RpList, Self::BlockingLister)> {
-        let path_label = self.get_path_label(path);
-        REQUESTS_TOTAL
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingList.into_static(),
-                path_label,
-            ])
-            .inc();
-
-        let timer = REQUESTS_DURATION_SECONDS
-            .with_label_values(&[
-                &self.scheme,
-                Operation::BlockingList.into_static(),
-                path_label,
-            ])
-            .start_timer();
-        let result = self.inner.blocking_list(path, args);
-        timer.observe_duration();
-
-        result.inspect_err(|e| {
-            increment_errors_total(Operation::BlockingList, e.kind());
-        })
-    }
-}
-
-pub struct PrometheusMetricWrapper<R> {
-    inner: R,
-
-    op: Operation,
-    bytes_counter: Histogram,
-    _requests_duration_timer: HistogramTimer,
-    bytes: u64,
-}
-
-impl<R> Drop for PrometheusMetricWrapper<R> {
-    fn drop(&mut self) {
-        self.bytes_counter.observe(self.bytes as f64);
-    }
-}
-
-impl<R> PrometheusMetricWrapper<R> {
-    fn new(
-        inner: R,
-        op: Operation,
-        bytes_counter: Histogram,
-        requests_duration_timer: HistogramTimer,
-    ) -> Self {
-        Self {
-            inner,
-            op,
-            bytes_counter,
-            _requests_duration_timer: requests_duration_timer,
-            bytes: 0,
-        }
-    }
-}
-
-impl<R: oio::Read> oio::Read for PrometheusMetricWrapper<R> {
-    async fn read(&mut self) -> Result<Buffer> {
-        self.inner.read().await.inspect_err(|err| {
-            increment_errors_total(self.op, err.kind());
-        })
-    }
-}
-
-impl<R: oio::BlockingRead> oio::BlockingRead for PrometheusMetricWrapper<R> {
-    fn read(&mut self) -> opendal::Result<Buffer> {
-        self.inner.read().inspect_err(|err| {
-            increment_errors_total(self.op, err.kind());
-        })
-    }
-}
-
-impl<R: oio::Write> oio::Write for PrometheusMetricWrapper<R> {
-    async fn write(&mut self, bs: Buffer) -> Result<()> {
-        let bytes = bs.len();
-        match self.inner.write(bs).await {
-            Ok(_) => {
-                self.bytes += bytes as u64;
-                Ok(())
-            }
-            Err(err) => {
-                increment_errors_total(self.op, err.kind());
-                Err(err)
-            }
-        }
-    }
-
-    async fn close(&mut self) -> Result<()> {
-        self.inner.close().await.inspect_err(|err| {
-            increment_errors_total(self.op, err.kind());
-        })
-    }
-
-    async fn abort(&mut self) -> Result<()> {
-        self.inner.close().await.inspect_err(|err| {
-            increment_errors_total(self.op, err.kind());
-        })
-    }
-}
-
-impl<R: oio::BlockingWrite> oio::BlockingWrite for PrometheusMetricWrapper<R> {
-    fn write(&mut self, bs: Buffer) -> Result<()> {
-        let bytes = bs.len();
-        self.inner
-            .write(bs)
-            .map(|_| {
-                self.bytes += bytes as u64;
-            })
-            .inspect_err(|err| {
-                increment_errors_total(self.op, err.kind());
-            })
-    }
-
-    fn close(&mut self) -> Result<()> {
-        self.inner.close().inspect_err(|err| {
-            increment_errors_total(self.op, err.kind());
-        })
-    }
-}
--- a/src/object-store/src/util.rs
+++ b/src/object-store/src/util.rs
@@ -15,19 +15,12 @@
 use std::fmt::Display;

 use common_telemetry::{debug, error, trace};
-use futures::TryStreamExt;
 use opendal::layers::{LoggingInterceptor, LoggingLayer, TracingLayer};
 use opendal::raw::{AccessorInfo, Operation};
-use opendal::{Entry, ErrorKind, Lister};
+use opendal::ErrorKind;

-use crate::layers::PrometheusMetricsLayer;
 use crate::ObjectStore;

-/// Collect all entries from the [Lister].
-pub async fn collect(stream: Lister) -> Result<Vec<Entry>, opendal::Error> {
-    stream.try_collect::<Vec<_>>().await
-}
-
 /// Join two paths and normalize the output dir.
 ///
 /// The output dir is always ends with `/`. e.g.
@@ -127,26 +120,12 @@ pub fn normalize_path(path: &str) -> String {
    p
 }

-// This logical tries to extract parent path from the object storage operation
-// the function also relies on assumption that the region path is built from
-// pattern `<data|index>/catalog/schema/table_id/....`
-//
-// this implementation tries to extract at most 3 levels of parent path
-pub(crate) fn extract_parent_path(path: &str) -> &str {
-    // split the path into `catalog`, `schema` and others
-    path.char_indices()
-        .filter(|&(_, c)| c == '/')
-        // we get the data/catalog/schema from path, split at the 3rd /
-        .nth(2)
-        .map_or(path, |(i, _)| &path[..i])
-}
-
 /// Attaches instrument layers to the object store.
 pub fn with_instrument_layers(object_store: ObjectStore, path_label: bool) -> ObjectStore {
    object_store
        .layer(LoggingLayer::new(DefaultLoggingInterceptor))
        .layer(TracingLayer)
-        .layer(PrometheusMetricsLayer::new(path_label))
+        .layer(crate::layers::build_prometheus_metrics_layer(path_label))
 }

 static LOGGING_TARGET: &str = "opendal::services";
@@ -263,28 +242,4 @@ mod tests {
        assert_eq!("/abc", join_path("//", "/abc"));
        assert_eq!("abc/def", join_path("abc/", "//def"));
    }
-
-    #[test]
-    fn test_path_extraction() {
-        assert_eq!(
-            "data/greptime/public",
-            extract_parent_path("data/greptime/public/1024/1024_0000000000/")
-        );
-
-        assert_eq!(
-            "data/greptime/public",
-            extract_parent_path("data/greptime/public/1/")
-        );
-
-        assert_eq!(
-            "data/greptime/public",
-            extract_parent_path("data/greptime/public")
-        );
-
-        assert_eq!("data/greptime/", extract_parent_path("data/greptime/"));
-
-        assert_eq!("data/", extract_parent_path("data/"));
-
-        assert_eq!("/", extract_parent_path("/"));
-    }
 }
--- a/src/object-store/tests/object_store_test.rs
+++ b/src/object-store/tests/object_store_test.rs
@@ -65,23 +65,38 @@ async fn test_object_list(store: &ObjectStore) -> Result<()> {
    store.write(p3, "Hello, object3!").await?;

    // List objects
-    let entries = store.list("/").await?;
+    let entries = store
+        .list("/")
+        .await?
+        .into_iter()
+        .filter(|x| x.metadata().mode() == EntryMode::FILE)
+        .collect::<Vec<_>>();
    assert_eq!(3, entries.len());

    store.delete(p1).await?;
    store.delete(p3).await?;

    // List objects again
-    // Only o2 is exists
-    let entries = store.list("/").await?;
+    // Only o2 and root exist
+    let entries = store
+        .list("/")
+        .await?
+        .into_iter()
+        .filter(|x| x.metadata().mode() == EntryMode::FILE)
+        .collect::<Vec<_>>();
    assert_eq!(1, entries.len());
-    assert_eq!(p2, entries.first().unwrap().path());
+    assert_eq!(p2, entries[0].path());

    let content = store.read(p2).await?;
    assert_eq!("Hello, object2!", String::from_utf8(content.to_vec())?);

    store.delete(p2).await?;
-    let entries = store.list("/").await?;
+    let entries = store
+        .list("/")
+        .await?
+        .into_iter()
+        .filter(|x| x.metadata().mode() == EntryMode::FILE)
+        .collect::<Vec<_>>();
    assert!(entries.is_empty());

    assert!(store.read(p1).await.is_err());
@@ -252,7 +267,7 @@ async fn test_file_backend_with_lru_cache() -> Result<()> {

 async fn assert_lru_cache<C: Access>(cache_layer: &LruCacheLayer<C>, file_names: &[&str]) {
    for file_name in file_names {
-        assert!(cache_layer.contains_file(file_name).await);
+        assert!(cache_layer.contains_file(file_name).await, "{file_name}");
    }
 }

@@ -264,7 +279,9 @@ async fn assert_cache_files<C: Access>(
    let (_, mut lister) = store.list("/", OpList::default()).await?;
    let mut objects = vec![];
    while let Some(e) = lister.next().await? {
-        objects.push(e);
+        if e.mode() == EntryMode::FILE {
+            objects.push(e);
+        }
    }

    // compare the cache file with the expected cache file; ignore orders
@@ -332,9 +349,9 @@ async fn test_object_store_cache_policy() -> Result<()> {
    assert_cache_files(
        &cache_store,
        &[
-            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14",
-            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=7-14",
-            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=0-14",
+            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-",
+            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=7-",
+            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=0-",
        ],
        &["Hello, object1!", "object2!", "Hello, object2!"],
    )
@@ -342,9 +359,9 @@ async fn test_object_store_cache_policy() -> Result<()> {
    assert_lru_cache(
        &cache_layer,
        &[
-            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14",
-            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=7-14",
-            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=0-14",
+            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-",
+            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=7-",
+            "ecfe0dce85de452eb0a325158e7bfb75.cache-bytes=0-",
        ],
    )
    .await;
@@ -355,13 +372,13 @@ async fn test_object_store_cache_policy() -> Result<()> {
    assert_eq!(cache_layer.read_cache_stat().await, (1, 15));
    assert_cache_files(
        &cache_store,
-        &["6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14"],
+        &["6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-"],
        &["Hello, object1!"],
    )
    .await?;
    assert_lru_cache(
        &cache_layer,
-        &["6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14"],
+        &["6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-"],
    )
    .await;

@@ -388,8 +405,8 @@ async fn test_object_store_cache_policy() -> Result<()> {
    assert_cache_files(
        &cache_store,
        &[
-            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14",
-            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-14",
+            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-",
+            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-",
            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-4",
        ],
        &["Hello, object1!", "Hello, object3!", "Hello"],
@@ -398,8 +415,8 @@ async fn test_object_store_cache_policy() -> Result<()> {
    assert_lru_cache(
        &cache_layer,
        &[
-            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-14",
-            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-14",
+            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=0-",
+            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-",
            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-4",
        ],
    )
@@ -416,7 +433,7 @@ async fn test_object_store_cache_policy() -> Result<()> {
        &cache_store,
        &[
            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=1-14",
-            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-14",
+            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-",
            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-4",
        ],
        &["ello, object1!", "Hello, object3!", "Hello"],
@@ -426,7 +443,7 @@ async fn test_object_store_cache_policy() -> Result<()> {
        &cache_layer,
        &[
            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=1-14",
-            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-14",
+            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-",
            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-4",
        ],
    )
@@ -448,7 +465,7 @@ async fn test_object_store_cache_policy() -> Result<()> {
        &cache_layer,
        &[
            "6d29752bdc6e4d5ba5483b96615d6c48.cache-bytes=1-14",
-            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-14",
+            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-",
            "a8b1dc21e24bb55974e3e68acc77ed52.cache-bytes=0-4",
        ],
    )
--- a/src/operator/src/insert.rs
+++ b/src/operator/src/insert.rs
@@ -576,9 +576,6 @@ impl Inserter {

                    for table in tables {
                        let table_info = table.table_info();
-                        if table_info.is_ttl_instant_table() {
-                            instant_table_ids.insert(table_info.table_id());
-                        }
                        table_name_to_ids.insert(table_info.name.clone(), table_info.table_id());
                    }
                }
@@ -599,9 +596,6 @@ impl Inserter {
                        .create_physical_table(create_table, ctx, statement_executor)
                        .await?;
                    let table_info = table.table_info();
-                    if table_info.is_ttl_instant_table() {
-                        instant_table_ids.insert(table_info.table_id());
-                    }
                    table_name_to_ids.insert(table_info.name.clone(), table_info.table_id());
                }
                for alter_expr in alter_tables.into_iter() {
--- a/tests-integration/tests/grpc.rs
+++ b/tests-integration/tests/grpc.rs
@@ -444,40 +444,6 @@ async fn insert_with_hints_and_assert(db: &Database) {
 +-------+-------------------------------------+\
 ";
    assert_eq!(pretty, expected);
-
-    // testing data with ttl=instant and auto_create_table = true can be handled correctly
-    let (expected_host_col, expected_cpu_col, expected_mem_col, expected_ts_col) = expect_data();
-
-    let request = InsertRequest {
-        table_name: "demo1".to_string(),
-        columns: vec![
-            expected_host_col.clone(),
-            expected_cpu_col.clone(),
-            expected_mem_col.clone(),
-            expected_ts_col.clone(),
-        ],
-        row_count: 4,
-    };
-    let result = db
-        .insert_with_hints(
-            InsertRequests {
-                inserts: vec![request],
-            },
-            &[("auto_create_table", "true"), ("ttl", "instant")],
-        )
-        .await;
-    assert_eq!(result.unwrap(), 0);
-
-    // check table is empty
-    let output = db.sql("SELECT * FROM demo1").await.unwrap();
-
-    let record_batches = match output.data {
-        OutputData::RecordBatches(record_batches) => record_batches,
-        OutputData::Stream(stream) => RecordBatches::try_collect(stream).await.unwrap(),
-        OutputData::AffectedRows(_) => unreachable!(),
-    };
-
-    assert!(record_batches.iter().all(|r| r.num_rows() == 0));
 }

 async fn insert_and_assert(db: &Database) {
--- a/tests/cases/standalone/common/flow/flow_basic.result
+++ b/tests/cases/standalone/common/flow/flow_basic.result
@@ -390,65 +390,6 @@ GROUP BY

 Affected Rows: 0

-INSERT INTO
-    bytes_log
-VALUES
-    (NULL, '2023-01-01 00:00:01'),
-    (300, '2023-01-01 00:00:29');
-
-Affected Rows: 2
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('find_approx_rate');
-
-+--------------------------------------+
-| ADMIN FLUSH_FLOW('find_approx_rate') |
-+--------------------------------------+
-|  FLOW_FLUSHED  |
-+--------------------------------------+
-
-SELECT
-    rate,
-    time_window
-FROM
-    approx_rate;
-
-+------+---------------------+
-| rate | time_window         |
-+------+---------------------+
-| 0.0  | 2023-01-01T00:00:00 |
-+------+---------------------+
-
-INSERT INTO
-    bytes_log
-VALUES
-    (NULL, '2022-01-01 00:00:01'),
-    (NULL, '2022-01-01 00:00:29');
-
-Affected Rows: 2
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('find_approx_rate');
-
-+--------------------------------------+
-| ADMIN FLUSH_FLOW('find_approx_rate') |
-+--------------------------------------+
-|  FLOW_FLUSHED  |
-+--------------------------------------+
-
-SELECT
-    rate,
-    time_window
-FROM
-    approx_rate;
-
-+------+---------------------+
-| rate | time_window         |
-+------+---------------------+
-|      | 2022-01-01T00:00:00 |
-| 0.0  | 2023-01-01T00:00:00 |
-+------+---------------------+
-
 INSERT INTO
    bytes_log
 VALUES
@@ -475,8 +416,6 @@ FROM
 +-------------------+---------------------+
 | rate              | time_window         |
 +-------------------+---------------------+
-|                   | 2022-01-01T00:00:00 |
-| 0.0               | 2023-01-01T00:00:00 |
 | 6.633333333333334 | 2025-01-01T00:00:00 |
 +-------------------+---------------------+

@@ -506,8 +445,6 @@ FROM
 +--------------------+---------------------+
 | rate               | time_window         |
 +--------------------+---------------------+
-|                    | 2022-01-01T00:00:00 |
-| 0.0                | 2023-01-01T00:00:00 |
 | 6.633333333333334  | 2025-01-01T00:00:00 |
 | 1.6666666666666667 | 2025-01-01T00:00:30 |
 +--------------------+---------------------+
@@ -1055,7 +992,6 @@ CREATE TABLE requests_without_ip (
    service_name STRING,
    val INT,
    ts TIMESTAMP TIME INDEX,
-    PRIMARY KEY(service_name)
 );

 Affected Rows: 0
@@ -1073,12 +1009,12 @@ Affected Rows: 0
 INSERT INTO
    requests
 VALUES
-    (NULL, "10.0.0.1", 100, "2024-10-18 19:00:00"),
+    ("svc1", "10.0.0.1", 100, "2024-10-18 19:00:00"),
    ("svc1", "10.0.0.2", 100, "2024-10-18 19:00:00"),
-    (NULL, "10.0.0.1", 200, "2024-10-18 19:00:30"),
+    ("svc1", "10.0.0.1", 200, "2024-10-18 19:00:30"),
    ("svc1", "10.0.0.2", 200, "2024-10-18 19:00:30"),
-    (NULL, "10.0.0.1", 300, "2024-10-18 19:01:00"),
-    (NULL, "10.0.0.2", 100, "2024-10-18 19:01:01"),
+    ("svc1", "10.0.0.1", 300, "2024-10-18 19:01:00"),
+    ("svc1", "10.0.0.2", 100, "2024-10-18 19:01:01"),
    ("svc1", "10.0.0.1", 400, "2024-10-18 19:01:30"),
    ("svc1", "10.0.0.2", 200, "2024-10-18 19:01:31");

@@ -1101,128 +1037,14 @@ FROM
 +--------------+-----+---------------------+
 | service_name | val | ts                  |
 +--------------+-----+---------------------+
-|              | 100 | 2024-10-18T19:00:00 |
-|              | 200 | 2024-10-18T19:00:30 |
-|              | 300 | 2024-10-18T19:01:00 |
-|              | 100 | 2024-10-18T19:01:01 |
 | svc1         | 100 | 2024-10-18T19:00:00 |
 | svc1         | 200 | 2024-10-18T19:00:30 |
+| svc1         | 300 | 2024-10-18T19:01:00 |
+| svc1         | 100 | 2024-10-18T19:01:01 |
 | svc1         | 400 | 2024-10-18T19:01:30 |
 | svc1         | 200 | 2024-10-18T19:01:31 |
 +--------------+-----+---------------------+

-- Test if FLOWS table works, but don't care about the result since it vary from runs
-SELECT
-    count(CASE WHEN state_size > 0 THEN 1 ELSE 0 END) as active_flows,
-FROM
-    INFORMATION_SCHEMA.FLOWS;
-
-+--------------+
-| active_flows |
-+--------------+
-| 1            |
-+--------------+
-
-INSERT INTO
-    requests
-VALUES
-    (null, "10.0.0.1", 100, "2024-10-19 19:00:00"),
-    (null, "10.0.0.2", 100, "2024-10-19 19:00:00"),
-    (null, "10.0.0.1", 200, "2024-10-19 19:00:30"),
-    (null, "10.0.0.2", 200, "2024-10-19 19:00:30"),
-    (null, "10.0.0.1", 300, "2024-10-19 19:01:00"),
-    (null, "10.0.0.2", 100, "2024-10-19 19:01:01"),
-    (null, "10.0.0.1", 400, "2024-10-19 19:01:30"),
-    (null, "10.0.0.2", 200, "2024-10-19 19:01:31");
-
-Affected Rows: 8
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('requests_long_term');
-
-+----------------------------------------+
-| ADMIN FLUSH_FLOW('requests_long_term') |
-+----------------------------------------+
-|  FLOW_FLUSHED  |
-+----------------------------------------+
-
-SELECT
-    *
-FROM
-    requests_without_ip;
-
-+--------------+-----+---------------------+
-| service_name | val | ts                  |
-+--------------+-----+---------------------+
-|              | 100 | 2024-10-18T19:00:00 |
-|              | 200 | 2024-10-18T19:00:30 |
-|              | 300 | 2024-10-18T19:01:00 |
-|              | 100 | 2024-10-18T19:01:01 |
-|              | 100 | 2024-10-19T19:00:00 |
-|              | 200 | 2024-10-19T19:00:30 |
-|              | 300 | 2024-10-19T19:01:00 |
-|              | 100 | 2024-10-19T19:01:01 |
-|              | 400 | 2024-10-19T19:01:30 |
-|              | 200 | 2024-10-19T19:01:31 |
-| svc1         | 100 | 2024-10-18T19:00:00 |
-| svc1         | 200 | 2024-10-18T19:00:30 |
-| svc1         | 400 | 2024-10-18T19:01:30 |
-| svc1         | 200 | 2024-10-18T19:01:31 |
-+--------------+-----+---------------------+
-
-INSERT INTO
-    requests
-VALUES
-    ("svc2", "10.0.0.1", 100, "2024-10-18 19:00:00"),
-    ("svc2", "10.0.0.2", 100, "2024-10-18 19:00:00"),
-    ("svc2", "10.0.0.1", 200, "2024-10-18 19:00:30"),
-    ("svc2", "10.0.0.2", 200, "2024-10-18 19:00:30"),
-    ("svc2", "10.0.0.1", 300, "2024-10-18 19:01:00"),
-    ("svc2", "10.0.0.2", 100, "2024-10-18 19:01:01"),
-    ("svc2", "10.0.0.1", 400, "2024-10-18 19:01:30"),
-    ("svc2", "10.0.0.2", 200, "2024-10-18 19:01:31");
-
-Affected Rows: 8
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('requests_long_term');
-
-+----------------------------------------+
-| ADMIN FLUSH_FLOW('requests_long_term') |
-+----------------------------------------+
-|  FLOW_FLUSHED  |
-+----------------------------------------+
-
-SELECT
-    *
-FROM
-    requests_without_ip;
-
-+--------------+-----+---------------------+
-| service_name | val | ts                  |
-+--------------+-----+---------------------+
-|              | 100 | 2024-10-18T19:00:00 |
-|              | 200 | 2024-10-18T19:00:30 |
-|              | 300 | 2024-10-18T19:01:00 |
-|              | 100 | 2024-10-18T19:01:01 |
-|              | 100 | 2024-10-19T19:00:00 |
-|              | 200 | 2024-10-19T19:00:30 |
-|              | 300 | 2024-10-19T19:01:00 |
-|              | 100 | 2024-10-19T19:01:01 |
-|              | 400 | 2024-10-19T19:01:30 |
-|              | 200 | 2024-10-19T19:01:31 |
-| svc1         | 100 | 2024-10-18T19:00:00 |
-| svc1         | 200 | 2024-10-18T19:00:30 |
-| svc1         | 400 | 2024-10-18T19:01:30 |
-| svc1         | 200 | 2024-10-18T19:01:31 |
-| svc2         | 100 | 2024-10-18T19:00:00 |
-| svc2         | 200 | 2024-10-18T19:00:30 |
-| svc2         | 300 | 2024-10-18T19:01:00 |
-| svc2         | 100 | 2024-10-18T19:01:01 |
-| svc2         | 400 | 2024-10-18T19:01:30 |
-| svc2         | 200 | 2024-10-18T19:01:31 |
-+--------------+-----+---------------------+
-
 DROP FLOW requests_long_term;

 Affected Rows: 0
--- a/tests/cases/standalone/common/flow/flow_basic.sql
+++ b/tests/cases/standalone/common/flow/flow_basic.sql
@@ -214,36 +214,6 @@ from
 GROUP BY
    time_window;

-INSERT INTO
-    bytes_log
-VALUES
-    (NULL, '2023-01-01 00:00:01'),
-    (300, '2023-01-01 00:00:29');
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('find_approx_rate');
-
-SELECT
-    rate,
-    time_window
-FROM
-    approx_rate;
-
-INSERT INTO
-    bytes_log
-VALUES
-    (NULL, '2022-01-01 00:00:01'),
-    (NULL, '2022-01-01 00:00:29');
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('find_approx_rate');
-
-SELECT
-    rate,
-    time_window
-FROM
-    approx_rate;
-
 INSERT INTO
    bytes_log
 VALUES
@@ -569,7 +539,6 @@ CREATE TABLE requests_without_ip (
    service_name STRING,
    val INT,
    ts TIMESTAMP TIME INDEX,
-    PRIMARY KEY(service_name)
 );

 CREATE FLOW requests_long_term SINK TO requests_without_ip AS
@@ -583,64 +552,18 @@ FROM
 INSERT INTO
    requests
 VALUES
-    (NULL, "10.0.0.1", 100, "2024-10-18 19:00:00"),
+    ("svc1", "10.0.0.1", 100, "2024-10-18 19:00:00"),
    ("svc1", "10.0.0.2", 100, "2024-10-18 19:00:00"),
-    (NULL, "10.0.0.1", 200, "2024-10-18 19:00:30"),
+    ("svc1", "10.0.0.1", 200, "2024-10-18 19:00:30"),
    ("svc1", "10.0.0.2", 200, "2024-10-18 19:00:30"),
-    (NULL, "10.0.0.1", 300, "2024-10-18 19:01:00"),
-    (NULL, "10.0.0.2", 100, "2024-10-18 19:01:01"),
+    ("svc1", "10.0.0.1", 300, "2024-10-18 19:01:00"),
+    ("svc1", "10.0.0.2", 100, "2024-10-18 19:01:01"),
    ("svc1", "10.0.0.1", 400, "2024-10-18 19:01:30"),
    ("svc1", "10.0.0.2", 200, "2024-10-18 19:01:31");

 -- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
 ADMIN FLUSH_FLOW('requests_long_term');

-SELECT
-    *
-FROM
-    requests_without_ip;
-
-- Test if FLOWS table works, but don't care about the result since it vary from runs
-SELECT
-    count(CASE WHEN state_size > 0 THEN 1 ELSE 0 END) as active_flows,
-FROM
-    INFORMATION_SCHEMA.FLOWS;
-
-INSERT INTO
-    requests
-VALUES
-    (null, "10.0.0.1", 100, "2024-10-19 19:00:00"),
-    (null, "10.0.0.2", 100, "2024-10-19 19:00:00"),
-    (null, "10.0.0.1", 200, "2024-10-19 19:00:30"),
-    (null, "10.0.0.2", 200, "2024-10-19 19:00:30"),
-    (null, "10.0.0.1", 300, "2024-10-19 19:01:00"),
-    (null, "10.0.0.2", 100, "2024-10-19 19:01:01"),
-    (null, "10.0.0.1", 400, "2024-10-19 19:01:30"),
-    (null, "10.0.0.2", 200, "2024-10-19 19:01:31");
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('requests_long_term');
-
-SELECT
-    *
-FROM
-    requests_without_ip;
-
-INSERT INTO
-    requests
-VALUES
-    ("svc2", "10.0.0.1", 100, "2024-10-18 19:00:00"),
-    ("svc2", "10.0.0.2", 100, "2024-10-18 19:00:00"),
-    ("svc2", "10.0.0.1", 200, "2024-10-18 19:00:30"),
-    ("svc2", "10.0.0.2", 200, "2024-10-18 19:00:30"),
-    ("svc2", "10.0.0.1", 300, "2024-10-18 19:01:00"),
-    ("svc2", "10.0.0.2", 100, "2024-10-18 19:01:01"),
-    ("svc2", "10.0.0.1", 400, "2024-10-18 19:01:30"),
-    ("svc2", "10.0.0.2", 200, "2024-10-18 19:01:31");
-
-- SQLNESS REPLACE (ADMIN\sFLUSH_FLOW\('\w+'\)\s+\|\n\+-+\+\n\|\s+)[0-9]+\s+\| $1 FLOW_FLUSHED  |
-ADMIN FLUSH_FLOW('requests_long_term');
-
 SELECT
    *
 FROM
--- a/tests/cases/standalone/common/system/information_schema.result
+++ b/tests/cases/standalone/common/system/information_schema.result
@@ -187,17 +187,16 @@ select * from information_schema.columns order by table_schema, table_name, colu
 | greptime      | information_schema | files                                 | update_count                      | 13               |                          |                        | 19                | 0             |                    |                    |                |            |       | select,insert |                       | Int64                | bigint          | FIELD         |                | No          | bigint          |                |        |
 | greptime      | information_schema | files                                 | update_time                       | 34               |                          |                        |                   |               | 3                  |                    |                |            |       | select,insert |                       | DateTime             | datetime        | FIELD         |                | No          | datetime        |                |        |
 | greptime      | information_schema | files                                 | version                           | 25               | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
-| greptime      | information_schema | flows                                 | comment                           | 6                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
-| greptime      | information_schema | flows                                 | expire_after                      | 7                |                          |                        | 19                | 0             |                    |                    |                |            |       | select,insert |                       | Int64                | bigint          | FIELD         |                | Yes         | bigint          |                |        |
-| greptime      | information_schema | flows                                 | flow_definition                   | 5                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
+| greptime      | information_schema | flows                                 | comment                           | 5                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
+| greptime      | information_schema | flows                                 | expire_after                      | 6                |                          |                        | 19                | 0             |                    |                    |                |            |       | select,insert |                       | Int64                | bigint          | FIELD         |                | Yes         | bigint          |                |        |
+| greptime      | information_schema | flows                                 | flow_definition                   | 4                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
 | greptime      | information_schema | flows                                 | flow_id                           | 2                |                          |                        | 10                | 0             |                    |                    |                |            |       | select,insert |                       | UInt32               | int unsigned    | FIELD         |                | No          | int unsigned    |                |        |
 | greptime      | information_schema | flows                                 | flow_name                         | 1                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
-| greptime      | information_schema | flows                                 | flownode_ids                      | 10               | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
-| greptime      | information_schema | flows                                 | options                           | 11               | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
-| greptime      | information_schema | flows                                 | sink_table_name                   | 9                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
-| greptime      | information_schema | flows                                 | source_table_ids                  | 8                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
-| greptime      | information_schema | flows                                 | state_size                        | 3                |                          |                        | 20                | 0             |                    |                    |                |            |       | select,insert |                       | UInt64               | bigint unsigned | FIELD         |                | Yes         | bigint unsigned |                |        |
-| greptime      | information_schema | flows                                 | table_catalog                     | 4                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
+| greptime      | information_schema | flows                                 | flownode_ids                      | 9                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
+| greptime      | information_schema | flows                                 | options                           | 10               | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
+| greptime      | information_schema | flows                                 | sink_table_name                   | 8                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
+| greptime      | information_schema | flows                                 | source_table_ids                  | 7                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | Yes         | string          |                |        |
+| greptime      | information_schema | flows                                 | table_catalog                     | 3                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
 | greptime      | information_schema | global_status                         | variable_name                     | 1                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
 | greptime      | information_schema | global_status                         | variable_value                    | 2                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
 | greptime      | information_schema | key_column_usage                      | column_name                       | 8                | 2147483647               | 2147483647             |                   |               |                    | utf8               | utf8_bin       |            |       | select,insert |                       | String               | string          | FIELD         |                | No          | string          |                |        |
Author	SHA1	Message	Date
evenyag	9d3dc2d311	chore: Merge branch 'main' into chore/bench-metrics	2024-12-19 16:07:43 +08:00
Ruihang Xia	422d18da8b	feat: bump opendal and switch prometheus layer to the upstream impl (#5179 ) * feat: bump opendal and switch prometheus layer to the upstream impl Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove unused files Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix tests Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove unused things Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove root dir on recovering cache Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * filter out non-files entry in test Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2024-12-19 03:42:05 +00:00
Weny Xu	66f0581f5b	fix: ensure table route metadata is eventually rolled back on failure (#5174 ) * fix: ensure table route metadata is eventually rolled back on procedure failure * fix(fuzz): enhance procedure condition checking * chore: add logs * feat: close downgraded leader region actively * chore: apply suggestions from CR	2024-12-19 03:29:34 +00:00
Ning Sun	c9ad8c7101	feat: show create postgresql foreign table (#5143 ) * feat: add show create table for pg in parser * feat: implement show create table operation * fix: adopt upstream changes	2024-12-18 15:15:55 +00:00
Ning Sun	2107737db1	chore: make nix compilation environment config more robust (#5183 ) * chore: improve nix-shell support * fix: add pkg-config * ci: add a github action to ensure build on clean system * ci: optimise dependencies of task * ci: move clean build to nightly	2024-12-18 12:41:24 +00:00
Ruihang Xia	548e1988ab	refactor: remove unused symbols (#5193 ) chore: remove unused symbols Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2024-12-18 11:24:43 +00:00
Yingwen	218236cc5b	docs: fix grafana dashboard row (#5192 )	2024-12-18 09:10:56 +00:00
Ruihang Xia	f04d380259	fix: validate matcher op for __name__ in promql (#5191 ) Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2024-12-18 08:51:46 +00:00
Lei, HUANG	fa773cf480	fix(sqlness): enforce order in union tests (#5190 ) Add ORDER BY clause to subquery union tests Updated the SQL and result files for subquery union tests to include an ORDER BY clause, ensuring consistent result ordering. This change aligns with the test case from the DuckDB repository.	2024-12-18 08:24:15 +00:00
jeremyhi	9b4e8555e2	feat: extract hints from http header (#5128 ) * feat: extract hints from http header * Update src/servers/src/http/hints.rs Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com> * chore: by comment * refactor: get instead of loop --------- Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>	2024-12-18 08:17:34 +00:00
Yingwen	c6b7caa2ec	feat: do not remove time filters in ScanRegion (#5180 ) * feat: do not remove time filters * chore: remove `time_range` from parquet reader * chore: print more message in the check script * chore: fix unused error	2024-12-18 06:39:49 +00:00
Yingwen	58d6982c93	feat: do not keep MemtableRefs in ScanInput (#5184 )	2024-12-18 06:37:22 +00:00
dennis zhuang	e662c241e6	feat: impl label_join and label_replace for promql (#5153 ) * feat: impl label_join and label_replace for promql * chore: style * fix: dst_label is eqauls to src_label * fix: forgot to sort the results * fix: processing empty source label	2024-12-18 06:35:45 +00:00
Lanqing Yang	266919c226	fix: display inverted and fulltext index in show index (#5169 )	2024-12-18 06:10:59 +00:00
Yohan Wal	7d1bcc9d49	feat: introduce Buffer for non-continuous bytes (#5164 ) * feat: introduce Buffer for non-continuous bytes * Update src/mito2/src/cache/index.rs Co-authored-by: Weny Xu <wenymedia@gmail.com> * chore: apply review comments * refactor: use opendal::Buffer --------- Co-authored-by: Weny Xu <wenymedia@gmail.com>	2024-12-18 03:45:38 +00:00
LFC	18e8c45384	refactor: produce BatchBuilder from a Batch to modify it again (#5186 ) chore: pub some mods	2024-12-18 02:42:33 +00:00
Lei, HUANG	c33cf59398	perf: avoid holding memtable during compaction (#5157 ) * perf/avoid-holding-memtable-during-compaction: Refactor Compaction Version Handling • Introduced CompactionVersion struct to encapsulate region version details for compaction, removing dependency on VersionRef. • Updated CompactionRequest and CompactionRegion to use CompactionVersion. • Modified open_compaction_region to construct CompactionVersion without memtables. • Adjusted WindowedCompactionPicker to work with CompactionVersion. • Enhanced flush logic in WriteBufferManager to improve memory usage checks and logging. * reformat code * chore: change log level * reformat code --------- Co-authored-by: Yingwen <realevenyag@gmail.com>	2024-12-17 07:06:07 +00:00
discord9	421088a868	test: sqlness upgrade compatibility tests (#5126 ) * feat: simple version switch * chore: remove debug print * chore: add common folder * tests: add drop table * feat: pull versioned binary * chore: don't use native-tls * chore: rm outdated docs * chore: new line * fix: save old bin dir * fix: switch version restart all node * feat: use etcd * fix: wait for election * fix: normal sqlness * refactor: hashmap for bin dir * test: past 3 major version compat crate table * refactor: allow using without setup etcd	2024-12-17 07:00:02 +00:00
Zhenchi	d821dc5a3e	feat(bloom-filter): add basic bloom filter creator (Part 1) (#5177 ) * feat(bloom-filter): add a simple bloom filter creator (Part 1) Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix: clippy Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix: header Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * docs: add format comment Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> --------- Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>	2024-12-17 06:55:42 +00:00
Yingwen	bfc777e6ac	fix: deletion between two put may not work in `last_non_null` mode (#5168 ) * fix: deletion between rows with the same key may not work * test: add sqlness test case * chore: comments	2024-12-17 04:01:32 +00:00
Yingwen	8a5384697b	chore: add aquamarine to dep lists (#5181 )	2024-12-17 01:45:50 +00:00
Weny Xu	d0245473a9	fix: correct `set_region_role_state_gracefully` behaviors (#5171 ) * fix: reduce default max rows for fuzz testing * chore: remove Postgres setup from fuzz test workflow * chore(fuzz): increase resource limits for GreptimeDB cluster * chore(fuzz): increase resource limits for kafka * fix: correct `set_region_role_state_gracefully` behaviors * chore: remove Postgres setup from fuzz test workflow * chore(fuzz): redue resource limits for GreptimeDB & kafka	2024-12-16 14:01:40 +00:00
discord9	043d0bd7c2	test: flow rebuild (#5162 ) * tests: rebuild flow * tests: more rebuild * tests: restart * chore: drop clean	2024-12-16 12:25:23 +00:00
Ning Sun	acedff030b	chore: add nix-shell configure for a minimal environment for development (#5175 ) * chore: add nix-shell development environment * chore: add rust-analyzer * chore: use .envrc as a private file	2024-12-16 11:47:18 +00:00
ZonaHe	88f7075a2a	feat: update dashboard to v0.7.3 (#5172 ) Co-authored-by: sunchanglong <sunchanglong@users.noreply.github.com>	2024-12-16 10:56:41 +00:00
Ruihang Xia	54698325b6	feat: introduce SKIPPING index (part 1) (#5155 ) * skip index parser Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * wip: sqlness Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * impl show create part Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * add empty line Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * change keyword to SKIPPING INDEX Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * rename local variables Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2024-12-16 09:21:00 +00:00
Lei, HUANG	5ffda7e971	chore: gauge for flush compaction (#5156 ) * add metrics * chore/bench-metrics: Add INFLIGHT_FLUSH_COUNT Metric to Flush Process • Introduced INFLIGHT_FLUSH_COUNT metric to track the number of ongoing flush operations. • Incremented INFLIGHT_FLUSH_COUNT in FlushScheduler to monitor active flushes. • Removed redundant increment of INFLIGHT_FLUSH_COUNT in RegionWorkerLoop to prevent double counting. * chore/bench-metrics: Add Metrics for Compaction and Flush Operations • Introduced INFLIGHT_COMPACTION_COUNT and INFLIGHT_FLUSH_COUNT metrics to track the number of ongoing compaction and flush operations. • Incremented INFLIGHT_COMPACTION_COUNT when scheduling remote and local compaction jobs, and decremented it upon completion. • Added INFLIGHT_FLUSH_COUNT increment and decrement logic around flush tasks to monitor active flush operations. • Removed redundant metric updates in worker.rs and handle_compaction.rs to streamline metric handling. * chore: add metrics for remote compaction jobs * chore: format * chore: also add dashbaord	2024-12-16 07:08:07 +00:00
Lin Yihai	f82af15eba	feat: Add `vector_scalar_mul` function. (#5166 )	2024-12-16 06:46:38 +00:00
shuiyisong	9d7fea902e	chore: remove unused dep (#5163 ) * chore: remove unused dep * chore: remove more unused dep	2024-12-16 06:17:27 +00:00
Niwaka	358d5e1d63	fix: support alter table ~ add ~ custom_type (#5165 )	2024-12-15 09:05:29 +00:00
Yingwen	579059d99f	ci: use 4xlarge for nightly build (#5158 )	2024-12-13 12:53:11 +00:00
localhost	53d55c0b6b	fix: loki write row len error (#5161 )	2024-12-13 10:10:59 +00:00
Yingwen	bef6896280	docs: Add index panels to standalone grafana dashboard (#5140 ) * docs: Add index panels to standalnoe grafana dashboard * docs: fix flush/compaction op	2024-12-13 08:17:49 +00:00
Yohan Wal	4b4c6dbb66	refactor: cache inverted index with fixed-size page (#5114 ) * feat: cache inverted index by page instead of file * fix: add unit test and fix bugs * chore: typo * chore: ci * fix: math * chore: apply review comments * chore: renames * test: add unit test for index key calculation * refactor: use ReadableSize * feat: add config for inverted index page size * chore: update config file * refactor: handle multiple range read and fix some related bugs * fix: add config * test: turn to a fs reader to match behaviors of object store	2024-12-13 07:34:24 +00:00
Lei, HUANG	0c302ba127	fix: handle stall metrics	2024-12-13 11:08:11 +08:00
Lei, HUANG	7139ba08c8	chore/bench-metrics: Add configurable slow threshold for region worker • Introduced slow_threshold environment variable to set a custom threshold for slow operations, defaulting to 1000 milliseconds. • Updated RegionWorkerLoop to use slow_threshold for performance monitoring. • Adjusted logic to include select_cost in the slow operation check.	2024-12-13 10:52:18 +08:00
Lei, HUANG	f3e0a31e5d	chore/bench-metrics: Add Metrics for Compaction and Flush Operations • Introduced INFLIGHT_COMPACTION_COUNT and INFLIGHT_FLUSH_COUNT metrics to track the number of ongoing compaction and flush operations. • Incremented INFLIGHT_COMPACTION_COUNT when scheduling remote and local compaction jobs, and decremented it upon completion. • Added INFLIGHT_FLUSH_COUNT increment and decrement logic around flush tasks to monitor active flush operations. • Removed redundant metric updates in worker.rs and handle_compaction.rs to streamline metric handling.	2024-12-12 21:38:13 +08:00
Lei, HUANG	36c82121fb	chore/bench-metrics: Add INFLIGHT_FLUSH_COUNT Metric to Flush Process • Introduced INFLIGHT_FLUSH_COUNT metric to track the number of ongoing flush operations. • Incremented INFLIGHT_FLUSH_COUNT in FlushScheduler to monitor active flushes. • Removed redundant increment of INFLIGHT_FLUSH_COUNT in RegionWorkerLoop to prevent double counting.	2024-12-12 21:38:13 +08:00
evenyag	716bb82d37	feat: add slow metrics for worker	2024-12-12 21:13:07 +08:00
Lei, HUANG	2bb450b09a	add metrics	2024-12-12 20:23:01 +08:00
localhost	e8e9526738	chore: pipeline dryrun api can currently receives pipeline raw content (#5142 ) * chore: pipeline dryrun api can currently receives pipeline raw content * chore: remove dryrun v1 and add test * chore: change dryrun pipeline api body schema * chore: remove useless struct PipelineInfo * chore: update PipelineDryrunParams doc * chore: increase code readability * chore: add some comment for pipeline dryrun test * Apply suggestions from code review Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com> * chore: format code --------- Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>	2024-12-12 11:47:21 +00:00
Yingwen	fee75a1fad	feat: collect reader metrics from prune reader (#5152 )	2024-12-12 11:27:22 +00:00
localhost	b8a78b7838	chore: decide tag column in log api follow table schema if table exists (#5138 ) * chore: decide tag column in log api follow table schema if table exists * chore: add more test for greptime_identity pipeline * chore: change pipeline get_table function signature * chore: change identity_pipeline_inner tag_column_names type	2024-12-12 09:01:21 +00:00
Weny Xu	2137c53274	feat(index): add `file_size_hint` for remote blob reader (#5147 ) feat(index): add file_size_hint for remote blob reader	2024-12-12 04:45:40 +00:00
Yohan Wal	03ad6e2a8d	feat(fuzz): add alter table options for alter fuzzer (#5074 ) * feat(fuzz): add set table options to alter fuzzer * chore: clippy is happy, I'm sad * chore: happy ci happy * fix: unit test * feat(fuzz): add unset table options to alter fuzzer * fix: unit test * feat(fuzz): add table option validator * fix: make clippy happy * chore: add comments * chore: apply review comments * fix: unit test * feat(fuzz): add more ttl options * fix: #5108 * chore: add comments * chore: add comments	2024-12-12 04:21:38 +00:00
Weny Xu	d53fbcb936	feat: introduce `PuffinMetadataCache` (#5148 ) * feat: introduce `PuffinMetadataCache` * refactor: remove too_many_arguments * chore: fmt toml	2024-12-12 04:09:36 +00:00
Weny Xu	8c1959c580	feat: add prefetch support to `InvertedIndexFooterReader` for reduced I/O time (#5146 ) * feat: add prefetch support to `InvertedIndeFooterReader` * chore: correct struct name * chore: apply suggestions from CR	2024-12-12 03:49:54 +00:00
Weny Xu	e2a41ccaec	feat: add prefetch support to `PuffinFileFooterReader` for reduced I/O time (#5145 ) * feat: introduce `PuffinFileFooterReader` * refactor: remove `SyncReader` trait and impl * refactor: replace `FooterParser` with `PuffinFileFooterReader` * chore: remove unused errors	2024-12-12 03:13:36 +00:00
Niwaka	a8012147ab	feat: support push down IN filter (#5129 ) * feat: support push down IN filter * chore: move tests to prune.sql	2024-12-11 13:46:23 +00:00
Ruihang Xia	60f8dbf7f0	feat: implement `v1/sql/parse` endpoint to parse GreptimeDB's SQL dialect (#5144 ) * derive ser/de Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * impl method Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix typo Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * remove deserialize Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2024-12-11 13:33:54 +00:00
ZonaHe	9da2e17d0e	feat: update dashboard to v0.7.2 (#5141 ) Co-authored-by: sunchanglong <sunchanglong@users.noreply.github.com>	2024-12-11 12:47:59 +00:00
Yohan Wal	1a8e77a480	test: part of parser test migrated from duckdb (#5125 ) * test: update test * fix: fix test	2024-12-11 09:28:13 +00:00
Zhenchi	e1e39993f7	feat(vector): add scalar add function (#5119 ) * refactor: extract implicit conversion helper functions of vector Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * feat(vector): add scalar add function Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix fmt Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> --------- Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>	2024-12-11 09:25:56 +00:00
Lei, HUANG	a30d918df2	perf: avoid cache during compaction (#5135 ) * Revert "refactor: Avoid wrapping Option for CacheManagerRef (#4996)" This reverts commit `42bf7e9965`. * fix: memory usage during log ingestion * fix: fmt	2024-12-11 08:24:41 +00:00
dennis zhuang	2c4ac76754	feat: adjust WAL purge default configurations (#5107 ) * feat: adjust WAL purge default configurations * fix: config * feat: change raft engine file_size default to 128Mib	2024-12-11 08:08:05 +00:00
jeremyhi	a6893aad42	chore: set store_key_prefix for all kvbackend (#5132 )	2024-12-11 08:04:02 +00:00
discord9	d91517688a	chore: fix aws_lc not in depend tree check in CI (#5121 ) * chore: fix aws_lc check in CI * chore: update lock file	2024-12-11 07:02:03 +00:00
shuiyisong	3d1b8c4fac	chore: add `/ready` api for health checking (#5124 ) * chore: add ready endpoint for health checking * chore: add test	2024-12-11 02:56:48 +00:00
Yingwen	7c69ca0502	chore: bump main branch version to 0.12 (#5133 ) chore: bump version to v0.12.0	2024-12-10 13:10:37 +00:00