Commit Graph

3347 Commits

Author SHA1 Message Date
Alex Chi Z
d558b547e8 probably better strategy?
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-10 08:04:59 -04:00
Alex Chi Z
569ed35c91 trivial move switch
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-07 15:07:33 -04:00
Alex Chi Z
f258f50b76 pagectl: separate xy margin for draw timeline
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-07 12:14:08 -04:00
Alex Chi Z
f31cc2394b compaction PoC: subcompaction (#4656)
This PR adds subcompaction support for compaction PoC. For compaction
job >= 4GB, it will be split into 4 threads.

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-07 12:13:44 -04:00
Alex Chi Z
7e7cdaa3eb handle name conflict
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-05 16:26:02 -04:00
Alex Chi Z
a079d250d9 compaction PoC: trivial move compaction (#4604)
reduce write amp. for bulk load, might also be useful for main branch

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-05 15:27:25 -04:00
Alex Chi Z
756319b0ce do not excldue last tier
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-07-03 14:02:53 -04:00
Alex Chi Z
8816fc98fc fix compaction algorithm
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 18:11:51 -04:00
Alex Chi Z
c3bcaa0551 rm println
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 15:02:30 -04:00
Alex Chi Z
8aede79abf fix
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 14:57:42 -04:00
Alex Chi Z
d28e309c06 fix
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 14:48:54 -04:00
Alex Chi Z
647b7a70a8 fix
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 14:31:00 -04:00
Alex Chi Z
e7955895d1 fix
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 14:18:46 -04:00
Alex Chi Z
dc9c842d21 true incremental
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 14:05:28 -04:00
Alex Chi Z
05719cb9cd debug
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 12:42:15 -04:00
Alex Chi Z
0051a6c931 debug
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 12:27:36 -04:00
Alex Chi Z
c9c40171cd fix layer map
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 11:29:41 -04:00
Alex Chi Z
376762e07e fewer logs
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 09:16:46 -04:00
Alex Chi Z
d4e262f646 delta with correct range
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-29 09:11:15 -04:00
Alex Chi Z
d279b4421e increase threshold
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 15:39:52 -04:00
Alex Chi Z
b1f0bbd12a add reduce num sorted run trigger
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 15:38:14 -04:00
Alex Chi Z
7d16a9f96f fix again
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 15:07:31 -04:00
Alex Chi Z
878627161c revert not
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 14:58:42 -04:00
Alex Chi Z
4db4f42dec correctly handle compaction
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 14:41:26 -04:00
Alex Chi Z
6cb149e3c3 enable tiered again
Signed-off-by: Alex Chi Z <chi@neon.tech>
2023-06-28 14:30:17 -04:00
Alex Chi
f3fdaf8ef1 parallel compaction
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 16:42:56 -04:00
Alex Chi
eb93e686ab fix deletion
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 14:47:04 -04:00
Alex Chi
2cb79ae3ff fix deletion
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 14:43:20 -04:00
Alex Chi
dfe8527806 remove assertion
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 13:52:12 -04:00
Alex Chi
335710cec6 bring back original compaction
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 13:38:02 -04:00
Alex Chi
a78008ad82 max_merge_width
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-27 13:30:23 -04:00
Alex Chi
30e7ffcd28 adjust compaction strategy
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-26 15:52:37 -04:00
Alex Chi
43d564ce0a incremental image layer
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-26 15:25:35 -04:00
Alex Chi
f86ff5e54b dump more
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-26 14:57:00 -04:00
Alex Chi
9ed6ad1d24 fix weak ptr
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-26 14:33:00 -04:00
Alex Chi
91f28cb516 include delta l0 in compaction, more metrics
Signed-off-by: Alex Chi <chi@neon.tech>
2023-06-26 13:56:29 -04:00
Alex Chi
0b459eb414 fix ratio compute
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 15:11:15 -04:00
Alex Chi
0865ed623c fix comment
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 15:01:25 -04:00
Alex Chi
9e0f103c7b insert at 0
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 15:00:58 -04:00
Alex Chi
9f216a78a1 print
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 15:00:10 -04:00
Alex Chi
6967b4837b fix
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 14:53:53 -04:00
Alex Chi
9b50350857 threshold = 3
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 14:37:29 -04:00
Alex Chi
8ebfa32a0c compaction l0 adds to sorted runs
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 14:24:57 -04:00
Alex Chi
9905d75715 dump file size
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 14:18:54 -04:00
Alex Chi
b0b616f3ac dump
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 14:12:33 -04:00
Alex Chi
820685fe92 remove all contents
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 13:52:34 -04:00
Alex Chi
a593d96b79 neon_local: support force init
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 13:52:28 -04:00
Alex Chi
867b656ef2 bypass ut
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-22 11:28:34 -04:00
Alex Chi
76b339b150 create partial image layers
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-21 14:38:11 -04:00
Alex Chi
9b3fa1a2e1 fix compile error
Signed-off-by: Alex Chi <iskyzh@gmail.com>
2023-06-21 14:16:44 -04:00