mirror of
https://github.com/l0ng-ai/openidea.git
synced 2026-11-19 00:10:00 +00:00
The initial scan parsed files in 512-file chunks (rayon join, then insert and redb commit), so every join idled the pool on its slowest file and insert/persist serialized between chunks. The scan now parses the whole todo list in one rayon pass, serializes cache records inside the parse, and streams them over a bounded channel to the index thread, which inserts and persists in batches of 2048 alongside the parse. Records stay byte-identical; the scan's log line carries walk / classify / extract / insert / persist / durable laps. Co-authored-by: akbar1214 <43771058+akbar1214@users.noreply.github.com>