Commit Graph

  • 84169c926a Merge branch 'problame/2024-02-walredo-work/prespawn/broken-tenants-no-walredo' into problame/2024-02-walredo-work/prespawn/heaver-once-cell-for-process-launch Christian Schwarz 2024-02-02 15:53:57 +00:00
  • acdebf2cec Merge branch 'problame/2024-02-walredo-work/prespawn/split-code' into problame/2024-02-walredo-work/prespawn/broken-tenants-no-walredo Christian Schwarz 2024-02-02 15:53:56 +00:00
  • 44cb5e5be6 Merge branch 'main' into problame/2024-02-walredo-work/prespawn/split-code Christian Schwarz 2024-02-02 15:53:55 +00:00
  • 46fb1a90ce pageserver: avoid calculating/sending logical sizes on shard !=0 (#6567) John Spray 2024-02-02 15:52:03 +00:00
  • 2ab2608d4c [DO NOT MERGE] refactor(walredo): use replace RwLock with heavier_once_cell Christian Schwarz 2024-02-02 15:36:15 +00:00
  • de7d366df3 wip Christian Schwarz 2024-02-02 15:25:39 +00:00
  • 7c1b2dc9ef Merge branch 'problame/2024-02-walredo-work/prespawn/broken-tenants-no-walredo' into problame/2024-02-walredo-work/prespawn/impl Christian Schwarz 2024-02-02 14:56:29 +00:00
  • f73aa3eb32 refactor(walredo): avoid the need for a WalRedoManager in broken tenants Christian Schwarz 2024-02-02 14:52:53 +00:00
  • 2374e1318e Merge branch 'main' into problame/2024-02-walredo-work/prespawn/split-code Christian Schwarz 2024-02-02 14:42:30 +00:00
  • 8fe3c9ff55 wip Christian Schwarz 2024-02-02 14:42:00 +00:00
  • 8e8890530c wip Christian Schwarz 2024-02-02 14:26:26 +00:00
  • 56171cbe8c pageserver: more permissive activation timeout when testing (#6564) John Spray 2024-02-02 14:14:42 +00:00
  • 48b05b7c50 Add a time_travel_remote_storage http endpoint (#6533) Arpad Müller 2024-02-02 14:52:12 +01:00
  • 0856fe6676 proxy: remove per client bytes (#5466) Conrad Ludgate 2024-02-02 12:28:48 +00:00
  • 014147a644 wip Christian Schwarz 2024-02-02 11:50:43 +00:00
  • aa0e9fdaef Merge branch 'main' into problame/2024-02-walredo-work/prespawn/split-code Christian Schwarz 2024-02-02 11:50:15 +00:00
  • 4133d14a77 Compute: pgbouncer 1.22.0 (#6582) Alexander Bayandin 2024-02-02 11:49:11 +00:00
  • 30c9e145d7 check-macos-build: switch job to macos-14 (M1) (#6539) Alexander Bayandin 2024-02-02 10:51:20 +00:00
  • 24e916d37f pageserver: fix a syntax error in swagger (#6566) John Spray 2024-02-02 10:35:09 +00:00
  • 23f58145ed Update wording for better readability (#6559) Andreas Scherbaum 2024-02-02 11:22:32 +01:00
  • 9b8aa270b8 cleanups Christian Schwarz 2024-02-02 10:19:18 +00:00
  • 4571db1750 extract NeonWalRecord apply logic Christian Schwarz 2024-02-02 10:14:50 +00:00
  • 6fe534fea3 move protocol ad child module of process, where it belongs Christian Schwarz 2024-02-02 10:05:50 +00:00
  • 8b258e20a0 move more stuff around Christian Schwarz 2024-02-02 10:03:40 +00:00
  • 29eec6c563 split off walredo process & protocol from walredo.rs Christian Schwarz 2024-02-02 09:59:31 +00:00
  • bbab418a7e proxy: remove otel subscriber proxy-remote-otel Conrad Ludgate 2024-02-02 08:28:53 +00:00
  • 350865392c Print checkpoint key contents with "pagectl print-layer-file" (#6541) Heikki Linnakangas 2024-02-02 01:35:31 +02:00
  • 1be5e564ce feat(walredo): use posix_spawn by moving close_fds() work to walredo C code (#6574) Christian Schwarz 2024-02-01 22:38:34 +01:00
  • 7a70ef991f feat(walredo): various observability improvements (#6573) Christian Schwarz 2024-02-01 21:59:40 +01:00
  • be30388901 Add retry to fetching basebackup (#6537) Sasha Krassovsky 2024-02-01 11:50:04 -09:00
  • 3525080031 Fix pgvector 0.6.0 with Neon. (#6571) Heikki Linnakangas 2024-02-01 22:48:31 +02:00
  • 527cdbc010 Don't require AWS access keys for S3 pytests (#6556) Arpad Müller 2024-02-01 21:18:07 +01:00
  • 39be2b0108 Makefile: set PQ_LIB_DIR to avoid linkage with system libpq (#6538) Alexander Bayandin 2024-02-01 17:34:48 +00:00
  • 91cec9ba48 add comment problame/2024-02-walredo-work/2-autokill-if-only-errors Christian Schwarz 2024-02-01 17:26:39 +00:00
  • 10df5a650b wip vlad/vectored-value-reconstruction Vlad Lazar 2024-02-01 16:26:43 +00:00
  • d37b99c7b1 pageserver: lift ancestor timeline logic from read path Vlad Lazar 2024-01-31 10:33:59 +00:00
  • d9f89f828d only update in apply_batch_postgres Christian Schwarz 2024-02-01 16:17:21 +00:00
  • 6d71a4fd31 Ignore zero sized cluster size in pageserver feedback. wp-ignore-reported-zero-size Arseny Sher 2024-02-01 16:58:14 +03:00
  • 74df4a7b76 fix(walredo): walredo process that causes errors is never killed Christian Schwarz 2024-02-01 13:41:33 +01:00
  • fa52cd575e Remove old tests results and old coverage collection (#6376) Alexander Bayandin 2024-02-01 13:36:55 +00:00
  • d2c410c748 pageserver_api: remove overlaps from KeySpace (#6544) Vlad Lazar 2024-02-01 13:14:35 +00:00
  • 221531c9db pageserver: lift ancestor timeline logic from read path (#6543) Vlad Lazar 2024-02-01 10:35:18 +00:00
  • 4c173456dc pagebench: fix percentiles reporting (#6547) Christian Schwarz 2024-02-01 00:29:48 +01:00
  • e82625b77d refactor(pageserver main): signal handling (#6554) Christian Schwarz 2024-02-01 00:25:57 +01:00
  • 0ac1e71524 update tokio-epoll-uring (#6558) Christian Schwarz 2024-01-31 23:54:54 +01:00
  • 271133d960 Proxy: reduce number of get role secret calls (#6557) Anna Khanova 2024-01-31 23:16:56 +01:00
  • 3d5fab127a rewrite Gate impl for better observability (#6542) Joonas Koivunen 2024-02-01 00:15:58 +02:00
  • 66719d7eaf logging: fix span usage (#6549) Joonas Koivunen 2024-01-31 22:52:00 +02:00
  • 9a9d9beaee Download SLRU segments on demand (#6151) Konstantin Knizhnik 2024-01-31 21:39:18 +02:00
  • 2bfc831c60 control_plane/attachment_service: make --path optional (#6545) John Spray 2024-01-31 17:02:41 +00:00
  • 96413743e4 move poison to utils and document Christian Schwarz 2024-01-31 16:24:19 +00:00
  • b1b8ca32c8 working impl Christian Schwarz 2024-01-31 16:08:40 +00:00
  • 70b37cf88f WIP poison Christian Schwarz 2024-01-31 15:57:34 +00:00
  • 799db161d3 tests: support for running on single pg version, use in one place (#6525) Joonas Koivunen 2024-01-31 17:37:25 +02:00
  • 47380be12d Remove version param from get_lsn_by_timestamp (#6551) Arpad Müller 2024-01-31 15:30:19 +01:00
  • c7b02ce8ec proxy: use jemalloc (#6531) Conrad Ludgate 2024-01-31 13:51:11 +00:00
  • 4160d407fb cfg(testing) still needs io::Write Christian Schwarz 2024-01-31 13:46:26 +00:00
  • 4010adf653 control_plane/attachment_service: complete APIs (#6394) John Spray 2024-01-31 12:23:06 +00:00
  • a29ac8b8a2 clippy (again?) Christian Schwarz 2024-01-31 13:14:36 +01:00
  • 639ed3cb3c clippy + compile errors Christian Schwarz 2024-01-31 12:26:27 +01:00
  • 2736f61604 error handling Christian Schwarz 2024-01-31 12:22:21 +01:00
  • 8012b80a45 some cleanup work Christian Schwarz 2024-01-31 12:17:05 +01:00
  • a93be15360 remove wal_redo_timeout Christian Schwarz 2024-01-31 12:15:18 +01:00
  • 2c1652a02b WIP: async walredo Christian Schwarz 2024-01-27 12:47:29 +00:00
  • 4c6627f3d9 pageserver_api: remove overlaps from KeySpace Vlad Lazar 2024-01-31 10:15:40 +00:00
  • e10a7ee391 Prevent to frequent reconnects in case of race condition errors returned by PS (tenant not found) (#6522) Konstantin Knizhnik 2024-01-31 09:17:32 +02:00
  • e8c9a51273 Allow creating subscriptions as neon_superuser (#6484) Sasha Krassovsky 2024-01-30 22:32:33 -08:00
  • 3c3ee8f3e8 Compute: add compatibility patch for pgvector (#6527) Alexander Bayandin 2024-01-30 17:33:24 +00:00
  • 6928a34f59 S3 DR: Large prefix improvements (#6515) Arpad Müller 2024-01-30 16:57:27 +01:00
  • f55021ee76 pageserver: fix noisy logging on tenant NotFound jcsp/page-service-not-found-spam John Spray 2024-01-30 15:18:16 +00:00
  • d121dd5806 implement request-scoped LRU cache problame/integrate-tokio-epoll-uring/benchmarking/2024-01-30--2--benchmark-against-main Christian Schwarz 2024-01-29 18:22:00 +00:00
  • a4ce99bc07 use O_DIRECT for VirtualFile reads Christian Schwarz 2024-01-29 16:20:24 +00:00
  • 74633a33ba rip out materialized page cache Christian Schwarz 2024-01-29 14:45:02 +00:00
  • f4fbaf0ba4 add back page cache but not for DeltaLayerValue and ImageLayerValue Christian Schwarz 2024-01-29 14:35:55 +00:00
  • ea36e4b874 rip out page cache Christian Schwarz 2023-08-29 15:17:40 +00:00
  • 9f24321d13 hack: use a single runtime in pageserver problame/integrate-tokio-epoll-uring/benchmarking/2024-01-30--2--benchmark-against-main--baseline Christian Schwarz 2024-01-29 12:23:25 +00:00
  • 87e501a273 hacky: ability to set io_engine via mgmt_api => pagebench Christian Schwarz 2024-01-29 10:50:20 +00:00
  • e3fb145f0f TODO/workaround: walredo quiescing broken with compaction_period=0 Christian Schwarz 2024-01-27 12:47:57 +00:00
  • 03abbaaddf WIP: async walredo Christian Schwarz 2024-01-27 12:47:29 +00:00
  • 6f488b3bd3 pagebench: fix percentiles reporting Christian Schwarz 2024-01-27 12:46:36 +00:00
  • 1daeba6d87 another attempt to reduce allocations, don't know if helpers, certainly didn't eliminate all of them problame/integrate-tokio-epoll-uring/benchmarking/2024-01-30--1-try-reduce-allocations Christian Schwarz 2024-01-30 14:10:20 +00:00
  • bc684e9d3b Make WAL segment init atomic. Arseny Sher 2024-01-26 16:51:41 +03:00
  • 08532231ee Fix find_end_of_wal busy loop. Arseny Sher 2024-01-24 10:39:53 +03:00
  • 79137a089f fix(#6366): pageserver: incorrect log level for Tenant not found during basebackup (#6400) Christian Schwarz 2024-01-30 14:10:48 +01:00
  • e3cb715e8a fix: capture initdb stderr, discard others (#6524) Joonas Koivunen 2024-01-30 15:07:58 +02:00
  • cfd78950e2 basic sk bench of pgbench init with perf fixtures sk-basic-bench Arseny Sher 2024-01-30 14:22:21 +03:00
  • c70bf9150f build(deps): bump aiohttp from 3.9.0 to 3.9.2 (#6518) dependabot[bot] 2024-01-30 10:46:49 +00:00
  • f3e1ae6740 try (and fail) to implement borrowed deserialize of Value Christian Schwarz 2024-01-30 10:43:24 +00:00
  • de8076d97d use smallvec & pooling to avoid allocations on reconstruction path Christian Schwarz 2024-01-30 09:37:53 +00:00
  • 8e4da52069 Compute: pgvector 0.6.0 (#6517) Alexander Bayandin 2024-01-30 09:29:45 +00:00
  • a28cdf1c28 wal-redo: consume reconstruct state as references (needed for next patch, useful indepdendently) Christian Schwarz 2024-01-30 09:11:16 +00:00
  • 9cdc9200cb Retroactively fix the nextXid on a known broken timeline (v3) one-off-nextxid-hack-3 Heikki Linnakangas 2024-01-30 11:01:33 +02:00
  • 3cd4f8aa59 possibly found the place where we do all those allocations, will check tomorrow problame/integrate-tokio-epoll-uring/benchmarking/2024-01-29 Christian Schwarz 2024-01-29 20:25:02 +00:00
  • c98215674c avoid Vec::new() in walredo code path; still no dramatic improvement over before_scratch.svg Christian Schwarz 2024-01-29 20:09:22 +00:00
  • 0e3561f6d1 WIP: try to eliminate the raw_vec::finish_grow and bytes::promotable_even-drop Christian Schwarz 2024-01-29 19:52:05 +00:00
  • 28a4247c97 rip out slot pinning, has about 5% speedup Christian Schwarz 2024-01-29 19:37:11 +00:00
  • 70bc01494c Revert "broken impl of a permit pool to shave off its allocations" Christian Schwarz 2024-01-29 19:23:09 +00:00
  • a1af2c7150 broken impl of a permit pool to shave off its allocations Christian Schwarz 2024-01-29 19:22:55 +00:00
  • 043ed5edea for posterity: RSS is about 18GB with previous bench at env.pageserver_config_override='page_cache_size=2097152;max_file_descriptors=500000;virtual_file_io_engine="tokio-epoll-uring"' Christian Schwarz 2024-01-29 18:35:07 +00:00
  • 6753ff089c results: req_lru_size=2 gives tokio-epoll-uring 16k GetPage/s@110kIOPs std-fs: 9.5 GetPage/s @ 65k IOPS Christian Schwarz 2024-01-29 18:22:18 +00:00