Commit Graph

  • 49a5e411d6 implement request-scoped LRU cache Christian Schwarz 2024-01-29 18:22:00 +00:00
  • 2ff1a5cecd Patch safekeeper control file on HTTP request (#6455) Arthur Petukhovsky 2024-01-29 18:20:57 +00:00
  • ec8dcc2231 flatten proxy flow (#6447) Conrad Ludgate 2024-01-29 17:38:03 +00:00
  • b844c6f0c7 Do pagination in list_object_versions call (#6500) Arpad Müller 2024-01-29 17:59:26 +01:00
  • 21a11822e8 results: tokio-epoll-uring 3.3kGetPage/s@240k IOPS, std-fs: 1.2kGetPage/s@80k IOPS Christian Schwarz 2024-01-29 16:21:22 +00:00
  • aca2d7bdea use O_DIRECT for VirtualFile reads Christian Schwarz 2024-01-29 16:20:24 +00:00
  • 6a85a06e1b Compute: build rdkit without freetype support (#6495) Alexander Bayandin 2024-01-29 16:16:37 +00:00
  • db44395ee2 rip out materialized page cache Christian Schwarz 2024-01-29 14:45:02 +00:00
  • 03874009ec add back page cache but not for DeltaLayerValue and ImageLayerValue Christian Schwarz 2024-01-29 14:35:55 +00:00
  • 0033b4c985 results: both tokio-epoll-uring and std-fs achieve about 4k GetPage/sec @ 60k IOPS Christian Schwarz 2024-01-29 13:53:13 +00:00
  • a608667301 rip out page cache Christian Schwarz 2023-08-29 15:17:40 +00:00
  • b04a6acd6c docker: add attachment_service binary (#6506) John Spray 2024-01-29 13:31:56 +00:00
  • b9b7670a3a hack: use a single runtime in pageserver Christian Schwarz 2024-01-29 12:23:25 +00:00
  • 62a3d87098 results under higher memory pressure show that tokio-epoll-uring pays off Christian Schwarz 2024-01-29 10:50:52 +00:00
  • 8d6ce71b29 hacky: ability to set io_engine via mgmt_api => pagebench Christian Schwarz 2024-01-29 10:50:20 +00:00
  • 1ec3e39d4e Merge pull request #6504 from neondatabase/releases/2024-01-29 release-4713 John Spray 2024-01-29 10:05:01 +00:00
  • 0c7b89235c pageserver: add range layer map search implementation (#6469) Vlad Lazar 2024-01-29 09:47:12 +00:00
  • 1e9a50bca8 disk_usage_eviction_task: cleanup summaries (#6490) Joonas Koivunen 2024-01-29 10:38:40 +02:00
  • 511e730cc0 hll experiment (#6312) Conrad Ludgate 2024-01-29 07:26:20 +00:00
  • c1148dc9ac Fix calculation of maximal multixact in ingest_multixact_create_record (#6502) Konstantin Knizhnik 2024-01-29 07:39:16 +02:00
  • 8253cf1931 proxy: Relax endpoint check (#6503) Anna Khanova 2024-01-28 22:27:14 +01:00
  • 3a82430432 fixup(#6492): also switch the benchmarks that runs on merge-to-main back to std-fs (#6501) Christian Schwarz 2024-01-28 00:15:11 +01:00
  • d23ea718ee 2min 3 tenants, 2000 req/s each; that is 0 IOPS workload (all in PS/Kernel page cache) problame/integrate-tokio-epoll-uring/benchmarking/2024-01-26 Christian Schwarz 2024-01-27 12:59:15 +00:00
  • 73a7ca38b3 same config, but, rate limit of 2/sec per tenant => bursty due to ticker behavior Christian Schwarz 2024-01-27 12:51:11 +00:00
  • 394e675db3 Print checkpoint key contents "pagectl print-layer-file" debug-checkpoints-2 Heikki Linnakangas 2024-01-27 14:49:59 +02:00
  • 7eb1d4cfa6 manual 2min test run including warmup Christian Schwarz 2024-01-27 12:48:32 +00:00
  • 6ebd683327 TODO/workaround: walredo quiescing broken with compaction_period=0 Christian Schwarz 2024-01-27 12:47:57 +00:00
  • b1ecdfe099 WIP: async walredo Christian Schwarz 2024-01-27 12:47:29 +00:00
  • 82a74d0e77 pagebench: fix percentiles reporting Christian Schwarz 2024-01-27 12:46:36 +00:00
  • 8fbecb1565 bump lsn a bit one-off-nextxid-hack Stas Kelvich 2024-01-27 13:48:06 +02:00
  • 715772bdd6 new values Stas Kelvich 2024-01-27 13:34:28 +02:00
  • 1626a1f333 Fix the test Heikki Linnakangas 2024-01-20 19:59:56 +02:00
  • 12e39001ce Apply the hack for all timelines of the target tenant Heikki Linnakangas 2024-01-20 19:26:53 +02:00
  • 65cd16de86 Fix the test Heikki Linnakangas 2024-01-20 19:04:27 +02:00
  • b308be20df Fix typos and formatting in test, per 'ruff' Heikki Linnakangas 2024-01-20 18:24:20 +02:00
  • a2d08cfc97 Fix formatting Heikki Linnakangas 2024-01-20 18:22:42 +02:00
  • 4ee11d9dfc Retroactively fix the nextXid on a known broken timeline Heikki Linnakangas 2024-01-20 17:58:35 +02:00
  • 734755eaca Enable nextest retries for the arm build (#6496) Arpad Müller 2024-01-27 05:16:11 +01:00
  • e34166a28f CI: switch back to std-fs io engine for soak time before next release (#6492) Christian Schwarz 2024-01-26 22:48:34 +01:00
  • 3a36a0a227 fix(test suite): some tests leak child processes (#6497) Christian Schwarz 2024-01-26 19:23:53 +01:00
  • 58f6cb649e control_plane: database persistence for attachment_service (#6468) John Spray 2024-01-26 17:20:44 +00:00
  • 49b43c75e2 run test_pageserver_max_throughput_getpage_at_latest_lsn with 1k tenants, compare std-fs with tokio-epoll-uring Christian Schwarz 2024-01-26 16:47:16 +00:00
  • dcc7610ad6 Do backoff::retry in s3 timetravel test (#6493) Arpad Müller 2024-01-26 17:43:56 +01:00
  • 4c245b0f5a update_build_tools_image.yml: Push build-tools image to Docker Hub (#6481) Alexander Bayandin 2024-01-26 16:12:49 +00:00
  • 55b7cde665 tests: add basic coverage for sharding (#6380) John Spray 2024-01-26 14:40:47 +00:00
  • 5b34d5f561 pageserver: add vectored get latency histogram (#6461) Vlad Lazar 2024-01-26 13:40:03 +00:00
  • 2978c839c6 avoid --privileged and blanket passwdless sudo problame/neon-env-builder-cgroup Christian Schwarz 2024-01-26 13:02:02 +00:00
  • 26c55b0255 Compute: fix rdkit extension build (#6488) Alexander Bayandin 2024-01-26 12:39:20 +00:00
  • 6aefbb1aa5 [DO NOT MERGE] more debug logging to prove hypothesis that it's the fsyncs problame/benchmarking/pr/repro-slow-claim-pidfile Christian Schwarz 2024-01-26 10:54:35 +00:00
  • 1fdde9e41e fixup cedc0376ff Christian Schwarz 2024-01-26 10:26:03 +00:00
  • 3c71003b7d Revert "[DO NOT MERGE] build only debug build for v14" Christian Schwarz 2024-01-26 10:18:45 +00:00
  • a36be87680 Merge remote-tracking branch 'origin/main' into problame/neon-env-builder-cgroup Christian Schwarz 2024-01-26 10:17:25 +00:00
  • 39490ddd6c fix other detected process leakage in the lowest-effort-way possible Christian Schwarz 2024-01-26 10:10:08 +00:00
  • 2b581eefa7 add todo to protect against race condition with leaked threads Christian Schwarz 2024-01-26 10:09:35 +00:00
  • cedc0376ff fix(neon_local): leaks compute_ctl child process if get_status() fails Christian Schwarz 2024-01-26 10:05:52 +00:00
  • 12e9b2a909 Update plv8 (#6465) Vadim Kharitonov 2024-01-26 10:56:11 +01:00
  • 918b03b3b0 integrate tokio-epoll-uring as alternative VirtualFile IO engine (#5824) Christian Schwarz 2024-01-26 09:25:07 +01:00
  • b2ec54b8ac Revert "run tests sequentially" Christian Schwarz 2024-01-25 20:31:20 +00:00
  • 193ba2384a Revert "[DO NOT MERGE] fail fast" Christian Schwarz 2024-01-25 20:31:04 +00:00
  • a7f8032d28 force CI run after adding run-benchmarks label Christian Schwarz 2024-01-25 20:13:13 +00:00
  • dac2f40b26 re-enable the n_tenants=100 Christian Schwarz 2024-01-25 20:11:17 +00:00
  • eb2088264f suspicion: the fsync after creating a lot of data is the culprit; add info! line to get a timestamp before we start claiming pidfile Christian Schwarz 2024-01-25 20:09:40 +00:00
  • 8749fe5131 Merge branch 'main' into problame/neon-local-pageserver-slow-starts Christian Schwarz 2024-01-25 21:00:26 +01:00
  • 51f1671c6b WIP: convert compute_ctl to use background_process problame/wip-switch-compute-ctl-to-background_process Christian Schwarz 2024-01-25 19:55:01 +00:00
  • bcb8bed875 Revert "guarantee leaked processes so we know this actually works" Christian Schwarz 2024-01-25 19:53:09 +00:00
  • d36623ad74 CI: cancel old e2e-tests on new commits (#6463) Alexander Bayandin 2024-01-25 19:25:29 +00:00
  • 23e36ae6a3 see if this fixes the permission denied issue Christian Schwarz 2024-01-25 19:02:35 +00:00
  • 194981c16b [DO NOT MERGE] fail fast Christian Schwarz 2024-01-25 19:01:04 +00:00
  • 415b489f18 neon_simple_env was not using test_cgroup_dir Christian Schwarz 2024-01-25 18:59:41 +00:00
  • cf916b9be8 unnest the match, as per review request Christian Schwarz 2024-01-25 18:28:05 +00:00
  • 689ad72e92 fix(neon_local): leaks child process if it fails to start & pass checks (#6474) Christian Schwarz 2024-01-25 19:20:02 +01:00
  • fd4cce9417 test_pageserver_max_throughput_getpage_at_latest_lsn: remove n_tenants=100 combination (#6477) Christian Schwarz 2024-01-25 19:17:53 +01:00
  • d52b81340f S3 based recovery (#6155) Arpad Müller 2024-01-25 18:23:18 +01:00
  • 8dee9908f8 fix(compaction_task): wrong log levels (#6442) Joonas Koivunen 2024-01-25 18:45:17 +02:00
  • f79e4a3a0b if it's not a connectivity issue, bail Christian Schwarz 2024-01-25 16:40:11 +00:00
  • 407c78cfaf run tests sequentially Christian Schwarz 2024-01-25 16:27:10 +00:00
  • 800d3d1cee [DO NOT MERGE] build only debug build for v14 Alexander Bayandin 2024-01-10 12:03:06 +00:00
  • 9fae0d9562 using /v1/tenant endpoint is actually not necessary, we don't care about status 503 Christian Schwarz 2024-01-25 15:48:54 +00:00
  • 50c4b83066 impmlement request timeouts using cancel-by-drop Christian Schwarz 2024-01-25 15:36:44 +00:00
  • 727412b094 Revert "send request with a timeout, extending mgmt_api crate to do that" Christian Schwarz 2024-01-25 15:31:33 +00:00
  • 73e86160f3 send request with a timeout, extending mgmt_api crate to do that Christian Schwarz 2024-01-25 15:31:10 +00:00
  • 1648639874 fix(neon_local): long init_tenant_mgr causes pageserver startup failure Christian Schwarz 2024-01-25 15:00:19 +00:00
  • 1dcb05c3d9 fix(neon_local): leaks child process if it fails to start & pass checks Christian Schwarz 2024-01-25 14:19:59 +00:00
  • 19ed230708 Add support for PS sharding in compute (#6205) Konstantin Knizhnik 2024-01-25 15:53:31 +02:00
  • 463b6a26b5 test: show relative order eviction with "fast growing tenant" (#6377) Joonas Koivunen 2024-01-25 15:38:28 +02:00
  • 4204a7dd59 guarantee leaked processes so we know this actually works Christian Schwarz 2024-01-25 13:02:39 +00:00
  • b602063f7f attempt to make it work in CI Christian Schwarz 2024-01-25 13:02:30 +00:00
  • bcfd333c98 feat(test suite): use cgroups to detect if a test leaks processes Christian Schwarz 2024-01-25 12:38:50 +00:00
  • c9b1657e4c pageserver: fixes for creation operations overlapping with shutdown/startup (#6436) John Spray 2024-01-25 12:35:52 +00:00
  • ae77e5355c pagseserver: add range layer map lookup Vlad Lazar 2024-01-25 10:55:46 +00:00
  • 493760bb7a pageserver: add ranged layer coverage for all intersected layers Vlad Lazar 2024-01-25 10:47:49 +00:00
  • b92be77e19 Make RemoteStorage not use async_trait (#6464) Arpad Müller 2024-01-24 21:27:54 +01:00
  • 8cb8c8d7b5 Allow remove_wal.rs to run on inactive timelines (#6462) Arthur Petukhovsky 2024-01-24 19:48:56 +03:00
  • 210700d0d9 proxy: add newtype wrappers for string based IDs (#6445) Conrad Ludgate 2024-01-24 16:38:10 +00:00
  • 2595dba1e3 CI: remove io engine parametrization, always run with tokio-epoll-uring problame/integrate-tokio-epoll-uring/backup-2024-01-26 Christian Schwarz 2024-01-24 09:47:58 +00:00
  • 2cf5a4c5ce CI: run tests with both Pageserver VirtualFile IO engines Christian Schwarz 2024-01-23 18:28:55 +00:00
  • 2b4ff3424e feature(test_runner): support choosing Pageserver VirtualFile IO engine Alexander Bayandin 2024-01-12 10:44:38 +00:00
  • 12609c11f6 feature(VirtualFile): implement tokio-epoll-uring IO engine Christian Schwarz 2024-01-12 10:42:44 +00:00
  • d05ef40f68 refactor(VirtualFile): configurable IO engine Christian Schwarz 2024-01-12 10:32:33 +00:00
  • ef8e7c8703 refactor(VirtualFile): use owned buffers for reads Christian Schwarz 2024-01-12 10:16:26 +00:00