Commit Graph
8465 Commits
Author SHA1 Message Date
Kosntantin Knizhnik 0226a67603 Restore timeout for test_unlogged_)build_test 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik d6d39f790b Reduce size of shared_buffer in test_unlogged_build to rertproduce the problem at main 2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas b36c02dda5 Update pgxn/neon/pagestore_smgr.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas ec89876612 Update test_runner/performance/test_unlogged_build.py
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas adf103a557 Update test_runner/performance/test_unlogged.py
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 388be47144 Update test_runner/performance/test_unlogged.py
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 9955d02a01 Update pgxn/neon/pagestore_smgr.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 657c63b9cb Update pgxn/neon/pagestore_smgr.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 4885621e55 Update pgxn/neon/pagestore_smgr.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 4580391963 Update pgxn/neon/pagestore_smgr.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 28ce584d01 Rename relkind to relpersistence 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik ae7b92abeb Undo check for INIT_FORKNUM 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 3c54a235dd Add test_unlogged_build.py 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik de33affb1f Fix merge conflicts 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik eabac14080 Fix merge conflicts 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 8e150568ec Handle init fork in specialk way 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 1ca23b47fd Add comment to the test 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 1c0f4d6f97 Replace spinlock with LWLock 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 67c31b61e8 Fix warning 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 9d12eea25a Fix merge problems 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik c1362cbf71 Fix empty list check 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 902ea0ccd9 Address review comments 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik fb6d7c4676 Fix merge conflict 2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 5d93a8cc71 Update pgxn/neon/relkind_cache.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas c3fdab3886 Update pgxn/neon/pagestore_client.h
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 1e4783f3f9 Update pgxn/neon/pagestore_client.h
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 20dea3aafb Move lwlock to pagestore_smgr 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik ca13e7ad7a Do not return from TRY/CATCH in determine_entry_relkind 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 87c9b067c2 Remove obsolete comment 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik e9df43abda Change return type of determine_entry_relkind to RelKind 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 840c73e3c4 Rename safe_mdexists to determine_entry_relkind and do unpin instead of unlock in it 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik a9e940e236 Add assertion to store_cached_relkind 2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 02ecb1ebbf Update pgxn/neon/pagestore_client.h
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin KnizhnikandHeikki Linnakangas 2c0a87af68 Update pgxn/neon/relkind_cache.c
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
2025-07-29 08:04:11 +03:00
Konstantin Knizhnik a9d4cbe242 Unpin entry in case of mdexists error 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik d5d41241fa Fix incorrect unpin condition in get_cached_relkind 2025-07-29 08:04:11 +03:00
Kosntantin Knizhnik 2e34fe03c7 Replace flags with enum 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 510c891ae5 Add comments 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik ac233dc9aa Fix access to uninitialized flag 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik c083765840 Address review comments 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 8884f55eee Increase number of updates in test_unlogged.py 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 1f93b664ad Add test_unlogged for measuring effect of relkind cache 2025-07-29 08:04:11 +03:00
Konstantin Knizhnik 883379f936 Add cache for relation kind 2025-07-29 08:04:11 +03:00
Ivan Efremov 6be572177c chore: Fix nightly lints (#12746)
- Remove some unused code
- Use `is_multiple_of()` instead of '%'
- Collapse consecuative "if let" statements
- Elided lifetime fixes

It is enough just to review the code of your team
2025-07-28 21:36:30 +00:00
Alex Chi Z. fe7a4e1ab6 fix(test): wait compaction in timeline offload test (#12673)
## Problem

close LKB-753. `test_pageserver_metrics_removed_after_offload` is
unstable and it sometimes leave the metrics behind after tenant
offloading. It turns out that we triggered an image compaction before
the offload and the job was stopped after the offload request was
completed.

## Summary of changes

Wait all background tasks to finish before checking the metrics.

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>
2025-07-28 16:27:55 +00:00
Heikki Linnakangas 40cae8cc36 Fix misc typos and some cosmetic code cleanup (#12695) 2025-07-28 16:21:35 +00:00
Heikki Linnakangas 02fc8b7c70 Add compatibility macros for MyProcNumber and PGIOAlignedBlock (#12715)
There were a few uses of these already, so collect them to the
compatibility header to avoid the repetition and scattered #ifdefs.

The definition of MyProcNumber is a little different from what was used
before, but the end result is the same. (PGPROC->pgprocno values were
just assigned sequentially to all PGPROC array members, see
InitProcGlobal(). That's a bit silly, which is why it was removed in
v17.)
2025-07-28 15:05:36 +00:00
John SprayandJohn Spray 60feb168e2 pageserver: decrease MAX_SHARDS in utilization (#12668)
## Problem

When tenants have a lot of timelines, the number of tenants that a
pageserver can comfortably handle goes down. Branching is much more
widely used in practice now than it was when this code was written, and
we generally run pageservers with a few thousand tenants (where each
tenant has many timelines), rather than the 10k-20k we might have done
historically.

This should really be something configurable, or a more direct proxy for
resource utilization (such as non-archived timeline count), but this
change should be a low effort improvement.

## Summary of changes

* Change the target shard count (MAX_SHARDS) to 2500 from 5000 when
calculating pageserver utilization (i.e. a 200% overcommit now
corresponds to 5000 shards, not 10000 shards)

Co-authored-by: John Spray <john.spray@databricks.com>
2025-07-28 13:50:18 +00:00
da596a5162 Update the versions for ClickHouse and Debezium (#12741)
## Problem
The test for logical replication used the year-old versions of
ClickHouse and Debezium so that we may miss problems related to
up-to-date versions.
## Summary of changes
The ClickHouse version has been updated to 24.8.
The Debezium version has been updated to the latest stable one,
3.1.3Final.
Some problems with locally running the Debezium test have been fixed.

---------

Co-authored-by: Alexey Masterov <alexey.masterov@databricks.com>
Co-authored-by: Alexander Bayandin <alexander@neon.tech>
2025-07-28 13:26:33 +00:00
Conrad Ludgate effd6bf829 [proxy] add metrics for caches (#12752)
Exposes metrics for caches. LKB-2594

This exposes a high level namespace, `cache`, that all cache metrics can
be added to - this makes it easier to make library panels for the caches
as I understand it.

To calculate the current cache fill ratio, you could use the following
query:

```
(
    cache_inserted_total{cache="node_info"}
  - sum (cache_evicted_total{cache="node_info"}) without (cause)
)
  / cache_capacity{cache="node_info"}
```

To calculate the cache hit ratio, you could use the following query:

```
  cache_request_total{cache="node_info", outcome="hit"}
/ sum (cache_request_total{cache="node_info"}) without (outcome)
```
2025-07-28 10:41:49 +00:00