Dimitri Fontaine
4d3b28bd2e
[Hadron] Always run databricks auth hook. ( #12683 )
2025-07-30 21:34:30 +00:00
Konstantin Knizhnik
b00a0096bf
Reintialize page in allocNewBuffer only when buffer is returned ( #12399 )
...
## Problem
See https://github.com/neondatabase/neon/issues/12387
`allocNewBuffer` initialise page with zeros
but not always return it because of parity checks.
In case of wrong parity the page is rejected and as a result we have
dirty page with zero LSN, which cause assertion failure on neon_write
when page is evicted from shared buffers.
## Summary of changes
Perform, page initialisation in `allocNewBuffer` only when buffer is
returned (parity check is passed).
Postgres PRs:
https://github.com/neondatabase/postgres/pull/661
https://github.com/neondatabase/postgres/pull/662
https://github.com/neondatabase/postgres/pull/663
https://github.com/neondatabase/postgres/pull/664
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
Co-authored-by: Kosntantin Knizhnik <konstantin.knizhnik@databricks.com >
2025-07-22 17:50:26 +00:00
Dimitri Fontaine
1e30b31fa7
Cherry pick: pg hooks for online table. ( #12654 )
...
## Problem
## Summary of changes
2025-07-21 11:10:10 +00:00
Shockingly Good
8e95455aef
Update the postgres submodules ( #12636 )
...
Synchronises the main branch's postgres submodules with the
`neondatabase/postgres` repository state.
2025-07-18 08:21:22 +00:00
Dimitri Fontaine
267fb49908
Update Postgres branches. ( #12628 )
...
## Problem
## Summary of changes
2025-07-16 18:39:54 +00:00
Alexey Kondratov
dd7fff655a
feat(compute): Introduce privileged_role_name parameter ( #12539 )
...
## Problem
Currently `neon_superuser` is hardcoded in many places. It makes it
harder to reuse the same code in different envs.
## Summary of changes
Parametrize `neon_superuser` in `compute_ctl` via
`--privileged-role-name` and in `neon` extensions via
`neon.privileged_role_name`, so it's now possible to use different
'superuser' role names if needed. Everything still defaults to
`neon_superuser`, so no control plane code changes are needed and I
intentionally do not touch regression and migrations tests.
Postgres PRs:
- https://github.com/neondatabase/postgres/pull/674
- https://github.com/neondatabase/postgres/pull/675
- https://github.com/neondatabase/postgres/pull/676
- https://github.com/neondatabase/postgres/pull/677
Cloud PR:
- https://github.com/neondatabase/cloud/pull/31138
2025-07-15 20:22:57 +00:00
Matthias van de Meent
4566b12a22
NEON: Finish Zenith->Neon rename ( #12566 )
...
Even though we're now part of Databricks, let's at least make this part
consistent.
## Summary of changes
- PG14: https://github.com/neondatabase/postgres/pull/669
- PG15: https://github.com/neondatabase/postgres/pull/670
- PG16: https://github.com/neondatabase/postgres/pull/671
- PG17: https://github.com/neondatabase/postgres/pull/672
---------
Co-authored-by: Arpad Müller <arpad-m@users.noreply.github.com >
2025-07-11 18:56:39 +00:00
Heikki Linnakangas
af46b5286f
Avoid recompiling postgres_ffi when there has been no changes ( #12292 )
...
Every time you run `make`, it runs `make install` on all the PostgreSQL
sources, which copies the header files. That in turn triggers a rebuild
of the `postgres_ffi` crate, and everything that depends on it. We had
worked around this earlier (see #2458 ), by passing a custom INSTALL
script to the Postgres makefiles, which refrains from updating the
modification timestamp on headers when they have not been changed, but
the v14 makefile didn't obey INSTALL for the header files. Backporting
c0a1d7621b to v14 fixes that.
This backports upstream PostgreSQL commit c0a1d7621b to v14.
Corresponding PR in the 'postgres' repo:
https://github.com/neondatabase/postgres/pull/660
2025-06-21 21:07:38 +00:00
Konstantin Knizhnik
9c6c780201
Replica promote ( #12090 )
...
## Problem
This PR is part of larger computes support activity:
https://www.notion.so/neondatabase/Larger-computes-114f189e00478080ba01e8651ab7da90
Epic: https://github.com/neondatabase/cloud/issues/19010
In case of planned node restart, we are going to
1. create new read-only replica
2. capture LFC state at primary
3. use this state to prewarm replica
4. stop old primary
5. promote replica to primary
Steps 1-3 are currently implemented and support from compute side.
This PR provides compute level implementation of replica promotion.
Support replica promotion
## Summary of changes
Right now replica promotion is done in three steps:
1. Set safekeepers list (now it is empty for replica)
2. Call `pg_promote()` top promote replica
3. Update endpoint setting to that it ids not more treated as replica.
May be all this three steps should be done by some function in
compute_ctl. But right now this logic is only implement5ed in test.
Postgres submodules PRs:
https://github.com/neondatabase/postgres/pull/648
https://github.com/neondatabase/postgres/pull/649
https://github.com/neondatabase/postgres/pull/650
https://github.com/neondatabase/postgres/pull/651
---------
Co-authored-by: Matthias van de Meent <matthias@neon.tech >
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-06-05 11:27:14 +00:00
Konstantin Knizhnik
81c557d87e
Unlogged build get smgr ( #11954 )
...
## Problem
See https://github.com/neondatabase/neon/issues/11910
and https://neondb.slack.com/archives/C04DGM6SMTM/p1747314649059129
## Summary of changes
Do not change persistence in `start_unlogged_build`
Postgres PRs:
https://github.com/neondatabase/postgres/pull/642
https://github.com/neondatabase/postgres/pull/641
https://github.com/neondatabase/postgres/pull/640
https://github.com/neondatabase/postgres/pull/639
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-05-18 05:02:47 +00:00
Konstantin Knizhnik
48b870bc07
Use unlogged build in GIST for storing root page ( #11892 )
...
## Problem
See https://github.com/neondatabase/neon/issues/11891
Newly added assert is first when root page of GIST index is written to
the disk as part of sorted build.
## Summary of changes
Wrap writing of root page in unlogged build.
https://github.com/neondatabase/postgres/pull/632
https://github.com/neondatabase/postgres/pull/633
https://github.com/neondatabase/postgres/pull/634
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-05-15 04:45:22 +00:00
Anastasia Lubennikova
25ab16ee24
chore(compute): Postgres 17.5, 16.9, 15.13 and 14.18 ( #11886 )
...
Bump all minor versions.
the only conflict was
src/backend/storage/smgr/smgr.c in v17
where our smgr changes conflicted with
ee578921b6
2025-05-13 13:30:09 +00:00
Konstantin Knizhnik
03d635b916
Add more guards for prefetch_pump_state ( #11859 )
...
## Problem
See https://neondb.slack.com/archives/C08PJ07BZ44/p1746566292750689
Looks like there are more cases when `prefetch_pump_state` can be called
in unexpected place and cause core dump.
## Summary of changes
Add more guards.
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-05-09 09:07:08 +00:00
Konstantin Knizhnik
101e115b38
Change prefetch logic in vacuum ( #11650 )
...
## Problem
See https://neondb.slack.com/archives/C03QLRH7PPD/p1745003314183649
Vacuum doesn't use prefetch because this strange logic in
`lazy_scan_heap`:
```
/* And only up to the next unskippable block */
if (next_prefetch_block + prefetch_budget > vacrel->next_unskippable_block)
prefetch_budget = vacrel->next_unskippable_block - next_prefetch_block;
```
## Summary of changes
Disable prefetch only if vacuum jumps to next skippable block (there is
SKIP_PAGES_THRESHOLD) which cancel seqscan and perform jump only if gap
is large enough).
Postgres PRs:
https://github.com/neondatabase/postgres/pull/620
https://github.com/neondatabase/postgres/pull/621
https://github.com/neondatabase/postgres/pull/622
https://github.com/neondatabase/postgres/pull/623
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-05-09 06:54:40 +00:00
Mark Novikov
d22377c754
Skip event triggers in dump-restore ( #11794 )
...
## Problem
Data import fails if the src db has any event triggers, because those
can only be restored by a superuser. Specifically imports from Heroku
and Supabase are guaranteed to fail.
Closes https://github.com/neondatabase/cloud/issues/27353
## Summary of changes
Depends on `pg_dump` patches per each supported PostgreSQL version:
- https://github.com/neondatabase/postgres/pull/630
- https://github.com/neondatabase/postgres/pull/629
- https://github.com/neondatabase/postgres/pull/627
- https://github.com/neondatabase/postgres/pull/628
2025-05-08 11:04:28 +00:00
Konstantin Knizhnik
f0e7b3e0ef
Use unlogged build for gist_indexsortbuild_flush_ready_pages ( #11753 )
...
## Problem
See https://github.com/neondatabase/neon/issues/11718
GIST index can be constructed in two ways: GIST_SORTED_BUILD and
GIST_BUFFERING.
We used unlogged build in the second case but not in the first.
## Summary of changes
Use unlogged build in `gist_indexsortbuild_flush_ready_pages`
Correspondent Postgres PRsL:
https://github.com/neondatabase/postgres/pull/624
https://github.com/neondatabase/postgres/pull/625
https://github.com/neondatabase/postgres/pull/626
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
Co-authored-by: Heikki Linnakangas <heikki@neon.tech >
2025-05-06 07:24:27 +00:00
Heikki Linnakangas
b4e26a6284
Set last-written LSN as part of smgr_end_unlogged_build() ( #11584 )
...
This way, the callers don't need to do it, reducing the footprint of
changes we've had to made to various index AM's build functions.
2025-04-16 12:34:18 +00:00
Konstantin Knizhnik
8eb701d706
Save FSM/VM pages on normal shutdown ( #11449 )
...
## Problem
See https://neondb.slack.com/archives/C03QLRH7PPD/p1743746717119179
We wallow FSM/VM pages when they are written to disk to persist them in
PS.
But it is not happen during shutdown checkpoint, because writing to WAL
during checkpoint cause Postgres panic.
## Summary of changes
Move `CheckPointBuffers` call to `PreCheckPointGuts`
Postgres PRs:
https://github.com/neondatabase/postgres/pull/615
https://github.com/neondatabase/postgres/pull/614
https://github.com/neondatabase/postgres/pull/613
https://github.com/neondatabase/postgres/pull/612
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-04-07 13:56:55 +00:00
Alexander Bayandin
8e1b5a9727
Fix Postgres build on macOS ( #11442 )
...
## Problem
Postgres build fails with the following error on macOS:
```
/Users/bayandin/work/neon//vendor/postgres-v14/src/port/snprintf.c:424:27: error: 'strchrnul' is only available on macOS 15.4 or newer [-Werror,-Wunguarded-availability-new]
424 | const char *next_pct = strchrnul(format + 1, '%');
| ^~~~~~~~~
/Users/bayandin/work/neon//vendor/postgres-v14/src/port/snprintf.c:376:14: note: 'strchrnul' has been marked as being introduced in macOS 15.4 here, but the deployment target is macOS 15.0.0
376 | extern char *strchrnul(const char *s, int c);
| ^
/Users/bayandin/work/neon//vendor/postgres-v14/src/port/snprintf.c:424:27: note: enclose 'strchrnul' in a __builtin_available check to silence this warning
424 | const char *next_pct = strchrnul(format + 1, '%');
| ^~~~~~~~~
425 |
426 | /* Dump literal data we just scanned over */
427 | dostr(format, next_pct - format, target);
428 | if (target->failed)
429 | break;
430 |
431 | if (*next_pct == '\0')
432 | break;
433 | format = next_pct;
|
1 error generated.
```
## Summary of changes
- Update Postgres fork to include changes from
6da2ba1d8ahttps://github.com/neondatabase/postgres/pull/608
- https://github.com/neondatabase/postgres/pull/609
- https://github.com/neondatabase/postgres/pull/610
- https://github.com/neondatabase/postgres/pull/611
2025-04-04 14:09:15 +00:00
Tristan Partin
497116b76d
Download extension if it does not exist on the filesystem ( #11315 )
...
Previously we attempted to download all extensions in CREATE EXTENSION
statements. Extensions like pg_stat_statements and neon are not remote
extensions, but still we were requesting them when
skip_pg_catalog_updates was set to false.
Fixes: https://github.com/neondatabase/neon/issues/11127
Signed-off-by: Tristan Partin <tristan@neon.tech >
2025-04-04 01:06:22 +00:00
Suhas Thalanki
5589efb6de
moving LastWrittenLSNCache to Neon Extension ( #11031 )
...
## Problem
We currently have this code duplicated across different PG versions.
Moving this to an extension would reduce duplication and simplify
maintenance.
## Summary of changes
Moving the LastWrittenLSN code from PG versions to the Neon extension
and linking it with hooks.
Related Postgres PR: https://github.com/neondatabase/postgres/pull/590
Closes: https://github.com/neondatabase/neon/issues/10973
---------
Co-authored-by: Tristan Partin <tristan@neon.tech >
2025-03-19 17:29:40 +00:00
Tristan Partin
1b8c4286c4
Fetch remote extension in ALTER EXTENSION UPDATE statements ( #11102 )
...
Previously, remote extensions were not fetched unless they were used in
some other manner. For instance, loading a BM25 index in pg_search
fetches the pg_search extension. However, if on a fresh compute with
pg_search 0.15.5 installed, the user ran `ALTER EXTENSION pg_search
UPDATE TO '0.15.6'` without first using the pg_search extension, we
would not fetch the extension and fail to find an update path.
Signed-off-by: Tristan Partin <tristan@neon.tech >
2025-03-09 17:29:44 +00:00
Anastasia Lubennikova
38a883118a
Skip dropping tablesync replication slots on the publisher from branch ( #11073 )
...
fixes https://github.com/neondatabase/cloud/issues/24292
Do not drop tablesync replication slots on the publisher,
when we're in the process of dropping subscriptions inherited by a neon
branch.
Because these slots are still needed by the parent branch subscriptions.
For regular slots we handle this by setting the slot_name to NONE
before calling DROP SUBSCRIPTION, but tablesync slots are not exposed to
SQL.
rely on GUC disable_logical_replication_subscribers=true
to know that we're in the Neon-specific process of dropping
subscriptions.
2025-03-05 11:29:46 +00:00
Konstantin Knizhnik
8f82c661d4
Move neon_pgstat_file_size_limit to the extension ( #10959 )
...
## Problem
PG14 uses separate backend for stats collector having no access to
shaerd memory.
As far as AUX mechanism requires access to shared memory, persisting
pgstat.stat file
is not supported at pg14. And so there is no definition of
`neon_pgstat_file_size_limit`
variable. It makes it impossible to provide same config for all Postgres
version.
## Summary of changes
Move neon_pgstat_file_size_limit to Neon extension.
Postgres submodules PR:
https://github.com/neondatabase/postgres/pull/587
https://github.com/neondatabase/postgres/pull/588
https://github.com/neondatabase/postgres/pull/589
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
Co-authored-by: Tristan Partin <tristan@neon.tech >
2025-02-25 12:23:04 +00:00
Konstantin Knizhnik
bd335fa751
Fix prototype of CheckPointReplicationState ( #10907 )
...
## Problem
Occasionally removed (void) from definition of
`CheckPointReplicationState` function
## Summary of changes
Restore function prototype.
https://github.com/neondatabase/postgres/pull/585
https://github.com/neondatabase/postgres/pull/586
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-02-20 18:29:14 +00:00
Konstantin Knizhnik
a6d8640d6f
Persist pg_stat information in pageserver ( #6560 )
...
## Problem
Statistic is saved in local file and so lost on compute restart.
Persist in in page server using the same AUX file mechanism used for
replication slots
See more about motivation in
https://neondb.slack.com/archives/C04DGM6SMTM/p1703077676522789
## Summary of changes
Persist postal file using AUX mechanism
Postgres PRs:
https://github.com/neondatabase/postgres/pull/547
https://github.com/neondatabase/postgres/pull/446
https://github.com/neondatabase/postgres/pull/445
Related to #6684 and #6228
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-02-20 06:38:55 +00:00
Heikki Linnakangas
f36ec5c84b
chore(compute): Postgres 17.4, 16.8, 15.12 and 14.17 ( #10868 )
...
Update all minor versions. No conflicts.
Postgres repository PRs:
- https://github.com/neondatabase/postgres/pull/584
- https://github.com/neondatabase/postgres/pull/583
- https://github.com/neondatabase/postgres/pull/582
- https://github.com/neondatabase/postgres/pull/581
2025-02-18 15:56:43 +00:00
Alexey Kondratov
8c2f85b209
chore(compute): Postgres 17.3, 16.7, 15.11 and 14.16 ( #10771 )
...
## Summary of changes
Bump all minor versions. The only non-trivial conflict was between
-
0350b876b0bd09a752f4c1c9df3159https://github.com/neondatabase/postgres/pull/580
- https://github.com/neondatabase/postgres/pull/579
- https://github.com/neondatabase/postgres/pull/577
- https://github.com/neondatabase/postgres/pull/578
2025-02-13 13:28:05 +00:00
Konstantin Knizhnik
0cf0119751
Add --save_records option to pg_waldump ( #10626 )
...
## Problem
Make it possible to dump WAL records in format recognised by walredo
process.
Intended usage:
```
pg_waldump -R 1663/5/16396 -B 771727 000000010000000100000034 --save-records=/tmp/walredo.records
postgres --wal-redo < /tmp/walredo.records > /tmp/page.img
```
## Summary of changes
Related Postgres PRs:
https://github.com/neondatabase/postgres/pull/575
https://github.com/neondatabase/postgres/pull/572
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-02-10 15:48:03 +00:00
Konstantin Knizhnik
7fc6953da4
Is neon superuser ( #10625 )
...
## Problem
is_neon_superuser() fiunction is public in pg14/pg15
but statically defined in publicationcmd.c in pg16/pg17
## Summary of changes
Make this function public for all Postgres version.
It is intended to be used not only in publicationcmd.c
See
https://github.com/neondatabase/postgres/pull/573
https://github.com/neondatabase/postgres/pull/576
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-02-06 05:42:14 +00:00
Alexander Lakhin
a7a706cff7
Fix submodule reference after #10473 ( #10577 )
2025-01-30 09:09:43 +00:00
alexanderlaw
4d2328ebe3
Fix C code to satisfy sanitizers ( #10473 )
2025-01-29 10:05:43 +00:00
Konstantin Knizhnik
9f1408fdf3
Do not assign max(lsn) to maxLastWrittenLsn in SetLastWrittenLSNForblokv ( #10474 )
...
## Problem
See https://github.com/neondatabase/neon/issues/10281
`SetLastWrittenLSNForBlockv` is assigning max(lsn) to
`maxLastWrittenLsn` while its should contain only max LSN not present in
LwLSN cache. It case unnecessary waits in PS.
## Summary of changes
Restore status-quo for pg17.
Related Postgres PR: https://github.com/neondatabase/postgres/pull/563
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-01-24 14:57:32 +00:00
Konstantin Knizhnik
d8ab6ddb0f
Check if relation has storage in calculate_relation_size ( #10477 )
...
## Problem
Parent of partitioned table has no storage, it relfilelocator is zero.
It cab be incorrectly hashed and produce wrong results.
See https://github.com/neondatabase/postgres/pull/518
## Summary of changes
This problem is already addressed in pg17.
Add the same check for all other PG versions.
Postgres PRs:
https://github.com/neondatabase/postgres/pull/566
https://github.com/neondatabase/postgres/pull/565
https://github.com/neondatabase/postgres/pull/564
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-01-24 12:43:52 +00:00
Anastasia Lubennikova
8e8df1b453
Disable logical replication subscribers ( #10249 )
...
Drop logical replication subscribers
before compute starts on a non-main branch.
Add new compute_ctl spec flag: drop_subscriptions_before_start
If it is set, drop all the subscriptions from the compute node
before it starts.
To avoid race on compute start, use new GUC
neon.disable_logical_replication_subscribers
to temporarily disable logical replication workers until we drop the
subscriptions.
Ensure that we drop subscriptions exactly once when endpoint starts on a
new branch.
It is essential, because otherwise, we may drop not only inherited, but
newly created subscriptions.
We cannot rely only on spec.drop_subscriptions_before_start flag,
because if for some reason compute restarts inside VM,
it will start again with the same spec and flag value.
To handle this, we save the fact of the operation in the database
in the neon.drop_subscriptions_done table.
If the table does not exist, we assume that the operation was never
performed, so we must do it.
If table exists, we check if the operation was performed on the current
timeline.
fixes: https://github.com/neondatabase/neon/issues/8790
2025-01-23 11:02:15 +00:00
Tristan Partin
3e529f124f
Remove leading slashes when downloading remote files ( #10396 )
...
Signed-off-by: Tristan Partin <tristan@neon.tech >
2025-01-15 15:29:52 +00:00
Konstantin Knizhnik
a039f8381f
Optimize vector get last written LSN ( #10360 )
...
## Problem
See https://github.com/neondatabase/neon/issues/10281
pg17 performs extra lock/unlock operation when fetching LwLSN.
## Summary of changes
Perform all lookups under one lock, moving initialization of not found
keys to separate loop.
Related Postgres PR:
https://github.com/neondatabase/postgres/pull/553
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-01-14 05:54:30 +00:00
Konstantin Knizhnik
ceacc29609
Start with minimal prefetch distance to minimize prefetch overhead for exact or limited index scans ( #10359 )
...
## Problem
See https://neondb.slack.com/archives/C04DGM6SMTM/p1736526089437179
In case of queries index scan with LIMIT clause, multiple backends can
concurrently send larger number of duplicated prefetch requests which
are not stored in LFC and so actually do useless job.
Current implementation of index prefetch starts with maximal prefetch
distance (10 by default now) when there are no key bounds, so in queries
with LIMIT clause like `select * from T order by pk limit 1` compute can
send a lot of useless prefetch requests to page server.
## Summary of changes
Always start with minimal prefetch distance even if there are not key
boundaries.
Related Postgres PRs:
https://github.com/neondatabase/postgres/pull/552
https://github.com/neondatabase/postgres/pull/551
https://github.com/neondatabase/postgres/pull/550
https://github.com/neondatabase/postgres/pull/549
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2025-01-13 14:26:11 +00:00
Konstantin Knizhnik
aaf980f70d
Online checkpoint replication state ( #9976 )
...
## Problem
See https://neondb.slack.com/archives/C04DGM6SMTM/p1733180965970089
Replication state is checkpointed only by shutdown checkpoint.
It means that replication snapshots are not removed till compute
shutdown.
## Summary of changes
Checkpoint replication state during online checkpoint
Related Postgres PR:
https://github.com/neondatabase/postgres/pull/546
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2024-12-18 09:34:38 +00:00
Konstantin Knizhnik
117c1b5dde
Do not perform prefetch for temp relations ( #10146 )
...
## Problem
See https://neondb.slack.com/archives/C04DGM6SMTM/p1734002916827019
With recent prefetch fixes for pg17 and `effective_io_concurrency=100`
pg_regress test stats.sql is failed when set temp_buffers to 100.
Stream API will try to lock all this 100 buffers for prefetch.
## Summary of changes
Disable such behaviour for temp relations.
Postgres PR: https://github.com/neondatabase/postgres/pull/548
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2024-12-16 06:03:53 +00:00
Konstantin Knizhnik
2521eba674
Check for invalid down link while prefetching B-Tree leave pages for index-only scan ( #9867 )
...
## Problem
See #9866
Index-only scan prefetch implementation doesn't take in account that
down link may be invalid
## Summary of changes
Check that downlink is valid block number
Correspondent Postgres PRs:
https://github.com/neondatabase/postgres/pull/534
https://github.com/neondatabase/postgres/pull/535
https://github.com/neondatabase/postgres/pull/536
https://github.com/neondatabase/postgres/pull/537
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2024-12-13 20:46:41 +00:00
Matthias van de Meent
597125e124
Disable readstream's reliance on seqscan readahead ( #9860 )
...
Neon doesn't have seqscan detection of its own, so stop read_stream from
trying to utilize that readahead, and instead make it issue readahead of
its own.
## Problem
@knizhnik noticed that we didn't issue smgrprefetch[v] calls for
seqscans in PG17 due to the move to the read_stream API, which assumes
that the underlying IO facilities do seqscan detection for readahead.
That is a wrong assumption when Neon is involved, so let's remove the
code that applies that assumption.
## Summary of changes
Remove the cases where seqscans are detected and prefetch is disabled as
a consequence, and instead don't do that detection.
PG PR: https://github.com/neondatabase/postgres/pull/532
2024-12-11 00:51:05 +00:00
Matthias van de Meent
e71d20d392
Emit nbtree vacuum cycle id in nbtree xlog through forced FPIs ( #9932 )
...
This fixes neondatabase/neon#9929 .
## Postgres repo PRS:
- PG17: https://github.com/neondatabase/postgres/pull/538
- PG16: https://github.com/neondatabase/postgres/pull/539
- PG15: https://github.com/neondatabase/postgres/pull/540
- PG14: https://github.com/neondatabase/postgres/pull/541
## Problem
see #9929
## Summary of changes
We update the split code to force the code to emit an FPI whenever the
cycle ID might be interesting for concurrent btree vacuum.
2024-12-10 19:42:52 +00:00
Konstantin Knizhnik
97a9abd181
Add GUC controlling whether to pause recovery if some critical GUCs at replica have smaller value than on primary ( #9057 )
...
## Problem
See https://github.com/neondatabase/neon/issues/9023
## Summary of changes
Ass GUC `recovery_pause_on_misconfig` allowing not to pause in case of
replica and primary configuration mismatch
See https://github.com/neondatabase/postgres/pull/501
See https://github.com/neondatabase/postgres/pull/502
See https://github.com/neondatabase/postgres/pull/503
See https://github.com/neondatabase/postgres/pull/504
## Checklist before requesting a review
- [ ] I have performed a self-review of my code.
- [ ] If it is a core feature, I have added thorough tests.
- [ ] Do we need to implement analytics? if so did you add the relevant
metrics to the dashboard?
- [ ] If this PR requires public announcement, mark it with
/release-notes label and add several sentences in this section.
## Checklist before merging
- [ ] Do not forget to reformat commit message to not include the above
checklist
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
Co-authored-by: Heikki Linnakangas <heikki@neon.tech >
2024-12-01 12:23:10 +00:00
Matthias van de Meent
973a8d2680
Fix timeout value used in XLogWaitForReplayOf ( #9937 )
...
The previous value assumed usec precision, while the timeout used is in
milliseconds, causing replica backends to wait for (potentially) many
hours for WAL replay without the expected progress reports in logs.
This fixes the issue.
Reported-By: Alexander Lakhin <exclusion@gmail.com >
## Problem
https://github.com/neondatabase/postgres/pull/279#issuecomment-2507671817
The timeout value was configured with the assumption the indicated value
would be microseconds, where it's actually milliseconds. That causes the
backend to wait for much longer (2h46m40s) before it emits the "I'm
waiting for recovery" message. While we do have wait events configured
on this, it's not great to have stuck backends without clear logs, so
this fixes the timeout value in all our PostgreSQL branches.
## PG PRs
* PG14: https://github.com/neondatabase/postgres/pull/542
* PG15: https://github.com/neondatabase/postgres/pull/543
* PG16: https://github.com/neondatabase/postgres/pull/544
* PG17: https://github.com/neondatabase/postgres/pull/545
2024-11-29 19:10:26 +00:00
Konstantin Knizhnik
0713ff3176
Bump Postgres version ( #9808 )
...
## Problem
I have made a mistake in merging Postgre PRs
## Summary of changes
Restore consistency of submodule referenced.
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2024-11-21 14:56:56 +00:00
Konstantin Knizhnik
770ac34ae6
Register custom xlog reader callbacks for on-demand WAL download in StartupDecodingContext ( #9007 )
...
## Problem
See https://github.com/neondatabase/neon/issues/8931
On-demand WAL download are not set in all cases where WAL is accessed by
logical replication
## Summary of changes
Set customer xlog reader handles in StartupDecodingContext
Related changes in Postgres modules:
https://github.com/neondatabase/postgres/pull/495
https://github.com/neondatabase/postgres/pull/496
https://github.com/neondatabase/postgres/pull/497
https://github.com/neondatabase/postgres/pull/498
## Checklist before requesting a review
- [ ] I have performed a self-review of my code.
- [ ] If it is a core feature, I have added thorough tests.
- [ ] Do we need to implement analytics? if so did you add the relevant
metrics to the dashboard?
- [ ] If this PR requires public announcement, mark it with
/release-notes label and add several sentences in this section.
## Checklist before merging
- [ ] Do not forget to reformat commit message to not include the above
checklist
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
2024-11-19 22:29:57 +02:00
Heikki Linnakangas
ada84400b7
PostgreSQL minor version updates (17.2, 16.6, 15.10, 14.15) ( #9795 )
...
The community decided to make a new off-schedule release due to ABI
breakage in last week's release. We're not affected by the ABI
breakage because we rebuild all extensions in our docker images, but
let's stay up-to-date. There were a few other fixes in the release
too.
2024-11-19 17:01:05 +02:00
Heikki Linnakangas
10aaa3677d
PostgreSQL minor version updates (17.1, 16.5, 15.9, 14.14) ( #9727 )
...
This includes a patch to temporarily disable one test in the pg_anon
test suite. It is an upstream issue, the test started failing with the
new PostgreSQL minor versions because of a change in the default
timezone used in tests. We don't want to block the release for this,
so just disable the test for now. See
199f0a392b (note_2148017485)https://github.com/neondatabase/postgres/pull/524
https://github.com/neondatabase/postgres/pull/525
https://github.com/neondatabase/postgres/pull/526
https://github.com/neondatabase/postgres/pull/527
2024-11-13 15:08:58 +02:00
Konstantin Knizhnik
1ff5333a1b
Do not wallog AUX files at replica ( #9457 )
...
## Problem
Attempt to persist LR stuff at replica cause cannot make new WAL entries
during recovery` error.
See https://neondb.slack.com/archives/C07S7RBFVRA/p1729280401283389
## Summary of changes
Do not wallog AUX files at replica.
Related Postgres PRs:
https://github.com/neondatabase/postgres/pull/517
https://github.com/neondatabase/postgres/pull/516
https://github.com/neondatabase/postgres/pull/515
https://github.com/neondatabase/postgres/pull/514
## Checklist before requesting a review
- [ ] I have performed a self-review of my code.
- [ ] If it is a core feature, I have added thorough tests.
- [ ] Do we need to implement analytics? if so did you add the relevant
metrics to the dashboard?
- [ ] If this PR requires public announcement, mark it with
/release-notes label and add several sentences in this section.
## Checklist before merging
- [ ] Do not forget to reformat commit message to not include the above
checklist
---------
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech >
Co-authored-by: Heikki Linnakangas <heikki@neon.tech >
2024-11-13 08:50:01 +02:00
