mirror of
https://github.com/neondatabase/neon.git
synced 2026-05-15 20:20:38 +00:00
6059801943
We need checksums to verify data integrity, when we read it from untrusted place (e.g. local disk) or via untrusted communication channel (e.g. network). At the same time, we trust pageserver <-> redo process communication channel, as it is just a pipe. Here we enable calculation of data checksums in the wal redo process and when we extract FPI during WAL injestion. Compute node (Postgres) will verify checksum of every page after receiving it back from pageserver. So it is pretty similar to how vanilla Postgres checks them. There are two other places where we should verify checksums to detect data corruption earlier: - when we receive WAL records from safekeepers (already implemented, see: WalStreamDecoder::poll_decode) - when we write layer files to disk and read back in memory from local disk or S3
22 lines
909 B
Bash
Executable File
22 lines
909 B
Bash
Executable File
#!/bin/bash
|
|
PG_BIN=$1
|
|
WAL_PATH=$2
|
|
DATA_DIR=$3
|
|
PORT=$4
|
|
SYSID=`od -A n -j 24 -N 8 -t d8 $WAL_PATH/000000010000000000000002* | cut -c 3-`
|
|
rm -fr $DATA_DIR
|
|
env -i LD_LIBRARY_PATH=$PG_BIN/../lib $PG_BIN/initdb -E utf8 -U cloud_admin -D $DATA_DIR --data-checksums --sysid=$SYSID
|
|
echo port=$PORT >> $DATA_DIR/postgresql.conf
|
|
REDO_POS=0x`$PG_BIN/pg_controldata -D $DATA_DIR | fgrep "REDO location"| cut -c 42-`
|
|
declare -i WAL_SIZE=$REDO_POS+114
|
|
$PG_BIN/pg_ctl -D $DATA_DIR -l logfile start
|
|
$PG_BIN/pg_ctl -D $DATA_DIR -l logfile stop -m immediate
|
|
cp $DATA_DIR/pg_wal/000000010000000000000001 .
|
|
cp $WAL_PATH/* $DATA_DIR/pg_wal/
|
|
if [ -f $DATA_DIR/pg_wal/*.partial ]
|
|
then
|
|
(cd $DATA_DIR/pg_wal ; for partial in \*.partial ; do mv $partial `basename $partial .partial` ; done)
|
|
fi
|
|
dd if=000000010000000000000001 of=$DATA_DIR/pg_wal/000000010000000000000001 bs=$WAL_SIZE count=1 conv=notrunc
|
|
rm -f 000000010000000000000001
|