Heikki Linnakangas 9809613c6f Don't try to read from two WAL files in one read() call.
That obviously won't work, you have to stop at the WAL file boundary,
and open the next file.
2021-04-19 10:34:51 +03:00
2021-04-18 19:15:06 -07:00
2021-04-14 20:12:50 +03:00
2021-04-16 15:36:36 -07:00
2021-04-02 10:38:51 +03:00
2021-04-13 18:58:22 +03:00

Zenith

Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

Running local installation

  1. Build zenith and patched postgres
git clone --recursive https://github.com/libzenith/zenith.git
cd zenith
./pgbuild.sh # builds postgres and installs it to ./tmp_install
cargo build
  1. Start pageserver and postggres on top of it (should be called from repo root):
# Create ~/.zenith with proper paths to binaries and data
# Later that would be responsibility of a package install script
>./target/debug/zenith init

# start pageserver
> ./target/debug/zenith start
Starting pageserver at '127.0.0.1:64000'

# create and configure postgres data dir
> ./target/debug/zenith pg create
Creating new postgres: path=/Users/user/code/zenith/tmp_check_cli/compute/pg1 port=55432
Database initialized

# start it
> ./target/debug/zenith pg start pg1

# look up status and connection info
> ./target/debug/zenith pg list     
NODE		ADDRESS				STATUS
pg1			127.0.0.1:55432		running
  1. Now it is possible to connect to postgres and run some queries:
> psql -p55432 -h 127.0.0.1 postgres
postgres=# CREATE TABLE t(key int primary key, value text);
CREATE TABLE
postgres=# insert into t values(1,1);
INSERT 0 1
postgres=# select * from t;
 key | value 
-----+-------
   1 | 1
(1 row)

Running tests

git clone --recursive https://github.com/libzenith/zenith.git
./pgbuild.sh # builds postgres and installs it to ./tmp_install
cargo test -- --test-threads=1

Source tree layout

/walkeeper:

WAL safekeeper. Written in Rust.

/pageserver:

Page Server. Written in Rust.

Depends on the modified 'postgres' binary for WAL redo.

/integration_tests:

Tests with different combinations of a Postgres compute node, WAL safekeeper and Page Server.

/mgmt-console:

Web UI to launch (modified) Postgres servers, using S3 as the backing store. Written in Python. This is somewhat outdated, as it doesn't use the WAL safekeeper or Page Servers.

/vendor/postgres:

PostgreSQL source tree, with the modifications needed for Zenith.

/vendor/postgres/src/bin/safekeeper:

Extension (safekeeper_proxy) that runs in the compute node, and connects to the WAL safekeepers and streams the WAL

Description
Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
Readme Apache-2.0 133 MiB
Languages
Rust 73.5%
Python 19.4%
C 5.2%
Dockerfile 0.8%
Shell 0.3%
Other 0.8%