6c825dcbaa
This is a big async -> sync conversion. Most of it is a pretty straightforward conversion of removing `async` and `.await` and swapping in the right std modules. I didn't find a thread-blocking version of `Notify` so I wrote one, and then realized that there was already a Mutex being used there, so I deleted my Notify and just used Condvar instead. There is one part that seems odd to me: in `handle_start_replication` there is a place where the previous code was doing a non-blocking read; there is no TcpStream::try_read() so I fell back on manually flipping the socket to non-blocking mode and then back again. This seems pretty gross, but I'm not sure exactly what to replace this with: a background thread? Extract the fd and run select() on it to first test if it's readable?
Zenith
Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes
Running local installation
- Build zenith and patched postgres
git clone --recursive https://github.com/libzenith/zenith.git
cd zenith
make
- Start pageserver and postggres on top of it (should be called from repo root):
# Create ~/.zenith with proper paths to binaries and data
# Later that would be responsibility of a package install script
>./target/debug/zenith init
# start pageserver
> ./target/debug/zenith pageserver start
Starting pageserver at '127.0.0.1:64000'
# create and configure postgres data dir
> ./target/debug/zenith pg create
Creating new postgres: path=/Users/user/code/zenith/tmp_check_cli/compute/pg1 port=55432
Database initialized
# start it
> ./target/debug/zenith pg start pg1
# look up status and connection info
> ./target/debug/zenith pg list
NODE ADDRESS STATUS
pg1 127.0.0.1:55432 running
- Now it is possible to connect to postgres and run some queries:
> psql -p55432 -h 127.0.0.1 postgres
postgres=# CREATE TABLE t(key int primary key, value text);
CREATE TABLE
postgres=# insert into t values(1,1);
INSERT 0 1
postgres=# select * from t;
key | value
-----+-------
1 | 1
(1 row)
Running tests
git clone --recursive https://github.com/libzenith/zenith.git
make # builds also postgres and installs it to ./tmp_install
cargo test -- --test-threads=1
Source tree layout
/walkeeper:
WAL safekeeper. Written in Rust.
/pageserver:
Page Server. Written in Rust.
Depends on the modified 'postgres' binary for WAL redo.
/integration_tests:
Tests with different combinations of a Postgres compute node, WAL safekeeper and Page Server.
/mgmt-console:
Web UI to launch (modified) Postgres servers, using S3 as the backing store. Written in Python. This is somewhat outdated, as it doesn't use the WAL safekeeper or Page Servers.
/vendor/postgres:
PostgreSQL source tree, with the modifications needed for Zenith.
/vendor/postgres/src/bin/safekeeper:
Extension (safekeeper_proxy) that runs in the compute node, and connects to the WAL safekeepers and streams the WAL