rust/neon

mirror of https://github.com/neondatabase/neon.git synced 2025-12-23 06:09:59 +00:00

Files

Tristan Partin 5bd8e2363a Enable all pyupgrade checks in ruff

This will help to keep us from using deprecated Python features going
forward.

Signed-off-by: Tristan Partin <tristan@neon.tech>

2024-10-08 14:32:26 -05:00

readme.md

Add script for safekeeper tenants cleanup (#3452 )

2023-02-09 13:28:20 +02:00

remote.yaml

Add script for safekeeper tenants cleanup (#3452 )

2023-02-09 13:28:20 +02:00

script.py

Enable all pyupgrade checks in ruff

2024-10-08 14:32:26 -05:00

readme.md

Cleanup script for safekeeper

This script can be used to remove tenant directories on safekeepers for projects which do not longer exist (deleted in console).

To run this script you need to upload it to safekeeper (i.e. with SSH), and run it with python3. Ansible can be used to run this script on multiple safekeepers.

NOTE: Console queries to check that project is deleted are slow and inefficient. If you want to run this script on safekeeper with many tenants, consider making PR to console repo to make projects search by tenant_id faster.

How to run on a single node

zsh nsh safekeeper-0.us-east-2.aws.neon.build

ls /storage/safekeeper/data/ | grep -v safekeeper > tenants.txt

mkdir -p /storage/neon-trash/2023-01-01--cleanup

 export CONSOLE_API_TOKEN=
python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME --dry-run

cat tenants.txt | python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME --dry-run

cat tenants.txt | python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME |& tee logs.txt

How to use ansible (staging)

cd ~/neon/.github/ansible

export AWS_DEFAULT_PROFILE=dev

ansible-playbook -i staging.us-east-2.hosts.yaml -e @ssm_config ../../scripts/sk_cleanup_tenants/remote.yaml

# add --extra-vars "api_token=" to set console api token

How to use ansible (prod)

Change endpoint in script.py to "https://console.neon.tech/api"

cd ~/neon/.github/ansible

export AWS_DEFAULT_PROFILE=prod

ansible-playbook -i prod.us-east-2.hosts.yaml -e @ssm_config ../../scripts/sk_cleanup_tenants/remote.yaml

# add --extra-vars "api_token=" to set console api token

Heavily inspired with script for pageserver cleanup: https://gist.github.com/problame/bafb6ca6334f0145757238e61380c3f1/9bef1845a8291ebfa1f3a51eb79c01d12498b2b5