mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-16 18:02:56 +00:00
7ed9eb4a56
This script can be used to remove tenant directories on safekeepers for projects which do not longer exist (deleted in the console). To run this script you need to upload it to safekeeper (i.e. with SSH), and run it with python3. Ansible can be used to run this script on multiple safekeepers. Fixes https://github.com/neondatabase/cloud/issues/3356
55 lines
2.0 KiB
Markdown
55 lines
2.0 KiB
Markdown
# Cleanup script for safekeeper
|
|
|
|
This script can be used to remove tenant directories on safekeepers for projects which do not longer exist (deleted in console).
|
|
|
|
To run this script you need to upload it to safekeeper (i.e. with SSH), and run it with python3. Ansible can be used to run this script on multiple safekeepers.
|
|
|
|
NOTE: Console queries to check that project is deleted are slow and inefficient.
|
|
If you want to run this script on safekeeper with many tenants, consider
|
|
making PR to console repo to make projects search by tenant_id faster.
|
|
|
|
## How to run on a single node
|
|
|
|
```
|
|
zsh nsh safekeeper-0.us-east-2.aws.neon.build
|
|
|
|
ls /storage/safekeeper/data/ | grep -v safekeeper > tenants.txt
|
|
|
|
mkdir -p /storage/neon-trash/2023-01-01--cleanup
|
|
|
|
export CONSOLE_API_TOKEN=
|
|
python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME --dry-run
|
|
|
|
cat tenants.txt | python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME --dry-run
|
|
|
|
cat tenants.txt | python3 script.py --trash-dir /storage/neon-trash/2023-01-01--cleanup --safekeeper-id $(cat /storage/safekeeper/data/safekeeper.id) --safekeeper-host $HOSTNAME |& tee logs.txt
|
|
```
|
|
|
|
## How to use ansible (staging)
|
|
|
|
```
|
|
cd ~/neon/.github/ansible
|
|
|
|
export AWS_DEFAULT_PROFILE=dev
|
|
|
|
ansible-playbook -i staging.us-east-2.hosts.yaml -e @ssm_config ../../scripts/sk_cleanup_tenants/remote.yaml
|
|
|
|
# add --extra-vars "api_token=" to set console api token
|
|
```
|
|
|
|
## How to use ansible (prod)
|
|
|
|
- Change `endpoint` in `script.py` to "https://console.neon.tech/api"
|
|
|
|
```
|
|
cd ~/neon/.github/ansible
|
|
|
|
export AWS_DEFAULT_PROFILE=prod
|
|
|
|
ansible-playbook -i prod.us-east-2.hosts.yaml -e @ssm_config ../../scripts/sk_cleanup_tenants/remote.yaml
|
|
|
|
# add --extra-vars "api_token=" to set console api token
|
|
```
|
|
|
|
|
|
> Heavily inspired with script for pageserver cleanup: https://gist.github.com/problame/bafb6ca6334f0145757238e61380c3f1/9bef1845a8291ebfa1f3a51eb79c01d12498b2b5 |