rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-10-28 22:46:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'alek_targz' of github.com:neondatabase/neon into alek_targz_default_on

Browse Source
This commit is contained in:
Alek Westover
2023-07-17 07:59:30 -04:00
parent 0a00869615 87eead5220
commit ca22453627
2 changed files with 29 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
compute_tools/src/extension_server.rs
+1 -1
View File
@@ -157,7 +157,7 @@ pub async fn download_extension(
.read_to_end(&mut write_data_buffer)
.await?;
let unzip_dest = pgbin.strip_suffix("/bin/postgres").expect("bad pgbin");
let tar = GzDecoder::new(std::io::Cursor::new(write_data_buffer));
let tar = GzDecoder::new(write_data_buffer.as_slice());
let mut archive = Archive::new(tar);
archive.unpack(unzip_dest)?;
info!("Download + unzip {:?} completed successfully", &ext_path);
docs/rfcs/024-extension-loading.md
+28 -24
View File
@@ -139,30 +139,34 @@ popular extensions.
## Extension Storage implementation
Extension Storage in our case is an S3 bucket with a "directory" per build and postgres version,
where extension files are stored as plain files in the bucket following the same directory structure as in the postgres.
i.e.
`s3://<the-bucket>/<build-version>/<postgres-version>/lib/postgis-3.1.so`
`s3://<the-bucket>/<build-version>/<postgres-version>/share/extension/postgis.control`
`s3://<the-bucket>/<build-version>/<postgres-version>/share/extension/postgis--3.1.sql`
To handle custom extensions, that available only to specific users, we use per-extension subdirectories:
i.e.
`s3://<the-bucket>/<build-version>/<postgres-version>/<custom-ext-prefix>/lib/ext-name.so`, etc.
`s3://<the-bucket>/<build-version>/<postgres-version>/<custom-ext-prefix>/share/extension/ext-name.control`, etc.
On compute start, `compute_ctl` accepts a list of custom_ext_prefixes.
To get the list of available extensions,`compute_ctl` downloads control files from all prefixes:
`s3://<the-bucket>/<build-version>/<postgres-version>/share/extension/`
`s3://<the-bucket>/<build-version>/<postgres-version>/<custom-ext-prefix1>/share/extension/`
`s3://<the-bucket>/<build-version>/<postgres-version>/<custom-ext-prefix2>/share/extension/`
The layout of the S3 bucket is as follows:
```
v14/ext_index.json
-- this contains information necessary to create control files
v14/extensions/test_ext1.tar.gz
-- this contains the library files and sql files necessary to create this extension
v14/extensions/custom_ext1.tar.gz
```
The difference between private and public extensions is determined by who can
load the extension. This is specified in `ext_index.json`.
Speicially, `ext_index.json` has a list of public extensions, and a list of
extensions enabled for specific tenant-ids. Here is an example `ext_index.json`:
```
{
"enabled_extensions": {
"123454321": [
"anon"
],
"public": [
"embedding"
]
},
"control_data": {
"embedding": "comment = 'hnsw index' \ndefault_version = '0.1.0' \nmodule_pathname = '$libdir/embedding' \nrelocatable = true \ntrusted = true",
"anon": "# PostgreSQL Anonymizer (anon) extension \ncomment = 'Data anonymization tools' \ndefault_version = '1.1.0' \ndirectory='extension/anon' \nrelocatable = false \nrequires = 'pgcrypto' \nsuperuser = false \nmodule_pathname = '$libdir/anon' \ntrusted = true \n"
}
}
```
### How to add new extension to the Extension Storage?