One Server

Docs

Files

Buckets and uploads, the difference between public and private, S3 access, moving files from Supabase, and resumable (TUS) uploads.

Files

POST   /storage/v1/bucket                    {"id":"avatars","public":true}
POST   /storage/v1/object/avatars/<name>      body: the file bytes
GET    /storage/v1/object/avatars/<name>
DELETE /storage/v1/object/avatars/<name>

Public and private buckets

Files in a bucket with public: true can be downloaded by anyone and are cached by the CDN. Files in any other bucket can be downloaded only by the person who uploaded them, and are not cached.

Things that are already public, such as profile pictures, go in a public bucket; everything else is private by default.

File policies written for Supabase work as written. storage.foldername(name), storage.filename(name), storage.extension(name) and the name and owner columns of storage.buckets exist, so a condition such as (storage.foldername(name))[1] = auth.uid()::text, added on the console's [Policies] screen, by a move from Supabase, or as create policy … on storage.objects in the SQL editor, run_sql or a migration, is put in this project's names and works. In SQL, One Server makes such a policy itself once the rest of the SQL has run, because storage's tables belong to the platform.

Using S3

Tools that speak S3, such as the AWS SDKs, the aws CLI and rclone, work with the same buckets. A file uploaded over REST shows up over S3, and the other way round.

endpoint:        https://<ref>.<zone>/storage/v1/s3
region:          ap-northeast-2
forcePathStyle:  true

Create an access key with create_s3_key (the secret access key is in that response only), see keys with list_s3_keys and revoke one with revoke_s3_key. An access key is the service role and bypasses access rules (RLS), so keep it on a server.

To act with an app user's permissions, use session-token mode: the project ref as the access key ID, a publishable key (bw_publishable_…) as the secret access key and the user's access token as the session token. That user's access rules apply as they are.

ListBuckets, CreateBucket, DeleteBucket, HeadBucket, ListObjectsV2, GetObject (Range), HeadObject, PutObject, CopyObject, DeleteObject(s) and multipart uploads are supported. Versioning, ACLs, tagging and lifecycle answer NotImplemented.

Moving files from Supabase

Supabase Storage speaks S3 too, so one rclone command moves a whole bucket. Paths stay the same, so file paths stored in your tables (avatars/123.png) need no change.

1. Make a bucket with the same name here first (create_bucket), public or private as it was on Supabase. 2. Make an S3 access key in Supabase's Storage settings, and one here with create_s3_key. 3. Put both in your rclone config (~/.config/rclone/rclone.conf).

[supabase]
type = s3
provider = Other
endpoint = https://<supabase-ref>.supabase.co/storage/v1/s3
region = <your Supabase project's region>
access_key_id = <Supabase access key ID>
secret_access_key = <Supabase secret access key>

[oneend]
type = s3
provider = Other
endpoint = https://<ref>.<zone>/storage/v1/s3
region = ap-northeast-2
access_key_id = <access key ID from create_s3_key>
secret_access_key = <secret access key from create_s3_key>

4. Copy each bucket. Running it again sends only what changed.

rclone copy supabase:avatars oneend:avatars --progress
rclone check supabase:avatars oneend:avatars   # nothing missing

With only the aws CLI, download and upload again.

aws s3 sync s3://avatars ./avatars --endpoint-url https://<supabase-ref>.supabase.co/storage/v1/s3
aws s3 sync ./avatars s3://avatars --endpoint-url https://<ref>.<zone>/storage/v1/s3

The plan sets the largest single file (Free 50MB, Pro 5GB). Public addresses become https://<ref>.<zone>/storage/v1/object/public/<bucket>/<path>, so if a table stores full addresses, change only those.

Resumable uploads (TUS)

Upload large files with TUS and they can carry on after a dropped connection. Use tus-js-client or Uppy as they are.

endpoint:  https://<ref>.<zone>/storage/v1/upload/resumable
headers:   Authorization: Bearer <key or access token>, x-upsert: true
metadata:  bucketName, objectName, contentType, cacheControl

An upload that receives nothing for a day expires and is removed. The plan sets the largest single file, for a standard upload as over S3 and TUS (Free 50MB, Pro 5GB). A smaller size limit on the bucket wins.

Files with tools

list_objects        list a folder of a bucket (prefix, search)
upload_object       upload one file (up to 2MB, binary as base64)
download_object     download one file (up to 2MB)
delete_object       delete one file
create_signed_url   an address a private file can be downloaded from for a set time (7 days at most)
create_upload_url   an address to PUT a large file (an image or a video) to (2 hours, one file)
update_bucket       change whether a bucket is public, its file size limit and the types it takes
empty_bucket        delete every file in a bucket (its name in confirm)
delete_bucket       delete an empty bucket (its name in confirm)

Upload and download bigger files through the /storage/v1 addresses above.

To put in an image or a video you have now, do not send its contents as a tool argument: get a URL with create_upload_url and PUT the file to that url: curl -X PUT -H 'content-type: video/mp4' --data-binary @./intro.mp4 '<url>'. One PUT takes up to max_bytes (the plan's per-file limit: Free 50MB, Pro 5GB; or the bucket's limit if smaller). For a large file on a slow line, the resumable (TUS) upload can pick up where it stopped. Over REST it is POST /v1/projects/<ref>/buckets/<bucket>/upload-urls.

When making a bucket (create_bucket) or later (update_bucket), file_size_limit (such as "5MB") and allowed_mime_types (such as ["image/*"]) set which files it takes. To delete a bucket, empty it first with empty_bucket, then put its name exactly in the confirm of delete_bucket. empty_bucket takes the bucket name in confirm too. Every tool that cannot be undone takes the name of what it removes in confirm the same way: empty_bucket, delete_queue, purge_queue, reset_branch, delete_branch, remove_function, remove_domain, delete_gpu_server, disable_extension, remove_user (email or id), and the project name for reset_database_password, set_spend_cap (turning it off) and rotate_signing_key (with grace_hours: 0). A wrong or empty one is refused with what to put in. Over REST it is PATCH /v1/projects/<ref>/buckets/<bucket>, POST /v1/projects/<ref>/buckets/<bucket>/empty and DELETE /v1/projects/<ref>/buckets/<bucket> {confirm}.

Storage settings, earlier versions, buckets connected to deployments

get_storage_settings      largest single file, image transformation, the S3 endpoint, versioning per bucket
set_storage_settings      the project's file size limit and image transformation (what is left out stays)
set_s3_enabled            turn the S3 endpoint on or off (keys are kept)
set_bucket_versioning     whether a bucket keeps overwritten and deleted files as earlier versions, and for how long
list_object_versions      earlier versions (with name, only that file's)
restore_object_version    make an earlier version the current file again
remove_object_version     remove an earlier version for good (the file name in confirm)
list_bucket_connections   buckets connected to deployments and the variables they set
connect_bucket            put an S3 key for one bucket into deployments as <prefix>_BUCKET and so on
disconnect_bucket         revoke a connection's key and take its variables out (the prefix in confirm)
list_storage_policies     access rules on the file table (objects) and the bucket table (buckets)
create_storage_policy     a rule from the console's templates (public-read, signed-in-read, own-folder ...) or written out
remove_storage_policy     remove a rule (its name in confirm; the built-in ones stay)

Rules on the file table cannot be made with run_sql, so use create_storage_policy. auth.uid() in a condition becomes uid(). A connection to production, and disconnecting it, takes owners and admins; the rest, developers and above. Not on the Free plan.