Skip to content
Build With SyntaxKit

Storage

S3-compatible uploads in packages/storage. The browser PUTs to a presigned URL; the server validates with sharp, writes a final key, and deletes the temp. Any S3-compatible store works; only endpoint and credentials change.

Last updated on

7 min read

On This Page

How An Upload Flows

Three round-trips for an upload: presign, direct PUT to bucket, finalize. The server validates and re-encodes during finalize, then deletes the temporary key.

Every image upload is three round-trips:

  1. Presign. The client asks the API for a short-lived PUT URL and a temporaryKey under tmp/images/.
  2. PUT. The browser uploads bytes straight to the bucket. They never pass through your Next.js server.
  3. Finalize. The server reads the temp object, validates and re-encodes it with sharp, writes the final key under images/, and deletes the temp.

That split keeps large bodies off the serverless request (no Vercel 4.5 MB body limit) while the server still decides what becomes permanent. Presigned URLs expire in 6 minutes.

Re-encoding through sharp is the security boundary. If sharp cannot decode the file as an image, finalize fails and the temp key is deleted. Original bytes never become the served file.

Package Layout

index.tsServer entry: client, keys, policy, image pipeline, upload workflow, cleanup, S3 commands
client.tsS3Client construction from AWS_REGION, AWS_ENDPOINT_URL_S3, AWS_S3_FORCE_PATH_STYLE
policy.tsALLOWED_MIME_TYPES, MAX_FILE_SIZE, Zod upload schemas. Client-safe via @syntaxkit/storage/policy
keys.tsimages/ and tmp/images/ prefixes, key builders, ownership checks, owner parsing
image.tssharp pipeline: validate and re-encode on finalize
upload.tspresignImageUpload + finalizeImageUpload: S3 head/get/put and temp-key lifecycle
errors.tsStorageError. The API router maps it to ORPCError
utils.tsisS3Key, delete helpers, prefix cleanup. Used by API and auth hooks
hooks.tsuseFileUpload React hook for upload UIs

packages/storage owns key conventions, policy, the sharp pipeline, and the presign/finalize workflow. presignImageUpload and finalizeImageUpload are plain functions that throw StorageError.

packages/api/src/router/storage.ts is a thin adapter: auth, abuse policy, bucket from env, then map StorageError to ORPCError. Storage owns the workflow; API owns the transport.

Configuring Storage

Provider env blocks (AWS S3, Cloudflare R2, MinIO) and a verification step live on Setup: Storage. Same SDK and code path for every S3-compatible store. Only the endpoint and credentials change.

Full per-variable reference: Environment Variables: Object Storage (S3).

File Constraints And Server-Side Validation

ConstraintValue
Allowed MIME typesimage/jpeg, image/png, image/webp, image/gif
Max file size1 MB
Max output dimension2048 px (longest side, fit inside)
Output formatJPEG (mozjpeg, q85) by default; PNG when alpha is present

The client rejects wrong size or MIME before presign. Server-side sharp is the source of truth: a file that does not decode, or exceeds the dimension limit, fails finalize and the temp object is removed. Final outputs are always re-encoded.

Presigned PUT URLs do not reliably enforce Content-Length across providers, so size is checked three times:

  1. Client and presign Zod schema reject anything above MAX_FILE_SIZE before a URL is signed.
  2. Finalize runs HeadObject and refuses a temp key whose Content-Length is missing or over the limit. No body download yet.
  3. GetObject is read with a hard byte cap. If the stream exceeds the cap (misstated size, or a rewrite between HEAD and GET), the read aborts and the temp key is deleted. The worker never holds more than MAX_FILE_SIZE in memory.

What Gets Uploaded

Required Bucket Configuration

The kit does not provision your bucket. Two settings are easy to miss because they are operational, not code.

CORS. The browser PUTs directly to a presigned URL. Without CORS allowing PUT from your app origin, uploads fail with an opaque toast. Recommended rule:

[
  {
    "AllowedOrigins": ["https://your-app.com"],
    "AllowedMethods": ["PUT"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"]
  }
]

Lifecycle rule for tmp/. The kit deletes temp objects when finalize succeeds, when image processing fails, or when the final PUT fails. It cannot catch a user who abandons the upload after presign: the server is never called again. Expire tmp/ after 1 day as a safety net:

{
  "Rules": [
    {
      "ID": "expire-tmp-uploads",
      "Status": "Enabled",
      "Filter": { "Prefix": "tmp/" },
      "Expiration": { "Days": 1 }
    }
  ]
}

Graceful Degradation

When the storage env block is missing, isStorageEnabled returns false:

  • Personal settings and organization settings replace the avatar form with a placeholder that says storage is not configured.
  • The AI chat composer hides the image-attachment button. Chat otherwise keeps working.

pnpm setup:doctor reports storage status with every other capability. See Environment Variables for the full env matrix.

Cleanup Behavior

Four helpers cover cleanup from a single key to everything a principal uploaded. The notes below are the gotchas that usually trip people up.

deleteS3ImageIfNeeded

Deletes a finalized image key. Skips external URLs. Best-effort: errors are logged, not thrown. Used when a user or org replaces an image, and from beforeUserDelete / afterOrganizationDelete for the current avatar or logo.

deleteS3Keys

Deletes an explicit list of keys in batches of up to 1000 (S3 DeleteObjects limit). Used from beforeOrganizationDelete: collect chat-attachment URLs from Message.parts, then delete before Prisma cascades the rows. Best-effort per chunk; one failed batch does not stop the rest.

deleteS3Prefix

Lists every object under a prefix (ListObjectsV2 pagination) and deletes via deleteS3Keys. Used from beforeUserDelete for images/{userId}/ and tmp/images/{userId}/. Wipes every byte that user uploaded, whether or not a message still references it.

deleteObjectIfPresent

Deletes an in-flight temp key inside finalizeImageUpload when validation, sharp processing, or the final PUT fails. Covers every server-observable failure path between the user's temp PUT and a successful final write.

Why the org-delete hook runs before the cascade

afterDeleteOrganization cannot read messages: Prisma has already cascaded Chat / Message rows. So Message.parts is read in beforeDeleteOrganization. Logo cleanup still runs in afterDeleteOrganization. Together they cover every S3 byte tied to the org.

Single-reference assumption

Each presign + finalize mints a fresh crypto.randomUUID() key. The chat composer never reuses a previous upload, so each key under images/{userId}/ is referenced by at most one message. Cleanup deletes by key without reference counting. If you add attachment reuse, track (key, userId, refCount) and only delete from S3 when refCount hits zero.

IAM

Cleanup needs list and delete on the bucket: s3:ListBucket, s3:DeleteObject, and s3:DeleteObjects on the bucket and object ARNs (arn:aws:s3:::bucket and arn:aws:s3:::bucket/*, or the R2/MinIO equivalent). Without s3:ListBucket, the user-delete prefix wipe becomes a silent no-op.

Adding A New Upload Surface

Same pattern for any new file-upload feature.

Reuse the presign and finalize procedures

Do not build a parallel upload route. orpc.storage.presign and orpc.storage.finalize already enforce auth, abuse policy, key ownership, MIME, size, and image validation.

Mirror the client pattern from AvatarUploader

Call orpc.storage.presign with file metadata, PUT bytes to the returned URL with the correct Content-Type, then call orpc.storage.finalize with the temporaryKey. Chat attachments use the same shape.

Decide where the resulting imageKey lives

For simple replacements, write through an existing mutation that calls deleteS3ImageIfNeeded on the old value. For new entities, store the key on the row and add a Prisma cascade or delete hook.

Render public URLs with getImageUrl

Composes NEXT_PUBLIC_S3_PUBLIC_URL plus the key, and passes external URLs through unchanged. Do not string-template the URL yourself.

Extend CSP if your upload host differs from the public host

getStorageConnectOrigins in apps/web/lib/storage-origins.ts covers the two hosts the kit uses. If storage uses a third hostname, extend the helper so the browser can fetch it.

Where To Go Next

Also useful: Authentication, AI, Database, and Environment Variables.

Was this page helpful?

On this page