chadtmiller.com/pds.js
An atproto PDS in JavaScript, for Cloudflare Workers and Node
0
0
# @pdsjs/blobs-s3
## 2.5.2
## 2.5.1
## 2.5.0
### Minor Changes
- 078c2df: The S3 backup target sends an object past 8 MiB as a multipart upload, and
accepts a blob as a stream. A single PUT carrying a whole blob is what a
provider answers HTTP 413 to, and one such blob failed the entire snapshot:
the blob loop runs before the repo CAR and the manifest, so the run wrote
nothing at all. Reading the blob whole also put a video-sized buffer in an
isolate that has 128 MB.
A backup now streams each blob from the store to the target one part at a
time, so neither side ever holds more than a part. `BackupTargetPort` gains
an optional `putStream`, and a run counts the bytes as they pass, which keeps
its heartbeat beating across the parts of one large blob.
## 2.4.3
## 2.4.2
## 2.4.1
## 2.4.0
### Patch Changes
- 6b79149: Video-ready blob hosting. Blob uploads and downloads stream: `uploadBlob` hashes and stages bytes chunk by chunk instead of buffering the body, and `getBlob` answers byte ranges from a streamed read, so a 300 MB video fits through a Durable Object and seeks in a browser. The blob port gains optional `createWriter` and `getStream`; the fs, R2, S3 and Deno stores implement both, and each platform injects its native SHA-256. Drive stores a file as one blob of its own type when the store streams, takes large files across several requests through upload sessions, and offers a Link dialog with the blob ref any record can embed. The Worker serves public blobs through the edge cache, and a site's `/.blobs/` route serves ranges and now applies the same space-privacy filter as `getBlob`. A new `videos.mjs` example app is a gallery over the drive's videos. The node server accepts a `blobUploadLimit` option (`PDS_BLOB_UPLOAD_LIMIT` in the example), which Bluesky video posting needs.
## 2.3.0
## 2.2.0
### Minor Changes
- 224cb48: `createS3FetchBackupTarget`: back up to any S3-compatible bucket over plain
fetch and WebCrypto — no AWS SDK, so it runs on Cloudflare Workers, Node, and
Deno alike. Signature Version 4 with path-style addressing and the four basic
operations, which is the subset every compatible provider (Bunny, Backblaze
B2, MinIO, R2, AWS) implements; the signer is verified against AWS's own
documented example vectors.
On Cloudflare this unlocks off-provider backups: set `BACKUP_S3_ENDPOINT`,
`BACKUP_S3_BUCKET`, `BACKUP_S3_REGION` as vars and the two keys as secrets,
and the target wins over the same-account `BACKUPS` R2 binding. The Node
example now uses it for `PDS_BACKUP_S3_*` too, dropping the
`@aws-sdk/client-s3` requirement.
- 224cb48: Scheduled backups, with a Backups page in the account UI.
A backup is a snapshot in a new `BackupTargetPort` — a directory
(`createFsBackupTarget`), an S3-compatible bucket (`createS3BackupTarget`), or
on Cloudflare an R2 bucket bound as `BACKUPS`. Each snapshot holds the full
signed repo CAR (the same bytes `com.atproto.sync.getRepo` serves) and a
manifest, written last so an interrupted snapshot never counts; blobs are
stored content-addressed at `blobs/<cid>` and shared across snapshots, so a
run only uploads what the target doesn't already hold. Retention keeps the
newest N snapshots and reaps blobs no kept manifest references.
Scheduling rides the maintenance tick each runtime already had: the hourly
interval on Node and Deno, and the Durable Object alarm on Cloudflare, which
now fires hourly to match. Settings and run history live in the existing
`metadata` table as one JSON document, read through new optional
`getBackupState`/`setBackupState` methods on `ActorStoragePort`.
The signing key stays out of snapshots unless opted in: entering a passphrase
on the Backups page seals a copy under PBKDF2 → AES-256-GCM (`key.enc` in each
snapshot), and the page warns when the key has rotated since it was sealed.
The page also offers one-click manual runs, run history, schedule and
retention settings, and a direct `.car` download that works even with no
target configured.
## 2.1.0
## 2.0.0
## 1.0.2
## 1.0.1
## 1.0.0