~ knotless Search/
↑↓ move↵ open⇧↵ all results

chadtmiller.com/pds.js

An atproto PDS in JavaScript, for Cloudflare Workers and Node

0
0
main
703 lines · 15.4 KB Raw History

Deno Support Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Add Deno runtime support to pds.js by abstracting the SQLite driver and creating a Deno HTTP/WebSocket adapter.

Architecture: Create a minimal SQLite driver interface that both better-sqlite3 (Node) and node:sqlite (Deno 2.2+) implement natively. Add a new @pdsjs/deno package for the HTTP server adapter using Deno.serve(). Reuse @pdsjs/storage-sqlite unchanged except for type annotations.

Tech Stack: Deno 2.2+, node:sqlite, Deno.serve(), TypeScript/JSDoc


Task 1: Add SQLite Driver Interface Types

Files:

  • Create: packages/storage-sqlite/driver.js

Step 1: Create the driver interface file

// @pdsjs/storage-sqlite/driver - SQLite driver interface types
// Both better-sqlite3 and node:sqlite implement this interface natively

/**
 * @typedef {Object} SQLiteStatement
 * @property {(...args: unknown[]) => unknown} get - Single row query
 * @property {(...args: unknown[]) => unknown[]} all - Multi-row query
 * @property {(...args: unknown[]) => {changes: number}} run - Execute INSERT/UPDATE/DELETE
 */

/**
 * @typedef {Object} SQLiteDatabase
 * @property {(sql: string) => void} exec - Execute raw SQL (DDL, multiple statements)
 * @property {(sql: string) => SQLiteStatement} prepare - Create prepared statement
 */

export {};

Step 2: Verify file created

Run: cat packages/storage-sqlite/driver.js Expected: File contents shown

Step 3: Commit

git add packages/storage-sqlite/driver.js
git commit -m "feat(storage-sqlite): add SQLite driver interface types"

Task 2: Update Storage Functions to Use Driver Interface

Files:

  • Modify: packages/storage-sqlite/index.js (lines 24-28 and 314-318)

Step 1: Update createActorStorage type annotation

Change line 28 from:

 * @param {import('better-sqlite3').Database} db

to:

 * @param {import('./driver.js').SQLiteDatabase} db

Step 2: Update createSharedStorage type annotation

Change line 318 from:

 * @param {import('better-sqlite3').Database} db

to:

 * @param {import('./driver.js').SQLiteDatabase} db

Step 3: Run existing tests to verify no breakage

Run: pnpm test Expected: All tests pass

Step 4: Commit

git add packages/storage-sqlite/index.js
git commit -m "feat(storage-sqlite): use driver interface instead of better-sqlite3 types"

Task 3: Export Driver Types from Package

Files:

  • Modify: packages/storage-sqlite/package.json

Step 1: Add driver.js to exports

Add to the exports field:

{
  "exports": {
    ".": "./index.js",
    "./driver": "./driver.js"
  }
}

Step 2: Verify package.json is valid

Run: node -e "console.log(JSON.parse(require('fs').readFileSync('packages/storage-sqlite/package.json')))" Expected: No JSON parse errors

Step 3: Commit

git add packages/storage-sqlite/package.json
git commit -m "feat(storage-sqlite): export driver interface types"

Task 4: Create @pdsjs/deno Package Structure

Files:

  • Create: packages/deno/package.json
  • Create: packages/deno/deno.json

Step 1: Create package.json

{
  "name": "@pdsjs/deno",
  "version": "0.7.0",
  "type": "module",
  "exports": {
    ".": "./index.js"
  },
  "dependencies": {
    "@pdsjs/core": "workspace:*"
  }
}

Step 2: Create deno.json

{
  "name": "@pdsjs/deno",
  "version": "0.7.0",
  "exports": "./index.js",
  "imports": {
    "@pdsjs/core": "../core/index.js",
    "@pdsjs/core/": "../core/"
  }
}

Step 3: Commit

git add packages/deno/package.json packages/deno/deno.json
git commit -m "feat(deno): create package structure"

Task 5: Implement Deno WebSocket Port

Files:

  • Create: packages/deno/index.js

Step 1: Create the WebSocket port implementation

// @pdsjs/deno - Deno HTTP server adapter

/**
 * Create WebSocket port for Deno
 * @param {Map<Request, {socket: WebSocket, response: Response}>} upgradeMap
 * @returns {import('@pdsjs/core/ports.js').WebSocketPort}
 */
export function createWebSocketPort(upgradeMap) {
  /** @type {Set<WebSocket>} */
  const clients = new Set();

  return {
    isUpgrade(request) {
      return request.headers.get('upgrade')?.toLowerCase() === 'websocket';
    },

    upgrade(request, onConnect) {
      const upgrade = Deno.upgradeWebSocket(request);
      const { socket, response } = upgrade;

      clients.add(socket);
      socket.onclose = () => clients.delete(socket);

      socket.onopen = () => {
        onConnect({
          send(data) {
            if (socket.readyState === WebSocket.OPEN) {
              socket.send(data);
            }
          },
          close() {
            socket.close();
          },
        });
      };

      return response;
    },

    broadcast(data) {
      for (const client of clients) {
        if (client.readyState === WebSocket.OPEN) {
          try {
            client.send(data);
          } catch {
            // Client may have disconnected
          }
        }
      }
    },
  };
}

Step 2: Commit

git add packages/deno/index.js
git commit -m "feat(deno): implement WebSocket port"

Task 6: Implement Deno HTTP Server

Files:

  • Modify: packages/deno/index.js

Step 1: Add createServer function

Append to packages/deno/index.js:

import { PersonalDataServer } from '@pdsjs/core';

/**
 * @typedef {Object} DenoServerOptions
 * @property {import('@pdsjs/core/ports.js').ActorStoragePort} actorStorage
 * @property {import('@pdsjs/core/ports.js').SharedStoragePort} sharedStorage
 * @property {import('@pdsjs/core/ports.js').BlobPort} blobs
 * @property {string} jwtSecret
 * @property {number} [port=3000]
 * @property {string} [hostname]
 * @property {string} [appviewUrl]
 * @property {string} [appviewDid]
 * @property {string} [relayUrl]
 * @property {string} [password]
 */

/**
 * @typedef {Object} DenoServer
 * @property {PersonalDataServer} pds
 * @property {Deno.HttpServer} server
 * @property {() => void} close
 */

/**
 * Create Deno PDS server
 * @param {DenoServerOptions} options
 * @returns {DenoServer}
 */
export function createServer({
  actorStorage,
  sharedStorage,
  blobs,
  jwtSecret,
  port = 3000,
  hostname,
  appviewUrl,
  appviewDid,
  relayUrl,
  password,
}) {
  /** @type {Map<Request, {socket: WebSocket, response: Response}>} */
  const upgradeMap = new Map();
  const webSocket = createWebSocketPort(upgradeMap);

  const pds = new PersonalDataServer({
    actorStorage,
    sharedStorage,
    blobs,
    webSocket,
    jwtSecret,
    hostname,
    appviewUrl,
    appviewDid,
    relayUrl,
    password,
  });

  const server = Deno.serve({ port }, async (request) => {
    // Handle WebSocket upgrade
    if (webSocket.isUpgrade(request)) {
      return pds.fetch(request);
    }

    // Handle regular HTTP
    return pds.fetch(request);
  });

  // Cleanup interval (every hour)
  const cleanupInterval = setInterval(
    async () => {
      await pds.runBlobCleanup();
      await sharedStorage.cleanupExpiredDpopJtis();
    },
    60 * 60 * 1000,
  );

  return {
    pds,
    server,
    port,
    close() {
      clearInterval(cleanupInterval);
      server.shutdown();
    },
  };
}

/**
 * Create server from environment variables
 * @param {Object} deps - Storage dependencies
 * @param {import('@pdsjs/core/ports.js').ActorStoragePort} deps.actorStorage
 * @param {import('@pdsjs/core/ports.js').SharedStoragePort} deps.sharedStorage
 * @param {import('@pdsjs/core/ports.js').BlobPort} deps.blobs
 * @returns {DenoServer}
 */
export function createServerFromEnv({ actorStorage, sharedStorage, blobs }) {
  return createServer({
    actorStorage,
    sharedStorage,
    blobs,
    jwtSecret: Deno.env.get('JWT_SECRET') || 'development-secret',
    port: parseInt(Deno.env.get('PORT') || '3000', 10),
    hostname: Deno.env.get('HOSTNAME'),
    appviewUrl: Deno.env.get('APPVIEW_URL'),
    appviewDid: Deno.env.get('APPVIEW_DID'),
    relayUrl: Deno.env.get('RELAY_URL'),
    password: Deno.env.get('PDS_PASSWORD'),
  });
}

Step 2: Run type check

Run: pnpm exec tsc --noEmit Expected: No type errors

Step 3: Commit

git add packages/deno/index.js
git commit -m "feat(deno): implement HTTP server with createServer"

Task 7: Create Deno Example

Files:

  • Create: examples/deno/main.ts
  • Create: examples/deno/deno.json

Step 1: Create deno.json

{
  "imports": {
    "@pdsjs/core": "../../packages/core/index.js",
    "@pdsjs/core/": "../../packages/core/",
    "@pdsjs/deno": "../../packages/deno/index.js",
    "@pdsjs/storage-sqlite": "../../packages/storage-sqlite/index.js",
    "@pdsjs/blobs-s3": "../../packages/blobs-s3/index.js"
  }
}

Step 2: Create main.ts

import { DatabaseSync } from 'node:sqlite';
import { createActorStorage, createSharedStorage } from '@pdsjs/storage-sqlite';
import { createServer } from '@pdsjs/deno';

// Initialize SQLite database
const dbPath = Deno.env.get('PDS_DB_PATH') || './pds.db';
const db = new DatabaseSync(dbPath);

// Create storage adapters
const actorStorage = createActorStorage(db);
const sharedStorage = createSharedStorage(db);

// For blobs, use S3 or implement Deno filesystem adapter
// This example uses a minimal in-memory blob store for demo purposes
const blobs = {
  async get(_did: string, _cid: string) {
    return null; // TODO: implement with Deno.readFile or S3
  },
  async put(_did: string, _cid: string, _data: Uint8Array, _mimeType: string) {
    // TODO: implement
  },
  async delete(_did: string, _cid: string) {
    // TODO: implement
  },
};

// Create and start server
const server = createServer({
  actorStorage,
  sharedStorage,
  blobs,
  jwtSecret: Deno.env.get('JWT_SECRET') || 'development-secret',
  port: parseInt(Deno.env.get('PORT') || '3000', 10),
  hostname: Deno.env.get('HOSTNAME'),
  appviewUrl: Deno.env.get('APPVIEW_URL'),
  appviewDid: Deno.env.get('APPVIEW_DID'),
  relayUrl: Deno.env.get('RELAY_URL'),
  password: Deno.env.get('PDS_PASSWORD'),
});

console.log(`PDS running on http://localhost:${server.port}`);

// Handle shutdown
Deno.addSignalListener('SIGINT', () => {
  console.log('Shutting down...');
  server.close();
  db.close();
  Deno.exit(0);
});

Step 3: Commit

git add examples/deno/main.ts examples/deno/deno.json
git commit -m "feat(examples): add Deno example"

Task 8: Add Deno Filesystem Blobs (Optional)

Files:

  • Create: packages/blobs-deno/index.js
  • Create: packages/blobs-deno/package.json
  • Create: packages/blobs-deno/deno.json

Step 1: Create package.json

{
  "name": "@pdsjs/blobs-deno",
  "version": "0.7.0",
  "type": "module",
  "exports": {
    ".": "./index.js"
  }
}

Step 2: Create deno.json

{
  "name": "@pdsjs/blobs-deno",
  "version": "0.7.0",
  "exports": "./index.js"
}

Step 3: Create index.js

// @pdsjs/blobs-deno - Deno filesystem blob storage

import { join, dirname } from 'node:path';

/**
 * Create filesystem blob storage for Deno
 * @param {string} baseDir - Base directory for blob storage
 * @returns {import('@pdsjs/core/ports.js').BlobPort}
 */
export function createDenoBlobs(baseDir) {
  /**
   * Get blob file path
   * @param {string} did
   * @param {string} cid
   */
  function blobPath(did, cid) {
    // Use first 2 chars of CID for sharding
    const shard = cid.slice(0, 2);
    return join(baseDir, did, shard, cid);
  }

  return {
    async get(did, cid) {
      try {
        const path = blobPath(did, cid);
        const data = await Deno.readFile(path);
        const metaPath = `${path}.meta`;
        const meta = JSON.parse(await Deno.readTextFile(metaPath));
        return { data, mimeType: meta.mimeType };
      } catch (e) {
        if (e instanceof Deno.errors.NotFound) {
          return null;
        }
        throw e;
      }
    },

    async put(did, cid, data, mimeType) {
      const path = blobPath(did, cid);
      await Deno.mkdir(dirname(path), { recursive: true });
      await Deno.writeFile(path, data);
      await Deno.writeTextFile(`${path}.meta`, JSON.stringify({ mimeType }));
    },

    async delete(did, cid) {
      try {
        const path = blobPath(did, cid);
        await Deno.remove(path);
        await Deno.remove(`${path}.meta`);
      } catch (e) {
        if (!(e instanceof Deno.errors.NotFound)) {
          throw e;
        }
      }
    },
  };
}

Step 4: Commit

git add packages/blobs-deno/
git commit -m "feat(blobs-deno): add Deno filesystem blob storage"

Task 9: Update Deno Example to Use Blobs

Files:

  • Modify: examples/deno/main.ts
  • Modify: examples/deno/deno.json

Step 1: Update deno.json imports

Add to imports:

"@pdsjs/blobs-deno": "../../packages/blobs-deno/index.js"

Step 2: Update main.ts to use blobs-deno

Replace the placeholder blobs with:

import { createDenoBlobs } from '@pdsjs/blobs-deno';

const blobsDir = Deno.env.get('PDS_BLOBS_DIR') || './blobs';
const blobs = createDenoBlobs(blobsDir);

Step 3: Commit

git add examples/deno/main.ts examples/deno/deno.json
git commit -m "feat(examples): use blobs-deno in Deno example"

Task 10: Add README for Deno Example

Files:

  • Create: examples/deno/README.md

Step 1: Create README

# Deno PDS Example

Minimal PDS server running on Deno.

## Requirements

- Deno 2.2+ (for `node:sqlite` support)

## Quick Start

```bash
cd examples/deno
deno run --allow-net --allow-read --allow-write --allow-env main.ts

Environment Variables

VariableDescriptionDefault
PDS_DB_PATHSQLite database path./pds.db
PDS_BLOBS_DIRBlob storage directory./blobs
JWT_SECRETJWT signing secretdevelopment-secret
PORTServer port3000
HOSTNAMEPDS hostname-
APPVIEW_URLAppView URL for proxying-
APPVIEW_DIDAppView DID for service auth-
RELAY_URLRelay URL for firehose-
PDS_PASSWORDPassword for createSession-

Permissions

The server requires these Deno permissions:

  • --allow-net - HTTP server and outbound requests
  • --allow-read - Read database and blob files
  • --allow-write - Write database and blob files
  • --allow-env - Read environment variables

**Step 2: Commit**

```bash
git add examples/deno/README.md
git commit -m "docs(examples): add Deno example README"

Task 11: Test Deno Example Manually

Step 1: Run the Deno example

cd examples/deno
deno run --allow-net --allow-read --allow-write --allow-env main.ts

Expected: Server starts, prints "PDS running on http://localhost:3000"

Step 2: Test health endpoint

curl http://localhost:3000/xrpc/_health

Expected: {"version":"..."}

Step 3: Test describe server

curl http://localhost:3000/xrpc/com.atproto.server.describeServer

Expected: JSON response with server description


Task 12: Update Root README

Files:

  • Modify: README.md

Step 1: Add Deno to supported platforms

Add to the platforms section:

### Deno

```bash
cd examples/deno
deno run --allow-net --allow-read --allow-write --allow-env main.ts

Requires Deno 2.2+ for node:sqlite support.


**Step 2: Commit**

```bash
git add README.md
git commit -m "docs: add Deno to supported platforms"