# MinIO S3 E2E Testing Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add MinIO to docker-compose and enable E2E testing of the `@pdsjs/blobs-s3` package via `BLOB_STORAGE=s3` environment variable.

**Architecture:** MinIO runs alongside existing docker services. The `@pdsjs/node` package accepts an optional `blobs` adapter parameter. Test helper configures S3 when `BLOB_STORAGE=s3` is set, otherwise uses filesystem.

**Tech Stack:** MinIO (S3-compatible storage), @aws-sdk/client-s3, @pdsjs/blobs-s3

---

### Task 1: Add MinIO to Docker Compose

**Files:**
- Modify: `docker-compose.yml`

**Step 1: Add MinIO service**

Add after the `postgres` service:

```yaml
  minio:
    image: minio/minio:latest
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      - MINIO_ROOT_USER=minioadmin
      - MINIO_ROOT_PASSWORD=minioadmin
    command: server /data --console-address ":9001"
    volumes:
      - minio_data:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 2s
      timeout: 5s
      retries: 10
```

**Step 2: Add minio_data volume**

Add to `volumes:` section:

```yaml
  minio_data:
```

**Step 3: Verify MinIO starts**

Run: `docker compose up minio -d && docker compose logs -f minio`
Expected: "API: http://0.0.0.0:9000" and "Console: http://0.0.0.0:9001"

**Step 4: Commit**

```bash
git add docker-compose.yml
git commit -m "feat: add MinIO to docker-compose for S3 testing"
```

---

### Task 2: Add AWS SDK Dev Dependency

**Files:**
- Modify: `package.json`

**Step 1: Install @aws-sdk/client-s3**

Run: `pnpm add -D @aws-sdk/client-s3`
Expected: Package added to devDependencies

**Step 2: Commit**

```bash
git add package.json pnpm-lock.yaml
git commit -m "chore: add @aws-sdk/client-s3 dev dependency"
```

---

### Task 3: Update @pdsjs/node to Accept Blobs Adapter

**Files:**
- Modify: `packages/node/index.js`
- Modify: `packages/node/index.d.ts`

**Step 1: Update createServer to accept blobs parameter**

In `packages/node/index.js`, change the function signature and body:

```javascript
export async function createServer({
  dbPath,
  blobsDir,
  blobs: blobsArg,
  jwtSecret,
  port = 3000,
  hostname,
  appviewUrl,
  appviewDid,
  relayUrl,
  password,
}) {
```

Then change line 106 from:

```javascript
  const blobs = createFsBlobs(blobsDir);
```

To:

```javascript
  const blobs = blobsArg ?? createFsBlobs(blobsDir);
```

**Step 2: Update type definitions**

In `packages/node/index.d.ts`, update ServerOptions interface to add blobs parameter:

```typescript
export function createServer(options: {
  dbPath: string;
  blobsDir?: string;
  blobs?: import('@pdsjs/core/ports').BlobPort;
  jwtSecret: string;
  port?: number;
  hostname?: string;
  appviewUrl?: string;
  appviewDid?: string;
  relayUrl?: string;
  password?: string;
}): Promise<{
  pds: import('@pdsjs/core').PersonalDataServer;
  server: import('node:http').Server;
  db: unknown;
  listen: () => Promise<import('node:http').Server>;
  close: () => Promise<void>;
}>;
```

**Step 3: Verify existing tests still pass**

Run: `PLATFORM=node npm test -- --run`
Expected: All tests pass (no S3 configured, uses blobsDir fallback)

**Step 4: Commit**

```bash
git add packages/node/index.js packages/node/index.d.ts
git commit -m "feat(node): accept optional blobs adapter parameter"
```

---

### Task 4: Update Docker Services Helper for MinIO

**Files:**
- Modify: `test/helpers/docker-services.js`

**Step 1: Add MinIO constants and readiness check**

Add after line 4 (`const PLC_URL = ...`):

```javascript
const MINIO_URL = 'http://localhost:9000';
const USE_S3 = process.env.BLOB_STORAGE === 's3';
```

**Step 2: Add MinIO wait in ensureDockerServices**

Add after `await waitForService(RELAY_URL, 'Relay');` (around line 42):

```javascript
  if (USE_S3) {
    await waitForService(`${MINIO_URL}/minio/health/ready`, 'MinIO');
  }
```

**Step 3: Export USE_S3**

Update the export at the end of the file:

```javascript
export { RELAY_URL, PLC_URL, MINIO_URL, USE_S3 };
```

**Step 4: Commit**

```bash
git add test/helpers/docker-services.js
git commit -m "feat(test): add MinIO readiness check when BLOB_STORAGE=s3"
```

---

### Task 5: Update Node Server Helper for S3 Blobs

**Files:**
- Modify: `test/helpers/node-server.js`

**Step 1: Add USE_S3 import and S3 configuration**

Replace the entire file with:

```javascript
// test/helpers/node-server.js
import { mkdirSync, rmSync } from 'node:fs';
import { createServer } from '@pdsjs/node';

const TEST_DATA_DIR = './test-data';
const TEST_PORT = 3000;
const USE_LOCAL_INFRA = process.env.USE_LOCAL_INFRA !== 'false';
const USE_S3 = process.env.BLOB_STORAGE === 's3';

/**
 * Create S3 blobs adapter for MinIO
 * @returns {Promise<import('@pdsjs/core/ports').BlobPort>}
 */
async function createMinioBlobs() {
  const {
    S3Client,
    GetObjectCommand,
    PutObjectCommand,
    DeleteObjectCommand,
    CreateBucketCommand,
  } = await import('@aws-sdk/client-s3');
  const { createS3Blobs } = await import('@pdsjs/blobs-s3');

  const client = new S3Client({
    endpoint: 'http://localhost:9000',
    region: 'us-east-1',
    credentials: {
      accessKeyId: 'minioadmin',
      secretAccessKey: 'minioadmin',
    },
    forcePathStyle: true,
  });

  // Ensure bucket exists
  try {
    await client.send(new CreateBucketCommand({ Bucket: 'pds-blobs' }));
    console.log('Created S3 bucket: pds-blobs');
  } catch (e) {
    if (e.name !== 'BucketAlreadyOwnedByYou' && e.name !== 'BucketAlreadyExists') {
      throw e;
    }
  }

  return createS3Blobs({
    client,
    bucket: 'pds-blobs',
    GetObjectCommand,
    PutObjectCommand,
    DeleteObjectCommand,
  });
}

/**
 * Start Node.js PDS server for e2e tests
 * Cleans test data directory for fresh state each run
 * @returns {Promise<{close: () => Promise<void>}>} Server instance with close() method
 */
export async function startNodeServer() {
  // Fresh data each run
  rmSync(TEST_DATA_DIR, { recursive: true, force: true });
  mkdirSync(TEST_DATA_DIR, { recursive: true });

  // Configure blobs adapter
  const blobs = USE_S3 ? await createMinioBlobs() : undefined;
  if (USE_S3) {
    console.log('Using S3 blob storage (MinIO)');
  }

  const server = await createServer({
    port: TEST_PORT,
    dbPath: `${TEST_DATA_DIR}/pds.db`,
    blobsDir: `${TEST_DATA_DIR}/blobs`,
    blobs,
    jwtSecret: 'test-secret-for-e2e',
    // Use HTTPS hostname (via Caddy proxy) when docker infra is enabled
    hostname: USE_LOCAL_INFRA
      ? 'host.docker.internal:3443'
      : `localhost:${TEST_PORT}`,
    password: 'test-password',
    // Use local relay when docker infrastructure is available
    relayUrl: USE_LOCAL_INFRA ? 'http://localhost:2470' : undefined,
    // Keep appview pointing to production (for proxy tests)
    appviewUrl: 'https://api.bsky.app',
    appviewDid: 'did:web:api.bsky.app',
  });

  await server.listen();
  return server;
}

/**
 * Stop Node.js PDS server
 * @param {{close: () => Promise<void>}} server - Server instance from startNodeServer
 */
export async function stopNodeServer(server) {
  await server.close();
}

export { USE_LOCAL_INFRA, TEST_PORT, USE_S3 };
```

**Step 2: Verify tests pass with filesystem**

Run: `docker compose up -d && PLATFORM=node USE_LOCAL_INFRA=true npm test -- --run`
Expected: All tests pass using filesystem blobs

**Step 3: Commit**

```bash
git add test/helpers/node-server.js
git commit -m "feat(test): configure S3 blobs when BLOB_STORAGE=s3"
```

---

### Task 6: End-to-End Verification

**Step 1: Start all docker services**

Run: `docker compose down -v && docker compose up -d`
Expected: All services start including MinIO

**Step 2: Run tests with S3 backend**

Run: `PLATFORM=node USE_LOCAL_INFRA=true BLOB_STORAGE=s3 npm test -- --run`
Expected: All tests pass, console shows "Using S3 blob storage (MinIO)"

**Step 3: Verify blobs in MinIO console (optional)**

Open: `http://localhost:9001` (login: minioadmin/minioadmin)
Expected: `pds-blobs` bucket exists with blob files after test run

**Step 4: Final commit (if any cleanup needed)**

```bash
git status
# If clean, no commit needed
```

---

## Summary

After completing all tasks:
- `docker compose up -d` starts MinIO alongside other services
- `PLATFORM=node npm test` runs with filesystem blobs (default)
- `PLATFORM=node BLOB_STORAGE=s3 npm test` runs with S3 blobs via MinIO
- Existing tests cover blob operations, validating the S3 adapter works E2E