# Lexicon Resolver Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add optional lexicon validation to record writes with dynamic schema resolution.

**Architecture:** Core defines a `LexiconResolverPort` interface and wires it into write handlers (createRecord, putRecord, applyWrites). A separate `@pdsjs/lexicon-resolver` package wraps `@atproto/lex-resolver` for schema resolution and `@atproto/lexicon` for validation.

**Tech Stack:** `@atproto/lexicon` (validation), `@atproto/lex-resolver` (DNS + PDS schema fetching)

---

## Task 1: Add LexiconResolverPort to ports.js

**Files:**
- Modify: `packages/core/src/ports.js:127-129`

**Step 1: Add the port typedef**

Add before the final `export {}`:

```javascript
/**
 * @typedef {'valid' | 'unknown'} ValidationStatus
 */

/**
 * Lexicon resolver port for validating records against schemas.
 * Implementations may resolve schemas dynamically (DNS + fetch) or use static schemas.
 *
 * @typedef {Object} LexiconResolverPort
 * @property {(collection: string, record: object) => Promise<ValidationStatus>} assertValid
 *   Resolves schema if needed, validates record.
 *   Returns 'valid' on success, 'unknown' if schema unresolvable.
 *   Throws Error with descriptive message if record is invalid against known schema.
 */
```

**Step 2: Commit**

```bash
git add packages/core/src/ports.js
git commit -m "feat(core): add LexiconResolverPort typedef

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 2: Add lexiconResolver to PDS constructor

**Files:**
- Modify: `packages/core/src/pds.js:300-340`

**Step 1: Add JSDoc for the new parameter**

Add to the constructor JSDoc (after line 311):

```javascript
 * @param {import('./ports.js').LexiconResolverPort} [config.lexiconResolver] - Optional lexicon resolver for record validation
```

**Step 2: Add to constructor destructuring**

Add `lexiconResolver` to the destructuring (after `webSocket,`):

```javascript
    webSocket,
    lexiconResolver,
```

**Step 3: Store on instance**

Add after `this.webSocket = webSocket;` (line 334):

```javascript
    this.lexiconResolver = lexiconResolver;
```

**Step 4: Commit**

```bash
git add packages/core/src/pds.js
git commit -m "feat(core): add lexiconResolver to PDS constructor

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 3: Add validation to handleCreateRecord

**Files:**
- Modify: `packages/core/src/pds.js:742-819`
- Test: `packages/core/test/lexicon.test.js` (create)

**Step 1: Write failing test**

Create `packages/core/test/lexicon.test.js`:

```javascript
import { describe, expect, test } from 'vitest';

// Mock resolver that tracks calls
function createMockResolver(behavior = 'valid') {
  const calls = [];
  return {
    calls,
    async assertValid(collection, record) {
      calls.push({ collection, record });
      if (behavior === 'valid') return 'valid';
      if (behavior === 'unknown') return 'unknown';
      if (behavior === 'invalid') throw new Error('Invalid record: missing required field');
      return 'valid';
    },
  };
}

describe('Lexicon Validation', () => {
  describe('LexiconResolverPort mock', () => {
    test('calls resolver with collection and record', async () => {
      const resolver = createMockResolver('valid');
      expect(resolver.calls).toHaveLength(0);

      const status = await resolver.assertValid('app.bsky.feed.post', { text: 'hello' });

      expect(resolver.calls).toHaveLength(1);
      expect(resolver.calls[0].collection).toBe('app.bsky.feed.post');
      expect(resolver.calls[0].record).toEqual({ text: 'hello' });
      expect(status).toBe('valid');
    });

    test('returns unknown when schema not found', async () => {
      const resolver = createMockResolver('unknown');
      const status = await resolver.assertValid('com.example.unknown', {});
      expect(status).toBe('unknown');
    });

    test('throws when record is invalid', async () => {
      const resolver = createMockResolver('invalid');
      await expect(
        resolver.assertValid('app.bsky.feed.post', {})
      ).rejects.toThrow('Invalid record');
    });
  });
});
```

**Step 2: Run test to verify it passes**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test packages/core/test/lexicon.test.js
```

Expected: PASS (tests the mock behavior)

**Step 3: Update handleCreateRecord to use resolver**

In `handleCreateRecord`, replace the current record handling (around lines 785-818) with:

```javascript
    const rkey = providedRkey || createTid();
    const uri = `at://${did}/${collection}/${rkey}`;

    // Ensure $type is set
    const recordWithType = { $type: collection, ...record };

    // Validate record against schema if resolver provided
    const shouldValidate = _validate !== false;
    let validationStatus;

    if (shouldValidate && this.lexiconResolver) {
      try {
        validationStatus = await this.lexiconResolver.assertValid(
          collection,
          recordWithType,
        );
      } catch (err) {
        return Response.json(
          {
            error: 'InvalidRecord',
            message: `Invalid ${collection} record: ${err.message}`,
          },
          { status: 400 },
        );
      }
    } else if (shouldValidate && !this.lexiconResolver) {
      validationStatus = 'unknown';
    }
    // If _validate === false, validationStatus remains undefined

    // Encode record and create CID
    const encoded = cborEncodeDagCbor(recordWithType);
    const cid = cidToString(await createCid(encoded));

    // Store block
    await this.actorStorage.putBlock(cid, encoded);

    // Store record
    await this.actorStorage.putRecord(uri, cid, collection, rkey, encoded);

    // Link blobs to record
    const blobRefs = findBlobRefs(record);
    for (const blobCid of blobRefs) {
      await this.actorStorage.linkBlobToRecord(blobCid, uri);
    }

    // Create commit
    await this.createCommit(did, [{ action: 'create', uri, cid }]);

    // Get commit info
    const latestCommit = await this.actorStorage.getLatestCommit();

    return Response.json({
      uri,
      cid,
      commit: latestCommit
        ? { cid: latestCommit.cid, rev: latestCommit.rev }
        : null,
      validationStatus,
    });
```

**Step 4: Run tests**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test
```

**Step 5: Commit**

```bash
git add packages/core/src/pds.js packages/core/test/lexicon.test.js
git commit -m "feat(core): add lexicon validation to createRecord

- Calls lexiconResolver.assertValid when resolver provided
- Returns InvalidRecord error when validation fails
- Returns validationStatus in response (valid/unknown/undefined)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 4: Add validation to handlePutRecord

**Files:**
- Modify: `packages/core/src/pds.js:915-1018`

**Step 1: Add validation logic to handlePutRecord**

Find the section after scope permissions check (around line 950) and add the same validation pattern. Update to include `_validate` in destructuring and add validation logic before encoding:

```javascript
    const {
      repo,
      collection,
      rkey,
      record,
      validate: _validate,
    } = await request.json();
```

Then after scope check, before encoding:

```javascript
    // Ensure $type is set
    const recordWithType = { $type: collection, ...record };

    // Validate record against schema if resolver provided
    const shouldValidate = _validate !== false;
    let validationStatus;

    if (shouldValidate && this.lexiconResolver) {
      try {
        validationStatus = await this.lexiconResolver.assertValid(
          collection,
          recordWithType,
        );
      } catch (err) {
        return Response.json(
          {
            error: 'InvalidRecord',
            message: `Invalid ${collection} record: ${err.message}`,
          },
          { status: 400 },
        );
      }
    } else if (shouldValidate && !this.lexiconResolver) {
      validationStatus = 'unknown';
    }
```

**Step 2: Update response to include validationStatus**

Add `validationStatus` to the response object.

**Step 3: Run tests**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test
```

**Step 4: Commit**

```bash
git add packages/core/src/pds.js
git commit -m "feat(core): add lexicon validation to putRecord

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 5: Add validation to handleApplyWrites

**Files:**
- Modify: `packages/core/src/pds.js:1021-1170`

**Step 1: Extract validate param from request body**

Add `validate` to the destructuring:

```javascript
    const { repo, writes, validate, swapCommit: _swapCommit } = await request.json();
```

**Step 2: Add validation inside create case**

For create case (around line 1070), add validation before storing:

```javascript
      if ($type === 'com.atproto.repo.applyWrites#create') {
        const { collection, rkey: providedRkey, value } = write;

        // Validate if resolver provided and validate !== false
        const shouldValidate = validate !== false;
        let validationStatus;
        const recordWithType = { $type: collection, ...value };

        if (shouldValidate && this.lexiconResolver) {
          try {
            validationStatus = await this.lexiconResolver.assertValid(
              collection,
              recordWithType,
            );
          } catch (err) {
            return Response.json(
              {
                error: 'InvalidRecord',
                message: `Invalid ${collection} record: ${err.message}`,
              },
              { status: 400 },
            );
          }
        } else if (shouldValidate && !this.lexiconResolver) {
          validationStatus = 'unknown';
        }

        // ... rest of create logic, using recordWithType for encoding
```

**Step 3: Add validation to update case**

Same pattern for the update case.

**Step 4: Add validationStatus to create/update results**

```javascript
        results.push({
          $type: 'com.atproto.repo.applyWrites#createResult',
          uri,
          cid,
          validationStatus,
        });
```

**Step 5: Run tests**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test
```

**Step 6: Commit**

```bash
git add packages/core/src/pds.js
git commit -m "feat(core): add lexicon validation to applyWrites

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 6: Create lexicon-resolver package

**Files:**
- Create: `packages/lexicon-resolver/package.json`
- Create: `packages/lexicon-resolver/src/index.js`
- Create: `packages/lexicon-resolver/src/resolver.js`

**Step 1: Create package.json**

```json
{
  "name": "@pdsjs/lexicon-resolver",
  "version": "0.1.0",
  "type": "module",
  "main": "src/index.js",
  "exports": {
    ".": "./src/index.js",
    "./resolver": "./src/resolver.js"
  },
  "dependencies": {
    "@atproto/lexicon": "^0.4.0",
    "@atproto/lex-resolver": "^0.1.0"
  },
  "devDependencies": {
    "vitest": "^2.1.8"
  }
}
```

**Step 2: Create src/index.js**

```javascript
export { LexiconResolver } from './resolver.js';
```

**Step 3: Create src/resolver.js**

```javascript
import { Lexicons } from '@atproto/lexicon';
import { LexResolver } from '@atproto/lex-resolver';

/**
 * @typedef {import('@pdsjs/core/ports').ValidationStatus} ValidationStatus
 */

/**
 * Lexicon resolver using @atproto/lex-resolver for schema resolution
 * and @atproto/lexicon for validation.
 * Implements LexiconResolverPort.
 */
export class LexiconResolver {
  /**
   * @param {Object} [opts]
   * @param {number} [opts.ttl=3600000] - Cache TTL in ms (default 1 hour)
   * @param {typeof fetch} [opts.fetch] - Fetch implementation
   * @param {string} [opts.plcDirectoryUrl] - PLC directory URL
   * @param {Array<object>} [opts.schemas] - Pre-loaded static schemas (LexiconDoc[])
   */
  constructor(opts = {}) {
    this.lexicons = new Lexicons();
    this.resolver = new LexResolver({
      fetch: opts.fetch,
      plcDirectoryUrl: opts.plcDirectoryUrl,
    });
    this.ttl = opts.ttl ?? 3600000;
    this.cache = new Map(); // nsid -> { fetchedAt }

    // Pre-load static schemas
    for (const schema of opts.schemas ?? []) {
      try {
        this.lexicons.add(schema);
        this.cache.set(schema.id, { fetchedAt: Date.now() });
      } catch {
        // Schema already added or invalid, skip
      }
    }
  }

  /**
   * Validate a record against its schema.
   * @param {string} collection - The collection/NSID (e.g., 'app.bsky.feed.post')
   * @param {object} record - The record to validate
   * @returns {Promise<ValidationStatus>}
   */
  async assertValid(collection, record) {
    // Ensure schema is loaded
    const hasSchema = await this.ensureSchema(collection);
    if (!hasSchema) {
      return 'unknown';
    }

    // Validate using @atproto/lexicon
    this.lexicons.assertValidRecord(collection, record);
    return 'valid';
  }

  /**
   * Ensure a schema is loaded, fetching if necessary.
   * @param {string} nsid
   * @returns {Promise<boolean>} - true if schema is available
   */
  async ensureSchema(nsid) {
    // Check if already loaded and not stale
    const cached = this.cache.get(nsid);
    if (cached && Date.now() - cached.fetchedAt < this.ttl) {
      return this.lexicons.get(nsid) !== undefined;
    }

    // Already have it but checking staleness
    if (this.lexicons.get(nsid)) {
      return true;
    }

    // Try to resolve
    try {
      const result = await this.resolver.get(nsid);
      if (result?.lexicon) {
        this.lexicons.add(result.lexicon);
        this.cache.set(nsid, { fetchedAt: Date.now() });
        return true;
      }
    } catch {
      // Resolution failed
    }

    return false;
  }
}
```

**Step 4: Commit**

```bash
git add packages/lexicon-resolver/
git commit -m "feat(lexicon-resolver): create package with @atproto/lexicon + lex-resolver

- LexiconResolver class implementing LexiconResolverPort
- Uses @atproto/lex-resolver for DNS + PDS schema fetching
- Uses @atproto/lexicon for validation
- In-memory caching with TTL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 7: Add tests for LexiconResolver

**Files:**
- Create: `packages/lexicon-resolver/test/resolver.test.js`

**Step 1: Create test file**

```javascript
import { describe, expect, test, vi } from 'vitest';
import { LexiconResolver } from '../src/resolver.js';

// Sample lexicon schema for testing
const postSchema = {
  lexicon: 1,
  id: 'app.bsky.feed.post',
  defs: {
    main: {
      type: 'record',
      key: 'tid',
      record: {
        type: 'object',
        required: ['text', 'createdAt'],
        properties: {
          text: { type: 'string', maxLength: 3000 },
          createdAt: { type: 'string', format: 'datetime' },
        },
      },
    },
  },
};

describe('LexiconResolver', () => {
  describe('with static schemas', () => {
    test('validates record against pre-loaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const status = await resolver.assertValid('app.bsky.feed.post', {
        $type: 'app.bsky.feed.post',
        text: 'Hello world',
        createdAt: new Date().toISOString(),
      });

      expect(status).toBe('valid');
    });

    test('throws on invalid record', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      await expect(
        resolver.assertValid('app.bsky.feed.post', {
          $type: 'app.bsky.feed.post',
          // missing required fields
        })
      ).rejects.toThrow();
    });

    test('returns unknown for unloaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const status = await resolver.assertValid('com.example.unknown', {
        $type: 'com.example.unknown',
      });

      expect(status).toBe('unknown');
    });
  });

  describe('ensureSchema', () => {
    test('returns true for pre-loaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const result = await resolver.ensureSchema('app.bsky.feed.post');
      expect(result).toBe(true);
    });

    test('returns false for unknown schema when resolution fails', async () => {
      const resolver = new LexiconResolver();
      // Mock the resolver to fail
      resolver.resolver.get = vi.fn().mockRejectedValue(new Error('Not found'));

      const result = await resolver.ensureSchema('com.example.unknown');
      expect(result).toBe(false);
    });
  });
});
```

**Step 2: Run tests**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test packages/lexicon-resolver/test/resolver.test.js
```

**Step 3: Commit**

```bash
git add packages/lexicon-resolver/test/
git commit -m "test(lexicon-resolver): add unit tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 8: Install dependencies and update workspace

**Step 1: Install dependencies**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm install
```

**Step 2: Run full test suite**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm test
```

**Step 3: Run linter**

```bash
cd /Users/chadmiller/code/pds-experiment && pnpm lint
```

**Step 4: Fix any issues and commit**

```bash
git add pnpm-lock.yaml
git commit -m "chore: install lexicon-resolver dependencies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Task 9: Update endpoint comparison docs

**Files:**
- Modify: `docs/endpoint-comparison.md`

**Step 1: Update the missing parameters table**

Change from:

```markdown
| repo.createRecord | collection, record, rkey | **repo**, **validate**, swapCommit |
| repo.putRecord | collection, rkey, record | **repo**, **validate**, swapCommit, swapRecord |
| repo.applyWrites | writes | **repo**, validate, swapCommit |
```

To:

```markdown
| repo.createRecord | collection, record, rkey, **validate** | **repo**, swapCommit |
| repo.putRecord | collection, rkey, record, **validate** | **repo**, swapCommit, swapRecord |
| repo.applyWrites | writes, **validate** | **repo**, swapCommit |
```

**Step 2: Commit**

```bash
git add docs/endpoint-comparison.md
git commit -m "docs: update endpoint comparison with validate param support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```

---

## Summary

After completing all tasks:

1. **@pdsjs/core** has:
   - `LexiconResolverPort` typedef in `ports.js`
   - Optional `lexiconResolver` in PDS constructor
   - `validate` param on createRecord, putRecord, applyWrites
   - `validationStatus` in responses

2. **@pdsjs/lexicon-resolver** provides:
   - `LexiconResolver` class implementing the port
   - Uses `@atproto/lex-resolver` for DNS + PDS schema fetching
   - Uses `@atproto/lexicon` for validation
   - In-memory caching with TTL
   - Pre-loadable static schemas

3. **Usage:**
   ```javascript
   import { PersonalDataServer } from '@pdsjs/core';
   import { LexiconResolver } from '@pdsjs/lexicon-resolver';

   const pds = new PersonalDataServer({
     // ... other config
     lexiconResolver: new LexiconResolver({
       schemas: [/* optional static schemas */],
     }),
   });
   ```