Lexicon Resolver Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Add optional lexicon validation to record writes with dynamic schema resolution.

Architecture: Core defines a LexiconResolverPort interface and wires it into write handlers (createRecord, putRecord, applyWrites). A separate @pdsjs/lexicon-resolver package wraps @atproto/lex-resolver for schema resolution and @atproto/lexicon for validation.

Tech Stack: @atproto/lexicon (validation), @atproto/lex-resolver (DNS + PDS schema fetching)

Task 1: Add LexiconResolverPort to ports.js

Files:

Modify: packages/core/src/ports.js:127-129

Step 1: Add the port typedef

Add before the final export {}:

/**
 * @typedef {'valid' | 'unknown'} ValidationStatus
 */

/**
 * Lexicon resolver port for validating records against schemas.
 * Implementations may resolve schemas dynamically (DNS + fetch) or use static schemas.
 *
 * @typedef {Object} LexiconResolverPort
 * @property {(collection: string, record: object) => Promise<ValidationStatus>} assertValid
 *   Resolves schema if needed, validates record.
 *   Returns 'valid' on success, 'unknown' if schema unresolvable.
 *   Throws Error with descriptive message if record is invalid against known schema.
 */

Step 2: Commit

git add packages/core/src/ports.js
git commit -m "feat(core): add LexiconResolverPort typedef

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 2: Add lexiconResolver to PDS constructor

Files:

Modify: packages/core/src/pds.js:300-340

Step 1: Add JSDoc for the new parameter

Add to the constructor JSDoc (after line 311):

 * @param {import('./ports.js').LexiconResolverPort} [config.lexiconResolver] - Optional lexicon resolver for record validation

Step 2: Add to constructor destructuring

Add lexiconResolver to the destructuring (after webSocket,):

    webSocket,
    lexiconResolver,

Step 3: Store on instance

Add after this.webSocket = webSocket; (line 334):

    this.lexiconResolver = lexiconResolver;

Step 4: Commit

git add packages/core/src/pds.js
git commit -m "feat(core): add lexiconResolver to PDS constructor

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 3: Add validation to handleCreateRecord

Files:

Modify: packages/core/src/pds.js:742-819
Test: packages/core/test/lexicon.test.js (create)

Step 1: Write failing test

Create packages/core/test/lexicon.test.js:

import { describe, expect, test } from 'vitest';

// Mock resolver that tracks calls
function createMockResolver(behavior = 'valid') {
  const calls = [];
  return {
    calls,
    async assertValid(collection, record) {
      calls.push({ collection, record });
      if (behavior === 'valid') return 'valid';
      if (behavior === 'unknown') return 'unknown';
      if (behavior === 'invalid') throw new Error('Invalid record: missing required field');
      return 'valid';
    },
  };
}

describe('Lexicon Validation', () => {
  describe('LexiconResolverPort mock', () => {
    test('calls resolver with collection and record', async () => {
      const resolver = createMockResolver('valid');
      expect(resolver.calls).toHaveLength(0);

      const status = await resolver.assertValid('app.bsky.feed.post', { text: 'hello' });

      expect(resolver.calls).toHaveLength(1);
      expect(resolver.calls[0].collection).toBe('app.bsky.feed.post');
      expect(resolver.calls[0].record).toEqual({ text: 'hello' });
      expect(status).toBe('valid');
    });

    test('returns unknown when schema not found', async () => {
      const resolver = createMockResolver('unknown');
      const status = await resolver.assertValid('com.example.unknown', {});
      expect(status).toBe('unknown');
    });

    test('throws when record is invalid', async () => {
      const resolver = createMockResolver('invalid');
      await expect(
        resolver.assertValid('app.bsky.feed.post', {})
      ).rejects.toThrow('Invalid record');
    });
  });
});

Step 2: Run test to verify it passes

cd /Users/chadmiller/code/pds-experiment && pnpm test packages/core/test/lexicon.test.js

Expected: PASS (tests the mock behavior)

Step 3: Update handleCreateRecord to use resolver

In handleCreateRecord, replace the current record handling (around lines 785-818) with:

    const rkey = providedRkey || createTid();
    const uri = `at://${did}/${collection}/${rkey}`;

    // Ensure $type is set
    const recordWithType = { $type: collection, ...record };

    // Validate record against schema if resolver provided
    const shouldValidate = _validate !== false;
    let validationStatus;

    if (shouldValidate && this.lexiconResolver) {
      try {
        validationStatus = await this.lexiconResolver.assertValid(
          collection,
          recordWithType,
        );
      } catch (err) {
        return Response.json(
          {
            error: 'InvalidRecord',
            message: `Invalid ${collection} record: ${err.message}`,
          },
          { status: 400 },
        );
      }
    } else if (shouldValidate && !this.lexiconResolver) {
      validationStatus = 'unknown';
    }
    // If _validate === false, validationStatus remains undefined

    // Encode record and create CID
    const encoded = cborEncodeDagCbor(recordWithType);
    const cid = cidToString(await createCid(encoded));

    // Store block
    await this.actorStorage.putBlock(cid, encoded);

    // Store record
    await this.actorStorage.putRecord(uri, cid, collection, rkey, encoded);

    // Link blobs to record
    const blobRefs = findBlobRefs(record);
    for (const blobCid of blobRefs) {
      await this.actorStorage.linkBlobToRecord(blobCid, uri);
    }

    // Create commit
    await this.createCommit(did, [{ action: 'create', uri, cid }]);

    // Get commit info
    const latestCommit = await this.actorStorage.getLatestCommit();

    return Response.json({
      uri,
      cid,
      commit: latestCommit
        ? { cid: latestCommit.cid, rev: latestCommit.rev }
        : null,
      validationStatus,
    });

Step 4: Run tests

cd /Users/chadmiller/code/pds-experiment && pnpm test

Step 5: Commit

git add packages/core/src/pds.js packages/core/test/lexicon.test.js
git commit -m "feat(core): add lexicon validation to createRecord

- Calls lexiconResolver.assertValid when resolver provided
- Returns InvalidRecord error when validation fails
- Returns validationStatus in response (valid/unknown/undefined)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 4: Add validation to handlePutRecord

Files:

Modify: packages/core/src/pds.js:915-1018

Step 1: Add validation logic to handlePutRecord

Find the section after scope permissions check (around line 950) and add the same validation pattern. Update to include _validate in destructuring and add validation logic before encoding:

    const {
      repo,
      collection,
      rkey,
      record,
      validate: _validate,
    } = await request.json();

Then after scope check, before encoding:

    // Ensure $type is set
    const recordWithType = { $type: collection, ...record };

    // Validate record against schema if resolver provided
    const shouldValidate = _validate !== false;
    let validationStatus;

    if (shouldValidate && this.lexiconResolver) {
      try {
        validationStatus = await this.lexiconResolver.assertValid(
          collection,
          recordWithType,
        );
      } catch (err) {
        return Response.json(
          {
            error: 'InvalidRecord',
            message: `Invalid ${collection} record: ${err.message}`,
          },
          { status: 400 },
        );
      }
    } else if (shouldValidate && !this.lexiconResolver) {
      validationStatus = 'unknown';
    }

Step 2: Update response to include validationStatus

Add validationStatus to the response object.

Step 3: Run tests

cd /Users/chadmiller/code/pds-experiment && pnpm test

Step 4: Commit

git add packages/core/src/pds.js
git commit -m "feat(core): add lexicon validation to putRecord

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 5: Add validation to handleApplyWrites

Files:

Modify: packages/core/src/pds.js:1021-1170

Step 1: Extract validate param from request body

Add validate to the destructuring:

    const { repo, writes, validate, swapCommit: _swapCommit } = await request.json();

Step 2: Add validation inside create case

For create case (around line 1070), add validation before storing:

      if ($type === 'com.atproto.repo.applyWrites#create') {
        const { collection, rkey: providedRkey, value } = write;

        // Validate if resolver provided and validate !== false
        const shouldValidate = validate !== false;
        let validationStatus;
        const recordWithType = { $type: collection, ...value };

        if (shouldValidate && this.lexiconResolver) {
          try {
            validationStatus = await this.lexiconResolver.assertValid(
              collection,
              recordWithType,
            );
          } catch (err) {
            return Response.json(
              {
                error: 'InvalidRecord',
                message: `Invalid ${collection} record: ${err.message}`,
              },
              { status: 400 },
            );
          }
        } else if (shouldValidate && !this.lexiconResolver) {
          validationStatus = 'unknown';
        }

        // ... rest of create logic, using recordWithType for encoding

Step 3: Add validation to update case

Same pattern for the update case.

Step 4: Add validationStatus to create/update results

        results.push({
          $type: 'com.atproto.repo.applyWrites#createResult',
          uri,
          cid,
          validationStatus,
        });

Step 5: Run tests

cd /Users/chadmiller/code/pds-experiment && pnpm test

Step 6: Commit

git add packages/core/src/pds.js
git commit -m "feat(core): add lexicon validation to applyWrites

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 6: Create lexicon-resolver package

Files:

Create: packages/lexicon-resolver/package.json
Create: packages/lexicon-resolver/src/index.js
Create: packages/lexicon-resolver/src/resolver.js

Step 1: Create package.json

{
  "name": "@pdsjs/lexicon-resolver",
  "version": "0.1.0",
  "type": "module",
  "main": "src/index.js",
  "exports": {
    ".": "./src/index.js",
    "./resolver": "./src/resolver.js"
  },
  "dependencies": {
    "@atproto/lexicon": "^0.4.0",
    "@atproto/lex-resolver": "^0.1.0"
  },
  "devDependencies": {
    "vitest": "^2.1.8"
  }
}

Step 2: Create src/index.js

export { LexiconResolver } from './resolver.js';

Step 3: Create src/resolver.js

import { Lexicons } from '@atproto/lexicon';
import { LexResolver } from '@atproto/lex-resolver';

/**
 * @typedef {import('@pdsjs/core/ports').ValidationStatus} ValidationStatus
 */

/**
 * Lexicon resolver using @atproto/lex-resolver for schema resolution
 * and @atproto/lexicon for validation.
 * Implements LexiconResolverPort.
 */
export class LexiconResolver {
  /**
   * @param {Object} [opts]
   * @param {number} [opts.ttl=3600000] - Cache TTL in ms (default 1 hour)
   * @param {typeof fetch} [opts.fetch] - Fetch implementation
   * @param {string} [opts.plcDirectoryUrl] - PLC directory URL
   * @param {Array<object>} [opts.schemas] - Pre-loaded static schemas (LexiconDoc[])
   */
  constructor(opts = {}) {
    this.lexicons = new Lexicons();
    this.resolver = new LexResolver({
      fetch: opts.fetch,
      plcDirectoryUrl: opts.plcDirectoryUrl,
    });
    this.ttl = opts.ttl ?? 3600000;
    this.cache = new Map(); // nsid -> { fetchedAt }

    // Pre-load static schemas
    for (const schema of opts.schemas ?? []) {
      try {
        this.lexicons.add(schema);
        this.cache.set(schema.id, { fetchedAt: Date.now() });
      } catch {
        // Schema already added or invalid, skip
      }
    }
  }

  /**
   * Validate a record against its schema.
   * @param {string} collection - The collection/NSID (e.g., 'app.bsky.feed.post')
   * @param {object} record - The record to validate
   * @returns {Promise<ValidationStatus>}
   */
  async assertValid(collection, record) {
    // Ensure schema is loaded
    const hasSchema = await this.ensureSchema(collection);
    if (!hasSchema) {
      return 'unknown';
    }

    // Validate using @atproto/lexicon
    this.lexicons.assertValidRecord(collection, record);
    return 'valid';
  }

  /**
   * Ensure a schema is loaded, fetching if necessary.
   * @param {string} nsid
   * @returns {Promise<boolean>} - true if schema is available
   */
  async ensureSchema(nsid) {
    // Check if already loaded and not stale
    const cached = this.cache.get(nsid);
    if (cached && Date.now() - cached.fetchedAt < this.ttl) {
      return this.lexicons.get(nsid) !== undefined;
    }

    // Already have it but checking staleness
    if (this.lexicons.get(nsid)) {
      return true;
    }

    // Try to resolve
    try {
      const result = await this.resolver.get(nsid);
      if (result?.lexicon) {
        this.lexicons.add(result.lexicon);
        this.cache.set(nsid, { fetchedAt: Date.now() });
        return true;
      }
    } catch {
      // Resolution failed
    }

    return false;
  }
}

Step 4: Commit

git add packages/lexicon-resolver/
git commit -m "feat(lexicon-resolver): create package with @atproto/lexicon + lex-resolver

- LexiconResolver class implementing LexiconResolverPort
- Uses @atproto/lex-resolver for DNS + PDS schema fetching
- Uses @atproto/lexicon for validation
- In-memory caching with TTL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 7: Add tests for LexiconResolver

Files:

Create: packages/lexicon-resolver/test/resolver.test.js

Step 1: Create test file

import { describe, expect, test, vi } from 'vitest';
import { LexiconResolver } from '../src/resolver.js';

// Sample lexicon schema for testing
const postSchema = {
  lexicon: 1,
  id: 'app.bsky.feed.post',
  defs: {
    main: {
      type: 'record',
      key: 'tid',
      record: {
        type: 'object',
        required: ['text', 'createdAt'],
        properties: {
          text: { type: 'string', maxLength: 3000 },
          createdAt: { type: 'string', format: 'datetime' },
        },
      },
    },
  },
};

describe('LexiconResolver', () => {
  describe('with static schemas', () => {
    test('validates record against pre-loaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const status = await resolver.assertValid('app.bsky.feed.post', {
        $type: 'app.bsky.feed.post',
        text: 'Hello world',
        createdAt: new Date().toISOString(),
      });

      expect(status).toBe('valid');
    });

    test('throws on invalid record', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      await expect(
        resolver.assertValid('app.bsky.feed.post', {
          $type: 'app.bsky.feed.post',
          // missing required fields
        })
      ).rejects.toThrow();
    });

    test('returns unknown for unloaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const status = await resolver.assertValid('com.example.unknown', {
        $type: 'com.example.unknown',
      });

      expect(status).toBe('unknown');
    });
  });

  describe('ensureSchema', () => {
    test('returns true for pre-loaded schema', async () => {
      const resolver = new LexiconResolver({
        schemas: [postSchema],
      });

      const result = await resolver.ensureSchema('app.bsky.feed.post');
      expect(result).toBe(true);
    });

    test('returns false for unknown schema when resolution fails', async () => {
      const resolver = new LexiconResolver();
      // Mock the resolver to fail
      resolver.resolver.get = vi.fn().mockRejectedValue(new Error('Not found'));

      const result = await resolver.ensureSchema('com.example.unknown');
      expect(result).toBe(false);
    });
  });
});

Step 2: Run tests

cd /Users/chadmiller/code/pds-experiment && pnpm test packages/lexicon-resolver/test/resolver.test.js

Step 3: Commit

git add packages/lexicon-resolver/test/
git commit -m "test(lexicon-resolver): add unit tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 8: Install dependencies and update workspace

Step 1: Install dependencies

cd /Users/chadmiller/code/pds-experiment && pnpm install

Step 2: Run full test suite

cd /Users/chadmiller/code/pds-experiment && pnpm test

Step 3: Run linter

cd /Users/chadmiller/code/pds-experiment && pnpm lint

Step 4: Fix any issues and commit

git add pnpm-lock.yaml
git commit -m "chore: install lexicon-resolver dependencies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Task 9: Update endpoint comparison docs

Files:

Modify: docs/endpoint-comparison.md

Step 1: Update the missing parameters table

Change from:

| repo.createRecord | collection, record, rkey | **repo**, **validate**, swapCommit |
| repo.putRecord | collection, rkey, record | **repo**, **validate**, swapCommit, swapRecord |
| repo.applyWrites | writes | **repo**, validate, swapCommit |

To:

| repo.createRecord | collection, record, rkey, **validate** | **repo**, swapCommit |
| repo.putRecord | collection, rkey, record, **validate** | **repo**, swapCommit, swapRecord |
| repo.applyWrites | writes, **validate** | **repo**, swapCommit |

Step 2: Commit

git add docs/endpoint-comparison.md
git commit -m "docs: update endpoint comparison with validate param support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"

Summary

After completing all tasks:

@pdsjs/core has:
- LexiconResolverPort typedef in ports.js
- Optional lexiconResolver in PDS constructor
- validate param on createRecord, putRecord, applyWrites
- validationStatus in responses
@pdsjs/lexicon-resolver provides:
- LexiconResolver class implementing the port
- Uses @atproto/lex-resolver for DNS + PDS schema fetching
- Uses @atproto/lexicon for validation
- In-memory caching with TTL
- Pre-loadable static schemas

Usage:

import { PersonalDataServer } from '@pdsjs/core';
import { LexiconResolver } from '@pdsjs/lexicon-resolver';

const pds = new PersonalDataServer({
  // ... other config
  lexiconResolver: new LexiconResolver({
    schemas: [/* optional static schemas */],
  }),
});

chadtmiller.com/pds.js Switch repository Loading…

Lexicon Resolver Implementation Plan

Task 1: Add LexiconResolverPort to ports.js

Task 2: Add lexiconResolver to PDS constructor

Task 3: Add validation to handleCreateRecord

Task 4: Add validation to handlePutRecord

Task 5: Add validation to handleApplyWrites

Task 6: Create lexicon-resolver package

Task 7: Add tests for LexiconResolver

Task 8: Install dependencies and update workspace

Task 9: Update endpoint comparison docs

Summary

chadtmiller.com/pds.js

Switch repository

Loading…