apcore-js 0.1.0

package/planning/schema-system/tasks/strict-mode.md

@@ -0,0 +1,140 @@
+# Task: Strict Mode
+
+## Goal
+
+Implement the strict-mode transformation utilities for converting JSON Schemas to OpenAI-compatible strict mode format (Algorithm A23). This includes `toStrictSchema()` for full strict conversion, `stripExtensions()` for removing vendor extensions and defaults, and `applyLlmDescriptions()` for substituting `x-llm-description` into `description` fields.
+
+## Files Involved
+
+- `src/schema/strict.ts` -- `toStrictSchema()`, `stripExtensions()`, `applyLlmDescriptions()`, internal `convertToStrict()`
+- `tests/schema/test-strict.test.ts` -- Unit tests
+
+## Steps
+
+### 1. Implement deepCopy utility
+
+```typescript
+function deepCopy<T>(obj: T): T {
+  return JSON.parse(JSON.stringify(obj));
+}
+```
+
+Used internally to ensure all transformations operate on copies, never mutating the original schema.
+
+### 2. Implement applyLlmDescriptions()
+
+Recursively walk the schema tree. At each object node, if both `x-llm-description` and `description` exist, replace `description` with the value of `x-llm-description`. Recurse into `properties`, `items`, `oneOf`/`anyOf`/`allOf`, and `definitions`/`$defs`.
+
+```typescript
+export function applyLlmDescriptions(node: unknown): void {
+  if (typeof node !== 'object' || node === null || Array.isArray(node)) return;
+
+  const obj = node as Record<string, unknown>;
+  if ('x-llm-description' in obj && 'description' in obj) {
+    obj['description'] = obj['x-llm-description'];
+  }
+
+  if ('properties' in obj && typeof obj['properties'] === 'object' && obj['properties'] !== null) {
+    for (const prop of Object.values(obj['properties'] as Record<string, unknown>)) {
+      applyLlmDescriptions(prop);
+    }
+  }
+  // ... recurse into items, oneOf, anyOf, allOf, definitions, $defs
+}
+```
+
+TDD: Test root-level description replacement. Test nested property description replacement. Test no-op when `x-llm-description` is absent.
+
+### 3. Implement stripExtensions()
+
+Recursively remove all keys starting with `x-` and the `default` key from every object node. Recurse into all nested objects and arrays.
+
+```typescript
+export function stripExtensions(node: unknown): void {
+  if (typeof node !== 'object' || node === null || Array.isArray(node)) return;
+
+  const obj = node as Record<string, unknown>;
+  const keysToRemove = Object.keys(obj).filter(
+    (k) => (typeof k === 'string' && k.startsWith('x-')) || k === 'default',
+  );
+  for (const k of keysToRemove) {
+    delete obj[k];
+  }
+
+  for (const value of Object.values(obj)) {
+    if (typeof value === 'object' && value !== null) {
+      if (Array.isArray(value)) {
+        for (const item of value) {
+          if (typeof item === 'object' && item !== null) {
+            stripExtensions(item);
+          }
+        }
+      } else {
+        stripExtensions(value);
+      }
+    }
+  }
+}
+```
+
+TDD: Test `x-sensitive`, `x-custom`, `x-llm-description` are removed. Test `default` is removed. Test nested structures are processed. Test non-extension keys survive.
+
+### 4. Implement convertToStrict() (internal)
+
+For each object node with `type: "object"` and `properties`:
+1. Set `additionalProperties: false`
+2. Collect existing `required` array into a Set
+3. Find optional properties (not in `required`)
+4. For optional properties with a `type` field: wrap type in array with `"null"` (e.g., `"string"` -> `["string", "null"]`)
+5. For optional properties without a `type` field: wrap in `{ oneOf: [originalSchema, { type: "null" }] }`
+6. Set `required` to sorted array of ALL property names
+
+Recurse into `properties`, `items`, `oneOf`/`anyOf`/`allOf`, and `definitions`/`$defs`.
+
+```typescript
+function convertToStrict(node: unknown): void {
+  // ... see implementation in src/schema/strict.ts
+}
+```
+
+TDD: Test `additionalProperties: false` is added. Test all properties become required. Test optional properties get null-union wrapping. Test already-required properties keep their original type.
+
+### 5. Implement toStrictSchema() composition
+
+Compose the three operations: deep-copy, strip extensions, then convert to strict.
+
+```typescript
+export function toStrictSchema(schema: Record<string, unknown>): Record<string, unknown> {
+  const result = deepCopy(schema);
+  stripExtensions(result);
+  convertToStrict(result);
+  return result;
+}
+```
+
+TDD: Test that the original schema is not mutated. Test full pipeline: extensions removed, `additionalProperties: false` set, all required, optionals nullable.
+
+## Acceptance Criteria
+
+- [x] `applyLlmDescriptions()` replaces `description` with `x-llm-description` where both exist
+- [x] `applyLlmDescriptions()` recurses into `properties`, `items`, `oneOf`/`anyOf`/`allOf`, `definitions`/`$defs`
+- [x] `applyLlmDescriptions()` is a no-op when `x-llm-description` is absent
+- [x] `stripExtensions()` removes all `x-` prefixed keys recursively
+- [x] `stripExtensions()` removes `default` keys recursively
+- [x] `stripExtensions()` preserves non-extension keys
+- [x] `stripExtensions()` handles nested objects and arrays
+- [x] `toStrictSchema()` sets `additionalProperties: false` on all object schemas
+- [x] `toStrictSchema()` makes all properties required (sorted alphabetically)
+- [x] `toStrictSchema()` wraps optional type-bearing properties with null type array
+- [x] `toStrictSchema()` wraps optional non-type properties with `oneOf` null union
+- [x] `toStrictSchema()` does not mutate the original schema (returns deep copy)
+- [x] `toStrictSchema()` strips extensions before strict conversion
+- [x] All tests pass with `vitest`
+
+## Dependencies
+
+- None (standalone utility, no internal dependencies)
+
+## Estimated Time
+
+2 hours

package/planning/schema-system/tasks/typebox-generation.md

@@ -0,0 +1,133 @@
+# Task: TypeBox Generation
+
+## Goal
+
+Implement the `jsonSchemaToTypeBox()` function that recursively converts a JSON Schema dictionary to a TypeBox `TSchema` object. This is the TypeScript equivalent of the Python `create_model()` approach, leveraging the fact that TypeBox schemas ARE valid JSON Schema.
+
+## Files Involved
+
+- `src/schema/loader.ts` -- `jsonSchemaToTypeBox()` function (exported, co-located with SchemaLoader)
+- `tests/schema/test-loader.test.ts` -- Unit tests for `jsonSchemaToTypeBox()`
+
+## Steps
+
+### 1. Handle object type with properties
+
+Convert `{ type: "object", properties: {...}, required: [...] }` to `Type.Object({...})`. Recursively convert each property. Wrap non-required properties with `Type.Optional()`.
+
+```typescript
+if (schemaType === 'object') {
+  const properties = schema['properties'] as Record<string, Record<string, unknown>> | undefined;
+  const required = new Set((schema['required'] as string[]) ?? []);
+
+  if (properties) {
+    const typeboxProps: Record<string, TSchema> = {};
+    for (const [name, propSchema] of Object.entries(properties)) {
+      const propType = jsonSchemaToTypeBox(propSchema);
+      typeboxProps[name] = required.has(name) ? propType : Type.Optional(propType);
+    }
+    return Type.Object(typeboxProps);
+  }
+  return Type.Record(Type.String(), Type.Unknown());
+}
+```
+
+TDD: Test object with required and optional properties. Verify `Value.Check()` accepts valid data and rejects missing required fields.
+
+### 2. Handle object type without properties
+
+Convert `{ type: "object" }` (no properties) to `Type.Record(Type.String(), Type.Unknown())`, accepting any string-keyed object.
+
+TDD: Test that `{ any: "value" }` passes validation.
+
+### 3. Handle array type
+
+Convert `{ type: "array", items: {...} }` to `Type.Array(jsonSchemaToTypeBox(items))`. Without items, use `Type.Array(Type.Unknown())`.
+
+```typescript
+if (schemaType === 'array') {
+  const items = schema['items'] as Record<string, unknown> | undefined;
+  return items ? Type.Array(jsonSchemaToTypeBox(items)) : Type.Array(Type.Unknown());
+}
+```
+
+TDD: Test typed array (string items), untyped array, valid and invalid data.
+
+### 4. Handle string type with constraints
+
+Convert `{ type: "string" }` to `Type.String()`. Pass through constraint options: `minLength`, `maxLength`, `pattern`, `format`.
+
+```typescript
+if (schemaType === 'string') {
+  const opts: Record<string, unknown> = {};
+  if ('minLength' in schema) opts['minLength'] = schema['minLength'];
+  if ('maxLength' in schema) opts['maxLength'] = schema['maxLength'];
+  if ('pattern' in schema) opts['pattern'] = schema['pattern'];
+  if ('format' in schema) opts['format'] = schema['format'];
+  return Type.String(opts);
+}
+```
+
+TDD: Test plain string, string with `minLength`/`maxLength`, verify constraint enforcement.
+
+### 5. Handle integer and number types with constraints
+
+Convert `{ type: "integer" }` to `Type.Integer()` and `{ type: "number" }` to `Type.Number()`. Pass through: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`.
+
+TDD: Test integer rejects floats, number accepts both, constraint boundaries work.
+
+### 6. Handle boolean and null types
+
+Convert `{ type: "boolean" }` to `Type.Boolean()` and `{ type: "null" }` to `Type.Null()`.
+
+TDD: Test boolean accepts `true`/`false`, rejects strings. Null accepts `null`, rejects `undefined`.
+
+### 7. Handle enum
+
+Convert `{ enum: ["a", "b", "c"] }` to `Type.Union(values.map(v => Type.Literal(v)))`.
+
+TDD: Test valid enum values accepted, invalid values rejected.
+
+### 8. Handle oneOf and anyOf
+
+Convert `{ oneOf: [...] }` and `{ anyOf: [...] }` to `Type.Union(schemas.map(s => jsonSchemaToTypeBox(s)))`.
+
+TDD: Test union of string and number accepts both types, rejects others.
+
+### 9. Handle allOf
+
+Convert `{ allOf: [...] }` to `Type.Intersect(schemas.map(s => jsonSchemaToTypeBox(s)))`.
+
+TDD: Test intersection of two object schemas requires properties from both.
+
+### 10. Handle unknown/unsupported schemas
+
+Any schema that does not match a known type falls through to `Type.Unknown()`, which accepts any value.
+
+TDD: Test empty schema `{}` accepts any value.
+
+## Acceptance Criteria
+
+- [x] `object` with properties converts to `Type.Object()` with correct required/optional handling
+- [x] `object` without properties converts to `Type.Record(Type.String(), Type.Unknown())`
+- [x] `array` with items converts to `Type.Array()` with recursive item conversion
+- [x] `array` without items converts to `Type.Array(Type.Unknown())`
+- [x] `string` converts to `Type.String()` with `minLength`/`maxLength`/`pattern`/`format` passthrough
+- [x] `integer` converts to `Type.Integer()` with numeric constraint passthrough
+- [x] `number` converts to `Type.Number()` with numeric constraint passthrough
+- [x] `boolean` converts to `Type.Boolean()`
+- [x] `null` converts to `Type.Null()`
+- [x] `enum` converts to `Type.Union(Type.Literal(...))` for each value
+- [x] `oneOf`/`anyOf` convert to `Type.Union()`
+- [x] `allOf` converts to `Type.Intersect()`
+- [x] Unknown schemas convert to `Type.Unknown()`
+- [x] All generated TypeBox schemas pass `Value.Check()` validation correctly
+- [x] All tests pass with `vitest`
+
+## Dependencies
+
+- types-and-annotations (for `TSchema` type from TypeBox)
+
+## Estimated Time
+
+3 hours

package/planning/schema-system/tasks/types-and-annotations.md

@@ -0,0 +1,160 @@
+# Task: Types and Annotations
+
+## Goal
+
+Define the core type interfaces, enums, and annotation merging utilities that form the foundation of the schema system. All other schema tasks depend on these types.
+
+## Files Involved
+
+- `src/schema/types.ts` -- `SchemaDefinition`, `ResolvedSchema`, `SchemaStrategy`, `ExportProfile`, `SchemaValidationErrorDetail`, `SchemaValidationResult`, `LLMExtensions`, `validationResultToError()`
+- `src/schema/annotations.ts` -- `mergeAnnotations()`, `mergeExamples()`, `mergeMetadata()`
+
+## Steps
+
+### 1. Define SchemaStrategy enum
+
+```typescript
+export enum SchemaStrategy {
+  YamlFirst = 'yaml_first',
+  NativeFirst = 'native_first',
+  YamlOnly = 'yaml_only',
+}
+```
+
+TDD: Write tests asserting enum values match the expected string representations.
+
+### 2. Define ExportProfile enum
+
+```typescript
+export enum ExportProfile {
+  MCP = 'mcp',
+  OpenAI = 'openai',
+  Anthropic = 'anthropic',
+  Generic = 'generic',
+}
+```
+
+TDD: Write tests asserting all four profiles are available and have correct string values.
+
+### 3. Define SchemaDefinition interface
+
+```typescript
+export interface SchemaDefinition {
+  moduleId: string;
+  description: string;
+  inputSchema: Record<string, unknown>;
+  outputSchema: Record<string, unknown>;
+  errorSchema?: Record<string, unknown> | null;
+  definitions: Record<string, unknown>;
+  version: string;
+  documentation?: string | null;
+  schemaUrl?: string | null;
+}
+```
+
+TDD: Create a conforming object and verify all fields are accessible with correct types.
+
+### 4. Define ResolvedSchema interface
+
+```typescript
+import type { TSchema } from '@sinclair/typebox';
+
+export interface ResolvedSchema {
+  jsonSchema: Record<string, unknown>;
+  schema: TSchema;
+  moduleId: string;
+  direction: string;
+}
+```
+
+TDD: Create a `ResolvedSchema` with `Type.Object({})` and verify `schema` field is a valid `TSchema`.
+
+### 5. Define validation result types
+
+```typescript
+export interface SchemaValidationErrorDetail {
+  path: string;
+  message: string;
+  constraint?: string | null;
+  expected?: unknown;
+  actual?: unknown;
+}
+
+export interface SchemaValidationResult {
+  valid: boolean;
+  errors: SchemaValidationErrorDetail[];
+}
+```
+
+TDD: Create valid and invalid results, verify field access.
+
+### 6. Implement validationResultToError()
+
+```typescript
+export function validationResultToError(result: SchemaValidationResult): SchemaValidationError {
+  if (result.valid) {
+    throw new Error('Cannot convert valid result to error');
+  }
+  const errorDicts = result.errors.map((e) => ({
+    path: e.path,
+    message: e.message,
+    constraint: e.constraint ?? null,
+    expected: e.expected ?? null,
+    actual: e.actual ?? null,
+  }));
+  return new SchemaValidationError('Schema validation failed', errorDicts);
+}
+```
+
+TDD: Verify conversion from `SchemaValidationResult` to `SchemaValidationError`, and verify throwing on valid result.
+
+### 7. Define LLMExtensions interface
+
+```typescript
+export interface LLMExtensions {
+  llmDescription?: string | null;
+  examples?: unknown[] | null;
+  sensitive: boolean;
+  constraints?: string | null;
+  deprecated?: Record<string, unknown> | null;
+}
+```
+
+### 8. Implement mergeAnnotations()
+
+Merges YAML-defined and code-defined module annotations with YAML-wins precedence over code, and code-wins precedence over defaults. Operates on the `ANNOTATION_FIELDS` list: `readonly`, `destructive`, `idempotent`, `requiresApproval`, `openWorld`.
+
+TDD: Test default-only, code-only, YAML-only, and YAML-overrides-code scenarios.
+
+### 9. Implement mergeExamples()
+
+YAML examples override code examples entirely. If no YAML examples, fall back to code examples. If neither, return empty array. YAML examples are mapped from raw records to `ModuleExample` objects.
+
+TDD: Test YAML-wins, code-fallback, and empty scenarios.
+
+### 10. Implement mergeMetadata()
+
+Shallow merge with YAML overrides. Start from code metadata (or empty), then `Object.assign()` YAML metadata on top.
+
+TDD: Test merge behavior with overlapping keys.
+
+## Acceptance Criteria
+
+- [x] `SchemaStrategy` enum has `YamlFirst`, `NativeFirst`, `YamlOnly` members
+- [x] `ExportProfile` enum has `MCP`, `OpenAI`, `Anthropic`, `Generic` members
+- [x] `SchemaDefinition` interface includes all required and optional fields
+- [x] `ResolvedSchema` interface includes `jsonSchema`, `schema` (TSchema), `moduleId`, `direction`
+- [x] `SchemaValidationErrorDetail` includes `path`, `message`, optional `constraint`/`expected`/`actual`
+- [x] `validationResultToError()` converts invalid results and throws on valid results
+- [x] `mergeAnnotations()` respects YAML > code > defaults precedence
+- [x] `mergeExamples()` respects YAML-wins-all precedence
+- [x] `mergeMetadata()` performs shallow merge with YAML overrides
+- [x] All types compile with `tsc --noEmit` under strict mode
+
+## Dependencies
+
+- None (foundation task)
+
+## Estimated Time
+
+2 hours

package/planning/schema-system/tasks/validator.md

@@ -0,0 +1,149 @@
+# Task: SchemaValidator
+
+## Goal
+
+Implement the `SchemaValidator` class for runtime validation of data against TypeBox `TSchema` objects. Supports a coercion toggle (`Value.Decode` vs `Value.Check`), detailed error collection via `Value.Errors()`, and separate `validateInput()`/`validateOutput()` entry points that return validated data or throw `SchemaValidationError`.
+
+## Files Involved
+
+- `src/schema/validator.ts` -- `SchemaValidator` class
+- `src/schema/types.ts` -- `SchemaValidationResult`, `SchemaValidationErrorDetail`, `validationResultToError()`
+- `src/errors.ts` -- `SchemaValidationError`
+- `tests/schema/test-validator.test.ts` -- Unit tests
+
+## Steps
+
+### 1. Implement constructor with coercion toggle
+
+```typescript
+export class SchemaValidator {
+  private _coerceTypes: boolean;
+
+  constructor(coerceTypes: boolean = true) {
+    this._coerceTypes = coerceTypes;
+  }
+}
+```
+
+TDD: Verify default `coerceTypes` is `true`. Verify explicit `false` disables coercion.
+
+### 2. Implement validate() method
+
+When coercion is enabled, attempt `Value.Decode()`. If it succeeds, return `{ valid: true, errors: [] }`. If it throws, collect errors and return `{ valid: false, errors: [...] }`.
+
+When coercion is disabled, use `Value.Check()`. If it returns `true`, return valid result. Otherwise, collect errors.
+
+```typescript
+validate(data: Record<string, unknown>, schema: TSchema): SchemaValidationResult {
+  if (this._coerceTypes) {
+    try {
+      Value.Decode(schema, data);
+      return { valid: true, errors: [] };
+    } catch {
+      return { valid: false, errors: this._collectErrors(schema, data) };
+    }
+  }
+
+  if (Value.Check(schema, data)) {
+    return { valid: true, errors: [] };
+  }
+  return { valid: false, errors: this._collectErrors(schema, data) };
+}
+```
+
+TDD: Test valid data returns `{ valid: true }`. Test invalid data returns `{ valid: false }` with non-empty errors array.
+
+### 3. Implement _collectErrors() private method
+
+Iterate over `Value.Errors(schema, data)` and map each `ValueError` to a `SchemaValidationErrorDetail`.
+
+```typescript
+private _collectErrors(schema: TSchema, data: unknown): SchemaValidationErrorDetail[] {
+  const errors: SchemaValidationErrorDetail[] = [];
+  for (const error of Value.Errors(schema, data)) {
+    errors.push(this._typeboxErrorToDetail(error));
+  }
+  return errors;
+}
+```
+
+TDD: Verify errors have populated `path` and `message` fields.
+
+### 4. Implement _typeboxErrorToDetail() mapping
+
+Map TypeBox `ValueError` fields to `SchemaValidationErrorDetail`:
+- `error.path` -> `path` (defaults to `'/'` if empty)
+- `error.message` -> `message`
+- `error.type` -> `constraint` (stringified)
+- `error.schema` -> `expected`
+- `error.value` -> `actual`
+
+TDD: Verify detail mapping for nested object errors includes correct JSON path.
+
+### 5. Implement validateInput() method
+
+Call `_validateAndReturn()`. On success, return the (potentially coerced) data. On failure, throw `SchemaValidationError` via `validationResultToError()`.
+
+```typescript
+validateInput(data: Record<string, unknown>, schema: TSchema): Record<string, unknown> {
+  return this._validateAndReturn(data, schema);
+}
+```
+
+TDD: Test valid input returns data. Test invalid input throws `SchemaValidationError`.
+
+### 6. Implement validateOutput() method
+
+Same as `validateInput()` -- delegates to `_validateAndReturn()`. Separate method for semantic clarity in the executor pipeline (step 5 vs step 8).
+
+TDD: Test valid output returns data. Test invalid output throws `SchemaValidationError`.
+
+### 7. Implement _validateAndReturn() private method
+
+Shared logic for `validateInput()` and `validateOutput()`. With coercion, use `Value.Decode()` and return the decoded value. Without coercion, use `Value.Check()` and return the original data. On failure, collect errors and throw.
+
+```typescript
+private _validateAndReturn(data: Record<string, unknown>, schema: TSchema): Record<string, unknown> {
+  if (this._coerceTypes) {
+    try {
+      return Value.Decode(schema, data) as Record<string, unknown>;
+    } catch {
+      const result: SchemaValidationResult = {
+        valid: false,
+        errors: this._collectErrors(schema, data),
+      };
+      throw validationResultToError(result);
+    }
+  }
+  if (Value.Check(schema, data)) {
+    return data;
+  }
+  const result: SchemaValidationResult = {
+    valid: false,
+    errors: this._collectErrors(schema, data),
+  };
+  throw validationResultToError(result);
+}
+```
+
+TDD: Test both coercion and strict paths for valid and invalid data.
+
+## Acceptance Criteria
+
+- [x] `validate()` returns `{ valid: true, errors: [] }` for conforming data
+- [x] `validate()` returns `{ valid: false, errors: [...] }` for non-conforming data with detailed errors
+- [x] Coercion mode (`coerceTypes: true`) uses `Value.Decode()` for type coercion
+- [x] Strict mode (`coerceTypes: false`) uses `Value.Check()` for exact type matching
+- [x] `validateInput()` returns data on success, throws `SchemaValidationError` on failure
+- [x] `validateOutput()` returns data on success, throws `SchemaValidationError` on failure
+- [x] Error details include `path`, `message`, `constraint`, `expected`, `actual`
+- [x] Nested object errors include correct JSON path (e.g., `/nested/x`)
+- [x] All tests pass with `vitest`
+
+## Dependencies
+
+- types-and-annotations (for `SchemaValidationResult`, `SchemaValidationErrorDetail`, `validationResultToError()`)
+
+## Estimated Time
+
+2 hours