tjs-lang 0.8.6 → 0.8.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/docs.js +3 -2
- package/demo/docs.json +2 -2
- package/dist/index.js +5 -5
- package/dist/index.js.map +3 -3
- package/dist/tjs-browser-from-ts.js +2 -2
- package/dist/tjs-browser-from-ts.js.map +2 -2
- package/dist/tjs-browser.js +3 -3
- package/dist/tjs-browser.js.map +3 -3
- package/dist/tjs-eval.js +2 -2
- package/dist/tjs-eval.js.map +3 -3
- package/dist/tjs-from-ts.js +2 -2
- package/dist/tjs-from-ts.js.map +2 -2
- package/dist/tjs-lang.js +3 -3
- package/dist/tjs-lang.js.map +3 -3
- package/dist/tjs-vm.js +2 -2
- package/dist/tjs-vm.js.map +3 -3
- package/package.json +1 -1
- package/src/lang/bare-assignments.test.ts +48 -0
- package/src/lang/doc-comment-position.test.ts +32 -0
- package/src/lang/docs.ts +6 -2
- package/src/lang/emitters/from-ts.ts +4 -1
- package/src/lang/parser-transforms.ts +11 -4
- package/src/lang/parser.ts +13 -4
package/bin/docs.js
CHANGED
|
@@ -149,8 +149,9 @@ function findMarkdownFiles(dirs, ignore) {
|
|
|
149
149
|
markdownFiles.push(doc)
|
|
150
150
|
} else if (['.ts', '.js'].includes(extname(file))) {
|
|
151
151
|
const content = readFileSync(filePath, 'utf8')
|
|
152
|
-
// Match /*# ... */ blocks (inline markdown
|
|
153
|
-
|
|
152
|
+
// Match line-start /*# ... */ blocks (inline markdown docs). A /*# after
|
|
153
|
+
// code on the line, or inside a string, is an ordinary block comment.
|
|
154
|
+
const docs = content.match(/(?<=^[ \t]*)\/\*#[\s\S]+?\*\//gm) || []
|
|
154
155
|
if (docs.length) {
|
|
155
156
|
const markdown = docs.map((s) => s.substring(3, s.length - 2).trim())
|
|
156
157
|
const text = markdown.join('\n\n---\n\n')
|
package/demo/docs.json
CHANGED
|
@@ -1289,7 +1289,7 @@
|
|
|
1289
1289
|
"title": "CLAUDE.md",
|
|
1290
1290
|
"filename": "CLAUDE.md",
|
|
1291
1291
|
"path": "CLAUDE.md",
|
|
1292
|
-
"text": "# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Project Overview\n\n**tjs-lang** (npm: `tjs-lang`) is a typed JavaScript platform — a language, runtime, and toolchain that transpiles TypeScript and TJS to JavaScript with runtime type validation, inline WASM, monadic errors, and safe eval. It also includes AJS, a gas-metered VM for executing untrusted agent code in any JavaScript environment.\n\n**Three pillars:**\n\n- **TJS** — TypeScript-like syntax where types are examples that survive to runtime as contracts, documentation, and tests. Transpiles TS → TJS → JS in a single fast pass.\n- **AJS** — Agent language that compiles to JSON AST for safe, sandboxed execution with fuel limits and injected capabilities. Code travels to data.\n- **Toolchain** — Compresses transpilation, linting, testing, and documentation generation into one pass. Includes inline WASM with SIMD, polymorphic dispatch, local class extensions, and a browser-based playground.\n\n> **TJS syntax is NOT TypeScript.** Full reference: [`CLAUDE-TJS-SYNTAX.md`](CLAUDE-TJS-SYNTAX.md). The single most common LLM mistake is treating `function foo(x: 'default')` as a TypeScript string-literal type. It is _not_ — the colon value is an **example**, and `'default'` widens to `string`. See the syntax doc before writing or modifying TJS source.\n\n## Common Commands\n\n```bash\n# Development\nbun run format # ESLint fix + Prettier\nbun run test:fast # Core tests (skips LLM & benchmarks)\nbun run make # Full build (clean, format, grammars, tsc, esbuild)\nbun run dev # Development server with file watcher\nbun run start # Build demo + start dev server\nbun run latest # Clean reinstall (rm node_modules + bun install)\n\n# Testing (framework: bun:test — describe/it/expect)\nbun test # Full test suite\nbun test src/path/to/file.test.ts # Single test file\nbun test --test-name-pattern \"pattern\" # Run tests matching pattern\nSKIP_LLM_TESTS=1 bun test # Skip LLM integration tests\nbun test --coverage # With coverage report\n\n# Efficient test debugging - capture once, query multiple times\nbun test 2>&1 | tee /tmp/test-results.txt | tail -20 # Run and save\ngrep -E \"^\\(fail\\)\" /tmp/test-results.txt # List failures\ngrep -A10 \"test name\" /tmp/test-results.txt # See specific error\n\n# CLI tools\nbun src/cli/tjs.ts check <file> # Parse and type check TJS file\nbun src/cli/tjs.ts run <file> # Transpile and execute\nbun src/cli/tjs.ts types <file> # Output type metadata as JSON\nbun src/cli/tjs.ts emit <file> # Output transpiled JavaScript\nbun src/cli/tjs.ts convert <file> # Convert TypeScript to TJS (--emit-tjs) or JS\nbun src/cli/tjs.ts test <file> # Run inline tests in a TJS file\n\n# Type checking & other\nbun run typecheck # tsc --noEmit (type check without emitting)\nbun run test:llm # LM Studio integration tests\nbun run bench # Vector search benchmarks\nbun run docs # Generate documentation\n\n# Build standalone CLI binaries\nbun run build:cli # Compiles tjs + tjsx to dist/\n\n# Compatibility testing — see scripts/compat-*.ts (zod, effect, radash, superstruct, ts-pattern, kysely)\n\n# Deployment (Firebase)\nbun run deploy # Build demo + deploy functions + hosting\nbun run deploy:hosting # Hosting only (serves from .demo/)\nbun run functions:deploy # Cloud functions only\nbun run functions:serve # Local functions emulator\n```\n\n## Architecture\n\n### Two-Layer Design\n\n1. **Builder Layer** (`src/builder.ts`): Fluent API that constructs AST nodes. Contains no execution logic.\n2. **Runtime Layer** (`src/vm/runtime.ts`): Executes AST nodes. Contains all atom implementations (~3024 lines, security-critical).\n\n### Key Source Files\n\n- `src/index.ts` - Main entry, re-exports everything\n- `src/vm/runtime.ts` - All atom implementations, expression evaluation, fuel charging (~3024 lines, security-critical)\n- `src/vm/vm.ts` - AgentVM class (~247 lines)\n- `src/vm/atoms/batteries.ts` - Battery atoms (vector search, LLM, store operations)\n- `src/builder.ts` - TypedBuilder fluent API (~757 lines / ~19KB)\n- `src/lang/parser.ts` - TJS parser with colon shorthand, unsafe markers, return type extraction\n- `src/lang/parser-transforms.ts` - Type, Generic, and FunctionPredicate block/function form transforms\n- `src/lang/emitters/ast.ts` - Emits Agent99 AST from parsed source\n- `src/lang/emitters/js.ts` - Emits JavaScript with `__tjs` metadata\n- `src/lang/emitters/from-ts.ts` - TypeScript to TJS/JS transpiler with class metadata extraction\n- `src/lang/emitters/dts.ts` - .d.ts declaration file generator from TJS transpilation results\n- `src/lang/inference.ts` - Type inference from example values\n- `src/lang/json-schema.ts` - JSON Schema generation from TypeDescriptors and example values\n- `src/lang/linter.ts` - Static analysis (unused vars, unreachable code, no-explicit-new)\n- `src/lang/predicate.ts` - Predicate-safety verifier: certifies a cluster of pure, synchronous, composable predicates (reads the atom `effects` tag via `effectfulFromAtoms`); verified predicates compile to native JS. Also `suggest()` — mines a cluster for autocomplete completions (keyword sets → `value`s, `startsWith` guards → open-ended `stub`s like `var(--`; mined values run through the compiled predicate so they're guaranteed valid). Exported from `tjs-lang/lang`. See `experiments/predicates/` (CSS torture set + perf + suggest demo)\n- `src/lang/predicate-schema.ts` - Predicate-aware JSON-Schema: the `$predicate` keyword (computational types). `compilePredicateSchema`/`validatePredicateSchema` — structure for naive validators, `$predicate` for aware ones (progressive enhancement). The serializable-into-JSON-Schema endgame; exported from `tjs-lang/lang`\n- `src/lang/runtime.ts` - TJS runtime (monadic errors, type checking, wrapClass)\n- `src/lang/wasm.ts` - WASM compiler (opcodes, disassembler, bytecode generation; multi-function module composition; wasm-to-wasm `call <index>` resolution)\n- `src/lang/emitters/js-wasm.ts` - JS bootstrap emitter for compiled wasm modules (one `WebAssembly.compile` per file, name→export-index table, type-aware wrappers)\n- `src/lang/module-loader.ts` - Transpile-time `.tjs`/`.ts`/`.js` module loader (Phase 0.75); used by cross-file `wasm function` composition\n- `src/linalg/` - `tjs-lang/linalg` stdlib subpath (f32x4 SIMD vector kernels)\n- `src/types/` - Type system definitions (Type.ts, Generic.ts)\n- `src/transpiler/` - AJS transpiler (source → AST)\n- `src/batteries/` - LM Studio integration (lazy init, model audit, vector search)\n- `src/store/` - Store implementations for persistence\n- `src/rbac/` - Role-based access control\n- `src/use-cases/` - Integration tests and real-world examples (30 test files)\n- `src/cli/tjs.ts` - CLI tool for check/run/types/emit/convert/test commands\n- `src/cli/tjsx.ts` - JSX/component runner\n- `src/cli/playground.ts` - Local playground server\n- `src/cli/create-app.ts` - Project scaffolding tool\n\n### Core APIs\n\n```typescript\n// Language\najs`...` // Parse AJS to AST\ntjs`...` // Parse TypeScript variant with type metadata\ntranspile(source, options) // Full transpilation with signature extraction\ncreateAgent(source, vm) // Creates callable agent\n\n// VM\nconst vm = new AgentVM(customAtoms)\nawait vm.run(ast, args, {\n fuel, capabilities, timeoutMs, trace,\n costOverrides: { atomOp: 5 }, // per-atom fuel cost override\n timeoutOverrides: { atomOp: 60_000 }, // per-atom wall-clock override (ms; 0 disables)\n})\n\n// Builder\nAgent.take(schema).varSet(...).httpFetch(...).return(schema)\nvm.Agent // Builder with custom atoms included\n\n// JSON Schema\nType('user', { name: '', age: 0 }).toJSONSchema() // → JSON Schema object\nType('user', { name: '', age: 0 }).strip(value) // → strip extra fields\nfunctionMetaToJSONSchema(fn.__tjs) // → { input, output } schemas\n\n// Error History (on by default, zero cost on happy path)\n__tjs.errors() // → recent MonadicErrors (ring buffer, max 64)\n__tjs.clearErrors() // → returns and clears\n__tjs.getErrorCount() // → total since last clear\n```\n\n### Package Entry Points\n\n```typescript\nimport { Agent, AgentVM, ajs, tjs } from 'tjs-lang' // Main entry\nimport { Eval, SafeFunction } from 'tjs-lang/eval' // Safe eval utilities\nimport { tjs, transpile } from 'tjs-lang/lang' // Language tools only\nimport { fromTS } from 'tjs-lang/lang/from-ts' // TypeScript transpilation\nimport { AgentVM } from 'tjs-lang/vm' // VM only (smaller bundle)\nimport { batteryAtoms } from 'tjs-lang/batteries' // LM Studio batteries\nimport { dot, norm_sq } from 'tjs-lang/linalg' // SIMD linear-algebra kernels\n// Editor integrations: 'tjs-lang/editors/monaco', '/codemirror', '/ace'\n```\n\n### Transpiler Chain (TS → TJS → JS)\n\nTJS supports transpiling TypeScript to JavaScript with runtime type validation. The pipeline has two distinct, independently testable steps:\n\n**Step 1: TypeScript → TJS** (`fromTS`)\n\n```typescript\nimport { fromTS } from 'tjs-lang/lang/from-ts'\n\nconst tsSource = `\nfunction greet(name: string): string {\n return \\`Hello, \\${name}!\\`\n}\n`\n\nconst result = fromTS(tsSource, { emitTJS: true })\n// result.code contains TJS:\n// function greet(name: ''): '' {\n// return \\`Hello, \\${name}!\\`\n// }\n```\n\n**Step 2: TJS → JavaScript** (`tjs`)\n\n```typescript\nimport { tjs } from 'tjs-lang/lang'\n\nconst tjsSource = `\nfunction greet(name: ''): '' {\n return \\`Hello, \\${name}!\\`\n}\n`\n\nconst jsResult = tjs(tjsSource)\n// jsResult.code contains JavaScript with __tjs metadata for runtime validation\n```\n\n**Full Chain Example:**\n\n```typescript\nimport { fromTS } from 'tjs-lang/lang/from-ts'\nimport { tjs } from 'tjs-lang/lang'\n\n// TypeScript source with type annotations\nconst tsSource = `\nfunction add(a: number, b: number): number {\n return a + b\n}\n`\n\n// Step 1: TS → TJS\nconst tjsResult = fromTS(tsSource, { emitTJS: true })\n\n// Step 2: TJS → JS (with runtime validation)\nconst jsCode = tjs(tjsResult.code)\n\n// Execute the result\nconst fn = new Function('__tjs', jsCode + '; return add')(__tjs_runtime)\nfn(1, 2) // Returns 3\nfn('a', 'b') // Returns { error: 'type mismatch', ... }\n```\n\n**Design Notes:**\n\n- The two steps are intentionally separate for tree-shaking (TS support is optional)\n- `fromTS` lives in a separate entry point (`tjs-lang/lang/from-ts`)\n- Import only what you need to keep bundle size minimal\n- Each step is independently testable (see `src/lang/codegen.test.ts`)\n- Constrained generics (`<T extends { id: number }>`) use the constraint as the example value instead of `any`\n- Generic defaults (`<T = string>`) use the default as the example value\n- Unconstrained generics (`<T>`) degrade to `any` — there's no information to use\n\n### Security Model\n\n- **Capability-based**: VM has zero IO by default; inject `fetch`, `store`, `llm` via capabilities\n- **Fuel metering**: Every atom has a cost; execution stops when fuel exhausted\n- **Timeout enforcement**: Default `fuel × 10ms`; explicit `timeoutMs` overrides\n- **Monadic errors**: Errors wrapped in `AgentError` (VM) / `MonadicError` (TJS), not thrown (prevents exception exploits). Use `isMonadicError()` to check — `isError()` is deprecated\n- **Expression sandboxing**: ExprNode AST evaluation, blocked prototype access\n\n### Expression Evaluation\n\nExpressions use AST nodes (`$expr`), not strings:\n\n```typescript\n{ $expr: 'binary', op: '+', left: {...}, right: {...} }\n{ $expr: 'ident', name: 'varName' }\n{ $expr: 'member', object: {...}, property: 'foo' }\n```\n\nEach node costs 0.01 fuel. Forbidden: function calls, `new`, `this`, `__proto__`, `constructor`.\n\n## AJS Expression Gotchas\n\nAJS expressions behave differently from JavaScript in several important ways:\n\n- **Null member access is safe by default**: `null.foo.bar` returns `undefined` silently (uses `?.` semantics internally). This differs from JavaScript which would throw `TypeError`.\n- **No computed member access with variables**: `items[i]` fails at transpile time with \"Computed member access with variables not yet supported\". Literal indices work (`items[0]`, `obj[\"key\"]`). Workaround: use `.map`/`.reduce` atoms instead.\n- **Unknown atom errors**: When an atom doesn't exist, the error is `\"Unknown Atom: <name>\"` with no listing of available atoms.\n- **TJS parameter syntax is NOT TypeScript**: `function foo(x: 'default')` means \"required param, example value 'default'\" — not a TypeScript string literal type. LLMs consistently generate `function foo(x: string)` which is wrong. The colon value is an _example_, not a _type annotation_.\n\n## Testing Strategy\n\n- Unit tests alongside source files (`*.test.ts`)\n- Integration tests in `src/use-cases/` (RAG, orchestration, malicious actors)\n- Security tests in `src/use-cases/malicious-actor.test.ts`\n- Language tests split across 18 files in `src/lang/` (lang.test.ts, features.test.ts, codegen.test.ts, parser.test.ts, from-ts.test.ts, wasm.test.ts, etc.)\n- LLM integration tests (run via full `bun test`, skipped by `SKIP_LLM_TESTS`) need a local **LM Studio** server with a chat + embedding model loaded. Setup and the hard-won gotchas (model load failures, leaked-VRAM stray `node` worker, updating runtimes, CORS, the audit-cache parallel race) are in [`docs/lm-studio-setup.md`](docs/lm-studio-setup.md).\n\nCoverage targets: 98% lines on `src/vm/runtime.ts` (security-critical), 80%+ overall.\n\n**Bug fix rule:** Always create a reproduction test case before fixing a bug.\n\n## Key Patterns\n\n### Adding a New Atom\n\n1. Define with `defineAtom(opCode, inputSchema, outputSchema, implementation, { cost, timeoutMs, docs, effects })`\n2. Add to `src/vm/atoms/` and export from `src/vm/atoms/index.ts`\n3. Add tests\n4. Run `bun run test:fast`\n\n**Atom implementation notes:**\n\n- `cost` can be static number or dynamic: `(input, ctx) => number`\n- `timeoutMs` defaults to 1000ms; use `0` for no timeout (e.g., `seq`)\n- Atoms are always async; fuel deduction is automatic in the `exec` wrapper\n- `effects` defaults to `'pure'`; **set `'io'` for any atom that touches `ctx.capabilities` (fetch/store/llm/agent/code), is nondeterministic (random/uuid), or has side effects (console)**. This drives predicate-safety (a predicate may only call pure atoms — see `experiments/predicates/`). Core IO atoms are tagged centrally via `EFFECTFUL_CORE_OPS` in `runtime.ts`; the invariant is guarded by `src/vm/atom-effects.test.ts`.\n\n### Debugging Agents\n\nEnable tracing: `vm.run(ast, args, { trace: true })` returns `TraceEvent[]` with execution path, fuel consumption, and state changes.\n\n### Custom Atoms Must\n\n- Be non-blocking (no synchronous CPU-heavy work)\n- Respect `ctx.signal` for cancellation\n- Access IO only via `ctx.capabilities`\n\n### Value Resolution\n\nThe `resolveValue()` function handles multiple input patterns:\n\n- `{ $kind: 'arg', path: 'varName' }` → lookup in `ctx.args`\n- `{ $expr: ... }` → evaluate ExprNode via `evaluateExpr()`\n- String with dots `'obj.foo.bar'` → traverse state with forbidden property checks\n- Bare strings → lookup in state, else return literal\n\n### Monadic Error Flow\n\nWhen `ctx.error` is set, subsequent atoms in a `seq` skip execution. Errors are wrapped in `AgentError`, not thrown. This prevents exception-based exploits.\n\n### TJS Syntax Reference\n\nFull syntax documentation is in [`CLAUDE-TJS-SYNTAX.md`](CLAUDE-TJS-SYNTAX.md). Key concepts:\n\n- **Colon shorthand**: `function foo(x: 'hello')` — colon value is an _example_, not a type. This is the most common LLM mistake.\n- **Numeric narrowing**: `3.14` = float, `42` = integer, `+0` = non-negative integer\n- **Return types**: `function add(a: 0, b: 0): 0 { ... }` (colon syntax, same as TypeScript)\n- **Safety markers**: `!` = unsafe (skip validation), `?` = safe (explicit validation)\n- **Mode defaults**: Native TJS has all modes ON by default (`TjsEquals`, `TjsClass`, `TjsDate`, `TjsNoeval`, `TjsNoVar`, `TjsStandard`). TS-originated code (`fromTS`) and AJS/VM code get modes OFF. `TjsCompat` directive explicitly disables all modes. `TjsStrict` enables all modes (useful for TS-originated code opting in).\n- **Source dialect (`dialect: 'js' | 'tjs'`)**: the public, programmatic way to set the modes-on/off default. `tjs(src, { dialect: 'js' })` preserves plain-JS semantics (modes OFF, `safety: 'none'`); `'tjs'` (or a bare string) is native TJS (modes ON). `dialect` is authoritative; otherwise inferred from the `fromTS` annotation / `vmTarget`. For file-based tooling, use the canonical extension→dialect helpers `dialectForFilename(filename)` / `sourceKindForFilename(filename)` from `tjs-lang/lang` (`.js`/`.mjs`/`.cjs` → `'js'`, `.tjs` → `'tjs'`, `.ts` → use `fromTS`). This upholds the **TJS ⊇ JS** invariant — see `PRINCIPLES.md`. (`.tjs` is a _better_ language, not just JS; choosing it is the opt-in to the modes.)\n- **Bang access**: `x!.foo` — returns MonadicError if `x` is null/undefined, otherwise bare `x.foo`. Chains propagate: `x!.foo!.bar`.\n- **Type/Generic/FunctionPredicate**: Three declaration forms for runtime type predicates\n- **`const!`**: Compile-time immutability, zero runtime cost\n- **Equality**: `==`/`!=` in native TJS (via `Eq`/`NotEq` under `TjsEquals`) are **footgun-free `===`** — they unwrap boxed primitives (`new Boolean(false) == false`) and treat `null`/`undefined` as equal, but do **NOT** coerce types (`'5' != 5`, `'' != false`) and are **NOT structural** (distinct objects/arrays are distinct: `{a:1} != {a:1}` — a real distinction, and structural `==` would be a silent O(n) hit). For deep structural comparison use the `Is`/`IsNot` function (or a type's `.Equals` hook). `===`/`!==` = strict identity.\n- **Polymorphic functions**: Multiple same-name declarations merge into arity/type dispatcher\n- **`extend` blocks**: Local class extensions without prototype pollution\n- **WASM blocks**: Inline WebAssembly compiled at transpile time, with SIMD intrinsics and `wasmBuffer()` zero-copy arrays\n- **`@tjs` annotations**: `/* @tjs ... */` comments in TS files enrich TJS output\n\n#### Runtime Configuration\n\n```typescript\nimport { configure } from 'tjs-lang/lang'\n\nconfigure({ logTypeErrors: true }) // Log type errors to console\nconfigure({ throwTypeErrors: true }) // Throw instead of return (debugging)\nconfigure({ callStacks: true }) // Track call stacks in errors (~2x overhead)\nconfigure({ trackErrors: false }) // Disable error history (on by default)\n```\n\n#### Error History\n\nType errors are tracked in a ring buffer (on by default, zero cost on happy path):\n\n```typescript\n__tjs.errors() // → recent MonadicErrors (newest last, max 64)\n__tjs.clearErrors() // → returns and clears the buffer\n__tjs.getErrorCount() // → total since last clear (survives buffer wrap)\n```\n\nUse for debugging (find silent failures), testing (`clearErrors()` → run → check), and monitoring.\n\n#### Standalone JS Output\n\nEmitted `.js` files work without any runtime setup. Each file includes an inline\nminimal runtime as fallback — only the functions actually used are included (~500\nbytes for a basic validated function). If `globalThis.__tjs` exists (shared runtime),\nit's used instead.\n\n## Dependencies\n\nRuntime (shipped): `acorn` (JS parser, ~30KB), `tosijs-schema` (validation, ~5KB). Both have zero transitive dependencies.\n\n## Forbidden Properties (Security)\n\nThe following property names are blocked in expression evaluation to prevent prototype pollution:\n\n- `__proto__`, `constructor`, `prototype`\n\nThese are hardcoded in `runtime.ts` and checked during member access in `evaluateExpr()`.\n\n## Batteries System\n\nThe batteries (`src/batteries/`) provide zero-config local AI development:\n\n- **Lazy initialization**: First import audits LM Studio models (cached 24 hours)\n- **HTTPS detection**: Blocks local LLM calls from HTTPS contexts (security)\n- **Capabilities interface**: `fetch`, `store` (KV + vector), `llmBattery` (predict/embed)\n\nRegister battery atoms: `new AgentVM(batteryAtoms)` then pass `{ capabilities: batteries }` to `run()`.\n\n### Capability Key Naming\n\nThe base `Capabilities` interface (`runtime.ts`) uses `llm` with `{ predict, embed? }`, but the battery atoms access capabilities via different keys:\n\n| Capability key | Used by | Contains |\n| -------------- | -------------------------------------------------------- | -------------------------------------------- |\n| `llmBattery` | `llmPredictBattery`, `llmVision` | Full `LLMCapability` (`predict` + `embed`) |\n| `vector` | `storeVectorize` | Just `{ embed }` (extracted from llmBattery) |\n| `store` | `storeSearch`, `storeCreateCollection`, `storeVectorAdd` | KV + vector store ops |\n\nBoth `llmBattery` and `vector` can be `undefined`/`null` if LM Studio isn't available or HTTPS is detected.\n\n### Battery Atom Return Types\n\n- **`llmPredictBattery`**: Returns OpenAI message format `{ role?, content?, tool_calls? }` — NOT a plain string\n- **`storeVectorize`**: Returns `number[]` (embedding vector)\n- **`storeSearch`**: Returns `any[]` (matched documents)\n\n## Development Configuration\n\n### Bun Plugin\n\n`bunfig.toml` preloads `src/bun-plugin/tjs-plugin.ts` which enables importing `.tjs` files directly in bun (transpiled on-the-fly). It also aliases `tjs-lang` to `./src/index.ts` for local development, so `import { tjs } from 'tjs-lang'` resolves to the source tree without needing `npm link` or a published package.\n\n### Code Style\n\n- **Prettier**: Single quotes, no semicolons, 2-space indentation, 80 char width, es5 trailing commas\n- Prefix unused variables with `_` (enforced by ESLint: `argsIgnorePattern: '^_'`)\n- `any` types are allowed (`@typescript-eslint/no-explicit-any: 0`)\n- Module type is ESM (`\"type\": \"module\"` in package.json)\n- Build output goes to `dist/` (declaration files only via `tsconfig.build.json`, bundles via `scripts/build.ts`)\n- Run `bun run format` before committing (ESLint fix + Prettier)\n\n### Firebase Deployment\n\nThe playground is hosted on Firebase (`tjs-platform.web.app`). Demo build output goes to `.demo/` (gitignored) which is the Firebase hosting root. Cloud Functions live in `functions/` with their own build process (`functions/src/*.tjs` → transpile → bundle). Firebase config: `firebase.json`, `.firebaserc`, `firestore.rules`.\n\nThe `docs/` directory contains real documentation (markdown), not build artifacts. See `docs/README.md` for the documentation index.\n\n### Playground Examples\n\nThe playground (https://tjs-platform.web.app) shows interactive TJS and AJS examples in a navigable sidebar. Examples live as markdown files with embedded code blocks, NOT as raw `.tjs` files.\n\n**Where they live:**\n\n- TJS playground examples: `guides/examples/tjs/<slug>.md`\n- AJS playground examples: `guides/examples/ajs/<slug>.md` (assumed parallel structure)\n\n**File format:**\n\n<!-- prettier-ignore -->\n```markdown\n<!--{\"section\":\"tjs\",\"type\":\"example\",\"group\":\"basics\",\"order\":16}-->\n\n# Example Title\n\nShort intro paragraph (plain markdown).\n\n```tjs\n/*#\n## Optional H2 — markdown rendered above the code in the playground\nExplain the concept here. Use markdown freely.\n*/\n\n// Then the actual TJS code\nfunction demo() { ... }\n\n/**\n * JSDoc-style /** ... */ blocks are also extracted as docs.\n * Leading ` * ` is stripped from each line; the rest renders as markdown.\n * Use this when porting TS source where JSDoc is already idiomatic.\n */\n\ntest 'a description' {\n expect(...).toBe(...)\n}\n```\n```\n\nFrontmatter fields: `section` (`tjs`/`ajs`), `type: \"example\"`, `group` (`basics`/`advanced`/etc.), `order` (numeric, controls sidebar position). The H1 becomes the example title in the nav.\n\n**Registration:**\n\nExamples are auto-discovered by `bin/docs.js` (run via `bun run docs`), which walks the markdown tree, parses frontmatter, extracts the `tjs`/`ajs` code block, and writes the result to `demo/docs.json`. The demo loads `docs.json` at runtime — no other registration step.\n\n**After adding/editing an example:** run `bun run docs` and commit the regenerated `demo/docs.json` alongside the `.md` file. (The docs builder also runs as part of `bun run build:demo` and `bun run deploy`.)\n\n**Testing playground examples:**\n\nThe CLI (`bun src/cli/tjs.ts run`) does NOT inject the test-block `expect` harness — that's a playground-only thing. So running an extracted code block via the CLI prints \"expect is not defined\" for any `test { expect(...) }` blocks even though they pass in the playground. To verify an example:\n\n1. **Console-log behavior** (works via CLI): extract the `tjs` code block and run it.\n\n ````bash\n awk '/^```tjs$/{flag=1; next} /^```$/{flag=0} flag' \\\n guides/examples/tjs/<slug>.md > /tmp/example.tjs\n bun src/cli/tjs.ts run /tmp/example.tjs\n ````\n\n Verify the printed output matches the expected behavior shown in the example's comments.\n\n2. **Test blocks**: spin up the dev server (`bun run start`) and load the example in the playground UI to confirm tests pass under the real `expect` harness.\n\n3. **Frontmatter / registration**: after `bun run docs`, grep `demo/docs.json` for the slug to confirm it was picked up with the right `section`/`group`/`order`.\n\n### Additional Directories\n\n- `tjs-src/` — TJS runtime written in TJS itself (self-hosting)\n- `guides/` — Usage patterns, benchmarks, examples (`patterns.md`, `benchmarks.md`, `tjs-examples.md`)\n- `examples/` — Standalone TJS example files (`hello.tjs`, `datetime.tjs`, `generic-demo.tjs`)\n- `editors/` — Syntax highlighting for Monaco, CodeMirror, Ace, VSCode\n\n### Additional Documentation\n\n- `PRINCIPLES.md` — **non-negotiable language invariants**: options-off TJS ⊇ JS, and TJS ⊇ AJS. A richer layer may do _more_ with the same source but must never make subset-legal code _illegal_ (e.g. un-runnable signature tests are inconclusive, not errors). Read before changing parser/transpiler acceptance or signature-test behavior; a subset violation is a bug.\n- `llms.txt` — agent-facing navigation index (ships in npm bundle); points to docs and source entry points\n- `guides/footguns.md` — JS footguns TJS fixes (boxed-primitive truthiness, `==` coercion, `typeof null`, uninitialized `let`, etc.). Demo: `examples/js-footguns-fixed.tjs`.\n- `guides/playground-imports.md` — how the playground/dev-server resolves bare imports: TFS service worker, default JSDelivr `/+esm` routing, esm.sh allowlist for peer-dep packages (React), CDN hints (`jsdelivr/`, `esmsh/`, `unpkg/`, `github/`), and full-URL passthrough.\n- `README.md` — Project intro, install, quick start\n- `DOCS-TJS.md` — TJS language guide\n- `DOCS-AJS.md` — AJS runtime guide\n- `TJS-FOR-JS.md` — TJS guide for JavaScript developers (syntax differences, gotchas)\n- `TJS-FOR-TS.md` — TJS guide for TypeScript developers (migration, interop)\n- `CONTEXT.md` — Architecture deep dive\n- `AGENTS.md` — Agent workflow instructions (session-completion checklist, push-before-done rule)\n- `TODO.md` — Open work, organized by area; move items to the **Completed** section when done\n- `PLAN.md` — Roadmap\n- `DOCS-WASM.md` — Canonical WASM reference: inline blocks, `wasm function` declarations, memory model, cross-file composition, `tjs-lang/linalg`, current limitations\n- `wasm-library-plan.md` — Cross-file WASM library design (composable `wasm function` declarations, transpile-time module composition, linalg stdlib). **Shipped in v0.8.0** — all phases (0.5, 0.75, 1, 1.5, 2, 3, 4, 5 MVP, 6) complete. See the plan for what's deferred (linalg expansion, i32/f32/v128 return types, etc.).\n- `MANIFESTO-BUILDER.md` / `MANIFESTO-ENTERPRISE.md` — Positioning docs (audience-targeted pitches)\n- `benchmarks.md` — Top-level benchmark results (separate from `guides/benchmarks.md`)\n\n### Keeping This File and `llms.txt` Current\n\nUpdate both files when you change something an agent needs to discover:\n\n- **New top-level markdown doc** → add to \"Additional Documentation\" here AND to the appropriate section of `llms.txt`.\n- **New package entry point** (subpath export in `package.json`) → add to \"Package Entry Points\" here AND to \"Package entry points\" in `llms.txt`.\n- **New CLI command or `bun run` script** → add to \"Common Commands\".\n- **Renamed or moved key source file** → update \"Key Source Files\" here AND \"Source map\" in `llms.txt`.\n- **New language mode / safety directive** → add to the TJS Syntax Reference section.\n- **New playground example** → add to `guides/examples/{tjs,ajs}/<slug>.md`, then `bun run docs` to regenerate `demo/docs.json`. See \"Playground Examples\" above.\n\nSkip stale-prone precision (exact line counts, file sizes) for new entries — they drift silently. The existing `~3024` etc. are kept current opportunistically, not on every commit.\n\n### Tracking Work\n\nWork is tracked in plain markdown — no external issue tracker. Open items live in `TODO.md` (organized by area). When you start a task, find or add the relevant entry; when you finish, check the box and (for substantial work) move it to the Completed section with a short note.\n\n### Landing the Plane (Session Completion Checklist)\n\nSee `AGENTS.md` for the canonical session-completion checklist. Hard rule: work is not complete until `git push` succeeds — never stop before pushing, never `--no-verify` to bypass hooks.\n\n### Common Gotchas\n\n- **`tjs(source)` returns an object, not a string.** It returns `{ code, types, metadata, testResults, ... }` — use `.code` for the transpiled JS string.\n- **Prettier mangles bare-expression code blocks in markdown.** Code blocks tagged ` ```js` get reformatted; bare expressions like `'5' == 5` and `[1] == 1` on consecutive lines collapse into one expression with ASI guards. Use `<!-- prettier-ignore -->` directly above the code fence to preserve hand-formatted JS examples (or tag the block as `text`/`tjs`/`ts` instead — Prettier ignores those).\n- **`tjs-lang` package alias only works inside the project** (set in `bunfig.toml`). Test scripts written in `/tmp` won't resolve `import { tjs } from 'tjs-lang/lang'` to the local source — they'll resolve to whatever's in `node_modules`. For ad-hoc experiments outside the repo, use absolute paths: `import { tjs } from '/Users/.../tjs-lang/src/lang/index'`.\n\n### Running Emitted TJS Code\n\nEmitted JS works standalone — no setup required. Each file includes an inline\nruntime fallback. If you want the shared runtime (e.g. for `isMonadicError` to\nwork across files), install it first:\n\n```typescript\nimport { installRuntime, createRuntime } from '../lang/runtime'\ninstallRuntime() // or: globalThis.__tjs = createRuntime()\n\nconst fn = new Function(result.code + '\\nreturn fnName')()\nfn('valid') // works\nfn(42) // returns MonadicError (not thrown)\n```\n"
|
|
1292
|
+
"text": "# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Project Overview\n\n**tjs-lang** (npm: `tjs-lang`) is a typed JavaScript platform — a language, runtime, and toolchain that transpiles TypeScript and TJS to JavaScript with runtime type validation, inline WASM, monadic errors, and safe eval. It also includes AJS, a gas-metered VM for executing untrusted agent code in any JavaScript environment.\n\n**Three pillars:**\n\n- **TJS** — TypeScript-like syntax where types are examples that survive to runtime as contracts, documentation, and tests. Transpiles TS → TJS → JS in a single fast pass.\n- **AJS** — Agent language that compiles to JSON AST for safe, sandboxed execution with fuel limits and injected capabilities. Code travels to data.\n- **Toolchain** — Compresses transpilation, linting, testing, and documentation generation into one pass. Includes inline WASM with SIMD, polymorphic dispatch, local class extensions, and a browser-based playground.\n\n> **TJS syntax is NOT TypeScript.** Full reference: [`CLAUDE-TJS-SYNTAX.md`](CLAUDE-TJS-SYNTAX.md). The single most common LLM mistake is treating `function foo(x: 'default')` as a TypeScript string-literal type. It is _not_ — the colon value is an **example**, and `'default'` widens to `string`. See the syntax doc before writing or modifying TJS source.\n\n## Common Commands\n\n```bash\n# Development\nbun run format # ESLint fix + Prettier\nbun run test:fast # Core tests (skips LLM & benchmarks)\nbun run make # Full build (clean, format, grammars, tsc, esbuild)\nbun run dev # Development server with file watcher\nbun run start # Build demo + start dev server\nbun run latest # Clean reinstall (rm node_modules + bun install)\n\n# Testing (framework: bun:test — describe/it/expect)\nbun test # Full test suite\nbun test src/path/to/file.test.ts # Single test file\nbun test --test-name-pattern \"pattern\" # Run tests matching pattern\nSKIP_LLM_TESTS=1 bun test # Skip LLM integration tests\nbun test --coverage # With coverage report\n\n# Efficient test debugging - capture once, query multiple times\nbun test 2>&1 | tee /tmp/test-results.txt | tail -20 # Run and save\ngrep -E \"^\\(fail\\)\" /tmp/test-results.txt # List failures\ngrep -A10 \"test name\" /tmp/test-results.txt # See specific error\n\n# CLI tools\nbun src/cli/tjs.ts check <file> # Parse and type check TJS file\nbun src/cli/tjs.ts run <file> # Transpile and execute\nbun src/cli/tjs.ts types <file> # Output type metadata as JSON\nbun src/cli/tjs.ts emit <file> # Output transpiled JavaScript\nbun src/cli/tjs.ts convert <file> # Convert TypeScript to TJS (--emit-tjs) or JS\nbun src/cli/tjs.ts test <file> # Run inline tests in a TJS file\n\n# Type checking & other\nbun run typecheck # tsc --noEmit (type check without emitting)\nbun run test:llm # LM Studio integration tests\nbun run bench # Vector search benchmarks\nbun run docs # Generate documentation\n\n# Build standalone CLI binaries\nbun run build:cli # Compiles tjs + tjsx to dist/\n\n# Compatibility testing — see scripts/compat-*.ts (zod, effect, radash, superstruct, ts-pattern, kysely)\n\n# Deployment (Firebase)\nbun run deploy # Build demo + deploy functions + hosting\nbun run deploy:hosting # Hosting only (serves from .demo/)\nbun run functions:deploy # Cloud functions only\nbun run functions:serve # Local functions emulator\n```\n\n## Architecture\n\n### Two-Layer Design\n\n1. **Builder Layer** (`src/builder.ts`): Fluent API that constructs AST nodes. Contains no execution logic.\n2. **Runtime Layer** (`src/vm/runtime.ts`): Executes AST nodes. Contains all atom implementations (~3024 lines, security-critical).\n\n### Key Source Files\n\n- `src/index.ts` - Main entry, re-exports everything\n- `src/vm/runtime.ts` - All atom implementations, expression evaluation, fuel charging (~3024 lines, security-critical)\n- `src/vm/vm.ts` - AgentVM class (~247 lines)\n- `src/vm/atoms/batteries.ts` - Battery atoms (vector search, LLM, store operations)\n- `src/builder.ts` - TypedBuilder fluent API (~757 lines / ~19KB)\n- `src/lang/parser.ts` - TJS parser with colon shorthand, unsafe markers, return type extraction\n- `src/lang/parser-transforms.ts` - Type, Generic, and FunctionPredicate block/function form transforms\n- `src/lang/emitters/ast.ts` - Emits Agent99 AST from parsed source\n- `src/lang/emitters/js.ts` - Emits JavaScript with `__tjs` metadata\n- `src/lang/emitters/from-ts.ts` - TypeScript to TJS/JS transpiler with class metadata extraction\n- `src/lang/emitters/dts.ts` - .d.ts declaration file generator from TJS transpilation results\n- `src/lang/inference.ts` - Type inference from example values\n- `src/lang/json-schema.ts` - JSON Schema generation from TypeDescriptors and example values\n- `src/lang/linter.ts` - Static analysis (unused vars, unreachable code, no-explicit-new)\n- `src/lang/predicate.ts` - Predicate-safety verifier: certifies a cluster of pure, synchronous, composable predicates (reads the atom `effects` tag via `effectfulFromAtoms`); verified predicates compile to native JS. Also `suggest()` — mines a cluster for autocomplete completions (keyword sets → `value`s, `startsWith` guards → open-ended `stub`s like `var(--`; mined values run through the compiled predicate so they're guaranteed valid). Exported from `tjs-lang/lang`. See `experiments/predicates/` (CSS torture set + perf + suggest demo)\n- `src/lang/predicate-schema.ts` - Predicate-aware JSON-Schema: the `$predicate` keyword (computational types). `compilePredicateSchema`/`validatePredicateSchema` — structure for naive validators, `$predicate` for aware ones (progressive enhancement). The serializable-into-JSON-Schema endgame; exported from `tjs-lang/lang`\n- `src/lang/runtime.ts` - TJS runtime (monadic errors, type checking, wrapClass)\n- `src/lang/wasm.ts` - WASM compiler (opcodes, disassembler, bytecode generation; multi-function module composition; wasm-to-wasm `call <index>` resolution)\n- `src/lang/emitters/js-wasm.ts` - JS bootstrap emitter for compiled wasm modules (one `WebAssembly.compile` per file, name→export-index table, type-aware wrappers)\n- `src/lang/module-loader.ts` - Transpile-time `.tjs`/`.ts`/`.js` module loader (Phase 0.75); used by cross-file `wasm function` composition\n- `src/linalg/` - `tjs-lang/linalg` stdlib subpath (f32x4 SIMD vector kernels)\n- `src/types/` - Type system definitions (Type.ts, Generic.ts)\n- `src/transpiler/` - AJS transpiler (source → AST)\n- `src/batteries/` - LM Studio integration (lazy init, model audit, vector search)\n- `src/store/` - Store implementations for persistence\n- `src/rbac/` - Role-based access control\n- `src/use-cases/` - Integration tests and real-world examples (30 test files)\n- `src/cli/tjs.ts` - CLI tool for check/run/types/emit/convert/test commands\n- `src/cli/tjsx.ts` - JSX/component runner\n- `src/cli/playground.ts` - Local playground server\n- `src/cli/create-app.ts` - Project scaffolding tool\n\n### Core APIs\n\n```typescript\n// Language\najs`...` // Parse AJS to AST\ntjs`...` // Parse TypeScript variant with type metadata\ntranspile(source, options) // Full transpilation with signature extraction\ncreateAgent(source, vm) // Creates callable agent\n\n// VM\nconst vm = new AgentVM(customAtoms)\nawait vm.run(ast, args, {\n fuel, capabilities, timeoutMs, trace,\n costOverrides: { atomOp: 5 }, // per-atom fuel cost override\n timeoutOverrides: { atomOp: 60_000 }, // per-atom wall-clock override (ms; 0 disables)\n})\n\n// Builder\nAgent.take(schema).varSet(...).httpFetch(...).return(schema)\nvm.Agent // Builder with custom atoms included\n\n// JSON Schema\nType('user', { name: '', age: 0 }).toJSONSchema() // → JSON Schema object\nType('user', { name: '', age: 0 }).strip(value) // → strip extra fields\nfunctionMetaToJSONSchema(fn.__tjs) // → { input, output } schemas\n\n// Error History (on by default, zero cost on happy path)\n__tjs.errors() // → recent MonadicErrors (ring buffer, max 64)\n__tjs.clearErrors() // → returns and clears\n__tjs.getErrorCount() // → total since last clear\n```\n\n### Package Entry Points\n\n```typescript\nimport { Agent, AgentVM, ajs, tjs } from 'tjs-lang' // Main entry\nimport { Eval, SafeFunction } from 'tjs-lang/eval' // Safe eval utilities\nimport { tjs, transpile } from 'tjs-lang/lang' // Language tools only\nimport { fromTS } from 'tjs-lang/lang/from-ts' // TypeScript transpilation\nimport { AgentVM } from 'tjs-lang/vm' // VM only (smaller bundle)\nimport { batteryAtoms } from 'tjs-lang/batteries' // LM Studio batteries\nimport { dot, norm_sq } from 'tjs-lang/linalg' // SIMD linear-algebra kernels\n// Editor integrations: 'tjs-lang/editors/monaco', '/codemirror', '/ace'\n\n// Self-contained BROWSER bundles — drop-in via import() from any CDN, no\n// import-map/config (acorn + tosijs-schema inlined):\nconst { tjs } = await import(\n 'https://cdn.jsdelivr.net/npm/tjs-lang/dist/tjs-browser.js'\n)\n// TS→TJS in the browser: lazy-loads the TypeScript compiler from a CDN on first call:\nconst { fromTS } = await import(\n 'https://cdn.jsdelivr.net/npm/tjs-lang/dist/tjs-browser-from-ts.js'\n)\n// → exports: 'tjs-lang/browser' (TJS/AJS) and 'tjs-lang/browser/from-ts' (TS).\n```\n\n**Browser bundles + CDN reality** (build targets `tjs-browser` / `tjs-browser-from-ts`\nin `scripts/build.ts`; `src/lang/browser.ts`, `browser-from-ts.ts`, `ts-cdn-shim.ts`):\nthe TJS/AJS bundle is fully self-contained → loads from **any** CDN (jsDelivr,\nunpkg, esm.sh). The TS path lazy-loads the `typescript` compiler from a CDN on\ndemand (Proxy shim aliased over `from-ts`'s `typescript` import; no source change).\n**Verified in a real browser: esm.sh is the ONLY CDN that reliably serves\n`typescript`** (~700ms) — jsDelivr `+esm` / esm.run time out on its ~10MB CJS,\nskypack is dead. So `DEFAULT_TYPESCRIPT_URL = https://esm.sh/typescript@5`,\noverridable via `fromTS(src, { typescriptUrl })` or by preloading\n`globalThis.__TJS_TS__`. Self-containment guarded by `src/lang/browser-bundle.test.ts`.\n\n### Transpiler Chain (TS → TJS → JS)\n\nTJS supports transpiling TypeScript to JavaScript with runtime type validation. The pipeline has two distinct, independently testable steps:\n\n**Step 1: TypeScript → TJS** (`fromTS`)\n\n```typescript\nimport { fromTS } from 'tjs-lang/lang/from-ts'\n\nconst tsSource = `\nfunction greet(name: string): string {\n return \\`Hello, \\${name}!\\`\n}\n`\n\nconst result = fromTS(tsSource, { emitTJS: true })\n// result.code contains TJS:\n// function greet(name: ''): '' {\n// return \\`Hello, \\${name}!\\`\n// }\n```\n\n**Step 2: TJS → JavaScript** (`tjs`)\n\n```typescript\nimport { tjs } from 'tjs-lang/lang'\n\nconst tjsSource = `\nfunction greet(name: ''): '' {\n return \\`Hello, \\${name}!\\`\n}\n`\n\nconst jsResult = tjs(tjsSource)\n// jsResult.code contains JavaScript with __tjs metadata for runtime validation\n```\n\n**Full Chain Example:**\n\n```typescript\nimport { fromTS } from 'tjs-lang/lang/from-ts'\nimport { tjs } from 'tjs-lang/lang'\n\n// TypeScript source with type annotations\nconst tsSource = `\nfunction add(a: number, b: number): number {\n return a + b\n}\n`\n\n// Step 1: TS → TJS\nconst tjsResult = fromTS(tsSource, { emitTJS: true })\n\n// Step 2: TJS → JS (with runtime validation)\nconst jsCode = tjs(tjsResult.code)\n\n// Execute the result\nconst fn = new Function('__tjs', jsCode + '; return add')(__tjs_runtime)\nfn(1, 2) // Returns 3\nfn('a', 'b') // Returns { error: 'type mismatch', ... }\n```\n\n**Design Notes:**\n\n- The two steps are intentionally separate for tree-shaking (TS support is optional)\n- `fromTS` lives in a separate entry point (`tjs-lang/lang/from-ts`)\n- Import only what you need to keep bundle size minimal\n- Each step is independently testable (see `src/lang/codegen.test.ts`)\n- Constrained generics (`<T extends { id: number }>`) use the constraint as the example value instead of `any`\n- Generic defaults (`<T = string>`) use the default as the example value\n- Unconstrained generics (`<T>`) degrade to `any` — there's no information to use\n\n### Security Model\n\n- **Capability-based**: VM has zero IO by default; inject `fetch`, `store`, `llm` via capabilities\n- **Fuel metering**: Every atom has a cost; execution stops when fuel exhausted\n- **Timeout enforcement**: Default `fuel × 10ms`; explicit `timeoutMs` overrides\n- **Monadic errors**: Errors wrapped in `AgentError` (VM) / `MonadicError` (TJS), not thrown (prevents exception exploits). Use `isMonadicError()` to check — `isError()` is deprecated\n- **Expression sandboxing**: ExprNode AST evaluation, blocked prototype access\n\n### Expression Evaluation\n\nExpressions use AST nodes (`$expr`), not strings:\n\n```typescript\n{ $expr: 'binary', op: '+', left: {...}, right: {...} }\n{ $expr: 'ident', name: 'varName' }\n{ $expr: 'member', object: {...}, property: 'foo' }\n```\n\nEach node costs 0.01 fuel. Forbidden: function calls, `new`, `this`, `__proto__`, `constructor`.\n\n## AJS Expression Gotchas\n\nAJS expressions behave differently from JavaScript in several important ways:\n\n- **Null member access is safe by default**: `null.foo.bar` returns `undefined` silently (uses `?.` semantics internally). This differs from JavaScript which would throw `TypeError`.\n- **No computed member access with variables**: `items[i]` fails at transpile time with \"Computed member access with variables not yet supported\". Literal indices work (`items[0]`, `obj[\"key\"]`). Workaround: use `.map`/`.reduce` atoms instead.\n- **Unknown atom errors**: When an atom doesn't exist, the error is `\"Unknown Atom: <name>\"` with no listing of available atoms.\n- **TJS parameter syntax is NOT TypeScript**: `function foo(x: 'default')` means \"required param, example value 'default'\" — not a TypeScript string literal type. LLMs consistently generate `function foo(x: string)` which is wrong. The colon value is an _example_, not a _type annotation_.\n\n## Testing Strategy\n\n- Unit tests alongside source files (`*.test.ts`)\n- Integration tests in `src/use-cases/` (RAG, orchestration, malicious actors)\n- Security tests in `src/use-cases/malicious-actor.test.ts`\n- Language tests split across 18 files in `src/lang/` (lang.test.ts, features.test.ts, codegen.test.ts, parser.test.ts, from-ts.test.ts, wasm.test.ts, etc.)\n- LLM integration tests (run via full `bun test`, skipped by `SKIP_LLM_TESTS`) need a local **LM Studio** server with a chat + embedding model loaded. Setup and the hard-won gotchas (model load failures, leaked-VRAM stray `node` worker, updating runtimes, CORS, the audit-cache parallel race) are in [`docs/lm-studio-setup.md`](docs/lm-studio-setup.md).\n\nCoverage targets: 98% lines on `src/vm/runtime.ts` (security-critical), 80%+ overall.\n\n**Bug fix rule:** Always create a reproduction test case before fixing a bug.\n\n## Key Patterns\n\n### Adding a New Atom\n\n1. Define with `defineAtom(opCode, inputSchema, outputSchema, implementation, { cost, timeoutMs, docs, effects })`\n2. Add to `src/vm/atoms/` and export from `src/vm/atoms/index.ts`\n3. Add tests\n4. Run `bun run test:fast`\n\n**Atom implementation notes:**\n\n- `cost` can be static number or dynamic: `(input, ctx) => number`\n- `timeoutMs` defaults to 1000ms; use `0` for no timeout (e.g., `seq`)\n- Atoms are always async; fuel deduction is automatic in the `exec` wrapper\n- `effects` defaults to `'pure'`; **set `'io'` for any atom that touches `ctx.capabilities` (fetch/store/llm/agent/code), is nondeterministic (random/uuid), or has side effects (console)**. This drives predicate-safety (a predicate may only call pure atoms — see `experiments/predicates/`). Core IO atoms are tagged centrally via `EFFECTFUL_CORE_OPS` in `runtime.ts`; the invariant is guarded by `src/vm/atom-effects.test.ts`.\n\n### Debugging Agents\n\nEnable tracing: `vm.run(ast, args, { trace: true })` returns `TraceEvent[]` with execution path, fuel consumption, and state changes.\n\n### Custom Atoms Must\n\n- Be non-blocking (no synchronous CPU-heavy work)\n- Respect `ctx.signal` for cancellation\n- Access IO only via `ctx.capabilities`\n\n### Value Resolution\n\nThe `resolveValue()` function handles multiple input patterns:\n\n- `{ $kind: 'arg', path: 'varName' }` → lookup in `ctx.args`\n- `{ $expr: ... }` → evaluate ExprNode via `evaluateExpr()`\n- String with dots `'obj.foo.bar'` → traverse state with forbidden property checks\n- Bare strings → lookup in state, else return literal\n\n### Monadic Error Flow\n\nWhen `ctx.error` is set, subsequent atoms in a `seq` skip execution. Errors are wrapped in `AgentError`, not thrown. This prevents exception-based exploits.\n\n### TJS Syntax Reference\n\nFull syntax documentation is in [`CLAUDE-TJS-SYNTAX.md`](CLAUDE-TJS-SYNTAX.md). Key concepts:\n\n- **Colon shorthand**: `function foo(x: 'hello')` — colon value is an _example_, not a type. This is the most common LLM mistake.\n- **Numeric narrowing**: `3.14` = float, `42` = integer, `+0` = non-negative integer\n- **Return types**: `function add(a: 0, b: 0): 0 { ... }` (colon syntax, same as TypeScript)\n- **Safety markers**: `!` = unsafe (skip validation), `?` = safe (explicit validation)\n- **Mode defaults**: Native TJS has all modes ON by default (`TjsEquals`, `TjsClass`, `TjsDate`, `TjsNoeval`, `TjsNoVar`, `TjsStandard`). TS-originated code (`fromTS`) and AJS/VM code get modes OFF. `TjsCompat` directive explicitly disables all modes. `TjsStrict` enables all modes (useful for TS-originated code opting in).\n- **Source dialect (`dialect: 'js' | 'tjs'`)**: the public, programmatic way to set the modes-on/off default. `tjs(src, { dialect: 'js' })` preserves plain-JS semantics (modes OFF, `safety: 'none'`); `'tjs'` (or a bare string) is native TJS (modes ON). `dialect` is authoritative; otherwise inferred from the `fromTS` annotation / `vmTarget`. For file-based tooling, use the canonical extension→dialect helpers `dialectForFilename(filename)` / `sourceKindForFilename(filename)` from `tjs-lang/lang` (`.js`/`.mjs`/`.cjs` → `'js'`, `.tjs` → `'tjs'`, `.ts` → use `fromTS`). This upholds the **TJS ⊇ JS** invariant — see `PRINCIPLES.md`. (`.tjs` is a _better_ language, not just JS; choosing it is the opt-in to the modes.)\n- **Bang access**: `x!.foo` — returns MonadicError if `x` is null/undefined, otherwise bare `x.foo`. Chains propagate: `x!.foo!.bar`.\n- **Type/Generic/FunctionPredicate**: Three declaration forms for runtime type predicates\n- **`const!`**: Compile-time immutability, zero runtime cost\n- **Equality**: `==`/`!=` in native TJS (via `Eq`/`NotEq` under `TjsEquals`) are **footgun-free `===`** — they unwrap boxed primitives (`new Boolean(false) == false`) and treat `null`/`undefined` as equal, but do **NOT** coerce types (`'5' != 5`, `'' != false`) and are **NOT structural** (distinct objects/arrays are distinct: `{a:1} != {a:1}` — a real distinction, and structural `==` would be a silent O(n) hit). For deep structural comparison use the `Is`/`IsNot` function (or a type's `.Equals` hook). `===`/`!==` = strict identity.\n- **Polymorphic functions**: Multiple same-name declarations merge into arity/type dispatcher\n- **`extend` blocks**: Local class extensions without prototype pollution\n- **WASM blocks**: Inline WebAssembly compiled at transpile time, with SIMD intrinsics and `wasmBuffer()` zero-copy arrays\n- **`@tjs` annotations**: `/* @tjs ... */` comments in TS files enrich TJS output\n\n#### Runtime Configuration\n\n```typescript\nimport { configure } from 'tjs-lang/lang'\n\nconfigure({ logTypeErrors: true }) // Log type errors to console\nconfigure({ throwTypeErrors: true }) // Throw instead of return (debugging)\nconfigure({ callStacks: true }) // Track call stacks in errors (~2x overhead)\nconfigure({ trackErrors: false }) // Disable error history (on by default)\n```\n\n#### Error History\n\nType errors are tracked in a ring buffer (on by default, zero cost on happy path):\n\n```typescript\n__tjs.errors() // → recent MonadicErrors (newest last, max 64)\n__tjs.clearErrors() // → returns and clears the buffer\n__tjs.getErrorCount() // → total since last clear (survives buffer wrap)\n```\n\nUse for debugging (find silent failures), testing (`clearErrors()` → run → check), and monitoring.\n\n#### Standalone JS Output\n\nEmitted `.js` files work without any runtime setup. Each file includes an inline\nminimal runtime as fallback — only the functions actually used are included (~500\nbytes for a basic validated function). If `globalThis.__tjs` exists (shared runtime),\nit's used instead.\n\n## Dependencies\n\nRuntime (shipped): `acorn` (JS parser, ~30KB), `tosijs-schema` (validation, ~5KB). Both have zero transitive dependencies.\n\n## Forbidden Properties (Security)\n\nThe following property names are blocked in expression evaluation to prevent prototype pollution:\n\n- `__proto__`, `constructor`, `prototype`\n\nThese are hardcoded in `runtime.ts` and checked during member access in `evaluateExpr()`.\n\n## Batteries System\n\nThe batteries (`src/batteries/`) provide zero-config local AI development:\n\n- **Lazy initialization**: First import audits LM Studio models (cached 24 hours)\n- **HTTPS detection**: Blocks local LLM calls from HTTPS contexts (security)\n- **Capabilities interface**: `fetch`, `store` (KV + vector), `llmBattery` (predict/embed)\n\nRegister battery atoms: `new AgentVM(batteryAtoms)` then pass `{ capabilities: batteries }` to `run()`.\n\n### Capability Key Naming\n\nThe base `Capabilities` interface (`runtime.ts`) uses `llm` with `{ predict, embed? }`, but the battery atoms access capabilities via different keys:\n\n| Capability key | Used by | Contains |\n| -------------- | -------------------------------------------------------- | -------------------------------------------- |\n| `llmBattery` | `llmPredictBattery`, `llmVision` | Full `LLMCapability` (`predict` + `embed`) |\n| `vector` | `storeVectorize` | Just `{ embed }` (extracted from llmBattery) |\n| `store` | `storeSearch`, `storeCreateCollection`, `storeVectorAdd` | KV + vector store ops |\n\nBoth `llmBattery` and `vector` can be `undefined`/`null` if LM Studio isn't available or HTTPS is detected.\n\n### Battery Atom Return Types\n\n- **`llmPredictBattery`**: Returns OpenAI message format `{ role?, content?, tool_calls? }` — NOT a plain string\n- **`storeVectorize`**: Returns `number[]` (embedding vector)\n- **`storeSearch`**: Returns `any[]` (matched documents)\n\n## Development Configuration\n\n### Bun Plugin\n\n`bunfig.toml` preloads `src/bun-plugin/tjs-plugin.ts` which enables importing `.tjs` files directly in bun (transpiled on-the-fly). It also aliases `tjs-lang` to `./src/index.ts` for local development, so `import { tjs } from 'tjs-lang'` resolves to the source tree without needing `npm link` or a published package.\n\n### Code Style\n\n- **Prettier**: Single quotes, no semicolons, 2-space indentation, 80 char width, es5 trailing commas\n- Prefix unused variables with `_` (enforced by ESLint: `argsIgnorePattern: '^_'`)\n- `any` types are allowed (`@typescript-eslint/no-explicit-any: 0`)\n- Module type is ESM (`\"type\": \"module\"` in package.json)\n- Build output goes to `dist/` (declaration files only via `tsconfig.build.json`, bundles via `scripts/build.ts`)\n- Run `bun run format` before committing (ESLint fix + Prettier)\n\n### Firebase Deployment\n\nThe playground is hosted on Firebase (`tjs-platform.web.app`). Demo build output goes to `.demo/` (gitignored) which is the Firebase hosting root. Cloud Functions live in `functions/` with their own build process (`functions/src/*.tjs` → transpile → bundle). Firebase config: `firebase.json`, `.firebaserc`, `firestore.rules`.\n\nThe `docs/` directory contains real documentation (markdown), not build artifacts. See `docs/README.md` for the documentation index.\n\n### Playground Examples\n\nThe playground (https://tjs-platform.web.app) shows interactive TJS and AJS examples in a navigable sidebar. Examples live as markdown files with embedded code blocks, NOT as raw `.tjs` files.\n\n**Where they live:**\n\n- TJS playground examples: `guides/examples/tjs/<slug>.md`\n- AJS playground examples: `guides/examples/ajs/<slug>.md` (assumed parallel structure)\n\n**File format:**\n\n<!-- prettier-ignore -->\n```markdown\n<!--{\"section\":\"tjs\",\"type\":\"example\",\"group\":\"basics\",\"order\":16}-->\n\n# Example Title\n\nShort intro paragraph (plain markdown).\n\n```tjs\n/*#\n## Optional H2 — markdown rendered above the code in the playground\nExplain the concept here. Use markdown freely.\n*/\n\n// Then the actual TJS code\nfunction demo() { ... }\n\n/**\n * JSDoc-style /** ... */ blocks are also extracted as docs.\n * Leading ` * ` is stripped from each line; the rest renders as markdown.\n * Use this when porting TS source where JSDoc is already idiomatic.\n */\n\ntest 'a description' {\n expect(...).toBe(...)\n}\n```\n```\n\nFrontmatter fields: `section` (`tjs`/`ajs`), `type: \"example\"`, `group` (`basics`/`advanced`/etc.), `order` (numeric, controls sidebar position). The H1 becomes the example title in the nav.\n\n**Registration:**\n\nExamples are auto-discovered by `bin/docs.js` (run via `bun run docs`), which walks the markdown tree, parses frontmatter, extracts the `tjs`/`ajs` code block, and writes the result to `demo/docs.json`. The demo loads `docs.json` at runtime — no other registration step.\n\n**After adding/editing an example:** run `bun run docs` and commit the regenerated `demo/docs.json` alongside the `.md` file. (The docs builder also runs as part of `bun run build:demo` and `bun run deploy`.)\n\n**Testing playground examples:**\n\nThe CLI (`bun src/cli/tjs.ts run`) does NOT inject the test-block `expect` harness — that's a playground-only thing. So running an extracted code block via the CLI prints \"expect is not defined\" for any `test { expect(...) }` blocks even though they pass in the playground. To verify an example:\n\n1. **Console-log behavior** (works via CLI): extract the `tjs` code block and run it.\n\n ````bash\n awk '/^```tjs$/{flag=1; next} /^```$/{flag=0} flag' \\\n guides/examples/tjs/<slug>.md > /tmp/example.tjs\n bun src/cli/tjs.ts run /tmp/example.tjs\n ````\n\n Verify the printed output matches the expected behavior shown in the example's comments.\n\n2. **Test blocks**: spin up the dev server (`bun run start`) and load the example in the playground UI to confirm tests pass under the real `expect` harness.\n\n3. **Frontmatter / registration**: after `bun run docs`, grep `demo/docs.json` for the slug to confirm it was picked up with the right `section`/`group`/`order`.\n\n### Additional Directories\n\n- `tjs-src/` — TJS runtime written in TJS itself (self-hosting)\n- `guides/` — Usage patterns, benchmarks, examples (`patterns.md`, `benchmarks.md`, `tjs-examples.md`)\n- `examples/` — Standalone TJS example files (`hello.tjs`, `datetime.tjs`, `generic-demo.tjs`)\n- `editors/` — Syntax highlighting for Monaco, CodeMirror, Ace, VSCode\n\n### Additional Documentation\n\n- `PRINCIPLES.md` — **non-negotiable language invariants**: options-off TJS ⊇ JS, and TJS ⊇ AJS. A richer layer may do _more_ with the same source but must never make subset-legal code _illegal_ (e.g. un-runnable signature tests are inconclusive, not errors). Read before changing parser/transpiler acceptance or signature-test behavior; a subset violation is a bug.\n- `llms.txt` — agent-facing navigation index (ships in npm bundle); points to docs and source entry points\n- `guides/footguns.md` — JS footguns TJS fixes (boxed-primitive truthiness, `==` coercion, `typeof null`, uninitialized `let`, etc.). Demo: `examples/js-footguns-fixed.tjs`.\n- `guides/playground-imports.md` — how the playground/dev-server resolves bare imports: TFS service worker, default JSDelivr `/+esm` routing, esm.sh allowlist for peer-dep packages (React), CDN hints (`jsdelivr/`, `esmsh/`, `unpkg/`, `github/`), and full-URL passthrough.\n- `README.md` — Project intro, install, quick start\n- `DOCS-TJS.md` — TJS language guide\n- `DOCS-AJS.md` — AJS runtime guide\n- `TJS-FOR-JS.md` — TJS guide for JavaScript developers (syntax differences, gotchas)\n- `TJS-FOR-TS.md` — TJS guide for TypeScript developers (migration, interop)\n- `CONTEXT.md` — Architecture deep dive\n- `AGENTS.md` — Agent workflow instructions (session-completion checklist, push-before-done rule)\n- `TODO.md` — Open work, organized by area; move items to the **Completed** section when done\n- `PLAN.md` — Roadmap\n- `DOCS-WASM.md` — Canonical WASM reference: inline blocks, `wasm function` declarations, memory model, cross-file composition, `tjs-lang/linalg`, current limitations\n- `wasm-library-plan.md` — Cross-file WASM library design (composable `wasm function` declarations, transpile-time module composition, linalg stdlib). **Shipped in v0.8.0** — all phases (0.5, 0.75, 1, 1.5, 2, 3, 4, 5 MVP, 6) complete. See the plan for what's deferred (linalg expansion, i32/f32/v128 return types, etc.).\n- `MANIFESTO-BUILDER.md` / `MANIFESTO-ENTERPRISE.md` — Positioning docs (audience-targeted pitches)\n- `benchmarks.md` — Top-level benchmark results (separate from `guides/benchmarks.md`)\n\n### Keeping This File and `llms.txt` Current\n\nUpdate both files when you change something an agent needs to discover:\n\n- **New top-level markdown doc** → add to \"Additional Documentation\" here AND to the appropriate section of `llms.txt`.\n- **New package entry point** (subpath export in `package.json`) → add to \"Package Entry Points\" here AND to \"Package entry points\" in `llms.txt`.\n- **New CLI command or `bun run` script** → add to \"Common Commands\".\n- **Renamed or moved key source file** → update \"Key Source Files\" here AND \"Source map\" in `llms.txt`.\n- **New language mode / safety directive** → add to the TJS Syntax Reference section.\n- **New playground example** → add to `guides/examples/{tjs,ajs}/<slug>.md`, then `bun run docs` to regenerate `demo/docs.json`. See \"Playground Examples\" above.\n\nSkip stale-prone precision (exact line counts, file sizes) for new entries — they drift silently. The existing `~3024` etc. are kept current opportunistically, not on every commit.\n\n### Tracking Work\n\nWork is tracked in plain markdown — no external issue tracker. Open items live in `TODO.md` (organized by area). When you start a task, find or add the relevant entry; when you finish, check the box and (for substantial work) move it to the Completed section with a short note.\n\n### Landing the Plane (Session Completion Checklist)\n\nSee `AGENTS.md` for the canonical session-completion checklist. Hard rule: work is not complete until `git push` succeeds — never stop before pushing, never `--no-verify` to bypass hooks.\n\n### Common Gotchas\n\n- **`tjs(source)` returns an object, not a string.** It returns `{ code, types, metadata, testResults, ... }` — use `.code` for the transpiled JS string.\n- **Prettier mangles bare-expression code blocks in markdown.** Code blocks tagged ` ```js` get reformatted; bare expressions like `'5' == 5` and `[1] == 1` on consecutive lines collapse into one expression with ASI guards. Use `<!-- prettier-ignore -->` directly above the code fence to preserve hand-formatted JS examples (or tag the block as `text`/`tjs`/`ts` instead — Prettier ignores those).\n- **`tjs-lang` package alias only works inside the project** (set in `bunfig.toml`). Test scripts written in `/tmp` won't resolve `import { tjs } from 'tjs-lang/lang'` to the local source — they'll resolve to whatever's in `node_modules`. For ad-hoc experiments outside the repo, use absolute paths: `import { tjs } from '/Users/.../tjs-lang/src/lang/index'`.\n\n### Running Emitted TJS Code\n\nEmitted JS works standalone — no setup required. Each file includes an inline\nruntime fallback. If you want the shared runtime (e.g. for `isMonadicError` to\nwork across files), install it first:\n\n```typescript\nimport { installRuntime, createRuntime } from '../lang/runtime'\ninstallRuntime() // or: globalThis.__tjs = createRuntime()\n\nconst fn = new Function(result.code + '\\nreturn fnName')()\nfn('valid') // works\nfn(42) // returns MonadicError (not thrown)\n```\n"
|
|
1293
1293
|
},
|
|
1294
1294
|
{
|
|
1295
1295
|
"title": "Context: Working with tosijs-schema",
|
|
@@ -1625,7 +1625,7 @@
|
|
|
1625
1625
|
"title": "TJS-Lang TODO",
|
|
1626
1626
|
"filename": "TODO.md",
|
|
1627
1627
|
"path": "TODO.md",
|
|
1628
|
-
"text": "# TJS-Lang TODO\n\n## Predicate types — \"AJS is JSON-Schema's missing piece\"\n\nThe thesis (see the blog draft): JSON-Schema / TS can't express types that need\n**computation**; verified-pure, composable AJS predicates can — serializable\n(the AJS AST), safe (no IO, fuel-bounded), and compiled to native JS so they're\nfast. CSS is the torture-test proof. The engine is built and green on `main`;\nwhat remains is **delivery, measurement, and reach** — not invention.\n\n**Done (engine):**\n\n- [x] Atom `effects: 'pure' | 'io'` keystone — classified, guarded (`src/vm/atom-effects.test.ts`).\n- [x] `verifyPredicate` / `compilePredicate` — `src/lang/predicate.ts`, exported from `tjs-lang/lang`. Transitive closure check, pure-method whitelist, registry-driven effects.\n- [x] Fuel-bounded, global-shadowed native compiler (loops rejected; `__fuel()` at function entry; per-call budget; stack-overflow normalized to `PredicateFuelExhausted`). Zero measurable perf cost.\n- [x] PoC + CSS torture set + perf ballpark in `experiments/predicates/` (theme ~0.13ms, ReDoS-linear).\n\n**Remaining (delivery / north star):**\n\n- [x] **#4 Autocomplete `suggest()` companion** — `src/lang/predicate.ts` (`suggest`, exported from `tjs-lang/lang`). Mines a cluster for completions: keyword sets (array literals + `==` literals) → `value` suggestions, `startsWith(...)` guards → open-ended `stub`s (`var(--`/`calc(`). Mined values are run through the compiled entry predicate so suggestions are _guaranteed valid_, not just enumerated. Beats both TS modes: a `string` fallback offers nothing, a finite union can't offer the open-ended stubs. Prefix-filtered + limited. Tests: `src/lang/suggest.test.ts`, demo `experiments/predicates/suggest.demo.test.ts`.\n- [ ] **#5 Wire into `FunctionPredicate` / `Type`** — predicate bodies authored in this verified-safe substrate; the real consumer.\n- [x] **#6 (tjs-lang side) the `$predicate` keyword + reference evaluator** — `src/lang/predicate-schema.ts` (`compilePredicateSchema` / `validatePredicateSchema`, exported from `tjs-lang/lang`). A JSON-Schema node carries `$predicate` (predicate-cluster _source_; trivially serializable, the verifier makes it safe to run). Structural keywords (type/properties/required/items) validate for everyone; `$predicate` runs only for aware validators → progressive enhancement. Demoed on CSS (`experiments/predicates/css-schema.demo.test.ts`): same JSON, naive sees `string`, aware validates var()/calc()/!important + recursion. Gotcha noted: embed predicate source via `String.raw` (regex backslashes) — moot in real JSON.\n- [ ] **#6 (production) wire `$predicate` into tosijs-schema** — the \"incoming version\" from the blog: tosijs-schema (separate repo) evaluates `$predicate` via this engine. The tjs-lang format + evaluator are ready to consume. **The remaining north-star step.**\n- [ ] **Real CSS predicate library** — productionize beyond the PoC corpus (the tosijs CSS replacement). Recursive structure is plain `$ref` JSON-Schema; only leaf value-grammar needs `$predicate` (progressive enhancement). Schema validates the _serialized/data_ form; `Color` instances + bare-number→`px` are runtime conveniences (duck-typeable via `.toString()` where wanted).\n- [ ] **Regex-linting in the verifier** — the ReDoS path-forward the blog commits to: reject catastrophic-backtracking patterns (the one unbounded primitive fuel can't interrupt). Until then, predicates are \"no worse than `pattern`, with the bounded-tokenizer option.\"\n\n## \"Safe is fast\" — the campaign (measurement + propagation, not invention)\n\nThe architecture already makes the safe path the fast path: boundary-level checks\n(a few comparisons per _call_, not per-op), verify→native for validation, inline\nWASM/SIMD for hot loops, zero-cost happy-path errors. The strip-the-safety\ntranspiler option is the Obj-C-`IMP`-cache / Rust-`unsafe` move — it wins the perf\nargument precisely because, in practice, you leave the safety on. What's left is\nto **prove it and spread it**:\n\n- [ ] **Systematic overhead benchmark** — TJS-checked function call vs raw call across representative code (not just predicates), so \"safe is fast\" is backed by numbers, not just architecture. (Doubles as the CSS-post perf data — re-run on the _real_ tosijs theme with the full predicate set, confirming the ~0.1ms claim on real data.)\n- [ ] **Propagate verify→native** — weave it under the type system / tosijs so the capability is pervasive, not just an engine + PoC.\n- [ ] Frame the announcement around data + a real framework running it, not a promise. The blog draft is the spec: its present-tense claims (#6, the CSS lib, the real-theme number) must be true before publishing.\n\n## Editors - published `.js` is stale (address sooner rather than later)\n\n- [ ] **The `tjs-lang/editors/*` subpaths ship hand-maintained `.js` files that\n are NOT built from the `.ts` sources.** `editors/codemirror/ajs-language.js`\n is from Jan 2026 (~7.5KB, an old standalone CDN-example impl) while\n `ajs-language.ts` is the real ~51KB implementation (used by the playground,\n which bundles from source). So none of the autocomplete work (scope model,\n introspection bridge, member completion) reaches npm consumers of\n `tjs-lang/editors/codemirror` — they get months-old code. Same for\n monaco/ace `.js`. **Fix: add a build step that compiles/bundles\n `editors/**/\\*.ts`→ the published`.js`(wire into`scripts/build.ts`/\n `bun run make`) so it stays current automatically.\\*\\* Not urgent for active\n use cases (user isn't consuming tjs-lang externally yet) but flagged to do\n soon. Context: tosijs-ui's live-example/doc engine now uses tjs (replacing\n sucrase) and is evolving into a portable embeddable playground/IDE — it'll\n switch from ACE to CodeMirror and will want working autocomplete, at which\n point this matters.\n\n## Playground - Introspection-driven autocomplete\n\nThe current completion provider was regex-based and useless on real examples\n(`extractVariables` matched only `const NAME =`, missing ALL destructuring — and\nthe tosijs examples bind everything via destructuring). Direction: introspection,\nChrome-console style — run the user's actual code and read real values; predicates\nfill the value-grammar leaf (a runtime string can't reveal valid CSS colors). See\nthe `introspection-autocomplete` memory.\n\n- [x] **Increment 1a — scope-aware symbol model** — `demo/src/scope-symbols.ts`\n (`collectScopeSymbols`, acorn + acorn-loose fallback, destructuring-aware,\n position-scoped, origin-tracking). Wired into `demo/src/autocomplete.ts`\n (replaces the regex extractors, with regex as never-go-blank fallback).\n `todoApp`, `h1`…`button` now complete; `h1` shows `∈ elements`. Tests:\n `demo/src/scope-symbols.test.ts` (11) + provider regression in\n `demo/autocomplete.test.ts`.\n- [x] **Increment 1b — introspection bridge** — done. (i) path-aware member\n resolution in `ajs-language.ts` (`getPathBeforeDot`/`resolvePath`/\n `getCompletionsFromPath`) so `todoApp.items.` resolves, not just `todoApp.`.\n (ii) `editors/introspect-value.ts` (serializable, self-contained, injectable) + async `AutocompleteConfig.getMembers` + `demo/src/introspection-bridge.ts`\n (hidden disposable iframe, reuses the run pipeline, direct-`eval` handle into\n module scope, caches last good sandbox) wired via `getMembers` in the\n playground. Tested headlessly through the real `tjsCompletionSource`.\n **Verified working well live** (destructured locals + `todoApp.items.push` + proxy members).\n- [ ] **Increment 2 — richer hints from real values** — function arity, `__tjs`\n metadata when present, signature help from the live function.\n- [ ] **Increment 3 — argument-type-driven completion (PINNED) — the convergence\n point.** Infer an argument's type from the callee and complete inside it:\n `h1({ style: { color: ⎸ } })` → arg0 is `ElementPart` → suggest `style` →\n CSS values. A vanilla JS function exposes only `.length`/`.name`, so this\n needs the callee to carry `__tjs` whose param is a type-as-example (itself an\n introspectable value — the bridge reads its keys; `style`'s value is a CSS\n predicate → `suggest()`). **Precondition / the pin: rewrite tosijs `style` +\n the elementCreator in TJS** so creators carry `__tjs` example-typed params —\n then it's pure introspection + `suggest()`, no special-casing, no `.d.ts`\n parsing (the \"smaller declaration files\" payoff). Works TODAY for the user's\n OWN example-typed object-param functions via `getMetadata`/`getSignatureHelp`;\n the one missing primitive is **call-context detection** (enclosing call +\n callee path + arg index + nested-key-vs-value) — unit-testable like\n `collectScopeSymbols`. The `elementParts`/`style` CSS leaf rides this via the\n predicate-schema + `suggest()` work (`src/lang/predicate.ts`).\n- [ ] **Increment 4 — completions-as-functions** — let a value/type carry a\n `suggest` hook (annotation / `__suggest`) the bridge calls; transpiles away\n under build options (dev-only, like the strip-safety pattern). The third leg\n of \"a language, not a type system.\"\n\n## Playground - Leverage tjs documentation system\n\n- [ ] tosijs-ui essentially encapsulates most of what we've done with playgrounds in a more reusable way\n- [ ] where necessary identify shortcomings in tosijs-ui's build / doc system\n- [ ] fold in anything we add / need beyond the new build / doc system\n\n## Playground - Error Navigation\n\n- [ ] Test errors: click should navigate to source location\n- [ ] Console errors: click should navigate to source location\n- [ ] Error in imported module: click through to source\n\n## Playground - Module Management\n\n- [ ] Import one example from another in playground\n- [ ] Save/Load TS examples (consistency with TJS examples)\n- [ ] File name should be linked to example name\n- [ ] New example button in playground\n- [ ] UI for managing stored modules (browse/delete IndexedDB)\n- [ ] Auto-discover and build local dependencies in module resolution\n- [ ] **Wire `ModuleLoader` into the playground's `tjs()` invocation** for transpile-time cross-file `wasm function` composition (Phase 3 of the wasm-library plan). Today the playground resolves imports at runtime via the local-module store — correct but uses the \"boundary form\" with a JS↔wasm crossing per call. With a ModuleLoader, imported `wasm function`s would be composed into the consumer's own `WebAssembly.Module` at transpile time, enabling wasm-to-wasm calls (single-digit nanosecond per-call cost). The `wasm-library-consumer.md` example flags this as a known gap. See `src/lang/module-loader.ts` (already shipped) and `wasm-library-plan.md` § Phase 3.\n\n## Language Features\n\n- [x] Honest boolean coercion (TjsStandard) — `Boolean(new Boolean(false))` and friends now return false. Source rewriter wraps every truthiness context (`if`/`while`/`for`/`do`/`!`/`&&`/`||`/`?:`, `Boolean(x)` calls) with `__tjs.toBool` which unwraps boxed primitives. Always-on under `TjsStandard`. Demo: `examples/js-footguns-fixed.tjs`. Doc: `guides/footguns.md`.\n- [ ] Intra-function type safety — bring TJS to parity with TS / good linters\n - [ ] **Tier 1 (lint):** `TjsTypedLet` mode — warn/error on `let` without type annotation. Follows the `TjsNoVar` precedent (`src/lang/parser.ts:214`). Severity gated by mode (info under `TjsStandard`, error under `TjsStrict`). ~30 lines in `src/lang/linter.ts`.\n - [ ] **Tier 2 (compile-time inference):** infer `TypeDescriptor` from initializer (already have `src/lang/inference.ts`), store per-decl in scope, walk subsequent `AssignmentExpression` nodes, warn on type-incompatible reassignment. ~200–300 lines, linter-only, no codegen changes.\n - [ ] **Tier 3 (runtime checks, long-term):** rewrite `let x = e` / `x = e` in the JS emitter to `__tjs.checkType(...)` so out-of-band assignments return MonadicError. Open design questions: closed-over `let`s, uninitialized `let x`, perf cost of per-assignment call. Defer until we see how Tier 1+2 land.\n- [ ] Audit monadic-error propagation when an error is nested inside a parameter (esp. arrays)\n - Rule: a MonadicError reaching a checked boundary should surface as ONE error, not as data containing an error (e.g. `[5, <error>, 7]`).\n - Caveat: if the function never inspects the param, no error needs to fire — propagation is on-check, not eager.\n - Partial coverage today: input-validation in emitted JS scans top-level array params for an embedded MonadicError and re-propagates it (commit `3db372d`). Other paths likely miss this — return values, deeper nesting (object fields, arrays-of-arrays), function-typed params whose callbacks return arrays containing errors, etc.\n - Investigate: where does a MonadicError survive past a boundary as data? Audit `checkType` in `src/lang/runtime.ts`, the emitted-JS validation prefix in `src/lang/emitters/js.ts`, and `checkFnShape` interaction with array returns.\n- [x] Portable Type predicates — expression-only AJS subset (no loops/async, serializable). **Done** as the predicate engine — see the \"Predicate types\" section above (`src/lang/predicate.ts`).\n- [x] Sync AJS / AJS-to-JS compilation — verified-pure predicates compile to native JS with fuel-injection points. **Done** (`compilePredicate`); see \"Predicate types\" above. (Generalizing this to arbitrary type-checked AJS beyond predicates is the future \"propagate verify→native\" item.)\n- [ ] Self-contained transpiler output (no runtime dependency)\n - Currently transpiled code references `globalThis.__tjs` for pushStack/popStack, typeError, Is/IsNot\n - Requires runtime to be installed or a stub (see playground's manual \\_\\_tjs stub)\n - Goal: TJS produces completely independent code, only needing semantic dependencies\n - Options: inline minimal runtime (~1KB), `{ standalone: true }` option, or tree-shake\n - See: src/lang/emitters/js.ts TODO comment for details\n- [x] WASM compilation at transpile time (not runtime)\n - [x] Compile wasm {} blocks during transpilation\n - [x] Embed base64-encoded WASM bytes in output\n - [x] Include WAT disassembly as comment for debugging/learning\n - [x] Self-contained async instantiation (no separate compileWasmBlocksForIframe)\n- [x] Expand WASM support beyond POC\n - [x] For loops with numeric bounds\n - [x] Conditionals (if/else)\n - [x] Local variables within block\n - [x] Typed array access (Float32Array, Float64Array, Int32Array, Uint8Array)\n - [x] Memory operations\n - [x] Continue/break statements\n - [x] Logical expressions (&& / ||)\n - [x] Math functions (sqrt, abs, floor, ceil, min, max, sin, cos, log, exp, pow)\n- [x] WASM SIMD support (v128/f32x4)\n - 12 f32x4 intrinsics: load, store, splat, extract_lane, replace_lane, add, sub, mul, div, neg, sqrt\n - Explicit intrinsic approach (users call f32x4\\_\\* in wasm blocks)\n - Disassembler handles 0xfd prefix with LEB128 sub-opcodes\n - 16-byte aligned memory for v128 loads/stores\n - Demos: starfield SIMD rotation, vector search cosine similarity\n- [ ] WASM SIMD vector search (batteries)\n - Replace JS vectorSearch battery with WASM SIMD implementation\n - SIMD cosine similarity demonstrated in vector search demo\n - TODO: integrate as a battery atom with auto-detect + fallback\n\n## Cross-file WASM Libraries (v0.8.0)\n\nShipped in v0.8.0 — design + history in `wasm-library-plan.md`, user-facing reference in `DOCS-WASM.md`.\n\n- [x] Module consolidation: one `WebAssembly.Module` per file with N exports (was N separate modules sharing memory)\n- [x] Transpile-time `ModuleLoader` (`src/lang/module-loader.ts`) — opt-in `.tjs`/`.ts`/`.js` resolution\n- [x] `(export)? wasm function NAME(params): RetType { body }` declaration syntax\n- [x] Purity enforcement (backend already rejects host imports) + `(!)` unsafe marker reserved\n- [x] Cross-file composition: `import { dot } from 'tjs-lang/linalg'` resolves at transpile time\n- [x] Wasm-to-wasm `call <index>` instructions (Phase 1.5) — no JS↔wasm boundary on intra-module calls\n- [x] Tree-shaking + transitive dep walking in cross-file composition (only reached functions get pulled in)\n- [x] Boundary distribution form: same source → self-contained `.js` for non-tjs consumers\n- [x] `tjs-lang/linalg` MVP — `dot`, `norm_sq`, `dot_at`, `norm_sq_at` (f32x4 SIMD)\n- [x] Canonical 3-way vector-search benchmark proves composed-WASM matches inline perf\n- [x] DOCS-WASM.md + TJS-FOR-JS.md additions + playground examples (`wasm-functions.md`, `wasm-library-author.md`, `wasm-library-consumer.md`)\n- [x] JSDoc `/** */` blocks extracted by playground docs renderer\n\n### Deferred follow-ups\n\n- [ ] Wire `ModuleLoader` into the playground's `tjs()` invocation so cross-file composition works inside the playground (today the playground resolves imports at runtime — works but uses the boundary form). See `Playground - Module Management` section above for the full note. **High priority — the canonical wasm-library demo runs at boundary-form perf in the playground until this lands.**\n- [ ] `i32` / `f32` / `v128` return types in wasm bytecode emitter (currently all returns are f64-or-void). Parsed today via `: RetType` annotation but not driving emission. Needed for top-K (i32 indices) and any wasm function that naturally returns f32 from SIMD.\n- [ ] `tjs-lang/linalg` expansion beyond MVP:\n - Vector: `norm`, `normalize`, `add`, `sub`, `scale`, `lerp` (use `out` parameter for buffer results)\n - Matrix: `matmul`, `transpose`, `identity`, `inverse_3x3`, `inverse_4x4`\n - 3D: `cross`, `quat_mul`, `mat4_from_quat`, `look_at`, `perspective`\n - Batched kernels: `cosine_search(corpus, query, count, dim) → bestIdx`, `top_k_cosine(corpus, query, count, dim, k, outIdx, outScores)` (one boundary crossing for the whole workload regardless of K)\n- [ ] gl-matrix benchmark — measure linalg vs the standard JS vector library at realistic scale\n- [ ] Production `dist/tjs-linalg.js` bundle wired into `scripts/build.ts` (currently `bun` resolves the `.tjs` source directly; production consumers need the pre-transpiled `.js`)\n- [ ] SIMD tail-loop for `n` not a multiple of 4 (today callers must pad)\n- [ ] Inline `wasm{}` blocks still subject to `==` → `Eq()` rewrite (the inline-block extractor runs after `transformEqualityToStructural`; the new `wasm function` extractor runs before it). Fix: move `extractWasmBlocks` earlier in `preprocess()` too. Pre-existing bug, not introduced by v0.8.0.\n\n## Editor\n\n- [ ] Embedded AJS syntax highlighting\n\n## Documentation / Examples\n\n- [ ] Create an endpoint example\n- [ ] Fold docs and tests into one panel, with passing tests collapsed by default (ts -> tjs inserts test; tjs -> js turns test blocks into documentation along with outcomes).\n- [ ] Dim/hide the preview tab if nothing ever changed it\n- [ ] Single source of truth for version number. I note the badge displayed in console is not matching the version. No hardwired versions -- version number is pulled from package.json and written to version.ts somewhere and that is the single source of truth.\n\n## Production integration feedback (snowfox-app)\n\nOutstanding items from real-world VM integration. See conversation notes; ranked by hours-burned.\n\n- [ ] **`resolveValue` doesn't recurse into plain object literals** — atoms with structured input get `{$expr}` children unresolved. Need canonical `deepResolve(value, ctx)` helper.\n- [ ] **Browser-safe entry point (`tjs-lang/browser`)** — main entry pulls `node:fs/promises` (CLI/playground); breaks webpack 4 and similar bundlers.\n- [ ] **`evaluateExpr` diagnostics** — when a node has missing required fields, wrap with op name + step location instead of raw `Cannot read properties of undefined`.\n- [ ] **`typescript` not resolvable from the main entry** _(confirmed live in 0.8.0–0.8.2 via fresh `npm install` + Node import)_ — `import 'tjs-lang'` throws `Cannot find package 'typescript' imported from dist/index.js`. Cause: `src/lang/index.ts:62` statically re-exports `fromTS`, dragging the TS compiler (~4MB) into `dist/index.js`; `typescript` is only a devDependency, never declared, so Node consumers without it can't import the main entry at all (and it pulls TS at import time → also crashes Cloud Run). The lean `tjs-lang/lang` entry is fine (no fromTS). **Not a 0.8.2 regression** — pre-existing. Fix options: (a) drop the static `fromTS` re-export from the main entry so it's reached only via `tjs-lang/lang/from-ts` (matches the documented usage; makes `import 'tjs-lang'` TS-free) — cleanest, mild breaking change for top-level `fromTS` importers; (b) declare `typescript` as a `peerDependency` (+ optional meta) or `optionalDependency` so it's at least provided/signalled. Recommend (a) + lazy-load. Cut in 0.8.3.\n- [ ] **`const` inside `while` loop body** — `constSet` re-runs each iteration and throws \"Cannot reassign const variable\". Either compile-time error or per-iteration scope.\n- [ ] **AgentVM: warn on unknown atoms referenced in source** — currently fails at execution time with `Unknown Atom: foo` and no hint about `batteryAtoms` / user-defined atoms.\n\n## Language subset invariant (TJS ⊇ AJS) — see PRINCIPLES.md\n\n**Invariant:** every legal AJS source must be legal TJS source (and options-off\nTJS ⊇ JS). TJS may do _more_ with the same source but must never _reject_ it.\nEngraved in `PRINCIPLES.md`. **Now holds** — restored via the signature-test\nchanges below; guarded by `src/lang/subset-invariant.test.ts`.\n\n- [x] **Signature tests: inconclusive (not error) when un-runnable** — a signature test that can't execute (undefined references like AJS atoms `httpFetch`, or a harness that can't run the module) is now reported as `inconclusive: true` (a warning carrying the reason), never a transpile error. Only a test that _runs and mismatches_ stays a hard failure. New `inconclusive` field on `TestResult`; the strict-mode throw in `js.ts` skips inconclusive results. (Playground: surface the `inconclusive` flag distinctly — see playground TODO.)\n- [x] **Multi-function signature-test harness** — the realistic newline-separated multi-function source already executed and validates correctly; only the _same-line_ `} function` edge case failed the harness (\"Unexpected keyword 'function'\"). That failure is now inconclusive (non-fatal) rather than a transpile error, so the invariant holds either way. (Making same-line two-functions actually execute is a nice-to-have, not required.)\n- [x] **Subset guard test** — `src/lang/subset-invariant.test.ts`: representative AJS snippets (helpers with typed sigs, atom-call + return type, helper calling an atom) asserted valid as _both_ AJS and TJS; plain JS asserted valid under options-off TJS; plus controls (un-runnable → inconclusive, genuine mismatch → still throws).\n\n- [x] **Playground: surface inconclusive signature tests** — `renderTestResults` (demo/src/playground-shared.ts) now counts inconclusive separately, renders them with a distinct amber `test-inconclusive`/`test-note` style and a `—` icon (not the ✗ failure), keeps them out of the failure count and editor error markers, and turns the tests-tab indicator amber when only inconclusive. Verified with a happy-dom unit test incl. real transpiler output (`demo/src/playground-test-results.test.ts`).\n- [x] **Source dialect (`dialect: 'js' | 'tjs'`)** — public transpile option that sets the modes-on/off default explicitly. `'js'` preserves plain-JS semantics; `'tjs'` (and the bare-string default) is native TJS. Plus extension→dialect helpers `dialectForFilename`/`sourceKindForFilename` from `tjs-lang/lang`, wired into the CLI (check/types/emit/run) so a `.js` file is never silently given TJS semantics. Makes plain JS first-class for hosts (e.g. the tosijs doc system replacing sucrase). `src/lang/dialect.ts`, `src/lang/dialect.test.ts`.\n- [ ] **`transpileSource` one-call `js | ts | tjs` sugar** — deferred. A single async call wrapping the route in PRINCIPLES.md (\"Routing all three dialects\"). It must NOT live in `tjs-lang/lang`: esbuild emits single-file bundles (no code-splitting), so a `fromTS` import — even a dynamic one — gets inlined and drags the TypeScript compiler into the lean, TS-free lang bundle (this broke the `tjs-lang`/`tjs-eval`/`tjs-vm` builds when first attempted). Correct home is a TS-aware entry (the main `tjs-lang` entry already bundles fromTS + externalizes typescript), or switch the bundler to code-splitting. Until then, consumers use the explicit recipe (tjs for js/tjs, fromTS+tjs for ts).\n\n### Deferred enrichment (parity, not invariant)\n\nAJS and TJS share one parser, so AJS already _accepts_ the full signature syntax — input `(!`/`(?` and return `)-!`/`)-?`/`)->` markers, colon/return examples — they just aren't _enforced_ in AJS. Closing that is a nice-to-have, separate from the subset invariant above.\n\nTJS return-marker semantics (reference for when AJS enforcement lands): `)-!` never checks the return + **bypasses the build-time signature test**; `)-?` always checks at runtime; `)->` checks only under global `safety: 'all'`; plain `): T` captures the type + runs the build-time signature test but isn't runtime-asserted (default `safety: 'inputs'`). In AJS today every signature behaves like `)-!` on the return and gets only coarse JSON-Schema validation on inputs (and `n: 0` integer examples currently emit a no-op `{}` schema — a bug).\n\n- [ ] **Signature-as-test in AJS** — TJS already runs the signature example as a transpile-time test (`scale(x:1.5,factor:0.5):0.75` with an inconsistent body fails with \"Expected 0.75, got 1.5\", `isSignatureTest:true`). AJS runs nothing. The VM can execute the function with the example inputs directly, so AJS is well-positioned to run the same check. Opt-in at first (don't break existing untested agents).\n- [ ] **Enrich AJS entry input schema** — `parametersToJsonSchema` currently coarsens examples (`1.5`→`{type:number}`) and, worse, `n: 0` (integer example) emits `{}` — a no-op that validates nothing. JSON Schema can express `{type:integer}` and `{minimum:0}`; capture int / non-negative / number distinctions so the entry contract isn't silently dropped. (Full predicate parity with TJS `checkType` isn't reachable in JSON Schema — defer.)\n- [ ] **Validate helper params** — helper bodies currently bind args by position with no validation (only arity is checked at transpile). For least-astonishment, helpers should honor their param examples like the entry function once AJS enforcement lands.\n\n### Completed in current session\n\n- [x] **Local helper functions / `TOOL_LIBRARY` pattern** — AJS agent source may now declare multiple top-level functions: the **last** is the entry point, the rest are helpers. Implemented **option 2** (by-reference `callLocal` + per-agent helper table), chosen over inlining because it supports recursion (bounded by fuel/timeout + a `MAX_CALL_DEPTH=256` host-stack guard) and keeps the AST compact (helper bodies stored once, not duplicated per call site — matters since AJS AST travels as data). Helpers run in isolated scopes (top-level siblings, no closure over caller locals). Helper calls must live at statement level (can't be nested in expressions, like template literals); recursion is a runtime loop, not a transpile error. See `src/use-cases/local-helpers.test.ts`, `extractFunctions` (parser), `ensureHelperTransformed`/`callLocal` emit (emitters/ast.ts), `callLocal` atom (vm/runtime.ts).\n- [x] `llmPredictBattery` now has `timeoutMs: 120000` (was using default 1000ms — broken for any real LLM call) + regression test in `batteries.test.ts`.\n- [x] `typesVersions` fallback in `package.json` so legacy `moduleResolution: node` consumers can resolve `tjs-lang/vm`, `tjs-lang/lang`, `tjs-lang/batteries` etc.\n- [x] **Per-atom `timeoutMs` override** — `vm.run({ timeoutOverrides: { llmPredictBattery: 60000 } })` now works, mirroring the existing `costOverrides` pattern. Supports `number` and `(input, ctx) => number`; `0` disables the per-atom timeout. New `TimeoutOverride` type exported from `tjs-lang/vm`. See `src/use-cases/timeout-overrides.test.ts`.\n- [x] **Replaced `vm.run` default `timeoutMs = fuel × 10ms` formula** — now derived from the registered atoms as `max(per-atom timeoutMs) × 2`, floored at 60s (`AgentVM.defaultRunTimeout`). A fixed 60s default (interim) was shorter than the 120s `llmVision`/`llmPredictBattery` budgets, so vision/LLM calls timed out mid-call on slower models; the atom-derived default always covers the slowest atom (and a chained pair) and self-adjusts to custom slow atoms. Updated timeout error message to point at `timeoutMs` / `timeoutOverrides` instead of \"increase fuel\".\n- [x] **`storeVectorize` / `storeVectorAdd` get `timeoutMs: 60000`** — both make embedding network calls but had the 1s atom default, so a cold embedding model timed out. (Same class as the llmVision/llmPredict 120s budgets, missed for the store atoms.) Local ops (`storeSearch`, `storeCreateCollection`) keep the default.\n\n- [x] **Vision-detection probe used a degenerate 1×1 PNG** — real vision preprocessors reject it (gemma-4-e4b: HTTP 400 \"Cannot handle this data type: (1,1,1)\"), so a genuinely multimodal model was false-negatived as `vision: false` and vision examples skipped with \"no vision model available\". Probe now uses a valid 32×32 PNG (gemma returns 200). `src/batteries/audit.ts`.\n\n### Deferred (surfaced this session)\n\n- [ ] **Model-audit vision detection still only checks `res.ok`** (`audit.ts` checkVision) — a text model that _tolerates_ the multimodal format without erroring would false-positive. Stronger: check the _response content_ (does the model actually describe the image?). Lower priority now that the 1×1 false-negative is fixed.\n\n## Infrastructure\n\n- [ ] Make playground components reusable for others\n- [ ] Web worker for transpiles (freezer - not needed yet)\n- [x] Retarget Firebase as host platform (vs GitHub Pages)\n- [ ] Universal LLM endpoint with real LLMs (OpenAI, Anthropic, etc.)\n- [ ] ESM-as-a-service: versioned library endpoints\n- [ ] User accounts (Google sign-in) for API key storage\n- [ ] AJS-based Firestore and Storage security rules\n- [ ] npx tjs-playground - run playground locally with LM Studio\n- [ ] Virtual subdomains for user apps (yourapp.tjs.land)\n - [ ] Wildcard DNS to Firebase\n - [ ] Subdomain routing in Cloud Function\n - [ ] Deploy button in playground\n - [ ] Public/private visibility toggle\n- [ ] Rate limiting / abuse prevention for LLM endpoint\n- [ ] Usage tracking / billing foundation (for future paid tiers)\n\n## Dependencies & Tooling\n\nFollow-ups from the ESLint 8 → 10 + typescript-eslint 5 → 8 flat-config migration:\n\n- [ ] **Decide package-lock.json policy.** Repo is bun-primary (bun.lock is canonical). The committed `package-lock.json` is stale (still references the old eslint v5 tree) and a fresh npm re-resolve balloons it by ~6k lines (full firebase-admin/google-cloud tree) and needs `--legacy-peer-deps` (pre-existing `tosijs-ui` wants `marked@^16` vs pinned `marked@9`). Either regenerate it in its own commit or remove it and let bun.lock be the sole lockfile.\n- [ ] **Clean up 22 pre-existing lint warnings** (unused vars/imports, prefer-const) — surfaced by `bun eslint src`, predate the migration (same `no-unused-vars`/`^_` config), all warnings not errors. Low-risk dead-code sweep across ~10 files.\n- [ ] **Dev-dependency vulns (none shipped to consumers).** `npm audit` shows 28, all dev/peer: Firebase SDK + admin stack, the vitest/vite/esbuild/rollup chain (vitest _critical_, genuinely used by 5 test files → needs v3 major), happy-dom, valibot, ws. Plus one eslint-transitive straggler: `flatted@3.3.3` via `file-entry-cache → flat-cache` (non-major fix).\n- [ ] **Resolve the `marked` peer conflict** — `tosijs-ui` peers on `marked@^16`, repo pins `marked@9.1.6` (bun warns + installs; npm refuses without `--legacy-peer-deps`).\n\n## Self-hosting (TS feature coverage)\n\nFour `it.skip` cases in `src/use-cases/self-hosting.test.ts` — advanced TS that the\nTS→TJS path can't yet handle. Un-skip as support lands:\n\n- [ ] Class with private fields and methods (gated on class support)\n- [ ] Builder pattern with method chaining (gated on class support)\n- [ ] Complex decorator patterns (requires `experimentalDecorators`)\n- [ ] Module augmentation (type-only, no runtime code)\n\n(Also 4 unconditional skips in `src/lang/metadata-cache.test.ts` — the transpile\nmetadata-cache feature is stubbed: store/retrieve, version-invalidation, merge,\nprune.)\n\n## Batteries / LLM tests\n\n- [ ] **Audit misclassifies models under concurrent probing.** Many test files call `LocalModels.audit()` at once, sharing one `.models.cache.json` (cwd, 24h TTL). Clearing the cache before a parallel `bun test` makes several audits probe LM Studio simultaneously and classifications come back scrambled (embedding models tagged `LLM`, an LLM tagged `Embedding`). Tests stay green only by luck of ordering. Fix: serialize the audit, harden the probes, or isolate the cache per run. Workaround documented in [`docs/lm-studio-setup.md`](docs/lm-studio-setup.md). Surfaced 2026-06-10 while getting the LLM suite green.\n\n---\n\n## Completed (this session)\n\n### Project Rename\n\n- [x] Rename from tosijs-agent to tjs-lang\n- [x] Update all references in package.json, docs, scripts\n\n### Timestamp & LegalDate Utilities\n\n- [x] Timestamp - pure functions, 1-based months, no Date warts (53 tests)\n - now, from, parse, tryParse\n - addDays/Hours/Minutes/Seconds/Weeks/Months/Years\n - diff, diffSeconds/Minutes/Hours/Days\n - year/month/day/hour/minute/second/millisecond/dayOfWeek\n - toLocal, format, formatDate, formatTime, toDate\n - isBefore/isAfter/isEqual/min/max\n - startOf/endOf Day/Month/Year\n- [x] LegalDate - pure functions, YYYY-MM-DD strings (55 tests)\n - today, todayIn, from, parse, tryParse\n - addDays/Weeks/Months/Years\n - diff, diffMonths, diffYears\n - year/month/day/dayOfWeek/weekOfYear/dayOfYear/quarter\n - isLeapYear, daysInMonth, daysInYear\n - toTimestamp, toUnix, fromUnix\n - format, formatLong, formatShort\n - isBefore/isAfter/isEqual/min/max/isBetween\n - startOf/endOf Month/Quarter/Year/Week\n- [x] Portable predicate helpers: isValidUrl, isValidTimestamp, isValidLegalDate\n\n### TJS Mode System (native TJS has all modes ON by default; TS-originated code defaults to OFF)\n\n- [x] Invert mode system - native TJS enables all modes; TS-originated/AJS code defaults to JS semantics\n- [x] TjsEquals directive - structural == and != (null == undefined)\n- [x] TjsClass directive - classes callable without new\n- [x] TjsDate directive - bans Date constructor/methods\n- [x] TjsNoeval directive - bans eval() and new Function()\n- [x] TjsStrict directive - enables all of the above\n- [x] TjsSafeEval directive - includes Eval/SafeFunction for dynamic code execution\n- [x] Updated Is() for nullish equality (null == undefined)\n- [x] Added Is/IsNot tests (structural equality, nullish handling)\n- [x] TjsStandard directive - newlines as statement terminators (prevents ASI footguns)\n- [x] WASM POC - wasm {} blocks with parsing, fallback mechanism, basic numeric compilation\n- [x] Eval/SafeFunction - proper VM-backed implementation with fuel metering and capabilities\n\n### Bundle Size Optimization\n\n- [x] Separated Eval/SafeFunction into standalone module (eval.ts)\n- [x] Created core.ts - AJS transpiler without TypeScript dependency\n- [x] Fixed tjs-transpiler bundle: 4.14MB → 88.9KB (27KB gzipped)\n- [x] Runtime is now ~5KB gzipped (just Is/IsNot, wrap, Type, etc.)\n- [x] Eval adds ~27KB gzipped (VM + AJS transpiler, no TypeScript)\n- [x] TypeScript only bundled in playground (5.8MB) for real-time TS transpilation\n"
|
|
1628
|
+
"text": "# TJS-Lang TODO\n\n## Predicate types — \"AJS is JSON-Schema's missing piece\"\n\nThe thesis (see the blog draft): JSON-Schema / TS can't express types that need\n**computation**; verified-pure, composable AJS predicates can — serializable\n(the AJS AST), safe (no IO, fuel-bounded), and compiled to native JS so they're\nfast. CSS is the torture-test proof. The engine is built and green on `main`;\nwhat remains is **delivery, measurement, and reach** — not invention.\n\n**Done (engine):**\n\n- [x] Atom `effects: 'pure' | 'io'` keystone — classified, guarded (`src/vm/atom-effects.test.ts`).\n- [x] `verifyPredicate` / `compilePredicate` — `src/lang/predicate.ts`, exported from `tjs-lang/lang`. Transitive closure check, pure-method whitelist, registry-driven effects.\n- [x] Fuel-bounded, global-shadowed native compiler (loops rejected; `__fuel()` at function entry; per-call budget; stack-overflow normalized to `PredicateFuelExhausted`). Zero measurable perf cost.\n- [x] PoC + CSS torture set + perf ballpark in `experiments/predicates/` (theme ~0.13ms, ReDoS-linear).\n\n**Remaining (delivery / north star):**\n\n- [x] **#4 Autocomplete `suggest()` companion** — `src/lang/predicate.ts` (`suggest`, exported from `tjs-lang/lang`). Mines a cluster for completions: keyword sets (array literals + `==` literals) → `value` suggestions, `startsWith(...)` guards → open-ended `stub`s (`var(--`/`calc(`). Mined values are run through the compiled entry predicate so suggestions are _guaranteed valid_, not just enumerated. Beats both TS modes: a `string` fallback offers nothing, a finite union can't offer the open-ended stubs. Prefix-filtered + limited. Tests: `src/lang/suggest.test.ts`, demo `experiments/predicates/suggest.demo.test.ts`.\n- [ ] **#5 Wire into `FunctionPredicate` / `Type`** — predicate bodies authored in this verified-safe substrate; the real consumer.\n- [x] **#6 (tjs-lang side) the `$predicate` keyword + reference evaluator** — `src/lang/predicate-schema.ts` (`compilePredicateSchema` / `validatePredicateSchema`, exported from `tjs-lang/lang`). A JSON-Schema node carries `$predicate` (predicate-cluster _source_; trivially serializable, the verifier makes it safe to run). Structural keywords (type/properties/required/items) validate for everyone; `$predicate` runs only for aware validators → progressive enhancement. Demoed on CSS (`experiments/predicates/css-schema.demo.test.ts`): same JSON, naive sees `string`, aware validates var()/calc()/!important + recursion. Gotcha noted: embed predicate source via `String.raw` (regex backslashes) — moot in real JSON.\n- [ ] **#6 (production) wire `$predicate` into tosijs-schema** — the \"incoming version\" from the blog: tosijs-schema (separate repo) evaluates `$predicate` via this engine. The tjs-lang format + evaluator are ready to consume. **The remaining north-star step.**\n- [ ] **Real CSS predicate library** — productionize beyond the PoC corpus (the tosijs CSS replacement). Recursive structure is plain `$ref` JSON-Schema; only leaf value-grammar needs `$predicate` (progressive enhancement). Schema validates the _serialized/data_ form; `Color` instances + bare-number→`px` are runtime conveniences (duck-typeable via `.toString()` where wanted).\n- [ ] **Regex-linting in the verifier** — the ReDoS path-forward the blog commits to: reject catastrophic-backtracking patterns (the one unbounded primitive fuel can't interrupt). Until then, predicates are \"no worse than `pattern`, with the bounded-tokenizer option.\"\n\n## \"Safe is fast\" — the campaign (measurement + propagation, not invention)\n\nThe architecture already makes the safe path the fast path: boundary-level checks\n(a few comparisons per _call_, not per-op), verify→native for validation, inline\nWASM/SIMD for hot loops, zero-cost happy-path errors. The strip-the-safety\ntranspiler option is the Obj-C-`IMP`-cache / Rust-`unsafe` move — it wins the perf\nargument precisely because, in practice, you leave the safety on. What's left is\nto **prove it and spread it**:\n\n- [ ] **Systematic overhead benchmark** — TJS-checked function call vs raw call across representative code (not just predicates), so \"safe is fast\" is backed by numbers, not just architecture. (Doubles as the CSS-post perf data — re-run on the _real_ tosijs theme with the full predicate set, confirming the ~0.1ms claim on real data.)\n- [ ] **Propagate verify→native** — weave it under the type system / tosijs so the capability is pervasive, not just an engine + PoC.\n- [ ] Frame the announcement around data + a real framework running it, not a promise. The blog draft is the spec: its present-tense claims (#6, the CSS lib, the real-theme number) must be true before publishing.\n\n## Testing - watch items (don't fix yet)\n\n- [ ] **Flaky LLM assertion (low priority — leave unless it recurs).**\n `src/batteries/models.integration.test.ts:55` asserts `res.content.length > 5`\n for `predict('the color of the sky is')`. These tests have been reliable for\n a long time; a one-off failure (2026-06) was traced to non-deterministic /\n terse model output (isolated re-run passed, probe returned a normal 32-char\n reply) — most likely just a poor model choice that run, NOT a code bug. If it\n starts failing regularly: harden to assert non-empty string + tolerate empty\n `content` when a reasoning field is present (or use a prompt that demands a\n full sentence). Until then, leave as-is.\n\n## Editors - published `.js` is stale (address sooner rather than later)\n\n- [ ] **The `tjs-lang/editors/*` subpaths ship hand-maintained `.js` files that\n are NOT built from the `.ts` sources.** `editors/codemirror/ajs-language.js`\n is from Jan 2026 (~7.5KB, an old standalone CDN-example impl) while\n `ajs-language.ts` is the real ~51KB implementation (used by the playground,\n which bundles from source). So none of the autocomplete work (scope model,\n introspection bridge, member completion) reaches npm consumers of\n `tjs-lang/editors/codemirror` — they get months-old code. Same for\n monaco/ace `.js`. **Fix: add a build step that compiles/bundles\n `editors/**/\\*.ts`→ the published`.js`(wire into`scripts/build.ts`/\n`bun run make`) so it stays current automatically.\\*\\* Not urgent for active\n use cases (user isn't consuming tjs-lang externally yet) but flagged to do\n soon. Context: tosijs-ui's live-example/doc engine now uses tjs (replacing\n sucrase) and is evolving into a portable embeddable playground/IDE — it'll\n switch from ACE to CodeMirror and will want working autocomplete, at which\n point this matters.\n\n## Playground - Introspection-driven autocomplete\n\nThe current completion provider was regex-based and useless on real examples\n(`extractVariables` matched only `const NAME =`, missing ALL destructuring — and\nthe tosijs examples bind everything via destructuring). Direction: introspection,\nChrome-console style — run the user's actual code and read real values; predicates\nfill the value-grammar leaf (a runtime string can't reveal valid CSS colors). See\nthe `introspection-autocomplete` memory.\n\n- [x] **Increment 1a — scope-aware symbol model** — `demo/src/scope-symbols.ts`\n (`collectScopeSymbols`, acorn + acorn-loose fallback, destructuring-aware,\n position-scoped, origin-tracking). Wired into `demo/src/autocomplete.ts`\n (replaces the regex extractors, with regex as never-go-blank fallback).\n `todoApp`, `h1`…`button` now complete; `h1` shows `∈ elements`. Tests:\n `demo/src/scope-symbols.test.ts` (11) + provider regression in\n `demo/autocomplete.test.ts`.\n- [x] **Increment 1b — introspection bridge** — done. (i) path-aware member\n resolution in `ajs-language.ts` (`getPathBeforeDot`/`resolvePath`/\n `getCompletionsFromPath`) so `todoApp.items.` resolves, not just `todoApp.`.\n (ii) `editors/introspect-value.ts` (serializable, self-contained, injectable) + async `AutocompleteConfig.getMembers` + `demo/src/introspection-bridge.ts`\n (hidden disposable iframe, reuses the run pipeline, direct-`eval` handle into\n module scope, caches last good sandbox) wired via `getMembers` in the\n playground. Tested headlessly through the real `tjsCompletionSource`.\n **Verified working well live** (destructured locals + `todoApp.items.push` + proxy members).\n- [ ] **Increment 2 — richer hints from real values** — function arity, `__tjs`\n metadata when present, signature help from the live function.\n- [ ] **Increment 3 — argument-type-driven completion (PINNED) — the convergence\n point.** Infer an argument's type from the callee and complete inside it:\n `h1({ style: { color: ⎸ } })` → arg0 is `ElementPart` → suggest `style` →\n CSS values. A vanilla JS function exposes only `.length`/`.name`, so this\n needs the callee to carry `__tjs` whose param is a type-as-example (itself an\n introspectable value — the bridge reads its keys; `style`'s value is a CSS\n predicate → `suggest()`). **Precondition / the pin: rewrite tosijs `style` +\n the elementCreator in TJS** so creators carry `__tjs` example-typed params —\n then it's pure introspection + `suggest()`, no special-casing, no `.d.ts`\n parsing (the \"smaller declaration files\" payoff). Works TODAY for the user's\n OWN example-typed object-param functions via `getMetadata`/`getSignatureHelp`;\n the one missing primitive is **call-context detection** (enclosing call +\n callee path + arg index + nested-key-vs-value) — unit-testable like\n `collectScopeSymbols`. The `elementParts`/`style` CSS leaf rides this via the\n predicate-schema + `suggest()` work (`src/lang/predicate.ts`).\n- [ ] **Increment 4 — completions-as-functions** — let a value/type carry a\n `suggest` hook (annotation / `__suggest`) the bridge calls; transpiles away\n under build options (dev-only, like the strip-safety pattern). The third leg\n of \"a language, not a type system.\"\n\n## Playground - Leverage tjs documentation system\n\n- [ ] tosijs-ui essentially encapsulates most of what we've done with playgrounds in a more reusable way\n- [ ] where necessary identify shortcomings in tosijs-ui's build / doc system\n- [ ] fold in anything we add / need beyond the new build / doc system\n\n## Playground - Error Navigation\n\n- [ ] Test errors: click should navigate to source location\n- [ ] Console errors: click should navigate to source location\n- [ ] Error in imported module: click through to source\n\n## Playground - Module Management\n\n- [ ] Import one example from another in playground\n- [ ] Save/Load TS examples (consistency with TJS examples)\n- [ ] File name should be linked to example name\n- [ ] New example button in playground\n- [ ] UI for managing stored modules (browse/delete IndexedDB)\n- [ ] Auto-discover and build local dependencies in module resolution\n- [ ] **Wire `ModuleLoader` into the playground's `tjs()` invocation** for transpile-time cross-file `wasm function` composition (Phase 3 of the wasm-library plan). Today the playground resolves imports at runtime via the local-module store — correct but uses the \"boundary form\" with a JS↔wasm crossing per call. With a ModuleLoader, imported `wasm function`s would be composed into the consumer's own `WebAssembly.Module` at transpile time, enabling wasm-to-wasm calls (single-digit nanosecond per-call cost). The `wasm-library-consumer.md` example flags this as a known gap. See `src/lang/module-loader.ts` (already shipped) and `wasm-library-plan.md` § Phase 3.\n\n## Language Features\n\n- [x] Honest boolean coercion (TjsStandard) — `Boolean(new Boolean(false))` and friends now return false. Source rewriter wraps every truthiness context (`if`/`while`/`for`/`do`/`!`/`&&`/`||`/`?:`, `Boolean(x)` calls) with `__tjs.toBool` which unwraps boxed primitives. Always-on under `TjsStandard`. Demo: `examples/js-footguns-fixed.tjs`. Doc: `guides/footguns.md`.\n- [ ] Intra-function type safety — bring TJS to parity with TS / good linters\n - [ ] **Tier 1 (lint):** `TjsTypedLet` mode — warn/error on `let` without type annotation. Follows the `TjsNoVar` precedent (`src/lang/parser.ts:214`). Severity gated by mode (info under `TjsStandard`, error under `TjsStrict`). ~30 lines in `src/lang/linter.ts`.\n - [ ] **Tier 2 (compile-time inference):** infer `TypeDescriptor` from initializer (already have `src/lang/inference.ts`), store per-decl in scope, walk subsequent `AssignmentExpression` nodes, warn on type-incompatible reassignment. ~200–300 lines, linter-only, no codegen changes.\n - [ ] **Tier 3 (runtime checks, long-term):** rewrite `let x = e` / `x = e` in the JS emitter to `__tjs.checkType(...)` so out-of-band assignments return MonadicError. Open design questions: closed-over `let`s, uninitialized `let x`, perf cost of per-assignment call. Defer until we see how Tier 1+2 land.\n- [ ] Audit monadic-error propagation when an error is nested inside a parameter (esp. arrays)\n - Rule: a MonadicError reaching a checked boundary should surface as ONE error, not as data containing an error (e.g. `[5, <error>, 7]`).\n - Caveat: if the function never inspects the param, no error needs to fire — propagation is on-check, not eager.\n - Partial coverage today: input-validation in emitted JS scans top-level array params for an embedded MonadicError and re-propagates it (commit `3db372d`). Other paths likely miss this — return values, deeper nesting (object fields, arrays-of-arrays), function-typed params whose callbacks return arrays containing errors, etc.\n - Investigate: where does a MonadicError survive past a boundary as data? Audit `checkType` in `src/lang/runtime.ts`, the emitted-JS validation prefix in `src/lang/emitters/js.ts`, and `checkFnShape` interaction with array returns.\n- [x] Portable Type predicates — expression-only AJS subset (no loops/async, serializable). **Done** as the predicate engine — see the \"Predicate types\" section above (`src/lang/predicate.ts`).\n- [x] Sync AJS / AJS-to-JS compilation — verified-pure predicates compile to native JS with fuel-injection points. **Done** (`compilePredicate`); see \"Predicate types\" above. (Generalizing this to arbitrary type-checked AJS beyond predicates is the future \"propagate verify→native\" item.)\n- [ ] Self-contained transpiler output (no runtime dependency)\n - Currently transpiled code references `globalThis.__tjs` for pushStack/popStack, typeError, Is/IsNot\n - Requires runtime to be installed or a stub (see playground's manual \\_\\_tjs stub)\n - Goal: TJS produces completely independent code, only needing semantic dependencies\n - Options: inline minimal runtime (~1KB), `{ standalone: true }` option, or tree-shake\n - See: src/lang/emitters/js.ts TODO comment for details\n- [x] WASM compilation at transpile time (not runtime)\n - [x] Compile wasm {} blocks during transpilation\n - [x] Embed base64-encoded WASM bytes in output\n - [x] Include WAT disassembly as comment for debugging/learning\n - [x] Self-contained async instantiation (no separate compileWasmBlocksForIframe)\n- [x] Expand WASM support beyond POC\n - [x] For loops with numeric bounds\n - [x] Conditionals (if/else)\n - [x] Local variables within block\n - [x] Typed array access (Float32Array, Float64Array, Int32Array, Uint8Array)\n - [x] Memory operations\n - [x] Continue/break statements\n - [x] Logical expressions (&& / ||)\n - [x] Math functions (sqrt, abs, floor, ceil, min, max, sin, cos, log, exp, pow)\n- [x] WASM SIMD support (v128/f32x4)\n - 12 f32x4 intrinsics: load, store, splat, extract_lane, replace_lane, add, sub, mul, div, neg, sqrt\n - Explicit intrinsic approach (users call f32x4\\_\\* in wasm blocks)\n - Disassembler handles 0xfd prefix with LEB128 sub-opcodes\n - 16-byte aligned memory for v128 loads/stores\n - Demos: starfield SIMD rotation, vector search cosine similarity\n- [ ] WASM SIMD vector search (batteries)\n - Replace JS vectorSearch battery with WASM SIMD implementation\n - SIMD cosine similarity demonstrated in vector search demo\n - TODO: integrate as a battery atom with auto-detect + fallback\n\n## Cross-file WASM Libraries (v0.8.0)\n\nShipped in v0.8.0 — design + history in `wasm-library-plan.md`, user-facing reference in `DOCS-WASM.md`.\n\n- [x] Module consolidation: one `WebAssembly.Module` per file with N exports (was N separate modules sharing memory)\n- [x] Transpile-time `ModuleLoader` (`src/lang/module-loader.ts`) — opt-in `.tjs`/`.ts`/`.js` resolution\n- [x] `(export)? wasm function NAME(params): RetType { body }` declaration syntax\n- [x] Purity enforcement (backend already rejects host imports) + `(!)` unsafe marker reserved\n- [x] Cross-file composition: `import { dot } from 'tjs-lang/linalg'` resolves at transpile time\n- [x] Wasm-to-wasm `call <index>` instructions (Phase 1.5) — no JS↔wasm boundary on intra-module calls\n- [x] Tree-shaking + transitive dep walking in cross-file composition (only reached functions get pulled in)\n- [x] Boundary distribution form: same source → self-contained `.js` for non-tjs consumers\n- [x] `tjs-lang/linalg` MVP — `dot`, `norm_sq`, `dot_at`, `norm_sq_at` (f32x4 SIMD)\n- [x] Canonical 3-way vector-search benchmark proves composed-WASM matches inline perf\n- [x] DOCS-WASM.md + TJS-FOR-JS.md additions + playground examples (`wasm-functions.md`, `wasm-library-author.md`, `wasm-library-consumer.md`)\n- [x] JSDoc `/** */` blocks extracted by playground docs renderer\n\n### Deferred follow-ups\n\n- [ ] Wire `ModuleLoader` into the playground's `tjs()` invocation so cross-file composition works inside the playground (today the playground resolves imports at runtime — works but uses the boundary form). See `Playground - Module Management` section above for the full note. **High priority — the canonical wasm-library demo runs at boundary-form perf in the playground until this lands.**\n- [ ] `i32` / `f32` / `v128` return types in wasm bytecode emitter (currently all returns are f64-or-void). Parsed today via `: RetType` annotation but not driving emission. Needed for top-K (i32 indices) and any wasm function that naturally returns f32 from SIMD.\n- [ ] `tjs-lang/linalg` expansion beyond MVP:\n - Vector: `norm`, `normalize`, `add`, `sub`, `scale`, `lerp` (use `out` parameter for buffer results)\n - Matrix: `matmul`, `transpose`, `identity`, `inverse_3x3`, `inverse_4x4`\n - 3D: `cross`, `quat_mul`, `mat4_from_quat`, `look_at`, `perspective`\n - Batched kernels: `cosine_search(corpus, query, count, dim) → bestIdx`, `top_k_cosine(corpus, query, count, dim, k, outIdx, outScores)` (one boundary crossing for the whole workload regardless of K)\n- [ ] gl-matrix benchmark — measure linalg vs the standard JS vector library at realistic scale\n- [ ] Production `dist/tjs-linalg.js` bundle wired into `scripts/build.ts` (currently `bun` resolves the `.tjs` source directly; production consumers need the pre-transpiled `.js`)\n- [ ] SIMD tail-loop for `n` not a multiple of 4 (today callers must pad)\n- [ ] Inline `wasm{}` blocks still subject to `==` → `Eq()` rewrite (the inline-block extractor runs after `transformEqualityToStructural`; the new `wasm function` extractor runs before it). Fix: move `extractWasmBlocks` earlier in `preprocess()` too. Pre-existing bug, not introduced by v0.8.0.\n\n## Editor\n\n- [ ] Embedded AJS syntax highlighting\n\n## Documentation / Examples\n\n- [ ] Create an endpoint example\n- [ ] Fold docs and tests into one panel, with passing tests collapsed by default (ts -> tjs inserts test; tjs -> js turns test blocks into documentation along with outcomes).\n- [ ] Dim/hide the preview tab if nothing ever changed it\n- [ ] Single source of truth for version number. I note the badge displayed in console is not matching the version. No hardwired versions -- version number is pulled from package.json and written to version.ts somewhere and that is the single source of truth.\n\n## Production integration feedback (snowfox-app)\n\nOutstanding items from real-world VM integration. See conversation notes; ranked by hours-burned.\n\n- [ ] **`resolveValue` doesn't recurse into plain object literals** — atoms with structured input get `{$expr}` children unresolved. Need canonical `deepResolve(value, ctx)` helper.\n- [ ] **Browser-safe entry point (`tjs-lang/browser`)** — main entry pulls `node:fs/promises` (CLI/playground); breaks webpack 4 and similar bundlers.\n- [ ] **`evaluateExpr` diagnostics** — when a node has missing required fields, wrap with op name + step location instead of raw `Cannot read properties of undefined`.\n- [ ] **`typescript` not resolvable from the main entry** _(confirmed live in 0.8.0–0.8.2 via fresh `npm install` + Node import)_ — `import 'tjs-lang'` throws `Cannot find package 'typescript' imported from dist/index.js`. Cause: `src/lang/index.ts:62` statically re-exports `fromTS`, dragging the TS compiler (~4MB) into `dist/index.js`; `typescript` is only a devDependency, never declared, so Node consumers without it can't import the main entry at all (and it pulls TS at import time → also crashes Cloud Run). The lean `tjs-lang/lang` entry is fine (no fromTS). **Not a 0.8.2 regression** — pre-existing. Fix options: (a) drop the static `fromTS` re-export from the main entry so it's reached only via `tjs-lang/lang/from-ts` (matches the documented usage; makes `import 'tjs-lang'` TS-free) — cleanest, mild breaking change for top-level `fromTS` importers; (b) declare `typescript` as a `peerDependency` (+ optional meta) or `optionalDependency` so it's at least provided/signalled. Recommend (a) + lazy-load. Cut in 0.8.3.\n- [ ] **`const` inside `while` loop body** — `constSet` re-runs each iteration and throws \"Cannot reassign const variable\". Either compile-time error or per-iteration scope.\n- [ ] **AgentVM: warn on unknown atoms referenced in source** — currently fails at execution time with `Unknown Atom: foo` and no hint about `batteryAtoms` / user-defined atoms.\n\n## Language subset invariant (TJS ⊇ AJS) — see PRINCIPLES.md\n\n**Invariant:** every legal AJS source must be legal TJS source (and options-off\nTJS ⊇ JS). TJS may do _more_ with the same source but must never _reject_ it.\nEngraved in `PRINCIPLES.md`. **Now holds** — restored via the signature-test\nchanges below; guarded by `src/lang/subset-invariant.test.ts`.\n\n- [x] **Signature tests: inconclusive (not error) when un-runnable** — a signature test that can't execute (undefined references like AJS atoms `httpFetch`, or a harness that can't run the module) is now reported as `inconclusive: true` (a warning carrying the reason), never a transpile error. Only a test that _runs and mismatches_ stays a hard failure. New `inconclusive` field on `TestResult`; the strict-mode throw in `js.ts` skips inconclusive results. (Playground: surface the `inconclusive` flag distinctly — see playground TODO.)\n- [x] **Multi-function signature-test harness** — the realistic newline-separated multi-function source already executed and validates correctly; only the _same-line_ `} function` edge case failed the harness (\"Unexpected keyword 'function'\"). That failure is now inconclusive (non-fatal) rather than a transpile error, so the invariant holds either way. (Making same-line two-functions actually execute is a nice-to-have, not required.)\n- [x] **Subset guard test** — `src/lang/subset-invariant.test.ts`: representative AJS snippets (helpers with typed sigs, atom-call + return type, helper calling an atom) asserted valid as _both_ AJS and TJS; plain JS asserted valid under options-off TJS; plus controls (un-runnable → inconclusive, genuine mismatch → still throws).\n\n- [x] **Playground: surface inconclusive signature tests** — `renderTestResults` (demo/src/playground-shared.ts) now counts inconclusive separately, renders them with a distinct amber `test-inconclusive`/`test-note` style and a `—` icon (not the ✗ failure), keeps them out of the failure count and editor error markers, and turns the tests-tab indicator amber when only inconclusive. Verified with a happy-dom unit test incl. real transpiler output (`demo/src/playground-test-results.test.ts`).\n- [x] **Source dialect (`dialect: 'js' | 'tjs'`)** — public transpile option that sets the modes-on/off default explicitly. `'js'` preserves plain-JS semantics; `'tjs'` (and the bare-string default) is native TJS. Plus extension→dialect helpers `dialectForFilename`/`sourceKindForFilename` from `tjs-lang/lang`, wired into the CLI (check/types/emit/run) so a `.js` file is never silently given TJS semantics. Makes plain JS first-class for hosts (e.g. the tosijs doc system replacing sucrase). `src/lang/dialect.ts`, `src/lang/dialect.test.ts`.\n- [ ] **`transpileSource` one-call `js | ts | tjs` sugar** — deferred. A single async call wrapping the route in PRINCIPLES.md (\"Routing all three dialects\"). It must NOT live in `tjs-lang/lang`: esbuild emits single-file bundles (no code-splitting), so a `fromTS` import — even a dynamic one — gets inlined and drags the TypeScript compiler into the lean, TS-free lang bundle (this broke the `tjs-lang`/`tjs-eval`/`tjs-vm` builds when first attempted). Correct home is a TS-aware entry (the main `tjs-lang` entry already bundles fromTS + externalizes typescript), or switch the bundler to code-splitting. Until then, consumers use the explicit recipe (tjs for js/tjs, fromTS+tjs for ts).\n\n### Deferred enrichment (parity, not invariant)\n\nAJS and TJS share one parser, so AJS already _accepts_ the full signature syntax — input `(!`/`(?` and return `)-!`/`)-?`/`)->` markers, colon/return examples — they just aren't _enforced_ in AJS. Closing that is a nice-to-have, separate from the subset invariant above.\n\nTJS return-marker semantics (reference for when AJS enforcement lands): `)-!` never checks the return + **bypasses the build-time signature test**; `)-?` always checks at runtime; `)->` checks only under global `safety: 'all'`; plain `): T` captures the type + runs the build-time signature test but isn't runtime-asserted (default `safety: 'inputs'`). In AJS today every signature behaves like `)-!` on the return and gets only coarse JSON-Schema validation on inputs (and `n: 0` integer examples currently emit a no-op `{}` schema — a bug).\n\n- [ ] **Signature-as-test in AJS** — TJS already runs the signature example as a transpile-time test (`scale(x:1.5,factor:0.5):0.75` with an inconsistent body fails with \"Expected 0.75, got 1.5\", `isSignatureTest:true`). AJS runs nothing. The VM can execute the function with the example inputs directly, so AJS is well-positioned to run the same check. Opt-in at first (don't break existing untested agents).\n- [ ] **Enrich AJS entry input schema** — `parametersToJsonSchema` currently coarsens examples (`1.5`→`{type:number}`) and, worse, `n: 0` (integer example) emits `{}` — a no-op that validates nothing. JSON Schema can express `{type:integer}` and `{minimum:0}`; capture int / non-negative / number distinctions so the entry contract isn't silently dropped. (Full predicate parity with TJS `checkType` isn't reachable in JSON Schema — defer.)\n- [ ] **Validate helper params** — helper bodies currently bind args by position with no validation (only arity is checked at transpile). For least-astonishment, helpers should honor their param examples like the entry function once AJS enforcement lands.\n\n### Completed in current session\n\n- [x] **Local helper functions / `TOOL_LIBRARY` pattern** — AJS agent source may now declare multiple top-level functions: the **last** is the entry point, the rest are helpers. Implemented **option 2** (by-reference `callLocal` + per-agent helper table), chosen over inlining because it supports recursion (bounded by fuel/timeout + a `MAX_CALL_DEPTH=256` host-stack guard) and keeps the AST compact (helper bodies stored once, not duplicated per call site — matters since AJS AST travels as data). Helpers run in isolated scopes (top-level siblings, no closure over caller locals). Helper calls must live at statement level (can't be nested in expressions, like template literals); recursion is a runtime loop, not a transpile error. See `src/use-cases/local-helpers.test.ts`, `extractFunctions` (parser), `ensureHelperTransformed`/`callLocal` emit (emitters/ast.ts), `callLocal` atom (vm/runtime.ts).\n- [x] `llmPredictBattery` now has `timeoutMs: 120000` (was using default 1000ms — broken for any real LLM call) + regression test in `batteries.test.ts`.\n- [x] `typesVersions` fallback in `package.json` so legacy `moduleResolution: node` consumers can resolve `tjs-lang/vm`, `tjs-lang/lang`, `tjs-lang/batteries` etc.\n- [x] **Per-atom `timeoutMs` override** — `vm.run({ timeoutOverrides: { llmPredictBattery: 60000 } })` now works, mirroring the existing `costOverrides` pattern. Supports `number` and `(input, ctx) => number`; `0` disables the per-atom timeout. New `TimeoutOverride` type exported from `tjs-lang/vm`. See `src/use-cases/timeout-overrides.test.ts`.\n- [x] **Replaced `vm.run` default `timeoutMs = fuel × 10ms` formula** — now derived from the registered atoms as `max(per-atom timeoutMs) × 2`, floored at 60s (`AgentVM.defaultRunTimeout`). A fixed 60s default (interim) was shorter than the 120s `llmVision`/`llmPredictBattery` budgets, so vision/LLM calls timed out mid-call on slower models; the atom-derived default always covers the slowest atom (and a chained pair) and self-adjusts to custom slow atoms. Updated timeout error message to point at `timeoutMs` / `timeoutOverrides` instead of \"increase fuel\".\n- [x] **`storeVectorize` / `storeVectorAdd` get `timeoutMs: 60000`** — both make embedding network calls but had the 1s atom default, so a cold embedding model timed out. (Same class as the llmVision/llmPredict 120s budgets, missed for the store atoms.) Local ops (`storeSearch`, `storeCreateCollection`) keep the default.\n\n- [x] **Vision-detection probe used a degenerate 1×1 PNG** — real vision preprocessors reject it (gemma-4-e4b: HTTP 400 \"Cannot handle this data type: (1,1,1)\"), so a genuinely multimodal model was false-negatived as `vision: false` and vision examples skipped with \"no vision model available\". Probe now uses a valid 32×32 PNG (gemma returns 200). `src/batteries/audit.ts`.\n\n### Deferred (surfaced this session)\n\n- [ ] **Model-audit vision detection still only checks `res.ok`** (`audit.ts` checkVision) — a text model that _tolerates_ the multimodal format without erroring would false-positive. Stronger: check the _response content_ (does the model actually describe the image?). Lower priority now that the 1×1 false-negative is fixed.\n\n## Infrastructure\n\n- [ ] Make playground components reusable for others\n- [ ] Web worker for transpiles (freezer - not needed yet)\n- [x] Retarget Firebase as host platform (vs GitHub Pages)\n- [ ] Universal LLM endpoint with real LLMs (OpenAI, Anthropic, etc.)\n- [ ] ESM-as-a-service: versioned library endpoints\n- [ ] User accounts (Google sign-in) for API key storage\n- [ ] AJS-based Firestore and Storage security rules\n- [ ] npx tjs-playground - run playground locally with LM Studio\n- [ ] Virtual subdomains for user apps (yourapp.tjs.land)\n - [ ] Wildcard DNS to Firebase\n - [ ] Subdomain routing in Cloud Function\n - [ ] Deploy button in playground\n - [ ] Public/private visibility toggle\n- [ ] Rate limiting / abuse prevention for LLM endpoint\n- [ ] Usage tracking / billing foundation (for future paid tiers)\n\n## Dependencies & Tooling\n\nFollow-ups from the ESLint 8 → 10 + typescript-eslint 5 → 8 flat-config migration:\n\n- [ ] **Decide package-lock.json policy.** Repo is bun-primary (bun.lock is canonical). The committed `package-lock.json` is stale (still references the old eslint v5 tree) and a fresh npm re-resolve balloons it by ~6k lines (full firebase-admin/google-cloud tree) and needs `--legacy-peer-deps` (pre-existing `tosijs-ui` wants `marked@^16` vs pinned `marked@9`). Either regenerate it in its own commit or remove it and let bun.lock be the sole lockfile.\n- [ ] **Clean up 22 pre-existing lint warnings** (unused vars/imports, prefer-const) — surfaced by `bun eslint src`, predate the migration (same `no-unused-vars`/`^_` config), all warnings not errors. Low-risk dead-code sweep across ~10 files.\n- [ ] **Dev-dependency vulns (none shipped to consumers).** `npm audit` shows 28, all dev/peer: Firebase SDK + admin stack, the vitest/vite/esbuild/rollup chain (vitest _critical_, genuinely used by 5 test files → needs v3 major), happy-dom, valibot, ws. Plus one eslint-transitive straggler: `flatted@3.3.3` via `file-entry-cache → flat-cache` (non-major fix).\n- [ ] **Resolve the `marked` peer conflict** — `tosijs-ui` peers on `marked@^16`, repo pins `marked@9.1.6` (bun warns + installs; npm refuses without `--legacy-peer-deps`).\n\n## Self-hosting (TS feature coverage)\n\nFour `it.skip` cases in `src/use-cases/self-hosting.test.ts` — advanced TS that the\nTS→TJS path can't yet handle. Un-skip as support lands:\n\n- [ ] Class with private fields and methods (gated on class support)\n- [ ] Builder pattern with method chaining (gated on class support)\n- [ ] Complex decorator patterns (requires `experimentalDecorators`)\n- [ ] Module augmentation (type-only, no runtime code)\n\n(Also 4 unconditional skips in `src/lang/metadata-cache.test.ts` — the transpile\nmetadata-cache feature is stubbed: store/retrieve, version-invalidation, merge,\nprune.)\n\n## Batteries / LLM tests\n\n- [ ] **Audit misclassifies models under concurrent probing.** Many test files call `LocalModels.audit()` at once, sharing one `.models.cache.json` (cwd, 24h TTL). Clearing the cache before a parallel `bun test` makes several audits probe LM Studio simultaneously and classifications come back scrambled (embedding models tagged `LLM`, an LLM tagged `Embedding`). Tests stay green only by luck of ordering. Fix: serialize the audit, harden the probes, or isolate the cache per run. Workaround documented in [`docs/lm-studio-setup.md`](docs/lm-studio-setup.md). Surfaced 2026-06-10 while getting the LLM suite green.\n\n---\n\n## Completed (this session)\n\n### Project Rename\n\n- [x] Rename from tosijs-agent to tjs-lang\n- [x] Update all references in package.json, docs, scripts\n\n### Timestamp & LegalDate Utilities\n\n- [x] Timestamp - pure functions, 1-based months, no Date warts (53 tests)\n - now, from, parse, tryParse\n - addDays/Hours/Minutes/Seconds/Weeks/Months/Years\n - diff, diffSeconds/Minutes/Hours/Days\n - year/month/day/hour/minute/second/millisecond/dayOfWeek\n - toLocal, format, formatDate, formatTime, toDate\n - isBefore/isAfter/isEqual/min/max\n - startOf/endOf Day/Month/Year\n- [x] LegalDate - pure functions, YYYY-MM-DD strings (55 tests)\n - today, todayIn, from, parse, tryParse\n - addDays/Weeks/Months/Years\n - diff, diffMonths, diffYears\n - year/month/day/dayOfWeek/weekOfYear/dayOfYear/quarter\n - isLeapYear, daysInMonth, daysInYear\n - toTimestamp, toUnix, fromUnix\n - format, formatLong, formatShort\n - isBefore/isAfter/isEqual/min/max/isBetween\n - startOf/endOf Month/Quarter/Year/Week\n- [x] Portable predicate helpers: isValidUrl, isValidTimestamp, isValidLegalDate\n\n### TJS Mode System (native TJS has all modes ON by default; TS-originated code defaults to OFF)\n\n- [x] Invert mode system - native TJS enables all modes; TS-originated/AJS code defaults to JS semantics\n- [x] TjsEquals directive - structural == and != (null == undefined)\n- [x] TjsClass directive - classes callable without new\n- [x] TjsDate directive - bans Date constructor/methods\n- [x] TjsNoeval directive - bans eval() and new Function()\n- [x] TjsStrict directive - enables all of the above\n- [x] TjsSafeEval directive - includes Eval/SafeFunction for dynamic code execution\n- [x] Updated Is() for nullish equality (null == undefined)\n- [x] Added Is/IsNot tests (structural equality, nullish handling)\n- [x] TjsStandard directive - newlines as statement terminators (prevents ASI footguns)\n- [x] WASM POC - wasm {} blocks with parsing, fallback mechanism, basic numeric compilation\n- [x] Eval/SafeFunction - proper VM-backed implementation with fuel metering and capabilities\n\n### Bundle Size Optimization\n\n- [x] Separated Eval/SafeFunction into standalone module (eval.ts)\n- [x] Created core.ts - AJS transpiler without TypeScript dependency\n- [x] Fixed tjs-transpiler bundle: 4.14MB → 88.9KB (27KB gzipped)\n- [x] Runtime is now ~5KB gzipped (just Is/IsNot, wrap, Type, etc.)\n- [x] Eval adds ~27KB gzipped (VM + AJS transpiler, no TypeScript)\n- [x] TypeScript only bundled in playground (5.8MB) for real-time TS transpilation\n"
|
|
1629
1629
|
},
|
|
1630
1630
|
{
|
|
1631
1631
|
"title": "TJS: Typed JavaScript",
|