@tangleai/linq 0.24.1

package/CHANGELOG.md

@@ -0,0 +1,27 @@
+# @tangleai/linq
+
+## 0.24.1
+
+No changes in this release.
+
+## 0.24.0
+
+No changes in this release.
+
+## 0.23.0
+
+No changes in this release.
+
+## 0.22.0
+
+### Minor Changes
+
+- Convert the migrated source, tests, benchmarks and hosts to strict TypeScript,
+  with JavaScript and declarations emitted through one release build. Move the
+  program pen from `@tangleai/jaren/program` and the Jaren integration barrel to
+  `@tangleai/linq/program`, preserving its JSON format and phantom binding types.
+  The new `@tangleai/linq` root exposes the program namespace and shared build error.
+
+  Match the embedder declarations to unknown widths before the first response,
+  retain precise ledger result variants, and enforce the refinement-pressure
+  instrument's stated 60-second deadline through the chat client's abort signal.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Joham (jklarenbeek@gmail.com)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,39 @@
+# @tangleai/linq
+
+Typed, immutable pens for Tangle AI document formats. A pen is a fluent authoring
+API whose deliverable is plain, deeply frozen JSON. It does not execute that
+document or duplicate its compiler. Shared snapshotting, option checks and query
+capture come from `@jarenjs/linq/authoring`.
+
+The package currently contains one pen, at `@tangleai/linq/program`:
+
+```ts
+import { program } from '@tangleai/linq/program';
+
+const document = program(['data'])
+  .select('data', 'count', value => value.get('items').count())
+  .answer('count')
+  .schema;
+```
+
+The root supports `import { program as p } from '@tangleai/linq'`, followed by
+`p.program(['data'])`. It also exports the same `LinqBuildError` class used by
+Jaren pens. Each future pen will have a separate subpath. There are no additional
+pens or query execution APIs in this package today; generic schema, model, flow,
+contract and other Jaren pens remain in `@jarenjs/linq`.
+
+The program pen tracks declared inputs, output names, slot/family kinds and the
+terminal answer in TypeScript. Its eight operations are `chunk`, `grep`,
+`select`, `stat`, `peek`, `map`, `reduce` and `answer`. Use `.schema` or `.toJSON()`
+to obtain the document, then validate/compile/run it with `@tangleai/agents`.
+Raw documents passed to `from()` make no binding-order or terminal-state inference.
+
+See [the program reference](docs/PROGRAM-PEN.md) for exact options, emitted JSON,
+refusals, type limits and executable examples. The package depends only on
+`@jarenjs/linq`; its pen bundle is checked to exclude runtime engines and Tangle
+model/context/agent mechanisms.
+
+Replace imports from `@tangleai/jaren/program` with `@tangleai/linq/program`.
+For pen symbols previously imported from the `@tangleai/jaren` barrel, import
+from `@tangleai/linq/program` directly. Workspace exports point to strict
+TypeScript; npm artifacts contain compiled ESM JavaScript and emitted declarations.

package/docs/PROGRAM-PEN.md

@@ -0,0 +1,102 @@
+# The AI program pen
+
+> `@tangleai/linq/program` — the public action program over environment slots. **Read it when**
+> you want typed fixtures or host-authored programs without a model client.
+
+## 1. What it writes
+
+The pen emits exactly `{ steps: [...] }`, the document consumed by the AI
+program schema, compiler and runner. It imports no AI engine or environment.
+`.schema` is an independent, deeply frozen JSON snapshot; `JSON.stringify()`
+writes that document. Updates return a new builder.
+
+`program(['corpus'])` declares input names for TypeScript; those names never
+become extra document members or an embedded registry. The compiler checks the
+real environment. Queries can be raw JSON or callbacks recorded over the input
+document, using the same capture as other pens and no runtime query engine.
+
+## 2. The mapping table
+
+Factories return frozen public steps. The same method on a program appends that
+step. Every non-answer step writes `op`, `from` and `as` before its options.
+
+| Factory / method | Emits | Type | Status |
+|---|---|---|---|
+| `program(slots?)` | empty `steps` | declared input names | native |
+| `chunk(from, as, options?)`, `.chunk(...)` | chunk; strategy and size | result family | native |
+| `grep(from, as, options)`, `.grep(...)` | grep; pattern, flags, limit | match-list slot | native |
+| `select(from, as, query)`, `.select(...)` | select; query JSON | result slot | native |
+| `stat(from, as)`, `.stat(...)` | stat | result slot | native |
+| `peek(from, as)`, `.peek(...)` | peek | result slot | native |
+| `map(from, as, prompt)`, `.map(...)` | map; bounded instruction | result family | native |
+| `reduce(from, as, query, options?)`, `.reduce(...)` | reduce; query JSON and optional `outputSchema` | result slot | native |
+| `answer(from, options?)`, `.answer(...)` | answer; optional chars, no as | terminal program | native |
+| `.step(step)` | appends a public step | tracks its input/result names | native |
+| `from(document)` | raw program | no binding-order inference | native |
+| `.schema`, `.toJSON()` | frozen public JSON | program document | native |
+
+`chunk` options are `strategy: 'size' | 'line' | 'separator'` and `size`.
+`grep` requires `pattern`, with optional `flags: 'i' | 'm' | 'im' | ''` and
+`limit`. `answer` takes `chars`. The schema owns numeric and string bounds.
+
+## 3. Worked examples
+
+```js
+import { program } from '@tangleai/linq/program';
+export const plan = program(['corpus'])
+  .chunk('corpus', 'pieces', { strategy: 'line', size: 200 })
+  .map('pieces', 'found', 'Return the number in this piece as JSON')
+  .reduce('found', 'count', { $count: '$[*]' })
+  .answer('count', { chars: 50 });
+```
+```json
+{"steps":[{"op":"chunk","from":"corpus","as":"pieces","strategy":"line","size":200},{"op":"map","from":"pieces","as":"found","prompt":"Return the number in this piece as JSON"},{"op":"reduce","from":"found","as":"count","query":{"$count":"$[*]"}},{"op":"answer","from":"count","chars":50}]}
+```
+
+Validate with `PROGRAM_SCHEMA` (or `programSchema({ queryRef })`), then
+`compileProgram`/`programGate` with the query compiler and known environment
+names. `createProgramRunner` does both shape and compile checks before execution.
+The standalone compile gate checks semantics; it does not replace the shape
+schema's numeric ranges, member closure or step-count cap.
+
+## 4. Refusals
+
+| Code | Condition |
+|---|---|
+| `JL0101` | invalid slot/result name shape, unknown option/operation, non-JSON input, or a malformed input-name list |
+| `JL0102` | appending a step after answer |
+| `JL0104` | a query callback reads an external; program queries bind only their input document |
+
+The schema/compiler own unknown environment names, duplicate result bindings,
+missing answers, limits, query validity and family/slot semantics. `from()`
+preserves raw programs for those checks rather than carrying a second compiler.
+
+## 5. The types
+
+`ProgramBuilder<Bindings, Done>` tracks names and whether answer has closed the
+program. `ProgramDocument`, `Step`, `StepDocument`, `StepOptions`, `Operation`,
+`Query`, `ChunkOptions`, `GrepOptions` and `AnswerOptions` are types only.
+
+A later step may read only a declared input or previous result. `select` and
+`answer` read individual slots, not chunk/map families; `reduce` reads map
+results. A result cannot be rebound, although an input slot can be shadowed as
+in the compiler. Standalone step factories retain literal names and `.step()`
+checks them on insertion. Raw documents claim neither binding order nor a
+terminal state. No inferred data schema, budget or successful-run guarantee is
+attached to a program.
+
+## 6. What it cannot spell
+
+This is the action document, not a prompt/client wrapper. It neither pastes
+corpus contents into a private envelope nor runs model calls in query expressions.
+Only the existing runner's map step calls a model. Cancellation, concurrency,
+sub-call budgets, storage and result interpretation remain runner concerns.
+
+## 7. Cost
+
+The isolated program pen costs **<!--fact:bundle.program-->15,691<!--/fact--> bytes**
+with Bun 1.4.0. `npm run test:tree-shaking` and the external packed consumer
+measure the same probe, enforce an 18,000-byte ceiling, and reject retained
+model mechanisms, runtime engines and the query chain. Nine modules remain.
+The original Jaren esbuild measurement was 15,472 bytes; the different bundler's
+output is a separate measurement, not a runtime performance comparison.

package/package.json

@@ -0,0 +1,48 @@
+{
+  "name": "@tangleai/linq",
+  "version": "0.24.1",
+  "description": "Typed, immutable document pens for Tangle AI, using the shared Jaren authoring primitives.",
+  "license": "MIT",
+  "type": "module",
+  "main": "./src/index.js",
+  "exports": {
+    ".": {
+      "types": "./src/index.d.ts",
+      "import": "./src/index.js",
+      "default": "./src/index.js"
+    },
+    "./program": {
+      "types": "./src/program.d.ts",
+      "import": "./src/program.js",
+      "default": "./src/program.js"
+    },
+    "./package.json": "./package.json"
+  },
+  "engines": {
+    "node": ">=24"
+  },
+  "sideEffects": false,
+  "dependencies": {
+    "@jarenjs/linq": "0.86.0"
+  },
+  "private": false,
+  "types": "./src/index.d.ts",
+  "files": [
+    "src/**/*.js",
+    "src/**/*.d.ts",
+    "docs/**/*.md",
+    "schemas/**/*.json",
+    "README.md",
+    "LICENSE",
+    "CHANGELOG.md"
+  ],
+  "publishConfig": {
+    "access": "public",
+    "registry": "https://registry.npmjs.org/"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/jklarenbeek/tangleai.git",
+    "directory": "packages/linq"
+  }
+}

package/src/errors.d.ts

@@ -0,0 +1,6 @@
+import { LinqBuildError as JarenLinqBuildError } from '@jarenjs/linq';
+/** The published coded constructor still inherits Error's signature in its declaration. */
+export declare const LinqBuildError: {
+    new (code: string, reason: string, docPath?: string, cause?: Error): JarenLinqBuildError;
+};
+export type LinqBuildError = JarenLinqBuildError;

package/src/errors.js

@@ -0,0 +1,3 @@
+import { LinqBuildError as JarenLinqBuildError } from '@jarenjs/linq';
+/** The published coded constructor still inherits Error's signature in its declaration. */
+export const LinqBuildError = JarenLinqBuildError;

package/src/index.d.ts

@@ -0,0 +1,3 @@
+/** Document authoring pens. Each pen also has its own subpath. */
+export * as program from './program.ts';
+export { LinqBuildError } from './errors.ts';

package/src/index.js

@@ -0,0 +1,3 @@
+/** Document authoring pens. Each pen also has its own subpath. */
+export * as program from "./program.js";
+export { LinqBuildError } from "./errors.js";

package/src/program.d.ts

@@ -0,0 +1,93 @@
+/** Immutable action documents; execution and semantic validation belong to @tangleai/agents. */
+import { DocumentBuilder } from '@jarenjs/linq/authoring';
+import type { Json } from '@jarenjs/linq/schema';
+import type { ExprBase, UnknownExpr } from '@jarenjs/linq';
+export type Query = Json | ((value: UnknownExpr) => ExprBase<unknown> | Json);
+export interface ChunkOptions {
+    readonly strategy?: 'size' | 'line' | 'separator';
+    readonly size?: number;
+}
+export interface GrepOptions {
+    readonly pattern: string;
+    readonly flags?: 'i' | 'm' | 'im' | '';
+    readonly limit?: number;
+}
+export interface AnswerOptions {
+    readonly chars?: number;
+}
+export interface ReduceOptions {
+    readonly outputSchema?: Json;
+}
+export interface StepOptions {
+    chunk: ChunkOptions;
+    grep: GrepOptions;
+    select: {
+        readonly query: Json;
+    };
+    stat: {};
+    peek: {};
+    map: {
+        readonly prompt: string;
+    };
+    reduce: {
+        readonly query: Json;
+    } & ReduceOptions;
+    answer: AnswerOptions;
+}
+export type Operation = keyof StepOptions;
+export type Step<K extends Operation, F extends string = string, N extends string = string> = {
+    readonly op: K;
+    readonly from: F;
+} & (K extends 'answer' ? {} : {
+    readonly as: N;
+}) & StepOptions[K];
+export type StepDocument = {
+    [K in Operation]: Step<K>;
+}[Operation];
+export interface ProgramDocument {
+    readonly steps: readonly StepDocument[];
+}
+type Kind = Exclude<Operation, 'answer'> | 'slot';
+type Bindings = Record<string, Kind>;
+type Names<B, K extends Kind = Kind> = {
+    [N in keyof B]: B[N] extends K ? N : never;
+}[keyof B] & string;
+type Single<B> = Names<B, Exclude<Kind, 'chunk' | 'map'>>;
+type Fresh<B, N extends string> = N extends keyof B ? B[N] extends 'slot' ? N : never : N;
+type Add<B, N extends string, K extends Kind> = Omit<B, N> & Record<N, K>;
+type InputFor<B, K extends Operation> = K extends 'reduce' ? Names<B, 'map'> : K extends 'select' | 'answer' ? Single<B> : Names<B>;
+type StepKeys<K extends Operation> = 'op' | 'from' | (K extends 'answer' ? never : 'as') | keyof StepOptions[K];
+type CheckStep<B, T extends StepDocument> = Exclude<keyof T, StepKeys<T['op']>> extends never ? T['from'] extends InputFor<B, T['op']> ? T extends {
+    readonly as: infer N extends string;
+} ? N extends Fresh<B, N> ? unknown : never : unknown : never : never;
+type AfterStep<B, T extends StepDocument> = T extends {
+    readonly as: infer N extends string;
+    readonly op: infer K extends Kind;
+} ? Add<B, N, K> : B;
+export declare function chunk<const F extends string, const N extends string>(from: F, as: N, options?: ChunkOptions): Step<'chunk', F, N>;
+export declare function grep<const F extends string, const N extends string>(from: F, as: N, options: GrepOptions): Step<'grep', F, N>;
+export declare function select<const F extends string, const N extends string>(from: F, as: N, query: Query): Step<'select', F, N>;
+export declare function stat<const F extends string, const N extends string>(from: F, as: N): Step<'stat', F, N>;
+export declare function peek<const F extends string, const N extends string>(from: F, as: N): Step<'peek', F, N>;
+export declare function map<const F extends string, const N extends string>(from: F, as: N, prompt: string): Step<'map', F, N>;
+export declare function reduce<const F extends string, const N extends string>(from: F, as: N, query: Query, options?: ReduceOptions): Step<'reduce', F, N>;
+export declare function answer<const F extends string>(from: F, options?: AnswerOptions): Step<'answer', F>;
+/** Binding kinds and terminal state are phantom types: neither enters the JSON document. */
+export declare class ProgramBuilder<B extends Bindings = {}, Done extends boolean = false> extends DocumentBuilder<ProgramDocument> {
+    readonly __bindings: B;
+    readonly __done: Done;
+    step<const T extends StepDocument>(this: ProgramBuilder<B, false>, step: T & CheckStep<B, T>): ProgramBuilder<AfterStep<B, T>, T['op'] extends 'answer' ? true : false>;
+    chunk<N extends string>(this: ProgramBuilder<B, false>, from: Names<B>, as: Fresh<B, N>, options?: ChunkOptions): ProgramBuilder<Add<B, N, 'chunk'>>;
+    grep<N extends string>(this: ProgramBuilder<B, false>, from: Names<B>, as: Fresh<B, N>, options: GrepOptions): ProgramBuilder<Add<B, N, 'grep'>>;
+    select<N extends string>(this: ProgramBuilder<B, false>, from: Single<B>, as: Fresh<B, N>, query: Query): ProgramBuilder<Add<B, N, 'select'>>;
+    stat<N extends string>(this: ProgramBuilder<B, false>, from: Names<B>, as: Fresh<B, N>): ProgramBuilder<Add<B, N, 'stat'>>;
+    peek<N extends string>(this: ProgramBuilder<B, false>, from: Names<B>, as: Fresh<B, N>): ProgramBuilder<Add<B, N, 'peek'>>;
+    map<N extends string>(this: ProgramBuilder<B, false>, from: Names<B>, as: Fresh<B, N>, prompt: string): ProgramBuilder<Add<B, N, 'map'>>;
+    reduce<N extends string>(this: ProgramBuilder<B, false>, from: Names<B, 'map'>, as: Fresh<B, N>, query: Query, options?: ReduceOptions): ProgramBuilder<Add<B, N, 'reduce'>>;
+    answer(this: ProgramBuilder<B, false>, from: Single<B>, options?: AnswerOptions): ProgramBuilder<B, true>;
+}
+/** Input names exist only in the binding type; they never become document members. */
+export declare function program<const S extends readonly string[] = []>(slots?: S): ProgramBuilder<Record<S[number], 'slot'>>;
+/** Raw documents make no binding-order or terminal-state inference. */
+export declare function from(document: ProgramDocument): ProgramBuilder<Record<string, Kind>, boolean>;
+export {};

package/src/program.js

@@ -0,0 +1,72 @@
+/** Immutable action documents; execution and semantic validation belong to @tangleai/agents. */
+import { DocumentBuilder, optionsOf, snapshot, captureQuery } from '@jarenjs/linq/authoring';
+import { LinqBuildError } from "./errors.js";
+const FIELDS = {
+    chunk: ['strategy', 'size'], grep: ['pattern', 'flags', 'limit'], select: ['query'],
+    stat: [], peek: [], map: ['prompt'], reduce: ['query', 'outputSchema'], answer: ['chars'],
+};
+function make(op, from, as, options) {
+    if (typeof from !== 'string' || !from || (op !== 'answer' && (typeof as !== 'string' || !as)))
+        throw new LinqBuildError('JL0101', 'A step needs a source name and, except answer, a result name');
+    return snapshot({ op, from, ...(op === 'answer' ? {} : { as }), ...optionsOf(options, FIELDS[op], `${op}()`) });
+}
+function expression(value) {
+    // Only the input document is bound; external captures retain the shared JL0104 refusal.
+    return (typeof value === 'function' ? captureQuery('program query', [], value, { fold: false }) : value);
+}
+export function chunk(from, as, options = {}) { return make('chunk', from, as, options); }
+export function grep(from, as, options) { return make('grep', from, as, options); }
+export function select(from, as, query) { return make('select', from, as, { query: expression(query) }); }
+export function stat(from, as) { return make('stat', from, as, {}); }
+export function peek(from, as) { return make('peek', from, as, {}); }
+export function map(from, as, prompt) { return make('map', from, as, { prompt }); }
+export function reduce(from, as, query, options = {}) { return make('reduce', from, as, { query: expression(query), ...optionsOf(options, ['outputSchema'], 'reduce()') }); }
+export function answer(from, options = {}) { return make('answer', from, undefined, options); }
+/** Binding kinds and terminal state are phantom types: neither enters the JSON document. */
+export class ProgramBuilder extends DocumentBuilder {
+    step(step) {
+        return append(this, step);
+    }
+    chunk(from, as, options = {}) {
+        return append(this, chunk(from, as, options));
+    }
+    grep(from, as, options) {
+        return append(this, grep(from, as, options));
+    }
+    select(from, as, query) {
+        return append(this, select(from, as, query));
+    }
+    stat(from, as) {
+        return append(this, stat(from, as));
+    }
+    peek(from, as) {
+        return append(this, peek(from, as));
+    }
+    map(from, as, prompt) {
+        return append(this, map(from, as, prompt));
+    }
+    reduce(from, as, query, options = {}) {
+        return append(this, reduce(from, as, query, options));
+    }
+    answer(from, options = {}) {
+        return append(this, answer(from, options));
+    }
+}
+/** Input names exist only in the binding type; they never become document members. */
+export function program(slots = []) {
+    if (!Array.isArray(slots) || slots.some((name) => typeof name !== 'string' || !name))
+        throw new LinqBuildError('JL0101', 'program() takes input slot names');
+    return new ProgramBuilder({ steps: [] });
+}
+/** Raw documents make no binding-order or terminal-state inference. */
+export function from(document) {
+    return new ProgramBuilder(document);
+}
+function append(builder, value) {
+    if (builder.schema.steps.at(-1)?.op === 'answer')
+        throw new LinqBuildError('JL0102', 'answer is terminal; start another program to append work');
+    if (value === null || typeof value !== 'object' || !Object.hasOwn(FIELDS, value.op))
+        throw new LinqBuildError('JL0101', 'step() needs a known program operation');
+    const step = optionsOf(value, ['op', 'from', ...(value.op === 'answer' ? [] : ['as']), ...FIELDS[value.op]], 'step()');
+    return builder.with({ steps: [...builder.schema.steps, step] });
+}