keri-ts 0.2.2 → 0.3.0

package/README.md

@@ -1,150 +1,74 @@
-# keri-ts : KERI TypeScript Library
+# keri-ts
 
-An aspiring full implementation of the KERI, ACDC, CESR, and PTEL specifications
-in TypeScript.
+KERI TypeScript runtime package with the `tufa` CLI.
 
-### Usage
+## Install
 
 ```bash
-# Run the CLI
-deno task tufa --help
-
-# Initialize a keystore
-deno task tufa init --name mykeystore
-
-# Get help for specific commands
-deno task tufa init --help
-
-# Show CLI version
-deno task tufa version
-deno task tufa --version
-
-# Annotate a CESR file
-deno task tufa annotate --in samples/cesr-streams/CESR_1_0-oor-auth-vc.cesr
-
-# Annotate CESR from stdin
-cat samples/cesr-streams/CESR_1_0-oor-auth-vc.cesr | deno task tufa annotate
-```
+# As a dependency
+npm install keri-ts
 
-### Install CLI (`tufa`)
-
-```bash
-# npm (global)
+# Global CLI install
 npm install -g keri-ts
-tufa --help
-
-# Deno (global)
-deno install --global --allow-all --unstable-ffi --name tufa mod.ts
-tufa --help
-
-# make sure to add the Deno bin path to your PATH var
-export PATH="$HOME/.deno/bin:$PATH"
 ```
 
-### Features
+## CLI quick start
 
-- TODO Fully integrates with KERIpy, KERIA, and SignifyTS, both CESR 1.0 and
-  CESR 2.0
-- TODO Creates and manages keystores
-- TODO Parses and packs CESR streams
-- ✅ Provides CLI for interaction with KERI keystores (basic implementation)
-- TODO Provides a mailbox agent, controller agent, direct mode, and indirect
-  mode agents.
-- TODO Creates, Issues, and Verifies ACDC credentials.
-- TODO Provides a JSON Schema server to host ACDC schemas.
-
-### Importing library
-
-TBD
-
-### Build scripts (Deno)
+`tufa version` and `tufa annotate` are the fastest way to verify your setup.
 
 ```bash
-# Run CLI commands
-deno task tufa init --name test
+# Version
+tufa version
+tufa --version
+
+# Annotate CESR from file
+tufa annotate --in ./mystream.cesr
 
-# Start server (legacy)
-deno task start
+# Annotate from stdin
+cat ./mystream.cesr | tufa annotate
 ```
 
-### Versioning and Releases
+## Benchmark CESR parser
 
-- `keri-ts` and `cesr-ts` are versioned independently with Changesets.
-- Runtime CLI version strings are generated from package manifest versions.
-- CI builds automatically append build metadata to CLI display versions.
+Benchmark any CESR stream from file or stdin:
 
 ```bash
-# Add a patch/minor/major release intent file
-deno task release:changeset
+# Benchmark a stream file
+tufa benchmark cesr --in ./mystream.cesr
 
-# Apply version bumps/changelog updates from pending changesets
-deno task release:version
-
-# Regenerate and verify runtime version modules
-deno task version:generate
-deno task version:check
+# Benchmark from stdin with explicit chunk simulation
+cat ./mystream.cesr | tufa benchmark cesr --chunk-size 256 --iterations 25 --warmup 5
 ```
 
-### LMDB v1 Format Requirement
-
-**KERIpy Compatibility**: This project requires LMDB data format v1 for
-interoperability with KERIpy databases. The `lmdb-js` package defaults to LMDB
-v2 format, using v0.9.90 of LMDB, which is incompatible with databases created
-by KERIpy (which uses `py-lmdb` with LMDB 0.9.33).
+- `--in <path>`: input file path (defaults to stdin)
+- `--iterations <count>`: measured benchmark runs (default `50`)
+- `--warmup <count>`: warmup runs before measurement (default `5`)
+- `--chunk-size <bytes>`: chunk size for streaming simulation (`0` = full
+  stream)
+- `--framed`: benchmark parser in framed mode
+- `--compat`: benchmark parser with compat attachment dispatch mode
+- `--allow-errors`: continue benchmark even when parse errors are emitted
+- `--json`: print one JSON result line
 
-**Rebuilding lmdb with v1 support**:
-
-After installing dependencies, rebuild the `lmdb` package with v1 data format
-support:
+## Annotate options
 
 ```bash
-cd node_modules/lmdb
-export LMDB_DATA_V1=true
-npm run recompile
-cd ../..
+tufa annotate --in <path> --out <path> --pretty
+tufa annotate --qb2 --in <binary.qb2> --out <annotation.txt>
 ```
 
-**When to rebuild**:
-
-- After initial `npm install`
-- After updating the `lmdb` package
-- If you encounter `malloc` errors or database format incompatibility errors
-  when opening KERIpy-created databases
+- `--in <path>`: input file path (defaults to stdin)
+- `--out <path>`: output file path (defaults to stdout)
+- `--qb2`: parse input as qb2 binary
+- `--pretty`: pretty-print annotation output
 
-The `malloc` error looks like the following:
+## Package entrypoints
 
 ```bash
-node(42074,0x1f6bde0c0) malloc: *** error for object 0x3c7e805c00e0: pointer being freed was not allocated
-node(42074,0x1f6bde0c0) malloc: *** set a breakpoint in malloc_error_break to debug
-```
-
-**Note**: The v1 format uses LMDB 0.9.29 (via `lmdb-data-v1`), which is
-compatible with KERIpy's LMDB 0.9.33. Some newer LMDB features (encryption,
-remapping) are not available with v1 format.
-
-## Development Notes
-
-### Deno install locally
+# CLI help
+npx tufa --help
 
-`tufa` will show a few warnings locally unless you install it with the
-`--allow-scripts` option like below:
-
-```zsh
-deno install --global \
-  --config /Users/kbull/code/keri/kentbull/keri-ts/deno.json \
-  --node-modules-dir=auto \
-  --allow-scripts=npm:lmdb,npm:msgpackr-extract \
-  --allow-all \
-  --unstable-ffi \
-  --name tufa \
-  /Users/kbull/code/keri/kentbull/keri-ts/mod.ts
-```
-
-This is because of how Deno blocks build/install scripts for installed
-dependencies unless explicitly allowed.
-
-You may need to warm dependencies once with the following command:
-
-```zsh
-deno install --node-modules-dir=auto --allow-scripts=npm:lmdb,npm:msgpackr-extract
+# From source in this repo
+deno task tufa --help
+deno task tufa version
 ```

package/esm/app/cli/agent.js

@@ -1,19 +1,5 @@
 import { ValidationError } from "../../core/errors.js";
 import { startServer } from "../server.js";
-function printAgentHelp() {
-    console.log(`
-tufa agent - Start the KERI agent server
-
-Usage: tufa agent [options]
-
-Options:
-  --port, -p <port>           Port number for the server (default: 8000)
-  --help, -h                  Show this help message
-
-The agent server provides HTTP endpoints for interacting with keri-ts.
-The server will run until interrupted with SIGINT (Ctrl+C).
-`);
-}
 /**
  * Agent command operation - starts the HTTP server
  *
@@ -21,11 +7,6 @@ The server will run until interrupted with SIGINT (Ctrl+C).
  * @returns Operation that runs the server until shutdown
  */
 export function* agentCommand(args) {
-    // Check for help flag
-    if (args.help || args.h) {
-        printAgentHelp();
-        return;
-    }
     // Extract port from args (default to 8000)
     const port = args.port ? Number(args.port) : 8000;
     // Validate port number

package/esm/app/cli/annotate.js

@@ -1,13 +1,18 @@
 import * as dntShim from "../../_dnt.shims.js";
 import { annotate } from "cesr-ts";
 const TEXT_DECODER = new TextDecoder();
+/**
+ * Reads bytes from stdin in chunks of 64KB and returns them as a single Uint8Array
+ * @returns All bytes read from stdin
+ */
 function readAllStdinSync() {
     const chunks = [];
     let total = 0;
     while (true) {
-        const chunk = new Uint8Array(64 * 1024);
-        const read = dntShim.Deno.stdin.readSync(chunk);
+        const chunk = new Uint8Array(64 * 1024); // 64KB chunk size
+        const read = dntShim.Deno.stdin.readSync(chunk); // reads up to 64KB chunk, returns num bytes
         if (read === null) {
+            // read until EOF (null)
             break;
         }
         const used = chunk.subarray(0, read);

package/esm/app/cli/benchmark.js

@@ -0,0 +1,210 @@
+import * as dntShim from "../../_dnt.shims.js";
+import { createParser } from "cesr-ts";
+/**
+ * Synchronously consume stdin for CLI benchmark mode.
+ *
+ * Maintainer note:
+ * - Synchronous read keeps CLI command implementation simple and deterministic.
+ * - This function should remain isolated; parser benchmark internals themselves
+ *   are IO-agnostic.
+ */
+function readAllStdinSync() {
+    const chunks = [];
+    let total = 0;
+    while (true) {
+        const chunk = new Uint8Array(64 * 1024);
+        const read = dntShim.Deno.stdin.readSync(chunk);
+        if (read === null) {
+            break;
+        }
+        const used = chunk.subarray(0, read);
+        chunks.push(used);
+        total += read;
+    }
+    const output = new Uint8Array(total);
+    let offset = 0;
+    for (const chunk of chunks) {
+        output.set(chunk, offset);
+        offset += chunk.length;
+    }
+    return output;
+}
+/**
+ * Validate positive integer options with explicit field-scoped errors.
+ */
+function asPositiveInteger(value, fallback, fieldName) {
+    if (value === undefined) {
+        return fallback;
+    }
+    if (!Number.isFinite(value) || !Number.isInteger(value) || value <= 0) {
+        throw new Error(`${fieldName} must be a positive integer`);
+    }
+    return value;
+}
+/**
+ * Validate non-negative integer options with explicit field-scoped errors.
+ */
+function asNonNegativeInteger(value, fallback, fieldName) {
+    if (value === undefined) {
+        return fallback;
+    }
+    if (!Number.isFinite(value) || !Number.isInteger(value) || value < 0) {
+        throw new Error(`${fieldName} must be a non-negative integer`);
+    }
+    return value;
+}
+/**
+ * Normalize optional chunk sizing into a legal parse feed shape.
+ */
+function normalizeChunkSize(chunkSize, inputLength) {
+    if (chunkSize <= 0 || chunkSize >= inputLength) {
+        return inputLength;
+    }
+    return chunkSize;
+}
+/**
+ * Build ordered chunk views over the input stream.
+ *
+ * Invariant:
+ * - uses `subarray` so benchmark execution does not add avoidable copy overhead.
+ */
+function createChunks(input, chunkSize) {
+    if (chunkSize >= input.length) {
+        return [input];
+    }
+    const chunks = [];
+    for (let offset = 0; offset < input.length; offset += chunkSize) {
+        chunks.push(input.subarray(offset, Math.min(offset + chunkSize, input.length)));
+    }
+    return chunks;
+}
+/**
+ * Execute one complete parser pass and aggregate frame/error events.
+ */
+function parseStreamOnce(input, chunkSize, parserOptions) {
+    const parser = createParser(parserOptions);
+    let frameCount = 0;
+    let errorCount = 0;
+    for (const chunk of createChunks(input, chunkSize)) {
+        for (const event of parser.feed(chunk)) {
+            if (event.type === "frame") {
+                frameCount++;
+            }
+            else {
+                errorCount++;
+            }
+        }
+    }
+    for (const event of parser.flush()) {
+        if (event.type === "frame") {
+            frameCount++;
+        }
+        else {
+            errorCount++;
+        }
+    }
+    return { frameCount, errorCount };
+}
+/**
+ * Run timed parser benchmark loop for `tufa benchmark cesr`.
+ *
+ * Boundary contract:
+ * - warmup runs are excluded from timing.
+ * - correctness remains first: parse errors fail the benchmark unless
+ *   explicitly allowed.
+ */
+function benchmarkParser(input, options) {
+    if (input.length === 0) {
+        throw new Error("Benchmark input stream must not be empty");
+    }
+    const iterations = asPositiveInteger(options.iterations, 50, "iterations");
+    const warmupIterations = asNonNegativeInteger(options.warmupIterations, 5, "warmup");
+    const parsedChunkSize = asNonNegativeInteger(options.chunkSize, 0, "chunkSize");
+    const chunkSize = normalizeChunkSize(parsedChunkSize, input.length);
+    const failOnParseError = !(options.allowErrors ?? false);
+    const parserOptions = {
+        framed: options.framed ?? false,
+        attachmentDispatchMode: options.compat ? "compat" : "strict",
+    };
+    for (let i = 0; i < warmupIterations; i++) {
+        parseStreamOnce(input, chunkSize, parserOptions);
+    }
+    let totalFrames = 0;
+    let totalErrors = 0;
+    const startMs = performance.now();
+    for (let i = 0; i < iterations; i++) {
+        const summary = parseStreamOnce(input, chunkSize, parserOptions);
+        totalFrames += summary.frameCount;
+        totalErrors += summary.errorCount;
+        if (failOnParseError && summary.errorCount > 0) {
+            throw new Error(`Benchmark run produced parse errors (run=${i + 1}, errorCount=${summary.errorCount})`);
+        }
+    }
+    const elapsedMs = Math.max(performance.now() - startMs, Number.EPSILON);
+    const throughputMiBPerSec = ((input.length * iterations * 1000) / elapsedMs) /
+        (1024 * 1024);
+    const framesPerSec = (totalFrames * 1000) / elapsedMs;
+    return {
+        iterations,
+        warmupIterations,
+        bytesPerIteration: input.length,
+        chunkSize,
+        totalFrames,
+        totalErrors,
+        elapsedMs,
+        avgIterationMs: elapsedMs / iterations,
+        throughputMiBPerSec,
+        framesPerSec,
+    };
+}
+/**
+ * Render maintainer-oriented benchmark output for terminal use.
+ */
+function formatBenchmarkText(sourceLabel, result) {
+    const chunkLabel = result.chunkSize === result.bytesPerIteration
+        ? "full stream"
+        : `${result.chunkSize} bytes`;
+    return [
+        "CESR parser benchmark",
+        `source: ${sourceLabel}`,
+        `iterations: ${result.iterations} (warmup: ${result.warmupIterations})`,
+        `input bytes/iteration: ${result.bytesPerIteration}`,
+        `chunk size: ${chunkLabel}`,
+        `frames/iteration: ${(result.totalFrames / result.iterations).toFixed(2)}`,
+        `errors/iteration: ${(result.totalErrors / result.iterations).toFixed(2)}`,
+        `avg iteration: ${result.avgIterationMs.toFixed(3)} ms`,
+        `throughput: ${result.throughputMiBPerSec.toFixed(3)} MiB/s`,
+        `frame rate: ${result.framesPerSec.toFixed(2)} frames/s`,
+    ].join("\n");
+}
+// deno-lint-ignore require-yield
+/**
+ * Effection command handler for `tufa benchmark cesr`.
+ *
+ * Responsibility boundary:
+ * - parse command options from dispatch args,
+ * - read stream input from file/stdin,
+ * - execute benchmark and emit one result payload (text or JSON).
+ */
+export function* benchmarkCommand(args) {
+    const options = {
+        inPath: args.inPath,
+        iterations: args.iterations,
+        warmupIterations: args.warmupIterations,
+        chunkSize: args.chunkSize,
+        framed: args.framed,
+        compat: args.compat,
+        allowErrors: args.allowErrors,
+        json: args.json,
+    };
+    const sourceLabel = options.inPath ? options.inPath : "stdin";
+    const inputBytes = options.inPath
+        ? dntShim.Deno.readFileSync(options.inPath)
+        : readAllStdinSync();
+    const result = benchmarkParser(inputBytes, options);
+    if (options.json) {
+        console.log(JSON.stringify({ source: sourceLabel, ...result }));
+        return;
+    }
+    console.log(formatBenchmarkText(sourceLabel, result));
+}

package/esm/app/cli/cli.js

@@ -1,9 +1,8 @@
 import * as dntShim from "../../_dnt.shims.js";
 import { Command } from "commander";
 import { AppError } from "../../core/errors.js";
-import { createCoreCommandHandlers, registerCoreCommands, } from "./command-definitions.js";
-import { createStubCommandHandlers, isStubCommandsEnabled, registerStubCommands, } from "./stub-commands.js";
 import { DISPLAY_VERSION } from "../version.js";
+import { createCmdHandlers, registerCmds } from "./command-definitions.js";
 /**
  * Create the CLI program with action handlers that signal command execution.
  * Command declaration is delegated to focused command modules.
@@ -13,68 +12,67 @@ function createCLIProgram(onCommand) {
     program.name("tufa").version(DISPLAY_VERSION).description("Trust Utilities for Agents CLI");
     // Prevent Commander from exiting automatically so we can run Effection operations
     program.exitOverride();
-    registerCoreCommands(program, onCommand);
-    if (isStubCommandsEnabled()) {
-        registerStubCommands(program, onCommand);
-    }
+    registerCmds(program, onCommand);
     return program;
 }
+function parseCLIArgs(program, args) {
+    // Commander expects full argv or args array.
+    // In Deno, Deno.args gives us the arguments without executable info.
+    const argsToParse = args.length > 0 ? args : dntShim.Deno.args;
+    program.parse(argsToParse, { from: "user" });
+}
+function isCommanderExitError(error) {
+    return !!(error && typeof error === "object" && "code" in error);
+}
+function isExpectedCommanderExit(code) {
+    return (code === "commander.help" ||
+        code === "commander.helpDisplayed" ||
+        code === "commander.version" ||
+        code === "commander.unknownCommand" ||
+        code === "commander.missingArgument");
+}
+function handleParseError(error) {
+    if (isCommanderExitError(error) && isExpectedCommanderExit(error.code)) {
+        // Commander already printed any relevant help/error output.
+        return;
+    }
+    const message = error instanceof Error ? error.message : String(error);
+    if (error instanceof AppError && error.context) {
+        console.error(`Error: ${message}`, error.context);
+    }
+    else {
+        console.error(`Error: ${message}`);
+    }
+    throw error;
+}
+function* runCmd(selection, commandHandlers) {
+    if (!selection) {
+        return;
+    }
+    const handler = commandHandlers.get(selection.name);
+    if (!handler) {
+        return;
+    }
+    // Execute command operation within Effection's structured concurrency.
+    yield* handler(selection.args);
+}
 /**
  * Main CLI operation - runs within Effection's structured concurrency
  * This is the outermost runtime, not JavaScript's event loop
  */
 export function* tufa(args = []) {
-    const executionContext = {};
-    const commandHandlers = createCoreCommandHandlers();
-    if (isStubCommandsEnabled()) {
-        for (const [key, handler] of createStubCommandHandlers()) {
-            commandHandlers.set(key, handler);
-        }
-    }
+    const dispatch = {};
+    const commandHandlers = createCmdHandlers();
     // Use Commander.js for all command parsing
     const program = createCLIProgram((next) => {
-        executionContext.selection = next;
+        dispatch.selection = next;
     });
     try {
-        // Parse arguments - Commander expects full argv or args array
-        // In Deno, Deno.args gives us the arguments without the executable info
-        const argsToParse = args.length > 0 ? args : dntShim.Deno.args;
-        program.parse(argsToParse, { from: "user" });
+        parseCLIArgs(program, args);
     }
     catch (error) {
-        // Handle Commander-specific errors
-        if (error && typeof error === "object" && "code" in error) {
-            const commanderError = error;
-            // Help was requested - Commander already printed it, just return
-            if (commanderError.code === "commander.help" ||
-                commanderError.code === "commander.helpDisplayed" ||
-                commanderError.code === "commander.version") {
-                return;
-            }
-            // Unknown command or other parsing errors - Commander already printed the error
-            if (commanderError.code === "commander.unknownCommand" ||
-                commanderError.code === "commander.missingArgument") {
-                // Commander already printed the error message, just exit gracefully
-                return;
-            }
-        }
-        // For other errors, log and rethrow
-        const message = error instanceof Error ? error.message : String(error);
-        if (error instanceof AppError && error.context) {
-            console.error(`Error: ${message}`, error.context);
-        }
-        else {
-            console.error(`Error: ${message}`);
-        }
-        throw error;
-    }
-    // Execute the appropriate command operation based on context
-    const selected = executionContext.selection;
-    if (selected) {
-        const handler = commandHandlers.get(selected.name);
-        if (handler) {
-            // Execute the command operation within Effection's structured concurrency
-            yield* handler(selected.args);
-        }
+        handleParseError(error);
+        return;
     }
+    yield* runCmd(dispatch.selection, commandHandlers);
 }