tryscript 0.1.6 → 0.1.7

package/README.md

@@ -1,25 +1,43 @@
 # tryscript
 
-[![CI](https://github.com/jlevy/tryscript/actions/workflows/ci.yml/badge.svg)](https://github.com/jlevy/tryscript/actions/runs/21106289057)
-[![Coverage](https://raw.githubusercontent.com/jlevy/tryscript/main/badges/packages/tryscript/coverage-total.svg)](https://github.com/jlevy/tryscript/actions/runs/21106289057)
+[![Follow @ojoshe on X](https://img.shields.io/badge/follow_%40ojoshe-black?logo=x&logoColor=white)](https://x.com/ojoshe)
+[![CI](https://github.com/jlevy/tryscript/actions/workflows/ci.yml/badge.svg)](https://github.com/jlevy/tryscript/actions/runs/22267140455)
+[![Coverage](https://raw.githubusercontent.com/jlevy/tryscript/main/badges/packages/tryscript/coverage-total.svg)](https://github.com/jlevy/tryscript/actions/runs/22267140455)
 [![npm version](https://img.shields.io/npm/v/tryscript)](https://www.npmjs.com/package/tryscript)
-[![X Follow](https://img.shields.io/twitter/follow/ojoshe)](https://x.com/ojoshe)
 
-Golden testing for CLI applications - a TypeScript port of [trycmd](https://github.com/assert-rs/trycmd).
+**Powerful, agent-friendly testing of CLI applications via golden tests**
 
 > [!NOTE]
 > 100% of the code and specs in this repository were written by Claude Code.
 > The design and management and prompting was by me ([jlevy](https://github.com/jlevy)) supported by  the workflows, agent rules,
-> and other research docs in [Speculate](https://github.com/jlevy/speculate).
+> and other research docs in [tbd](https://github.com/jlevy/tbd).
 > 
-> You can see what you think, but I find the code quality higher than most agent-written code I've
+> I find the code quality higher than most agent-written code I've
 > seen because of the spec-driven process.
 > You can review the architecture doc and all of the specs all of the specs in [docs/project](docs/project).
 > The general research, guideline, and rules docs I use are in [docs/general](docs/general).
 
+## Why?
+
+Write CLI tests as Markdown. tryscript runs commands, captures output, and compares against expected results:
+
+- Tests are clear and maintainable for agents and humans: tests become documentation; documentation becomes tests
+- Inner state and working can be exposed for greater test coverage at no extra cost
+- Things are quick to implement or test using arbitrary shell commands
+
+This began as a TypeScript port of [trycmd](https://github.com/assert-rs/trycmd) but I (well, Claude and friends)
+have since enhanced it to be more agent-friendly and self-documenting as a CLI.
+
+For a bit more philosophy on why golden tests are so useful, you (or your friendly agent)
+should read [tbd](https://github.com/jlevy/tbd)’s guidelines doc:
+
+```bash
+npx --yes get-tbd@latest guidelines golden-testing-guidelines
+```
+
 ## What It Does
 
-Write CLI tests as Markdown. tryscript runs commands, captures output, and compares against expected results. Tests become documentation; documentation becomes tests.
+An example test:
 
 ````markdown
 ---
@@ -68,6 +86,16 @@ $ my-cli process data.json > output.txt && grep "success" output.txt
 
 The `[..]` matches any text on that line. The `...` matches zero or more lines. These "elision patterns" let tests handle dynamic output gracefully. Any shell command works - pipes, redirects, environment variables, etc.
 
+### Wildcard Categories
+
+Tryscript supports three categories of wildcards, in order of preference:
+
+1. **Named patterns** (`[HASH]`, `[VERSION]`, `[CWD]`, etc.) -- Typed dynamic values with specific meaning. Preferred when the output has a known structure.
+2. **Unknown wildcards** (`[??]`, `???`) -- Temporary placeholders for output you haven't filled in yet. Intended to be expanded with `--expand` before finalizing tests.
+3. **Generic wildcards** (`[..]`, `...`) -- Intentional omission of unpredictable or irrelevant output. Use when the exact value doesn't matter for the test.
+
+Use `--expand` to automatically fill in unknown wildcards with actual output after a successful run.
+
 ## Quick Start
 
 ```bash
@@ -90,7 +118,8 @@ npx tryscript run --update tests/
 ## Features
 
 - **Markdown format** - Tests are readable documentation
-- **Elision patterns** - Handle variable output: `[..]`, `...`, `[CWD]`, `[ROOT]`, `[EXE]`
+- **Elision patterns** - Handle variable output: `[..]`, `...`, `[??]`, `???`, `[CWD]`, `[ROOT]`, `[EXE]`
+- **Wildcard expansion** - Fill in `[??]`/`???` placeholders with actual output via `--expand`
 - **Custom patterns** - Define regex patterns for timestamps, versions, UUIDs
 - **Update mode** - Regenerate expected output with `--update`
 - **Sandbox mode** - Isolate tests in temp directories
@@ -113,6 +142,10 @@ For complete syntax reference, run `tryscript docs` or see the [reference docume
 | Option | Description |
 | --- | --- |
 | `--update` | Update test files with actual output |
+| `--expand` | Expand unknown wildcards (`???`/`[??]`) with actual output |
+| `--expand-generic` | Expand unknown + generic wildcards |
+| `--expand-all` | Expand all wildcards (including named patterns) |
+| `--capture-log <path>` | Write wildcard capture log to YAML file |
 | `--fail-fast` | Stop on first failure |
 | `--filter <regex>` | Filter tests by name |
 | `--verbose` | Show detailed output |

package/dist/bin.cjs

@@ -1,20 +1,20 @@
 #!/usr/bin/env node
 
 
-const require_src = require('./src-1oEnK7GG.cjs');
+const require_src = require('./src-BIZMxxIt.cjs');
 let node_url = require("node:url");
 let node_fs = require("node:fs");
 let node_path = require("node:path");
 let node_child_process = require("node:child_process");
 let node_fs_promises = require("node:fs/promises");
 let node_os = require("node:os");
+let atomically = require("atomically");
 let commander = require("commander");
 let fast_glob = require("fast-glob");
 fast_glob = require_src.__toESM(fast_glob);
 let picocolors = require("picocolors");
 picocolors = require_src.__toESM(picocolors);
 let diff = require("diff");
-let atomically = require("atomically");
 
 //#region src/cli/lib/shared.ts
 /**
@@ -136,9 +136,10 @@ function reportSummary(summary, _options) {
 async function updateTestFile(file, results) {
 	let content = file.rawContent;
 	const changes = [];
-	const blocksWithResults = file.blocks.map((block, i) => ({
+	const resultByBlock = new Map(results.map((result) => [result.block, result]));
+	const blocksWithResults = [...file.blocks].map((block) => ({
 		block,
-		result: results[i]
+		result: resultByBlock.get(block)
 	})).reverse();
 	for (const { block, result } of blocksWithResults) {
 		if (!result) continue;
@@ -161,12 +162,14 @@ async function updateTestFile(file, results) {
 * Build an updated console block with new expected output.
 */
 function buildUpdatedBlock(block, result) {
-	const lines = ["```console", ...block.command.split("\n").map((line, i) => {
+	const fence = "`".repeat(/^(`+)/.exec(block.rawContent)?.[1]?.length ?? 3);
+	const commandLines = block.command.split("\n").map((line, i) => {
 		return i === 0 ? `$ ${line}` : `> ${line}`;
-	})];
+	});
+	const lines = [`${fence}console`, ...commandLines];
 	const trimmedOutput = result.actualOutput.trimEnd();
 	if (trimmedOutput) lines.push(trimmedOutput);
-	lines.push(`? ${result.actualExitCode}`, "```");
+	lines.push(`? ${result.actualExitCode}`, fence);
 	return lines.join("\n");
 }
 
@@ -540,10 +543,32 @@ function mergeExternalCoverage(reportsDir, externalLcovPath) {
 * Register the run command.
 */
 function registerRunCommand(program) {
-	program.command("run").description("Run golden tests").argument("[files...]", "Test files to run (default: **/*.tryscript.md)").option("--update", "Update golden files with actual output").option("--diff", "Show diff on failure (default: true)").option("--no-diff", "Hide diff on failure").option("--fail-fast", "Stop on first failure").option("--filter <pattern>", "Filter tests by name pattern").option("--verbose", "Show detailed output including passing test output").option("--quiet", "Suppress non-essential output (only show failures)").option("--coverage", "Enable code coverage collection (requires c8)").option("--coverage-dir <dir>", "Coverage output directory (default: coverage-tryscript)").option("--coverage-reporter <reporter...>", "Coverage reporters (default: text, html). Can be specified multiple times.").option("--coverage-exclude <pattern...>", "Patterns to exclude from coverage (c8 --exclude). Can be specified multiple times.").option("--coverage-exclude-node-modules", "Exclude node_modules from coverage (c8 --exclude-node-modules, default: true)").option("--no-coverage-exclude-node-modules", "Include node_modules in coverage (c8 --no-exclude-node-modules)").option("--coverage-exclude-after-remap", "Apply exclude logic after sourcemap remapping (c8 --exclude-after-remap)").option("--coverage-skip-full", "Hide files with 100% coverage (c8 --skip-full)").option("--coverage-allow-external", "Allow files from outside cwd (c8 --allowExternal)").option("--coverage-monocart", "Use monocart for accurate line counts, better for merging with vitest (c8 --experimental-monocart)").option("--merge-lcov <path>", "Merge coverage from an existing LCOV file (e.g., from vitest --coverage)").action(runCommand$1);
+	program.command("run").description("Run golden tests").argument("[files...]", "Test files to run (default: **/*.tryscript.md)").option("--update", "Update golden files with actual output").option("--diff", "Show diff on failure (default: true)").option("--no-diff", "Hide diff on failure").option("--fail-fast", "Stop on first failure").option("--filter <pattern>", "Filter tests by name pattern").option("--verbose", "Show detailed output including passing test output").option("--quiet", "Suppress non-essential output (only show failures)").option("--expand", "Expand unknown wildcards (??? and [??]) with actual output").option("--expand-generic", "Expand unknown and generic wildcards with actual output").option("--expand-all", "Expand all wildcards (including named patterns) with actual output").option("--capture-log <path>", "Write wildcard capture log to YAML file").option("--coverage", "Enable code coverage collection (requires c8)").option("--coverage-dir <dir>", "Coverage output directory (default: coverage-tryscript)").option("--coverage-reporter <reporter...>", "Coverage reporters (default: text, html). Can be specified multiple times.").option("--coverage-exclude <pattern...>", "Patterns to exclude from coverage (c8 --exclude). Can be specified multiple times.").option("--coverage-exclude-node-modules", "Exclude node_modules from coverage (c8 --exclude-node-modules, default: true)").option("--no-coverage-exclude-node-modules", "Include node_modules in coverage (c8 --no-exclude-node-modules)").option("--coverage-exclude-after-remap", "Apply exclude logic after sourcemap remapping (c8 --exclude-after-remap)").option("--coverage-skip-full", "Hide files with 100% coverage (c8 --skip-full)").option("--coverage-allow-external", "Allow files from outside cwd (c8 --allowExternal)").option("--coverage-monocart", "Use monocart for accurate line counts, better for merging with vitest (c8 --experimental-monocart)").option("--merge-lcov <path>", "Merge coverage from an existing LCOV file (e.g., from vitest --coverage)").action(runCommand$1);
+}
+/**
+* Count unknown wildcard tokens (`???` and `[??]`) in expected output.
+*/
+function countUnknownWildcards(expectedOutput) {
+	return (expectedOutput.match(/\[\?\?]/g) ?? []).length + (expectedOutput.match(/\?\?\?\n/g) ?? []).length;
 }
 async function runCommand$1(files, options) {
 	const startTime = Date.now();
+	if ([
+		options.expand,
+		options.expandGeneric,
+		options.expandAll
+	].filter(Boolean).length > 1) {
+		logError("--expand, --expand-generic, and --expand-all are mutually exclusive");
+		process.exit(1);
+	}
+	let expandLevel;
+	if (options.expand) expandLevel = "unknown";
+	else if (options.expandGeneric) expandLevel = "generic";
+	else if (options.expandAll) expandLevel = "all";
+	if (expandLevel && options.update) {
+		logError("--expand* flags and --update are mutually exclusive");
+		process.exit(1);
+	}
 	const opts = {
 		diff: options.diff !== false,
 		verbose: options.verbose ?? false,
@@ -593,6 +618,8 @@ async function runCommand$1(files, options) {
 		coverageEnv = getCoverageEnv(coverageCtx);
 	}
 	const fileResults = [];
+	const fileContexts = /* @__PURE__ */ new Map();
+	const filePatterns = /* @__PURE__ */ new Map();
 	let shouldStop = false;
 	for (const filePath of testFiles) {
 		if (shouldStop) break;
@@ -608,6 +635,7 @@ async function runCommand$1(files, options) {
 		if (blocksToRun.length === 0) continue;
 		const ctx = await require_src.createExecutionContext(config, filePath, coverageEnv);
 		const results = [];
+		let fileContext;
 		try {
 			for (const block of blocksToRun) {
 				const result = await require_src.runBlock(block, ctx);
@@ -634,6 +662,12 @@ async function runCommand$1(files, options) {
 				}
 			}
 			await require_src.runAfterHook(ctx);
+			fileContext = {
+				root: ctx.testDir,
+				cwd: ctx.cwd
+			};
+			fileContexts.set(filePath, fileContext);
+			filePatterns.set(filePath, config.patterns ?? {});
 		} finally {
 			await require_src.cleanupExecutionContext(ctx);
 		}
@@ -649,7 +683,14 @@ async function runCommand$1(files, options) {
 			const { updated, changes } = await updateTestFile(testFile, results);
 			if (updated) console.error(colors.warn(`  ${status.update} Updated: ${changes.join(", ")}`));
 		}
+		if (expandLevel && fileContext) {
+			const { expanded, expandedCount, changes } = await require_src.expandTestFile(testFile, results, expandLevel, fileContext, config.patterns ?? {});
+			if (expanded) console.error(colors.warn(`  ${status.update} Expanded ${expandedCount} wildcard(s): ${changes.join(", ")}`));
+		}
 	}
+	let totalUnknownWildcards = 0;
+	for (const fr of fileResults) for (const block of fr.file.blocks) totalUnknownWildcards += countUnknownWildcards(block.expectedOutput);
+	if (totalUnknownWildcards > 0) logWarn(`${totalUnknownWildcards} unknown wildcard(s) found (??? or [??]). These are temporary and should be expanded. Use --expand to fill them in.`);
 	const summary = {
 		files: fileResults,
 		totalPassed: fileResults.reduce((sum, f) => sum + f.results.filter((r) => r.passed).length, 0),
@@ -658,6 +699,15 @@ async function runCommand$1(files, options) {
 		duration: Date.now() - startTime
 	};
 	reportSummary(summary, opts);
+	if (options.captureLog) try {
+		await require_src.writeCaptureLog(options.captureLog, fileResults, (file) => fileContexts.get(file.path) ?? {
+			root: process.cwd(),
+			cwd: process.cwd()
+		}, (file) => filePatterns.get(file.path) ?? {});
+		console.error(colors.info(`Capture log written to ${options.captureLog}`));
+	} catch (error) {
+		logError(`Failed to write capture log: ${error instanceof Error ? error.message : String(error)}`);
+	}
 	if (coverageCtx) {
 		console.error("\nGenerating coverage report...");
 		try {