markform 0.1.8 → 0.1.9

package/README.md

@@ -1,5 +1,8 @@
 # Markform
 
+[![CI](https://github.com/jlevy/markform/actions/workflows/ci.yml/badge.svg)](https://github.com/jlevy/markform/actions/workflows/ci.yml)
+![Coverage](./badges/coverage-total.svg)
+
 **Markform** is a text format for defining structured forms that humans can read,
 machines can parse, and agents can fill via tool calls.
 
@@ -113,21 +116,23 @@ Fields have types defined by the attributes.
 Values are filled in incrementally, just like any form.
 Once filled in, values appear directly inside the tags, in Markdown format:
 
-```jinja
+````jinja
 {% field kind="string" id="movie" label="Movie" role="user"
    required=true minLength=1 maxLength=300 %}
+```value
 The Shawshank Redemption
+```
 {% /field %}
 
 {% field kind="single_select" id="mpaa_rating" role="agent" label="MPAA Rating" %}
 - [ ] G {% #g %}
 - [ ] PG {% #pg %}
 - [ ] PG-13 {% #pg_13 %}
-- [X] R {% #r %}
+- [x] R {% #r %}
 - [ ] NC-17 {% #nc_17 %}
 - [ ] NR/Unrated {% #nr %}
 {% /field %}
-```
+````
 
 Note fields can have a `role="user"` to indicate they are filled interactively by the
 user, or a `role="agent"` to indicate an agent should fill them in.
@@ -283,7 +288,10 @@ Enter the movie title (add year or details for disambiguation).
 {% /field %}
 
 {% instructions ref="ratings_table" %}
-Fill in scores and vote counts from each source:IMDB: Rating (1.0-10.0 scale), vote countRT Critics: Tomatometer (0-100%), review countRT Audience: Audience Score (0-100%), rating count
+Fill in scores and vote counts from each source:
+- IMDB: Rating (1.0-10.0 scale), vote count
+- RT Critics: Tomatometer (0-100%), review count
+- RT Audience: Audience Score (0-100%), rating count
 {% /instructions %}
 
 {% /group %}
@@ -371,14 +379,37 @@ View them with `markform examples --list`, copy with `markform examples`, and ru
 - [`movie-research-demo.form.md`](https://github.com/jlevy/markform/blob/main/packages/markform/examples/movie-research/movie-research-demo.form.md)
   \- The quick example above.
 
-- [`movie-research-basic.form.md`](https://github.com/jlevy/markform/blob/main/packages/markform/examples/movie-research/movie-research-basic.form.md)
-  \- Standard movie research with IMDB, Rotten Tomatoes, Metacritic.
+- [`movie-deep-research.form.md`](https://github.com/jlevy/markform/blob/main/packages/markform/examples/movie-research/movie-deep-research.form.md)
+  \- Comprehensive movie analysis with streaming, box office, technical specs.
 
-- [`movie-research-deep.form.md`](https://github.com/jlevy/markform/blob/main/packages/markform/examples/movie-research/movie-research-deep.form.md)
-  \- Comprehensive movie analysis with streaming, box office, analysis.
+### Why a New Format?
+
+The closest alternatives I’ve seen are:
+
+- Plain Markdown docs can be used as templates and filled in by agents.
+  These are more expressive, but it is hard to edit them programmatically or use LLMs to
+  update them reliably.
+
+- JSON + JSON Schema which are good for struture but terrible for additional
+  unstructured context like instructions Markdown.
+
+- Agent to-do lists are part of many chat or coding interfaces and are programmatically
+  edited by agents. But these are limited to simple checklists, not forms with other
+  fields.
+
+- Numerous tools like Typeform, Google Forms, PDF forms, and Docusign offer
+  human-friendly UI. But these do not have a human-friendly text format for use by
+  agents as well as humans.
+
+| Approach | Usable GUI editor | Human-readable source format | Agent-editable | APIs and validation rules |
+| --- | :---: | :---: | :---: | :---: |
+| Plain Markdown | ✅<br>IDEs/editors | ✅ | ⚠️<br>fragile | ❌ |
+| JSON + JSON Schema | ✅<br>IDEs/editors | ⚠️<br>no free text | ✅ | ✅ |
+| SaaS tools (Typeform, Docusign, PDF forms) | ✅ | ⚠️<br>rarely | ⚠️<br>sometimes | ⚠️<br>sometimes |
+| HTML/web Forms | ✅<br>IDEs/editors | ⚠️<br>HTML+code | ⚠️<br>coding agent | ✅ |
+| Excel/Google Sheets | ✅<br>app | ❌<br>.csv/.xlsx | ⚠️<br>with tools | ✅<br>with some coding |
+| **Markform** | ✅<br>IDEs/editors | ✅ | ✅<br>with this package | ✅<br>with this package |
 
-- [`earnings-analysis.form.md`](https://github.com/jlevy/markform/blob/main/packages/markform/examples/earnings-analysis/earnings-analysis.form.md)
-  \- Financial analysis form.
 ## Architecture
 
 This repo has a specification and an implementation.
@@ -453,10 +484,8 @@ flowchart LR
 ```bash
 # Copy all bundled examples to ./forms/
 markform examples
-
 # List available examples
 markform examples --list
-
 # Copy a specific example
 markform examples --name movie-research-demo
 ```
@@ -466,7 +495,6 @@ markform examples --name movie-research-demo
 ```bash
 # Browse forms in ./forms/ and run one interactively
 markform run
-
 # Run a specific form directly
 markform run forms/movie-research-demo.form.md
 ```
@@ -483,7 +511,6 @@ markform status my-form.form.md
 ```bash
 # View form structure, progress, and validation issues
 markform inspect my-form.form.md
-
 # Output as JSON
 markform inspect my-form.form.md --format=json
 ```
@@ -493,10 +520,8 @@ markform inspect my-form.form.md --format=json
 ```bash
 # Interactive mode: fill user-role fields via prompts
 markform fill my-form.form.md --interactive
-
 # Agent mode: use an LLM to fill agent-role fields
 markform fill my-form.form.md --model=anthropic/claude-sonnet-4-5
-
 # Mock agent for testing (uses pre-filled form as source)
 markform fill my-form.form.md --mock --mock-source filled.form.md
 ```
@@ -506,13 +531,10 @@ markform fill my-form.form.md --mock --mock-source filled.form.md
 ```bash
 # Export as readable markdown (strips Markdoc tags)
 markform export my-form.form.md --format=markdown
-
 # Export values as JSON
 markform export my-form.form.md --format=json
-
 # Export values as YAML
 markform export my-form.form.md --format=yaml
-
 # Dump just the current values
 markform dump my-form.form.md
 ```
@@ -522,10 +544,8 @@ markform dump my-form.form.md
 ```bash
 # Export form structure as JSON Schema (for validation, code generation, etc.)
 markform schema my-form.form.md
-
 # Pure JSON Schema without Markform extensions
 markform schema my-form.form.md --pure
-
 # Specify JSON Schema draft version
 markform schema my-form.form.md --draft draft-07
 ```
@@ -549,19 +569,14 @@ markform serve my-form.form.md
 ```bash
 # Quick reference for writing forms (agent-friendly)
 markform docs
-
 # Full specification
 markform spec
-
 # TypeScript and AI SDK API documentation
 markform apis
-
 # This README
 markform readme
-
 # See supported AI providers and example models
 markform models
-
 # See all commands
 markform --help
 ```
@@ -594,17 +609,17 @@ applications.
 ### Basic Parsing
 
 ```typescript
-import { parseForm, serializeForm } from "markform";
+import { parseForm, serialize } from "markform";
 
 // Parse a .form.md file
 const form = parseForm(markdownContent);
 
 // Access schema and values
 console.log(form.schema.title);
-console.log(form.values);
+console.log(form.responsesByFieldId);
 
 // Serialize back to markdown
-const output = serializeForm(form);
+const output = serialize(form);
 ```
 
 ### AI SDK Integration
@@ -763,32 +778,6 @@ This enables powerful AI workflows that assemble information in a defined struct
 - An **agent execution harness** for step-by-step form filling, enabling deep research
   agents that assemble validated output in a structured format.
 
-### Does anything like this already exist?
-
-Not that I have seen.
-The closest alternatives are:
-
-- Plain Markdown docs can be used as templates and filled in by agents.
-  These are more expressive, but it is hard to edit them programmatically or use LLMs to
-  update them reliably.
-
-- Agent to-do lists are part of many chat or coding interfaces and are programmatically
-  edited by agents. But these are limited to simple checklists, not forms with other
-  fields.
-
-- Numerous tools like Typeform, Google Forms, PDF forms, and Docusign offer
-  human-friendly UI. But these do not have a human-friendly text format for use by
-  agents as well as humans.
-
-| Approach | Usable GUI editor | Human-readable source format | Agent-editable | APIs and validation rules |
-| --- | :---: | :---: | :---: | :---: |
-| Plain Markdown | ✅ IDEs/editors | ✅ | ⚠️ fragile | ❌ |
-| JSON + JSON Schema | ✅ IDEs/editors | ⚠️ no free text | ✅ | ✅ |
-| SaaS tools (Typeform, Docusign, PDF forms) | ✅ | ⚠️ rarely | ⚠️ sometimes | ⚠️ sometimes |
-| HTML/web Forms | ✅ IDEs/editors | ⚠️ HTML+code | ⚠️ coding agent | ✅ |
-| Excel/Google Sheets | ✅ app | ❌ .csv/.xlsx | ⚠️ with tools | ✅ with some coding |
-| **Markform** | ✅ IDEs/editors | ✅ | ✅ with this package | ✅ with this package |
-
 ### What are example use cases?
 
 - Deep research tools where agents need to follow codified processes to assemble

package/dist/ai-sdk.d.mts

@@ -1,5 +1,5 @@
 
-import { At as PatchSchema, Dt as ParsedForm, Ot as Patch, Q as Id, V as FieldResponse, dr as ValidatorRegistry, q as FormSchema, r as ApplyResult, rt as InspectResult } from "./coreTypes-BSPJ9H27.mjs";
+import { At as PatchSchema, Dt as ParsedForm, Ot as Patch, Q as Id, V as FieldResponse, dr as ValidatorRegistry, q as FormSchema, r as ApplyResult, rt as InspectResult } from "./coreTypes-JCPm418M.mjs";
 import { z } from "zod";
 
 //#region src/integrations/toolTypes.d.ts

package/dist/ai-sdk.mjs

@@ -1,6 +1,6 @@
 
-import { L as PatchSchema } from "./coreTypes-DJtu8OOp.mjs";
-import { d as serialize, i as inspect, t as applyPatches } from "./apply-BUU2QcJ2.mjs";
+import { L as PatchSchema } from "./coreTypes-B1oI7qvV.mjs";
+import { d as serialize, i as inspect, t as applyPatches } from "./apply-B2kt6C2z.mjs";
 import { z } from "zod";
 
 //#region src/integrations/vercelAiSdkTools.ts

package/dist/{apply-BUU2QcJ2.mjs → apply-B2kt6C2z.mjs}

@@ -789,6 +789,10 @@ function serializeField(field, responses) {
 		case "date": return serializeDateField(field, response);
 		case "year": return serializeYearField(field, response);
 		case "table": return serializeTableField(field, response);
+		default: {
+			const _exhaustive = field;
+			throw new Error(`Unhandled field kind: ${_exhaustive.kind}`);
+		}
 	}
 }
 /**
@@ -1250,6 +1254,10 @@ function isFieldSubmitted(field, value) {
 		}
 		case "year": return value.value !== null;
 		case "table": return value.rows.length > 0;
+		default: {
+			const _exhaustive = field;
+			throw new Error(`Unhandled field kind: ${_exhaustive.kind}`);
+		}
 	}
 }
 /**
@@ -1982,6 +1990,10 @@ function validateField(field, responses) {
 		case "date": return validateDateField(field, value);
 		case "year": return validateYearField(field, value);
 		case "table": return validateTableField(field, value);
+		default: {
+			const _exhaustive = field;
+			throw new Error(`Unhandled field kind: ${_exhaustive.kind}`);
+		}
 	}
 }
 /**
@@ -2130,7 +2142,7 @@ function inspect(form, options = {}) {
 	const structureSummary = computeStructureSummary(form.schema);
 	const progressSummary = computeProgressSummary(form.schema, form.responsesByFieldId, form.notes, validationInspectIssues);
 	const formState = computeFormState(progressSummary);
-	const issues = filterIssuesByRole(sortAndAssignPriorities(addOptionalEmptyIssues(validationInspectIssues, form, progressSummary.fields), form), form, options.targetRoles);
+	const issues = filterIssuesByRole(sortAndAssignPriorities(addOptionalUnansweredIssues(validationInspectIssues, form, progressSummary.fields), form), form, options.targetRoles);
 	return {
 		structureSummary,
 		progressSummary,
@@ -2153,19 +2165,29 @@ function convertValidationIssues(validationIssues, form) {
 	}));
 }
 /**
-* Add issues for empty optional fields that don't already have issues.
-* Fields that have been explicitly skipped do not get optional_empty issues.
+* Add issues for unanswered optional fields.
+*
+* An `optional_unanswered` issue is only added when:
+* - The field is optional (not required)
+* - The field has no value (empty=true)
+* - The field is unanswered (answerState='unanswered')
+*
+* Fields that have been addressed (answered, skipped, or aborted) do NOT get
+* optional_unanswered issues, even if their value is empty. For example:
+* - A multi_select answered with no selections (selected=[])
+* - A string_list answered with no items (items=[])
+* These are intentional "none" answers, not missing data.
 */
-function addOptionalEmptyIssues(existingIssues, form, fieldProgress) {
+function addOptionalUnansweredIssues(existingIssues, form, fieldProgress) {
 	const issues = [...existingIssues];
 	const fieldsWithIssues = new Set(existingIssues.map((i) => i.ref));
 	for (const [fieldId, progress] of Object.entries(fieldProgress)) {
-		if (progress.answerState === "skipped" || progress.answerState === "aborted") continue;
+		if (progress.answerState !== "unanswered") continue;
 		if (progress.empty && !fieldsWithIssues.has(fieldId) && !isRequiredField(fieldId, form)) issues.push({
 			ref: fieldId,
 			scope: "field",
-			reason: "optional_empty",
-			message: "Optional field has no value",
+			reason: "optional_unanswered",
+			message: "Optional field not yet addressed",
 			severity: "recommended",
 			priority: 0
 		});
@@ -2211,7 +2233,7 @@ function isRequiredField(fieldId, form) {
 * - validation_error: 2
 * - checkbox_incomplete: 3 (when required), 2 (when recommended)
 * - min_items_not_met: 2
-* - optional_empty: 1
+* - optional_unanswered: 1
 *
 * Total score = field_priority_weight + issue_type_score
 *
@@ -2232,7 +2254,7 @@ const ISSUE_TYPE_SCORES = {
 	validation_error: 2,
 	checkbox_incomplete: 2,
 	min_items_not_met: 2,
-	optional_empty: 1
+	optional_unanswered: 1
 };
 /**
 * Calculate the priority tier (1-5) from a score.
@@ -2257,7 +2279,7 @@ function getIssueTypeScore(reason, severity) {
 *
 * Priority is computed as a tier (1-5, P1-P5) based on:
 * - Field priority weight (high=3, medium=2, low=1)
-* - Issue type score (required_missing=3, validation_error=2, optional_empty=1)
+* - Issue type score (required_missing=3, validation_error=2, optional_unanswered=1)
 *
 * Within each tier, issues are sorted by severity (required first) then by ref.
 */
@@ -2860,7 +2882,7 @@ function convertToInspectIssues(form) {
 	for (const vi of result.issues) issues.push({
 		ref: vi.ref ?? "",
 		scope: "field",
-		reason: vi.severity === "error" ? "validation_error" : "optional_empty",
+		reason: vi.severity === "error" ? "validation_error" : "optional_unanswered",
 		message: vi.message,
 		severity: vi.severity === "error" ? "required" : "recommended",
 		priority: priority++
@@ -2911,4 +2933,4 @@ function applyPatches(form, patches) {
 }
 
 //#endregion
-export { deriveSchemaPath as A, DEFAULT_ROLES as C, USER_ROLE as D, REPORT_EXTENSION as E, formatSuggestedLlms as F, getWebSearchConfig as I, hasWebSearchSupport as L, parseRolesFlag as M, SUGGESTED_LLMS as N, deriveExportPath as O, WEB_SEARCH_CONFIG as P, parseModelIdForDisplay as R, DEFAULT_RESEARCH_MAX_PATCHES_PER_TURN as S, MAX_FORMS_IN_MENU as T, DEFAULT_MAX_PATCHES_PER_TURN as _, validate as a, DEFAULT_PRIORITY as b, computeProgressSummary as c, serialize as d, serializeRawMarkdown as f, DEFAULT_MAX_ISSUES_PER_TURN as g, DEFAULT_FORMS_DIR as h, inspect as i, detectFileType as j, deriveReportPath as k, computeStructureSummary as l, AGENT_ROLE as m, getAllFields as n, computeAllSummaries as o, serializeReportMarkdown as p, getFieldsForRoles as r, computeFormState as s, applyPatches as t, isFormComplete as u, DEFAULT_MAX_TURNS as v, DEFAULT_ROLE_INSTRUCTIONS as w, DEFAULT_RESEARCH_MAX_ISSUES_PER_TURN as x, DEFAULT_PORT as y };
+export { deriveReportPath as A, DEFAULT_RESEARCH_MAX_PATCHES_PER_TURN as C, REPORT_EXTENSION as D, MAX_FORMS_IN_MENU as E, WEB_SEARCH_CONFIG as F, formatSuggestedLlms as I, getWebSearchConfig as L, detectFileType as M, parseRolesFlag as N, USER_ROLE as O, SUGGESTED_LLMS as P, hasWebSearchSupport as R, DEFAULT_RESEARCH_MAX_ISSUES_PER_TURN as S, DEFAULT_ROLE_INSTRUCTIONS as T, DEFAULT_MAX_ISSUES_PER_TURN as _, validate as a, DEFAULT_PORT as b, computeProgressSummary as c, serialize as d, serializeRawMarkdown as f, DEFAULT_FORMS_DIR as g, ALL_EXTENSIONS as h, inspect as i, deriveSchemaPath as j, deriveExportPath as k, computeStructureSummary as l, AGENT_ROLE as m, getAllFields as n, computeAllSummaries as o, serializeReportMarkdown as p, getFieldsForRoles as r, computeFormState as s, applyPatches as t, isFormComplete as u, DEFAULT_MAX_PATCHES_PER_TURN as v, DEFAULT_ROLES as w, DEFAULT_PRIORITY as x, DEFAULT_MAX_TURNS as y, parseModelIdForDisplay as z };

package/dist/bin.mjs

@@ -1,7 +1,7 @@
 #!/usr/bin/env node
 
 
-import { t as runCli } from "./cli-BZh25bvy.mjs";
+import { t as runCli } from "./cli-Dt_PlYi_.mjs";
 import { resolve } from "node:path";
 import { existsSync } from "node:fs";
 import { config } from "dotenv";