flex-md 4.6.0 → 4.7.0

package/README.md

@@ -170,14 +170,72 @@ Compliance levels measure how much detail you provide in system parts:
 "Lines: ~50. Provide a complete working example."
 ```
 
+## Input vs Output Format Specs
+
+Flex-MD supports both **Input Format Specs** and **Output Format Specs**, but they serve different purposes:
+
+### Input Format Specs (Planning & Design)
+
+Input Format Specs (defined with `## Input format`) are **design-time tools** for planning and documentation:
+
+- ✅ **Documentation**: Clearly specify what input structure your LLM expects
+- ✅ **Planning**: Help design your prompt structure and data flow
+- ✅ **Coding**: Guide developers on how to structure inputs
+- ✅ **Validation**: Can be used to validate input data (optional)
+
+**Input format specs are NOT used for runtime token estimation** — they're for human understanding and system design.
+
+### Output Format Specs (Runtime)
+
+Output Format Specs (defined with `## Output format`) are **runtime tools**:
+
+- ✅ **Token Estimation**: Used to calculate `max_tokens` for API calls
+- ✅ **Validation**: Enforce structure on LLM responses
+- ✅ **Extraction**: Parse structured data from responses
+- ✅ **Guidance**: Tell the LLM exactly what format to return
+
+**Output format specs ARE used for runtime token estimation** — they directly impact API call parameters.
+
+### Example: Using Both
+
+```typescript
+const instructions = `
+## Input format
+- User Query — text (required)
+  The question or request from the user.
+- Context — list (optional)
+  Items: 1-5. Additional context items.
+
+## Output format
+- Answer — text (required)
+  Length: 2-3 paragraphs. Provide a comprehensive answer.
+- Sources — list (required)
+  Items: 3-7. List information sources used.
+`;
+
+// Parse both formats
+import { parseFormatSpecs } from 'flex-md';
+const { input, output } = parseFormatSpecs(instructions);
+
+// Input format: Use for documentation/planning
+console.log('Expected input structure:', input);
+
+// Output format: Use for runtime token estimation
+const maxTokens = getMaxTokens(output);
+```
+
 ## Smart Toolbox
 
 ### Token Estimation
 
+#### Planning-Time Estimation
+
+For estimating tokens from format specs directly (useful during development):
+
 ```typescript
 import { getMaxTokens, estimateSpecTokens } from 'flex-md';
 
-// Quick estimate
+// Quick estimate from output format spec
 const maxTokens = getMaxTokens(spec);
 
 // Detailed estimate with options
@@ -195,6 +253,64 @@ console.log(estimate);
 // }
 ```
 
+#### Runtime Estimation
+
+For estimating tokens at runtime with actual prompt, context, and instructions:
+
+```typescript
+import { runtimeEstimateTokens } from 'flex-md';
+
+const estimate = runtimeEstimateTokens({
+  prompt: 'Analyze this code and explain what it does.',
+  context: 'Previous conversation about TypeScript...',
+  instructions: `
+    You are a helpful coding assistant.
+    
+    ## Output format
+    - Analysis — text (required)
+      Length: 2-3 paragraphs. Explain the code functionality.
+    - Code Quality — list (required)
+      Items: 3-5. List quality observations.
+  `,
+  options: {
+    safetyMultiplier: 1.2,
+    strategy: 'average',
+    additionalOverhead: 50  // For system messages, formatting, etc.
+  }
+});
+
+console.log(estimate.maxTokens);  // Recommended max_tokens for API call (output tokens only)
+console.log(estimate.breakdown);
+// {
+//   prompt: 12,                    // Input tokens (for budgeting)
+//   context: 8,                    // Input tokens (for budgeting)
+//   instructions: 45,              // Input tokens (for budgeting)
+//   output: { total: { estimated: 450, ... }, ... },  // Output token estimate
+//   additionalOverhead: 50,
+//   total: 565                     // Total tokens (input + output) for budgeting
+// }
+
+// Use in API call
+const response = await fetch('https://api.anthropic.com/v1/messages', {
+  method: 'POST',
+  body: JSON.stringify({
+    model: 'claude-sonnet-4-20250514',
+    max_tokens: estimate.maxTokens,  // Output tokens only (extracted from output format spec)
+    messages: [{
+      role: 'user',
+      content: prompt + '\n\n' + instructions
+    }]
+  })
+});
+```
+
+The `runtimeEstimateTokens` function:
+- **Extracts output format specs from instructions automatically** (input format specs are ignored)
+- Estimates tokens for prompt, context, and instructions text (for budgeting/planning)
+- **Estimates output tokens based on the output format spec** (this becomes `max_tokens`)
+- Provides a breakdown showing input tokens (for budgeting) and output tokens (for API parameter)
+- Returns `maxTokens` which is the value to use for the `max_tokens` API parameter (output only)
+
 ### Compliance Checking
 
 ```typescript
@@ -410,14 +526,18 @@ const v3 = `
 ## API Reference
 
 ### Core Functions
-- `parseOutputFormatSpec(markdown)` - Parse spec from markdown
+- `parseOutputFormatSpec(markdown)` - Parse output format spec from markdown
+- `parseInputFormatSpec(markdown)` - Parse input format spec from markdown
+- `parseFormatSpecs(instructions)` - Extract both input and output format specs from instructions
 - `stringifyOutputFormatSpec(spec)` - Convert spec to markdown
 - `buildMarkdownGuidance(spec, options)` - Generate LLM instructions
 - `enforceFlexMd(text, spec, options)` - Validate and repair LLM output
 
 ### Token Estimation (v4.0)
-- `getMaxTokens(spec, options?)` - Get estimated max_tokens
-- `estimateSpecTokens(spec, options?)` - Detailed token estimate
+- `getMaxTokens(spec, options?)` - Get estimated max_tokens from spec
+- `getMaxTokensFromInstructions(instructions, options?)` - Extract format specs and estimate tokens
+- `estimateSpecTokens(spec, options?)` - Detailed token estimate from spec
+- `runtimeEstimateTokens(params)` - Runtime estimation with prompt, context, and instructions
 - `parseSystemPart(instruction, kind)` - Parse system part from instruction
 - `estimateTokens(systemPart)` - Estimate tokens for system part
 

package/dist/index.cjs

@@ -14,7 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.jsonToMarkdown = exports.Schema = exports.MarkdownParser = exports.JSONTransformer = exports.enforceFlexMd = exports.repairToMarkdownLevel = exports.detectResponseKind = exports.buildIssuesEnvelopeAuto = exports.buildIssuesEnvelope = exports.parseIssuesEnvelope = exports.processResponseMarkdown = exports.extractFromMarkdown = exports.checkConnection = exports.hasFlexMdContract = exports.checkCompliance = exports.validateMarkdownAgainstOfs = exports.enrichInstructionsWithFlexMd = exports.enrichInstructions = exports.buildMarkdownGuidance = exports.stringifyOutputFormatSpec = exports.recall = exports.remember = exports.transformWithOfs = exports.ofsToSchema = exports.validateFormat = exports.parseOutputFormatSpec = exports.buildOutline = exports.logger = void 0;
+exports.jsonToMarkdown = exports.Schema = exports.MarkdownParser = exports.JSONTransformer = exports.enforceFlexMd = exports.repairToMarkdownLevel = exports.detectResponseKind = exports.buildIssuesEnvelopeAuto = exports.buildIssuesEnvelope = exports.parseIssuesEnvelope = exports.processResponseMarkdown = exports.extractFromMarkdown = exports.checkConnection = exports.hasFlexMdContract = exports.checkCompliance = exports.validateMarkdownAgainstOfs = exports.enrichInstructionsWithFlexMd = exports.enrichInstructions = exports.buildMarkdownGuidance = exports.stringifyOutputFormatSpec = exports.recall = exports.remember = exports.transformWithOfs = exports.ofsToSchema = exports.validateFormat = exports.parseFormatSpecs = exports.parseInputFormatSpec = exports.parseOutputFormatSpec = exports.buildOutline = exports.logger = void 0;
 // Core SFMD Types
 __exportStar(require("./types.js"), exports);
 __exportStar(require("./strictness/types.js"), exports);
@@ -28,6 +28,8 @@ Object.defineProperty(exports, "buildOutline", { enumerable: true, get: function
 // Output Format Spec (OFS)
 var parser_js_1 = require("./ofs/parser.js");
 Object.defineProperty(exports, "parseOutputFormatSpec", { enumerable: true, get: function () { return parser_js_1.parseOutputFormatSpec; } });
+Object.defineProperty(exports, "parseInputFormatSpec", { enumerable: true, get: function () { return parser_js_1.parseInputFormatSpec; } });
+Object.defineProperty(exports, "parseFormatSpecs", { enumerable: true, get: function () { return parser_js_1.parseFormatSpecs; } });
 Object.defineProperty(exports, "validateFormat", { enumerable: true, get: function () { return parser_js_1.validateFormat; } });
 var adapter_js_1 = require("./ofs/adapter.js");
 Object.defineProperty(exports, "ofsToSchema", { enumerable: true, get: function () { return adapter_js_1.ofsToSchema; } });

package/dist/index.d.ts

@@ -3,7 +3,7 @@ export * from "./strictness/types.js";
 export { logger } from "./logger.js";
 export * from "./md/parse.js";
 export { buildOutline } from "./md/outline.js";
-export { parseOutputFormatSpec, validateFormat } from "./ofs/parser.js";
+export { parseOutputFormatSpec, parseInputFormatSpec, parseFormatSpecs, validateFormat } from "./ofs/parser.js";
 export { ofsToSchema, transformWithOfs } from "./ofs/adapter.js";
 export { remember, recall } from "./ofs/memory.js";
 export { stringifyOutputFormatSpec } from "./ofs/stringify.js";

package/dist/index.js

@@ -14,7 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.jsonToMarkdown = exports.Schema = exports.MarkdownParser = exports.JSONTransformer = exports.enforceFlexMd = exports.repairToMarkdownLevel = exports.detectResponseKind = exports.buildIssuesEnvelopeAuto = exports.buildIssuesEnvelope = exports.parseIssuesEnvelope = exports.processResponseMarkdown = exports.extractFromMarkdown = exports.checkConnection = exports.hasFlexMdContract = exports.checkCompliance = exports.validateMarkdownAgainstOfs = exports.enrichInstructionsWithFlexMd = exports.enrichInstructions = exports.buildMarkdownGuidance = exports.stringifyOutputFormatSpec = exports.recall = exports.remember = exports.transformWithOfs = exports.ofsToSchema = exports.validateFormat = exports.parseOutputFormatSpec = exports.buildOutline = exports.logger = void 0;
+exports.jsonToMarkdown = exports.Schema = exports.MarkdownParser = exports.JSONTransformer = exports.enforceFlexMd = exports.repairToMarkdownLevel = exports.detectResponseKind = exports.buildIssuesEnvelopeAuto = exports.buildIssuesEnvelope = exports.parseIssuesEnvelope = exports.processResponseMarkdown = exports.extractFromMarkdown = exports.checkConnection = exports.hasFlexMdContract = exports.checkCompliance = exports.validateMarkdownAgainstOfs = exports.enrichInstructionsWithFlexMd = exports.enrichInstructions = exports.buildMarkdownGuidance = exports.stringifyOutputFormatSpec = exports.recall = exports.remember = exports.transformWithOfs = exports.ofsToSchema = exports.validateFormat = exports.parseFormatSpecs = exports.parseInputFormatSpec = exports.parseOutputFormatSpec = exports.buildOutline = exports.logger = void 0;
 // Core SFMD Types
 __exportStar(require("./types.js"), exports);
 __exportStar(require("./strictness/types.js"), exports);
@@ -28,6 +28,8 @@ Object.defineProperty(exports, "buildOutline", { enumerable: true, get: function
 // Output Format Spec (OFS)
 var parser_js_1 = require("./ofs/parser.js");
 Object.defineProperty(exports, "parseOutputFormatSpec", { enumerable: true, get: function () { return parser_js_1.parseOutputFormatSpec; } });
+Object.defineProperty(exports, "parseInputFormatSpec", { enumerable: true, get: function () { return parser_js_1.parseInputFormatSpec; } });
+Object.defineProperty(exports, "parseFormatSpecs", { enumerable: true, get: function () { return parser_js_1.parseFormatSpecs; } });
 Object.defineProperty(exports, "validateFormat", { enumerable: true, get: function () { return parser_js_1.validateFormat; } });
 var adapter_js_1 = require("./ofs/adapter.js");
 Object.defineProperty(exports, "ofsToSchema", { enumerable: true, get: function () { return adapter_js_1.ofsToSchema; } });

package/dist/ofs/parser.d.ts

@@ -1,4 +1,4 @@
-import type { OutputFormatSpec, FormatValidationResult } from "../types.js";
+import type { OutputFormatSpec, InputFormatSpec, FormatSpecs, FormatValidationResult } from "../types.js";
 /**
  * Validate a format specification.
  * Returns detailed validation results.
@@ -12,3 +12,13 @@ export interface ParseOfsOptions {
     allowDelimiterFallbacks?: boolean;
 }
 export declare function parseOutputFormatSpec(md: string, opts?: ParseOfsOptions): OutputFormatSpec | null;
+/**
+ * Parse an Input Format Spec block from Markdown.
+ * Uses the same parsing logic as Output Format Spec.
+ */
+export declare function parseInputFormatSpec(md: string, opts?: ParseOfsOptions): InputFormatSpec | null;
+/**
+ * Extract both Input and Output Format Specs from instructions text.
+ * This function searches for both "## Input format" and "## Output format" sections.
+ */
+export declare function parseFormatSpecs(instructions: string, opts?: ParseOfsOptions): FormatSpecs;

package/dist/ofs/parser.js

@@ -2,6 +2,8 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.validateFormat = validateFormat;
 exports.parseOutputFormatSpec = parseOutputFormatSpec;
+exports.parseInputFormatSpec = parseInputFormatSpec;
+exports.parseFormatSpecs = parseFormatSpecs;
 const parse_js_1 = require("../md/parse.js");
 /**
  * Validate a format specification.
@@ -246,3 +248,126 @@ function parseRequiredOptional(s) {
         return false;
     return undefined;
 }
+/**
+ * Parse an Input Format Spec block from Markdown.
+ * Uses the same parsing logic as Output Format Spec.
+ */
+function parseInputFormatSpec(md, opts = {}) {
+    const headingRx = opts.headingRegex ?? /^##\s*Input format\b/i;
+    const lines = md.split("\n");
+    // find Input Format Spec start
+    let start = -1;
+    for (let i = 0; i < lines.length; i++) {
+        const line = lines[i] ?? "";
+        if (headingRx.test(line)) {
+            start = i;
+            break;
+        }
+    }
+    if (start === -1)
+        return null;
+    // capture block until next H2 (##) or end
+    let end = lines.length;
+    for (let i = start + 1; i < lines.length; i++) {
+        const line = lines[i] ?? "";
+        if (/^##\s+/.test(line)) {
+            end = i;
+            break;
+        }
+    }
+    const block = lines.slice(start, end).join("\n");
+    const sections = [];
+    const tables = [];
+    let emptySectionValue;
+    let inTables = false;
+    let currentSection = null;
+    for (const rawLine of block.split("\n")) {
+        const line = rawLine.trim();
+        if (/^tables\b/i.test(line)) {
+            inTables = true;
+            currentSection = null;
+            continue;
+        }
+        if (/^empty sections\b/i.test(line)) {
+            inTables = false;
+            currentSection = null;
+            continue;
+        }
+        // Empty section rule
+        const mNone = line.match(/write\s+`([^`]+)`/i);
+        if (/empty/i.test(line) && mNone) {
+            emptySectionValue = mNone[1];
+            currentSection = null;
+            continue;
+        }
+        // bullet items
+        const bullet = line.match(/^- (.+)$/);
+        if (bullet) {
+            const item = bullet[1];
+            if (inTables) {
+                const t = parseTableDecl(item);
+                if (t)
+                    tables.push(t);
+                currentSection = null;
+            }
+            else {
+                const s = parseSectionDecl(item, !!opts.allowDelimiterFallbacks);
+                if (s) {
+                    sections.push(s);
+                    currentSection = s;
+                }
+                else {
+                    currentSection = null;
+                }
+            }
+            continue;
+        }
+        // heading items (e.g. ### Short Answer)
+        const headingMatch = line.match(/^#{1,6}\s+(.+)$/);
+        if (headingMatch) {
+            const name = headingMatch[1].trim();
+            // Don't re-parse "Input format" itself if it somehow gets in here
+            if ((0, parse_js_1.normalizeName)(name) !== "input format") {
+                const s = { name, kind: "text" };
+                sections.push(s);
+                currentSection = s;
+            }
+            continue;
+        }
+        // If not a bullet and we have a current section, it's an instruction
+        if (currentSection && line.length > 0) {
+            // Support "Columns: A, B, C" in instructions for tables
+            const colMatch = line.match(/^Columns:\s*(.+)$/i);
+            if (colMatch && (currentSection.kind === "table" || currentSection.kind === "ordered_table")) {
+                currentSection.columns = colMatch[1].split(",").map(c => c.trim()).filter(Boolean);
+            }
+            else {
+                const existing = currentSection.instruction || "";
+                currentSection.instruction = existing ? `${existing} ${line}` : line;
+            }
+        }
+    }
+    if (!sections.length)
+        return null;
+    return {
+        descriptorType: "input_format_spec",
+        format: "markdown",
+        sectionOrderMatters: false,
+        sections,
+        tablesOptional: true,
+        tables,
+        emptySectionValue: emptySectionValue ?? "None"
+    };
+}
+/**
+ * Extract both Input and Output Format Specs from instructions text.
+ * This function searches for both "## Input format" and "## Output format" sections.
+ */
+function parseFormatSpecs(instructions, opts = {}) {
+    const input = parseInputFormatSpec(instructions, opts);
+    const output = parseOutputFormatSpec(instructions, opts);
+    return {
+        input: input || undefined,
+        output: output || undefined
+    };
+}

package/dist/tokens/estimator.d.ts

@@ -7,6 +7,12 @@ export declare const TOKEN_CONSTANTS: {
     readonly perCodeLine: 5;
     readonly headingOverhead: 10;
     readonly baseOverhead: 50;
+    readonly charsPerToken: 4;
 };
 export declare function estimateTokens(systemPart: SystemPart): TokenEstimate;
 export declare function getFallbackEstimate(kind: SectionKind, required: boolean): TokenEstimate;
+/**
+ * Estimate tokens from plain text using character-based approximation.
+ * Uses a rule of thumb: ~4 characters per token (varies by model, but good approximation).
+ */
+export declare function estimateTextTokens(text: string): number;

package/dist/tokens/estimator.js

@@ -3,6 +3,7 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.TOKEN_CONSTANTS = void 0;
 exports.estimateTokens = estimateTokens;
 exports.getFallbackEstimate = getFallbackEstimate;
+exports.estimateTextTokens = estimateTextTokens;
 // Calibrated token constants
 exports.TOKEN_CONSTANTS = {
     // Text lengths (enumerated)
@@ -22,7 +23,9 @@ exports.TOKEN_CONSTANTS = {
     perCodeLine: 5,
     // Overhead
     headingOverhead: 10,
-    baseOverhead: 50
+    baseOverhead: 50,
+    // Text token estimation (rough approximation: 1 token ≈ 4 characters)
+    charsPerToken: 4
 };
 function estimateTokens(systemPart) {
     const { parsed } = systemPart;
@@ -141,3 +144,14 @@ function getFallbackEstimate(kind, required) {
         confidence: 'low'
     };
 }
+/**
+ * Estimate tokens from plain text using character-based approximation.
+ * Uses a rule of thumb: ~4 characters per token (varies by model, but good approximation).
+ */
+function estimateTextTokens(text) {
+    if (!text || text.length === 0)
+        return 0;
+    // Rough approximation: 1 token ≈ 4 characters for English text
+    // This is a common rule of thumb and works reasonably well for most cases
+    return Math.ceil(text.length / exports.TOKEN_CONSTANTS.charsPerToken);
+}

package/dist/tokens/index.d.ts

@@ -3,6 +3,7 @@ export * from './types.js';
 export * from './parser.js';
 export * from './estimator.js';
 export * from './spec-estimator.js';
+export * from './runtime-estimator.js';
 export * from './validator.js';
 export * from './compliance.js';
 export * from './cognitive-cost.js';
@@ -13,7 +14,7 @@ export * from './auto-fix.js';
 /**
  * Convenience function: estimate max_tokens from spec
  *
- * @param spec - OutputFormatSpec object or markdown string
+ * @param spec - OutputFormatSpec object, markdown string, or full instructions text
  * @param options - Estimation options
  * @returns Estimated max_tokens value
  */
@@ -21,4 +22,23 @@ export declare function getMaxTokens(spec: OutputFormatSpec | string | null | un
     includeOptional?: boolean;
     safetyMultiplier?: number;
     strategy?: 'conservative' | 'average' | 'generous';
+    /**
+     * If true, treats the input as full instructions and extracts both
+     * input and output format specs. If false, only looks for output format.
+     * @default false (backwards compatible)
+     */
+    extractFromInstructions?: boolean;
+}): number;
+/**
+ * Estimate max tokens from entire instructions text.
+ * Automatically extracts both input and output format specs if present.
+ *
+ * @param instructions - Full instructions text that may contain format specs
+ * @param options - Estimation options
+ * @returns Estimated max_tokens value based on both input and output formats
+ */
+export declare function getMaxTokensFromInstructions(instructions: string, options?: {
+    includeOptional?: boolean;
+    safetyMultiplier?: number;
+    strategy?: 'conservative' | 'average' | 'generous';
 }): number;

package/dist/tokens/index.js

@@ -15,12 +15,14 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.getMaxTokens = getMaxTokens;
+exports.getMaxTokensFromInstructions = getMaxTokensFromInstructions;
 const parser_js_1 = require("../ofs/parser.js");
 const spec_estimator_js_1 = require("./spec-estimator.js");
 __exportStar(require("./types.js"), exports);
 __exportStar(require("./parser.js"), exports);
 __exportStar(require("./estimator.js"), exports);
 __exportStar(require("./spec-estimator.js"), exports);
+__exportStar(require("./runtime-estimator.js"), exports);
 __exportStar(require("./validator.js"), exports);
 __exportStar(require("./compliance.js"), exports);
 __exportStar(require("./cognitive-cost.js"), exports);
@@ -31,13 +33,24 @@ __exportStar(require("./auto-fix.js"), exports);
 /**
  * Convenience function: estimate max_tokens from spec
  *
- * @param spec - OutputFormatSpec object or markdown string
+ * @param spec - OutputFormatSpec object, markdown string, or full instructions text
  * @param options - Estimation options
  * @returns Estimated max_tokens value
  */
 function getMaxTokens(spec, options) {
     if (!spec)
         return 0;
+    const extractFromInstructions = options?.extractFromInstructions ?? false;
+    // If extractFromInstructions is true and spec is a string, try to extract both formats
+    if (extractFromInstructions && typeof spec === 'string') {
+        const formatSpecs = (0, parser_js_1.parseFormatSpecs)(spec);
+        if (formatSpecs.input || formatSpecs.output) {
+            const estimate = (0, spec_estimator_js_1.estimateFormatSpecsTokens)(formatSpecs.input, formatSpecs.output, options);
+            return estimate.total.estimated;
+        }
+        // Fall through to output-only parsing if no formats found
+    }
+    // Original behavior: parse as output format spec
     const parsedSpec = typeof spec === 'string'
         ? (0, parser_js_1.parseOutputFormatSpec)(spec)
         : spec;
@@ -46,3 +59,16 @@ function getMaxTokens(spec, options) {
     const estimate = (0, spec_estimator_js_1.estimateSpecTokens)(parsedSpec, options);
     return estimate.total.estimated;
 }
+/**
+ * Estimate max tokens from entire instructions text.
+ * Automatically extracts both input and output format specs if present.
+ *
+ * @param instructions - Full instructions text that may contain format specs
+ * @param options - Estimation options
+ * @returns Estimated max_tokens value based on both input and output formats
+ */
+function getMaxTokensFromInstructions(instructions, options) {
+    const formatSpecs = (0, parser_js_1.parseFormatSpecs)(instructions);
+    const estimate = (0, spec_estimator_js_1.estimateFormatSpecsTokens)(formatSpecs.input, formatSpecs.output, options);
+    return estimate.total.estimated;
+}

package/dist/tokens/runtime-estimator.d.ts

@@ -0,0 +1,92 @@
+import type { SpecTokenEstimate } from './types.js';
+export interface RuntimeEstimateOptions {
+    /**
+     * Whether to include optional sections in output format estimation
+     * @default true
+     */
+    includeOptional?: boolean;
+    /**
+     * Safety multiplier to add headroom (e.g., 1.2 = 20% extra)
+     * @default 1.2
+     */
+    safetyMultiplier?: number;
+    /**
+     * Estimation strategy
+     * @default 'average'
+     */
+    strategy?: 'conservative' | 'average' | 'generous';
+    /**
+     * Whether to estimate tokens for the prompt text
+     * @default true
+     */
+    estimatePrompt?: boolean;
+    /**
+     * Whether to estimate tokens for the context text
+     * @default true
+     */
+    estimateContext?: boolean;
+    /**
+     * Whether to estimate tokens for the instructions text
+     * @default true
+     */
+    estimateInstructions?: boolean;
+    /**
+     * Additional overhead tokens to add (for system messages, formatting, etc.)
+     * @default 0
+     */
+    additionalOverhead?: number;
+}
+export interface RuntimeTokenEstimate {
+    /**
+     * Estimated max_tokens needed for the LLM response
+     */
+    maxTokens: number;
+    /**
+     * Breakdown of token estimates
+     */
+    breakdown: {
+        prompt: number;
+        context: number;
+        instructions: number;
+        output: SpecTokenEstimate;
+        additionalOverhead: number;
+        total: number;
+    };
+    /**
+     * Confidence level of the estimate
+     */
+    confidence: 'high' | 'medium' | 'low';
+}
+/**
+ * Estimate max_tokens for an LLM API call at runtime.
+ *
+ * This function estimates the total tokens needed by considering:
+ * - The prompt text (user input)
+ * - Context (previous messages, system messages, etc.)
+ * - Instructions (which may contain output format specs)
+ * - Expected output format (extracted from instructions)
+ *
+ * Note: Input format specs in instructions are ignored (they're for planning/design only).
+ * Only output format specs are used for runtime token estimation.
+ *
+ * @param params - Runtime estimation parameters
+ * @returns Token estimate with breakdown
+ */
+export declare function runtimeEstimateTokens(params: {
+    /**
+     * The user prompt/message text
+     */
+    prompt?: string;
+    /**
+     * Context text (system messages, previous conversation, etc.)
+     */
+    context?: string;
+    /**
+     * Instructions text (may contain format specs)
+     */
+    instructions?: string;
+    /**
+     * Estimation options
+     */
+    options?: RuntimeEstimateOptions;
+}): RuntimeTokenEstimate;

package/dist/tokens/runtime-estimator.js

@@ -0,0 +1,74 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.runtimeEstimateTokens = runtimeEstimateTokens;
+const parser_js_1 = require("../ofs/parser.js");
+const spec_estimator_js_1 = require("./spec-estimator.js");
+const estimator_js_1 = require("./estimator.js");
+/**
+ * Estimate max_tokens for an LLM API call at runtime.
+ *
+ * This function estimates the total tokens needed by considering:
+ * - The prompt text (user input)
+ * - Context (previous messages, system messages, etc.)
+ * - Instructions (which may contain output format specs)
+ * - Expected output format (extracted from instructions)
+ *
+ * Note: Input format specs in instructions are ignored (they're for planning/design only).
+ * Only output format specs are used for runtime token estimation.
+ *
+ * @param params - Runtime estimation parameters
+ * @returns Token estimate with breakdown
+ */
+function runtimeEstimateTokens(params) {
+    const { prompt = '', context = '', instructions = '' } = params;
+    const opts = {
+        includeOptional: params.options?.includeOptional ?? true,
+        safetyMultiplier: params.options?.safetyMultiplier ?? 1.2,
+        strategy: params.options?.strategy ?? 'average',
+        estimatePrompt: params.options?.estimatePrompt ?? true,
+        estimateContext: params.options?.estimateContext ?? true,
+        estimateInstructions: params.options?.estimateInstructions ?? true,
+        additionalOverhead: params.options?.additionalOverhead ?? 0
+    };
+    // Extract format specs from instructions (only output format is used at runtime)
+    const formatSpecs = (0, parser_js_1.parseFormatSpecs)(instructions);
+    // Estimate tokens for each component
+    const promptTokens = opts.estimatePrompt ? (0, estimator_js_1.estimateTextTokens)(prompt) : 0;
+    const contextTokens = opts.estimateContext ? (0, estimator_js_1.estimateTextTokens)(context) : 0;
+    const instructionsTokens = opts.estimateInstructions ? (0, estimator_js_1.estimateTextTokens)(instructions) : 0;
+    // Estimate output tokens from output format spec (input format is for planning only)
+    const outputEstimate = formatSpecs.output
+        ? (0, spec_estimator_js_1.estimateFormatSpecsTokens)(undefined, formatSpecs.output, {
+            includeOptional: opts.includeOptional,
+            safetyMultiplier: 1.0, // Don't apply safety multiplier here, apply at the end
+            strategy: opts.strategy
+        })
+        : {
+            total: { estimated: 0, min: 0, max: 0, confidence: 'low' },
+            bySectionName: {},
+            overhead: 0
+        };
+    // Calculate total input tokens (prompt + context + instructions)
+    const inputTokens = promptTokens + contextTokens + instructionsTokens;
+    // Calculate total output tokens (from format spec)
+    // Note: max_tokens parameter in LLM APIs refers to OUTPUT tokens only
+    const outputTokens = outputEstimate.total.estimated;
+    // Apply safety multiplier to output tokens only (this is what max_tokens represents)
+    const maxTokens = Math.ceil(outputTokens * opts.safetyMultiplier) + opts.additionalOverhead;
+    // Total tokens for budgeting/planning (input + output)
+    const totalTokens = inputTokens + outputTokens + opts.additionalOverhead;
+    // Determine confidence based on output spec confidence
+    const confidence = outputEstimate.total.confidence;
+    return {
+        maxTokens, // This is the value to use for max_tokens API parameter (output only)
+        breakdown: {
+            prompt: promptTokens,
+            context: contextTokens,
+            instructions: instructionsTokens,
+            output: outputEstimate,
+            additionalOverhead: opts.additionalOverhead,
+            total: totalTokens // Total for budgeting (input + output)
+        },
+        confidence
+    };
+}

package/dist/tokens/spec-estimator.d.ts

@@ -1,7 +1,25 @@
 import type { SpecTokenEstimate } from './types.js';
-import type { OutputFormatSpec } from '../types.js';
+import type { OutputFormatSpec, InputFormatSpec } from '../types.js';
 export declare function estimateSpecTokens(spec: OutputFormatSpec, options?: {
     includeOptional?: boolean;
     safetyMultiplier?: number;
     strategy?: 'conservative' | 'average' | 'generous';
 }): SpecTokenEstimate;
+/**
+ * Estimate tokens for an Input Format Spec.
+ * Uses the same logic as output format estimation.
+ */
+export declare function estimateInputSpecTokens(spec: InputFormatSpec, options?: {
+    includeOptional?: boolean;
+    safetyMultiplier?: number;
+    strategy?: 'conservative' | 'average' | 'generous';
+}): SpecTokenEstimate;
+/**
+ * Estimate tokens for both input and output format specs.
+ * Combines estimates from both if present.
+ */
+export declare function estimateFormatSpecsTokens(inputSpec: InputFormatSpec | undefined, outputSpec: OutputFormatSpec | undefined, options?: {
+    includeOptional?: boolean;
+    safetyMultiplier?: number;
+    strategy?: 'conservative' | 'average' | 'generous';
+}): SpecTokenEstimate;

package/dist/tokens/spec-estimator.js

@@ -1,6 +1,8 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.estimateSpecTokens = estimateSpecTokens;
+exports.estimateInputSpecTokens = estimateInputSpecTokens;
+exports.estimateFormatSpecsTokens = estimateFormatSpecsTokens;
 const parser_js_1 = require("./parser.js");
 const estimator_js_1 = require("./estimator.js");
 function estimateSpecTokens(spec, options = {}) {
@@ -69,3 +71,55 @@ function estimateSpecTokens(spec, options = {}) {
         overhead
     };
 }
+/**
+ * Estimate tokens for an Input Format Spec.
+ * Uses the same logic as output format estimation.
+ */
+function estimateInputSpecTokens(spec, options = {}) {
+    // Input format uses the same structure as output format, so reuse the same logic
+    const outputSpec = {
+        ...spec,
+        descriptorType: "output_format_spec"
+    };
+    return estimateSpecTokens(outputSpec, options);
+}
+/**
+ * Estimate tokens for both input and output format specs.
+ * Combines estimates from both if present.
+ */
+function estimateFormatSpecsTokens(inputSpec, outputSpec, options = {}) {
+    const inputEstimate = inputSpec ? estimateInputSpecTokens(inputSpec, options) : null;
+    const outputEstimate = outputSpec ? estimateSpecTokens(outputSpec, options) : null;
+    // If only one is present, return it
+    if (!inputEstimate && outputEstimate)
+        return outputEstimate;
+    if (inputEstimate && !outputEstimate)
+        return inputEstimate;
+    if (!inputEstimate && !outputEstimate) {
+        // Return empty estimate
+        return {
+            total: { estimated: 0, min: 0, max: 0, confidence: 'low' },
+            bySectionName: {},
+            overhead: 0
+        };
+    }
+    // Combine both estimates
+    const combined = {
+        total: {
+            estimated: inputEstimate.total.estimated + outputEstimate.total.estimated,
+            min: inputEstimate.total.min + outputEstimate.total.min,
+            max: inputEstimate.total.max + outputEstimate.total.max,
+            confidence: inputEstimate.total.confidence === 'high' && outputEstimate.total.confidence === 'high'
+                ? 'high'
+                : inputEstimate.total.confidence === 'low' || outputEstimate.total.confidence === 'low'
+                    ? 'low'
+                    : 'medium'
+        },
+        bySectionName: {
+            ...Object.fromEntries(Object.entries(inputEstimate.bySectionName).map(([k, v]) => [`input:${k}`, v])),
+            ...Object.fromEntries(Object.entries(outputEstimate.bySectionName).map(([k, v]) => [`output:${k}`, v]))
+        },
+        overhead: inputEstimate.overhead + outputEstimate.overhead
+    };
+    return combined;
+}

package/dist/types.d.ts

@@ -112,6 +112,20 @@ export interface OutputFormatSpec {
     tablesOptional?: boolean;
     tables?: OfsTable[];
 }
+export interface InputFormatSpec {
+    description?: string;
+    sections: OutputSectionSpec[];
+    emptySectionValue?: string;
+    descriptorType?: "input_format_spec";
+    format?: "markdown";
+    sectionOrderMatters?: boolean;
+    tablesOptional?: boolean;
+    tables?: OfsTable[];
+}
+export interface FormatSpecs {
+    input?: InputFormatSpec;
+    output?: OutputFormatSpec;
+}
 export interface MdNode {
     title: string;
     level: number;
@@ -199,6 +213,8 @@ export interface ComplianceCheckResult {
         hasRequiredSections?: boolean;
         sectionCount?: number;
         containerType?: "fenced-block" | "none";
+        hasInputFormat?: boolean;
+        hasOutputFormat?: boolean;
     };
 }
 export interface EnhancementChange {

package/dist/validate/compliance.d.ts

@@ -2,6 +2,7 @@ import { ComplianceCheckResult } from "../types.js";
 /**
  * Check if instructions meet the required flex-md compliance level.
  * Returns detailed results including what's missing or wrong.
+ * Now also checks for Input Format Spec if present.
  */
 export declare function checkCompliance(instructions: string, complianceLevel: "L0" | "L1" | "L2" | "L3"): Promise<ComplianceCheckResult>;
 /**

package/dist/validate/compliance.js

@@ -2,14 +2,20 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.checkCompliance = checkCompliance;
 exports.hasFlexMdContract = hasFlexMdContract;
+const parser_js_1 = require("../ofs/parser.js");
 /**
  * Check if instructions meet the required flex-md compliance level.
  * Returns detailed results including what's missing or wrong.
+ * Now also checks for Input Format Spec if present.
  */
 async function checkCompliance(instructions, complianceLevel) {
     const issues = [];
     const suggestions = [];
     const lower = instructions.toLowerCase();
+    // Extract both input and output format specs
+    const formatSpecs = (0, parser_js_1.parseFormatSpecs)(instructions);
+    const hasInputFormat = !!formatSpecs.input;
+    const hasOutputFormat = !!formatSpecs.output;
     const hasMarkdownMention = lower.includes("markdown");
     const hasSectionMention = lower.includes("section") || lower.includes("heading");
     const hasContainerMention = (lower.includes("fenced block") || lower.includes("```")) && (lower.includes("inside") || lower.includes("wrapped"));
@@ -81,7 +87,9 @@ async function checkCompliance(instructions, complianceLevel) {
         metadata: {
             hasContainer: hasContainerMention,
             hasRequiredSections: hasSectionMention,
-            containerType: hasContainerMention ? "fenced-block" : "none"
+            containerType: hasContainerMention ? "fenced-block" : "none",
+            hasInputFormat,
+            hasOutputFormat
         }
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "flex-md",
-  "version": "4.6.0",
+  "version": "4.7.0",
   "description": "Parse and stringify FlexMD: semi-structured Markdown with three powerful layers - Frames, Output Format Spec (OFS), and Detection/Extraction.",
   "license": "MIT",
   "author": "",