nl2time 0.1.0 → 0.3.0

package/README.md

@@ -24,6 +24,8 @@ describe(Temporal.Instant.from('2026-07-20T04:00:00Z'), ctx).text;
 // 'yesterday at 9:00 PM'
 ```
 
+> **New here?** Read the **[step-by-step walkthrough](docs/walkthrough.md)** — two fully-traced examples showing exactly what happens at each stage in both directions (tokens → rules → IR → deterministic resolution, and instant → framing → IR → rendered speech).
+
 ## Why another date library?
 
 Existing tools do one direction, half-way:
@@ -60,6 +62,10 @@ const result = await parseWithFallback(text, ctx, async (text, ctxSummary) => {
 
 See [docs/agents.md](docs/agents.md) for reference architectures (analytics agents, tool-calling, deferred resolution).
 
+## Domain adaptation
+
+Domains with conflicting or novel vocabulary ("FY26", "EOD means 5pm", "swing shift") extend the parser **without forking** via declarative JSON packs — phrase→IR templates with integer/year captures that compete with (and can shadow) the built-in rules, plus a `disable` list and a code-level `Rule` escape hatch. Packs ship their own golden cases, runnable with `nl2time/corpus`. See [docs/extending.md](docs/extending.md) and the worked [fiscal-calendar example](examples/fiscal-july/).
+
 ## Policy, not guesses
 
 Cultural/ambiguous semantics are explicit `TimeContext` knobs with CLDR-derived defaults:
@@ -73,11 +79,46 @@ Cultural/ambiguous semantics are explicit `TimeContext` knobs with CLDR-derived
 | Does "last 3 days" include today? | `partialPeriod: 'include' \| 'exclude'` | `include` |
 | "At 4" — am or pm? | — | both candidates, plausibility-ordered |
 
+## Install
+
+[![npm](https://img.shields.io/npm/v/nl2time)](https://www.npmjs.com/package/nl2time)
+
+```
+npm install nl2time
+```
+
+## Languages
+
+Parsing is multilingual, dispatched by the context's locale. Each language climbs its own imported conformance corpus (Microsoft Recognizers-Text specs, MIT) with a CI-gated baseline:
+
+| Language | Corpus cases | Passing |
+|---|---|---|
+| English (en-US / en-GB) | 1,031 | 84% |
+| German | 157 | 97% |
+| Japanese | 393 | 96% |
+| Chinese (Simplified) | 175 | 95% |
+| French | 406 | 90% |
+| Spanish | 579 | 96% |
+
+Latin-script languages share a parameterized rule factory (`makeLatinRules` + a lexicon); CJK languages use per-character tokenization with bespoke rule modules. `describe()` output is currently English; localized rendering is on the roadmap. Remaining failure mass is dominated by documented upstream divergences (e.g. issue #14).
+
 ## Status
 
-v0.1 — English parsing, en-* describe, core engine with a DST/edge-case battery. Recurrence (`every Tuesday`) is representable in the IR but resolves in v2.
+v0.3 — multilingual parsing (6 locales), en-* describe, core engine with a DST/edge-case battery, holidays (fixed-date, nth-weekday, computed Easter), business-day spans, domain packs. Recurrence (`every Tuesday`) is representable in the IR but resolves in v2.
+
+**Corpus** ([corpus/](corpus/), runner exported as `nl2time/corpus`): bidirectional golden sets with per-case license provenance — hand-authored forward + reverse sets (100% passing, including the machine-inverted reverse set), and ~2,700 gradeable imported cases across six languages with per-language CI regression baselines (`npm run eval`, `npm run baselines`). See [corpus/ATTRIBUTIONS.md](corpus/ATTRIBUTIONS.md) and [docs/porting.md](docs/porting.md).
+
+## Test-data provenance
+
+Every imported conformance case carries machine-readable provenance (upstream project, license, file path, pinned commit, index), and upstream licenses are vendored verbatim beside the data. **[corpus/ATTRIBUTIONS.md](corpus/ATTRIBUTIONS.md)** is the authoritative record: what is vendored (Microsoft Recognizers-Text, MIT), what is original (the hand-authored and machine-inverted golden sets), what is used eval-only and never redistributed (TempEval-3 Platinum, WikiWars), what was deliberately excluded on license grounds (GPL/LDC/ODbL sources), and the academic lineage (TimeML/TIMEX3, SCATE, TempEval-3, and the LLM date-arithmetic benchmarks that motivated the deterministic-IR design). Known upstream data defects are documented there too, with the exclusion rules applied at import.
+
+## Python
+
+A Python engine port lives in [`python/`](python/) (PyPI packaging prepared; `whenever`-based time model): IR validation, `TimeContext`, and the full deterministic resolver at **100% bit-exact parity** with the JS reference across all 2,760 engine-parity fixtures (`corpus/ir/`), enforced in CI on every push. Parsers and `describe()` are JS-only so far — [docs/porting.md](docs/porting.md) has the strategy and the divergence gates.
+
+## Release history
 
-**Corpus** ([corpus/](corpus/), runner exported as `nl2time/corpus`): a bidirectional golden set with per-case license provenance — 41 hand-authored NL→time cases (all passing), a 65-case datetime→NL golden set (all passing; to our knowledge the first published eval set for humanized time generation), and 693 cases imported from Microsoft Recognizers-Text (MIT) of which 12.6% currently pass — the honest coverage baseline the parser climbs against (`npm run eval`, `npm run baselines`). See [corpus/ATTRIBUTIONS.md](corpus/ATTRIBUTIONS.md) and [docs/porting.md](docs/porting.md).
+See **[CHANGELOG.md](CHANGELOG.md)** and [GitHub releases](https://github.com/AndyFooBlah/nl2time/releases). Current: v0.3.0 (multilingual parsing).
 
 ## Development
 

package/dist/corpus/runner.d.ts

@@ -1,6 +1,7 @@
 import { type TimeContextOptions } from '../context.js';
 import { type DescribeOptions } from '../describe/index.js';
 import type { CalendarAmount, Grain } from '../ir/types.js';
+import type { DomainPack } from '../packs/index.js';
 export interface SourceInfo {
     name: string;
     license: string;
@@ -36,6 +37,8 @@ export interface ForwardCase {
     text: string;
     ctx: TimeContextOptions;
     expect: ForwardExpect;
+    /** Domain packs to activate for this case (docs/extending.md). */
+    packs?: DomainPack[];
     /** Default: 'core'. */
     level?: 'core' | 'aspirational';
     source?: SourceInfo;

package/dist/corpus/runner.js

@@ -119,7 +119,7 @@ export function runForwardCase(c) {
     catch (e) {
         return { id: c.id, pass: false, detail: `context: ${String(e)}` };
     }
-    const { matches } = parse(c.text, ctx);
+    const { matches } = parse(c.text, ctx, c.packs ? { packs: c.packs } : undefined);
     if (c.expect.noMatch) {
         return matches.length === 0
             ? { id: c.id, pass: true }

package/dist/index.d.ts

@@ -3,6 +3,10 @@ export type { TimeExpr, Grain, Unit, Weekday, DayPeriod, CalendarAmount, Mod, Ta
 export { IR_VERSION } from './ir/types.js';
 export { validateExpr, IRValidationError } from './ir/validate.js';
 export { resolve, resolveOne, NotResolvableError, AmbiguityError, type TimeValue, type Resolution, } from './engine/resolve.js';
-export { parse, type ParseResult, type ParseMatch } from './parse/index.js';
+export { parse, createParser, type ParseResult, type ParseMatch, type ParseOptions, } from './parse/index.js';
+export { EN_RULE_ENTRIES, type Rule, type RuleMatch } from './parse/en.js';
+export { SUPPORTED_LANGUAGES, languageDef } from './parse/languages.js';
+export { makeLatinRules, type LatinLexicon } from './parse/latin.js';
+export { compilePack, validatePack, PackError, type DomainPack, type VocabEntry, } from './packs/index.js';
 export { describe, type Description, type DescribeOptions, type Framing, } from './describe/index.js';
 export { Temporal } from './clock/index.js';

package/dist/index.js

@@ -2,6 +2,10 @@ export { TimeContext, ConfigError, } from './context.js';
 export { IR_VERSION } from './ir/types.js';
 export { validateExpr, IRValidationError } from './ir/validate.js';
 export { resolve, resolveOne, NotResolvableError, AmbiguityError, } from './engine/resolve.js';
-export { parse } from './parse/index.js';
+export { parse, createParser, } from './parse/index.js';
+export { EN_RULE_ENTRIES } from './parse/en.js';
+export { SUPPORTED_LANGUAGES, languageDef } from './parse/languages.js';
+export { makeLatinRules } from './parse/latin.js';
+export { compilePack, validatePack, PackError, } from './packs/index.js';
 export { describe, } from './describe/index.js';
 export { Temporal } from './clock/index.js';

package/dist/packs/index.d.ts

@@ -0,0 +1,39 @@
+import type { Rule } from '../parse/en.js';
+export interface VocabEntry {
+    /** Alternative phrasings, matched case-insensitively over tokens. */
+    phrases: string[];
+    /** TimeExpr template; integers may be capture references (see module doc). */
+    expr?: unknown;
+    /**
+     * Shorthand for a named time-of-day interval in local 24h hours
+     * ("swing shift" 16→24). `before` ≤ `from` wraps past midnight.
+     * Exactly one of `expr` / `period` is required.
+     */
+    period?: {
+        from: number;
+        before: number;
+    };
+    /** How the match combines with neighbors (default: 'date'; periods: 'time'). */
+    role?: 'date' | 'time' | 'datetime' | 'duration';
+    /** Default 0.98 — above built-ins, so equal-length matches shadow them. */
+    confidence?: number;
+}
+export interface DomainPack {
+    name: string;
+    /** Built-in rule names to turn off entirely (see EN_RULE_ENTRIES). */
+    disable?: string[];
+    vocabulary: VocabEntry[];
+    /**
+     * Suggested TimeContext options for this domain (weekStart, dayPeriods, …).
+     * Not applied automatically — merge into TimeContext.make() yourself, since
+     * the context may also carry per-user settings.
+     */
+    context?: Record<string, unknown>;
+}
+export declare class PackError extends Error {
+    constructor(message: string);
+}
+/** Compile a pack's vocabulary into a single scanner rule. */
+export declare function compilePack(pack: DomainPack): Rule;
+/** Structural validation for untrusted pack JSON (config files, uploads). */
+export declare function validatePack(input: unknown): DomainPack;

package/dist/packs/index.js

@@ -0,0 +1,156 @@
+import { IRValidationError, validateExpr } from '../ir/validate.js';
+export class PackError extends Error {
+    constructor(message) {
+        super(message);
+        this.name = 'PackError';
+    }
+}
+const CAPTURE_RE = /\{(n|yr)\}/g;
+function compilePhrase(phrase) {
+    return phrase
+        .trim()
+        .toLowerCase()
+        .split(/\s+/)
+        .map((seg) => {
+        const captures = [...seg.matchAll(CAPTURE_RE)].map((m) => m[1]);
+        if (captures.length === 0)
+            return { kind: 'word', value: seg };
+        if (seg === '{n}' || seg === '{yr}')
+            return { kind: 'number', capture: captures[0] };
+        // Embedded captures: "fy{yr}" → /^fy(\d{1,4})$/
+        const source = seg
+            .replace(/[.*+?^${}()|[\]\\]/g, (c) => (c === '{' || c === '}' ? c : `\\${c}`))
+            .replace(CAPTURE_RE, '(\\d{1,4})');
+        return { kind: 'embedded', regex: new RegExp(`^${source}$`), captures };
+    });
+}
+function normalizeCapture(name, raw) {
+    if (name === 'yr' && raw < 100)
+        return 2000 + raw;
+    return raw;
+}
+/** Deep-substitute capture references in a template; returns a fresh tree. */
+function substitute(node, captures) {
+    if (Array.isArray(node))
+        return node.map((x) => substitute(x, captures));
+    if (typeof node === 'object' && node !== null) {
+        const obj = node;
+        if (typeof obj['$'] === 'string') {
+            const base = captures[obj['$']];
+            if (base === undefined)
+                throw new PackError(`unbound capture "${obj['$']}"`);
+            const scale = typeof obj['scale'] === 'number' ? obj['scale'] : 1;
+            const offset = typeof obj['offset'] === 'number' ? obj['offset'] : 0;
+            return base * scale + offset;
+        }
+        const out = {};
+        for (const [k, v] of Object.entries(obj))
+            out[k] = substitute(v, captures);
+        return out;
+    }
+    return node;
+}
+function periodTemplate(period) {
+    return {
+        op: 'between',
+        start: { op: 'literal', time: { hour: period.from } },
+        end: { op: 'literal', time: { hour: period.before === 24 ? 0 : period.before } },
+    };
+}
+function word(t) {
+    return t?.type === 'word' ? t.value : undefined;
+}
+function compileEntry(entry, packName, index) {
+    if ((entry.expr === undefined) === (entry.period === undefined)) {
+        throw new PackError(`${packName}[${index}]: exactly one of expr/period is required`);
+    }
+    const template = entry.expr ?? periodTemplate(entry.period);
+    const role = entry.role ?? (entry.period ? 'time' : 'date');
+    const confidence = entry.confidence ?? 0.98;
+    // Fail fast: validate the template with sample captures.
+    try {
+        validateExpr(substitute(template, { n: 1, yr: 2026 }));
+    }
+    catch (e) {
+        if (e instanceof IRValidationError || e instanceof PackError) {
+            throw new PackError(`${packName}[${index}]: invalid template — ${e.message}`);
+        }
+        throw e;
+    }
+    return entry.phrases.map((p) => ({
+        segments: compilePhrase(p),
+        template,
+        role,
+        confidence,
+    }));
+}
+/** Compile a pack's vocabulary into a single scanner rule. */
+export function compilePack(pack) {
+    const entries = pack.vocabulary.flatMap((e, idx) => compileEntry(e, pack.name, idx));
+    return (tokens, i) => {
+        let best;
+        for (const entry of entries) {
+            const captures = {};
+            let ok = true;
+            for (const [si, seg] of entry.segments.entries()) {
+                const t = tokens[i + si];
+                if (seg.kind === 'word') {
+                    const w = word(t);
+                    // Literal numeric segments ("4") also match number tokens.
+                    if (w !== seg.value && !(t?.type === 'number' && String(t.value) === seg.value)) {
+                        ok = false;
+                        break;
+                    }
+                }
+                else if (seg.kind === 'number') {
+                    if (t?.type !== 'number') {
+                        ok = false;
+                        break;
+                    }
+                    captures[seg.capture] = normalizeCapture(seg.capture, t.value);
+                }
+                else {
+                    const w = word(t);
+                    const m = w?.match(seg.regex);
+                    if (!m) {
+                        ok = false;
+                        break;
+                    }
+                    seg.captures.forEach((name, ci) => {
+                        captures[name] = normalizeCapture(name, Number(m[ci + 1]));
+                    });
+                }
+            }
+            if (!ok)
+                continue;
+            const consumed = entry.segments.length;
+            if (best && consumed <= best.consumed)
+                continue;
+            best = {
+                expr: substitute(entry.template, captures),
+                consumed,
+                confidence: entry.confidence,
+                role: entry.role,
+            };
+        }
+        return best;
+    };
+}
+/** Structural validation for untrusted pack JSON (config files, uploads). */
+export function validatePack(input) {
+    if (typeof input !== 'object' || input === null)
+        throw new PackError('pack must be an object');
+    const p = input;
+    if (typeof p.name !== 'string' || p.name === '')
+        throw new PackError('pack.name required');
+    if (!Array.isArray(p.vocabulary) || p.vocabulary.length === 0) {
+        throw new PackError('pack.vocabulary must be a non-empty array');
+    }
+    if (p.disable !== undefined && !Array.isArray(p.disable)) {
+        throw new PackError('pack.disable must be an array of rule names');
+    }
+    // compileEntry performs the deep checks (and template validation).
+    const pack = input;
+    pack.vocabulary.forEach((e, idx) => compileEntry(e, pack.name, idx));
+    return pack;
+}

package/dist/parse/de.d.ts

@@ -0,0 +1,9 @@
+import type { Rule } from './en.js';
+import { type LatinLexicon } from './latin.js';
+export declare const DE_LEXICON: LatinLexicon;
+export declare const DE_RULE_ENTRIES: {
+    name: string;
+    rule: Rule;
+}[];
+/** Connector words allowed between merged date & time parts. */
+export declare const DE_CONNECTORS: string[];