extract-cli 0.1.6__tar.gz → 0.1.9__tar.gz

extract_cli-0.1.9/AGENTS.md

@@ -0,0 +1,87 @@
+# Agents
+
+Drive `extract-cli` from an LLM agent or non-interactive client. Same agent
+contract as the rest of the contract-ops suite: a stable machine-readable
+catalog, JSON on stdout, humans on stderr, and a small documented exit-code set.
+
+`extract-cli` is the suite's **open-loop front door**: hand it any contract
+(`.md` / `.txt` / `.html` / `.docx` / `.pdf`, yours or a counterparty's) and it
+returns structured JSON the rest of the pipeline can consume. Every field
+carries a `confidence` and a `source` — **verify, don't trust**.
+
+## Output contract
+
+- **Success**: a single JSON object to **stdout**, exit `0`. This is the machine
+  payload; it's the default (no `--json` needed, though `--json` forces it).
+- Every extracted scalar is the envelope `{value, confidence, source}`;
+  "not found" is the canonical `{value: null, confidence: 0.0, source: "none"}`.
+  Lists (`parties`, `clauses`, `defined_terms`) carry per-item
+  `confidence`/`source`. `source ∈ {deterministic, llm, none}`.
+- `_meta` records `extractor_version`, `tiers_used`, and `llm_used`.
+- The output shape is locked by a JSON Schema —
+  [`docs/spec/extract-output.schema.json`](docs/spec/extract-output.schema.json),
+  also printed by `extract schema`. Validate against it instead of trusting
+  field shapes by convention. (Note: the `--no-confidence` projection is a
+  reduced convenience view, **not** governed by the schema.)
+- **stderr** is for humans only: `--why` rationale, warnings, and errors.
+  stdout stays clean JSON even under `--why`.
+- **Failure**: a one-line `error: <message>` on **stderr**, non-zero exit.
+  The error shape is a flat string (the suite is not uniform on error-object
+  shape) — **branch on the exit code, never on the human-readable message.**
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Success. |
+| `1` | Low-signal document — no high-signal fields (parties/clauses/dates) could be extracted; e.g. a scanned/image-only or empty file. A **finding**, not a crash: valid JSON is still emitted on stdout. |
+| `2` | Bad usage / user-actionable error (unreadable path, bad flag value, unsupported completion shell). |
+
+## Discovery
+
+Never hardcode command or flag names — call the catalog at startup:
+
+```bash
+extract --catalog json    # {name, bin, version, description, commands[], exitCodes}
+```
+
+`--catalog json` is the suite-wide discovery contract (parallel to
+`nda-review-cli --catalog json`, `docx2pdf --catalog json`,
+`sign --catalog json`). It is **complete, accurate, and stable across minor
+versions** — a test asserts it never drifts from the real parser.
+
+Tool-specific discovery extras:
+
+```bash
+extract schema            # the output JSON Schema (the cross-CLI data contract)
+extract fields            # extractable fields and the tier that produces each
+extract fields --json     # ...as JSON
+extract demo              # run on a bundled fixture (zero-config first run)
+extract --version
+```
+
+## Failure → recovery
+
+| Symptom | Diagnose | Recover |
+|---|---|---|
+| Exit `1`, warning "no high-signal fields" | The document is likely scanned/image-only or has no recognizable structure. JSON is still emitted. | OCR the source first, or feed a text/`.docx`/`.md` version. The empty-but-valid JSON is safe to pass downstream. |
+| Exit `2`, `error: ...` | `extract --catalog json` (or `extract <cmd> --help`) for the real surface. | Fix the path/flag and retry. |
+| `clauses: []` on a real contract | The `.docx` likely auto-numbers via Word's numbering with no heading style (its numbers live only in `numbering.xml`), so the deterministic cascade sees no headings. | Re-run with `--llm` (opt-in): when no clauses are detected, the LLM is asked for section headings, normalized through the same canonical vocabulary and emitted with `tier: "llm"`, `source: "llm"`, and a modest confidence. Requires `~/.config/contract-ops/llm.json`. |
+| Low-fidelity `.docx`/`.pdf` text | The stdlib best-effort reader ran (no extras installed). | `pip install "extract-cli[docx]"` and/or `"extract-cli[pdf]"` for higher fidelity. The core always works without them. |
+| `--llm` only printed a warning | No LLM config found. | Copy [`config/llm.json.example`](config/llm.json.example) to `~/.config/contract-ops/llm.json`. Without it, deterministic output is still returned in full. |
+
+## Recommended usage
+
+```bash
+# Inspect any contract's structure, one tool for five formats.
+extract counterparty.docx | jq '{parties: [.parties[].name],
+  governing_law: .governing_law.value, clauses: [.clauses[].canonical_title]}'
+
+# Gate a workflow on extraction confidence (non-zero exit if any clause is shaky).
+extract draft.docx | jq -e '.clauses | all(.confidence > 0.7)' && echo ok
+```
+
+The integration contract is the **output schema** + the **shared canonical
+clause vocabulary** (`canonical_title` values match what `template-vault-cli`
+detects and `nda-review-cli` keys policy on) — not per-tool flags. See
+[`docs/INTEROP.md`](docs/INTEROP.md).

{extract_cli-0.1.6 → extract_cli-0.1.9}/CHANGELOG.md

@@ -6,6 +6,79 @@ to [Semantic Versioning](https://semver.org/). Per the suite convention
 (see [`docs/INTEROP.md`](docs/INTEROP.md)), **backward-incompatible changes to
 the output schema require a major version bump**; new optional fields are minor.
 
+## [0.1.9] - 2026-05-22
+
+### Security / robustness
+- **Resource bounds for untrusted input.** A hard on-disk file cap
+  (`MAX_INPUT_BYTES`, 100 MB) and a decompressed-size cap
+  (`MAX_DECOMPRESSED_BYTES`, 200 MB) so a zip-bomb `.docx` or zlib-bomb `.pdf`
+  can't exhaust memory: the DOCX reader checks `word/document.xml`'s
+  uncompressed size before reading, and the PDF reader decompresses streams
+  with a bounded budget. Both degrade gracefully (warning, empty text), never
+  crash. (Verified fast/bounded on a 2 MB doc: ~0.6 s, ~10 MB peak.)
+
+### Added (output schema — minor, backward-compatible additions)
+- **`jurisdiction`** — governing law normalized to a stable code
+  (`State of Delaware` → `US-DE`, `Province of Ontario` → `CA-ON`, …).
+- **`amounts[]`** — every distinct monetary amount (`value` remains the headline one).
+- **`signatories[]`** — `{name, title}` from signature blocks (`By:` / `Name:` /
+  `Title:`); empty on unsigned templates.
+
+### Changed
+- **Clause vocabulary round 2** (from the corpus survey): canonical
+  `Suspension`, `Support`, `Service Levels` + `invoicing`→Payment,
+  customer-data/protection-by-* → Data Protection. Noise filter now also drops
+  recitals/preamble/signature sections, definition fragments (a title starting
+  with a quote), and unfilled placeholders (`[ # ]%`). Mapped clause coverage
+  across the 58-document corpus rose from 57% → **64%**, no over-matching.
+- Test coverage raised to **92%** (94% with the `[docx]`/`[pdf]` extras).
+
+## [0.1.8] - 2026-05-22
+
+Clause-detection breadth, driven by a 58-document real-corpus survey.
+
+### Added
+- **Auto-numbered DOCX clauses.** The DOCX reader now treats `w:numPr` list
+  paragraphs (no heading style; number generated from `numbering.xml`) as
+  clause-heading candidates, run through the same run-in/heading-likeness filter
+  as heading styles. Real agreements that number clauses this way (data
+  processing / design-partner agreements) get a clause map where they previously
+  got none; deep numbered body sentences are still excluded. New `numbered_docx`
+  fixture + tests.
+- **Two-line `ARTICLE N` headings.** A bare `ARTICLE N` / `SECTION N` line whose
+  title sits on the next line (common in formal agreements) is detected as a
+  pair — recovering, e.g., a real SEC services agreement's clause map (0 → 8).
+  Fires only with >= 2 well-formed pairs; reported under the `numbered` tier (no
+  schema change).
+- **Expanded canonical clause vocabulary** from the corpus survey: new canonical
+  clauses `Exclusions`, `Remedies`, `Restrictions`, `Taxes`,
+  `Reservation of Rights`, `Third-Party Beneficiaries`, `Feedback`,
+  `Miscellaneous`, plus aliases for `Compliance with Laws` (anti-bribery, export
+  controls) and `Data Protection` (customer data/content). ~155 more clauses map
+  across the corpus, with no observed over-matching.
+- **`CLAUDE.md`** — codebase development notes (complements AGENTS.md).
+
+No output-schema change.
+
+## [0.1.7] - 2026-05-22
+
+### Added
+- **`extract --catalog json` — the suite's shared discovery contract.** Emits
+  `{name, bin, version, description, commands[], exitCodes}` (mirroring
+  `nda-review-cli --catalog json` / `docx2pdf --catalog json` /
+  `sign --catalog json`) so agents can learn every command and flag at startup
+  instead of hardcoding them. A test asserts the catalog never drifts from the
+  real argparse parser. Also added to the bash/zsh completion flag lists.
+- **`AGENTS.md`** — the agent contract in the suite's canonical section order
+  (output contract / exit codes / discovery / failure → recovery).
+- **`llms.txt`** — machine-readable tool summary at the repo root.
+
+### Changed
+- Packaging: added the suite-standard keywords (`contract-ops`, `agent-first`,
+  `legal-tech`); README now opens with `## Run this` / `## Where to go next`;
+  `--catalog json` documented in the README and `docs/INTEROP.md`. No schema or
+  extraction-logic change (`extractor_version` unchanged).
+
 ## [0.1.6] - 2026-05-21
 
 ### Docs
@@ -198,6 +271,9 @@ Initial release — the open-loop front door of the contract-ops CLI suite.
   intentionally *not* governed by the output schema (the schema describes the
   full default output).
 
+[0.1.9]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.9
+[0.1.8]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.8
+[0.1.7]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.7
 [0.1.6]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.6
 [0.1.5]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.5
 [0.1.4]: https://github.com/DrBaher/extract-cli/releases/tag/v0.1.4

{extract_cli-0.1.6 → extract_cli-0.1.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: extract-cli
-Version: 0.1.6
+Version: 0.1.9
 Summary: Open-loop front door of the contract-ops CLI suite: ingest any contract (.md/.txt/.html/.docx/.pdf) and emit structured JSON.
 Project-URL: Homepage, https://cli.drbaher.com/
 Project-URL: Repository, https://github.com/DrBaher/extract-cli
@@ -8,7 +8,7 @@ Project-URL: Suite interop, https://github.com/DrBaher/extract-cli/blob/main/doc
 Author-email: DrBaher <Drbaher@gmail.com>
 License: MIT
 License-File: LICENSE
-Keywords: clause,cli,contract,extraction,json,legal,nda
+Keywords: agent-first,clause,cli,contract,contract-ops,extraction,json,legal,legal-tech,nda
 Classifier: Development Status :: 4 - Beta
 Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
@@ -61,6 +61,30 @@ ingest (extract) → review → diff → convert → sign
    ^you are here
 ```
 
+## Run this
+
+```bash
+pipx run extract-cli demo        # zero-config: extract a bundled NDA → structured JSON
+# or, installed:  pip install extract-cli && extract demo
+```
+
+That prints the full output contract — parties, dates, term, governing law, and
+a clause map normalized onto the suite's canonical vocabulary — for a bundled
+fixture, with no setup and no network. Point it at your own file with
+`extract path/to/contract.docx`.
+
+## Where to go next
+
+- **New here?** Keep reading — [What it does](#what-it-does) and
+  [The two extraction tiers](#the-two-extraction-tiers).
+- **Driving it from an agent?** See [`AGENTS.md`](AGENTS.md) and call
+  `extract --catalog json` at startup to discover commands/flags. The output
+  shape is locked by [`docs/spec/extract-output.schema.json`](docs/spec/extract-output.schema.json).
+- **Wiring it into the pipeline?** See [`docs/INTEROP.md`](docs/INTEROP.md) — the
+  contract is the output schema + the shared clause vocabulary.
+- **Contributing / building a sibling CLI?** [`CONTRIBUTING.md`](CONTRIBUTING.md)
+  and [ARCHITECTURE.md](ARCHITECTURE.md).
+
 ## What it does
 
 Give it a contract in **`.md` / `.txt` / `.html`** (native), **`.docx`**, or
@@ -115,6 +139,7 @@ for them.
 
 ```bash
 extract <path>            # parse a document → structured JSON on stdout (default)
+extract --catalog json    # machine-readable catalog of commands/flags (agents call at startup)
 extract schema            # print the output JSON Schema (the cross-CLI contract)
 extract fields            # list extractable fields and their tier
 extract demo              # run on a bundled fixture and show the narrative
@@ -125,6 +150,7 @@ extract completion bash   # emit a shell-completion script (bash|zsh)
 
 | Flag | Meaning |
 |---|---|
+| `--catalog json` | Print the machine-readable command/flag catalog and exit (the suite discovery contract; agents call this at startup) |
 | `--llm` | Opt-in LLM enrichment of fuzzy fields (off by default) |
 | `--fields a,b,c` | Emit only a subset of top-level fields (e.g. `parties,clauses`) |
 | `--format json\|table` | Output format (default `json`) |
@@ -148,10 +174,13 @@ Streams follow the suite convention: **stdout** is the machine payload (JSON),
   "dates":      { "effective": { "value": "2024-03-01", "confidence": 0.85, "source": "deterministic" }, "expiration": { "value": null, "confidence": 0.0, "source": "none" } },
   "term":       { "length": { "value": "3 years", ... }, "auto_renew": { "value": true, ... }, "notice_period_days": { "value": 60, ... } },
   "governing_law": { "value": "State of Delaware", "confidence": 0.85, "source": "deterministic" },
+  "jurisdiction": { "value": "US-DE", "confidence": 0.8, "source": "deterministic" },
   "clauses":    [ { "canonical_title": "Confidentiality", "detected_title": "## Confidentiality Obligations", "tier": "h2", "span": {"start": 0, "end": 120}, "confidence": 0.95, "source": "deterministic", "mapped": true } ],
   "defined_terms": [ { "term": "Confidential Information", "confidence": 0.6, "source": "deterministic" } ],
   "value":      { "value": "$50,000", "confidence": 0.6, "source": "deterministic" },
-  "_meta":      { "extractor_version": "0.1.0", "tiers_used": ["deterministic"], "llm_used": false }
+  "amounts":    [ { "value": "$50,000", "confidence": 0.6, "source": "deterministic" } ],
+  "signatories": [ { "name": "Jane Doe", "title": "CEO", "confidence": 0.55, "source": "deterministic" } ],
+  "_meta":      { "extractor_version": "0.1.9", "tiers_used": ["deterministic"], "llm_used": false }
 }
 ```
 

{extract_cli-0.1.6 → extract_cli-0.1.9}/README.md

@@ -23,6 +23,30 @@ ingest (extract) → review → diff → convert → sign
    ^you are here
 ```
 
+## Run this
+
+```bash
+pipx run extract-cli demo        # zero-config: extract a bundled NDA → structured JSON
+# or, installed:  pip install extract-cli && extract demo
+```
+
+That prints the full output contract — parties, dates, term, governing law, and
+a clause map normalized onto the suite's canonical vocabulary — for a bundled
+fixture, with no setup and no network. Point it at your own file with
+`extract path/to/contract.docx`.
+
+## Where to go next
+
+- **New here?** Keep reading — [What it does](#what-it-does) and
+  [The two extraction tiers](#the-two-extraction-tiers).
+- **Driving it from an agent?** See [`AGENTS.md`](AGENTS.md) and call
+  `extract --catalog json` at startup to discover commands/flags. The output
+  shape is locked by [`docs/spec/extract-output.schema.json`](docs/spec/extract-output.schema.json).
+- **Wiring it into the pipeline?** See [`docs/INTEROP.md`](docs/INTEROP.md) — the
+  contract is the output schema + the shared clause vocabulary.
+- **Contributing / building a sibling CLI?** [`CONTRIBUTING.md`](CONTRIBUTING.md)
+  and [ARCHITECTURE.md](ARCHITECTURE.md).
+
 ## What it does
 
 Give it a contract in **`.md` / `.txt` / `.html`** (native), **`.docx`**, or
@@ -77,6 +101,7 @@ for them.
 
 ```bash
 extract <path>            # parse a document → structured JSON on stdout (default)
+extract --catalog json    # machine-readable catalog of commands/flags (agents call at startup)
 extract schema            # print the output JSON Schema (the cross-CLI contract)
 extract fields            # list extractable fields and their tier
 extract demo              # run on a bundled fixture and show the narrative
@@ -87,6 +112,7 @@ extract completion bash   # emit a shell-completion script (bash|zsh)
 
 | Flag | Meaning |
 |---|---|
+| `--catalog json` | Print the machine-readable command/flag catalog and exit (the suite discovery contract; agents call this at startup) |
 | `--llm` | Opt-in LLM enrichment of fuzzy fields (off by default) |
 | `--fields a,b,c` | Emit only a subset of top-level fields (e.g. `parties,clauses`) |
 | `--format json\|table` | Output format (default `json`) |
@@ -110,10 +136,13 @@ Streams follow the suite convention: **stdout** is the machine payload (JSON),
   "dates":      { "effective": { "value": "2024-03-01", "confidence": 0.85, "source": "deterministic" }, "expiration": { "value": null, "confidence": 0.0, "source": "none" } },
   "term":       { "length": { "value": "3 years", ... }, "auto_renew": { "value": true, ... }, "notice_period_days": { "value": 60, ... } },
   "governing_law": { "value": "State of Delaware", "confidence": 0.85, "source": "deterministic" },
+  "jurisdiction": { "value": "US-DE", "confidence": 0.8, "source": "deterministic" },
   "clauses":    [ { "canonical_title": "Confidentiality", "detected_title": "## Confidentiality Obligations", "tier": "h2", "span": {"start": 0, "end": 120}, "confidence": 0.95, "source": "deterministic", "mapped": true } ],
   "defined_terms": [ { "term": "Confidential Information", "confidence": 0.6, "source": "deterministic" } ],
   "value":      { "value": "$50,000", "confidence": 0.6, "source": "deterministic" },
-  "_meta":      { "extractor_version": "0.1.0", "tiers_used": ["deterministic"], "llm_used": false }
+  "amounts":    [ { "value": "$50,000", "confidence": 0.6, "source": "deterministic" } ],
+  "signatories": [ { "name": "Jane Doe", "title": "CEO", "confidence": 0.55, "source": "deterministic" } ],
+  "_meta":      { "extractor_version": "0.1.9", "tiers_used": ["deterministic"], "llm_used": false }
 }
 ```
 

{extract_cli-0.1.6 → extract_cli-0.1.9}/docs/INTEROP.md

@@ -52,10 +52,12 @@ is a self-contained reference validator.
 
 Top-level keys: `document` {title, format, sha256, source_path}, `parties[]`,
 `dates` {effective, expiration}, `term` {length, auto_renew,
-notice_period_days, *renewal_mechanics?*}, `governing_law`, `clauses[]`
-{canonical_title, detected_title, tier, span, confidence, source, mapped},
-`defined_terms[]`, `value`, *`obligations[]?`*, and `_meta` {extractor_version,
-tiers_used, llm_used}. **Every extracted field carries a `confidence` (0–1) and
+notice_period_days, *renewal_mechanics?*}, `governing_law`, `jurisdiction`
+(normalized code, e.g. `US-DE`), `clauses[]` {canonical_title, detected_title,
+tier, span, confidence, source, mapped}, `defined_terms[]`, `value`,
+`amounts[]` (all monetary amounts), `signatories[]` {name, title}, *`obligations[]?`*,
+and `_meta` {extractor_version, tiers_used, llm_used}. Formats: markdown, text,
+html, docx, pdf. **Every extracted field carries a `confidence` (0–1) and
 a `source` ∈ {deterministic, llm, none}.** Scalar fields use the envelope
 `{value, confidence, source}`; "not found" is `{value: null, confidence: 0.0,
 source: "none"}`. Italic fields are added only under `--llm`.
@@ -118,6 +120,7 @@ only stdlib `urllib`, so there is no runtime dependency.
 | Concern | Convention |
 |---|---|
 | Primary result | **stdout** (JSON payload, default) |
+| Discovery | `extract --catalog json` (commands/flags, the suite contract) + `extract schema` / `extract fields --json` |
 | `--why`, warnings, errors | **stderr** |
 | `--why` envelope | plain-text `[why] <header>` block (as in template-vault-cli / draft-cli) |
 | Quiet | `-q` / `--silent` / `--quiet` aliases |

{extract_cli-0.1.6 → extract_cli-0.1.9}/docs/spec/extract-output.schema.json

@@ -10,9 +10,12 @@
     "dates",
     "term",
     "governing_law",
+    "jurisdiction",
     "clauses",
     "defined_terms",
     "value",
+    "amounts",
+    "signatories",
     "_meta"
   ],
   "additionalProperties": false,
@@ -157,6 +160,9 @@
     "governing_law": {
       "$ref": "#/$defs/field"
     },
+    "jurisdiction": {
+      "$ref": "#/$defs/field"
+    },
     "clauses": {
       "type": "array",
       "items": {
@@ -247,6 +253,58 @@
     "value": {
       "$ref": "#/$defs/field"
     },
+    "amounts": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": [
+          "value",
+          "confidence",
+          "source"
+        ],
+        "properties": {
+          "value": {
+            "type": "string"
+          },
+          "confidence": {
+            "$ref": "#/$defs/confidence"
+          },
+          "source": {
+            "$ref": "#/$defs/source"
+          }
+        },
+        "additionalProperties": false
+      }
+    },
+    "signatories": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": [
+          "name",
+          "confidence",
+          "source"
+        ],
+        "properties": {
+          "name": {
+            "type": "string"
+          },
+          "title": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "confidence": {
+            "$ref": "#/$defs/confidence"
+          },
+          "source": {
+            "$ref": "#/$defs/source"
+          }
+        },
+        "additionalProperties": false
+      }
+    },
     "obligations": {
       "type": "array",
       "items": {