paratext-cli 0.2.2__tar.gz → 0.2.4__tar.gz

paratext_cli-0.2.4/.helical/intro.md

@@ -0,0 +1 @@
+A framework for extracting structured metadata from digitised material with multimodal models, and reviewing what it produces. Public and Apache-2.0; the NLS-specific projects and data live downstream in paratext-nls.

paratext_cli-0.2.4/.helical/tasks/PTX-1 split-the-framework-out-of-paratext-nls-and-release-it.md

@@ -0,0 +1,15 @@
+---
+id: PTX-1
+title: Split the framework out of paratext-nls and release it
+status: done
+horizon: now
+flow: clear
+created: '2026-06-28'
+updated: '2026-06-28'
+---
+
+The engine, CLI, config and plug-in contract were entangled with NLS-specific projects and data. Separating them is what makes the framework publishable at all, and forces the plug-in boundary to be real rather than notional.
+
+## Notes
+
+- Initial release of the framework: engine, CLI, config, plug-in contract, paratext.cards and paratext.sources, with a generic cards starter.

paratext_cli-0.2.4/.helical/tasks/PTX-10 remove-the-watttime-provider.md

@@ -0,0 +1,17 @@
+---
+id: PTX-10
+title: Remove the WattTime provider
+status: done
+horizon: now
+flow: clear
+labels:
+  - carbon
+created: '2026-07-01'
+updated: '2026-07-01'
+---
+
+WattTime was added for US coverage using MOER-percentile gating, then its free tier turned out to serve only CAISO_NORTH — too narrow to present as US support.
+
+## Notes
+
+- Provider removed rather than shipped with a misleading scope. Confirmed limitation documented.

paratext_cli-0.2.4/.helical/tasks/PTX-11 make-paratext-new-self-installing.md

@@ -0,0 +1,17 @@
+---
+id: PTX-11
+title: Make paratext new self-installing
+status: done
+horizon: now
+flow: clear
+labels:
+  - cli
+created: '2026-07-01'
+updated: '2026-07-01'
+---
+
+A scaffolded project that does not appear in the entry-point registry until the user runs two more commands is a scaffold that half-works by default.
+
+## Notes
+
+- `paratext new` registers the entry point and runs uv sync automatically, with --no-install to opt out. Offers to write the project config block.

paratext_cli-0.2.4/.helical/tasks/PTX-12 add-paratext-guide-so-agents-can-self-orient.md

@@ -0,0 +1,17 @@
+---
+id: PTX-12
+title: Add paratext guide so agents can self-orient
+status: done
+horizon: now
+flow: clear
+labels:
+  - cli
+created: '2026-07-03'
+updated: '2026-07-03'
+---
+
+An agent that finds paratext on the PATH has no way to learn the project contract without being told. A guide command makes the tool explain itself.
+
+## Notes
+
+- `paratext guide` added, readable from the PATH without repo access.

paratext_cli-0.2.4/.helical/tasks/PTX-13 guard-schema-prompt-and-view-drift.md

@@ -0,0 +1,15 @@
+---
+id: PTX-13
+title: Guard schema, prompt and view drift
+status: done
+horizon: now
+flow: clear
+created: '2026-07-09'
+updated: '2026-07-09'
+---
+
+A field can be added to the schema, missed in the prompt, and silently absent from the view — three files that must agree and nothing checking that they do.
+
+## Notes
+
+- audit_project added to catch the drift, the scaffolder now seeds a drift-guard test, and the discipline is documented in the README.

paratext_cli-0.2.4/.helical/tasks/PTX-14 build-the-human-corrected-gold-eval-set-in-review.md

@@ -0,0 +1,18 @@
+---
+id: PTX-14
+title: Build the human-corrected gold eval set in review
+status: done
+horizon: now
+flow: clear
+labels:
+  - evaluation
+  - ui
+created: '2026-07-16'
+updated: '2026-07-16'
+---
+
+Verdicts say whether an extraction was good enough. They do not say what the right answer was, which is what an eval set needs.
+
+## Notes
+
+- Eval-set builder in the review UI writing human-corrected gold labels, reachable from Stats, with the reviewer note sticky under the image.

paratext_cli-0.2.4/.helical/tasks/PTX-15 add-marc-and-dublin-core-export-formats.md

@@ -0,0 +1,17 @@
+---
+id: PTX-15
+title: Add MARC and Dublin Core export formats
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+created: '2026-07-16'
+updated: '2026-07-16'
+---
+
+A Hugging Face dataset serves machine-learning reuse. A library needs the same round as catalogue records.
+
+## Notes
+
+- `--format {hf,marc,dc}` added. Rights statement override included, and export accuracy reframed as a model baseline rather than a quality claim.

paratext_cli-0.2.4/.helical/tasks/PTX-16 ship-schema-json-with-each-exported-dataset.md

@@ -0,0 +1,17 @@
+---
+id: PTX-16
+title: Ship schema.json with each exported dataset
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+created: '2026-07-16'
+updated: '2026-07-16'
+---
+
+A dataset whose response_format is only described in prose cannot be reused programmatically.
+
+## Notes
+
+- Machine-readable schema.json shipped with every exported dataset.

paratext_cli-0.2.4/.helical/tasks/PTX-17 add-the-project-inspector-and-fix-sqlite-thread-safety.md

@@ -0,0 +1,17 @@
+---
+id: PTX-17
+title: Add the project inspector and fix sqlite thread safety
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-07-21'
+updated: '2026-07-21'
+---
+
+Diagnosing a misconfigured project meant reading files by hand, and the review server was touching sqlite across threads.
+
+## Notes
+
+- Project inspector added (later renamed Project configuration and linked from a homepage footer), sqlite thread safety fixed, and inspect/notices/troubleshooting documented.

paratext_cli-0.2.4/.helical/tasks/PTX-18 point-the-default-card-detector-at-the-published-nls-repo.md

@@ -0,0 +1,17 @@
+---
+id: PTX-18
+title: Point the default card detector at the published NLS repo
+status: done
+horizon: now
+flow: clear
+labels:
+  - index-cards
+created: '2026-07-21'
+updated: '2026-07-21'
+---
+
+The default detector weights were a local path, so the cards extra only worked on the machine that trained them.
+
+## Notes
+
+- Default detector points at the published NationalLibraryOfScotland repo; a local path remains supported. Card tooling made opt-in and the crop fallback fixed.

paratext_cli-0.2.4/.helical/tasks/PTX-19 add-the-scope-aware-export-modal.md

@@ -0,0 +1,18 @@
+---
+id: PTX-19
+title: Add the scope-aware export modal
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+  - ui
+created: '2026-07-23'
+updated: '2026-07-23'
+---
+
+Export from the CLI requires knowing the flags. From the review UI it should be a choice of scope and format, with the field mapping visible and editable.
+
+## Notes
+
+- Export API plus a modal offering MARC, DC and JSONL with an editable mapping, skip toggles, and JSONL scoped to the full round.

paratext_cli-0.2.4/.helical/tasks/PTX-2 add-the-inbuilt-review-web-ui.md

@@ -0,0 +1,17 @@
+---
+id: PTX-2
+title: Add the inbuilt review web UI
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-06-28'
+updated: '2026-06-28'
+---
+
+Extraction output is unreadable as JSONL. Reviewing it needs the image and the extracted fields side by side, which means a server ships with the framework rather than beside it.
+
+## Notes
+
+- `paratext review` added, defaulting to a ./review root that lists all projects and detecting an already-running server.

paratext_cli-0.2.4/.helical/tasks/PTX-20 move-the-review-ui-onto-oat.md

@@ -0,0 +1,17 @@
+---
+id: PTX-20
+title: Move the review UI onto Oat
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-07-23'
+updated: '2026-07-23'
+---
+
+The review UI had accumulated custom CSS duplicating what Oat already provides, and in places fighting it. Auditing every rule against Oat's own components is cheaper than maintaining a parallel design system.
+
+## Notes
+
+- Custom pill, entry-row, muted and box styling replaced by Oat's .badge, .card, .text-light, .vstack, semantic dialog, and pre/code. Two Oat bugs worked around: the bundled switch omits appearance:none, and table input styling painted over the skip switch.

paratext_cli-0.2.4/.helical/tasks/PTX-21 render-entries-as-tables-read-only-and-editable.md

@@ -0,0 +1,17 @@
+---
+id: PTX-21
+title: Render entries as tables, read-only and editable
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-07-24'
+updated: '2026-07-24'
+---
+
+Entries are tabular data and were being shown as stacked fields, which makes comparing rows across a record impossible.
+
+## Notes
+
+- Read-only and editable entry tables matching each other, with autogrowing cells. table-layout:fixed was tried and reverted — it cramped the editor. Collapsed and auxiliary fields are omitted from the eval editor.

paratext_cli-0.2.4/.helical/tasks/PTX-22 publish-gold-to-hugging-face-with-sign-in-with-hf.md

@@ -0,0 +1,17 @@
+---
+id: PTX-22
+title: Publish gold to Hugging Face with Sign in with HF
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+created: '2026-07-24'
+updated: '2026-07-24'
+---
+
+Publishing from the review UI should not require the reviewer to hold or paste an API token.
+
+## Notes
+
+- OAuth PKCE flow added. The authorisation code is delivered over BroadcastChannel because Hugging Face's COOP header severs window.opener.

paratext_cli-0.2.4/.helical/tasks/PTX-23 warn-on-incomplete-provenance-without-blocking-publication.md

@@ -0,0 +1,17 @@
+---
+id: PTX-23
+title: Warn on incomplete provenance without blocking publication
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+created: '2026-07-24'
+updated: '2026-07-24'
+---
+
+Missing provenance should be visible, but refusing to publish over it turns a documentation gap into a blocked workflow.
+
+## Notes
+
+- Non-blocking warning at export; the gap can be filled in on the dataset card.

paratext_cli-0.2.4/.helical/tasks/PTX-24 add-the-paratext-logo-and-favicon.md

@@ -0,0 +1,17 @@
+---
+id: PTX-24
+title: Add the paratext logo and favicon
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-07-24'
+updated: '2026-07-24'
+---
+
+A deployed review UI with no favicon is indistinguishable from every other tab.
+
+## Notes
+
+- Pilcrow-and-t mark added as favicon with the svg mime type served, and the logo placed in the README header.

paratext_cli-0.2.4/.helical/tasks/PTX-25 remove-duplicated-helpers-across-the-codebase.md

@@ -0,0 +1,15 @@
+---
+id: PTX-25
+title: Remove duplicated helpers across the codebase
+status: done
+horizon: now
+flow: clear
+created: '2026-07-27'
+updated: '2026-07-27'
+---
+
+The split from paratext-nls left copies of the same helper in projects and in the review server, which drift apart silently.
+
+## Notes
+
+- Single humanise() shared between projects and the review server, image_source reusing packaging.default_materialise, one config template read from the packaged example.toml, and Counter for the excluded/skipped tallies.

paratext_cli-0.2.4/.helical/tasks/PTX-26 split-store-and-dataset-loading-out-of-the-review-server.md

@@ -0,0 +1,17 @@
+---
+id: PTX-26
+title: Split Store and dataset loading out of the review server
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-07-27'
+updated: '2026-07-27'
+---
+
+The review server had grown to hold its own persistence and dataset loading, which makes both untestable in isolation.
+
+## Notes
+
+- store.py and datasets.py extracted and noted in AGENTS.md.

paratext_cli-0.2.4/.helical/tasks/PTX-27 lean-the-readme-and-split-the-reference-into-docs.md

@@ -0,0 +1,17 @@
+---
+id: PTX-27
+title: Lean the README and split the reference into docs/
+status: done
+horizon: now
+flow: clear
+labels:
+  - docs
+created: '2026-07-27'
+updated: '2026-07-27'
+---
+
+A README that documents every flag stops being an introduction. Reference material belongs somewhere a reader chooses to go.
+
+## Notes
+
+- README reduced to a narrative introduction, reference split into docs/, and config documented as kebab-case only.

paratext_cli-0.2.4/.helical/tasks/PTX-28 add-notice-and-make-the-repository-public.md

@@ -0,0 +1,17 @@
+---
+id: PTX-28
+title: Add NOTICE and make the repository public
+status: done
+horizon: now
+flow: clear
+labels:
+  - packaging
+created: '2026-08-12'
+updated: '2026-08-12'
+---
+
+The Apache-2.0 LICENSE was correct from the initial release, but no copyright holder was named anywhere in the repository — which a downstream user needs in order to attribute correctly.
+
+## Notes
+
+- NOTICE added naming NLS as copyright holder and shipped in the wheel via license-files. Repository made public 2026-08-12 after licensing sign-off, verified by anonymous clone and raw hotlink.

paratext_cli-0.2.4/.helical/tasks/PTX-29 rename-the-distribution-to-paratext-cli-and-publish-to-pypi.md

@@ -0,0 +1,17 @@
+---
+id: PTX-29
+title: Rename the distribution to paratext-cli and publish to PyPI
+status: done
+horizon: now
+flow: clear
+labels:
+  - packaging
+created: '2026-08-12'
+updated: '2026-08-12'
+---
+
+The PyPI name `paratext` belongs to an unrelated package, so the distribution needs a different name while the import package and console script stay as they are.
+
+## Notes
+
+- Distribution renamed to paratext-cli; `import paratext` and the `paratext` command are unchanged. Published the same day via PyPI trusted publishing, with release docs and a publish workflow.

paratext_cli-0.2.4/.helical/tasks/PTX-3 make-the-review-server-configurable.md

@@ -0,0 +1,17 @@
+---
+id: PTX-3
+title: Make the review server configurable
+status: done
+horizon: now
+flow: clear
+labels:
+  - ui
+created: '2026-06-28'
+updated: '2026-06-28'
+---
+
+The review server has to coexist with whatever else is on the host, and be reachable from off-box when it is deployed rather than run locally.
+
+## Notes
+
+- Configurable review-port (defaulting to 5050 to avoid ai-verify on 4000), plus --host and --db flags.

paratext_cli-0.2.4/.helical/tasks/PTX-30 make-project-discovery-work-from-any-environment.md

@@ -0,0 +1,18 @@
+---
+id: PTX-30
+title: Make project discovery work from any environment
+status: done
+horizon: now
+flow: clear
+labels:
+  - cli
+  - packaging
+created: '2026-08-12'
+updated: '2026-08-12'
+---
+
+Projects are found per environment through entry points, and a bare `paratext` invocation was resolving to the wrong interpreter and reporting a confusing argparse error for an unknown project.
+
+## Notes
+
+- The CLI hands over to a project's .venv so a bare `paratext` finds its projects. Unknown projects point at `uv run paratext` instead of argparse's invalid-choice message. The scaffolder nests into the installed package and bootstraps an empty directory.

paratext_cli-0.2.4/.helical/tasks/PTX-31 give-clear-errors-for-provider-and-source-misconfiguration.md

@@ -0,0 +1,18 @@
+---
+id: PTX-31
+title: Give clear errors for provider and source misconfiguration
+status: done
+horizon: now
+flow: clear
+labels:
+  - cli
+  - packaging
+created: '2026-08-12'
+updated: '2026-08-12'
+---
+
+A malformed base URL or a missing source directory produced a failure far from its cause.
+
+## Notes
+
+- Base URLs trimmed of endpoint paths and given a missing scheme, preflight 4xx diagnosed, a warning raised when a provider token sits in paratext.toml rather than the environment, and a clean error for a missing or non-directory source.

paratext_cli-0.2.4/.helical/tasks/PTX-32 add-a-configurable-output-budget-and-provider-extra-body-pas.md

@@ -0,0 +1,17 @@
+---
+id: PTX-32
+title: Add a configurable output budget and provider extra-body passthrough
+status: done
+horizon: now
+flow: clear
+labels:
+  - packaging
+created: '2026-08-19'
+updated: '2026-08-19'
+---
+
+Different providers expose different knobs, and a fixed output budget truncates long extractions on some models while wasting tokens on others.
+
+## Notes
+
+- Output budget configurable; provider-specific parameters passed through via extra_body.

paratext_cli-0.2.4/.helical/tasks/PTX-33 bring-marc-output-up-to-isbd-punctuation-and-leader-conventi.md

@@ -0,0 +1,18 @@
+---
+id: PTX-33
+title: Bring MARC output up to ISBD punctuation and leader conventions
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+  - marc
+created: '2026-08-19'
+updated: '2026-08-19'
+---
+
+MARC that is structurally valid but punctuated wrongly still fails cataloguer review. Nonfiling indicators, ISBD separators and the leader all carry meaning.
+
+## Notes
+
+- Nonfiling indicators, ISBD title separator, corporate name order, ISBD punctuation on the 264 imprint, encoding level 5 and ISBD form in the leader, and closing full stops on 245 and 264$c.

paratext_cli-0.2.4/.helical/tasks/PTX-34 replace-the-blind-margin-crop-with-a-content-aware-fallback.md

@@ -0,0 +1,17 @@
+---
+id: PTX-34
+title: Replace the blind margin crop with a content-aware fallback
+status: done
+horizon: now
+flow: clear
+labels:
+  - index-cards
+created: '2026-08-20'
+updated: '2026-08-20'
+---
+
+When the detector finds nothing, cropping a fixed margin discards real content on some cards and keeps background on others.
+
+## Notes
+
+- Content-aware crop fallback; stale 'uniform crop' references cleared from the detector messages.

paratext_cli-0.2.4/.helical/tasks/PTX-35 add-the-opt-in-ai-assistance-note-to-marc-and-dc-export.md

@@ -0,0 +1,18 @@
+---
+id: PTX-35
+title: Add the opt-in AI-assistance note to MARC and DC export
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+  - marc
+created: '2026-08-24'
+updated: '2026-08-24'
+---
+
+A record derived from machine extraction should be able to say so, but whether to assert it is a cataloguing policy decision, not the tool's.
+
+## Notes
+
+- AI-assistance note added as an opt-in on CLI export and as a control in the review export modal.

paratext_cli-0.2.4/.helical/tasks/PTX-36 emit-only-one-main-entry-per-marc-record.md

@@ -0,0 +1,18 @@
+---
+id: PTX-36
+title: Emit only one main entry per MARC record
+status: done
+horizon: now
+flow: clear
+labels:
+  - export
+  - marc
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+Records were being emitted with more than one 1xx field, which is invalid MARC and was caught by cataloguers in review.
+
+## Notes
+
+- A single main entry per record enforced at export.

paratext_cli-0.2.4/.helical/tasks/PTX-37 extend-cli-test-coverage-to-argparse-wiring-round-resolution.md

@@ -0,0 +1,15 @@
+---
+id: PTX-37
+title: Extend CLI test coverage to argparse wiring, round resolution and config layering
+status: todo
+horizon: now
+flow: clear
+priority: high
+labels:
+  - cli
+  - tests
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+tests/test_cli.py now exists and covers -v/--version plus a guard that paratext.__version__ matches installed metadata, since __init__.py and pyproject.toml declare the version separately. The parts that motivated the task are still uncovered: argparse wiring, _resolve_round, and config layering. These are where a change breaks a user without breaking a test.

paratext_cli-0.2.4/.helical/tasks/PTX-38 walk-a-real-run-from-a-fresh-scaffold.md

@@ -0,0 +1,15 @@
+---
+id: PTX-38
+title: Walk a real run from a fresh scaffold
+status: todo
+horizon: now
+flow: clear
+priority: high
+labels:
+  - cli
+  - tests
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+`paratext new` has been walked end to end and fixed, but a real `paratext run` from a fresh scaffold has still never been done — it needs a live endpoint. Until it has, the first thing a new user does after scaffolding is untested.

paratext_cli-0.2.4/.helical/tasks/PTX-39 add-the-paratext-dependency-when-scaffolding-into-an-existin.md

@@ -0,0 +1,15 @@
+---
+id: PTX-39
+title: Add the paratext dependency when scaffolding into an existing project
+status: todo
+horizon: next
+flow: clear
+priority: med
+labels:
+  - cli
+  - tests
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+The empty-directory bootstrap writes a pyproject declaring paratext-cli, but scaffolding into a project that already has one adds no dependency. The scaffolded audit test therefore only runs if the user happened to have paratext installed already.

paratext_cli-0.2.4/.helical/tasks/PTX-4 genericise-the-defaults-away-from-local-inference-servers.md

@@ -0,0 +1,18 @@
+---
+id: PTX-4
+title: Genericise the defaults away from local inference servers
+status: done
+horizon: now
+flow: clear
+priority: high
+labels:
+  - docs
+created: '2026-06-28'
+updated: '2026-09-02'
+---
+
+Defaults naming Lemonade and LM Studio make the framework look like it only runs against one person's setup.
+
+## Notes
+
+- Provider defaults genericised; README documents remote and hosted endpoints including Hugging Face.

paratext_cli-0.2.4/.helical/tasks/PTX-40 fix-cross-batch-accumulation-overwriting-metadata-jsonl.md

@@ -0,0 +1,14 @@
+---
+id: PTX-40
+title: Fix cross-batch accumulation overwriting metadata.jsonl
+status: todo
+horizon: next
+flow: clear
+priority: med
+labels:
+  - export
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+Exporting a second round overwrites the first round's metadata.jsonl, so a dataset cannot accumulate across batches.

paratext_cli-0.2.4/.helical/tasks/PTX-41 remove-the-silent-hf-default-from-format.md

@@ -0,0 +1,15 @@
+---
+id: PTX-41
+title: Remove the silent hf default from --format
+status: todo
+horizon: next
+flow: clear
+priority: med
+labels:
+  - cli
+  - export
+created: '2026-09-02'
+updated: '2026-09-02'
+---
+
+`--format` defaults silently to hf, so a user who meant MARC gets a Hugging Face dataset without being told. On a TTY the right behaviour is a menu; otherwise an error.