gramps-web-toolkit 0.2.0__py3-none-any.whl

gramps_web_toolkit-0.2.0.dist-info/METADATA

@@ -0,0 +1,89 @@
+Metadata-Version: 2.5
+Name: gramps-web-toolkit
+Version: 0.2.0
+Summary: Command-line tools for running AI research agents against a Gramps Web family tree without letting them write guesses into it
+Project-URL: Homepage, https://github.com/GeiserX/gramps-web-toolkit
+Project-URL: Documentation, https://geiserx.github.io/gramps-web-toolkit/
+Project-URL: Issues, https://github.com/GeiserX/gramps-web-toolkit/issues
+License-Expression: GPL-3.0-or-later
+License-File: LICENSE
+Keywords: ai-agents,family-tree,genealogy,gramps,gramps-web
+Classifier: Environment :: Console
+Classifier: Operating System :: POSIX
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Sociology :: Genealogy
+Requires-Python: >=3.11
+Requires-Dist: requests>=2.28
+Provides-Extra: mh
+Requires-Dist: playwright>=1.40; extra == 'mh'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="docs/images/banner.svg" alt="gramps-web-toolkit banner" width="100%">
+</p>
+
+<h1 align="center">gramps-web-toolkit</h1>
+
+<p align="center">
+  <a href="https://github.com/GeiserX/gramps-web-toolkit/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/GeiserX/gramps-web-toolkit/ci.yml?style=flat-square&label=CI" alt="CI"></a>
+  <a href="https://geiserx.github.io/gramps-web-toolkit/"><img src="https://img.shields.io/badge/docs-site-e8b04a?style=flat-square" alt="Docs"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/github/license/GeiserX/gramps-web-toolkit?style=flat-square" alt="License"></a>
+</p>
+
+gramps-web-toolkit is a set of command-line tools for running AI research agents against a family tree in [Gramps Web](https://www.grampsweb.org) without letting them write guesses into it. Each tool fixes a failure we hit while running about 30 research rounds over two trees of roughly 10,000 people each. The [pipeline design](docs/pipeline.md) and [every trap it taught us](docs/hard-won.md) sit next to the code.
+
+The toolkit works for genealogy in any country. One module, `spanish-poller`, is Spain-only: it watches the Spanish Ministry of Justice sede and the DEHú notification inbox. In `gwt-fetch`, `iiif` works with any IIIF archive, while `digibib`, `gazeta` and `geo` cover Spanish sources and `europeana` searches every provider in any country (`--provider Hispana` narrows it to Spain).
+
+## Features
+
+- `gwt-apply`: applies a JSON Lines change list to a tree, reads before every write, skips what is already there, and stays dry-run until you pass `--apply`
+- `gwt-check`: reads the whole tree after an apply and reports orphans, dangling links, broken birth and death pointers, likely duplicates, dates that contradict each other and what changed since the last copy, with a self-test that plants every error kind
+- `gwt-import-excel`: turns a research workbook into a gwt-apply change list for review
+- `gwt-fetch`: downloads archive images and text without a browser, with one manifest row per file: any IIIF archive, Europeana search, plus Spanish sources (DIGIBIB, the historical Gaceta de Madrid, CartoCiudad geocoding)
+- `gwt-ledger`: a shared "already searched" ledger, so parallel agents stop repeating each other's negative searches
+- `gwt-queue`: a durable "what to search next" queue that hands prioritized tasks to lanes, reopens the claims of dead lanes and backs off the tasks that keep failing
+- `gwt-leads`: reads a tree dump offline and lists the gaps (lines that stop, missing vitals, uncited facts), ranked with the root's direct ancestors first, as `gwt-queue` tasks
+- `gwt-fscache`: an on-disk HTTP cache for rate-limited genealogy APIs that never stores a block or an auth header
+- `gwt-rank`: orders record-search results by how well they fit a person in the tree, from the cached FamilySearch JSON or from page text, and never hides a row
+- `gwt-mh`: MyHeritage record search, record pages and originals through a signed-in browser
+- `gwt-read`: record search and records on FamilySearch and PARES, slow and cached so the account is not suspended
+- `gwt-gate`: one request pace shared by every agent on a machine, which backs off on a block and lets priority lanes go first
+- `gwt-rawstore`: keeps source images in an S3-compatible bucket and only a small manifest in git
+- `gwt-spanish-poller` (Spain only): watches Spanish civil-registry requests and the DEHú inbox read-only, and flags cases that vanish
+- `ocr-bench`: scores OCR, HTR and LLM readings of the same pages on how many names they recover
+
+## Quick start
+
+Watch the toolkit refuse a duplicate in two minutes, with no server. The tree in `examples/` is a snapshot of 30 invented people, and `changes.jsonl` adds a wife and a son for Tomás Ortega Sanz, but adds Tomás again as a new person although he is already in the tree.
+
+<!-- quick-start:begin -->
+```bash
+git clone https://github.com/GeiserX/gramps-web-toolkit
+cd gramps-web-toolkit
+pipx install .                    # Python 3.11+, macOS or Linux; every Python tool, needs only requests
+cd examples
+gwt check selftest                # plants every kind of error in a made-up tree and finds each one
+gwt apply changes.jsonl --snapshot tree.json.gz --log /tmp/gwt-demo.jsonl         # refused: Tomás is already I0008
+gwt apply changes-fixed.jsonl --snapshot tree.json.gz --log /tmp/gwt-demo.jsonl   # uses I0008: the plan goes through
+gwt check check tree.json.gz      # the whole tree, offline
+gwt leads --root I0001 --limit 10 tree.json.gz   # the gaps to research next, Lucía's ancestors first
+```
+<!-- quick-start:end -->
+
+`./demo.sh` in `examples/` runs the same commands and checks each exit code. Nothing above writes to a tree: without `--apply`, `gwt apply` only prints the plan. Every tool runs as `gwt <tool>` and keeps its old `gwt-<tool>` command too. `gwt mh` needs the `mh` extra (`pipx install '.[mh]'`) and a Chrome it reaches over CDP. `gwt-spanish-poller` needs Node 18+ and Playwright and is not in the package. PyPI gets the package with the first release. [Getting started](https://geiserx.github.io/gramps-web-toolkit/getting-started/) connects the tools to your own tree.
+
+## Documentation
+
+The full documentation is at https://geiserx.github.io/gramps-web-toolkit/
+
+- [Getting started](https://geiserx.github.io/gramps-web-toolkit/getting-started/): what each tool needs, install, connecting to a tree, a first dry run
+- [The research pipeline](https://geiserx.github.io/gramps-web-toolkit/pipeline/): how agents, verifiers, a judge and a writer share one tree
+- [The autonomous research loop](https://geiserx.github.io/gramps-web-toolkit/agent-loop/): the queue, leads, ledger, gate and cache wired into one loop an agent runs on its own
+- [Hard-won lessons](https://geiserx.github.io/gramps-web-toolkit/hard-won/): each failure that cost us time, data or an account, and the rule that followed
+- One page per tool: [gwt-apply](https://geiserx.github.io/gramps-web-toolkit/apply/), [gwt-check](https://geiserx.github.io/gramps-web-toolkit/check/), [gwt-import-excel](https://geiserx.github.io/gramps-web-toolkit/import-excel/), [gwt-leads](https://geiserx.github.io/gramps-web-toolkit/leads/), [gwt-queue](https://geiserx.github.io/gramps-web-toolkit/queue/), [gwt-ledger](https://geiserx.github.io/gramps-web-toolkit/ledger/), [gwt-rank](https://geiserx.github.io/gramps-web-toolkit/rank/), [gwt-fetch](https://geiserx.github.io/gramps-web-toolkit/fetch/), [gwt-mh](https://geiserx.github.io/gramps-web-toolkit/myheritage/), [gwt-rawstore](https://geiserx.github.io/gramps-web-toolkit/rawstore/), [ocr-bench](https://geiserx.github.io/gramps-web-toolkit/ocr-bench/), [gwt-spanish-poller](https://geiserx.github.io/gramps-web-toolkit/spanish-poller/), [gwt-gate](https://geiserx.github.io/gramps-web-toolkit/gate/), [gwt-fscache](https://geiserx.github.io/gramps-web-toolkit/fscache/)
+- [Development](https://geiserx.github.io/gramps-web-toolkit/development/): running the tests, building the site, the rules for a change
+
+## License
+
+[GPL-3.0-or-later](LICENSE)

gramps_web_toolkit-0.2.0.dist-info/RECORD

@@ -0,0 +1,53 @@
+gwt_apply/__init__.py,sha256=QLRPRzG1km4dvNLZ7EcN5Gmm5RK83OI4tqejG3KDYrg,487
+gwt_apply/__main__.py,sha256=k1ocEWawweo1qCJWNFAAvyxz3tcY13dzvCenHszij30,48
+gwt_apply/cli.py,sha256=DaGZRMTEvicsMbUf4q4P-hTCGEb8NenlsiZof2QyVlA,19881
+gwt_apply/client.py,sha256=OYxJycPBsgb-9RHNLB21f0hWNqHcm9VPfvMs3jBqNgM,24230
+gwt_apply/dates.py,sha256=VOPgfkiBNGRolYjhKEGu0OHs8osGcOX0nLa9GRFQnBg,4064
+gwt_apply/dupes.py,sha256=YP4PoMgI2d-GQlVJEF3Ju-Cls2uq31gyAiLerv6aa6U,12686
+gwt_apply/lock.py,sha256=9Cy6YHK4vEq3q7Dh3AMUhDZfZsSE6TVeNYJtAce75Kc,4970
+gwt_apply/ops.py,sha256=ox_axeYnND71zhuN72JoDu1HhoohtN-3yHrDIwt5qmc,100914
+gwt_apply/revert.py,sha256=NHorYE9wRsMvdgsEwS2lltA5cCdCxIpjPIcjAUfucDM,20073
+gwt_apply/snapshot.py,sha256=psfShVQwgvXWd0YmrW1qzHfNxpaZ-Uby7qbaNu-Hlng,6191
+gwt_check/__init__.py,sha256=tx35gRrKv3V97e-g0q6V8PHpgkd5NPy-4cSmNbVKoZQ,480
+gwt_check/__main__.py,sha256=vbbfAie_PIhAqDsVuH2guoAgXfs-Tghw3ail-wgzfEY,59
+gwt_check/checks.py,sha256=JWElFaLddT8MofSq8QvkBd3eo21OgEeHPk-v9Sven58,38255
+gwt_check/cli.py,sha256=Cfc25KjbSwFEcloeFumt6LIF8rH7HTR--3Ifj8w41b8,24510
+gwt_check/client.py,sha256=tz6Ps5c-siJ2XvH5WXAqqn7m4lV0HNLAtx7z6n0GCN0,10928
+gwt_check/compare.py,sha256=nxzQIjC3rtpvYJ5D6469cVurNOkj513cxMRk7Yr8Liw,5138
+gwt_check/dates.py,sha256=46wZEUAvGKkdiBq-6B3XEXrOXdCuOzGprdW5NmZ32Sk,4781
+gwt_check/fill.py,sha256=oJoLnaYIsgVlyK3aG82K4yf7SsHOPjqrw9tsOfcYFVg,5361
+gwt_check/repos.py,sha256=owljO5hhY3BV75rQRr37u1Azq0ufScUcKBuQzWyuF4Y,2824
+gwt_check/selftest.py,sha256=AAXiA1ONZF6nnZnOohEqmMLUgJOZNSmkJlOODtVxW3M,27703
+gwt_check/tree.py,sha256=X3vnMYXv2vMw8Ynoy2vLOn_BrcVSvHw3PmTjHFclCmE,7328
+gwt_core/__init__.py,sha256=toI7Uc_XBYLtbTjfhcu89-VawdDFe72Dw8GVlv43g7o,140
+gwt_core/cli.py,sha256=wBHxJ-XT1hDpqI7pnBjAJCsgpYX110_YLWpafA9rA8Y,2718
+gwt_core/version.py,sha256=nXqqbGZO6uD8x32mPRr21QiWgcXT2oWHbUWaAClO3iY,1346
+gwt_fetch/__init__.py,sha256=N-xmIYQaqj8INAyWFbgsSbuY5nNXbIX8QWv2XLOmOe0,87
+gwt_fetch/__main__.py,sha256=E6Gls0DNz8GQK2K-kOUIx8cYhgANW_CH54VKrfCfs14,52
+gwt_fetch/cli.py,sha256=SGrgp49pwO_A8EAR6EugDu309OSUFNJznqPpsARJniI,3664
+gwt_fetch/digibib.py,sha256=Fs7jYr6kzuqTbmE0B7wlFWizTnSRRGFeWaWe9Pd2IvA,3504
+gwt_fetch/europeana.py,sha256=3uMgQpRWdc6HJxjcwqNiCEygcr9p07K6PNJREFktHOI,3130
+gwt_fetch/gazeta.py,sha256=Ca7Xw_ARKI4kA5D9w4qXvOS5vSuXbjAJtjDRrDpr-bI,6023
+gwt_fetch/geo.py,sha256=4l2kRxZ9N3mGS4ioJeCPqTV6ZBblMmZ8BTg-uDoqt1k,2043
+gwt_fetch/iiif.py,sha256=_x_uSPGyoLq44hfXSh5H7zih3odCNIyLZ4OhxtfwoeI,6574
+gwt_fetch/net.py,sha256=wmy8v_aT4HbxrNFsRVV4MgJJ4MeUDwuSPy0QiL6p6BY,2751
+gwt_fetch/store.py,sha256=TpORZRXcyLDmfRKfxEpNWWk_gESwTzKNUCTg7sQgQuU,3610
+gwt_fscache.py,sha256=oJ25BQM3-x0xovDv_B-9Uog2t5rwNkYYP78HhJzUBCU,9223
+gwt_gate.py,sha256=da0HMhPEmSBS0xKnqBazKU9IvZs0cobyzdiBzswDZ2s,11506
+gwt_import_excel.py,sha256=TJiarAtNxDGZfEWdEBgKkzVHCXGQu-pu4o3ipWugnUA,27589
+gwt_leads.py,sha256=UAtjjdwO70MVu-CoaNCp5OLQPRPRDfD_SrHwWVqeXMc,21707
+gwt_ledger.py,sha256=fGNVtEt1vNDnjawiB1aWzEXzbElwEEbzTSvzBfJaVyU,11750
+gwt_mh.py,sha256=gCFtcpTsxgC2X5SDSv5ELvTTgB4ojuMqeiC__fdWW2c,40918
+gwt_mh_browser.py,sha256=miKkvMP1DYecxOziSFFqoPWceuLC2I8OPldXOc6XYxI,14120
+gwt_mh_cli.py,sha256=i47XB5FuzIH_ggVV2x5zCxm_XPXfcCHSTDB9Q4rJOgY,21361
+gwt_pares.py,sha256=lkcFAE85w3cEIbPMnnxje9UGSRUe3ImUTsNwkEQHvhQ,5372
+gwt_queue.py,sha256=67oJau5lnqVzaXW63Uxzpc2iGp1BGp6gv8EcVu6LlCc,23329
+gwt_rank.py,sha256=QTih_kPRSiqlxMnr-ytwbElx3pov2l5QmsGYBoQS6Jk,21247
+gwt_rawstore.py,sha256=42VteqJxkJCSpcHpsjiG4bz6znprotFhPwZjRfU5Ii0,20870
+gwt_read.py,sha256=9aJwi8lIdaReHw3yl6YjTkjIALgjchsWnKQgfiQOYlY,15104
+gwt_reader.py,sha256=iZG0FITiSOrcaGdBiWZrlB2IHufuYhcnBxEnJlR0PkE,13582
+gramps_web_toolkit-0.2.0.dist-info/METADATA,sha256=J8jR0JFkKMEOi6m1veHOp9DqveUq09XDe82LHPgZdic,8050
+gramps_web_toolkit-0.2.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+gramps_web_toolkit-0.2.0.dist-info/entry_points.txt,sha256=wkhGHl9Mqty0Y36GXlIT5uzdShwAkO0KXD6PuVyhxuQ,423
+gramps_web_toolkit-0.2.0.dist-info/licenses/LICENSE,sha256=5X8cMguM-HmKfS_4Om-eBqM6A1hfbgZf6pfx2G24QFI,35150
+gramps_web_toolkit-0.2.0.dist-info/RECORD,,

gramps_web_toolkit-0.2.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.4
+Root-Is-Purelib: true
+Tag: py3-none-any

gramps_web_toolkit-0.2.0.dist-info/entry_points.txt

@@ -0,0 +1,15 @@
+[console_scripts]
+gwt = gwt_core.cli:main
+gwt-apply = gwt_apply.cli:main
+gwt-check = gwt_check.cli:main
+gwt-fetch = gwt_fetch.cli:main
+gwt-fscache = gwt_fscache:main
+gwt-gate = gwt_gate:main
+gwt-import-excel = gwt_import_excel:main
+gwt-leads = gwt_leads:main
+gwt-ledger = gwt_ledger:main
+gwt-mh = gwt_core.cli:mh
+gwt-queue = gwt_queue:main
+gwt-rank = gwt_rank:main
+gwt-rawstore = gwt_rawstore:main
+gwt-read = gwt_read:main