ccrt 0.1.0__tar.gz

ccrt-0.1.0/CHANGELOG.md

@@ -0,0 +1,44 @@
+# Changelog
+
+## 0.1.0
+
+- Added source-preserving UCDP GED CSV/API and ACLED event CSV adapters, validation
+  reports, explicit snapshot versions, and input fingerprints.
+- Added bounded authenticated API paging/retries with mocked failure tests and
+  verified the live endpoint's zero-based page numbering.
+- Preserved published UCDP fatality estimates that fall outside low/best/high
+  numerical order, with a warning for each affected record.
+- Replaced automatic one-to-one assignment with all candidate links, date intervals,
+  precision warnings, coarse-location handling, and explicit review decisions.
+- Added JSON run/review bundles, review CSV import/export, nullable pandas tables,
+  and metadata-preserving Parquet.
+- Added a resumable terminal reviewer that groups competing candidates and displays
+  the original source accounts while saving each decision.
+- Added an optional experimental full-screen review dashboard using terminal curses,
+  isolated from the review data model for easy removal.
+- Added review reports with candidate coverage, unique-record counts, unlinked
+  reasons, and accepted-link diagnostics. Partial reviews never produce precision
+  or recall metrics.
+- Added installed-wheel workflow checks for core and tables installations, and
+  extended package CI beyond the CLI help check.
+- Added MIT licensing, CLI subcommands, tutorial notebook, synthetic evaluation,
+  benchmark tooling, CI, and package build checks.
+
+### Migration from the unreleased initial prototype
+
+`Incident` is replaced by `SourceRecord`; `load_csv` now returns a `Dataset`.
+`reconcile(left, right, config=MatchConfig(...))` accepts datasets and returns
+`.candidates` plus `.unlinked`, replacing greedy `.matches` and unmatched lists.
+Use `total_deaths_low/best/high` and `civilian_deaths` explicitly. The CLI now uses
+`import`, `fetch-ucdp`, `match`, `review`, and `export` subcommands. These changes
+precede the first public release; no backward-compatibility shim is provided.
+
+### Validation scope
+
+`0.1.0` is an experimental library. Independent real-data linkage
+evaluation is not a publication gate for the library, but remains necessary before
+claiming empirical matching accuracy. A prior UCDP GED 26.1 Iraq 2020 query
+retrieved 282 records with 10 validation warnings; a real ACLED export supplied
+6,306 filtered Iraq 2020 records with no import issues. Candidate generation
+produced 1,329 pairs. Partial manual reviews do not establish precision or recall.
+See `docs/releasing.md` for the publication procedure.

ccrt-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,24 @@
+# Contributing and learning
+
+Install `.[tables,dev]` into a virtual environment. Run `python -m pytest`,
+`ruff check .`, and `ruff format --check .` before proposing a change. Tests use
+invented examples and mocked HTTP; they must never require private data or API keys.
+
+Work on one behaviour at a time: explain the requirement, inspect its source
+contract, write a test that demonstrates the behaviour, implement it, and review
+its effect on real research assumptions. Good first exercises are in the notebook.
+
+New source adapters need a documented field mapping and source codebook reference,
+fixtures that can be distributed, validation for malformed input, and a round-trip
+provenance test. Do not infer civilian deaths from total deaths. Changes to date
+uncertainty or candidate ranking need an updated algorithm ID and evaluation notes.
+Reports of matching quality need independently adjudicated reference labels from a
+declared sample; synthetic metrics and partially reviewed candidates are not
+real-data accuracy estimates. Keep source records and evidence within their access
+and redistribution terms.
+
+Open a GitHub issue with the version, a minimal synthetic reproduction, expected
+behaviour, and actual behaviour. Remove credentials and private source material.
+We aim to review contributions as maintainer time allows; no support response SLA
+is offered for the preview. Python 3.10–3.14 are tested in CI; optional pandas and
+PyArrow support is tested separately from the core installation.

ccrt-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Fabian Figueroa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ccrt-0.1.0/MANIFEST.in

@@ -0,0 +1,5 @@
+include LICENSE CHANGELOG.md CONTRIBUTING.md
+include docs/api.md docs/releasing.md docs/source-mappings.md docs/usage.md docs/validation.md
+recursive-include examples *.csv *.ipynb
+recursive-include tests *.py
+recursive-include benchmarks *.py

ccrt-0.1.0/PKG-INFO

@@ -0,0 +1,118 @@
+Metadata-Version: 2.4
+Name: ccrt
+Version: 0.1.0
+Summary: Experimental toolkit for reviewing candidate links between conflict incident records
+Author: Fabian Figueroa
+License-Expression: MIT
+Project-URL: Repository, https://github.com/FabianFig/ccrt
+Project-URL: Issues, https://github.com/FabianFig/ccrt/issues
+Keywords: conflict,record-linkage,ucdp,acled,research
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: httpx<1,>=0.27
+Requires-Dist: python-dotenv<2,>=1
+Provides-Extra: tables
+Requires-Dist: pandas>=2.2; extra == "tables"
+Requires-Dist: pyarrow>=15; extra == "tables"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.9; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=6; extra == "dev"
+Requires-Dist: nbformat>=5.10; extra == "dev"
+Requires-Dist: nbclient>=0.10; extra == "dev"
+Requires-Dist: ipykernel>=6; extra == "dev"
+Dynamic: license-file
+
+# Conflict Casualty Reconciliation Toolkit (CCRT)
+
+CCRT compares source accounts of conflict incidents. It imports UCDP GED and ACLED
+records, proposes possible links, supports human review, and exports results while
+preserving each source's original claims.
+
+**Status:** `0.1.0` is an experimental research library.
+
+## Installation
+
+CCRT supports Python 3.10–3.14. To install a published release:
+
+```bash
+python -m pip install ccrt
+```
+
+Install `ccrt[tables]` for pandas and Parquet support. From a checkout, use
+`python -m pip install -e .`, or `python -m pip install -e '.[tables,dev]'` for the
+project's development checks.
+
+## Quickstart
+
+The example files contain invented incidents in Exampleland. From the repository
+root, with an unused `work/` directory:
+
+```bash
+mkdir -p work
+ccrt import ucdp examples/ucdp_synthetic.csv work/ucdp.json --version 26.1
+ccrt import acled examples/acled_synthetic.csv work/acled.json --version synthetic-2026-09-24
+ccrt match work/ucdp.json work/acled.json work/matches
+ccrt report work/matches/run.json --review work/matches/review.json
+```
+
+The match run contains three candidate links and four unlinked source records.
+Inspect `work/matches/candidates.csv`, then use the [usage guide](docs/usage.md) to
+record decisions and export reviewed results. CCRT does not overwrite existing
+outputs.
+
+## Python API
+
+```python
+from ccrt import MatchConfig, load_acled_csv, load_ucdp_csv, reconcile
+
+ucdp = load_ucdp_csv("examples/ucdp_synthetic.csv", version="26.1")
+acled = load_acled_csv("examples/acled_synthetic.csv", version="synthetic-2026-09-24")
+result = reconcile(ucdp, acled, config=MatchConfig(max_days=3, max_km=25))
+
+for candidate in result.candidates:
+    print(candidate.left.source_id, candidate.right.source_id, candidate.score)
+```
+
+The [API reference](docs/api.md) covers review decisions, saved runs, reports, and
+exports.
+
+## Sources and capabilities
+
+- UCDP GED 25.1 and 26.1 CSV files with modern field names, and authenticated GED
+  API access.
+- ACLED event CSV exports with current lowercase field names. The supplied version
+  is a label for the exact export snapshot.
+- Source-preserving JSON bundles, candidate generation, CSV and terminal review,
+  review reports, CSV export, and optional Parquet export.
+
+See [source mappings](docs/source-mappings.md) for supported fields, precision
+handling, and provenance. The [teaching notebook](examples/reconciliation_walkthrough.ipynb)
+provides another offline example.
+
+## Limitations
+
+CCRT compares source accounts; it does not establish objective incident identity.
+Candidate scores are uncalibrated proximity rankings, and generating a candidate
+does not confirm correspondence. Real-world linkage accuracy has not been
+independently evaluated. CCRT does not reconcile casualty totals or estimate
+underreporting. See [validation notes](docs/validation.md) for software checks and
+research limits.
+
+## Contributing and licence
+
+See [CONTRIBUTING.md](CONTRIBUTING.md), the [changelog](CHANGELOG.md), and the
+[release procedure](docs/releasing.md). The code and synthetic examples are under
+the [MIT licence](LICENSE). UCDP and ACLED data have separate access, attribution,
+and redistribution terms; obtain source access and check the terms for your snapshot.

ccrt-0.1.0/README.md

@@ -0,0 +1,82 @@
+# Conflict Casualty Reconciliation Toolkit (CCRT)
+
+CCRT compares source accounts of conflict incidents. It imports UCDP GED and ACLED
+records, proposes possible links, supports human review, and exports results while
+preserving each source's original claims.
+
+**Status:** `0.1.0` is an experimental research library.
+
+## Installation
+
+CCRT supports Python 3.10–3.14. To install a published release:
+
+```bash
+python -m pip install ccrt
+```
+
+Install `ccrt[tables]` for pandas and Parquet support. From a checkout, use
+`python -m pip install -e .`, or `python -m pip install -e '.[tables,dev]'` for the
+project's development checks.
+
+## Quickstart
+
+The example files contain invented incidents in Exampleland. From the repository
+root, with an unused `work/` directory:
+
+```bash
+mkdir -p work
+ccrt import ucdp examples/ucdp_synthetic.csv work/ucdp.json --version 26.1
+ccrt import acled examples/acled_synthetic.csv work/acled.json --version synthetic-2026-09-24
+ccrt match work/ucdp.json work/acled.json work/matches
+ccrt report work/matches/run.json --review work/matches/review.json
+```
+
+The match run contains three candidate links and four unlinked source records.
+Inspect `work/matches/candidates.csv`, then use the [usage guide](docs/usage.md) to
+record decisions and export reviewed results. CCRT does not overwrite existing
+outputs.
+
+## Python API
+
+```python
+from ccrt import MatchConfig, load_acled_csv, load_ucdp_csv, reconcile
+
+ucdp = load_ucdp_csv("examples/ucdp_synthetic.csv", version="26.1")
+acled = load_acled_csv("examples/acled_synthetic.csv", version="synthetic-2026-09-24")
+result = reconcile(ucdp, acled, config=MatchConfig(max_days=3, max_km=25))
+
+for candidate in result.candidates:
+    print(candidate.left.source_id, candidate.right.source_id, candidate.score)
+```
+
+The [API reference](docs/api.md) covers review decisions, saved runs, reports, and
+exports.
+
+## Sources and capabilities
+
+- UCDP GED 25.1 and 26.1 CSV files with modern field names, and authenticated GED
+  API access.
+- ACLED event CSV exports with current lowercase field names. The supplied version
+  is a label for the exact export snapshot.
+- Source-preserving JSON bundles, candidate generation, CSV and terminal review,
+  review reports, CSV export, and optional Parquet export.
+
+See [source mappings](docs/source-mappings.md) for supported fields, precision
+handling, and provenance. The [teaching notebook](examples/reconciliation_walkthrough.ipynb)
+provides another offline example.
+
+## Limitations
+
+CCRT compares source accounts; it does not establish objective incident identity.
+Candidate scores are uncalibrated proximity rankings, and generating a candidate
+does not confirm correspondence. Real-world linkage accuracy has not been
+independently evaluated. CCRT does not reconcile casualty totals or estimate
+underreporting. See [validation notes](docs/validation.md) for software checks and
+research limits.
+
+## Contributing and licence
+
+See [CONTRIBUTING.md](CONTRIBUTING.md), the [changelog](CHANGELOG.md), and the
+[release procedure](docs/releasing.md). The code and synthetic examples are under
+the [MIT licence](LICENSE). UCDP and ACLED data have separate access, attribution,
+and redistribution terms; obtain source access and check the terms for your snapshot.

ccrt-0.1.0/benchmarks/benchmark.py

@@ -0,0 +1,57 @@
+"""Reproducible synthetic throughput baseline. Run after installing CCRT."""
+
+import argparse
+import json
+import platform
+import time
+import tracemalloc
+from datetime import date, timedelta
+
+from ccrt import Dataset, MatchConfig, SourceRecord, reconcile
+from ccrt.schema import digest
+
+
+def run(count):
+    start = date(2000, 1, 1)
+    datasets = []
+    for source in ("left", "right"):
+        records = tuple(
+            SourceRecord(
+                source,
+                str(i),
+                "synthetic-v1",
+                start + timedelta(days=i % 3650),
+                start + timedelta(days=i % 3650),
+                latitude=(i % 90) - 45,
+                longitude=(i % 180) - 90,
+                country="Exampleland",
+                date_precision=1,
+                geo_precision=1,
+            )
+            for i in range(count)
+        )
+        datasets.append(Dataset(source, "synthetic-v1", records, digest([r.to_dict() for r in records])))
+    tracemalloc.start()
+    began = time.perf_counter()
+    result = reconcile(*datasets, config=MatchConfig(max_days=1, max_km=5))
+    elapsed = time.perf_counter() - began
+    _, peak = tracemalloc.get_traced_memory()
+    tracemalloc.stop()
+    return {
+        "records_per_source": count,
+        "candidates": len(result.candidates),
+        "elapsed_seconds": round(elapsed, 3),
+        "peak_python_mib": round(peak / 1024**2, 2),
+        "python": platform.python_version(),
+        "platform": platform.platform(),
+        "distribution": "3650 dates, repeated deterministic coordinates; max_days=1, max_km=5",
+    }
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--records", type=int, default=10000)
+    args = parser.parse_args()
+    if not 1 <= args.records <= 100000:
+        parser.error("--records must be between 1 and 100000")
+    print(json.dumps(run(args.records), indent=2))

ccrt-0.1.0/benchmarks/evaluate.py

@@ -0,0 +1,42 @@
+"""Sensitivity analysis on exhaustively labeled, bounded inputs."""
+
+import argparse
+import csv
+import json
+from dataclasses import asdict
+
+from ccrt import MatchConfig, load_dataset, reconcile
+from ccrt.evaluation import evaluate_candidates
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("left")
+    parser.add_argument("right")
+    parser.add_argument("truth_csv", help="All true pairs: left_id,right_id")
+    parser.add_argument(
+        "--exhaustive",
+        action="store_true",
+        required=True,
+        help="Acknowledge that unlabeled pairs in these inputs are negative",
+    )
+    args = parser.parse_args()
+    with open(args.truth_csv, newline="", encoding="utf-8") as stream:
+        truth = {(row["left_id"], row["right_id"]) for row in csv.DictReader(stream)}
+    left, right = load_dataset(args.left), load_dataset(args.right)
+    reports = []
+    for max_days, max_km, coarse in ((0, 1, False), (3, 25, False), (3, 25, True)):
+        config = MatchConfig(max_days=max_days, max_km=max_km, include_coarse_locations=coarse)
+        result = reconcile(left, right, config=config)
+        reports.append(
+            {
+                "config": config.to_dict(),
+                "run_id": result.run_id,
+                **asdict(evaluate_candidates(result, truth)),
+            }
+        )
+    print(json.dumps(reports, indent=2))
+
+
+if __name__ == "__main__":
+    main()

ccrt-0.1.0/benchmarks/smoke_install.py

@@ -0,0 +1,124 @@
+"""Exercise an installed wheel outside its checkout, with optional table support.
+
+Run using a fresh environment's Python from a directory outside this repository:
+    /tmp/ccrt-smoke/bin/python /path/to/repo/benchmarks/smoke_install.py
+Then install the wheel's tables extra and repeat with --tables.
+"""
+
+import argparse
+import csv
+import importlib.util
+import json
+import os
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+import ccrt
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--tables", action="store_true")
+    args = parser.parse_args()
+    checkout = Path(__file__).resolve().parents[1]
+    installed = Path(ccrt.__file__).resolve()
+    if installed.is_relative_to(checkout):
+        raise RuntimeError("smoke test imported the checkout; use a fresh wheel installation")
+    if not args.tables:
+        assert importlib.util.find_spec("pandas") is None, "core smoke environment contains pandas"
+        assert importlib.util.find_spec("pyarrow") is None, "core smoke environment contains PyArrow"
+    console = Path(sys.executable).parent / ("ccrt.exe" if os.name == "nt" else "ccrt")
+    child_env = dict(os.environ)
+    child_env.pop("PYTHONPATH", None)
+
+    with tempfile.TemporaryDirectory(prefix="ccrt-installed-workflow-") as directory:
+        work = Path(directory)
+
+        def run(*arguments):
+            completed = subprocess.run(
+                [str(console), *map(str, arguments)],
+                cwd=work,
+                env=child_env,
+                capture_output=True,
+                text=True,
+            )
+            if completed.returncode:
+                raise RuntimeError(f"ccrt {' '.join(map(str, arguments))}: {completed.stderr}")
+            return completed.stdout
+
+        assert "Reviewable" in run("--help")
+        assert run("--version").strip() == ccrt.__version__
+        run(
+            "import",
+            "ucdp",
+            checkout / "examples/ucdp_synthetic.csv",
+            work / "ucdp.json",
+            "--version",
+            "26.1",
+        )
+        run(
+            "import",
+            "acled",
+            checkout / "examples/acled_synthetic.csv",
+            work / "acled.json",
+            "--version",
+            "smoke",
+        )
+        run("match", work / "ucdp.json", work / "acled.json", work / "matches")
+        result = ccrt.load_run(work / "matches/run.json")
+        assert len(result.candidates) == 3
+        candidate = next(
+            c for c in result.candidates if (c.left.source_id, c.right.source_id) == ("U1", "A1")
+        )
+        run(
+            "review",
+            work / "matches/run.json",
+            work / "review.json",
+            "--candidate",
+            candidate.candidate_id,
+            "--status",
+            "accepted",
+            "--reviewer",
+            "Smoke test",
+        )
+        run(
+            "report",
+            work / "matches/run.json",
+            "--review",
+            work / "review.json",
+            "--output",
+            work / "report.json",
+        )
+        report = json.loads((work / "report.json").read_text())
+        assert (report["accepted_count"], report["unresolved_count"]) == (1, 2)
+        run(
+            "export",
+            work / "matches/run.json",
+            work / "review.csv",
+            "--format",
+            "review-csv",
+            "--review",
+            work / "review.json",
+        )
+        with (work / "review.csv").open(newline="", encoding="utf-8") as stream:
+            rows = list(csv.DictReader(stream))
+        assert sum(row["status"] == "accepted" for row in rows) == 1
+        assert sum(row["status"] == "unlinked" for row in rows) == 4
+        run("export", work / "ucdp.json", work / "records.csv", "--format", "csv")
+        if args.tables:
+            from ccrt.export import read_parquet
+
+            run("export", work / "ucdp.json", work / "ucdp.parquet", "--format", "parquet")
+            original = ccrt.load_dataset(work / "ucdp.json")
+            assert read_parquet(work / "ucdp.parquet") == original
+            assert len(original.to_pandas()) == len(original)
+
+    subprocess.run([sys.executable, "-m", "pip", "check"], check=True, env=child_env)
+    mode = "core + tables" if args.tables else "core"
+    print(f"Installed-package workflow passed ({mode}): {installed}")
+
+
+if __name__ == "__main__":
+    main()

ccrt-0.1.0/ccrt/__init__.py

@@ -0,0 +1,37 @@
+"""Public CCRT API. Source accounts stay separate from proposed event links."""
+
+from ccrt._version import __version__
+from ccrt.client import UCDPClient, UCDPError
+from ccrt.loaders import ValidationError, load_acled_csv, load_csv, load_ucdp_csv
+from ccrt.merge import Candidate, MatchConfig, Reconciliation, reconcile
+from ccrt.report import ReviewReport, summarize_review
+from ccrt.review import Decision, Review
+from ccrt.schema import Dataset, SourceRecord, ValidationIssue
+from ccrt.storage import load_dataset, load_review, load_run, save_dataset, save_review, save_run
+
+__all__ = [
+    "__version__",
+    "SourceRecord",
+    "Dataset",
+    "ValidationIssue",
+    "ValidationError",
+    "UCDPClient",
+    "UCDPError",
+    "load_ucdp_csv",
+    "load_acled_csv",
+    "load_csv",
+    "MatchConfig",
+    "Candidate",
+    "Reconciliation",
+    "reconcile",
+    "Decision",
+    "Review",
+    "ReviewReport",
+    "summarize_review",
+    "load_dataset",
+    "save_dataset",
+    "load_run",
+    "save_run",
+    "load_review",
+    "save_review",
+]

ccrt-0.1.0/ccrt/__main__.py

@@ -0,0 +1,3 @@
+from ccrt.cli import main
+
+raise SystemExit(main())

ccrt-0.1.0/ccrt/_version.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"