reckoner-er 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. reckoner_er-0.3.0/CHANGELOG.md +37 -0
  2. reckoner_er-0.3.0/LICENSE +21 -0
  3. reckoner_er-0.3.0/MANIFEST.in +8 -0
  4. reckoner_er-0.3.0/PKG-INFO +100 -0
  5. reckoner_er-0.3.0/README.md +268 -0
  6. reckoner_er-0.3.0/benchmarks/data/gleif_us_sample.csv +2923 -0
  7. reckoner_er-0.3.0/benchmarks/data/gleif_us_sample.meta.json +50 -0
  8. reckoner_er-0.3.0/benchmarks/fetch_gleif_sample.py +162 -0
  9. reckoner_er-0.3.0/benchmarks/run_benchmark.py +142 -0
  10. reckoner_er-0.3.0/docs/benchmark.md +91 -0
  11. reckoner_er-0.3.0/docs/pypi.md +70 -0
  12. reckoner_er-0.3.0/docs/python-api.md +109 -0
  13. reckoner_er-0.3.0/examples/api.py +31 -0
  14. reckoner_er-0.3.0/examples/quickstart.py +34 -0
  15. reckoner_er-0.3.0/examples/vendors.csv +10 -0
  16. reckoner_er-0.3.0/pyproject.toml +63 -0
  17. reckoner_er-0.3.0/reckoner/__init__.py +77 -0
  18. reckoner_er-0.3.0/reckoner/__main__.py +7 -0
  19. reckoner_er-0.3.0/reckoner/api.py +221 -0
  20. reckoner_er-0.3.0/reckoner/cli.py +474 -0
  21. reckoner_er-0.3.0/reckoner/evaluate.py +89 -0
  22. reckoner_er-0.3.0/reckoner/py.typed +0 -0
  23. reckoner_er-0.3.0/reckoner/resolver.py +341 -0
  24. reckoner_er-0.3.0/reckoner_er.egg-info/PKG-INFO +100 -0
  25. reckoner_er-0.3.0/reckoner_er.egg-info/SOURCES.txt +30 -0
  26. reckoner_er-0.3.0/reckoner_er.egg-info/dependency_links.txt +1 -0
  27. reckoner_er-0.3.0/reckoner_er.egg-info/entry_points.txt +2 -0
  28. reckoner_er-0.3.0/reckoner_er.egg-info/top_level.txt +1 -0
  29. reckoner_er-0.3.0/setup.cfg +4 -0
  30. reckoner_er-0.3.0/tests/test_api.py +299 -0
  31. reckoner_er-0.3.0/tests/test_cli.py +127 -0
  32. reckoner_er-0.3.0/tests/test_resolver.py +89 -0
@@ -0,0 +1,37 @@
1
+ # Changelog
2
+
3
+ ## [0.3.0] - 2026-11-14
4
+
5
+ - On PyPI as `reckoner-er` (`pip install reckoner-er`). The name `reckoner` there belongs to an unrelated Helm tool; the import name and the command stay `reckoner`.
6
+ - New two-function API: `resolve(names)` returns typed `Cluster` objects (members, names, canonical name, identifiers, confidence, and an `explanations` sentence for every merge and refusal); `match(a, b)` returns a `MatchResult` with `matched`, `score`, `method` and `reasons`. Both take plain strings or records with identifiers. `Cluster` and `MatchResult` are frozen and hashable. See `docs/python-api.md`.
7
+ - New `reckoner match "A" "B"` command, with `--a-cik`/`--b-cik` (and UEI, EIN, CAGE, ticker, domain), `--agency`, `--color` and `--json`. Exits 0 on a match and 1 otherwise. `match` is recognised as the first argument that is not an option, so `reckoner --color never match A B` works too. Because of this, a file literally named `match` is now read with `reckoner ./match` or `reckoner -- match`.
8
+ - `pairwise_scores(clusters, labels)` scores a resolution against labelled data (pairwise precision, recall, F1).
9
+ - Benchmark on real GLEIF names (1312 entities, 2922 names, CC0 snapshot committed): precision 0.966 and recall 0.969 on formatting variants; renames and trade names are not linked, by design. `python benchmarks/run_benchmark.py`; details in `docs/benchmark.md`. CI fails if precision or recall drops below 0.95.
10
+ - Fix: dotted legal suffixes at the end of a name are now stripped like the plain ones. "Oracle Partners, L.P." and "ORACLE PARTNERS LP" both normalize to `oracle partners` (they did not match before), as do "L.L.C.", "P.A." and similar. Each dotted abbreviation is read on its own from the right, so "Acme U.S. L.P." matches "Acme U.S. LP" and "Foo L.P. L.L.C." matches "Foo LP LLC". Only letters that spell a legal suffix are joined, so initials such as "A.G. Edwards" and "U.S." normalize exactly as before. On the benchmark this raised formatting recall from 0.910 to 0.969 with precision unchanged.
11
+ - Type hints throughout; ships `py.typed` and passes `mypy --strict`.
12
+ - `publish.yml` builds, checks and smoke-tests the sdist and wheel, then uploads them to PyPI with trusted publishing when a `v*` tag is pushed.
13
+ - The PyPI page comes from `docs/pypi.md` (absolute image URLs, `pip install reckoner-er`). The sdist includes `benchmarks/`, `docs/` and `examples/`, so the full test suite runs from an unpacked sdist; the benchmark test skips itself when `benchmarks/` is absent.
14
+
15
+ ## [0.2.2] - 2026-09-25
16
+
17
+ - Fix: `link_confidence` no longer claims an identifier's confidence when only one record in the cluster carries that identifier. Such clusters were joined by name, so they now report 0.60 (for example GE with a CIK on one record only: was 0.99, now 0.60).
18
+ - The summary now shows every record in each group and why it joined (`= CIK 907471`, `= name 'general electric'`), aligned, with a count of merges and refusals.
19
+ - Color in the terminal (off when piped, with `NO_COLOR`, or `--color never`).
20
+ - Running `reckoner` with no file prints what to try next; a file with none of the recognised columns stops with a message naming the columns found and what to rename; a missing file suggests `--demo`.
21
+ - `--help` ends with copy-paste examples.
22
+ - `install.sh` and `install.ps1` one-line installers; `examples/vendors.csv`.
23
+
24
+ ## [0.2.1] - 2026-09-25
25
+
26
+ - The CLI now stops with a clear message when a CSV row has more fields than the header (usually an unquoted comma in a company name) instead of writing a garbled extra column.
27
+
28
+ ## [0.2.0] - 2026-09-25
29
+
30
+ - New `reckoner` command line tool: resolve a CSV, JSON or JSON Lines file (or stdin) and get a readable summary, the full JSON result with every receipt, or a CSV of your rows with `entity_id`, `canonical_name` and `link_confidence` appended. `--agency` for agency names, `--demo` for a built-in example.
31
+ - Prebuilt single-file executables for Windows, macOS (Apple Silicon and Intel) and Linux on every GitHub Release. No Python needed.
32
+ - `python -m reckoner` works too.
33
+ - CI runs the tests and the CLI on Linux and Windows.
34
+
35
+ ## [0.1.0]
36
+
37
+ - Initial release: `EntityResolver`, identifier-first merges, exact-normalized name links, refusal receipts, agency mode.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Matthew Busel
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,8 @@
1
+ # The sdist carries what the test suite needs: tests/test_api.py runs the
2
+ # benchmark on the committed GLEIF snapshot.
3
+ recursive-include tests *.py
4
+ recursive-include benchmarks *.py *.csv *.json
5
+ recursive-include docs *.md
6
+ recursive-include examples *.py *.csv
7
+ include CHANGELOG.md
8
+ global-exclude __pycache__ *.py[cod]
@@ -0,0 +1,100 @@
1
+ Metadata-Version: 2.4
2
+ Name: reckoner-er
3
+ Version: 0.3.0
4
+ Summary: Deterministic, explainable entity resolution for organization names. No fuzzy matching. Every merge comes with a receipt.
5
+ Author-email: Matthew Busel <mattbusel@gmail.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/Mattbusel/reckoner
8
+ Project-URL: Issues, https://github.com/Mattbusel/reckoner/issues
9
+ Project-URL: Changelog, https://github.com/Mattbusel/reckoner/blob/main/CHANGELOG.md
10
+ Project-URL: Documentation, https://github.com/Mattbusel/reckoner/blob/main/docs/python-api.md
11
+ Keywords: entity-resolution,record-linkage,deduplication,company-names,data-cleaning,data-engineering,master-data-management,name-matching,lei
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Environment :: Console
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3 :: Only
18
+ Classifier: Programming Language :: Python :: 3.9
19
+ Classifier: Programming Language :: Python :: 3.10
20
+ Classifier: Programming Language :: Python :: 3.11
21
+ Classifier: Programming Language :: Python :: 3.12
22
+ Classifier: Programming Language :: Python :: 3.13
23
+ Classifier: Typing :: Typed
24
+ Classifier: Topic :: Scientific/Engineering :: Information Analysis
25
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
26
+ Requires-Python: >=3.9
27
+ Description-Content-Type: text/markdown
28
+ License-File: LICENSE
29
+ Dynamic: license-file
30
+
31
+ <p align="center"><img src="https://raw.githubusercontent.com/Mattbusel/reckoner/v0.3.0/assets/banner.png" alt="reckoner: input records on the left, joined to the companies they name on the right, each line labeled with the reason; one pair stamped REFUSED" width="100%"></p>
32
+
33
+ # reckoner
34
+
35
+ **Give it a list of company names from different sources, and it tells you which ones are the same company, and why.** It only merges on proof (a shared ID, or the exact same name once spelling noise is removed), never on a guess, and it writes down a receipt for every merge and every refusal.
36
+
37
+ ```
38
+ pip install reckoner-er
39
+ ```
40
+
41
+ The package is called `reckoner-er` on PyPI because `reckoner` there is an unrelated Helm tool, so `pip install reckoner` installs the wrong thing. The import name and the command are both `reckoner`. Do not install both packages in one environment. Python 3.9 or newer, no dependencies, ships `py.typed`.
42
+
43
+ <p align="center"><img src="https://raw.githubusercontent.com/Mattbusel/reckoner/v0.3.0/assets/demo.gif" alt="Terminal recording: reckoner --demo, then reckoner vendors.csv grouping 9 rows into 5 companies, then writing resolved.csv with an entity_id column" width="900"></p>
44
+
45
+ ## From Python
46
+
47
+ ```python
48
+ from reckoner import resolve, match
49
+
50
+ clusters = resolve([
51
+ "GENERAL ELECTRIC CO",
52
+ "General Electric Company",
53
+ {"name": "GE", "cik": "0000040545"},
54
+ {"name": "General Electric", "cik": "40545"},
55
+ "Boeing",
56
+ ])
57
+ for c in clusters:
58
+ print(c.canonical_name, c.members, c.link_confidence)
59
+ # General Electric Company (0, 1, 2, 3) 0.99
60
+ # Boeing (4,) 1.0
61
+
62
+ m = match("THE SHERWIN-WILLIAMS COMPANY", "Sherwin Williams Co")
63
+ m.matched, m.score, m.method # (True, 0.6, 'name_exact_normalized')
64
+
65
+ match({"name": "Meta Platforms, Inc.", "cik": "1326801"},
66
+ {"name": "META PLATFORMS INC", "cik": "907471"}).reasons
67
+ # ('identical normalized names but conflicting CIK - refusing to merge',)
68
+ ```
69
+
70
+ Every `Cluster` carries its members, identifiers, confidence and an `explanations` sentence for every merge and refusal. Full reference: [docs/python-api.md](https://github.com/Mattbusel/reckoner/blob/v0.3.0/docs/python-api.md).
71
+
72
+ ## From the command line
73
+
74
+ ```bash
75
+ reckoner --demo # see it work on a built-in example
76
+ reckoner vendors.csv # CSV, JSON or JSON Lines with a name column
77
+ reckoner vendors.csv -f csv -o resolved.csv # your rows + entity_id, canonical_name, link_confidence
78
+ reckoner match "GENERAL ELECTRIC CO" "General Electric Company" # exit 0 on a match, 1 otherwise
79
+ ```
80
+
81
+ Recognised columns (case-insensitive): `name`, `agency`, `cik`, `uei`, `ein`, `cage`, `ticker`, `domain`, `source`, `state`. `--agency` handles government agency names (`DoD`, `EPA`, `U.S.` prefixes). `reckoner --help` lists every option.
82
+
83
+ ## How it decides
84
+
85
+ <p align="center"><img src="https://raw.githubusercontent.com/Mattbusel/reckoner/v0.3.0/assets/how-it-decides.svg" alt="How reckoner decides: pass 1 merges on a shared ID; pass 2 links exact normalized names at 0.60 unless the records carry conflicting IDs, in which case it refuses with a reason" width="880"></p>
86
+
87
+ - **Identifiers merge first.** A shared CIK, UEI, EIN, CAGE, ticker or domain merges two records outright. Registry IDs (CIK, UEI, EIN, CAGE) link at 0.97 to 0.99; reassignable ones (ticker 0.90, domain 0.85) lower.
88
+ - **Names link only on an exact match after conservative cleanup.** Case, punctuation, `&`, a leading `The`, and trailing legal suffixes (`Inc`, `Corp`, `LLC`, `L.P.`, `GmbH`, `plc`, ...) are removed. Distinguishing words are never dropped. Such links score 0.60.
89
+ - **No fuzzy scoring anywhere.** A typo does not match. There is no similarity knob to tune.
90
+ - **Conflicting registry IDs block a merge.** Same normalized name but different CIKs: refused, with the reason recorded.
91
+
92
+ On a committed snapshot of real GLEIF legal names (1312 entities, 2922 names), formatting variants resolve at precision 0.966 and recall 0.969. Renames and trade names are not linked, by design. Method and numbers: [docs/benchmark.md](https://github.com/Mattbusel/reckoner/blob/v0.3.0/docs/benchmark.md).
93
+
94
+ ## Links
95
+
96
+ - Source, install scripts and prebuilt binaries (no Python needed): https://github.com/Mattbusel/reckoner
97
+ - Changelog: https://github.com/Mattbusel/reckoner/blob/main/CHANGELOG.md
98
+ - Issues: https://github.com/Mattbusel/reckoner/issues
99
+
100
+ MIT licensed.
@@ -0,0 +1,268 @@
1
+ <p align="center"><img src="assets/banner.png" alt="reckoner: input records on the left, joined to the companies they name on the right, each line labeled with the reason; one pair stamped REFUSED" width="100%"></p>
2
+
3
+ # reckoner
4
+
5
+ **Give it a spreadsheet of company names from different sources, and it tells you which rows are the same company, and why.** It only merges on proof (a shared ID, or the exact same name once spelling noise is removed), never on a guess, and it writes down a receipt for every merge and every refusal.
6
+
7
+ <p align="center"><img src="assets/demo.gif" alt="Terminal recording: reckoner --demo, then reckoner vendors.csv grouping 9 rows into 5 companies, then writing resolved.csv with an entity_id column" width="900"></p>
8
+
9
+ ## Install
10
+
11
+ | Your system | One line |
12
+ | --- | --- |
13
+ | Windows (PowerShell) | `irm https://raw.githubusercontent.com/Mattbusel/reckoner/main/install.ps1 \| iex` |
14
+ | macOS or Linux | `curl -fsSL https://raw.githubusercontent.com/Mattbusel/reckoner/main/install.sh \| sh` |
15
+ | Homebrew (macOS, Linux) | `brew install mattbusel/tap/reckoner` |
16
+ | Scoop (Windows) | `scoop bucket add mattbusel https://github.com/Mattbusel/scoop-bucket; scoop install mattbusel/reckoner` |
17
+ | Python (pipx) | `pipx install git+https://github.com/Mattbusel/reckoner` |
18
+ | Just the file | [Download for Windows, macOS or Linux](https://github.com/Mattbusel/reckoner/releases/latest), unzip, run |
19
+
20
+ No Python is needed for anything except the pipx line. The install scripts check the download against the release's `SHA256SUMS.txt` before installing: `install.sh` puts `reckoner` in `~/.local/bin`, `install.ps1` puts `reckoner.exe` in `%LOCALAPPDATA%\Programs\reckoner` and adds it to your PATH.
21
+
22
+ <details>
23
+ <summary>Unsigned binary warnings, and why not <code>pip install reckoner</code></summary>
24
+
25
+ The binaries are unsigned. Windows SmartScreen may say "unknown publisher": click **More info**, then **Run anyway**. On macOS, if a downloaded binary is blocked, run `xattr -d com.apple.quarantine reckoner` (the install script does this for you).
26
+
27
+ The name `reckoner` on PyPI belongs to an unrelated Helm tool, so `pip install reckoner` installs the wrong thing. Use the pipx line above, or `pip install git+https://github.com/Mattbusel/reckoner` for the Python library.
28
+ </details>
29
+
30
+ ## Use it in 3 steps
31
+
32
+ **1. See it work on built-in data.**
33
+
34
+ ```bash
35
+ reckoner --demo
36
+ ```
37
+
38
+ You get 7 records grouped into 4 companies, and one refused merge: two records that are both called "Meta Platforms" but carry different SEC CIK numbers, so reckoner will not join them.
39
+
40
+ **2. Point it at your own file.** A CSV, JSON or JSON Lines file with a `name` column (and, if you have them, ID columns like `cik`, `ein`, `uei`, `ticker`, `domain`):
41
+
42
+ ```bash
43
+ reckoner vendors.csv
44
+ ```
45
+
46
+ Every group is listed with the rows in it and the reason each row joined.
47
+
48
+ **3. Save the answer as a spreadsheet.**
49
+
50
+ ```bash
51
+ reckoner vendors.csv -f csv -o resolved.csv
52
+ ```
53
+
54
+ `resolved.csv` is your file, same rows, same order, with three columns added: `entity_id` (rows with the same ID are the same company), `canonical_name` and `link_confidence`.
55
+
56
+ ## Results
57
+
58
+ This is the real output for [`examples/vendors.csv`](examples/vendors.csv), nine rows from three made-up feeds:
59
+
60
+ ```
61
+ $ reckoner examples/vendors.csv
62
+ 9 records -> 5 entities 4 merges, 0 refused
63
+
64
+ ent-0 General Electric Company 0.60 TICKER GE
65
+ #0 GENERAL ELECTRIC CO sec
66
+ #1 General Electric Company usaspending = name 'general electric'
67
+ ent-2 The Boeing Company 0.60 TICKER BA
68
+ #2 The Boeing Company sec
69
+ #3 BOEING CO usaspending = name 'boeing'
70
+ ent-4 Lockheed Martin Corporation 0.90 TICKER LMT
71
+ #4 Lockheed Martin Corporation sec
72
+ #5 LOCKHEED MARTIN CORP usaspending = name 'lockheed martin'
73
+ #6 Lockheed Martin Corp. sam = TICKER LMT
74
+ ent-7 Microsoft Corporation 1.00 TICKER MSFT
75
+ ent-8 Micrsoft Corp 1.00
76
+
77
+ #n is the input record. '= CIK 123', '= TICKER X': joined on a shared ID.
78
+ '= name ...' joined on the exact normalized name (0.60, never fuzzy).
79
+ ```
80
+
81
+ Read it like this:
82
+
83
+ - **Lockheed Martin**: row 6 joined because it has the same ticker as row 4 (0.90). Row 5 has no ticker, but its name is the same once case and "CORP" are removed, so it links at 0.60.
84
+ - **Boeing**: "The Boeing Company" and "BOEING CO" both become `boeing`.
85
+ - **Micrsoft Corp stays on its own.** It is a typo, and reckoner does not guess. A wrong merge is worse than a missed one.
86
+
87
+ And when two rows look like the same name but carry different registry IDs, the merge is refused and the reason is kept (from `reckoner --demo`):
88
+
89
+ ```
90
+ Refused merges (1):
91
+ #4 + #6 'Meta Platforms, Inc.' vs 'META PLATFORMS INC'
92
+ identical normalized names but conflicting CIK - refusing to merge
93
+ ```
94
+
95
+ <p align="center"><img src="assets/how-it-decides.svg" alt="How reckoner decides: pass 1 merges on a shared ID; pass 2 links exact normalized names at 0.60 unless the records carry conflicting IDs, in which case it refuses with a reason" width="880"></p>
96
+
97
+ ## Why it works this way
98
+
99
+ The same company is spelled differently in every system it appears in:
100
+
101
+ | One source says | Another says |
102
+ | --- | --- |
103
+ | `GENERAL ELECTRIC CO` | `GENERAL ELECTRIC COMPANY` |
104
+ | `BECTON DICKINSON & CO` | `BECTON, DICKINSON AND COMPANY` |
105
+ | `THE SHERWIN-WILLIAMS COMPANY` | `Sherwin Williams Co` |
106
+ | `Medtronic plc` | `MEDTRONIC INC` |
107
+
108
+ Join two datasets on the raw name and you silently drop many true matches. Turn on a fuzzy matcher instead and you start quietly merging companies that are not the same, with no way to explain any single decision. reckoner takes the opposite stance:
109
+
110
+ - **Identifiers merge first.** A shared CIK, UEI, EIN, CAGE, ticker or domain is the only thing that merges two records outright. Registry IDs (CIK, UEI, EIN, CAGE) link at 0.97 to 0.99; reassignable ones (ticker 0.90, domain 0.85) lower.
111
+ - **Names link only on an exact match after conservative cleanup.** Case, punctuation, `&`, a leading `The`, and trailing legal suffixes (`Inc`, `Corp`, `Company`, `LLC`, `GmbH`, `plc`, ...) are removed. Distinguishing words are never dropped. Such links score 0.60.
112
+ - **No fuzzy scoring anywhere.** A typo does not match. There is no similarity knob to tune.
113
+ - **Conflicting registry IDs block a merge.** Same normalized name but different CIKs: refused, with the reason recorded.
114
+ - **Everything is a receipt.** Every link and refusal is a `MatchRecord` with the original values, the normalized value, the method, the confidence and the evidence.
115
+
116
+ It does not do fuzzy, edit-distance or embedding matching, machine learning, or network calls. If you need probabilistic linkage across millions of noisy consumer records, use a heavier library. reckoner is for when a wrong merge is worse than a missed one and every decision has to be explainable to an analyst, an auditor or a regulator.
117
+
118
+ Pure Python standard library, zero dependencies, one file of logic ([`reckoner/resolver.py`](reckoner/resolver.py)) you can read in ten minutes.
119
+
120
+ <details>
121
+ <summary><b>Command line reference</b></summary>
122
+
123
+ ```
124
+ reckoner [-h] [-f {summary,json,csv}] [-o OUTPUT] [--agency] [--demo]
125
+ [--color {auto,always,never}] [--version] [input]
126
+ ```
127
+
128
+ | Option | What it does |
129
+ | --- | --- |
130
+ | `input` | CSV, JSON array or JSON Lines file, or `-` for stdin |
131
+ | `-f summary` | readable report (default) |
132
+ | `-f json` | the full result with every match receipt |
133
+ | `-f csv` | each input row plus `entity_id`, `canonical_name`, `link_confidence` |
134
+ | `-o FILE` | write to a file instead of the terminal |
135
+ | `--agency` | government agency names: strips `U.S.` prefixes and expands `DoD`, `EPA`, ... |
136
+ | `--color` | `auto` (default: color in a terminal, off when piped or when `NO_COLOR` is set), `always`, `never` |
137
+
138
+ Recognised columns (case-insensitive): `name`, `agency`, `cik`, `uei`, `ein`, `cage`, `ticker`, `domain`, `source`, `state`. Other columns are kept in CSV output. If none of your columns are recognised, reckoner stops and tells you which columns it found and what to rename.
139
+
140
+ ```bash
141
+ reckoner records.jsonl --format json # full result with every receipt
142
+ reckoner agencies.csv --agency # DoD, EPA, U.S. prefixes
143
+ cat companies.csv | reckoner - # read stdin
144
+ ```
145
+ </details>
146
+
147
+ <details>
148
+ <summary><b>Use it from Python</b></summary>
149
+
150
+ ```python
151
+ from reckoner import EntityResolver
152
+
153
+ records = [
154
+ {"name": "GENERAL ELECTRIC CO", "cik": "0000040545", "source": "sec"},
155
+ {"name": "General Electric Company", "source": "usaspending"},
156
+ {"name": "THE SHERWIN-WILLIAMS COMPANY", "source": "usaspending"},
157
+ {"name": "Sherwin Williams Co", "source": "sec"},
158
+ {"name": "Meta Platforms, Inc.", "cik": "1326801", "source": "sec"},
159
+ {"name": "Meta Financial Group, Inc.", "cik": "907471", "source": "sec"},
160
+ ]
161
+
162
+ result = EntityResolver().resolve(records)
163
+
164
+ for e in result["entities"]:
165
+ print(e["canonical_name"], "<-", e["aliases"], f"(conf {e['link_confidence']})")
166
+ ```
167
+
168
+ ```
169
+ General Electric Company <- ['GENERAL ELECTRIC CO', 'General Electric Company'] (conf 0.6)
170
+ THE SHERWIN-WILLIAMS COMPANY <- ['THE SHERWIN-WILLIAMS COMPANY', 'Sherwin Williams Co'] (conf 0.6)
171
+ Meta Platforms, Inc. <- ['Meta Platforms, Inc.'] (conf 1.0)
172
+ Meta Financial Group, Inc. <- ['Meta Financial Group, Inc.'] (conf 1.0)
173
+ ```
174
+
175
+ Both GE and Sherwin-Williams link on the normalized name (only one GE record carries the CIK, so the CIK cannot prove the link). The two Metas normalize to different names and carry different CIKs, so they never link.
176
+
177
+ When two records normalize to the same name but carry conflicting identifiers, the merge is refused and the refusal is kept:
178
+
179
+ ```python
180
+ result = EntityResolver().resolve([
181
+ {"name": "Meta Platforms, Inc.", "cik": "1326801"},
182
+ {"name": "META PLATFORMS INC", "cik": "907471"},
183
+ ])
184
+ result["entities_out"] # 2, with a refusal receipt in result["matches"]
185
+ ```
186
+
187
+ Agency names: `EntityResolver(agency_mode=True)` adds US-prefix stripping and a small alias table (`DoD` to `department of defense`, `EPA` to `environmental protection agency`, ...):
188
+
189
+ ```python
190
+ EntityResolver(agency_mode=True).resolve([
191
+ {"agency": "DoD"},
192
+ {"agency": "U.S. Department of Defense"},
193
+ ]) # -> 1 entity
194
+ ```
195
+
196
+ Run the full example with `PYTHONPATH=. python examples/quickstart.py` from the repo root.
197
+ </details>
198
+
199
+ <details>
200
+ <summary><b>The output shape</b></summary>
201
+
202
+ `resolve(records)` returns a dict:
203
+
204
+ ```python
205
+ {
206
+ "entities": [ ... ], # one canonical entity per cluster
207
+ "clusters": [[0, 1], [2, 3], ...], # input indices grouped
208
+ "matches": [ ... ], # every MatchRecord, merged and refused
209
+ "records_in": 6,
210
+ "entities_out": 4,
211
+ }
212
+ ```
213
+
214
+ Each entity:
215
+
216
+ ```python
217
+ {
218
+ "entity_id": "ent-0",
219
+ "canonical_name": "General Electric Company",
220
+ "normalized_name": "general electric",
221
+ "identifiers": {"cik": "40545"},
222
+ "aliases": ["GENERAL ELECTRIC CO", "General Electric Company"],
223
+ "member_indices": [0, 1],
224
+ "sources": ["sec", "usaspending"],
225
+ "link_confidence": 0.6,
226
+ }
227
+ ```
228
+
229
+ `link_confidence` is the strongest evidence that actually merged the cluster: an identifier's confidence if an identifier shared between members merged them, otherwise 0.60 for a name link. A single-record entity is 1.0.
230
+
231
+ Each match or refusal receipt:
232
+
233
+ ```python
234
+ {
235
+ "left_index": 0, "right_index": 1,
236
+ "method": "name_exact_normalized",
237
+ "confidence": 0.0, "merged": False,
238
+ "left_original": "Meta Platforms, Inc.",
239
+ "right_original": "META PLATFORMS INC",
240
+ "normalized": "meta platforms",
241
+ "evidence": [],
242
+ "refusal_reason": "identical normalized names but conflicting CIK - refusing to merge",
243
+ }
244
+ ```
245
+ </details>
246
+
247
+ <details>
248
+ <summary><b>Extending it, and running the tests</b></summary>
249
+
250
+ The normalization is deliberately small and readable. Add legal suffixes to `LEGAL_SUFFIXES`, agency aliases to `AGENCY_ALIASES`, or new identifier types to `IDENTIFIERS` with a normalizer and a confidence. Everything is exact-match on the normalized value; there is nothing fuzzy to tune.
251
+
252
+ ```bash
253
+ git clone https://github.com/Mattbusel/reckoner
254
+ cd reckoner
255
+ python -m unittest discover -s tests
256
+ python -m reckoner --demo
257
+ ```
258
+
259
+ Or just copy `reckoner/resolver.py` into your project. It is standard library only.
260
+ </details>
261
+
262
+ ## License
263
+
264
+ MIT, see [LICENSE](LICENSE).
265
+
266
+ ---
267
+
268
+ Built by the team behind [Tensorust](https://tensorust-site.vercel.app/), where reconciling public records across systems that each spell reality slightly differently is the whole job.