bibauthors 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. bibauthors-0.1.0/LICENSE +21 -0
  2. bibauthors-0.1.0/PKG-INFO +542 -0
  3. bibauthors-0.1.0/README.md +510 -0
  4. bibauthors-0.1.0/pyproject.toml +114 -0
  5. bibauthors-0.1.0/setup.cfg +4 -0
  6. bibauthors-0.1.0/src/bibauthors/__init__.py +108 -0
  7. bibauthors-0.1.0/src/bibauthors/cache.py +199 -0
  8. bibauthors-0.1.0/src/bibauthors/cli.py +297 -0
  9. bibauthors-0.1.0/src/bibauthors/config.py +191 -0
  10. bibauthors-0.1.0/src/bibauthors/entry.py +142 -0
  11. bibauthors-0.1.0/src/bibauthors/http.py +380 -0
  12. bibauthors-0.1.0/src/bibauthors/latex.py +399 -0
  13. bibauthors-0.1.0/src/bibauthors/names.py +1424 -0
  14. bibauthors-0.1.0/src/bibauthors/parsers/__init__.py +148 -0
  15. bibauthors-0.1.0/src/bibauthors/parsers/bibtex.py +593 -0
  16. bibauthors-0.1.0/src/bibauthors/parsers/csljson.py +220 -0
  17. bibauthors-0.1.0/src/bibauthors/providers/__init__.py +239 -0
  18. bibauthors-0.1.0/src/bibauthors/providers/arxiv.py +209 -0
  19. bibauthors-0.1.0/src/bibauthors/providers/crossref.py +254 -0
  20. bibauthors-0.1.0/src/bibauthors/providers/openalex.py +212 -0
  21. bibauthors-0.1.0/src/bibauthors/providers/semanticscholar.py +160 -0
  22. bibauthors-0.1.0/src/bibauthors/report.py +371 -0
  23. bibauthors-0.1.0/src/bibauthors/verdict.py +153 -0
  24. bibauthors-0.1.0/src/bibauthors/verify.py +397 -0
  25. bibauthors-0.1.0/src/bibauthors.egg-info/PKG-INFO +542 -0
  26. bibauthors-0.1.0/src/bibauthors.egg-info/SOURCES.txt +33 -0
  27. bibauthors-0.1.0/src/bibauthors.egg-info/dependency_links.txt +1 -0
  28. bibauthors-0.1.0/src/bibauthors.egg-info/entry_points.txt +2 -0
  29. bibauthors-0.1.0/src/bibauthors.egg-info/requires.txt +6 -0
  30. bibauthors-0.1.0/src/bibauthors.egg-info/top_level.txt +1 -0
  31. bibauthors-0.1.0/tests/test_accuracy.py +645 -0
  32. bibauthors-0.1.0/tests/test_cli.py +478 -0
  33. bibauthors-0.1.0/tests/test_names.py +752 -0
  34. bibauthors-0.1.0/tests/test_parsers.py +837 -0
  35. bibauthors-0.1.0/tests/test_providers.py +615 -0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Manu Nicholas Jacob
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,542 @@
1
+ Metadata-Version: 2.4
2
+ Name: bibauthors
3
+ Version: 0.1.0
4
+ Summary: Verify bibliography AUTHOR LISTS against authoritative metadata, so a chimeric citation cannot pass as a clean one.
5
+ Author: Manu Nicholas Jacob
6
+ Maintainer: Manu Nicholas Jacob
7
+ License: MIT
8
+ Project-URL: Homepage, https://github.com/manunicholasjacob/bibauthors
9
+ Project-URL: Source, https://github.com/manunicholasjacob/bibauthors
10
+ Project-URL: Issues, https://github.com/manunicholasjacob/bibauthors/issues
11
+ Project-URL: Changelog, https://github.com/manunicholasjacob/bibauthors/blob/main/CHANGELOG.md
12
+ Project-URL: ORCID, https://orcid.org/0009-0007-6589-6572
13
+ Keywords: bibliography,bibtex,citations,author verification,hallucination,crossref,openalex,arxiv,reproducibility,latex
14
+ Classifier: Development Status :: 4 - Beta
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: License :: OSI Approved :: MIT License
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.9
19
+ Classifier: Programming Language :: Python :: 3.10
20
+ Classifier: Programming Language :: Python :: 3.11
21
+ Classifier: Programming Language :: Python :: 3.12
22
+ Classifier: Programming Language :: Python :: 3.13
23
+ Classifier: Topic :: Text Processing :: Markup :: LaTeX
24
+ Classifier: Topic :: Scientific/Engineering
25
+ Requires-Python: >=3.9
26
+ Description-Content-Type: text/markdown
27
+ License-File: LICENSE
28
+ Requires-Dist: tomli>=1.2; python_version < "3.11"
29
+ Provides-Extra: dev
30
+ Requires-Dist: pytest; extra == "dev"
31
+ Dynamic: license-file
32
+
33
+ # bibauthors
34
+
35
+ Verify the **author list** of every entry in a bibliography against
36
+ authoritative metadata, and fail when the authors do not belong to the work.
37
+
38
+ Deliberately narrow. Every citation checker matches on title and DOI, and
39
+ most of them do that well. This one checks the author list, which is the part
40
+ they all do worst.
41
+
42
+ > A `DISJOINT` verdict is a prompt to check an entry by hand. It is **not** an
43
+ > accusation of fabrication. See [Limitations](#limitations).
44
+
45
+ - [The chimeric citation](#the-chimeric-citation)
46
+ - [Install](#install)
47
+ - [Usage](#usage)
48
+ - [Verdicts](#verdicts)
49
+ - [Thresholds](#thresholds)
50
+ - [Measured accuracy](#measured-accuracy)
51
+ - [Sources](#sources)
52
+ - [Related work](#related-work)
53
+ - [Limitations](#limitations)
54
+
55
+ ## The chimeric citation
56
+
57
+ The most dangerous hallucinated citation is not the one that invents a paper.
58
+ An invented paper is easy to catch: no DOI resolves, no title matches, and
59
+ every existing tool flags it.
60
+
61
+ The dangerous one is the **chimeric citation**: a real title, a real venue, a
62
+ real year, and an **invented author list**. It passes a title-matching checker
63
+ with a clean report. It looks real on Google Scholar. A reader who recognises
64
+ the title has no reason to look twice, and a reader who recognises one of the
65
+ names has every reason to trust it.
66
+
67
+ This is not hypothetical. On 29 October 2026 one turned up in this author's
68
+ own bibliography. The entry for *Evaluating LLM Agent Collusion in Double
69
+ Auctions* (arXiv 2507.01413) carried the authors:
70
+
71
+ Agrawal, Liu, Fried, Sycara, Jacobs, Oesterheld, Conitzer
72
+
73
+ The real author list is:
74
+
75
+ Kushal Agrawal, Verona Teo, Juan J. Vazquez,
76
+ Sudarsh Kunnavakkam, Vishak Srikanth, Andy Liu
77
+
78
+ Two of the seven surnames are real authors of the paper. Five are not. The
79
+ title, the arXiv id, the venue and the year were all correct. The checker in
80
+ use at the time reported the entry as "suspicious, found in 1 source", which
81
+ is the same label it gives every legitimate arXiv preprint, so the label
82
+ carried no signal about the actual error.
83
+
84
+ `bibauthors` reports that entry like this:
85
+
86
+ ```
87
+ FAIL collusion2025:9 PARTIAL_OVERLAP via arxiv
88
+ title: Evaluating LLM Agent Collusion in Double Auctions
89
+ record: http://arxiv.org/abs/2507.01413v1
90
+ cited (7):
91
+ [0] Agrawal, Kushal -> [0] Kushal Agrawal
92
+ [1] Liu, Andy -> [5] Andy Liu
93
+ [2] Fried, Daniel <-- NOT ON THIS WORK
94
+ [3] Sycara, Katia <-- NOT ON THIS WORK
95
+ [4] Jacobs, Abigail <-- NOT ON THIS WORK
96
+ [5] Oesterheld, Caspar <-- NOT ON THIS WORK
97
+ [6] Conitzer, Vincent <-- NOT ON THIS WORK
98
+ record authors not cited (4 of 6):
99
+ [1] Verona Teo
100
+ [2] Juan J. Vazquez
101
+ [3] Sudarsh Kunnavakkam
102
+ [4] Vishak Srikanth
103
+ coverage: 29% of cited authors matched
104
+ ```
105
+
106
+ That entry is committed as `tests/fixtures/bib/chimeric.bib`, and the test
107
+ suite asserts the verdict on every run.
108
+
109
+ ## Install
110
+
111
+ ```sh
112
+ pip install bibauthors
113
+ ```
114
+
115
+ `bibauthors` has **no required runtime dependencies**. It is standard library
116
+ only, so it drops into a minimal CI image. (`tomli` is pulled in only on
117
+ Python 3.9 and 3.10, and only if you use a TOML config file.) Python 3.9+.
118
+
119
+ ### As a GitHub Action
120
+
121
+ This is the form most paper repositories want. Add one step:
122
+
123
+ ```yaml
124
+ name: bibliography
125
+ on: [push, pull_request]
126
+ permissions:
127
+ contents: read
128
+ jobs:
129
+ authors:
130
+ runs-on: ubuntu-latest
131
+ steps:
132
+ - uses: actions/checkout@v4
133
+ - uses: manunicholasjacob/bibauthors@v0.1.0
134
+ with:
135
+ paths: refs.bib
136
+ ```
137
+
138
+ Findings land as annotations on the pull request diff, at the line of the
139
+ offending entry. Inputs: `paths` (default `**/*.bib`), `offline`, `cache-dir`,
140
+ `providers`, `fail-on` (`mismatch`, `unverifiable` or `never`), `strict`,
141
+ `json-report`, `annotations`, `python-version`, `version`. Outputs:
142
+ `exit-code`, `mismatches`, `unverifiable`, `report-path`.
143
+
144
+ To keep CI hermetic and off the metadata APIs entirely, commit your cache and
145
+ run `offline: true` with `cache-dir` pointed at it.
146
+
147
+ ### As a pre-commit hook
148
+
149
+ ```yaml
150
+ repos:
151
+ - repo: https://github.com/manunicholasjacob/bibauthors
152
+ rev: v0.1.0
153
+ hooks:
154
+ - id: bibauthors
155
+ ```
156
+
157
+ There is also `bibauthors-offline` for cache-only, network-free commits. Note
158
+ that pre-commit treats any non-zero exit as a failure, so an unverifiable
159
+ entry (exit 2) will block a commit there, unlike the Action's default policy.
160
+
161
+ ## Usage
162
+
163
+ ```sh
164
+ bibauthors refs.bib # human report
165
+ bibauthors refs.bib --json report.json # machine-readable too
166
+ bibauthors refs.bib --offline # cache only, no network
167
+ bibauthors refs.bib --strict # also fail on order and truncation
168
+ bibauthors refs.bib --set name_variant=0.75 # tune a threshold
169
+ bibauthors --list-thresholds # every knob and its default
170
+ cat refs.bib | bibauthors # reads stdin
171
+ ```
172
+
173
+ Accepts BibTeX and CSL-JSON, sniffed by default.
174
+
175
+ ### Exit codes
176
+
177
+ | Code | Meaning |
178
+ | ---- | ------- |
179
+ | 0 | clean: every entry `MATCH` or `MATCH_WITH_VARIANTS` |
180
+ | 1 | author mismatch found |
181
+ | 2 | no mismatch, but at least one entry could not be verified |
182
+ | 3 | usage or input error |
183
+
184
+ Code 1 wins over code 2 when both apply: the mismatch is the finding the tool
185
+ exists to report. Code 2 is separate on purpose. "I could not check this" and
186
+ "this is wrong" are different statements, and a checker that conflates them
187
+ teaches you to ignore it.
188
+
189
+ ### As a library
190
+
191
+ ```python
192
+ from bibauthors import Config, Verdict, verify_bibliography
193
+
194
+ report = verify_bibliography("refs.bib", Config(offline=True))
195
+
196
+ for result in report.results:
197
+ if result.verdict in (Verdict.DISJOINT, Verdict.PARTIAL_OVERLAP):
198
+ print(result.entry.key, result.verdict)
199
+ for index in result.comparison.unmatched_cited:
200
+ print(" not on this work:", result.entry.authors[index])
201
+
202
+ raise SystemExit(report.exit_code())
203
+ ```
204
+
205
+ The name matcher is usable on its own, without any network or bibliography:
206
+
207
+ ```python
208
+ from bibauthors import Thresholds, compare_names, parse_name
209
+
210
+ a = parse_name("van der Berg, J.")
211
+ b = parse_name("Jan van der Berg")
212
+ result = compare_names(a, b, Thresholds())
213
+ print(result.score, result.variants) # 0.9 ['initials']
214
+ ```
215
+
216
+ Stable public API: `verify_bibliography`, `verify_entries`, `verify_entry`,
217
+ `Verifier`, `load_bibliography`, `parse_bibtex`, `parse_csl_json`, `Entry`,
218
+ `parse_name`, `parse_names`, `fold`, `compare_names`, `compare_author_lists`,
219
+ `PersonName`, `NameComparison`, `ListComparison`, `Verdict`, `Thresholds`,
220
+ `Config`, `EntryResult`, `Report`, `YearFinding`, `format_text`,
221
+ `format_json`, `format_github_annotations`, `to_dict`, `Provider`, `Record`,
222
+ `build_providers`, `HttpClient`, `Cache`.
223
+
224
+ The JSON report carries a `schema_version` (currently `1.0`), versioned
225
+ separately from the package. Within a major schema version, fields are added
226
+ but never renamed or removed.
227
+
228
+ ## Verdicts
229
+
230
+ The verdict is graded, not boolean. A bibliography is full of legitimate
231
+ variation, and a tool that answers only yes or no either cries wolf or stays
232
+ silent.
233
+
234
+ | Verdict | Severity | Meaning | Fails? |
235
+ | ------- | -------- | ------- | ------ |
236
+ | `MATCH` | 0 | Cited authors and record authors agree exactly. | no |
237
+ | `MATCH_WITH_VARIANTS` | 0 | The same authors, written differently: initials, diacritics, particles, transliteration. | no |
238
+ | `TRUNCATED` | 1 | The cited list is a truncated prefix of the record, as `et al.` promises. | with `--strict` |
239
+ | `ORDER_DIFFERS` | 2 | The same authors, in a different order. Reported separately from a wrong author, because it usually means a transcription slip, not a fabrication. | with `--strict` |
240
+ | `UNVERIFIABLE` | 2 | No authoritative record was found to compare against. Not a finding about the entry. | exit 2 |
241
+ | `PARTIAL_OVERLAP` | 3 | Some cited authors are on the work and some are not. **This is where the real chimeric case lands**, because a fabricated list usually keeps a real name or two. | yes |
242
+ | `DISJOINT` | 4 | Not one cited author is on this work. | yes |
243
+
244
+ `DISJOINT` is deliberately hard to reach. It requires that nothing matched at
245
+ all, that the record lists at least `disjoint_min_record_authors` authors, that
246
+ the entry was not marked truncated, and that the record was identified by a
247
+ DOI or arXiv id or by a title similarity at or above `title_similarity`. If
248
+ nothing matched but any of those guards fails, the verdict falls back to
249
+ `PARTIAL_OVERLAP` with a note explaining the downgrade. A record found by
250
+ search with a weak title match is never allowed to drive a mismatch verdict
251
+ at all, because judging an author list against the wrong paper is the fastest
252
+ way to manufacture a false accusation.
253
+
254
+ Dates are a separate axis, reported as notes rather than verdicts. An arXiv
255
+ entry whose year matches the latest version but not v1 is surfaced like this:
256
+
257
+ ```
258
+ year: stored year 2025 matches the latest version (2025-01-03),
259
+ but v1 is 2024 (2024-12-19). Decide which version you mean.
260
+ ```
261
+
262
+ That is the second real case from the 29 October pass, committed as
263
+ `tests/fixtures/bib/wrong_year.bib` (the Qwen2.5 Technical Report, arXiv
264
+ 2412.15115: v1 19 December 2024, v2 3 January 2025).
265
+
266
+ ## Thresholds
267
+
268
+ Everything is configurable, from a `[tool.bibauthors]` table in
269
+ `pyproject.toml`, from a `bibauthors.toml`, or from repeatable `--set
270
+ NAME=VALUE` flags. `bibauthors --list-thresholds` prints them all.
271
+
272
+ | Threshold | Default | What it does |
273
+ | --------- | ------- | ------------ |
274
+ | `name_match` | 0.92 | At or above this, two names are the same with no variant worth reporting. |
275
+ | `name_variant` | 0.70 | At or above this, two names are the same person, reached by one or more variant rules. Below it, they are different people. |
276
+ | `partial_coverage` | 0.50 | Fraction of cited authors that must match before coverage is called adequate. |
277
+ | `title_similarity` | 0.80 | How close a record's title must be before the record may drive a mismatch verdict. |
278
+ | `disjoint_min_record_authors` | 2 | A record with fewer known authors can never produce `DISJOINT`. |
279
+ | `disjoint_max_coverage` | 0.0 | Coverage must be at or below this for `DISJOINT`. At the default, one matched author downgrades to `PARTIAL_OVERLAP`. |
280
+ | `surname_only_penalty` | 0.18 | Subtracted when one side has no given name, so a bare surname cannot reach `name_match` alone. |
281
+ | `initial_vs_full_given` | 0.90 | Ceiling for a given-name pair agreeing only on the initial. |
282
+ | `promote_order_differs` | false | Count `ORDER_DIFFERS` as a failure. |
283
+ | `promote_truncated` | false | Count `TRUNCATED` as a failure. |
284
+
285
+ `name_variant` is the consequential one. Lowering it hides real fabrications;
286
+ raising it manufactures them. The [measured accuracy](#measured-accuracy)
287
+ section includes a real sweep of it so the choice can be made on numbers
288
+ rather than taste.
289
+
290
+ The scoring model, for anyone who needs to reason about a number:
291
+ `score = 0.65 * family + 0.35 * given`, less `surname_only_penalty` when
292
+ either side has no given name, then clamped by the ceiling of every variant
293
+ rule that fired. The weights are load-bearing: a pair whose first given names
294
+ disagree can reach at most 0.65, which is below the default `name_variant`, so
295
+ "John Smith" against "Robert Smith" is a non-match on the given name alone no
296
+ matter how perfectly the surnames agree.
297
+
298
+ Variant rules and their ceilings: spacing and diacritics 1.0, dropped middle
299
+ name and dropped suffix 0.95, fuzzy group match 0.95, order swapped and
300
+ particle moved and German `ue` expansion and general spelling 0.90, initials
301
+ `initial_vs_full_given`, Mc/Mac 0.88, compound surname 0.85, conflicting
302
+ suffix and conflicting middle initial 0.85, transliteration 0.80, vowel
303
+ romanization 0.78.
304
+
305
+ ## Measured accuracy
306
+
307
+ **These numbers come from a test set this project built. They are not an
308
+ independent benchmark, and no figure here is borrowed from any other
309
+ project.** Reproduce them with:
310
+
311
+ ```sh
312
+ PYTHONPATH=src python tools/evaluate.py
313
+ ```
314
+
315
+ ### The set
316
+
317
+ | | |
318
+ | --- | --- |
319
+ | Name pairs | **154** labelled pairs: 89 the same human being, 65 different people |
320
+ | Categories | 19: exact, initials, middle_name, middle_initial_conflict, particle, mc_mac, apostrophe, hyphenated, diacritics, transliteration, cjk_order, cjk_script, suffix, comma_order, group, near_miss_surname, different_person, same_surname_different_person, initial_collision |
321
+ | Entries | **27** labelled bibliography entries, covering all 7 verdicts (5 MATCH, 5 MATCH_WITH_VARIANTS, 3 ORDER_DIFFERS, 4 TRUNCATED, 3 PARTIAL_OVERLAP, 5 DISJOINT, 2 UNVERIFIABLE) |
322
+ | Ground truth | Author lists from recorded CrossRef, OpenAlex, arXiv and Semantic Scholar responses, plus public record for the confusable-people pairs |
323
+ | Provenance | `tests/fixtures/eval/README-DATA.txt` |
324
+
325
+ The set is **small and deliberately weighted towards hard cases**. 34 of the
326
+ 65 negative pairs are real confusable people or genuinely distinct near-miss
327
+ surnames; there are no easy negatives. One row moves pair accuracy by 0.65 of
328
+ a percentage point. **These figures are not an estimate of accuracy on an
329
+ ordinary bibliography, where the tool scores considerably higher.** They are a
330
+ measurement of where it breaks.
331
+
332
+ ### Results, at the shipped defaults
333
+
334
+ | Metric | Name pairs (n=154) | Entries (n=27) |
335
+ | ------ | ------------------ | -------------- |
336
+ | Accuracy | 0.8766 (135/154) | 0.9630 (26/27) |
337
+ | Precision | 0.8571 | |
338
+ | Recall | 0.9438 | |
339
+ | F1 | 0.8984 | |
340
+ | **False-accusation rate** | **0.0562** (5 of 89) | **0.0588** (1 of 17) |
341
+ | **Miss rate** | **0.2154** (14 of 65) | **0.0000** (0 of 8) |
342
+
343
+ The last two rows are the ones that matter, and they are reported separately
344
+ and never averaged into one figure, because the costs are not symmetric. The
345
+ false-accusation rate is how often something genuinely correct is called
346
+ wrong. The miss rate is how often something genuinely wrong is not caught.
347
+
348
+ All 8 entries with a genuinely wrong author list were caught, including the
349
+ real chimeric case and four fully fabricated lists. One entry out of 17 with a
350
+ correct author list was wrongly given `PARTIAL_OVERLAP`, and the cause is
351
+ instructive: CrossRef deposits Aidan N. Gomez of *Attention Is All You Need*
352
+ with the family name `N.Gomez`, so a correctly spelled citation scores 0.909
353
+ against the record, just under the spelling floor. The defect is in the
354
+ deposited metadata, the bibliography is innocent, and the tool still reports
355
+ it. That is the shape most false accusations will take.
356
+
357
+ ### Per category
358
+
359
+ | Category | n | Accuracy | | Category | n | Accuracy |
360
+ | -------- | - | -------- |-| -------- | - | -------- |
361
+ | apostrophe | 6 | 1.0000 | | initials | 10 | 1.0000 |
362
+ | cjk_order | 8 | 1.0000 | | mc_mac | 6 | 0.5000 |
363
+ | cjk_script | 6 | 0.8333 | | middle_initial_conflict | 6 | 0.3333 |
364
+ | comma_order | 6 | 1.0000 | | middle_name | 8 | 1.0000 |
365
+ | diacritics | 8 | 1.0000 | | near_miss_surname | 12 | 0.9167 |
366
+ | different_person | 12 | 1.0000 | | particle | 10 | 0.9000 |
367
+ | exact | 6 | 1.0000 | | same_surname_different_person | 10 | 1.0000 |
368
+ | group | 8 | 0.8750 | | suffix | 6 | 0.5000 |
369
+ | hyphenated | 8 | 0.7500 | | transliteration | 8 | 0.6250 |
370
+ | initial_collision | 10 | 1.0000 | | **ALL** | **154** | **0.8766** |
371
+
372
+ The three weak categories are honest and two of them are deliberate.
373
+ `middle_initial_conflict` (0.3333) and `suffix` (0.5000) are low because a
374
+ contradicting middle initial or generational suffix is scored as weak evidence
375
+ that two names are different people, never as proof. The consequence, measured:
376
+ George W. Bush and George H. W. Bush are treated as one person, as are the two
377
+ Michael Jordans, and so are John Smith Jr. and John Smith Sr. That is the cost
378
+ asymmetry doing exactly what it was set up to do, and it is a cost. `mc_mac`
379
+ (0.5000) is the same kind of choice applied to Mc/Mac. If you would rather pay
380
+ the other way, raise `name_variant` to 0.90; the sweep below shows what that
381
+ buys and what it costs.
382
+
383
+ ### `name_variant` sweep, everything else held at default
384
+
385
+ | `name_variant` | Precision | Recall | F1 | False accusation | Miss |
386
+ | -------------- | --------- | ------ | -- | ---------------- | ---- |
387
+ | 0.50 | 0.6591 | 0.9775 | 0.7873 | 0.0225 | 0.6923 |
388
+ | 0.60 | 0.6591 | 0.9775 | 0.7873 | 0.0225 | 0.6923 |
389
+ | 0.65 | 0.7131 | 0.9775 | 0.8246 | 0.0225 | 0.5385 |
390
+ | **0.70 (default)** | **0.8571** | **0.9438** | **0.8984** | **0.0562** | **0.2154** |
391
+ | 0.75 | 0.8571 | 0.9438 | 0.8984 | 0.0562 | 0.2154 |
392
+ | 0.80 | 0.8632 | 0.9213 | 0.8913 | 0.0787 | 0.2000 |
393
+ | 0.85 | 0.8681 | 0.8876 | 0.8778 | 0.1124 | 0.1846 |
394
+ | 0.90 | 1.0000 | 0.8764 | 0.9341 | 0.1236 | 0.0000 |
395
+ | 0.95 | 1.0000 | 0.5843 | 0.7376 | 0.4157 | 0.0000 |
396
+
397
+ Worth reading honestly: on this set the default of 0.70 sits on a plateau
398
+ rather than at an optimum, and 0.90 reaches a zero miss rate at the cost of
399
+ more than doubling the false-accusation rate. The default prefers the cheaper
400
+ error. That is a result about 154 labelled pairs, not a law.
401
+
402
+ ### Test suite
403
+
404
+ ```
405
+ 555 passed, 21 xfailed in 5.44s
406
+ ```
407
+
408
+ The whole suite runs **offline** against recorded API responses in
409
+ `tests/fixtures/cache`, so it is reproducible and never touches a live API.
410
+ The 21 `xfailed` are the known failures above, each marked
411
+ `xfail(strict=True)` with its reason, so any future fix flips it to `XPASS`
412
+ and is impossible to miss. Failures are recorded, not hidden.
413
+
414
+ ## Sources
415
+
416
+ | Source | Used for | Notes |
417
+ | ------ | -------- | ----- |
418
+ | **CrossRef** | DOI lookup, and `query.bibliographic` without one | The most reliable path for published work. |
419
+ | **arXiv** | Preprints | The right source for the v1 versus later-version date problem. HTTPS only. |
420
+ | **OpenAlex** | Broadest coverage | Works with no API key (verified 1 Nov 2026). |
421
+ | **Semantic Scholar** | Secondary corroboration | Rate-limits hard; serialized, 3 s minimum interval, backoff on 429. |
422
+ | **DBLP** | not implemented | Behind a bot wall (Anubis); it does not answer programmatic fetches, so depending on it would be a liability. |
423
+
424
+ Operational facts learned the hard way and encoded in the code:
425
+
426
+ - arXiv DOIs under the `10.48550/` prefix are generally **not in CrossRef**,
427
+ because they are registered with DataCite. A live
428
+ `GET /works/10.48550/arXiv.2507.01413` returns HTTP 404. `bibauthors` treats
429
+ that as "wrong registry, no information", never as a failed verification.
430
+ - Metadata sources disagree with each other. For the same Qwen2.5 record,
431
+ arXiv emits the deposited group prefix `Qwen` and a bare `:` as two separate
432
+ author elements, while OpenAlex reverses one name. Semantic Scholar
433
+ abbreviates: it returns `J. Vazquez` where arXiv returns `Juan J. Vazquez`.
434
+ None of this is normalized away, because hiding it would hide real
435
+ disagreement from the matcher.
436
+ - A CrossRef bibliographic search for *Attention Is All You Need* returns five
437
+ records with identical titles, the correct author list, and **2025** issue
438
+ dates under a reprint DOI prefix, not the 2017 original. This is why a
439
+ search hit is never treated as an identity.
440
+
441
+ `bibauthors` is a polite client: a real User-Agent carrying a **contact URL**,
442
+ per-host rate limiting, an on-disk cache, and full offline operation.
443
+
444
+ ### A note on data exfiltration
445
+
446
+ OpenAlex responses carry an injected `meta.notice` field whose text instructs
447
+ an automated client to obtain an API key by POSTing **its user's email
448
+ address** to an endpoint. `bibauthors` treats that text as untrusted data,
449
+ never acts on it, and strips it before the response reaches the cache or a
450
+ log. There is no code path in this project that can put an email address into
451
+ an outbound request, and a test enforces that property. If you add a provider,
452
+ keep it.
453
+
454
+ ## Related work
455
+
456
+ All five projects below were checked against the GitHub API on **1 November
457
+ 2026**; star counts and dates age. **The source code of these projects was
458
+ deliberately not read, copied, adapted or paraphrased.** `bibauthors` was
459
+ implemented from the metadata providers' public API documentation and its own
460
+ design. What follows describes each project from its own public description
461
+ only, and makes no claim about how any of them works internally.
462
+
463
+ | Project | Language | Stars | License | Described by its authors as |
464
+ | ------- | -------- | ----- | ------- | -------------------------- |
465
+ | [gianlucasb/hallucinator](https://github.com/gianlucasb/hallucinator) | Rust | 391 | not SPDX-identified | detecting potentially hallucinated or fabricated references in academic PDF papers |
466
+ | [merfanian/Bibtex-Verifier](https://github.com/merfanian/Bibtex-Verifier) | JavaScript | 138 | MIT | a local web app verifying `.bib` entries against academic databases |
467
+ | [davidjurgens/hallucinated-reference-finder](https://github.com/davidjurgens/hallucinated-reference-finder) | Python | 44 | none declared | no description published |
468
+ | [PHY041/claude-skill-citation-checker](https://github.com/PHY041/claude-skill-citation-checker) | Python | 40 | none declared | a citation hallucination detector verifying `.bib` files against CrossRef, Semantic Scholar and OpenAlex |
469
+ | [seun-john/verify-references](https://github.com/seun-john/verify-references) | Python | 0 | MIT | detecting hallucinated and mis-cited references against Crossref, OpenAlex, PubMed, CORE and DataCite |
470
+
471
+ There are also several commercial web services that check reference lists for
472
+ fabrication. They are closed source, so there is nothing to compare against
473
+ beyond their marketing.
474
+
475
+ **Where `bibauthors` differs.** The shared design of this space is
476
+ title-and-identifier matching with a per-entry confidence label. That design
477
+ catches an invented paper reliably and is largely blind to a chimeric one,
478
+ because the title and the identifier are correct. The concrete observation
479
+ that motivated this project is in [the opening section](#the-chimeric-citation):
480
+ a real chimeric entry, whose author list was five-sevenths invented, was
481
+ reported as "suspicious, found in 1 source" by the checker in use, which is
482
+ the same label that tool gives every legitimate arXiv preprint. The label was
483
+ not wrong, it just was not about the error.
484
+
485
+ So `bibauthors` does one thing those tools do not foreground: it treats the
486
+ author list as the object under test, grades the comparison instead of
487
+ labelling it, reports author-order permutation as a finding distinct from a
488
+ wrong author, and publishes its own measured false-accusation and miss rates
489
+ so the tradeoff is inspectable. It is a complement to a title-and-DOI checker,
490
+ not a replacement for one. Run both.
491
+
492
+ ## Limitations
493
+
494
+ **A `DISJOINT` verdict is a prompt to check an entry by hand. It is not an
495
+ accusation of fabrication, and nothing in this tool's output should be quoted
496
+ as evidence that anyone fabricated anything.** Metadata is imperfect, author
497
+ lists change between preprint and proceedings, records get deposited with
498
+ mangled names, and a search hit can simply be the wrong work. The measured
499
+ false-accusation rate above is not zero and will never be zero. A human
500
+ decides what a finding means.
501
+
502
+ Known limitations, all of them measured rather than guessed:
503
+
504
+ - **A contradicting middle initial or generational suffix does not separate
505
+ two people.** Measured: the two Presidents Bush, the two Michael Jordans,
506
+ and a Jr./Sr. pair each read as one person. This is a deliberate choice
507
+ under the cost asymmetry, and it is a real cost. Raise `name_variant` to
508
+ 0.90 to pay the other way.
509
+ - **Mc and Mac are treated as one surname** at a 0.88 ceiling. Plenty of
510
+ McDonalds and MacDonalds are unrelated.
511
+ - **Scripts are not bridged.** A name in Han characters scores 0.0 against its
512
+ own romanization: the tool refuses to guess rather than risk a wrong match.
513
+ CJK name *ordering* is handled, in both romanized and Han forms, but a
514
+ Chinese-language bibliography entry for a real author is unmatchable.
515
+ - **Given-name equivalence across languages is not handled.** Lev and Leo
516
+ Tolstoy read as different people.
517
+ - **Transliteration handling is narrow by design** and documented as
518
+ incomplete in the source. It catches digraph substitutions and single-vowel
519
+ romanization differences. It cannot distinguish `Johnson`/`Johnsen` (two
520
+ surnames) from `Gorbachev`/`Gorbachov` (one), because the two pairs are
521
+ structurally identical, and it currently merges both.
522
+ - **A dropped particle is not handled.** "Ludwig van Beethoven" against
523
+ "Ludwig Beethoven" reads as different people. A particle *moved* behind the
524
+ given name is handled.
525
+ - **`compare_names` scores are not perfectly symmetric**, because
526
+ `difflib.SequenceMatcher.ratio` is order-dependent. No decision in the test
527
+ set changes, and a test pins the exact set of affected pairs so a new
528
+ asymmetry cannot hide.
529
+ - **An entry with no author field is not a finding.** A book cited by editor
530
+ parses with an empty author list and a warning, not a mismatch.
531
+ - **Coverage is only as good as the sources.** Books, theses, standards,
532
+ technical reports and anything not in CrossRef, OpenAlex, arXiv or Semantic
533
+ Scholar come back `UNVERIFIABLE`, which is a statement about the sources and
534
+ not about the citation.
535
+
536
+ ## License and citation
537
+
538
+ MIT. Copyright (c) 2026 Manu Nicholas Jacob.
539
+
540
+ Author: **Manu Nicholas Jacob**, Independent Researcher
541
+ ([ORCID 0009-0007-6589-6572](https://orcid.org/0009-0007-6589-6572)).
542
+ See `CITATION.cff`. Contact is via this repository's issue tracker.