claimcite 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- claimcite-0.1.0/.github/ISSUE_TEMPLATE/false-result.yml +22 -0
- claimcite-0.1.0/.github/ISSUE_TEMPLATE/new-source.yml +16 -0
- claimcite-0.1.0/.github/pull_request_template.md +8 -0
- claimcite-0.1.0/.github/workflows/ci.yml +28 -0
- claimcite-0.1.0/.github/workflows/release.yml +51 -0
- claimcite-0.1.0/.gitignore +7 -0
- claimcite-0.1.0/.pre-commit-hooks.yaml +7 -0
- claimcite-0.1.0/AGENTS.md +109 -0
- claimcite-0.1.0/CHANGELOG.md +16 -0
- claimcite-0.1.0/CLAUDE.md +1 -0
- claimcite-0.1.0/CONTRIBUTING.md +41 -0
- claimcite-0.1.0/LICENSE +21 -0
- claimcite-0.1.0/PKG-INFO +155 -0
- claimcite-0.1.0/README.md +132 -0
- claimcite-0.1.0/README.zh-CN.md +118 -0
- claimcite-0.1.0/action.yml +43 -0
- claimcite-0.1.0/examples/sample.md +11 -0
- claimcite-0.1.0/pyproject.toml +45 -0
- claimcite-0.1.0/src/claimcite/__init__.py +1 -0
- claimcite-0.1.0/src/claimcite/__main__.py +5 -0
- claimcite-0.1.0/src/claimcite/cache.py +118 -0
- claimcite-0.1.0/src/claimcite/check.py +82 -0
- claimcite-0.1.0/src/claimcite/cli.py +63 -0
- claimcite-0.1.0/src/claimcite/evidence.py +47 -0
- claimcite-0.1.0/src/claimcite/extract.py +150 -0
- claimcite-0.1.0/src/claimcite/models.py +64 -0
- claimcite-0.1.0/src/claimcite/net.py +61 -0
- claimcite-0.1.0/src/claimcite/report.py +157 -0
- claimcite-0.1.0/src/claimcite/sources.py +209 -0
- claimcite-0.1.0/src/claimcite/text.py +34 -0
- claimcite-0.1.0/src/claimcite/web.py +93 -0
- claimcite-0.1.0/tests/conftest.py +42 -0
- claimcite-0.1.0/tests/fixtures/crossref_work.json +1 -0
- claimcite-0.1.0/tests/fixtures/esearch_doi_found.json +1 -0
- claimcite-0.1.0/tests/fixtures/esearch_doi_missing.json +1 -0
- claimcite-0.1.0/tests/fixtures/handle_found.json +1 -0
- claimcite-0.1.0/tests/fixtures/handle_missing.json +1 -0
- claimcite-0.1.0/tests/fixtures/pubmed_efetch.xml +5 -0
- claimcite-0.1.0/tests/fixtures/pubmed_missing.xml +3 -0
- claimcite-0.1.0/tests/fixtures/web_page.html +15 -0
- claimcite-0.1.0/tests/fixtures/web_robots_allow.txt +2 -0
- claimcite-0.1.0/tests/fixtures/web_robots_deny.txt +2 -0
- claimcite-0.1.0/tests/test_cache.py +152 -0
- claimcite-0.1.0/tests/test_check.py +121 -0
- claimcite-0.1.0/tests/test_cli.py +64 -0
- claimcite-0.1.0/tests/test_extract.py +57 -0
- claimcite-0.1.0/tests/test_net.py +31 -0
- claimcite-0.1.0/tests/test_numbered.py +73 -0
- claimcite-0.1.0/tests/test_real_world.py +101 -0
- claimcite-0.1.0/tests/test_web.py +170 -0
- claimcite-0.1.0/uv.lock +202 -0
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
name: Wrong result · 结果不对
|
|
2
|
+
description: claimcite flagged a correct citation, or missed a wrong one · 正确的引用被报错,或错误的引用没被发现
|
|
3
|
+
labels: [accuracy]
|
|
4
|
+
body:
|
|
5
|
+
- type: textarea
|
|
6
|
+
attributes:
|
|
7
|
+
label: Citation text · 引用原文
|
|
8
|
+
description: Paste the exact line from your document · 粘贴文档里的原始那一行
|
|
9
|
+
render: markdown
|
|
10
|
+
validations:
|
|
11
|
+
required: true
|
|
12
|
+
- type: textarea
|
|
13
|
+
attributes:
|
|
14
|
+
label: What claimcite said · claimcite 的输出
|
|
15
|
+
render: text
|
|
16
|
+
validations:
|
|
17
|
+
required: true
|
|
18
|
+
- type: textarea
|
|
19
|
+
attributes:
|
|
20
|
+
label: What it should have said, and why · 应该是什么结果,为什么
|
|
21
|
+
validations:
|
|
22
|
+
required: true
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
name: New source or format · 新数据源或新格式
|
|
2
|
+
description: Propose a registry (OpenAlex, CNKI…) or a citation format (footnotes, BibTeX…) · 提议新的数据源或引用格式
|
|
3
|
+
labels: [enhancement]
|
|
4
|
+
body:
|
|
5
|
+
- type: input
|
|
6
|
+
attributes:
|
|
7
|
+
label: Source or format · 数据源或格式
|
|
8
|
+
validations:
|
|
9
|
+
required: true
|
|
10
|
+
- type: textarea
|
|
11
|
+
attributes:
|
|
12
|
+
label: Example citations · 引用示例
|
|
13
|
+
description: Real examples help most · 真实例子最有用
|
|
14
|
+
- type: textarea
|
|
15
|
+
attributes:
|
|
16
|
+
label: API / docs · 接口文档
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
## What and why · 改了什么,为什么
|
|
2
|
+
|
|
3
|
+
## Tests · 测试
|
|
4
|
+
- [ ] `uv run pytest` and `uv run ruff check src tests` pass
|
|
5
|
+
- [ ] New network responses are recorded in `tests/fixtures/` · 新的网络响应已录进 fixtures
|
|
6
|
+
|
|
7
|
+
## AI assistance · AI 辅助
|
|
8
|
+
<!-- Did you use AI tools? Say which, and confirm you read and ran the result. · 是否用了 AI 工具?说明是哪个,并确认你读过、跑过。 -->
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
on:
|
|
3
|
+
push:
|
|
4
|
+
branches: [main]
|
|
5
|
+
pull_request:
|
|
6
|
+
jobs:
|
|
7
|
+
test:
|
|
8
|
+
runs-on: ubuntu-latest
|
|
9
|
+
strategy:
|
|
10
|
+
matrix:
|
|
11
|
+
python: ["3.10", "3.12", "3.13"]
|
|
12
|
+
steps:
|
|
13
|
+
- uses: actions/checkout@v4
|
|
14
|
+
- uses: astral-sh/setup-uv@v6
|
|
15
|
+
with:
|
|
16
|
+
python-version: ${{ matrix.python }}
|
|
17
|
+
- run: uv sync
|
|
18
|
+
- run: uv run ruff check src tests
|
|
19
|
+
- run: uv run pytest -q
|
|
20
|
+
action:
|
|
21
|
+
runs-on: ubuntu-latest
|
|
22
|
+
steps:
|
|
23
|
+
- uses: actions/checkout@v4
|
|
24
|
+
- uses: ./
|
|
25
|
+
with:
|
|
26
|
+
files: examples/*.md
|
|
27
|
+
fail-on: never
|
|
28
|
+
web: "false"
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Push a tag such as v0.1.0 to publish. The version in src/claimcite/__init__.py must match the tag.
|
|
2
|
+
# PyPI side (once): add a trusted publisher for this repo, workflow release.yml, environment "pypi".
|
|
3
|
+
name: Release
|
|
4
|
+
on:
|
|
5
|
+
push:
|
|
6
|
+
tags: ["v*"]
|
|
7
|
+
jobs:
|
|
8
|
+
build:
|
|
9
|
+
runs-on: ubuntu-latest
|
|
10
|
+
steps:
|
|
11
|
+
- uses: actions/checkout@v4
|
|
12
|
+
- uses: astral-sh/setup-uv@v6
|
|
13
|
+
with:
|
|
14
|
+
python-version: "3.12"
|
|
15
|
+
- run: uv sync
|
|
16
|
+
- run: uv run pytest -q
|
|
17
|
+
- name: Tag matches the package version
|
|
18
|
+
run: test "v$(uv run python -c 'import claimcite; print(claimcite.__version__)')" = "$GITHUB_REF_NAME"
|
|
19
|
+
- run: uv build
|
|
20
|
+
- uses: actions/upload-artifact@v4
|
|
21
|
+
with:
|
|
22
|
+
name: dist
|
|
23
|
+
path: dist/
|
|
24
|
+
pypi:
|
|
25
|
+
needs: build
|
|
26
|
+
runs-on: ubuntu-latest
|
|
27
|
+
environment: pypi
|
|
28
|
+
permissions:
|
|
29
|
+
id-token: write
|
|
30
|
+
steps:
|
|
31
|
+
- uses: actions/download-artifact@v4
|
|
32
|
+
with:
|
|
33
|
+
name: dist
|
|
34
|
+
path: dist/
|
|
35
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
|
36
|
+
github-release:
|
|
37
|
+
needs: pypi
|
|
38
|
+
runs-on: ubuntu-latest
|
|
39
|
+
permissions:
|
|
40
|
+
contents: write
|
|
41
|
+
steps:
|
|
42
|
+
- uses: actions/checkout@v4
|
|
43
|
+
- uses: actions/download-artifact@v4
|
|
44
|
+
with:
|
|
45
|
+
name: dist
|
|
46
|
+
path: dist/
|
|
47
|
+
- name: Release notes from CHANGELOG.md
|
|
48
|
+
run: awk -v v="${GITHUB_REF_NAME#v}" '/^## /{if (p) exit; if (index($0, "## " v)) {p=1; next}} p' CHANGELOG.md > notes.md
|
|
49
|
+
- run: gh release create "$GITHUB_REF_NAME" dist/* --title "claimcite $GITHUB_REF_NAME" --notes-file notes.md
|
|
50
|
+
env:
|
|
51
|
+
GH_TOKEN: ${{ github.token }}
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# Instructions for AI agents · AI 代理须知
|
|
2
|
+
|
|
3
|
+
[English](#english) · [中文](#中文)
|
|
4
|
+
|
|
5
|
+
These rules apply to any AI coding agent working on claimcite. Human contributors: see [CONTRIBUTING.md](CONTRIBUTING.md), which these rules extend.
|
|
6
|
+
|
|
7
|
+
## English
|
|
8
|
+
|
|
9
|
+
### Think from first principles
|
|
10
|
+
|
|
11
|
+
Start from what is true, not from habit or what other projects do.
|
|
12
|
+
|
|
13
|
+
- **Start from the goal.** claimcite exists so a reader can trust that each citation is real, points to the paper the text names, and contains the quoted words. Judge every change by whether it makes that verdict more correct, or easier to get.
|
|
14
|
+
- **Get facts by running things.** Read the code, run it, and look at what the real service returns before you decide. Do not guess a field's meaning from its name. For example, Crossref's `update-to` sits on the retraction notice, not on the retracted paper. Only the real record shows that.
|
|
15
|
+
- **Question the starting point.** Issues, labels, earlier reviews and this file can be wrong or out of date. Check that the problem is real and still unsolved before you fix it.
|
|
16
|
+
- **Build the simplest fix that the facts require.** Do not add layers, options or dependencies the problem does not need.
|
|
17
|
+
- **Show your reasoning.** In the PR, say which facts you relied on (a recorded response, a failing test, a measurement) and how they lead to the change.
|
|
18
|
+
|
|
19
|
+
### Check before you commit
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
uv sync
|
|
23
|
+
uv run pytest
|
|
24
|
+
uv run ruff check src tests
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Both must pass. Tests run offline in well under a second.
|
|
28
|
+
|
|
29
|
+
### Where things live
|
|
30
|
+
|
|
31
|
+
| File | Job |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `src/claimcite/extract.py` | Find citations (DOI, PMID, URL) and the title, year and quotes written next to them |
|
|
34
|
+
| `src/claimcite/sources.py` | Talk to doi.org, PubMed and Crossref; parse their responses into `Record` |
|
|
35
|
+
| `src/claimcite/check.py` | Compare what the text says with the `Record`; decide ok / warning / error / unchecked |
|
|
36
|
+
| `src/claimcite/report.py` | Markdown (English and Chinese) and JSON output |
|
|
37
|
+
| `src/claimcite/net.py` | HTTP with retries and per-host rate limits |
|
|
38
|
+
| `src/claimcite/cache.py` | On-disk cache of successful responses |
|
|
39
|
+
| `src/claimcite/web.py` | Fetch web pages (respecting robots.txt) and look for quoted text |
|
|
40
|
+
| `tests/fixtures/` | Real responses recorded from the services |
|
|
41
|
+
|
|
42
|
+
### Rules
|
|
43
|
+
|
|
44
|
+
1. **Bug fixes start with a failing test.** Write the test, watch it fail, then fix.
|
|
45
|
+
2. **Fixtures are recorded, not invented.** Save real responses from the service, including "not found". In `tests/conftest.py`, route only the exact identifiers a test uses. A broad pattern such as a whole DOI prefix makes unrelated tests pass by accident.
|
|
46
|
+
3. **Test the path you changed.** If a paper is found in PubMed first, a test through `check_reference` never reaches the Crossref parser. Test the parser directly as well.
|
|
47
|
+
4. **Assertions must be able to fail.** `assert a or b` that is true in the broken case proves nothing.
|
|
48
|
+
5. **False alarms are worse than misses.** Report `error` only when the registry answer is definite. If you are unsure, report `warning`. If the lookup itself failed, report `unchecked` and fill in `Finding.error`. Never report "does not exist" because a request failed.
|
|
49
|
+
6. **No secrets on disk or in history.** API keys, tokens and email addresses must not appear in the cache, logs, fixtures, test URLs or commits. The cache key strips `api_key`. Keep it that way for any new credential.
|
|
50
|
+
7. **Optional parts must not crash the run.** If the cache, or a single source, cannot be used, print one line to stderr and continue.
|
|
51
|
+
8. **Be polite to free services.** Batch requests, go through `net.http_get` for its rate limiting, and send the claimcite User-Agent. Do not put anyone's contact details in requests.
|
|
52
|
+
9. **Standard library only at runtime**, Python 3.10+. Discuss a new runtime dependency in an issue first. Dev tools go in the `dev` group.
|
|
53
|
+
10. **Both languages.** Every user-facing message in `report.py` needs English and Chinese. README changes go into both `README.md` and `README.zh-CN.md`.
|
|
54
|
+
11. **No traces of your own environment.** Do not leave comments, workarounds or paths that only make sense on the machine you ran on.
|
|
55
|
+
12. **One issue per pull request.** Comment on the issue before you start. Say in the PR that it is AI-assisted.
|
|
56
|
+
13. **Commit identity.** Do not commit with a personal email address that should stay private. Use a GitHub noreply address.
|
|
57
|
+
14. **Run a new check on a real document before calling it done.** Read every warning it gives and decide whether the document or the tool is wrong. Offline tests passed for the first web check, but a run on a real 34-chapter guide found 7 false warnings out of 15 in one chapter (curly vs straight quotes, extra spaces), plus a false warning on every PDF with a quote. Note the run time, too.
|
|
58
|
+
|
|
59
|
+
## 中文
|
|
60
|
+
|
|
61
|
+
### 第一性原理
|
|
62
|
+
|
|
63
|
+
从事实出发推导,不照搬习惯,也不照搬别的项目的做法。
|
|
64
|
+
|
|
65
|
+
- **先问目标。** claimcite 的目标是让读者能相信每条引用:论文真实存在,就是文中说的那篇,引号里的话确实出自它。每个改动都看它是否让这个判断更准确,或更容易得到。
|
|
66
|
+
- **事实靠运行得到。** 动手前先读代码、跑代码,看真实服务返回什么。不要凭字段名猜意思。比如 Crossref 的 `update-to` 在撤稿通知上,不在被撤的论文上,只有看真实记录才知道。
|
|
67
|
+
- **质疑出发点。** issue、标签、之前的 review,包括这份文件,都可能是错的或过时的。修之前先确认问题真实存在、还没人解决。
|
|
68
|
+
- **按事实需要做最简单的方案。** 问题不需要的抽象层、选项和依赖都不要加。
|
|
69
|
+
- **说清推理。** 在 PR 里写明你依据的事实(录下来的响应、失败的测试、实测数据),以及它们怎么推出这个改动。
|
|
70
|
+
|
|
71
|
+
### 提交前检查
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
uv sync
|
|
75
|
+
uv run pytest
|
|
76
|
+
uv run ruff check src tests
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
两项都必须通过。测试不联网,一秒内跑完。
|
|
80
|
+
|
|
81
|
+
### 代码分工
|
|
82
|
+
|
|
83
|
+
| 文件 | 负责 |
|
|
84
|
+
|---|---|
|
|
85
|
+
| `src/claimcite/extract.py` | 找出引用(DOI、PMID、网址),以及旁边写的标题、年份和引文 |
|
|
86
|
+
| `src/claimcite/sources.py` | 访问 doi.org、PubMed、Crossref,把返回结果解析成 `Record` |
|
|
87
|
+
| `src/claimcite/check.py` | 比对文中写的和 `Record`,判定通过、警告、错误或未核查 |
|
|
88
|
+
| `src/claimcite/report.py` | Markdown(中英文)和 JSON 输出 |
|
|
89
|
+
| `src/claimcite/net.py` | 带重试和按站点限速的网络请求 |
|
|
90
|
+
| `src/claimcite/cache.py` | 把查询成功的结果缓存到磁盘 |
|
|
91
|
+
| `src/claimcite/web.py` | 抓取网页(遵守 robots.txt),查找引文 |
|
|
92
|
+
| `tests/fixtures/` | 从真实服务录下来的响应 |
|
|
93
|
+
|
|
94
|
+
### 规则
|
|
95
|
+
|
|
96
|
+
1. **修 bug 先写测试。** 先写测试,确认它失败,再修。
|
|
97
|
+
2. **测试数据要录,不要编。** 保存服务的真实响应,包括"查不到"的情况。在 `tests/conftest.py` 里只为测试用到的具体编号配路由。范围太宽的规则(比如整个 DOI 前缀)会让无关的测试碰巧通过。
|
|
98
|
+
3. **测到你改的那条路径。** 如果论文先在 PubMed 查到,经过 `check_reference` 的测试就走不到 Crossref 的解析代码,要对解析函数单独写测试。
|
|
99
|
+
4. **断言必须有可能失败。** 像 `assert a or b` 这样在代码出错时也成立的断言,什么也证明不了。
|
|
100
|
+
5. **误报比漏报更糟。** 只有数据源给出确定答案时才报"错误"。不确定就报"警告"。查询本身失败时报"未核查",并填写 `Finding.error`。不能因为请求失败就报"编号不存在"。
|
|
101
|
+
6. **密钥不落盘,也不进历史。** API key、token、邮箱不能出现在缓存、日志、测试数据、测试网址或提交里。缓存键会去掉 `api_key`,以后新增任何凭据也要这样处理。
|
|
102
|
+
7. **可选部分不能让整个程序崩溃。** 缓存或某个数据源用不了时,往 stderr 打一行提示,然后继续。
|
|
103
|
+
8. **善待免费服务。** 批量请求,通过 `net.http_get` 发请求以遵守限速,并带上 claimcite 的 User-Agent。请求里不要放任何人的联系方式。
|
|
104
|
+
9. **运行时只用标准库**,Python 3.10 及以上。要加运行时依赖,先开 issue 讨论。开发工具放进 `dev` 组。
|
|
105
|
+
10. **中英双语。** `report.py` 里每条面向用户的提示都要有中文和英文。改 README 要同时改 `README.md` 和 `README.zh-CN.md`。
|
|
106
|
+
11. **不留下你自己运行环境的痕迹。** 只在你那台机器上才说得通的注释、绕路写法和路径,都不要提交。
|
|
107
|
+
12. **一个 PR 只做一个 issue。** 开工前先在 issue 下留言。在 PR 里说明是 AI 辅助完成的。
|
|
108
|
+
13. **提交身份。** 不要用需要保密的个人邮箱提交,用 GitHub 的 noreply 邮箱。
|
|
109
|
+
14. **新的检查先在真实文档上跑一遍再算完成。** 逐条看它报的警告,判断是文档错了还是工具错了。网页链接检查第一版的离线测试全部通过,但在一本 34 章的真实指南上一跑,一章的 15 条警告里有 7 条是误报(直引号和弯引号、多出的空格),带引文的 PDF 也每条都误报。顺便记下运行时间。
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 (unreleased)
|
|
4
|
+
|
|
5
|
+
First release on PyPI: `pipx install claimcite`.
|
|
6
|
+
|
|
7
|
+
- Check that each DOI and PMID exists (doi.org handle API for every registrar, PubMed).
|
|
8
|
+
- Compare the title and year written in the text with the registry (PubMed, then Crossref).
|
|
9
|
+
- Look for quoted text (`"…"`, `“…”`, `「…」`) in the abstract.
|
|
10
|
+
- Show the study type PubMed records: meta-analysis, RCT, observational, animal study…
|
|
11
|
+
- Web links: check that the page is alive and contains the quoted text, respecting `robots.txt`.
|
|
12
|
+
- Numbered citations: a quote followed by `[1]`, `[^1]`, `[1, 2]` or `[1-3]` is checked against that
|
|
13
|
+
entry in the reference list.
|
|
14
|
+
- Check several files in one run; GitHub Action and pre-commit hook.
|
|
15
|
+
- Markdown report in English or Chinese, JSON for scripts, exit code 1 on errors for CI.
|
|
16
|
+
- 30-day on-disk cache of registry answers.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
@AGENTS.md
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Contributing · 参与贡献
|
|
2
|
+
|
|
3
|
+
[English](#english) · [中文](#中文)
|
|
4
|
+
|
|
5
|
+
## English
|
|
6
|
+
|
|
7
|
+
1. **Pick something.** Roadmap items in the README and issues labelled `good first issue` are ready to start.
|
|
8
|
+
Comment on the issue so two people don't do the same work.
|
|
9
|
+
2. **Set up.**
|
|
10
|
+
```bash
|
|
11
|
+
git clone https://github.com/wasabihu/claimcite && cd claimcite
|
|
12
|
+
uv sync
|
|
13
|
+
uv run pytest && uv run ruff check src tests
|
|
14
|
+
```
|
|
15
|
+
3. **Tests run offline.** Every network response used in tests is a file in `tests/fixtures/`, recorded from
|
|
16
|
+
the real service. When you add a registry, record real responses (including "not found") and route them in
|
|
17
|
+
`tests/conftest.py`.
|
|
18
|
+
4. **Bug fixes start with a failing test** that reproduces the bug, then the fix.
|
|
19
|
+
5. **Keep false alarms rare.** A checker that cries wolf gets ignored. If a check is uncertain, report a
|
|
20
|
+
warning or "not checked" rather than an error.
|
|
21
|
+
6. **Be polite to free services.** Batch requests, respect rate limits, send the claimcite User-Agent.
|
|
22
|
+
|
|
23
|
+
**AI-assisted contributions are welcome.** Say so in the pull request, and make sure you have read and run
|
|
24
|
+
what you submit. You are responsible for it, not the tool.
|
|
25
|
+
AI agents: read [AGENTS.md](AGENTS.md) first.
|
|
26
|
+
|
|
27
|
+
## 中文
|
|
28
|
+
|
|
29
|
+
1. **选一项来做。** README 里的路线图和带 `good first issue` 标签的 issue 都可以直接开始做。先在 issue 下留言说你在做,免得两个人做重复了。
|
|
30
|
+
2. **搭环境。**
|
|
31
|
+
```bash
|
|
32
|
+
git clone https://github.com/wasabihu/claimcite && cd claimcite
|
|
33
|
+
uv sync
|
|
34
|
+
uv run pytest && uv run ruff check src tests
|
|
35
|
+
```
|
|
36
|
+
3. **测试不联网。** 测试用到的每个网络响应都是 `tests/fixtures/` 里的文件,录自真实服务。新增数据源时,把真实响应(包括"查不到"的情况)录下来,并在 `tests/conftest.py` 里配好路由。
|
|
37
|
+
4. **修 bug 先写测试。** 先写一个能复现 bug、会失败的测试,再修。
|
|
38
|
+
5. **少误报。** 一个老是误报的检查工具,最后没人会理会。不确定的情况报"警告"或"未核查",不要报"错误"。
|
|
39
|
+
6. **善待免费服务。** 尽量批量请求,遵守访问频率限制,请求时带上 claimcite 的 User-Agent。
|
|
40
|
+
|
|
41
|
+
**欢迎 AI 辅助完成的贡献。** 在 PR 里说明,并确保提交的内容你自己读过、跑过。对内容负责的是你,不是工具。AI 代理请先读 [AGENTS.md](AGENTS.md)。
|
claimcite-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 claimcite contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
claimcite-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: claimcite
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Check that every citation in a document is real, matches what the text says it is, and supports the quoted words.
|
|
5
|
+
Project-URL: Homepage, https://github.com/wasabihu/claimcite
|
|
6
|
+
Project-URL: Issues, https://github.com/wasabihu/claimcite/issues
|
|
7
|
+
Project-URL: Changelog, https://github.com/wasabihu/claimcite/blob/main/CHANGELOG.md
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: citation,crossref,doi,evidence-based,fact-checking,hallucination,pubmed
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
|
|
20
|
+
Classifier: Topic :: Text Processing
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
|
|
24
|
+
# claimcite
|
|
25
|
+
|
|
26
|
+
**English** · [中文](README.zh-CN.md)
|
|
27
|
+
|
|
28
|
+
Check that every citation in a document is real, points to the paper the text says it does, and actually contains the words quoted from it.
|
|
29
|
+
|
|
30
|
+
AI-written articles, health advice and even papers increasingly cite sources that don't exist, or attach a real DOI to the wrong paper. Readers can't check dozens of references by hand. claimcite does it in one command.
|
|
31
|
+
|
|
32
|
+
```console
|
|
33
|
+
$ claimcite check examples/sample.md
|
|
34
|
+
# Citation check
|
|
35
|
+
|
|
36
|
+
1 ok · 2 warnings · 2 errors · 0 not checked
|
|
37
|
+
|
|
38
|
+
- **Error** · line 7 · `doi:10.1016/s0140-6736(10)61388-8`
|
|
39
|
+
- Title does not match this identifier. Registry title: Worldwide burden of disease from
|
|
40
|
+
exposure to second-hand smoke: a retrospective analysis of data from 192 countries.
|
|
41
|
+
- Year differs: document 2021, registry 2011
|
|
42
|
+
- **Error** · line 9 · `doi:10.1001/jama.2022.99999`
|
|
43
|
+
- Identifier does not exist
|
|
44
|
+
- **Warning** · line 11 · `pmid:33237064`
|
|
45
|
+
- Study type (as tagged by PubMed): Systematic review / meta-analysis
|
|
46
|
+
- Quoted text not found in the abstract (it may be in the full text): “calcium and vitamin D
|
|
47
|
+
supplementation cured osteoporosis in all women”
|
|
48
|
+
...
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## What it checks today
|
|
52
|
+
|
|
53
|
+
| Check | How |
|
|
54
|
+
|---|---|
|
|
55
|
+
| The DOI / PMID exists | doi.org handle API (every registrar), PubMed |
|
|
56
|
+
| The title written in the text matches the identifier | PubMed, then Crossref; fuzzy match |
|
|
57
|
+
| The year matches (±1 for online-first vs print) | same |
|
|
58
|
+
| Text in quotes (`"…"`, `“…”`, `「…」`) appears in the abstract | PubMed / Crossref abstract |
|
|
59
|
+
| Study type: meta-analysis, RCT, observational, animal… | PubMed publication types and MeSH |
|
|
60
|
+
| A web link is alive and contains the quoted text | fetch the page (respecting `robots.txt`), strip HTML, normalise whitespace |
|
|
61
|
+
|
|
62
|
+
It reads Markdown or plain text and understands the common
|
|
63
|
+
`Author (Year). Title. Journal: "quote". https://doi.org/…` pattern, plus `doi:…`, bare DOIs,
|
|
64
|
+
`PMID: …` and PubMed links. Numbered citations work too: a quote in the text followed by `[1]`, `[^1]`,
|
|
65
|
+
`[1, 2]` or `[1-3]` is checked against the matching entry in the reference list (`[1] …`, `1. …` or
|
|
66
|
+
`[^1]: …`). Code blocks and inline code are skipped. Other web links are fetched (respecting `robots.txt`) and checked for
|
|
67
|
+
availability and for the quoted text. A dead link or a missing quote is a **warning**, never an error —
|
|
68
|
+
pages change and some sites block bots. When a page cannot be read the link stays *not checked*.
|
|
69
|
+
|
|
70
|
+
**What it does not do (yet):** decide whether a source *supports* a claim. That needs a language model and a
|
|
71
|
+
labelled test set. See the roadmap. claimcite only confirms that sources exist and match what the text says
|
|
72
|
+
about them. It is not medical advice.
|
|
73
|
+
|
|
74
|
+
## Install and run
|
|
75
|
+
|
|
76
|
+
Requires Python 3.10+. No API keys needed.
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
pipx install claimcite # or: uv tool install claimcite
|
|
80
|
+
claimcite check article.md # Markdown report
|
|
81
|
+
claimcite check article.md --lang zh # 中文报告
|
|
82
|
+
claimcite check article.md --format json # for scripts and CI
|
|
83
|
+
cat article.md | claimcite check -
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The exit code is 1 when any citation has an error, so it works as a CI gate (`--fail-on warning|never` to tune).
|
|
87
|
+
Set `NCBI_API_KEY` to raise PubMed's rate limit.
|
|
88
|
+
|
|
89
|
+
### Lookup cache
|
|
90
|
+
|
|
91
|
+
Successful registry responses are cached on disk for **30 days** (default `~/.cache/claimcite/v1/`,
|
|
92
|
+
or `$XDG_CACHE_HOME/claimcite/v1/`), so re-running after a one-line edit does not repeat every
|
|
93
|
+
request. Lookup failures are never cached. Use `--no-cache` to bypass the cache for one run.
|
|
94
|
+
|
|
95
|
+
Web pages are fetched at most once a second per site, so the first run over a document with hundreds of
|
|
96
|
+
links takes minutes. Use `--no-web` to check only DOIs and PMIDs. Quotes in PDFs are not checked yet.
|
|
97
|
+
|
|
98
|
+
### In CI: GitHub Action
|
|
99
|
+
|
|
100
|
+
```yaml
|
|
101
|
+
# .github/workflows/citations.yml
|
|
102
|
+
on: [push, pull_request]
|
|
103
|
+
jobs:
|
|
104
|
+
claimcite:
|
|
105
|
+
runs-on: ubuntu-latest
|
|
106
|
+
steps:
|
|
107
|
+
- uses: actions/checkout@v4
|
|
108
|
+
- uses: wasabihu/claimcite@v0.1.0
|
|
109
|
+
with:
|
|
110
|
+
files: "docs/**/*.md" # default: every Markdown file
|
|
111
|
+
fail-on: error # or: warning, never
|
|
112
|
+
web: "true" # "false" checks DOIs and PMIDs only
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The report also appears in the job summary. Runs on Linux and macOS runners.
|
|
116
|
+
|
|
117
|
+
### Before each commit: pre-commit
|
|
118
|
+
|
|
119
|
+
```yaml
|
|
120
|
+
# .pre-commit-config.yaml
|
|
121
|
+
repos:
|
|
122
|
+
- repo: https://github.com/wasabihu/claimcite
|
|
123
|
+
rev: v0.1.0
|
|
124
|
+
hooks:
|
|
125
|
+
- id: claimcite
|
|
126
|
+
# args: [--no-web] # faster: skip ordinary web links
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Every changed Markdown file is checked; the commit stops when a citation has an error. You can also pass
|
|
130
|
+
several files on the command line: `claimcite check docs/*.md`.
|
|
131
|
+
|
|
132
|
+
## Roadmap
|
|
133
|
+
|
|
134
|
+
Each item is a self-contained piece of work. Pick one and open an issue to say you're on it.
|
|
135
|
+
|
|
136
|
+
- [x] **Web links**: check that the page is alive and contains the quoted text
|
|
137
|
+
- [ ] **More registries**: OpenAlex, Semantic Scholar, arXiv, CNKI/Wanfang metadata, DataCite
|
|
138
|
+
- [ ] **Retractions**: flag retracted papers (Crossref / Retraction Watch data)
|
|
139
|
+
- [ ] **Full text**: look for quotes in open-access full text (PubMed Central, Europe PMC), not only abstracts
|
|
140
|
+
- [ ] **Support check**: does the source support the sentence? Pluggable LLM judge, local models first
|
|
141
|
+
- [ ] **Benchmark**: a public, labelled set of real and fabricated citations to measure every check
|
|
142
|
+
- [x] **Numbered citations**: `[1]`, `[1-3]` and footnotes `[^1]` linked to the reference list
|
|
143
|
+
- [ ] **More input formats**: BibTeX, Word / PDF input
|
|
144
|
+
- [x] **Integrations**: GitHub Action, pre-commit hook
|
|
145
|
+
- [ ] Browser extension
|
|
146
|
+
- [x] **Caching** between runs (30-day on-disk cache, `--no-cache` to bypass)
|
|
147
|
+
- [ ] Faster batch lookups
|
|
148
|
+
|
|
149
|
+
## Contributing
|
|
150
|
+
|
|
151
|
+
Contributions of every size are welcome, including AI-assisted ones. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
152
|
+
|
|
153
|
+
## License
|
|
154
|
+
|
|
155
|
+
MIT
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# claimcite
|
|
2
|
+
|
|
3
|
+
**English** · [中文](README.zh-CN.md)
|
|
4
|
+
|
|
5
|
+
Check that every citation in a document is real, points to the paper the text says it does, and actually contains the words quoted from it.
|
|
6
|
+
|
|
7
|
+
AI-written articles, health advice and even papers increasingly cite sources that don't exist, or attach a real DOI to the wrong paper. Readers can't check dozens of references by hand. claimcite does it in one command.
|
|
8
|
+
|
|
9
|
+
```console
|
|
10
|
+
$ claimcite check examples/sample.md
|
|
11
|
+
# Citation check
|
|
12
|
+
|
|
13
|
+
1 ok · 2 warnings · 2 errors · 0 not checked
|
|
14
|
+
|
|
15
|
+
- **Error** · line 7 · `doi:10.1016/s0140-6736(10)61388-8`
|
|
16
|
+
- Title does not match this identifier. Registry title: Worldwide burden of disease from
|
|
17
|
+
exposure to second-hand smoke: a retrospective analysis of data from 192 countries.
|
|
18
|
+
- Year differs: document 2021, registry 2011
|
|
19
|
+
- **Error** · line 9 · `doi:10.1001/jama.2022.99999`
|
|
20
|
+
- Identifier does not exist
|
|
21
|
+
- **Warning** · line 11 · `pmid:33237064`
|
|
22
|
+
- Study type (as tagged by PubMed): Systematic review / meta-analysis
|
|
23
|
+
- Quoted text not found in the abstract (it may be in the full text): “calcium and vitamin D
|
|
24
|
+
supplementation cured osteoporosis in all women”
|
|
25
|
+
...
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## What it checks today
|
|
29
|
+
|
|
30
|
+
| Check | How |
|
|
31
|
+
|---|---|
|
|
32
|
+
| The DOI / PMID exists | doi.org handle API (every registrar), PubMed |
|
|
33
|
+
| The title written in the text matches the identifier | PubMed, then Crossref; fuzzy match |
|
|
34
|
+
| The year matches (±1 for online-first vs print) | same |
|
|
35
|
+
| Text in quotes (`"…"`, `“…”`, `「…」`) appears in the abstract | PubMed / Crossref abstract |
|
|
36
|
+
| Study type: meta-analysis, RCT, observational, animal… | PubMed publication types and MeSH |
|
|
37
|
+
| A web link is alive and contains the quoted text | fetch the page (respecting `robots.txt`), strip HTML, normalise whitespace |
|
|
38
|
+
|
|
39
|
+
It reads Markdown or plain text and understands the common
|
|
40
|
+
`Author (Year). Title. Journal: "quote". https://doi.org/…` pattern, plus `doi:…`, bare DOIs,
|
|
41
|
+
`PMID: …` and PubMed links. Numbered citations work too: a quote in the text followed by `[1]`, `[^1]`,
|
|
42
|
+
`[1, 2]` or `[1-3]` is checked against the matching entry in the reference list (`[1] …`, `1. …` or
|
|
43
|
+
`[^1]: …`). Code blocks and inline code are skipped. Other web links are fetched (respecting `robots.txt`) and checked for
|
|
44
|
+
availability and for the quoted text. A dead link or a missing quote is a **warning**, never an error —
|
|
45
|
+
pages change and some sites block bots. When a page cannot be read the link stays *not checked*.
|
|
46
|
+
|
|
47
|
+
**What it does not do (yet):** decide whether a source *supports* a claim. That needs a language model and a
|
|
48
|
+
labelled test set. See the roadmap. claimcite only confirms that sources exist and match what the text says
|
|
49
|
+
about them. It is not medical advice.
|
|
50
|
+
|
|
51
|
+
## Install and run
|
|
52
|
+
|
|
53
|
+
Requires Python 3.10+. No API keys needed.
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
pipx install claimcite # or: uv tool install claimcite
|
|
57
|
+
claimcite check article.md # Markdown report
|
|
58
|
+
claimcite check article.md --lang zh # 中文报告
|
|
59
|
+
claimcite check article.md --format json # for scripts and CI
|
|
60
|
+
cat article.md | claimcite check -
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The exit code is 1 when any citation has an error, so it works as a CI gate (`--fail-on warning|never` to tune).
|
|
64
|
+
Set `NCBI_API_KEY` to raise PubMed's rate limit.
|
|
65
|
+
|
|
66
|
+
### Lookup cache
|
|
67
|
+
|
|
68
|
+
Successful registry responses are cached on disk for **30 days** (default `~/.cache/claimcite/v1/`,
|
|
69
|
+
or `$XDG_CACHE_HOME/claimcite/v1/`), so re-running after a one-line edit does not repeat every
|
|
70
|
+
request. Lookup failures are never cached. Use `--no-cache` to bypass the cache for one run.
|
|
71
|
+
|
|
72
|
+
Web pages are fetched at most once a second per site, so the first run over a document with hundreds of
|
|
73
|
+
links takes minutes. Use `--no-web` to check only DOIs and PMIDs. Quotes in PDFs are not checked yet.
|
|
74
|
+
|
|
75
|
+
### In CI: GitHub Action
|
|
76
|
+
|
|
77
|
+
```yaml
|
|
78
|
+
# .github/workflows/citations.yml
|
|
79
|
+
on: [push, pull_request]
|
|
80
|
+
jobs:
|
|
81
|
+
claimcite:
|
|
82
|
+
runs-on: ubuntu-latest
|
|
83
|
+
steps:
|
|
84
|
+
- uses: actions/checkout@v4
|
|
85
|
+
- uses: wasabihu/claimcite@v0.1.0
|
|
86
|
+
with:
|
|
87
|
+
files: "docs/**/*.md" # default: every Markdown file
|
|
88
|
+
fail-on: error # or: warning, never
|
|
89
|
+
web: "true" # "false" checks DOIs and PMIDs only
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
The report also appears in the job summary. Runs on Linux and macOS runners.
|
|
93
|
+
|
|
94
|
+
### Before each commit: pre-commit
|
|
95
|
+
|
|
96
|
+
```yaml
|
|
97
|
+
# .pre-commit-config.yaml
|
|
98
|
+
repos:
|
|
99
|
+
- repo: https://github.com/wasabihu/claimcite
|
|
100
|
+
rev: v0.1.0
|
|
101
|
+
hooks:
|
|
102
|
+
- id: claimcite
|
|
103
|
+
# args: [--no-web] # faster: skip ordinary web links
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Every changed Markdown file is checked; the commit stops when a citation has an error. You can also pass
|
|
107
|
+
several files on the command line: `claimcite check docs/*.md`.
|
|
108
|
+
|
|
109
|
+
## Roadmap
|
|
110
|
+
|
|
111
|
+
Each item is a self-contained piece of work. Pick one and open an issue to say you're on it.
|
|
112
|
+
|
|
113
|
+
- [x] **Web links**: check that the page is alive and contains the quoted text
|
|
114
|
+
- [ ] **More registries**: OpenAlex, Semantic Scholar, arXiv, CNKI/Wanfang metadata, DataCite
|
|
115
|
+
- [ ] **Retractions**: flag retracted papers (Crossref / Retraction Watch data)
|
|
116
|
+
- [ ] **Full text**: look for quotes in open-access full text (PubMed Central, Europe PMC), not only abstracts
|
|
117
|
+
- [ ] **Support check**: does the source support the sentence? Pluggable LLM judge, local models first
|
|
118
|
+
- [ ] **Benchmark**: a public, labelled set of real and fabricated citations to measure every check
|
|
119
|
+
- [x] **Numbered citations**: `[1]`, `[1-3]` and footnotes `[^1]` linked to the reference list
|
|
120
|
+
- [ ] **More input formats**: BibTeX, Word / PDF input
|
|
121
|
+
- [x] **Integrations**: GitHub Action, pre-commit hook
|
|
122
|
+
- [ ] Browser extension
|
|
123
|
+
- [x] **Caching** between runs (30-day on-disk cache, `--no-cache` to bypass)
|
|
124
|
+
- [ ] Faster batch lookups
|
|
125
|
+
|
|
126
|
+
## Contributing
|
|
127
|
+
|
|
128
|
+
Contributions of every size are welcome, including AI-assisted ones. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
129
|
+
|
|
130
|
+
## License
|
|
131
|
+
|
|
132
|
+
MIT
|