zotkit 0.4.0__tar.gz → 0.4.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: zotkit
3
- Version: 0.4.0
3
+ Version: 0.4.2
4
4
  Summary: Headless Zotero library management: Web API CRUD plus direct WebDAV attachment upload/download — no desktop app required
5
5
  Author: Shawn
6
6
  License-Expression: MIT
@@ -104,6 +104,8 @@ zotkit find --tag status:to-read
104
104
 
105
105
  zotkit create --arxiv 2401.12345 # fetch arXiv metadata, dry-run preview
106
106
  zotkit create --arxiv 2401.12345 --apply --tags field:ai # create + download & attach the PDF
107
+ zotkit create --arxiv 2401.12345 1706.03762 math/0211159 --apply # batch: one metadata
108
+ # request, PDFs politely spaced ≥3 s apart
107
109
  zotkit create --doi 10.1038/nature14539 # fetch CrossRef metadata by DOI (no PDF —
108
110
  # usually paywalled; attach manually after)
109
111
 
@@ -122,12 +124,27 @@ zotkit backup # full JSON snapshot -> backups/
122
124
  zotkit lint field:physics topic:new-idea # offline tag check
123
125
  ```
124
126
 
125
- `--arxiv` takes an id or an abs/pdf URL and maps the full record (all authors,
126
- abstract, date, DOI, `preprint` item type); `--doi` maps CrossRef records
127
- (journal articles, conference papers, books, chapters, …) and refuses to guess
128
- on CrossRef types it doesn't know. Both accept `--collection` and `--tags`, and
129
- `--no-pdf` skips the arXiv PDF. Fetching metadata from arbitrary web pages is
130
- out of scope (zotkit stays a daemon-free CLI — no translation-server).
127
+ `--arxiv` takes ids or abs/pdf URLs (several, space- or comma-separated) and maps
128
+ the full record (all authors, abstract, date, DOI); `--doi`
129
+ maps CrossRef records (journal articles, conference papers, books, chapters, …)
130
+ and refuses to guess on CrossRef types it doesn't know.
131
+
132
+ **Version of record**: when arXiv reports a *journal* DOI (the paper was formally
133
+ published — as opposed to arXiv's own `10.48550/*` DataCite DOI), `--arxiv`
134
+ builds the journal record from CrossRef instead of a `preprint`: proper item
135
+ type, venue, volume/pages, formal date. The arXiv identity is kept — `arXiv: <id>`
136
+ goes in Extra, the `url` stays the open-access abs page (the journal link lives in
137
+ the DOI field), the arXiv abstract fills in when CrossRef has none, and the PDF
138
+ still comes from arXiv. If the CrossRef lookup fails, the item falls back to the
139
+ preprint record with a warning rather than failing. Items whose abstract zotkit
140
+ wrote also carry an `abstract-source: arxiv|crossref` line in Extra, naming where
141
+ it actually came from. Both accept `--collection`
142
+ and `--tags`, and `--no-pdf` skips the arXiv PDFs. Rate limiting is built into the
143
+ request layer — batches use one arXiv metadata request and space PDF downloads
144
+ per arXiv's terms of use, so callers (humans or agents) never pace themselves. A
145
+ bad id fails alone, not the batch; the exit code is non-zero only if something
146
+ failed. Fetching metadata from arbitrary web pages is out of scope (zotkit stays
147
+ a daemon-free CLI — no translation-server).
131
148
 
132
149
  Item JSON for `zotkit create --file` (a list, one object per reference):
133
150
 
@@ -84,6 +84,8 @@ zotkit find --tag status:to-read
84
84
 
85
85
  zotkit create --arxiv 2401.12345 # fetch arXiv metadata, dry-run preview
86
86
  zotkit create --arxiv 2401.12345 --apply --tags field:ai # create + download & attach the PDF
87
+ zotkit create --arxiv 2401.12345 1706.03762 math/0211159 --apply # batch: one metadata
88
+ # request, PDFs politely spaced ≥3 s apart
87
89
  zotkit create --doi 10.1038/nature14539 # fetch CrossRef metadata by DOI (no PDF —
88
90
  # usually paywalled; attach manually after)
89
91
 
@@ -102,12 +104,27 @@ zotkit backup # full JSON snapshot -> backups/
102
104
  zotkit lint field:physics topic:new-idea # offline tag check
103
105
  ```
104
106
 
105
- `--arxiv` takes an id or an abs/pdf URL and maps the full record (all authors,
106
- abstract, date, DOI, `preprint` item type); `--doi` maps CrossRef records
107
- (journal articles, conference papers, books, chapters, …) and refuses to guess
108
- on CrossRef types it doesn't know. Both accept `--collection` and `--tags`, and
109
- `--no-pdf` skips the arXiv PDF. Fetching metadata from arbitrary web pages is
110
- out of scope (zotkit stays a daemon-free CLI — no translation-server).
107
+ `--arxiv` takes ids or abs/pdf URLs (several, space- or comma-separated) and maps
108
+ the full record (all authors, abstract, date, DOI); `--doi`
109
+ maps CrossRef records (journal articles, conference papers, books, chapters, …)
110
+ and refuses to guess on CrossRef types it doesn't know.
111
+
112
+ **Version of record**: when arXiv reports a *journal* DOI (the paper was formally
113
+ published — as opposed to arXiv's own `10.48550/*` DataCite DOI), `--arxiv`
114
+ builds the journal record from CrossRef instead of a `preprint`: proper item
115
+ type, venue, volume/pages, formal date. The arXiv identity is kept — `arXiv: <id>`
116
+ goes in Extra, the `url` stays the open-access abs page (the journal link lives in
117
+ the DOI field), the arXiv abstract fills in when CrossRef has none, and the PDF
118
+ still comes from arXiv. If the CrossRef lookup fails, the item falls back to the
119
+ preprint record with a warning rather than failing. Items whose abstract zotkit
120
+ wrote also carry an `abstract-source: arxiv|crossref` line in Extra, naming where
121
+ it actually came from. Both accept `--collection`
122
+ and `--tags`, and `--no-pdf` skips the arXiv PDFs. Rate limiting is built into the
123
+ request layer — batches use one arXiv metadata request and space PDF downloads
124
+ per arXiv's terms of use, so callers (humans or agents) never pace themselves. A
125
+ bad id fails alone, not the batch; the exit code is non-zero only if something
126
+ failed. Fetching metadata from arbitrary web pages is out of scope (zotkit stays
127
+ a daemon-free CLI — no translation-server).
111
128
 
112
129
  Item JSON for `zotkit create --file` (a list, one object per reference):
113
130
 
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "zotkit"
7
- version = "0.4.0"
7
+ version = "0.4.2"
8
8
  description = "Headless Zotero library management: Web API CRUD plus direct WebDAV attachment upload/download — no desktop app required"
9
9
  readme = "README.md"
10
10
  license = "MIT"
@@ -2,4 +2,4 @@
2
2
  from .core import Conventions, TagConventionError, Zot, lint_tags, load_conventions
3
3
 
4
4
  __all__ = ["Zot", "lint_tags", "load_conventions", "Conventions", "TagConventionError"]
5
- __version__ = "0.4.0"
5
+ __version__ = "0.4.2"
@@ -81,11 +81,13 @@ def main(argv=None):
81
81
  "or a DOI (dry unless --apply)")
82
82
  src = p.add_mutually_exclusive_group(required=True)
83
83
  src.add_argument("--file")
84
- src.add_argument("--arxiv", metavar="ID_OR_URL",
85
- help="arXiv id (2401.12345) or abs/pdf URL; fetches metadata, "
86
- "and with --apply also downloads + attaches the PDF")
87
- src.add_argument("--doi", metavar="DOI",
88
- help="DOI (or doi.org URL); fetches metadata from CrossRef")
84
+ src.add_argument("--arxiv", metavar="ID_OR_URL", nargs="+",
85
+ help="arXiv ids (2401.12345) or abs/pdf URLs — several allowed, "
86
+ "space- or comma-separated; fetches metadata in one batched "
87
+ "request, and with --apply also downloads + attaches the PDFs")
88
+ src.add_argument("--doi", metavar="DOI", nargs="+",
89
+ help="DOIs (or doi.org URLs) — several allowed, space- or "
90
+ "comma-separated; fetches metadata from CrossRef")
89
91
  p.add_argument("--apply", action="store_true")
90
92
  p.add_argument("--no-dedup", action="store_true")
91
93
  p.add_argument("--loose-tags", action="store_true", help="warn instead of error on tag violations")
@@ -141,25 +143,47 @@ def main(argv=None):
141
143
  _print_items(zot.find(a.title, a.tag, a.collection))
142
144
 
143
145
  elif a.cmd == "create":
144
- pdf_url = None
146
+ pdfs: dict[str, tuple[str, str]] = {} # title -> (arXiv id, pdf url)
147
+ failures: list[tuple[str, str]] = [] # (identifier, why)
145
148
  if a.file:
146
149
  data = json.load(open(a.file, encoding="utf-8"))
147
150
  items = data["items"] if isinstance(data, dict) else data
148
151
  else:
149
- from .metadata import MetadataError, fetch_arxiv, fetch_doi
152
+ from .metadata import MetadataError, fetch_arxiv_batch, fetch_doi
153
+ wanted = [w.strip() for arg in (a.arxiv or a.doi)
154
+ for w in arg.split(",") if w.strip()]
155
+ items = []
150
156
  try:
151
157
  if a.arxiv:
152
- item, pdf_url = fetch_arxiv(a.arxiv)
158
+ for r in fetch_arxiv_batch(wanted):
159
+ if r.get("error"):
160
+ failures.append((r["id"], r["error"]))
161
+ continue
162
+ if r.get("note"): # version-of-record upgrade happened
163
+ print(r["note"])
164
+ if r.get("warning"):
165
+ print(f"warning: {r['warning']}", file=sys.stderr)
166
+ items.append(r["item"])
167
+ pdfs[r["item"]["title"]] = (r["id"], r["pdf_url"])
153
168
  else:
154
- item = fetch_doi(a.doi)
169
+ for d in wanted:
170
+ try:
171
+ items.append(fetch_doi(d))
172
+ except MetadataError as e:
173
+ failures.append((d, str(e)))
155
174
  except MetadataError as e:
156
175
  print(f"error: {e}", file=sys.stderr)
157
176
  return 1
158
- if a.collection:
159
- item["collection"] = a.collection
160
- if a.tags:
161
- item["tags"] = [t.strip() for t in a.tags.split(",") if t.strip()]
162
- items = [item]
177
+ for ident, why in failures:
178
+ prefix = f"{ident}: " if len(wanted) > 1 else ""
179
+ print(f"error: {prefix}{why}", file=sys.stderr)
180
+ for item in items:
181
+ if a.collection:
182
+ item["collection"] = a.collection
183
+ if a.tags:
184
+ item["tags"] = [t.strip() for t in a.tags.split(",") if t.strip()]
185
+ if not items:
186
+ return 1
163
187
  if not a.apply:
164
188
  for d in items:
165
189
  problems = lint_tags(d.get("tags", []), conventions=zot.conventions,
@@ -171,42 +195,51 @@ def main(argv=None):
171
195
  continue
172
196
  flag = (" !! " + "; ".join(problems)) if problems else ""
173
197
  print(f" [dry] {d.get('collection')} | {d.get('title','')[:60]}{flag}")
174
- print(f"{len(items)} item(s). DRY — add --apply to create.")
175
- return 0
198
+ tail = f", {len(failures)} failed to fetch" if failures else ""
199
+ print(f"{len(items)} item(s){tail}. DRY — add --apply to create.")
200
+ return 1 if failures else 0
176
201
  created = zot.create_items(items, dedup=not a.no_dedup, strict_tags=not a.loose_tags)
177
202
  new = [c for c in created if c.get("key")]
203
+ dup = len(created) - len(new)
178
204
  if a.file:
179
205
  out = Path(a.file).with_suffix(".created.json")
180
206
  json.dump(new, open(out, "w", encoding="utf-8"), ensure_ascii=False, indent=2)
181
- print(f"created {len(new)} (skipped {len(created)-len(new)} dup) -> {out}")
207
+ print(f"created {len(new)} (skipped {dup} dup) -> {out}")
182
208
  if any(c.get("file_path") for c in new):
183
209
  print(f"attach PDFs: zotkit attach --from {out} --all")
184
210
  return 0
185
- # identifier mode: one item
186
- if not new:
211
+ # identifier mode
212
+ if not new and not failures:
187
213
  print("skipped: an item with this DOI/title already exists (use --no-dedup "
188
214
  "to force)")
189
215
  return 0
190
- key = new[0]["key"]
191
- print(f"created {key} {items[0].get('title', '')[:70]}")
192
- if pdf_url and not a.no_pdf:
193
- from .metadata import download_pdf
194
- import tempfile
195
- with tempfile.TemporaryDirectory() as td:
196
- pdf = Path(td) / (items[0]["archiveID"].replace("arXiv:", "arXiv-")
197
- .replace("/", "_") + ".pdf")
198
- try:
199
- download_pdf(pdf_url, pdf)
200
- except MetadataError as e:
201
- print(f"item created, but PDF failed: {e}\n"
202
- f" attach manually: zotkit attach --key {key} --pdf <file>",
203
- file=sys.stderr)
204
- return 1
205
- info = zot.attach(key, pdf)
206
- print(f"attached {info['filename']} ({info['storage']})")
207
- elif a.doi:
216
+ for c in new:
217
+ key = c["key"]
218
+ print(f"created {key} {str(c.get('title', ''))[:70]}")
219
+ aid_pdf = pdfs.get(c.get("title"))
220
+ if aid_pdf and not a.no_pdf:
221
+ from .metadata import download_pdf
222
+ import tempfile
223
+ aid, pdf_url = aid_pdf
224
+ with tempfile.TemporaryDirectory() as td:
225
+ pdf = Path(td) / f"arXiv-{aid.replace('/', '_')}.pdf"
226
+ try:
227
+ download_pdf(pdf_url, pdf) # spaced ≥3 s apart by metadata._get
228
+ except MetadataError as e:
229
+ failures.append((aid, f"item {key} created, but PDF failed: {e}"))
230
+ print(f"error: item {key} created, but PDF failed: {e}\n"
231
+ f" attach manually: zotkit attach --key {key} --pdf <file>",
232
+ file=sys.stderr)
233
+ continue
234
+ info = zot.attach(key, pdf)
235
+ print(f"attached {info['filename']} ({info['storage']})")
236
+ if a.doi and new:
237
+ hint = new[0]["key"] if len(new) == 1 else "<key>"
208
238
  print("no PDF downloaded (publisher PDFs are usually paywalled) — "
209
- f"attach one with: zotkit attach --key {key} --pdf <file>")
239
+ f"attach one with: zotkit attach --key {hint} --pdf <file>")
240
+ if len(wanted) > 1:
241
+ print(f"batch: created {len(new)}, skipped {dup} dup, failed {len(failures)}")
242
+ return 1 if failures else 0
210
243
 
211
244
  elif a.cmd == "attach":
212
245
  if a.key and a.pdf:
@@ -6,6 +6,11 @@ Scope is deliberately narrow: stable JSON/Atom APIs only — no translation-serv
6
6
  no arbitrary-URL scraping (zotkit stays a daemon-free CLI).
7
7
 
8
8
  Abstracts are stored verbatim (line breaks, LaTeX, JATS markup untouched).
9
+
10
+ Rate limiting lives here, in the request layer: every request through `_get`
11
+ waits out the per-host minimum interval (arXiv's terms: 1 request / 3 s on a
12
+ single connection; CrossRef polite pool: 1 s). Callers — CLI users, agents,
13
+ batch loops — never need to pace themselves.
9
14
  """
10
15
  from __future__ import annotations
11
16
 
@@ -56,7 +61,17 @@ class MetadataError(RuntimeError):
56
61
  unknown record, unsupported type, network trouble)."""
57
62
 
58
63
 
64
+ # minimum seconds between requests to the same host (arXiv terms of use: 3 s,
65
+ # covering the export API and PDF downloads; CrossRef polite pool: 1 s)
66
+ _MIN_INTERVAL = {"export.arxiv.org": 3.0, "arxiv.org": 3.0, "api.crossref.org": 1.0}
67
+ _next_ok: dict[str, float] = {}
68
+
69
+
59
70
  def _get(url: str, **kw) -> httpx.Response:
71
+ host = httpx.URL(url).host
72
+ wait = _next_ok.get(host, 0.0) - time.monotonic()
73
+ if wait > 0:
74
+ time.sleep(wait)
60
75
  try:
61
76
  r = httpx.get(url, headers={"User-Agent": USER_AGENT}, timeout=30,
62
77
  follow_redirects=True, **kw)
@@ -67,10 +82,11 @@ def _get(url: str, **kw) -> httpx.Response:
67
82
  follow_redirects=True, **kw)
68
83
  if r.status_code == 429:
69
84
  raise MetadataError(
70
- f"{httpx.URL(url).host} is rate-limiting (HTTP 429) — "
71
- "wait a minute and retry")
85
+ f"{host} is rate-limiting (HTTP 429) — wait a minute and retry")
72
86
  except httpx.HTTPError as e:
73
- raise MetadataError(f"network error talking to {httpx.URL(url).host}: {e}") from e
87
+ raise MetadataError(f"network error talking to {host}: {e}") from e
88
+ finally:
89
+ _next_ok[host] = time.monotonic() + _MIN_INTERVAL.get(host, 0.0)
74
90
  return r
75
91
 
76
92
 
@@ -79,6 +95,11 @@ def _squash(s: str) -> str:
79
95
  return re.sub(r"\s+", " ", s or "").strip()
80
96
 
81
97
 
98
+ def _add_extra(item: dict, line: str) -> None:
99
+ """Append a line to the item's Extra field, never overwriting what's there."""
100
+ item["extra"] = f"{item['extra']}\n{line}" if item.get("extra") else line
101
+
102
+
82
103
  def _person(name: str) -> dict:
83
104
  """Split a display name into first/last on the final space; single-token
84
105
  names go in lastName alone (matches Zotero's single-field mode)."""
@@ -106,29 +127,19 @@ def arxiv_id(id_or_url: str) -> str:
106
127
  return m.group(1)
107
128
 
108
129
 
109
- def fetch_arxiv(id_or_url: str) -> tuple[dict, str]:
110
- """arXiv export API → (item dict, pdf url)."""
111
- aid = arxiv_id(id_or_url)
112
- r = _get("https://export.arxiv.org/api/query", params={"id_list": aid})
113
- if r.status_code != 200:
114
- raise MetadataError(f"arXiv API returned HTTP {r.status_code} for '{aid}'")
115
- try:
116
- feed = ET.fromstring(r.text)
117
- except ET.ParseError as e:
118
- raise MetadataError(f"arXiv API returned unparseable XML: {e}") from e
119
- entry = feed.find(f"{_ATOM}entry")
120
- title = _squash(entry.findtext(f"{_ATOM}title", "")) if entry is not None else ""
121
- if entry is None or not title:
122
- raise MetadataError(f"arXiv has no record for '{aid}' — check the id")
123
- if title == "Error":
124
- raise MetadataError(f"arXiv API error for '{aid}': "
125
- f"{_squash(entry.findtext(f'{_ATOM}summary', ''))}")
126
-
127
- bare = re.sub(r"v\d+$", "", aid)
128
- doi = _squash(entry.findtext(f"{_ARXIV}doi", "")) or f"10.48550/arXiv.{bare}"
130
+ _ARXIV_CHUNK = 50 # ids per id_list request; chunks are spaced out by _get
131
+
132
+
133
+ def _bare(aid: str) -> str:
134
+ return re.sub(r"v\d+$", "", aid, flags=re.IGNORECASE)
135
+
136
+
137
+ def _arxiv_item(entry, aid: str) -> tuple[dict, str]:
138
+ doi = (_squash(entry.findtext(f"{_ARXIV}doi", ""))
139
+ or f"10.48550/arXiv.{_bare(aid)}")
129
140
  item = {
130
141
  "itemType": "preprint",
131
- "title": title,
142
+ "title": _squash(entry.findtext(f"{_ATOM}title", "")),
132
143
  "creators": [_person(n.text or "")
133
144
  for n in entry.findall(f"{_ATOM}author/{_ATOM}name")],
134
145
  "abstractNote": (entry.findtext(f"{_ATOM}summary", "") or "").strip("\n "),
@@ -139,9 +150,99 @@ def fetch_arxiv(id_or_url: str) -> tuple[dict, str]:
139
150
  "archiveID": f"arXiv:{aid}",
140
151
  "libraryCatalog": "arXiv.org",
141
152
  }
153
+ if item["abstractNote"]:
154
+ _add_extra(item, "abstract-source: arxiv")
142
155
  return item, f"https://arxiv.org/pdf/{aid}"
143
156
 
144
157
 
158
+ _DATACITE_PREFIX = "10.48550/" # arXiv's own DOIs; anything else is a journal DOI
159
+
160
+
161
+ def _journal_record(pre: dict, aid: str) -> dict:
162
+ """Version-of-record upgrade: the arXiv record carries a journal DOI, so
163
+ build the item from CrossRef instead (venue/volume/pages/formal date),
164
+ keeping the arXiv identity: abstract fallback when CrossRef has none,
165
+ `arXiv: <id>` in Extra, and the open-access abs page as url (the journal
166
+ link is carried by the DOI field). Raises MetadataError on lookup failure —
167
+ the caller falls back to the preprint record."""
168
+ j = fetch_doi(pre["DOI"])
169
+ if not j.get("abstractNote") and pre.get("abstractNote"):
170
+ j["abstractNote"] = pre["abstractNote"]
171
+ _add_extra(j, "abstract-source: arxiv")
172
+ j["url"] = pre["url"]
173
+ _add_extra(j, f"arXiv: {aid}")
174
+ return j
175
+
176
+
177
+ def fetch_arxiv_batch(ids_or_urls: list[str]) -> list[dict]:
178
+ """arXiv export API, one id_list request per ≤50 ids → results in input
179
+ order, each {"id", "item", "pdf_url"} or {"id", "error"}. A bad id never
180
+ fails the batch; only transport-level trouble raises MetadataError.
181
+
182
+ Version of record: when arXiv reports a journal DOI (not its own
183
+ 10.48550/* DataCite DOI) the item is rebuilt from CrossRef ("note" key
184
+ says so); if that lookup fails the preprint record stands ("warning" key).
185
+
186
+ The response can't be trusted positionally: entries come back in arbitrary
187
+ order (and with resolved version numbers), so they are mapped back to the
188
+ requested ids by version-stripped id.
189
+ """
190
+ results: list[dict] = []
191
+ valid: list[dict] = [] # results entries still awaiting an API answer
192
+ for s in ids_or_urls:
193
+ try:
194
+ results.append({"id": arxiv_id(s)})
195
+ valid.append(results[-1])
196
+ except MetadataError as e:
197
+ results.append({"id": s, "error": str(e)})
198
+
199
+ for i in range(0, len(valid), _ARXIV_CHUNK):
200
+ chunk = valid[i:i + _ARXIV_CHUNK]
201
+ # max_results defaults to 10 — without it, batches >10 silently truncate
202
+ r = _get("https://export.arxiv.org/api/query",
203
+ params={"id_list": ",".join(c["id"] for c in chunk),
204
+ "max_results": len(chunk)})
205
+ if r.status_code != 200:
206
+ raise MetadataError(f"arXiv API returned HTTP {r.status_code}")
207
+ try:
208
+ feed = ET.fromstring(r.text)
209
+ except ET.ParseError as e:
210
+ raise MetadataError(f"arXiv API returned unparseable XML: {e}") from e
211
+ emap = {}
212
+ for entry in feed.findall(f"{_ATOM}entry"):
213
+ title = _squash(entry.findtext(f"{_ATOM}title", ""))
214
+ if not title or title == "Error": # error/placeholder entries: unmappable
215
+ continue
216
+ try:
217
+ emap[_bare(arxiv_id(entry.findtext(f"{_ATOM}id", "")))] = entry
218
+ except MetadataError:
219
+ continue
220
+ for c in chunk:
221
+ entry = emap.get(_bare(c["id"]))
222
+ if entry is None:
223
+ c["error"] = f"arXiv has no record for '{c['id']}' — check the id"
224
+ continue
225
+ c["item"], c["pdf_url"] = _arxiv_item(entry, c["id"])
226
+ doi = c["item"]["DOI"]
227
+ if not doi.startswith(_DATACITE_PREFIX):
228
+ try:
229
+ c["item"] = _journal_record(c["item"], c["id"])
230
+ c["note"] = (f"{c['id']} → journal DOI {doi}, "
231
+ "building journal record")
232
+ except MetadataError as e:
233
+ c["warning"] = (f"{c['id']}: journal DOI {doi} lookup failed "
234
+ f"({e}) — keeping preprint record")
235
+ return results
236
+
237
+
238
+ def fetch_arxiv(id_or_url: str) -> tuple[dict, str]:
239
+ """arXiv export API → (item dict, pdf url); raises on any failure."""
240
+ res = fetch_arxiv_batch([id_or_url])[0]
241
+ if res.get("error"):
242
+ raise MetadataError(res["error"])
243
+ return res["item"], res["pdf_url"]
244
+
245
+
145
246
  def download_pdf(url: str, dest) -> None:
146
247
  """Download a PDF to `dest`, refusing anything that isn't actually a PDF."""
147
248
  r = _get(url)
@@ -206,7 +307,7 @@ def fetch_doi(doi: str) -> dict:
206
307
  "url": msg.get("URL") or f"https://doi.org/{doi}",
207
308
  "volume": msg.get("volume", ""),
208
309
  "issue": msg.get("issue", ""),
209
- "pages": msg.get("page", ""),
310
+ "pages": msg.get("page") or msg.get("article-number", ""),
210
311
  "publisher": _squash(msg.get("publisher", "")),
211
312
  "language": msg.get("language", ""),
212
313
  "ISSN": (msg.get("ISSN") or [""])[0],
@@ -220,4 +321,7 @@ def fetch_doi(doi: str) -> dict:
220
321
  if not item["title"]:
221
322
  raise MetadataError(f"CrossRef record for '{doi}' has no title — refusing to "
222
323
  "create an empty item")
223
- return {k: v for k, v in item.items() if v not in ("", [])}
324
+ item = {k: v for k, v in item.items() if v not in ("", [])}
325
+ if item.get("abstractNote"):
326
+ _add_extra(item, "abstract-source: crossref")
327
+ return item
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: zotkit
3
- Version: 0.4.0
3
+ Version: 0.4.2
4
4
  Summary: Headless Zotero library management: Web API CRUD plus direct WebDAV attachment upload/download — no desktop app required
5
5
  Author: Shawn
6
6
  License-Expression: MIT
@@ -104,6 +104,8 @@ zotkit find --tag status:to-read
104
104
 
105
105
  zotkit create --arxiv 2401.12345 # fetch arXiv metadata, dry-run preview
106
106
  zotkit create --arxiv 2401.12345 --apply --tags field:ai # create + download & attach the PDF
107
+ zotkit create --arxiv 2401.12345 1706.03762 math/0211159 --apply # batch: one metadata
108
+ # request, PDFs politely spaced ≥3 s apart
107
109
  zotkit create --doi 10.1038/nature14539 # fetch CrossRef metadata by DOI (no PDF —
108
110
  # usually paywalled; attach manually after)
109
111
 
@@ -122,12 +124,27 @@ zotkit backup # full JSON snapshot -> backups/
122
124
  zotkit lint field:physics topic:new-idea # offline tag check
123
125
  ```
124
126
 
125
- `--arxiv` takes an id or an abs/pdf URL and maps the full record (all authors,
126
- abstract, date, DOI, `preprint` item type); `--doi` maps CrossRef records
127
- (journal articles, conference papers, books, chapters, …) and refuses to guess
128
- on CrossRef types it doesn't know. Both accept `--collection` and `--tags`, and
129
- `--no-pdf` skips the arXiv PDF. Fetching metadata from arbitrary web pages is
130
- out of scope (zotkit stays a daemon-free CLI — no translation-server).
127
+ `--arxiv` takes ids or abs/pdf URLs (several, space- or comma-separated) and maps
128
+ the full record (all authors, abstract, date, DOI); `--doi`
129
+ maps CrossRef records (journal articles, conference papers, books, chapters, …)
130
+ and refuses to guess on CrossRef types it doesn't know.
131
+
132
+ **Version of record**: when arXiv reports a *journal* DOI (the paper was formally
133
+ published — as opposed to arXiv's own `10.48550/*` DataCite DOI), `--arxiv`
134
+ builds the journal record from CrossRef instead of a `preprint`: proper item
135
+ type, venue, volume/pages, formal date. The arXiv identity is kept — `arXiv: <id>`
136
+ goes in Extra, the `url` stays the open-access abs page (the journal link lives in
137
+ the DOI field), the arXiv abstract fills in when CrossRef has none, and the PDF
138
+ still comes from arXiv. If the CrossRef lookup fails, the item falls back to the
139
+ preprint record with a warning rather than failing. Items whose abstract zotkit
140
+ wrote also carry an `abstract-source: arxiv|crossref` line in Extra, naming where
141
+ it actually came from. Both accept `--collection`
142
+ and `--tags`, and `--no-pdf` skips the arXiv PDFs. Rate limiting is built into the
143
+ request layer — batches use one arXiv metadata request and space PDF downloads
144
+ per arXiv's terms of use, so callers (humans or agents) never pace themselves. A
145
+ bad id fails alone, not the batch; the exit code is non-zero only if something
146
+ failed. Fetching metadata from arbitrary web pages is out of scope (zotkit stays
147
+ a daemon-free CLI — no translation-server).
131
148
 
132
149
  Item JSON for `zotkit create --file` (a list, one object per reference):
133
150
 
File without changes
File without changes
File without changes