fetchpdf 0.1.3__tar.gz → 0.1.4__tar.gz

{fetchpdf-0.1.3/fetchpdf.egg-info → fetchpdf-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fetchpdf
-Version: 0.1.3
+Version: 0.1.4
 Summary: A comprehensive tool to download PDFs from DOIs using multiple sources
 Author-email: Dan Elton <delton17@gmail.com>
 License-Expression: MIT
@@ -56,7 +56,7 @@ A comprehensive Python package to download academic papers (PDFs) from DOIs (or
 - [Supplementary Material](#supplementary-material)
 - [`--pull-everything`: the second layer](#--pull-everything-the-second-layer)
 - [Figure Images](#figure-images)
-- [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions) — papers your library subscribes to (`fetchpdf cookies setup`)
+- [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions) — papers your library subscribes to (`fetchpdf fetchcookies`)
 - [Calling fetchpdf from a script or an agent](#calling-fetchpdf-from-a-script-or-an-agent) — `--json`
 - [API Reference](#api-reference)
 - [Download Sources](#download-sources-in-order-of-priority)
@@ -85,7 +85,7 @@ A comprehensive Python package to download academic papers (PDFs) from DOIs (or
 - 🖨️ **[XML to HTML / PDF](#rendering-the-xml-as-html-or-pdf)**: `--generate-html-from-xml` / `--generate-pdf-from-xml` render a retrieved XML as one self-contained page, figures and tables embedded
 - 🔗 **[Linked Datasets & Code](#linked-datasets-and-code-stem_linked_artifactsjson)**: the same pass discovers each paper's external datasets/software via ScholeXplorer, Europe PMC and DataCite, downloads ownership-confirmed deposits from figshare/Zenodo/OSF/Dryad/Dataverse, and records every link in a sidecar
 - 🤖 **[LLM-assisted retrieval](#--pull-everything-the-second-layer)**: `--pull-everything` adds a second layer — a model reads the paper itself and goes after the SI and datasets the APIs missed
-- 🔐 **[Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions)** (opt-in): a one-time `fetchpdf cookies setup` lets fetchpdf download papers your library subscribes to
+- 🔐 **[Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions)** (opt-in): a one-time `fetchpdf fetchcookies` lets fetchpdf download papers your library subscribes to
 - 🚀 **Batch Processing**: Process multiple DOIs/PMIDs from CSV files or lists
 - ⚡ **Parallel Execution**: Download multiple papers simultaneously with configurable workers
 
@@ -184,7 +184,7 @@ fetchpdf '10.1002/(SICI)1521-3773(19981116)37:21<2922::AID-ANIE2922>3.0.CO;2-1,
 #   ...but single-quote any list holding ( ) < > ; -- the shell would read them as syntax
 ```
 
-**At a university?** Run `fetchpdf cookies setup` once and fetchpdf will also fetch
+**At a university?** Run `fetchpdf fetchcookies` once and fetchpdf will also fetch
 papers your library subscribes to. See
 [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions).
 
@@ -1005,11 +1005,17 @@ fetchpdf-render ./out --html --pdf --fetch-figures   # XML you already have
 
 **A rendering, not the publisher's PDF**, and never written as `{stem}.pdf`:
 that name means the typeset article to the skip check and the validators. The
-page opens with a banner naming the source file, the DOI, and how many figures
-were embedded. A figure whose image could not be had keeps a visible
+page opens with a banner naming the source file, the fetchpdf version and the
+DOI, plus a note for anything that could not be shown: figures without an image,
+tables published only as an image that is missing, or no reference list. A figure whose image could not be had keeps a visible
 placeholder, so an incomplete rendering looks incomplete. The prose and
-tables come from the same walker as `--to-markdown`, so the two agree;
-references are omitted and formulas stay as text.
+tables come from the same walker as `--to-markdown`, so the two agree, except
+that the page ends with the reference list (with DOI links), which the Markdown
+leaves out. Formulas stay as text. The XML never says how large a figure was
+typeset, so each is given about the same area (a square figure ~10.5 cm across),
+shaped by its aspect ratio and capped at the column width. Under the title come
+the authors, their affiliations and the journal citation (volume, issue, pages).
+Tables always fit the page width in the PDF; very wide ones print smaller.
 
 Only records whose XML was kept are rendered — add `--get-xml-or-html` to keep
 the XML beside the PDF.
@@ -1026,7 +1032,7 @@ library, for the papers that have no open-access copy.
 Browser cookie support is included in the standard fetchpdf installation.
 
 ```bash
-fetchpdf cookies setup
+fetchpdf fetchcookies
 ```
 
 The wizard asks two questions, then opens some tabs:
@@ -1058,7 +1064,7 @@ session exists but the test article was refused: the session lapsed, or your
 library does not take that particular journal. Access is turned on only if at
 least one publisher works.
 
-**Cookies often expire within hours**. You likely will have to rerun `fetchpdf cookies setup` periodically to restore access.
+**Cookies often expire within hours**. You likely will have to rerun `fetchpdf fetchcookies` periodically to restore access.
 
 ### Living with it
 
@@ -1092,7 +1098,7 @@ run. This works the same for a single DOI and for a batch:
 ```
 
 Runs that cannot be asked (pipes, cron, `--json`) print the command instead
-(`fetchpdf cookies setup --publishers wiley,tandf`, or `refresh --publishers ...`).
+(`fetchpdf fetchcookies --publishers wiley,tandf`, or `refresh --publishers ...`).
 `--no-cookies` and `--cookies FILE` turn the offer off. Royal Society papers are
 not offered, because there is no test article to sign in on.
 

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/README.md

@@ -17,7 +17,7 @@ A comprehensive Python package to download academic papers (PDFs) from DOIs (or
 - [Supplementary Material](#supplementary-material)
 - [`--pull-everything`: the second layer](#--pull-everything-the-second-layer)
 - [Figure Images](#figure-images)
-- [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions) — papers your library subscribes to (`fetchpdf cookies setup`)
+- [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions) — papers your library subscribes to (`fetchpdf fetchcookies`)
 - [Calling fetchpdf from a script or an agent](#calling-fetchpdf-from-a-script-or-an-agent) — `--json`
 - [API Reference](#api-reference)
 - [Download Sources](#download-sources-in-order-of-priority)
@@ -46,7 +46,7 @@ A comprehensive Python package to download academic papers (PDFs) from DOIs (or
 - 🖨️ **[XML to HTML / PDF](#rendering-the-xml-as-html-or-pdf)**: `--generate-html-from-xml` / `--generate-pdf-from-xml` render a retrieved XML as one self-contained page, figures and tables embedded
 - 🔗 **[Linked Datasets & Code](#linked-datasets-and-code-stem_linked_artifactsjson)**: the same pass discovers each paper's external datasets/software via ScholeXplorer, Europe PMC and DataCite, downloads ownership-confirmed deposits from figshare/Zenodo/OSF/Dryad/Dataverse, and records every link in a sidecar
 - 🤖 **[LLM-assisted retrieval](#--pull-everything-the-second-layer)**: `--pull-everything` adds a second layer — a model reads the paper itself and goes after the SI and datasets the APIs missed
-- 🔐 **[Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions)** (opt-in): a one-time `fetchpdf cookies setup` lets fetchpdf download papers your library subscribes to
+- 🔐 **[Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions)** (opt-in): a one-time `fetchpdf fetchcookies` lets fetchpdf download papers your library subscribes to
 - 🚀 **Batch Processing**: Process multiple DOIs/PMIDs from CSV files or lists
 - ⚡ **Parallel Execution**: Download multiple papers simultaneously with configurable workers
 
@@ -145,7 +145,7 @@ fetchpdf '10.1002/(SICI)1521-3773(19981116)37:21<2922::AID-ANIE2922>3.0.CO;2-1,
 #   ...but single-quote any list holding ( ) < > ; -- the shell would read them as syntax
 ```
 
-**At a university?** Run `fetchpdf cookies setup` once and fetchpdf will also fetch
+**At a university?** Run `fetchpdf fetchcookies` once and fetchpdf will also fetch
 papers your library subscribes to. See
 [Institutional access](#institutional-access-via-cookies-ie-using-your-librarys-subscriptions).
 
@@ -966,11 +966,17 @@ fetchpdf-render ./out --html --pdf --fetch-figures   # XML you already have
 
 **A rendering, not the publisher's PDF**, and never written as `{stem}.pdf`:
 that name means the typeset article to the skip check and the validators. The
-page opens with a banner naming the source file, the DOI, and how many figures
-were embedded. A figure whose image could not be had keeps a visible
+page opens with a banner naming the source file, the fetchpdf version and the
+DOI, plus a note for anything that could not be shown: figures without an image,
+tables published only as an image that is missing, or no reference list. A figure whose image could not be had keeps a visible
 placeholder, so an incomplete rendering looks incomplete. The prose and
-tables come from the same walker as `--to-markdown`, so the two agree;
-references are omitted and formulas stay as text.
+tables come from the same walker as `--to-markdown`, so the two agree, except
+that the page ends with the reference list (with DOI links), which the Markdown
+leaves out. Formulas stay as text. The XML never says how large a figure was
+typeset, so each is given about the same area (a square figure ~10.5 cm across),
+shaped by its aspect ratio and capped at the column width. Under the title come
+the authors, their affiliations and the journal citation (volume, issue, pages).
+Tables always fit the page width in the PDF; very wide ones print smaller.
 
 Only records whose XML was kept are rendered — add `--get-xml-or-html` to keep
 the XML beside the PDF.
@@ -987,7 +993,7 @@ library, for the papers that have no open-access copy.
 Browser cookie support is included in the standard fetchpdf installation.
 
 ```bash
-fetchpdf cookies setup
+fetchpdf fetchcookies
 ```
 
 The wizard asks two questions, then opens some tabs:
@@ -1019,7 +1025,7 @@ session exists but the test article was refused: the session lapsed, or your
 library does not take that particular journal. Access is turned on only if at
 least one publisher works.
 
-**Cookies often expire within hours**. You likely will have to rerun `fetchpdf cookies setup` periodically to restore access.
+**Cookies often expire within hours**. You likely will have to rerun `fetchpdf fetchcookies` periodically to restore access.
 
 ### Living with it
 
@@ -1053,7 +1059,7 @@ run. This works the same for a single DOI and for a batch:
 ```
 
 Runs that cannot be asked (pipes, cron, `--json`) print the command instead
-(`fetchpdf cookies setup --publishers wiley,tandf`, or `refresh --publishers ...`).
+(`fetchpdf fetchcookies --publishers wiley,tandf`, or `refresh --publishers ...`).
 `--no-cookies` and `--cookies FILE` turn the offer off. Royal Society papers are
 not offered, because there is no test article to sign in on.
 

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/fetchpdf/__init__.py

@@ -15,7 +15,7 @@ from .fetch_metadata_from_doi import fetch_metadata_from_doi
 # behavioural difference, it is the same object.
 fetch_pdf_from_doi = fetch_pdf
 
-__version__ = "0.1.3"
+__version__ = "0.1.4"
 __all__ = [
     "fetch_pdf",
     "fetch_pdf_from_doi",  # deprecated alias for fetch_pdf

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/fetchpdf/fetchpdf.py

@@ -4024,7 +4024,7 @@ def _try_institutional(save_path, resolved, cookies_file=None, verbose=False,
                        _source_out=None):
     """The institutional-access route. Returns the saved path, or None.
 
-    Off unless the user opted in -- `fetchpdf cookies setup`, or --cookies FILE --
+    Off unless the user opted in -- `fetchpdf fetchcookies`, or --cookies FILE --
     and silent for a publisher the route does not cover, so a run over APA or
     Elsevier DOIs carries no extra noise.
     """
@@ -5131,7 +5131,7 @@ def _fetch_pdf_chain(doi,
     # tail (Crossref landing pages, CORE, DOAJ, DataCite, PMID, Elsevier) that
     # rarely finds what a subscription serves in a second or two. Grey sources
     # (fetchpdf_grey) still run after the whole walk. A no-op unless the user
-    # ran `fetchpdf cookies setup` or passed --cookies, and for publishers it does
+    # ran `fetchpdf fetchcookies` or passed --cookies, and for publishers it does
     # not cover. Once per record: not on related-DOI re-entries.
     if _top_level and not (xml_only or xml_html_only):
         institutional_path = _try_institutional(
@@ -5809,6 +5809,14 @@ def _prune_empty_record_dir(record_dir):
     """
     if not record_dir:
         return
+    # The resolution cache is written beside the artifact, so a record that got
+    # nothing still holds one; alone, it is not something the record wrote.
+    from .retrieval.cache import CACHE_FILENAME
+    try:
+        if os.listdir(record_dir) == [CACHE_FILENAME]:
+            os.unlink(os.path.join(record_dir, CACHE_FILENAME))
+    except OSError:
+        pass
     try:
         os.rmdir(record_dir)
     except OSError:
@@ -6423,7 +6431,11 @@ def _batch_fetch_pdfs_inner(dois, output_dir, email=None, verbose=False, delay=0
         # path, or a record with four supplementary PDFs would report "pdf 5" and
         # corrupt the format composition tally.
         _pull_si(display_id, raw_identifier, canonical_doi, save_path)
-        _pull_figures(display_id, raw_identifier, canonical_doi, save_path)
+        # Unlike the supplements, figures belong to an article we have: with no
+        # artifact there is nothing for them to illustrate, and the pass would
+        # leave an empty `_figures/` and manifest in a record that failed.
+        if success:
+            _pull_figures(display_id, raw_identifier, canonical_doi, save_path)
         _render_xml(display_id, save_path)
         _pull_images(result or save_path, display_id)
 
@@ -7000,7 +7012,7 @@ def _ask_yes(question: str) -> bool:
 
 
 def _import_cookies(cfg, publishers) -> bool:
-    """Run `fetchpdf cookies setup` (no config yet) or `refresh` in-process,
+    """Run `fetchpdf fetchcookies` (no config yet) or `refresh` in-process,
     for *publishers* only -- the ones the failed papers are at.
 
     True when the retry is worth making: setup succeeded, or refresh found a
@@ -7064,7 +7076,7 @@ def _offer_cookie_import(failed, retry):
     if cfg is None:
         print(f"🔐 {n} failed {noun} ({where}) may be available through your "
               f"library's subscriptions, by importing your browser's cookies.")
-        command = f"fetchpdf cookies setup --publishers {','.join(publishers)}"
+        command = f"fetchpdf fetchcookies --publishers {','.join(publishers)}"
     else:
         print(f"🔐 {n} failed {noun} ({where}) are at publishers your library access "
               f"covers. If your library sign-in there has lapsed, signing in again may get them.")
@@ -7319,6 +7331,9 @@ def main(argv=None):
     if argv and argv[0] == "cookies":
         from .get_cookies import main as cookies_main
         return cookies_main(argv[1:])
+    if argv and argv[0] == "fetchcookies":
+        from .get_cookies import main as cookies_main
+        return cookies_main(["setup"] + argv[1:])
 
     pre = argparse.ArgumentParser(add_help=False)
     pre.add_argument("--json", action="store_true")
@@ -7410,7 +7425,7 @@ def _main(argv=None, json_out=None):
         description="Download PDFs from DOIs using multiple fallback sources.",
         epilog=(
             "Institutional access:\n"
-            "  fetchpdf cookies setup    # sign in through your library\n"
+            "  fetchpdf fetchcookies    # sign in through your library\n"
             "  fetchpdf cookies --help   # all cookie commands\n\n"
             "Examples:\n"
             "  fetchpdf papers.csv -o ./pdfs                      # batch from CSV\n"
@@ -7824,7 +7839,7 @@ def _main(argv=None, json_out=None):
         metavar="FILE",
         default=None,
         help="Institutional access from this cookie file instead of the one "
-             "`fetchpdf cookies setup` configured. Without it, access is used only "
+             "`fetchpdf fetchcookies` configured. Without it, access is used only "
              "if setup has been run. Covers publishers whose PDF URL follows "
              "from the DOI (Wiley, T&F, SAGE, Springer, Royal Society, Hogrefe, "
              "INFORMS); tried after the fast open-access lookups. Only cookies "
@@ -7890,7 +7905,7 @@ def _main(argv=None, json_out=None):
     if args.cookies_only:
         if args.no_cookies or not _institutional.active_cookies_file():
             print("❌ --cookies-only: institutional access is not set up. Run "
-                  "`fetchpdf cookies setup` (or pass --cookies FILE).")
+                  "`fetchpdf fetchcookies` (or pass --cookies FILE).")
             return 2
         _institutional.set_cookies_only(True)
         # "Alone" includes the grey last resorts an installed fetchpdf_grey adds.
@@ -8295,7 +8310,9 @@ def _main(argv=None, json_out=None):
             except Exception as e:
                 print(f"\n📎 supplementary pass failed: {str(e)[:200]}")
 
-        if args.pull_figures:
+        # Only for a record that was retrieved: figures with no article to go
+        # with them are an empty `_figures/` and manifest in a failed record.
+        if args.pull_figures and result:
             # A separate call above fetch_pdf, for the same reason the
             # supplementary pass is one: the tiered engine re-enters the chain
             # at T5 with a temporary save_path, and a flag threaded through
@@ -8337,6 +8354,8 @@ def _main(argv=None, json_out=None):
                         print(f"\n🖨️  render: {outcome['error']}")
                 except Exception as e:
                     print(f"\n🖨️  render failed: {str(e)[:200]}")
+            elif not result:
+                print("\n🖨️  no XML could be retrieved for this record, so nothing to render")
             else:
                 print("\n🖨️  no .xml was kept for this record, so nothing to render "
                       "(add --get-xml-or-html)")
@@ -8391,7 +8410,8 @@ def _main(argv=None, json_out=None):
             return 0
         else:
             _prune_empty_record_dir(made_record_dir)
-            print(f"\n❌ Failed to download PDF for {args.doi}")
+            wanted = "XML" if args.xml_only else ("XML or HTML" if args.xml_html_only else "PDF")
+            print(f"\n❌ Failed to download {wanted} for {args.doi}")
             _emit_json(json_out, args.doi, resolved_identifier, None,
                        JSON_STATUS_FAILED, identity=identity,
                        reasons=reasons or ["no source produced a usable artifact"],

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/fetchpdf/get_cookies.py

@@ -1,12 +1,12 @@
 """fetchpdf cookies: set up institutional access for fetchpdf, once.
 
-    fetchpdf cookies setup        # one-time wizard
+    fetchpdf fetchcookies        # one-time wizard
     fetchpdf cookies check        # which publishers your sessions currently open
     fetchpdf cookies refresh      # re-open sign-in tabs for the ones that lapsed
     fetchpdf cookies disable      # stop using institutional access
 
 Institutional access lets fetchpdf download a paper your library subscribes to
-when no open-access copy exists. It is OFF until you run ``fetchpdf cookies setup``.
+when no open-access copy exists. It is OFF until you run ``fetchpdf fetchcookies``.
 
 How it works. Each publisher keeps your library sign-in as a cookie on its own
 site (wiley.com, tandfonline.com, ...). The wizard opens one subscription
@@ -378,7 +378,7 @@ def cmd_setup(args) -> int:
         import browser_cookie3  # noqa: F401
     except ImportError:
         print("  A required dependency is missing: browser_cookie3.\n"
-              "  Run pip install 'browser_cookie3>=0.19', then run fetchpdf cookies setup again.")
+              "  Run pip install 'browser_cookie3>=0.19', then run fetchpdf fetchcookies again.")
         return 2
 
     # 1. Library
@@ -443,7 +443,7 @@ def cmd_setup(args) -> int:
 
     if not any(st == "ok" for st in status.values()):
         print("  No publisher served a PDF, so institutional access is NOT turned on.\n"
-              "  Check you can open a PDF in your browser, then run fetchpdf cookies setup again.")
+              "  Check you can open a PDF in your browser, then run fetchpdf fetchcookies again.")
         return 1
 
     cfg["publishers"] = status
@@ -463,7 +463,7 @@ def cmd_setup(args) -> int:
 def _require_config() -> Optional[dict]:
     cfg = inst.load_access_config()
     if not cfg:
-        print("Institutional access is not set up. Run:  fetchpdf cookies setup")
+        print("Institutional access is not set up. Run:  fetchpdf fetchcookies")
     return cfg
 
 
@@ -516,7 +516,7 @@ def cmd_disable(args) -> int:
         return 0
     cfg["enabled"] = False
     _write_private_json(ACCESS_CONFIG, cfg)
-    print(f"Institutional access is OFF. Turn it back on with: fetchpdf cookies setup")
+    print(f"Institutional access is OFF. Turn it back on with: fetchpdf fetchcookies")
     return 0
 
 

fetchpdf-0.1.4/fetchpdf/retrieval/byline.py

@@ -0,0 +1,257 @@
+"""Authors, affiliations and the journal citation, for under the rendered title.
+
+Like references.py, this is for the page a person reads (to_html.py) and is
+kept out of the Markdown walker, whose readers do not need a byline.
+
+JATS carries them in `<article-meta>`: `<contrib-group>` (each author's
+`<xref ref-type="aff">` points at an `<aff>` by id), and the volume, issue and
+pages as its direct children, with the journal title in `<journal-meta>`.
+Elsevier's full-text XML carries `<ce:author-group>` (`<ce:cross-ref refid>`
+to `<ce:affiliation>`) in the article head and the citation in `prism:`
+elements of its `coredata`; when the head is absent, `dc:creator` still names
+the authors.
+"""
+
+import html as html_lib
+import xml.etree.ElementTree as ET
+from dataclasses import dataclass, field
+from typing import List, Optional
+
+from ._util import localname
+from .references import _compose
+from .to_markdown import _clean, _first_text
+
+
+@dataclass
+class Byline:
+    authors: List[tuple] = field(default_factory=list)       # (name, [markers])
+    affiliations: List[tuple] = field(default_factory=list)  # (label, text)
+    journal: str = ""                                         # HTML
+
+    def html(self) -> str:
+        if not (self.authors or self.affiliations or self.journal):
+            return ""
+        out = ['<div class="byline">']
+        if self.authors:
+            names = [_esc(name) + (f"<sup>{_esc(','.join(marks))}</sup>" if marks else "")
+                     for name, marks in self.authors]
+            out.append(f'<p class="authors">{", ".join(names)}</p>')
+        if self.affiliations:
+            out.append('<ul class="affiliations">' + "".join(
+                "<li>" + (f"<sup>{_esc(label)}</sup> " if label else "") + f"{_esc(text)}</li>"
+                for label, text in self.affiliations) + "</ul>")
+        if self.journal:
+            out.append(f'<p class="journal">{self.journal}</p>')
+        out.append("</div>")
+        return "\n".join(out) + "\n"
+
+
+def byline_from_xml(content: bytes) -> Byline:
+    try:
+        root = ET.fromstring(content)
+    except ET.ParseError:
+        return Byline()
+    article = next((e for e in root.iter() if localname(e.tag) == "article"), None)
+    front = _child(article, "front")
+    meta = _child(front, "article-meta")
+    if meta is not None:
+        return _jats(front, meta)
+    return _elsevier(root)
+
+
+# -- JATS ---------------------------------------------------------------------
+
+
+def _jats(front, meta) -> Byline:
+    byline = Byline()
+    affs = []                                   # (id, label, text)
+    for aff in _jats_affs(meta):
+        label = _first_text(aff, "label", direct_only=True) or _first_text(aff, "sup", True)
+        text = _clean(" ".join(_text_without(aff, ("label", "sup", "institution-id"))))
+        if text:
+            affs.append((aff.get("id") or "", label, text.strip(" ,;")))
+
+    authors = []                                # (name, [(rid, kind, text)])
+    for group in meta:
+        if localname(group.tag) != "contrib-group":
+            continue
+        for contrib in group:
+            if localname(contrib.tag) != "contrib":
+                continue
+            if (contrib.get("contrib-type") or "author").lower() != "author":
+                continue
+            name = _jats_name(contrib)
+            if name:
+                refs = [(x.get("rid") or "", (x.get("ref-type") or "").lower(),
+                         _clean("".join(x.itertext())))
+                        for x in contrib.iter() if localname(x.tag) == "xref"]
+                authors.append((name, refs))
+
+    # An editor's or reviewer's <aff> sits beside the authors'; when authors
+    # point at theirs, only those belong in the byline.
+    cited = {rid for _, refs in authors for rid, kind, _ in refs if kind == "aff"}
+    if cited & {aff_id for aff_id, _, _ in affs}:
+        affs = [a for a in affs if a[0] in cited]
+    if len(affs) > 1 and not any(label for _, label, _ in affs):
+        affs = [(aff_id, str(n), text) for n, (aff_id, _, text) in enumerate(affs, 1)]
+    labels = {aff_id: label for aff_id, label, _ in affs if aff_id}
+    byline.affiliations = [(label, text) for _, label, text in affs]
+
+    for name, refs in authors:
+        marks = []
+        for rid, kind, text in refs:
+            if kind not in ("aff", "corresp", "fn", "author-notes"):
+                continue
+            mark = labels.get(rid) if kind == "aff" and rid in labels else text
+            if mark and mark not in marks:
+                marks.append(mark)
+        byline.authors.append((name, marks))
+
+    journal_meta = _child(front, "journal-meta")
+    journal = ""
+    if journal_meta is not None:
+        journal = (_first_text(journal_meta, "journal-title")
+                   or _first_text(journal_meta, "abbrev-journal-title"))
+    fpage, lpage = _direct(meta, "fpage"), _direct(meta, "lpage")
+    byline.journal = _citation(journal, _jats_year(meta), _direct(meta, "volume"),
+                               _direct(meta, "issue"),
+                               _pages(fpage, lpage) or _direct(meta, "elocation-id"))
+    return byline
+
+
+def _jats_affs(meta):
+    """Each `<aff>` once: under the contrib-groups or the meta, first of any alternatives."""
+    found = []
+    def visit(element):
+        for child in element:
+            tag = localname(child.tag)
+            if tag == "aff":
+                found.append(child)
+            elif tag == "aff-alternatives":
+                first = _child(child, "aff")
+                if first is not None:
+                    found.append(first)
+            elif tag == "contrib-group":
+                visit(child)
+    visit(meta)
+    return found
+
+
+def _jats_name(contrib) -> str:
+    for el in contrib:
+        tag = localname(el.tag)
+        if tag == "name-alternatives":
+            el = next((n for n in el if localname(n.tag) in ("name", "string-name")), None)
+            if el is None:
+                continue
+            tag = localname(el.tag)
+        if tag == "name":
+            return " ".join(filter(None, (_first_text(el, "prefix"), _first_text(el, "given-names"),
+                                          _first_text(el, "surname"), _first_text(el, "suffix"))))
+        if tag in ("string-name", "collab"):
+            return _clean(" ".join(el.itertext()))
+    return ""
+
+
+def _jats_year(meta) -> str:
+    dates = [d for d in meta if localname(d.tag) == "pub-date"]
+    def kind(d):
+        return (d.get("pub-type") or d.get("date-type") or "").lower()
+    for wanted in ("ppub", "collection", "pub", "epub"):
+        for date in dates:
+            if kind(date) == wanted and _first_text(date, "year"):
+                return _first_text(date, "year")
+    return next((_first_text(d, "year") for d in dates if _first_text(d, "year")), "")
+
+
+# -- Elsevier -----------------------------------------------------------------
+
+
+def _elsevier(root) -> Byline:
+    byline = Byline()
+    labels = {}
+    groups = [e for e in root.iter() if localname(e.tag) == "author-group"]
+    for group in groups:
+        for aff in group:
+            if localname(aff.tag) != "affiliation":
+                continue
+            label = _first_text(aff, "label", direct_only=True)
+            text = _first_text(aff, "textfn") or _clean(
+                " ".join(_text_without(aff, ("label", "affiliation"))))
+            if text:
+                byline.affiliations.append((label, text))
+                if aff.get("id"):
+                    labels[aff.get("id")] = label
+    for group in groups:
+        for person in group:
+            tag = localname(person.tag)
+            if tag == "author":
+                name = " ".join(filter(None, (_first_text(person, "given-name"),
+                                              _first_text(person, "surname"))))
+            elif tag == "collaboration":
+                name = _first_text(person, "text") or _clean(" ".join(person.itertext()))
+            else:
+                continue
+            marks = []
+            for ref in person.iter():
+                if localname(ref.tag) == "cross-ref":
+                    # The affiliation's own label wins: some XML prints "1"
+                    # at the author and "a" at the affiliation it points to.
+                    mark = labels.get(ref.get("refid") or "") or _clean("".join(ref.itertext()))
+                    if mark and mark not in marks:
+                        marks.append(mark)
+            if name:
+                byline.authors.append((name, marks))
+    if not byline.authors:
+        for el in root.iter():
+            if localname(el.tag) == "creator":
+                surname, _, given = _clean("".join(el.itertext())).partition(", ")
+                byline.authors.append((" ".join(filter(None, (given, surname))), []))
+
+    coredata = next((e for e in root.iter() if localname(e.tag) == "coredata"), root)
+    first, last = _first_text(coredata, "startingpage"), _first_text(coredata, "endingpage")
+    byline.journal = _citation(
+        _first_text(coredata, "publicationname"), _first_text(coredata, "coverdate")[:4],
+        _first_text(coredata, "volume"), _first_text(coredata, "issueidentifier"),
+        _pages(first, last) or _first_text(coredata, "pagerange"))
+    return byline
+
+
+# -- shared -------------------------------------------------------------------
+
+
+def _citation(journal: str, year: str, volume: str, issue: str, pages: str) -> str:
+    """`<i>Journal.</i> 2026;21(11):5256–5260.`, or "" when there is no journal."""
+    if not journal:
+        return ""
+    return _compose(authors=[], etal=False, title="", editors=[], source=journal,
+                    publisher="", year=year, volume=volume, issue=issue, pages=pages,
+                    doi="", url="", comment="")
+
+
+def _pages(first: str, last: str) -> str:
+    return f"{first}–{last}" if first and last and first != last else first
+
+
+def _text_without(element, skip) -> List[str]:
+    """Text of `element`, leaving out the subtrees of the named descendants (tails kept)."""
+    parts = [element.text or ""]
+    for child in element:
+        if localname(child.tag) not in skip:
+            parts.extend(_text_without(child, skip))
+        parts.append(child.tail or "")
+    return parts
+
+
+def _direct(element, name: str) -> str:
+    return _first_text(element, name, direct_only=True)
+
+
+def _child(element, name: str):
+    if element is None:
+        return None
+    return next((c for c in element if localname(c.tag) == name), None)
+
+
+def _esc(text: Optional[str]) -> str:
+    return html_lib.escape(text or "", quote=True)

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/fetchpdf/retrieval/figures.py

@@ -403,6 +403,12 @@ def _run(manifest, manifest_path, directory, stem, ids, ctx, http,
 
     def finish() -> FiguresSummary:
         manifest["completed_at"] = _iso(time.time())
+        # The directory is made before each download; when every one failed it
+        # is left empty, and an empty `_figures/` reads as "figures here".
+        try:
+            os.rmdir(directory)
+        except OSError:
+            pass                                # holds figures, or never made
         path = write_manifest(manifest_path, manifest, ctx.verbose)
         refusal = (manifest.get("refusal") or {}).get("kind", "")
         rows = manifest["figures"]

{fetchpdf-0.1.3 → fetchpdf-0.1.4}/fetchpdf/retrieval/institutional.py

@@ -48,7 +48,7 @@ SOURCE = "institutional_cookies"
 # --------------------------------------------------------------------------
 # Off until the user opts in. Two ways to opt in:
 #
-# * `fetchpdf cookies setup` writes ACCESS_CONFIG (library + everyday browser). From
+# * `fetchpdf fetchcookies` writes ACCESS_CONFIG (library + everyday browser). From
 #   then on every fetchpdf run re-reads the publisher cookies straight from that
 #   browser, once per process, so sessions the user keeps alive by ordinary
 #   browsing are picked up without re-exporting anything.
@@ -69,7 +69,7 @@ _live_resolved = False
 
 
 def load_access_config() -> Optional[dict]:
-    """The `fetchpdf cookies setup` configuration, or None if access is not set up."""
+    """The `fetchpdf fetchcookies` configuration, or None if access is not set up."""
     try:
         cfg = json.loads(ACCESS_CONFIG.read_text(encoding="utf-8"))
     except (OSError, ValueError):
@@ -132,7 +132,7 @@ def cookies_user_managed() -> bool:
 def active_cookies_file() -> Optional[str]:
     """The cookie file this process should use, or None when access is off.
 
-    An explicit ``--cookies`` wins. Otherwise, if `fetchpdf cookies setup` has been
+    An explicit ``--cookies`` wins. Otherwise, if `fetchpdf fetchcookies` has been
     run, the configured browser's cookies are exported once per process to the
     config dir and that file is used; if the browser cannot be read (locked
     store, keyring prompt declined) the last export is used instead.