flowapy 0.3.0__tar.gz → 0.4.0__tar.gz

{flowapy-0.3.0 → flowapy-0.4.0}/.pre-commit-config.yaml

@@ -113,6 +113,14 @@ repos:
       # demo-gateway is its own uv project (separate venv + ruff/mypy config),
       # so these cd in and run that project's pinned tools, mirroring the pytest
       # hook below.
+      #
+      # demo-gateway installs flowapy as an editable path dep (`path = "../.."`),
+      # so a change to `src/flowa/` can break its mypy/pytest gates even when no
+      # demo-gateway file is touched. The type+behaviour gates therefore also
+      # fire on `src/flowa/**.py`; the formatters don't, since ruff only inspects
+      # demo-gateway's own files. (Without this, a flowapy-only commit skips these
+      # locally — pre-commit runs hooks against staged files — and the break only
+      # surfaces in CI's `--all-files` run.)
       - id: ruff-format-demo-gateway
         name: ruff format (demo-gateway)
         entry: bash -c 'cd examples/demo-gateway && uv run --no-sync ruff format --check src/ tests/'
@@ -130,10 +138,10 @@ repos:
         entry: bash -c 'cd examples/demo-gateway && uv run --no-sync mypy src/ tests/'
         language: system
         pass_filenames: false
-        files: ^examples/demo-gateway/.*\.py$
+        files: ^(examples/demo-gateway/.*|src/flowa/.*)\.py$
       - id: pytest-demo-gateway
         name: pytest (demo-gateway)
         entry: bash -c 'cd examples/demo-gateway && uv run --no-sync pytest tests/'
         language: system
         pass_filenames: false
-        files: ^examples/demo-gateway/.*\.py$
+        files: ^(examples/demo-gateway/.*|src/flowa/.*)\.py$

{flowapy-0.3.0 → flowapy-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flowapy
-Version: 0.3.0
+Version: 0.4.0
 Summary: Variant literature assessment pipeline with AI extraction
 Project-URL: Homepage, https://github.com/populationgenomics/flowa
 Project-URL: Source, https://github.com/populationgenomics/flowa

{flowapy-0.3.0 → flowapy-0.4.0}/examples/demo/README.md

@@ -166,10 +166,12 @@ VARIANT=NM_001035_3-c_14174A_G
 rm -f assessments/$VARIANT/aggregation.json \
       assessments/$VARIANT/aggregation_raw.json
 rm -rf assessments/$VARIANT/extractions/ assessments/$VARIANT/runs/
-# Re-runs flowa.convert (which uses anchorite for PDF chunking and
-# builds pdf_index.pkl.zst). Drop this line to reuse the cached markdown
-# + index and only redo extract + aggregate.
-rm -f papers/*/markdown.md papers/*/convert_raw.json papers/*/pdf_index.pkl.zst
+# Re-runs flowa.convert (which transcribes each PDF, builds merged.pdf when
+# there are PDF supplements, and builds pdf_index.pkl.zst). Drop this line to
+# reuse the cached transcriptions + index and only redo extract + aggregate.
+rm -f papers/*/main.md papers/*/merged.md papers/*/merged.pdf \
+      papers/*/convert_raw.json papers/*/pdf_index.pkl.zst \
+      papers/*/supplements/*.pdf.md
 ```
 
 Then drive the pipeline. The demo's `scripts/start.ts` translates the
@@ -207,11 +209,11 @@ and not needed by anything downstream.
 
 For papers whose source license blocks redistribution (CC-BY-NC-ND,
 paywalled; see `fixtures/LICENSES.md` for the rule), do **not** delete
-the whole `papers/{encodedDoi}/` directory — only delete `source.pdf`,
-`markdown.md`, `convert_raw.json`, and `pdf_index.pkl.zst`. The
-`pdf_index.pkl.zst` embeds the PDF's extracted text (anchorite's char
-index), so it carries the same copyright as `source.pdf` and must not
-ship in the open-source repo. Keep `metadata.json` (the bibliographic
+the whole `papers/{encodedDoi}/` directory — only delete `main.pdf`,
+`merged.pdf`, `main.md`, `merged.md`, `convert_raw.json`, `pdf_index.pkl.zst`,
+and the `supplements/` directory. The `pdf_index.pkl.zst` embeds the PDF's
+extracted text (anchorite's char index), so it carries the same copyright as
+`main.pdf` and must not ship in the open-source repo. Keep `metadata.json` (the bibliographic
 fields are factual data, not copyrightable) but replace its `abstract`
 field with a sentinel string, so the omission reads as deliberate (not
 a missing-data bug) when the literature view renders the row:

{flowapy-0.3.0 → flowapy-0.4.0}/examples/demo/fixtures/LICENSES.md

@@ -2,7 +2,7 @@
 
 The fixture under `examples/demo/fixtures/papers/` ships three kinds of entries, all keyed by the encoded DOI:
 
-1. **Full-content papers** (15) — `source.pdf` + `markdown.md` + `metadata.json`. The pipeline downloaded the PDF from PMC, converted it to Markdown, and ran extraction. Three also carry `source.md` (the PDF transcription) and a `supplements/` directory of xlsx/docx supplements, which are converted and appended into `markdown.md`. All are CC-BY (any version) or CC0, so the PDF, the derivative Markdown, and the supplements are redistributable provided attribution is preserved (entries below).
+1. **Full-content papers** (15) — `main.pdf` + `main.md` + `metadata.json` + `pdf_index.pkl.zst`. The pipeline downloaded the main PDF from PMC, transcribed it to Markdown (`main.md`), and ran extraction. Papers with supplements also carry a `supplements/` directory and an assembled `merged.md` (`main.md` + the supplement transcriptions/conversions): **3** carry an xlsx/docx (office) supplement, and **2** carry a separate PDF supplement (which is additionally concatenated into a derived `merged.pdf` and transcribed). All are CC-BY (any version) or CC0, so the PDFs, the derivative Markdown, and the supplements are redistributable provided attribution is preserved (entries below).
 2. **Metadata-only, license-restricted** (5) — only `metadata.json`; the source paper is paywalled or under a no-derivatives / non-commercial licence (e.g. CC-BY-NC-ND). The `abstract` field is replaced with a sentinel string explaining the omission, since the abstract is the author's prose and is subject to publisher copyright. Bibliographic facts (DOI, PMID, title, authors, journal, date) are not copyrightable and ship as-is.
 3. **Metadata-only, CC-BY not in PMC** (1) — only `metadata.json`; the source paper IS CC-BY-licensed but PMC's OA subset doesn't carry it, so flowa's PMC-only download path silently skipped it. Abstract is preserved (CC-BY permits redistribution); PDF/Markdown are absent. A curator running the demo locally can fetch the PDF from the journal site.
 
@@ -101,6 +101,7 @@ For all three kinds, the demo's literature page renders the row from `metadata.j
 - **DOI:** [10.1002/humu.23878](https://doi.org/10.1002/humu.23878)
 - **PMID:** 31342611
 - **License:** [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)
+- **Supplement:** `supplements/000_HUMU-40-2146-s001.pdf` (supplementary data PDF; inherits the article's CC-BY 4.0).
 
 ### `10.1186%2Fs12881-019-0878-8/`
 
@@ -147,12 +148,18 @@ For all three kinds, the demo's literature page renders the row from `metadata.j
 - **DOI:** [10.3389/fped.2021.729824](https://doi.org/10.3389/fped.2021.729824)
 - **PMID:** 34966699
 - **License:** [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)
+- **Supplement:** `supplements/000_Image_1.PDF` (supplementary figure PDF; inherits the article's CC-BY 4.0).
 
 ### Derivative artifacts
 
 For each full-content paper the fixture also ships the pipeline's derivatives:
 
-- `markdown.md` — Markdown extracted from the PDF by the conversion stage.
+- `main.md` — the main PDF transcribed to Markdown by the conversion stage.
+- `merged.md` — present only for papers with supplements: `main.md` + the
+  PDF-supplement transcriptions + the converted office supplements.
+- `merged.pdf` — present only for the two papers with a PDF supplement: `main.pdf`
+  concatenated with the supplement PDF (what the viewer renders + the index is built from).
+- `pdf_index.pkl.zst` — the anchorite PDF index (embeds the PDF's extracted text).
 - `metadata.json` — bibliographic metadata (DOI, PMID, title, authors, etc.).
 
 These derivatives are released under the same CC-BY licence as the underlying PDFs.

{flowapy-0.3.0 → flowapy-0.4.0}/examples/demo/fixtures/assessments/NM_000152_5-c_1935C_A/aggregation.json

@@ -416,8 +416,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 110375,
-                  "end": 110422
+                  "start": 110789,
+                  "end": 110836
                 }
               }
             }
@@ -454,8 +454,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 26931,
-                  "end": 27066
+                  "start": 36529,
+                  "end": 36664
                 }
               }
             }
@@ -485,8 +485,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 53026,
-                  "end": 53074
+                  "start": 53390,
+                  "end": 53438
                 }
               }
             }
@@ -551,8 +551,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 65886,
-                  "end": 66324
+                  "start": 66258,
+                  "end": 66696
                 }
               }
             }
@@ -624,8 +624,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 54842,
-                  "end": 55327
+                  "start": 55216,
+                  "end": 55701
                 }
               }
             }
@@ -2810,8 +2810,8 @@
                   }
                 ],
                 "markdown_anchor": {
-                  "start": 6305,
-                  "end": 6462
+                  "start": 6304,
+                  "end": 6461
                 }
               }
             },