spirotrace 0.0.1__tar.gz

spirotrace-0.0.1/PKG-INFO

@@ -0,0 +1,66 @@
+Metadata-Version: 2.1
+Name: spirotrace
+Version: 0.0.1
+Summary: Traceable extraction of lung function data from printed and scanned reports
+Author: Kirsty Hawke
+Project-URL: Homepage, https://github.com/hawkek/spirotrace
+Project-URL: Issues, https://github.com/hawkek/spirotrace/issues
+Keywords: spirometry,lung function,PDF extraction,OCR,data quality,GLI
+Classifier: Development Status :: 1 - Planning
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+
+# spirotrace
+
+**Traceable extraction of lung function data from printed and scanned reports.**
+
+> Status: early development. Nothing here is ready for use yet. The working prototype that this package replaces is not public; this repository is being built up from it module by module, with tests.
+
+## Statement of need
+
+Lung function results (spirometry, lung volumes, gas transfer) often reach researchers only as printed or scanned PDF reports, not as structured data. Typing them by hand is slow and error-prone, and generic PDF or OCR extraction gives values with no indication of which ones are wrong.
+
+spirotrace turns such reports into an audited, analysis-ready dataset with a measured error rate:
+
+- **Extraction.** Report layouts are described in template files (geometry and vocabulary); one reader handles every layout.
+- **Auditability.** Every value keeps its raw text, its position on the page, how it was read, and every human correction made to it (who, when, before, after).
+- **Physiology-informed checks.** Values are checked against each other and against reference equations (for example, the printed z-score against a GLI recalculation, FEV1/FVC against FEV1 and FVC, LLN against the sign of the z-score), so errors that look like plausible numbers are still caught.
+- **Measured reliability.** Built-in tools compare the extraction with a blinded manual reference sample and report accuracy, flag sensitivity and residual (silent) errors.
+- **Local by default.** Everything runs on the user's machine. Nothing is uploaded; there is no telemetry.
+
+Similar tools exist ([PFT-Extractor](https://github.com/lindseyboulet/PFT-Extractor), an R app for Vmax reports; [pft-extractor](https://github.com/Automate-Medical/pft-extractor), a cloud pipeline using Amazon Textract). Neither describes validation of the extracted values or detection of extraction errors.
+
+## Planned scope
+
+| Tier | Contents |
+| --- | --- |
+| Research MVP | Two report layouts, deterministic extraction, provenance on every value, core checks, review corrections as an event log, derived variables (ATS/ERS 2022 pattern, severity, PRISm, bronchodilator response) after review, GLI recalculation via [rspiro](https://cran.r-project.org/package=rspiro), reference-standard evaluation |
+| v1.0 | Add-your-own layout templates, a local browser review app, documentation |
+| Later | Installers for non-programmers, template editor, further OCR engines |
+
+## Data protection
+
+spirotrace is meant for clinical reports. Please:
+
+- **never** attach a report, a screenshot of one, or extracted values to an issue or pull request;
+- report layout problems with the masked, structure-only survey output instead (instructions will be in the docs);
+- check that your ethics and data-access approvals cover your use.
+
+Outputs written to a project's `share/` folder are masked or aggregate by default. Whether any file may be shared is a decision for your study's governance, not something the software can certify.
+
+## Installation
+
+Not yet. When the first usable release is out: `pip install spirotrace`.
+
+## Citation
+
+See [CITATION.cff](CITATION.cff). A software paper is planned.
+
+## Licence
+
+To be confirmed (an OSI-approved permissive licence is intended).

spirotrace-0.0.1/README.md

@@ -0,0 +1,49 @@
+# spirotrace
+
+**Traceable extraction of lung function data from printed and scanned reports.**
+
+> Status: early development. Nothing here is ready for use yet. The working prototype that this package replaces is not public; this repository is being built up from it module by module, with tests.
+
+## Statement of need
+
+Lung function results (spirometry, lung volumes, gas transfer) often reach researchers only as printed or scanned PDF reports, not as structured data. Typing them by hand is slow and error-prone, and generic PDF or OCR extraction gives values with no indication of which ones are wrong.
+
+spirotrace turns such reports into an audited, analysis-ready dataset with a measured error rate:
+
+- **Extraction.** Report layouts are described in template files (geometry and vocabulary); one reader handles every layout.
+- **Auditability.** Every value keeps its raw text, its position on the page, how it was read, and every human correction made to it (who, when, before, after).
+- **Physiology-informed checks.** Values are checked against each other and against reference equations (for example, the printed z-score against a GLI recalculation, FEV1/FVC against FEV1 and FVC, LLN against the sign of the z-score), so errors that look like plausible numbers are still caught.
+- **Measured reliability.** Built-in tools compare the extraction with a blinded manual reference sample and report accuracy, flag sensitivity and residual (silent) errors.
+- **Local by default.** Everything runs on the user's machine. Nothing is uploaded; there is no telemetry.
+
+Similar tools exist ([PFT-Extractor](https://github.com/lindseyboulet/PFT-Extractor), an R app for Vmax reports; [pft-extractor](https://github.com/Automate-Medical/pft-extractor), a cloud pipeline using Amazon Textract). Neither describes validation of the extracted values or detection of extraction errors.
+
+## Planned scope
+
+| Tier | Contents |
+| --- | --- |
+| Research MVP | Two report layouts, deterministic extraction, provenance on every value, core checks, review corrections as an event log, derived variables (ATS/ERS 2022 pattern, severity, PRISm, bronchodilator response) after review, GLI recalculation via [rspiro](https://cran.r-project.org/package=rspiro), reference-standard evaluation |
+| v1.0 | Add-your-own layout templates, a local browser review app, documentation |
+| Later | Installers for non-programmers, template editor, further OCR engines |
+
+## Data protection
+
+spirotrace is meant for clinical reports. Please:
+
+- **never** attach a report, a screenshot of one, or extracted values to an issue or pull request;
+- report layout problems with the masked, structure-only survey output instead (instructions will be in the docs);
+- check that your ethics and data-access approvals cover your use.
+
+Outputs written to a project's `share/` folder are masked or aggregate by default. Whether any file may be shared is a decision for your study's governance, not something the software can certify.
+
+## Installation
+
+Not yet. When the first usable release is out: `pip install spirotrace`.
+
+## Citation
+
+See [CITATION.cff](CITATION.cff). A software paper is planned.
+
+## Licence
+
+To be confirmed (an OSI-approved permissive licence is intended).

spirotrace-0.0.1/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "spirotrace"
+version = "0.0.1"
+description = "Traceable extraction of lung function data from printed and scanned reports"
+readme = "README.md"
+requires-python = ">=3.11"
+authors = [{ name = "Kirsty Hawke" }]
+keywords = ["spirometry", "lung function", "PDF extraction", "OCR", "data quality", "GLI"]
+classifiers = [
+    "Development Status :: 1 - Planning",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Medical Science Apps.",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+]
+# Dependencies are added as modules are ported (planned: pdfplumber, pypdfium2, Pillow, numpy, openpyxl, PyYAML).
+dependencies = []
+
+[project.optional-dependencies]
+dev = ["pytest>=7"]
+
+[project.urls]
+Homepage = "https://github.com/hawkek/spirotrace"
+Issues = "https://github.com/hawkek/spirotrace/issues"
+
+[project.scripts]
+spirotrace = "spirotrace.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+"spirotrace.layouts" = ["templates/*.yaml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

spirotrace-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

spirotrace-0.0.1/src/spirotrace/__init__.py

@@ -0,0 +1,3 @@
+"""spirotrace: traceable extraction of lung function data from printed and scanned reports."""
+
+__version__ = "0.0.1"

spirotrace-0.0.1/src/spirotrace/checks/__init__.py

@@ -0,0 +1,5 @@
+"""Rule registry. Every check is a named rule with an id, a layer (ocr, range, layout, image, consistency, gli,
+cohort, free_text), a severity and a one-sentence rationale, and can be switched off by name.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/cli.py

@@ -0,0 +1,32 @@
+"""Command line entry point: spirotrace <command>."""
+import argparse
+import sys
+
+from . import __version__
+
+COMMANDS = {
+    "init": "create a study project folder",
+    "run": "extract and check all reports in a project",
+    "review": "open the local review app",
+    "export": "write the dataset, data dictionary and QC report",
+    "template": "draft, test or survey a layout template",
+    "eval": "evaluate against a manually coded reference sample",
+}
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog="spirotrace", description=__doc__)
+    ap.add_argument("--version", action="version", version=f"spirotrace {__version__}")
+    sub = ap.add_subparsers(dest="command", metavar="command")
+    for name, help_ in COMMANDS.items():
+        sub.add_parser(name, help=help_)
+    a = ap.parse_args(argv)
+    if a.command is None:
+        ap.print_help()
+        return 0
+    print(f"spirotrace {__version__}: '{a.command}' is not implemented yet.")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

spirotrace-0.0.1/src/spirotrace/derive/__init__.py

@@ -0,0 +1,5 @@
+"""Derived variables computed from reviewed values only: GLI recalculation (rspiro), ATS/ERS 2022 pattern,
+severity, PRISm, bronchodilator response. Each derived value records the observations it came from.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/evaluate/__init__.py

@@ -0,0 +1,5 @@
+"""Reference-standard evaluation: sampling, coding sheets, agreement and adjudication, accuracy, flag
+sensitivity and silent errors, ablation by check layer.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/ingest/__init__.py

@@ -0,0 +1,4 @@
+"""Read each PDF once: page rotation and deskew, the text layer or an OCR engine, page images; cached by file hash.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/layouts/__init__.py

@@ -0,0 +1,6 @@
+"""Layout templates (YAML: geometry and vocabulary only), page classification, table, field and free-text readers.
+
+Behaviour belongs in Python: a layout quirk is a named, tested handler that a template switches on.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/privacy/__init__.py

@@ -0,0 +1,4 @@
+"""Masking and the export guard: everything written to share/ passes through here.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace/review/__init__.py

@@ -0,0 +1,4 @@
+"""Local review app (served on 127.0.0.1 only). Every reviewer action is appended to an event log.
+
+Not implemented yet.
+"""

spirotrace-0.0.1/src/spirotrace.egg-info/PKG-INFO

@@ -0,0 +1,66 @@
+Metadata-Version: 2.1
+Name: spirotrace
+Version: 0.0.1
+Summary: Traceable extraction of lung function data from printed and scanned reports
+Author: Kirsty Hawke
+Project-URL: Homepage, https://github.com/hawkek/spirotrace
+Project-URL: Issues, https://github.com/hawkek/spirotrace/issues
+Keywords: spirometry,lung function,PDF extraction,OCR,data quality,GLI
+Classifier: Development Status :: 1 - Planning
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+
+# spirotrace
+
+**Traceable extraction of lung function data from printed and scanned reports.**
+
+> Status: early development. Nothing here is ready for use yet. The working prototype that this package replaces is not public; this repository is being built up from it module by module, with tests.
+
+## Statement of need
+
+Lung function results (spirometry, lung volumes, gas transfer) often reach researchers only as printed or scanned PDF reports, not as structured data. Typing them by hand is slow and error-prone, and generic PDF or OCR extraction gives values with no indication of which ones are wrong.
+
+spirotrace turns such reports into an audited, analysis-ready dataset with a measured error rate:
+
+- **Extraction.** Report layouts are described in template files (geometry and vocabulary); one reader handles every layout.
+- **Auditability.** Every value keeps its raw text, its position on the page, how it was read, and every human correction made to it (who, when, before, after).
+- **Physiology-informed checks.** Values are checked against each other and against reference equations (for example, the printed z-score against a GLI recalculation, FEV1/FVC against FEV1 and FVC, LLN against the sign of the z-score), so errors that look like plausible numbers are still caught.
+- **Measured reliability.** Built-in tools compare the extraction with a blinded manual reference sample and report accuracy, flag sensitivity and residual (silent) errors.
+- **Local by default.** Everything runs on the user's machine. Nothing is uploaded; there is no telemetry.
+
+Similar tools exist ([PFT-Extractor](https://github.com/lindseyboulet/PFT-Extractor), an R app for Vmax reports; [pft-extractor](https://github.com/Automate-Medical/pft-extractor), a cloud pipeline using Amazon Textract). Neither describes validation of the extracted values or detection of extraction errors.
+
+## Planned scope
+
+| Tier | Contents |
+| --- | --- |
+| Research MVP | Two report layouts, deterministic extraction, provenance on every value, core checks, review corrections as an event log, derived variables (ATS/ERS 2022 pattern, severity, PRISm, bronchodilator response) after review, GLI recalculation via [rspiro](https://cran.r-project.org/package=rspiro), reference-standard evaluation |
+| v1.0 | Add-your-own layout templates, a local browser review app, documentation |
+| Later | Installers for non-programmers, template editor, further OCR engines |
+
+## Data protection
+
+spirotrace is meant for clinical reports. Please:
+
+- **never** attach a report, a screenshot of one, or extracted values to an issue or pull request;
+- report layout problems with the masked, structure-only survey output instead (instructions will be in the docs);
+- check that your ethics and data-access approvals cover your use.
+
+Outputs written to a project's `share/` folder are masked or aggregate by default. Whether any file may be shared is a decision for your study's governance, not something the software can certify.
+
+## Installation
+
+Not yet. When the first usable release is out: `pip install spirotrace`.
+
+## Citation
+
+See [CITATION.cff](CITATION.cff). A software paper is planned.
+
+## Licence
+
+To be confirmed (an OSI-approved permissive licence is intended).

spirotrace-0.0.1/src/spirotrace.egg-info/SOURCES.txt

@@ -0,0 +1,18 @@
+README.md
+pyproject.toml
+src/spirotrace/__init__.py
+src/spirotrace/cli.py
+src/spirotrace.egg-info/PKG-INFO
+src/spirotrace.egg-info/SOURCES.txt
+src/spirotrace.egg-info/dependency_links.txt
+src/spirotrace.egg-info/entry_points.txt
+src/spirotrace.egg-info/requires.txt
+src/spirotrace.egg-info/top_level.txt
+src/spirotrace/checks/__init__.py
+src/spirotrace/derive/__init__.py
+src/spirotrace/evaluate/__init__.py
+src/spirotrace/ingest/__init__.py
+src/spirotrace/layouts/__init__.py
+src/spirotrace/privacy/__init__.py
+src/spirotrace/review/__init__.py
+tests/test_skeleton.py

spirotrace-0.0.1/src/spirotrace.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

spirotrace-0.0.1/src/spirotrace.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+spirotrace = spirotrace.cli:main

spirotrace-0.0.1/src/spirotrace.egg-info/requires.txt

@@ -0,0 +1,3 @@
+
+[dev]
+pytest>=7

spirotrace-0.0.1/src/spirotrace.egg-info/top_level.txt

@@ -0,0 +1 @@
+spirotrace

spirotrace-0.0.1/tests/test_skeleton.py

@@ -0,0 +1,16 @@
+import spirotrace
+from spirotrace import cli
+
+
+def test_version_is_set():
+    assert spirotrace.__version__ == "0.0.1"
+
+
+def test_cli_without_command_prints_help(capsys):
+    assert cli.main([]) == 0
+    assert "usage" in capsys.readouterr().out
+
+
+def test_unimplemented_command_says_so(capsys):
+    assert cli.main(["run"]) == 1
+    assert "not implemented" in capsys.readouterr().out