pubmed-proto 0.1.0__tar.gz

pubmed_proto-0.1.0/.gitignore

@@ -0,0 +1,5 @@
+generated/
+dist/
+__pycache__/
+*.pyc
+.venv/

pubmed_proto-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Centre for Population Genomics
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pubmed_proto-0.1.0/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.4
+Name: pubmed-proto
+Version: 0.1.0
+Summary: Typed protobuf + pydantic models and converters for NLM PubMed XML.
+Project-URL: Repository, https://github.com/populationgenomics/pubmed-proto
+Project-URL: Issues, https://github.com/populationgenomics/pubmed-proto/issues
+Author: Centre for Population Genomics
+License-Expression: MIT
+License-File: LICENSE
+Keywords: bioinformatics,ncbi,protobuf,pubmed,pydantic,xml
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: lxml
+Requires-Dist: protobuf>=6.32.1
+Requires-Dist: pydantic>=2
+Description-Content-Type: text/markdown
+
+# pubmed-proto
+
+Generates **`pubmed_proto`** — a typed Python package for parsing NLM PubMed XML
+into protobuf and pydantic models — from the PubMed DTD.
+
+This repository is the **generator**, not the package. It holds the inputs
+(`pubmed.dtd`, `pubmed_transforms.yaml`) and drives
+[`xsd-former`](https://github.com/populationgenomics/xsd-former) (the
+`xsdformer` CLI) to emit the `pubmed_proto` source tree, which is then built into
+a wheel and published to PyPI. The generated tree (`generated/`) and build
+outputs (`dist/`) are gitignored — only the inputs are version-controlled.
+
+## Consuming `pubmed_proto`
+
+Depend on the published wheel, not this repo:
+
+```
+pip install pubmed_proto      # or: uv add pubmed_proto
+```
+
+```python
+from lxml import etree
+from pubmed_proto import xml_converter, pydantic_converter, models
+
+tree = etree.parse("efetch_output.xml")
+article_el = tree.getroot().find("PubmedArticle")
+
+proto = xml_converter.PubmedArticle(article_el)     # XML  -> protobuf
+model = pydantic_converter.PubmedArticle_from_proto(proto)  # protobuf -> pydantic
+json_str = model.model_dump_json()                  # pydantic -> JSON
+```
+
+The package exposes four modules (all typed; ships `py.typed`):
+
+| module               | purpose                                            |
+| -------------------- | -------------------------------------------------- |
+| `pubmed_pb2`         | compiled protobuf messages (`Article`, `Author`, …) |
+| `models`             | pydantic models mirroring the protobuf schema       |
+| `xml_converter`      | PubMed XML → protobuf (per-message factory funcs)   |
+| `pydantic_converter` | protobuf ↔ pydantic (`X_from_proto` / `X_to_proto`) |
+
+## Developing the generator
+
+Requires [`uv`](https://docs.astral.sh/uv/).
+
+```
+make generate   # DTD + transforms -> generated/pubmed_proto/
+make build      # generate, then build the wheel into dist/
+make clean      # remove generated/ and dist/
+uv run --group test pytest   # round-trip gate over real PubMed records
+```
+
+Shaping the output is done in **`pubmed_transforms.yaml`** — dropping
+book/admin types, flattening list wrappers, coercing booleans/timestamps, and
+serializing rich-text fields to markdown. See the
+[`xsd-former`](https://github.com/populationgenomics/xsd-former) docs for the
+transform reference.
+
+## Provenance & attribution
+
+`pubmed.dtd` is derived from the **U.S. National Library of Medicine PubMed
+DTD**, version `pubmed_250101` (dated 2024-08-28):
+
+<https://dtd.nlm.nih.gov/ncbi/pubmed/out/pubmed_250101.dtd>
+
+NLM DTDs are U.S. Government works and public domain in the United States; the
+MIT `LICENSE` in this repo covers CPG's own files (transforms, generator wiring,
+tests), not the NLM DTD.
+
+**Local modification:** the external MathML module include
+(`<!ENTITY % mathml-in-pubmed SYSTEM "mathml-in-pubmed.mod">` and its reference)
+was removed so the DTD is self-contained for schema generation — MathML markup
+in titles/abstracts is not modelled. With that one include removed, the file is
+byte-identical to upstream `pubmed_250101`.
+
+The DTD is **vendored deliberately, not fetched at build time**: it's a modified
+derivative (so a fetch wouldn't reproduce it), and pinning the exact bytes keeps
+the generated schema reproducible. The full provenance also lives in a comment
+at the top of `pubmed.dtd`.
+
+## Releasing
+
+The published version is **`build.version` in `pubmed_transforms.yaml`** (what
+`xsdformer` stamps into the wheel). To release:
+
+1. Bump `build.version` in `pubmed_transforms.yaml`.
+2. Publish a GitHub Release tagged `vX.Y.Z` matching that version.
+
+The `release` workflow generates, builds, and publishes to PyPI via Trusted
+Publishing (OIDC). It fails if the tag and `build.version` disagree.

pubmed_proto-0.1.0/README.md

@@ -0,0 +1,90 @@
+# pubmed-proto
+
+Generates **`pubmed_proto`** — a typed Python package for parsing NLM PubMed XML
+into protobuf and pydantic models — from the PubMed DTD.
+
+This repository is the **generator**, not the package. It holds the inputs
+(`pubmed.dtd`, `pubmed_transforms.yaml`) and drives
+[`xsd-former`](https://github.com/populationgenomics/xsd-former) (the
+`xsdformer` CLI) to emit the `pubmed_proto` source tree, which is then built into
+a wheel and published to PyPI. The generated tree (`generated/`) and build
+outputs (`dist/`) are gitignored — only the inputs are version-controlled.
+
+## Consuming `pubmed_proto`
+
+Depend on the published wheel, not this repo:
+
+```
+pip install pubmed_proto      # or: uv add pubmed_proto
+```
+
+```python
+from lxml import etree
+from pubmed_proto import xml_converter, pydantic_converter, models
+
+tree = etree.parse("efetch_output.xml")
+article_el = tree.getroot().find("PubmedArticle")
+
+proto = xml_converter.PubmedArticle(article_el)     # XML  -> protobuf
+model = pydantic_converter.PubmedArticle_from_proto(proto)  # protobuf -> pydantic
+json_str = model.model_dump_json()                  # pydantic -> JSON
+```
+
+The package exposes four modules (all typed; ships `py.typed`):
+
+| module               | purpose                                            |
+| -------------------- | -------------------------------------------------- |
+| `pubmed_pb2`         | compiled protobuf messages (`Article`, `Author`, …) |
+| `models`             | pydantic models mirroring the protobuf schema       |
+| `xml_converter`      | PubMed XML → protobuf (per-message factory funcs)   |
+| `pydantic_converter` | protobuf ↔ pydantic (`X_from_proto` / `X_to_proto`) |
+
+## Developing the generator
+
+Requires [`uv`](https://docs.astral.sh/uv/).
+
+```
+make generate   # DTD + transforms -> generated/pubmed_proto/
+make build      # generate, then build the wheel into dist/
+make clean      # remove generated/ and dist/
+uv run --group test pytest   # round-trip gate over real PubMed records
+```
+
+Shaping the output is done in **`pubmed_transforms.yaml`** — dropping
+book/admin types, flattening list wrappers, coercing booleans/timestamps, and
+serializing rich-text fields to markdown. See the
+[`xsd-former`](https://github.com/populationgenomics/xsd-former) docs for the
+transform reference.
+
+## Provenance & attribution
+
+`pubmed.dtd` is derived from the **U.S. National Library of Medicine PubMed
+DTD**, version `pubmed_250101` (dated 2024-08-28):
+
+<https://dtd.nlm.nih.gov/ncbi/pubmed/out/pubmed_250101.dtd>
+
+NLM DTDs are U.S. Government works and public domain in the United States; the
+MIT `LICENSE` in this repo covers CPG's own files (transforms, generator wiring,
+tests), not the NLM DTD.
+
+**Local modification:** the external MathML module include
+(`<!ENTITY % mathml-in-pubmed SYSTEM "mathml-in-pubmed.mod">` and its reference)
+was removed so the DTD is self-contained for schema generation — MathML markup
+in titles/abstracts is not modelled. With that one include removed, the file is
+byte-identical to upstream `pubmed_250101`.
+
+The DTD is **vendored deliberately, not fetched at build time**: it's a modified
+derivative (so a fetch wouldn't reproduce it), and pinning the exact bytes keeps
+the generated schema reproducible. The full provenance also lives in a comment
+at the top of `pubmed.dtd`.
+
+## Releasing
+
+The published version is **`build.version` in `pubmed_transforms.yaml`** (what
+`xsdformer` stamps into the wheel). To release:
+
+1. Bump `build.version` in `pubmed_transforms.yaml`.
+2. Publish a GitHub Release tagged `vX.Y.Z` matching that version.
+
+The `release` workflow generates, builds, and publishes to PyPI via Trusted
+Publishing (OIDC). It fails if the tag and `build.version` disagree.

pubmed_proto-0.1.0/pubmed_proto/__init__.py

@@ -0,0 +1,4 @@
+from pubmed_proto import pubmed_pb2 as pubmed_pb2
+from pubmed_proto import models as models
+from pubmed_proto import pydantic_converter as pydantic_converter
+from pubmed_proto import xml_converter as xml_converter