sciwrite-lint 0.3.0__tar.gz → 0.5.0__tar.gz

{sciwrite_lint-0.3.0 → sciwrite_lint-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sciwrite-lint
-Version: 0.3.0
+Version: 0.5.0
 Summary: A linter for scientific manuscripts — reference verification, consistency checking, and structural validation.
 Author: Sergey Samsonau
 Maintainer-email: Authentic Research Partners LLC <info@arpconnect.com>
@@ -48,9 +48,20 @@ Requires-Dist: pdf2image>=1.17
 Requires-Dist: lxml>=6.1
 Requires-Dist: pypandoc-binary>=1.15
 Requires-Dist: rfc3986>=2.0
+Requires-Dist: psutil>=5.9
+Requires-Dist: annotated-types>=0.6
 Provides-Extra: dev
 Requires-Dist: pytest>=9.0.3; extra == "dev"
 Requires-Dist: pytest-xdist>=3.0; extra == "dev"
+Requires-Dist: ruff>=0.15; extra == "dev"
+Requires-Dist: mypy>=1.18; extra == "dev"
+Requires-Dist: ast-grep-cli>=0.35; extra == "dev"
+Requires-Dist: deptry>=0.25; extra == "dev"
+Requires-Dist: bandit>=1.9; extra == "dev"
+Requires-Dist: pip-audit>=2.10; extra == "dev"
+Requires-Dist: pip-licenses>=5.5; extra == "dev"
+Requires-Dist: twine>=6.0; extra == "dev"
+Requires-Dist: build>=1.0; extra == "dev"
 Dynamic: license-file
 
 # sciwrite-lint
@@ -136,22 +147,22 @@ sciwrite-lint contributions paper.pdf --format json
 
 ### Optimizations
 
-Three models — a vision model (Qwen3-VL-2B default, or 8B FP8 via `--vision-backend vllm` for +15% accuracy), an embedding model (Arctic Embed), and an 8B reasoning LLM (Qwen3 via vLLM) — share a single consumer GPU. Models run in separate pipeline stages; on WSL2, CUDA memory virtualization pages idle allocations to system RAM, letting all three share physical VRAM. FP8 weights and KV cache (Ada Lovelace+) and per-paper SQLite caching with hash-based invalidation are baseline. On top of that:
+Three models — a vision model (Qwen3-VL-2B default, or 8B FP8 via `--vision-backend vllm` for +15% accuracy), an embedding model (Arctic Embed), and an 8B reasoning LLM (Qwen3 via vLLM) — share a single consumer GPU. The pipeline runs each in its own stage and explicitly stops one before starting the next, so only one ever owns GPU memory at a time (same code path on WSL2 and native Linux). FP8 weights and KV cache (Ada Lovelace+) and per-paper SQLite caching are baseline; every cache layer pins its inputs by hash so editing the paper, replacing a figure, or refreshing a reference recomputes only what's affected (see the *Iterative editing* subsection under Usage — `--fresh` is not part of the normal loop). On top of that:
 
-- **Semantic section filtering** — embedding-based KNN retrieval sends only the ~5 most relevant sections per claim to the LLM, reducing LLM calls ~4x
+- **Cost-aware verify-claim ladder** — sentence chunk → paragraph chunk → whole section. Each level fans out top-N candidates in parallel; stops on a conclusive verdict. Most claims resolve at the cheap sentence level (~200-token prompts) instead of paying for whole sections (~5K tokens), with full-section escalation reserved for hard cases
 - **Prefix-first prompt structure** — shared context placed before variable content in all prompts, maximizing vLLM's automatic prefix caching
-- **Per-call-site thinking budgets** — each LLM call site has empirically tuned (max_tokens, thinking_preset) pairs, measured via grid search to maximize detection quality
+- **Per-call-site thinking budgets** — each LLM call site has its own (max_tokens, thinking_preset) pair tuned for that check's prompt and output shape
 - **Adaptive embedding batches** — token-aware batch sizing gives ~50x speedup over CPU while staying within VRAM limits
 - **Batch-staged multi-paper pipeline** — when checking 2+ papers, GPU models load once per batch (vision/embedding/cited-vision) and vLLM/network stages run concurrently, giving a meaningful speedup over sequential per-paper runs. Tune via `--concurrency` (default 2, validated up to 4 on a single consumer GPU)
 - **Phased API resolution** — citations flow through OpenAlex → Semantic Scholar → CrossRef → Open Library/LoC, each phase only processing what previous phases didn't resolve
 - **14-source full-text cascade** — early exit on first successful download across arXiv, Semantic Scholar, OpenAlex, PubMed Central, Europe PMC, Unpaywall, bioRxiv, NBER, RePEc/IDEAS, HAL, ERIC, NASA ADS, OSF Preprints, CORE
 - **Live monitoring** *(advanced)* — `sciwrite-lint containers monitor` shows service health, VRAM usage, and KV cache utilization in a terminal dashboard
 
-Full pipeline on a 50-reference paper: ~30 minutes initial (dominated by downloads and claim verification), minutes on cached reruns. On native Linux, the pipeline automatically swaps vLLM containers to free GPU for embedding and vision stages (~50x faster than CPU). *(Native Linux GPU swap is preliminary — tested on WSL2 only; expected to work, may need minor fixes.)*
+Full pipeline on a 50-reference paper: ~30 minutes initial (dominated by downloads and claim verification), minutes on cached reruns. The pipeline automatically swaps vLLM containers to free GPU for embedding and vision stages (~50× faster than CPU) — same code path on WSL2 and native Linux.
 
 ## Install
 
-**Assumed setup:** A workstation with an NVIDIA GPU (16+ GB VRAM). Developed and tested on Windows (WSL2). Native Linux is likely to work with GPU memory allocation tuning (see [docs/services.md](https://github.com/authentic-research-partners/sciwrite-lint/blob/main/docs/services.md#embedding-device)). Not tested on macOS.
+**Assumed setup:** A workstation with an NVIDIA GPU (16+ GB VRAM). Tested on WSL2 with NVIDIA driver 546.01+. Native Linux is likely to work with GPU memory allocation tuning (see [docs/services.md](https://github.com/authentic-research-partners/sciwrite-lint/blob/main/docs/services.md#embedding-device)). Not tested on macOS.
 
 Requires [uv](https://docs.astral.sh/uv/getting-started/installation/), a container runtime (podman or docker), CUDA drivers, and [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
 
@@ -196,6 +207,14 @@ sciwrite-lint containers start         # start GROBID + vLLM (needs GPU for vLLM
 sciwrite-lint containers monitor       # live dashboard: service health, VRAM, KV cache
 ```
 
+### WSL2: disable CUDA Sysmem Fallback (recommended)
+
+By default Windows silently spills GPU overflow into system RAM via PCIe (30–100× slower than VRAM); you'll see this as a sudden throughput collapse during the heaviest pipeline stages with no error in the logs. Tell the NVIDIA driver to fail loudly instead:
+
+> NVIDIA Control Panel → 3D Settings → Manage 3D Settings → *CUDA - Sysmem Fallback Policy* → **Prefer No Sysmem Fallback** → Apply.
+
+Driver 546.01+. After this, WSL2 behaves like native Linux on overflow (CUDA OOM error). The pipeline already sequences GPU consumers so OOM should not happen in normal use; the setting just prevents silent slowdowns when something does go wrong. No setting needed on native Linux — `cudaMalloc` is already loud there.
+
 ![Monitor dashboard](https://github.com/authentic-research-partners/sciwrite-lint/raw/main/docs/monitor.png)
 
 `init` detects `.tex` files and their `.bib` references and generates a `.sciwrite-lint.toml` config. Review to confirm the right files were detected.
@@ -263,7 +282,18 @@ sciwrite-lint contributions paper.pdf           # standalone file scoring
 
 `check` runs the full pipeline in one command: text checks → figure analysis → LLM consistency → reference verification via APIs → download and parse cited papers → claim verification → consistency checks on cited papers → bibliography verification → aggregate reliability scores → SciLint Score. An initial run on a 50-reference paper takes up to 30 minutes (dominated by downloads and claim verification); subsequent cached runs complete in minutes.
 
-Use `--fresh` to start from scratch (backs up the existing workspace before recreating it).
+### Iterative editing — no `--fresh` needed
+
+`sciwrite-lint check` invalidates cached results whenever the inputs they were derived from change, so rerunning after an edit just works. Concretely:
+
+- **Edit a sentence or paragraph** → the manuscript-LLM checks (prose-quality, cross-section consistency, etc.) re-run only on the touched sentences and their paragraph siblings; untouched prose stays cached
+- **Edit a paragraph around a citation** → the claim verifier reruns on the edited prose
+- **Replace a figure** (overwrite the PNG/PDF in your tree) → the vision cache re-describes it
+- **Swap the local LLM model** (e.g. `qwen3` ↔ `gemma3` via `--model`) → manuscript-check rows and claim verdicts no longer count as cache hits
+- **Overwrite a reference PDF** in your drop folder, or **re-fetch** an open-access PDF that has new bytes → the parser, embeddings, and claim verifier all rerun against the new content
+- **Upgrade a summary `.md` to a real PDF** in your drop folder → claims are re-verified against the richer evidence
+
+Use `--fresh` only when you want to wipe the workspace deliberately (it backs up the existing workspace before recreating it). It is not the answer to "I edited the paper and a finding looks stale" — if you ever see that, file an issue; a cache invariant is wrong and we'll fix it.
 
 Use `--checks ID[,ID...]` to run only the listed checks (comma-separated), disabling the rest for this invocation. Useful for fast iteration during editing (`--checks prose-quality` runs just the prose review), for debugging a single check, or for scripting a two-pass workflow. Unknown check IDs fail loudly — run `sciwrite-lint checks` to list available IDs.
 

sciwrite_lint-0.3.0/sciwrite_lint.egg-info/PKG-INFO → sciwrite_lint-0.5.0/README.md

@@ -1,58 +1,3 @@
-Metadata-Version: 2.4
-Name: sciwrite-lint
-Version: 0.3.0
-Summary: A linter for scientific manuscripts — reference verification, consistency checking, and structural validation.
-Author: Sergey Samsonau
-Maintainer-email: Authentic Research Partners LLC <info@arpconnect.com>
-License-Expression: MIT
-Project-URL: Homepage, https://arpconnect.com
-Project-URL: Repository, https://github.com/authentic-research-partners/sciwrite-lint
-Project-URL: Documentation, https://github.com/authentic-research-partners/sciwrite-lint#readme
-Project-URL: Issues, https://github.com/authentic-research-partners/sciwrite-lint/issues
-Project-URL: Paper, https://arxiv.org/abs/2604.08501
-Keywords: linter,latex,bibtex,pdf,scientific-writing,academic-writing,citation-checker,reference-verification,claim-verification,manuscript,peer-review,arxiv,vllm,qwen
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: Topic :: Scientific/Engineering
-Classifier: Topic :: Software Development :: Quality Assurance
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Typing :: Typed
-Requires-Python: <3.14,>=3.13
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: httpx>=0.28
-Requires-Dist: tenacity>=9.0
-Requires-Dist: trafilatura>=2.0
-Requires-Dist: aiosqlite>=0.22
-Requires-Dist: loguru>=0.7
-Requires-Dist: anyascii>=0.3
-Requires-Dist: rich>=14.0
-Requires-Dist: defusedxml>=0.7
-Requires-Dist: pydantic>=2.10
-Requires-Dist: rapidfuzz>=3.0
-Requires-Dist: bibtexparser>=1.4
-Requires-Dist: pdfplumber>=0.11
-Requires-Dist: openai>=2.0
-Requires-Dist: grobid-tei-xml>=0.1
-Requires-Dist: sentence-transformers>=5.0
-Requires-Dist: numpy>=2.0
-Requires-Dist: sqlite-vec>=0.1.6
-Requires-Dist: torch>=2.5
-Requires-Dist: transformers>=4.50
-Requires-Dist: Pillow>=12.2
-Requires-Dist: pypdf>=6.10.2
-Requires-Dist: pypdfium2>=5.0
-Requires-Dist: pdf2image>=1.17
-Requires-Dist: lxml>=6.1
-Requires-Dist: pypandoc-binary>=1.15
-Requires-Dist: rfc3986>=2.0
-Provides-Extra: dev
-Requires-Dist: pytest>=9.0.3; extra == "dev"
-Requires-Dist: pytest-xdist>=3.0; extra == "dev"
-Dynamic: license-file
-
 # sciwrite-lint
 
 [![arXiv](https://img.shields.io/badge/arXiv-2604.08501-b31b1b.svg)](https://arxiv.org/abs/2604.08501)
@@ -136,22 +81,22 @@ sciwrite-lint contributions paper.pdf --format json
 
 ### Optimizations
 
-Three models — a vision model (Qwen3-VL-2B default, or 8B FP8 via `--vision-backend vllm` for +15% accuracy), an embedding model (Arctic Embed), and an 8B reasoning LLM (Qwen3 via vLLM) — share a single consumer GPU. Models run in separate pipeline stages; on WSL2, CUDA memory virtualization pages idle allocations to system RAM, letting all three share physical VRAM. FP8 weights and KV cache (Ada Lovelace+) and per-paper SQLite caching with hash-based invalidation are baseline. On top of that:
+Three models — a vision model (Qwen3-VL-2B default, or 8B FP8 via `--vision-backend vllm` for +15% accuracy), an embedding model (Arctic Embed), and an 8B reasoning LLM (Qwen3 via vLLM) — share a single consumer GPU. The pipeline runs each in its own stage and explicitly stops one before starting the next, so only one ever owns GPU memory at a time (same code path on WSL2 and native Linux). FP8 weights and KV cache (Ada Lovelace+) and per-paper SQLite caching are baseline; every cache layer pins its inputs by hash so editing the paper, replacing a figure, or refreshing a reference recomputes only what's affected (see the *Iterative editing* subsection under Usage — `--fresh` is not part of the normal loop). On top of that:
 
-- **Semantic section filtering** — embedding-based KNN retrieval sends only the ~5 most relevant sections per claim to the LLM, reducing LLM calls ~4x
+- **Cost-aware verify-claim ladder** — sentence chunk → paragraph chunk → whole section. Each level fans out top-N candidates in parallel; stops on a conclusive verdict. Most claims resolve at the cheap sentence level (~200-token prompts) instead of paying for whole sections (~5K tokens), with full-section escalation reserved for hard cases
 - **Prefix-first prompt structure** — shared context placed before variable content in all prompts, maximizing vLLM's automatic prefix caching
-- **Per-call-site thinking budgets** — each LLM call site has empirically tuned (max_tokens, thinking_preset) pairs, measured via grid search to maximize detection quality
+- **Per-call-site thinking budgets** — each LLM call site has its own (max_tokens, thinking_preset) pair tuned for that check's prompt and output shape
 - **Adaptive embedding batches** — token-aware batch sizing gives ~50x speedup over CPU while staying within VRAM limits
 - **Batch-staged multi-paper pipeline** — when checking 2+ papers, GPU models load once per batch (vision/embedding/cited-vision) and vLLM/network stages run concurrently, giving a meaningful speedup over sequential per-paper runs. Tune via `--concurrency` (default 2, validated up to 4 on a single consumer GPU)
 - **Phased API resolution** — citations flow through OpenAlex → Semantic Scholar → CrossRef → Open Library/LoC, each phase only processing what previous phases didn't resolve
 - **14-source full-text cascade** — early exit on first successful download across arXiv, Semantic Scholar, OpenAlex, PubMed Central, Europe PMC, Unpaywall, bioRxiv, NBER, RePEc/IDEAS, HAL, ERIC, NASA ADS, OSF Preprints, CORE
 - **Live monitoring** *(advanced)* — `sciwrite-lint containers monitor` shows service health, VRAM usage, and KV cache utilization in a terminal dashboard
 
-Full pipeline on a 50-reference paper: ~30 minutes initial (dominated by downloads and claim verification), minutes on cached reruns. On native Linux, the pipeline automatically swaps vLLM containers to free GPU for embedding and vision stages (~50x faster than CPU). *(Native Linux GPU swap is preliminary — tested on WSL2 only; expected to work, may need minor fixes.)*
+Full pipeline on a 50-reference paper: ~30 minutes initial (dominated by downloads and claim verification), minutes on cached reruns. The pipeline automatically swaps vLLM containers to free GPU for embedding and vision stages (~50× faster than CPU) — same code path on WSL2 and native Linux.
 
 ## Install
 
-**Assumed setup:** A workstation with an NVIDIA GPU (16+ GB VRAM). Developed and tested on Windows (WSL2). Native Linux is likely to work with GPU memory allocation tuning (see [docs/services.md](https://github.com/authentic-research-partners/sciwrite-lint/blob/main/docs/services.md#embedding-device)). Not tested on macOS.
+**Assumed setup:** A workstation with an NVIDIA GPU (16+ GB VRAM). Tested on WSL2 with NVIDIA driver 546.01+. Native Linux is likely to work with GPU memory allocation tuning (see [docs/services.md](https://github.com/authentic-research-partners/sciwrite-lint/blob/main/docs/services.md#embedding-device)). Not tested on macOS.
 
 Requires [uv](https://docs.astral.sh/uv/getting-started/installation/), a container runtime (podman or docker), CUDA drivers, and [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
 
@@ -196,6 +141,14 @@ sciwrite-lint containers start         # start GROBID + vLLM (needs GPU for vLLM
 sciwrite-lint containers monitor       # live dashboard: service health, VRAM, KV cache
 ```
 
+### WSL2: disable CUDA Sysmem Fallback (recommended)
+
+By default Windows silently spills GPU overflow into system RAM via PCIe (30–100× slower than VRAM); you'll see this as a sudden throughput collapse during the heaviest pipeline stages with no error in the logs. Tell the NVIDIA driver to fail loudly instead:
+
+> NVIDIA Control Panel → 3D Settings → Manage 3D Settings → *CUDA - Sysmem Fallback Policy* → **Prefer No Sysmem Fallback** → Apply.
+
+Driver 546.01+. After this, WSL2 behaves like native Linux on overflow (CUDA OOM error). The pipeline already sequences GPU consumers so OOM should not happen in normal use; the setting just prevents silent slowdowns when something does go wrong. No setting needed on native Linux — `cudaMalloc` is already loud there.
+
 ![Monitor dashboard](https://github.com/authentic-research-partners/sciwrite-lint/raw/main/docs/monitor.png)
 
 `init` detects `.tex` files and their `.bib` references and generates a `.sciwrite-lint.toml` config. Review to confirm the right files were detected.
@@ -263,7 +216,18 @@ sciwrite-lint contributions paper.pdf           # standalone file scoring
 
 `check` runs the full pipeline in one command: text checks → figure analysis → LLM consistency → reference verification via APIs → download and parse cited papers → claim verification → consistency checks on cited papers → bibliography verification → aggregate reliability scores → SciLint Score. An initial run on a 50-reference paper takes up to 30 minutes (dominated by downloads and claim verification); subsequent cached runs complete in minutes.
 
-Use `--fresh` to start from scratch (backs up the existing workspace before recreating it).
+### Iterative editing — no `--fresh` needed
+
+`sciwrite-lint check` invalidates cached results whenever the inputs they were derived from change, so rerunning after an edit just works. Concretely:
+
+- **Edit a sentence or paragraph** → the manuscript-LLM checks (prose-quality, cross-section consistency, etc.) re-run only on the touched sentences and their paragraph siblings; untouched prose stays cached
+- **Edit a paragraph around a citation** → the claim verifier reruns on the edited prose
+- **Replace a figure** (overwrite the PNG/PDF in your tree) → the vision cache re-describes it
+- **Swap the local LLM model** (e.g. `qwen3` ↔ `gemma3` via `--model`) → manuscript-check rows and claim verdicts no longer count as cache hits
+- **Overwrite a reference PDF** in your drop folder, or **re-fetch** an open-access PDF that has new bytes → the parser, embeddings, and claim verifier all rerun against the new content
+- **Upgrade a summary `.md` to a real PDF** in your drop folder → claims are re-verified against the richer evidence
+
+Use `--fresh` only when you want to wipe the workspace deliberately (it backs up the existing workspace before recreating it). It is not the answer to "I edited the paper and a finding looks stale" — if you ever see that, file an issue; a cache invariant is wrong and we'll fix it.
 
 Use `--checks ID[,ID...]` to run only the listed checks (comma-separated), disabling the rest for this invocation. Useful for fast iteration during editing (`--checks prose-quality` runs just the prose review), for debugging a single check, or for scripting a two-pass workflow. Unknown check IDs fail loudly — run `sciwrite-lint checks` to list available IDs.
 

{sciwrite_lint-0.3.0 → sciwrite_lint-0.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "sciwrite-lint"
-version = "0.3.0"
+version = "0.5.0"
 description = "A linter for scientific manuscripts — reference verification, consistency checking, and structural validation."
 requires-python = ">=3.13,<3.14"
 license = "MIT"
@@ -49,6 +49,8 @@ dependencies = [
     "lxml>=6.1",
     "pypandoc-binary>=1.15",
     "rfc3986>=2.0",
+    "psutil>=5.9",
+    "annotated-types>=0.6",
 ]
 
 [project.urls]
@@ -59,9 +61,23 @@ Issues = "https://github.com/authentic-research-partners/sciwrite-lint/issues"
 Paper = "https://arxiv.org/abs/2604.08501"
 
 [project.optional-dependencies]
+# `pip install -e ".[dev]"` brings in every tool used by the mandatory checks
+# and the release pipeline. Exception: the `claude` CLI is a Node app —
+# install separately per https://docs.anthropic.com/en/docs/claude-code.
 dev = [
+    # Mandatory checks — run after every code change
     "pytest>=9.0.3",
     "pytest-xdist>=3.0",
+    "ruff>=0.15",
+    "mypy>=1.18",
+    "ast-grep-cli>=0.35",
+    # Release pipeline (scripts/release.sh phases 3–6)
+    "deptry>=0.25",
+    "bandit>=1.9",
+    "pip-audit>=2.10",
+    "pip-licenses>=5.5",
+    "twine>=6.0",
+    "build>=1.0",
 ]
 
 [project.scripts]
@@ -70,44 +86,5 @@ sciwrite-lint = "sciwrite_lint.__main__:main"
 [tool.setuptools.packages.find]
 include = ["sciwrite_lint*"]
 
-[tool.pytest.ini_options]
-testpaths = ["tests"]
-pythonpath = ["."]
-markers = [
-    "integration: tests that read actual paper files",
-    "network: tests that make real network connections (slow)",
-]
-# PID-unique basetemp set in conftest.py — prevents FileExistsError
-# when multiple pytest processes run concurrently.
-addopts = "-n auto -p no:cacheprovider"
-
-[tool.bandit]
-skips = [
-    "B101",  # assert — used for type narrowing and internal invariants
-    "B104",  # bind 0.0.0.0 — container start commands for GROBID/vLLM
-    "B314",  # xml.etree.fromstring — parsing trusted GROBID TEI output
-    "B404",  # subprocess import — CLI tool, subprocess is core functionality
-    "B405",  # xml.etree import — parsing trusted GROBID TEI output
-    "B603",  # subprocess call — CLI invokes grobid/vllm/claude by design
-    "B607",  # partial path — CLI tools resolved via PATH
-    "B608",  # SQL string — internal usage.db, no user input in queries
-    "B615",  # HF from_pretrained without revision pin — trusted publisher (Qwen), research tool
-]
-
-[tool.deptry]
-optional_dependencies_dev_groups = ["dev"]
-
-[tool.deptry.per_rule_ignores]
-# Self-imports of shipped non-package directories
-DEP001 = ["evals", "eval_real_world", "bench"]
-# Lazy/conditional imports in optional groups — deptry can't detect these
-DEP002 = ["bibtexparser", "pdfplumber", "sentence-transformers", "numpy", "sqlite-vec", "grobid-tei-xml", "torch", "transformers", "Pillow", "pypdf", "pdf2image"]
-# Intra-package imports
-DEP003 = ["sciwrite_lint", "evals", "eval_real_world"]
+[tool.setuptools.package-data]
 
-[tool.mypy]
-plugins = ["pydantic.mypy"]
-[[tool.mypy.overrides]]
-# No py.typed marker or typeshed stubs available
-module = ["bibtexparser", "sqlite_vec", "pypdf", "pypdfium2", "pypdfium2.raw", "transformers", "pdf2image", "lxml", "lxml.html", "lxml.html.etree", "rfc3986", "pypandoc"]
-ignore_missing_imports = true

{sciwrite_lint-0.3.0 → sciwrite_lint-0.5.0}/sciwrite_lint/__init__.py

@@ -2,7 +2,7 @@
 
 from sciwrite_lint.exceptions import LLMConnectionError, SciWriteLintError
 
-__version__ = "0.3.0"
+__version__ = "0.5.0"
 
 from sciwrite_lint.oa import (
     DownloadResult,

{sciwrite_lint-0.3.0 → sciwrite_lint-0.5.0}/sciwrite_lint/__main__.py

@@ -14,113 +14,55 @@ import argparse
 import sys
 from pathlib import Path
 
-from loguru import logger
-
 from sciwrite_lint import __version__
-from sciwrite_lint.config import LintConfig, PaperConfig, load_config
+from sciwrite_lint.cli._common import (
+    _classify_verify_issue as _classify_verify_issue,
+    _load_config as _load_config,
+    _paper_names as _paper_names,
+    _resolve_input_files as _resolve_input_files,
+    _resolve_paper as _resolve_paper,
+    _setup_logging,
+)
+from sciwrite_lint.config import LintConfig, load_config
 from sciwrite_lint.vllm.vllm_server import MODELS as _VLLM_MODELS
 
 _ALL_MODELS = sorted(_VLLM_MODELS.keys())
 _TEXT_MODELS = [k for k, v in _VLLM_MODELS.items() if v["kind"] == "text"]
 
+_REPLACE_HELP = (
+    "Stop any other running vLLM container first (two vLLMs can't "
+    "share the GPU safely; without this flag the start is refused)"
+)
+
 
 # ---------------------------------------------------------------------------
-# Helpers (used by CLI submodules)
+# Main parser
 # ---------------------------------------------------------------------------
 
 
-def _load_config(args: argparse.Namespace) -> LintConfig:
-    """Load config from --config flag or auto-discovery."""
-    if getattr(args, "config", None):
-        return load_config(Path(args.config))
-
-    config = load_config(None)
-    if config.config_path is None:
-        from sciwrite_lint.config import _detect_papers
-
-        logger.error("No .sciwrite-lint.toml found.")
-        detected = _detect_papers()
-        if detected:
-            logger.error("  Detected .tex files:")
-            for p in detected:
-                bib = f" (bib: {p['bib']})" if p.get("bib") else ""
-                logger.error(f"    {p['file_path']}{bib}")
-        logger.error("  Run: sciwrite-lint init")
-        logger.error("  Then review .sciwrite-lint.toml before running checks.")
-    return config
-
-
-def _resolve_paper(config: LintConfig, name: str) -> PaperConfig | None:
-    """Resolve a paper name to its config. Print error if not found."""
-    pc = config.get_paper(name)
-    if not pc:
-        if config.papers:
-            names = ", ".join(p.name for p in config.papers)
-            logger.error(f"Unknown paper '{name}'. Registered: {names}")
-        else:
-            logger.error(f"Unknown paper '{name}'. No papers registered.")
-            logger.error(
-                f"  Add [[papers]] to {config.config_path or '.sciwrite-lint.toml'}"
-            )
-    return pc
-
-
-def _paper_names(config: LintConfig) -> list[str]:
-    return [p.name for p in config.papers]
-
+def _register_optional_extensions(
+    sub: argparse._SubParsersAction,  # type: ignore[type-arg]
+) -> None:
+    """Load CLI extensions discovered under ``sciwrite_lint``.
 
-def _resolve_input_files(
-    args: argparse.Namespace, config: LintConfig
-) -> list[tuple[str, Path]]:
-    """Resolve which files to check (.tex or .pdf).
-
-    Priority: positional file > --paper (from config) > all papers in config.
-    Returns list of (name, path) pairs.
+    An underscore-prefixed subpackage is a CLI extension when its
+    ``__init__.py`` defines ``register_cli(sub)``. Discovery walks
+    ``sciwrite_lint.__path__`` via ``pkgutil``, so no extension package
+    names appear in this source — adding or removing one is a filesystem
+    change.
     """
-    if hasattr(args, "file") and args.file:
-        p = Path(args.file)
-        return [(p.stem, p)]
-
-    paper_filter = getattr(args, "paper", None)
-    if paper_filter:
-        pc = _resolve_paper(config, paper_filter)
-        if not pc:
-            return []
-        return [(pc.name, pc.file_path)]
-
-    if config.papers:
-        return [(pc.name, pc.file_path) for pc in config.papers]
-
-    logger.error("No papers registered. Either:")
-    logger.error("  sciwrite-lint check <file.tex|file.pdf> — check a specific file")
-    logger.error(
-        "  sciwrite-lint init                      — set up project with [[papers]]"
-    )
-    return []
-
+    import importlib
+    import pkgutil
 
-def _classify_verify_issue(issue: str) -> tuple[str, str]:
-    """Classify a verify issue string into (level, rule_id) for findings."""
-    from sciwrite_lint.pipeline import _classify_verify_issue as _classify
+    import sciwrite_lint as _pkg
 
-    return _classify(issue)
-
-
-def _setup_logging(config: LintConfig) -> None:
-    """Configure loguru rotating file sink from config."""
-    logger.add(
-        "logs/sciwrite-lint.log",
-        rotation="10 MB",
-        retention="30 days",
-        compression="gz",
-        level=config.log_level,
-        format="{time:YYYY-MM-DD HH:mm:ss} | {level:<8} | {name}:{function}:{line} | {message}",
-    )
-
-
-# ---------------------------------------------------------------------------
-# Main parser
-# ---------------------------------------------------------------------------
+    for _finder, name, ispkg in pkgutil.iter_modules(_pkg.__path__):
+        if not ispkg or not name.startswith("_"):
+            continue
+        ext = importlib.import_module(f"{_pkg.__name__}.{name}")
+        register = getattr(ext, "register_cli", None)
+        if register is not None:
+            register(sub)
 
 
 def main(argv: list[str] | None = None) -> int:
@@ -206,6 +148,9 @@ def main(argv: list[str] | None = None) -> int:
     p_checks = sub.add_parser("checks", help="List all registered checks")
     p_checks.set_defaults(func=run_checks_list)
 
+    # Load optional underscore-prefixed CLI extension subpackages (if any).
+    _register_optional_extensions(sub)
+
     # --- init ---
     p_init = sub.add_parser(
         "init", help="Initialize project: config + references directory"
@@ -309,12 +254,6 @@ def main(argv: list[str] | None = None) -> int:
     # --- verify-claims ---
     p_vc = sub.add_parser("verify-claims", help="Verify claims against cited sources")
     p_vc.add_argument("--paper", required=True, help="Paper to verify")
-    p_vc.add_argument(
-        "--backend",
-        choices=["vllm", "claude"],
-        default="vllm",
-        help="vllm (local LLM, fast) or claude (Claude CLI, deep)",
-    )
     p_vc.add_argument(
         "--model",
         choices=_TEXT_MODELS,
@@ -457,6 +396,7 @@ def main(argv: list[str] | None = None) -> int:
         action="store_true",
         help="Also start vision vLLM (qwen3-vl-8b-fp8 on port 5002)",
     )
+    p_containers.add_argument("--replace", action="store_true", help=_REPLACE_HELP)
     p_containers.add_argument("--config", help="Path to .sciwrite-lint.toml")
     p_containers.set_defaults(func=run_containers)
 
@@ -488,6 +428,7 @@ def main(argv: list[str] | None = None) -> int:
         action="store_true",
         help="Pull latest vLLM image before starting",
     )
+    p_vllm_start.add_argument("--replace", action="store_true", help=_REPLACE_HELP)
     p_vllm_start.add_argument("--config", help="Path to .sciwrite-lint.toml")
     p_vllm_start.set_defaults(func=run_vllm)
 
@@ -536,14 +477,32 @@ def main(argv: list[str] | None = None) -> int:
         p_vllm.print_help()
         return 0
 
-    # Set up file logging from config (stderr sink stays at INFO by default)
-    try:
-        config = load_config(
-            Path(args.config) if getattr(args, "config", None) else None
-        )
-    except Exception as e:
-        logger.debug("Config load failed, using defaults: {}", e)
+    # Set up file logging from config (stderr sink stays at INFO by default).
+    # ``load_config`` already returns ``LintConfig()`` when no config file is
+    # discovered; reaching the exception path means a file WAS found (or
+    # explicitly passed via --config) but failed to parse or validate.
+    # Surface that instead of silently substituting hardcoded defaults — a
+    # broken TOML otherwise produces results computed against the wrong
+    # workspace, paper list, or API keys.
+    #
+    # The ``init`` command is the one exception: its job is to scaffold a
+    # config, so it must run against a missing, broken, or sentinel TOML
+    # without aborting. Use ``LintConfig()`` for logging setup only and let
+    # init proceed to (over)write the file.
+    if args.command == "init":
         config = LintConfig()
+    else:
+        try:
+            config = load_config(
+                Path(args.config) if getattr(args, "config", None) else None
+            )
+        except Exception as e:
+            cfg_arg = getattr(args, "config", None)
+            cfg_hint = f" ({cfg_arg})" if cfg_arg else " (.sciwrite-lint.toml)"
+            raise SystemExit(
+                f"ERROR: failed to load configuration{cfg_hint}: {type(e).__name__}: {e}\n"
+                "  Fix the TOML or remove the file to use built-in defaults."
+            ) from e
     _setup_logging(config)
 
     return args.func(args)

{sciwrite_lint-0.3.0 → sciwrite_lint-0.5.0}/sciwrite_lint/api.py

@@ -116,8 +116,14 @@ class CitationAPI:
                 client=self._client,
             )
         except Exception as e:
+            # The waterfall caller distinguishes ``{"error": …}`` from
+            # ``None`` (not-found): error dicts mean "lookup itself
+            # failed" and stay visible, ``None`` means "no result for
+            # this citation". All API lookups in this module return the
+            # error shape so transient network failures don't masquerade
+            # as misses.
             logger.debug("CrossRef lookup failed for {}: {}", citation.key, e)
-            return None
+            return {"error": str(e), "source": "crossref"}
 
     # ------------------------------------------------------------------
     # OpenAlex
@@ -502,6 +508,13 @@ async def batch_openalex(
                 label="OpenAlex batch",
             )
             if resp.status_code != 200:
+                # API failure ≠ "no matches". Log at WARNING so the operator
+                # sees the API was unreachable; the verify-cascade still
+                # falls through to S2/CrossRef per the pipeline contract.
+                logger.warning(
+                    "OpenAlex batch returned status {}; cascading to next provider",
+                    resp.status_code,
+                )
                 return results
             works = resp.json().get("results", [])
             for work in works:
@@ -552,7 +565,7 @@ async def batch_s2(
             async with tracked("semantic_scholar", ids=len(ids)):
                 try:
                     resp = await retry_on_transient(
-                        lambda: client.post(
+                        lambda ids=ids: client.post(
                             "https://api.semanticscholar.org/graph/v1/paper/batch",
                             json={"ids": ids},
                             params={"fields": _S2_FIELDS},
@@ -560,13 +573,27 @@ async def batch_s2(
                         label="S2 batch",
                     )
                     if resp.status_code != 200:
+                        # API failure ≠ "no matches". Log at WARNING so the
+                        # operator sees the API was unreachable.
+                        logger.warning(
+                            "S2 batch returned status {} for {} IDs; "
+                            "cascading to next provider",
+                            resp.status_code,
+                            len(ids),
+                        )
                         continue
                     papers = resp.json()
                     for paper, (key, _s2_id) in zip(papers, batch):
                         if paper:  # S2 returns null for not-found
                             results[key] = _parse_s2(paper)
                 except httpx.HTTPError as e:
-                    logger.debug("S2 batch request failed: {}", e)
+                    # Network error ≠ "no matches". WARNING so the operator
+                    # can see API was unreachable; cascade continues.
+                    logger.warning(
+                        "S2 batch request failed ({}: {}); cascading to next provider",
+                        type(e).__name__,
+                        e,
+                    )
                     continue
 
     return results
@@ -606,10 +633,28 @@ async def parallel_crossref(
                             polite_email=config.polite_email,
                             client=client,
                         )
-                        if result and not result.get("error"):
-                            results[c.key] = result
+                        if result is None:
+                            return  # not-found — fine, next provider handles
+                        if result.get("error"):
+                            # API failure ≠ "no match". Log at WARNING so
+                            # the operator sees CrossRef was unreachable;
+                            # don't propagate the error dict into ``results``
+                            # because callers expect parsed-result shape.
+                            logger.warning(
+                                "CrossRef lookup failed for {}: {}",
+                                c.key,
+                                result["error"],
+                            )
+                            return
+                        results[c.key] = result
                     except Exception as e:
-                        logger.debug("CrossRef lookup failed for {}: {}", c.key, e)
+                        # Network error ≠ "no match". WARNING so visible.
+                        logger.warning(
+                            "CrossRef lookup raised for {} ({}: {})",
+                            c.key,
+                            type(e).__name__,
+                            e,
+                        )
 
         await asyncio.gather(*[_lookup_one(c) for c in citations])
     return results