bmlibrarian-lite 0.1.0__tar.gz

bmlibrarian_lite-0.1.0/.gitignore

@@ -0,0 +1,89 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+env/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+.project
+.pydevproject
+.settings/
+
+# Testing
+.tox/
+.coverage
+.coverage.*
+htmlcov/
+.pytest_cache/
+.mypy_cache/
+
+# Jupyter
+.ipynb_checkpoints/
+
+# Data directories
+data/
+*.db
+*.sqlite
+*.sqlite3
+
+# ChromaDB
+chroma/
+.chroma/
+
+# User data
+~/.bmlibrarian_lite/
+
+# Logs
+*.log
+logs/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Environment
+.env
+.env.local
+*.env
+
+# Secrets
+secrets/
+*.pem
+*.key
+
+# Temporary files
+tmp/
+temp/
+*.tmp
+*.temp
+
+# Lock files (keep uv.lock if using uv)
+# uv.lock
+.claude/settings.local.json

bmlibrarian_lite-0.1.0/CLAUDE.md

@@ -0,0 +1,175 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+BMLibrarian Lite is a lightweight biomedical literature research tool that provides AI-powered systematic review and document interrogation capabilities. It uses ChromaDB + SQLite for storage (no PostgreSQL), FastEmbed for CPU-optimized embeddings, and supports Anthropic Claude or Ollama for LLM inference.
+
+## Common Commands
+
+```bash
+# Create virtual environment and install with uv
+uv venv
+source .venv/bin/activate
+uv pip install -e ".[dev]"
+
+# Run the GUI (default)
+python bmlibrarian_lite.py
+
+# Run tests
+pytest
+
+# Run a single test file
+pytest tests/test_foo.py
+
+# Run a specific test
+pytest tests/test_foo.py::test_function_name
+
+# Run linting
+ruff check .
+
+# Run type checking
+mypy src/
+
+# CLI commands
+python bmlibrarian_lite.py stats        # Storage statistics
+python bmlibrarian_lite.py validate     # Validate configuration
+python bmlibrarian_lite.py config       # Show configuration
+python bmlibrarian_lite.py clear        # Clear all data
+```
+
+## Code Style Requirements
+
+- **Docstrings are mandatory**: Use Google-style docstrings on all public functions, classes, and methods
+- **Type hints are mandatory**: All function parameters and return types must be annotated
+- **No magic numbers**: Always use named constants or configuration values; never hardcode numbers
+- **Prefer pure functions**: Write small, reusable pure functions in focused modules over complex long files
+- **Constants go in `constants.py`**: Centralize numeric and string constants
+- **No inline stylesheets**: All stylesheets via the centralised styling system
+- **No hardcoded pixel values**: Use `scaled()` from `dpi_scale.py` for DPI-aware dimensions
+
+## Architecture
+
+### Entry Points
+- `bmlibrarian_lite.py` - CLI entry point with subcommands (gui, stats, validate, config, clear)
+- `src/bmlibrarian_lite/gui/app.py` - PySide6 main window with tabbed interface
+
+### Core Layers
+
+**Configuration & Storage** (`config.py`, `storage.py`)
+- `LiteConfig` - Dataclass-based configuration with JSON persistence in `~/.bmlibrarian_lite/`
+- `LiteStorage` - Manages ChromaDB collections and SQLite metadata
+
+**LLM Integration** (`llm/`)
+- `LLMClient` - Unified client supporting both Anthropic and Ollama providers
+- Model strings use format: `provider:model` (e.g., `anthropic:claude-sonnet-4-20250514`, `ollama:llama3.2`)
+
+**Agent System** (`agents/`)
+- `LiteBaseAgent` - Base class providing LLM communication via `_chat()` method
+- Specialized agents: `SearchAgent`, `ScoringAgent`, `CitationAgent`, `ReportingAgent`, `InterrogationAgent`
+- Agents inherit config and share LLM client instance
+
+**PubMed Integration** (`pubmed/`)
+- `PubMedSearchClient` - Wrapper around NCBI E-utilities API
+- Converts natural language to PubMed queries
+- Europe PMC full-text XML retrieval support
+
+**PDF Discovery** (`pdf_discovery.py`)
+- `PDFDiscoverer` - Finds and downloads PDFs from PubMed Central, Unpaywall, and DOI resolution
+
+**Quality Assessment** (`quality/`)
+- Study classification, evidence grading, and quality scoring
+- `QualityManager` orchestrates the assessment workflow
+- `QualityAssessment` dataclass with study design, quality tier, and extraction details
+
+### GUI Structure (`gui/`)
+
+The PySide6 GUI uses a three-tab design:
+- `SystematicReviewTab` - Search PubMed, score documents, extract citations, generate reports
+- `DocumentInterrogationTab` - Load documents and perform Q&A
+- `AuditTrailTab` - Real-time workflow visibility (NEW)
+
+#### Audit Trail Tab (NEW)
+
+The Audit Trail provides transparency into the systematic review workflow with three sub-tabs:
+
+- **Queries Tab** (`audit_queries_tab.py`): Displays generated PubMed queries with statistics (docs found, scored, citations extracted)
+- **Literature Tab** (`audit_literature_tab.py`): Scrollable document cards with relevance scores and quality badges
+- **Citations Tab** (`audit_citations_tab.py`): Document cards with highlighted citation passages
+
+**Document Cards** (`document_card.py`):
+- Collapsible cards - click to expand and show abstract
+- Quality badges (RCT, SR, etc.) with color-coded study design
+- Score badges (1-5) with color gradients
+- LLM rationale display for scoring and quality decisions
+- Right-click context menu to send documents to interrogator
+- Pale blue header background for visual distinction
+
+**Key Audit Trail Signals:**
+- `workflow_started` - Clears previous audit data
+- `query_generated(str)` - New PubMed query generated
+- `documents_found(list[LiteDocument])` - Documents retrieved
+- `document_scored(ScoredDocument)` - Relevance score assigned
+- `quality_assessed(str, QualityAssessment)` - Quality assessment complete
+- `citation_extracted(Citation)` - Citation passage extracted
+
+Background operations use `QThread` workers in `workers.py` to keep the UI responsive.
+
+### Data Flow
+
+1. User enters research question → `SearchAgent` converts to PubMed query
+2. `PubMedSearchClient` fetches articles → stored in `LiteStorage`
+3. `LiteEmbedder` (FastEmbed) creates embeddings → stored in ChromaDB
+4. `QualityManager` assesses study quality (optional)
+5. `ScoringAgent` scores relevance → `CitationAgent` extracts citations
+6. `ReportingAgent` generates final report
+7. Audit Trail tab displays real-time progress throughout
+
+## Key Patterns
+
+- **Lazy initialization**: LLM clients and embedders are created on first use
+- **Qt signals/slots**: GUI components communicate via Qt signal system
+- **Dataclasses throughout**: `LiteDocument`, `LiteChunk`, `SearchSession`, etc.
+- **Thread-safe updates**: Use `threading.RLock()` for concurrent access
+- **DPI scaling**: Use `scaled()` function for all pixel dimensions
+
+## Constants (from `constants.py`)
+
+Key audit trail constants:
+- `AUDIT_CARD_PADDING`, `AUDIT_CARD_SPACING` - Card layout dimensions
+- `AUDIT_CARD_BORDER_RADIUS`, `AUDIT_CARD_MIN_HEIGHT` - Card styling
+- `AUDIT_CARD_HEADER_COLOR` - Pale light blue header (`#E3F2FD`)
+- `AUDIT_RATIONALE_COLOR` - Muted text for LLM rationales (`#555555`)
+- `AUDIT_ABSTRACT_MAX_LINES` - Maximum abstract lines before scroll (15)
+- `MAX_AUTHORS_BEFORE_ET_AL` - Authors to show before "et al." (3)
+
+## Environment Variables
+
+- `ANTHROPIC_API_KEY` - Required for Claude API
+- `OLLAMA_HOST` - Ollama server URL (default: http://localhost:11434)
+- `NCBI_EMAIL` - Recommended for PubMed API
+- `TOKENIZERS_PARALLELISM=false` - Set automatically to avoid HuggingFace warnings
+
+## Documentation Structure
+
+```
+doc/
+├── user/           # End-user documentation
+│   └── guide.md    # User guide
+├── developer/      # Developer documentation
+│   └── guide.md    # Developer guide
+├── llm/            # LLM assistant context
+│   ├── context.md  # General LLM context
+│   ├── database-schema.md  # Database schema reference
+│   └── golden_rules.md     # Coding standards
+└── planning/       # Planning documents
+```
+
+## Related Documentation
+
+- `doc/llm/golden_rules.md` - MUST READ: Coding standards and rules
+- `doc/llm/database-schema.md` - Database schema reference
+- `doc/llm/context.md` - Extended LLM context
+- `doc/developer/guide.md` - Developer guide
+- `doc/user/guide.md` - User guide

bmlibrarian_lite-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Horst Herb
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

bmlibrarian_lite-0.1.0/PKG-INFO

@@ -0,0 +1,288 @@
+Metadata-Version: 2.4
+Name: bmlibrarian-lite
+Version: 0.1.0
+Summary: Lightweight biomedical literature research tool - no PostgreSQL required
+Project-URL: Homepage, https://github.com/hherb/bmlibrarian-lite
+Project-URL: Documentation, https://github.com/hherb/bmlibrarian-lite#readme
+Project-URL: Repository, https://github.com/hherb/bmlibrarian-lite.git
+Project-URL: Issues, https://github.com/hherb/bmlibrarian-lite/issues
+Author-email: Horst Herb <bmlibrarian@proton.me>
+License: MIT
+License-File: LICENSE
+Keywords: ai,biomedical,literature,llm,pubmed,research,systematic-review
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Healthcare Industry
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Requires-Python: >=3.12
+Requires-Dist: anthropic>=0.34.0
+Requires-Dist: backoff>=2.2.1
+Requires-Dist: chromadb>=0.5.0
+Requires-Dist: fastembed>=0.3.0
+Requires-Dist: markdown>=3.5.0
+Requires-Dist: ollama>=0.3.0
+Requires-Dist: pygments>=2.17.0
+Requires-Dist: pymupdf>=1.23.0
+Requires-Dist: pysbd>=0.3.4
+Requires-Dist: pyside6>=6.6.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: requests>=2.31.0
+Requires-Dist: tokenizers>=0.19.0
+Provides-Extra: dev
+Requires-Dist: mypy>=1.9.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.4.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# BMLibrarian Lite
+
+A lightweight biomedical literature research tool - no PostgreSQL required.
+
+BMLibrarian Lite is a simplified version of BMLibrarian that provides AI-powered literature search and analysis capabilities without requiring a PostgreSQL database. It uses only the PubMed E-utilities API for searching and fetching article metadata instead of a local database. It uses ChromaDB for vector storage and SQLite for metadata, making it easy to install and use on any machine. Also, in order to allow it to run on computers with limited resources, it uses FastEmbed for local embeddings and allows to use cloud LLM providers like Anthropic Claude instead of relying exclusively on local inference.
+
+
+## Features
+
+- **Systematic Literature Review**: Search PubMed, score documents, extract citations, and generate reports
+- **Document Interrogation**: Interactive Q&A with loaded documents
+- **PDF Discovery & Download**: Automatic PDF retrieval from PubMed Central, Unpaywall, and DOI resolution
+- **Quality Assessment**: Automated study quality evaluation with evidence grading
+- **Audit Trail**: Real-time visibility into the review workflow with LLM reasoning transparency
+- **Multiple LLM Providers**: Support for both Anthropic Claude (online) and Ollama (local)
+- **Embedded Storage**: ChromaDB + SQLite - no external database needed
+- **Cross-Platform GUI**: PySide6-based desktop application
+
+## Quick Start
+
+### Installation
+
+**From PyPI (recommended):**
+
+```bash
+pip install bmlibrarian-lite
+```
+
+**From source:**
+
+```bash
+# Clone the repository
+git clone https://github.com/hherb/bmlibrarian-lite.git
+cd bmlibrarian-lite
+
+# Create virtual environment and install
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+pip install -e .
+```
+
+### Configuration
+
+1. **Set your Anthropic API key** (for Claude):
+   ```bash
+   export ANTHROPIC_API_KEY="your-api-key-here"
+   ```
+
+2. **Or configure Ollama** (for local models):
+   ```bash
+   # Install Ollama: https://ollama.ai
+   ollama pull llama3.2
+   export OLLAMA_HOST="http://localhost:11434"
+   ```
+
+3. **Optional: Set your email for PubMed** (recommended):
+   ```bash
+   export NCBI_EMAIL="your@email.com"
+   ```
+
+4. **Optional: Configure PDF Discovery** (for Unpaywall API):
+   - Go to Settings in the GUI
+   - Enter your email address for Unpaywall API access
+   - Configure OpenAthens if you have institutional access
+
+### Running the Application
+
+```bash
+# Launch the GUI (short command)
+bmll
+
+# Or using the full name
+bmlibrarian-lite
+
+# Or using Python module
+python -m bmlibrarian_lite
+```
+
+### CLI Commands
+
+```bash
+# Show storage statistics
+bmll stats
+
+# Validate configuration
+bmll validate --verbose
+
+# Show current configuration
+bmll config
+
+# Clear all data
+bmll clear
+
+# Show version
+bmll --version
+```
+
+## Usage
+
+### Systematic Review Workflow
+
+1. **Enter your research question** in the main text area
+2. **Click "Search"** to query PubMed
+3. **Review found articles** and adjust filters as needed
+4. **Score documents** for relevance (1-5 scale)
+5. **Extract citations** from high-scoring documents
+6. **Generate a report** synthesizing the evidence
+
+### Audit Trail
+
+The Audit Trail tab provides real-time visibility into the systematic review workflow:
+
+- **Queries Tab**: View generated PubMed queries and search statistics
+- **Literature Tab**: Browse document cards with relevance scores and quality badges
+  - Click cards to expand and view abstracts
+  - See LLM rationales for scoring and quality decisions
+  - Quality badges show study design (RCT, Systematic Review, etc.)
+- **Citations Tab**: View extracted citation passages with highlighting
+
+Right-click any document card to send it to the Document Interrogator for deeper analysis.
+
+### Document Interrogation
+
+1. **Switch to the "Document Interrogation" tab**
+2. **Load a document** (PDF, TXT, or MD file)
+3. **Ask questions** about the document content
+4. **Get AI-powered answers** with source references
+
+## Configuration
+
+Configuration is stored in `~/.bmlibrarian_lite/config.json`:
+
+```json
+{
+  "llm": {
+    "provider": "anthropic",
+    "model": "claude-sonnet-4-20250514",
+    "temperature": 0.7,
+    "max_tokens": 4096
+  },
+  "embeddings": {
+    "model": "BAAI/bge-small-en-v1.5"
+  },
+  "pubmed": {
+    "email": "your@email.com"
+  },
+  "search": {
+    "chunk_size": 512,
+    "chunk_overlap": 50,
+    "similarity_threshold": 0.7,
+    "max_results": 100
+  }
+}
+```
+
+### LLM Providers
+
+**Anthropic Claude** (default):
+```json
+{
+  "llm": {
+    "provider": "anthropic",
+    "model": "claude-sonnet-4-20250514"
+  }
+}
+```
+
+**Ollama** (local):
+```json
+{
+  "llm": {
+    "provider": "ollama",
+    "model": "llama3.2"
+  }
+}
+```
+
+You can also use the model string format: `anthropic:claude-sonnet-4-20250514` or `ollama:llama3.2`
+
+## Requirements
+
+- Python 3.12+
+- Internet connection (for PubMed search and Claude API)
+- ~500MB disk space for embeddings cache
+
+### Dependencies
+
+- **chromadb**: Vector storage
+- **fastembed**: CPU-optimized embeddings
+- **anthropic**: Claude API client
+- **ollama**: Ollama API client
+- **PySide6**: GUI framework
+- **PyMuPDF**: PDF processing
+
+## Differences from Full BMLibrarian
+
+BMLibrarian Lite is designed for ease of use and portability:
+
+| Feature | BMLibrarian | BMLibrarian Lite |
+|---------|-------------|------------------|
+| Database | PostgreSQL + pgvector | ChromaDB + SQLite |
+| Embeddings | Ollama (local) | FastEmbed (CPU) |
+| PDF Discovery | Full (Unpaywall, PMC, OpenAthens) | Included (PMC, Unpaywall, DOI) |
+| Multi-Agent Workflow | Full orchestration | Simplified workflow |
+| Audit Trail | N/A | Real-time workflow visibility |
+| Installation | Complex | Simple `pip install` |
+
+## Documentation
+
+Documentation is organized into three categories:
+
+- **User Documentation** (`doc/user/`): End-user guides and tutorials
+- **Developer Documentation** (`doc/developer/`): Architecture, API, and contribution guides
+- **LLM Context** (`doc/llm/`): Context for AI assistants working with the codebase
+  - `golden_rules.md` - Coding standards
+  - `database-schema.md` - Database schema reference
+
+## Development
+
+```bash
+# Install development dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run linting
+ruff check .
+
+# Run type checking
+mypy src/
+```
+
+## License
+
+MIT License - see LICENSE file for details.
+
+## Acknowledgments
+
+BMLibrarian Lite is derived from [BMLibrarian](https://github.com/hherb/bmlibrarian), a comprehensive biomedical literature research platform.
+
+## Support
+
+- **Issues**: [GitHub Issues](https://github.com/hherb/bmlibrarian-lite/issues)
+- **Documentation**: See the `doc/` directory for detailed guides