gptmed 0.3.4__tar.gz → 0.4.0__tar.gz

{gptmed-0.3.4/gptmed.egg-info → gptmed-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gptmed
-Version: 0.3.4
+Version: 0.4.0
 Summary: A lightweight GPT-based language model framework for training custom question-answering models on any domain
 Author-email: Sanjog Sigdel <sigdelsanjog@gmail.com>
 Maintainer-email: Sanjog Sigdel <sigdelsanjog@gmail.com>
@@ -10,7 +10,7 @@ Project-URL: Documentation, https://github.com/sigdelsanjog/gptmed#readme
 Project-URL: Repository, https://github.com/sigdelsanjog/gptmed
 Project-URL: Issues, https://github.com/sigdelsanjog/gptmed/issues
 Keywords: nlp,language-model,transformer,gpt,pytorch,qa,question-answering,training,deep-learning,custom-model
-Classifier: Development Status :: 3 - Alpha
+Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
 Classifier: Intended Audience :: Education
@@ -38,28 +38,64 @@ Requires-Dist: mypy>=0.950; extra == "dev"
 Provides-Extra: training
 Requires-Dist: tensorboard>=2.10.0; extra == "training"
 Requires-Dist: wandb>=0.13.0; extra == "training"
+Provides-Extra: visualization
+Requires-Dist: matplotlib>=3.5.0; extra == "visualization"
+Requires-Dist: seaborn>=0.12.0; extra == "visualization"
+Provides-Extra: xai
+Requires-Dist: matplotlib>=3.5.0; extra == "xai"
+Requires-Dist: seaborn>=0.12.0; extra == "xai"
+Requires-Dist: captum>=0.6.0; extra == "xai"
+Requires-Dist: scikit-learn>=1.0.0; extra == "xai"
 Dynamic: license-file
 
 # GptMed 🤖
 
-A lightweight GPT-based language model framework for training custom question-answering models on any domain. This package provides a transformer-based GPT architecture that you can train on your own Q&A datasets - whether it's casual conversations, technical support, education, or any other domain.
-
+[![Downloads](https://static.pepy.tech/badge/gptmed)](https://pepy.tech/project/gptmed)
+[![Downloads/Month](https://static.pepy.tech/badge/gptmed/month)](https://pepy.tech/project/gptmed)
 [![PyPI version](https://badge.fury.io/py/gptmed.svg)](https://badge.fury.io/py/gptmed)
 [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-## 📖 [Complete User Manual](USER_MANUAL.md) | [Quick Start](#quick-start)
+A lightweight GPT-based language model framework for training custom question-answering models on any domain. This package provides a transformer-based GPT architecture that you can train on your own Q&A datasets - whether it's casual conversations, technical support, education, or any other domain.
 
-> **New to GptMed?** Check out the [**step-by-step User Manual**](USER_MANUAL.md) for a complete guide on training your own model!
+## Citation
 
-## Features
+If you use this model in your research, please cite:
 
-- 🧠 **Custom GPT Architecture**: Lightweight transformer model for any Q&A domain
-- 🎯 **Domain-Agnostic**: Train on any question-answering dataset (casual chat, tech support, education, etc.)
-- ⚡ **Fast Inference**: Optimized for quick question answering
-- 🔧 **Flexible Training**: Easy to train on your own custom datasets
-- 📦 **Lightweight**: Small model size suitable for edge deployment
-- 🛠️ **Complete Toolkit**: Includes tokenizer training, model training, and inference utilities
+```bibtex
+@software{gptmed_2026,
+  author = {Sanjog Sigdel},
+  title = {GptMed: A custom causal question answering general purpose GPT Transformer Architecture Model},
+  year = {2026},
+  url = {https://github.com/sigdelsanjog/gptmed}
+}
+```
+
+## Table of Contents
+
+- [Installation](#installation)
+  - [From PyPI (Recommended)](#from-pypi-recommended)
+  - [From Source](#from-source)
+  - [With Optional Dependencies](#with-optional-dependencies)
+- [Quick Start](#quick-start)
+  - [Using the High-Level API](#using-the-high-level-api)
+  - [Inference (Generate Answers)](#inference-generate-answers)
+  - [Using Command Line](#using-command-line)
+  - [Training Your Own Model](#training-your-own-model)
+- [Model Architecture](#model-architecture)
+- [Configuration](#configuration)
+  - [Model Sizes](#model-sizes)
+  - [Training Configuration](#training-configuration)
+- [Observability](#observability)
+- [Project Structure](#project-structure)
+- [Requirements](#requirements)
+- [Documentation](#documentation)
+- [Performance](#performance)
+- [Examples](#examples)
+- [Contributing](#contributing)
+- [Citation](#citation)
+- [License](#license)
+- [Support](#support)
 
 ## Installation
 
@@ -83,15 +119,49 @@ pip install -e .
 # For development
 pip install gptmed[dev]
 
-# For training
+# For training with logging integrations
 pip install gptmed[training]
 
+# For visualization (loss curves, metrics plots)
+pip install gptmed[visualization]
+
+# For Explainable AI features
+pip install gptmed[xai]
+
 # All dependencies
-pip install gptmed[dev,training]
+pip install gptmed[dev,training,visualization,xai]
 ```
 
 ## Quick Start
 
+### Using the High-Level API
+
+The easiest way to use GptMed is through the high-level API:
+
+```python
+import gptmed
+
+# 1. Create a training configuration
+gptmed.create_config('my_config.yaml')
+
+# 2. Edit my_config.yaml with your settings (data paths, model size, etc.)
+
+# 3. Train the model
+gptmed.train_from_config('my_config.yaml')
+
+# 4. Generate answers
+answer = gptmed.generate(
+    checkpoint='model/checkpoints/best_model.pt',
+    tokenizer='tokenizer/my_tokenizer.model',
+    prompt='What is machine learning?',
+    max_length=150,
+    temperature=0.7
+)
+print(answer)
+```
+
+For a complete API testing workflow, see the [gptmed-api folder](https://github.com/sigdelsanjog/gptmed/tree/main/gptmed-api) with ready-to-run examples.
+
 ### Inference (Generate Answers)
 
 ```python
@@ -187,6 +257,50 @@ config = TrainingConfig(
 )
 ```
 
+## Observability
+
+**New in v0.4.0**: Built-in training monitoring with Observer Pattern architecture.
+
+### Features
+
+- 📊 **Loss Curves**: Track training/validation loss over time
+- 📈 **Metrics Tracking**: Perplexity, gradient norms, learning rates
+- 🔔 **Callbacks**: Console output, JSON logging, early stopping
+- 📁 **Export**: CSV export, matplotlib visualizations
+- 🔌 **Extensible**: Add custom observers for integrations (W&B, TensorBoard)
+
+### Quick Example
+
+```python
+from gptmed.observability import MetricsTracker, ConsoleCallback, EarlyStoppingCallback
+
+# Create observers
+tracker = MetricsTracker(output_dir='./metrics')
+console = ConsoleCallback(print_every=50)
+early_stop = EarlyStoppingCallback(patience=3)
+
+# Use with TrainingService (automatic)
+from gptmed.services import TrainingService
+service = TrainingService(config_path='config.yaml')
+service.train()  # Automatically creates MetricsTracker
+
+# Or use with Trainer directly
+trainer = Trainer(model, train_loader, config, observers=[tracker, console])
+trainer.train()
+```
+
+### Available Observers
+
+| Observer                | Description                                               |
+| ----------------------- | --------------------------------------------------------- |
+| `MetricsTracker`        | Comprehensive metrics collection with export capabilities |
+| `ConsoleCallback`       | Real-time console output with progress bars               |
+| `JSONLoggerCallback`    | Structured JSON logging for analysis                      |
+| `EarlyStoppingCallback` | Stop training when validation loss plateaus               |
+| `LRSchedulerCallback`   | Learning rate scheduling integration                      |
+
+See [XAI.md](XAI.md) for future Explainable AI features roadmap.
+
 ## Project Structure
 
 ```
@@ -201,10 +315,16 @@ gptmed/
 │   ├── train.py          # Training script
 │   ├── trainer.py        # Training loop
 │   └── dataset.py        # Data loading
+├── observability/         # Training monitoring & XAI (v0.4.0+)
+│   ├── base.py           # Observer pattern interfaces
+│   ├── metrics_tracker.py # Loss curves & metrics
+│   └── callbacks.py      # Console, JSON, early stopping
 ├── tokenizer/
 │   └── train_tokenizer.py # SentencePiece tokenizer
 ├── configs/
 │   └── train_config.py   # Training configurations
+├── services/
+│   └── training_service.py # High-level training orchestration
 └── utils/
     ├── checkpoints.py    # Model checkpointing
     └── logging.py        # Training logging
@@ -226,6 +346,7 @@ gptmed/
 
 - [User Manual](USER_MANUAL.md) - **Start here!** Complete training pipeline guide
 - [Architecture Guide](ARCHITECTURE_EXTENSION_GUIDE.md) - Understanding the model architecture
+- [XAI Roadmap](XAI.md) - Explainable AI features & implementation guide
 - [Deployment Guide](DEPLOYMENT_GUIDE.md) - Publishing to PyPI
 - [Changelog](CHANGELOG.md) - Version history
 
@@ -241,20 +362,53 @@ _Tested on GTX 1080 8GB_
 
 ## Examples
 
-### Medical Question Answering
+### Domain-Agnostic Usage
+
+GptMed works with **any domain** - just train on your own Q&A data:
 
 ```python
-# Example 1: Symptoms inquiry
-question = "What are the early signs of Alzheimer's disease?"
+# Technical Support Bot
+question = "How do I reset my WiFi router?"
 answer = generator.generate(question, temperature=0.7)
 
-# Example 2: Treatment information
-question = "How is Type 2 diabetes treated?"
+# Educational Assistant
+question = "Explain the water cycle in simple terms"
 answer = generator.generate(question, temperature=0.6)
 
-# Example 3: Medical definitions
-question = "What is hypertension?"
+# Customer Service
+question = "What is your return policy?"
 answer = generator.generate(question, temperature=0.5)
+
+# Medical Q&A (example domain)
+question = "What are the symptoms of flu?"
+answer = generator.generate(question, temperature=0.7)
+```
+
+### Training Observability (v0.4.0+)
+
+Monitor your training with built-in observability:
+
+```python
+from gptmed.observability import MetricsTracker, ConsoleCallback
+
+# Create observers
+tracker = MetricsTracker(output_dir='./metrics')
+console = ConsoleCallback(print_every=10)
+
+# Train with observability
+gptmed.train_from_config(
+    'my_config.yaml',
+    observers=[tracker, console]
+)
+
+# After training - get the report
+report = tracker.get_report()
+print(f"Final Loss: {report['final_loss']:.4f}")
+print(f"Total Steps: {report['total_steps']}")
+
+# Export metrics
+tracker.export_to_csv('training_metrics.csv')
+tracker.plot_loss_curves('loss_curves.png')  # Requires matplotlib
 ```
 
 ## Contributing
@@ -267,19 +421,6 @@ Contributions are welcome! Please feel free to submit a Pull Request.
 4. Push to the branch (`git push origin feature/AmazingFeature`)
 5. Open a Pull Request
 
-## Citation
-
-If you use this model in your research, please cite:
-
-```bibtex
-@software{llm_med_2026,
-  author = {Sanjog Sigdel},
-  title = {GptMed: A custom causal question answering general purpose GPT Transformer Architecture Model},
-  year = {2026},
-  url = {https://github.com/sigdelsanjog/gptmed}
-}
-```
-
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
@@ -289,16 +430,12 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 - MedQuAD dataset creators
 - PyTorch team
 
-## Disclaimer
-
-⚠️ **Medical Disclaimer**: This model is for research and educational purposes only. It should NOT be used for actual medical diagnosis or treatment decisions. Always consult qualified healthcare professionals for medical advice.
-
 ## Support
 
-- � **[User Manual](USER_MANUAL.md)** - Complete step-by-step training guide
-- �📫 Issues: [GitHub Issues](https://github.com/sigdelsanjog/gptmed/issues)
+- 📫 [User Manual](USER_MANUAL.md)\*\* - Complete step-by-step training guide
+- 📫 Issues: [GitHub Issues](https://github.com/sigdelsanjog/gptmed/issues)
 - 💬 Discussions: [GitHub Discussions](https://github.com/sigdelsanjog/gptmed/discussions)
-- 📧 Email: sanjog.sigdel@ku.edu.np
+- 📧 Email: sigdelsanjog@gmail.com | sanjog.sigdel@ku.edu.np
 
 ## Changelog
 
@@ -306,4 +443,4 @@ See [CHANGELOG.md](CHANGELOG.md) for version history.
 
 ---
 
-Made with ❤️ for learning purpose
+#### Made with ❤️ from Nepal

{gptmed-0.3.4 → gptmed-0.4.0}/README.md

@@ -1,23 +1,51 @@
 # GptMed 🤖
 
-A lightweight GPT-based language model framework for training custom question-answering models on any domain. This package provides a transformer-based GPT architecture that you can train on your own Q&A datasets - whether it's casual conversations, technical support, education, or any other domain.
-
+[![Downloads](https://static.pepy.tech/badge/gptmed)](https://pepy.tech/project/gptmed)
+[![Downloads/Month](https://static.pepy.tech/badge/gptmed/month)](https://pepy.tech/project/gptmed)
 [![PyPI version](https://badge.fury.io/py/gptmed.svg)](https://badge.fury.io/py/gptmed)
 [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-## 📖 [Complete User Manual](USER_MANUAL.md) | [Quick Start](#quick-start)
+A lightweight GPT-based language model framework for training custom question-answering models on any domain. This package provides a transformer-based GPT architecture that you can train on your own Q&A datasets - whether it's casual conversations, technical support, education, or any other domain.
 
-> **New to GptMed?** Check out the [**step-by-step User Manual**](USER_MANUAL.md) for a complete guide on training your own model!
+## Citation
 
-## Features
+If you use this model in your research, please cite:
 
-- 🧠 **Custom GPT Architecture**: Lightweight transformer model for any Q&A domain
-- 🎯 **Domain-Agnostic**: Train on any question-answering dataset (casual chat, tech support, education, etc.)
-- ⚡ **Fast Inference**: Optimized for quick question answering
-- 🔧 **Flexible Training**: Easy to train on your own custom datasets
-- 📦 **Lightweight**: Small model size suitable for edge deployment
-- 🛠️ **Complete Toolkit**: Includes tokenizer training, model training, and inference utilities
+```bibtex
+@software{gptmed_2026,
+  author = {Sanjog Sigdel},
+  title = {GptMed: A custom causal question answering general purpose GPT Transformer Architecture Model},
+  year = {2026},
+  url = {https://github.com/sigdelsanjog/gptmed}
+}
+```
+
+## Table of Contents
+
+- [Installation](#installation)
+  - [From PyPI (Recommended)](#from-pypi-recommended)
+  - [From Source](#from-source)
+  - [With Optional Dependencies](#with-optional-dependencies)
+- [Quick Start](#quick-start)
+  - [Using the High-Level API](#using-the-high-level-api)
+  - [Inference (Generate Answers)](#inference-generate-answers)
+  - [Using Command Line](#using-command-line)
+  - [Training Your Own Model](#training-your-own-model)
+- [Model Architecture](#model-architecture)
+- [Configuration](#configuration)
+  - [Model Sizes](#model-sizes)
+  - [Training Configuration](#training-configuration)
+- [Observability](#observability)
+- [Project Structure](#project-structure)
+- [Requirements](#requirements)
+- [Documentation](#documentation)
+- [Performance](#performance)
+- [Examples](#examples)
+- [Contributing](#contributing)
+- [Citation](#citation)
+- [License](#license)
+- [Support](#support)
 
 ## Installation
 
@@ -41,15 +69,49 @@ pip install -e .
 # For development
 pip install gptmed[dev]
 
-# For training
+# For training with logging integrations
 pip install gptmed[training]
 
+# For visualization (loss curves, metrics plots)
+pip install gptmed[visualization]
+
+# For Explainable AI features
+pip install gptmed[xai]
+
 # All dependencies
-pip install gptmed[dev,training]
+pip install gptmed[dev,training,visualization,xai]
 ```
 
 ## Quick Start
 
+### Using the High-Level API
+
+The easiest way to use GptMed is through the high-level API:
+
+```python
+import gptmed
+
+# 1. Create a training configuration
+gptmed.create_config('my_config.yaml')
+
+# 2. Edit my_config.yaml with your settings (data paths, model size, etc.)
+
+# 3. Train the model
+gptmed.train_from_config('my_config.yaml')
+
+# 4. Generate answers
+answer = gptmed.generate(
+    checkpoint='model/checkpoints/best_model.pt',
+    tokenizer='tokenizer/my_tokenizer.model',
+    prompt='What is machine learning?',
+    max_length=150,
+    temperature=0.7
+)
+print(answer)
+```
+
+For a complete API testing workflow, see the [gptmed-api folder](https://github.com/sigdelsanjog/gptmed/tree/main/gptmed-api) with ready-to-run examples.
+
 ### Inference (Generate Answers)
 
 ```python
@@ -145,6 +207,50 @@ config = TrainingConfig(
 )
 ```
 
+## Observability
+
+**New in v0.4.0**: Built-in training monitoring with Observer Pattern architecture.
+
+### Features
+
+- 📊 **Loss Curves**: Track training/validation loss over time
+- 📈 **Metrics Tracking**: Perplexity, gradient norms, learning rates
+- 🔔 **Callbacks**: Console output, JSON logging, early stopping
+- 📁 **Export**: CSV export, matplotlib visualizations
+- 🔌 **Extensible**: Add custom observers for integrations (W&B, TensorBoard)
+
+### Quick Example
+
+```python
+from gptmed.observability import MetricsTracker, ConsoleCallback, EarlyStoppingCallback
+
+# Create observers
+tracker = MetricsTracker(output_dir='./metrics')
+console = ConsoleCallback(print_every=50)
+early_stop = EarlyStoppingCallback(patience=3)
+
+# Use with TrainingService (automatic)
+from gptmed.services import TrainingService
+service = TrainingService(config_path='config.yaml')
+service.train()  # Automatically creates MetricsTracker
+
+# Or use with Trainer directly
+trainer = Trainer(model, train_loader, config, observers=[tracker, console])
+trainer.train()
+```
+
+### Available Observers
+
+| Observer                | Description                                               |
+| ----------------------- | --------------------------------------------------------- |
+| `MetricsTracker`        | Comprehensive metrics collection with export capabilities |
+| `ConsoleCallback`       | Real-time console output with progress bars               |
+| `JSONLoggerCallback`    | Structured JSON logging for analysis                      |
+| `EarlyStoppingCallback` | Stop training when validation loss plateaus               |
+| `LRSchedulerCallback`   | Learning rate scheduling integration                      |
+
+See [XAI.md](XAI.md) for future Explainable AI features roadmap.
+
 ## Project Structure
 
 ```
@@ -159,10 +265,16 @@ gptmed/
 │   ├── train.py          # Training script
 │   ├── trainer.py        # Training loop
 │   └── dataset.py        # Data loading
+├── observability/         # Training monitoring & XAI (v0.4.0+)
+│   ├── base.py           # Observer pattern interfaces
+│   ├── metrics_tracker.py # Loss curves & metrics
+│   └── callbacks.py      # Console, JSON, early stopping
 ├── tokenizer/
 │   └── train_tokenizer.py # SentencePiece tokenizer
 ├── configs/
 │   └── train_config.py   # Training configurations
+├── services/
+│   └── training_service.py # High-level training orchestration
 └── utils/
     ├── checkpoints.py    # Model checkpointing
     └── logging.py        # Training logging
@@ -184,6 +296,7 @@ gptmed/
 
 - [User Manual](USER_MANUAL.md) - **Start here!** Complete training pipeline guide
 - [Architecture Guide](ARCHITECTURE_EXTENSION_GUIDE.md) - Understanding the model architecture
+- [XAI Roadmap](XAI.md) - Explainable AI features & implementation guide
 - [Deployment Guide](DEPLOYMENT_GUIDE.md) - Publishing to PyPI
 - [Changelog](CHANGELOG.md) - Version history
 
@@ -199,20 +312,53 @@ _Tested on GTX 1080 8GB_
 
 ## Examples
 
-### Medical Question Answering
+### Domain-Agnostic Usage
+
+GptMed works with **any domain** - just train on your own Q&A data:
 
 ```python
-# Example 1: Symptoms inquiry
-question = "What are the early signs of Alzheimer's disease?"
+# Technical Support Bot
+question = "How do I reset my WiFi router?"
 answer = generator.generate(question, temperature=0.7)
 
-# Example 2: Treatment information
-question = "How is Type 2 diabetes treated?"
+# Educational Assistant
+question = "Explain the water cycle in simple terms"
 answer = generator.generate(question, temperature=0.6)
 
-# Example 3: Medical definitions
-question = "What is hypertension?"
+# Customer Service
+question = "What is your return policy?"
 answer = generator.generate(question, temperature=0.5)
+
+# Medical Q&A (example domain)
+question = "What are the symptoms of flu?"
+answer = generator.generate(question, temperature=0.7)
+```
+
+### Training Observability (v0.4.0+)
+
+Monitor your training with built-in observability:
+
+```python
+from gptmed.observability import MetricsTracker, ConsoleCallback
+
+# Create observers
+tracker = MetricsTracker(output_dir='./metrics')
+console = ConsoleCallback(print_every=10)
+
+# Train with observability
+gptmed.train_from_config(
+    'my_config.yaml',
+    observers=[tracker, console]
+)
+
+# After training - get the report
+report = tracker.get_report()
+print(f"Final Loss: {report['final_loss']:.4f}")
+print(f"Total Steps: {report['total_steps']}")
+
+# Export metrics
+tracker.export_to_csv('training_metrics.csv')
+tracker.plot_loss_curves('loss_curves.png')  # Requires matplotlib
 ```
 
 ## Contributing
@@ -225,19 +371,6 @@ Contributions are welcome! Please feel free to submit a Pull Request.
 4. Push to the branch (`git push origin feature/AmazingFeature`)
 5. Open a Pull Request
 
-## Citation
-
-If you use this model in your research, please cite:
-
-```bibtex
-@software{llm_med_2026,
-  author = {Sanjog Sigdel},
-  title = {GptMed: A custom causal question answering general purpose GPT Transformer Architecture Model},
-  year = {2026},
-  url = {https://github.com/sigdelsanjog/gptmed}
-}
-```
-
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
@@ -247,16 +380,12 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 - MedQuAD dataset creators
 - PyTorch team
 
-## Disclaimer
-
-⚠️ **Medical Disclaimer**: This model is for research and educational purposes only. It should NOT be used for actual medical diagnosis or treatment decisions. Always consult qualified healthcare professionals for medical advice.
-
 ## Support
 
-- � **[User Manual](USER_MANUAL.md)** - Complete step-by-step training guide
-- �📫 Issues: [GitHub Issues](https://github.com/sigdelsanjog/gptmed/issues)
+- 📫 [User Manual](USER_MANUAL.md)\*\* - Complete step-by-step training guide
+- 📫 Issues: [GitHub Issues](https://github.com/sigdelsanjog/gptmed/issues)
 - 💬 Discussions: [GitHub Discussions](https://github.com/sigdelsanjog/gptmed/discussions)
-- 📧 Email: sanjog.sigdel@ku.edu.np
+- 📧 Email: sigdelsanjog@gmail.com | sanjog.sigdel@ku.edu.np
 
 ## Changelog
 
@@ -264,4 +393,4 @@ See [CHANGELOG.md](CHANGELOG.md) for version history.
 
 ---
 
-Made with ❤️ for learning purpose
+#### Made with ❤️ from Nepal