mi-crow 1.0.0__py3-none-any.whl → 1.0.0.post2__py3-none-any.whl

{mi_crow-1.0.0.dist-info → mi_crow-1.0.0.post2.dist-info}/METADATA

@@ -1,8 +1,11 @@
 Metadata-Version: 2.4
 Name: mi-crow
-Version: 1.0.0
-Summary: Engineer Thesis: Explaining and modifying LLM responses using SAE and concepts.
-Author-email: Hubert Kowalski <your.email@example.com>, Adam Kaniasty <adam.kaniasty@gmail.com>
+Version: 1.0.0.post2
+Summary: Python library for mechanistic interpretability research on Large Language Models. Designed for researchers, provides unified interface for SAE training, activation hooks, and concept manipulation.
+Author: Hubert Kowalski
+Author-email: Adam Kaniasty <adam.kaniasty@gmail.com>
+Maintainer: Hubert Kowalski
+Maintainer-email: Adam Kaniasty <adam.kaniasty@gmail.com>
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: accelerate>=1.10.1
@@ -19,6 +22,7 @@ Requires-Dist: wandb>=0.22.1
 Requires-Dist: pytest>=8.4.2
 Requires-Dist: pytest-xdist>=3.8.0
 Requires-Dist: seaborn>=0.13.2
+Requires-Dist: numpy<2.0,>=1.20.0
 Provides-Extra: dev
 Requires-Dist: pre-commit>=4.3.0; extra == "dev"
 Requires-Dist: ruff>=0.13.2; extra == "dev"
@@ -36,10 +40,153 @@ Requires-Dist: mkdocs-literate-nav>=0.6; extra == "docs"
 Requires-Dist: mkdocs-gen-files>=0.5; extra == "docs"
 Requires-Dist: mike>=2.1; extra == "docs"
 
-![CI](https://github.com/AdamKaniasty/Inzynierka/actions/workflows/tests.yml/badge.svg?branch=main)
+<div align="center">
+  <img src="docs/logo.svg" alt="Mi-Crow Logo" width="200">
+</div>
+
+# Mi-Crow: Mechanistic Interpretability for Large Language Models
+
+[![CI](https://github.com/AdamKaniasty/Inzynierka/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/AdamKaniasty/Inzynierka/actions)
+[![PyPI](https://img.shields.io/pypi/v/mi-crow)](https://pypi.org/project/mi-crow/)
 [![Docs](https://img.shields.io/badge/docs-gh--pages-blue)](https://adamkaniasty.github.io/Inzynierka/)
 
-## Running Tests
+**Mi-Crow** is a Python library for mechanistic interpretability research on Large Language Models (LLMs). Designed for researchers, it provides a unified interface for analyzing and controlling model behavior through Sparse Autoencoders (SAEs), activation hooks, and concept manipulation.
+
+## Features
+
+- **Unified Model Interface** - Work with any HuggingFace language model through a consistent API
+- **Sparse Autoencoder Training** - Train SAEs to extract interpretable features from model activations
+- **Hook System** - Intercept and manipulate model activations with minimal performance overhead
+- **Concept Discovery & Manipulation** - Discover and control model behavior through learned concepts
+- **Hierarchical Data Persistence** - Efficient storage and management of large-scale experiment data
+- **Research Focused** - Comprehensive testing (85%+ coverage), extensive documentation, and designed for interpretability research workflows
+
+## Installation
+
+### Install from PyPI
+
+```bash
+pip install mi-crow
+```
+
+### Install from Source
+
+```bash
+git clone https://github.com/AdamKaniasty/Mi-Crow.git
+cd Mi-Crow
+pip install -e .
+```
+
+### Requirements
+
+- **Python 3.12+** (required for modern type hints and features)
+- **PyTorch** - Tensor operations and neural networks
+- **Transformers** - Model loading and tokenization
+- **Accelerate** - Distributed and mixed-precision training
+- **Datasets** - Dataset loading and processing
+- **overcomplete** - SAE implementations
+
+## Quick Start
+
+### Basic Usage
+
+```python
+from mi_crow.language_model import LanguageModel
+
+# Initialize a language model
+lm = LanguageModel(model_id="bielik")
+
+# Run inference
+outputs = lm.forwards(["Hello, world!"])
+
+# Access activations and outputs
+print(outputs.logits)
+```
+
+### Training an SAE
+
+```python
+from mi_crow.language_model import LanguageModel
+from mi_crow.mechanistic.sae import SaeTrainer
+from mi_crow.mechanistic.sae.modules import TopKSae
+
+# Load model and collect activations
+lm = LanguageModel(model_id="bielik")
+activations = lm.save_activations(
+    dataset=["Your text data here"],
+    layers=["transformer_h_0_attn_c_attn"]
+)
+
+# Train SAE
+trainer = SaeTrainer(
+    model=lm,
+    layer="transformer_h_0_attn_c_attn",
+    sae_class=TopKSae,
+    hyperparams={"epochs": 10, "batch_size": 256}
+)
+sae = trainer.train(activations)
+```
+
+### Concept Manipulation
+
+```python
+# Load concepts and manipulate model behavior
+concepts = lm.load_concepts(sae_id="your_sae_id")
+concepts.manipulate(neuron_idx=0, scale_factor=1.5)
+
+# Run inference with concept manipulation
+outputs = lm.forwards(
+    ["Your prompt"],
+    with_controllers=True,
+    concept_config=concepts
+)
+```
+
+## Documentation
+
+- **Full Documentation**: [adamkaniasty.github.io/Inzynierka](https://adamkaniasty.github.io/Inzynierka/)
+- **GitHub Repository**: [github.com/AdamKaniasty/Mi-Crow](https://github.com/AdamKaniasty/Mi-Crow/)
+- **Example Notebooks**: See `examples/` directory for Jupyter notebook tutorials
+
+## Architecture
+
+Mi-Crow follows a modular design with five core components:
+
+1. **`language_model/`** - Unified interface for language models
+   - Model initialization from HuggingFace Hub or local files
+   - Unified inference interface with mixed-precision support
+   - Architecture-agnostic layer abstraction
+
+2. **`hooks/`** - Flexible hook system for activation interception
+   - Detectors for observing activations
+   - Controllers for modifying model behavior
+   - Support for FORWARD and PRE_FORWARD hooks
+
+3. **`mechanistic/`** - SAE training and concept manipulation
+   - Sparse Autoencoder training (TopK, L1 variants)
+   - Concept dictionary management
+   - Concept-based model steering
+
+4. **`store/`** - Hierarchical data persistence
+   - Efficient tensor storage in safetensors format
+   - Batch iteration for large datasets
+   - Metadata management
+
+5. **`datasets/`** - Dataset loading and processing
+   - HuggingFace dataset integration
+   - Local file dataset support
+
+## Example Workflow
+
+See the example notebooks in the `examples/` directory:
+
+1. **`01_train_sae_model.ipynb`** - Train an SAE on model activations
+2. **`02_attach_sae_and_save_texts.ipynb`** - Collect top activating texts
+3. **`03_load_concepts.ipynb`** - Load and manipulate concepts
+
+## Development
+
+### Running Tests
 
 The project uses pytest for testing. Tests are organized into unit tests and end-to-end tests.
 
@@ -77,6 +224,45 @@ pytest tests/e2e -q
 
 The test suite is configured to require at least 85% code coverage. Coverage reports are generated in both terminal and XML formats.
 
+The project maintains **85%+ code coverage** requirement.
+
+### Code Quality
+
+- **Linting**: Ruff for code formatting and linting
+- **Pre-commit Hooks**: Automated quality checks
+- **Type Hints**: Extensive use of Python type annotations
+- **CI/CD**: GitHub Actions for automated testing and deployment
+
+## Citation
+
+If you use Mi-Crow in your research, please cite:
+
+```bibtex
+@thesis{kaniasty2025microw,
+  title={Mechanistic Interpretability for Large Language Models: A Production-Ready Framework},
+  author={Kaniasty, Adam and Kowalski, Hubert},
+  year={2025},
+  school={Warsaw University of Technology},
+  note={Engineering Thesis}
+}
+```
+
+## License
+
+See the main repository for license information: [Mi-Crow License](https://github.com/AdamKaniasty/Mi-Crow/)
+
+## Contact
+
+- **Maintainers**: Adam Kaniasty, Hubert Kowalski
+- **Email**: adam.kaniasty@gmail.com
+- **GitHub**: [@AdamKaniasty](https://github.com/AdamKaniasty)
+
+## Acknowledgments
+
+This work was developed in collaboration with the **Bielik** team and represents a contribution to the open-source mechanistic interpretability community.
+
+---
+
 ## Backend (FastAPI) quickstart
 
 Install server-only dependencies (kept out of the core library) with uv:

{mi_crow-1.0.0.dist-info → mi_crow-1.0.0.post2.dist-info}/RECORD

@@ -1,52 +1,54 @@
 mi_crow/__init__.py,sha256=J7aXVlAicbjvk5630rhDxx0ATsvZnihud5u_aQpAwY8,487
 mi_crow/utils.py,sha256=LTfh2Ep87lAgPBaZkrQPP9caXFJoS9zUxu4qFuV4kzM,1549
 mi_crow/datasets/__init__.py,sha256=lCAc3nFlvoERrBPAan6C9YFmDx86W2gbIAy267Rb2Sk,349
-mi_crow/datasets/base_dataset.py,sha256=fSs63o0LDNXt1-i0HmKTxaP33CMcS6rWJ3C4vpB8qR0,22527
-mi_crow/datasets/classification_dataset.py,sha256=c2peu0kWVY3N8XlEi9GjRuE6MmVV8f8vz8gtPmxI70s,21721
+mi_crow/datasets/base_dataset.py,sha256=ki8C_GKNte6_Csgezh_l3E1-rc15uiwKlgUtFTiOB50,25343
+mi_crow/datasets/classification_dataset.py,sha256=uD2dmf6lcLgc6R3fjhyXPkC8mrEhBPTVLWWihrbdDlM,25876
 mi_crow/datasets/loading_strategy.py,sha256=17VM3Td8lqDllGIx9DHI6WiXmSKKQHDHbfe4ZeM8ATA,1206
-mi_crow/datasets/text_dataset.py,sha256=dk5RzWy-T7ssQNwv84FrgEZ83zuDgpq91XgUfbwvk8c,16796
+mi_crow/datasets/text_dataset.py,sha256=pEslx7_Of7zUS5umdraKYxJyeeilt4-1Ydjfjn5BKrE,21922
 mi_crow/hooks/__init__.py,sha256=KYy5qcbEpnJceNH86ofy43Suu_36QXjj0HYl79rVyls,693
-mi_crow/hooks/controller.py,sha256=eo8LMERORXYUjH4-_R6DHk5JKN6O8SW6PlnuBFrlNqg,6063
+mi_crow/hooks/controller.py,sha256=pt00-9XBvX13OLWGst3Y0RNTttkMCVjlWj3EaP4Wtv4,6198
 mi_crow/hooks/detector.py,sha256=Bj3xz56cSgRvbcoQBsHIdlJdf0dtgVLw3l1pOSRvRAg,3114
 mi_crow/hooks/hook.py,sha256=JrCyPptXzHICAToxug7FD8zKWKZcLxbXfIe7UHCkh34,7542
-mi_crow/hooks/utils.py,sha256=GdUAqL9InCsthjBVWbZtVQp2VtQLLOMaJy8S8Tc7WX0,2024
+mi_crow/hooks/utils.py,sha256=z6SaGWDrwRSUpO-TeUuxeiz62kg9dlmt-wTC3juzOjA,5342
 mi_crow/hooks/implementations/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 mi_crow/hooks/implementations/function_controller.py,sha256=Rwu1Ghffqm-Jc4mqniRdIsXQeV24JKlx9CMO8jP8iv0,3092
-mi_crow/hooks/implementations/layer_activation_detector.py,sha256=mFqLkWrBlNYzYYvwI8O-ymnpwP1UJSX7eWIQ7Fp-lxE,3167
-mi_crow/hooks/implementations/model_input_detector.py,sha256=4AFA88eQ-zM9Kz8bpv4Bl5cvlZF5ZyIRXNi7Znd6V98,10078
-mi_crow/hooks/implementations/model_output_detector.py,sha256=BmD8oJ1k_nv4hIzG-9SGxSCRARQkLRI4d_XQmKs6jjE,4883
+mi_crow/hooks/implementations/layer_activation_detector.py,sha256=yk1uTyxsQLHQOlCk0k9_FqPyEs8tpi5V98eU9UwwCDo,3360
+mi_crow/hooks/implementations/model_input_detector.py,sha256=W-CX5FzExfWI0JcY43tPMCr67LlaRXvnP3GRE2BGWc8,10168
+mi_crow/hooks/implementations/model_output_detector.py,sha256=_x-4CmQXw7nRJRM9f7_iSJi11kyNYbnY7s2IF1Zu5h4,5111
 mi_crow/language_model/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-mi_crow/language_model/activations.py,sha256=mJhuXkQ_JKzQsrFossnXVTdq4WgBI10Gi0ZYeQ9USFY,16989
+mi_crow/language_model/activations.py,sha256=bZ_2OjTQE4itfLoynJywGYV88PxA9jxV5P_aAGqCGXU,20539
 mi_crow/language_model/context.py,sha256=TJSe6IruGktePpQ0dtHIoeatSY72qAhEKeTvLizCdlw,1124
 mi_crow/language_model/contracts.py,sha256=6ij7rzJcpSAKgYx-fiefg0Fi8TsFugaM5IlihEm_2VM,268
+mi_crow/language_model/device_manager.py,sha256=nxGgrZGD5MTZACJe45_CIzp1YZNZmNUSrTFz4iG7bSo,3730
 mi_crow/language_model/hook_metadata.py,sha256=GACZjZUneo2l5j7DCFycLAunTm0etdMQ2YB_xgueUuk,1394
-mi_crow/language_model/inference.py,sha256=-Kpm85jM8y6-GyDgrvIczitBIwGh8grJP8aYuXsLV-g,19082
-mi_crow/language_model/initialization.py,sha256=e_Vkk-p9KWRt6-Hmkm6I29dTf20jzEAyNF9CG4nc48M,3704
-mi_crow/language_model/language_model.py,sha256=b4KniweHXauPHPgHba9cAlLK86z2atfeoetnG-UHoSo,13998
-mi_crow/language_model/layers.py,sha256=7RdVU1mXt5oy2OhzNoKDeD9Omm899ZC-BJqpcTfxP2w,15933
-mi_crow/language_model/persistence.py,sha256=9wQE6tRvLg7BgdLlkKRTOfRwXb5Q0LsEgg8B9J7Yos0,5881
+mi_crow/language_model/inference.py,sha256=41hff-1MuL3JwSKF5p1z-j-KxgBhwPDaNIqubodV0pA,19623
+mi_crow/language_model/initialization.py,sha256=q4zBS_BemeIGSMsE3iXZ8BG_ZnT3nSd2hYtBxa01KoY,4025
+mi_crow/language_model/language_model.py,sha256=rblruse5sDEi2a0r9YzvgMDhytHpVwwDSBB44_OGh90,13331
+mi_crow/language_model/layers.py,sha256=tnl1xBmpHNBgapXP1o8QB34uMkd2o2ufNYDQA0SD8M0,15893
+mi_crow/language_model/persistence.py,sha256=n7NKo_J-KgB2zE4Tt6S7n9cSox3v768ac47tqq2ALM4,6030
 mi_crow/language_model/tokenizer.py,sha256=uZbMDVNnzu8WZINUaR1tLFXiuk9V5pAoahwnJOUvEuE,7379
-mi_crow/language_model/utils.py,sha256=5Y7scRvvudUjKDV8QPhC3HAc2S-dCuqbm6xEjRr0fRM,2630
+mi_crow/language_model/utils.py,sha256=dwql6LEcJ4kKzl7wy-qL5uet4xFFGlGhCdlm65nUJLU,2577
 mi_crow/mechanistic/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 mi_crow/mechanistic/sae/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 mi_crow/mechanistic/sae/autoencoder_context.py,sha256=u5WzSlLb8_HaJF9LwGqe-J_rE-iRCoXbvwEhTZIArkw,947
-mi_crow/mechanistic/sae/sae.py,sha256=R2IckZ-UDVXFURUSRoIdX0MGLe0TWU5JCVeDXPz1Wv4,6053
-mi_crow/mechanistic/sae/sae_trainer.py,sha256=VqN9UgEPtgYItzAS0RqMfOhTnTxIEuModO0noHYieFA,26327
+mi_crow/mechanistic/sae/sae.py,sha256=i1TO19MY7WBUVfRGMi3WPkgDaHmoNr8Ul2ZZWwo6KCk,6180
+mi_crow/mechanistic/sae/sae_trainer.py,sha256=bgIzGwvrJBNLOAXIIfXJZftxJc5LBH379Pas_DSob-g,41342
 mi_crow/mechanistic/sae/concepts/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-mi_crow/mechanistic/sae/concepts/autoencoder_concepts.py,sha256=SE7A6y6cZiMGhylljNdZz0D8lhrulUh1FYiKhnPPRvY,14387
-mi_crow/mechanistic/sae/concepts/concept_dictionary.py,sha256=aUXTe4Fy7Oe7iYJOcQImEXYMZAXhnkfAszZNwivk4eg,7673
+mi_crow/mechanistic/sae/concepts/autoencoder_concepts.py,sha256=cT5hL6NGkpULpOjDCR2EZyYDgxwsrJCnp2RMXlB56k0,17135
+mi_crow/mechanistic/sae/concepts/concept_dictionary.py,sha256=te_tSDGIMEqItTDiB1T-rE2YlFfN-eVVxgR4t5GlZH8,8001
 mi_crow/mechanistic/sae/concepts/concept_models.py,sha256=HGyPoMSmj8CAg9joIa6fV_FfWAY2XN4LFPIkNQiJWbY,133
 mi_crow/mechanistic/sae/concepts/input_tracker.py,sha256=kIiqt7guv_-9-UPYtefAFJbHkWtAS_mnqYVvRU4eb2o,1890
+mi_crow/mechanistic/sae/concepts/text_heap.py,sha256=4NAI0JF7iOmiwrLxoqjdFLFtfjdwIbCyzr26jsy5C-E,5437
 mi_crow/mechanistic/sae/modules/__init__.py,sha256=e0lkCALQZcJN7KpYyTtXx3OD2NhBxV_kOZLLJ6EWaTE,243
 mi_crow/mechanistic/sae/modules/l1_sae.py,sha256=qqw0iTWLSmWAlz5kgfw_mex8LeecFWM1FobyUteMqmM,15388
-mi_crow/mechanistic/sae/modules/topk_sae.py,sha256=TIueJ4ftJm0XfU2UyHm0n3bIwL-W1RUgnpYym4xBxpg,17328
+mi_crow/mechanistic/sae/modules/topk_sae.py,sha256=SndyeLQrYjTjNsJndU7Iw820nZBB5GeBWmC8TZNWQvs,17461
 mi_crow/mechanistic/sae/training/__init__.py,sha256=5flCJVkOyKizY0FZy1OP5v0EI6bPEayunpnUPp82a6s,140
 mi_crow/mechanistic/sae/training/wandb_logger.py,sha256=YlSJd5CaNa35RmIgf1FD_gSEDyhGRa2UdHo_Ofrplos,8558
 mi_crow/store/__init__.py,sha256=DrYTpdgzrRzjHm9bigy-GiP0BGxzjmD3-lJCthtgxbE,123
-mi_crow/store/local_store.py,sha256=XmguFvdrUi6NHzvV_bLaDJzpk5KWU_-ObkzhICcLu6g,17216
-mi_crow/store/store.py,sha256=VuDe9Git0glND3TTHh0zhDJNxdQY3dCp0cURhApYQbU,9334
+mi_crow/store/local_store.py,sha256=0Djh3zIVYWx_NASUv6VXfVLV3gRzTduFFgO7wswYcZA,17571
+mi_crow/store/store.py,sha256=ZxxIvhSPunUG7XsVGiDyqxk71Cmb0h-vSwLbZbDCRko,11299
 mi_crow/store/store_dataloader.py,sha256=UkZhHCOTg56ozomPtU9vHBhxIMOPcOiyfMqiAxgqtQs,4341
-mi_crow-1.0.0.dist-info/METADATA,sha256=wKP7S0wv8GRvTyvD-wjCAUicATKY5tvQopbYxrJW9GY,6577
-mi_crow-1.0.0.dist-info/WHEEL,sha256=qELbo2s1Yzl39ZmrAibXA2jjPLUYfnVhUNTlyF1rq0Y,92
-mi_crow-1.0.0.dist-info/top_level.txt,sha256=DTuNo2VWgrH6jQKY19NciReSpLwGKKIRzJ3WbpspLlE,8
-mi_crow-1.0.0.dist-info/RECORD,,
+mi_crow-1.0.0.post2.dist-info/METADATA,sha256=BUbZvRNArEuUWzk5fDV7z_CzzLgyrDNr-MAqiwE5ZiI,12475
+mi_crow-1.0.0.post2.dist-info/WHEEL,sha256=qELbo2s1Yzl39ZmrAibXA2jjPLUYfnVhUNTlyF1rq0Y,92
+mi_crow-1.0.0.post2.dist-info/top_level.txt,sha256=DTuNo2VWgrH6jQKY19NciReSpLwGKKIRzJ3WbpspLlE,8
+mi_crow-1.0.0.post2.dist-info/RECORD,,