jeffy-classify 0.1.0a8__tar.gz

jeffy_classify-0.1.0a8/.gitignore

@@ -0,0 +1,56 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+*.egg
+.eggs/
+
+# Virtual environments
+venv/
+.venv/
+env/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Large model files (encoder weights - not tracked)
+models/encoder/
+*.safetensors
+*.bin
+*.onnx
+
+# Dataset downloads and caches
+data/downloads/
+data/cache/
+.cache/
+
+# Sensitive
+.env
+*.credentials
+*.token
+
+# ViZDoom artifacts (not part of Jeffy)
+_vizdoom.ini
+*.wad
+
+# Large per-example prediction files (reproduce with scripts)
+# Keep summaries and comparison JSON in git
+data/eval_results/*_predictions.jsonl
+
+# Build output (regenerate with jeffy-build)
+data/model_pack/
+
+# Drafts and conversation artifacts (private)
+SHOW_HN_DRAFT.md
+JEFFY_HANDOFF.md
+snapshot.json

jeffy_classify-0.1.0a8/ATTRIBUTION.md

@@ -0,0 +1,76 @@
+# Attribution and Upstream Licenses
+
+## Jeffy Code
+
+MIT License. See LICENSE.
+
+## Shared Encoder
+
+**BAAI/bge-large-en-v1.5**
+- Source: https://huggingface.co/BAAI/bge-large-en-v1.5
+- License: MIT
+- Downloaded at runtime by sentence-transformers; not included in this repository.
+
+## Inspiration
+
+**Jeff** by Mathias Strasser
+- Source: https://github.com/firelex/jeff
+- License: MIT (code), Apache 2.0 (model weights)
+- Jeffy's /v1/systemone endpoint and playground design are inspired by Jeff.
+  No Jeff code is included in this repository.
+
+## Pretrained Head Artifacts
+
+The model pack contains logistic regression coefficients and scaler statistics
+trained on public datasets. These are derived model parameters — numerical
+arrays that do not contain any training text.
+
+Each artifact's `manifest.json` records the source dataset. Source terms were
+checked on HuggingFace and original project sites (Sep 2026).
+
+**What we distribute:** Numerical classifier parameters (coefficients, scaler means/scales).
+**What we do NOT distribute:** Training text, dataset copies, or dataset downloads.
+**Reproduction:** Each head can be retrained from its source dataset via `jeffy-build`.
+
+### Verified: explicit license permitting derivative works
+
+| Dataset | License | Source | Checked |
+|---------|---------|--------|---------|
+| banking77 | CC BY 4.0 | [HF](https://huggingface.co/datasets/legacy-datasets/banking77) | HF metadata |
+| clinc_oos | CC BY 3.0 | [HF](https://huggingface.co/datasets/clinc_oos) | HF metadata |
+| massive_intent | CC BY 4.0 | [HF](https://huggingface.co/datasets/mteb/amazon_massive_intent) | HF metadata |
+| sms_spam | CC BY 4.0 | [HF](https://huggingface.co/datasets/ucirvine/sms_spam) | HF metadata |
+| snli | CC BY-SA 4.0 | [HF](https://huggingface.co/datasets/stanfordnlp/snli) | HF metadata |
+| dbpedia | CC BY-SA 3.0 | [HF](https://huggingface.co/datasets/fancyzhx/dbpedia_14) | HF metadata |
+| tweet_eval_sentiment | CC BY 3.0 | [HF](https://huggingface.co/datasets/cardiffnlp/tweet_eval) | Listed per-subset on HF card |
+
+### No explicit license on HF; no restriction on trained weights found
+
+| Dataset | HF License Field | Original Source | Finding |
+|---------|-----------------|-----------------|---------|
+| ag_news | "unknown" | AG corpus (original site unreachable) | No terms found. Academic paper origin, widely redistributed. |
+| imdb | "other" | [Stanford](https://ai.stanford.edu/~amaas/data/sentiment/) | Original site requests citation only. No use restrictions stated. |
+| sst2 | "unknown" | Stanford NLP | No terms on HF or original page beyond citation. |
+| emotion | "other" | [HF](https://huggingface.co/datasets/dair-ai/emotion) | HF card states "educational and research purposes only." |
+
+### Unresolved: source terms require further review
+
+| Dataset | Issue | Detail |
+|---------|-------|--------|
+| emotion | HF card says "educational and research purposes only" | This may restrict commercial use of the dataset. Whether it applies to derived model weights (which contain no text) is not established. |
+| tweet_eval_emotion | Twitter API TOS required; per-subset license undefined | All tweet_eval subsets require Twitter TOS compliance. Our weights contain no tweet text. The sentiment subset is CC BY 3.0; the emotion and offensive subsets have undefined per-subset licenses. |
+| tweet_eval_offensive | Twitter API TOS required; per-subset license undefined | Same as above. |
+
+No dataset in this inventory explicitly prohibits distributing trained model
+weights. The three unresolved cases involve ambiguous or restrictive dataset-use
+terms whose applicability to derived numerical parameters has not been reviewed.
+
+## scikit-learn
+
+Bundled pretrained artifacts use numpy's .npz format for portability.
+Custom-trained models also save a joblib/pickle backup.
+scikit-learn (BSD-3-Clause) is required at runtime to reconstruct classifiers.
+
+## sentence-transformers
+
+The encoder is loaded via sentence-transformers, which is Apache 2.0 licensed.

jeffy_classify-0.1.0a8/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nicolas Brenner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

jeffy_classify-0.1.0a8/PKG-INFO

@@ -0,0 +1,325 @@
+Metadata-Version: 2.5
+Name: jeffy-classify
+Version: 0.1.0a8
+Summary: Pretrained text classifiers: 13 ready-to-use heads, train your own in seconds
+Project-URL: Homepage, https://github.com/nicobrenner/jeffy
+Project-URL: Repository, https://github.com/nicobrenner/jeffy
+Project-URL: Issues, https://github.com/nicobrenner/jeffy/issues
+Author: Nico Brenner
+License-Expression: MIT
+License-File: LICENSE
+Keywords: embeddings,machine-learning,nlp,pretrained,text-classification
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.11
+Requires-Dist: fastapi>=0.100.0
+Requires-Dist: joblib>=1.3.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: scipy>=1.10.0
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: uvicorn>=0.20.0
+Provides-Extra: build
+Requires-Dist: datasets>=2.14.0; extra == 'build'
+Provides-Extra: dev
+Requires-Dist: httpx>=0.24.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Jeffy
+
+Pretrained text classifiers you can run and retrain on CPU.
+
+<p align="center">
+  <img src="examples/inbox_demo.gif" width="100%" alt="Inbox Router">
+</p>
+<p align="center">
+  <img src="examples/doom_battle.gif" width="49%" alt="Doom Battle">
+  <img src="examples/doom_defend.gif" width="49%" alt="Defend the Center">
+</p>
+
+## Try it
+
+[Install uv](https://docs.astral.sh/uv/getting-started/installation/), then:
+
+```bash
+uvx --python 3.12 \
+  --from "jeffy-classify @ git+https://github.com/nicobrenner/jeffy.git@v0.1.0-alpha.7" \
+  jeffy-serve
+```
+
+Open http://localhost:8400, pick a classifier, and paste one of these:
+
+| Classifier | Try this text |
+|------------|---------------|
+| banking77 | I was charged twice for the same transaction |
+| sms_spam | WINNER! You have been selected for a free cruise. Reply YES to claim. |
+| ag_news | The Federal Reserve raised interest rates by 25 basis points on Wednesday |
+
+<p align="center">
+  <img src="examples/playground.png" width="100%" alt="Jeffy Playground">
+</p>
+
+## Pretrained capabilities
+
+13 classifiers ship with the package. Weights are logistic regression coefficients (derived model parameters, not copies of training data). Source datasets and licenses are documented in `ATTRIBUTION.md`.
+
+| Task | What it does | Classes | Test Acc | Test F1 |
+|------|-------------|---------|----------|---------|
+| sms_spam | SMS spam detection | 2 | 99.1% | 98.0% |
+| dbpedia | Wikipedia article category | 14 | 96.0% | 95.9% |
+| imdb | Movie review sentiment (long text) | 2 | 94.8% | 94.8% |
+| banking77 | Banking customer intent | 77 | 94.3% | 94.3% |
+| ag_news | News topic (world/sports/business/tech) | 4 | 90.5% | 90.5% |
+| sst2 | Movie review sentiment | 2 | 90.1% | 90.1% |
+| clinc_oos | Voice assistant intent + out-of-scope | 151 | 88.4% | 92.1% |
+| massive_intent | Smart home voice commands | 60 | 88.1% | 86.4% |
+| tweet_eval_offensive | Offensive language | 2 | 81.0% | 74.8% |
+| tweet_eval_emotion | Tweet emotion | 4 | 78.1% | 74.7% |
+| emotion | Text emotion (6 emotions) | 6 | 75.5% | 67.8% |
+| tweet_eval_sentiment | Tweet sentiment (3-way) | 3 | 66.2% | 65.7% |
+| snli | Natural language inference | 3 | 65.6% | 65.2% |
+
+Test accuracy on held-out splits. Details in `data/eval_results/benchmark.json`.
+
+**Weaknesses:** SNLI (65.6%) and tweet_eval_sentiment (66.2%) are below what task-specific models achieve. Emotion (75.5%) has limited class coverage. Probabilities are uncalibrated.
+
+## Train a custom classifier
+
+```bash
+uvx --python 3.12 \
+  --from "jeffy-classify @ git+https://github.com/nicobrenner/jeffy.git@v0.1.0-alpha.7" \
+  jeffy-train --example --save-dir my_models
+```
+
+```
+Loaded 24 examples from reviews.csv
+Training 'reviews': 24 examples, 2 classes
+  Split: 19 train, 5 test
+  Test accuracy: 100.0%
+Saved to my_models/reviews/
+```
+
+`--example` uses a bundled 24-row product review CSV. To bring your own:
+
+```bash
+uvx --python 3.12 \
+  --from "jeffy-classify @ git+https://github.com/nicobrenner/jeffy.git@v0.1.0-alpha.7" \
+  jeffy-train --input your_data.csv --text-col text --label-col label \
+  --task-id your_task --save-dir my_models
+```
+
+Supports `.csv`, `.tsv`, and `.jsonl`.
+
+## Serve a custom model
+
+```bash
+JEFFY_PACK_DIR=my_models uvx --python 3.12 \
+  --from "jeffy-classify @ git+https://github.com/nicobrenner/jeffy.git@v0.1.0-alpha.7" \
+  jeffy-serve
+```
+
+```bash
+curl -s -X POST http://localhost:8400/v1/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text": "The battery life is amazing", "task": "reviews"}'
+# → {"label": "positive", "confidence": 0.87, ...}
+```
+
+## Install from source
+
+```bash
+git clone https://github.com/nicobrenner/jeffy.git
+cd jeffy
+
+# With uv (recommended)
+uv venv && uv pip install -e .
+
+# Or with pip
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+```
+
+The first prediction downloads the shared encoder ([bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5), ~1.2 GB, cached afterward).
+
+## API
+
+### Python
+
+```python
+from jeffy.engine import Engine
+
+engine = Engine()
+engine.load()
+
+# List available classifiers
+for name, cap in engine.capabilities.items():
+    print(f"{name}: {cap.description} ({cap.n_classes} classes)")
+
+# Classify text
+result = engine.predict("banking77", "I was charged twice for the same transaction")
+print(result["label"])          # "transaction_charged_twice"
+print(result["confidence"])     # 0.999
+print(result["probabilities"])  # {"transaction_charged_twice": 0.999, ...}
+```
+
+### SDK walkthrough
+
+![SDK walkthrough](examples/demo.gif)
+
+### HTTP
+
+```bash
+# Banking intent
+curl -s -X POST http://localhost:8400/v1/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text": "I was charged twice for the same transaction", "task": "banking77"}'
+# → {"label": "transaction_charged_twice", "confidence": 0.999, ...}
+
+# Spam detection
+curl -s -X POST http://localhost:8400/v1/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text": "WINNER! You have been selected for a free cruise. Reply YES to claim.", "task": "sms_spam"}'
+# → {"label": "spam", "confidence": 0.91, ...}
+
+# News topic
+curl -s -X POST http://localhost:8400/v1/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text": "The Federal Reserve raised interest rates by 25 basis points on Wednesday", "task": "ag_news"}'
+# → {"label": "Business", "confidence": 0.86, ...}
+```
+
+### List capabilities
+
+```bash
+curl -s http://localhost:8400/v1/capabilities | python3 -c "
+import json, sys
+for c in json.load(sys.stdin)['capabilities']:
+    print(f\"{c['task_id']:25s} {c['n_classes']:3d} classes  {c['test_accuracy']:.1%}  {c['name']}\")"
+```
+
+### Per-capability metadata
+
+Each shipped classifier has a `manifest.json` with label names, source dataset, HuggingFace path, stated license, encoder identity, training/test counts, and integrity hashes.
+
+```bash
+curl -s http://localhost:8400/v1/capabilities/banking77 | python3 -m json.tool
+```
+
+## Train from Python
+
+```python
+from jeffy.train import train_classifier
+
+clf = train_classifier(
+    texts=["great product!", "terrible service", "fast shipping", "broken on arrival"],
+    labels=["positive", "negative", "positive", "negative"],
+    task_id="my_reviews",
+)
+
+result = clf.predict("the quality exceeded my expectations")
+print(result["label"])  # "positive"
+
+clf.save("my_models")
+```
+
+### Tuning options
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `C` | 0.01 | How aggressively the model fits your data. Low (0.001) = conservative, keeps predictions closer to "I'm not sure." High (1.0) = trusts individual training examples more. If the model is great on training data but bad on new data (overfitting), lower C. |
+| `test_size` | 0.2 | What fraction of your data to hold back for testing. With 100 examples at 0.2, it trains on 80 and tests on 20. Set to 0 to train on everything (useful when you have very little data and will test manually). |
+| `cv_folds` | 3 | Cross-validation: splits your training data into 3 parts, trains on 2 and tests on 1, rotates three times, averages the scores. Gives a more reliable accuracy estimate than a single split. Set to 0 to skip (faster, less reliable estimate). |
+
+Start with the defaults. With <50 examples per class, expect noisy estimates.
+
+## Reproduce the evaluation
+
+```bash
+# Install with build dependencies
+uv pip install -e ".[build]"
+# or: pip install -e ".[build]"
+
+# Retrain all 13 heads from source datasets (~40 min, downloads ~5 GB)
+jeffy-build --out data/model_pack
+
+# Evaluate on held-out test sets with tuned baselines
+jeffy-evaluate --baselines --latency --device cpu
+```
+
+## Evaluation notes
+
+- **SST-2**: Evaluated on `validation` split (official test labels are not public).
+- **SMS Spam**: Random split (test_size=0.2, seed=42); no standard benchmark split.
+- **SNLI**: Input encoded as `premise [SEP] hypothesis`. Label -1 filtered.
+- **CLINC-OOS**: 151 classes including out-of-scope. In-scope accuracy 96.5%, OOS detection 51.7%.
+- **MASSIVE**: English only (config `en`).
+
+## Deployment
+
+| Component | Size | Required for |
+|-----------|------|-------------|
+| Jeffy package (wheel) | 1.5 MB | Always (includes all 13 heads) |
+| Encoder (bge-large-en-v1.5) | ~1.2 GB | Inference (downloaded on first use) |
+| `datasets` package | ~100 MB | Retraining from HuggingFace only |
+
+Runtime memory: ~2 GB (encoder loaded once, shared across all heads).
+
+**Latency** (CPU, single example, Linux aarch64):
+
+| Stage | p50 | Notes |
+|-------|-----|-------|
+| Embedding | 50–80 ms | Dominates; varies with input length |
+| Classifier | <1 ms | Negligible |
+| Total | 50–80 ms | End-to-end |
+
+## Model security
+
+Bundled pretrained artifacts use numpy `.npz` format (portable, no pickle). Custom-trained models also save a joblib pickle backup. **Only load custom pickle artifacts from trusted sources.** Each artifact's `manifest.json` includes integrity hashes verified on load.
+
+## Provenance and licensing
+
+Jeffy code is MIT-licensed. Head artifacts are derived from public datasets; redistribution permissions have **not been independently verified** for all sources. See `ATTRIBUTION.md` for per-dataset license status.
+
+| Dataset | Stated license |
+|---------|---------------|
+| banking77, massive_intent, sms_spam | CC BY 4.0 |
+| clinc_oos | CC BY 3.0 |
+| dbpedia | CC BY-SA 3.0 |
+| snli | CC BY-SA 4.0 |
+| ag_news, imdb | Academic / non-commercial |
+| sst2 | Stanford academic license |
+| emotion | Academic |
+| tweet_eval_* | Twitter TOS / academic |
+
+The encoder ([bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5)) is MIT-licensed.
+
+## Tested
+
+Verified with clean-environment wheel and sdist install on Linux aarch64, Python 3.12, scikit-learn 1.9+, sentence-transformers 6.1+, numpy 2.5+. Pretrained artifacts use numpy `.npz` format, avoiding sklearn version coupling.
+
+## What's not included
+
+- **No zero-shot / general classification.** Each task needs a trained head. Unknown tasks return an error.
+- **No LLM fallback.** This release is pure embedding + classifier.
+- **No automatic task routing.** You must specify which classifier to use.
+
+## Roadmap
+
+| Status | Milestone |
+|--------|-----------|
+| **Available** | Pretrained classifier library, SDK/API, playground, custom training from CSV/JSONL |
+| **Next** | Landing page, PyPI release |
+| **Planned** | Broader classifier catalog, released in verified batches |
+| **Planned** | Automatic routing among supported classifiers |
+| **Planned** | Optional local/API LLM fallback for unsupported tasks |
+| **Planned** | Non-text classifiers (game state, sensor data, structured features) |
+| **Exploring** | Assisted labeling, retraining from corrections, classifier sharing |
+
+Suggestions for datasets, capabilities, or workflows are welcome as issues.