sigx 0.1.1__tar.gz

sigx-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 fengrru
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sigx-0.1.1/PKG-INFO

@@ -0,0 +1,549 @@
+Metadata-Version: 2.4
+Name: sigx
+Version: 0.1.1
+Summary: Implicit Feedback Signal Extraction for LLM Alignment
+Author: fengrru
+License: MIT
+Project-URL: Homepage, https://github.com/fengrru/sigx
+Project-URL: Repository, https://github.com/fengrru/sigx
+Project-URL: Issues, https://github.com/fengrru/sigx/issues
+Keywords: llm,alignment,dpo,kto,rlhf,preference-learning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.21.0
+Requires-Dist: scikit-learn>=1.0.0
+Provides-Extra: wildchat
+Requires-Dist: datasets>=2.0.0; extra == "wildchat"
+Provides-Extra: llm
+Requires-Dist: openai>=1.0.0; extra == "llm"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: pyright>=1.0.0; extra == "dev"
+Dynamic: license-file
+
+<div align="center">
+
+[![CI](https://github.com/Fengrru/sigx/actions/workflows/ci.yml/badge.svg)](https://github.com/Fengrru/sigx/actions/workflows/ci.yml)
+[![codecov](https://codecov.io/gh/Fengrru/sigx/branch/main/graph/badge.svg)](https://codecov.io/gh/Fengrru/sigx)
+![Python](https://img.shields.io/badge/python-3.8%2B-blue?style=flat&logo=python&logoColor=white)
+![License](https://img.shields.io/badge/license-MIT-green?style=flat)
+![F1](https://img.shields.io/badge/benchmark-F1%200.83-orange?style=flat)
+
+# SigX
+
+### Implicit Feedback Signal Extraction for LLM Alignment
+
+**Extract preference signals from conversation logs — no human annotation required.**
+
+[SigX Documentation](docs/index.md) | [Quick Start](#quick-start) | [API Reference](docs/api/pipeline.md) | [Contributing](CONTRIBUTING.md)
+
+</div>
+
+---
+
+## Why SigX?
+
+Traditional LLM alignment relies on **expensive human annotations** or **synthetic preferences from GPT-4**. But every production chat log already contains rich preference signals:
+
+| What Users Do | What It Means | SigX Signal |
+|:---|:---|:---|
+| Re-ask the same question | Previous answer was unhelpful | `rephrase` |
+| Say "Actually I meant..." | Model misunderstood | `correction` |
+| Say "That's wrong" | Explicit dissatisfaction | `negative` |
+| Say "Thanks! Perfect!" | Explicit satisfaction | `positive` |
+| Say "Never mind" and leave | User gave up | `abandon` |
+
+**SigX mines these signals at zero marginal cost** — turning your existing chat logs into DPO/KTO training data.
+
+> Inspired by [WildFeedback](https://arxiv.org/abs/2408.15549) (Microsoft Research, 2024) and the broader literature on learning from implicit preferences.
+
+---
+
+## Comparison with Related Tools
+
+| Feature | **SigX** | TRL (HuggingFace) | VeRL |
+|:---|:---:|:---:|:---:|
+| **Extract signals from raw logs** | **Yes** | No (expects labeled pairs) | No |
+| **Implicit feedback detection** | **Yes** | No | No |
+| **DPO/KTO output format** | **Yes** | Yes (training only) | Yes (training only) |
+| **Auto-infer chosen responses** | **Yes** | No | No |
+| **Pluggable extractors** | **Yes** | No | No |
+| **Built-in evaluation metrics** | **Yes** | No | No |
+| **Zero GPU required** | **Yes** | No | No |
+
+> SigX is not a training framework — it sits **upstream** of TRL/VeRL, producing the preference data they consume.
+
+---
+
+## Highlights
+
+<table>
+<tr>
+<td width="50%">
+
+### Core Capabilities
+
+- **Pluggable Extractors** — Mix regex, TF-IDF, ML, and LLM detectors
+- **Smart DPO Conversion** — Auto-infer `chosen` from subsequent positive turns
+- **Multi-Format Output** — DPO, KTO, rejection sampling for TRL/VeRL
+- **Built-in Evaluation** — Per-type precision/recall/F1 against benchmarks
+
+</td>
+<td width="50%">
+
+### Design Principles
+
+- **Lightweight** — Only `numpy` + `scikit-learn`. No GPU needed.
+- **Extensible** — Subclass `BaseExtractor` for custom detectors
+- **Production-ready** — Type annotations, 92 tests, CI/CD
+- **Framework-agnostic** — Output compatible with TRL, VeRL, and more
+
+</td>
+</tr>
+</table>
+
+---
+
+## Quick Start
+
+```bash
+pip install git+https://github.com/Fengrru/sigx.git
+```
+
+### 1. Extract Signals from Conversations
+
+```python
+from sigx import Pipeline, RephraseDetector, SentimentDetector
+
+pipeline = Pipeline([
+    RephraseDetector(similarity_threshold=0.6),
+    SentimentDetector(min_confidence=0.6),
+])
+
+conversations = [
+    {
+        "conversation_id": "1",
+        "conversation": [
+            {"role": "user", "content": "What is Python?"},
+            {"role": "assistant", "content": "Python is a type of snake."},
+            {"role": "user", "content": "That's not what I asked. I meant the programming language."},
+            {"role": "assistant", "content": "Python is a high-level programming language created by Guido van Rossum."},
+            {"role": "user", "content": "Thanks! That's exactly what I needed."},
+        ]
+    }
+]
+
+signals = pipeline.run(conversations)
+for s in signals:
+    print(f"[{s.signal_type}] confidence={s.confidence:.2f} | {s.evidence[:60]}...")
+```
+
+**Output:**
+```
+[negative] confidence=0.95 | That's not what I asked. I meant the programming language....
+[positive] confidence=0.80 | Thanks! That's exactly what I needed....
+```
+
+### 2. Convert to DPO Training Pairs
+
+```python
+pairs = pipeline.to_dpo(conversations)
+for p in pairs:
+    print(f"Prompt:   {p.prompt[:80]}...")
+    print(f"Rejected: {p.rejected[:80]}...")
+    print(f"Chosen:   {p.chosen[:80] if p.chosen else 'None'}...")
+```
+
+**Output:**
+```
+Prompt:   User: What is Python?...
+Rejected: Python is a type of snake....
+Chosen:   Python is a high-level programming language created by Guido van Rossum....
+```
+
+### 3. Feed to TRL for Training
+
+```python
+from trl import DPOTrainer
+
+trainer = DPOTrainer(
+    model=model,
+    train_dataset=pairs,  # SigX output works directly with TRL
+)
+trainer.train()
+```
+
+---
+
+## Installation
+
+> PyPI release coming soon. For now, install from source:
+
+| Install Command | Includes |
+|:---|:---|
+| `pip install git+https://github.com/Fengrru/sigx.git` | Core (numpy, scikit-learn) |
+| `pip install "sigx[llm] @ git+https://github.com/Fengrru/sigx.git"` | + OpenAI API for LLMExtractor |
+| `pip install "sigx[wildchat] @ git+https://github.com/Fengrru/sigx.git"` | + HuggingFace datasets for WildChat |
+| `git clone https://github.com/Fengrru/sigx.git && pip install -e ".[dev]"` | + pytest, ruff, mypy, pyright |
+
+---
+
+## Signal Types
+
+SigX detects five categories of implicit feedback:
+
+| Signal | Trigger Pattern | Confidence Range | Training Implication |
+|:---|:---|:---:|:---|
+| `rephrase` | User re-asks similar question (TF-IDF cosine sim) | 0.60 – 1.00 | Previous response → `rejected` |
+| `correction` | "Actually I meant...", "No, that's not..." | 0.60 – 0.95 | Model output → `rejected` |
+| `negative` | "That's wrong", "Not helpful" | 0.65 – 0.95 | Explicit dissatisfaction → `rejected` |
+| `positive` | "Thanks!", "Exactly what I needed" | 0.60 – 0.90 | Explicit satisfaction → `chosen` / `label=True` |
+| `abandon` | "Never mind", "I give up", trailing assistant | 0.35 – 0.90 | User gave up → `rejected` |
+
+---
+
+## Extractors
+
+### RephraseDetector
+
+Detects when users rephrase or repeat a question using TF-IDF cosine similarity.
+For CJK text (Chinese/Japanese/Korean) it automatically switches to character
+n-grams, since whitespace tokenization does not apply to those languages.
+
+```python
+from sigx import RephraseDetector
+
+detector = RephraseDetector(
+    similarity_threshold=0.6,   # Cosine similarity threshold
+    min_turn_length=20,         # Skip user turns shorter than this (chars)
+    skip_acknowledgments=True,  # Skip "thanks", "ok", etc.
+)
+```
+
+### SentimentDetector
+
+Hybrid regex + ML detection with 41 built-in patterns for corrections, negatives, positives, and sarcasm.
+
+```python
+from sigx import SentimentDetector
+
+# Rule-only mode (default, no extra deps)
+detector = SentimentDetector(min_confidence=0.6)
+
+# ML-enhanced mode (broader coverage)
+detector = SentimentDetector(use_ml=True, min_confidence=0.5)
+detector.fit(
+    texts=["that's wrong", "thanks!", "actually I meant..."],
+    labels=["negative", "positive", "correction"],
+)
+```
+
+### AbandonDetector
+
+Detects when users give up via explicit patterns or conversation structure analysis.
+
+```python
+from sigx import AbandonDetector
+
+detector = AbandonDetector(
+    min_assistant_length=300,           # Min chars for trailing-assistant
+    min_turns=3,                        # Min conversation length
+    require_unanswered_question=True,   # Only flag if user asked a question
+)
+```
+
+### LLMExtractor
+
+High-accuracy classification using any OpenAI-compatible LLM.
+
+```python
+from sigx import LLMExtractor
+
+detector = LLMExtractor(
+    model="gpt-4o-mini",            # Or "qwen2.5-7b-instruct" via vLLM
+    base_url=None,                  # Defaults to OpenAI; set for Ollama/vLLM
+    api_key="sk-...",               # Or set OPENAI_API_KEY env var
+    min_confidence=0.6,
+)
+```
+
+### Custom Extractors
+
+```python
+from sigx.extractors import BaseExtractor
+from sigx.types import Signal
+
+class ToxicityExtractor(BaseExtractor):
+    name = "toxicity"
+
+    def extract(self, conversation: dict) -> list[Signal]:
+        signals = []
+        # Your detection logic here
+        return signals
+
+pipeline = Pipeline([ToxicityExtractor()])
+```
+
+---
+
+## DPO Chosen Strategies
+
+A key innovation: SigX automatically infers the `chosen` response from subsequent conversation turns.
+
+| Strategy | Behavior | Use Case |
+|:---|:---|:---|
+| `subsequent` *(default)* | Find positive turn after negative signal; use assistant response before it as `chosen` | Best quality |
+| `last_assistant` | Use the final assistant response as `chosen` | Simple fallback |
+| `none` | `chosen=None` | Backward compatible |
+
+```python
+from sigx import Pipeline, CHOSEN_SUBSEQUENT, CHOSEN_NONE
+
+# Default: smart chosen inference
+pipeline = Pipeline(extractors=[...], chosen_strategy=CHOSEN_SUBSEQUENT)
+
+# Backward compatible: no chosen inference
+pipeline = Pipeline(extractors=[...], chosen_strategy=CHOSEN_NONE)
+```
+
+---
+
+## Data Loading
+
+SigX supports multiple conversation formats:
+
+```python
+from sigx import load_conversations, load_wildchat, stream_wildchat
+
+# ShareGPT format
+convos = load_conversations("sharegpt_data.json", format="sharegpt")
+
+# OpenAI chat format
+convos = load_conversations("openai_data.json", format="openai")
+
+# Generic JSONL
+convos = load_conversations("logs.jsonl", format="jsonl")
+
+# WildChat from HuggingFace (1M+ conversations)
+convos = load_wildchat(n=1000)
+
+# Stream large datasets (memory efficient)
+for convo in stream_wildchat(n=10000):
+    signals = pipeline.run([convo])
+```
+
+---
+
+## Output Formats
+
+### DPO (Direct Preference Optimization)
+
+```python
+pairs = pipeline.to_dpo(conversations)
+# List[PreferencePair] — compatible with TRL's DPOTrainer
+```
+
+### KTO (Kahneman-Tversky Optimization)
+
+```python
+examples = pipeline.to_kto(conversations)
+# List[KTOExample] — binary labels: True (desirable), False (undesirable)
+```
+
+### Rejection Sampling
+
+```python
+pairs = pipeline.to_rejection(conversations)
+# List[dict] with keys: prompt, rejected, signal_type, confidence
+```
+
+---
+
+## End-to-End Training Pipeline
+
+```mermaid
+%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#e3f2fd', 'primaryTextColor': '#1565c0', 'primaryBorderColor': '#1565c0', 'lineColor': '#90a4ae', 'secondaryColor': '#fff3e0', 'tertiaryColor': '#e8f5e9'}}}%%
+
+graph LR
+    subgraph INPUT [" "]
+        direction TB
+        I1["JSONL"]
+        I2["ShareGPT"]
+        I3["WildChat"]
+        I4["OpenAI"]
+    end
+
+    subgraph SIGX ["SigX"]
+        direction LR
+        S1["Extract"] --> S2["Filter"] --> S3["Convert"]
+    end
+
+    subgraph OUTPUT [" "]
+        direction TB
+        O1["DPO Pairs"]
+        O2["KTO Examples"]
+    end
+
+    INPUT --> SIGX --> OUTPUT
+
+    style INPUT fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#0d47a1
+    style SIGX fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#bf360c
+    style OUTPUT fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#1b5e20
+```
+
+```python
+from sigx import Pipeline, SentimentDetector, RephraseDetector, AbandonDetector
+from sigx.io import load_wildchat
+
+# 1. Configure pipeline
+pipeline = Pipeline([
+    SentimentDetector(min_confidence=0.6),
+    RephraseDetector(similarity_threshold=0.6),
+    AbandonDetector(min_turns=3),
+])
+
+# 2. Load conversation data
+convos = load_wildchat(n=10000)
+
+# 3. Convert to DPO training pairs
+dpo_pairs = pipeline.to_dpo(convos)
+
+# 4. Feed directly to TRL
+from trl import DPOTrainer
+trainer = DPOTrainer(model=model, train_dataset=dpo_pairs)
+trainer.train()
+```
+
+---
+
+## Evaluation
+
+Evaluate extraction quality against a labeled benchmark:
+
+```python
+metrics = pipeline.evaluate("benchmark.json")
+
+print(f"Overall F1: {metrics['overall']['f1']:.4f}")
+# Per-type breakdown
+for signal_type, m in metrics["per_type"].items():
+    print(f"  {signal_type}: P={m['precision']:.4f} R={m['recall']:.4f} F1={m['f1']:.4f}")
+```
+
+### Benchmark Results
+
+Evaluated on a 20-conversation benchmark with regex-based extractors (default settings, no LLM):
+
+![Benchmark results](docs/assets/benchmark.png)
+
+> Reproduce with `pipeline.evaluate("tests/benchmark.json")`; regenerate the chart with `python scripts/generate_assets.py`. Adding `LLMExtractor` significantly improves recall on `negative` and `rephrase` types.
+
+---
+
+## Architecture
+
+```
+sigx/
+├── pipeline.py          # Orchestration: extract → filter → convert
+├── types.py             # Core dataclasses: Signal, PreferencePair, KTOExample
+├── exceptions.py        # Custom exception hierarchy
+├── extractors/
+│   ├── base.py          # Abstract BaseExtractor
+│   ├── rephrase.py      # TF-IDF cosine similarity rephrase detection
+│   ├── sentiment.py     # Regex + optional ML sentiment classifier
+│   ├── abandon.py       # Frustration pattern + trailing-assistant detection
+│   └── llm.py           # OpenAI-compatible LLM classifier
+├── filters/
+│   └── quality.py       # Confidence threshold, dedup, per-conv limits
+├── converters/
+│   └── preference.py    # DPO / KTO / rejection-sampling converters
+└── io/
+    └── loader.py        # ShareGPT, OpenAI, WildChat, JSONL loaders
+```
+
+---
+
+## Research Background
+
+SigX is built on growing evidence that **implicit feedback from real user interactions produces better alignment data** than synthetic alternatives:
+
+| Paper | Institution | Year | Key Finding |
+|:---|:---|:---:|:---|
+| [WildFeedback](https://arxiv.org/abs/2408.15549) | Microsoft Research | 2024 | 20K preference pairs from 148K ChatGPT conversations outperformed UltraFeedback on AlpacaEval 2, Arena-Hard, MT-Bench |
+
+**The key insight**: Real user feedback captures nuance that synthetic data misses, and the signals are already present in existing chat logs — they just need to be extracted.
+
+### How SigX Relates
+
+| Project | Focus | SigX Connection |
+|:---|:---|:---|
+| [WildFeedback](https://arxiv.org/abs/2408.15549) | Extracting preference pairs from in-situ feedback | SigX implements the extraction pipeline in Python |
+| [TRL](https://github.com/huggingface/trl) | Post-training (DPO/KTO/PPO) | SigX output is TRL-compatible |
+| [VeRL](https://github.com/volcengine/verl) | RLHF framework | SigX output works with VeRL |
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/fengrru/sigx.git
+cd sigx
+pip install -e ".[dev]"
+
+# Run tests
+pytest                    # 92 tests
+pytest --cov=sigx         # With coverage
+
+# Lint
+ruff check .              # Lint
+ruff format .             # Format
+
+# Type check
+mypy sigx/                # mypy
+pyright                   # pyright
+```
+
+---
+
+## Citation
+
+```bibtex
+@software{sigx2026,
+  title   = {{SigX: Implicit Feedback Signal Extraction for LLM Alignment}},
+  author  = {fengrru},
+  license = {MIT},
+  url     = {https://github.com/fengrru/sigx},
+  year    = {2026},
+}
+```
+
+---
+
+## License
+
+MIT License — see [LICENSE](LICENSE) for details.
+
+---
+
+<div align="center">
+
+**Built with care for the LLM alignment community.**
+
+[Report Bug](https://github.com/fengrru/sigx/issues) · [Request Feature](https://github.com/fengrru/sigx/issues) · [Contributing Guide](CONTRIBUTING.md)
+
+</div>