language-ninja 2.2.0__tar.gz

language_ninja-2.2.0/PKG-INFO

@@ -0,0 +1,113 @@
+Metadata-Version: 2.4
+Name: language-ninja
+Version: 2.2.0
+Summary: Keyboard-first Textual NLP sentiment training, analysis, notes, and export workstation.
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: textual>=0.70
+Requires-Dist: scikit-learn>=1.3
+
+# 🥷 Language Trainer Ninja v2.2
+
+A keyboard-first Textual NLP workstation for sentiment analysis, model training, dataset inspection, notes, history, SQLite queries, and CSV/JSON export.
+
+## v2.2 model
+
+The bundled model is now a calibrated classical NLP pipeline rather than the earlier small Naive Bayes/logistic baseline:
+
+```text
+Word TF-IDF (1–3 grams) ─┐
+                         ├─ FeatureUnion ─ Linear SVM ─ 5-fold sigmoid calibration ─ probabilities
+Char TF-IDF (3–5 grams) ─┘
+```
+
+Why both feature families:
+
+- Word n-grams capture phrases such as `not good`, `really love`, and longer contrast patterns.
+- Character n-grams improve handling of contractions, punctuation, spelling variation, and unseen word forms.
+- A balanced Linear SVM supplies a strong decision boundary for sparse text.
+- Sigmoid calibration turns SVM margins into class probabilities for the confidence display.
+
+The starter corpus contains **1,500 balanced examples**:
+
+```text
+positive  500
+neutral   500
+negative  500
+```
+
+It includes straightforward sentiment, conversational phrasing, negation, contrast, and neutral operational language.
+
+> The built-in holdout score is useful as a regression check, but the starter corpus contains generated/template-assisted examples. Treat its very high holdout accuracy as an internal sanity metric, not as proof of production-level real-world accuracy.
+
+## Install
+
+```bash
+cd ~/Downloads/language-trainer-ninja-master
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+python -m pip install -e .
+language-ninja
+```
+
+A trained `models/sentiment_model.pkl` is included, so Analyze can work immediately. Use **Train → Train Model** after changing the dataset.
+
+## Confidence
+
+The TUI displays all three calibrated probabilities:
+
+```text
+Prediction: NEGATIVE • HIGH CONFIDENCE
+Winning probability: 82.8%
+POS 9.5%   NEU 7.6%   NEG 82.8%
+```
+
+UI guidance:
+
+```text
+HIGH       75–100%
+MODERATE   55–74%
+LOW        below 55%
+```
+
+These labels describe the model's probability separation, not a guarantee of correctness.
+
+## Pages
+
+### Analyze
+Paste conversational text and inspect the winning class plus positive, neutral, and negative probabilities. Every prediction is written to History.
+
+### Train
+Train from a `text,label` CSV. The status panel reports row counts, algorithm, feature count, holdout accuracy, macro F1, and per-class precision/recall/F1.
+
+### Dataset
+Inspect the examples used to train the model and export them to CSV or JSON.
+
+### Notes
+Save experiment notes or attach a note to the most recent prediction.
+
+### History
+Review every prediction with **POS / NEU / NEG probabilities** and a HIGH/MODERATE/LOW confidence level.
+
+### Database
+Run read-only `SELECT`, `WITH`, and `PRAGMA` queries against the SQLite database.
+
+### Export
+Export predictions, notes, datasets, and query results to `./exports/`.
+
+## Rebuild the bundled corpus
+
+The deterministic corpus generator is included:
+
+```bash
+python tools/build_starter_corpus.py
+```
+
+Then retrain from the TUI, or use the Python ML module.
+
+## Run tests
+
+```bash
+python -m pytest -q
+```

language_ninja-2.2.0/README.md

@@ -0,0 +1,104 @@
+# 🥷 Language Trainer Ninja v2.2
+
+A keyboard-first Textual NLP workstation for sentiment analysis, model training, dataset inspection, notes, history, SQLite queries, and CSV/JSON export.
+
+## v2.2 model
+
+The bundled model is now a calibrated classical NLP pipeline rather than the earlier small Naive Bayes/logistic baseline:
+
+```text
+Word TF-IDF (1–3 grams) ─┐
+                         ├─ FeatureUnion ─ Linear SVM ─ 5-fold sigmoid calibration ─ probabilities
+Char TF-IDF (3–5 grams) ─┘
+```
+
+Why both feature families:
+
+- Word n-grams capture phrases such as `not good`, `really love`, and longer contrast patterns.
+- Character n-grams improve handling of contractions, punctuation, spelling variation, and unseen word forms.
+- A balanced Linear SVM supplies a strong decision boundary for sparse text.
+- Sigmoid calibration turns SVM margins into class probabilities for the confidence display.
+
+The starter corpus contains **1,500 balanced examples**:
+
+```text
+positive  500
+neutral   500
+negative  500
+```
+
+It includes straightforward sentiment, conversational phrasing, negation, contrast, and neutral operational language.
+
+> The built-in holdout score is useful as a regression check, but the starter corpus contains generated/template-assisted examples. Treat its very high holdout accuracy as an internal sanity metric, not as proof of production-level real-world accuracy.
+
+## Install
+
+```bash
+cd ~/Downloads/language-trainer-ninja-master
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+python -m pip install -e .
+language-ninja
+```
+
+A trained `models/sentiment_model.pkl` is included, so Analyze can work immediately. Use **Train → Train Model** after changing the dataset.
+
+## Confidence
+
+The TUI displays all three calibrated probabilities:
+
+```text
+Prediction: NEGATIVE • HIGH CONFIDENCE
+Winning probability: 82.8%
+POS 9.5%   NEU 7.6%   NEG 82.8%
+```
+
+UI guidance:
+
+```text
+HIGH       75–100%
+MODERATE   55–74%
+LOW        below 55%
+```
+
+These labels describe the model's probability separation, not a guarantee of correctness.
+
+## Pages
+
+### Analyze
+Paste conversational text and inspect the winning class plus positive, neutral, and negative probabilities. Every prediction is written to History.
+
+### Train
+Train from a `text,label` CSV. The status panel reports row counts, algorithm, feature count, holdout accuracy, macro F1, and per-class precision/recall/F1.
+
+### Dataset
+Inspect the examples used to train the model and export them to CSV or JSON.
+
+### Notes
+Save experiment notes or attach a note to the most recent prediction.
+
+### History
+Review every prediction with **POS / NEU / NEG probabilities** and a HIGH/MODERATE/LOW confidence level.
+
+### Database
+Run read-only `SELECT`, `WITH`, and `PRAGMA` queries against the SQLite database.
+
+### Export
+Export predictions, notes, datasets, and query results to `./exports/`.
+
+## Rebuild the bundled corpus
+
+The deterministic corpus generator is included:
+
+```bash
+python tools/build_starter_corpus.py
+```
+
+Then retrain from the TUI, or use the Python ML module.
+
+## Run tests
+
+```bash
+python -m pytest -q
+```

language_ninja-2.2.0/language_ninja/__init__.py

@@ -0,0 +1,3 @@
+"""Language Trainer Ninja TUI."""
+
+__version__ = "2.2.0"

language_ninja-2.2.0/language_ninja/app.py

@@ -0,0 +1,420 @@
+from __future__ import annotations
+
+import csv
+from pathlib import Path
+
+from textual.app import App, ComposeResult
+from textual.containers import Horizontal, Vertical
+from textual.widgets import (
+    Button,
+    DataTable,
+    Footer,
+    Header,
+    Input,
+    Label,
+    Select,
+    Static,
+    TabbedContent,
+    TabPane,
+    TextArea,
+)
+
+from .ml import load_dataset, load_model, predict, save_model, train_and_evaluate
+from .storage import Storage, export_records, export_table
+
+
+ROOT = Path.cwd()
+DATA_DIR = ROOT / "data"
+MODEL_DIR = ROOT / "models"
+EXPORT_DIR = ROOT / "exports"
+DB_PATH = ROOT / "language_ninja.db"
+DEFAULT_DATASET = DATA_DIR / "propositions.csv"
+DEFAULT_MODEL = MODEL_DIR / "sentiment_model.pkl"
+
+
+class LanguageNinja(App):
+    TITLE = "🥷 Language Trainer Ninja"
+    SUB_TITLE = "NLP Training • Analysis • Notes • Export"
+
+    CSS = """
+    Screen { layout: vertical; }
+    TabPane { padding: 1 2; }
+    .section { height: auto; margin-bottom: 1; }
+    .panel { border: round $primary; padding: 1; height: auto; }
+    .instructions { border: round $secondary; padding: 1; margin-bottom: 1; height: auto; }
+    .grow { height: 1fr; }
+    .status { height: 7; border: round $accent; padding: 1; }
+    #train_status { height: 14; }
+    .compact { width: 1fr; }
+    Button { margin-right: 1; }
+    DataTable { height: 1fr; }
+    TextArea { height: 10; }
+    #analysis_input { height: 8; }
+    #note_body { height: 12; }
+    #sql_input { height: 5; }
+    """
+
+    BINDINGS = [
+        ("a", "tab('analyze')", "Analyze"),
+        ("t", "tab('train')", "Train"),
+        ("d", "tab('dataset')", "Dataset"),
+        ("n", "tab('notes')", "Notes"),
+        ("h", "tab('history')", "History"),
+        ("e", "tab('export')", "Export"),
+        ("q", "quit", "Quit"),
+    ]
+
+    def __init__(self):
+        super().__init__()
+        self.storage = Storage(DB_PATH)
+        self.classifier = None
+        self.vectorizer = None
+        self.active_model = DEFAULT_MODEL
+        self.last_prediction_id: int | None = None
+        self.query_columns: list[str] = []
+        self.query_rows: list[tuple] = []
+
+    def compose(self) -> ComposeResult:
+        yield Header(show_clock=True)
+        with TabbedContent(initial="analyze", id="tabs"):
+            with TabPane("Analyze", id="analyze"):
+                yield Static(
+                    "HOW TO USE • Type or paste a sentence below, then choose Analyze. "
+                    "The highest calibrated probability becomes the prediction. HIGH means 75%+, MODERATE 55–74%, and LOW below 55%. "
+                    "All three probabilities are saved to History so uncertainty stays visible.",
+                    classes="instructions",
+                )
+                yield Label("Text to analyze")
+                yield TextArea(id="analysis_input")
+                with Horizontal(classes="section"):
+                    yield Button("Analyze", id="analyze_btn", variant="primary")
+                    yield Button("Clear", id="clear_analysis")
+                yield Static("Ready. Train or load a model to begin.", id="analysis_result", classes="status")
+
+            with TabPane("Train", id="train"):
+                yield Static(
+                    "HOW TO USE • Choose a CSV containing text,label columns and select Train Model. "
+                    "The v2.2 trainer uses word/phrase TF-IDF + character TF-IDF feeding a calibrated Linear SVM. "
+                    "It evaluates on a stratified holdout set, reports per-class quality, then retrains the active model on all rows.",
+                    classes="instructions",
+                )
+                yield Label("Dataset path")
+                yield Input(str(DEFAULT_DATASET), id="dataset_path")
+                yield Label("Model output path")
+                yield Input(str(DEFAULT_MODEL), id="model_path")
+                with Horizontal(classes="section"):
+                    yield Button("Train Model", id="train_btn", variant="primary")
+                    yield Button("Load Model", id="load_model_btn")
+                yield Static("No training run yet.", id="train_status", classes="status")
+
+            with TabPane("Dataset", id="dataset"):
+                yield Static(
+                    "HOW TO USE • Review the examples that teach the model. Each row needs text and a sentiment label. "
+                    "The bundled corpus contains 1,500 balanced conversational examples, including negation and mixed-language cases. "
+                    "Refresh after editing the CSV externally, or export a copy for backup.",
+                    classes="instructions",
+                )
+                with Horizontal(classes="section"):
+                    yield Button("Refresh", id="dataset_refresh", variant="primary")
+                    yield Button("Export CSV", id="dataset_export_csv")
+                    yield Button("Export JSON", id="dataset_export_json")
+                yield DataTable(id="dataset_table", zebra_stripes=True)
+
+            with TabPane("Notes", id="notes"):
+                yield Static(
+                    "HOW TO USE • Record observations, experiments, or corrections. Save Note stores a general note. "
+                    "Link to Last Prediction attaches the note to the most recent analysis so you can document mistakes or edge cases.",
+                    classes="instructions",
+                )
+                yield Label("Title")
+                yield Input(placeholder="Experiment, correction, observation...", id="note_title")
+                yield Label("Note")
+                yield TextArea(id="note_body")
+                with Horizontal(classes="section"):
+                    yield Button("Save Note", id="save_note", variant="primary")
+                    yield Button("Link to Last Prediction", id="link_note")
+                    yield Button("Export Notes", id="notes_export")
+                yield DataTable(id="notes_table", zebra_stripes=True)
+
+            with TabPane("History", id="history"):
+                yield Static(
+                    "HOW TO USE • Review past predictions, confidence scores, and the model used. "
+                    "Use this page to spot repeated low-confidence cases and export prediction history for deeper analysis.",
+                    classes="instructions",
+                )
+                with Horizontal(classes="section"):
+                    yield Button("Refresh", id="history_refresh", variant="primary")
+                    yield Button("Export CSV", id="history_export_csv")
+                    yield Button("Export JSON", id="history_export_json")
+                yield DataTable(id="history_table", zebra_stripes=True)
+
+            with TabPane("Database", id="database"):
+                yield Static(
+                    "HOW TO USE • Run read-only SELECT queries against the app database. "
+                    "Use this for advanced inspection of predictions and notes. Query results can be exported to CSV or JSON.",
+                    classes="instructions",
+                )
+                yield Label("Read-only SQL")
+                yield TextArea("SELECT * FROM predictions ORDER BY id DESC LIMIT 25;", id="sql_input")
+                with Horizontal(classes="section"):
+                    yield Button("Run Query", id="query_run", variant="primary")
+                    yield Button("Export Query CSV", id="query_export_csv")
+                    yield Button("Export Query JSON", id="query_export_json")
+                yield DataTable(id="query_table", zebra_stripes=True)
+
+            with TabPane("Export", id="export"):
+                yield Static(
+                    "HOW TO USE • Choose what you want to export and select CSV or JSON, then choose Export Now. "
+                    "Files are written to the project's ./exports/ directory.",
+                    classes="instructions",
+                )
+                yield Static(
+                    "Exports are written to ./exports/\n\n"
+                    "Available exports:\n"
+                    "• Prediction history → CSV / JSON\n"
+                    "• Notes → JSON\n"
+                    "• Dataset → CSV / JSON\n"
+                    "• Database query results → CSV / JSON\n",
+                    classes="panel",
+                )
+                with Horizontal(classes="section"):
+                    yield Select(
+                        [("Predictions", "predictions"), ("Notes", "notes"), ("Dataset", "dataset")],
+                        value="predictions",
+                        id="export_source",
+                    )
+                    yield Select([("CSV", "csv"), ("JSON", "json")], value="csv", id="export_format")
+                    yield Button("Export Now", id="export_now", variant="primary")
+                yield Static("Nothing exported yet.", id="export_status", classes="status")
+        yield Footer()
+
+    def on_mount(self) -> None:
+        self._setup_tables()
+        self.refresh_dataset()
+        self.refresh_history()
+        self.refresh_notes()
+        if self.active_model.exists():
+            self._load_model(self.active_model)
+
+    def _setup_tables(self) -> None:
+        dataset = self.query_one("#dataset_table", DataTable)
+        dataset.add_columns("#", "Text", "Label")
+        history = self.query_one("#history_table", DataTable)
+        history.add_columns("ID", "Time", "Text", "Result", "POS", "NEU", "NEG", "Level")
+        notes = self.query_one("#notes_table", DataTable)
+        notes.add_columns("ID", "Time", "Title", "Body", "Prediction")
+
+    def action_tab(self, tab_id: str) -> None:
+        self.query_one("#tabs", TabbedContent).active = tab_id
+
+    def _notify_error(self, message: str) -> None:
+        self.notify(message, severity="error", timeout=5)
+
+    def _load_model(self, path: Path) -> None:
+        try:
+            self.classifier, self.vectorizer = load_model(path)
+            self.active_model = path
+            self.query_one("#train_status", Static).update(f"Loaded model: {path}")
+        except Exception as exc:
+            self._notify_error(f"Could not load model: {exc}")
+
+    def refresh_dataset(self) -> None:
+        table = self.query_one("#dataset_table", DataTable)
+        table.clear()
+        path = Path(self.query_one("#dataset_path", Input).value or DEFAULT_DATASET)
+        try:
+            with path.open("r", encoding="utf-8", newline="") as handle:
+                reader = csv.reader(handle)
+                next(reader, None)
+                for index, row in enumerate(reader, start=1):
+                    if len(row) >= 2:
+                        table.add_row(str(index), row[0], row[1])
+        except Exception as exc:
+            self._notify_error(f"Dataset: {exc}")
+
+    def refresh_history(self) -> None:
+        table = self.query_one("#history_table", DataTable)
+        table.clear()
+        for row in self.storage.recent_predictions():
+            table.add_row(
+                str(row["id"]),
+                row["timestamp"],
+                row["text"],
+                row["sentiment"].upper(),
+                f'{row["positive_probability"]:.1%}',
+                f'{row["neutral_probability"]:.1%}',
+                f'{row["negative_probability"]:.1%}',
+                row["confidence_level"],
+            )
+
+    def refresh_notes(self) -> None:
+        table = self.query_one("#notes_table", DataTable)
+        table.clear()
+        for row in self.storage.recent_notes():
+            table.add_row(
+                str(row["id"]), row["timestamp"], row["title"], row["body"], str(row["linked_prediction_id"] or "")
+            )
+
+    def _export_path(self, stem: str, fmt: str) -> Path:
+        EXPORT_DIR.mkdir(parents=True, exist_ok=True)
+        return EXPORT_DIR / f"{stem}.{fmt}"
+
+    def export_predictions(self, fmt: str) -> Path:
+        return export_records(self.storage.recent_predictions(100000), self._export_path("predictions", fmt), fmt)
+
+    def export_notes(self, fmt: str) -> Path:
+        return export_records(self.storage.recent_notes(100000), self._export_path("notes", fmt), fmt)
+
+    def export_dataset(self, fmt: str) -> Path:
+        path = Path(self.query_one("#dataset_path", Input).value or DEFAULT_DATASET)
+        with path.open("r", encoding="utf-8", newline="") as handle:
+            reader = csv.DictReader(handle)
+            records = list(reader)
+        return export_records(records, self._export_path("dataset", fmt), fmt)
+
+    async def on_button_pressed(self, event: Button.Pressed) -> None:
+        button_id = event.button.id
+        try:
+            if button_id == "analyze_btn":
+                self.analyze_text()
+            elif button_id == "clear_analysis":
+                self.query_one("#analysis_input", TextArea).text = ""
+                self.query_one("#analysis_result", Static).update("Ready.")
+            elif button_id == "train_btn":
+                self.train_active_dataset()
+            elif button_id == "load_model_btn":
+                self._load_model(Path(self.query_one("#model_path", Input).value))
+            elif button_id == "dataset_refresh":
+                self.refresh_dataset()
+            elif button_id == "history_refresh":
+                self.refresh_history()
+            elif button_id == "save_note":
+                self.save_note(link=False)
+            elif button_id == "link_note":
+                self.save_note(link=True)
+            elif button_id == "notes_export":
+                path = self.export_notes("json")
+                self.notify(f"Exported {path}")
+            elif button_id == "dataset_export_csv":
+                self.notify(f"Exported {self.export_dataset('csv')}")
+            elif button_id == "dataset_export_json":
+                self.notify(f"Exported {self.export_dataset('json')}")
+            elif button_id == "history_export_csv":
+                self.notify(f"Exported {self.export_predictions('csv')}")
+            elif button_id == "history_export_json":
+                self.notify(f"Exported {self.export_predictions('json')}")
+            elif button_id == "query_run":
+                self.run_query()
+            elif button_id == "query_export_csv":
+                self.export_query("csv")
+            elif button_id == "query_export_json":
+                self.export_query("json")
+            elif button_id == "export_now":
+                self.export_selected()
+        except Exception as exc:
+            self._notify_error(str(exc))
+
+    def train_active_dataset(self) -> None:
+        dataset_path = Path(self.query_one("#dataset_path", Input).value)
+        model_path = Path(self.query_one("#model_path", Input).value)
+        texts, labels = load_dataset(dataset_path)
+        classifier, vectorizer, report = train_and_evaluate(texts, labels)
+        save_model(model_path, classifier, vectorizer)
+        self.classifier, self.vectorizer = classifier, vectorizer
+        self.active_model = model_path
+        if report.accuracy is None:
+            metrics = "Evaluation: add more examples per class for a reliable holdout score"
+            per_class = ""
+        else:
+            metrics = f"Holdout accuracy: {report.accuracy:.1%} • Macro F1: {report.macro_f1:.1%}"
+            per_class = "\n".join(
+                f"{label.upper():8} P {values['precision']:.1%}  R {values['recall']:.1%}  F1 {values['f1']:.1%}"
+                for label, values in report.class_metrics.items()
+            )
+        self.query_one("#train_status", Static).update(
+            "TRAINING COMPLETE\n"
+            f"Examples: {report.rows} • {report.labels}\n"
+            f"Model: {report.algorithm}\n"
+            f"Features: {report.feature_count:,} • Word TF-IDF 1–3 grams • Char TF-IDF 3–5 grams\n"
+            f"{metrics}\n{per_class}\n"
+            f"Saved: {model_path}"
+        )
+        self.notify("Model trained and activated.")
+
+    def analyze_text(self) -> None:
+        text = self.query_one("#analysis_input", TextArea).text.strip()
+        if not text:
+            raise ValueError("Enter text to analyze.")
+        if self.classifier is None or self.vectorizer is None:
+            raise ValueError("Train or load a model first.")
+        result = predict(text, self.classifier, self.vectorizer)
+        probabilities = result.probabilities
+        self.query_one("#analysis_result", Static).update(
+            f"Prediction: {result.label.upper()} • {result.confidence_level} CONFIDENCE\n"
+            f"Winning probability: {result.confidence:.1%}\n"
+            f"POS {probabilities.get('positive', 0.0):.1%}   "
+            f"NEU {probabilities.get('neutral', 0.0):.1%}   "
+            f"NEG {probabilities.get('negative', 0.0):.1%}"
+        )
+        self.last_prediction_id = self.storage.add_prediction(
+            self.active_model.name,
+            text,
+            result.label,
+            result.confidence,
+            probabilities=result.probabilities,
+            confidence_level=result.confidence_level,
+        )
+        self.refresh_history()
+
+    def save_note(self, link: bool) -> None:
+        title = self.query_one("#note_title", Input).value.strip() or "Untitled note"
+        body = self.query_one("#note_body", TextArea).text.strip()
+        if not body:
+            raise ValueError("Write a note before saving.")
+        linked = self.last_prediction_id if link else None
+        if link and linked is None:
+            raise ValueError("There is no prediction to link this note to yet.")
+        self.storage.add_note(title, body, linked)
+        self.query_one("#note_title", Input).value = ""
+        self.query_one("#note_body", TextArea).text = ""
+        self.refresh_notes()
+        self.notify("Note saved.")
+
+    def run_query(self) -> None:
+        query = self.query_one("#sql_input", TextArea).text.strip()
+        columns, rows = self.storage.execute_read_query(query)
+        self.query_columns = list(columns)
+        self.query_rows = [tuple(row) for row in rows]
+        table = self.query_one("#query_table", DataTable)
+        table.clear(columns=True)
+        if columns:
+            table.add_columns(*columns)
+        for row in rows:
+            table.add_row(*[str(value) for value in row])
+        self.notify(f"Query returned {len(rows)} row(s).")
+
+    def export_query(self, fmt: str) -> None:
+        if not self.query_columns:
+            self.run_query()
+        path = export_table(self.query_columns, self.query_rows, self._export_path("query-results", fmt), fmt)
+        self.notify(f"Exported {path}")
+
+    def export_selected(self) -> None:
+        source = self.query_one("#export_source", Select).value
+        fmt = self.query_one("#export_format", Select).value
+        if source == "predictions":
+            path = self.export_predictions(str(fmt))
+        elif source == "notes":
+            path = self.export_notes(str(fmt))
+        else:
+            path = self.export_dataset(str(fmt))
+        self.query_one("#export_status", Static).update(f"Export complete:\n{path}")
+        self.notify(f"Exported {path}")
+
+
+def main() -> None:
+    LanguageNinja().run()
+
+
+if __name__ == "__main__":
+    main()

language_ninja-2.2.0/language_ninja/ml.py

@@ -0,0 +1,224 @@
+from __future__ import annotations
+
+import csv
+import pickle
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Iterable
+
+from sklearn.calibration import CalibratedClassifierCV
+from sklearn.feature_extraction.text import TfidfVectorizer
+from sklearn.linear_model import LogisticRegression
+from sklearn.metrics import accuracy_score, classification_report, confusion_matrix, f1_score
+from sklearn.model_selection import train_test_split
+from sklearn.pipeline import FeatureUnion
+from sklearn.svm import LinearSVC
+
+MODEL_VERSION = "2.2"
+
+
+@dataclass(slots=True)
+class Prediction:
+    label: str
+    confidence: float
+    probabilities: dict[str, float]
+
+    @property
+    def confidence_level(self) -> str:
+        if self.confidence >= 0.75:
+            return "HIGH"
+        if self.confidence >= 0.55:
+            return "MODERATE"
+        return "LOW"
+
+
+@dataclass(slots=True)
+class TrainingReport:
+    rows: int
+    labels: dict[str, int]
+    accuracy: float | None
+    macro_f1: float | None
+    report: str
+    feature_count: int
+    algorithm: str
+    class_metrics: dict[str, dict[str, float]] = field(default_factory=dict)
+    confusion: list[list[int]] = field(default_factory=list)
+    label_order: list[str] = field(default_factory=list)
+
+
+def load_dataset(path: str | Path) -> tuple[list[str], list[str]]:
+    texts: list[str] = []
+    labels: list[str] = []
+    with Path(path).open("r", encoding="utf-8", newline="") as handle:
+        reader = csv.DictReader(handle)
+        if not reader.fieldnames or "text" not in reader.fieldnames or "label" not in reader.fieldnames:
+            raise ValueError("Dataset must contain text,label columns.")
+        for row in reader:
+            text = (row.get("text") or "").strip()
+            label = (row.get("label") or "").strip().lower()
+            if not text or not label:
+                continue
+            texts.append(text)
+            labels.append(label)
+    if not texts:
+        raise ValueError("Dataset contains no usable text/label rows.")
+    if len(set(labels)) < 2:
+        raise ValueError("Dataset needs at least two different labels.")
+    return texts, labels
+
+
+def build_vectorizer() -> FeatureUnion:
+    # Word n-grams capture phrases such as "not good", "really love", and
+    # contrast constructions. Character n-grams improve resilience to spelling,
+    # contractions, punctuation, and unseen word forms.
+    word = TfidfVectorizer(
+        lowercase=True,
+        analyzer="word",
+        ngram_range=(1, 3),
+        sublinear_tf=True,
+        min_df=1,
+        max_df=0.995,
+        max_features=30000,
+        strip_accents="unicode",
+    )
+    char = TfidfVectorizer(
+        lowercase=True,
+        analyzer="char_wb",
+        ngram_range=(3, 5),
+        sublinear_tf=True,
+        min_df=1,
+        max_features=30000,
+        strip_accents="unicode",
+    )
+    return FeatureUnion(
+        [("word", word), ("char", char)],
+        transformer_weights={"word": 1.4, "char": 0.7},
+    )
+
+
+def _fit_classifier(matrix, labels: list[str]):
+    counts = {label: labels.count(label) for label in set(labels)}
+    min_class = min(counts.values())
+
+    if min_class >= 5:
+        # Linear SVM gives a strong text decision boundary. Sigmoid calibration
+        # converts its margins into probabilities that are more meaningful for
+        # the TUI confidence display.
+        base = LinearSVC(C=1.5, class_weight="balanced", random_state=42)
+        classifier = CalibratedClassifierCV(base, method="sigmoid", cv=5)
+        classifier.fit(matrix, labels)
+        return classifier, "Calibrated Linear SVM (5-fold sigmoid)"
+
+    # Small custom datasets cannot support 5-fold calibration. Keep a reliable
+    # probability-capable fallback so users can still train their own corpus.
+    classifier = LogisticRegression(
+        max_iter=3000,
+        C=2.0,
+        class_weight="balanced",
+        solver="lbfgs",
+        random_state=42,
+    )
+    classifier.fit(matrix, labels)
+    return classifier, "Balanced Logistic Regression (small-data fallback)"
+
+
+def train_model(texts: Iterable[str], labels: Iterable[str]):
+    texts = list(texts)
+    labels = list(labels)
+    vectorizer = build_vectorizer()
+    matrix = vectorizer.fit_transform(texts)
+    classifier, algorithm = _fit_classifier(matrix, labels)
+    return classifier, vectorizer, matrix.shape[1], algorithm
+
+
+def train_and_evaluate(texts: Iterable[str], labels: Iterable[str]):
+    texts = list(texts)
+    labels = list(labels)
+    label_order = sorted(set(labels))
+    label_counts = {label: labels.count(label) for label in label_order}
+
+    accuracy = None
+    macro_f1 = None
+    report_text = "Dataset is too small for a reliable holdout evaluation. Add more examples per label."
+    class_metrics: dict[str, dict[str, float]] = {}
+    matrix_rows: list[list[int]] = []
+
+    min_class = min(label_counts.values())
+    if len(texts) >= 30 and min_class >= 8:
+        x_train, x_test, y_train, y_test = train_test_split(
+            texts,
+            labels,
+            test_size=0.20,
+            random_state=31415,
+            stratify=labels,
+        )
+        eval_clf, eval_vec, _, _ = train_model(x_train, y_train)
+        transformed = eval_vec.transform(x_test)
+        predictions = eval_clf.predict(transformed)
+        accuracy = float(accuracy_score(y_test, predictions))
+        macro_f1 = float(f1_score(y_test, predictions, average="macro", zero_division=0))
+        report_text = classification_report(y_test, predictions, labels=label_order, zero_division=0)
+        report_dict = classification_report(
+            y_test, predictions, labels=label_order, output_dict=True, zero_division=0
+        )
+        class_metrics = {
+            label: {
+                "precision": float(report_dict[label]["precision"]),
+                "recall": float(report_dict[label]["recall"]),
+                "f1": float(report_dict[label]["f1-score"]),
+                "support": float(report_dict[label]["support"]),
+            }
+            for label in label_order
+        }
+        matrix_rows = confusion_matrix(y_test, predictions, labels=label_order).tolist()
+
+    classifier, vectorizer, feature_count, algorithm = train_model(texts, labels)
+    return classifier, vectorizer, TrainingReport(
+        rows=len(texts),
+        labels=label_counts,
+        accuracy=accuracy,
+        macro_f1=macro_f1,
+        report=report_text,
+        feature_count=feature_count,
+        algorithm=algorithm,
+        class_metrics=class_metrics,
+        confusion=matrix_rows,
+        label_order=label_order,
+    )
+
+
+def save_model(path: str | Path, classifier, vectorizer) -> None:
+    path = Path(path)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    payload = {
+        "version": MODEL_VERSION,
+        "classifier": classifier,
+        "vectorizer": vectorizer,
+    }
+    with path.open("wb") as handle:
+        pickle.dump(payload, handle)
+
+
+def load_model(path: str | Path):
+    with Path(path).open("rb") as handle:
+        payload = pickle.load(handle)
+    # Backward compatibility with v2.0/v2.1 tuple model files.
+    if isinstance(payload, tuple) and len(payload) == 2:
+        return payload
+    if isinstance(payload, dict) and "classifier" in payload and "vectorizer" in payload:
+        return payload["classifier"], payload["vectorizer"]
+    raise ValueError("Unsupported model file format.")
+
+
+def predict(text: str, classifier, vectorizer) -> Prediction:
+    matrix = vectorizer.transform([text])
+    label = str(classifier.predict(matrix)[0])
+    probabilities: dict[str, float] = {}
+    confidence = 1.0
+    if hasattr(classifier, "predict_proba"):
+        values = classifier.predict_proba(matrix)[0]
+        probabilities = {
+            str(name): float(value) for name, value in zip(classifier.classes_, values)
+        }
+        confidence = probabilities.get(label, max(probabilities.values(), default=1.0))
+    return Prediction(label=label, confidence=confidence, probabilities=probabilities)

language_ninja-2.2.0/language_ninja/storage.py

@@ -0,0 +1,145 @@
+from __future__ import annotations
+
+import csv
+import json
+import sqlite3
+from datetime import datetime
+from pathlib import Path
+from typing import Iterable, Mapping, Sequence
+
+
+class Storage:
+    def __init__(self, db_path: str | Path):
+        self.db_path = Path(db_path)
+        self.db_path.parent.mkdir(parents=True, exist_ok=True)
+        self.connection = sqlite3.connect(self.db_path)
+        self.connection.row_factory = sqlite3.Row
+        self._migrate()
+
+    def _columns(self, table: str) -> set[str]:
+        rows = self.connection.execute(f"PRAGMA table_info({table})").fetchall()
+        return {str(row["name"]) for row in rows}
+
+    def _migrate(self) -> None:
+        self.connection.executescript(
+            """
+            CREATE TABLE IF NOT EXISTS predictions (
+                id INTEGER PRIMARY KEY AUTOINCREMENT,
+                timestamp TEXT NOT NULL,
+                model TEXT NOT NULL,
+                text TEXT NOT NULL,
+                sentiment TEXT NOT NULL,
+                confidence REAL NOT NULL DEFAULT 0
+            );
+            CREATE TABLE IF NOT EXISTS notes (
+                id INTEGER PRIMARY KEY AUTOINCREMENT,
+                timestamp TEXT NOT NULL,
+                title TEXT NOT NULL,
+                body TEXT NOT NULL,
+                linked_prediction_id INTEGER,
+                FOREIGN KEY(linked_prediction_id) REFERENCES predictions(id)
+            );
+            """
+        )
+        prediction_columns = self._columns("predictions")
+        additions = {
+            "positive_probability": "REAL NOT NULL DEFAULT 0",
+            "neutral_probability": "REAL NOT NULL DEFAULT 0",
+            "negative_probability": "REAL NOT NULL DEFAULT 0",
+            "confidence_level": "TEXT NOT NULL DEFAULT 'LOW'",
+        }
+        for name, definition in additions.items():
+            if name not in prediction_columns:
+                self.connection.execute(f"ALTER TABLE predictions ADD COLUMN {name} {definition}")
+        self.connection.commit()
+
+    def add_prediction(
+        self,
+        model: str,
+        text: str,
+        sentiment: str,
+        confidence: float,
+        probabilities: Mapping[str, float] | None = None,
+        confidence_level: str = "LOW",
+    ) -> int:
+        probabilities = probabilities or {}
+        cursor = self.connection.execute(
+            """
+            INSERT INTO predictions(
+                timestamp, model, text, sentiment, confidence,
+                positive_probability, neutral_probability, negative_probability,
+                confidence_level
+            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+            """,
+            (
+                datetime.now().isoformat(timespec="seconds"),
+                model,
+                text,
+                sentiment,
+                confidence,
+                float(probabilities.get("positive", 0.0)),
+                float(probabilities.get("neutral", 0.0)),
+                float(probabilities.get("negative", 0.0)),
+                confidence_level,
+            ),
+        )
+        self.connection.commit()
+        return int(cursor.lastrowid)
+
+    def add_note(self, title: str, body: str, linked_prediction_id: int | None = None) -> int:
+        cursor = self.connection.execute(
+            "INSERT INTO notes(timestamp, title, body, linked_prediction_id) VALUES (?, ?, ?, ?)",
+            (datetime.now().isoformat(timespec="seconds"), title, body, linked_prediction_id),
+        )
+        self.connection.commit()
+        return int(cursor.lastrowid)
+
+    def recent_predictions(self, limit: int = 250):
+        return self.connection.execute(
+            """
+            SELECT id, timestamp, model, text, sentiment, confidence,
+                   positive_probability, neutral_probability, negative_probability,
+                   confidence_level
+            FROM predictions ORDER BY id DESC LIMIT ?
+            """,
+            (limit,),
+        ).fetchall()
+
+    def recent_notes(self, limit: int = 250):
+        return self.connection.execute(
+            "SELECT id, timestamp, title, body, linked_prediction_id FROM notes ORDER BY id DESC LIMIT ?",
+            (limit,),
+        ).fetchall()
+
+    def execute_read_query(self, query: str):
+        statement = query.strip().lower()
+        if not (statement.startswith("select") or statement.startswith("pragma") or statement.startswith("with")):
+            raise ValueError("Database tab is read-only. Use SELECT, WITH, or PRAGMA queries.")
+        cursor = self.connection.execute(query)
+        columns = [item[0] for item in cursor.description or []]
+        rows = cursor.fetchall()
+        return columns, rows
+
+
+def export_records(records: Iterable[Mapping], path: str | Path, fmt: str) -> Path:
+    path = Path(path)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    rows = [dict(row) for row in records]
+    fmt = fmt.lower()
+    if fmt == "json":
+        path.write_text(json.dumps(rows, indent=2, ensure_ascii=False), encoding="utf-8")
+        return path
+    if fmt == "csv":
+        fields = list(rows[0].keys()) if rows else []
+        with path.open("w", newline="", encoding="utf-8") as handle:
+            writer = csv.DictWriter(handle, fieldnames=fields)
+            if fields:
+                writer.writeheader()
+                writer.writerows(rows)
+        return path
+    raise ValueError(f"Unsupported export format: {fmt}")
+
+
+def export_table(columns: Sequence[str], rows: Sequence[Sequence], path: str | Path, fmt: str) -> Path:
+    dictionaries = [dict(zip(columns, row)) for row in rows]
+    return export_records(dictionaries, path, fmt)

language_ninja-2.2.0/language_ninja.egg-info/PKG-INFO

@@ -0,0 +1,113 @@
+Metadata-Version: 2.4
+Name: language-ninja
+Version: 2.2.0
+Summary: Keyboard-first Textual NLP sentiment training, analysis, notes, and export workstation.
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: textual>=0.70
+Requires-Dist: scikit-learn>=1.3
+
+# 🥷 Language Trainer Ninja v2.2
+
+A keyboard-first Textual NLP workstation for sentiment analysis, model training, dataset inspection, notes, history, SQLite queries, and CSV/JSON export.
+
+## v2.2 model
+
+The bundled model is now a calibrated classical NLP pipeline rather than the earlier small Naive Bayes/logistic baseline:
+
+```text
+Word TF-IDF (1–3 grams) ─┐
+                         ├─ FeatureUnion ─ Linear SVM ─ 5-fold sigmoid calibration ─ probabilities
+Char TF-IDF (3–5 grams) ─┘
+```
+
+Why both feature families:
+
+- Word n-grams capture phrases such as `not good`, `really love`, and longer contrast patterns.
+- Character n-grams improve handling of contractions, punctuation, spelling variation, and unseen word forms.
+- A balanced Linear SVM supplies a strong decision boundary for sparse text.
+- Sigmoid calibration turns SVM margins into class probabilities for the confidence display.
+
+The starter corpus contains **1,500 balanced examples**:
+
+```text
+positive  500
+neutral   500
+negative  500
+```
+
+It includes straightforward sentiment, conversational phrasing, negation, contrast, and neutral operational language.
+
+> The built-in holdout score is useful as a regression check, but the starter corpus contains generated/template-assisted examples. Treat its very high holdout accuracy as an internal sanity metric, not as proof of production-level real-world accuracy.
+
+## Install
+
+```bash
+cd ~/Downloads/language-trainer-ninja-master
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+python -m pip install -e .
+language-ninja
+```
+
+A trained `models/sentiment_model.pkl` is included, so Analyze can work immediately. Use **Train → Train Model** after changing the dataset.
+
+## Confidence
+
+The TUI displays all three calibrated probabilities:
+
+```text
+Prediction: NEGATIVE • HIGH CONFIDENCE
+Winning probability: 82.8%
+POS 9.5%   NEU 7.6%   NEG 82.8%
+```
+
+UI guidance:
+
+```text
+HIGH       75–100%
+MODERATE   55–74%
+LOW        below 55%
+```
+
+These labels describe the model's probability separation, not a guarantee of correctness.
+
+## Pages
+
+### Analyze
+Paste conversational text and inspect the winning class plus positive, neutral, and negative probabilities. Every prediction is written to History.
+
+### Train
+Train from a `text,label` CSV. The status panel reports row counts, algorithm, feature count, holdout accuracy, macro F1, and per-class precision/recall/F1.
+
+### Dataset
+Inspect the examples used to train the model and export them to CSV or JSON.
+
+### Notes
+Save experiment notes or attach a note to the most recent prediction.
+
+### History
+Review every prediction with **POS / NEU / NEG probabilities** and a HIGH/MODERATE/LOW confidence level.
+
+### Database
+Run read-only `SELECT`, `WITH`, and `PRAGMA` queries against the SQLite database.
+
+### Export
+Export predictions, notes, datasets, and query results to `./exports/`.
+
+## Rebuild the bundled corpus
+
+The deterministic corpus generator is included:
+
+```bash
+python tools/build_starter_corpus.py
+```
+
+Then retrain from the TUI, or use the Python ML module.
+
+## Run tests
+
+```bash
+python -m pytest -q
+```

language_ninja-2.2.0/language_ninja.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+README.md
+pyproject.toml
+language_ninja/__init__.py
+language_ninja/app.py
+language_ninja/ml.py
+language_ninja/storage.py
+language_ninja.egg-info/PKG-INFO
+language_ninja.egg-info/SOURCES.txt
+language_ninja.egg-info/dependency_links.txt
+language_ninja.egg-info/entry_points.txt
+language_ninja.egg-info/requires.txt
+language_ninja.egg-info/top_level.txt
+tests/test_model.py
+tests/test_storage.py

language_ninja-2.2.0/language_ninja.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

language_ninja-2.2.0/language_ninja.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+language-ninja = language_ninja.app:main

language_ninja-2.2.0/language_ninja.egg-info/requires.txt

@@ -0,0 +1,2 @@
+textual>=0.70
+scikit-learn>=1.3

language_ninja-2.2.0/language_ninja.egg-info/top_level.txt

@@ -0,0 +1 @@
+language_ninja

language_ninja-2.2.0/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "language-ninja"
+version = "2.2.0"
+description = "Keyboard-first Textual NLP sentiment training, analysis, notes, and export workstation."
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+  "textual>=0.70",
+  "scikit-learn>=1.3",
+]
+
+[project.scripts]
+language-ninja = "language_ninja.app:main"
+
+[tool.setuptools.packages.find]
+include = ["language_ninja*"]

language_ninja-2.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

language_ninja-2.2.0/tests/test_model.py

@@ -0,0 +1,26 @@
+from pathlib import Path
+
+from language_ninja.ml import load_dataset, load_model, predict
+
+
+def test_bundled_dataset_is_balanced():
+    texts, labels = load_dataset(Path('data/propositions.csv'))
+    assert len(texts) == 1500
+    assert labels.count('positive') == 500
+    assert labels.count('neutral') == 500
+    assert labels.count('negative') == 500
+
+
+def test_bundled_model_handles_baselines():
+    classifier, vectorizer = load_model(Path('models/sentiment_model.pkl'))
+    cases = [
+        ('I absolutely love this application', 'positive'),
+        ('I absolutely hate this application', 'negative'),
+        ('The application opened at nine this morning', 'neutral'),
+        ('You should speak more like a human instead of a robot', 'negative'),
+        ('You did well, your little business is alright', 'positive'),
+    ]
+    for text, expected in cases:
+        result = predict(text, classifier, vectorizer)
+        assert result.label == expected
+        assert result.confidence >= 0.75

language_ninja-2.2.0/tests/test_storage.py

@@ -0,0 +1,13 @@
+from pathlib import Path
+
+from language_ninja.storage import Storage, export_records
+
+
+def test_prediction_note_and_export(tmp_path: Path):
+    db = Storage(tmp_path / "test.db")
+    prediction_id = db.add_prediction("demo.pkl", "great", "positive", 0.9)
+    db.add_note("review", "Looks correct", prediction_id)
+    assert len(db.recent_predictions()) == 1
+    assert len(db.recent_notes()) == 1
+    out = export_records(db.recent_predictions(), tmp_path / "predictions.json", "json")
+    assert out.exists()