taskclf 0.3.0__tar.gz → 0.3.2__tar.gz

taskclf-0.3.2/PKG-INFO

@@ -0,0 +1,294 @@
+Metadata-Version: 2.3
+Name: taskclf
+Version: 0.3.2
+Summary: Local-first task classifier that infers your work type from computer activity signals
+Author: fruitiecutiepie
+Author-email: fruitiecutiepie <104437268+fruitiecutiepie@users.noreply.github.com>
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Office/Business
+Classifier: Topic :: System :: Monitoring
+Requires-Dist: duckdb>=1.4.4
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: python-multipart>=0.0.18
+Requires-Dist: lightgbm>=4.6.0
+Requires-Dist: matplotlib>=3.10.8
+Requires-Dist: pandas>=3.0.0
+Requires-Dist: pyarrow>=23.0.0
+Requires-Dist: pydantic>=2.12.5
+Requires-Dist: rich>=14.3.2
+Requires-Dist: scipy>=1.15.0
+Requires-Dist: scikit-learn>=1.8.0
+Requires-Dist: pywebview>=5.0
+Requires-Dist: typer>=0.23.1
+Requires-Dist: uvicorn[standard]>=0.34.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: pystray>=0.19.5
+Requires-Dist: pillow>=11.0.0
+Requires-Python: >=3.14
+Project-URL: Homepage, https://github.com/fruitiecutiepie/taskclf
+Project-URL: Documentation, https://fruitiecutiepie.github.io/taskclf
+Project-URL: Repository, https://github.com/fruitiecutiepie/taskclf
+Description-Content-Type: text/markdown
+
+# taskclf — Task Type Classifier from Local Activity Signals
+
+[![PyPI version](https://img.shields.io/pypi/v/taskclf)](https://pypi.org/project/taskclf/)
+[![Python](https://img.shields.io/pypi/pyversions/taskclf)](https://pypi.org/project/taskclf/)
+
+Train and run a personal task-type classifier (e.g. coding / writing / meetings) using privacy-preserving computer activity signals such as foreground app/window metadata and aggregated input statistics (counts/rates only).
+
+This project is intentionally scoped as a **personalized classifier** (single-user first). The architecture keeps:
+- **Collectors** (platform/tool dependent) isolated behind adapters
+- **Features** as a versioned, validated contract
+- **Models** as bundled artifacts with schema checks
+- **Inference** as a small, stable loop that emits task segments and daily summaries
+
+## Goals
+- Fast iteration: first useful model in < 1 week of data
+- Privacy: no raw keystrokes, no raw window titles persisted
+- Stability: feature schema versioning + schema hash gates
+- Extensibility: add new collectors and models without breaking consumers
+
+## Non-Goals
+- Universal (multi-user) generalization out of the box
+- Storing or analyzing raw typed content
+- "Perfect" labeling UI (start minimal, iterate later)
+
+---
+
+## Labels (v1)
+
+Eight core labels defined in `schema/labels_v1.json`:
+
+| ID | Label | Description |
+|----|-------|-------------|
+| 0 | `Build` | Writing or implementing structured content in editor/terminal |
+| 1 | `Debug` | Investigating issues, terminal-heavy troubleshooting |
+| 2 | `Review` | Reviewing technical material or diffs with light edits |
+| 3 | `Write` | Writing structured non-code content |
+| 4 | `ReadResearch` | Consuming information with minimal production |
+| 5 | `Communicate` | Asynchronous coordination (chat/email) |
+| 6 | `Meet` | Synchronous meetings or calls |
+| 7 | `BreakIdle` | Idle or break period |
+
+Labels are stored as **time spans** (not per-keystroke events). Users can remap
+core labels to personal categories via a **taxonomy config**
+(see `configs/user_taxonomy_example.yaml`).
+
+---
+
+## Data Flow Overview
+
+### Structures (pipelines)
+* ETL pipeline reads raw → produces features parquet
+* Training pipeline reads features + labels → produces model
+* Inference pipeline reads new events → emits predictions + segments
+
+### Batch (repeatable)
+1. **Ingest**: pull ActivityWatch export → `data/raw/aw/`
+2. **Feature build**: events → per-minute features → `data/processed/features_v1/`
+3. **Label import**: label spans → `data/processed/labels_v1/`
+4. **Build dataset**: join features + labels, split by time → training arrays
+5. **Train**: fit model → `models/<run_id>/`
+6. **Evaluate**: metrics, acceptance checks, calibration
+7. **Report**: daily summaries → `artifacts/`
+
+### Online (real-time)
+Every N seconds:
+- read the last minute(s) of events
+- compute the latest feature bucket
+- predict + smooth (with optional calibration and taxonomy mapping)
+- append predictions → `artifacts/`
+
+At end-of-day:
+- produce report
+
+---
+
+## Privacy & Safety
+This repo enforces the following:
+- **No raw keystrokes** are stored (only aggregate counts/rates).
+- **No raw window titles** are stored by default.
+  - Titles are hashed or locally tokenized; you can keep a local mapping if you choose.
+- Dataset artifacts stay **local-first**.
+
+---
+
+## Quick Start
+
+### Requirements
+- Python >= 3.14
+- `uv` installed
+
+### Install
+
+```bash
+uv tool install taskclf
+```
+
+For development (from source):
+
+```bash
+uv sync
+uv run taskclf --help
+```
+
+### Ingest (ActivityWatch)
+
+```bash
+uv run taskclf ingest aw --input /path/to/activitywatch-export.json
+```
+
+This parses an ActivityWatch JSON export, normalizes app names to reverse-domain
+identifiers, hashes window titles (never storing raw text), and writes
+privacy-safe events to `data/raw/aw/<YYYY-MM-DD>/events.parquet` partitioned by
+date.
+
+Options:
+- `--out-dir` — output directory (default: `data/raw/aw`)
+- `--title-salt` — salt for hashing window titles (default: `taskclf-default-salt`)
+
+### Build features
+
+```bash
+uv run taskclf features build --date 2026-02-16
+```
+
+### Import labels
+
+```bash
+uv run taskclf labels import --file labels.csv
+```
+
+Or add individual label blocks:
+
+```bash
+uv run taskclf labels add-block \
+  --start 2026-02-16T09:00:00 --end 2026-02-16T10:00:00 --label Build
+```
+
+Or label what you're doing right now (no timestamps needed):
+
+```bash
+uv run taskclf labels label-now --minutes 10 --label Build
+```
+
+This queries ActivityWatch for a live summary of apps used in the last N minutes
+and creates the label span automatically.
+
+### Train
+
+```bash
+uv run taskclf train lgbm --from 2026-02-01 --to 2026-02-16
+```
+
+### Run batch inference
+
+```bash
+uv run taskclf infer batch --model-dir models/<run_id> --from 2026-02-01 --to 2026-02-16
+```
+
+### Run online inference
+
+```bash
+uv run taskclf infer online --model-dir models/<run_id>
+```
+
+Starts a polling loop that queries a running ActivityWatch server, builds
+feature rows from live window events, predicts task types using a trained model,
+smooths predictions, and writes running outputs to `artifacts/`. Press Ctrl+C
+to stop; a final daily report is generated on shutdown.
+
+Options:
+- `--poll-seconds` — seconds between polls (default: 60)
+- `--aw-host` — ActivityWatch server URL (default: `http://localhost:5600`)
+- `--smooth-window` — rolling majority window size (default: 3)
+- `--title-salt` — salt for hashing window titles (default: `taskclf-default-salt`)
+- `--out-dir` — output directory (default: `artifacts`)
+- `--label-queue` / `--no-label-queue` — auto-enqueue low-confidence predictions for manual labeling
+- `--label-confidence` — confidence threshold for auto-enqueue (default: 0.55)
+
+### Run baseline (no model needed)
+
+```bash
+uv run taskclf infer baseline --from 2026-02-01 --to 2026-02-16
+```
+
+Rule-based classifier useful for day-1 bootstrapping before you have a trained model.
+
+### Produce report
+
+```bash
+uv run taskclf report daily --segments-file artifacts/segments.json
+```
+
+---
+
+## CLI Reference
+
+All commands: `uv run taskclf --help`
+
+| Group | Commands | Purpose |
+|-------|----------|---------|
+| `ingest` | `aw` | Import ActivityWatch exports |
+| `features` | `build` | Build per-minute feature rows |
+| `labels` | `import`, `add-block`, `label-now`, `show-queue`, `project` | Manage label spans and labeling queue |
+| `train` | `build-dataset`, `lgbm`, `evaluate`, `tune-reject`, `calibrate`, `retrain`, `check-retrain` | Training, evaluation, and retraining pipeline |
+| `taxonomy` | `validate`, `show`, `init` | User-defined label groupings |
+| `infer` | `batch`, `online`, `baseline`, `compare` | Prediction (ML, rule-based, comparison) |
+| `report` | `daily` | Daily summaries (JSON/CSV/Parquet) |
+| `monitor` | `drift-check`, `telemetry`, `show` | Feature drift and telemetry tracking |
+| *(top-level)* | `tray` | System tray labeling app with activity transition detection |
+| *(top-level)* | `ui` | Web UI for labeling, queue, and live prediction streaming |
+
+Full CLI docs: `docs/api/cli/main.md`
+
+---
+
+## Repo Layout
+
+* `src/taskclf/` — application code (adapters, core, features, labels, train, infer, report, ui)
+* `schema/` — versioned JSON schemas for features and labels
+* `configs/` — configuration files (model params, retrain policy, taxonomy examples)
+* `docs/` — API reference and guides (served via `make docs-serve`)
+* `data/` — raw and processed datasets (local, gitignored)
+* `models/` — trained model bundles (one folder per run)
+* `artifacts/` — predictions, segments, reports, evaluation outputs
+* `tests/` — test suite
+
+---
+
+## Model Artifact Contract
+
+Every saved model bundle (`models/<run_id>/`) contains:
+
+* the model file
+* `metadata.json`: feature schema version + hash, label set, training date range, params, dataset hash
+* `metrics.json`: macro/weighted F1, per-class metrics
+* `confusion_matrix.csv`
+* categorical encoders (if applicable)
+
+Inference refuses to run if the schema hash mismatches the model bundle.
+
+---
+
+## Development
+
+Common tasks are in the `Makefile`:
+
+```bash
+make lint        # ruff check .
+make test        # pytest
+make typecheck   # mypy src
+make docs-serve  # local preview at http://127.0.0.1:8000
+make docs-build  # static site in site/
+```
+
+---
+
+## License
+
+TBD (local-first personal project by default).

taskclf-0.3.2/pyproject.toml

@@ -0,0 +1,67 @@
+[project]
+name = "taskclf"
+version = "0.3.2"
+description = "Local-first task classifier that infers your work type from computer activity signals"
+readme = "README.md"
+license = {text = "MIT"}
+authors = [
+    { name = "fruitiecutiepie", email = "104437268+fruitiecutiepie@users.noreply.github.com" }
+]
+requires-python = ">=3.14"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Office/Business",
+    "Topic :: System :: Monitoring",
+]
+dependencies = [
+    "duckdb>=1.4.4",
+    "fastapi>=0.115.0",
+    "python-multipart>=0.0.18",
+    "lightgbm>=4.6.0",
+    "matplotlib>=3.10.8",
+    "pandas>=3.0.0",
+    "pyarrow>=23.0.0",
+    "pydantic>=2.12.5",
+    "rich>=14.3.2",
+    "scipy>=1.15.0",
+    "scikit-learn>=1.8.0",
+    "pywebview>=5.0",
+    "typer>=0.23.1",
+    "uvicorn[standard]>=0.34.0",
+    "pyyaml>=6.0",
+    "pystray>=0.19.5",
+    "Pillow>=11.0.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/fruitiecutiepie/taskclf"
+Documentation = "https://fruitiecutiepie.github.io/taskclf"
+Repository = "https://github.com/fruitiecutiepie/taskclf"
+
+[project.scripts]
+taskclf = "taskclf.cli.main:app"
+
+[tool.uv_build]
+data-includes = ["src/taskclf/ui/static/**/*"]
+
+[build-system]
+requires = ["uv_build>=0.9.5,<0.10.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "black>=26.1.0",
+    "httpx>=0.28.0",
+    "mypy>=1.19.1",
+    "pandas-stubs>=3.0.0.260204",
+    "pre-commit>=4.5.1",
+    "pytest>=9.0.2",
+    "pytest-xdist>=3.8.0",
+    "ruff>=0.15.1",
+]
+docs = [
+    "zensical",
+    "mkdocstrings-python",
+]

taskclf-0.3.2/src/taskclf/adapters/activitywatch/client.py

@@ -0,0 +1,298 @@
+"""ActivityWatch data access: JSON export parsing and REST API client.
+
+Provides two data-ingestion paths:
+
+* **File-based** -- :func:`parse_aw_export` reads an AW JSON export
+  (the format produced by *Export all buckets as JSON* in the AW web UI
+  or ``GET /api/0/export``).
+* **REST-based** -- :func:`fetch_aw_events` queries a running
+  ``aw-server`` instance for events in a time range.
+
+Both paths normalize application names via
+:func:`~taskclf.adapters.activitywatch.mapping.normalize_app` and
+replace raw window titles with salted hashes so that no sensitive text
+is ever persisted.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import urllib.request
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+from taskclf.adapters.activitywatch.mapping import normalize_app
+from taskclf.adapters.activitywatch.types import AWEvent, AWInputEvent
+from taskclf.core.defaults import DEFAULT_AW_TIMEOUT_SECONDS
+from taskclf.core.hashing import salted_hash
+
+logger = logging.getLogger(__name__)
+
+_CURRENTWINDOW_TYPE = "currentwindow"
+_INPUT_TYPE = "os.hid.input"
+
+
+def _parse_timestamp(raw: str) -> datetime:
+    """Parse an ISO-8601 timestamp from AW into a timezone-aware UTC datetime."""
+    ts = datetime.fromisoformat(raw)
+    if ts.tzinfo is not None:
+        ts = ts.astimezone(timezone.utc)
+    else:
+        ts = ts.replace(tzinfo=timezone.utc)
+    return ts
+
+
+def _raw_event_to_aw_event(raw: dict[str, Any], *, title_salt: str) -> AWEvent:
+    """Convert a single raw AW event dict into a normalized :class:`AWEvent`."""
+    data = raw.get("data", {})
+    app_name = data.get("app", "unknown")
+    title = data.get("title", "")
+
+    app_id, is_browser, is_editor, is_terminal, app_category = normalize_app(app_name)
+    title_hash = salted_hash(title, salt=title_salt)
+
+    return AWEvent(
+        timestamp=_parse_timestamp(raw["timestamp"]),
+        duration_seconds=float(raw.get("duration", 0)),
+        app_id=app_id,
+        window_title_hash=title_hash,
+        is_browser=is_browser,
+        is_editor=is_editor,
+        is_terminal=is_terminal,
+        app_category=app_category,
+    )
+
+
+# ---------------------------------------------------------------------------
+# File-based ingestion
+# ---------------------------------------------------------------------------
+
+
+def parse_aw_export(path: Path, *, title_salt: str) -> list[AWEvent]:
+    """Parse an ActivityWatch JSON export file into normalized events.
+
+    Filters for buckets of type ``currentwindow`` (i.e.
+    ``aw-watcher-window`` data).  Each event's application name is
+    normalized and its window title is replaced with a salted hash.
+
+    Args:
+        path: Path to the AW export JSON file.
+        title_salt: Salt used for hashing window titles.
+
+    Returns:
+        Sorted (by timestamp) list of :class:`AWEvent` instances.
+
+    Raises:
+        FileNotFoundError: If *path* does not exist.
+        KeyError: If the JSON structure is missing expected keys.
+    """
+    raw = json.loads(path.read_text(encoding="utf-8"))
+
+    buckets: dict[str, Any] = raw.get("buckets", raw)
+
+    events: list[AWEvent] = []
+    for bucket_id, bucket in buckets.items():
+        bucket_type = bucket.get("type", "")
+        if bucket_type != _CURRENTWINDOW_TYPE:
+            logger.debug("Skipping bucket %s (type=%s)", bucket_id, bucket_type)
+            continue
+
+        logger.info(
+            "Processing bucket %s (%d events)",
+            bucket_id,
+            len(bucket.get("events", [])),
+        )
+        for raw_event in bucket.get("events", []):
+            events.append(_raw_event_to_aw_event(raw_event, title_salt=title_salt))
+
+    events.sort(key=lambda e: e.timestamp)
+    return events
+
+
+def _raw_to_input_event(raw: dict[str, Any]) -> AWInputEvent:
+    """Convert a single raw AW input event dict into an :class:`AWInputEvent`."""
+    data = raw.get("data", {})
+    return AWInputEvent(
+        timestamp=_parse_timestamp(raw["timestamp"]),
+        duration_seconds=float(raw.get("duration", 0)),
+        presses=int(data.get("presses", 0)),
+        clicks=int(data.get("clicks", 0)),
+        delta_x=int(data.get("deltaX", 0)),
+        delta_y=int(data.get("deltaY", 0)),
+        scroll_x=int(data.get("scrollX", 0)),
+        scroll_y=int(data.get("scrollY", 0)),
+    )
+
+
+def parse_aw_input_export(path: Path) -> list[AWInputEvent]:
+    """Parse ``aw-watcher-input`` events from an AW JSON export.
+
+    Filters for buckets of type ``os.hid.input``.  These events carry
+    only aggregate counts (key presses, mouse clicks, movement, scroll)
+    and contain no sensitive payload.
+
+    Args:
+        path: Path to the AW export JSON file.
+
+    Returns:
+        Sorted (by timestamp) list of :class:`AWInputEvent` instances.
+        Empty if no ``os.hid.input`` bucket exists in the export.
+    """
+    raw = json.loads(path.read_text(encoding="utf-8"))
+    buckets: dict[str, Any] = raw.get("buckets", raw)
+
+    events: list[AWInputEvent] = []
+    for bucket_id, bucket in buckets.items():
+        bucket_type = bucket.get("type", "")
+        if bucket_type != _INPUT_TYPE:
+            continue
+
+        logger.info(
+            "Processing input bucket %s (%d events)",
+            bucket_id,
+            len(bucket.get("events", [])),
+        )
+        for raw_event in bucket.get("events", []):
+            events.append(_raw_to_input_event(raw_event))
+
+    events.sort(key=lambda e: e.timestamp)
+    return events
+
+
+# ---------------------------------------------------------------------------
+# REST API helpers
+# ---------------------------------------------------------------------------
+
+
+def _api_get(url: str) -> Any:
+    """Issue a GET request and return the parsed JSON body."""
+    req = urllib.request.Request(url, headers={"Accept": "application/json"})
+    with urllib.request.urlopen(req, timeout=DEFAULT_AW_TIMEOUT_SECONDS) as resp:
+        return json.loads(resp.read().decode("utf-8"))
+
+
+def list_aw_buckets(host: str) -> dict[str, dict]:
+    """List all buckets from a running AW server.
+
+    Args:
+        host: Base URL of the AW server (e.g. ``"http://localhost:5600"``).
+
+    Returns:
+        Dict mapping bucket IDs to their metadata.
+    """
+    url = f"{host.rstrip('/')}/api/0/buckets/"
+    return _api_get(url)
+
+
+def find_window_bucket_id(host: str) -> str:
+    """Auto-discover the ``aw-watcher-window`` bucket on *host*.
+
+    Args:
+        host: Base URL of the AW server.
+
+    Returns:
+        The bucket ID whose ``type`` is ``currentwindow``.
+
+    Raises:
+        ValueError: If no ``currentwindow`` bucket exists on the server.
+    """
+    buckets = list_aw_buckets(host)
+    for bucket_id, meta in buckets.items():
+        if meta.get("type") == _CURRENTWINDOW_TYPE:
+            return bucket_id
+    raise ValueError(
+        f"No bucket with type={_CURRENTWINDOW_TYPE!r} found on {host}. "
+        f"Available: {list(buckets.keys())}"
+    )
+
+
+def find_input_bucket_id(host: str) -> str | None:
+    """Auto-discover the ``aw-watcher-input`` bucket on *host*.
+
+    Unlike :func:`find_window_bucket_id`, this returns ``None`` when no
+    input bucket exists because ``aw-watcher-input`` is an optional
+    watcher that many users don't run.
+
+    Args:
+        host: Base URL of the AW server.
+
+    Returns:
+        The bucket ID whose ``type`` is ``os.hid.input``, or ``None``.
+    """
+    buckets = list_aw_buckets(host)
+    for bucket_id, meta in buckets.items():
+        if meta.get("type") == _INPUT_TYPE:
+            return bucket_id
+    return None
+
+
+def fetch_aw_events(
+    host: str,
+    bucket_id: str,
+    start: datetime,
+    end: datetime,
+    *,
+    title_salt: str,
+) -> list[AWEvent]:
+    """Fetch events from the AW REST API for a time range.
+
+    Args:
+        host: Base URL of the AW server (e.g. ``"http://localhost:5600"``).
+        bucket_id: Bucket to query (e.g. ``"aw-watcher-window_myhostname"``).
+        start: Inclusive start of the query window (UTC).
+        end: Exclusive end of the query window (UTC).
+        title_salt: Salt used for hashing window titles.
+
+    Returns:
+        Sorted list of :class:`AWEvent` instances.
+    """
+    base = host.rstrip("/")
+    start_iso = start.isoformat() + "Z" if start.tzinfo is None else start.isoformat()
+    end_iso = end.isoformat() + "Z" if end.tzinfo is None else end.isoformat()
+
+    url = (
+        f"{base}/api/0/buckets/{bucket_id}/events"
+        f"?start={start_iso}&end={end_iso}"
+    )
+    raw_events: list[dict] = _api_get(url)
+
+    events = [
+        _raw_event_to_aw_event(e, title_salt=title_salt) for e in raw_events
+    ]
+    events.sort(key=lambda e: e.timestamp)
+    return events
+
+
+def fetch_aw_input_events(
+    host: str,
+    bucket_id: str,
+    start: datetime,
+    end: datetime,
+) -> list[AWInputEvent]:
+    """Fetch input events from the AW REST API for a time range.
+
+    Args:
+        host: Base URL of the AW server.
+        bucket_id: Input bucket to query (e.g.
+            ``"aw-watcher-input_myhostname"``).
+        start: Inclusive start of the query window (UTC).
+        end: Exclusive end of the query window (UTC).
+
+    Returns:
+        Sorted list of :class:`AWInputEvent` instances.
+    """
+    base = host.rstrip("/")
+    start_iso = start.isoformat() + "Z" if start.tzinfo is None else start.isoformat()
+    end_iso = end.isoformat() + "Z" if end.tzinfo is None else end.isoformat()
+
+    url = (
+        f"{base}/api/0/buckets/{bucket_id}/events"
+        f"?start={start_iso}&end={end_iso}"
+    )
+    raw_events: list[dict] = _api_get(url)
+
+    events = [_raw_to_input_event(e) for e in raw_events]
+    events.sort(key=lambda e: e.timestamp)
+    return events