watfile 0.1.0__tar.gz

watfile-0.1.0/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: watfile
+Version: 0.1.0
+Summary: Classify documents with the TypeSafe Jev decision model and sort them into folders
+Keywords: classification,files,llm,typesafe,cli
+Author: Michael Hunger
+Author-email: Michael Hunger <github@jexp.de>
+License-Expression: MIT
+Classifier: Environment :: Console
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Utilities
+Requires-Dist: liteparse>=2.14.6
+Requires-Dist: typesafe-sdk>=0.7.0
+Requires-Python: >=3.12
+Project-URL: Repository, https://github.com/jexp/watfile
+Description-Content-Type: text/markdown
+
+# watfile
+
+Classify files with a decision-making AI model and sort them into category folders.
+
+watfile sends each document's text (title/abstract-grade extract) to a
+**TypeSafe AI Jev** (System One) [Choice](https://docs.typesafe.ai/primitives/choice)
+question, gets back a typed answer with a selected category, per-category
+probabilities and confidence, then moves the file into the matching folder.
+A `Classifier` abstraction keeps the backend pluggable — local MLX (laya) and
+other backends slot in later.
+
+## Install
+
+Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/).
+
+### From PyPI (once published)
+
+```sh
+# one-off run, no install
+uvx watfile --help
+
+# persistent CLI on your PATH
+uv tool install watfile
+watfile --help
+```
+
+### From source
+
+```sh
+git clone <repo> && cd watfile
+uv sync            # create venv + install deps (typesafe-sdk, liteparse)
+uv run watfile --help
+
+# or install the local checkout as a tool
+uv tool install --from . watfile
+```
+
+### Configuration
+
+watfile resolves its TypeSafe API key (create one at <https://console.typesafe.ai/>)
+with this precedence — first match wins:
+
+1. `TYPESAFE_API_KEY` environment variable
+2. `.env` file in the current directory (gitignored; `TYPESAFE_API_KEY=...`)
+3. `~/.config/watfile/config.toml` (`api_key = "..."`, also `base_url`, `model`;
+   `$WATFILE_CONFIG` or `$XDG_CONFIG_HOME` can relocate it)
+
+```sh
+export TYPESAFE_API_KEY=...        # option 1
+echo 'TYPESAFE_API_KEY=...' > .env # option 2
+cat > ~/.config/watfile/config.toml <<'EOF'   # option 3
+api_key = "..."
+EOF
+```
+
+## Usage
+
+Point watfile at files or folders, and either give a comma-separated category
+list (`-c`) or a target folder whose subfolders are the categories (`-d`):
+
+```sh
+# explicit categories, files moved into ./sorted/<category>/
+uv run watfile ~/Downloads/invoice.pdf -c invoice,donation,apartment
+
+# folder input, recursive; categories = existing subfolders of -d
+mkdir -p ~/docs/{invoice,donation,apartment}
+uv run watfile ~/Downloads -r -d ~/docs
+
+# preview without touching anything
+uv run watfile ~/Downloads -r -d ~/docs -n
+
+# actually move the files (default is symlinking into the category folders)
+uv run watfile ~/Downloads -r -d ~/docs -m
+
+# copy instead
+uv run watfile ~/Downloads -r -d ~/docs --copy
+
+# custom output root with -c
+uv run watfile *.pdf -c computerscience,biology -o ~/sorted
+```
+
+Output per file:
+
+```
+bill.pdf: invoice (conf 0.94) -> symlink to ~/docs/invoice/bill.pdf
+```
+
+Files that can't be classified (unsupported extension, no extractable text) are
+skipped with a warning; name collisions get a `_1`, `_2`… suffix.
+
+### Supported inputs
+
+- **Text formats** (read directly): `.txt .md .markdown .rst .log .csv .json`
+- **PDF** (via [liteparse](https://github.com/run-llama/liteparse)): only the
+  first 2 pages are parsed, OCR disabled — enough for classification, ~1000x
+  faster than a full parse. Scanned/image-only PDFs are skipped.
+
+### Options
+
+```
+usage: watfile [-h] [-r] (-c CATEGORIES | -d DIRECTORY) [-o OUTPUT]
+               [--backend {jev,laya}] [-n] [--copy]
+               inputs [inputs ...]
+
+positional arguments:
+  inputs                files and/or folders to process
+
+options:
+  -h, --help            show this help message and exit
+  -r, --recursive       recurse into folder inputs
+  -c CATEGORIES, --categories CATEGORIES
+                        comma-separated categories, e.g. invoice,donation,apartment
+  -d DIRECTORY, --directory
+                        target folder whose existing subfolders are the categories
+  -o OUTPUT, --output   output root for sorted files (default: same as -d, or ./sorted with -c)
+  --backend {jev,laya}  classifier backend (default: jev)
+  -n, --dry-run         print decisions without placing files
+  -m, --move            move files into the category folder (default: symlink)
+  --copy                copy files instead of symlinking
+  --symlink             create symlinks in category folders (default)
+```
+
+## Development
+
+```sh
+uv sync
+uv run pytest              # unit tests; live API tests skip without TYPESAFE_API_KEY
+```
+
+`tests/fixture/` contains 4 real arXiv PDFs with ground-truth categories
+(derived from their arXiv subject tags) used by the integration tests.
+
+## Roadmap
+
+- `laya` local backend (MLX via OpenAI-compatible HTTP)
+- batching: classify 25/50/100 files in a single API call

watfile-0.1.0/README.md

@@ -0,0 +1,136 @@
+# watfile
+
+Classify files with a decision-making AI model and sort them into category folders.
+
+watfile sends each document's text (title/abstract-grade extract) to a
+**TypeSafe AI Jev** (System One) [Choice](https://docs.typesafe.ai/primitives/choice)
+question, gets back a typed answer with a selected category, per-category
+probabilities and confidence, then moves the file into the matching folder.
+A `Classifier` abstraction keeps the backend pluggable — local MLX (laya) and
+other backends slot in later.
+
+## Install
+
+Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/).
+
+### From PyPI (once published)
+
+```sh
+# one-off run, no install
+uvx watfile --help
+
+# persistent CLI on your PATH
+uv tool install watfile
+watfile --help
+```
+
+### From source
+
+```sh
+git clone <repo> && cd watfile
+uv sync            # create venv + install deps (typesafe-sdk, liteparse)
+uv run watfile --help
+
+# or install the local checkout as a tool
+uv tool install --from . watfile
+```
+
+### Configuration
+
+watfile resolves its TypeSafe API key (create one at <https://console.typesafe.ai/>)
+with this precedence — first match wins:
+
+1. `TYPESAFE_API_KEY` environment variable
+2. `.env` file in the current directory (gitignored; `TYPESAFE_API_KEY=...`)
+3. `~/.config/watfile/config.toml` (`api_key = "..."`, also `base_url`, `model`;
+   `$WATFILE_CONFIG` or `$XDG_CONFIG_HOME` can relocate it)
+
+```sh
+export TYPESAFE_API_KEY=...        # option 1
+echo 'TYPESAFE_API_KEY=...' > .env # option 2
+cat > ~/.config/watfile/config.toml <<'EOF'   # option 3
+api_key = "..."
+EOF
+```
+
+## Usage
+
+Point watfile at files or folders, and either give a comma-separated category
+list (`-c`) or a target folder whose subfolders are the categories (`-d`):
+
+```sh
+# explicit categories, files moved into ./sorted/<category>/
+uv run watfile ~/Downloads/invoice.pdf -c invoice,donation,apartment
+
+# folder input, recursive; categories = existing subfolders of -d
+mkdir -p ~/docs/{invoice,donation,apartment}
+uv run watfile ~/Downloads -r -d ~/docs
+
+# preview without touching anything
+uv run watfile ~/Downloads -r -d ~/docs -n
+
+# actually move the files (default is symlinking into the category folders)
+uv run watfile ~/Downloads -r -d ~/docs -m
+
+# copy instead
+uv run watfile ~/Downloads -r -d ~/docs --copy
+
+# custom output root with -c
+uv run watfile *.pdf -c computerscience,biology -o ~/sorted
+```
+
+Output per file:
+
+```
+bill.pdf: invoice (conf 0.94) -> symlink to ~/docs/invoice/bill.pdf
+```
+
+Files that can't be classified (unsupported extension, no extractable text) are
+skipped with a warning; name collisions get a `_1`, `_2`… suffix.
+
+### Supported inputs
+
+- **Text formats** (read directly): `.txt .md .markdown .rst .log .csv .json`
+- **PDF** (via [liteparse](https://github.com/run-llama/liteparse)): only the
+  first 2 pages are parsed, OCR disabled — enough for classification, ~1000x
+  faster than a full parse. Scanned/image-only PDFs are skipped.
+
+### Options
+
+```
+usage: watfile [-h] [-r] (-c CATEGORIES | -d DIRECTORY) [-o OUTPUT]
+               [--backend {jev,laya}] [-n] [--copy]
+               inputs [inputs ...]
+
+positional arguments:
+  inputs                files and/or folders to process
+
+options:
+  -h, --help            show this help message and exit
+  -r, --recursive       recurse into folder inputs
+  -c CATEGORIES, --categories CATEGORIES
+                        comma-separated categories, e.g. invoice,donation,apartment
+  -d DIRECTORY, --directory
+                        target folder whose existing subfolders are the categories
+  -o OUTPUT, --output   output root for sorted files (default: same as -d, or ./sorted with -c)
+  --backend {jev,laya}  classifier backend (default: jev)
+  -n, --dry-run         print decisions without placing files
+  -m, --move            move files into the category folder (default: symlink)
+  --copy                copy files instead of symlinking
+  --symlink             create symlinks in category folders (default)
+```
+
+## Development
+
+```sh
+uv sync
+uv run pytest              # unit tests; live API tests skip without TYPESAFE_API_KEY
+```
+
+`tests/fixture/` contains 4 real arXiv PDFs with ground-truth categories
+(derived from their arXiv subject tags) used by the integration tests.
+
+## Roadmap
+
+- `laya` local backend (MLX via OpenAI-compatible HTTP)
+- batching: classify 25/50/100 files in a single API call

watfile-0.1.0/pyproject.toml

@@ -0,0 +1,42 @@
+[project]
+name = "watfile"
+version = "0.1.0"
+description = "Classify documents with the TypeSafe Jev decision model and sort them into folders"
+license = "MIT"
+readme = "README.md"
+keywords = [
+    "classification",
+    "files",
+    "llm",
+    "typesafe",
+    "cli",
+]
+classifiers = [
+    "Environment :: Console",
+    "Intended Audience :: End Users/Desktop",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Utilities",
+]
+requires-python = ">=3.12"
+dependencies = [
+    "liteparse>=2.14.6",
+    "typesafe-sdk>=0.7.0",
+]
+
+[[project.authors]]
+name = "Michael Hunger"
+email = "github@jexp.de"
+
+[project.scripts]
+watfile = "watfile:main"
+
+[project.urls]
+Repository = "https://github.com/jexp/watfile"
+
+[build-system]
+requires = ["uv_build>=0.11.21,<0.12.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = ["pytest>=9.1.1"]

watfile-0.1.0/pyproject.toml.orig

@@ -0,0 +1,37 @@
+[project]
+name = "watfile"
+version = "0.1.0"
+description = "Classify documents with the TypeSafe Jev decision model and sort them into folders"
+authors = [
+    { name = "Michael Hunger", email = "github@jexp.de" }
+]
+license = "MIT"
+readme = "README.md"
+keywords = ["classification", "files", "llm", "typesafe", "cli"]
+classifiers = [
+    "Environment :: Console",
+    "Intended Audience :: End Users/Desktop",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Utilities",
+]
+requires-python = ">=3.12"
+dependencies = [
+    "liteparse>=2.14.6",
+    "typesafe-sdk>=0.7.0",
+]
+
+[project.scripts]
+watfile = "watfile:main"
+
+[build-system]
+requires = ["uv_build>=0.11.21,<0.12.0"]
+build-backend = "uv_build"
+
+[project.urls]
+Repository = "https://github.com/jexp/watfile"
+
+[dependency-groups]
+dev = [
+    "pytest>=9.1.1",
+]

watfile-0.1.0/src/watfile/__init__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+__all__ = ["main"]

watfile-0.1.0/src/watfile/classifier/base.py

@@ -0,0 +1,24 @@
+"""Classifier abstraction — backend-agnostic file categorisation."""
+
+from abc import ABC, abstractmethod
+from dataclasses import dataclass
+from typing import Sequence
+
+
+@dataclass(frozen=True)
+class Verdict:
+    """Classification result for one document."""
+
+    category: str
+    confidence: float  # backend-reported certainty, 0..1
+    probabilities: dict[str, float]  # category -> probability
+
+
+class Classifier(ABC):
+    """Backend contract. Implementations must be stateless per call so batching
+    (classify_many) can be added without changing the interface."""
+
+    @abstractmethod
+    def classify(self, text: str, categories: Sequence[str]) -> Verdict:
+        """Classify one document's text into exactly one of *categories*."""
+        ...

watfile-0.1.0/src/watfile/classifier/jev.py

@@ -0,0 +1,33 @@
+"""TypeSafe AI Jev (System One) classifier backend."""
+
+from typing import Sequence
+
+from typesafe_sdk import Choice, TypeSafeClient
+
+from .base import Classifier, Verdict
+
+_QUESTION_KEY = "category"
+
+
+class JevClassifier(Classifier):
+    def __init__(self, model: str = "jev-latest") -> None:
+        self._client = TypeSafeClient(model=model)
+
+    def classify(self, text: str, categories: Sequence[str]) -> Verdict:
+        question = Choice(
+            instructions=(
+                "Which folder/category does this document belong to? "
+                "Judge by content, not filename. Pick exactly one."
+            ),
+            criteria={cat: None for cat in categories},
+        )
+        response = self._client.system_one(
+            state={"document": text},
+            questions={_QUESTION_KEY: question},
+        )
+        answer = response.choices[_QUESTION_KEY]
+        return Verdict(
+            category=answer.choice,
+            confidence=answer.confidence,
+            probabilities=dict(answer.probabilities),
+        )

watfile-0.1.0/src/watfile/cli.py

@@ -0,0 +1,149 @@
+"""watfile CLI entry point."""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+from typing import Sequence
+
+from .classifier.base import Classifier, Verdict
+from .classifier.jev import JevClassifier
+from .config import apply_config, load_config
+from .extract import UnsupportedFileTypeError, extract_text
+from .sorter import PLACEMENT_COPY, PLACEMENT_MOVE, PLACEMENT_SYMLINK, place_file
+
+
+def _collect_files(inputs: Sequence[str], recursive: bool) -> list[Path]:
+    files: list[Path] = []
+    for raw in inputs:
+        path = Path(raw).expanduser()
+        if path.is_dir():
+            pattern = "**/*" if recursive else "*"
+            files.extend(p for p in sorted(path.glob(pattern)) if p.is_file())
+        elif path.is_file():
+            files.append(path)
+        else:
+            print(f"warning: not found, skipped: {path}", file=sys.stderr)
+    # de-duplicate, keep order
+    seen: set[Path] = set()
+    unique: list[Path] = []
+    for f in files:
+        resolved = f.resolve()
+        if resolved not in seen:
+            seen.add(resolved)
+            unique.append(f)
+    return unique
+
+
+def _categories_from_dir(target: Path) -> list[str]:
+    if not target.is_dir():
+        raise SystemExit(f"target folder does not exist: {target}")
+    subdirs = [p.name for p in sorted(target.iterdir()) if p.is_dir()]
+    if not subdirs:
+        raise SystemExit(
+            f"target folder {target} has no subfolders to use as categories; "
+            "create one folder per category or pass -c instead"
+        )
+    return subdirs
+
+
+def _parse_categories_arg(raw: str) -> list[str]:
+    cats = [c.strip() for c in raw.split(",") if c.strip()]
+    if len(cats) < 2:
+        raise SystemExit("need at least 2 categories for -c")
+    return cats
+
+
+def _build_classifier(name: str, config) -> Classifier:
+    if name == "jev":
+        if not config.api_key:
+            raise SystemExit(
+                "no TYPESAFE_API_KEY found.\n"
+                "Set the environment variable, or create a gitignored .env file with\n"
+                "TYPESAFE_API_KEY=..., or ~/.config/watfile/config.toml with:\n"
+                'api_key = "..."\n'
+                "Get a key at https://console.typesafe.ai/"
+            )
+        apply_config(config)
+        model = config.model or "jev-latest"
+        return JevClassifier(model=model)
+    if name == "laya":
+        raise SystemExit("laya backend not yet implemented (step 80 in PLAN.md)")
+    raise SystemExit(f"unknown backend: {name}")
+
+
+def _classify_one(classifier: Classifier, path: Path, categories: Sequence[str]) -> Verdict | None:
+    try:
+        text = extract_text(path)
+    except UnsupportedFileTypeError as exc:
+        print(f"  skipped ({exc})", file=sys.stderr)
+        return None
+    if not text.strip():
+        print("  skipped (no extractable text)", file=sys.stderr)
+        return None
+    return classifier.classify(text, categories)
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="watfile",
+        description="Classify files with a decision model and sort them into category folders.",
+    )
+    parser.add_argument("inputs", nargs="+", help="files and/or folders to process")
+    parser.add_argument("-r", "--recursive", action="store_true", help="recurse into folder inputs")
+    target = parser.add_mutually_exclusive_group(required=True)
+    target.add_argument("-c", "--categories", help="comma-separated categories, e.g. invoice,donation,apartment")
+    target.add_argument("-d", "--directory", help="target folder whose existing subfolders are the categories")
+    parser.add_argument("-o", "--output", help="output root for sorted files (default: same as -d, or ./sorted with -c)")
+    parser.add_argument("--backend", default="jev", choices=["jev", "laya"], help="classifier backend (default: jev)")
+    parser.add_argument("-n", "--dry-run", action="store_true", help="print decisions without placing files")
+    placement = parser.add_mutually_exclusive_group()
+    placement.add_argument("-m", "--move", action="store_true", help="move files into the category folder (default: symlink)")
+    placement.add_argument("--copy", action="store_true", help="copy files instead of symlinking")
+    placement.add_argument("--symlink", action="store_true", help="create symlinks in category folders (default)")
+    args = parser.parse_args(argv)
+
+    if args.categories:
+        categories = _parse_categories_arg(args.categories)
+        target_root = Path(args.output) if args.output else Path("sorted")
+    else:
+        target_root = Path(args.directory).expanduser()
+        categories = _categories_from_dir(target_root)
+        if args.output:
+            target_root = Path(args.output).expanduser()
+
+    files = _collect_files(args.inputs, args.recursive)
+    if not files:
+        print("no files to process", file=sys.stderr)
+        return 1
+
+    print(f"categories: {', '.join(categories)}")
+    print(f"files: {len(files)}  backend: {args.backend}  target: {target_root}"
+          + ("  (dry-run)" if args.dry_run else ""))
+
+    classifier = _build_classifier(args.backend, load_config())
+
+    failures = 0
+    for path in files:
+        print(f"{path.name}: ", end="", flush=True)
+        verdict = _classify_one(classifier, path, categories)
+        if verdict is None:
+            failures += 1
+            continue
+        placement = (
+            PLACEMENT_MOVE if args.move else PLACEMENT_COPY if args.copy else PLACEMENT_SYMLINK
+        )
+        result = place_file(path, verdict.category, target_root, dry_run=args.dry_run, placement=placement)
+        action = ("would " if args.dry_run else "") + {
+            PLACEMENT_MOVE: "move",
+            PLACEMENT_COPY: "copy",
+            PLACEMENT_SYMLINK: "symlink",
+        }[placement]
+        print(f"{verdict.category} (conf {verdict.confidence:.2f}) -> {action} to {result.destination}")
+
+    return 1 if failures == len(files) else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

watfile-0.1.0/src/watfile/config.py

@@ -0,0 +1,108 @@
+"""Configuration for watfile: API key and model resolution.
+
+Resolution order (first wins):
+  1. TYPESAFE_API_KEY environment variable
+  2. .env file in the current directory (gitignored)
+  3. ~/.config/watfile/config.toml  (XDG config home; $WATFILE_CONFIG overrides)
+     keys: api_key, base_url, model
+
+Uses stdlib tomllib only (Python 3.11+), no extra dependency.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import tomllib
+from dataclasses import dataclass
+from pathlib import Path
+
+#: env var the TypeSafe SDK itself reads; used as source and final sink.
+API_KEY_ENV = "TYPESAFE_API_KEY"
+
+#: project-local env file, expected gitignored.
+DOTENV_FILE = ".env"
+
+#: config file name inside the config dir.
+CONFIG_FILENAME = "config.toml"
+
+
+def default_config_path() -> Path:
+    """XDG-style config path: $XDG_CONFIG_HOME/watfile/config.toml (default ~/.config/...)."""
+    base = os.environ.get("XDG_CONFIG_HOME")
+    root = Path(base) if base else Path.home() / ".config"
+    return root / "watfile" / CONFIG_FILENAME
+
+
+def _load_toml(path: Path) -> dict:
+    try:
+        with open(path, "rb") as fh:
+            return tomllib.load(fh)
+    except FileNotFoundError:
+        return {}
+    except tomllib.TOMLDecodeError as exc:
+        print(f"warning: ignoring invalid config {path}: {exc}", file=sys.stderr)
+        return {}
+
+
+@dataclass(frozen=True)
+class Config:
+    api_key: str | None = None
+    base_url: str | None = None
+    model: str | None = None
+
+
+#: TOML config uses friendly field names (api_key/base_url/model);
+#: .env and environment use the SDK's variable names.
+_TOML_TO_ENV = {
+    "api_key": API_KEY_ENV,
+    "base_url": "TYPESAFE_BASE_URL",
+    "model": "TYPESAFE_DEFAULT_MODEL",
+}
+
+
+def load_config() -> Config:
+    """Merge TOML config, ./.env, and environment into a Config (env wins)."""
+    values: dict[str, str] = {}
+
+    config_path = Path(os.environ.get("WATFILE_CONFIG") or default_config_path())
+    for key, value in _load_toml(config_path).items():
+        if isinstance(value, str) and key in _TOML_TO_ENV:
+            values[_TOML_TO_ENV[key]] = value
+
+    dotenv_path = Path(DOTENV_FILE)
+    if dotenv_path.is_file():
+        for line in dotenv_path.read_text().splitlines():
+            line = line.strip()
+            if not line or line.startswith("#") or "=" not in line:
+                continue
+            key, _, value = line.partition("=")
+            # .env overrides TOML (project-local beats user-global), but not real env
+            if key.strip() not in os.environ:
+                values[key.strip()] = value.strip().strip("'\"")
+
+    # real environment always wins
+    for key in (API_KEY_ENV, "TYPESAFE_BASE_URL", "TYPESAFE_DEFAULT_MODEL"):
+        if os.environ.get(key):
+            values[key] = os.environ[key]
+
+    return Config(
+        api_key=values.get(API_KEY_ENV),
+        base_url=values.get("TYPESAFE_BASE_URL"),
+        model=values.get("TYPESAFE_DEFAULT_MODEL"),
+    )
+
+
+def apply_config(config: Config) -> None:
+    """Export resolved values into the environment so the typesafe-sdk picks them up."""
+    if config.api_key:
+        os.environ[API_KEY_ENV] = config.api_key
+    if config.base_url:
+        os.environ["TYPESAFE_BASE_URL"] = config.base_url
+    if config.model:
+        os.environ["TYPESAFE_DEFAULT_MODEL"] = config.model
+
+
+def api_key_available() -> bool:
+    """True if a key is resolvable from env, ./.env, or the config file."""
+    return load_config().api_key is not None

watfile-0.1.0/src/watfile/extract.py

@@ -0,0 +1,41 @@
+"""Text extraction from files for classification."""
+
+from pathlib import Path
+
+from liteparse import LiteParse
+
+#: Extensions read directly as text.
+TEXT_EXTENSIONS = {".txt", ".md", ".markdown", ".rst", ".log", ".csv", ".json"}
+
+#: Extensions parsed via liteparse.
+PARSED_EXTENSIONS = {".pdf"}
+
+#: Cap extracted text sent to the classifier (chars). Jev works on decision-relevant
+#: context; whole documents are unnecessary and slow.
+MAX_TEXT_CHARS = 8_000
+
+#: Only parse the first pages of PDFs — title/abstract carry the classification signal,
+#: and parsing hundreds of pages wastes seconds per file.
+MAX_PDF_PAGES = 2
+
+
+class UnsupportedFileTypeError(ValueError):
+    pass
+
+
+def extract_text(path: Path) -> str:
+    """Return text content of *path*, truncated to MAX_TEXT_CHARS.
+
+    Plain-text formats are read directly; PDFs go through liteparse.
+    """
+    ext = path.suffix.lower()
+    if ext in TEXT_EXTENSIONS:
+        text = path.read_text(encoding="utf-8", errors="replace")
+    elif ext in PARSED_EXTENSIONS:
+        # OCR off: classification only needs embedded text, and OCR adds 1.5-7s/page.
+        # Scanned-image PDFs will come back empty and are skipped by the caller.
+        result = LiteParse(ocr_enabled=False, max_pages=MAX_PDF_PAGES, quiet=True).parse(path)
+        text = result.text
+    else:
+        raise UnsupportedFileTypeError(f"unsupported extension: {ext or '<none>'} ({path.name})")
+    return text[:MAX_TEXT_CHARS]

watfile-0.1.0/src/watfile/sorter.py

@@ -0,0 +1,72 @@
+"""Move classified files into category folders."""
+
+import os
+import shutil
+from dataclasses import dataclass
+from pathlib import Path
+
+_UNCATEGORIZED = "uncategorized"
+
+
+@dataclass(frozen=True)
+class MoveResult:
+    source: Path
+    destination: Path
+    moved: bool  # False in dry-run or on failure
+
+
+#: How a classified file is placed into its category folder.
+PLACEMENT_SYMLINK = "symlink"  # default: leave original in place, link in category folder
+PLACEMENT_MOVE = "move"
+PLACEMENT_COPY = "copy"
+
+
+def sanitize_category(name: str) -> str:
+    """Make a category safe as a folder name."""
+    keep = "-_. ()"
+    cleaned = "".join(c if (c.isalnum() or c in keep) else "_" for c in name.strip())
+    return cleaned.strip(". ") or _UNCATEGORIZED
+
+
+def _resolve_collision(destination: Path) -> Path:
+    """Return a non-existing variant of *destination* by appending a counter.
+
+    Uses lexists so dangling symlinks also count as occupied.
+    """
+    if not os.path.lexists(destination):
+        return destination
+    stem, suffix = destination.stem, destination.suffix
+    for i in range(1, 1000):
+        candidate = destination.with_name(f"{stem}_{i}{suffix}")
+        if not os.path.lexists(candidate):
+            return candidate
+    raise FileExistsError(f"could not find free name for {destination}")
+
+
+def place_file(
+    source: Path,
+    category: str,
+    target_root: Path,
+    *,
+    dry_run: bool = False,
+    placement: str = PLACEMENT_SYMLINK,
+) -> MoveResult:
+    """Place *source* into target_root/<category>/ as symlink (default), move, or copy.
+
+    Symlinks point at the absolute original location; move leaves the original
+    gone; copy duplicates the file.
+    """
+    category_dir = target_root / sanitize_category(category)
+    destination = _resolve_collision(category_dir / source.name)
+    if dry_run:
+        return MoveResult(source=source, destination=destination, moved=False)
+    category_dir.mkdir(parents=True, exist_ok=True)
+    if placement == PLACEMENT_SYMLINK:
+        os.symlink(source.resolve(), destination)
+    elif placement == PLACEMENT_COPY:
+        shutil.copy2(source, destination)
+    elif placement == PLACEMENT_MOVE:
+        shutil.move(str(source), destination)
+    else:
+        raise ValueError(f"unknown placement: {placement}")
+    return MoveResult(source=source, destination=destination, moved=True)