footprint-trace 0.1.0 → 0.2.0

package/README.md

@@ -25,19 +25,18 @@ trace → chat with claude → collect → train → install → /footprint in O
 
 | | trace / collect | train / serve |
 |---|---|---|
-| **macOS (Apple Silicon)** | ✅ | ✅ (MLX) |
-| **macOS (Intel)** | ✅ | ❌ |
-| **Linux** | ✅ | ❌ (for now) |
-| **Windows** | ✅ | ❌ (for now) |
+| **macOS (Apple Silicon)** | ✅ | ✅ MLX backend |
+| **Linux** | ✅ | ✅ torch backend (CUDA or CPU) |
+| **Windows** | ✅ | ✅ torch backend (CUDA or CPU) |
+| **macOS (Intel)** | ✅ | ✅ torch backend (CPU) |
 
 - [Claude Code](https://claude.com/claude-code) installed and used at least once
 - Python 3.9+
 - Node.js 18+ (only if installing via npm)
 
-Training and serving use [MLX](https://github.com/ml-explore/mlx), which is
-Apple-Silicon-only today. On Linux/Windows you can still trace and collect —
-your `data/` folder is portable; train it on any Apple Silicon Mac. Other
-backends (llama.cpp / unsloth) are welcome contributions.
+footprint picks the backend automatically: [MLX](https://github.com/ml-explore/mlx)
+on Apple Silicon, PyTorch + PEFT everywhere else (uses CUDA when available;
+CPU works but trains slowly — prefer a GPU or drop `FOOTPRINT_ITERS`).
 
 ## Install
 
@@ -61,7 +60,7 @@ python3 footprint.py setup
 npm install -g footprint-trace      # or the git clone above
 ```
 
-`setup` skips MLX automatically. `trace` and `collect` work; copy `data/` to a Mac to train.
+First run installs the torch backend (one-time, ~2 GB). CUDA GPU picked up automatically.
 
 ### Windows
 
@@ -71,7 +70,7 @@ Use PowerShell (Python 3 from python.org or the Microsoft Store):
 npm install -g footprint-trace
 ```
 
-Same scope as Linux: trace + collect. WSL2 works the same way.
+Same as Linux: full trace/collect/train/serve via the torch backend. WSL2 also works.
 
 ## Getting started
 
@@ -100,14 +99,15 @@ footprint collect     # transcripts -> data/train.jsonl + data/valid.jsonl
 footprint train       # LoRA fine-tune (~10 min, downloads ~1 GB base model first time)
 ```
 
-**4. Install the always-on server (macOS).**
+**4. Install the always-on server.**
 
 ```sh
 footprint install
 ```
 
-This registers a launchd agent — the server starts at login, restarts if it
-crashes, and you never run it by hand. It also wires up OpenCode:
+This registers the server with your OS — launchd (macOS), systemd (Linux) or
+Task Scheduler (Windows) — so it starts at login, restarts if it crashes, and
+you never run it by hand. It also wires up OpenCode:
 
 - provider `footprint` at `http://127.0.0.1:8399/v1`
 - a `/footprint <task>` command inside OpenCode
@@ -136,7 +136,7 @@ Or point any OpenAI-compatible tool at the server:
 | `footprint trace` | arm tracing (run before starting Claude) |
 | `footprint collect [dir]` | parse transcripts of a project (default: current dir; unknown dir = all projects) |
 | `footprint train` | LoRA fine-tune on collected data |
-| `footprint install` | launchd auto-server + OpenCode integration (macOS) |
+| `footprint install` | auto-start server (launchd / systemd / Task Scheduler) + OpenCode integration |
 | `footprint serve` | run the server manually (fallback; `install` makes this unnecessary) |
 | `footprint status` | model, example count, adapter, tracing, server state |
 
@@ -144,15 +144,15 @@ Or point any OpenAI-compatible tool at the server:
 
 | Env var | Default | |
 |---|---|---|
-| `FOOTPRINT_MODEL` | `mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit` | any MLX chat model |
+| `FOOTPRINT_MODEL` | `mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit` (mac) / `Qwen/Qwen2.5-Coder-1.5B-Instruct` | any chat model of the backend |
 | `FOOTPRINT_ITERS` | `300` | training iterations |
 | `FOOTPRINT_PORT` | `8399` | server port |
 
 ## Troubleshooting
 
 - **`no transcripts found`** — either you haven't used Claude Code in this project, or the trace marker is newer than all sessions. Chat first, or `rm ~/.claude/footprint-trace`.
-- **server not responding** — check `/tmp/footprint-serve.log`; `launchctl kickstart -k gui/$UID/com.footprint.serve` restarts it.
-- **`mlx_lm` import fails** — re-run `footprint setup` (Apple Silicon only).
+- **server not responding** — macOS: `/tmp/footprint-serve.log`, `launchctl kickstart -k gui/$UID/com.footprint.serve`; Linux: `journalctl --user -u footprint`, `systemctl --user restart footprint`; Windows: `schtasks /Run /TN footprint-serve`.
+- **backend import fails** (`mlx_lm` / `torch`) — re-run `footprint setup`.
 - **quality is rough** — more data beats more iterations: keep tracing, re-collect, re-train. 1.5B is small; try `FOOTPRINT_MODEL=mlx-community/Qwen2.5-Coder-7B-Instruct-4bit` if you have ≥16 GB RAM.
 
 ## Privacy

package/bin/footprint.js

@@ -5,7 +5,10 @@ const fs = require("fs");
 
 const root = path.join(__dirname, "..");
 const script = path.join(root, "footprint.py");
-const venvPy = path.join(root, ".venv", "bin", "python");
+const win = process.platform === "win32";
+const venvPy = win
+  ? path.join(root, ".venv", "Scripts", "python.exe")
+  : path.join(root, ".venv", "bin", "python");
 const args = process.argv.slice(2);
 
 if (args.length === 0) {
@@ -13,8 +16,8 @@ if (args.length === 0) {
 }
 
 if (!fs.existsSync(venvPy)) {
-  console.log("first run — setting up (venv + mlx-lm, takes a minute)…");
-  const s = spawnSync("python3", [script, "setup"], { stdio: "inherit" });
+  console.log("first run — setting up (venv + model backend)…");
+  const s = spawnSync(win ? "python" : "python3", [script, "setup"], { stdio: "inherit" });
   if (s.status) process.exit(s.status);
 }
 

package/footprint.py

@@ -1,216 +1,6 @@
 #!/usr/bin/env python3
-"""footprint — learn a local SLM from your Claude Code transcripts.
-
-Commands:
-  setup                 create .venv, install mlx-lm, link /footprint skill
-  trace                 arm tracing: sessions from now on become training data (run before chatting)
-  collect [project]     parse ~/.claude/projects transcripts -> data/{train,valid}.jsonl
-  train                 LoRA fine-tune base model on collected data
-  serve                 OpenAI-compatible API at localhost:8399 (launchd runs this; users don't)
-  install               launchd agent (server always on) + OpenCode /footprint command
-  status                what's collected/trained
-"""
-import json, os, re, subprocess, sys, glob, random, time, plistlib
-
-ROOT = os.path.dirname(os.path.abspath(__file__))
-DATA = os.path.join(ROOT, "data")
-ADAPTERS = os.path.join(ROOT, "adapters")
-PROJECTS = os.path.expanduser("~/.claude/projects")
-TRACE_MARK = os.path.expanduser("~/.claude/footprint-trace")
-LAUNCHD = os.path.expanduser("~/Library/LaunchAgents/com.footprint.serve.plist")
-OPENCODE = os.path.expanduser("~/.config/opencode")
-MODEL = os.environ.get("FOOTPRINT_MODEL", "mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit")
-CTX_BUDGET = 6000      # chars of history per training example
-RESULT_CAP = 1500      # chars kept per tool result
-
-SYSTEM = ("You are footprint, a local coding agent trained on Claude Code sessions. "
-          "To act, emit tool calls as <tool name=\"NAME\">{json args}</tool>. "
-          "Say DONE when the task is complete.")
-
-
-def _text(content, cap=None):
-    """Flatten a Claude message content (str or block list) to training text."""
-    if isinstance(content, str):
-        out = content
-    else:
-        parts = []
-        for b in content or []:
-            t = b.get("type")
-            if t == "text":
-                parts.append(b["text"])
-            elif t == "tool_use":
-                parts.append(f'<tool name="{b["name"]}">{json.dumps(b.get("input", {}))}</tool>')
-            elif t == "tool_result":
-                c = b.get("content")
-                c = "".join(x.get("text", "") for x in c) if isinstance(c, list) else str(c or "")
-                parts.append(f"<tool_result>{c[:RESULT_CAP]}</tool_result>")
-        out = "\n".join(p for p in parts if p.strip())
-    out = re.sub(r"<system-reminder>.*?</system-reminder>", "", out, flags=re.S)
-    out = re.sub(r"<local-command-caveat>.*?</local-command-caveat>", "", out, flags=re.S)
-    return (out[:cap] if cap else out).strip()
-
-
-def parse_session(path):
-    """One transcript file -> list of alternating {role, content} messages."""
-    msgs = []
-    for line in open(path, errors="replace"):
-        try:
-            rec = json.loads(line)
-        except json.JSONDecodeError:
-            continue
-        if rec.get("type") not in ("user", "assistant") or rec.get("isSidechain") or rec.get("isMeta"):
-            continue
-        m = rec.get("message") or {}
-        role = m.get("role")
-        txt = _text(m.get("content"))
-        if not role or not txt:
-            continue
-        if msgs and msgs[-1]["role"] == role:
-            msgs[-1]["content"] += "\n" + txt
-        else:
-            msgs.append({"role": role, "content": txt})
-    return msgs
-
-
-def trace():
-    """Arm tracing. Claude Code already logs every session to ~/.claude/projects;
-    the marker makes collect use only sessions started after this moment."""
-    os.makedirs(os.path.dirname(TRACE_MARK), exist_ok=True)
-    with open(TRACE_MARK, "w") as f:
-        f.write(str(time.time()))
-    print("tracing armed. chat with Claude Code as usual — those sessions become training data.\n"
-          "then: footprint.py collect && footprint.py train\n"
-          "(delete ~/.claude/footprint-trace to collect ALL history again)")
-
-
-def collect(project=None):
-    since = float(open(TRACE_MARK).read()) if os.path.exists(TRACE_MARK) else 0.0
-    slug = "-" + os.path.abspath(project or os.getcwd()).strip("/").replace("/", "-").replace(".", "-")
-    dirs = [os.path.join(PROJECTS, slug)] if os.path.isdir(os.path.join(PROJECTS, slug)) \
-        else glob.glob(os.path.join(PROJECTS, "*"))
-    examples = []
-    for d in dirs:
-        for f in glob.glob(os.path.join(d, "*.jsonl")):
-            if os.path.getmtime(f) < since:
-                continue
-            msgs = parse_session(f)
-            # one example per assistant turn, with preceding context under budget
-            for i, m in enumerate(msgs):
-                if m["role"] != "assistant":
-                    continue
-                ctx, size = [], 0
-                for prev in reversed(msgs[:i]):
-                    size += len(prev["content"])
-                    if size > CTX_BUDGET:
-                        break
-                    ctx.insert(0, prev)
-                if not ctx or ctx[0]["role"] != "user":
-                    continue
-                examples.append({"messages": [{"role": "system", "content": SYSTEM}, *ctx, m]})
-    if not examples:
-        hint = " (trace marker active — only sessions after `trace` count; delete ~/.claude/footprint-trace for all history)" if since else ""
-        sys.exit(f"no transcripts found under {PROJECTS}{hint}")
-    random.seed(0)
-    random.shuffle(examples)
-    n_valid = max(1, len(examples) // 10)
-    os.makedirs(DATA, exist_ok=True)
-    for name, chunk in [("valid", examples[:n_valid]), ("train", examples[n_valid:])]:
-        with open(os.path.join(DATA, f"{name}.jsonl"), "w") as fh:
-            for e in chunk:
-                fh.write(json.dumps(e) + "\n")
-    print(f"collected {len(examples)} examples ({len(examples)-n_valid} train / {n_valid} valid) -> {DATA}")
-
-
-def train():
-    if not os.path.exists(os.path.join(DATA, "train.jsonl")):
-        sys.exit("no data. run: footprint.py collect")
-    subprocess.run([sys.executable, "-m", "mlx_lm", "lora", "--model", MODEL, "--train",
-                    "--data", DATA, "--adapter-path", ADAPTERS,
-                    "--iters", os.environ.get("FOOTPRINT_ITERS", "300"),
-                    "--max-seq-length", "4096", "--batch-size", "1"], check=True)
-    print(f"adapter saved -> {ADAPTERS}")
-
-
-def serve():
-    port = os.environ.get("FOOTPRINT_PORT", "8399")
-    args = [sys.executable, "-m", "mlx_lm", "server", "--model", MODEL,
-            "--host", "127.0.0.1", "--port", port]
-    if os.path.exists(os.path.join(ADAPTERS, "adapters.safetensors")):
-        args += ["--adapter-path", ADAPTERS]
-    else:
-        print("(no adapter yet — serving base model; run train first for footprint behavior)")
-    print(f"OpenAI-compatible API: http://127.0.0.1:{port}/v1  (model name: footprint)\n"
-          "point Cursor / OpenCode / Codex / any OpenAI-compatible client here, api key: none")
-    subprocess.run(args, check=True)
-
-
-def install():
-    """Server runs itself (launchd, KeepAlive) + OpenCode /footprint command. User never runs serve."""
-    if sys.platform != "darwin":
-        sys.exit("install is macOS-only (launchd + MLX). on Linux/Windows run serve manually once MLX alternatives land.")
-    py = os.path.join(ROOT, ".venv", "bin", "python")
-    if not os.path.exists(py):
-        sys.exit("no .venv. run: python3 footprint.py setup")
-    port = os.environ.get("FOOTPRINT_PORT", "8399")
-    os.makedirs(os.path.dirname(LAUNCHD), exist_ok=True)
-    with open(LAUNCHD, "wb") as f:
-        plistlib.dump({"Label": "com.footprint.serve",
-                       "ProgramArguments": [py, os.path.join(ROOT, "footprint.py"), "serve"],
-                       "RunAtLoad": True, "KeepAlive": True,
-                       "StandardOutPath": "/tmp/footprint-serve.log",
-                       "StandardErrorPath": "/tmp/footprint-serve.log"}, f)
-    subprocess.run(["launchctl", "unload", LAUNCHD], capture_output=True)
-    subprocess.run(["launchctl", "load", "-w", LAUNCHD], check=True)
-
-    cfg_path = os.path.join(OPENCODE, "opencode.json")
-    cfg = json.load(open(cfg_path)) if os.path.exists(cfg_path) else {"$schema": "https://opencode.ai/config.json"}
-    cfg.setdefault("provider", {})["footprint"] = {
-        "npm": "@ai-sdk/openai-compatible", "name": "footprint",
-        "options": {"baseURL": f"http://127.0.0.1:{port}/v1"},
-        "models": {"footprint": {"name": "footprint"}}}
-    os.makedirs(os.path.join(OPENCODE, "command"), exist_ok=True)
-    with open(cfg_path, "w") as f:
-        json.dump(cfg, f, indent=2)
-    with open(os.path.join(OPENCODE, "command", "footprint.md"), "w") as f:
-        f.write("---\ndescription: do task with footprint — local model trained on your Claude sessions\n"
-                "model: footprint/footprint\n---\n$ARGUMENTS\n")
-    print(f"installed:\n"
-          f"  server: launchd keeps http://127.0.0.1:{port}/v1 running (log: /tmp/footprint-serve.log)\n"
-          f"  opencode: /footprint <task> uses it; or pick model 'footprint' via /models\n"
-          f"  cursor/codex: base URL http://127.0.0.1:{port}/v1, any api key")
-
-
-def status():
-    tr = os.path.join(DATA, "train.jsonl")
-    n = sum(1 for _ in open(tr)) if os.path.exists(tr) else 0
-    print(f"model: {MODEL}\nexamples: {n}\nadapter: "
-          f"{'yes' if os.path.exists(os.path.join(ADAPTERS, 'adapters.safetensors')) else 'no'}\n"
-          f"tracing: {'armed since ' + time.ctime(float(open(TRACE_MARK).read())) if os.path.exists(TRACE_MARK) else 'off (collect uses all history)'}\n"
-          f"server: {'launchd-managed' if os.path.exists(LAUNCHD) else 'not installed (run install)'}")
-
-
-def setup():
-    venv = os.path.join(ROOT, ".venv")
-    if not os.path.exists(venv):
-        subprocess.run([sys.executable, "-m", "venv", venv], check=True)
-    if sys.platform == "darwin":
-        subprocess.run([os.path.join(venv, "bin", "pip"), "install", "-q", "mlx-lm"], check=True)
-    else:
-        print("mlx-lm skipped (Apple Silicon only). trace/collect work here; "
-              "copy data/ to a Mac for train/serve.")
-    link = os.path.expanduser("~/.claude/skills/footprint")
-    os.makedirs(os.path.dirname(link), exist_ok=True)
-    if not os.path.exists(link):
-        os.symlink(os.path.join(ROOT, "skills", "footprint"), link)
-    print(f"ready. /footprint skill linked -> {link}\nuse: {venv}/bin/python footprint.py collect|train|run")
-
+"""footprint entry point — real code lives in footprint_core/."""
+from footprint_core.cli import main
 
 if __name__ == "__main__":
-    cmd = sys.argv[1] if len(sys.argv) > 1 else "status"
-    {"collect": lambda: collect(sys.argv[2] if len(sys.argv) > 2 else None),
-     "trace": trace,
-     "train": train,
-     "serve": serve,
-     "install": install,
-     "status": status,
-     "setup": setup}.get(cmd, lambda: sys.exit(__doc__))()
+    main()

package/footprint_core/backends/__init__.py

@@ -0,0 +1,9 @@
+from .. import config
+
+
+def get():
+    if config.IS_MAC:
+        from . import mlx_backend as b
+    else:
+        from . import torch_backend as b
+    return b

package/footprint_core/backends/mlx_backend.py

@@ -0,0 +1,23 @@
+"""Apple Silicon backend: mlx-lm (train + OpenAI-compatible server)."""
+import os, subprocess, sys
+
+from .. import config
+
+
+def train():
+    subprocess.run([sys.executable, "-m", "mlx_lm", "lora", "--model", config.MODEL, "--train",
+                    "--data", config.DATA, "--adapter-path", config.ADAPTERS,
+                    "--iters", config.ITERS, "--max-seq-length", "4096",
+                    "--batch-size", "1", "--grad-checkpoint"], check=True)
+    print(f"adapter saved -> {config.ADAPTERS}")
+
+
+def serve():
+    args = [sys.executable, "-m", "mlx_lm", "server", "--model", config.MODEL,
+            "--host", "127.0.0.1", "--port", config.PORT]
+    if os.path.exists(os.path.join(config.ADAPTERS, "adapters.safetensors")):
+        args += ["--adapter-path", config.ADAPTERS]
+    else:
+        print("(no adapter yet — serving base model; run train first for footprint behavior)")
+    print(f"OpenAI-compatible API: http://127.0.0.1:{config.PORT}/v1  (model name: footprint)")
+    subprocess.run(args, check=True)

package/footprint_core/backends/torch_backend.py

@@ -0,0 +1,93 @@
+"""Linux/Windows backend: transformers + peft (CUDA if available, else CPU)."""
+import json, os
+
+from .. import config
+
+
+def train():
+    from datasets import load_dataset
+    from peft import LoraConfig
+    from trl import SFTConfig, SFTTrainer
+    ds = load_dataset("json", data_files={"train": os.path.join(config.DATA, "train.jsonl"),
+                                          "valid": os.path.join(config.DATA, "valid.jsonl")})
+    trainer = SFTTrainer(
+        model=config.MODEL,
+        args=SFTConfig(output_dir=config.ADAPTERS, max_steps=int(config.ITERS),
+                       per_device_train_batch_size=1, gradient_checkpointing=True,
+                       max_length=4096, logging_steps=25, report_to=[]),
+        train_dataset=ds["train"], eval_dataset=ds["valid"],
+        peft_config=LoraConfig(r=8, lora_alpha=16, target_modules="all-linear"))
+    trainer.train()
+    trainer.save_model(config.ADAPTERS)
+    print(f"adapter saved -> {config.ADAPTERS}")
+
+
+def serve():
+    import torch
+    from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+
+    tok = AutoTokenizer.from_pretrained(config.MODEL)
+    model = AutoModelForCausalLM.from_pretrained(config.MODEL, torch_dtype="auto", device_map="auto")
+    if config.has_adapter():
+        from peft import PeftModel
+        model = PeftModel.from_pretrained(model, config.ADAPTERS)
+    else:
+        print("(no adapter yet — serving base model; run train first for footprint behavior)")
+    model.eval()
+
+    def generate(messages, max_tokens, temperature):
+        ids = tok.apply_chat_template(messages, add_generation_prompt=True,
+                                      return_tensors="pt").to(model.device)
+        with torch.no_grad():
+            out = model.generate(ids, max_new_tokens=max_tokens, do_sample=temperature > 0,
+                                 temperature=max(temperature, 1e-3),
+                                 pad_token_id=tok.eos_token_id)
+        return tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
+
+    class Handler(BaseHTTPRequestHandler):
+        def _json(self, obj, ctype="application/json"):
+            body = json.dumps(obj).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", ctype)
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+        def do_GET(self):
+            if self.path.startswith("/v1/models"):
+                self._json({"object": "list", "data": [{"id": "footprint", "object": "model"}]})
+            else:
+                self.send_error(404)
+
+        def do_POST(self):
+            if self.path.rstrip("/") != "/v1/chat/completions":
+                self.send_error(404)
+                return
+            req = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
+            text = generate(req.get("messages", []), req.get("max_tokens") or 1024,
+                            req.get("temperature") if req.get("temperature") is not None else 0.7)
+            if req.get("stream"):
+                # ponytail: whole answer as one SSE chunk; true token streaming if UX needs it
+                self.send_response(200)
+                self.send_header("Content-Type", "text/event-stream")
+                self.end_headers()
+                for payload in (
+                        {"id": "fp", "object": "chat.completion.chunk", "model": "footprint",
+                         "choices": [{"index": 0, "delta": {"role": "assistant", "content": text},
+                                      "finish_reason": None}]},
+                        {"id": "fp", "object": "chat.completion.chunk", "model": "footprint",
+                         "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]}):
+                    self.wfile.write(f"data: {json.dumps(payload)}\n\n".encode())
+                self.wfile.write(b"data: [DONE]\n\n")
+            else:
+                self._json({"id": "fp", "object": "chat.completion", "model": "footprint",
+                            "choices": [{"index": 0, "message": {"role": "assistant", "content": text},
+                                         "finish_reason": "stop"}]})
+
+        def log_message(self, *a):
+            pass
+
+    print(f"OpenAI-compatible API: http://127.0.0.1:{config.PORT}/v1  (model name: footprint)\n"
+          f"device: {'cuda' if torch.cuda.is_available() else 'cpu'}")
+    ThreadingHTTPServer(("127.0.0.1", int(config.PORT)), Handler).serve_forever()

package/footprint_core/cli.py

@@ -0,0 +1,29 @@
+"""footprint — learn a local SLM from your Claude Code transcripts.
+
+Commands:
+  setup                 create .venv + install the platform backend (mlx / torch)
+  trace                 arm tracing: sessions from now on become training data (run before chatting)
+  collect [project]     parse ~/.claude/projects transcripts -> data/{train,valid}.jsonl
+  train                 LoRA fine-tune base model on collected data
+  serve                 OpenAI-compatible API at localhost:8399 (install runs this for you)
+  install               auto-start server (launchd / systemd / Task Scheduler) + OpenCode /footprint
+  status                what's collected/trained
+"""
+import os, sys
+
+from . import config, service, transcripts
+from .backends import get
+
+
+def main(argv=None):
+    argv = sys.argv[1:] if argv is None else argv
+    cmd = argv[0] if argv else "status"
+    if cmd == "train" and not os.path.exists(os.path.join(config.DATA, "train.jsonl")):
+        sys.exit("no data. run: footprint collect")
+    {"collect": lambda: transcripts.collect(argv[1] if len(argv) > 1 else None),
+     "trace": service.trace,
+     "train": lambda: get().train(),
+     "serve": lambda: get().serve(),
+     "install": service.install,
+     "status": service.status,
+     "setup": service.setup}.get(cmd, lambda: sys.exit(__doc__))()

package/footprint_core/config.py

@@ -0,0 +1,33 @@
+import os, sys
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+DATA = os.path.join(ROOT, "data")
+ADAPTERS = os.path.join(ROOT, "adapters")
+PROJECTS = os.path.expanduser("~/.claude/projects")
+TRACE_MARK = os.path.expanduser("~/.claude/footprint-trace")
+OPENCODE = os.path.expanduser("~/.config/opencode")
+LAUNCHD = os.path.expanduser("~/Library/LaunchAgents/com.footprint.serve.plist")
+SYSTEMD = os.path.expanduser("~/.config/systemd/user/footprint.service")
+
+IS_MAC = sys.platform == "darwin"
+IS_WIN = sys.platform == "win32"
+VENV_PY = os.path.join(ROOT, ".venv", "Scripts", "python.exe") if IS_WIN \
+    else os.path.join(ROOT, ".venv", "bin", "python")
+ENTRY = os.path.join(ROOT, "footprint.py")
+
+MODEL = os.environ.get("FOOTPRINT_MODEL",
+                       "mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit" if IS_MAC
+                       else "Qwen/Qwen2.5-Coder-1.5B-Instruct")
+PORT = os.environ.get("FOOTPRINT_PORT", "8399")
+ITERS = os.environ.get("FOOTPRINT_ITERS", "300")
+CTX_BUDGET = 6000      # chars of history per training example
+RESULT_CAP = 1500      # chars kept per tool result
+
+SYSTEM = ("You are footprint, a local coding agent trained on Claude Code sessions. "
+          "To act, emit tool calls as <tool name=\"NAME\">{json args}</tool>. "
+          "Say DONE when the task is complete.")
+
+
+def has_adapter():
+    return any(os.path.exists(os.path.join(ADAPTERS, f))
+               for f in ("adapters.safetensors", "adapter_model.safetensors"))

package/footprint_core/service.py

@@ -0,0 +1,97 @@
+"""trace / status / setup / install — everything that isn't parsing or the model."""
+import json, os, plistlib, subprocess, sys, time
+
+from . import config
+
+
+def trace():
+    """Arm tracing. Claude Code already logs every session to ~/.claude/projects;
+    the marker makes collect use only sessions started after this moment."""
+    os.makedirs(os.path.dirname(config.TRACE_MARK), exist_ok=True)
+    with open(config.TRACE_MARK, "w") as f:
+        f.write(str(time.time()))
+    print("tracing armed. chat with Claude Code as usual — those sessions become training data.\n"
+          "then: footprint collect && footprint train\n"
+          "(delete ~/.claude/footprint-trace to collect ALL history again)")
+
+
+def status():
+    tr = os.path.join(config.DATA, "train.jsonl")
+    n = sum(1 for _ in open(tr)) if os.path.exists(tr) else 0
+    installed = os.path.exists(config.LAUNCHD) if config.IS_MAC else \
+        os.path.exists(config.SYSTEMD) if not config.IS_WIN else \
+        subprocess.run(["schtasks", "/Query", "/TN", "footprint-serve"],
+                       capture_output=True).returncode == 0
+    print(f"backend: {'mlx' if config.IS_MAC else 'torch'}\nmodel: {config.MODEL}\nexamples: {n}\n"
+          f"adapter: {'yes' if config.has_adapter() else 'no'}\n"
+          f"tracing: {'armed since ' + time.ctime(float(open(config.TRACE_MARK).read())) if os.path.exists(config.TRACE_MARK) else 'off (collect uses all history)'}\n"
+          f"server: {'auto-managed' if installed else 'not installed (run install)'}")
+
+
+def setup():
+    venv = os.path.join(config.ROOT, ".venv")
+    if not os.path.exists(venv):
+        subprocess.run([sys.executable, "-m", "venv", venv], check=True)
+    pkgs = ["mlx-lm"] if config.IS_MAC else ["torch", "transformers", "peft", "trl", "datasets"]
+    if not config.IS_MAC:
+        print("installing torch + transformers stack (one-time, ~2 GB)…")
+    subprocess.run([config.VENV_PY, "-m", "pip", "install", "-q", *pkgs], check=True)
+    link = os.path.expanduser("~/.claude/skills/footprint")
+    try:
+        os.makedirs(os.path.dirname(link), exist_ok=True)
+        if not os.path.exists(link):
+            os.symlink(os.path.join(config.ROOT, "skills", "footprint"), link)
+    except OSError:
+        pass  # Windows without symlink rights — skill is optional
+    print(f"ready. next: footprint trace")
+
+
+def _opencode():
+    cfg_path = os.path.join(config.OPENCODE, "opencode.json")
+    cfg = json.load(open(cfg_path)) if os.path.exists(cfg_path) else {"$schema": "https://opencode.ai/config.json"}
+    cfg.setdefault("provider", {})["footprint"] = {
+        "npm": "@ai-sdk/openai-compatible", "name": "footprint",
+        "options": {"baseURL": f"http://127.0.0.1:{config.PORT}/v1"},
+        "models": {"footprint": {"name": "footprint"}}}
+    os.makedirs(os.path.join(config.OPENCODE, "command"), exist_ok=True)
+    with open(cfg_path, "w") as f:
+        json.dump(cfg, f, indent=2)
+    with open(os.path.join(config.OPENCODE, "command", "footprint.md"), "w") as f:
+        f.write("---\ndescription: do task with footprint — local model trained on your Claude sessions\n"
+                "model: footprint/footprint\n---\n$ARGUMENTS\n")
+
+
+def install():
+    """Register the server with the OS so it runs itself. User never runs serve."""
+    if not os.path.exists(config.VENV_PY):
+        sys.exit("no .venv. run: footprint setup")
+    if config.IS_MAC:
+        os.makedirs(os.path.dirname(config.LAUNCHD), exist_ok=True)
+        with open(config.LAUNCHD, "wb") as f:
+            plistlib.dump({"Label": "com.footprint.serve",
+                           "ProgramArguments": [config.VENV_PY, config.ENTRY, "serve"],
+                           "RunAtLoad": True, "KeepAlive": True,
+                           "StandardOutPath": "/tmp/footprint-serve.log",
+                           "StandardErrorPath": "/tmp/footprint-serve.log"}, f)
+        subprocess.run(["launchctl", "unload", config.LAUNCHD], capture_output=True)
+        subprocess.run(["launchctl", "load", "-w", config.LAUNCHD], check=True)
+        how = "launchd (log: /tmp/footprint-serve.log)"
+    elif config.IS_WIN:
+        subprocess.run(["schtasks", "/Create", "/F", "/TN", "footprint-serve", "/SC", "ONLOGON",
+                        "/TR", f'"{config.VENV_PY}" "{config.ENTRY}" serve'], check=True)
+        subprocess.run(["schtasks", "/Run", "/TN", "footprint-serve"], check=True)
+        how = "Task Scheduler (task: footprint-serve)"
+    else:
+        os.makedirs(os.path.dirname(config.SYSTEMD), exist_ok=True)
+        with open(config.SYSTEMD, "w") as f:
+            f.write("[Unit]\nDescription=footprint local model server\n\n"
+                    f"[Service]\nExecStart={config.VENV_PY} {config.ENTRY} serve\nRestart=always\n\n"
+                    "[Install]\nWantedBy=default.target\n")
+        subprocess.run(["systemctl", "--user", "daemon-reload"], check=True)
+        subprocess.run(["systemctl", "--user", "enable", "--now", "footprint"], check=True)
+        how = "systemd --user (journalctl --user -u footprint)"
+    _opencode()
+    print(f"installed:\n"
+          f"  server: {how} keeps http://127.0.0.1:{config.PORT}/v1 running\n"
+          f"  opencode: /footprint <task> uses it; or pick model 'footprint' via /models\n"
+          f"  cursor/codex: base URL http://127.0.0.1:{config.PORT}/v1, any api key")

package/footprint_core/transcripts.py

@@ -0,0 +1,95 @@
+"""Parse Claude Code transcripts into training examples."""
+import glob, json, os, random, re, sys
+
+from . import config
+
+
+def _text(content, cap=None):
+    """Flatten a Claude message content (str or block list) to training text."""
+    if isinstance(content, str):
+        out = content
+    else:
+        parts = []
+        for b in content or []:
+            t = b.get("type")
+            if t == "text":
+                parts.append(b["text"])
+            elif t == "tool_use":
+                parts.append(f'<tool name="{b["name"]}">{json.dumps(b.get("input", {}))}</tool>')
+            elif t == "tool_result":
+                c = b.get("content")
+                c = "".join(x.get("text", "") for x in c) if isinstance(c, list) else str(c or "")
+                parts.append(f"<tool_result>{c[:config.RESULT_CAP]}</tool_result>")
+        out = "\n".join(p for p in parts if p.strip())
+    out = re.sub(r"<system-reminder>.*?</system-reminder>", "", out, flags=re.S)
+    out = re.sub(r"<local-command-caveat>.*?</local-command-caveat>", "", out, flags=re.S)
+    return (out[:cap] if cap else out).strip()
+
+
+def parse_session(path):
+    """One transcript file -> list of alternating {role, content} messages."""
+    msgs = []
+    for line in open(path, errors="replace"):
+        try:
+            rec = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        if rec.get("type") not in ("user", "assistant") or rec.get("isSidechain") or rec.get("isMeta"):
+            continue
+        m = rec.get("message") or {}
+        role = m.get("role")
+        txt = _text(m.get("content"))
+        if not role or not txt:
+            continue
+        if msgs and msgs[-1]["role"] == role:
+            msgs[-1]["content"] += "\n" + txt
+        else:
+            msgs.append({"role": role, "content": txt})
+    return msgs
+
+
+def collect(project=None):
+    since = float(open(config.TRACE_MARK).read()) if os.path.exists(config.TRACE_MARK) else 0.0
+    slug = "-" + os.path.abspath(project or os.getcwd()).strip("/").replace("/", "-").replace(".", "-")
+    dirs = [os.path.join(config.PROJECTS, slug)] if os.path.isdir(os.path.join(config.PROJECTS, slug)) \
+        else glob.glob(os.path.join(config.PROJECTS, "*"))
+    examples, seen, dupes = [], set(), 0
+    for d in dirs:
+        for f in glob.glob(os.path.join(d, "*.jsonl")):
+            if os.path.getmtime(f) < since:
+                continue
+            msgs = parse_session(f)
+            if len(msgs) < 2:
+                continue
+            # one example per assistant turn, with preceding context under budget
+            for i, m in enumerate(msgs):
+                if m["role"] != "assistant":
+                    continue
+                ctx, size = [], 0
+                for prev in reversed(msgs[:i]):
+                    size += len(prev["content"])
+                    if size > config.CTX_BUDGET:
+                        break
+                    ctx.insert(0, prev)
+                if not ctx or ctx[0]["role"] != "user":
+                    continue
+                ex = {"messages": [{"role": "system", "content": config.SYSTEM}, *ctx, m]}
+                sig = hash(json.dumps(ex["messages"], sort_keys=True))
+                if sig in seen:
+                    dupes += 1
+                    continue
+                seen.add(sig)
+                examples.append(ex)
+    if not examples:
+        hint = " (trace marker active — only sessions after `trace` count; delete ~/.claude/footprint-trace for all history)" if since else ""
+        sys.exit(f"no transcripts found under {config.PROJECTS}{hint}")
+    random.seed(0)
+    random.shuffle(examples)
+    n_valid = max(1, len(examples) // 10)
+    os.makedirs(config.DATA, exist_ok=True)
+    for name, chunk in [("valid", examples[:n_valid]), ("train", examples[n_valid:])]:
+        with open(os.path.join(config.DATA, f"{name}.jsonl"), "w") as fh:
+            for e in chunk:
+                fh.write(json.dumps(e) + "\n")
+    print(f"collected {len(examples)} examples ({len(examples)-n_valid} train / {n_valid} valid, "
+          f"{dupes} duplicates skipped) -> {config.DATA}")

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "footprint-trace",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Local SLM trained on your Claude Code sessions. When Claude quota runs out, OpenCode keeps working like Claude.",
   "bin": {
     "footprint": "bin/footprint.js"
@@ -11,6 +11,8 @@
   "files": [
     "bin",
     "footprint.py",
+    "footprint_core/*.py",
+    "footprint_core/backends/*.py",
     "skills"
   ],
   "license": "MIT"

package/skills/footprint/SKILL.md

@@ -6,8 +6,10 @@ description: Local SLM that learns from Claude Code transcripts ("footprint") an
 # Footprint
 
 Footprint collects this machine's Claude Code session transcripts, fine-tunes a
-small local model (MLX LoRA, Apple Silicon) on them, and can then execute tasks
-locally with a tiny bash-tool agent loop — no API needed.
+small local model on them (MLX LoRA on Apple Silicon, torch+PEFT elsewhere),
+and serves it as an OpenAI-compatible API for OpenCode/Cursor/Codex.
+Code layout: `footprint.py` (entry shim) -> `footprint_core/` (cli, config,
+transcripts, service, backends/{mlx,torch}_backend.py).
 
 Repo: `/Users/aman/Developer/footprint`. Python: `/Users/aman/Developer/footprint/.venv/bin/python`
 (if `.venv` missing, run `setup` first with system `python3`).