typesafeai-cli 0.3.1__py3-none-any.whl

typesafe_cli/data/typesafe-cli/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: typesafe-cli
+description: >
+  Drive Jev through the typesafe CLI. Use when you need a typed snap judgment
+  over local task context: route, yes/no, score, search a file, rerank hits,
+  extract a span, verify a claim, screen a message, or pick a skill.
+  Collect privacy-safe state yourself, then run typesafe find/rank/extract/
+  verify/screen/suggest-skill/decide or ask. Do not curl TypeSafe, do not write
+  throwaway SDK scripts, do not read TYPESAFE_* env vars or key files.
+  Triggers: typesafe CLI, Jev, noul, choice, score, find, rank, extract, verify,
+  screen, suggest-skill, decide, typed judgment, which skill, is this urgent.
+---
+
+# typesafe CLI
+
+Jev judges only the `state` and `questions` you send. It cannot see the repo, the diff, AGENTS.md, skills, MCP, memory, or prior tool results. A label like "the code change" or a file path with no body is not context. If the decision is about code, you must copy the relevant slices into `state`.
+
+You are the context adapter. The CLI is the pipe.
+
+## Scratch files
+
+Never write `state.json`, `questions.json`, or other TypeSafe payloads into the git working tree.
+
+Put them in a **private temp directory** scoped to this project:
+
+```bash
+ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
+WORKDIR="${TMPDIR:-/tmp}/codex/$(basename "$ROOT")"
+mkdir -p "$WORKDIR"
+```
+
+Then `$WORKDIR/state.json` and `$WORKDIR/questions.json`. For a one-line state, prefer `--state "…"` and skip files.
+
+Deleting those files after the call is good. Leaving them is fine. Do not commit them.
+
+## Secrets
+
+The CLI loads credentials. You never do.
+
+Run `typesafe auth status`. Read `data.has_key` only. If it is false, ask the human. Do not open `~/.config/typesafe/env`, `.env.local`, or print `TYPESAFE_*`.
+
+## Procedure
+
+### 1. Confirm the binary and a key
+
+```bash
+typesafe auth status
+```
+
+Done when `has_key` is true. If the command is missing, tell the human to install typesafeai-cli.
+
+### 2. Name the decision, then write the questions
+
+Do not start by dumping context. TypeSafe's contract is the questions: they name the fields Jev is allowed to look at. When a dedicated command exists (`find`, `rank`, `extract`, `verify`, `screen`, `suggest-skill`, `decide`), **its flags are the contract** — fill those instead of a free-form dump.
+
+One sentence: what you will **do** with the answer. If that is not a noul / choice / score, decide locally.
+
+Then write `$WORKDIR/questions.json`. Each question's `instructions` must point at named paths with backticks (`` `task` ``, `` `files[0].hunk` ``, `` `tools` ``). Those paths **are** the collection list. There is no catalog of agent jobs to encode in this skill.
+
+- `noul` — is this statement true?
+- `choice` — pick one; **≥2** options; include `other` / `none` / `abstain`
+- `score` — ordered rubric; **≥2** levels
+
+Batch independent questions. One judgment each. Meaning lives in `instructions`, not in the id.
+
+Done when every question names the fields it needs.
+
+### 3. Fill only those fields
+
+Read the repo, diff, tests, or tool list **here**. Put into `$WORKDIR/state.json` **only** the paths the questions reference.
+
+Completeness: a stranger could answer the questions from `state.json` alone.
+
+If a path is code (`` `files[0].hunk` ``), the value must be the slice (path + line range + body), not a filename and not "see the PR." Prefer under ~8k tokens of code. Do not send the tree.
+
+Redact secrets. If a required slice cannot be redacted, **do not call TypeSafe**.
+
+### 4. Call the CLI
+
+```bash
+typesafe ask --state-file "$WORKDIR/state.json" --questions-file "$WORKDIR/questions.json"
+```
+
+One-shot: `typesafe noul "…" --state "…"` / `choice` / `score`. Prefer `ask` for more than one question.
+
+Read stdout JSON: `data.model` should be a Jev id (`jev-1.13.0`). `data.answers` is the result. Do not parse prose; there is none.
+
+### 5. Apply the answer here
+
+Thresholds are yours, not Jev's.
+
+- Noul near 0.5 is **unsure**, not medium. Do not automate.
+- Choice/score `confidence` low → abstain or ask the human.
+- Then you pick the skill, run the tool, or stop. Jev does not choose the next action.
+
+## Combine calls (do not loop)
+
+Jev answers are independent. Accuracy comes from **the right sequence**, not from more chat. Local work first when the cookbook does; one `system_one` for every independent question about the same state.
+
+| Goal | Do this | Do not |
+| --- | --- | --- |
+| Many questions, one document | One `typesafe ask` with all of them | `noul` in a shell loop (pays for the document N times) |
+| Search a file | `typesafe find --file --query` (or `ask` with line ids as a Choice **and** an exists Noul in the same request) | Rank lines without asking whether an answer exists |
+| Many candidates | `typesafe rank` (or one batched `ask`) | One HTTP call per hit |
+| Pull a value out of text | Find candidates locally (regex/roster), then `typesafe extract` / Choice among those spans plus `none` | Ask Jev to generate the email/amount |
+| Check a claim | String-match the quote locally; if missing → fabricated. Else `typesafe verify` | Send only the claim with no source |
+| Gate a message | `typesafe screen` then your policy | Skip the gate and “be careful” |
+| Which skill to load | `typesafe suggest-skill` (cheap rank, then reread top 3). Treat the name as a hint | Load three skills because the names look similar |
+| 0.49 vs 0.51 | `typesafe decide` / a review band in code | Flip automation on a coin-flip noul |
+
+If a dedicated verb is the wrong fit, compose the same sequence with `ask`: batch questions, put candidates in `state`, run local checks before HTTP.
+
+Second request only when the first answer is required to fetch more evidence or to shrink options (skill shortlist, Choice over 255+ lines). Speculative extras belong in the **first** call.
+
+## Commands
+
+```bash
+typesafe auth status
+typesafe agent schema
+typesafe ask --state-file "$WORKDIR/state.json" --questions-file "$WORKDIR/questions.json"
+typesafe noul "…" --state "…"
+typesafe choice "…" --option a --option b --state "…"
+typesafe score "…" --level low --level high --state "…"
+typesafe find --file path --query "…"
+typesafe rank --query "…" --candidates-file items.json
+typesafe extract --file doc.txt --pattern email --question "…"
+typesafe verify --claim "…" --source-file rfc.txt
+typesafe screen --text-file msg.txt
+typesafe suggest-skill --task "…" --skills-dir ~/.agents/skills
+typesafe decide --answers-file "$WORKDIR/last.json"
+```
+
+Discover flags with `typesafe agent schema`. Do not invent request fields.
+
+## Official TypeSafe skill
+
+`typesafe-ai` is for **designing** TypeSafe into application code (SDK). This skill is for **running** Jev now through the CLI. Live evaluations in a coding session use this skill.

typesafe_cli/detect.py

@@ -0,0 +1,83 @@
+from __future__ import annotations
+
+import os
+import sys
+from contextvars import ContextVar
+from dataclasses import dataclass
+
+_force_agent: ContextVar[bool | None] = ContextVar("typesafe_force_agent", default=None)
+
+
+def set_force_agent(value: bool | None) -> None:
+    _force_agent.set(value)
+
+DETECTORS: tuple[tuple[str, tuple[str, ...]], ...] = (
+    ("claude-code", ("CLAUDECODE", "CLAUDE_CODE")),
+    ("cursor", ("CURSOR_AGENT",)),
+    ("codex", ("CODEX", "OPENAI_CODEX")),
+    ("opencode", ("OPENCODE",)),
+    ("aider", ("AIDER",)),
+    ("cline", ("CLINE",)),
+    ("windsurf", ("WINDSURF_AGENT",)),
+    ("github-copilot", ("GITHUB_COPILOT",)),
+    ("amazon-q", ("AMAZON_Q", "AWS_Q_DEVELOPER")),
+    ("gemini-code", ("GEMINI_CODE_ASSIST",)),
+    ("sourcegraph-cody", ("SRC_CODY",)),
+    ("grok", ("GROK", "GROK_CODE")),
+    ("generic-agent", ("AGENT",)),
+)
+
+ALL_ENV_VARS: tuple[str, ...] = tuple(var for _, vars_ in DETECTORS for var in vars_) + (
+    "FORCE_AGENT_MODE",
+)
+
+
+@dataclass(frozen=True)
+class AgentInfo:
+    name: str
+    detected: bool
+
+
+def _truthy(key: str) -> bool:
+    value = os.environ.get(key)
+    if value is None:
+        return False
+    return value.lower() in {"1", "true", "yes"}
+
+
+def detect_agent_info() -> AgentInfo:
+    for name, env_vars in DETECTORS:
+        if any(_truthy(var) for var in env_vars):
+            return AgentInfo(name=name, detected=True)
+    return AgentInfo(name="", detected=False)
+
+
+def is_agent_mode(*, argv: list[str] | None = None) -> bool:
+    forced = _force_agent.get()
+    if forced is not None:
+        return forced
+    args = sys.argv if argv is None else argv
+    if "--no-agent" in args:
+        return False
+    if "--agent" in args:
+        return True
+    if _truthy("FORCE_AGENT_MODE"):
+        return True
+    return detect_agent_info().detected
+
+
+def resolve_agent(*, target: str | None = None) -> str:
+    if target and target not in {"auto", ""}:
+        aliases = {
+            "claude": "claude-code",
+            "claude-code": "claude-code",
+            "cursor": "cursor",
+            "codex": "codex",
+            "opencode": "opencode",
+            "grok": "grok",
+            "gemini": "gemini-code",
+            "windsurf": "windsurf",
+        }
+        return aliases.get(target, target)
+    info = detect_agent_info()
+    return info.name or "generic-agent"

typesafe_cli/format.py

@@ -0,0 +1,22 @@
+from __future__ import annotations
+
+import json
+from typing import Any
+
+
+def success_envelope(*, data: dict[str, Any], command: str) -> dict[str, Any]:
+    return {"status": "success", "data": data, "metadata": {"command": command}}
+
+
+def error_envelope(*, code: str, message: str) -> dict[str, Any]:
+    return {"status": "error", "error": {"code": code, "message": message}}
+
+
+def dumps(obj: dict[str, Any]) -> str:
+    return json.dumps(obj, indent=2, default=_json_default) + "\n"
+
+
+def _json_default(value: Any) -> Any:
+    if isinstance(value, dict):
+        return {str(k): v for k, v in value.items()}
+    return str(value)

typesafe_cli/help.py

@@ -0,0 +1,36 @@
+from __future__ import annotations
+
+from typer.core import TyperCommand, TyperGroup
+
+from typesafe_cli.detect import is_agent_mode
+from typesafe_cli.format import dumps, success_envelope
+from typesafe_cli.schema import schema_for_help
+
+
+def _help_is_agent(ctx) -> bool:  # noqa: ANN001 — click Context
+    if ctx.params.get("no_agent"):
+        return False
+    if ctx.params.get("agent"):
+        return True
+    parent = ctx.parent
+    while parent is not None:
+        if parent.params.get("no_agent"):
+            return False
+        if parent.params.get("agent"):
+            return True
+        parent = parent.parent
+    return is_agent_mode()
+
+
+class AgentGroup(TyperGroup):
+    def get_help(self, ctx) -> str:  # noqa: ANN001
+        if _help_is_agent(ctx):
+            return dumps(success_envelope(data=schema_for_help(ctx), command="help")).rstrip("\n")
+        return super().get_help(ctx)
+
+
+class AgentCommand(TyperCommand):
+    def get_help(self, ctx) -> str:  # noqa: ANN001
+        if _help_is_agent(ctx):
+            return dumps(success_envelope(data=schema_for_help(ctx), command="help")).rstrip("\n")
+        return super().get_help(ctx)

typesafe_cli/io.py

@@ -0,0 +1,17 @@
+from __future__ import annotations
+
+import sys
+from typing import Any, NoReturn
+
+import typer
+
+from typesafe_cli.format import dumps, error_envelope, success_envelope
+
+
+def emit_success(*, data: dict[str, Any], command: str) -> None:
+    sys.stdout.write(dumps(success_envelope(data=data, command=command)))
+
+
+def fail(*, code: str, message: str, exit_code: int) -> NoReturn:
+    sys.stderr.write(dumps(error_envelope(code=code, message=message)))
+    raise typer.Exit(exit_code)

typesafe_cli/questions.py

@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+
+class QuestionError(ValueError):
+    pass
+
+
+ALLOWED_TYPES = {"noul", "choice", "score"}
+
+
+def load_state(raw: str | dict | list) -> str | dict | list:
+    return raw
+
+
+def parse_state_text(text: str) -> str | dict | list:
+    stripped = text.strip()
+    if not stripped:
+        raise QuestionError("state is empty")
+    if stripped[0] in "{[":
+        try:
+            return json.loads(stripped)
+        except json.JSONDecodeError:
+            return text
+    return text
+
+
+def load_state_file(path: Path) -> str | dict | list:
+    return parse_state_text(path.read_text(encoding="utf-8"))
+
+
+def load_questions_file(path: Path) -> tuple[dict[str, dict[str, Any]], str | dict | list | None]:
+    payload = json.loads(path.read_text(encoding="utf-8"))
+    if not isinstance(payload, dict):
+        raise QuestionError("questions file must be a JSON object")
+    embedded_state = payload.get("state")
+    return load_questions(payload), embedded_state
+
+
+def load_questions(data: dict[str, Any]) -> dict[str, dict[str, Any]]:
+    if "questions" in data:
+        raw = data["questions"]
+        if isinstance(raw, list):
+            mapping: dict[str, Any] = {}
+            for item in raw:
+                if not isinstance(item, dict) or "id" not in item:
+                    raise QuestionError("list questions must each have an id")
+                qid = item["id"]
+                if qid in mapping:
+                    raise QuestionError(f"duplicate question id: {qid}")
+                mapping[qid] = {k: v for k, v in item.items() if k != "id"}
+            raw = mapping
+        if not isinstance(raw, dict):
+            raise QuestionError("questions must be an object or a list")
+        source = raw
+    else:
+        source = {k: v for k, v in data.items() if k != "state"}
+
+    if not source:
+        raise QuestionError("no questions provided")
+
+    out: dict[str, dict[str, Any]] = {}
+    for qid, question in source.items():
+        if not isinstance(qid, str) or not qid:
+            raise QuestionError("question ids must be non-empty strings")
+        if not isinstance(question, dict):
+            raise QuestionError(f"question {qid} must be an object")
+        out[qid] = _validate_question(qid, question)
+    return out
+
+
+def _validate_question(qid: str, question: dict[str, Any]) -> dict[str, Any]:
+    qtype = question.get("type")
+    if qtype not in ALLOWED_TYPES:
+        raise QuestionError(f"{qid}: type must be noul, choice, or score")
+    instructions = question.get("instructions")
+    if instructions is None or instructions == "":
+        raise QuestionError(f"{qid}: instructions are required")
+
+    criteria = question.get("criteria")
+    if qtype == "choice":
+        if not isinstance(criteria, dict) or len(criteria) < 2:
+            raise QuestionError(f"{qid}: choice criteria must be a map with at least two options")
+    elif qtype == "score":
+        if not isinstance(criteria, list) or len(criteria) < 2:
+            raise QuestionError(f"{qid}: score criteria must be a list with at least two levels")
+    elif criteria is not None and not isinstance(criteria, dict):
+        raise QuestionError(f"{qid}: noul criteria must be an object if present")
+
+    validated: dict[str, Any] = {"type": qtype, "instructions": instructions}
+    if criteria is not None:
+        validated["criteria"] = criteria
+    return validated

typesafe_cli/recipes/__init__.py

@@ -0,0 +1,3 @@
+from typesafe_cli.recipes.thresholds import CHOICE_MAX_OPTIONS
+
+__all__ = ["CHOICE_MAX_OPTIONS"]

typesafe_cli/recipes/decide.py

@@ -0,0 +1,64 @@
+from __future__ import annotations
+
+from typing import Any
+
+from typesafe_cli.questions import QuestionError
+from typesafe_cli.recipes.thresholds import CHOICE_MIN_TOP_P, NOUL_UNCERTAIN_HIGH, NOUL_UNCERTAIN_LOW
+
+
+def parse_noul_band(raw: str) -> tuple[float, float]:
+    try:
+        low_s, high_s = raw.split(":", 1)
+        low, high = float(low_s), float(high_s)
+    except ValueError as exc:
+        raise QuestionError("noul band must be low:high, e.g. 0.30:0.70") from exc
+    if not 0.0 <= low <= high <= 1.0:
+        raise QuestionError("noul band must satisfy 0 <= low <= high <= 1")
+    return low, high
+
+
+def extract_answers(payload: Any) -> dict[str, Any]:
+    if not isinstance(payload, dict):
+        raise QuestionError("answers file must be a JSON object")
+    if "answers" in payload and isinstance(payload["answers"], dict):
+        return payload["answers"]
+    data = payload.get("data")
+    if isinstance(data, dict) and isinstance(data.get("answers"), dict):
+        return data["answers"]
+    if payload and all(isinstance(v, dict) and "type" in v for v in payload.values()):
+        return payload
+    raise QuestionError("could not find answers in file (expected data.answers or answers)")
+
+
+def apply_decide(
+    answers: dict[str, Any],
+    *,
+    noul_low: float = NOUL_UNCERTAIN_LOW,
+    noul_high: float = NOUL_UNCERTAIN_HIGH,
+    choice_min_p: float = CHOICE_MIN_TOP_P,
+) -> dict[str, Any]:
+    out: dict[str, Any] = {}
+    for qid, answer in answers.items():
+        if not isinstance(answer, dict):
+            raise QuestionError(f"{qid}: answer must be an object")
+        qtype = answer.get("type")
+        if qtype == "noul":
+            probability = float(answer["noul"])
+            if probability < noul_low:
+                decision = "no"
+            elif probability > noul_high:
+                decision = "yes"
+            else:
+                decision = "uncertain"
+            out[qid] = {**answer, "decision": decision, "noul": probability}
+        elif qtype == "choice":
+            probabilities = {str(k): float(v) for k, v in dict(answer.get("probabilities") or {}).items()}
+            top_p = max(probabilities.values()) if probabilities else 0.0
+            winner = answer.get("choice")
+            decision = winner if top_p >= choice_min_p else "uncertain"
+            out[qid] = {**answer, "decision": decision, "top_probability": top_p}
+        elif qtype == "score":
+            out[qid] = {**answer, "decision": answer.get("score")}
+        else:
+            raise QuestionError(f"{qid}: unsupported answer type {qtype!r}")
+    return out

typesafe_cli/recipes/extract.py

@@ -0,0 +1,42 @@
+from __future__ import annotations
+
+import re
+from typing import Any
+
+from typesafe_cli.questions import QuestionError
+
+PATTERN_NAMES = ("email", "phone", "money")
+
+_PATTERNS = {
+    "email": re.compile(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}"),
+    "phone": re.compile(r"\+?\d[\d.\-\s()]{7,}\d"),
+    "money": re.compile(r"\$\d{1,3}(?:,\d{3})*(?:\.\d{2})?|\b\d+\.\d{2}\b"),
+}
+
+
+def find_spans(text: str, pattern: str) -> list[str]:
+    if pattern not in _PATTERNS:
+        raise QuestionError(f"unknown pattern {pattern!r}; use {', '.join(PATTERN_NAMES)} or --candidates")
+    seen: list[str] = []
+    for match in _PATTERNS[pattern].findall(text):
+        span = match.strip()
+        if span and span not in seen:
+            seen.append(span)
+    return seen
+
+
+def extract_questions(spans: list[str], question: str) -> dict[str, Any]:
+    criteria: dict[str, str | None] = {span: None for span in spans}
+    criteria["none"] = "None of the candidate spans is the requested value"
+    if len(criteria) < 2:
+        raise QuestionError("extract needs at least one candidate span plus none")
+    return {
+        "value": {
+            "type": "choice",
+            "instructions": (
+                f"{question} Pick the verbatim candidate span. "
+                "If none of the candidates is correct, choose none."
+            ),
+            "criteria": criteria,
+        }
+    }

typesafe_cli/recipes/find.py

@@ -0,0 +1,90 @@
+from __future__ import annotations
+
+from collections.abc import Iterable, Iterator, Sequence
+from typing import Any
+
+from typesafe_cli.recipes.thresholds import FIND_CHUNK, FIND_EXISTS_ANSWERED, FIND_EXISTS_PARTIAL
+
+
+def tag_lines(text: str) -> list[dict[str, str]]:
+    raw_lines = text.splitlines()
+    width = max(3, len(str(max(len(raw_lines) - 1, 0))))
+    return [{"id": f"L{index:0{width}d}", "text": line} for index, line in enumerate(raw_lines)]
+
+
+def tagged_state(lines: Sequence[dict[str, str]]) -> str:
+    return "\n".join(f"{line['id']}| {line['text']}" for line in lines)
+
+
+def windows(items: Sequence[Any], *, size: int) -> Iterator[list[Any]]:
+    if size < 1:
+        raise ValueError("window size must be >= 1")
+    for start in range(0, len(items), size):
+        yield list(items[start : start + size])
+
+
+def find_questions(lines: Sequence[dict[str, str]], query: str) -> dict[str, Any]:
+    criteria: dict[str, str | None] = {line["id"]: None for line in lines}
+    if len(criteria) < 2:
+        criteria["none"] = "No line in this window answers the query"
+    return {
+        "line": {
+            "type": "choice",
+            "instructions": (
+                "Which tagged line best answers the query? "
+                f"Query: {query}. Prefer the most specific matching line id."
+            ),
+            "criteria": criteria,
+        },
+        "exists": {
+            "type": "noul",
+            "instructions": f"Does this document contain an answer to the query `{query}`?",
+        },
+    }
+
+
+def verdict_for_exists(exists: float) -> str:
+    if exists >= FIND_EXISTS_ANSWERED:
+        return "answered"
+    if exists >= FIND_EXISTS_PARTIAL:
+        return "partial"
+    return "absent"
+
+
+def merge_find_results(
+    *,
+    lines_by_id: dict[str, str],
+    chunks: Iterable[dict[str, Any]],
+) -> dict[str, Any]:
+    combined: dict[str, float] = {}
+    exists_values: list[float] = []
+    model = None
+    usage = {"input_tokens": 0, "output_tokens": 0}
+    for chunk in chunks:
+        model = chunk.get("model") or model
+        chunk_usage = chunk.get("usage") or {}
+        usage["input_tokens"] += int(chunk_usage.get("input_tokens") or 0)
+        usage["output_tokens"] += int(chunk_usage.get("output_tokens") or 0)
+        answers = chunk["answers"]
+        exists_values.append(float(answers["exists"]["noul"]))
+        probabilities = answers["line"].get("probabilities") or {}
+        for line_id, probability in probabilities.items():
+            if line_id == "none":
+                continue
+            combined[str(line_id)] = max(combined.get(str(line_id), 0.0), float(probability))
+    ranked = sorted(combined.items(), key=lambda item: item[1], reverse=True)
+    exists = max(exists_values) if exists_values else 0.0
+    return {
+        "model": model,
+        "query_lines": [
+            {"id": line_id, "text": lines_by_id.get(line_id, ""), "probability": probability}
+            for line_id, probability in ranked
+            if line_id in lines_by_id
+        ],
+        "exists": exists,
+        "verdict": verdict_for_exists(exists),
+        "usage": usage,
+    }
+
+
+FIND_CHUNK = FIND_CHUNK