pythinker-code 0.56.0__py3-none-any.whl → 0.58.0__py3-none-any.whl

pythinker_code/CHANGELOG.md

@@ -15,6 +15,79 @@ GitHub Releases page; `0.8.0` is the new starting line.
 
 ## Unreleased
 
+## 0.58.0 (2026-07-11)
+
+- **Agent-spec loading is more defensive and truthful.** Subagent `path`, `extend`, and
+  `system_prompt_path` references that resolve outside their spec's directory (or the built-in
+  agents directory) are now rejected instead of loaded, and the markdown agent catalogue no longer
+  reclassifies an unexpected parser error as a harmless "invalid field" skip — only genuinely
+  malformed frontmatter is skipped.
+- **Thinking and subagent activity now render cleanly in the terminal.** Live reasoning previews
+  render complete Markdown without exposing top-level HTML comments, activity-tree rows remain
+  visually stable, and the coral shimmer is reserved for the active verb spinner.
+- **Agent request compatibility is now executable and reviewable.** Provider handoff,
+  prompt ordering, persisted-versus-effective history, context JSONL restoration,
+  agent projections, and Toolset lifecycle behavior now have explicit compatibility
+  contracts guarding future agent-core changes.
+- **Skill discovery is bounded without making skills unreachable.** Pythinker now
+  searches one deterministic `SkillCatalog`, keeps exhaustive exact-name resolution,
+  and sends only task-relevant candidates to the model within an 8,000-character
+  request budget. The exhaustive `Runtime.skills` mapping remains available during
+  the compatibility window.
+- **Agent requests now have one observable assembly path.** Required guidance fails
+  closed, optional guidance reports sanitized degradation outcomes, and the new
+  `/prompt-manifest` command explains the latest request composition without storing
+  raw prompts, user text, or provenance paths.
+- **Conversation history updates are transactional.** Normal appends persist before
+  changing memory, while compaction, pruning, revert, and clear flows use atomic
+  replacement with coherent cancellation and rollback behavior. Concurrent revert
+  conflicts now stop after a bounded retry budget instead of starving indefinitely.
+  Existing JSONL records and restoration behavior remain compatible.
+- **Agent definitions now resolve through one source-aware catalogue.** YAML and
+  Markdown definitions share deterministic precedence, collision diagnostics, and
+  safe provenance handling. Unknown fields warn in this release, become errors in
+  the following minor release, and the `LaborMarket`, `AgentTypeDefinition`, and
+  generated-wrapper adapters remain through that strict-default release.
+- **Tool execution and MCP lifecycle behavior now have deterministic fault coverage.**
+  Publication rebuilds preserve the previous MCP tool registry if registration
+  fails. Characterization crossed the execution-overhead threshold, but a controlled
+  private extraction measured slightly worse and was reverted, so
+  `PythinkerToolset` remains the implementation boundary.
+- **Agent-core seams hardened from review.** Persisted usage/checkpoint records reject
+  boolean and negative token counts, `update_token_count` validates at the boundary, a
+  temporary system-prompt descriptor is closed if `fdopen` fails, request finalization
+  surfaces every provider acknowledgement failure, a failed skill projection is always
+  recorded as failed (never blurred to not-applicable), and request-assembly telemetry no
+  longer emits unbounded per-request token values as metric attributes.
+
+Upgrade with `pythinker update`, `pip install --upgrade pythinker-code==0.58.0`, or use the native installer for your platform from the [Releases page](https://github.com/Pythoughts-labs/pythinker-code/releases/latest).
+
+## 0.57.0 (2026-07-05)
+
+- **No more ghost/duplicate input prompt while the agent works.** After
+  submitting a prompt, the input card no longer fossilizes above the stream as
+  a second, ghostly prompt. The prompt bar and top border now stay visible
+  through turn-start and lazy-load frames, and running-agent output renders
+  through a stable diff-rendered scene (ported from Pi) so the input card no
+  longer jumps. An experimental Focus TUI mode (`tui.focus_mode`) pins the
+  composer and hides file activity behind a compact live shelf for a cleaner
+  fullscreen view, and active subagent tool work now surfaces in the pinned
+  status tail instead of a generic composing spinner.
+- **Added a native `/benchmark` command suite.** Run deterministic local
+  Pythinker model evaluations with bundled smoke tasks and SWE-style JSONL
+  tasks (`/benchmark:swe`), plus `/benchmark compare`, `/benchmark export`,
+  and `/benchmark discover` actions that produce replayable artifacts,
+  activity metrics, and branded markdown reports.
+- **Hardened benchmark and skill security edges.** SWE verification commands
+  are shape-validated before execution, benchmark discovery requires explicit
+  network opt-in, run IDs avoid clock collisions, runtime overrides restore
+  after setup failures, and active-skill deactivation failures now surface
+  instead of being swallowed. Explicitly invoked skills stay active across
+  later turns through a compact reminder until stopped by name or
+  "normal mode".
+
+Upgrade with `pythinker update`, `pip install --upgrade pythinker-code==0.57.0`, or use the native installer for your platform from the [Releases page](https://github.com/Pythoughts-labs/pythinker-code/releases/latest).
+
 ## 0.56.0 (2026-07-02)
 
 - **Windows shell UI recovers from mid-session console blanking.** The TUI's

pythinker_code/agents/default/system.md

@@ -127,7 +127,7 @@ Default to `ImplementAndJudge` for non-trivial scoped edits (see §4.4); reserve
 
 **Background shell** (root agent only). Launch long-running commands via `Shell` with `run_in_background=true` and a short `description`; the system notifies you at terminal states. `TaskList` re-enumerates active tasks (especially after context compaction); `TaskOutput` gives non-blocking snapshots (`block=true` only to intentionally wait); `TaskStop` cancels. After starting a background task, default to returning control to the user. The only task-management slash command for users is `/task` — never invent subcommands like `/task list` or `/tasks`. Subagents and sessions without these tools must not assume background-task control.
 
-**Skills (`ReadSkill`).** Load a skill's exact instructions before applying its workflow — mandatory for `review-pr`, `diagnose-ci-failures`, `fix-errors`, `implement-specs`, `spec-driven-implementation`, `check-impl-against-spec`, `resolve-merge-conflicts`, and `create-pr`. Read skill details only when needed, to conserve context. Catalog and scope precedence in §12.
+**Skills (`ReadSkill`).** Load a skill's exact instructions before applying its workflow — mandatory for `review-pr`, `diagnose-ci-failures`, `fix-errors`, `implement-specs`, `spec-driven-implementation`, `check-impl-against-spec`, `resolve-merge-conflicts`, and `create-pr`. If the user explicitly invokes or names a skill as an active mode, keep applying that loaded skill until they ask to stop it or return to normal mode. Read skill details only when needed, to conserve context. Catalog and scope precedence in §12.
 
 **Inline `/command` references.** Slash commands execute only as their own message starting with `/`. A `/command` or `/skill:<name>` mentioned mid-message did not auto-run, but it still expresses intent — act on it rather than leading your reply by reporting it as failed. For `/plan`, call `EnterPlanMode` (a real tool in your toolset, not just prose); for `/goal`, pursue the described objective until it is verifiably done; for `/skill:<name>`, load it via `ReadSkill` and apply it; for other guidance commands, apply the equivalent guidance yourself. Mention invoking the real command only when genuinely needed. Never silently drop such a reference.
 
@@ -255,8 +255,8 @@ Precedence per §2. `README`/`README.md` files are optional supplementary contex
 
 ## 12. Skills
 
-Skills are reusable, self-contained capability directories, each with a `SKILL.md` of instructions, examples, scripts, and reference material — specialized domain knowledge, workflow patterns, pre-configured tool chains, and templates. They are grouped by scope (`Project`, `User`, `Extra`, `Built-in`); when scopes define the same name, the more specific wins: **Project › User › Extra › Built-in.**
+Skills are reusable, self-contained capability directories, each with a `SKILL.md` of instructions, examples, scripts, and reference material — specialized domain knowledge, workflow patterns, pre-configured tool chains, and templates. When scopes define the same name, the more specific wins: **Project › User › Extra › Built-in.**
 
 ${PYTHINKER_SKILLS}
 
-Identify the skills relevant to the current task and read their `SKILL.md` before applying the workflow (§5). If a skill `<name>` has a companion `<name>-local`, treat it as local project specialization applied after the core skill. Read skill details only when needed, to conserve the context window.
+Identify the skills relevant to the current task and read their `SKILL.md` before applying the workflow (§5). If a skill `<name>` has a companion `<name>-local`, treat it as local project specialization applied after the core skill. Read skill details only when needed, to conserve the context window.

pythinker_code/agentspec.py

@@ -1,11 +1,13 @@
 from __future__ import annotations
 
+import hashlib
+import re
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, Literal, NamedTuple, cast
 
 import yaml
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, ValidationError
 
 from pythinker_code.exception import AgentSpecError
 
@@ -89,6 +91,14 @@ class ResolvedAgentSpec:
     subagents: dict[str, SubagentSpec]
 
 
+@dataclass(frozen=True, slots=True, kw_only=True)
+class AgentSpecSourceValidation:
+    """Unknown fields observed in one canonical YAML source before projection."""
+
+    source_path: Path
+    field_paths: tuple[str, ...]
+
+
 def load_agent_spec(agent_file: Path) -> ResolvedAgentSpec:
     """
     Load agent specification from file.
@@ -97,7 +107,22 @@ def load_agent_spec(agent_file: Path) -> ResolvedAgentSpec:
         FileNotFoundError: If the agent spec file is not found.
         AgentSpecError: If the agent spec is not valid.
     """
-    agent_spec = _load_agent_spec(agent_file)
+    agent_spec, _ = load_agent_spec_validated(agent_file)
+    return agent_spec
+
+
+def load_agent_spec_validated(
+    agent_file: Path,
+    *,
+    forbid_unknown_fields: bool = False,
+) -> tuple[ResolvedAgentSpec, tuple[AgentSpecSourceValidation, ...]]:
+    """Load a spec and report raw unknown fields from every inherited source."""
+    validations: dict[Path, AgentSpecSourceValidation] = {}
+    agent_spec = _load_agent_spec(
+        agent_file,
+        _validations=validations,
+        _forbid_unknown_fields=forbid_unknown_fields,
+    )
     assert agent_spec.extend is None, "agent extension should be recursively resolved"
     if isinstance(agent_spec.name, Inherit):
         raise AgentSpecError("Agent name is required")
@@ -111,7 +136,7 @@ def load_agent_spec(agent_file: Path) -> ResolvedAgentSpec:
         agent_spec.exclude_tools = []
     if isinstance(agent_spec.subagents, Inherit):
         agent_spec.subagents = {}
-    return ResolvedAgentSpec(
+    resolved = ResolvedAgentSpec(
         name=agent_spec.name,
         system_prompt_path=agent_spec.system_prompt_path,
         system_prompt_args=agent_spec.system_prompt_args,
@@ -127,9 +152,36 @@ def load_agent_spec(agent_file: Path) -> ResolvedAgentSpec:
         exclude_tools=agent_spec.exclude_tools or [],
         subagents=agent_spec.subagents or {},
     )
+    return resolved, tuple(validations.values())
+
+
+def _resolve_within_agent_roots(agent_file: Path, declared: str | Path) -> Path:
+    """Join *declared* onto *agent_file*'s directory, rejecting escaping paths.
+
+    Trusted local agent specs reference sibling files with relative paths. A
+    ``..`` traversal or symlink whose canonical target lands outside both the
+    originating spec's directory and the built-in agents directory is rejected
+    fail-closed as defense-in-depth, since the joined path is otherwise opened
+    or recursively loaded directly. The stored value keeps its ``.absolute()``
+    form, so permitted paths are unchanged.
+    """
+    parent = agent_file.parent
+    absolute = (parent / declared).absolute()
+    allowed_roots = (parent.resolve(), get_agents_dir().resolve())
+    if not any(absolute.resolve().is_relative_to(root) for root in allowed_roots):
+        raise AgentSpecError(
+            f"Agent spec reference {declared!r} resolves outside the permitted agent directories"
+        )
+    return absolute
 
 
-def _load_agent_spec(agent_file: Path, _visited: set[Path] | None = None) -> AgentSpec:
+def _load_agent_spec(
+    agent_file: Path,
+    _visited: set[Path] | None = None,
+    *,
+    _validations: dict[Path, AgentSpecSourceValidation] | None = None,
+    _forbid_unknown_fields: bool = False,
+) -> AgentSpec:
     resolved = agent_file.resolve()
     if _visited is None:
         _visited = set()
@@ -149,24 +201,46 @@ def _load_agent_spec(agent_file: Path, _visited: set[Path] | None = None) -> Age
         raise AgentSpecError(f"Agent spec file must contain a mapping: {agent_file}")
     data = cast("dict[str, Any]", data)
 
+    unknown_fields, has_invalid_field_key = _unknown_agent_spec_fields(data)
+    if unknown_fields:
+        if has_invalid_field_key:
+            fields = ", ".join(unknown_fields)
+            raise AgentSpecError(f"Invalid agent field key: {fields}")
+        if _forbid_unknown_fields:
+            fields = ", ".join(unknown_fields)
+            raise AgentSpecError(f"Unknown fields in required agent source: {fields}")
+        if _validations is not None and resolved not in _validations:
+            _validations[resolved] = AgentSpecSourceValidation(
+                source_path=resolved,
+                field_paths=unknown_fields,
+            )
+
     version = str(data.get("version", DEFAULT_AGENT_SPEC_VERSION))
     if version not in SUPPORTED_AGENT_SPEC_VERSIONS:
         raise AgentSpecError(f"Unsupported agent spec version: {version}")
 
-    agent_spec = AgentSpec(**data.get("agent", {}))
+    try:
+        agent_spec = AgentSpec(**data.get("agent", {}))
+    except (TypeError, ValidationError) as exc:
+        raise AgentSpecError("Agent spec contains an invalid known field") from exc
     if isinstance(agent_spec.system_prompt_path, Path):
-        agent_spec.system_prompt_path = (
-            agent_file.parent / agent_spec.system_prompt_path
-        ).absolute()
+        agent_spec.system_prompt_path = _resolve_within_agent_roots(
+            agent_file, agent_spec.system_prompt_path
+        )
     if isinstance(agent_spec.subagents, dict):
         for v in agent_spec.subagents.values():
-            v.path = (agent_file.parent / v.path).absolute()
+            v.path = _resolve_within_agent_roots(agent_file, v.path)
     if agent_spec.extend:
         if agent_spec.extend == "default":
             base_agent_file = DEFAULT_AGENT_FILE
         else:
-            base_agent_file = (agent_file.parent / agent_spec.extend).absolute()
-        base_agent_spec = _load_agent_spec(base_agent_file, _visited)
+            base_agent_file = _resolve_within_agent_roots(agent_file, agent_spec.extend)
+        base_agent_spec = _load_agent_spec(
+            base_agent_file,
+            _visited,
+            _validations=_validations,
+            _forbid_unknown_fields=_forbid_unknown_fields,
+        )
         if not isinstance(agent_spec.name, Inherit):
             base_agent_spec.name = agent_spec.name
         if not isinstance(agent_spec.system_prompt_path, Inherit):
@@ -207,3 +281,89 @@ def _load_agent_spec(agent_file: Path, _visited: set[Path] | None = None) -> Age
                 base_agent_spec.subagents = agent_spec.subagents
         agent_spec = base_agent_spec
     return agent_spec
+
+
+def _unknown_agent_spec_fields(data: dict[str, Any]) -> tuple[tuple[str, ...], bool]:
+    unknown: list[str] = []
+    has_invalid_key = False
+    for key in cast("dict[object, object]", data):
+        if isinstance(key, str) and key in {"version", "agent"}:
+            continue
+        rendered = render_agent_field_segment(key)
+        unknown.append(rendered.text)
+        has_invalid_key = has_invalid_key or rendered.structurally_invalid
+    raw_agent = data.get("agent")
+    if not isinstance(raw_agent, dict):
+        return tuple(sorted(unknown)), has_invalid_key
+    agent = cast("dict[str, Any]", raw_agent)
+    known_agent_fields = set(AgentSpec.model_fields)
+    for key in cast("dict[object, object]", agent):
+        if isinstance(key, str) and key in known_agent_fields:
+            continue
+        rendered = render_agent_field_segment(key)
+        unknown.append(f"agent.{rendered.text}")
+        has_invalid_key = has_invalid_key or rendered.structurally_invalid
+    raw_subagents = agent.get("subagents")
+    if isinstance(raw_subagents, dict):
+        known_subagent_fields = set(SubagentSpec.model_fields)
+        for name, raw_subagent in cast("dict[object, object]", raw_subagents).items():
+            rendered_name = render_agent_field_segment(name)
+            has_invalid_key = has_invalid_key or rendered_name.structurally_invalid
+            if rendered_name.structurally_invalid:
+                unknown.append(f"agent.subagents.{rendered_name.text}")
+            if not isinstance(raw_subagent, dict):
+                continue
+            for key in cast("dict[object, object]", raw_subagent):
+                if isinstance(key, str) and key in known_subagent_fields:
+                    continue
+                rendered = render_agent_field_segment(key)
+                unknown.append(f"agent.subagents.{rendered_name.text}.{rendered.text}")
+                has_invalid_key = has_invalid_key or rendered.structurally_invalid
+    return tuple(sorted(unknown)), has_invalid_key
+
+
+_FIELD_IDENTIFIER_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_-]*")
+_SENSITIVE_FIELD_HINTS = (
+    "password",
+    "passwd",
+    "secret",
+    "token",
+    "api_key",
+    "apikey",
+    "access_key",
+    "private_key",
+    "credential",
+    "auth",
+)
+
+
+@dataclass(frozen=True, slots=True, kw_only=True)
+class AgentFieldSegment:
+    text: str
+    redacted_for_safety: bool
+    structurally_invalid: bool
+
+
+def render_agent_field_segment(value: object) -> AgentFieldSegment:
+    """Render a stable field segment without conflating redaction and validity."""
+    if isinstance(value, str):
+        lowered = value.casefold()
+        if _FIELD_IDENTIFIER_RE.fullmatch(value) and not any(
+            hint in lowered for hint in _SENSITIVE_FIELD_HINTS
+        ):
+            return AgentFieldSegment(
+                text=value,
+                redacted_for_safety=False,
+                structurally_invalid=False,
+            )
+        digest_input = f"str:{value}"
+        structurally_invalid = _FIELD_IDENTIFIER_RE.fullmatch(value) is None
+    else:
+        digest_input = f"{type(value).__qualname__}:{value!r}"
+        structurally_invalid = True
+    digest = hashlib.sha256(digest_input.encode(encoding="utf-8")).hexdigest()[:12]
+    return AgentFieldSegment(
+        text=f"field[{digest}]",
+        redacted_for_safety=True,
+        structurally_invalid=structurally_invalid,
+    )

pythinker_code/benchmark/__init__.py

@@ -0,0 +1,5 @@
+"""Native Pythinker Benchmark support."""
+
+from pythinker_code.benchmark.commands import benchmark_usage
+
+__all__ = ["benchmark_usage"]

pythinker_code/benchmark/activity.py

@@ -0,0 +1,88 @@
+from __future__ import annotations
+
+import difflib
+import json
+from collections.abc import Mapping
+from pathlib import Path
+from typing import cast
+
+
+def summarize_benchmark_activity(
+    before: Mapping[str, str],
+    after: Mapping[str, str],
+    wire_file: Path,
+    wire_offset: int,
+) -> dict[str, object]:
+    added, removed = _changed_line_counts(before, after)
+    tool_calls_by_name = _tool_calls_by_name(wire_file, wire_offset)
+    shell_tool_calls = sum(
+        count for name, count in tool_calls_by_name.items() if name.casefold() in {"bash", "shell"}
+    )
+    return {
+        "changed_files_count": len(_changed_file_names(before, after)),
+        "added_lines": added,
+        "removed_lines": removed,
+        "tool_calls_by_name": dict(sorted(tool_calls_by_name.items())),
+        "shell_tool_calls": shell_tool_calls,
+    }
+
+
+def _changed_file_names(before: Mapping[str, str], after: Mapping[str, str]) -> set[str]:
+    names = set(before) | set(after)
+    return {name for name in names if before.get(name) != after.get(name)}
+
+
+def _changed_line_counts(before: Mapping[str, str], after: Mapping[str, str]) -> tuple[int, int]:
+    added = 0
+    removed = 0
+    for name in sorted(_changed_file_names(before, after)):
+        diff = difflib.ndiff(
+            before.get(name, "").splitlines(),
+            after.get(name, "").splitlines(),
+        )
+        for line in diff:
+            if line.startswith("+ "):
+                added += 1
+            elif line.startswith("- "):
+                removed += 1
+    return added, removed
+
+
+def _tool_calls_by_name(wire_file: Path, offset: int) -> dict[str, int]:
+    counts: dict[str, int] = {}
+    if not wire_file.exists():
+        return counts
+    try:
+        with wire_file.open("r", encoding="utf-8", errors="replace") as f:
+            f.seek(offset)
+            for line in f:
+                try:
+                    raw: object = json.loads(line)
+                except json.JSONDecodeError:
+                    continue
+                if not isinstance(raw, dict):
+                    continue
+                message = cast(dict[str, object], raw).get("message")
+                if not isinstance(message, dict):
+                    continue
+                message_data = cast(dict[str, object], message)
+                if message_data.get("type") != "ToolCall":
+                    continue
+                payload = message_data.get("payload")
+                if not isinstance(payload, dict):
+                    continue
+                name = _tool_name(cast(dict[str, object], payload))
+                if isinstance(name, str) and name:
+                    counts[name] = counts.get(name, 0) + 1
+    except OSError:
+        return {}
+    return counts
+
+
+def _tool_name(payload: Mapping[str, object]) -> str | None:
+    function = payload.get("function")
+    if isinstance(function, dict):
+        name = cast(dict[str, object], function).get("name")
+        return name if isinstance(name, str) else None
+    legacy_name = payload.get("name")
+    return legacy_name if isinstance(legacy_name, str) else None

pythinker_code/benchmark/bundled/__init__.py

@@ -0,0 +1 @@
+"""Bundled benchmark definitions."""

pythinker_code/benchmark/bundled/suites/__init__.py

@@ -0,0 +1 @@
+"""Bundled benchmark suites."""

pythinker_code/benchmark/bundled/suites/pythinker-core.json

@@ -0,0 +1,11 @@
+{
+  "name": "pythinker-core",
+  "title": "Pythinker Core Suite",
+  "description": "Deterministic local coding tasks covering common agent benchmark failure modes.",
+  "tasks": [
+    "core-atomic-transfer",
+    "core-dedup-order",
+    "core-explicit-none-metadata",
+    "core-safe-path-join"
+  ]
+}

pythinker_code/benchmark/bundled/suites/pythinker-smoke.json

@@ -0,0 +1,10 @@
+{
+  "name": "pythinker-smoke",
+  "title": "Pythinker Smoke Suite",
+  "description": "Small deterministic tasks for validating the native benchmark runner.",
+  "tasks": [
+    "smoke-edit-readme",
+    "smoke-fix-python-test",
+    "smoke-add-small-function"
+  ]
+}

pythinker_code/benchmark/bundled/tasks/__init__.py

@@ -0,0 +1 @@
+"""Bundled benchmark tasks."""

pythinker_code/benchmark/bundled/tasks/core-atomic-transfer.json

@@ -0,0 +1,21 @@
+{
+  "id": "core-atomic-transfer",
+  "title": "Atomic Ledger Transfer",
+  "description": "Fix a money-transfer bug without leaving partial state behind.",
+  "prompt": "Fix ledger.py. Ledger.transfer(src, dst, amount) must move funds atomically: reject missing source accounts, missing destination accounts, non-positive amounts, and insufficient funds without changing any balance. Keep the public API minimal.",
+  "workspace": {
+    "files": {
+      "ledger.py": "class Ledger:\n    def __init__(self, balances):\n        self.balances = dict(balances)\n\n    def transfer(self, src, dst, amount):\n        self.balances[src] -= amount\n        if self.balances[src] < 0:\n            raise ValueError('insufficient funds')\n        self.balances[dst] += amount\n",
+      "test_ledger.py": "import pytest\n\nfrom ledger import Ledger\n\n\ndef test_transfer_moves_funds():\n    ledger = Ledger({'a': 10, 'b': 1})\n    ledger.transfer('a', 'b', 4)\n    assert ledger.balances == {'a': 6, 'b': 5}\n\n\ndef test_insufficient_funds_rolls_back():\n    ledger = Ledger({'a': 3, 'b': 1})\n    with pytest.raises(ValueError):\n        ledger.transfer('a', 'b', 5)\n    assert ledger.balances == {'a': 3, 'b': 1}\n\n\ndef test_missing_destination_does_not_create_accounts():\n    ledger = Ledger({'a': 3})\n    with pytest.raises((KeyError, ValueError)):\n        ledger.transfer('a', 'missing', 1)\n    assert ledger.balances == {'a': 3}\n\n\ndef test_missing_source_rolls_back():\n    ledger = Ledger({'a': 3, 'b': 1})\n    with pytest.raises((KeyError, ValueError)):\n        ledger.transfer('missing', 'b', 1)\n    assert ledger.balances == {'a': 3, 'b': 1}\n\n\n@pytest.mark.parametrize('amount', [0, -1])\ndef test_non_positive_amount_rejected(amount):\n    ledger = Ledger({'a': 3, 'b': 1})\n    with pytest.raises(ValueError):\n        ledger.transfer('a', 'b', amount)\n    assert ledger.balances == {'a': 3, 'b': 1}\n\n\ndef test_non_positive_amount_checked_before_accounts():\n    ledger = Ledger({'a': 3, 'b': 1})\n    with pytest.raises(ValueError):\n        ledger.transfer('missing', 'b', -1)\n    assert ledger.balances == {'a': 3, 'b': 1}\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_ledger.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 180,
+    "max_steps": 60
+  },
+  "tags": ["core", "python", "state", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/core-dedup-order.json

@@ -0,0 +1,21 @@
+{
+  "id": "core-dedup-order",
+  "title": "Stable Deduplication",
+  "description": "Preserve first-seen order while removing duplicates.",
+  "prompt": "Fix seqkit.py. unique(items) must return unique values from any iterable in first-seen order. It must support unhashable values such as dictionaries and must not mutate the input.",
+  "workspace": {
+    "files": {
+      "seqkit.py": "def unique(items):\n    return sorted(set(items))\n",
+      "test_seqkit.py": "from seqkit import unique\n\n\ndef test_unique_preserves_first_seen_order():\n    assert unique(['b', 'a', 'b', 'c', 'a']) == ['b', 'a', 'c']\n\n\ndef test_unique_supports_unhashable_dicts():\n    first = {'id': 1, 'name': 'first'}\n    duplicate = {'id': 1, 'name': 'first'}\n    second = {'id': 2, 'name': 'second'}\n    assert unique([first, duplicate, second, first]) == [first, second]\n\n\ndef test_unique_does_not_mutate_input():\n    values = ['b', 'a', 'b']\n    assert unique(values) == ['b', 'a']\n    assert values == ['b', 'a', 'b']\n\n\ndef test_unique_accepts_generators():\n    assert unique(x for x in ['a', 'a', 'b']) == ['a', 'b']\n\n\ndef test_unique_keeps_falsey_values_distinct():\n    assert unique([0, False, '', None, 0, None]) == [0, '', None]\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_seqkit.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 180,
+    "max_steps": 60
+  },
+  "tags": ["core", "python", "algorithm", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/core-explicit-none-metadata.json

@@ -0,0 +1,21 @@
+{
+  "id": "core-explicit-none-metadata",
+  "title": "Explicit None Metadata",
+  "description": "Distinguish omitted metadata from explicit None.",
+  "prompt": "Fix metadata.py. build_driver_info must distinguish omitted name/version from explicitly passing None: omitted fields use defaults, explicit None suppresses that field, and if both are explicitly None the result is empty. Other explicit falsey values, such as empty string, False, or 0, must be preserved.",
+  "workspace": {
+    "files": {
+      "metadata.py": "DEFAULT_NAME = 'pythinker-client'\nDEFAULT_VERSION = '1.0'\n\n\ndef build_driver_info(name=None, version=None):\n    return {\n        'name': name or DEFAULT_NAME,\n        'version': version or DEFAULT_VERSION,\n    }\n",
+      "test_metadata.py": "from metadata import build_driver_info\n\n\ndef test_omitted_fields_use_defaults():\n    assert build_driver_info() == {'name': 'pythinker-client', 'version': '1.0'}\n\n\ndef test_explicit_none_suppresses_one_field():\n    assert build_driver_info(name=None) == {'version': '1.0'}\n    assert build_driver_info(version=None) == {'name': 'pythinker-client'}\n\n\ndef test_explicit_none_for_both_fields_returns_empty_metadata():\n    assert build_driver_info(name=None, version=None) == {}\n\n\ndef test_explicit_values_are_used():\n    assert build_driver_info(name='wrapper', version='2.5') == {'name': 'wrapper', 'version': '2.5'}\n\n\ndef test_falsey_explicit_values_are_preserved():\n    assert build_driver_info(name='', version='0') == {'name': '', 'version': '0'}\n    assert build_driver_info(name=False, version=0) == {'name': False, 'version': 0}\n    assert build_driver_info(name=None, version='') == {'version': ''}\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_metadata.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 180,
+    "max_steps": 60
+  },
+  "tags": ["core", "python", "api-semantics", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/core-safe-path-join.json

@@ -0,0 +1,21 @@
+{
+  "id": "core-safe-path-join",
+  "title": "Safe Path Join",
+  "description": "Reject path traversal while keeping normal relative paths usable.",
+  "prompt": "Fix paths.py. safe_join(root, user_path) must return a Path inside root for normal relative paths, and reject absolute paths, sibling-prefix paths, parent traversal, or symlink traversal outside root with ValueError.",
+  "workspace": {
+    "files": {
+      "paths.py": "from pathlib import Path\n\n\ndef safe_join(root, user_path):\n    return Path(root) / user_path\n",
+      "test_paths.py": "from pathlib import Path\n\nimport pytest\n\nfrom paths import safe_join\n\n\ndef test_safe_join_allows_nested_relative_path(tmp_path):\n    root = tmp_path / 'root'\n    root.mkdir()\n    assert safe_join(root, 'logs/app.txt') == root.resolve() / 'logs/app.txt'\n\n\ndef test_safe_join_allows_root_relative_path(tmp_path):\n    root = tmp_path / 'root'\n    root.mkdir()\n    assert safe_join(root, '.') == root.resolve()\n\n\ndef test_safe_join_rejects_parent_traversal(tmp_path):\n    root = tmp_path / 'root'\n    root.mkdir()\n    with pytest.raises(ValueError):\n        safe_join(root, '../secret.txt')\n\n\ndef test_safe_join_rejects_absolute_path(tmp_path):\n    root = tmp_path / 'root'\n    root.mkdir()\n    with pytest.raises(ValueError):\n        safe_join(root, Path('/tmp/secret.txt'))\n\n\ndef test_safe_join_rejects_sibling_prefix_absolute_path(tmp_path):\n    root = tmp_path / 'root'\n    sibling = tmp_path / 'root-sibling'\n    root.mkdir()\n    sibling.mkdir()\n    with pytest.raises(ValueError):\n        safe_join(root, sibling / 'secret.txt')\n\n\ndef test_safe_join_rejects_symlink_escape(tmp_path):\n    root = tmp_path / 'root'\n    outside = tmp_path / 'outside'\n    root.mkdir()\n    outside.mkdir()\n    link = root / 'link'\n    try:\n        link.symlink_to(outside, target_is_directory=True)\n    except OSError:\n        pytest.skip('symlinks unavailable')\n    with pytest.raises(ValueError):\n        safe_join(root, 'link/secret.txt')\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_paths.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 180,
+    "max_steps": 60
+  },
+  "tags": ["core", "python", "security", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/smoke-add-small-function.json

@@ -0,0 +1,21 @@
+{
+  "id": "smoke-add-small-function",
+  "title": "Add Small Function",
+  "description": "Add a tiny function required by a deterministic check.",
+  "prompt": "Add a slugify(text) function to strings.py. It should lowercase text, strip leading/trailing whitespace, and replace spaces with hyphens.",
+  "workspace": {
+    "files": {
+      "strings.py": "",
+      "test_strings.py": "from strings import slugify\n\n\ndef test_slugify():\n    assert slugify(' Hello Benchmark ') == 'hello-benchmark'\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_strings.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 120,
+    "max_steps": 30
+  },
+  "tags": ["smoke", "offline", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/smoke-edit-readme.json

@@ -0,0 +1,20 @@
+{
+  "id": "smoke-edit-readme",
+  "title": "Edit README",
+  "description": "Update a README with a requested sentence.",
+  "prompt": "Update README.md to include a short sentence explaining that this project is used for a Pythinker benchmark smoke test.",
+  "workspace": {
+    "files": {
+      "README.md": "# Example Project\n\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python - <<'PY'\nfrom pathlib import Path\ntext = Path('README.md').read_text(encoding='utf-8')\nassert 'Pythinker benchmark smoke test' in text\nPY"
+  },
+  "limits": {
+    "timeout_seconds": 120,
+    "max_steps": 30
+  },
+  "tags": ["smoke", "offline", "deterministic"]
+}

pythinker_code/benchmark/bundled/tasks/smoke-fix-python-test.json

@@ -0,0 +1,21 @@
+{
+  "id": "smoke-fix-python-test",
+  "title": "Fix Python Test",
+  "description": "Fix a small Python function so the included test passes.",
+  "prompt": "Fix calc.py so python -m pytest test_calc.py passes. Keep the change minimal.",
+  "workspace": {
+    "files": {
+      "calc.py": "def add(a, b):\n    return a - b\n",
+      "test_calc.py": "from calc import add\n\n\ndef test_add():\n    assert add(2, 3) == 5\n"
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_calc.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 120,
+    "max_steps": 30
+  },
+  "tags": ["smoke", "offline", "deterministic"]
+}