shipit-agent 2.0.7__tar.gz → 2.2.0__tar.gz

{shipit_agent-2.0.7/shipit_agent.egg-info → shipit_agent-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: shipit-agent
-Version: 2.0.7
+Version: 2.2.0
 Summary: A clean, powerful agent runtime with tools, MCP support, profiles, and streaming events.
 Author: Rahul Raj
 License: MIT
@@ -251,6 +251,73 @@ Both are backwards compatible and covered by the test suite. See the
 
 ---
 
+## Automatic tool discovery
+
+Ordinary `Agent(llm=llm, tools=tools, mcps=servers)` now defaults to
+`deferred_tools="auto"`. Catalogs that fit ten tools and approximately 4,096
+schema tokens stay visible. Otherwise, the initial request exposes up to nine
+working tools plus `search_tools`, within that schema budget.
+An existing `ToolSearchTool` is reused under its configured name.
+
+Core selection prefers a tool's numeric `discovery_priority`, then Shipit's
+core capabilities, then registration order. Assign higher priorities to your
+application's frequently used tools. Up to three recently discovered schemas
+can be reused within the same session, competing for the same initial slots.
+Changed schemas and statically denied tools are excluded from reuse; argument
+permissions are still checked at execution. Session caches are bounded to 32
+sessions per Agent instance and are not persisted across process restarts.
+The model can search descriptions, names, and MCP server metadata; matching
+tools become callable with their full schemas on the next step. Search does
+not execute them or grant additional permissions. Loaded tools remain visible
+for the rest of the run, so ten is an initial budget, not an execution limit.
+
+Auto mode includes a compact capability-family index instead of listing every
+deferred tool name. Ranking runs locally without additional LLM calls.
+Exact-name searches load only the named tool; names can use underscores,
+camelCase or Unicode. Newly loaded schemas append after those already shown.
+This works through `run()`, `stream()`, and the async runtime using ordinary
+function schemas, including providers without native tool-search support.
+
+Use `deferred_tools=False` to keep every schema visible, `True` for the existing
+core-only policy, or a list of names to defer specific tools. Code mode retains
+its own discovery policy. Measure total request tokens and task completion in
+your workload: discovery adds an LLM step, so schema savings are not a promise
+of equal total-token savings.
+
+See [the discovery review and offline benchmark](docs/tool-discovery-review.md)
+for measured payload reductions and remaining tradeoffs.
+
+Ordinary `Agent` also supports opt-in `progressive_skills=True` for metadata-first
+skill loading and `max_task_tokens=50_000` for a soft per-run token threshold.
+See [progressive skills, budget limits and validation](docs/progressive-agent-controls.md)
+for setup, isolation guarantees and provider-usage caveats.
+
+```python
+from shipit_agent.deferral import DiscoveryPolicy
+
+agent = Agent(
+    llm=llm, tools=tools, mcps=servers,
+    deferred_tools=DiscoveryPolicy(initial_tools=10, schema_tokens=4096, reuse_tools=3),
+)
+```
+
+Discovery itself remains visible even if its schema exceeds a very small
+configured budget. Definitions loaded during a run are not evicted mid-run.
+For large structured results, tools may return
+`ToolOutput.from_records(rows, fields=["id", "evidence"], limit=20)`.
+The model receives an explicit page and total count; callers retain the full
+JSON. This is context reduction, not security redaction.
+
+Run summaries distinguish `finished` from `incomplete` provider output, expose
+finish reasons, failed-tool counts, and recovery counts. `finished` means the
+runtime ended normally, not that an independent grader verified the answer.
+Provider-reported usage counters remain unchanged; cache ratios and cost
+estimates account for whether cached tokens are included in prompt tokens.
+
+An opt-in, billed multi-turn evaluation is available in
+`scripts/evaluate_agent_session.py`. It uses safe fixture tools and an
+in-process MCP server, not production data or production MCP transport.
+
 ## What's new in v1.7.0 — the working set
 
 The biggest capability release yet: do the powerful thing without burning tokens or trust.

{shipit_agent-2.0.7 → shipit_agent-2.2.0}/README.md

@@ -125,6 +125,73 @@ Both are backwards compatible and covered by the test suite. See the
 
 ---
 
+## Automatic tool discovery
+
+Ordinary `Agent(llm=llm, tools=tools, mcps=servers)` now defaults to
+`deferred_tools="auto"`. Catalogs that fit ten tools and approximately 4,096
+schema tokens stay visible. Otherwise, the initial request exposes up to nine
+working tools plus `search_tools`, within that schema budget.
+An existing `ToolSearchTool` is reused under its configured name.
+
+Core selection prefers a tool's numeric `discovery_priority`, then Shipit's
+core capabilities, then registration order. Assign higher priorities to your
+application's frequently used tools. Up to three recently discovered schemas
+can be reused within the same session, competing for the same initial slots.
+Changed schemas and statically denied tools are excluded from reuse; argument
+permissions are still checked at execution. Session caches are bounded to 32
+sessions per Agent instance and are not persisted across process restarts.
+The model can search descriptions, names, and MCP server metadata; matching
+tools become callable with their full schemas on the next step. Search does
+not execute them or grant additional permissions. Loaded tools remain visible
+for the rest of the run, so ten is an initial budget, not an execution limit.
+
+Auto mode includes a compact capability-family index instead of listing every
+deferred tool name. Ranking runs locally without additional LLM calls.
+Exact-name searches load only the named tool; names can use underscores,
+camelCase or Unicode. Newly loaded schemas append after those already shown.
+This works through `run()`, `stream()`, and the async runtime using ordinary
+function schemas, including providers without native tool-search support.
+
+Use `deferred_tools=False` to keep every schema visible, `True` for the existing
+core-only policy, or a list of names to defer specific tools. Code mode retains
+its own discovery policy. Measure total request tokens and task completion in
+your workload: discovery adds an LLM step, so schema savings are not a promise
+of equal total-token savings.
+
+See [the discovery review and offline benchmark](docs/tool-discovery-review.md)
+for measured payload reductions and remaining tradeoffs.
+
+Ordinary `Agent` also supports opt-in `progressive_skills=True` for metadata-first
+skill loading and `max_task_tokens=50_000` for a soft per-run token threshold.
+See [progressive skills, budget limits and validation](docs/progressive-agent-controls.md)
+for setup, isolation guarantees and provider-usage caveats.
+
+```python
+from shipit_agent.deferral import DiscoveryPolicy
+
+agent = Agent(
+    llm=llm, tools=tools, mcps=servers,
+    deferred_tools=DiscoveryPolicy(initial_tools=10, schema_tokens=4096, reuse_tools=3),
+)
+```
+
+Discovery itself remains visible even if its schema exceeds a very small
+configured budget. Definitions loaded during a run are not evicted mid-run.
+For large structured results, tools may return
+`ToolOutput.from_records(rows, fields=["id", "evidence"], limit=20)`.
+The model receives an explicit page and total count; callers retain the full
+JSON. This is context reduction, not security redaction.
+
+Run summaries distinguish `finished` from `incomplete` provider output, expose
+finish reasons, failed-tool counts, and recovery counts. `finished` means the
+runtime ended normally, not that an independent grader verified the answer.
+Provider-reported usage counters remain unchanged; cache ratios and cost
+estimates account for whether cached tokens are included in prompt tokens.
+
+An opt-in, billed multi-turn evaluation is available in
+`scripts/evaluate_agent_session.py`. It uses safe fixture tools and an
+in-process MCP server, not production data or production MCP transport.
+
 ## What's new in v1.7.0 — the working set
 
 The biggest capability release yet: do the powerful thing without burning tokens or trust.

shipit_agent-2.2.0/notebooks/openrouter_deep_test.ipynb

@@ -0,0 +1,309 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# OpenRouter deep test \u2014 `stealth/ox-alpha`\n",
+    "\n",
+    "End-to-end tests for the OpenRouter OpenAI-compatible endpoint: **text, streaming, vision, tool-calling (single + multi-arg), the shipit adapter, and a multi-agent example.**\n",
+    "\n",
+    "> Run cells top-to-bottom. The key is entered securely at runtime (never saved in the notebook).\n",
+    "> Model note: `stealth/ox-alpha` is a fast text/vision model but **does not accept video URLs** (returns 404). Images and tools work.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 0 \u00b7 Setup\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "# pip install openai  (already present in most envs)\n",
+    "import os, json, time, getpass\n",
+    "from openai import OpenAI\n",
+    "\n",
+    "# Enter your OpenRouter key securely (not stored in the notebook).\n",
+    "API_KEY = os.getenv('OPENROUTER_API_KEY') or getpass.getpass('OpenRouter API key: ')\n",
+    "MODEL   = 'stealth/ox-alpha'\n",
+    "\n",
+    "client = OpenAI(base_url='https://openrouter.ai/api/v1', api_key=API_KEY,\n",
+    "                timeout=120, max_retries=1)\n",
+    "HEADERS = {'HTTP-Referer': 'https://aftrdrk.dev', 'X-Title': 'OpenRouter deep test'}\n",
+    "print('client ready \u00b7', MODEL)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1 \u00b7 Basic text + latency\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "t0 = time.time()\n",
+    "r = client.chat.completions.create(\n",
+    "    model=MODEL, extra_headers=HEADERS, max_tokens=60,\n",
+    "    messages=[{'role':'user','content':'In one sentence, what is a ransomware double-extortion attack?'}])\n",
+    "print('latency: %.1fs' % (time.time()-t0))\n",
+    "print('content:', r.choices[0].message.content)\n",
+    "print('usage  :', r.usage.model_dump())\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2 \u00b7 Streaming (token-by-token)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "stream = client.chat.completions.create(\n",
+    "    model=MODEL, extra_headers=HEADERS, max_tokens=120, stream=True,\n",
+    "    messages=[{'role':'user','content':'List 3 detection tips for the Akira ransomware group, briefly.'}])\n",
+    "for chunk in stream:\n",
+    "    d = chunk.choices[0].delta.content if chunk.choices else None\n",
+    "    if d: print(d, end='')\n",
+    "print('\\n\\n[stream done]')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3 \u00b7 Vision (image)\n",
+    "`stealth/ox-alpha` can read images. It can be slow (30-90s) \u2014 be patient.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "t0 = time.time()\n",
+    "r = client.chat.completions.create(\n",
+    "    model=MODEL, extra_headers=HEADERS, max_tokens=150,\n",
+    "    messages=[{'role':'user','content':[\n",
+    "        {'type':'text','text':'What is in this image? One sentence.'},\n",
+    "        {'type':'image_url','image_url':{'url':'https://live.staticflickr.com/3851/14825276609_098cac593d_b.jpg'}},\n",
+    "    ]}])\n",
+    "print('latency: %.1fs' % (time.time()-t0))\n",
+    "print(r.choices[0].message.content)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4 \u00b7 Tool calling \u2014 single tool\n",
+    "Native OpenAI function-calling. The model should return a structured `tool_call`.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "tools = [{'type':'function','function':{\n",
+    "    'name':'get_weather','description':'Current weather for a city',\n",
+    "    'parameters':{'type':'object','properties':{'city':{'type':'string'}},'required':['city']}}}]\n",
+    "r = client.chat.completions.create(\n",
+    "    model=MODEL, extra_headers=HEADERS, tools=tools, tool_choice='auto', max_tokens=100,\n",
+    "    messages=[{'role':'user','content':\"What's the weather in Berlin?\"}])\n",
+    "m = r.choices[0].message\n",
+    "print('tool_calls:', [(t.function.name, t.function.arguments) for t in (m.tool_calls or [])] or 'none')\n",
+    "print('text      :', m.content)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5 \u00b7 Multi-argument tool (MCP-style)\n",
+    "The real agent test: can it fill several arguments in one structured call?\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "tools = [{'type':'function','function':{\n",
+    "    'name':'query_records','description':'Query internal records with multiple filters',\n",
+    "    'parameters':{'type':'object','properties':{\n",
+    "        'domain':{'type':'string','description':'e.g. ransomlook'},\n",
+    "        'entity':{'type':'string','description':'e.g. groups'},\n",
+    "        'entity_id':{'type':'string','description':'e.g. akira'}},\n",
+    "    'required':['domain','entity','entity_id']}}}]\n",
+    "r = client.chat.completions.create(\n",
+    "    model=MODEL, extra_headers=HEADERS, tools=tools, tool_choice='required', max_tokens=120,\n",
+    "    messages=[{'role':'user','content':'Pull the RansomLook group profile for Akira.'}])\n",
+    "m = r.choices[0].message\n",
+    "for t in (m.tool_calls or []):\n",
+    "    print(t.function.name, '->', json.loads(t.function.arguments))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6 \u00b7 Through the shipit adapter (`OpenAIChatLLM`)\n",
+    "Same call, but via shipit's OpenAI-compatible adapter \u2014 proving OpenRouter works inside the agent framework.\n",
+    "Requires `shipit_agent` importable (run in an env where it's installed).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    from shipit_agent.llms.openai_adapter import OpenAIChatLLM\n",
+    "    from shipit_agent.models import Message\n",
+    "    llm = OpenAIChatLLM(model=MODEL, api_key=API_KEY,\n",
+    "                        base_url='https://openrouter.ai/api/v1',\n",
+    "                        default_headers=HEADERS)\n",
+    "    resp = llm.complete(messages=[Message(role='user', content='Name one famous ransomware group in 2 words.')], tools=None)\n",
+    "    print('content:', resp.content)\n",
+    "    print('usage  :', resp.usage)   # cache_read_input_tokens surfaces here when warm\n",
+    "except ImportError as e:\n",
+    "    print('shipit_agent not importable in this kernel \u2014 skip:', e)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7 \u00b7 Multi-agent example\n",
+    "A tiny two-agent pipeline on OpenRouter: **Researcher** proposes findings, **Critic** reviews them, **Editor** writes the final brief. Pure OpenAI-compatible calls \u2014 swap in real tools/MCP as needed.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "def ask(system, user, max_tokens=300):\n",
+    "    r = client.chat.completions.create(\n",
+    "        model=MODEL, extra_headers=HEADERS, max_tokens=max_tokens, temperature=0.3,\n",
+    "        messages=[{'role':'system','content':system},{'role':'user','content':user}])\n",
+    "    return r.choices[0].message.content or ''\n",
+    "\n",
+    "TOPIC = 'the Akira ransomware group and its threat to manufacturers'\n",
+    "\n",
+    "print('\u2500\u2500 RESEARCHER \u2500\u2500')\n",
+    "research = ask('You are a threat-intel researcher. Be concrete and factual.',\n",
+    "               f'Give 5 key findings about {TOPIC}. Bullet points.')\n",
+    "print(research)\n",
+    "\n",
+    "print('\\n\u2500\u2500 CRITIC \u2500\u2500')\n",
+    "critique = ask('You are a skeptical reviewer. Flag anything unsupported or vague.',\n",
+    "               f'Review these findings and note weaknesses:\\n{research}')\n",
+    "print(critique)\n",
+    "\n",
+    "print('\\n\u2500\u2500 EDITOR \u2500\u2500')\n",
+    "brief = ask('You are a senior analyst writing for a client board.',\n",
+    "            f'Using the findings and the critique, write a tight 1-paragraph brief.\\n\\nFINDINGS:\\n{research}\\n\\nCRITIQUE:\\n{critique}')\n",
+    "print(brief)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 8 \u00b7 Agentic tool loop (multi-turn, one agent + tools)\n",
+    "A minimal manual agent loop: the model calls a tool, we execute it, feed the result back, and it answers. This is the core of any MCP agent.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "def fake_lookup(domain, entity, entity_id):\n",
+    "    # stand-in for a real MCP tool\n",
+    "    return json.dumps({'domain':domain,'entity':entity,'id':entity_id,\n",
+    "        'first_seen':'2023-03','model':'RaaS','notable':'targets VMware ESXi'})\n",
+    "\n",
+    "tools = [{'type':'function','function':{'name':'query_records',\n",
+    "    'parameters':{'type':'object','properties':{'domain':{'type':'string'},\n",
+    "    'entity':{'type':'string'},'entity_id':{'type':'string'}},\n",
+    "    'required':['domain','entity','entity_id']},'description':'Query internal records'}}]\n",
+    "\n",
+    "msgs = [{'role':'user','content':'Look up the Akira group profile in ransomlook and summarize it.'}]\n",
+    "for step in range(4):\n",
+    "    r = client.chat.completions.create(model=MODEL, extra_headers=HEADERS,\n",
+    "        tools=tools, tool_choice='auto', max_tokens=300, messages=msgs)\n",
+    "    m = r.choices[0].message\n",
+    "    if m.tool_calls:\n",
+    "        msgs.append({'role':'assistant','content':m.content or '','tool_calls':[t.model_dump() for t in m.tool_calls]})\n",
+    "        for t in m.tool_calls:\n",
+    "            args = json.loads(t.function.arguments)\n",
+    "            result = fake_lookup(**args)\n",
+    "            print(f'[tool] {t.function.name}({args}) -> {result[:60]}...')\n",
+    "            msgs.append({'role':'tool','tool_call_id':t.id,'content':result})\n",
+    "    else:\n",
+    "        print('\\n[final answer]\\n' + (m.content or ''))\n",
+    "        break\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 9 \u00b7 Latency / cache benchmark\n",
+    "Send the same large prefix 3x back-to-back and watch `cached_tokens` climb (implicit caching).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": [],
+   "source": [
+    "PREFIX = 'Background: ' + ('Akira is a ransomware group. ' * 300)\n",
+    "for i in range(3):\n",
+    "    t0 = time.time()\n",
+    "    r = client.chat.completions.create(model=MODEL, extra_headers=HEADERS, max_tokens=8,\n",
+    "        messages=[{'role':'user','content':PREFIX + f'\\n\\nSay the number {i}.'}])\n",
+    "    u = r.usage.model_dump(); pd = u.get('prompt_tokens_details',{})\n",
+    "    print(f'call {i}: {time.time()-t0:.1f}s \u00b7 prompt={u[\"prompt_tokens\"]} \u00b7 cached={pd.get(\"cached_tokens\")}')\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

{shipit_agent-2.0.7 → shipit_agent-2.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "shipit-agent"
-version = "2.0.7"
+version = "2.2.0"
 description = "A clean, powerful agent runtime with tools, MCP support, profiles, and streaming events."
 readme = "README.md"
 requires-python = ">=3.11"

{shipit_agent-2.0.7 → shipit_agent-2.2.0}/shipit_agent/agent.py

@@ -286,13 +286,17 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
     code_mode: bool = False
 
     # ── deferred tool loading (v1.7) ──────────────────────────────────
-    # Tool paging: core tools keep their full schema in
+    # Default "auto": above ten tools, keep nine core capabilities and a
+    # discovery tool. discovery_priority on a tool overrides core ranking;
+    # ties follow registration order. False opts out. Core tools keep schemas in
     # every request; everything else is listed by NAME only until
     # `tool_search` (or a direct call) loads it. Cuts the fixed per-step
     # schema cost to the working set. ``True`` defers everything outside
     # the core set; a list of names defers exactly those. Ignored when
     # ``code_mode`` is on (code mode collapses the catalogue its own way).
-    deferred_tools: Any = False
+    deferred_tools: Any = "auto"
+    # Pass deferral.DiscoveryPolicy(...) for initial schema-token/count budgets
+    # and bounded session reuse. Auto uses 10 tools / ~4096 schema tokens.
 
     # ── lockdown (exfiltration guard) ─────────────────────────────────
     # Once a tool reports it returned sensitive data, the run may only make
@@ -338,6 +342,11 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
     skills: list[SkillLike] = field(default_factory=list)  # always-on skills
     default_skill_ids: list[str] = field(default_factory=list)
     skill_match_limit: int = 3  # max auto-matched
+    progressive_skills: bool = False  # metadata-first, model-selected loading
+    max_task_tokens: int | None = None  # soft per-run stop between model steps
+    # Soft stop across every run of this agent (or one chat session), checked
+    # between steps. Clones start fresh; reset with reset_session_usage().
+    max_session_tokens: int | None = None
 
     # ── rules (see shipit_agent/rules/) ───────────────────────────────
     # Durable behavioural policy (AGENTS.md house-style), scoped to paths/tools
@@ -408,13 +417,32 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
         """Release provider/MCP resources without blocking an event loop."""
         await asyncio.to_thread(self.close)
 
+    @property
+    def session_token_usage(self) -> int:
+        """Tokens spent across every run of this agent's session so far."""
+        return int(self._session_runtime_state.get("session_tokens", 0))
+
+    def reset_session_usage(self) -> None:
+        """Start the session budget over — e.g. for a long-lived server agent
+        at the start of a new billing window."""
+        self._session_runtime_state.pop("session_tokens", None)
+        self._session_runtime_state.pop("session_usage_complete", None)
+
     def clone(self, **changes: Any) -> "Agent":
         """Copy this agent with selected configuration overrides.
 
         Mutable configuration containers are copied, while LLMs and durable
         stores are intentionally shared unless explicitly replaced.
         """
-        from dataclasses import replace
+        from dataclasses import fields, replace
+
+        hooks = self.hooks
+        if hooks is not None:
+            hooks = replace(hooks, **{
+                item.name: list(getattr(hooks, item.name))
+                for item in fields(hooks)
+                if isinstance(getattr(hooks, item.name), list)
+            })
 
         defaults = {
             "tools": list(self.tools),
@@ -425,9 +453,16 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
             "skills": list(self.skills),
             "default_skill_ids": list(self.default_skill_ids),
             "rules": list(self.rules),
+            "hooks": hooks,
+            # Contributions are already present in tools/hooks. Re-activation
+            # would duplicate tools and mutate callbacks on every chat turn.
+            "plugins": [],
         }
         defaults.update(changes)
-        return replace(self, **defaults)
+        cloned = replace(self, **defaults)
+        if "plugins" not in changes:
+            cloned.plugins = list(self.plugins)
+        return cloned
 
     def as_tool(
         self,
@@ -472,6 +507,14 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
             self.memory_store = InMemoryMemoryStore()
         if self.stream_queue_maxsize < 1:
             raise ValueError("stream_queue_maxsize must be positive")
+        if self.max_task_tokens is not None and (
+            type(self.max_task_tokens) is not int or self.max_task_tokens < 1
+        ):
+            raise ValueError("max_task_tokens must be a positive integer or None")
+        if self.max_session_tokens is not None and (
+            type(self.max_session_tokens) is not int or self.max_session_tokens < 1
+        ):
+            raise ValueError("max_session_tokens must be a positive integer or None")
         if self.stream_join_timeout < 0:
             raise ValueError("stream_join_timeout cannot be negative")
 
@@ -861,7 +904,7 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
             "progress_summaries": self.progress_summaries,
             "reminder": self.reminder,
             "evict_prior_tool_outputs": self.evict_prior_tool_outputs,
-            "prompt": self._effective_prompt(user_prompt),
+            "prompt": self._effective_prompt(user_prompt, selected_skills=selected_skills),
             "tools": effective_tools,
             "mcps": self.mcps,
             "required_tools": self.required_tools,
@@ -873,6 +916,8 @@ class Agent(AgentPreparationMixin, UpgradeMixin):
                 "agent_name": self.name,
                 "agent_description": self.description,
                 **self.metadata,
+                "max_task_tokens": self.max_task_tokens,
+                "max_session_tokens": self.max_session_tokens,
                 "used_skills": skill_ids,
                 "used_skill_tools": skill_tool_names,
                 "selected_skills": skill_details,

{shipit_agent-2.0.7 → shipit_agent-2.2.0}/shipit_agent/agent_preparation.py

@@ -56,7 +56,7 @@ class AgentPreparationMixin:
         seen: set[str] = set()
         candidates = [*self.skills]
         candidates.extend(self._resolve_skill_ref(i) for i in self.default_skill_ids)
-        if self.auto_use_skills and self.skill_registry is not None:
+        if self.auto_use_skills and not self.progressive_skills and self.skill_registry is not None:
             candidates.extend(
                 find_relevant_skills(
                     self.skill_registry,
@@ -70,15 +70,34 @@ class AgentPreparationMixin:
                 seen.add(skill.id)
         return selected
 
-    def _effective_prompt(self, user_prompt: str) -> str:
+    def _effective_prompt(
+        self, user_prompt: str, *, selected_skills: list[Skill] | None = None
+    ) -> str:
         effective = self.prompt
-        for skill in self._selected_skills(user_prompt):
+        if selected_skills is None:
+            selected_skills = self._selected_skills(user_prompt)
+        for skill in selected_skills:
             holder = type("PromptHolder", (), {"prompt": effective})()
             apply_skill(holder, skill)
             effective = holder.prompt
+        catalog = self._progressive_skill_catalog(selected_skills)
+        if catalog:
+            effective += "\n\n# Available skills\nLoad relevant instructions with load_skill before applying them. " \
+                "Skills provide guidance, not permission to access new tools or data.\n"
+            effective += "\n".join(f"- {entry.id}: {entry.description}" for entry in catalog)
         rules = self._rules_block()
         return f"{effective}\n\n{rules}" if rules and rules not in effective else effective
 
+    def _progressive_skill_catalog(self, selected_skills):
+        if not self.progressive_skills or self.skill_registry is None:
+            return []
+        from shipit_agent.skills.catalog import build_catalog
+        return build_catalog(
+            (skill for skill in self.skill_registry.list()
+             if skill.is_visible and skill.is_admin_enabled and skill.is_user_enabled),
+            exclude=[skill.id for skill in selected_skills],
+        )
+
     def _rules_block(self) -> str:
         from shipit_agent.rules import RuleSet, collect_tool_rules
 
@@ -138,13 +157,29 @@ class AgentPreparationMixin:
             if self._auto_sub_agent is not None:
                 effective["sub_agent"] = self._auto_sub_agent
 
-        if selected_skills:
+        missing_skill_tools = [
+            name for name in tool_names_for_skills(selected_skills)
+            if name not in effective
+        ]
+        if missing_skill_tools:
             builtins = get_builtin_tool_map(
                 llm=self.llm, project_root=str(self.project_root)
             )
-            for name in tool_names_for_skills(selected_skills):
+            for name in missing_skill_tools:
                 if name in builtins:
-                    effective[name] = builtins[name]
+                    # Host-supplied tools may enforce tenant scoping or custom
+                    # behavior. A skill adds missing capabilities, not overrides.
+                    effective.setdefault(name, builtins[name])
+        catalog = self._progressive_skill_catalog(selected_skills)
+        if catalog:
+            from shipit_agent.skills.catalog import LoadSkillTool, SkillSession
+            if "load_skill" in effective:
+                raise ValueError("progressive_skills reserves the tool name 'load_skill'")
+            # Fresh state per run: never store mutable loading state on an
+            # Agent shared by several chats. Previous evidence stays in history;
+            # reloading remains possible after history eviction/compaction.
+            lookup = {entry.id: self.skill_registry.get(entry.id) for entry in catalog}
+            effective["load_skill"] = LoadSkillTool(lookup, SkillSession(), available_tools=[])
         tools = list(effective.values())
         if self.verifier is not None and hasattr(self.verifier, "wrap_tools"):
             tools = self.verifier.wrap_tools(tools)