gemcode 0.3.70__tar.gz → 0.3.71__tar.gz

{gemcode-0.3.70/src/gemcode.egg-info → gemcode-0.3.71}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gemcode
-Version: 0.3.70
+Version: 0.3.71
 Summary: Local-first coding agent on Google Gemini + ADK
 Author: GemCode Contributors
 License:                                  Apache License
@@ -217,6 +217,16 @@ gemcode --yes "Add a module docstring to src/foo.py"
 gemcode --session mysess --yes "Continue: run tests and fix failures"
 ```
 
+### What GemCode writes to `.gemcode/`
+
+GemCode keeps project-local state under `.gemcode/`:
+
+- **`sessions.sqlite`**: session events/history (ADK `SqliteSessionService`)
+- **`audit.log`**: JSONL audit trail for tool usage + model usage + stop reasons
+- **`tool-results/`**: oversized tool outputs offloaded to stable refs (`tool_result:<sha>`)
+- **`artifacts/`**: file artifacts (ADK `FileArtifactService`)
+- **`policy.json`**: self-tuning per-repo profile used to calibrate dynamic budgets
+
 - **`--yes`**: allow mutating tools (`write_file`, `search_replace`). Shell execution is still restricted by the `.env.example` allowlist.
 - **`--session`**: Conversation history is stored under `.gemcode/sessions.sqlite` (ADK `SqliteSessionService`). Reuse the same `--session` id to continue.
 - **`--max-llm-calls`**: cap model↔tool iterations for this message (maps to ADK `RunConfig.max_llm_calls`). You can also set `GEMCODE_MAX_LLM_CALLS`.
@@ -243,6 +253,33 @@ gemcode --session mysess --yes "Continue: run tests and fix failures"
 - **Recovery-loop**: ADK `ReflectAndRetryToolPlugin`-based retries on tool failures.
   - Set `GEMCODE_ENABLE_TOOL_RECOVERY_RETRY=0` to disable.
   - Set `GEMCODE_TOOL_REFLECT_MAX_RETRIES=1` to control retries per tool.
+
+### Token efficiency (dynamic + intelligent)
+
+GemCode optimizes tokens without losing capability by using a dynamic policy:
+
+- **Context-pressure aware**: tool caps tighten when context is tight, loosen when there is room.
+- **Risk/complexity aware**: caps increase for risky tasks (writes, shell, failures, many files).
+- **Self-tuning per repo**: `.gemcode/policy.json` calibrates baseline evidence budgets over time.
+
+Key env toggles:
+
+- `GEMCODE_DYNAMIC_TOKEN_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_BOOST=<float>` (default `0.6`)
+- `GEMCODE_TOOL_RESULT_OFFLOAD=0|1` (default `1`)
+
+Live telemetry:
+
+- `/status` shows `risk_score`, `context_percent_left`, and profile EMAs.
+
+### Stable tool output offloading (OpenClaude-style)
+
+Oversized tool outputs are automatically offloaded and replaced with stable refs:
+
+- Stored in: `.gemcode/tool-results/`
+- References: `tool_result:<sha256>`
+- Load on demand: `load_tool_result(ref)`
 - **Gemini thinking controls (Claude-like)**:
   - By default GemCode lets Gemini use its dynamic/adaptive thinking behavior.
   - Set `GEMCODE_DISABLE_THINKING=1` to force a best-effort “low thinking” mode:
@@ -301,12 +338,24 @@ the user’s project.
   - `list_directory`
   - `glob_files`
   - `grep_content`
+  - `repo_map`
+  - `web_search`
+  - `web_fetch`
+  - `notebook_read`
+  - `notebook_edit`
 - Mutating tools (require `--yes` unless your policy blocks them):
   - `write_file`
   - `search_replace`
 - Shell execution:
   - `run_command` (guarded by `GEMCODE_ALLOW_COMMANDS` from `.env.example` and
     `GEMCODE_PERMISSION_MODE`).
+  - `bash` (pipelines + redirects; supports `background=True`)
+
+- Background task management (for processes started via `bash(..., background=True)`):
+  - `list_tasks`, `task_output`, `kill_task`
+
+- Tool offload loader:
+  - `load_tool_result(ref)`
 
 Tool execution is still controlled by permission gates and then governed by
 GemCode’s circuit breaker + recovery behavior.
@@ -434,6 +483,23 @@ pip install -e ".[dev]"
 pytest
 ```
 
+## Release workflow (GitHub Actions → PyPI)
+
+This repository publishes to PyPI on tag pushes:
+
+- `.github/workflows/publish-pypi.yml` triggers on tags matching `v*`
+
+Typical release steps:
+
+```bash
+# 1) bump gemcode/pyproject.toml version
+git add -A
+git commit -m "release: vX.Y.Z"
+git tag -a vX.Y.Z -m "vX.Y.Z"
+git push origin HEAD
+git push origin vX.Y.Z
+```
+
 ## References (local only)
 
 Do not commit proprietary leaked trees into this package. Keep `claude-code-leaked/` and similar folders outside version control or in a private mirror.

{gemcode-0.3.70 → gemcode-0.3.71}/README.md

@@ -28,6 +28,16 @@ gemcode --yes "Add a module docstring to src/foo.py"
 gemcode --session mysess --yes "Continue: run tests and fix failures"
 ```
 
+### What GemCode writes to `.gemcode/`
+
+GemCode keeps project-local state under `.gemcode/`:
+
+- **`sessions.sqlite`**: session events/history (ADK `SqliteSessionService`)
+- **`audit.log`**: JSONL audit trail for tool usage + model usage + stop reasons
+- **`tool-results/`**: oversized tool outputs offloaded to stable refs (`tool_result:<sha>`)
+- **`artifacts/`**: file artifacts (ADK `FileArtifactService`)
+- **`policy.json`**: self-tuning per-repo profile used to calibrate dynamic budgets
+
 - **`--yes`**: allow mutating tools (`write_file`, `search_replace`). Shell execution is still restricted by the `.env.example` allowlist.
 - **`--session`**: Conversation history is stored under `.gemcode/sessions.sqlite` (ADK `SqliteSessionService`). Reuse the same `--session` id to continue.
 - **`--max-llm-calls`**: cap model↔tool iterations for this message (maps to ADK `RunConfig.max_llm_calls`). You can also set `GEMCODE_MAX_LLM_CALLS`.
@@ -54,6 +64,33 @@ gemcode --session mysess --yes "Continue: run tests and fix failures"
 - **Recovery-loop**: ADK `ReflectAndRetryToolPlugin`-based retries on tool failures.
   - Set `GEMCODE_ENABLE_TOOL_RECOVERY_RETRY=0` to disable.
   - Set `GEMCODE_TOOL_REFLECT_MAX_RETRIES=1` to control retries per tool.
+
+### Token efficiency (dynamic + intelligent)
+
+GemCode optimizes tokens without losing capability by using a dynamic policy:
+
+- **Context-pressure aware**: tool caps tighten when context is tight, loosen when there is room.
+- **Risk/complexity aware**: caps increase for risky tasks (writes, shell, failures, many files).
+- **Self-tuning per repo**: `.gemcode/policy.json` calibrates baseline evidence budgets over time.
+
+Key env toggles:
+
+- `GEMCODE_DYNAMIC_TOKEN_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_BOOST=<float>` (default `0.6`)
+- `GEMCODE_TOOL_RESULT_OFFLOAD=0|1` (default `1`)
+
+Live telemetry:
+
+- `/status` shows `risk_score`, `context_percent_left`, and profile EMAs.
+
+### Stable tool output offloading (OpenClaude-style)
+
+Oversized tool outputs are automatically offloaded and replaced with stable refs:
+
+- Stored in: `.gemcode/tool-results/`
+- References: `tool_result:<sha256>`
+- Load on demand: `load_tool_result(ref)`
 - **Gemini thinking controls (Claude-like)**:
   - By default GemCode lets Gemini use its dynamic/adaptive thinking behavior.
   - Set `GEMCODE_DISABLE_THINKING=1` to force a best-effort “low thinking” mode:
@@ -112,12 +149,24 @@ the user’s project.
   - `list_directory`
   - `glob_files`
   - `grep_content`
+  - `repo_map`
+  - `web_search`
+  - `web_fetch`
+  - `notebook_read`
+  - `notebook_edit`
 - Mutating tools (require `--yes` unless your policy blocks them):
   - `write_file`
   - `search_replace`
 - Shell execution:
   - `run_command` (guarded by `GEMCODE_ALLOW_COMMANDS` from `.env.example` and
     `GEMCODE_PERMISSION_MODE`).
+  - `bash` (pipelines + redirects; supports `background=True`)
+
+- Background task management (for processes started via `bash(..., background=True)`):
+  - `list_tasks`, `task_output`, `kill_task`
+
+- Tool offload loader:
+  - `load_tool_result(ref)`
 
 Tool execution is still controlled by permission gates and then governed by
 GemCode’s circuit breaker + recovery behavior.
@@ -245,6 +294,23 @@ pip install -e ".[dev]"
 pytest
 ```
 
+## Release workflow (GitHub Actions → PyPI)
+
+This repository publishes to PyPI on tag pushes:
+
+- `.github/workflows/publish-pypi.yml` triggers on tags matching `v*`
+
+Typical release steps:
+
+```bash
+# 1) bump gemcode/pyproject.toml version
+git add -A
+git commit -m "release: vX.Y.Z"
+git tag -a vX.Y.Z -m "vX.Y.Z"
+git push origin HEAD
+git push origin vX.Y.Z
+```
+
 ## References (local only)
 
 Do not commit proprietary leaked trees into this package. Keep `claude-code-leaked/` and similar folders outside version control or in a private mirror.

{gemcode-0.3.70 → gemcode-0.3.71}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gemcode"
-version = "0.3.70"
+version = "0.3.71"
 description = "Local-first coding agent on Google Gemini + ADK"
 readme = "README.md"
 requires-python = ">=3.11"

{gemcode-0.3.70 → gemcode-0.3.71}/src/gemcode/callbacks.py

@@ -48,6 +48,9 @@ _RISK_TOOL_CALLS = "gemcode:risk_tool_calls"
 _RISK_HAD_SHELL = "gemcode:risk_had_shell"
 _RISK_HAD_WRITE = "gemcode:risk_had_write"
 _RISK_HAD_FAILURE = "gemcode:risk_had_failure"
+_TOOL_GROUP_CHARS = "gemcode:tool_group_chars"
+_TOOL_GROUP_EXCEEDED = "gemcode:tool_group_budget_exceeded"
+_TOOL_SEQ = "gemcode:tool_seq"
 
 def _truthy_env(name: str, *, default: bool = False) -> bool:
   v = os.environ.get(name)
@@ -150,6 +153,8 @@ def make_before_tool_callback(cfg: GemCodeConfig):
     try:
       if tool_context is not None:
         st = tool_context.state
+        # Per-turn tool sequence (used for stable tool-result replacement keys).
+        st[_TOOL_SEQ] = int(st.get(_TOOL_SEQ, 0) or 0) + 1
         st[_RISK_TOOL_CALLS] = int(st.get(_RISK_TOOL_CALLS, 0) or 0) + 1
         if name == "read_file":
           p = (args or {}).get("path")
@@ -360,18 +365,40 @@ def make_after_tool_callback(cfg: GemCodeConfig):
     except Exception:
       pass
 
+    # Aggregate per-turn tool-result budget: if we already exceeded the budget,
+    # tighten caps further for the rest of this user message.
+    try:
+      if tool_context is not None:
+        st = tool_context.state
+        if bool(st.get(_TOOL_GROUP_EXCEEDED, False)):
+          effective_tool_chars = max(1500, int(effective_tool_chars * 0.5))
+    except Exception:
+      pass
+
     if (
       isinstance(tool_response, dict)
       and getattr(cfg, "tool_result_offload_enabled", False)
       and effective_tool_chars > 0
     ):
       try:
-        from gemcode.tool_result_store import maybe_offload_tool_result
-        new_payload, did = maybe_offload_tool_result(
+        from gemcode.tool_result_store import maybe_offload_tool_result_stable
+        seq = None
+        st = None
+        try:
+          if tool_context is not None:
+            st = tool_context.state
+            seq = int(st.get(_TOOL_SEQ, 0) or 0)
+        except Exception:
+          st = None
+          seq = None
+        new_payload, did = maybe_offload_tool_result_stable(
           project_root=cfg.project_root,
           tool_name=name,
+          args=args or {},
           payload=tool_response,
           max_inline_chars=int(effective_tool_chars),
+          state=st,
+          seq=seq,
         )
         if did and isinstance(new_payload, dict):
           tool_response = new_payload
@@ -392,6 +419,19 @@ def make_after_tool_callback(cfg: GemCodeConfig):
       st = tool_context.state
     except Exception:
       return tool_response if (truncated or offloaded) else None
+
+    # Update aggregate per-turn budget counters (best-effort).
+    try:
+      from gemcode.context_budget import estimate_obj_string_chars
+      budget = int(getattr(cfg, "tool_result_group_budget_chars", 0) or 0)
+      if budget > 0:
+        used = int(st.get(_TOOL_GROUP_CHARS, 0) or 0)
+        used += int(estimate_obj_string_chars(tool_response))
+        st[_TOOL_GROUP_CHARS] = used
+        if used >= budget:
+          st[_TOOL_GROUP_EXCEEDED] = True
+    except Exception:
+      pass
     err = isinstance(tool_response, dict) and tool_response.get("error")
     err_kind = (
       isinstance(tool_response, dict) and tool_response.get("error_kind")

{gemcode-0.3.70 → gemcode-0.3.71}/src/gemcode/config.py

@@ -60,6 +60,10 @@ def token_budget_invocation_reset() -> dict:
       "gemcode:bt_t0": t,
       "gemcode:bt_base_total_tokens": -1,
       "gemcode:bt_token_budget_stop": False,
+      # Tool-result aggregate budget (per user message)
+      "gemcode:tool_group_chars": 0,
+      "gemcode:tool_group_budget_exceeded": False,
+      "gemcode:tool_seq": 0,
   }
 
 
@@ -131,6 +135,12 @@ class GemCodeConfig:
     )
   )
 
+  # Aggregate tool-result budget per user message (approx. characters of tool payloads).
+  # When exceeded, GemCode tightens subsequent tool output caps for the remainder of the turn.
+  tool_result_group_budget_chars: int = field(
+    default_factory=lambda: int(os.environ.get("GEMCODE_TOOL_RESULT_GROUP_BUDGET_CHARS", "60000"))
+  )
+
   # When enabled, oversized tool outputs are offloaded to disk under
   # .gemcode/tool-results/ and replaced in history with stable refs + previews.
   # This reduces context bloat and improves prompt-cache stability.

{gemcode-0.3.70 → gemcode-0.3.71}/src/gemcode/tool_result_store.py

@@ -18,6 +18,73 @@ from pathlib import Path
 from typing import Any
 
 _REF_PREFIX = "tool_result:"
+_REPL_STATE_KEY = "gemcode:tool_replacement_state"
+
+
+def _stable_key(tool_name: str, args: dict[str, Any] | None, seq: int | None) -> str:
+  """
+  Build a stable key for replacement decisions.
+
+  ADK does not expose tool_use_id directly to callbacks in all versions, so we use:
+  - per-turn tool sequence number (preferred when available)
+  - tool name
+  - a stable hash of args (best-effort)
+  """
+  import json
+  try:
+    args_s = json.dumps(args or {}, sort_keys=True, ensure_ascii=False)
+  except Exception:
+    args_s = str(args or {})
+  b = (f"{seq or 0}:{tool_name}:{args_s}").encode("utf-8", errors="replace")
+  return _sha256_bytes(b)
+
+
+def maybe_offload_tool_result_stable(
+  *,
+  project_root: Path,
+  tool_name: str,
+  args: dict[str, Any] | None,
+  payload: Any,
+  max_inline_chars: int,
+  state: dict[str, Any] | None,
+  seq: int | None,
+) -> tuple[Any, bool]:
+  """
+  Stable offload wrapper.
+
+  - If we've already processed an identical tool call in this session, we re-apply
+    the exact same replacement structure to preserve prompt byte stability.
+  - Otherwise, we apply offload and remember the replacement result.
+  """
+  if state is None:
+    return maybe_offload_tool_result(
+      project_root=project_root,
+      tool_name=tool_name,
+      payload=payload,
+      max_inline_chars=max_inline_chars,
+    )
+
+  repl_state = state.get(_REPL_STATE_KEY)
+  if not isinstance(repl_state, dict):
+    repl_state = {}
+    state[_REPL_STATE_KEY] = repl_state
+
+  key = _stable_key(tool_name, args, seq)
+  if key in repl_state:
+    return repl_state[key], False
+
+  new_payload, did = maybe_offload_tool_result(
+    project_root=project_root,
+    tool_name=tool_name,
+    payload=payload,
+    max_inline_chars=max_inline_chars,
+  )
+  if did:
+    repl_state[key] = new_payload
+  else:
+    # Freeze "no replacement" decision too (prevents later shape drift).
+    repl_state[key] = payload
+  return new_payload, did
 
 
 def _store_dir(project_root: Path) -> Path:

{gemcode-0.3.70 → gemcode-0.3.71/src/gemcode.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gemcode
-Version: 0.3.70
+Version: 0.3.71
 Summary: Local-first coding agent on Google Gemini + ADK
 Author: GemCode Contributors
 License:                                  Apache License
@@ -217,6 +217,16 @@ gemcode --yes "Add a module docstring to src/foo.py"
 gemcode --session mysess --yes "Continue: run tests and fix failures"
 ```
 
+### What GemCode writes to `.gemcode/`
+
+GemCode keeps project-local state under `.gemcode/`:
+
+- **`sessions.sqlite`**: session events/history (ADK `SqliteSessionService`)
+- **`audit.log`**: JSONL audit trail for tool usage + model usage + stop reasons
+- **`tool-results/`**: oversized tool outputs offloaded to stable refs (`tool_result:<sha>`)
+- **`artifacts/`**: file artifacts (ADK `FileArtifactService`)
+- **`policy.json`**: self-tuning per-repo profile used to calibrate dynamic budgets
+
 - **`--yes`**: allow mutating tools (`write_file`, `search_replace`). Shell execution is still restricted by the `.env.example` allowlist.
 - **`--session`**: Conversation history is stored under `.gemcode/sessions.sqlite` (ADK `SqliteSessionService`). Reuse the same `--session` id to continue.
 - **`--max-llm-calls`**: cap model↔tool iterations for this message (maps to ADK `RunConfig.max_llm_calls`). You can also set `GEMCODE_MAX_LLM_CALLS`.
@@ -243,6 +253,33 @@ gemcode --session mysess --yes "Continue: run tests and fix failures"
 - **Recovery-loop**: ADK `ReflectAndRetryToolPlugin`-based retries on tool failures.
   - Set `GEMCODE_ENABLE_TOOL_RECOVERY_RETRY=0` to disable.
   - Set `GEMCODE_TOOL_REFLECT_MAX_RETRIES=1` to control retries per tool.
+
+### Token efficiency (dynamic + intelligent)
+
+GemCode optimizes tokens without losing capability by using a dynamic policy:
+
+- **Context-pressure aware**: tool caps tighten when context is tight, loosen when there is room.
+- **Risk/complexity aware**: caps increase for risky tasks (writes, shell, failures, many files).
+- **Self-tuning per repo**: `.gemcode/policy.json` calibrates baseline evidence budgets over time.
+
+Key env toggles:
+
+- `GEMCODE_DYNAMIC_TOKEN_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_POLICY=0|1`
+- `GEMCODE_DYNAMIC_RISK_BOOST=<float>` (default `0.6`)
+- `GEMCODE_TOOL_RESULT_OFFLOAD=0|1` (default `1`)
+
+Live telemetry:
+
+- `/status` shows `risk_score`, `context_percent_left`, and profile EMAs.
+
+### Stable tool output offloading (OpenClaude-style)
+
+Oversized tool outputs are automatically offloaded and replaced with stable refs:
+
+- Stored in: `.gemcode/tool-results/`
+- References: `tool_result:<sha256>`
+- Load on demand: `load_tool_result(ref)`
 - **Gemini thinking controls (Claude-like)**:
   - By default GemCode lets Gemini use its dynamic/adaptive thinking behavior.
   - Set `GEMCODE_DISABLE_THINKING=1` to force a best-effort “low thinking” mode:
@@ -301,12 +338,24 @@ the user’s project.
   - `list_directory`
   - `glob_files`
   - `grep_content`
+  - `repo_map`
+  - `web_search`
+  - `web_fetch`
+  - `notebook_read`
+  - `notebook_edit`
 - Mutating tools (require `--yes` unless your policy blocks them):
   - `write_file`
   - `search_replace`
 - Shell execution:
   - `run_command` (guarded by `GEMCODE_ALLOW_COMMANDS` from `.env.example` and
     `GEMCODE_PERMISSION_MODE`).
+  - `bash` (pipelines + redirects; supports `background=True`)
+
+- Background task management (for processes started via `bash(..., background=True)`):
+  - `list_tasks`, `task_output`, `kill_task`
+
+- Tool offload loader:
+  - `load_tool_result(ref)`
 
 Tool execution is still controlled by permission gates and then governed by
 GemCode’s circuit breaker + recovery behavior.
@@ -434,6 +483,23 @@ pip install -e ".[dev]"
 pytest
 ```
 
+## Release workflow (GitHub Actions → PyPI)
+
+This repository publishes to PyPI on tag pushes:
+
+- `.github/workflows/publish-pypi.yml` triggers on tags matching `v*`
+
+Typical release steps:
+
+```bash
+# 1) bump gemcode/pyproject.toml version
+git add -A
+git commit -m "release: vX.Y.Z"
+git tag -a vX.Y.Z -m "vX.Y.Z"
+git push origin HEAD
+git push origin vX.Y.Z
+```
+
 ## References (local only)
 
 Do not commit proprietary leaked trees into this package. Keep `claude-code-leaked/` and similar folders outside version control or in a private mirror.