@oneciel-ai/claude-any 0.1.34 → 0.1.36

package/README.md

@@ -48,7 +48,7 @@ arguments through unchanged.
 
 Credits: One Ciel LLC
 
-Current version: `0.1.34`
+Current version: `0.1.36`
 
 ## Why This Exists
 
@@ -381,6 +381,23 @@ steps under that larger model's supervision.
 
 ## Changelog
 
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router calls now use upstream
+  `stream=true`, so long responses can flow as chunks instead of waiting for a
+  full non-streaming completion.
+- **Stream retry diagnostics**: streamed NVIDIA calls keep the same retry and
+  request-size activity status used by the statusline.
+
+### 0.1.35
+
+- **NVIDIA router context guard**: NVIDIA hosted now defaults to a 32K router
+  context window and LLM presets may tune that cap, reducing timeout-prone
+  payload growth in long Claude Code sessions.
+- **Upstream activity status**: the router records current request, retry,
+  success, and error state with estimated token/byte size so the statusline can
+  distinguish active upstream waits from idle sessions.
+
 ### 0.1.34
 
 - **Complete headless configuration path**: add `--ca-env-file`,

package/claude_any.py

@@ -39,8 +39,9 @@ LOG_LEVEL_PATH = CONFIG_DIR / "log-level"
 REQUEST_DUMP_PATH = CONFIG_DIR / "requests.jsonl"
 RESPONSE_DUMP_PATH = CONFIG_DIR / "responses.jsonl"
 TOOL_CALL_LOG_PATH = CONFIG_DIR / "tool-calls.jsonl"
-RATE_LIMIT_STATE_PATH = CONFIG_DIR / "rate-limit-state.json"
-CHAT_MESSAGES_PATH = CONFIG_DIR / "chat-messages.jsonl"
+RATE_LIMIT_STATE_PATH = CONFIG_DIR / "rate-limit-state.json"
+ROUTER_ACTIVITY_PATH = CONFIG_DIR / "router-activity.json"
+CHAT_MESSAGES_PATH = CONFIG_DIR / "chat-messages.jsonl"
 CHAT_FILES_DIR = CONFIG_DIR / "chat-files"
 PLAN_ARTIFACTS_DIR = CONFIG_DIR / "plan-artifacts"
 PID_PATH = CONFIG_DIR / "router.pid"
@@ -84,7 +85,7 @@ PROVIDER_LABELS = {
     "self-hosted-nim": "Self Hosted NIM",
 }
 APP_NAME = "Claude Any"
-VERSION = "0.1.34"
+VERSION = "0.1.36"
 CREDITS = "Credits: One Ciel LLC"
 
 LOG_LEVELS = {"SILENT": 0, "ERROR": 1, "WARN": 2, "INFO": 3, "DEBUG": 4, "TRACE": 5}
@@ -712,17 +713,18 @@ DEFAULT_CONFIG: dict[str, Any] = {
             "stream_enabled": True,
             "stream_word_chunking": False,
         },
-        "nvidia-hosted": {
-            "base_url": "https://integrate.api.nvidia.com/v1",
-            "api_key": "not-used",
-            "current_model": "qwen/qwen3-coder-480b-a35b-instruct",
+        "nvidia-hosted": {
+            "base_url": "https://integrate.api.nvidia.com/v1",
+            "api_key": "not-used",
+            "current_model": "qwen/qwen3-coder-480b-a35b-instruct",
             "advisor_model": "",
             "custom_models": [],
             "native_compat": False,
-            "rate_limit_rpm": 40,
-            "rate_limit_status": True,
-            "max_output_tokens": 4096,
-            "temperature": 0.7,
+            "rate_limit_rpm": 40,
+            "rate_limit_status": True,
+            "context_window": 32768,
+            "max_output_tokens": 4096,
+            "temperature": 0.7,
             "top_p": 0.8,
             "request_timeout_ms": 300000,
             "stream_enabled": True,
@@ -773,14 +775,21 @@ def apply_config_migrations(cfg: dict[str, Any]) -> None:
             pcfg["native_compat"] = False
         migrations[marker] = True
 
-    marker = "default_timeout_5m_20260513"
-    if not migrations.get(marker):
-        for pcfg in (cfg.get("providers") or {}).values():
-            if not isinstance(pcfg, dict):
-                continue
-            if positive_int(pcfg.get("request_timeout_ms")) in (600000, 1800000):
-                pcfg["request_timeout_ms"] = 300000
-        migrations[marker] = True
+    marker = "default_timeout_5m_20260513"
+    if not migrations.get(marker):
+        for pcfg in (cfg.get("providers") or {}).values():
+            if not isinstance(pcfg, dict):
+                continue
+            if positive_int(pcfg.get("request_timeout_ms")) in (600000, 1800000):
+                pcfg["request_timeout_ms"] = 300000
+        migrations[marker] = True
+
+    marker = "nvidia_context_window_32k_20260513"
+    if not migrations.get(marker):
+        pcfg = cfg.get("providers", {}).get("nvidia-hosted", {})
+        if isinstance(pcfg, dict) and not positive_int(pcfg.get("context_window")):
+            pcfg["context_window"] = 32768
+        migrations[marker] = True
 
 
 _config_cache: dict[str, Any] | None = None
@@ -1174,8 +1183,9 @@ from pathlib import Path
 HOME = Path.home()
 CONFIG_DIR = Path(os.environ.get("CLAUDE_ANY_CONFIG_DIR") or (HOME / ".config" / "claude-any"))
 CONFIG_PATH = CONFIG_DIR / "config.json"
-STATE_PATH = CONFIG_DIR / "rate-limit-state.json"
-PALETTE = (203, 209, 215, 221, 229, 187, 151, 116, 111, 147, 183, 219)
+STATE_PATH = CONFIG_DIR / "rate-limit-state.json"
+ACTIVITY_PATH = CONFIG_DIR / "router-activity.json"
+PALETTE = (203, 209, 215, 221, 229, 187, 151, 116, 111, 147, 183, 219)
 
 
 def load_json(path, default):
@@ -1225,8 +1235,9 @@ def main():
         rpm = int(raw_rpm)
     except Exception:
         rpm = 40
-    state = load_json(STATE_PATH, {})
-    now = time.time()
+    state = load_json(STATE_PATH, {})
+    activity = load_json(ACTIVITY_PATH, {})
+    now = time.time()
     key = f"{provider}:__global__" if provider else ""
     entry = state.get(key) if key else None
     if not isinstance(entry, dict):
@@ -1288,9 +1299,25 @@ def main():
             rpm_text += " | server " + ", ".join(parts)
     if penalty_until > now:
         rpm_text += f" | wait {max(0.0, penalty_until - now):.0f}s"
-    elif last_wait >= 0.5 and 0.0 <= now - updated_at < 60.0:
-        rpm_text += f" | wait {last_wait:.1f}s"
-    print(f"{left} | {color(rpm_text)}")
+    elif last_wait >= 0.5 and 0.0 <= now - updated_at < 60.0:
+        rpm_text += f" | wait {last_wait:.1f}s"
+    if isinstance(activity, dict):
+        try:
+            age = now - float(activity.get("updated_at") or 0)
+        except Exception:
+            age = 999999
+        if 0 <= age < 180:
+            event = str(activity.get("event") or "")
+            if event == "retry":
+                rpm_text += f" | retry {activity.get('attempt')}/{activity.get('total')}"
+            elif event == "request":
+                tokens = activity.get("tokens")
+                rpm_text += f" | upstream {age:.0f}s"
+                if tokens:
+                    rpm_text += f" {tokens}tok"
+            elif event in ("success", "error"):
+                rpm_text += f" | {event} {age:.0f}s"
+    print(f"{left} | {color(rpm_text)}")
 
 
 if __name__ == "__main__":
@@ -2834,16 +2861,34 @@ def native_anthropic_base_url(provider: str, pcfg: dict[str, Any]) -> str:
     return base
 
 
-def write_json(handler: BaseHTTPRequestHandler, obj: Any, status: int = 200) -> None:
-    body = json.dumps(obj).encode("utf-8")
-    handler.send_response(status)
-    handler.send_header("content-type", "application/json")
-    handler.send_header("content-length", str(len(body)))
-    handler.end_headers()
-    handler.wfile.write(body)
-
-
-def write_text_response(handler: BaseHTTPRequestHandler, text: str, status: int = 200, content_type: str = "text/plain; charset=utf-8") -> None:
+def write_json(handler: BaseHTTPRequestHandler, obj: Any, status: int = 200) -> None:
+    body = json.dumps(obj).encode("utf-8")
+    handler.send_response(status)
+    handler.send_header("content-type", "application/json")
+    handler.send_header("content-length", str(len(body)))
+    handler.end_headers()
+    handler.wfile.write(body)
+
+
+def write_router_activity(event: str, provider: str, model: str | None = None, **fields: Any) -> None:
+    try:
+        CONFIG_DIR.mkdir(parents=True, exist_ok=True)
+        data = {
+            "updated_at": time.time(),
+            "time": time.strftime("%Y-%m-%dT%H:%M:%S"),
+            "event": event,
+            "provider": provider,
+            "model": model or "",
+        }
+        data.update(fields)
+        tmp = ROUTER_ACTIVITY_PATH.with_name(f"{ROUTER_ACTIVITY_PATH.name}.{os.getpid()}.{time.time_ns()}.tmp")
+        tmp.write_text(json.dumps(data, ensure_ascii=False, separators=(",", ":")), encoding="utf-8")
+        tmp.replace(ROUTER_ACTIVITY_PATH)
+    except Exception:
+        pass
+
+
+def write_text_response(handler: BaseHTTPRequestHandler, text: str, status: int = 200, content_type: str = "text/plain; charset=utf-8") -> None:
     body = text.encode("utf-8")
     handler.send_response(status)
     handler.send_header("content-type", content_type)
@@ -3556,11 +3601,20 @@ def cap_output_tokens_for_context(
     return max(1, min(configured, available))
 
 
-def ollama_context_limit_for_budget(pcfg: dict[str, Any]) -> int:
-    raw = pcfg.get("num_ctx", "auto")
-    if isinstance(raw, str) and raw.strip().lower() == "auto":
-        return positive_int(pcfg.get("num_ctx_max")) or 65536
-    return positive_int(raw) or positive_int(pcfg.get("num_ctx_max")) or 65536
+def ollama_context_limit_for_budget(pcfg: dict[str, Any]) -> int:
+    raw = pcfg.get("num_ctx", "auto")
+    if isinstance(raw, str) and raw.strip().lower() == "auto":
+        return positive_int(pcfg.get("num_ctx_max")) or 65536
+    return positive_int(raw) or positive_int(pcfg.get("num_ctx_max")) or 65536
+
+
+def openai_context_limit_for_budget(provider: str, pcfg: dict[str, Any]) -> int:
+    configured = positive_int(pcfg.get("context_window")) or positive_int(pcfg.get("max_model_len"))
+    if configured:
+        return configured
+    if provider == "nvidia-hosted":
+        return 32768
+    return 65536
 
 
 def compact_ollama_messages_for_budget(
@@ -3695,10 +3749,10 @@ def ollama_chat_request(model: str, body: dict[str, Any], pcfg: dict[str, Any],
     return req
 
 
-def openai_compatible_chat_request(model: str, body: dict[str, Any], pcfg: dict[str, Any], stream: bool = False) -> dict[str, Any]:
-    messages = anthropic_messages_to_openai(body)
-    tools = anthropic_tools_to_ollama(body.get("tools"))
-    context_limit = positive_int(pcfg.get("context_window")) or positive_int(pcfg.get("max_model_len")) or 65536
+def openai_compatible_chat_request(provider: str, model: str, body: dict[str, Any], pcfg: dict[str, Any], stream: bool = False) -> dict[str, Any]:
+    messages = anthropic_messages_to_openai(body)
+    tools = anthropic_tools_to_ollama(body.get("tools"))
+    context_limit = openai_context_limit_for_budget(provider, pcfg)
     configured = configured_output_tokens(pcfg, body)
     reserve = positive_int(pcfg.get("context_reserve_tokens")) or 1024
     output_reserve = configured or positive_int(body.get("max_tokens")) or 4096
@@ -4525,7 +4579,7 @@ def forward_ollama_api_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg
     write_json(handler, message)
 
 
-def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
+def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
     choice = {}
     choices = data.get("choices")
     if isinstance(choices, list) and choices:
@@ -4538,10 +4592,202 @@ def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict
         },
         "done_reason": "length" if choice.get("finish_reason") == "length" else "stop",
     }
-    return ollama_chat_to_anthropic(wrapped, model, source_body=source_body)
-
-
-def upstream_http_error_message(exc: urllib.error.HTTPError, raw: str | None = None) -> str:
+    return ollama_chat_to_anthropic(wrapped, model, source_body=source_body)
+
+
+def stream_openai_chat_to_anthropic_sse(
+    handler: BaseHTTPRequestHandler,
+    resp: Any,
+    model: str,
+    provider: str,
+    source_body: dict[str, Any] | None = None,
+    start_index: int = 0,
+    word_chunking: bool = False,
+) -> None:
+    next_content_index = start_index
+    text_started = False
+    text_suppressed_for_plan = False
+    text_index: int | None = None
+    text_so_far = ""
+    text_buffer = ""
+    tool_fragments: dict[int, dict[str, Any]] = {}
+    output_tokens = 0
+    finish_reason = "stop"
+
+    def emit(event_name: str, payload: dict[str, Any]) -> None:
+        handler.wfile.write(f"event: {event_name}\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n".encode())
+        handler.wfile.flush()
+
+    def ensure_text_started() -> int:
+        nonlocal text_started, text_index, next_content_index
+        if text_started and text_index is not None:
+            return text_index
+        text_started = True
+        text_index = next_content_index
+        next_content_index += 1
+        emit(
+            "content_block_start",
+            {"type": "content_block_start", "index": text_index, "content_block": {"type": "text", "text": ""}},
+        )
+        return text_index
+
+    def emit_text_delta(text: str) -> None:
+        if not text:
+            return
+        idx = ensure_text_started()
+        emit(
+            "content_block_delta",
+            {"type": "content_block_delta", "index": idx, "delta": {"type": "text_delta", "text": text}},
+        )
+
+    try:
+        for raw_line in resp:
+            line = raw_line.decode("utf-8", errors="ignore").strip()
+            if not line or line.startswith(":"):
+                continue
+            if line.startswith("data:"):
+                line = line[5:].strip()
+            if not line or line == "[DONE]":
+                break
+            try:
+                event = json.loads(line)
+            except Exception:
+                continue
+            if not isinstance(event, dict):
+                continue
+            usage = event.get("usage")
+            if isinstance(usage, dict):
+                output_tokens = max(output_tokens, positive_int(usage.get("completion_tokens")) or 0)
+            choices = event.get("choices")
+            if not isinstance(choices, list) or not choices:
+                continue
+            choice = choices[0] if isinstance(choices[0], dict) else {}
+            if choice.get("finish_reason"):
+                finish_reason = str(choice.get("finish_reason"))
+            delta = choice.get("delta") if isinstance(choice.get("delta"), dict) else {}
+            text_chunk = delta.get("content") or ""
+            if text_chunk:
+                if source_body is not None and not text_started and not tool_fragments and should_auto_enter_plan_mode(source_body, text_so_far + text_chunk, []):
+                    text_so_far += text_chunk
+                    text_suppressed_for_plan = True
+                    continue
+                text_so_far += text_chunk
+                if word_chunking:
+                    text_buffer += text_chunk
+                    to_flush, text_buffer = _split_word_buffer(text_buffer, force=False)
+                    emit_text_delta(to_flush)
+                else:
+                    emit_text_delta(text_chunk)
+            for call in delta.get("tool_calls") or []:
+                if not isinstance(call, dict):
+                    continue
+                try:
+                    call_index = int(call.get("index"))
+                except Exception:
+                    call_index = len(tool_fragments)
+                slot = tool_fragments.setdefault(call_index, {"id": "", "name": "", "arguments": ""})
+                if call.get("id"):
+                    slot["id"] = str(call.get("id"))
+                fn = call.get("function") if isinstance(call.get("function"), dict) else {}
+                if fn.get("name"):
+                    slot["name"] += str(fn.get("name"))
+                if fn.get("arguments"):
+                    slot["arguments"] += str(fn.get("arguments"))
+        if word_chunking and text_buffer:
+            to_flush, text_buffer = _split_word_buffer(text_buffer, force=True)
+            emit_text_delta(to_flush)
+
+        tool_calls: list[dict[str, Any]] = []
+        for _, fragment in sorted(tool_fragments.items()):
+            raw_name = str(fragment.get("name") or "")
+            if not raw_name:
+                continue
+            matched_name = _fuzzy_match_tool_name(raw_name) or raw_name
+            normalized_args = normalize_tool_arguments(matched_name, fragment.get("arguments") or {})
+            fixed_input = _validate_and_fix_tool_input(matched_name, normalized_args)
+            if source_body is not None:
+                matched_name, fixed_input = plan_mode_tool_name_for_emit(source_body, matched_name, fixed_input)
+                if matched_name is None:
+                    continue
+            tool_calls.append({"function": {"name": matched_name, "arguments": fixed_input}})
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_id = str(fragment.get("id") or f"toolu_openai_{int(time.time() * 1000)}_{tool_index}")
+            append_tool_call_log(
+                "openai_stream_tool_call",
+                {
+                    "model": model,
+                    "raw_name": raw_name,
+                    "matched_name": matched_name,
+                    "raw_arguments": fragment.get("arguments"),
+                    "emitted_input": fixed_input,
+                    "sse_index": tool_index,
+                },
+            )
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": tool_id, "name": matched_name, "input": {}},
+                },
+            )
+            emit(
+                "content_block_delta",
+                {
+                    "type": "content_block_delta",
+                    "index": tool_index,
+                    "delta": {"type": "input_json_delta", "partial_json": json.dumps(fixed_input, ensure_ascii=False)},
+                },
+            )
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+
+        if source_body is not None and should_auto_enter_plan_mode(source_body, text_so_far, tool_calls):
+            router_log("WARN", "auto-synthesized EnterPlanMode from short/empty upstream OpenAI stream")
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_calls.append({"function": {"name": "EnterPlanMode", "arguments": {}}})
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": f"toolu_openai_plan_{int(time.time() * 1000)}", "name": "EnterPlanMode", "input": {}},
+                },
+            )
+            emit("content_block_delta", {"type": "content_block_delta", "index": tool_index, "delta": {"type": "input_json_delta", "partial_json": "{}"}})
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+        elif text_suppressed_for_plan and text_so_far:
+            emit_text_delta(text_so_far)
+
+        if source_body is not None and should_keep_work_alive_with_tasklist(source_body, text_so_far, tool_calls):
+            router_log("WARN", "auto-synthesized TaskList to keep work moving after OpenAI stream")
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_calls.append({"function": {"name": "TaskList", "arguments": {}}})
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": f"toolu_openai_keepalive_{int(time.time() * 1000)}", "name": "TaskList", "input": {}},
+                },
+            )
+            emit("content_block_delta", {"type": "content_block_delta", "index": tool_index, "delta": {"type": "input_json_delta", "partial_json": "{}"}})
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+
+        if text_started and text_index is not None:
+            emit("content_block_stop", {"type": "content_block_stop", "index": text_index})
+        stop_reason = "tool_use" if tool_calls else ("max_tokens" if finish_reason == "length" else "end_turn")
+        write_anthropic_open_stream_stop(handler, {"stop_reason": stop_reason, "usage": {"output_tokens": output_tokens or max(1, len(text_so_far) // 4)}})
+    finally:
+        try:
+            resp.close()
+        except Exception:
+            pass
+
+
+def upstream_http_error_message(exc: urllib.error.HTTPError, raw: str | None = None) -> str:
     if raw is None:
         raw = exc.read().decode("utf-8", errors="ignore")
     msg = raw.strip() or str(exc)
@@ -4582,7 +4828,7 @@ def retryable_timeout_exception(exc: BaseException) -> bool:
     return "timed out" in text or "timeout" in text
 
 
-def post_json_with_rate_retry(
+def post_json_with_rate_retry(
     url: str,
     req_body: Any,
     headers: dict[str, str],
@@ -4592,81 +4838,180 @@ def post_json_with_rate_retry(
     model: str,
     retry_notice: Callable[[str], None] | None = None,
 ) -> Any:
-    gateway_retries = positive_int(pcfg.get("gateway_retries")) or 2
-    max_attempts = max(1, gateway_retries + 1)
-    for attempt in range(max_attempts):
-        try:
-            data_bytes = json.dumps(req_body).encode("utf-8")
-            req = urllib.request.Request(url, data=data_bytes, headers=headers, method="POST")
-            with urllib.request.urlopen(req, timeout=timeout) as resp:
-                learn_router_rate_limit_headers(provider, pcfg, model, resp.headers)
-                return json.loads(resp.read().decode("utf-8"))
-        except urllib.error.HTTPError as exc:
-            raw = exc.read().decode("utf-8", errors="ignore")
-            learn_router_rate_limit_headers(provider, pcfg, model, exc.headers)
+    gateway_retries = positive_int(pcfg.get("gateway_retries")) or 2
+    max_attempts = max(1, gateway_retries + 1)
+    token_estimate = estimate_tokens(req_body)
+    byte_estimate = len(json.dumps(req_body, ensure_ascii=False).encode("utf-8"))
+    for attempt in range(max_attempts):
+        try:
+            write_router_activity(
+                "request",
+                provider,
+                model,
+                attempt=attempt + 1,
+                total=max_attempts,
+                tokens=token_estimate,
+                bytes=byte_estimate,
+                timeout=timeout,
+            )
+            router_log("INFO", f"upstream_request provider={provider} model={model} attempt={attempt + 1}/{max_attempts} tokens={token_estimate} bytes={byte_estimate} timeout={timeout}")
+            data_bytes = json.dumps(req_body).encode("utf-8")
+            req = urllib.request.Request(url, data=data_bytes, headers=headers, method="POST")
+            with urllib.request.urlopen(req, timeout=timeout) as resp:
+                learn_router_rate_limit_headers(provider, pcfg, model, resp.headers)
+                data = json.loads(resp.read().decode("utf-8"))
+                write_router_activity("success", provider, model, attempt=attempt + 1, tokens=token_estimate, bytes=byte_estimate)
+                return data
+        except urllib.error.HTTPError as exc:
+            raw = exc.read().decode("utf-8", errors="ignore")
+            learn_router_rate_limit_headers(provider, pcfg, model, exc.headers)
             if exc.code == 429 and attempt == 0:
                 wait = register_router_rate_limit_backoff(provider, pcfg, model, exc.headers.get("Retry-After"))
                 time.sleep(wait)
                 continue
-            if exc.code in UPSTREAM_RETRY_HTTP_CODES and attempt + 1 < max_attempts:
-                retry_no = attempt + 1
-                if retry_notice:
-                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
-                time.sleep(upstream_retry_wait_seconds(retry_no))
-                continue
-            raise RuntimeError(upstream_http_error_message(exc, raw)) from exc
-        except (TimeoutError, urllib.error.URLError) as exc:
-            if retryable_timeout_exception(exc) and attempt + 1 < max_attempts:
-                retry_no = attempt + 1
-                if retry_notice:
-                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
-                time.sleep(upstream_retry_wait_seconds(retry_no))
-                continue
-            raise RuntimeError(f"{type(exc).__name__}: {exc}") from exc
-    raise RuntimeError("upstream request failed")
-
-
-def forward_openai_compatible_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg: dict[str, Any], body: dict[str, Any]) -> None:
-    _update_tool_schema_registry(body.get("tools"))
-    model = resolve_requested_model(provider, pcfg, body.get("model"))
-    if provider == "nvidia-hosted":
-        model = ncp_model_id_for_nvidia_hosted(model)
-    req_body = openai_compatible_chat_request(model, body, pcfg, stream=False)
-    url = join_url(provider_upstream_request_base(provider, pcfg), "/chat/completions")
-    waited, rpm_used, rpm_limit = apply_router_rate_limit(provider, pcfg, model)
-    stream = bool(body.get("stream", True))
-    notice = rate_limit_notice(waited, rpm_used, rpm_limit, bool(pcfg.get("rate_limit_status", True)))
-    if stream:
-        write_anthropic_open_stream_start(handler, model)
-        index = 0
-        if notice:
-            index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": notice}], index)
-        try:
+            if exc.code in UPSTREAM_RETRY_HTTP_CODES and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, code=exc.code, tokens=token_estimate, bytes=byte_estimate)
+                router_log("WARN", f"upstream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} code={exc.code} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, code=exc.code, tokens=token_estimate, bytes=byte_estimate)
+            raise RuntimeError(upstream_http_error_message(exc, raw)) from exc
+        except (TimeoutError, urllib.error.URLError) as exc:
+            if retryable_timeout_exception(exc) and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate)
+                router_log("WARN", f"upstream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} error={type(exc).__name__} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate)
+            raise RuntimeError(f"{type(exc).__name__}: {exc}") from exc
+    raise RuntimeError("upstream request failed")
+
+
+def open_openai_stream_with_rate_retry(
+    url: str,
+    req_body: Any,
+    headers: dict[str, str],
+    timeout: float,
+    provider: str,
+    pcfg: dict[str, Any],
+    model: str,
+    retry_notice: Callable[[str], None] | None = None,
+) -> Any:
+    gateway_retries = positive_int(pcfg.get("gateway_retries")) or 2
+    max_attempts = max(1, gateway_retries + 1)
+    token_estimate = estimate_tokens(req_body)
+    byte_estimate = len(json.dumps(req_body, ensure_ascii=False).encode("utf-8"))
+    data_bytes = json.dumps(req_body).encode("utf-8")
+    for attempt in range(max_attempts):
+        try:
+            write_router_activity(
+                "request",
+                provider,
+                model,
+                attempt=attempt + 1,
+                total=max_attempts,
+                tokens=token_estimate,
+                bytes=byte_estimate,
+                timeout=timeout,
+                stream=True,
+            )
+            router_log("INFO", f"upstream_stream_request provider={provider} model={model} attempt={attempt + 1}/{max_attempts} tokens={token_estimate} bytes={byte_estimate} timeout={timeout}")
+            req = urllib.request.Request(url, data=data_bytes, headers=headers, method="POST")
+            resp = urllib.request.urlopen(req, timeout=timeout)
+            learn_router_rate_limit_headers(provider, pcfg, model, resp.headers)
+            return resp
+        except urllib.error.HTTPError as exc:
+            raw = exc.read().decode("utf-8", errors="ignore")
+            learn_router_rate_limit_headers(provider, pcfg, model, exc.headers)
+            if exc.code == 429 and attempt == 0:
+                wait = register_router_rate_limit_backoff(provider, pcfg, model, exc.headers.get("Retry-After"))
+                time.sleep(wait)
+                continue
+            if exc.code in UPSTREAM_RETRY_HTTP_CODES and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, code=exc.code, tokens=token_estimate, bytes=byte_estimate, stream=True)
+                router_log("WARN", f"upstream_stream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} code={exc.code} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, code=exc.code, tokens=token_estimate, bytes=byte_estimate, stream=True)
+            raise RuntimeError(upstream_http_error_message(exc, raw)) from exc
+        except (TimeoutError, urllib.error.URLError) as exc:
+            if retryable_timeout_exception(exc) and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate, stream=True)
+                router_log("WARN", f"upstream_stream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} error={type(exc).__name__} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate, stream=True)
+            raise RuntimeError(f"{type(exc).__name__}: {exc}") from exc
+    raise RuntimeError("upstream stream request failed")
+
+
+def forward_openai_compatible_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg: dict[str, Any], body: dict[str, Any]) -> None:
+    _update_tool_schema_registry(body.get("tools"))
+    model = resolve_requested_model(provider, pcfg, body.get("model"))
+    if provider == "nvidia-hosted":
+        model = ncp_model_id_for_nvidia_hosted(model)
+    url = join_url(provider_upstream_request_base(provider, pcfg), "/chat/completions")
+    waited, rpm_used, rpm_limit = apply_router_rate_limit(provider, pcfg, model)
+    stream = bool(body.get("stream", True))
+    notice = rate_limit_notice(waited, rpm_used, rpm_limit, bool(pcfg.get("rate_limit_status", True)))
+    if stream:
+        req_body = openai_compatible_chat_request(provider, model, body, pcfg, stream=True)
+        write_anthropic_open_stream_start(handler, model)
+        index = 0
+        if notice:
+            index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": notice}], index)
+        try:
             def emit_retry_notice(text: str) -> None:
                 nonlocal index
                 index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": text + "\n"}], index)
 
-            data = post_json_with_rate_retry(
-                url,
-                req_body,
-                provider_headers(provider, pcfg),
-                provider_request_timeout_seconds(pcfg),
-                provider,
+            resp = open_openai_stream_with_rate_retry(
+                url,
+                req_body,
+                provider_headers(provider, pcfg),
+                provider_request_timeout_seconds(pcfg),
+                provider,
                 pcfg,
-                model,
-                emit_retry_notice,
-            )
-        except RuntimeError as exc:
-            msg = str(exc)
-            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
-            write_anthropic_open_stream_stop(handler)
-            return
-        message = openai_chat_to_anthropic(data, model, source_body=body)
-        write_anthropic_stream_blocks(handler, list(message.get("content") or []), index)
-        write_anthropic_open_stream_stop(handler, message)
-        return
-    try:
-        data = post_json_with_rate_retry(
+                model,
+                emit_retry_notice,
+            )
+            stream_openai_chat_to_anthropic_sse(
+                handler,
+                resp,
+                model,
+                provider,
+                source_body=body,
+                start_index=index,
+                word_chunking=bool(pcfg.get("stream_word_chunking", False)),
+            )
+            write_router_activity("success", provider, model, tokens=estimate_tokens(req_body), bytes=len(json.dumps(req_body, ensure_ascii=False).encode("utf-8")), stream=True)
+        except RuntimeError as exc:
+            msg = str(exc)
+            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
+            write_anthropic_open_stream_stop(handler)
+            return
+        except Exception as exc:
+            msg = f"{type(exc).__name__}: {exc}"
+            write_router_activity("error", provider, model, error=type(exc).__name__, stream=True)
+            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
+            write_anthropic_open_stream_stop(handler)
+            return
+        return
+    req_body = openai_compatible_chat_request(provider, model, body, pcfg, stream=False)
+    try:
+        data = post_json_with_rate_retry(
             url,
             req_body,
             provider_headers(provider, pcfg),
@@ -5112,7 +5457,7 @@ def status_lines() -> list[str]:
         *([f"keep_alive: {pcfg.get('keep_alive', 'default')}"] if provider in ("ollama", "ollama-cloud") else []),
         *([f"think: {bool(pcfg.get('think', False))}"] if provider in ("ollama", "ollama-cloud") else []),
         *([f"request_timeout_ms: {pcfg.get('request_timeout_ms', 'default')}"] if provider in ("ollama", "ollama-cloud") else []),
-        *([f"context_window: {pcfg.get('context_window', 'default')}"] if provider in ("vllm", "self-hosted-nim") else []),
+        *([f"context_window: {pcfg.get('context_window', 'default')}"] if provider in ("vllm", "nvidia-hosted", "self-hosted-nim") else []),
         *([f"context_reserve_tokens: {pcfg.get('context_reserve_tokens', 'default')}"] if provider in ("vllm", "self-hosted-nim") else []),
         *([f"max_output_tokens: {pcfg.get('max_output_tokens', 'default')}"] if provider in ("vllm", "nvidia-hosted", "self-hosted-nim") else []),
         *([f"request_timeout_ms: {pcfg.get('request_timeout_ms', 'default')}"] if provider in ("vllm", "nvidia-hosted", "self-hosted-nim") else []),
@@ -5421,9 +5766,9 @@ def provider_options_status(provider: str, pcfg: dict[str, Any]) -> str:
             if limit is not None:
                 suffix = f"{used}/{limit}" if limit > 0 else f"{used}/min(unlimited)"
                 parts.append(f"rpm_used={suffix}")
-    if provider in ("vllm", "self-hosted-nim"):
-        parts.insert(0, f"context_window={pcfg.get('context_window', 'default')}")
-        parts.insert(1, f"reserve={pcfg.get('context_reserve_tokens', 'default')}")
+    if provider in ("vllm", "nvidia-hosted", "self-hosted-nim"):
+        parts.insert(0, f"context_window={pcfg.get('context_window', 'default')}")
+        parts.insert(1, f"reserve={pcfg.get('context_reserve_tokens', 'default')}")
     if provider in ("vllm", "self-hosted-nim"):
         native_default = False if provider == "nvidia-hosted" else True
         parts.append(f"native={bool(pcfg.get('native_compat', native_default))}")
@@ -5741,10 +6086,10 @@ def apply_llm_preset_to_provider(provider: str, pcfg: dict[str, Any], preset_id:
                 f"native={native_default}",
             ],
         }
-        for token in tokens_by_preset[preset_id]:
-            if provider == "nvidia-hosted" and token.startswith(("context_window=", "reserve=", "native=")):
-                continue
-            apply_provider_option(provider, pcfg, token)
+        for token in tokens_by_preset[preset_id]:
+            if provider == "nvidia-hosted" and token.startswith("native="):
+                continue
+            apply_provider_option(provider, pcfg, token)
         if server_limit:
             requested_context = positive_int(pcfg.get("context_window"))
             if requested_context and requested_context > server_limit:

package/docs/README.ja.md

@@ -47,7 +47,7 @@ vLLM、NVIDIA hosted、self-hosted NIM を選択し、通常の Claude Code 引
 
 Credits: One Ciel LLC
 
-現在のバージョン: `0.1.34`
+現在のバージョン: `0.1.36`
 
 ## 作られた理由
 
@@ -351,6 +351,23 @@ Windows/Linux 管理、クリーンアップスクリプト、定期的なセキ
 
 ## 変更履歴
 
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router 呼び出しは upstream にも
+  `stream=true` を使用します。長い応答を完全な non-streaming completion まで
+  待たず、chunk として流せます。
+- **Stream retry diagnostics**: streaming NVIDIA 呼び出しでも statusline 用の
+  retry/request size activity 状態を維持します。
+
+### 0.1.35
+
+- **NVIDIA router context guard**: NVIDIA hosted の router context 既定値を 32K
+  に下げ、LLM preset がこの cap を調整できるようにしました。長い Claude Code
+  セッションで payload が肥大して timeout する状況を減らします。
+- **Upstream activity status**: router が現在の request/retry/success/error
+  状態と推定 token/byte サイズを記録し、statusline で upstream 待機と idle を
+  判別できるようにしました。
+
 ### 0.1.34
 
 - **完全な headless 設定経路**: `--ca-env-file`、環境変数マッピング、Advisor

package/docs/README.ko.md

@@ -47,7 +47,7 @@ NVIDIA hosted, self-hosted NIM을 선택하고, Claude Code의 일반 인자는
 
 Credits: One Ciel LLC
 
-현재 버전: `0.1.34`
+현재 버전: `0.1.36`
 
 ## 왜 만들었나
 
@@ -351,6 +351,23 @@ Windows 이벤트 로그 리뷰, 바이러스/랜섬웨어 침입 시도 정리,
 
 ## 변경 이력
 
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router 호출은 이제 upstream에도
+  `stream=true`를 사용합니다. 긴 응답을 전체 완료까지 기다리지 않고 chunk로
+  흘려보내 timeout 가능성을 낮춥니다.
+- **Stream retry diagnostics**: streaming NVIDIA 호출도 statusline에서 쓰는
+  retry/request size activity 상태를 유지합니다.
+
+### 0.1.35
+
+- **NVIDIA router context guard**: NVIDIA hosted의 router context 기본값을 32K로
+  낮추고 LLM preset이 이 cap을 조정할 수 있게 하여, 긴 Claude Code 세션에서
+  payload가 커져 timeout이 나는 상황을 줄였습니다.
+- **Upstream activity status**: router가 현재 request/retry/success/error 상태와
+  추정 token/byte 크기를 기록하여, statusline에서 upstream 대기와 idle 상태를
+  구분할 수 있습니다.
+
 ### 0.1.34
 
 - **완전한 headless 설정 경로**: `--ca-env-file`, 환경변수 매핑, Advisor model,

package/docs/README.zh.md

@@ -47,7 +47,7 @@ NIM，并把普通 Claude Code 参数原样传递。
 
 Credits: One Ciel LLC
 
-当前版本: `0.1.34`
+当前版本: `0.1.36`
 
 ## 为什么存在
 
@@ -337,6 +337,21 @@ Hermes 格式模型或部分较旧的 Qwen tool template。
 
 ## 更新日志
 
+### 0.1.36
+
+- **NVIDIA upstream streaming**：NVIDIA hosted router 调用现在也会向 upstream
+  使用 `stream=true`，长响应可以按 chunk 流出，不再等待完整的非流式 completion。
+- **Stream retry diagnostics**：streaming NVIDIA 调用也保留 statusline 使用的
+  retry/request size activity 状态。
+
+### 0.1.35
+
+- **NVIDIA router context guard**：NVIDIA hosted 的 router context 默认值改为
+  32K，并允许 LLM preset 调整该 cap，减少长 Claude Code 会话中 payload 变大后
+  触发 timeout 的情况。
+- **Upstream activity status**：router 会记录当前 request/retry/success/error
+  状态和估算 token/byte 大小，statusline 可以区分正在等待 upstream 还是已 idle。
+
 ### 0.1.34
 
 - **完整 headless 配置路径**：新增 `--ca-env-file`、环境变量映射、Advisor

package/docs/manual.md

@@ -10,7 +10,7 @@ Code starts, while passing normal Claude Code arguments through unchanged.
 
 Credits: One Ciel LLC
 
-Current version: `0.1.34`
+Current version: `0.1.36`
 
 ## Install
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@oneciel-ai/claude-any",
-  "version": "0.1.34",
+  "version": "0.1.36",
   "description": "Claude Code provider selector for Anthropic, Ollama, Ollama Cloud, vLLM, NVIDIA hosted, and self-hosted NIM.",
   "license": "MIT",
   "author": "One Ciel LLC",