dsh-openai-shim 0.2.0__tar.gz

dsh_openai_shim-0.2.0/.dockerignore

@@ -0,0 +1,8 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+.pytest_cache/
+dist/
+build/
+.git/

dsh_openai_shim-0.2.0/.gitignore

@@ -0,0 +1,8 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+.pytest_cache/
+dist/
+build/
+.git/

dsh_openai_shim-0.2.0/CHANGELOG.md

@@ -0,0 +1,66 @@
+# dsh-openai-shim Changelog
+
+All notable changes to the `dsh-openai-shim` package are documented here.
+This project follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## [0.2.0] — 2026-09-21
+
+### Added
+- **Context-window-aware `max_tokens` (fixes the sglang/vLLM "context length" 400).**
+  Providers such as sglang/vLLM report `max_model_len` as the *total* context
+  window (input + output), not the output cap. A non-trivial prompt plus a large
+  `max_tokens` overflowed the window and 400'd with
+  *"exceeds the model's maximum context length of N tokens. You requested a
+  total of X tokens: I tokens from the input messages and N tokens for
+  completion."* This version handles that in two layers:
+  - **Proactive clamp** — `proxy.conf` now stores both `token_cap` (output) and
+    `context_window` (total); the shim clamps
+    `max_tokens = min(token_cap, context_window − est_input − 128)` so a long
+    prompt never overflows. `dsh-proxy use --base` auto-discovers **both**
+    limits from `/v1/models`.
+  - **Self-heal backstop** — if a context-window 400 still occurs, the shim
+    parses the server's *exact* input-token count from the error, retries once
+    with `context_window − input − 128`, and persists the discovered
+    `context_window` so the next request clamps proactively (no further 400).
+
+### Changed
+- `resolve_upstream` / `ShimConfig` now carry the context window alongside the
+  output cap (4-tuple resolution). The build-time deployment-agnostic gate is
+  updated for the 4-tuple.
+- Provider discovery reports both `token_cap` and `context_window`
+  (`discover_model_limits`); `discover_completion_cap` remains as a
+  back-compatible wrapper.
+
+### Fixed
+- Requests with a large prompt + a large `max_tokens` no longer fail against
+  sglang/vLLM endpoints (the second, context-window 400 that the 0.1.x
+  output-cap-only self-heal did not catch).
+
+## [0.1.1] — 2026-09
+
+### Added
+- **Per-provider completion-token cap, auto-discovered.** `dsh-proxy use --base`
+  probes `/v1/models` to learn the upstream's output limit and stores it as the
+  provider's `token_cap`; the shim clamps `max_tokens` to it and self-heals on a
+  *completion-cap* 400 (parses the server-reported cap, retries, persists).
+- **`dsh-proxy use` restarts the shim** (was stop-only, leaving a dead endpoint).
+
+### Fixed
+- `dsh-proxy status` exit code and the `serve` pidfile handling (two
+  boot-breaking bugs).
+- Default provider is now `litellm` (DiveAI 404s on `/chat/completions` and is
+  not a valid default).
+
+## [0.1.0] — 2026-09
+
+### Added
+- Initial release: a tiny, **zero-dependency (stdlib-only)** OpenAI-compatible
+  proxy that adapts DeepSeek Harness (dsh) to any OpenAI-compatible endpoint
+  (Socrates / LiteLLM / any sglang/vLLM endpoint).
+- Safe request rewrites: `reasoning_effort` (remap to a supported value or drop)
+  and `max_tokens` (clamp).
+- `dsh-openai-shim` (serve) and `dsh-proxy` (use/list/show/status/restart/sync)
+  command-line entry points.
+
+[0.2.0]: https://github.com/dive4dec/dsh-openai-shim/compare/v0.1.1...v0.2.0
+[0.1.1]: https://github.com/dive4dec/dsh-openai-shim/compare/v0.1.0...v0.1.1

dsh_openai_shim-0.2.0/LICENSE

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2026 dive4dec
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.

dsh_openai_shim-0.2.0/PKG-INFO

@@ -0,0 +1,228 @@
+Metadata-Version: 2.5
+Name: dsh-openai-shim
+Version: 0.2.0
+Summary: Zero-dependency OpenAI-compatible proxy that adapts DeepSeek Harness (dsh) to any OpenAI-compatible endpoint (Socrates/LiteLLM/sglang/vLLM)
+Project-URL: Homepage, https://github.com/dive4dec/dsh-openai-shim
+Project-URL: Source Code, https://github.com/dive4dec/dsh-openai-shim
+Author: dive4dec
+License: MIT
+License-File: LICENSE
+Keywords: deepseek,dsh,harness,litellm,openai,proxy,reasoning,sglang,vllm
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Provides-Extra: dev
+Requires-Dist: pytest; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# dsh-openai-shim
+
+A **zero-dependency, stdlib-only** OpenAI-compatible proxy that adapts
+[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) to
+**any** OpenAI-compatible endpoint — Socrates, LiteLLM, sglang, vLLM, etc.
+
+## Why
+
+`dsh`'s bundled `deepseek-official` provider is written for the DeepSeek API and
+does two things that break against general OpenAI endpoints like Socrates:
+
+1. It **hard-injects `reasoning_effort: "high"`** into every request. Socrates
+   (sglang) rejects `"high"` — it only accepts `xhigh` / `medium` / `low`.
+   → `HTTP 400: Unexpected reasoning effort high.`
+2. It requests a **default `max_tokens` of 256000**, which — combined with a
+   large system prompt — can exceed the model's context window.
+   → `HTTP 400: Requested token count exceeds the model's maximum context length.`
+
+Both are *request-parameter* mismatches, not protocol mismatches. `dsh` is not
+DeepSeek-locked; it connects, authenticates, routes, and runs its full agent
+tool loop against any OpenAI endpoint. The only friction is these two params.
+
+Rather than fight dsh's `cordis.patch.yml` / model-catalog (dev-preview,
+compiled into a native binary, version-churny), this shim is the **single
+compatibility chokepoint**: a tiny proxy dsh points at, which forwards to your
+endpoint and rewrites the two params. It is:
+
+- **agent-agnostic** — works for every dsh profile (`sdk`, `sdk-minimal`,
+  `acp`, `web`) and any OpenAI client.
+- **version-immune** — if dsh changes its defaults, the shim still fixes them.
+- **transparent** — passes through auth headers, streaming, `/models`,
+  tool-calling, everything else unchanged.
+
+## What it rewrites
+
+| Param | Default behavior | Options |
+|-------|------------------|---------|
+| `reasoning_effort` | `high → medium` (map to a supported value) | `--effort-mode map` (default) / `drop` (remove it) / `off` |
+| `max_tokens` | clamp to the provider's `token_cap` (env default `DSH_SHIM_TOKEN_CAP`, `100000`) | per-provider cap (auto-discovered); `0` disables clamping |
+
+The `max_tokens` cap is **per-provider**, not global: different models have
+different completion ceilings (a local sglang/vLLM server may allow 64000, a
+big hosted endpoint 100000+). The cap is discovered automatically when a
+provider is added, corrected automatically if the upstream rejects it, and can
+always be overridden with `--token-cap` — see [Per-user providers](#per-user-providers-dsh-proxy).
+
+## Install
+
+```sh
+pip install dsh-openai-shim
+# or from source
+pip install -e .
+```
+
+No third-party runtime dependencies — pure Python stdlib.
+
+## Use
+
+Run the shim, then point dsh at it:
+
+```sh
+# 1. start the shim (maps high→medium, caps max_tokens at 100000)
+dsh-openai-shim serve \
+    --upstream https://socratic.cs.cityu.edu.hk/ai-test \
+    --port 8090
+
+# 2. point dsh at the shim (not the upstream directly)
+DSH_HOME=/tmp/dsh python - <<'PY'
+from deepseek_harness import DeepSeekHarness
+with DeepSeekHarness(
+    dsh_home="/tmp/dsh",
+    cwd="/workspace",
+    provider="deepseek-official",
+    model="Socrates",                 # your endpoint's model id
+    base_url="http://127.0.0.1:8090/v1",   # NOTE: points at the shim
+    api_key="...",
+    max_tokens=100000,
+) as h:
+    print(h.run("Say hi.", session_id="s").final_response)
+PY
+```
+
+> **Path note:** if the client appends `/v1/...` to `base_url`, set
+> `--upstream` to the path **without** a trailing `/v1`.
+
+### Useful flags
+
+```
+--effort-mode {map,drop,off}   how to handle reasoning_effort (default: map)
+--token-cap N                  clamp completion max_tokens to N (default: 100000, 0 = off)
+--upstream-key Bearer          force a specific Authorization key upstream
+--host / --port                listen address (default 127.0.0.1:8090)
+```
+
+### One-shot rewrite (no server) — for tests / debugging
+
+```sh
+echo '{"reasoning_effort":"high","max_tokens":256000}' | \
+    dsh-openai-shim rewrite --effort-mode map
+# {"reasoning_effort": "medium", "max_tokens": 100000}
+```
+
+## Per-user providers (`dsh-proxy`)
+
+Beyond the one-shot `dsh-openai-shim serve --upstream …`, the package ships a
+**per-user provider manager** (`dsh-proxy`) for deployments that run one pod
+per student (e.g. JupyterHub). The student's choice of upstream is persisted
+in `~/.dsh/proxy.conf` (NFS home) so it survives pod restarts — the same
+file-based model as Hermes' `~/.hermes/config.yaml`:
+
+```json
+{
+  "provider": "litellm",
+  "providers": {
+    "diveai":   {"base_url": "https://…/ai/v1",   "api_key": "…"},
+    "litellm":  {"base_url": "https://…/litellm/v1", "api_key": "…"},
+    "myprov":   {"base_url": "http://10.0.0.5:30000/v1", "api_key": "…",
+                 "token_cap": 64000}
+  }
+}
+```
+
+- **The deployment's providers** (e.g. `diveai`, `litellm`) are upserted by
+  `dsh-proxy sync` on every notebook start — it reads the endpoint + key from
+  the pod environment (`DIVEAI_API_BASE/_KEY`, `LITELLM_API_BASE/_KEY`) and
+  refreshes those entries (key rotation, endpoint moves), without touching
+  student-added providers. The first-run default provider is set once and
+  never clobbered afterwards. A student-set `token_cap` survives a re-sync.
+- **Custom providers** are the student's own: `dsh-proxy use <name> --base
+  <url> [--key <key>]` adds ANY name (as many as you like) and switches to
+  it. Omit `--key` to forward the caller's `Authorization` header as-is.
+- **Completion-token cap is auto-discovered** (the student never has to know
+  the model's limit). On `use <name> --base <url>`, the shim probes
+  `GET /v1/models` on the new endpoint and reads the model's completion ceiling
+  (`max_completion_tokens` / `max_model_len`) into `token_cap`. This is the same
+  technique Hermes uses when you add a custom endpoint and skip the context
+  length. If the probe can't reach the endpoint (or it doesn't advertise a
+  limit), the cap is left at the env default — and the shim **self-heals at
+  request time**: if the upstream rejects a request with an "N too large … at
+  most M tokens" 400, the shim parses `M`, lowers the cap, retries the request
+  once, and persists `M` to proxy.conf. So a wrong/missing cap corrects itself
+  on the first failed turn. Override either step with `--token-cap N`
+  (or `--no-discover` to skip the probe entirely).
+- **Resolution is file-based and env-inert** — the daemon reads only
+  proxy.conf. A trailing `/v1` on a base is stripped automatically (the shim
+  re-appends it when forwarding).
+
+```sh
+dsh-proxy list                      # all providers, current flagged (*), caps shown
+dsh-proxy use litellm               # switch to an existing provider
+dsh-proxy use myprov --base https://host/v1 --key sk-...   # add + switch (cap auto-detected)
+dsh-proxy use spark --base http://10.0.0.5:30000/v1 --key sk-...   # cap auto-detected
+dsh-proxy use myprov --base https://host/v1 --key sk-... --token-cap 40000  # force a cap
+dsh-proxy show                      # current provider + upstream + effective cap (key masked)
+dsh-proxy sync                      # (in-pod) refresh deployment providers
+dsh-proxy status                    # is the shim daemon running?
+dsh-proxy restart                   # restart the shim daemon from proxy.conf
+```
+
+`use` writes the file and then **restarts** the running shim so the new
+provider takes effect immediately (there is no lazy auto-start in the request
+path — dsh makes plain HTTP calls to the loopback port). `sync` never
+restarts anything; the caller (the pod startup hook) restarts the daemon
+only when `sync` prints `changed=yes`.
+
+The package itself is **deployment-agnostic**: no provider catalog, no
+endpoints, no env-var names beyond the two raw `DIVEAI_*` / `LITELLM_*` pair
+the sync reads. A build-time gate (`dsh_shim_deployment_agnostic_gate.py` in
+the Jupyter image) fails the build if a deployment host ever leaks into the
+installed package or if resolution starts reading the environment.
+
+## Library use
+
+```python
+from dsh_openai_shim import ShimConfig, apply_rewrites, make_handler, serve
+
+cfg = ShimConfig(upstream="http://127.0.0.1:9999/v1",
+                 effort_mode="map", token_cap=100000)
+new_body, changes = apply_rewrites(body, cfg)   # pure, testable
+serve(cfg)                                       # run the server
+```
+
+## Security
+
+- By default the shim **forwards the client's `Authorization` header** to the
+  upstream. Use `--upstream-key` to replace it with a server-side key so the
+  client never holds the real endpoint credential.
+- Bind to `127.0.0.1` (default). Do not expose the shim to untrusted networks
+  without auth.
+
+## Tests
+
+```sh
+python -m pytest tests/ -q
+```
+
+The end-to-end tests run a local fake OpenAI upstream (no external network, no
+dsh required) and assert the shim rewrites `reasoning_effort` and `max_tokens`
+while passing through auth and the response.
+
+## Status
+
+v0.1.0 — spike-proven against Socrates (dsh → shim → Socrates → `pong`).
+Zero deps, MIT.

dsh_openai_shim-0.2.0/README.md

@@ -0,0 +1,204 @@
+# dsh-openai-shim
+
+A **zero-dependency, stdlib-only** OpenAI-compatible proxy that adapts
+[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) to
+**any** OpenAI-compatible endpoint — Socrates, LiteLLM, sglang, vLLM, etc.
+
+## Why
+
+`dsh`'s bundled `deepseek-official` provider is written for the DeepSeek API and
+does two things that break against general OpenAI endpoints like Socrates:
+
+1. It **hard-injects `reasoning_effort: "high"`** into every request. Socrates
+   (sglang) rejects `"high"` — it only accepts `xhigh` / `medium` / `low`.
+   → `HTTP 400: Unexpected reasoning effort high.`
+2. It requests a **default `max_tokens` of 256000**, which — combined with a
+   large system prompt — can exceed the model's context window.
+   → `HTTP 400: Requested token count exceeds the model's maximum context length.`
+
+Both are *request-parameter* mismatches, not protocol mismatches. `dsh` is not
+DeepSeek-locked; it connects, authenticates, routes, and runs its full agent
+tool loop against any OpenAI endpoint. The only friction is these two params.
+
+Rather than fight dsh's `cordis.patch.yml` / model-catalog (dev-preview,
+compiled into a native binary, version-churny), this shim is the **single
+compatibility chokepoint**: a tiny proxy dsh points at, which forwards to your
+endpoint and rewrites the two params. It is:
+
+- **agent-agnostic** — works for every dsh profile (`sdk`, `sdk-minimal`,
+  `acp`, `web`) and any OpenAI client.
+- **version-immune** — if dsh changes its defaults, the shim still fixes them.
+- **transparent** — passes through auth headers, streaming, `/models`,
+  tool-calling, everything else unchanged.
+
+## What it rewrites
+
+| Param | Default behavior | Options |
+|-------|------------------|---------|
+| `reasoning_effort` | `high → medium` (map to a supported value) | `--effort-mode map` (default) / `drop` (remove it) / `off` |
+| `max_tokens` | clamp to the provider's `token_cap` (env default `DSH_SHIM_TOKEN_CAP`, `100000`) | per-provider cap (auto-discovered); `0` disables clamping |
+
+The `max_tokens` cap is **per-provider**, not global: different models have
+different completion ceilings (a local sglang/vLLM server may allow 64000, a
+big hosted endpoint 100000+). The cap is discovered automatically when a
+provider is added, corrected automatically if the upstream rejects it, and can
+always be overridden with `--token-cap` — see [Per-user providers](#per-user-providers-dsh-proxy).
+
+## Install
+
+```sh
+pip install dsh-openai-shim
+# or from source
+pip install -e .
+```
+
+No third-party runtime dependencies — pure Python stdlib.
+
+## Use
+
+Run the shim, then point dsh at it:
+
+```sh
+# 1. start the shim (maps high→medium, caps max_tokens at 100000)
+dsh-openai-shim serve \
+    --upstream https://socratic.cs.cityu.edu.hk/ai-test \
+    --port 8090
+
+# 2. point dsh at the shim (not the upstream directly)
+DSH_HOME=/tmp/dsh python - <<'PY'
+from deepseek_harness import DeepSeekHarness
+with DeepSeekHarness(
+    dsh_home="/tmp/dsh",
+    cwd="/workspace",
+    provider="deepseek-official",
+    model="Socrates",                 # your endpoint's model id
+    base_url="http://127.0.0.1:8090/v1",   # NOTE: points at the shim
+    api_key="...",
+    max_tokens=100000,
+) as h:
+    print(h.run("Say hi.", session_id="s").final_response)
+PY
+```
+
+> **Path note:** if the client appends `/v1/...` to `base_url`, set
+> `--upstream` to the path **without** a trailing `/v1`.
+
+### Useful flags
+
+```
+--effort-mode {map,drop,off}   how to handle reasoning_effort (default: map)
+--token-cap N                  clamp completion max_tokens to N (default: 100000, 0 = off)
+--upstream-key Bearer          force a specific Authorization key upstream
+--host / --port                listen address (default 127.0.0.1:8090)
+```
+
+### One-shot rewrite (no server) — for tests / debugging
+
+```sh
+echo '{"reasoning_effort":"high","max_tokens":256000}' | \
+    dsh-openai-shim rewrite --effort-mode map
+# {"reasoning_effort": "medium", "max_tokens": 100000}
+```
+
+## Per-user providers (`dsh-proxy`)
+
+Beyond the one-shot `dsh-openai-shim serve --upstream …`, the package ships a
+**per-user provider manager** (`dsh-proxy`) for deployments that run one pod
+per student (e.g. JupyterHub). The student's choice of upstream is persisted
+in `~/.dsh/proxy.conf` (NFS home) so it survives pod restarts — the same
+file-based model as Hermes' `~/.hermes/config.yaml`:
+
+```json
+{
+  "provider": "litellm",
+  "providers": {
+    "diveai":   {"base_url": "https://…/ai/v1",   "api_key": "…"},
+    "litellm":  {"base_url": "https://…/litellm/v1", "api_key": "…"},
+    "myprov":   {"base_url": "http://10.0.0.5:30000/v1", "api_key": "…",
+                 "token_cap": 64000}
+  }
+}
+```
+
+- **The deployment's providers** (e.g. `diveai`, `litellm`) are upserted by
+  `dsh-proxy sync` on every notebook start — it reads the endpoint + key from
+  the pod environment (`DIVEAI_API_BASE/_KEY`, `LITELLM_API_BASE/_KEY`) and
+  refreshes those entries (key rotation, endpoint moves), without touching
+  student-added providers. The first-run default provider is set once and
+  never clobbered afterwards. A student-set `token_cap` survives a re-sync.
+- **Custom providers** are the student's own: `dsh-proxy use <name> --base
+  <url> [--key <key>]` adds ANY name (as many as you like) and switches to
+  it. Omit `--key` to forward the caller's `Authorization` header as-is.
+- **Completion-token cap is auto-discovered** (the student never has to know
+  the model's limit). On `use <name> --base <url>`, the shim probes
+  `GET /v1/models` on the new endpoint and reads the model's completion ceiling
+  (`max_completion_tokens` / `max_model_len`) into `token_cap`. This is the same
+  technique Hermes uses when you add a custom endpoint and skip the context
+  length. If the probe can't reach the endpoint (or it doesn't advertise a
+  limit), the cap is left at the env default — and the shim **self-heals at
+  request time**: if the upstream rejects a request with an "N too large … at
+  most M tokens" 400, the shim parses `M`, lowers the cap, retries the request
+  once, and persists `M` to proxy.conf. So a wrong/missing cap corrects itself
+  on the first failed turn. Override either step with `--token-cap N`
+  (or `--no-discover` to skip the probe entirely).
+- **Resolution is file-based and env-inert** — the daemon reads only
+  proxy.conf. A trailing `/v1` on a base is stripped automatically (the shim
+  re-appends it when forwarding).
+
+```sh
+dsh-proxy list                      # all providers, current flagged (*), caps shown
+dsh-proxy use litellm               # switch to an existing provider
+dsh-proxy use myprov --base https://host/v1 --key sk-...   # add + switch (cap auto-detected)
+dsh-proxy use spark --base http://10.0.0.5:30000/v1 --key sk-...   # cap auto-detected
+dsh-proxy use myprov --base https://host/v1 --key sk-... --token-cap 40000  # force a cap
+dsh-proxy show                      # current provider + upstream + effective cap (key masked)
+dsh-proxy sync                      # (in-pod) refresh deployment providers
+dsh-proxy status                    # is the shim daemon running?
+dsh-proxy restart                   # restart the shim daemon from proxy.conf
+```
+
+`use` writes the file and then **restarts** the running shim so the new
+provider takes effect immediately (there is no lazy auto-start in the request
+path — dsh makes plain HTTP calls to the loopback port). `sync` never
+restarts anything; the caller (the pod startup hook) restarts the daemon
+only when `sync` prints `changed=yes`.
+
+The package itself is **deployment-agnostic**: no provider catalog, no
+endpoints, no env-var names beyond the two raw `DIVEAI_*` / `LITELLM_*` pair
+the sync reads. A build-time gate (`dsh_shim_deployment_agnostic_gate.py` in
+the Jupyter image) fails the build if a deployment host ever leaks into the
+installed package or if resolution starts reading the environment.
+
+## Library use
+
+```python
+from dsh_openai_shim import ShimConfig, apply_rewrites, make_handler, serve
+
+cfg = ShimConfig(upstream="http://127.0.0.1:9999/v1",
+                 effort_mode="map", token_cap=100000)
+new_body, changes = apply_rewrites(body, cfg)   # pure, testable
+serve(cfg)                                       # run the server
+```
+
+## Security
+
+- By default the shim **forwards the client's `Authorization` header** to the
+  upstream. Use `--upstream-key` to replace it with a server-side key so the
+  client never holds the real endpoint credential.
+- Bind to `127.0.0.1` (default). Do not expose the shim to untrusted networks
+  without auth.
+
+## Tests
+
+```sh
+python -m pytest tests/ -q
+```
+
+The end-to-end tests run a local fake OpenAI upstream (no external network, no
+dsh required) and assert the shim rewrites `reasoning_effort` and `max_tokens`
+while passing through auth and the response.
+
+## Status
+
+v0.1.0 — spike-proven against Socrates (dsh → shim → Socrates → `pong`).
+Zero deps, MIT.