opencode-llmstack 0.7.2__tar.gz → 0.8.0__tar.gz

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opencode-llmstack
-Version: 0.7.2
+Version: 0.8.0
 Summary: Multi-tier local LLM stack: llama-swap + FastAPI auto-router + opencode wiring.
 Author: llmstack
 License: MIT
@@ -542,8 +542,8 @@ All knobs are env vars; defaults are picked up by `llmstack start`.
 | `ROUTER_ULTRA_MODEL` | `code-ultra` | short-context top tier → here (gated on availability) |
 | `ROUTER_PLAN_MODEL` | `plan` | design/discussion verbs → here |
 | `ROUTER_UNCENSORED_MODEL` | `plan-uncensored` | `[nofilter]` triggers → here |
-| `ROUTER_HIGH_FIDELITY_CEILING` | `8000` | tokens; at or below this, route to top tier (ultra → smart fallback) |
-| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast` |
+| `ROUTER_HIGH_FIDELITY_CEILING` | `12000` | tokens; at or below this, route to top tier (ultra → smart fallback). Paired with `code-ultra.ctx_size = 24000` (2x). |
+| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast`. Paired with `code-smart.ctx_size = 64000` (2x). |
 | `ROUTER_MULTI_TURN` | `6` | turn count that floors the long-context rung at `code-smart` |
 | `ROUTER_HOST` / `ROUTER_PORT` | `127.0.0.1` / `10101` | listen address |
 | `LOG_LEVEL` | `info` | router log level |

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/README.md

@@ -507,8 +507,8 @@ All knobs are env vars; defaults are picked up by `llmstack start`.
 | `ROUTER_ULTRA_MODEL` | `code-ultra` | short-context top tier → here (gated on availability) |
 | `ROUTER_PLAN_MODEL` | `plan` | design/discussion verbs → here |
 | `ROUTER_UNCENSORED_MODEL` | `plan-uncensored` | `[nofilter]` triggers → here |
-| `ROUTER_HIGH_FIDELITY_CEILING` | `8000` | tokens; at or below this, route to top tier (ultra → smart fallback) |
-| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast` |
+| `ROUTER_HIGH_FIDELITY_CEILING` | `12000` | tokens; at or below this, route to top tier (ultra → smart fallback). Paired with `code-ultra.ctx_size = 24000` (2x). |
+| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast`. Paired with `code-smart.ctx_size = 64000` (2x). |
 | `ROUTER_MULTI_TURN` | `6` | turn count that floors the long-context rung at `code-smart` |
 | `ROUTER_HOST` / `ROUTER_PORT` | `127.0.0.1` / `10101` | listen address |
 | `LOG_LEVEL` | `info` | router log level |

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/llmstack/app.py

@@ -74,7 +74,18 @@ Routing decision tree (first match wins):
   2. Explicit "ultra" trigger (``[ultra]``, ``[opus]``,
      ``ultra:``, ``opus:``) AND ultra tier configured -> code-ultra
   3. PLAN signal words AND no code-block / agent verbs / tools
-     (design discussion, no implementation pending)   -> plan
+     AND estimated tokens <= ``[plan]`` tier's ctx_size
+     (pure design discussion that fits the planner's
+     window)                                          -> plan
+                                                         (if the planner's
+                                                          ctx_size is breached
+                                                          we fall through to
+                                                          the coding ladder
+                                                          rather than send a
+                                                          request that won't
+                                                          fit -- the coding
+                                                          tiers cover larger
+                                                          windows by design)
   4. Estimated input tokens <= HIGH_FIDELITY_CEILING
      ("reasonable context still being built")         -> code-ultra
                                                          (else code-smart)
@@ -90,6 +101,12 @@ Routing decision tree (first match wins):
                                                           since 3B models
                                                           tool-call unreliably)
 
+The auto router's effective max context window is
+``[code-fast].ctx_size`` -- fast is the bottom of the step-down
+ladder, so any context that would overflow the tiers above lands on
+fast. Inputs longer than fast's window have no safe home and should
+be considered out of scope for ``model = auto``.
+
 Ultra-tier routing is gated on availability: rule (2) and the
 "high-fidelity" rung of (4) first check that the tier is loaded
 from ``models.ini`` (i.e. present in :data:`TIER_BY_ALIAS`). When
@@ -137,14 +154,21 @@ UNCENSORED_MODEL = os.getenv("ROUTER_UNCENSORED_MODEL", "plan-uncensored")
 #   est <= MID_FIDELITY_CEILING   -> code-smart
 #   est >  MID_FIDELITY_CEILING   -> code-fast (or smart with tools/loop)
 #
+# Each ceiling is half of the corresponding tier's ``ctx_size`` in
+# models.ini -- the ceiling marks where the tier still has comfortable
+# headroom, and double the ceiling is where the router has already
+# stepped down to the next tier (so the upper tier never has to handle
+# inputs at its own limit).
+#
 # Defaults:
-#   HIGH 8000  - "reasonable context built": a couple of files loaded,
+#   HIGH 12000 - "reasonable context built": a couple of files loaded,
 #                instructions clear, top-tier still cheap+fast here.
-#   MID  32000 - half of code-smart's 65k window; past this, hosted
+#                Pairs with a 24k ctx_size on code-ultra.
+#   MID  32000 - half of code-smart's 64k window; past this, hosted
 #                top-tier latency/$cost balloons and code-smart starts
 #                getting cramped, while code-fast's 128k YaRN window
 #                still has comfortable headroom.
-HIGH_FIDELITY_CEILING = int(os.getenv("ROUTER_HIGH_FIDELITY_CEILING", "8000"))
+HIGH_FIDELITY_CEILING = int(os.getenv("ROUTER_HIGH_FIDELITY_CEILING", "12000"))
 MID_FIDELITY_CEILING = int(os.getenv("ROUTER_MID_FIDELITY_CEILING", "32000"))
 # Floor the long-context rung at code-smart whenever a tool-call
 # protocol is in play -- 3B models tool-call unreliably regardless of
@@ -339,18 +363,30 @@ def classify(body: dict[str, Any]) -> tuple[str, str]:
         or _matches(AGENT_SIGNALS, messages, prompt)
     )
 
+    est = _estimate_tokens(messages, prompt)
+
     # Plan track is orthogonal to the code fidelity ladder: ``plan`` is a
     # chat-tuned model meant for design / "should we" discussions. Only
     # take it when nothing about the request says "I'm about to write
-    # code" (no triple-backticks, no agent verbs, no tool calls).
+    # code" (no triple-backticks, no agent verbs, no tool calls). And
+    # only if the input fits in the planner's ctx_size -- past that we'd
+    # be sending a request the planner can't hold, so we fall through
+    # to the coding ladder, which has tiers (smart, fast) explicitly
+    # sized for larger contexts.
     if (
         not has_tools
         and not has_code_signal
         and _matches(PLAN_SIGNALS, messages, prompt)
     ):
-        return PLAN_MODEL, "plan-signal"
-
-    est = _estimate_tokens(messages, prompt)
+        plan_tier = TIER_BY_ALIAS.get(PLAN_MODEL)
+        plan_ctx = plan_tier.ctx_size if plan_tier else 0
+        if not plan_ctx or est <= plan_ctx:
+            return PLAN_MODEL, "plan-signal"
+        log.info(
+            "plan-signal but tokens~%d > %s.ctx_size %d; "
+            "falling through to coding ladder",
+            est, PLAN_MODEL, plan_ctx,
+        )
 
     # Rung 1: short context -- start at the top.
     if est <= HIGH_FIDELITY_CEILING:

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/llmstack/generators/opencode.py

@@ -172,9 +172,22 @@ def build_config(
 
     tier_sections = [s for s in cfg.sections() if s != "ROUTING"]
 
-    # `auto` context = MIN across all tiers so opencode never packs a prompt
-    # that overflows the tier the router actually picks.
-    auto_ctx = min(
+    # `auto` context = the fast tier's ctx_size. The router runs a
+    # step-DOWN ladder (ultra -> smart -> fast as context grows), so
+    # the largest window in the ladder is fast's, and that's the
+    # effective ceiling for `model = auto` -- anything bigger has no
+    # tier to land on. Using `min(...)` here would clip opencode to
+    # the smallest tier's window even though the router would never
+    # actually send a long prompt to that tier.
+    fast_ctx = next(
+        (
+            _int(cfg[s].get("ctx_size", ""), 0)
+            for s in tier_sections
+            if (cfg[s].get("role") or "").strip() == "fast"
+        ),
+        0,
+    )
+    auto_ctx = fast_ctx or max(
         (_int(cfg[s].get("ctx_size", ""), 0) for s in tier_sections),
         default=8192,
     ) or 8192

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/llmstack/models.ini

@@ -51,7 +51,6 @@
 [DEFAULT]
 host         = 127.0.0.1
 router_port  = 10101        ; FastAPI auto-router (what opencode hits)
-swap_port    = 10102        ; llama-swap manager UI + raw model endpoints
 n_gpu_layers = 999          ; offload everything to Metal on Apple Silicon
 flash_attn   = on
 jinja        = true
@@ -68,31 +67,60 @@ tier         = code
 role         = fast
 hf_repo      = bartowski/Qwen2.5-Coder-3B-Instruct-GGUF
 hf_file      = Qwen2.5-Coder-3B-Instruct-Q5_K_M.gguf
-ctx_size     = 131072       ; native 32k extended via YaRN (factor 4)
+ctx_size     = 131072       ; native 32k extended via YaRN (factor 4); also defines the auto router's effective max -- contexts longer than this have nowhere to land, since fast is the bottom of the step-down ladder
 rope_scaling = yarn (scale=4, orig_ctx=32768)
 size_gb      = 2.5
 quant        = Q5_K_M
-status       = downloading      ; queued by `llmstack.sh download`
-opencode_use = small_model + auto-fast tier
 sampler      = temp=0.2, top_p=0.95, top_k=40, min_p=0.05      ; deterministic
 description  = Qwen2.5-Coder 3B - autocomplete / FIM / quick Q&A
 
+; Bedrock alternative for code-fast -- comment out the [code-fast] block above
+; and uncomment the block below to swap to a hosted fast tier (Claude Haiku
+; 4.5: cheapest + fastest Anthropic model with tool calling, sub-second TTFT).
+; See "BEDROCK NOTES" at the bottom of this file for profile / sampler /
+; access-form details.
+;
+; [code-fast]
+; tier         = code
+; role         = fast
+; backend      = bedrock
+; aws_model_id = eu.anthropic.claude-haiku-4-5-20251001-v1:0
+; aws_region   = eu-central-1
+; aws_profile  = bedrock-prod
+; ctx_size     = 200000
+; sampler      = temp=0.2     ; deterministic; Haiku 4.5 accepts ONE of temp / top_p
+; description  = Claude Haiku 4.5 on Bedrock - hosted fast tier for autocomplete / FIM / quick Q&A
+
 [code-smart]
 tier         = code
 role         = agent
 hf_repo      = unsloth/Qwen3-Coder-Next-GGUF
 hf_file      = Qwen3-Coder-Next-Q4_K_M.gguf
 hf_file_next = Qwen3-Coder-Next-UD-Q4_K_XL.gguf
-ctx_size     = 65536
+ctx_size     = 64000        ; 2x mid_fidelity_ceiling -- router steps down to code-fast past this, so a larger window would never get used
 size_gb      = 45
 size_gb_next = 50
 quant        = Q4_K_M
 quant_next   = UD-Q4_K_XL
-status       = ready (Q4_K_M); UD-Q4_K_XL queued
-opencode_use = agent.build + auto-agent tier
 sampler      = temp=0.5, top_p=0.85, top_k=20, min_p=0.05, rep_pen=1.05  ; balanced agent
 description  = Qwen3-Coder-Next 80B-A3B MoE - heavy coder for agent loops
 
+; Bedrock alternative for code-smart -- comment out the [code-smart] block
+; above and uncomment the block below to swap to a hosted heavy coder
+; (Claude Sonnet 4.6: agent-loop workhorse, heavy tool calling, multi-file
+; edits). See "BEDROCK NOTES" at the bottom of this file.
+;
+; [code-smart]
+; tier         = code
+; role         = agent
+; backend      = bedrock
+; aws_model_id = eu.anthropic.claude-sonnet-4-6
+; aws_region   = eu-central-1
+; aws_profile  = bedrock-prod
+; ctx_size     = 64000        ; 2x mid_fidelity_ceiling -- intentionally well below Sonnet 4.6's 200k native window; router steps down to code-fast past this rather than paying Sonnet's long-context $cost/latency
+; sampler      = temp=0.5     ; Sonnet 4.6 accepts ONE of temp / top_p; pick `temp` for agent work
+; description  = Claude Sonnet 4.6 on Bedrock - heavy coder for agent loops
+
 ; Top-tier hosted coder. Shipped disabled because it requires boto3 +
 ; AWS Bedrock access. `llmstack install` auto-uncomments the block
 ; below (by stripping the leading "; " from each line and dropping
@@ -107,10 +135,9 @@ description  = Qwen3-Coder-Next 80B-A3B MoE - heavy coder for agent loops
 ; role         = ultra
 ; backend      = bedrock
 ; aws_model_id = global.anthropic.claude-opus-4-7   ; global.* cross-region inference profile
-; aws_region   = us-east-1                          ; API anchor region; global.* auto-routes inference cross-region
-; aws_profile = bedrock-prod                        ; uncomment + set your own profile name; falls back to default cred chain otherwise
-; ctx_size     = 200000
-; opencode_use = on-demand top-tier coder for hard agent tasks
+; aws_region   = eu-central-1                       ; API anchor region; global.* auto-routes inference cross-region (set EU as the anchor for residency)
+; aws_profile  = bedrock-prod                       ; conventional profile name; configure once with `aws configure --profile bedrock-prod` (or change to your own and run `llmstack install`)
+; ctx_size     = 24000        ; 2x high_fidelity_ceiling -- intentionally well below Opus 4.7's 200k native window; we only invoke ultra for short prompts, where it's still cheap+fast, then step down past this
 ; ; NB: no `sampler =` line. Claude Opus 4.7 explicitly rejects all
 ; ;     sampler params (temperature, top_p, top_k) -- per the Bedrock
 ; ;     model card, "the recommended migration path is to omit these
@@ -134,11 +161,26 @@ size_gb      = 9.2
 size_gb_next = 12.1
 quant        = Q4_K_M
 quant_next   = Q6_K
-status       = ready (Q4_K_M); Q6_K queued
-opencode_use = agent.plan + auto-plan tier
 sampler      = temp=0.7, top_p=0.9, top_k=40, min_p=0.05       ; creative thinking
 description  = Qwopus GLM 18B - planning, design discussions, architecture
 
+; Bedrock alternative for plan -- comment out the [plan] block above and
+; uncomment the block below to swap to a hosted planner (Claude Opus 4.6:
+; deep reasoning for design discussions and architecture). Opus 4.6 still
+; accepts both temperature and top_p (unlike 4.7), so the local sampler
+; maps over cleanly. See "BEDROCK NOTES" at the bottom of this file.
+;
+; [plan]
+; tier         = chat
+; role         = plan
+; backend      = bedrock
+; aws_model_id = eu.anthropic.claude-opus-4-6-v1
+; aws_region   = eu-central-1
+; aws_profile  = bedrock-prod
+; ctx_size     = 200000
+; sampler      = temp=0.7, top_p=0.9     ; creative; Opus 4.6 accepts both
+; description  = Claude Opus 4.6 on Bedrock - planning, design discussions, architecture
+
 [plan-uncensored]
 tier         = chat
 role         = plan-uncensored
@@ -150,11 +192,51 @@ size_gb      = 13
 size_gb_next = 20
 quant        = i1-Q4_K_M
 quant_next   = i1-Q6_K
-status       = ready (i1-Q4_K_M); i1-Q6_K queued
-opencode_use = agent.plan-nofilter + auto via [nofilter] trigger
 sampler      = temp=0.85, top_p=0.95, top_k=50, min_p=0.05     ; max exploration
 description  = Mistral-Small 3.2 24B Heretic - no-filter planning
 
+; Bedrock alternative for plan-uncensored -- comment out the [plan-uncensored]
+; block above and uncomment ONE of the blocks below. Anthropic models on
+; Bedrock are filtered, so for the uncensored slot we pick the largest
+; open-weights model on Bedrock: Llama 3.1 405B has minimal safety post-
+; training and matches the spirit of the local Heretic tier. NOTE: Meta
+; models do NOT require the AWS use-case form, so this swap unblocks
+; plan-uncensored on a fresh AWS account.
+;
+; REGION CAVEAT: unlike the other tiers above, Llama 3.1 405B has NO
+; cross-region inference profile (no eu.* / global.*) and is only
+; deployed in US regions. Pin to us-west-2 even when the rest of the
+; stack is anchored in eu-central-1. If EU residency is mandatory for
+; this tier, switch to one of the eu.anthropic.* IDs at the cost of
+; losing the "uncensored" property. See "BEDROCK NOTES" at the bottom
+; of this file.
+;
+; [plan-uncensored]
+; tier         = chat
+; role         = plan-uncensored
+; backend      = bedrock
+; aws_model_id = meta.llama3-1-405b-instruct-v1:0
+; aws_region   = us-west-2     ; Llama 405B has no EU deployment; keep on US
+; aws_profile  = bedrock-prod
+; ctx_size     = 128000
+; sampler      = temp=0.85, top_p=0.95   ; max exploration
+; description  = Llama 3.1 405B on Bedrock - no-filter planning
+;
+; ...or, if your org locks Bedrock access to a VPC endpoint, use this
+; variant instead (same model + sampler, with aws_endpoint_url set):
+;
+; [plan-uncensored]
+; tier             = chat
+; role             = plan-uncensored
+; backend          = bedrock
+; aws_model_id     = meta.llama3-1-405b-instruct-v1:0
+; aws_region       = us-west-2     ; Llama 405B has no EU deployment
+; aws_profile      = bedrock-prod
+; aws_endpoint_url = https://bedrock-runtime.us-west-2.vpce.amazonaws.com
+; ctx_size         = 128000
+; sampler          = temp=0.85, top_p=0.95
+; description      = Llama 3.1 405B on Bedrock (VPC) - no-filter planning
+
 ;------------------------------------------------------------------------------
 [ROUTING]
 ; STEP-DOWN ladder: start at the top of the fidelity ladder for short
@@ -178,7 +260,11 @@ description  = Mistral-Small 3.2 24B Heretic - no-filter planning
 ;   2. "[ultra]" / "[opus]" / "ultra:" trigger AND code-ultra
 ;      tier configured                                            -> code-ultra
 ;   3. PLAN signal words AND no code-block / agent verbs / tools
-;      (pure design discussion)                                   -> plan
+;      AND tokens <= [plan].ctx_size (pure design discussion that
+;      still fits the planner's window)                           -> plan
+;      ...if the plan tier's ctx_size is breached, the request
+;      falls through to the coding ladder below rather than being
+;      sent to a planner whose window can't hold the input.
 ;   4. tokens <= high_fidelity_ceiling AND code-ultra configured  -> code-ultra
 ;      tokens <= high_fidelity_ceiling AND no code-ultra          -> code-smart
 ;   5. tokens <= mid_fidelity_ceiling                             -> code-smart
@@ -186,6 +272,24 @@ description  = Mistral-Small 3.2 24B Heretic - no-filter planning
 ;        - if tools[] OR turns >= multi_turn (3B tool-calls badly) -> code-smart
 ;        - else                                                    -> code-fast
 ;
+; AUTO ROUTER MAX CONTEXT = [code-fast].ctx_size. The fast tier sits at
+; the bottom of the step-down ladder, so any context too big for the
+; tiers above lands on fast. Anything beyond fast's window has no
+; safe home and should be considered out of scope for `model = auto`
+; -- callers with such payloads should pass an explicit model name
+; instead of relying on the router.
+;
+; FIDELITY-CEILING <-> CTX-SIZE COUPLING. Each "fidelity" rung's
+; ceiling is half of the corresponding tier's ctx_size:
+;   high_fidelity_ceiling x 2 == [code-ultra].ctx_size
+;   mid_fidelity_ceiling  x 2 == [code-smart].ctx_size
+; This is deliberate: the ceiling marks where the tier still has
+; comfortable headroom; double the ceiling is where we'd be packing
+; the tier to its limit (and where the router has already stepped
+; down to the next tier). If you bump a ceiling, bump the matching
+; ctx_size in the tier section too -- otherwise the router will
+; route requests up to a tier whose window can't hold them.
+;
 ; The "high-fidelity" rung is gated on availability: when the
 ; [code-ultra] section is absent (or fails to load), rules (2) and (4)
 ; silently fall back to code-smart instead of routing to a tier that
@@ -194,8 +298,8 @@ description  = Mistral-Small 3.2 24B Heretic - no-filter planning
 ; ROUTER_HIGH_FIDELITY_CEILING / ROUTER_MID_FIDELITY_CEILING /
 ; ROUTER_MULTI_TURN.
 ;
-high_fidelity_ceiling = 8000     ; tokens; below this, top-tier model is still cheap+fast
-mid_fidelity_ceiling  = 32000    ; tokens; smart's sweet spot up to here, then step down to fast
+high_fidelity_ceiling = 12000    ; tokens; below this, top-tier model is still cheap+fast (and ultra ctx_size = 2 * this)
+mid_fidelity_ceiling  = 32000    ; tokens; smart's sweet spot up to here, then step down to fast (smart ctx_size = 2 * this)
 multi_turn            = 6        ; turn count that floors the long-context rung at code-smart
 agent_signal_words    = implement, fix bug, write a function, refactor, edit, patch, debug, run tests, build it
 plan_signal_words     = design, architect, approach, trade-off, should we, how would you, explain why, think through, compare options, brainstorm, root cause
@@ -203,30 +307,46 @@ uncensored_triggers   = [nofilter], [uncensored], [heretic], "uncensored:", "nof
 ultra_triggers        = [ultra], [opus], "ultra:", "opus:" (line start)
 
 ;------------------------------------------------------------------------------
-; BEDROCK EXAMPLES (commented out -- copy / uncomment to adopt)
+; BEDROCK NOTES  (referenced by the commented-out alternatives above)
 ;------------------------------------------------------------------------------
-; To swap one of the local GGUF tiers above for an AWS Bedrock model, COMMENT
-; OUT the existing tier of the same name and uncomment one of these. The router
-; auto-detects backend=bedrock from the presence of `aws_model_id` -- no other
-; flag needed. llama-swap won't load it; the router calls Bedrock directly via
-; boto3 (`pip install 'llmstack[bedrock]'`).
+; Each tier section above carries a "Bedrock alternative for <tier>" block
+; directly underneath it (commented out by default). To swap a tier:
+;
+;   1. comment out the active local section (GGUF by default);
+;   2. uncomment the Bedrock-alternative block beneath it;
+;   3. run `llmstack install` (and `llmstack restart` if the tier was
+;      already loaded -- bedrock creds aren't picked up live).
 ;
-; Credentials: this file ONLY names a profile. The actual keys / SSO /
-; role chaining live in the standard AWS config files. One-time setup:
+; The router auto-detects backend=bedrock from `aws_model_id`, but every
+; alternative block also sets `backend = bedrock` explicitly so the intent
+; is obvious. llama-swap won't load bedrock tiers; the router calls
+; Bedrock directly via boto3 (`pip install 'llmstack[bedrock]'`).
+;
+; PROFILE: every alternative uses `aws_profile = bedrock-prod`, the
+; conventional profile name for this stack. The actual keys / SSO /
+; role chaining live in the standard AWS config files (this file ONLY
+; names a profile -- never put credentials here). One-time setup:
 ;
 ;     aws configure --profile bedrock-prod
-;     # for SSO:    aws configure sso --profile bedrock-prod
-;     # for role chaining, edit ~/.aws/config and add a profile with:
+;     # SSO:           aws configure sso --profile bedrock-prod
+;     # role chaining: edit ~/.aws/config and add:
+;     #   [profile bedrock-prod]
 ;     #   role_arn       = arn:aws:iam::123456789012:role/llmstack-bedrock
-;     #   source_profile = bedrock-prod
+;     #   source_profile = bedrock-prod-base
+;
+; To use a different profile name, edit the `aws_profile` line. To fall
+; back on boto3's default chain (env vars, default profile, instance
+; role), remove the line entirely.
 ;
-; Then reference the profile name from your tier with `aws_profile = ...`.
-; If you omit `aws_profile`, boto3's default chain applies (env vars,
-; default profile, instance role -- whatever boto3 normally finds).
+; UPGRADE PRE-STAGING: optional `aws_model_id_next` (+ `aws_region_next`)
+; is the queued upgrade target -- mirrors gguf `hf_file_next`. The router
+; uses it only when `llmstack start --next` is in effect; permanent
+; promotion is the same as gguf: edit `aws_model_id` and re-run
+; `llmstack install`.
 ;
-; SAMPLER NOTE: the `sampler = temp=..., top_p=..., top_k=..., ...`
-; line on each tier is the SINGLE SOURCE OF TRUTH for sampling, but how
-; it gets applied depends on the backend:
+; SAMPLER: the `sampler = temp=..., top_p=..., top_k=..., ...` line on
+; each tier is the SINGLE SOURCE OF TRUTH for sampling, but how it gets
+; applied depends on the backend:
 ;
 ;   * gguf tiers -- the llama-swap generator bakes the sampler keys
 ;     into the llama-server startup command line as `--temp`,
@@ -248,57 +368,21 @@ ultra_triggers        = [ultra], [opus], "ultra:", "opus:" (line start)
 ; opencode.json is sampler-free in both cases by design (the
 ; opencode.json generator never emits sampler params on agents).
 ;
-; Per-Bedrock-family rules (as of 2026):
+; Per-Bedrock-family sampler rules (as of 2026):
 ;
 ;   * Claude Opus 4.7+ -- rejects all sampler params; OMIT `sampler =`
 ;     entirely (the router will then pass requests through untouched).
-;   * Claude Sonnet 4.5 / Haiku 4.5 -- accept `temp` OR `top_p`, never
-;     both; pick one.
+;   * Claude Sonnet 4.5 / 4.6 / Haiku 4.5 -- accept `temp` OR `top_p`,
+;     never both; pick one.
 ;   * Claude Opus 4.x (4.1, 4.5, 4.6) -- accept `temp` and `top_p`.
-;   * Llama / Titan / Cohere / etc. -- accept `temp` + `top_p`; check
-;     the model card if in doubt.
-;
-; Example A: top-tier coder on Bedrock (us-west-2), default cred chain.
-; Optional `aws_model_id_next` (and optional `aws_region_next`) is the
-; queued upgrade target -- mirrors gguf `hf_file_next`. The router uses
-; it only when `--next` is in effect; permanent promotion is the same
-; as gguf: edit `aws_model_id` and re-run `llmstack install`.
-;
-; [code-smart]
-; tier              = code
-; role              = agent
-; backend           = bedrock
-; aws_model_id      = anthropic.claude-sonnet-4-5-20250929-v1:0
-; aws_region        = us-west-2
-; aws_model_id_next = anthropic.claude-sonnet-5-20260201-v1:0   ; queued
-; aws_region_next   = us-east-1                                  ; (optional) different region for the new model
-; ctx_size          = 200000
-; sampler           = temp=0.5      ; Sonnet 4.5 accepts ONE of temp / top_p; pick `temp` for agent work
-; description       = Claude Sonnet 4.5 on Bedrock - heavy coder for agent loops
-;
-; Example B: planner in a different AWS account, accessed via a named
-; profile that itself uses role-chaining + SSO under ~/.aws/config.
-; (Different tier => different profile name; different account/region.)
+;   * Llama / Titan / Mistral / Cohere / Nova / etc. -- accept `temp`
+;     + `top_p`; check the model card if in doubt.
 ;
-; [plan]
-; tier         = chat
-; role         = plan
-; aws_model_id = us.anthropic.claude-opus-4-1-20250805-v1:0
-; aws_region   = us-east-1
-; aws_profile  = bedrock-planning
-; ctx_size     = 200000
-; sampler      = temp=0.7, top_p=0.9
-; description  = Claude Opus 4.1 on Bedrock - planning, design discussions
-;
-; Example C: large model behind a VPC endpoint.
-;
-; [plan-uncensored]
-; tier             = chat
-; role             = plan-uncensored
-; aws_model_id     = meta.llama3-1-405b-instruct-v1:0
-; aws_region       = us-west-2
-; aws_profile      = bedrock-prod
-; aws_endpoint_url = https://bedrock-runtime.us-west-2.vpce.amazonaws.com
-; ctx_size         = 128000
-; sampler          = temp=0.85, top_p=0.95
-; description      = Llama 3.1 405B on Bedrock - max-exploration planning
+; ACCESS: Anthropic Claude on Bedrock requires a one-time use-case-form
+; approval per AWS account (Bedrock console -> Model catalog -> pick the
+; model -> fill the form). Approval is account-level and persists; once
+; granted, every Claude variant works (bare ID, us./eu./global. cross-
+; region profile, application inference profile ARN). To skip the form
+; entirely, use the Llama 3.1 405B variant under [plan-uncensored] (Meta
+; models don't require the form) or pick another non-Anthropic family
+; (Amazon Nova, Mistral, Cohere, Titan).

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/opencode_llmstack.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opencode-llmstack
-Version: 0.7.2
+Version: 0.8.0
 Summary: Multi-tier local LLM stack: llama-swap + FastAPI auto-router + opencode wiring.
 Author: llmstack
 License: MIT
@@ -542,8 +542,8 @@ All knobs are env vars; defaults are picked up by `llmstack start`.
 | `ROUTER_ULTRA_MODEL` | `code-ultra` | short-context top tier → here (gated on availability) |
 | `ROUTER_PLAN_MODEL` | `plan` | design/discussion verbs → here |
 | `ROUTER_UNCENSORED_MODEL` | `plan-uncensored` | `[nofilter]` triggers → here |
-| `ROUTER_HIGH_FIDELITY_CEILING` | `8000` | tokens; at or below this, route to top tier (ultra → smart fallback) |
-| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast` |
+| `ROUTER_HIGH_FIDELITY_CEILING` | `12000` | tokens; at or below this, route to top tier (ultra → smart fallback). Paired with `code-ultra.ctx_size = 24000` (2x). |
+| `ROUTER_MID_FIDELITY_CEILING` | `32000` | tokens; at or below this, route to `code-smart`; beyond, step down to `code-fast`. Paired with `code-smart.ctx_size = 64000` (2x). |
 | `ROUTER_MULTI_TURN` | `6` | turn count that floors the long-context rung at `code-smart` |
 | `ROUTER_HOST` / `ROUTER_PORT` | `127.0.0.1` / `10101` | listen address |
 | `LOG_LEVEL` | `info` | router log level |

{opencode_llmstack-0.7.2 → opencode_llmstack-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "opencode-llmstack"
-version = "0.7.2"
+version = "0.8.0"
 description = "Multi-tier local LLM stack: llama-swap + FastAPI auto-router + opencode wiring."
 readme = "README.md"
 requires-python = ">=3.11"