@tuned-tensor/cli 0.26.2 → 0.27.0

package/docs/local-runtime/decision.md

@@ -135,7 +135,8 @@ print(answer["choice"], answer["probabilities"])
 Pass the same instructions and criteria the model was trained with: they are
 part of its input. `resolved-workflow.json` records them under `question`.
 The tuned model is not registered with `tt models` and is not served through
-vLLM.
+vLLM. To move inference off your machine, use [serverless hosting](hosting.md)
+with `tt cloud endpoints create`.
 
 ## Limits
 

package/docs/local-runtime/hosting.md

@@ -0,0 +1,121 @@
+# Host a decision model
+
+Use a serverless endpoint when a small decision model receives occasional
+requests and you want to keep it off your own machine. Hosting currently supports
+the pinned English Laya base checkpoint and complete TT decision fine-tunes.
+GPU models, adapters, custom tokenizers and custom calibration are not supported.
+The endpoint uses 6 GB of CPU memory and scales to zero when idle; the first
+request after an idle period may be slow or fail while it starts.
+
+You need a TT account token and prepaid credits. Your application does not need
+AWS credentials. The hosted API must be enabled by the service operator before
+these commands work.
+
+```bash
+tt auth login
+tt balance
+# Use a completed decision run's checkpoint.
+tt cloud endpoints create \
+  --model .tuned-tensor/decision-runs/<run-id>/train/model \
+  --name support-triage
+```
+
+The command uploads only `model.safetensors` (at most 2 GB), without loading it
+into a model runtime. The serving image supplies the trusted Laya architecture
+and tokenizer. You do not need to upload the base model as well. To host the base
+model instead, point `--model` at its local checkpoint directory. There is no
+question or other inference configuration attached to a deployment. Send the
+complete model request each time you invoke it.
+
+Save the endpoint ID, HTTPS inference URL and inference key returned by create.
+The key is shown only once. It can invoke only this endpoint; your account token
+continues to manage it. Check status until it reports `ready`:
+
+```bash
+tt cloud endpoints get <endpoint-id>
+tt cloud endpoints list
+tt cloud endpoints invoke <endpoint-id> --payload request.json
+```
+
+For the current Laya runtime, `request.json` contains both the input and question:
+
+```json
+{
+  "input": "My card payment was declined",
+  "question": {
+    "type": "choice",
+    "instructions": "Which support queue should handle this message?",
+    "criteria": {
+      "lost_card": "A lost or stolen bank card",
+      "payment_declined": "A refused card payment"
+    }
+  }
+}
+```
+
+You may send a different question on the next request. The CLI sends this JSON
+object as-is; model-specific fields belong in the request, not deployment options.
+The hosting API bounds requests to 128 KiB; Laya validates its own fields.
+
+Each successful request costs the fixed number of US cents displayed at creation.
+The price stays fixed for that endpoint. Failed requests are refunded. This
+feature does not debit an idle compute or deployment fee; the operator still
+incurs AWS storage and related costs. Use `tt balance` and `tt balance --help`
+for your account balance and billing commands. Each account may have five
+non-deleted endpoints and create at most ten endpoints in 24 hours.
+
+A request has an idempotency UUID, printed by `invoke`. If a connection drops or
+the API says the request is still running, retry the **same complete payload** with that ID:
+
+```bash
+tt cloud endpoints invoke <endpoint-id> --payload request.json \
+  --request-id <original-request-uuid>
+```
+
+Replays return the recorded answer without a second charge. Reusing the ID with
+a different payload (including question or options) is rejected. A failed/refunded request remains failed; use a new
+ID for a fresh attempt. Interrupted requests are refunded by periodic recovery
+within roughly 15 minutes. For Laya, input text is limited to 8192 characters; each request
+includes the question it should answer. After three failures in five
+minutes, wait five minutes before a new attempt. After 100 failures across the
+account in 24 hours, new requests pause until earlier failures age out; cached
+replays remain available.
+
+For another application, call the URL from its backend:
+
+```bash
+curl "$TT_INFERENCE_URL" \
+  -H "Authorization: Bearer $TT_INFERENCE_KEY" \
+  -H "Idempotency-Key: $(uuidgen)" \
+  -H "Content-Type: application/json" \
+  --data-binary @request.json
+```
+
+The response's `data.result` is Laya's typed answer with probabilities. Store keys
+in your backend's secret storage; do not put them in public browser code. The
+service stores results for idempotent replay and request hashes rather than the request
+body. You can also open **Dashboard → Endpoints**, choose your endpoint, and click
+**Generate new inference key**. Confirm replacement, then copy the new key shown
+once. The old key stops working immediately; update your backend's secret.
+This page also lets you copy the inference URL and refresh deployment status.
+Settings continues to manage account access tokens, which have broader access.
+
+Endpoint keys can also be rotated from the CLI:
+
+```bash
+tt cloud endpoints rotate-key <endpoint-id>
+tt cloud endpoints delete <endpoint-id>
+tt cloud endpoints get <endpoint-id>
+```
+
+Deletion immediately blocks new requests. Cleanup removes the model, weights
+and SageMaker resources asynchronously; an endpoint still provisioning must finish
+that transition before AWS permits deletion. Late uploads go to temporary storage
+with automatic expiry and cannot recreate the serving artifact. Poll until
+`deleted`. Uploads abandoned for an hour, deployments that cannot become ready
+within an hour, and failed endpoints older than a day are cleaned automatically.
+Account deletion requires endpoint cleanup and accepted requests to finish.
+
+If creation fails after returning an endpoint ID, inspect that ID or delete it.
+If a deployment exists but the one-time key was lost, rotate it. Do not repeat
+`create` simply to retrieve the key: that allocates another endpoint.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tuned-tensor/cli",
-  "version": "0.26.2",
+  "version": "0.27.0",
   "description": "Local TT agent and CUDA fine-tuning CLI for Tuned Tensor workflows",
   "repository": {
     "type": "git",

package/training/serving/uv.lock

@@ -457,11 +457,11 @@ wheels = [
 
 [[package]]
 name = "dnspython"
-version = "2.8.0"
+version = "2.9.0"
 source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/8c/8b/57666417c0f90f08bcafa776861060426765fdb422eb10212086fb811d26/dnspython-2.8.0.tar.gz", hash = "sha256:181d3c6996452cb1189c4046c61599b84a5a86e099562ffde77d26984ff26d0f", size = 368251, upload-time = "2025-09-07T18:58:00.022Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/ef/4a/50822184bd67cc6493f0fb6a880749158fcd31ab3fa07409acfd91f9fc85/dnspython-2.9.0.tar.gz", hash = "sha256:b44dc6b18f07a8b1c56676a19fbfdb5209415b046a9cece286baafa87ff3f7f1", size = 423560, upload-time = "2026-10-09T00:07:24.352Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/ba/5a/18ad964b0086c6e62e2e7500f7edc89e3faa45033c71c1893d34eed2b2de/dnspython-2.8.0-py3-none-any.whl", hash = "sha256:01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af", size = 331094, upload-time = "2025-09-07T18:57:58.071Z" },
+    { url = "https://files.pythonhosted.org/packages/10/02/cdcc9b7c051786a103c3b09e1003a82fa0c66bcb91ffbdabcfbf7b4163b9/dnspython-2.9.0-py3-none-any.whl", hash = "sha256:9a4aedb833c3c1b49214d04d44d3032ab7a9135f7c1d29a549b4ff78fd82fda9", size = 354822, upload-time = "2026-10-09T00:07:22.622Z" },
 ]
 
 [[package]]