@tuned-tensor/cli 0.28.1 → 0.30.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (99) hide show
  1. package/README.md +76 -180
  2. package/dist/chunk-6CETSU2H.js +260 -0
  3. package/dist/chunk-6CETSU2H.js.map +1 -0
  4. package/dist/chunk-B2O6P7EX.js +1623 -0
  5. package/dist/chunk-B2O6P7EX.js.map +1 -0
  6. package/dist/{chunk-7CO5DBCY.js → chunk-GMBZ2FE6.js} +5046 -5014
  7. package/dist/chunk-GMBZ2FE6.js.map +1 -0
  8. package/dist/chunk-MM75IAY3.js +138 -0
  9. package/dist/chunk-MM75IAY3.js.map +1 -0
  10. package/dist/chunk-ST6ZSEUH.js +1272 -0
  11. package/dist/chunk-ST6ZSEUH.js.map +1 -0
  12. package/dist/contracts-SHBUXEOQ.js +72 -0
  13. package/dist/dataset-Y5NEUKI5.js +22 -0
  14. package/dist/dataset-Y5NEUKI5.js.map +1 -0
  15. package/dist/index.js +4530 -9349
  16. package/dist/index.js.map +1 -1
  17. package/dist/local-runtime.js +84 -467
  18. package/dist/local-runtime.js.map +1 -1
  19. package/dist/model-server-IWCRFAOO.js +18 -0
  20. package/dist/model-server-IWCRFAOO.js.map +1 -0
  21. package/dist/{prefetch-LBIOO7ZO.js → prefetch-Q2ZA3SJ7.js} +3 -2
  22. package/dist/prefetch-Q2ZA3SJ7.js.map +1 -0
  23. package/docs/local-runtime/architecture.md +4 -4
  24. package/docs/local-runtime/aws-gpu.md +4 -5
  25. package/docs/local-runtime/decision.md +2 -2
  26. package/docs/local-runtime/hosting.md +103 -92
  27. package/docs/local-runtime/model-quality.md +103 -0
  28. package/docs/local-runtime/serving.md +1 -6
  29. package/docs/local-runtime/spark.md +2 -3
  30. package/docs/spec-architecture.html +2 -26
  31. package/docs/spec-workflow.md +9 -0
  32. package/node_modules/@isaacs/nonrecursive-glob-match/LICENSE.md +55 -0
  33. package/node_modules/@isaacs/nonrecursive-glob-match/README.md +364 -0
  34. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.d.ts +123 -0
  35. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.d.ts.map +1 -0
  36. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.js +778 -0
  37. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.js.map +1 -0
  38. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.min.js +3 -0
  39. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.min.js.map +7 -0
  40. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.d.ts +15 -0
  41. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.d.ts.map +1 -0
  42. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.js +76 -0
  43. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.js.map +1 -0
  44. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/package.json +3 -0
  45. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.d.ts +4 -0
  46. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.d.ts.map +1 -0
  47. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.js +142 -0
  48. package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.js.map +1 -0
  49. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.d.ts +123 -0
  50. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.d.ts.map +1 -0
  51. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.js +774 -0
  52. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.js.map +1 -0
  53. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.min.js +3 -0
  54. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.min.js.map +7 -0
  55. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.d.ts +15 -0
  56. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.d.ts.map +1 -0
  57. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.js +72 -0
  58. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.js.map +1 -0
  59. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/package.json +3 -0
  60. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.d.ts +4 -0
  61. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.d.ts.map +1 -0
  62. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.js +138 -0
  63. package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.js.map +1 -0
  64. package/node_modules/@isaacs/nonrecursive-glob-match/package.json +56 -0
  65. package/node_modules/@isaacs/nonrecursive-glob-match/src/index.ts +932 -0
  66. package/node_modules/@isaacs/nonrecursive-glob-match/src/numeric-range.ts +79 -0
  67. package/node_modules/@isaacs/nonrecursive-glob-match/src/parse-class.ts +158 -0
  68. package/node_modules/minimatch/README.md +125 -107
  69. package/node_modules/minimatch/dist/commonjs/index.d.ts +49 -38
  70. package/node_modules/minimatch/dist/commonjs/index.d.ts.map +1 -1
  71. package/node_modules/minimatch/dist/commonjs/index.js +782 -359
  72. package/node_modules/minimatch/dist/commonjs/index.js.map +1 -1
  73. package/node_modules/minimatch/dist/esm/index.d.ts +49 -38
  74. package/node_modules/minimatch/dist/esm/index.d.ts.map +1 -1
  75. package/node_modules/minimatch/dist/esm/index.js +781 -356
  76. package/node_modules/minimatch/dist/esm/index.js.map +1 -1
  77. package/node_modules/minimatch/package.json +5 -4
  78. package/package.json +4 -5
  79. package/training/adapter/src/train.py +16 -1
  80. package/dist/chunk-7CO5DBCY.js.map +0 -1
  81. package/dist/chunk-FSVTMX4L.js +0 -2820
  82. package/dist/chunk-FSVTMX4L.js.map +0 -1
  83. package/node_modules/minimatch/dist/commonjs/ast.d.ts +0 -22
  84. package/node_modules/minimatch/dist/commonjs/ast.d.ts.map +0 -1
  85. package/node_modules/minimatch/dist/commonjs/ast.js +0 -845
  86. package/node_modules/minimatch/dist/commonjs/ast.js.map +0 -1
  87. package/node_modules/minimatch/dist/commonjs/brace-expressions.d.ts +0 -8
  88. package/node_modules/minimatch/dist/commonjs/brace-expressions.d.ts.map +0 -1
  89. package/node_modules/minimatch/dist/commonjs/brace-expressions.js +0 -150
  90. package/node_modules/minimatch/dist/commonjs/brace-expressions.js.map +0 -1
  91. package/node_modules/minimatch/dist/esm/ast.d.ts +0 -22
  92. package/node_modules/minimatch/dist/esm/ast.d.ts.map +0 -1
  93. package/node_modules/minimatch/dist/esm/ast.js +0 -841
  94. package/node_modules/minimatch/dist/esm/ast.js.map +0 -1
  95. package/node_modules/minimatch/dist/esm/brace-expressions.d.ts +0 -8
  96. package/node_modules/minimatch/dist/esm/brace-expressions.d.ts.map +0 -1
  97. package/node_modules/minimatch/dist/esm/brace-expressions.js +0 -146
  98. package/node_modules/minimatch/dist/esm/brace-expressions.js.map +0 -1
  99. /package/dist/{prefetch-LBIOO7ZO.js.map → contracts-SHBUXEOQ.js.map} +0 -0
@@ -0,0 +1,18 @@
1
+ #!/usr/bin/env node
2
+ import {
3
+ buildFoundationModelServerLaunch,
4
+ buildLocalBaseModelServerLaunch,
5
+ buildLocalModelServerLaunch,
6
+ serveLocalModel,
7
+ verifyLocalModelServerLaunch
8
+ } from "./chunk-6CETSU2H.js";
9
+ import "./chunk-ST6ZSEUH.js";
10
+ import "./chunk-B2O6P7EX.js";
11
+ export {
12
+ buildFoundationModelServerLaunch,
13
+ buildLocalBaseModelServerLaunch,
14
+ buildLocalModelServerLaunch,
15
+ serveLocalModel,
16
+ verifyLocalModelServerLaunch
17
+ };
18
+ //# sourceMappingURL=model-server-IWCRFAOO.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}
@@ -6,7 +6,8 @@ import {
6
6
  safeName,
7
7
  verifyLocalBaseModel,
8
8
  verifyModelPrefetchCacheReport
9
- } from "./chunk-FSVTMX4L.js";
9
+ } from "./chunk-ST6ZSEUH.js";
10
+ import "./chunk-B2O6P7EX.js";
10
11
  export {
11
12
  buildModelPrefetchPayload,
12
13
  buildModelSourcePrefetchPayload,
@@ -15,4 +16,4 @@ export {
15
16
  verifyLocalBaseModel,
16
17
  verifyModelPrefetchCacheReport
17
18
  };
18
- //# sourceMappingURL=prefetch-LBIOO7ZO.js.map
19
+ //# sourceMappingURL=prefetch-Q2ZA3SJ7.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}
@@ -1,10 +1,10 @@
1
1
  # Architecture
2
2
 
3
3
  This document describes the local CUDA runtime bundled with the unified `tt`
4
- agent and CLI. The standalone `tt-local` product surface is deprecated. Local
5
- workflow commands need no TT token. Managed agent inference reuses a TT access
6
- token; BYO inference uses the selected provider's credentials. Cloud operations
7
- are explicit through `tt cloud` and use the TT account token.
4
+ agent and CLI. The standalone `tt-local` product surface is deprecated. Every
5
+ command runs on machines the user controls; there is no Tuned Tensor account,
6
+ token or hosted service. The optional conversational agent uses the model
7
+ provider and credentials the user configures.
8
8
 
9
9
  The local adapter runtime boundary is text SFT of a registry-certified
10
10
  Transformers causal LM into a LoRA adapter on CUDA. Certified checkpoints
@@ -81,7 +81,7 @@ GPU/driver, uv, rsync 3.2+, bash, setsid, and GNU timeout. TT
81
81
  copies the bundled Python source and lockfile and runs that exact runtime
82
82
  with `uv run --frozen`; there is no separate training image or Step Functions
83
83
  pipeline. Internet access on the instance is needed to install the locked
84
- runtime. No TT token is needed for training.
84
+ runtime. No Tuned Tensor account is involved.
85
85
 
86
86
  `profile` and `region` are optional. AWS CLI uses its normal credential chain,
87
87
  including environment credentials, configured profiles and SSO. For SSO, log
@@ -248,9 +248,8 @@ terminate or resize instances, request quotas, choose fallback instances or
248
248
  fall back to another account. An AWS profile alone cannot create capacity.
249
249
  The instance remains running after the job; its lifecycle is yours to manage.
250
250
 
251
- Hosted training submission (`tt cloud runs start` and `estimate`) is retired.
252
- Existing cloud reports and cancellation of older hosted jobs remain available.
253
- This CLI change does not delete deployed AWS infrastructure or historical data.
251
+ Tuned Tensor does not run a hosted training service. GPU work runs only on
252
+ machines you own or rent.
254
253
 
255
254
  ## Troubleshooting a first run
256
255
 
@@ -258,7 +257,7 @@ This CLI change does not delete deployed AWS infrastructure or historical data.
258
257
  | --- | --- |
259
258
  | AWS credentials expired or access denied | Renew your SSO login, check the selected profile/account, and ask for `ec2:DescribeInstances` permission. |
260
259
  | Instance not found or not running | Confirm the instance ID and region, and start the instance through AWS. |
261
- | GPU quota or capacity error during launch | Check the regional quota and AWS capacity; a TT token or credit top-up will not fix it. |
260
+ | GPU quota or capacity error during launch | Check the regional quota and AWS capacity with AWS. |
262
261
  | SSH timeout or permission denied | Check IP routing, security group TCP 22 access, SSH username, key and agent using the manual SSH command above. |
263
262
  | Host key verification failed | Verify the host fingerprint through AWS or your administrator; do not disable strict host checking. |
264
263
  | `uv` missing or CUDA probe fails | Check the remote login-shell PATH and GPU driver; rerun `tt doctor` after correcting the host. |
@@ -135,8 +135,8 @@ print(answer["choice"], answer["probabilities"])
135
135
  Pass the same instructions and criteria the model was trained with: they are
136
136
  part of its input. `resolved-workflow.json` records them under `question`.
137
137
  The tuned model is not registered with `tt models` and is not served through
138
- vLLM. To move inference off your machine, use [serverless hosting](hosting.md)
139
- with `tt cloud endpoints create`.
138
+ vLLM. To answer requests from another application, [host it
139
+ yourself](hosting.md) on infrastructure you control.
140
140
 
141
141
  ## Limits
142
142
 
@@ -1,121 +1,132 @@
1
- # Host a decision model
1
+ # Host a decision model yourself
2
2
 
3
- Use a serverless endpoint when a small decision model receives occasional
4
- requests and you want to keep it off your own machine. Hosting currently supports
5
- the pinned English Laya base checkpoint and complete TT decision fine-tunes.
6
- GPU models, adapters, custom tokenizers and custom calibration are not supported.
7
- The endpoint uses 6 GB of CPU memory and scales to zero when idle; the first
8
- request after an idle period may be slow or fail while it starts.
3
+ A tuned decision model is an ordinary directory on your machine:
4
+ `.tuned-tensor/decision-runs/<run-id>/train/model`. Tuned Tensor does not
5
+ operate a hosting service. To answer requests from another application, run the
6
+ model on infrastructure you control: the same machine, a server, a container
7
+ platform or a cloud account of your own.
9
8
 
10
- You need a TT account token and prepaid credits. Your application does not need
11
- AWS credentials. The hosted API must be enabled by the service operator before
12
- these commands work.
9
+ Laya runs on CPU, so no GPU is needed for inference. The former hosted service
10
+ ran one Laya checkpoint with 6 GB of CPU memory; use that as a starting point and
11
+ measure your own latency and memory before you size anything.
13
12
 
14
- ```bash
15
- tt auth login
16
- tt balance
17
- # Use a completed decision run's checkpoint.
18
- tt cloud endpoints create \
19
- --model .tuned-tensor/decision-runs/<run-id>/train/model \
20
- --name support-triage
21
- ```
22
-
23
- The command uploads only `model.safetensors` (at most 2 GB), without loading it
24
- into a model runtime. The serving image supplies the trusted Laya architecture
25
- and tokenizer. You do not need to upload the base model as well. To host the base
26
- model instead, point `--model` at its local checkpoint directory. There is no
27
- question or other inference configuration attached to a deployment. Send the
28
- complete model request each time you invoke it.
13
+ ## Load the model
29
14
 
30
- Save the endpoint ID, HTTPS inference URL and inference key returned by create.
31
- The key is shown only once. It can invoke only this endpoint; your account token
32
- continues to manage it. Check status until it reports `ready`:
15
+ Install the Laya version TT trains with, then load the run's model directory:
33
16
 
34
17
  ```bash
35
- tt cloud endpoints get <endpoint-id>
36
- tt cloud endpoints list
37
- tt cloud endpoints invoke <endpoint-id> --payload request.json
18
+ python -m pip install "laya==0.3.22"
38
19
  ```
39
20
 
40
- For the current Laya runtime, `request.json` contains both the input and question:
21
+ ```python
22
+ import laya
41
23
 
42
- ```json
43
- {
44
- "input": "My card payment was declined",
45
- "question": {
24
+ agent = laya.load("path/to/decision-runs/<run-id>/train/model")
25
+ question = {
46
26
  "type": "choice",
47
27
  "instructions": "Which support queue should handle this message?",
48
28
  "criteria": {
49
- "lost_card": "A lost or stolen bank card",
50
- "payment_declined": "A refused card payment"
51
- }
52
- }
29
+ "lost_card": "A lost or stolen bank card",
30
+ "payment_declined": "A refused card payment",
31
+ },
53
32
  }
33
+ answer = agent.predict("My card payment was declined", {"queue": question})["answers"]["queue"]
34
+ print(answer["choice"], answer["probabilities"])
54
35
  ```
55
36
 
56
- You may send a different question on the next request. The CLI sends this JSON
57
- object as-is; model-specific fields belong in the request, not deployment options.
58
- The hosting API bounds requests to 128 KiB; Laya validates its own fields.
37
+ Send the same instructions and criteria the model was trained with; they are
38
+ part of its input. The run's `resolved-workflow.json` records them under
39
+ `question`.
59
40
 
60
- Each successful request costs the fixed number of US cents displayed at creation.
61
- The price stays fixed for that endpoint. Failed requests are refunded. This
62
- feature does not debit an idle compute or deployment fee; the operator still
63
- incurs AWS storage and related costs. Use `tt balance` and `tt balance --help`
64
- for your account balance and billing commands. Each account may have five
65
- non-deleted endpoints and create at most ten endpoints in 24 hours.
41
+ ## A minimal HTTP service
66
42
 
67
- A request has an idempotency UUID, printed by `invoke`. If a connection drops or
68
- the API says the request is still running, retry the **same complete payload** with that ID:
43
+ The example below uses only the Python standard library besides Laya. It loads
44
+ the model once, accepts `POST` requests whose body holds `input` and `question`,
45
+ and returns Laya's typed answer.
46
+
47
+ ```python
48
+ # serve_decision.py
49
+ import json
50
+ import os
51
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
52
+
53
+ import laya
54
+
55
+ MODEL_DIR = os.environ["TT_DECISION_MODEL"]
56
+ API_KEY = os.environ["TT_DECISION_API_KEY"]
57
+ MAX_BODY_BYTES = 128 * 1024
58
+
59
+ agent = laya.load(MODEL_DIR)
60
+
61
+
62
+ class Handler(BaseHTTPRequestHandler):
63
+ def do_POST(self):
64
+ if self.headers.get("Authorization") != f"Bearer {API_KEY}":
65
+ return self.reply(401, {"error": "unauthorized"})
66
+ length = int(self.headers.get("Content-Length") or 0)
67
+ if length <= 0 or length > MAX_BODY_BYTES:
68
+ return self.reply(413, {"error": "body too large or empty"})
69
+ try:
70
+ body = json.loads(self.rfile.read(length))
71
+ answer = agent.predict(body["input"], {"answer": body["question"]})["answers"]["answer"]
72
+ except (KeyError, TypeError, ValueError) as error:
73
+ return self.reply(400, {"error": str(error)})
74
+ self.reply(200, {"result": answer})
75
+
76
+ def reply(self, status, payload):
77
+ data = json.dumps(payload).encode()
78
+ self.send_response(status)
79
+ self.send_header("Content-Type", "application/json")
80
+ self.send_header("Content-Length", str(len(data)))
81
+ self.end_headers()
82
+ self.wfile.write(data)
69
83
 
70
- ```bash
71
- tt cloud endpoints invoke <endpoint-id> --payload request.json \
72
- --request-id <original-request-uuid>
73
- ```
74
84
 
75
- Replays return the recorded answer without a second charge. Reusing the ID with
76
- a different payload (including question or options) is rejected. A failed/refunded request remains failed; use a new
77
- ID for a fresh attempt. Interrupted requests are refunded by periodic recovery
78
- within roughly 15 minutes. For Laya, input text is limited to 8192 characters; each request
79
- includes the question it should answer. After three failures in five
80
- minutes, wait five minutes before a new attempt. After 100 failures across the
81
- account in 24 hours, new requests pause until earlier failures age out; cached
82
- replays remain available.
85
+ ThreadingHTTPServer(("127.0.0.1", 8080), Handler).serve_forever()
86
+ ```
83
87
 
84
- For another application, call the URL from its backend:
88
+ ```bash
89
+ export TT_DECISION_MODEL=.tuned-tensor/decision-runs/<run-id>/train/model
90
+ export TT_DECISION_API_KEY="$(openssl rand -hex 32)"
91
+ python serve_decision.py
92
+ ```
85
93
 
86
94
  ```bash
87
- curl "$TT_INFERENCE_URL" \
88
- -H "Authorization: Bearer $TT_INFERENCE_KEY" \
89
- -H "Idempotency-Key: $(uuidgen)" \
95
+ curl http://127.0.0.1:8080 \
96
+ -H "Authorization: Bearer $TT_DECISION_API_KEY" \
90
97
  -H "Content-Type: application/json" \
91
98
  --data-binary @request.json
92
99
  ```
93
100
 
94
- The response's `data.result` is Laya's typed answer with probabilities. Store keys
95
- in your backend's secret storage; do not put them in public browser code. The
96
- service stores results for idempotent replay and request hashes rather than the request
97
- body. You can also open **Dashboard → Endpoints**, choose your endpoint, and click
98
- **Generate new inference key**. Confirm replacement, then copy the new key shown
99
- once. The old key stops working immediately; update your backend's secret.
100
- This page also lets you copy the inference URL and refresh deployment status.
101
- Settings continues to manage account access tokens, which have broader access.
101
+ `request.json` contains both the input and the question:
102
102
 
103
- Endpoint keys can also be rotated from the CLI:
104
-
105
- ```bash
106
- tt cloud endpoints rotate-key <endpoint-id>
107
- tt cloud endpoints delete <endpoint-id>
108
- tt cloud endpoints get <endpoint-id>
103
+ ```json
104
+ {
105
+ "input": "My card payment was declined",
106
+ "question": {
107
+ "type": "choice",
108
+ "instructions": "Which support queue should handle this message?",
109
+ "criteria": {
110
+ "lost_card": "A lost or stolen bank card",
111
+ "payment_declined": "A refused card payment"
112
+ }
113
+ }
114
+ }
109
115
  ```
110
116
 
111
- Deletion immediately blocks new requests. Cleanup removes the model, weights
112
- and SageMaker resources asynchronously; an endpoint still provisioning must finish
113
- that transition before AWS permits deletion. Late uploads go to temporary storage
114
- with automatic expiry and cannot recreate the serving artifact. Poll until
115
- `deleted`. Uploads abandoned for an hour, deployments that cannot become ready
116
- within an hour, and failed endpoints older than a day are cleaned automatically.
117
- Account deletion requires endpoint cleanup and accepted requests to finish.
118
-
119
- If creation fails after returning an endpoint ID, inspect that ID or delete it.
120
- If a deployment exists but the one-time key was lost, rotate it. Do not repeat
121
- `create` simply to retrieve the key: that allocates another endpoint.
117
+ This is a starting point, not a production server. Before exposing it beyond
118
+ your machine:
119
+
120
+ - Keep the listener on `127.0.0.1` and put a TLS-terminating reverse proxy or
121
+ your platform's load balancer in front of it.
122
+ - Store the API key in your platform's secret storage, never in browser code.
123
+ - Add the rate limits, logging and health checks your platform expects.
124
+ - Keep the model directory read-only and pin `laya==0.3.22` in your image.
125
+
126
+ ## Packaging for your own cloud
127
+
128
+ Any platform that runs a Python container works. Copy the model directory and
129
+ the script into the image, install `laya==0.3.22`, and start
130
+ `python serve_decision.py` with the two environment variables set. Bind to
131
+ `0.0.0.0` only inside a container whose port is reachable solely through your
132
+ proxy or load balancer. You own the account, the costs and the shutdown.
@@ -0,0 +1,103 @@
1
+ # Review and use a customized model through the agent
2
+
3
+ Adapter projects can now inspect data, review completed runs, activate a reviewed
4
+ model, serve it on the current CUDA machine, stop that service, and roll back
5
+ through conversation. Each change uses the existing reviewed-action approval
6
+ flow; inspection needs no approval.
7
+
8
+ ## Agree on success before training
9
+
10
+ Keep acceptance criteria in `tunedtensor.json`, alongside the existing scoring
11
+ and general-regression settings. These values are illustrative, not universal
12
+ quality standards:
13
+
14
+ ```json
15
+ {
16
+ "evaluation": {
17
+ "scoring": { "mode": "json_fields" },
18
+ "taskQuality": {
19
+ "minScore": 0.85,
20
+ "minPassRate": 0.85,
21
+ "minExamples": 40,
22
+ "minScoreDelta": 0
23
+ }
24
+ }
25
+ }
26
+ ```
27
+
28
+ Choose the scorer and thresholds for the actual task, using representative,
29
+ disjoint held-out examples. Keep a separate test set when selecting recipes
30
+ on validation results. `minScore` and `minPassRate` range from 0 to 1;
31
+ `minExamples` is the required evaluated count; `minScoreDelta` is the required
32
+ improvement over the baseline and defaults to zero. A pass-rate regression is
33
+ always rejected. Baseline and candidate must use the same prompts/references.
34
+ Training examples and dry-run output cannot establish acceptance.
35
+
36
+ The report records the policy used by that run. Agent activation and serving
37
+ require that policy, passing task and general-regression checks, unchanged
38
+ current criteria, and verified artifacts. Editing the spec cannot attach new
39
+ criteria to an old report. CLI activation of older reports without a policy
40
+ remains supported, but now also rejects task-score/pass-rate regressions.
41
+ Explicit CLI serving by model ID remains a diagnostic path without this quality
42
+ gate; integrity verification alone is not a quality assessment.
43
+
44
+ These policy and agent lifecycle features currently apply to adapter projects.
45
+ Foundation and decision engines retain their existing engine-specific workflows.
46
+
47
+ ## Conversation workflow
48
+
49
+ 1. “Inspect training.jsonl and validation.jsonl for duplicate prompts, conflicting
50
+ answers and overlap.” `assess_local_dataset` scans supported single-turn chat
51
+ JSONL, including pinned cached split sources. It reports exact normalized
52
+ duplicates, length distributions and bounded answer counts. This does not
53
+ establish semantic accuracy, license suitability, PII safety or business coverage.
54
+ 2. Agree on the task criteria, update the spec through its reviewed edit flow,
55
+ and preview/run training using the existing pipeline tools.
56
+ 3. “Review my latest run. Activate it if it meets our agreed criteria.”
57
+ `list_local_runs` discovers actual run IDs and `inspect_local_run` returns
58
+ verified scores, the recorded evaluation split, gates and a bounded set of failures. The activation proposal
59
+ seals the selected run, report, manifest, project configuration and current
60
+ active selection. Approval rechecks these; changes require a new proposal.
61
+ 4. “Serve this reviewed model on port 8000.” Approval starts loading; the agent
62
+ checks `get_local_service` for health and the exact `/v1/models` identity.
63
+ Starting a process is not reported as a ready service. The evaluated system
64
+ instruction is restored from the saved run, not taken from a later spec edit.
65
+ 5. “Stop this service” releases its process group. “Roll back the active model”
66
+ restores the previous verified selection, or clears it if none existed.
67
+
68
+ Activation changes the configured store's active pointer. If projects explicitly
69
+ share a store, undoing this project's activation may restore the exact previous
70
+ model from another project; its identity and evidence are verified again.
71
+ Unrelated projects cannot be selected through the report or activation tools.
72
+
73
+ Activation changes the active pointer. Starting/stopping a service is a separate
74
+ operation; rollback does not replace a running server. The service binds to
75
+ `127.0.0.1` only, needs Linux/CUDA and uses the pinned serving runtime. Its preview
76
+ shows a 2,048-token context, 256-token output default, one concurrent request and
77
+ GPU memory utilization 0.3. These conservative defaults are not hardware-optimal
78
+ benchmarks; use explicit CLI serving settings for larger contexts or deployment.
79
+
80
+ The agent owns one server per CLI process. It does not adopt an existing server
81
+ or arbitrary PID. Normal process exit stops its server; this is an interactive
82
+ local session, not a persistent deployment manager. The run's `agent-serving.log`
83
+ contains startup errors. Port conflicts and identity mismatch never establish
84
+ readiness. Approval results persist so subsequent turns can use the actual
85
+ model and server IDs without asking the user to paste terminal output.
86
+
87
+ An abrupt OS kill (for example SIGKILL) cannot run application cleanup; this
88
+ session manager is not a crash-recovery supervisor.
89
+
90
+ ## Repeating a training experiment
91
+
92
+ Adapter specs accept `hyperparameters.seed` (an integer from 0 through
93
+ 4,294,967,295; default 42). The runner seeds before loading the model and
94
+ initializing LoRA, then uses the same seed for Trainer and data sampling.
95
+ `training-metrics.json` records the effective seed, trainable parameter count
96
+ and peak CUDA allocated/reserved bytes. These are PyTorch allocator peaks,
97
+ not total device or Spark unified-memory usage. A seed improves repeatability;
98
+ it does not guarantee identical GPU kernels across hardware or library versions.
99
+ Keep model revision, data hashes, runtime versions and evaluation policy fixed
100
+ when comparing backends. Never treat training speed alone as model quality.
101
+
102
+ For a measured Spark comparison with Unsloth, including startup cost, memory,
103
+ quality and repeatability limits, see the [backend review](https://github.com/tunedtensor/tuned-tensor-cli/blob/main/docs/reviews/2026-11-10-unsloth-comparison.md).
@@ -2,15 +2,10 @@
2
2
 
3
3
  `tt serve` verifies TT artifacts and launches **vLLM 0.32.0**. vLLM owns model
4
4
  execution, batching, KV/prefix caching, streaming, and native tool-call parsing.
5
- This local serving path connects the harness directly to vLLM; TT's managed
6
- agent proxy is a separate inference option. The serving dependencies are locked
5
+ This local serving path connects the harness directly to vLLM. The serving dependencies are locked
7
6
  separately from training, in an external cache environment; an npm upgrade does
8
7
  not mutate the training runtime.
9
8
 
10
- The legacy `tt cloud models serve --managed` wrapper accepts completion bodies
11
- up to 8 MiB and up to eight simultaneous requests, including uploads and queued
12
- generations. It returns HTTP 413 for larger bodies and HTTP 503 when full.
13
-
14
9
  ## Requirements and scope
15
10
 
16
11
  - The packaged serving runtime requires **Linux and NVIDIA CUDA**. CPU/macOS
@@ -27,9 +27,8 @@ trainable; consumer cards typically only fit Qwen LoRA. `tt status` and the
27
27
  shell agent reuse the cached report.
28
28
 
29
29
  Local TT training requires Node 22.19+, `uv`, working CUDA PyTorch, and enough free space
30
- for the Hugging Face cache plus run artifacts. Local commands need no TT
31
- access token. Agent inference can use your TT token through the managed proxy,
32
- or your own provider credentials and model.
30
+ for the Hugging Face cache plus run artifacts. No Tuned Tensor account is
31
+ needed. Agent inference uses your own provider credentials and model.
33
32
 
34
33
  ## Create a project
35
34
 
@@ -107,7 +107,7 @@
107
107
  <div class="tt-unit"><button class="btn" type="button" data-component="serve" aria-pressed="false">Serve verified model</button><small class="text-small">vLLM → OpenAI-compatible clients</small></div>
108
108
  </div>
109
109
  <div class="tt-arrow text-small">↺ evaluation evidence informs the next reviewed spec edit</div>
110
- <div class="viz-row"><button class="btn btn-ghost" type="button" data-component="cli" aria-pressed="false">CLI entry points</button><button class="btn btn-ghost" type="button" data-component="publish" aria-pressed="false">Optional publish → dashboard</button></div>
110
+ <div class="viz-row"><button class="btn btn-ghost" type="button" data-component="cli" aria-pressed="false">CLI entry points</button></div>
111
111
  </div>
112
112
  <section class="tt-detail" aria-label="Selected component details">
113
113
  <hr>
@@ -125,7 +125,7 @@
125
125
  <div class="tt-tree">
126
126
  <details open><summary><code>src/</code> · TypeScript CLI and orchestration</summary>
127
127
  <ul><li><code>index.ts → cli.ts</code> — executable and command registration</li><li><code>shell.ts / agent.ts</code> — terminal routing and conversation control</li><li><code>local-agent-client.ts / agent-model.ts</code> — local agent loop and provider selection</li><li><code>agent-tools.ts / agent-approval.ts</code> — tool schemas, proposals and approval</li><li><code>local-pipeline-action.ts / spec-workspace.ts</code> — pipeline previews and reviewed spec edits</li><li><code>agent-store.ts / config.ts / paths.ts</code> — conversation, credentials and paths</li><li><code>pipeline.ts</code> — shared pipeline contract and execution planning</li><li><code>client.ts</code> — account HTTP API</li></ul>
128
- <details><summary><code>commands/</code> · user-facing actions</summary><ul><li><code>pipeline.ts</code> — init / validate / plan / run</li><li><code>spec.ts</code> — shared CLI and shell review</li><li><code>local.ts</code> — local runtime command forwarding</li><li><code>cloud.ts / auth.ts / publish.ts</code> — account operations</li></ul></details>
128
+ <details><summary><code>commands/</code> · user-facing actions</summary><ul><li><code>pipeline.ts</code> — init / validate / plan / run</li><li><code>spec.ts</code> — shared CLI and shell review</li><li><code>local.ts</code> — local runtime command forwarding</li><li><code>agent.ts</code> — agent provider and model selection</li></ul></details>
129
129
  <details><summary><code>local-runtime/</code> · execution machinery</summary><ul><li><code>contracts.ts / local-project.ts</code> — strict schemas, parsing and shared spec validation</li><li><code>orchestrator.ts / foundation-runner.ts</code> — separate engine coordinators</li><li><code>dataset.ts / evaluation.ts</code> — compile data, score predictions</li><li><code>process-training.ts / process-runner.ts / gpu-executor.ts</code> — process lifecycle and transport</li><li><code>store.ts / artifacts.ts / model-registry.ts</code> — records and artifact integrity</li><li><code>model-server.ts / serving-runtime.ts</code> — verified serving process</li></ul></details>
130
130
  </details>
131
131
  <details open><summary><code>training/</code> · independently locked runtimes</summary>
@@ -474,30 +474,6 @@
474
474
  ],
475
475
  "example": "tt serve local-<run-id>\n# Or a foundation checkpoint:\ntt serve foundation --checkpoint <model-dir> --tokenizer <tokenizer.json>"
476
476
  },
477
- "publish": {
478
- "title": "Publish & account operations",
479
- "location": "Laptop HTTP client → TT service",
480
- "summary": "Optionally publish adapter run evidence to the dashboard. Account commands separately inspect hosted records and manage authentication and usage.",
481
- "input": "Stored spec plus run report, including prompts and model outputs.",
482
- "output": "A dashboard run record and URL.",
483
- "boundary": "Publishing uploads evidence, not model weights. It needs a TT token. Hosted training start/estimate are retired; EC2 training uses pipeline run.",
484
- "flow": [
485
- "local store",
486
- "spec + report",
487
- "POST /publish/runs",
488
- "dashboard"
489
- ],
490
- "path": "src/commands/publish.ts",
491
- "line": 172,
492
- "code": " const report = runReportSchema.parse(rawReport);\n const specRecord = await store.getSpec(run.behavior_spec_id);\n const exampleCount = report.candidate.results.length\n || report.baseline.results.length;\n\n const payload = {\n spec: {\n id: specRecord.id,\n ...specRecord.spec,\n },\n report,",
493
- "files": [
494
- "src/commands/publish.ts",
495
- "src/client.ts",
496
- "src/commands/cloud.ts",
497
- "src/commands/auth.ts"
498
- ],
499
- "example": "tt publish <run-id> --dry-run\n# Optional upload, after reviewing the payload:\ntt publish <run-id>\ntt cloud runs list\ntt usage"
500
- },
501
477
  "review": {
502
478
  "title": "Direct review & history",
503
479
  "location": "Terminal · no model call",
@@ -233,3 +233,12 @@ Splitting a multi-gigabyte master does not remove that training-loader limit.
233
233
  Do not treat a successful split preview as proof that an arbitrarily large split
234
234
  can train; streaming training ingestion or bounded dataset selection is separate
235
235
  work. Use `tt validate` on the saved spec to validate the actual training inputs.
236
+
237
+ ## Task acceptance and conversational model use
238
+
239
+ Adapter specs can set `evaluation.taskQuality` before training. The report records
240
+ the criteria; agent activation and serving require passing task and general
241
+ regression checks, an unchanged policy, and verified artifacts. The agent can
242
+ inspect local datasets and completed runs directly, then prepare activation,
243
+ serving, stopping and rollback. See [model quality](local-runtime/model-quality.md)
244
+ for policy fields, lifecycle behavior and limits.
@@ -0,0 +1,55 @@
1
+ # Blue Oak Model License
2
+
3
+ Version 1.0.0
4
+
5
+ ## Purpose
6
+
7
+ This license gives everyone as much permission to work with
8
+ this software as possible, while protecting contributors
9
+ from liability.
10
+
11
+ ## Acceptance
12
+
13
+ In order to receive this license, you must agree to its
14
+ rules. The rules of this license are both obligations
15
+ under that agreement and conditions to your license.
16
+ You must not do anything with this software that triggers
17
+ a rule that you cannot or will not follow.
18
+
19
+ ## Copyright
20
+
21
+ Each contributor licenses you to do everything with this
22
+ software that would otherwise infringe that contributor's
23
+ copyright in it.
24
+
25
+ ## Notices
26
+
27
+ You must ensure that everyone who gets a copy of
28
+ any part of this software from you, with or without
29
+ changes, also gets the text of this license or a link to
30
+ <https://blueoakcouncil.org/license/1.0.0>.
31
+
32
+ ## Excuse
33
+
34
+ If anyone notifies you in writing that you have not
35
+ complied with [Notices](#notices), you can keep your
36
+ license by taking all practical steps to comply within 30
37
+ days after the notice. If you do not do so, your license
38
+ ends immediately.
39
+
40
+ ## Patent
41
+
42
+ Each contributor licenses you to do everything with this
43
+ software that would otherwise infringe any patent claims
44
+ they can license or become able to license.
45
+
46
+ ## Reliability
47
+
48
+ No contributor can revoke this license.
49
+
50
+ ## No Liability
51
+
52
+ ***As far as the law allows, this software comes as is,
53
+ without any warranty or condition, and no contributor
54
+ will be liable to anyone for any damages related to this
55
+ software or this license, under any kind of legal claim.***