@tuned-tensor/cli 0.28.1 → 0.30.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +76 -180
- package/dist/chunk-6CETSU2H.js +260 -0
- package/dist/chunk-6CETSU2H.js.map +1 -0
- package/dist/chunk-B2O6P7EX.js +1623 -0
- package/dist/chunk-B2O6P7EX.js.map +1 -0
- package/dist/{chunk-7CO5DBCY.js → chunk-GMBZ2FE6.js} +5046 -5014
- package/dist/chunk-GMBZ2FE6.js.map +1 -0
- package/dist/chunk-MM75IAY3.js +138 -0
- package/dist/chunk-MM75IAY3.js.map +1 -0
- package/dist/chunk-ST6ZSEUH.js +1272 -0
- package/dist/chunk-ST6ZSEUH.js.map +1 -0
- package/dist/contracts-SHBUXEOQ.js +72 -0
- package/dist/dataset-Y5NEUKI5.js +22 -0
- package/dist/dataset-Y5NEUKI5.js.map +1 -0
- package/dist/index.js +4530 -9349
- package/dist/index.js.map +1 -1
- package/dist/local-runtime.js +84 -467
- package/dist/local-runtime.js.map +1 -1
- package/dist/model-server-IWCRFAOO.js +18 -0
- package/dist/model-server-IWCRFAOO.js.map +1 -0
- package/dist/{prefetch-LBIOO7ZO.js → prefetch-Q2ZA3SJ7.js} +3 -2
- package/dist/prefetch-Q2ZA3SJ7.js.map +1 -0
- package/docs/local-runtime/architecture.md +4 -4
- package/docs/local-runtime/aws-gpu.md +4 -5
- package/docs/local-runtime/decision.md +2 -2
- package/docs/local-runtime/hosting.md +103 -92
- package/docs/local-runtime/model-quality.md +103 -0
- package/docs/local-runtime/serving.md +1 -6
- package/docs/local-runtime/spark.md +2 -3
- package/docs/spec-architecture.html +2 -26
- package/docs/spec-workflow.md +9 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/LICENSE.md +55 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/README.md +364 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.d.ts +123 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.js +778 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.min.js +3 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/index.min.js.map +7 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.d.ts +15 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.js +76 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/numeric-range.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/package.json +3 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.d.ts +4 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.js +142 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/commonjs/parse-class.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.d.ts +123 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.js +774 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.min.js +3 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/index.min.js.map +7 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.d.ts +15 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.js +72 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/numeric-range.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/package.json +3 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.d.ts +4 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.d.ts.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.js +138 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/dist/esm/parse-class.js.map +1 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/package.json +56 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/src/index.ts +932 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/src/numeric-range.ts +79 -0
- package/node_modules/@isaacs/nonrecursive-glob-match/src/parse-class.ts +158 -0
- package/node_modules/minimatch/README.md +125 -107
- package/node_modules/minimatch/dist/commonjs/index.d.ts +49 -38
- package/node_modules/minimatch/dist/commonjs/index.d.ts.map +1 -1
- package/node_modules/minimatch/dist/commonjs/index.js +782 -359
- package/node_modules/minimatch/dist/commonjs/index.js.map +1 -1
- package/node_modules/minimatch/dist/esm/index.d.ts +49 -38
- package/node_modules/minimatch/dist/esm/index.d.ts.map +1 -1
- package/node_modules/minimatch/dist/esm/index.js +781 -356
- package/node_modules/minimatch/dist/esm/index.js.map +1 -1
- package/node_modules/minimatch/package.json +5 -4
- package/package.json +4 -5
- package/training/adapter/src/train.py +16 -1
- package/dist/chunk-7CO5DBCY.js.map +0 -1
- package/dist/chunk-FSVTMX4L.js +0 -2820
- package/dist/chunk-FSVTMX4L.js.map +0 -1
- package/node_modules/minimatch/dist/commonjs/ast.d.ts +0 -22
- package/node_modules/minimatch/dist/commonjs/ast.d.ts.map +0 -1
- package/node_modules/minimatch/dist/commonjs/ast.js +0 -845
- package/node_modules/minimatch/dist/commonjs/ast.js.map +0 -1
- package/node_modules/minimatch/dist/commonjs/brace-expressions.d.ts +0 -8
- package/node_modules/minimatch/dist/commonjs/brace-expressions.d.ts.map +0 -1
- package/node_modules/minimatch/dist/commonjs/brace-expressions.js +0 -150
- package/node_modules/minimatch/dist/commonjs/brace-expressions.js.map +0 -1
- package/node_modules/minimatch/dist/esm/ast.d.ts +0 -22
- package/node_modules/minimatch/dist/esm/ast.d.ts.map +0 -1
- package/node_modules/minimatch/dist/esm/ast.js +0 -841
- package/node_modules/minimatch/dist/esm/ast.js.map +0 -1
- package/node_modules/minimatch/dist/esm/brace-expressions.d.ts +0 -8
- package/node_modules/minimatch/dist/esm/brace-expressions.d.ts.map +0 -1
- package/node_modules/minimatch/dist/esm/brace-expressions.js +0 -146
- package/node_modules/minimatch/dist/esm/brace-expressions.js.map +0 -1
- /package/dist/{prefetch-LBIOO7ZO.js.map → contracts-SHBUXEOQ.js.map} +0 -0
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import {
|
|
3
|
+
buildFoundationModelServerLaunch,
|
|
4
|
+
buildLocalBaseModelServerLaunch,
|
|
5
|
+
buildLocalModelServerLaunch,
|
|
6
|
+
serveLocalModel,
|
|
7
|
+
verifyLocalModelServerLaunch
|
|
8
|
+
} from "./chunk-6CETSU2H.js";
|
|
9
|
+
import "./chunk-ST6ZSEUH.js";
|
|
10
|
+
import "./chunk-B2O6P7EX.js";
|
|
11
|
+
export {
|
|
12
|
+
buildFoundationModelServerLaunch,
|
|
13
|
+
buildLocalBaseModelServerLaunch,
|
|
14
|
+
buildLocalModelServerLaunch,
|
|
15
|
+
serveLocalModel,
|
|
16
|
+
verifyLocalModelServerLaunch
|
|
17
|
+
};
|
|
18
|
+
//# sourceMappingURL=model-server-IWCRFAOO.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}
|
|
@@ -6,7 +6,8 @@ import {
|
|
|
6
6
|
safeName,
|
|
7
7
|
verifyLocalBaseModel,
|
|
8
8
|
verifyModelPrefetchCacheReport
|
|
9
|
-
} from "./chunk-
|
|
9
|
+
} from "./chunk-ST6ZSEUH.js";
|
|
10
|
+
import "./chunk-B2O6P7EX.js";
|
|
10
11
|
export {
|
|
11
12
|
buildModelPrefetchPayload,
|
|
12
13
|
buildModelSourcePrefetchPayload,
|
|
@@ -15,4 +16,4 @@ export {
|
|
|
15
16
|
verifyLocalBaseModel,
|
|
16
17
|
verifyModelPrefetchCacheReport
|
|
17
18
|
};
|
|
18
|
-
//# sourceMappingURL=prefetch-
|
|
19
|
+
//# sourceMappingURL=prefetch-Q2ZA3SJ7.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Architecture
|
|
2
2
|
|
|
3
3
|
This document describes the local CUDA runtime bundled with the unified `tt`
|
|
4
|
-
agent and CLI. The standalone `tt-local` product surface is deprecated.
|
|
5
|
-
|
|
6
|
-
token
|
|
7
|
-
|
|
4
|
+
agent and CLI. The standalone `tt-local` product surface is deprecated. Every
|
|
5
|
+
command runs on machines the user controls; there is no Tuned Tensor account,
|
|
6
|
+
token or hosted service. The optional conversational agent uses the model
|
|
7
|
+
provider and credentials the user configures.
|
|
8
8
|
|
|
9
9
|
The local adapter runtime boundary is text SFT of a registry-certified
|
|
10
10
|
Transformers causal LM into a LoRA adapter on CUDA. Certified checkpoints
|
|
@@ -81,7 +81,7 @@ GPU/driver, uv, rsync 3.2+, bash, setsid, and GNU timeout. TT
|
|
|
81
81
|
copies the bundled Python source and lockfile and runs that exact runtime
|
|
82
82
|
with `uv run --frozen`; there is no separate training image or Step Functions
|
|
83
83
|
pipeline. Internet access on the instance is needed to install the locked
|
|
84
|
-
runtime. No
|
|
84
|
+
runtime. No Tuned Tensor account is involved.
|
|
85
85
|
|
|
86
86
|
`profile` and `region` are optional. AWS CLI uses its normal credential chain,
|
|
87
87
|
including environment credentials, configured profiles and SSO. For SSO, log
|
|
@@ -248,9 +248,8 @@ terminate or resize instances, request quotas, choose fallback instances or
|
|
|
248
248
|
fall back to another account. An AWS profile alone cannot create capacity.
|
|
249
249
|
The instance remains running after the job; its lifecycle is yours to manage.
|
|
250
250
|
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
This CLI change does not delete deployed AWS infrastructure or historical data.
|
|
251
|
+
Tuned Tensor does not run a hosted training service. GPU work runs only on
|
|
252
|
+
machines you own or rent.
|
|
254
253
|
|
|
255
254
|
## Troubleshooting a first run
|
|
256
255
|
|
|
@@ -258,7 +257,7 @@ This CLI change does not delete deployed AWS infrastructure or historical data.
|
|
|
258
257
|
| --- | --- |
|
|
259
258
|
| AWS credentials expired or access denied | Renew your SSO login, check the selected profile/account, and ask for `ec2:DescribeInstances` permission. |
|
|
260
259
|
| Instance not found or not running | Confirm the instance ID and region, and start the instance through AWS. |
|
|
261
|
-
| GPU quota or capacity error during launch | Check the regional quota and AWS capacity
|
|
260
|
+
| GPU quota or capacity error during launch | Check the regional quota and AWS capacity with AWS. |
|
|
262
261
|
| SSH timeout or permission denied | Check IP routing, security group TCP 22 access, SSH username, key and agent using the manual SSH command above. |
|
|
263
262
|
| Host key verification failed | Verify the host fingerprint through AWS or your administrator; do not disable strict host checking. |
|
|
264
263
|
| `uv` missing or CUDA probe fails | Check the remote login-shell PATH and GPU driver; rerun `tt doctor` after correcting the host. |
|
|
@@ -135,8 +135,8 @@ print(answer["choice"], answer["probabilities"])
|
|
|
135
135
|
Pass the same instructions and criteria the model was trained with: they are
|
|
136
136
|
part of its input. `resolved-workflow.json` records them under `question`.
|
|
137
137
|
The tuned model is not registered with `tt models` and is not served through
|
|
138
|
-
vLLM. To
|
|
139
|
-
|
|
138
|
+
vLLM. To answer requests from another application, [host it
|
|
139
|
+
yourself](hosting.md) on infrastructure you control.
|
|
140
140
|
|
|
141
141
|
## Limits
|
|
142
142
|
|
|
@@ -1,121 +1,132 @@
|
|
|
1
|
-
# Host a decision model
|
|
1
|
+
# Host a decision model yourself
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
request after an idle period may be slow or fail while it starts.
|
|
3
|
+
A tuned decision model is an ordinary directory on your machine:
|
|
4
|
+
`.tuned-tensor/decision-runs/<run-id>/train/model`. Tuned Tensor does not
|
|
5
|
+
operate a hosting service. To answer requests from another application, run the
|
|
6
|
+
model on infrastructure you control: the same machine, a server, a container
|
|
7
|
+
platform or a cloud account of your own.
|
|
9
8
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
9
|
+
Laya runs on CPU, so no GPU is needed for inference. The former hosted service
|
|
10
|
+
ran one Laya checkpoint with 6 GB of CPU memory; use that as a starting point and
|
|
11
|
+
measure your own latency and memory before you size anything.
|
|
13
12
|
|
|
14
|
-
|
|
15
|
-
tt auth login
|
|
16
|
-
tt balance
|
|
17
|
-
# Use a completed decision run's checkpoint.
|
|
18
|
-
tt cloud endpoints create \
|
|
19
|
-
--model .tuned-tensor/decision-runs/<run-id>/train/model \
|
|
20
|
-
--name support-triage
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
The command uploads only `model.safetensors` (at most 2 GB), without loading it
|
|
24
|
-
into a model runtime. The serving image supplies the trusted Laya architecture
|
|
25
|
-
and tokenizer. You do not need to upload the base model as well. To host the base
|
|
26
|
-
model instead, point `--model` at its local checkpoint directory. There is no
|
|
27
|
-
question or other inference configuration attached to a deployment. Send the
|
|
28
|
-
complete model request each time you invoke it.
|
|
13
|
+
## Load the model
|
|
29
14
|
|
|
30
|
-
|
|
31
|
-
The key is shown only once. It can invoke only this endpoint; your account token
|
|
32
|
-
continues to manage it. Check status until it reports `ready`:
|
|
15
|
+
Install the Laya version TT trains with, then load the run's model directory:
|
|
33
16
|
|
|
34
17
|
```bash
|
|
35
|
-
|
|
36
|
-
tt cloud endpoints list
|
|
37
|
-
tt cloud endpoints invoke <endpoint-id> --payload request.json
|
|
18
|
+
python -m pip install "laya==0.3.22"
|
|
38
19
|
```
|
|
39
20
|
|
|
40
|
-
|
|
21
|
+
```python
|
|
22
|
+
import laya
|
|
41
23
|
|
|
42
|
-
|
|
43
|
-
{
|
|
44
|
-
"input": "My card payment was declined",
|
|
45
|
-
"question": {
|
|
24
|
+
agent = laya.load("path/to/decision-runs/<run-id>/train/model")
|
|
25
|
+
question = {
|
|
46
26
|
"type": "choice",
|
|
47
27
|
"instructions": "Which support queue should handle this message?",
|
|
48
28
|
"criteria": {
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
}
|
|
52
|
-
}
|
|
29
|
+
"lost_card": "A lost or stolen bank card",
|
|
30
|
+
"payment_declined": "A refused card payment",
|
|
31
|
+
},
|
|
53
32
|
}
|
|
33
|
+
answer = agent.predict("My card payment was declined", {"queue": question})["answers"]["queue"]
|
|
34
|
+
print(answer["choice"], answer["probabilities"])
|
|
54
35
|
```
|
|
55
36
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
37
|
+
Send the same instructions and criteria the model was trained with; they are
|
|
38
|
+
part of its input. The run's `resolved-workflow.json` records them under
|
|
39
|
+
`question`.
|
|
59
40
|
|
|
60
|
-
|
|
61
|
-
The price stays fixed for that endpoint. Failed requests are refunded. This
|
|
62
|
-
feature does not debit an idle compute or deployment fee; the operator still
|
|
63
|
-
incurs AWS storage and related costs. Use `tt balance` and `tt balance --help`
|
|
64
|
-
for your account balance and billing commands. Each account may have five
|
|
65
|
-
non-deleted endpoints and create at most ten endpoints in 24 hours.
|
|
41
|
+
## A minimal HTTP service
|
|
66
42
|
|
|
67
|
-
|
|
68
|
-
the
|
|
43
|
+
The example below uses only the Python standard library besides Laya. It loads
|
|
44
|
+
the model once, accepts `POST` requests whose body holds `input` and `question`,
|
|
45
|
+
and returns Laya's typed answer.
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
# serve_decision.py
|
|
49
|
+
import json
|
|
50
|
+
import os
|
|
51
|
+
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
52
|
+
|
|
53
|
+
import laya
|
|
54
|
+
|
|
55
|
+
MODEL_DIR = os.environ["TT_DECISION_MODEL"]
|
|
56
|
+
API_KEY = os.environ["TT_DECISION_API_KEY"]
|
|
57
|
+
MAX_BODY_BYTES = 128 * 1024
|
|
58
|
+
|
|
59
|
+
agent = laya.load(MODEL_DIR)
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
class Handler(BaseHTTPRequestHandler):
|
|
63
|
+
def do_POST(self):
|
|
64
|
+
if self.headers.get("Authorization") != f"Bearer {API_KEY}":
|
|
65
|
+
return self.reply(401, {"error": "unauthorized"})
|
|
66
|
+
length = int(self.headers.get("Content-Length") or 0)
|
|
67
|
+
if length <= 0 or length > MAX_BODY_BYTES:
|
|
68
|
+
return self.reply(413, {"error": "body too large or empty"})
|
|
69
|
+
try:
|
|
70
|
+
body = json.loads(self.rfile.read(length))
|
|
71
|
+
answer = agent.predict(body["input"], {"answer": body["question"]})["answers"]["answer"]
|
|
72
|
+
except (KeyError, TypeError, ValueError) as error:
|
|
73
|
+
return self.reply(400, {"error": str(error)})
|
|
74
|
+
self.reply(200, {"result": answer})
|
|
75
|
+
|
|
76
|
+
def reply(self, status, payload):
|
|
77
|
+
data = json.dumps(payload).encode()
|
|
78
|
+
self.send_response(status)
|
|
79
|
+
self.send_header("Content-Type", "application/json")
|
|
80
|
+
self.send_header("Content-Length", str(len(data)))
|
|
81
|
+
self.end_headers()
|
|
82
|
+
self.wfile.write(data)
|
|
69
83
|
|
|
70
|
-
```bash
|
|
71
|
-
tt cloud endpoints invoke <endpoint-id> --payload request.json \
|
|
72
|
-
--request-id <original-request-uuid>
|
|
73
|
-
```
|
|
74
84
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
ID for a fresh attempt. Interrupted requests are refunded by periodic recovery
|
|
78
|
-
within roughly 15 minutes. For Laya, input text is limited to 8192 characters; each request
|
|
79
|
-
includes the question it should answer. After three failures in five
|
|
80
|
-
minutes, wait five minutes before a new attempt. After 100 failures across the
|
|
81
|
-
account in 24 hours, new requests pause until earlier failures age out; cached
|
|
82
|
-
replays remain available.
|
|
85
|
+
ThreadingHTTPServer(("127.0.0.1", 8080), Handler).serve_forever()
|
|
86
|
+
```
|
|
83
87
|
|
|
84
|
-
|
|
88
|
+
```bash
|
|
89
|
+
export TT_DECISION_MODEL=.tuned-tensor/decision-runs/<run-id>/train/model
|
|
90
|
+
export TT_DECISION_API_KEY="$(openssl rand -hex 32)"
|
|
91
|
+
python serve_decision.py
|
|
92
|
+
```
|
|
85
93
|
|
|
86
94
|
```bash
|
|
87
|
-
curl
|
|
88
|
-
-H "Authorization: Bearer $
|
|
89
|
-
-H "Idempotency-Key: $(uuidgen)" \
|
|
95
|
+
curl http://127.0.0.1:8080 \
|
|
96
|
+
-H "Authorization: Bearer $TT_DECISION_API_KEY" \
|
|
90
97
|
-H "Content-Type: application/json" \
|
|
91
98
|
--data-binary @request.json
|
|
92
99
|
```
|
|
93
100
|
|
|
94
|
-
|
|
95
|
-
in your backend's secret storage; do not put them in public browser code. The
|
|
96
|
-
service stores results for idempotent replay and request hashes rather than the request
|
|
97
|
-
body. You can also open **Dashboard → Endpoints**, choose your endpoint, and click
|
|
98
|
-
**Generate new inference key**. Confirm replacement, then copy the new key shown
|
|
99
|
-
once. The old key stops working immediately; update your backend's secret.
|
|
100
|
-
This page also lets you copy the inference URL and refresh deployment status.
|
|
101
|
-
Settings continues to manage account access tokens, which have broader access.
|
|
101
|
+
`request.json` contains both the input and the question:
|
|
102
102
|
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
103
|
+
```json
|
|
104
|
+
{
|
|
105
|
+
"input": "My card payment was declined",
|
|
106
|
+
"question": {
|
|
107
|
+
"type": "choice",
|
|
108
|
+
"instructions": "Which support queue should handle this message?",
|
|
109
|
+
"criteria": {
|
|
110
|
+
"lost_card": "A lost or stolen bank card",
|
|
111
|
+
"payment_declined": "A refused card payment"
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
}
|
|
109
115
|
```
|
|
110
116
|
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
117
|
+
This is a starting point, not a production server. Before exposing it beyond
|
|
118
|
+
your machine:
|
|
119
|
+
|
|
120
|
+
- Keep the listener on `127.0.0.1` and put a TLS-terminating reverse proxy or
|
|
121
|
+
your platform's load balancer in front of it.
|
|
122
|
+
- Store the API key in your platform's secret storage, never in browser code.
|
|
123
|
+
- Add the rate limits, logging and health checks your platform expects.
|
|
124
|
+
- Keep the model directory read-only and pin `laya==0.3.22` in your image.
|
|
125
|
+
|
|
126
|
+
## Packaging for your own cloud
|
|
127
|
+
|
|
128
|
+
Any platform that runs a Python container works. Copy the model directory and
|
|
129
|
+
the script into the image, install `laya==0.3.22`, and start
|
|
130
|
+
`python serve_decision.py` with the two environment variables set. Bind to
|
|
131
|
+
`0.0.0.0` only inside a container whose port is reachable solely through your
|
|
132
|
+
proxy or load balancer. You own the account, the costs and the shutdown.
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# Review and use a customized model through the agent
|
|
2
|
+
|
|
3
|
+
Adapter projects can now inspect data, review completed runs, activate a reviewed
|
|
4
|
+
model, serve it on the current CUDA machine, stop that service, and roll back
|
|
5
|
+
through conversation. Each change uses the existing reviewed-action approval
|
|
6
|
+
flow; inspection needs no approval.
|
|
7
|
+
|
|
8
|
+
## Agree on success before training
|
|
9
|
+
|
|
10
|
+
Keep acceptance criteria in `tunedtensor.json`, alongside the existing scoring
|
|
11
|
+
and general-regression settings. These values are illustrative, not universal
|
|
12
|
+
quality standards:
|
|
13
|
+
|
|
14
|
+
```json
|
|
15
|
+
{
|
|
16
|
+
"evaluation": {
|
|
17
|
+
"scoring": { "mode": "json_fields" },
|
|
18
|
+
"taskQuality": {
|
|
19
|
+
"minScore": 0.85,
|
|
20
|
+
"minPassRate": 0.85,
|
|
21
|
+
"minExamples": 40,
|
|
22
|
+
"minScoreDelta": 0
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
}
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Choose the scorer and thresholds for the actual task, using representative,
|
|
29
|
+
disjoint held-out examples. Keep a separate test set when selecting recipes
|
|
30
|
+
on validation results. `minScore` and `minPassRate` range from 0 to 1;
|
|
31
|
+
`minExamples` is the required evaluated count; `minScoreDelta` is the required
|
|
32
|
+
improvement over the baseline and defaults to zero. A pass-rate regression is
|
|
33
|
+
always rejected. Baseline and candidate must use the same prompts/references.
|
|
34
|
+
Training examples and dry-run output cannot establish acceptance.
|
|
35
|
+
|
|
36
|
+
The report records the policy used by that run. Agent activation and serving
|
|
37
|
+
require that policy, passing task and general-regression checks, unchanged
|
|
38
|
+
current criteria, and verified artifacts. Editing the spec cannot attach new
|
|
39
|
+
criteria to an old report. CLI activation of older reports without a policy
|
|
40
|
+
remains supported, but now also rejects task-score/pass-rate regressions.
|
|
41
|
+
Explicit CLI serving by model ID remains a diagnostic path without this quality
|
|
42
|
+
gate; integrity verification alone is not a quality assessment.
|
|
43
|
+
|
|
44
|
+
These policy and agent lifecycle features currently apply to adapter projects.
|
|
45
|
+
Foundation and decision engines retain their existing engine-specific workflows.
|
|
46
|
+
|
|
47
|
+
## Conversation workflow
|
|
48
|
+
|
|
49
|
+
1. “Inspect training.jsonl and validation.jsonl for duplicate prompts, conflicting
|
|
50
|
+
answers and overlap.” `assess_local_dataset` scans supported single-turn chat
|
|
51
|
+
JSONL, including pinned cached split sources. It reports exact normalized
|
|
52
|
+
duplicates, length distributions and bounded answer counts. This does not
|
|
53
|
+
establish semantic accuracy, license suitability, PII safety or business coverage.
|
|
54
|
+
2. Agree on the task criteria, update the spec through its reviewed edit flow,
|
|
55
|
+
and preview/run training using the existing pipeline tools.
|
|
56
|
+
3. “Review my latest run. Activate it if it meets our agreed criteria.”
|
|
57
|
+
`list_local_runs` discovers actual run IDs and `inspect_local_run` returns
|
|
58
|
+
verified scores, the recorded evaluation split, gates and a bounded set of failures. The activation proposal
|
|
59
|
+
seals the selected run, report, manifest, project configuration and current
|
|
60
|
+
active selection. Approval rechecks these; changes require a new proposal.
|
|
61
|
+
4. “Serve this reviewed model on port 8000.” Approval starts loading; the agent
|
|
62
|
+
checks `get_local_service` for health and the exact `/v1/models` identity.
|
|
63
|
+
Starting a process is not reported as a ready service. The evaluated system
|
|
64
|
+
instruction is restored from the saved run, not taken from a later spec edit.
|
|
65
|
+
5. “Stop this service” releases its process group. “Roll back the active model”
|
|
66
|
+
restores the previous verified selection, or clears it if none existed.
|
|
67
|
+
|
|
68
|
+
Activation changes the configured store's active pointer. If projects explicitly
|
|
69
|
+
share a store, undoing this project's activation may restore the exact previous
|
|
70
|
+
model from another project; its identity and evidence are verified again.
|
|
71
|
+
Unrelated projects cannot be selected through the report or activation tools.
|
|
72
|
+
|
|
73
|
+
Activation changes the active pointer. Starting/stopping a service is a separate
|
|
74
|
+
operation; rollback does not replace a running server. The service binds to
|
|
75
|
+
`127.0.0.1` only, needs Linux/CUDA and uses the pinned serving runtime. Its preview
|
|
76
|
+
shows a 2,048-token context, 256-token output default, one concurrent request and
|
|
77
|
+
GPU memory utilization 0.3. These conservative defaults are not hardware-optimal
|
|
78
|
+
benchmarks; use explicit CLI serving settings for larger contexts or deployment.
|
|
79
|
+
|
|
80
|
+
The agent owns one server per CLI process. It does not adopt an existing server
|
|
81
|
+
or arbitrary PID. Normal process exit stops its server; this is an interactive
|
|
82
|
+
local session, not a persistent deployment manager. The run's `agent-serving.log`
|
|
83
|
+
contains startup errors. Port conflicts and identity mismatch never establish
|
|
84
|
+
readiness. Approval results persist so subsequent turns can use the actual
|
|
85
|
+
model and server IDs without asking the user to paste terminal output.
|
|
86
|
+
|
|
87
|
+
An abrupt OS kill (for example SIGKILL) cannot run application cleanup; this
|
|
88
|
+
session manager is not a crash-recovery supervisor.
|
|
89
|
+
|
|
90
|
+
## Repeating a training experiment
|
|
91
|
+
|
|
92
|
+
Adapter specs accept `hyperparameters.seed` (an integer from 0 through
|
|
93
|
+
4,294,967,295; default 42). The runner seeds before loading the model and
|
|
94
|
+
initializing LoRA, then uses the same seed for Trainer and data sampling.
|
|
95
|
+
`training-metrics.json` records the effective seed, trainable parameter count
|
|
96
|
+
and peak CUDA allocated/reserved bytes. These are PyTorch allocator peaks,
|
|
97
|
+
not total device or Spark unified-memory usage. A seed improves repeatability;
|
|
98
|
+
it does not guarantee identical GPU kernels across hardware or library versions.
|
|
99
|
+
Keep model revision, data hashes, runtime versions and evaluation policy fixed
|
|
100
|
+
when comparing backends. Never treat training speed alone as model quality.
|
|
101
|
+
|
|
102
|
+
For a measured Spark comparison with Unsloth, including startup cost, memory,
|
|
103
|
+
quality and repeatability limits, see the [backend review](https://github.com/tunedtensor/tuned-tensor-cli/blob/main/docs/reviews/2026-11-10-unsloth-comparison.md).
|
|
@@ -2,15 +2,10 @@
|
|
|
2
2
|
|
|
3
3
|
`tt serve` verifies TT artifacts and launches **vLLM 0.32.0**. vLLM owns model
|
|
4
4
|
execution, batching, KV/prefix caching, streaming, and native tool-call parsing.
|
|
5
|
-
This local serving path connects the harness directly to vLLM
|
|
6
|
-
agent proxy is a separate inference option. The serving dependencies are locked
|
|
5
|
+
This local serving path connects the harness directly to vLLM. The serving dependencies are locked
|
|
7
6
|
separately from training, in an external cache environment; an npm upgrade does
|
|
8
7
|
not mutate the training runtime.
|
|
9
8
|
|
|
10
|
-
The legacy `tt cloud models serve --managed` wrapper accepts completion bodies
|
|
11
|
-
up to 8 MiB and up to eight simultaneous requests, including uploads and queued
|
|
12
|
-
generations. It returns HTTP 413 for larger bodies and HTTP 503 when full.
|
|
13
|
-
|
|
14
9
|
## Requirements and scope
|
|
15
10
|
|
|
16
11
|
- The packaged serving runtime requires **Linux and NVIDIA CUDA**. CPU/macOS
|
|
@@ -27,9 +27,8 @@ trainable; consumer cards typically only fit Qwen LoRA. `tt status` and the
|
|
|
27
27
|
shell agent reuse the cached report.
|
|
28
28
|
|
|
29
29
|
Local TT training requires Node 22.19+, `uv`, working CUDA PyTorch, and enough free space
|
|
30
|
-
for the Hugging Face cache plus run artifacts.
|
|
31
|
-
|
|
32
|
-
or your own provider credentials and model.
|
|
30
|
+
for the Hugging Face cache plus run artifacts. No Tuned Tensor account is
|
|
31
|
+
needed. Agent inference uses your own provider credentials and model.
|
|
33
32
|
|
|
34
33
|
## Create a project
|
|
35
34
|
|
|
@@ -107,7 +107,7 @@
|
|
|
107
107
|
<div class="tt-unit"><button class="btn" type="button" data-component="serve" aria-pressed="false">Serve verified model</button><small class="text-small">vLLM → OpenAI-compatible clients</small></div>
|
|
108
108
|
</div>
|
|
109
109
|
<div class="tt-arrow text-small">↺ evaluation evidence informs the next reviewed spec edit</div>
|
|
110
|
-
<div class="viz-row"><button class="btn btn-ghost" type="button" data-component="cli" aria-pressed="false">CLI entry points</button
|
|
110
|
+
<div class="viz-row"><button class="btn btn-ghost" type="button" data-component="cli" aria-pressed="false">CLI entry points</button></div>
|
|
111
111
|
</div>
|
|
112
112
|
<section class="tt-detail" aria-label="Selected component details">
|
|
113
113
|
<hr>
|
|
@@ -125,7 +125,7 @@
|
|
|
125
125
|
<div class="tt-tree">
|
|
126
126
|
<details open><summary><code>src/</code> · TypeScript CLI and orchestration</summary>
|
|
127
127
|
<ul><li><code>index.ts → cli.ts</code> — executable and command registration</li><li><code>shell.ts / agent.ts</code> — terminal routing and conversation control</li><li><code>local-agent-client.ts / agent-model.ts</code> — local agent loop and provider selection</li><li><code>agent-tools.ts / agent-approval.ts</code> — tool schemas, proposals and approval</li><li><code>local-pipeline-action.ts / spec-workspace.ts</code> — pipeline previews and reviewed spec edits</li><li><code>agent-store.ts / config.ts / paths.ts</code> — conversation, credentials and paths</li><li><code>pipeline.ts</code> — shared pipeline contract and execution planning</li><li><code>client.ts</code> — account HTTP API</li></ul>
|
|
128
|
-
<details><summary><code>commands/</code> · user-facing actions</summary><ul><li><code>pipeline.ts</code> — init / validate / plan / run</li><li><code>spec.ts</code> — shared CLI and shell review</li><li><code>local.ts</code> — local runtime command forwarding</li><li><code>
|
|
128
|
+
<details><summary><code>commands/</code> · user-facing actions</summary><ul><li><code>pipeline.ts</code> — init / validate / plan / run</li><li><code>spec.ts</code> — shared CLI and shell review</li><li><code>local.ts</code> — local runtime command forwarding</li><li><code>agent.ts</code> — agent provider and model selection</li></ul></details>
|
|
129
129
|
<details><summary><code>local-runtime/</code> · execution machinery</summary><ul><li><code>contracts.ts / local-project.ts</code> — strict schemas, parsing and shared spec validation</li><li><code>orchestrator.ts / foundation-runner.ts</code> — separate engine coordinators</li><li><code>dataset.ts / evaluation.ts</code> — compile data, score predictions</li><li><code>process-training.ts / process-runner.ts / gpu-executor.ts</code> — process lifecycle and transport</li><li><code>store.ts / artifacts.ts / model-registry.ts</code> — records and artifact integrity</li><li><code>model-server.ts / serving-runtime.ts</code> — verified serving process</li></ul></details>
|
|
130
130
|
</details>
|
|
131
131
|
<details open><summary><code>training/</code> · independently locked runtimes</summary>
|
|
@@ -474,30 +474,6 @@
|
|
|
474
474
|
],
|
|
475
475
|
"example": "tt serve local-<run-id>\n# Or a foundation checkpoint:\ntt serve foundation --checkpoint <model-dir> --tokenizer <tokenizer.json>"
|
|
476
476
|
},
|
|
477
|
-
"publish": {
|
|
478
|
-
"title": "Publish & account operations",
|
|
479
|
-
"location": "Laptop HTTP client → TT service",
|
|
480
|
-
"summary": "Optionally publish adapter run evidence to the dashboard. Account commands separately inspect hosted records and manage authentication and usage.",
|
|
481
|
-
"input": "Stored spec plus run report, including prompts and model outputs.",
|
|
482
|
-
"output": "A dashboard run record and URL.",
|
|
483
|
-
"boundary": "Publishing uploads evidence, not model weights. It needs a TT token. Hosted training start/estimate are retired; EC2 training uses pipeline run.",
|
|
484
|
-
"flow": [
|
|
485
|
-
"local store",
|
|
486
|
-
"spec + report",
|
|
487
|
-
"POST /publish/runs",
|
|
488
|
-
"dashboard"
|
|
489
|
-
],
|
|
490
|
-
"path": "src/commands/publish.ts",
|
|
491
|
-
"line": 172,
|
|
492
|
-
"code": " const report = runReportSchema.parse(rawReport);\n const specRecord = await store.getSpec(run.behavior_spec_id);\n const exampleCount = report.candidate.results.length\n || report.baseline.results.length;\n\n const payload = {\n spec: {\n id: specRecord.id,\n ...specRecord.spec,\n },\n report,",
|
|
493
|
-
"files": [
|
|
494
|
-
"src/commands/publish.ts",
|
|
495
|
-
"src/client.ts",
|
|
496
|
-
"src/commands/cloud.ts",
|
|
497
|
-
"src/commands/auth.ts"
|
|
498
|
-
],
|
|
499
|
-
"example": "tt publish <run-id> --dry-run\n# Optional upload, after reviewing the payload:\ntt publish <run-id>\ntt cloud runs list\ntt usage"
|
|
500
|
-
},
|
|
501
477
|
"review": {
|
|
502
478
|
"title": "Direct review & history",
|
|
503
479
|
"location": "Terminal · no model call",
|
package/docs/spec-workflow.md
CHANGED
|
@@ -233,3 +233,12 @@ Splitting a multi-gigabyte master does not remove that training-loader limit.
|
|
|
233
233
|
Do not treat a successful split preview as proof that an arbitrarily large split
|
|
234
234
|
can train; streaming training ingestion or bounded dataset selection is separate
|
|
235
235
|
work. Use `tt validate` on the saved spec to validate the actual training inputs.
|
|
236
|
+
|
|
237
|
+
## Task acceptance and conversational model use
|
|
238
|
+
|
|
239
|
+
Adapter specs can set `evaluation.taskQuality` before training. The report records
|
|
240
|
+
the criteria; agent activation and serving require passing task and general
|
|
241
|
+
regression checks, an unchanged policy, and verified artifacts. The agent can
|
|
242
|
+
inspect local datasets and completed runs directly, then prepare activation,
|
|
243
|
+
serving, stopping and rollback. See [model quality](local-runtime/model-quality.md)
|
|
244
|
+
for policy fields, lifecycle behavior and limits.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Blue Oak Model License
|
|
2
|
+
|
|
3
|
+
Version 1.0.0
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
This license gives everyone as much permission to work with
|
|
8
|
+
this software as possible, while protecting contributors
|
|
9
|
+
from liability.
|
|
10
|
+
|
|
11
|
+
## Acceptance
|
|
12
|
+
|
|
13
|
+
In order to receive this license, you must agree to its
|
|
14
|
+
rules. The rules of this license are both obligations
|
|
15
|
+
under that agreement and conditions to your license.
|
|
16
|
+
You must not do anything with this software that triggers
|
|
17
|
+
a rule that you cannot or will not follow.
|
|
18
|
+
|
|
19
|
+
## Copyright
|
|
20
|
+
|
|
21
|
+
Each contributor licenses you to do everything with this
|
|
22
|
+
software that would otherwise infringe that contributor's
|
|
23
|
+
copyright in it.
|
|
24
|
+
|
|
25
|
+
## Notices
|
|
26
|
+
|
|
27
|
+
You must ensure that everyone who gets a copy of
|
|
28
|
+
any part of this software from you, with or without
|
|
29
|
+
changes, also gets the text of this license or a link to
|
|
30
|
+
<https://blueoakcouncil.org/license/1.0.0>.
|
|
31
|
+
|
|
32
|
+
## Excuse
|
|
33
|
+
|
|
34
|
+
If anyone notifies you in writing that you have not
|
|
35
|
+
complied with [Notices](#notices), you can keep your
|
|
36
|
+
license by taking all practical steps to comply within 30
|
|
37
|
+
days after the notice. If you do not do so, your license
|
|
38
|
+
ends immediately.
|
|
39
|
+
|
|
40
|
+
## Patent
|
|
41
|
+
|
|
42
|
+
Each contributor licenses you to do everything with this
|
|
43
|
+
software that would otherwise infringe any patent claims
|
|
44
|
+
they can license or become able to license.
|
|
45
|
+
|
|
46
|
+
## Reliability
|
|
47
|
+
|
|
48
|
+
No contributor can revoke this license.
|
|
49
|
+
|
|
50
|
+
## No Liability
|
|
51
|
+
|
|
52
|
+
***As far as the law allows, this software comes as is,
|
|
53
|
+
without any warranty or condition, and no contributor
|
|
54
|
+
will be liable to anyone for any damages related to this
|
|
55
|
+
software or this license, under any kind of legal claim.***
|