@almyty/models 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +201 -0
- package/dist/exit-codes.d.ts +49 -0
- package/dist/exit-codes.js +90 -0
- package/dist/index.d.ts +71 -0
- package/dist/index.js +804 -0
- package/dist/version.d.ts +2 -0
- package/dist/version.js +22 -0
- package/package.json +48 -0
package/README.md
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
# @almyty/models
|
|
2
|
+
|
|
3
|
+
The almyty model catalog from your terminal: which models exist, which the
|
|
4
|
+
router may pick and **why not** when it may not, what a routing policy would
|
|
5
|
+
choose right now, what everything costs, and where self-hosted weights run.
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
npx @almyty/auth login
|
|
9
|
+
npx @almyty/models list
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## What makes a model usable
|
|
13
|
+
|
|
14
|
+
Support in almyty is registry data, never a code list. A model is usable when
|
|
15
|
+
its **card** exists in your organization's catalog and:
|
|
16
|
+
|
|
17
|
+
1. something can call it — a stored LLM provider row, or an endpoint URL from
|
|
18
|
+
a deployment,
|
|
19
|
+
2. its status is `active`, and
|
|
20
|
+
3. one **validation run** has passed: a real, short call, recorded.
|
|
21
|
+
|
|
22
|
+
`list` and `get` report which of those is missing, so a card that will not be
|
|
23
|
+
picked says so instead of looking like any other row:
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
Llama 3 8B [llama-3-8b] private_cloud/eu-central $0.1/$0.2 per M (feed:litellm)
|
|
27
|
+
9c2f… not selectable: no passed validation run (pending) — run: almyty models validate 9c2f…
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Commands
|
|
31
|
+
|
|
32
|
+
Every read command takes `--json` and writes undecorated JSON to stdout.
|
|
33
|
+
|
|
34
|
+
### Catalog
|
|
35
|
+
|
|
36
|
+
| Command | What it does |
|
|
37
|
+
|---|---|
|
|
38
|
+
| `list [--selectable] [--status s] [--tier t] [--provider id]` | Model cards, each line saying selectable or why not |
|
|
39
|
+
| `get <id>` | One card in full: what can call it, capabilities, pricing, the last validation run, measured latency |
|
|
40
|
+
| `register --name <n> --provider <providerId> --model <vendorModelId> [--tier t] [--region r] [--context n]` | Register a card against a stored LLM provider |
|
|
41
|
+
| `register-endpoint --name <n> --url <baseUrl> --model <vendorModelId> [--api-key-stdin] [--tier t] [--region r] [--context n]` | Register any OpenAI-compatible server you run |
|
|
42
|
+
| `set <id> [--name n] [--tier t] [--region r] [--context n] [--status s] [--price-in n --price-out n] [--clear-price]` | Change a card; a price pair is an override that wins over the automatic feed |
|
|
43
|
+
| `sync [providerId]` | Import what a provider lists, as unvalidated cards. With no id, every active provider |
|
|
44
|
+
| `validate <id>` | One real short call. Passing is what makes a card selectable |
|
|
45
|
+
| `delete <id>` | Remove a card |
|
|
46
|
+
|
|
47
|
+
Cards mostly arrive on their own: creating an LLM provider, changing it, and
|
|
48
|
+
every passing health check import what that provider currently lists. `sync`
|
|
49
|
+
is the same import by hand.
|
|
50
|
+
|
|
51
|
+
### Routing
|
|
52
|
+
|
|
53
|
+
`route` is the honest answer to "why did it not pick that model". It takes the
|
|
54
|
+
same policy an `llm_call` node carries, and **calls nothing** — it plans.
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
npx @almyty/models route --objective cheapest --tier private_cloud \
|
|
58
|
+
--regions eu-central --needs tools --budget-headroom 500
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
2 candidate(s), in the order they would be tried:
|
|
63
|
+
1. Llama 3 8B [llama-3-8b] openai private_cloud/eu-central $0.1 per M blended
|
|
64
|
+
9c2f… cheapest at $0.10 blended
|
|
65
|
+
2. Mixtral [mixtral-8x7b] openai private_cloud/eu-central $0.24 per M blended
|
|
66
|
+
4a71… next cheapest
|
|
67
|
+
|
|
68
|
+
Rejected 3:
|
|
69
|
+
1d0e… no callable provider
|
|
70
|
+
7bb2… privacy tier public exceeds the ceiling private_cloud
|
|
71
|
+
e551… no passed validation run
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
| Flag | Meaning |
|
|
75
|
+
|---|---|
|
|
76
|
+
| `--objective cheapest\|fastest\|pinned` | `cheapest` by blended feed price, `fastest` by measured p50 |
|
|
77
|
+
| `--tier local\|private_cloud\|public` | Privacy ceiling; anything above it is rejected |
|
|
78
|
+
| `--regions <a,b>` | Allowed regions |
|
|
79
|
+
| `--needs tools,vision,reasoning` | Capabilities that must be true |
|
|
80
|
+
| `--capabilities '<json>'` | The full capability object, when `--needs` is not enough |
|
|
81
|
+
| `--pinned <id>` | One card or vendor model id, by name |
|
|
82
|
+
| `--chain <a,b>` | An explicit fallback order |
|
|
83
|
+
| `--budget-headroom <cents>` | Reject anything that would not fit |
|
|
84
|
+
| `--prefer <providerId or type,...>` | Providers to prefer, best first, applied before cost |
|
|
85
|
+
|
|
86
|
+
It exits 5 when no card satisfies the policy, so a check can be a check.
|
|
87
|
+
|
|
88
|
+
### Versions, adapters, deployments
|
|
89
|
+
|
|
90
|
+
Registering a version is optional: do it when you want lineage, a manifest
|
|
91
|
+
digest and evaluation history attached to your own artifact. Skip it to just
|
|
92
|
+
run a model that already lives somewhere.
|
|
93
|
+
|
|
94
|
+
| Command | What it does |
|
|
95
|
+
|---|---|
|
|
96
|
+
| `versions` | Registered model versions |
|
|
97
|
+
| `register-version --name <n> --uri <pinned uri> [--base b] [--quantizations q1,q2]` | `hf://org/repo@sha`, `s3://bucket/key@etag`, `gs://bucket/key@gen`, `file:///path@sha` |
|
|
98
|
+
| `adapters` | Every adapter: what it can run (`modelSchemes`), its capabilities, which config fields are secret |
|
|
99
|
+
| `deploy <model> --adapter <key> [...]` | Run a model on a provider's managed product |
|
|
100
|
+
| `deployments` | Desired vs actual, state, spend |
|
|
101
|
+
| `deployment <id>` | One deployment in full, including its endpoint and rate |
|
|
102
|
+
| `scale <id> <replicas>` | Set desired replicas; `0` scales to zero |
|
|
103
|
+
| `teardown <id>` | Tear the endpoint down; weights stay in the registry |
|
|
104
|
+
|
|
105
|
+
Naming the model is configuration, so it is the positional argument:
|
|
106
|
+
|
|
107
|
+
```sh
|
|
108
|
+
npx @almyty/models deploy hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints
|
|
109
|
+
npx @almyty/models deploy fireworks://accounts/acme/models/qwen3-tuned --adapter fireworks
|
|
110
|
+
npx @almyty/models deploy --model-version <id> --adapter modal --desired '{"replicas":1}'
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
An **artifact** reference points at bytes and is pinned, so the deployment is
|
|
114
|
+
reproducible. A **provider reference** (`bedrock://`, `vertex://`,
|
|
115
|
+
`fireworks://`, …) names a model that already exists on a platform, which
|
|
116
|
+
versions it itself. The two do not mix freely: `adapters` lists what each
|
|
117
|
+
provider can really read, and a mismatch is refused at submit with
|
|
118
|
+
`ADAPTER_UNSUPPORTED_SOURCE` rather than as a provider error later.
|
|
119
|
+
|
|
120
|
+
## Secrets never travel on argv
|
|
121
|
+
|
|
122
|
+
`ps` shows every process's arguments to every user on the machine, shell
|
|
123
|
+
history keeps them, and most CI runners echo them. So this tool does not take
|
|
124
|
+
a secret as a flag value.
|
|
125
|
+
|
|
126
|
+
**An endpoint key.** Prompted without echo, or read from stdin:
|
|
127
|
+
|
|
128
|
+
```sh
|
|
129
|
+
npx @almyty/models register-endpoint --name vllm-box --url https://vllm.internal/v1 --model llama-3-8b
|
|
130
|
+
# API key for the endpoint (empty for none): ······
|
|
131
|
+
|
|
132
|
+
pass show vllm/key | npx @almyty/models register-endpoint \
|
|
133
|
+
--name vllm-box --url https://vllm.internal/v1 --model llama-3-8b --api-key-stdin
|
|
134
|
+
|
|
135
|
+
# an endpoint with no key at all
|
|
136
|
+
npx @almyty/models register-endpoint --name open-box --url https://box/v1 --model m --api-key ""
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
`--api-key <value>` is refused, and says this.
|
|
140
|
+
|
|
141
|
+
**Adapter configuration.** Best is not to paste one at all: connect the
|
|
142
|
+
provider account once and name the connection.
|
|
143
|
+
|
|
144
|
+
```sh
|
|
145
|
+
npx @almyty/connections connect huggingface
|
|
146
|
+
npx @almyty/models deploy hf://Qwen/Qwen3-0.6B@main \
|
|
147
|
+
--adapter huggingface-endpoints --credential <connectionId>
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Otherwise pass the object from a file or stdin:
|
|
151
|
+
|
|
152
|
+
```sh
|
|
153
|
+
npx @almyty/models deploy hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-file hf.json
|
|
154
|
+
cat hf.json | npx @almyty/models deploy hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-stdin
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
`--config` still works for the fields an adapter does **not** mark secret, and
|
|
158
|
+
is refused the moment it carries one that is. `adapters` prints which fields
|
|
159
|
+
those are.
|
|
160
|
+
|
|
161
|
+
## Pricing
|
|
162
|
+
|
|
163
|
+
Automatic. A daily job loads the LiteLLM cost map and cross-checks OpenRouter;
|
|
164
|
+
a disagreement above 25% is kept on the card and `get` prints it. `pricingSource`
|
|
165
|
+
says where a number came from: `feed:litellm`, `feed:openrouter`, `native`,
|
|
166
|
+
`adapter`, `manual` (an override you set with `set --price-in/--price-out`), or
|
|
167
|
+
`unpriced`.
|
|
168
|
+
|
|
169
|
+
## Environment
|
|
170
|
+
|
|
171
|
+
| Variable | Meaning |
|
|
172
|
+
|---|---|
|
|
173
|
+
| `ALMYTY_TOKEN` | Token override; skips `~/.almyty/credentials.json` |
|
|
174
|
+
| `ALMYTY_URL` | API URL override |
|
|
175
|
+
| `NO_COLOR` | Honoured: this tool never colours its output |
|
|
176
|
+
|
|
177
|
+
## Exit codes
|
|
178
|
+
|
|
179
|
+
| Code | Meaning |
|
|
180
|
+
|---|---|
|
|
181
|
+
| 0 | success |
|
|
182
|
+
| 1 | unexpected error |
|
|
183
|
+
| 2 | usage error (bad flags, missing argument, unknown command) |
|
|
184
|
+
| 3 | not authenticated — run `npx @almyty/auth login` |
|
|
185
|
+
| 4 | not found |
|
|
186
|
+
| 5 | the operation ran and failed (a validation run that did not pass, a policy that resolves to nothing) |
|
|
187
|
+
|
|
188
|
+
The same table in every `@almyty/*` CLI.
|
|
189
|
+
|
|
190
|
+
## About almyty
|
|
191
|
+
|
|
192
|
+
almyty is the platform for AI agents, agnostic by design: any LLM, any API
|
|
193
|
+
turned into tools, served over MCP, A2A, UTCP and Agent Skills.
|
|
194
|
+
|
|
195
|
+
- Website: https://almyty.com
|
|
196
|
+
- Design notes: `docs/models.md` in the almyty repository
|
|
197
|
+
- Source: https://github.com/almyty-inc/almyty
|
|
198
|
+
|
|
199
|
+
Run `npx @almyty/models --help` for the full surface.
|
|
200
|
+
|
|
201
|
+
Apache-2.0 © Almyty Inc.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Exit codes shared by every almyty CLI.
|
|
3
|
+
*
|
|
4
|
+
* Scripts need to tell "you are not logged in" apart from "that card does
|
|
5
|
+
* not exist" apart from "the validation run failed" without grepping
|
|
6
|
+
* stderr. Every almyty CLI uses this same table, so
|
|
7
|
+
* `almyty models validate x || case $? in 3) almyty auth login;; esac`
|
|
8
|
+
* behaves the same whichever binary produced the code.
|
|
9
|
+
*
|
|
10
|
+
* Kept as a copy rather than an import: these CLIs are published
|
|
11
|
+
* separately and this table is six numbers that must never drift, which a
|
|
12
|
+
* test in each package pins.
|
|
13
|
+
*/
|
|
14
|
+
export declare const EXIT: {
|
|
15
|
+
/** Success. */
|
|
16
|
+
readonly OK: 0;
|
|
17
|
+
/** Unexpected failure (a thrown error with no better classification). */
|
|
18
|
+
readonly ERROR: 1;
|
|
19
|
+
/** Bad or missing arguments, or an unknown command. */
|
|
20
|
+
readonly USAGE: 2;
|
|
21
|
+
/** No stored credential, or the API rejected the one we had. */
|
|
22
|
+
readonly AUTH: 3;
|
|
23
|
+
/** The named card, deployment, connection or grant does not exist. */
|
|
24
|
+
readonly NOT_FOUND: 4;
|
|
25
|
+
/** The command ran; the operation it asked for failed. */
|
|
26
|
+
readonly FAILED: 5;
|
|
27
|
+
};
|
|
28
|
+
export type ExitCode = (typeof EXIT)[keyof typeof EXIT];
|
|
29
|
+
/** One line per code, for `--help` output and READMEs. */
|
|
30
|
+
export declare const EXIT_CODE_HELP: string;
|
|
31
|
+
/** Errors the CLI raises itself for a bad invocation, so they exit 2. */
|
|
32
|
+
export declare class UsageError extends Error {
|
|
33
|
+
}
|
|
34
|
+
/**
|
|
35
|
+
* Classify a thrown error into an exit code.
|
|
36
|
+
*
|
|
37
|
+
* The shared client turns a 401 into "Authentication failed. Run: npx
|
|
38
|
+
* @almyty/auth login" and everything else into "API error <status>: <body>",
|
|
39
|
+
* so the status is what there is to go on.
|
|
40
|
+
*/
|
|
41
|
+
export declare function exitCodeFor(err: unknown): ExitCode;
|
|
42
|
+
/**
|
|
43
|
+
* What to print when a command fails.
|
|
44
|
+
*
|
|
45
|
+
* Node's fetch says `fetch failed` and nothing else, which tells the reader
|
|
46
|
+
* neither which host was unreachable nor that the URL is theirs to change.
|
|
47
|
+
* Every message here names the next thing to do.
|
|
48
|
+
*/
|
|
49
|
+
export declare function describeError(err: unknown, apiUrl?: string): string;
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Exit codes shared by every almyty CLI.
|
|
3
|
+
*
|
|
4
|
+
* Scripts need to tell "you are not logged in" apart from "that card does
|
|
5
|
+
* not exist" apart from "the validation run failed" without grepping
|
|
6
|
+
* stderr. Every almyty CLI uses this same table, so
|
|
7
|
+
* `almyty models validate x || case $? in 3) almyty auth login;; esac`
|
|
8
|
+
* behaves the same whichever binary produced the code.
|
|
9
|
+
*
|
|
10
|
+
* Kept as a copy rather than an import: these CLIs are published
|
|
11
|
+
* separately and this table is six numbers that must never drift, which a
|
|
12
|
+
* test in each package pins.
|
|
13
|
+
*/
|
|
14
|
+
export const EXIT = {
|
|
15
|
+
/** Success. */
|
|
16
|
+
OK: 0,
|
|
17
|
+
/** Unexpected failure (a thrown error with no better classification). */
|
|
18
|
+
ERROR: 1,
|
|
19
|
+
/** Bad or missing arguments, or an unknown command. */
|
|
20
|
+
USAGE: 2,
|
|
21
|
+
/** No stored credential, or the API rejected the one we had. */
|
|
22
|
+
AUTH: 3,
|
|
23
|
+
/** The named card, deployment, connection or grant does not exist. */
|
|
24
|
+
NOT_FOUND: 4,
|
|
25
|
+
/** The command ran; the operation it asked for failed. */
|
|
26
|
+
FAILED: 5,
|
|
27
|
+
};
|
|
28
|
+
/** One line per code, for `--help` output and READMEs. */
|
|
29
|
+
export const EXIT_CODE_HELP = [
|
|
30
|
+
' 0 success',
|
|
31
|
+
' 1 unexpected error',
|
|
32
|
+
' 2 usage error (bad flags, unknown command)',
|
|
33
|
+
' 3 not authenticated — run `almyty auth login`',
|
|
34
|
+
' 4 not found',
|
|
35
|
+
' 5 the operation ran and failed',
|
|
36
|
+
].join('\n');
|
|
37
|
+
/** Errors the CLI raises itself for a bad invocation, so they exit 2. */
|
|
38
|
+
export class UsageError extends Error {
|
|
39
|
+
}
|
|
40
|
+
/**
|
|
41
|
+
* Classify a thrown error into an exit code.
|
|
42
|
+
*
|
|
43
|
+
* The shared client turns a 401 into "Authentication failed. Run: npx
|
|
44
|
+
* @almyty/auth login" and everything else into "API error <status>: <body>",
|
|
45
|
+
* so the status is what there is to go on.
|
|
46
|
+
*/
|
|
47
|
+
export function exitCodeFor(err) {
|
|
48
|
+
if (err instanceof UsageError)
|
|
49
|
+
return EXIT.USAGE;
|
|
50
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
51
|
+
if (/^--|is required|must be|contradict|nothing to set|unknown command/i.test(message))
|
|
52
|
+
return EXIT.USAGE;
|
|
53
|
+
if (/Authentication failed|\bAPI error 401\b|\b403\b/.test(message))
|
|
54
|
+
return EXIT.AUTH;
|
|
55
|
+
if (/\bAPI error 404\b/.test(message))
|
|
56
|
+
return EXIT.NOT_FOUND;
|
|
57
|
+
if (/\bAPI error 4\d\d\b|\bAPI error 5\d\d\b/.test(message))
|
|
58
|
+
return EXIT.FAILED;
|
|
59
|
+
return EXIT.ERROR;
|
|
60
|
+
}
|
|
61
|
+
/**
|
|
62
|
+
* What to print when a command fails.
|
|
63
|
+
*
|
|
64
|
+
* Node's fetch says `fetch failed` and nothing else, which tells the reader
|
|
65
|
+
* neither which host was unreachable nor that the URL is theirs to change.
|
|
66
|
+
* Every message here names the next thing to do.
|
|
67
|
+
*/
|
|
68
|
+
export function describeError(err, apiUrl) {
|
|
69
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
70
|
+
const where = apiUrl ? ` at ${apiUrl}` : '';
|
|
71
|
+
if (/Authentication failed|\bAPI error 401\b/.test(message)) {
|
|
72
|
+
return 'Not authenticated: the stored token is missing or has expired.\n Run: npx @almyty/auth login (or set ALMYTY_TOKEN)';
|
|
73
|
+
}
|
|
74
|
+
if (/\bAPI error 403\b/.test(message)) {
|
|
75
|
+
return `${message}\n The token is valid but lacks permission here. An organization connection needs connections:manage (admin or owner).`;
|
|
76
|
+
}
|
|
77
|
+
if (/\bAPI error 404\b/.test(message)) {
|
|
78
|
+
return `${message}\n No such id, or it belongs to another organization.`;
|
|
79
|
+
}
|
|
80
|
+
if (/ECONNREFUSED|ENOTFOUND|EAI_AGAIN|fetch failed|other side closed|socket hang up/i.test(message)) {
|
|
81
|
+
return `Could not reach the almyty API${where}: ${message}\n Check ALMYTY_URL and the network, then try again.`;
|
|
82
|
+
}
|
|
83
|
+
if (/certificate|self.signed|SSL/i.test(message)) {
|
|
84
|
+
return `TLS handshake with the almyty API${where} failed: ${message}`;
|
|
85
|
+
}
|
|
86
|
+
if (/ENOENT/.test(message)) {
|
|
87
|
+
return `${message}\n The file named by --input-file or --config-file does not exist.`;
|
|
88
|
+
}
|
|
89
|
+
return message;
|
|
90
|
+
}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
export interface ParsedArgs {
|
|
3
|
+
command?: string;
|
|
4
|
+
positional: string[];
|
|
5
|
+
flags: Record<string, string | boolean>;
|
|
6
|
+
}
|
|
7
|
+
export declare function parseArgs(argv: string[]): ParsedArgs;
|
|
8
|
+
/**
|
|
9
|
+
* A number, or a message. `--context abc` used to become NaN, which
|
|
10
|
+
* JSON.stringify turns into null, so the API saw a field it could not
|
|
11
|
+
* explain and answered about the wrong thing.
|
|
12
|
+
*/
|
|
13
|
+
export declare function num(flags: ParsedArgs['flags'], key: string): number | undefined;
|
|
14
|
+
/** Say which positional is missing instead of sending "undefined" to the API. */
|
|
15
|
+
export declare function needArg(positional: string[], index: number, name: string, usage: string): string;
|
|
16
|
+
export declare function parseJsonObject(raw: string, flag: string): Record<string, unknown>;
|
|
17
|
+
/** A comma-separated list, trimmed, without the empties. */
|
|
18
|
+
export declare function csv(flags: ParsedArgs['flags'], key: string): string[] | undefined;
|
|
19
|
+
/** Names of the `x-secret` properties of an adapter's config schema. */
|
|
20
|
+
export declare function secretFields(schema: any): string[];
|
|
21
|
+
/**
|
|
22
|
+
* Refuse a secret that arrived on the command line. argv is readable by any
|
|
23
|
+
* process through `ps`, is written to shell history and is echoed by most CI
|
|
24
|
+
* runners, so a key that travelled that way has to be treated as disclosed.
|
|
25
|
+
*/
|
|
26
|
+
export declare function assertNoArgvSecrets(schema: any, config: Record<string, unknown>, flag: string, alternatives: string[]): void;
|
|
27
|
+
/** Request bodies are built from flags here so they can be checked without a network. */
|
|
28
|
+
export declare function registerBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
29
|
+
export declare function registerEndpointBody(flags: ParsedArgs['flags'], apiKey?: string): Record<string, unknown>;
|
|
30
|
+
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
31
|
+
export declare function setBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
32
|
+
/**
|
|
33
|
+
* A routing policy built from flags, for `route`. Same shape as the policy an
|
|
34
|
+
* llm_call node carries, so what this previews is what a run would do.
|
|
35
|
+
*/
|
|
36
|
+
export declare function routePolicy(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
37
|
+
/**
|
|
38
|
+
* Naming the model is configuration, so the model reference is the
|
|
39
|
+
* positional argument: `deploy hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
40
|
+
* `--model-version` is the other way in, for people who registered an
|
|
41
|
+
* artifact to get lineage and evaluation history with it.
|
|
42
|
+
*/
|
|
43
|
+
export declare function deployBody(flags: ParsedArgs['flags'], positional?: string[], providerConfig?: Record<string, unknown>): Record<string, unknown>;
|
|
44
|
+
export declare function registerVersionBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
45
|
+
export declare function formatVersion(v: any): string;
|
|
46
|
+
/**
|
|
47
|
+
* Why a card is not selectable, in the order the platform checks it. A card
|
|
48
|
+
* that lists but cannot be picked is the thing people actually need
|
|
49
|
+
* explained, and "validation: pending" alone does not explain a retired
|
|
50
|
+
* model or one whose provider row went away.
|
|
51
|
+
*/
|
|
52
|
+
export declare function unselectableReason(c: any): string;
|
|
53
|
+
export declare function formatCard(c: any): string;
|
|
54
|
+
/** The detail view, for `get`: everything that decides whether the router may pick it. */
|
|
55
|
+
export declare function formatCardDetail(c: any): string;
|
|
56
|
+
/**
|
|
57
|
+
* The route preview. Candidates in the order the router would try them, then
|
|
58
|
+
* every card it would not try and why, which is the only honest answer to
|
|
59
|
+
* "why did it not pick that model".
|
|
60
|
+
*/
|
|
61
|
+
export declare function formatRoutePlan(plan: any): string;
|
|
62
|
+
export declare function formatAdapter(a: any): string;
|
|
63
|
+
export declare function formatDeployment(d: any): string;
|
|
64
|
+
export declare function formatDeploymentDetail(d: any): string;
|
|
65
|
+
/** `sync` reports every kind of change, not only what it created. */
|
|
66
|
+
export declare function formatSync(data: any): string;
|
|
67
|
+
/**
|
|
68
|
+
* Reading stdin when stdin is the terminal means waiting for a person to
|
|
69
|
+
* type JSON and press ctrl-D, which looks exactly like a hang. Say so.
|
|
70
|
+
*/
|
|
71
|
+
export declare function assertStdinIsPiped(flag: string): void;
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,804 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* @almyty/models: the model catalog from the terminal.
|
|
4
|
+
*
|
|
5
|
+
* Support in almyty is registry data, never a code list: a model is usable
|
|
6
|
+
* when its card exists, has something that can call it, is active, and has
|
|
7
|
+
* one passed validation run. `list` and `get` say which of those is missing,
|
|
8
|
+
* and `route` answers the question a list cannot — what a routing policy
|
|
9
|
+
* would pick right now, and why it rejected the rest. See docs/models.md.
|
|
10
|
+
*
|
|
11
|
+
* Adapter configuration and endpoint keys are secrets, so they are never
|
|
12
|
+
* taken from argv: argv is readable through `ps` and lands in shell history.
|
|
13
|
+
*/
|
|
14
|
+
import { createInterface } from 'readline';
|
|
15
|
+
import { readFileSync } from 'fs';
|
|
16
|
+
import { AlmytyClient, resolveCredentialsOrExit } from '@almyty/client';
|
|
17
|
+
import { EXIT, EXIT_CODE_HELP, UsageError, describeError, exitCodeFor } from './exit-codes.js';
|
|
18
|
+
import { VERSION } from './version.js';
|
|
19
|
+
/** Flags that never take a value, so they never swallow the next argument. */
|
|
20
|
+
const BOOLEAN_FLAGS = new Set(['json', 'selectable', 'config-stdin', 'api-key-stdin', 'clear-price']);
|
|
21
|
+
export function parseArgs(argv) {
|
|
22
|
+
const result = { positional: [], flags: {} };
|
|
23
|
+
for (let i = 0; i < argv.length; i++) {
|
|
24
|
+
const arg = argv[i];
|
|
25
|
+
if (arg === '-h') {
|
|
26
|
+
result.flags.help = true;
|
|
27
|
+
continue;
|
|
28
|
+
}
|
|
29
|
+
if (arg === '-v') {
|
|
30
|
+
result.flags.version = true;
|
|
31
|
+
continue;
|
|
32
|
+
}
|
|
33
|
+
if (arg === '--') {
|
|
34
|
+
// Everything after `--` is positional, flags included.
|
|
35
|
+
result.positional.push(...argv.slice(i + 1));
|
|
36
|
+
break;
|
|
37
|
+
}
|
|
38
|
+
if (arg.startsWith('--')) {
|
|
39
|
+
const body = arg.slice(2);
|
|
40
|
+
// `--flag=value` as well as `--flag value`. With only the space form,
|
|
41
|
+
// `--input='{"a":1}'` became a flag literally named `input={"a":1}`
|
|
42
|
+
// and the value was dropped without a word.
|
|
43
|
+
const eq = body.indexOf('=');
|
|
44
|
+
if (eq !== -1) {
|
|
45
|
+
result.flags[body.slice(0, eq)] = body.slice(eq + 1);
|
|
46
|
+
continue;
|
|
47
|
+
}
|
|
48
|
+
if (body === 'help' || body === 'version') {
|
|
49
|
+
result.flags[body] = true;
|
|
50
|
+
continue;
|
|
51
|
+
}
|
|
52
|
+
if (BOOLEAN_FLAGS.has(body)) {
|
|
53
|
+
result.flags[body] = true;
|
|
54
|
+
continue;
|
|
55
|
+
}
|
|
56
|
+
const next = argv[i + 1];
|
|
57
|
+
if (next !== undefined && !next.startsWith('--')) {
|
|
58
|
+
result.flags[body] = next;
|
|
59
|
+
i++;
|
|
60
|
+
}
|
|
61
|
+
else
|
|
62
|
+
result.flags[body] = true;
|
|
63
|
+
continue;
|
|
64
|
+
}
|
|
65
|
+
if (!result.command)
|
|
66
|
+
result.command = arg;
|
|
67
|
+
else
|
|
68
|
+
result.positional.push(arg);
|
|
69
|
+
}
|
|
70
|
+
return result;
|
|
71
|
+
}
|
|
72
|
+
function printHelp() {
|
|
73
|
+
console.log(`
|
|
74
|
+
@almyty/models v${VERSION}
|
|
75
|
+
|
|
76
|
+
A model is usable when its card is active, has something that can call it,
|
|
77
|
+
and has one passed validation run. Nothing else makes it selectable, so
|
|
78
|
+
\`list\` and \`get\` report which of those is missing rather than a name alone.
|
|
79
|
+
|
|
80
|
+
Usage:
|
|
81
|
+
npx @almyty/models <command> [options]
|
|
82
|
+
|
|
83
|
+
Catalog:
|
|
84
|
+
list [--selectable] [--status active|inactive|error|deploying]
|
|
85
|
+
[--tier public|private_cloud|local] [--provider <providerId>]
|
|
86
|
+
List cards; each line says selectable, or why not
|
|
87
|
+
get <id> One card in full: capabilities, pricing, validation run
|
|
88
|
+
register --name <n> --provider <providerId> --model <vendorModelId>
|
|
89
|
+
[--tier public|private_cloud|local] [--region <r>] [--context <n>]
|
|
90
|
+
Register a card against a stored LLM provider
|
|
91
|
+
register-endpoint --name <n> --url <baseUrl> --model <vendorModelId>
|
|
92
|
+
[--api-key-stdin] [--tier <t>] [--region <r>] [--context <n>]
|
|
93
|
+
Register any OpenAI-compatible server you run.
|
|
94
|
+
The key is prompted, or read with --api-key-stdin.
|
|
95
|
+
set <id> [--name <n>] [--tier <t>] [--region <r>] [--context <n>]
|
|
96
|
+
[--status active|inactive|error|deploying]
|
|
97
|
+
[--price-in <usdPerMTok> --price-out <usdPerMTok>] [--clear-price]
|
|
98
|
+
Change a card. A price pair is an override that
|
|
99
|
+
wins over the automatic feed; --clear-price drops it.
|
|
100
|
+
sync [providerId] Import what a provider lists as unvalidated cards.
|
|
101
|
+
With no id, every active provider of the organization.
|
|
102
|
+
validate <id> One real short call. Passing is what makes a card
|
|
103
|
+
selectable. Exits non-zero when it fails.
|
|
104
|
+
delete <id> Remove a card
|
|
105
|
+
|
|
106
|
+
Routing (nothing is called; this plans):
|
|
107
|
+
route [--objective cheapest|fastest|pinned] [--tier <t>] [--regions <a,b>]
|
|
108
|
+
[--needs tools,vision,reasoning] [--capabilities '<json>']
|
|
109
|
+
[--pinned <card id or vendor model id>] [--chain <a,b>]
|
|
110
|
+
[--budget-headroom <cents>] [--prefer <providerId or type,...>]
|
|
111
|
+
What this policy would choose right now, in order,
|
|
112
|
+
and every card it rejected with the reason.
|
|
113
|
+
This is how to answer "why not that model".
|
|
114
|
+
|
|
115
|
+
Versions (optional: register an artifact only for lineage and evals on it):
|
|
116
|
+
versions Registered model versions
|
|
117
|
+
register-version --name <n> --uri <pinned uri> [--base <b>] [--quantizations <q1,q2>]
|
|
118
|
+
hf://org/repo@sha | s3://bucket/key@etag
|
|
119
|
+
gs://bucket/key@gen | file:///path@sha
|
|
120
|
+
|
|
121
|
+
Deployments:
|
|
122
|
+
adapters Registered adapters: what each can run (modelSchemes),
|
|
123
|
+
its capabilities, and which config fields are secret
|
|
124
|
+
deploy <model> --adapter <key> [--base <b>] [--config-file <path>] [--config-stdin]
|
|
125
|
+
[--desired '<json>'] [--credential <connectionId>] [--budget <id>] [--card <cardId>]
|
|
126
|
+
deploy --model-version <id> --adapter <key> [...]
|
|
127
|
+
<model> is where the model lives:
|
|
128
|
+
hf://org/repo@sha a Hugging Face repository
|
|
129
|
+
s3://bucket/prefix@etag, gs://bucket/prefix@gen,
|
|
130
|
+
file:///path@sha
|
|
131
|
+
bedrock:// sagemaker:// vertex:// foundry://
|
|
132
|
+
azureml:// fireworks:// together:// baseten://
|
|
133
|
+
a model already on that platform
|
|
134
|
+
\`adapters\` lists what each provider accepts; one that
|
|
135
|
+
cannot read your source is refused before anything runs.
|
|
136
|
+
Prefer --credential (a connection made with
|
|
137
|
+
\`almyty connections connect\`) over pasting a key.
|
|
138
|
+
deployments List deployments: desired vs actual, state, spend
|
|
139
|
+
deployment <id> One deployment in full
|
|
140
|
+
scale <deploymentId> <replicas> Set desired replicas; 0 scales to zero
|
|
141
|
+
teardown <deploymentId> Tear the endpoint down; weights stay in the registry
|
|
142
|
+
|
|
143
|
+
Options:
|
|
144
|
+
--json Undecorated JSON on stdout, for scripts
|
|
145
|
+
-h, --help This help
|
|
146
|
+
-v, --version Print the version
|
|
147
|
+
|
|
148
|
+
Environment:
|
|
149
|
+
ALMYTY_TOKEN Token override (skips ~/.almyty/credentials.json)
|
|
150
|
+
ALMYTY_URL API URL override
|
|
151
|
+
NO_COLOR Honoured: this tool never colours its output
|
|
152
|
+
|
|
153
|
+
Exit codes:
|
|
154
|
+
${EXIT_CODE_HELP}
|
|
155
|
+
`);
|
|
156
|
+
}
|
|
157
|
+
function str(flags, key) {
|
|
158
|
+
const v = flags[key];
|
|
159
|
+
return typeof v === 'string' ? v : undefined;
|
|
160
|
+
}
|
|
161
|
+
function need(flags, key) {
|
|
162
|
+
const v = str(flags, key);
|
|
163
|
+
if (!v)
|
|
164
|
+
throw new UsageError(`--${key} is required`);
|
|
165
|
+
return v;
|
|
166
|
+
}
|
|
167
|
+
/**
|
|
168
|
+
* A number, or a message. `--context abc` used to become NaN, which
|
|
169
|
+
* JSON.stringify turns into null, so the API saw a field it could not
|
|
170
|
+
* explain and answered about the wrong thing.
|
|
171
|
+
*/
|
|
172
|
+
export function num(flags, key) {
|
|
173
|
+
const raw = str(flags, key);
|
|
174
|
+
if (raw === undefined)
|
|
175
|
+
return undefined;
|
|
176
|
+
const n = Number(raw);
|
|
177
|
+
if (!Number.isFinite(n))
|
|
178
|
+
throw new UsageError(`--${key} must be a number, got ${raw}`);
|
|
179
|
+
return n;
|
|
180
|
+
}
|
|
181
|
+
/** Say which positional is missing instead of sending "undefined" to the API. */
|
|
182
|
+
export function needArg(positional, index, name, usage) {
|
|
183
|
+
const v = positional[index];
|
|
184
|
+
if (!v)
|
|
185
|
+
throw new UsageError(`${name} is required\n usage: almyty models ${usage}`);
|
|
186
|
+
return v;
|
|
187
|
+
}
|
|
188
|
+
export function parseJsonObject(raw, flag) {
|
|
189
|
+
let parsed;
|
|
190
|
+
try {
|
|
191
|
+
parsed = JSON.parse(raw);
|
|
192
|
+
}
|
|
193
|
+
catch {
|
|
194
|
+
throw new UsageError(`${flag} must be valid JSON`);
|
|
195
|
+
}
|
|
196
|
+
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed))
|
|
197
|
+
throw new UsageError(`${flag} must be a JSON object`);
|
|
198
|
+
return parsed;
|
|
199
|
+
}
|
|
200
|
+
function json(flags, key) {
|
|
201
|
+
const v = str(flags, key);
|
|
202
|
+
if (!v)
|
|
203
|
+
return undefined;
|
|
204
|
+
try {
|
|
205
|
+
return parseJsonObject(v, `--${key}`);
|
|
206
|
+
}
|
|
207
|
+
catch {
|
|
208
|
+
throw new UsageError(`--${key} must be valid JSON`);
|
|
209
|
+
}
|
|
210
|
+
}
|
|
211
|
+
/** A comma-separated list, trimmed, without the empties. */
|
|
212
|
+
export function csv(flags, key) {
|
|
213
|
+
const raw = str(flags, key);
|
|
214
|
+
if (!raw)
|
|
215
|
+
return undefined;
|
|
216
|
+
const parts = raw.split(',').map((s) => s.trim()).filter(Boolean);
|
|
217
|
+
return parts.length ? parts : undefined;
|
|
218
|
+
}
|
|
219
|
+
/** Names of the `x-secret` properties of an adapter's config schema. */
|
|
220
|
+
export function secretFields(schema) {
|
|
221
|
+
const props = schema?.properties ?? {};
|
|
222
|
+
return Object.entries(props).filter(([, def]) => def?.['x-secret'] === true).map(([key]) => key);
|
|
223
|
+
}
|
|
224
|
+
/**
|
|
225
|
+
* Refuse a secret that arrived on the command line. argv is readable by any
|
|
226
|
+
* process through `ps`, is written to shell history and is echoed by most CI
|
|
227
|
+
* runners, so a key that travelled that way has to be treated as disclosed.
|
|
228
|
+
*/
|
|
229
|
+
export function assertNoArgvSecrets(schema, config, flag, alternatives) {
|
|
230
|
+
const offending = secretFields(schema).filter((f) => config[f] !== undefined);
|
|
231
|
+
if (offending.length === 0)
|
|
232
|
+
return;
|
|
233
|
+
throw new UsageError(`${offending.join(', ')} ${offending.length === 1 ? 'is a secret' : 'are secrets'} and ${flag} puts it in your shell history and in \`ps\`.\n` +
|
|
234
|
+
alternatives.map((a) => ` ${a}`).join('\n'));
|
|
235
|
+
}
|
|
236
|
+
/** Request bodies are built from flags here so they can be checked without a network. */
|
|
237
|
+
export function registerBody(flags) {
|
|
238
|
+
const body = {
|
|
239
|
+
name: need(flags, 'name'),
|
|
240
|
+
providerId: need(flags, 'provider'),
|
|
241
|
+
vendorModelId: need(flags, 'model'),
|
|
242
|
+
};
|
|
243
|
+
if (str(flags, 'tier'))
|
|
244
|
+
body.privacyTier = str(flags, 'tier');
|
|
245
|
+
if (str(flags, 'region'))
|
|
246
|
+
body.region = str(flags, 'region');
|
|
247
|
+
const context = num(flags, 'context');
|
|
248
|
+
if (context !== undefined)
|
|
249
|
+
body.contextLength = context;
|
|
250
|
+
return body;
|
|
251
|
+
}
|
|
252
|
+
export function registerEndpointBody(flags, apiKey) {
|
|
253
|
+
const body = {
|
|
254
|
+
name: need(flags, 'name'),
|
|
255
|
+
url: need(flags, 'url'),
|
|
256
|
+
vendorModelId: need(flags, 'model'),
|
|
257
|
+
};
|
|
258
|
+
if (apiKey)
|
|
259
|
+
body.apiKey = apiKey;
|
|
260
|
+
if (str(flags, 'tier'))
|
|
261
|
+
body.privacyTier = str(flags, 'tier');
|
|
262
|
+
if (str(flags, 'region'))
|
|
263
|
+
body.region = str(flags, 'region');
|
|
264
|
+
const context = num(flags, 'context');
|
|
265
|
+
if (context !== undefined)
|
|
266
|
+
body.contextLength = context;
|
|
267
|
+
return body;
|
|
268
|
+
}
|
|
269
|
+
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
270
|
+
export function setBody(flags) {
|
|
271
|
+
const body = {};
|
|
272
|
+
if (str(flags, 'name'))
|
|
273
|
+
body.name = str(flags, 'name');
|
|
274
|
+
if (str(flags, 'tier'))
|
|
275
|
+
body.privacyTier = str(flags, 'tier');
|
|
276
|
+
if (str(flags, 'region'))
|
|
277
|
+
body.region = str(flags, 'region');
|
|
278
|
+
if (str(flags, 'status'))
|
|
279
|
+
body.status = str(flags, 'status');
|
|
280
|
+
const context = num(flags, 'context');
|
|
281
|
+
if (context !== undefined)
|
|
282
|
+
body.contextLength = context;
|
|
283
|
+
const inPerMTok = num(flags, 'price-in');
|
|
284
|
+
const outPerMTok = num(flags, 'price-out');
|
|
285
|
+
if (flags['clear-price']) {
|
|
286
|
+
if (inPerMTok !== undefined || outPerMTok !== undefined)
|
|
287
|
+
throw new UsageError('--clear-price and --price-in/--price-out contradict each other');
|
|
288
|
+
body.pricingOverride = null;
|
|
289
|
+
}
|
|
290
|
+
else if (inPerMTok !== undefined || outPerMTok !== undefined) {
|
|
291
|
+
// A one-sided override would silently price half the call from the feed
|
|
292
|
+
// and half by hand, which nobody means.
|
|
293
|
+
if (inPerMTok === undefined || outPerMTok === undefined)
|
|
294
|
+
throw new UsageError('a price override needs both --price-in and --price-out');
|
|
295
|
+
body.pricingOverride = { inPerMTok, outPerMTok };
|
|
296
|
+
}
|
|
297
|
+
if (Object.keys(body).length === 0)
|
|
298
|
+
throw new UsageError('nothing to set; pass at least one of --name --tier --region --context --status --price-in/--price-out --clear-price');
|
|
299
|
+
return body;
|
|
300
|
+
}
|
|
301
|
+
/**
|
|
302
|
+
* A routing policy built from flags, for `route`. Same shape as the policy an
|
|
303
|
+
* llm_call node carries, so what this previews is what a run would do.
|
|
304
|
+
*/
|
|
305
|
+
export function routePolicy(flags) {
|
|
306
|
+
const policy = {};
|
|
307
|
+
if (str(flags, 'objective'))
|
|
308
|
+
policy.objective = str(flags, 'objective');
|
|
309
|
+
if (str(flags, 'tier'))
|
|
310
|
+
policy.privacyTier = str(flags, 'tier');
|
|
311
|
+
const regions = csv(flags, 'regions');
|
|
312
|
+
if (regions)
|
|
313
|
+
policy.regions = regions;
|
|
314
|
+
const chain = csv(flags, 'chain');
|
|
315
|
+
if (chain)
|
|
316
|
+
policy.fallbackChain = chain;
|
|
317
|
+
const prefer = csv(flags, 'prefer');
|
|
318
|
+
if (prefer)
|
|
319
|
+
policy.connectionPreference = prefer;
|
|
320
|
+
if (str(flags, 'pinned'))
|
|
321
|
+
policy.pinnedModel = str(flags, 'pinned');
|
|
322
|
+
const headroom = num(flags, 'budget-headroom');
|
|
323
|
+
if (headroom !== undefined)
|
|
324
|
+
policy.budgetHeadroomCents = headroom;
|
|
325
|
+
// Two ways to say the same thing: a JSON object for the full shape, and
|
|
326
|
+
// --needs for the common case of "these must be true".
|
|
327
|
+
const capabilities = json(flags, 'capabilities');
|
|
328
|
+
const needs = csv(flags, 'needs');
|
|
329
|
+
if (capabilities && needs)
|
|
330
|
+
throw new UsageError('--capabilities and --needs contradict each other; use one');
|
|
331
|
+
if (capabilities)
|
|
332
|
+
policy.capabilities = capabilities;
|
|
333
|
+
else if (needs)
|
|
334
|
+
policy.capabilities = Object.fromEntries(needs.map((n) => [n, true]));
|
|
335
|
+
return policy;
|
|
336
|
+
}
|
|
337
|
+
/**
|
|
338
|
+
* Naming the model is configuration, so the model reference is the
|
|
339
|
+
* positional argument: `deploy hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
340
|
+
* `--model-version` is the other way in, for people who registered an
|
|
341
|
+
* artifact to get lineage and evaluation history with it.
|
|
342
|
+
*/
|
|
343
|
+
export function deployBody(flags, positional = [], providerConfig) {
|
|
344
|
+
const model = positional[0] ?? str(flags, 'model');
|
|
345
|
+
const modelVersion = str(flags, 'model-version');
|
|
346
|
+
if (!model && !modelVersion) {
|
|
347
|
+
throw new UsageError('Name the model to run (deploy hf://org/repo@sha --adapter <key>), or pass --model-version <id>.');
|
|
348
|
+
}
|
|
349
|
+
const body = { providerType: need(flags, 'adapter') };
|
|
350
|
+
if (modelVersion)
|
|
351
|
+
body.modelVersionId = modelVersion;
|
|
352
|
+
else
|
|
353
|
+
body.model = model;
|
|
354
|
+
if (str(flags, 'base'))
|
|
355
|
+
body.base = str(flags, 'base');
|
|
356
|
+
const desired = json(flags, 'desired');
|
|
357
|
+
if (providerConfig && Object.keys(providerConfig).length > 0)
|
|
358
|
+
body.providerConfig = providerConfig;
|
|
359
|
+
if (desired)
|
|
360
|
+
body.desired = desired;
|
|
361
|
+
if (str(flags, 'credential'))
|
|
362
|
+
body.credentialId = str(flags, 'credential');
|
|
363
|
+
if (str(flags, 'budget'))
|
|
364
|
+
body.budgetId = str(flags, 'budget');
|
|
365
|
+
if (str(flags, 'card'))
|
|
366
|
+
body.modelId = str(flags, 'card');
|
|
367
|
+
return body;
|
|
368
|
+
}
|
|
369
|
+
export function registerVersionBody(flags) {
|
|
370
|
+
const body = { name: need(flags, 'name'), registryUri: need(flags, 'uri') };
|
|
371
|
+
if (str(flags, 'base'))
|
|
372
|
+
body.base = str(flags, 'base');
|
|
373
|
+
if (str(flags, 'quantizations'))
|
|
374
|
+
body.quantizations = String(str(flags, 'quantizations')).split(',').map((s) => s.trim()).filter(Boolean);
|
|
375
|
+
return body;
|
|
376
|
+
}
|
|
377
|
+
export function formatVersion(v) {
|
|
378
|
+
const size = v.sizeBytes ? ` ${(Number(v.sizeBytes) / 1e9).toFixed(2)} GB` : '';
|
|
379
|
+
const q = v.quantizations?.length ? ` [${v.quantizations.join(', ')}]` : '';
|
|
380
|
+
return `${v.name} ${v.base}${size}${q}\n ${v.id} ${v.registryUri}`;
|
|
381
|
+
}
|
|
382
|
+
/**
|
|
383
|
+
* Why a card is not selectable, in the order the platform checks it. A card
|
|
384
|
+
* that lists but cannot be picked is the thing people actually need
|
|
385
|
+
* explained, and "validation: pending" alone does not explain a retired
|
|
386
|
+
* model or one whose provider row went away.
|
|
387
|
+
*/
|
|
388
|
+
export function unselectableReason(c) {
|
|
389
|
+
if (c.status && c.status !== 'active') {
|
|
390
|
+
const retired = c.metadata?.retiredReason;
|
|
391
|
+
return `status is ${c.status}${retired ? ` (${retired})` : ''}`;
|
|
392
|
+
}
|
|
393
|
+
if (!c.providerId && !c.endpointRef?.url)
|
|
394
|
+
return 'nothing can call it: no provider row and no endpoint URL';
|
|
395
|
+
if (c.validationStatus !== 'passed') {
|
|
396
|
+
const err = c.lastValidationError ? `: ${c.lastValidationError}` : '';
|
|
397
|
+
const status = c.validationStatus ?? 'pending';
|
|
398
|
+
return `no passed validation run (${status}${err}) — run: almyty models validate ${c.id}`;
|
|
399
|
+
}
|
|
400
|
+
return 'the catalog does not consider it selectable';
|
|
401
|
+
}
|
|
402
|
+
export function formatCard(c) {
|
|
403
|
+
const price = c.effectivePricing ? `$${c.effectivePricing.inPerMTok}/$${c.effectivePricing.outPerMTok} per M (${c.pricingSource})` : 'unpriced';
|
|
404
|
+
const flag = c.selectable ? 'selectable' : `not selectable: ${unselectableReason(c)}`;
|
|
405
|
+
return `${c.name} [${c.vendorModelId}] ${c.privacyTier}${c.region ? `/${c.region}` : ''} ${price}\n ${c.id} ${flag}`;
|
|
406
|
+
}
|
|
407
|
+
/** The detail view, for `get`: everything that decides whether the router may pick it. */
|
|
408
|
+
export function formatCardDetail(c) {
|
|
409
|
+
const caps = c.capabilities && Object.keys(c.capabilities).length
|
|
410
|
+
? Object.entries(c.capabilities).filter(([, v]) => v).map(([k]) => k).join(', ') || '(none declared true)'
|
|
411
|
+
: '(none declared)';
|
|
412
|
+
const lines = [
|
|
413
|
+
`${c.name}`,
|
|
414
|
+
` id ${c.id}`,
|
|
415
|
+
` vendor id ${c.vendorModelId}`,
|
|
416
|
+
` status ${c.status ?? 'unknown'}`,
|
|
417
|
+
` selectable ${c.selectable ? 'yes' : `no — ${unselectableReason(c)}`}`,
|
|
418
|
+
` called via ${c.providerId ? `provider ${c.providerId}${c.providerType ? ` (${c.providerType})` : ''}` : c.endpointRef?.url ? `endpoint ${c.endpointRef.url}` : 'nothing'}`,
|
|
419
|
+
` privacy ${c.privacyTier}${c.region ? ` / ${c.region}` : ''}`,
|
|
420
|
+
` capabilities ${caps}`,
|
|
421
|
+
` context ${c.contextLength ?? 'unknown'}`,
|
|
422
|
+
` pricing ${c.effectivePricing ? `$${c.effectivePricing.inPerMTok} in / $${c.effectivePricing.outPerMTok} out per M tokens (${c.pricingSource})` : 'unpriced'}`,
|
|
423
|
+
];
|
|
424
|
+
if (c.pricingOverride)
|
|
425
|
+
lines.push(' (a manual override is set; --clear-price drops it back to the feed)');
|
|
426
|
+
if (c.metadata?.pricingDisagreement)
|
|
427
|
+
lines.push(` price warning the feeds disagree: ${JSON.stringify(c.metadata.pricingDisagreement)}`);
|
|
428
|
+
lines.push(` validation ${c.validationStatus ?? 'pending'}${c.lastValidatedAt ? `, last ${c.lastValidatedAt}` : ', never run'}`);
|
|
429
|
+
if (c.lastValidationError)
|
|
430
|
+
lines.push(` last error ${c.lastValidationError}`);
|
|
431
|
+
if (c.measuredLatencyMs)
|
|
432
|
+
lines.push(` latency p50 ${c.measuredLatencyMs.p50 ?? '?'} ms, p95 ${c.measuredLatencyMs.p95 ?? '?'} ms`);
|
|
433
|
+
if (c.deploymentId)
|
|
434
|
+
lines.push(` deployment ${c.deploymentId}`);
|
|
435
|
+
if (c.modelVersionId)
|
|
436
|
+
lines.push(` version ${c.modelVersionId}`);
|
|
437
|
+
if (!c.selectable)
|
|
438
|
+
lines.push('', `Not a routing candidate yet. ${unselectableReason(c)}`);
|
|
439
|
+
return lines.join('\n');
|
|
440
|
+
}
|
|
441
|
+
/**
|
|
442
|
+
* The route preview. Candidates in the order the router would try them, then
|
|
443
|
+
* every card it would not try and why, which is the only honest answer to
|
|
444
|
+
* "why did it not pick that model".
|
|
445
|
+
*/
|
|
446
|
+
export function formatRoutePlan(plan) {
|
|
447
|
+
const lines = [];
|
|
448
|
+
if (!plan.candidates?.length) {
|
|
449
|
+
lines.push('No model satisfies this policy.');
|
|
450
|
+
}
|
|
451
|
+
else {
|
|
452
|
+
lines.push(`${plan.candidates.length} candidate(s), in the order they would be tried:`);
|
|
453
|
+
plan.candidates.forEach((c, i) => {
|
|
454
|
+
const price = c.blendedPricePerMTok != null ? `$${c.blendedPricePerMTok} per M blended` : 'unpriced';
|
|
455
|
+
lines.push(` ${i + 1}. ${c.name} [${c.vendorModelId}] ${c.providerType ?? 'no provider type'} ${c.privacyTier}${c.region ? `/${c.region}` : ''} ${price}`);
|
|
456
|
+
lines.push(` ${c.modelId} ${c.rationale}`);
|
|
457
|
+
});
|
|
458
|
+
}
|
|
459
|
+
if (plan.rejected?.length) {
|
|
460
|
+
lines.push('', `Rejected ${plan.rejected.length}:`);
|
|
461
|
+
for (const r of plan.rejected)
|
|
462
|
+
lines.push(` ${r.modelId} ${r.reason}`);
|
|
463
|
+
}
|
|
464
|
+
else {
|
|
465
|
+
lines.push('', 'Nothing rejected.');
|
|
466
|
+
}
|
|
467
|
+
return lines.join('\n');
|
|
468
|
+
}
|
|
469
|
+
export function formatAdapter(a) {
|
|
470
|
+
const caps = Object.entries(a.capabilities ?? {}).map(([k, v]) => `${k}=${JSON.stringify(v)}`).join(' ');
|
|
471
|
+
const schemes = a.modelSchemes?.length ? a.modelSchemes.join(' ') : '(none declared)';
|
|
472
|
+
const secrets = secretFields(a.configSchema);
|
|
473
|
+
const lines = [
|
|
474
|
+
`${a.key} ${a.displayName}`,
|
|
475
|
+
` runs ${schemes}`,
|
|
476
|
+
` capabilities ${caps}`,
|
|
477
|
+
];
|
|
478
|
+
if (secrets.length)
|
|
479
|
+
lines.push(` secret config ${secrets.join(', ')} (pass with --config-file or --config-stdin, or use --credential)`);
|
|
480
|
+
return lines.join('\n');
|
|
481
|
+
}
|
|
482
|
+
export function formatDeployment(d) {
|
|
483
|
+
const spend = d.actual?.spentCents != null ? ` spent ${(d.actual.spentCents / 100).toFixed(2)} USD` : '';
|
|
484
|
+
return `${d.id} ${d.providerType} ${d.state} replicas ${d.actual?.replicas ?? '?'}/${d.desired?.replicas ?? '?'}${spend}${d.lastError ? `\n ${d.lastError}` : ''}`;
|
|
485
|
+
}
|
|
486
|
+
export function formatDeploymentDetail(d) {
|
|
487
|
+
const lines = [
|
|
488
|
+
`${d.id}`,
|
|
489
|
+
` adapter ${d.providerType}`,
|
|
490
|
+
` state ${d.state}`,
|
|
491
|
+
` model ${d.modelRef ?? d.modelVersionId ?? 'unknown'}${d.modelBase ? ` base ${d.modelBase}` : ''}`,
|
|
492
|
+
` desired ${JSON.stringify(d.desired ?? {})}`,
|
|
493
|
+
` actual ${d.actual ? `${d.actual.state ?? '?'} replicas ${d.actual.replicas ?? '?'}${d.actual.hardware ? ` ${d.actual.hardware}` : ''}${d.actual.region ? ` ${d.actual.region}` : ''}` : '(not reconciled yet)'}`,
|
|
494
|
+
];
|
|
495
|
+
if (d.actual?.message)
|
|
496
|
+
lines.push(` message ${d.actual.message}`);
|
|
497
|
+
if (d.actual?.url)
|
|
498
|
+
lines.push(` endpoint ${d.actual.url}`);
|
|
499
|
+
if (d.actual?.openAiBase)
|
|
500
|
+
lines.push(` openai base ${d.actual.openAiBase}`);
|
|
501
|
+
if (d.modelId)
|
|
502
|
+
lines.push(` card ${d.modelId}`);
|
|
503
|
+
if (d.budgetId)
|
|
504
|
+
lines.push(` budget ${d.budgetId} (reaching it scales this to zero)`);
|
|
505
|
+
if (d.actual?.spentCents != null)
|
|
506
|
+
lines.push(` spend ${(d.actual.spentCents / 100).toFixed(2)} USD${d.actual.ratePerHourCents != null ? `, ${(d.actual.ratePerHourCents / 100).toFixed(2)} USD per hour` : ''}`);
|
|
507
|
+
if (d.lastReconcileAt)
|
|
508
|
+
lines.push(` reconciled ${d.lastReconcileAt}`);
|
|
509
|
+
if (d.lastError)
|
|
510
|
+
lines.push(` last error ${d.lastError}`);
|
|
511
|
+
return lines.join('\n');
|
|
512
|
+
}
|
|
513
|
+
/** `sync` reports every kind of change, not only what it created. */
|
|
514
|
+
export function formatSync(data) {
|
|
515
|
+
const lines = [
|
|
516
|
+
`${data.created?.length ?? 0} card(s) created, ${data.skipped ?? 0} already present, ` +
|
|
517
|
+
`${data.retired?.length ?? 0} retired, ${data.reinstated?.length ?? 0} reinstated.`,
|
|
518
|
+
];
|
|
519
|
+
for (const c of data.created ?? [])
|
|
520
|
+
lines.push(` + ${c.name} [${c.vendorModelId}] ${c.id}`);
|
|
521
|
+
for (const c of data.retired ?? [])
|
|
522
|
+
lines.push(` - ${c.name} [${c.vendorModelId}] ${c.metadata?.retiredReason ?? 'retired'}`);
|
|
523
|
+
for (const c of data.reinstated ?? [])
|
|
524
|
+
lines.push(` ~ ${c.name} [${c.vendorModelId}] listed again`);
|
|
525
|
+
if (data.providers) {
|
|
526
|
+
lines.push('', 'Per provider:');
|
|
527
|
+
for (const [id, summary] of Object.entries(data.providers)) {
|
|
528
|
+
lines.push(` ${id} ${summary?.error ? `error: ${summary.error}` : `created ${summary?.created ?? 0}, skipped ${summary?.skipped ?? 0}, retired ${summary?.retired ?? 0}, reinstated ${summary?.reinstated ?? 0}`}`);
|
|
529
|
+
}
|
|
530
|
+
}
|
|
531
|
+
if ((data.created?.length ?? 0) === 0 && (data.retired?.length ?? 0) === 0) {
|
|
532
|
+
lines.push('', 'Cards from a sync are unvalidated. Run `almyty models validate <id>` to make one selectable.');
|
|
533
|
+
}
|
|
534
|
+
return lines.join('\n');
|
|
535
|
+
}
|
|
536
|
+
function listQuery(flags) {
|
|
537
|
+
const params = new URLSearchParams();
|
|
538
|
+
if (flags.selectable)
|
|
539
|
+
params.set('selectable', 'true');
|
|
540
|
+
if (str(flags, 'status'))
|
|
541
|
+
params.set('status', str(flags, 'status'));
|
|
542
|
+
if (str(flags, 'tier'))
|
|
543
|
+
params.set('privacyTier', str(flags, 'tier'));
|
|
544
|
+
if (str(flags, 'provider'))
|
|
545
|
+
params.set('providerId', str(flags, 'provider'));
|
|
546
|
+
const q = params.toString();
|
|
547
|
+
return q ? `?${q}` : '';
|
|
548
|
+
}
|
|
549
|
+
/**
|
|
550
|
+
* Reading stdin when stdin is the terminal means waiting for a person to
|
|
551
|
+
* type JSON and press ctrl-D, which looks exactly like a hang. Say so.
|
|
552
|
+
*/
|
|
553
|
+
export function assertStdinIsPiped(flag) {
|
|
554
|
+
if (!process.stdin.isTTY)
|
|
555
|
+
return;
|
|
556
|
+
throw new UsageError(`${flag} reads stdin, and stdin is your terminal, so it would wait forever.\n` +
|
|
557
|
+
` Pipe it in: cat config.json | almyty models deploy ... ${flag}`);
|
|
558
|
+
}
|
|
559
|
+
function readStdin() {
|
|
560
|
+
return new Promise((resolve, reject) => {
|
|
561
|
+
let data = '';
|
|
562
|
+
process.stdin.setEncoding('utf8');
|
|
563
|
+
process.stdin.on('data', (chunk) => { data += chunk; });
|
|
564
|
+
process.stdin.on('end', () => resolve(data));
|
|
565
|
+
process.stdin.on('error', reject);
|
|
566
|
+
});
|
|
567
|
+
}
|
|
568
|
+
function askHidden(label) {
|
|
569
|
+
return new Promise((resolve) => {
|
|
570
|
+
const rl = createInterface({ input: process.stdin, output: process.stdout, terminal: true });
|
|
571
|
+
const anyRl = rl;
|
|
572
|
+
anyRl._writeToOutput = (s) => {
|
|
573
|
+
if (s.includes(label))
|
|
574
|
+
anyRl.output.write(label);
|
|
575
|
+
};
|
|
576
|
+
rl.question(label, (answer) => {
|
|
577
|
+
rl.close();
|
|
578
|
+
process.stdout.write('\n');
|
|
579
|
+
resolve(answer.trim());
|
|
580
|
+
});
|
|
581
|
+
});
|
|
582
|
+
}
|
|
583
|
+
/**
|
|
584
|
+
* The endpoint key, from the safest place it can come from. `--api-key` is
|
|
585
|
+
* refused because argv is world-readable; `-` means stdin, which is what a
|
|
586
|
+
* script should use.
|
|
587
|
+
*/
|
|
588
|
+
async function endpointApiKey(flags) {
|
|
589
|
+
const inline = str(flags, 'api-key');
|
|
590
|
+
if (inline && inline !== '-') {
|
|
591
|
+
throw new UsageError('--api-key puts the key in your shell history and in `ps`.\n' +
|
|
592
|
+
' Leave it off and the key is prompted without echo, or read it from stdin:\n' +
|
|
593
|
+
' --api-key-stdin (also: --api-key -)\n' +
|
|
594
|
+
' An endpoint with no key at all: --api-key ""');
|
|
595
|
+
}
|
|
596
|
+
if (flags['api-key-stdin'] || inline === '-') {
|
|
597
|
+
assertStdinIsPiped('--api-key-stdin');
|
|
598
|
+
return (await readStdin()).trim() || undefined;
|
|
599
|
+
}
|
|
600
|
+
// `--api-key` with no value, or an explicitly empty one: an open endpoint.
|
|
601
|
+
if (flags['api-key'] === true || inline === '')
|
|
602
|
+
return undefined;
|
|
603
|
+
if (!process.stdin.isTTY)
|
|
604
|
+
return undefined; // unattended and none supplied: an open endpoint
|
|
605
|
+
const typed = await askHidden('API key for the endpoint (empty for none): ');
|
|
606
|
+
return typed || undefined;
|
|
607
|
+
}
|
|
608
|
+
const CONFIG_ALTERNATIVES = [
|
|
609
|
+
'--config-file <path> read the JSON object from a file',
|
|
610
|
+
'--config-stdin read the JSON object from stdin',
|
|
611
|
+
'--credential <id> use a connection made with `almyty connections connect`',
|
|
612
|
+
];
|
|
613
|
+
/**
|
|
614
|
+
* Adapter configuration, from a file or stdin. Never a secret from argv.
|
|
615
|
+
*
|
|
616
|
+
* `schemaKnown` is false when the adapter catalog could not be read, so
|
|
617
|
+
* which fields are `x-secret` is unknown. `--config` is then refused rather
|
|
618
|
+
* than sent blind: failing open here would make an unreachable catalog the
|
|
619
|
+
* way to get a secret onto the command line.
|
|
620
|
+
*/
|
|
621
|
+
async function deployConfig(flags, adapterSchema, schemaKnown) {
|
|
622
|
+
const file = str(flags, 'config-file');
|
|
623
|
+
if (file)
|
|
624
|
+
return parseJsonObject(readFileSync(file, 'utf8'), `--config-file ${file}`);
|
|
625
|
+
if (flags['config-stdin']) {
|
|
626
|
+
assertStdinIsPiped('--config-stdin');
|
|
627
|
+
return parseJsonObject(await readStdin(), '--config-stdin');
|
|
628
|
+
}
|
|
629
|
+
const inline = json(flags, 'config');
|
|
630
|
+
if (!inline)
|
|
631
|
+
return undefined;
|
|
632
|
+
if (!schemaKnown) {
|
|
633
|
+
throw new UsageError('--config cannot be screened: the adapter list could not be read, so which of these fields\n' +
|
|
634
|
+
'are secret is unknown, and argv is in your shell history and in `ps`.\n' +
|
|
635
|
+
CONFIG_ALTERNATIVES.map((a) => ` ${a}`).join('\n'));
|
|
636
|
+
}
|
|
637
|
+
assertNoArgvSecrets(adapterSchema, inline, '--config', CONFIG_ALTERNATIVES);
|
|
638
|
+
return inline;
|
|
639
|
+
}
|
|
640
|
+
function newClient() {
|
|
641
|
+
const creds = resolveCredentialsOrExit();
|
|
642
|
+
return new AlmytyClient(creds.url, creds.token);
|
|
643
|
+
}
|
|
644
|
+
function out(args, data, pretty) {
|
|
645
|
+
console.log(args.flags.json ? JSON.stringify(data, null, 2) : pretty());
|
|
646
|
+
}
|
|
647
|
+
async function main() {
|
|
648
|
+
const args = parseArgs(process.argv.slice(2));
|
|
649
|
+
if (args.flags.version) {
|
|
650
|
+
console.log(VERSION);
|
|
651
|
+
return;
|
|
652
|
+
}
|
|
653
|
+
if (!args.command || args.command === 'help' || args.flags.help) {
|
|
654
|
+
printHelp();
|
|
655
|
+
return;
|
|
656
|
+
}
|
|
657
|
+
const client = newClient();
|
|
658
|
+
const q = (path, init) => client.request(path, init);
|
|
659
|
+
const post = (path, body) => q(path, { method: 'POST', body: JSON.stringify(body) });
|
|
660
|
+
switch (args.command) {
|
|
661
|
+
case 'list': {
|
|
662
|
+
const res = await q(`/models${listQuery(args.flags)}`);
|
|
663
|
+
out(args, res.data, () => (res.data.length
|
|
664
|
+
? res.data.map(formatCard).join('\n')
|
|
665
|
+
: args.flags.selectable
|
|
666
|
+
? 'No selectable model cards. A card becomes selectable when one validation run passes: almyty models validate <id>'
|
|
667
|
+
: 'No model cards yet. Register one (almyty models register) or import a provider\'s list (almyty models sync).'));
|
|
668
|
+
return;
|
|
669
|
+
}
|
|
670
|
+
case 'get': {
|
|
671
|
+
const id = needArg(args.positional, 0, 'card id', 'get <id>');
|
|
672
|
+
const res = await q(`/models/${id}`);
|
|
673
|
+
out(args, res.data, () => formatCardDetail(res.data));
|
|
674
|
+
return;
|
|
675
|
+
}
|
|
676
|
+
case 'route': {
|
|
677
|
+
const policy = routePolicy(args.flags);
|
|
678
|
+
const res = await post('/models/route-preview', policy);
|
|
679
|
+
out(args, res.data, () => formatRoutePlan(res.data));
|
|
680
|
+
// No candidate means nothing would answer a call under this policy.
|
|
681
|
+
if (!res.data?.candidates?.length)
|
|
682
|
+
process.exitCode = EXIT.FAILED;
|
|
683
|
+
return;
|
|
684
|
+
}
|
|
685
|
+
case 'register': {
|
|
686
|
+
const res = await post('/models', registerBody(args.flags));
|
|
687
|
+
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
688
|
+
return;
|
|
689
|
+
}
|
|
690
|
+
case 'register-endpoint': {
|
|
691
|
+
const apiKey = await endpointApiKey(args.flags);
|
|
692
|
+
const res = await post('/models/register-endpoint', registerEndpointBody(args.flags, apiKey));
|
|
693
|
+
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
694
|
+
return;
|
|
695
|
+
}
|
|
696
|
+
case 'set': {
|
|
697
|
+
const id = needArg(args.positional, 0, 'card id', 'set <id> [--tier t] [--price-in n --price-out n] ...');
|
|
698
|
+
const res = await q(`/models/${id}`, { method: 'PATCH', body: JSON.stringify(setBody(args.flags)) });
|
|
699
|
+
out(args, res.data, () => `Updated.\n${formatCardDetail(res.data)}`);
|
|
700
|
+
return;
|
|
701
|
+
}
|
|
702
|
+
case 'sync': {
|
|
703
|
+
const providerId = args.positional[0];
|
|
704
|
+
const res = await post('/models/sync', providerId ? { providerId } : {});
|
|
705
|
+
out(args, res.data, () => formatSync(res.data));
|
|
706
|
+
return;
|
|
707
|
+
}
|
|
708
|
+
case 'validate': {
|
|
709
|
+
const id = needArg(args.positional, 0, 'card id', 'validate <id>');
|
|
710
|
+
const res = await post(`/models/${id}/validate`, {});
|
|
711
|
+
out(args, res.data, () => (res.data.passed
|
|
712
|
+
? `Passed in ${res.data.latencyMs} ms. The card is selectable now.\n${formatCard(res.data.model)}`
|
|
713
|
+
: `Failed: ${res.data.error}\nThe card stays unselectable until a run passes.`));
|
|
714
|
+
if (!res.data.passed)
|
|
715
|
+
process.exitCode = EXIT.FAILED;
|
|
716
|
+
return;
|
|
717
|
+
}
|
|
718
|
+
case 'delete': {
|
|
719
|
+
const id = needArg(args.positional, 0, 'card id', 'delete <id>');
|
|
720
|
+
const res = await q(`/models/${id}`, { method: 'DELETE' });
|
|
721
|
+
out(args, res?.data ?? { id, deleted: true }, () => 'Deleted.');
|
|
722
|
+
return;
|
|
723
|
+
}
|
|
724
|
+
case 'versions': {
|
|
725
|
+
const res = await q('/model-versions');
|
|
726
|
+
out(args, res.data, () => (res.data.length ? res.data.map(formatVersion).join('\n') : 'No versions registered. Registering one is optional: it buys lineage and evaluation history on your own artifact.'));
|
|
727
|
+
return;
|
|
728
|
+
}
|
|
729
|
+
case 'register-version': {
|
|
730
|
+
const res = await post('/model-versions', registerVersionBody(args.flags));
|
|
731
|
+
out(args, res.data, () => `Registered.\n${formatVersion(res.data)}`);
|
|
732
|
+
return;
|
|
733
|
+
}
|
|
734
|
+
case 'adapters': {
|
|
735
|
+
const res = await q('/model-adapters');
|
|
736
|
+
out(args, res.data, () => (res.data.length ? res.data.map(formatAdapter).join('\n') : 'No adapters registered.'));
|
|
737
|
+
return;
|
|
738
|
+
}
|
|
739
|
+
case 'deploy': {
|
|
740
|
+
// The adapter's schema says which config fields are secret, so the
|
|
741
|
+
// check happens before anything is sent.
|
|
742
|
+
const adapterKey = need(args.flags, 'adapter');
|
|
743
|
+
let adapterSchema;
|
|
744
|
+
let schemaKnown = false;
|
|
745
|
+
try {
|
|
746
|
+
const adapters = await q('/model-adapters');
|
|
747
|
+
const adapter = adapters.data?.find((a) => a.key === adapterKey);
|
|
748
|
+
if (!adapter) {
|
|
749
|
+
throw new UsageError(`no adapter named ${adapterKey}. Registered: ${(adapters.data ?? []).map((a) => a.key).join(', ') || 'none'}`);
|
|
750
|
+
}
|
|
751
|
+
adapterSchema = adapter.configSchema;
|
|
752
|
+
schemaKnown = true;
|
|
753
|
+
}
|
|
754
|
+
catch (err) {
|
|
755
|
+
if (err instanceof UsageError)
|
|
756
|
+
throw err;
|
|
757
|
+
// The catalog could not be read. The API still validates the body,
|
|
758
|
+
// but --config can no longer be screened, so deployConfig refuses it.
|
|
759
|
+
}
|
|
760
|
+
const providerConfig = await deployConfig(args.flags, adapterSchema, schemaKnown);
|
|
761
|
+
const res = await post('/model-deployments', deployBody(args.flags, args.positional, providerConfig));
|
|
762
|
+
out(args, res.data, () => `Queued. Reconcile picks it up within a couple of minutes.\n${formatDeployment(res.data)}`);
|
|
763
|
+
return;
|
|
764
|
+
}
|
|
765
|
+
case 'deployments': {
|
|
766
|
+
const res = await q('/model-deployments');
|
|
767
|
+
out(args, res.data, () => (res.data.length ? res.data.map(formatDeployment).join('\n') : 'No deployments.'));
|
|
768
|
+
return;
|
|
769
|
+
}
|
|
770
|
+
case 'deployment': {
|
|
771
|
+
const id = needArg(args.positional, 0, 'deployment id', 'deployment <id>');
|
|
772
|
+
const res = await q(`/model-deployments/${id}`);
|
|
773
|
+
out(args, res.data, () => formatDeploymentDetail(res.data));
|
|
774
|
+
return;
|
|
775
|
+
}
|
|
776
|
+
case 'scale': {
|
|
777
|
+
const id = needArg(args.positional, 0, 'deployment id', 'scale <deploymentId> <replicas>');
|
|
778
|
+
const raw = needArg(args.positional, 1, 'replica count', 'scale <deploymentId> <replicas>');
|
|
779
|
+
const replicas = Number(raw);
|
|
780
|
+
if (!Number.isInteger(replicas) || replicas < 0)
|
|
781
|
+
throw new UsageError(`replicas must be a whole number of zero or more, got ${raw}`);
|
|
782
|
+
const res = await post(`/model-deployments/${id}/scale`, { replicas });
|
|
783
|
+
out(args, res.data, () => formatDeployment(res.data));
|
|
784
|
+
return;
|
|
785
|
+
}
|
|
786
|
+
case 'teardown': {
|
|
787
|
+
const id = needArg(args.positional, 0, 'deployment id', 'teardown <deploymentId>');
|
|
788
|
+
const res = await post(`/model-deployments/${id}/teardown`, {});
|
|
789
|
+
out(args, res.data, () => formatDeployment(res.data));
|
|
790
|
+
return;
|
|
791
|
+
}
|
|
792
|
+
default:
|
|
793
|
+
console.error(`Unknown command: ${args.command}\n`);
|
|
794
|
+
printHelp();
|
|
795
|
+
process.exit(EXIT.USAGE);
|
|
796
|
+
}
|
|
797
|
+
}
|
|
798
|
+
const invokedDirectly = process.argv[1] && /models-cli|almyty-models|dist\/index\.js|src\/index\.ts/.test(process.argv[1]) && !process.env.VITEST;
|
|
799
|
+
if (invokedDirectly) {
|
|
800
|
+
main().catch((err) => {
|
|
801
|
+
console.error(describeError(err, process.env.ALMYTY_URL));
|
|
802
|
+
process.exit(exitCodeFor(err));
|
|
803
|
+
});
|
|
804
|
+
}
|
package/dist/version.js
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The CLI's own version, read from its package.json at startup.
|
|
3
|
+
*
|
|
4
|
+
* It used to be a hardcoded string, which drifted: `--version` answered
|
|
5
|
+
* 0.1.0 while the published package was 1.2.0, so a bug report never
|
|
6
|
+
* identified the build it came from. Both `dist/index.js` and `src/index.ts`
|
|
7
|
+
* sit one directory below the package root, so the same relative path
|
|
8
|
+
* resolves for the built bin and for `tsx src/index.ts`.
|
|
9
|
+
*/
|
|
10
|
+
import { readFileSync } from 'node:fs';
|
|
11
|
+
export function readVersion(fallback = '0.0.0') {
|
|
12
|
+
try {
|
|
13
|
+
const pkg = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf-8'));
|
|
14
|
+
return typeof pkg.version === 'string' && pkg.version.length > 0
|
|
15
|
+
? pkg.version
|
|
16
|
+
: fallback;
|
|
17
|
+
}
|
|
18
|
+
catch {
|
|
19
|
+
return fallback;
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
export const VERSION = readVersion();
|
package/package.json
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@almyty/models",
|
|
3
|
+
"version": "1.2.0",
|
|
4
|
+
"publishConfig": {
|
|
5
|
+
"access": "public"
|
|
6
|
+
},
|
|
7
|
+
"description": "Manage the almyty model catalog from your terminal: register cards, validate them, deploy weights, watch spend.",
|
|
8
|
+
"type": "module",
|
|
9
|
+
"main": "dist/index.js",
|
|
10
|
+
"bin": {
|
|
11
|
+
"almyty-models": "dist/index.js"
|
|
12
|
+
},
|
|
13
|
+
"files": [
|
|
14
|
+
"dist"
|
|
15
|
+
],
|
|
16
|
+
"scripts": {
|
|
17
|
+
"build": "tsc && chmod +x dist/index.js",
|
|
18
|
+
"dev": "tsx src/index.ts",
|
|
19
|
+
"test": "vitest run",
|
|
20
|
+
"test:watch": "vitest",
|
|
21
|
+
"prepublishOnly": "npm run build"
|
|
22
|
+
},
|
|
23
|
+
"keywords": [
|
|
24
|
+
"almyty",
|
|
25
|
+
"models",
|
|
26
|
+
"cli"
|
|
27
|
+
],
|
|
28
|
+
"author": "almyty",
|
|
29
|
+
"license": "Apache-2.0",
|
|
30
|
+
"dependencies": {
|
|
31
|
+
"@almyty/client": "^1.2.0"
|
|
32
|
+
},
|
|
33
|
+
"devDependencies": {
|
|
34
|
+
"@types/node": "^25.4.0",
|
|
35
|
+
"tsx": "^4.7.0",
|
|
36
|
+
"typescript": "^5.3.0",
|
|
37
|
+
"vitest": "^4.1.0"
|
|
38
|
+
},
|
|
39
|
+
"homepage": "https://almyty.com",
|
|
40
|
+
"repository": {
|
|
41
|
+
"type": "git",
|
|
42
|
+
"url": "git+https://github.com/almyty-inc/almyty.git",
|
|
43
|
+
"directory": "packages/models-cli"
|
|
44
|
+
},
|
|
45
|
+
"bugs": {
|
|
46
|
+
"url": "https://github.com/almyty-inc/almyty/issues"
|
|
47
|
+
}
|
|
48
|
+
}
|