@almyty/models 1.3.0 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -44
- package/dist/index.d.ts +16 -2
- package/dist/index.js +94 -118
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -11,20 +11,25 @@ npx @almyty/models list
|
|
|
11
11
|
|
|
12
12
|
## What makes a model usable
|
|
13
13
|
|
|
14
|
-
Support in almyty is registry data, never a code list.
|
|
15
|
-
|
|
14
|
+
Support in almyty is registry data, never a code list. You connect a provider
|
|
15
|
+
once and every model it lists shows up. A model is usable when its **card**
|
|
16
|
+
exists in your organization's catalog and:
|
|
16
17
|
|
|
17
18
|
1. something can call it — a stored LLM provider row, or an endpoint URL from
|
|
18
|
-
a
|
|
19
|
+
a hosted model,
|
|
19
20
|
2. its status is `active`, and
|
|
20
|
-
3.
|
|
21
|
+
3. it is checked: its provider's key check has passed (one real call with the
|
|
22
|
+
key covers every model that provider lists), or, for an endpoint with no
|
|
23
|
+
provider row, a real short call through that model has.
|
|
21
24
|
|
|
22
|
-
|
|
23
|
-
|
|
25
|
+
A model the provider stops listing goes `inactive`; one the vendor answers
|
|
26
|
+
"model not found" for is marked unavailable. `list` and `get` report which of
|
|
27
|
+
those applies, so a card that will not be picked says so instead of looking
|
|
28
|
+
like any other row:
|
|
24
29
|
|
|
25
30
|
```
|
|
26
31
|
Llama 3 8B [llama-3-8b] private_cloud/eu-central $0.1/$0.2 per M (feed:litellm)
|
|
27
|
-
9c2f… not selectable:
|
|
32
|
+
9c2f… not selectable: waiting for its provider's key check (check the provider on the Models page)
|
|
28
33
|
```
|
|
29
34
|
|
|
30
35
|
## Commands
|
|
@@ -36,17 +41,16 @@ Every read command takes `--json` and writes undecorated JSON to stdout.
|
|
|
36
41
|
| Command | What it does |
|
|
37
42
|
|---|---|
|
|
38
43
|
| `list [--selectable] [--status s] [--tier t] [--provider id]` | Model cards, each line saying selectable or why not |
|
|
39
|
-
| `get <id>` | One card in full: what can call it, capabilities, pricing, the last
|
|
40
|
-
| `register --name <n> --provider <providerId> --model <vendorModelId> [--tier t] [--region r] [--context n]` | Register a card against a stored LLM provider |
|
|
41
|
-
| `register-endpoint --name <n> --url <baseUrl> --model <vendorModelId> [--api-key-stdin] [--tier t] [--region r] [--context n]` | Register any OpenAI-compatible server you run |
|
|
44
|
+
| `get <id>` | One card in full: what can call it, capabilities, pricing, the last check, measured latency |
|
|
45
|
+
| `register --name <n> --provider <providerId> --model <vendorModelId> [--tier t] [--region r] [--context n]` | Register a card against a stored LLM provider. A server you run (vLLM, TGI, llama.cpp) is added as a `custom` LLM provider first, then registered against like any other |
|
|
42
46
|
| `set <id> [--name n] [--tier t] [--region r] [--context n] [--status s] [--price-in n --price-out n] [--clear-price]` | Change a card; a price pair is an override that wins over the automatic feed |
|
|
43
|
-
| `sync [providerId]` | Import what a provider lists
|
|
44
|
-
| `validate <id>` | One real short call.
|
|
47
|
+
| `sync [providerId]` | Import what a provider lists; selectable at once when the provider's key check has passed. With no id, every active provider |
|
|
48
|
+
| `validate <id>` | One real short call through one model. A provider's key check already covers its models; this is for an endpoint with no provider row |
|
|
45
49
|
| `delete <id>` | Remove a card |
|
|
46
50
|
|
|
47
|
-
Cards
|
|
48
|
-
every
|
|
49
|
-
is the same import by hand.
|
|
51
|
+
Cards arrive on their own: connecting a provider, changing it, every passing
|
|
52
|
+
key check and a sweep every six hours import what that provider currently
|
|
53
|
+
lists. `sync` is the same import by hand.
|
|
50
54
|
|
|
51
55
|
### Routing
|
|
52
56
|
|
|
@@ -68,7 +72,7 @@ npx @almyty/models route --objective cheapest --tier private_cloud \
|
|
|
68
72
|
Rejected 3:
|
|
69
73
|
1d0e… no callable provider
|
|
70
74
|
7bb2… privacy tier public exceeds the ceiling private_cloud
|
|
71
|
-
e551…
|
|
75
|
+
e551… not usable yet
|
|
72
76
|
```
|
|
73
77
|
|
|
74
78
|
| Flag | Meaning |
|
|
@@ -85,7 +89,7 @@ Rejected 3:
|
|
|
85
89
|
|
|
86
90
|
It exits 5 when no card satisfies the policy, so a check can be a check.
|
|
87
91
|
|
|
88
|
-
### Versions, adapters,
|
|
92
|
+
### Versions, adapters, hosting
|
|
89
93
|
|
|
90
94
|
Registering a version is optional: do it when you want lineage, a manifest
|
|
91
95
|
digest and evaluation history attached to your own artifact. Skip it to just
|
|
@@ -96,21 +100,24 @@ run a model that already lives somewhere.
|
|
|
96
100
|
| `versions` | Registered model versions |
|
|
97
101
|
| `register-version --name <n> --uri <pinned uri> [--base b] [--quantizations q1,q2]` | `hf://org/repo@sha`, `s3://bucket/key@etag`, `gs://bucket/key@gen`, `file:///path@sha` |
|
|
98
102
|
| `adapters` | Every adapter: what it can run (`modelSchemes`), its capabilities, which config fields are secret |
|
|
99
|
-
| `
|
|
100
|
-
| `
|
|
101
|
-
| `
|
|
103
|
+
| `host <model> --adapter <key> [...]` | Run a model on a provider's managed product |
|
|
104
|
+
| `hosted` | Hosted models: desired vs actual, state, spend |
|
|
105
|
+
| `hosted <id>` | One hosted model in full, including its endpoint and rate |
|
|
102
106
|
| `scale <id> <replicas>` | Set desired replicas; `0` scales to zero |
|
|
103
107
|
| `teardown <id>` | Tear the endpoint down; weights stay in the registry |
|
|
104
108
|
|
|
109
|
+
`deploy`, `deployments` and `deployment <id>` also work, as other names for
|
|
110
|
+
`host`, `hosted` and `hosted <id>`.
|
|
111
|
+
|
|
105
112
|
Naming the model is configuration, so it is the positional argument:
|
|
106
113
|
|
|
107
114
|
```sh
|
|
108
|
-
npx @almyty/models
|
|
109
|
-
npx @almyty/models
|
|
110
|
-
npx @almyty/models
|
|
115
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints
|
|
116
|
+
npx @almyty/models host fireworks://accounts/acme/models/qwen3-tuned --adapter fireworks
|
|
117
|
+
npx @almyty/models host --model-version <id> --adapter modal --desired '{"replicas":1}'
|
|
111
118
|
```
|
|
112
119
|
|
|
113
|
-
An **artifact** reference points at bytes and is pinned, so the
|
|
120
|
+
An **artifact** reference points at bytes and is pinned, so the hosted model is
|
|
114
121
|
reproducible. A **provider reference** (`bedrock://`, `vertex://`,
|
|
115
122
|
`fireworks://`, …) names a model that already exists on a platform, which
|
|
116
123
|
versions it itself. The two do not mix freely: `adapters` lists what each
|
|
@@ -123,35 +130,20 @@ provider can really read, and a mismatch is refused at submit with
|
|
|
123
130
|
history keeps them, and most CI runners echo them. So this tool does not take
|
|
124
131
|
a secret as a flag value.
|
|
125
132
|
|
|
126
|
-
**An endpoint key.** Prompted without echo, or read from stdin:
|
|
127
|
-
|
|
128
|
-
```sh
|
|
129
|
-
npx @almyty/models register-endpoint --name vllm-box --url https://vllm.internal/v1 --model llama-3-8b
|
|
130
|
-
# API key for the endpoint (empty for none): ······
|
|
131
|
-
|
|
132
|
-
pass show vllm/key | npx @almyty/models register-endpoint \
|
|
133
|
-
--name vllm-box --url https://vllm.internal/v1 --model llama-3-8b --api-key-stdin
|
|
134
|
-
|
|
135
|
-
# an endpoint with no key at all
|
|
136
|
-
npx @almyty/models register-endpoint --name open-box --url https://box/v1 --model m --api-key ""
|
|
137
|
-
```
|
|
138
|
-
|
|
139
|
-
`--api-key <value>` is refused, and says this.
|
|
140
|
-
|
|
141
133
|
**Adapter configuration.** Best is not to paste one at all: connect the
|
|
142
134
|
provider account once and name the connection.
|
|
143
135
|
|
|
144
136
|
```sh
|
|
145
|
-
npx @almyty/
|
|
146
|
-
npx @almyty/models
|
|
137
|
+
npx @almyty/credentials add huggingface
|
|
138
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main \
|
|
147
139
|
--adapter huggingface-endpoints --credential <connectionId>
|
|
148
140
|
```
|
|
149
141
|
|
|
150
142
|
Otherwise pass the object from a file or stdin:
|
|
151
143
|
|
|
152
144
|
```sh
|
|
153
|
-
npx @almyty/models
|
|
154
|
-
cat hf.json | npx @almyty/models
|
|
145
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-file hf.json
|
|
146
|
+
cat hf.json | npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-stdin
|
|
155
147
|
```
|
|
156
148
|
|
|
157
149
|
`--config` still works for the fields an adapter does **not** mark secret, and
|
|
@@ -183,7 +175,7 @@ says where a number came from: `feed:litellm`, `feed:openrouter`, `native`,
|
|
|
183
175
|
| 2 | usage error (bad flags, missing argument, unknown command) |
|
|
184
176
|
| 3 | not authenticated — run `npx @almyty/auth login` |
|
|
185
177
|
| 4 | not found |
|
|
186
|
-
| 5 | the operation ran and failed (a
|
|
178
|
+
| 5 | the operation ran and failed (a check that did not pass, a policy that resolves to nothing) |
|
|
187
179
|
|
|
188
180
|
The same table in every `@almyty/*` CLI.
|
|
189
181
|
|
package/dist/index.d.ts
CHANGED
|
@@ -5,6 +5,14 @@ export interface ParsedArgs {
|
|
|
5
5
|
flags: Record<string, string | boolean>;
|
|
6
6
|
}
|
|
7
7
|
export declare function parseArgs(argv: string[]): ParsedArgs;
|
|
8
|
+
/**
|
|
9
|
+
* Other names the hosting commands answer to, matching the API's
|
|
10
|
+
* /model-deployments routes. They work but stay out of --help, which speaks
|
|
11
|
+
* of hosting. `deployment <id>` is `hosted <id>`.
|
|
12
|
+
*/
|
|
13
|
+
export declare const COMMAND_ALIASES: Readonly<Record<string, string>>;
|
|
14
|
+
export declare function resolveCommand(command: string | undefined): string | undefined;
|
|
15
|
+
export declare function helpText(): string;
|
|
8
16
|
/**
|
|
9
17
|
* A number, or a message. `--context abc` used to become NaN, which
|
|
10
18
|
* JSON.stringify turns into null, so the API saw a field it could not
|
|
@@ -26,7 +34,6 @@ export declare function secretFields(schema: any): string[];
|
|
|
26
34
|
export declare function assertNoArgvSecrets(schema: any, config: Record<string, unknown>, flag: string, alternatives: string[]): void;
|
|
27
35
|
/** Request bodies are built from flags here so they can be checked without a network. */
|
|
28
36
|
export declare function registerBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
29
|
-
export declare function registerEndpointBody(flags: ParsedArgs['flags'], apiKey?: string): Record<string, unknown>;
|
|
30
37
|
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
31
38
|
export declare function setBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
32
39
|
/**
|
|
@@ -36,7 +43,7 @@ export declare function setBody(flags: ParsedArgs['flags']): Record<string, unkn
|
|
|
36
43
|
export declare function routePolicy(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
37
44
|
/**
|
|
38
45
|
* Naming the model is configuration, so the model reference is the
|
|
39
|
-
* positional argument: `
|
|
46
|
+
* positional argument: `host hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
40
47
|
* `--model-version` is the other way in, for people who registered an
|
|
41
48
|
* artifact to get lineage and evaluation history with it.
|
|
42
49
|
*/
|
|
@@ -69,3 +76,10 @@ export declare function formatSync(data: any): string;
|
|
|
69
76
|
* type JSON and press ctrl-D, which looks exactly like a hang. Say so.
|
|
70
77
|
*/
|
|
71
78
|
export declare function assertStdinIsPiped(flag: string): void;
|
|
79
|
+
/**
|
|
80
|
+
* True when `argv1` is this module, directly or through the npm bin
|
|
81
|
+
* symlink. The old check matched the path against a regex and bailed out
|
|
82
|
+
* when VITEST was set, so a smoke test that spawned the built CLI from
|
|
83
|
+
* vitest inherited VITEST and got no output at all.
|
|
84
|
+
*/
|
|
85
|
+
export declare function isEntrypoint(argv1: string | undefined, moduleUrl: string): boolean;
|
package/dist/index.js
CHANGED
|
@@ -3,21 +3,21 @@
|
|
|
3
3
|
* @almyty/models: the model catalog from the terminal.
|
|
4
4
|
*
|
|
5
5
|
* Support in almyty is registry data, never a code list: a model is usable
|
|
6
|
-
* when its card exists, has something that can call it, is active, and
|
|
7
|
-
*
|
|
6
|
+
* when its card exists, has something that can call it, is active, and is
|
|
7
|
+
* checked (its provider's key check passed). `list` and `get` say which is missing,
|
|
8
8
|
* and `route` answers the question a list cannot — what a routing policy
|
|
9
9
|
* would pick right now, and why it rejected the rest. See docs/models.md.
|
|
10
10
|
*
|
|
11
11
|
* Adapter configuration and endpoint keys are secrets, so they are never
|
|
12
12
|
* taken from argv: argv is readable through `ps` and lands in shell history.
|
|
13
13
|
*/
|
|
14
|
-
import {
|
|
15
|
-
import {
|
|
14
|
+
import { readFileSync, realpathSync } from 'fs';
|
|
15
|
+
import { fileURLToPath } from 'url';
|
|
16
16
|
import { AlmytyClient, resolveCredentialsOrExit } from '@almyty/client';
|
|
17
17
|
import { EXIT, EXIT_CODE_HELP, UsageError, describeError, exitCodeFor } from './exit-codes.js';
|
|
18
18
|
import { VERSION } from './version.js';
|
|
19
19
|
/** Flags that never take a value, so they never swallow the next argument. */
|
|
20
|
-
const BOOLEAN_FLAGS = new Set(['json', 'selectable', 'config-stdin', '
|
|
20
|
+
const BOOLEAN_FLAGS = new Set(['json', 'selectable', 'config-stdin', 'clear-price']);
|
|
21
21
|
export function parseArgs(argv) {
|
|
22
22
|
const result = { positional: [], flags: {} };
|
|
23
23
|
for (let i = 0; i < argv.length; i++) {
|
|
@@ -69,12 +69,29 @@ export function parseArgs(argv) {
|
|
|
69
69
|
}
|
|
70
70
|
return result;
|
|
71
71
|
}
|
|
72
|
+
/**
|
|
73
|
+
* Other names the hosting commands answer to, matching the API's
|
|
74
|
+
* /model-deployments routes. They work but stay out of --help, which speaks
|
|
75
|
+
* of hosting. `deployment <id>` is `hosted <id>`.
|
|
76
|
+
*/
|
|
77
|
+
export const COMMAND_ALIASES = {
|
|
78
|
+
deploy: 'host',
|
|
79
|
+
deployments: 'hosted',
|
|
80
|
+
deployment: 'hosted',
|
|
81
|
+
};
|
|
82
|
+
export function resolveCommand(command) {
|
|
83
|
+
return command && Object.prototype.hasOwnProperty.call(COMMAND_ALIASES, command) ? COMMAND_ALIASES[command] : command;
|
|
84
|
+
}
|
|
72
85
|
function printHelp() {
|
|
73
|
-
console.log(
|
|
86
|
+
console.log(helpText());
|
|
87
|
+
}
|
|
88
|
+
export function helpText() {
|
|
89
|
+
return `
|
|
74
90
|
@almyty/models v${VERSION}
|
|
75
91
|
|
|
76
92
|
A model is usable when its card is active, has something that can call it,
|
|
77
|
-
and
|
|
93
|
+
and is checked: its provider's key check passed (every model of a provider at
|
|
94
|
+
once), or, for an endpoint with no provider, its own check. So
|
|
78
95
|
\`list\` and \`get\` report which of those is missing rather than a name alone.
|
|
79
96
|
|
|
80
97
|
Usage:
|
|
@@ -84,23 +101,23 @@ Catalog:
|
|
|
84
101
|
list [--selectable] [--status active|inactive|error|deploying]
|
|
85
102
|
[--tier public|private_cloud|local] [--provider <providerId>]
|
|
86
103
|
List cards; each line says selectable, or why not
|
|
87
|
-
get <id> One card in full: capabilities, pricing,
|
|
104
|
+
get <id> One card in full: capabilities, pricing, last check
|
|
88
105
|
register --name <n> --provider <providerId> --model <vendorModelId>
|
|
89
106
|
[--tier public|private_cloud|local] [--region <r>] [--context <n>]
|
|
90
|
-
Register a card against a stored LLM provider
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
Register any OpenAI-compatible server you run.
|
|
94
|
-
The key is prompted, or read with --api-key-stdin.
|
|
107
|
+
Register a card against a stored LLM provider.
|
|
108
|
+
A server you run (vLLM, TGI, llama.cpp) is a
|
|
109
|
+
\`custom\` LLM provider; register against that.
|
|
95
110
|
set <id> [--name <n>] [--tier <t>] [--region <r>] [--context <n>]
|
|
96
111
|
[--status active|inactive|error|deploying]
|
|
97
112
|
[--price-in <usdPerMTok> --price-out <usdPerMTok>] [--clear-price]
|
|
98
113
|
Change a card. A price pair is an override that
|
|
99
114
|
wins over the automatic feed; --clear-price drops it.
|
|
100
|
-
sync [providerId] Import what a provider lists
|
|
115
|
+
sync [providerId] Import what a provider lists. Selectable at once when
|
|
116
|
+
the provider's key check has passed.
|
|
101
117
|
With no id, every active provider of the organization.
|
|
102
|
-
validate <id> One real short call
|
|
103
|
-
|
|
118
|
+
validate <id> One real short call through one model. A provider's
|
|
119
|
+
key check covers its models; use this for an endpoint.
|
|
120
|
+
Exits non-zero when it fails.
|
|
104
121
|
delete <id> Remove a card
|
|
105
122
|
|
|
106
123
|
Routing (nothing is called; this plans):
|
|
@@ -118,14 +135,16 @@ Versions (optional: register an artifact only for lineage and evals on it):
|
|
|
118
135
|
hf://org/repo@sha | s3://bucket/key@etag
|
|
119
136
|
gs://bucket/key@gen | file:///path@sha
|
|
120
137
|
|
|
121
|
-
|
|
138
|
+
Hosting:
|
|
122
139
|
adapters Registered adapters: what each can run (modelSchemes),
|
|
123
140
|
its capabilities, and which config fields are secret
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
141
|
+
host <model> --adapter <key> [--base <b>] [--config-file <path>] [--config-stdin]
|
|
142
|
+
[--desired '<json>'] [--credential <connectionId>] [--budget <id>] [--card <cardId>]
|
|
143
|
+
host --model-version <id> --adapter <key> [...]
|
|
127
144
|
<model> is where the model lives:
|
|
128
|
-
hf://org/repo@
|
|
145
|
+
hf://org/repo[@rev] a Hugging Face repository; a
|
|
146
|
+
branch, tag or nothing is
|
|
147
|
+
pinned to its commit for you
|
|
129
148
|
s3://bucket/prefix@etag, gs://bucket/prefix@gen,
|
|
130
149
|
file:///path@sha
|
|
131
150
|
bedrock:// sagemaker:// vertex:// foundry://
|
|
@@ -133,12 +152,12 @@ Deployments:
|
|
|
133
152
|
a model already on that platform
|
|
134
153
|
\`adapters\` lists what each provider accepts; one that
|
|
135
154
|
cannot read your source is refused before anything runs.
|
|
136
|
-
Prefer --credential (a
|
|
137
|
-
\`almyty
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
scale <
|
|
141
|
-
teardown <
|
|
155
|
+
Prefer --credential (a credential added with
|
|
156
|
+
\`almyty credentials add\`) over pasting a key.
|
|
157
|
+
hosted List hosted models: desired vs actual, state, spend
|
|
158
|
+
hosted <id> One hosted model in full
|
|
159
|
+
scale <hostedId> <replicas> Set desired replicas; 0 scales to zero
|
|
160
|
+
teardown <hostedId> Tear the endpoint down; weights stay in the registry
|
|
142
161
|
|
|
143
162
|
Options:
|
|
144
163
|
--json Undecorated JSON on stdout, for scripts
|
|
@@ -152,7 +171,7 @@ Environment:
|
|
|
152
171
|
|
|
153
172
|
Exit codes:
|
|
154
173
|
${EXIT_CODE_HELP}
|
|
155
|
-
|
|
174
|
+
`;
|
|
156
175
|
}
|
|
157
176
|
function str(flags, key) {
|
|
158
177
|
const v = flags[key];
|
|
@@ -249,23 +268,6 @@ export function registerBody(flags) {
|
|
|
249
268
|
body.contextLength = context;
|
|
250
269
|
return body;
|
|
251
270
|
}
|
|
252
|
-
export function registerEndpointBody(flags, apiKey) {
|
|
253
|
-
const body = {
|
|
254
|
-
name: need(flags, 'name'),
|
|
255
|
-
url: need(flags, 'url'),
|
|
256
|
-
vendorModelId: need(flags, 'model'),
|
|
257
|
-
};
|
|
258
|
-
if (apiKey)
|
|
259
|
-
body.apiKey = apiKey;
|
|
260
|
-
if (str(flags, 'tier'))
|
|
261
|
-
body.privacyTier = str(flags, 'tier');
|
|
262
|
-
if (str(flags, 'region'))
|
|
263
|
-
body.region = str(flags, 'region');
|
|
264
|
-
const context = num(flags, 'context');
|
|
265
|
-
if (context !== undefined)
|
|
266
|
-
body.contextLength = context;
|
|
267
|
-
return body;
|
|
268
|
-
}
|
|
269
271
|
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
270
272
|
export function setBody(flags) {
|
|
271
273
|
const body = {};
|
|
@@ -336,7 +338,7 @@ export function routePolicy(flags) {
|
|
|
336
338
|
}
|
|
337
339
|
/**
|
|
338
340
|
* Naming the model is configuration, so the model reference is the
|
|
339
|
-
* positional argument: `
|
|
341
|
+
* positional argument: `host hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
340
342
|
* `--model-version` is the other way in, for people who registered an
|
|
341
343
|
* artifact to get lineage and evaluation history with it.
|
|
342
344
|
*/
|
|
@@ -344,7 +346,7 @@ export function deployBody(flags, positional = [], providerConfig) {
|
|
|
344
346
|
const model = positional[0] ?? str(flags, 'model');
|
|
345
347
|
const modelVersion = str(flags, 'model-version');
|
|
346
348
|
if (!model && !modelVersion) {
|
|
347
|
-
throw new UsageError('Name the model to run (
|
|
349
|
+
throw new UsageError('Name the model to run (host hf://org/repo@sha --adapter <key>), or pass --model-version <id>.');
|
|
348
350
|
}
|
|
349
351
|
const body = { providerType: need(flags, 'adapter') };
|
|
350
352
|
if (modelVersion)
|
|
@@ -394,8 +396,13 @@ export function unselectableReason(c) {
|
|
|
394
396
|
return 'nothing can call it: no provider row and no endpoint URL';
|
|
395
397
|
if (c.validationStatus !== 'passed') {
|
|
396
398
|
const err = c.lastValidationError ? `: ${c.lastValidationError}` : '';
|
|
397
|
-
|
|
398
|
-
|
|
399
|
+
if (c.validationStatus === 'failed')
|
|
400
|
+
return `the provider says this model is not available${err}`;
|
|
401
|
+
// A provider's models are usable once the provider's key check passes;
|
|
402
|
+
// an endpoint with no provider row is checked on its own.
|
|
403
|
+
if (c.providerId)
|
|
404
|
+
return `waiting for its provider's key check${err} (check the provider on the Models page)`;
|
|
405
|
+
return `not checked yet${err} — run: almyty models validate ${c.id}`;
|
|
399
406
|
}
|
|
400
407
|
return 'the catalog does not consider it selectable';
|
|
401
408
|
}
|
|
@@ -431,7 +438,7 @@ export function formatCardDetail(c) {
|
|
|
431
438
|
if (c.measuredLatencyMs)
|
|
432
439
|
lines.push(` latency p50 ${c.measuredLatencyMs.p50 ?? '?'} ms, p95 ${c.measuredLatencyMs.p95 ?? '?'} ms`);
|
|
433
440
|
if (c.deploymentId)
|
|
434
|
-
lines.push(`
|
|
441
|
+
lines.push(` hosted as ${c.deploymentId}`);
|
|
435
442
|
if (c.modelVersionId)
|
|
436
443
|
lines.push(` version ${c.modelVersionId}`);
|
|
437
444
|
if (!c.selectable)
|
|
@@ -529,7 +536,7 @@ export function formatSync(data) {
|
|
|
529
536
|
}
|
|
530
537
|
}
|
|
531
538
|
if ((data.created?.length ?? 0) === 0 && (data.retired?.length ?? 0) === 0) {
|
|
532
|
-
lines.push(
|
|
539
|
+
lines.push("", "A provider's models are selectable once its key check passes; nothing is needed per model.");
|
|
533
540
|
}
|
|
534
541
|
return lines.join('\n');
|
|
535
542
|
}
|
|
@@ -554,7 +561,7 @@ export function assertStdinIsPiped(flag) {
|
|
|
554
561
|
if (!process.stdin.isTTY)
|
|
555
562
|
return;
|
|
556
563
|
throw new UsageError(`${flag} reads stdin, and stdin is your terminal, so it would wait forever.\n` +
|
|
557
|
-
` Pipe it in: cat config.json | almyty models
|
|
564
|
+
` Pipe it in: cat config.json | almyty models host ... ${flag}`);
|
|
558
565
|
}
|
|
559
566
|
function readStdin() {
|
|
560
567
|
return new Promise((resolve, reject) => {
|
|
@@ -565,50 +572,10 @@ function readStdin() {
|
|
|
565
572
|
process.stdin.on('error', reject);
|
|
566
573
|
});
|
|
567
574
|
}
|
|
568
|
-
function askHidden(label) {
|
|
569
|
-
return new Promise((resolve) => {
|
|
570
|
-
const rl = createInterface({ input: process.stdin, output: process.stdout, terminal: true });
|
|
571
|
-
const anyRl = rl;
|
|
572
|
-
anyRl._writeToOutput = (s) => {
|
|
573
|
-
if (s.includes(label))
|
|
574
|
-
anyRl.output.write(label);
|
|
575
|
-
};
|
|
576
|
-
rl.question(label, (answer) => {
|
|
577
|
-
rl.close();
|
|
578
|
-
process.stdout.write('\n');
|
|
579
|
-
resolve(answer.trim());
|
|
580
|
-
});
|
|
581
|
-
});
|
|
582
|
-
}
|
|
583
|
-
/**
|
|
584
|
-
* The endpoint key, from the safest place it can come from. `--api-key` is
|
|
585
|
-
* refused because argv is world-readable; `-` means stdin, which is what a
|
|
586
|
-
* script should use.
|
|
587
|
-
*/
|
|
588
|
-
async function endpointApiKey(flags) {
|
|
589
|
-
const inline = str(flags, 'api-key');
|
|
590
|
-
if (inline && inline !== '-') {
|
|
591
|
-
throw new UsageError('--api-key puts the key in your shell history and in `ps`.\n' +
|
|
592
|
-
' Leave it off and the key is prompted without echo, or read it from stdin:\n' +
|
|
593
|
-
' --api-key-stdin (also: --api-key -)\n' +
|
|
594
|
-
' An endpoint with no key at all: --api-key ""');
|
|
595
|
-
}
|
|
596
|
-
if (flags['api-key-stdin'] || inline === '-') {
|
|
597
|
-
assertStdinIsPiped('--api-key-stdin');
|
|
598
|
-
return (await readStdin()).trim() || undefined;
|
|
599
|
-
}
|
|
600
|
-
// `--api-key` with no value, or an explicitly empty one: an open endpoint.
|
|
601
|
-
if (flags['api-key'] === true || inline === '')
|
|
602
|
-
return undefined;
|
|
603
|
-
if (!process.stdin.isTTY)
|
|
604
|
-
return undefined; // unattended and none supplied: an open endpoint
|
|
605
|
-
const typed = await askHidden('API key for the endpoint (empty for none): ');
|
|
606
|
-
return typed || undefined;
|
|
607
|
-
}
|
|
608
575
|
const CONFIG_ALTERNATIVES = [
|
|
609
576
|
'--config-file <path> read the JSON object from a file',
|
|
610
577
|
'--config-stdin read the JSON object from stdin',
|
|
611
|
-
'--credential <id> use a
|
|
578
|
+
'--credential <id> use a credential added with `almyty credentials add`',
|
|
612
579
|
];
|
|
613
580
|
/**
|
|
614
581
|
* Adapter configuration, from a file or stdin. Never a secret from argv.
|
|
@@ -618,7 +585,7 @@ const CONFIG_ALTERNATIVES = [
|
|
|
618
585
|
* than sent blind: failing open here would make an unreachable catalog the
|
|
619
586
|
* way to get a secret onto the command line.
|
|
620
587
|
*/
|
|
621
|
-
async function
|
|
588
|
+
async function hostConfig(flags, adapterSchema, schemaKnown) {
|
|
622
589
|
const file = str(flags, 'config-file');
|
|
623
590
|
if (file)
|
|
624
591
|
return parseJsonObject(readFileSync(file, 'utf8'), `--config-file ${file}`);
|
|
@@ -657,13 +624,13 @@ async function main() {
|
|
|
657
624
|
const client = newClient();
|
|
658
625
|
const q = (path, init) => client.request(path, init);
|
|
659
626
|
const post = (path, body) => q(path, { method: 'POST', body: JSON.stringify(body) });
|
|
660
|
-
switch (args.command) {
|
|
627
|
+
switch (resolveCommand(args.command)) {
|
|
661
628
|
case 'list': {
|
|
662
629
|
const res = await q(`/models${listQuery(args.flags)}`);
|
|
663
630
|
out(args, res.data, () => (res.data.length
|
|
664
631
|
? res.data.map(formatCard).join('\n')
|
|
665
632
|
: args.flags.selectable
|
|
666
|
-
?
|
|
633
|
+
? "No selectable models. A provider's models become selectable once its key check passes: connect one on the Models page"
|
|
667
634
|
: 'No model cards yet. Register one (almyty models register) or import a provider\'s list (almyty models sync).'));
|
|
668
635
|
return;
|
|
669
636
|
}
|
|
@@ -687,12 +654,6 @@ async function main() {
|
|
|
687
654
|
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
688
655
|
return;
|
|
689
656
|
}
|
|
690
|
-
case 'register-endpoint': {
|
|
691
|
-
const apiKey = await endpointApiKey(args.flags);
|
|
692
|
-
const res = await post('/models/register-endpoint', registerEndpointBody(args.flags, apiKey));
|
|
693
|
-
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
694
|
-
return;
|
|
695
|
-
}
|
|
696
657
|
case 'set': {
|
|
697
658
|
const id = needArg(args.positional, 0, 'card id', 'set <id> [--tier t] [--price-in n --price-out n] ...');
|
|
698
659
|
const res = await q(`/models/${id}`, { method: 'PATCH', body: JSON.stringify(setBody(args.flags)) });
|
|
@@ -736,7 +697,7 @@ async function main() {
|
|
|
736
697
|
out(args, res.data, () => (res.data.length ? res.data.map(formatAdapter).join('\n') : 'No adapters registered.'));
|
|
737
698
|
return;
|
|
738
699
|
}
|
|
739
|
-
case '
|
|
700
|
+
case 'host': {
|
|
740
701
|
// The adapter's schema says which config fields are secret, so the
|
|
741
702
|
// check happens before anything is sent.
|
|
742
703
|
const adapterKey = need(args.flags, 'adapter');
|
|
@@ -755,27 +716,27 @@ async function main() {
|
|
|
755
716
|
if (err instanceof UsageError)
|
|
756
717
|
throw err;
|
|
757
718
|
// The catalog could not be read. The API still validates the body,
|
|
758
|
-
// but --config can no longer be screened, so
|
|
719
|
+
// but --config can no longer be screened, so hostConfig refuses it.
|
|
759
720
|
}
|
|
760
|
-
const providerConfig = await
|
|
721
|
+
const providerConfig = await hostConfig(args.flags, adapterSchema, schemaKnown);
|
|
761
722
|
const res = await post('/model-deployments', deployBody(args.flags, args.positional, providerConfig));
|
|
762
723
|
out(args, res.data, () => `Queued. Reconcile picks it up within a couple of minutes.\n${formatDeployment(res.data)}`);
|
|
763
724
|
return;
|
|
764
725
|
}
|
|
765
|
-
case '
|
|
726
|
+
case 'hosted': {
|
|
727
|
+
const id = args.positional[0];
|
|
728
|
+
if (id) {
|
|
729
|
+
const res = await q(`/model-deployments/${id}`);
|
|
730
|
+
out(args, res.data, () => formatDeploymentDetail(res.data));
|
|
731
|
+
return;
|
|
732
|
+
}
|
|
766
733
|
const res = await q('/model-deployments');
|
|
767
|
-
out(args, res.data, () => (res.data.length ? res.data.map(formatDeployment).join('\n') : 'No
|
|
768
|
-
return;
|
|
769
|
-
}
|
|
770
|
-
case 'deployment': {
|
|
771
|
-
const id = needArg(args.positional, 0, 'deployment id', 'deployment <id>');
|
|
772
|
-
const res = await q(`/model-deployments/${id}`);
|
|
773
|
-
out(args, res.data, () => formatDeploymentDetail(res.data));
|
|
734
|
+
out(args, res.data, () => (res.data.length ? res.data.map(formatDeployment).join('\n') : 'No hosted models.'));
|
|
774
735
|
return;
|
|
775
736
|
}
|
|
776
737
|
case 'scale': {
|
|
777
|
-
const id = needArg(args.positional, 0, '
|
|
778
|
-
const raw = needArg(args.positional, 1, 'replica count', 'scale <
|
|
738
|
+
const id = needArg(args.positional, 0, 'hosted model id', 'scale <hostedId> <replicas>');
|
|
739
|
+
const raw = needArg(args.positional, 1, 'replica count', 'scale <hostedId> <replicas>');
|
|
779
740
|
const replicas = Number(raw);
|
|
780
741
|
if (!Number.isInteger(replicas) || replicas < 0)
|
|
781
742
|
throw new UsageError(`replicas must be a whole number of zero or more, got ${raw}`);
|
|
@@ -784,7 +745,7 @@ async function main() {
|
|
|
784
745
|
return;
|
|
785
746
|
}
|
|
786
747
|
case 'teardown': {
|
|
787
|
-
const id = needArg(args.positional, 0, '
|
|
748
|
+
const id = needArg(args.positional, 0, 'hosted model id', 'teardown <hostedId>');
|
|
788
749
|
const res = await post(`/model-deployments/${id}/teardown`, {});
|
|
789
750
|
out(args, res.data, () => formatDeployment(res.data));
|
|
790
751
|
return;
|
|
@@ -795,8 +756,23 @@ async function main() {
|
|
|
795
756
|
process.exit(EXIT.USAGE);
|
|
796
757
|
}
|
|
797
758
|
}
|
|
798
|
-
|
|
799
|
-
|
|
759
|
+
/**
|
|
760
|
+
* True when `argv1` is this module, directly or through the npm bin
|
|
761
|
+
* symlink. The old check matched the path against a regex and bailed out
|
|
762
|
+
* when VITEST was set, so a smoke test that spawned the built CLI from
|
|
763
|
+
* vitest inherited VITEST and got no output at all.
|
|
764
|
+
*/
|
|
765
|
+
export function isEntrypoint(argv1, moduleUrl) {
|
|
766
|
+
if (!argv1)
|
|
767
|
+
return false;
|
|
768
|
+
try {
|
|
769
|
+
return realpathSync(argv1) === realpathSync(fileURLToPath(moduleUrl));
|
|
770
|
+
}
|
|
771
|
+
catch {
|
|
772
|
+
return false;
|
|
773
|
+
}
|
|
774
|
+
}
|
|
775
|
+
if (isEntrypoint(process.argv[1], import.meta.url)) {
|
|
800
776
|
main().catch((err) => {
|
|
801
777
|
console.error(describeError(err, process.env.ALMYTY_URL));
|
|
802
778
|
process.exit(exitCodeFor(err));
|
package/package.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@almyty/models",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.5.1",
|
|
4
4
|
"publishConfig": {
|
|
5
5
|
"access": "public"
|
|
6
6
|
},
|
|
7
|
-
"description": "Manage the almyty model catalog from your terminal: register cards, validate them,
|
|
7
|
+
"description": "Manage the almyty model catalog from your terminal: register cards, validate them, host weights, watch spend.",
|
|
8
8
|
"type": "module",
|
|
9
9
|
"main": "dist/index.js",
|
|
10
10
|
"bin": {
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
"author": "almyty",
|
|
29
29
|
"license": "Apache-2.0",
|
|
30
30
|
"dependencies": {
|
|
31
|
-
"@almyty/client": "^1.
|
|
31
|
+
"@almyty/client": "^1.5.0"
|
|
32
32
|
},
|
|
33
33
|
"devDependencies": {
|
|
34
34
|
"@types/node": "^25.4.0",
|