@almyty/models 1.5.0 → 1.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -44
- package/dist/index.d.ts +16 -2
- package/dist/index.js +95 -118
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -11,20 +11,25 @@ npx @almyty/models list
|
|
|
11
11
|
|
|
12
12
|
## What makes a model usable
|
|
13
13
|
|
|
14
|
-
Support in almyty is registry data, never a code list.
|
|
15
|
-
|
|
14
|
+
Support in almyty is registry data, never a code list. You connect a provider
|
|
15
|
+
once and every model it lists shows up. A model is usable when its **card**
|
|
16
|
+
exists in your organization's catalog and:
|
|
16
17
|
|
|
17
18
|
1. something can call it — a stored LLM provider row, or an endpoint URL from
|
|
18
|
-
a
|
|
19
|
+
a hosted model,
|
|
19
20
|
2. its status is `active`, and
|
|
20
|
-
3.
|
|
21
|
+
3. it is checked: its provider's key check has passed (one real call with the
|
|
22
|
+
key covers every model that provider lists), or, for an endpoint with no
|
|
23
|
+
provider row, a real short call through that model has.
|
|
21
24
|
|
|
22
|
-
|
|
23
|
-
|
|
25
|
+
A model the provider stops listing goes `inactive`; one the vendor answers
|
|
26
|
+
"model not found" for is marked unavailable. `list` and `get` report which of
|
|
27
|
+
those applies, so a card that will not be picked says so instead of looking
|
|
28
|
+
like any other row:
|
|
24
29
|
|
|
25
30
|
```
|
|
26
31
|
Llama 3 8B [llama-3-8b] private_cloud/eu-central $0.1/$0.2 per M (feed:litellm)
|
|
27
|
-
9c2f… not selectable:
|
|
32
|
+
9c2f… not selectable: waiting for its provider's key check (check the provider on the Models page)
|
|
28
33
|
```
|
|
29
34
|
|
|
30
35
|
## Commands
|
|
@@ -36,17 +41,16 @@ Every read command takes `--json` and writes undecorated JSON to stdout.
|
|
|
36
41
|
| Command | What it does |
|
|
37
42
|
|---|---|
|
|
38
43
|
| `list [--selectable] [--status s] [--tier t] [--provider id]` | Model cards, each line saying selectable or why not |
|
|
39
|
-
| `get <id>` | One card in full: what can call it, capabilities, pricing, the last
|
|
40
|
-
| `register --name <n> --provider <providerId> --model <vendorModelId> [--tier t] [--region r] [--context n]` | Register a card against a stored LLM provider |
|
|
41
|
-
| `register-endpoint --name <n> --url <baseUrl> --model <vendorModelId> [--api-key-stdin] [--tier t] [--region r] [--context n]` | Register any OpenAI-compatible server you run |
|
|
44
|
+
| `get <id>` | One card in full: what can call it, capabilities, pricing, the last check, measured latency |
|
|
45
|
+
| `register --name <n> --provider <providerId> --model <vendorModelId> [--tier t] [--region r] [--context n]` | Register a card against a stored LLM provider. A server you run (vLLM, TGI, llama.cpp) is added as a `custom` LLM provider first, then registered against like any other |
|
|
42
46
|
| `set <id> [--name n] [--tier t] [--region r] [--context n] [--status s] [--price-in n --price-out n] [--clear-price]` | Change a card; a price pair is an override that wins over the automatic feed |
|
|
43
|
-
| `sync [providerId]` | Import what a provider lists
|
|
44
|
-
| `validate <id>` | One real short call.
|
|
47
|
+
| `sync [providerId]` | Import what a provider lists; selectable at once when the provider's key check has passed. With no id, every active provider |
|
|
48
|
+
| `validate <id>` | One real short call through one model. A provider's key check already covers its models; this is for an endpoint with no provider row |
|
|
45
49
|
| `delete <id>` | Remove a card |
|
|
46
50
|
|
|
47
|
-
Cards
|
|
48
|
-
every
|
|
49
|
-
is the same import by hand.
|
|
51
|
+
Cards arrive on their own: connecting a provider, changing it, every passing
|
|
52
|
+
key check and a sweep every six hours import what that provider currently
|
|
53
|
+
lists. `sync` is the same import by hand.
|
|
50
54
|
|
|
51
55
|
### Routing
|
|
52
56
|
|
|
@@ -68,7 +72,7 @@ npx @almyty/models route --objective cheapest --tier private_cloud \
|
|
|
68
72
|
Rejected 3:
|
|
69
73
|
1d0e… no callable provider
|
|
70
74
|
7bb2… privacy tier public exceeds the ceiling private_cloud
|
|
71
|
-
e551…
|
|
75
|
+
e551… not usable yet
|
|
72
76
|
```
|
|
73
77
|
|
|
74
78
|
| Flag | Meaning |
|
|
@@ -85,7 +89,7 @@ Rejected 3:
|
|
|
85
89
|
|
|
86
90
|
It exits 5 when no card satisfies the policy, so a check can be a check.
|
|
87
91
|
|
|
88
|
-
### Versions, adapters,
|
|
92
|
+
### Versions, adapters, hosting
|
|
89
93
|
|
|
90
94
|
Registering a version is optional: do it when you want lineage, a manifest
|
|
91
95
|
digest and evaluation history attached to your own artifact. Skip it to just
|
|
@@ -96,21 +100,24 @@ run a model that already lives somewhere.
|
|
|
96
100
|
| `versions` | Registered model versions |
|
|
97
101
|
| `register-version --name <n> --uri <pinned uri> [--base b] [--quantizations q1,q2]` | `hf://org/repo@sha`, `s3://bucket/key@etag`, `gs://bucket/key@gen`, `file:///path@sha` |
|
|
98
102
|
| `adapters` | Every adapter: what it can run (`modelSchemes`), its capabilities, which config fields are secret |
|
|
99
|
-
| `
|
|
100
|
-
| `
|
|
101
|
-
| `
|
|
103
|
+
| `host <model> --adapter <key> [...]` | Run a model on a provider's managed product |
|
|
104
|
+
| `hosted` | Hosted models: desired vs actual, state, spend |
|
|
105
|
+
| `hosted <id>` | One hosted model in full, including its endpoint and rate |
|
|
102
106
|
| `scale <id> <replicas>` | Set desired replicas; `0` scales to zero |
|
|
103
107
|
| `teardown <id>` | Tear the endpoint down; weights stay in the registry |
|
|
104
108
|
|
|
109
|
+
`deploy`, `deployments` and `deployment <id>` also work, as other names for
|
|
110
|
+
`host`, `hosted` and `hosted <id>`.
|
|
111
|
+
|
|
105
112
|
Naming the model is configuration, so it is the positional argument:
|
|
106
113
|
|
|
107
114
|
```sh
|
|
108
|
-
npx @almyty/models
|
|
109
|
-
npx @almyty/models
|
|
110
|
-
npx @almyty/models
|
|
115
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints
|
|
116
|
+
npx @almyty/models host fireworks://accounts/acme/models/qwen3-tuned --adapter fireworks
|
|
117
|
+
npx @almyty/models host --model-version <id> --adapter modal --desired '{"replicas":1}'
|
|
111
118
|
```
|
|
112
119
|
|
|
113
|
-
An **artifact** reference points at bytes and is pinned, so the
|
|
120
|
+
An **artifact** reference points at bytes and is pinned, so the hosted model is
|
|
114
121
|
reproducible. A **provider reference** (`bedrock://`, `vertex://`,
|
|
115
122
|
`fireworks://`, …) names a model that already exists on a platform, which
|
|
116
123
|
versions it itself. The two do not mix freely: `adapters` lists what each
|
|
@@ -123,35 +130,20 @@ provider can really read, and a mismatch is refused at submit with
|
|
|
123
130
|
history keeps them, and most CI runners echo them. So this tool does not take
|
|
124
131
|
a secret as a flag value.
|
|
125
132
|
|
|
126
|
-
**An endpoint key.** Prompted without echo, or read from stdin:
|
|
127
|
-
|
|
128
|
-
```sh
|
|
129
|
-
npx @almyty/models register-endpoint --name vllm-box --url https://vllm.internal/v1 --model llama-3-8b
|
|
130
|
-
# API key for the endpoint (empty for none): ······
|
|
131
|
-
|
|
132
|
-
pass show vllm/key | npx @almyty/models register-endpoint \
|
|
133
|
-
--name vllm-box --url https://vllm.internal/v1 --model llama-3-8b --api-key-stdin
|
|
134
|
-
|
|
135
|
-
# an endpoint with no key at all
|
|
136
|
-
npx @almyty/models register-endpoint --name open-box --url https://box/v1 --model m --api-key ""
|
|
137
|
-
```
|
|
138
|
-
|
|
139
|
-
`--api-key <value>` is refused, and says this.
|
|
140
|
-
|
|
141
133
|
**Adapter configuration.** Best is not to paste one at all: connect the
|
|
142
134
|
provider account once and name the connection.
|
|
143
135
|
|
|
144
136
|
```sh
|
|
145
|
-
npx @almyty/
|
|
146
|
-
npx @almyty/models
|
|
137
|
+
npx @almyty/credentials add huggingface
|
|
138
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main \
|
|
147
139
|
--adapter huggingface-endpoints --credential <connectionId>
|
|
148
140
|
```
|
|
149
141
|
|
|
150
142
|
Otherwise pass the object from a file or stdin:
|
|
151
143
|
|
|
152
144
|
```sh
|
|
153
|
-
npx @almyty/models
|
|
154
|
-
cat hf.json | npx @almyty/models
|
|
145
|
+
npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-file hf.json
|
|
146
|
+
cat hf.json | npx @almyty/models host hf://Qwen/Qwen3-0.6B@main --adapter huggingface-endpoints --config-stdin
|
|
155
147
|
```
|
|
156
148
|
|
|
157
149
|
`--config` still works for the fields an adapter does **not** mark secret, and
|
|
@@ -183,7 +175,7 @@ says where a number came from: `feed:litellm`, `feed:openrouter`, `native`,
|
|
|
183
175
|
| 2 | usage error (bad flags, missing argument, unknown command) |
|
|
184
176
|
| 3 | not authenticated — run `npx @almyty/auth login` |
|
|
185
177
|
| 4 | not found |
|
|
186
|
-
| 5 | the operation ran and failed (a
|
|
178
|
+
| 5 | the operation ran and failed (a check that did not pass, a policy that resolves to nothing) |
|
|
187
179
|
|
|
188
180
|
The same table in every `@almyty/*` CLI.
|
|
189
181
|
|
package/dist/index.d.ts
CHANGED
|
@@ -5,6 +5,14 @@ export interface ParsedArgs {
|
|
|
5
5
|
flags: Record<string, string | boolean>;
|
|
6
6
|
}
|
|
7
7
|
export declare function parseArgs(argv: string[]): ParsedArgs;
|
|
8
|
+
/**
|
|
9
|
+
* Other names the hosting commands answer to, matching the API's
|
|
10
|
+
* /model-deployments routes. They work but stay out of --help, which speaks
|
|
11
|
+
* of hosting. `deployment <id>` is `hosted <id>`.
|
|
12
|
+
*/
|
|
13
|
+
export declare const COMMAND_ALIASES: Readonly<Record<string, string>>;
|
|
14
|
+
export declare function resolveCommand(command: string | undefined): string | undefined;
|
|
15
|
+
export declare function helpText(): string;
|
|
8
16
|
/**
|
|
9
17
|
* A number, or a message. `--context abc` used to become NaN, which
|
|
10
18
|
* JSON.stringify turns into null, so the API saw a field it could not
|
|
@@ -26,7 +34,6 @@ export declare function secretFields(schema: any): string[];
|
|
|
26
34
|
export declare function assertNoArgvSecrets(schema: any, config: Record<string, unknown>, flag: string, alternatives: string[]): void;
|
|
27
35
|
/** Request bodies are built from flags here so they can be checked without a network. */
|
|
28
36
|
export declare function registerBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
29
|
-
export declare function registerEndpointBody(flags: ParsedArgs['flags'], apiKey?: string): Record<string, unknown>;
|
|
30
37
|
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
31
38
|
export declare function setBody(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
32
39
|
/**
|
|
@@ -36,7 +43,7 @@ export declare function setBody(flags: ParsedArgs['flags']): Record<string, unkn
|
|
|
36
43
|
export declare function routePolicy(flags: ParsedArgs['flags']): Record<string, unknown>;
|
|
37
44
|
/**
|
|
38
45
|
* Naming the model is configuration, so the model reference is the
|
|
39
|
-
* positional argument: `
|
|
46
|
+
* positional argument: `host hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
40
47
|
* `--model-version` is the other way in, for people who registered an
|
|
41
48
|
* artifact to get lineage and evaluation history with it.
|
|
42
49
|
*/
|
|
@@ -69,3 +76,10 @@ export declare function formatSync(data: any): string;
|
|
|
69
76
|
* type JSON and press ctrl-D, which looks exactly like a hang. Say so.
|
|
70
77
|
*/
|
|
71
78
|
export declare function assertStdinIsPiped(flag: string): void;
|
|
79
|
+
/**
|
|
80
|
+
* True when `argv1` is this module, directly or through the npm bin
|
|
81
|
+
* symlink. The old check matched the path against a regex and bailed out
|
|
82
|
+
* when VITEST was set, so a smoke test that spawned the built CLI from
|
|
83
|
+
* vitest inherited VITEST and got no output at all.
|
|
84
|
+
*/
|
|
85
|
+
export declare function isEntrypoint(argv1: string | undefined, moduleUrl: string): boolean;
|
package/dist/index.js
CHANGED
|
@@ -3,21 +3,21 @@
|
|
|
3
3
|
* @almyty/models: the model catalog from the terminal.
|
|
4
4
|
*
|
|
5
5
|
* Support in almyty is registry data, never a code list: a model is usable
|
|
6
|
-
* when its card exists, has something that can call it, is active, and
|
|
7
|
-
*
|
|
6
|
+
* when its card exists, has something that can call it, is active, and is
|
|
7
|
+
* checked (its provider's key check passed). `list` and `get` say which is missing,
|
|
8
8
|
* and `route` answers the question a list cannot — what a routing policy
|
|
9
9
|
* would pick right now, and why it rejected the rest. See docs/models.md.
|
|
10
10
|
*
|
|
11
11
|
* Adapter configuration and endpoint keys are secrets, so they are never
|
|
12
12
|
* taken from argv: argv is readable through `ps` and lands in shell history.
|
|
13
13
|
*/
|
|
14
|
-
import {
|
|
15
|
-
import {
|
|
14
|
+
import { readFileSync, realpathSync } from 'fs';
|
|
15
|
+
import { fileURLToPath } from 'url';
|
|
16
16
|
import { AlmytyClient, resolveCredentialsOrExit } from '@almyty/client';
|
|
17
17
|
import { EXIT, EXIT_CODE_HELP, UsageError, describeError, exitCodeFor } from './exit-codes.js';
|
|
18
18
|
import { VERSION } from './version.js';
|
|
19
19
|
/** Flags that never take a value, so they never swallow the next argument. */
|
|
20
|
-
const BOOLEAN_FLAGS = new Set(['json', 'selectable', 'config-stdin', '
|
|
20
|
+
const BOOLEAN_FLAGS = new Set(['json', 'selectable', 'config-stdin', 'clear-price']);
|
|
21
21
|
export function parseArgs(argv) {
|
|
22
22
|
const result = { positional: [], flags: {} };
|
|
23
23
|
for (let i = 0; i < argv.length; i++) {
|
|
@@ -69,12 +69,29 @@ export function parseArgs(argv) {
|
|
|
69
69
|
}
|
|
70
70
|
return result;
|
|
71
71
|
}
|
|
72
|
+
/**
|
|
73
|
+
* Other names the hosting commands answer to, matching the API's
|
|
74
|
+
* /model-deployments routes. They work but stay out of --help, which speaks
|
|
75
|
+
* of hosting. `deployment <id>` is `hosted <id>`.
|
|
76
|
+
*/
|
|
77
|
+
export const COMMAND_ALIASES = {
|
|
78
|
+
deploy: 'host',
|
|
79
|
+
deployments: 'hosted',
|
|
80
|
+
deployment: 'hosted',
|
|
81
|
+
};
|
|
82
|
+
export function resolveCommand(command) {
|
|
83
|
+
return command && Object.prototype.hasOwnProperty.call(COMMAND_ALIASES, command) ? COMMAND_ALIASES[command] : command;
|
|
84
|
+
}
|
|
72
85
|
function printHelp() {
|
|
73
|
-
console.log(
|
|
86
|
+
console.log(helpText());
|
|
87
|
+
}
|
|
88
|
+
export function helpText() {
|
|
89
|
+
return `
|
|
74
90
|
@almyty/models v${VERSION}
|
|
75
91
|
|
|
76
92
|
A model is usable when its card is active, has something that can call it,
|
|
77
|
-
and
|
|
93
|
+
and is checked: its provider's key check passed (every model of a provider at
|
|
94
|
+
once), or, for an endpoint with no provider, its own check. So
|
|
78
95
|
\`list\` and \`get\` report which of those is missing rather than a name alone.
|
|
79
96
|
|
|
80
97
|
Usage:
|
|
@@ -84,23 +101,23 @@ Catalog:
|
|
|
84
101
|
list [--selectable] [--status active|inactive|error|deploying]
|
|
85
102
|
[--tier public|private_cloud|local] [--provider <providerId>]
|
|
86
103
|
List cards; each line says selectable, or why not
|
|
87
|
-
get <id> One card in full: capabilities, pricing,
|
|
104
|
+
get <id> One card in full: capabilities, pricing, last check
|
|
88
105
|
register --name <n> --provider <providerId> --model <vendorModelId>
|
|
89
106
|
[--tier public|private_cloud|local] [--region <r>] [--context <n>]
|
|
90
|
-
Register a card against a stored LLM provider
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
Register any OpenAI-compatible server you run.
|
|
94
|
-
The key is prompted, or read with --api-key-stdin.
|
|
107
|
+
Register a card against a stored LLM provider.
|
|
108
|
+
A server you run (vLLM, TGI, llama.cpp) is a
|
|
109
|
+
\`custom\` LLM provider; register against that.
|
|
95
110
|
set <id> [--name <n>] [--tier <t>] [--region <r>] [--context <n>]
|
|
96
111
|
[--status active|inactive|error|deploying]
|
|
97
112
|
[--price-in <usdPerMTok> --price-out <usdPerMTok>] [--clear-price]
|
|
98
113
|
Change a card. A price pair is an override that
|
|
99
114
|
wins over the automatic feed; --clear-price drops it.
|
|
100
|
-
sync [providerId] Import what a provider lists
|
|
115
|
+
sync [providerId] Import what a provider lists. Selectable at once when
|
|
116
|
+
the provider's key check has passed.
|
|
101
117
|
With no id, every active provider of the organization.
|
|
102
|
-
validate <id> One real short call
|
|
103
|
-
|
|
118
|
+
validate <id> One real short call through one model. A provider's
|
|
119
|
+
key check covers its models; use this for an endpoint.
|
|
120
|
+
Exits non-zero when it fails.
|
|
104
121
|
delete <id> Remove a card
|
|
105
122
|
|
|
106
123
|
Routing (nothing is called; this plans):
|
|
@@ -118,14 +135,16 @@ Versions (optional: register an artifact only for lineage and evals on it):
|
|
|
118
135
|
hf://org/repo@sha | s3://bucket/key@etag
|
|
119
136
|
gs://bucket/key@gen | file:///path@sha
|
|
120
137
|
|
|
121
|
-
|
|
138
|
+
Hosting:
|
|
122
139
|
adapters Registered adapters: what each can run (modelSchemes),
|
|
123
140
|
its capabilities, and which config fields are secret
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
141
|
+
host <model> --adapter <key> [--base <b>] [--config-file <path>] [--config-stdin]
|
|
142
|
+
[--desired '<json>'] [--credential <connectionId>] [--budget <id>] [--card <cardId>]
|
|
143
|
+
host --model-version <id> --adapter <key> [...]
|
|
127
144
|
<model> is where the model lives:
|
|
128
|
-
hf://org/repo@
|
|
145
|
+
hf://org/repo[@rev] a Hugging Face repository; a
|
|
146
|
+
branch, tag or nothing is
|
|
147
|
+
pinned to its commit for you
|
|
129
148
|
s3://bucket/prefix@etag, gs://bucket/prefix@gen,
|
|
130
149
|
file:///path@sha
|
|
131
150
|
bedrock:// sagemaker:// vertex:// foundry://
|
|
@@ -133,12 +152,12 @@ Deployments:
|
|
|
133
152
|
a model already on that platform
|
|
134
153
|
\`adapters\` lists what each provider accepts; one that
|
|
135
154
|
cannot read your source is refused before anything runs.
|
|
136
|
-
Prefer --credential (a
|
|
137
|
-
\`almyty
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
scale <
|
|
141
|
-
teardown <
|
|
155
|
+
Prefer --credential (a credential added with
|
|
156
|
+
\`almyty credentials add\`) over pasting a key.
|
|
157
|
+
hosted List hosted models: desired vs actual, state, spend
|
|
158
|
+
hosted <id> One hosted model in full
|
|
159
|
+
scale <hostedId> <replicas> Set desired replicas; 0 scales to zero
|
|
160
|
+
teardown <hostedId> Tear the endpoint down; weights stay in the registry
|
|
142
161
|
|
|
143
162
|
Options:
|
|
144
163
|
--json Undecorated JSON on stdout, for scripts
|
|
@@ -152,7 +171,7 @@ Environment:
|
|
|
152
171
|
|
|
153
172
|
Exit codes:
|
|
154
173
|
${EXIT_CODE_HELP}
|
|
155
|
-
|
|
174
|
+
`;
|
|
156
175
|
}
|
|
157
176
|
function str(flags, key) {
|
|
158
177
|
const v = flags[key];
|
|
@@ -249,23 +268,6 @@ export function registerBody(flags) {
|
|
|
249
268
|
body.contextLength = context;
|
|
250
269
|
return body;
|
|
251
270
|
}
|
|
252
|
-
export function registerEndpointBody(flags, apiKey) {
|
|
253
|
-
const body = {
|
|
254
|
-
name: need(flags, 'name'),
|
|
255
|
-
url: need(flags, 'url'),
|
|
256
|
-
vendorModelId: need(flags, 'model'),
|
|
257
|
-
};
|
|
258
|
-
if (apiKey)
|
|
259
|
-
body.apiKey = apiKey;
|
|
260
|
-
if (str(flags, 'tier'))
|
|
261
|
-
body.privacyTier = str(flags, 'tier');
|
|
262
|
-
if (str(flags, 'region'))
|
|
263
|
-
body.region = str(flags, 'region');
|
|
264
|
-
const context = num(flags, 'context');
|
|
265
|
-
if (context !== undefined)
|
|
266
|
-
body.contextLength = context;
|
|
267
|
-
return body;
|
|
268
|
-
}
|
|
269
271
|
/** A card update. Refuses an empty one rather than sending a PATCH that does nothing. */
|
|
270
272
|
export function setBody(flags) {
|
|
271
273
|
const body = {};
|
|
@@ -336,7 +338,7 @@ export function routePolicy(flags) {
|
|
|
336
338
|
}
|
|
337
339
|
/**
|
|
338
340
|
* Naming the model is configuration, so the model reference is the
|
|
339
|
-
* positional argument: `
|
|
341
|
+
* positional argument: `host hf://org/repo@sha --adapter huggingface-endpoints`.
|
|
340
342
|
* `--model-version` is the other way in, for people who registered an
|
|
341
343
|
* artifact to get lineage and evaluation history with it.
|
|
342
344
|
*/
|
|
@@ -344,7 +346,7 @@ export function deployBody(flags, positional = [], providerConfig) {
|
|
|
344
346
|
const model = positional[0] ?? str(flags, 'model');
|
|
345
347
|
const modelVersion = str(flags, 'model-version');
|
|
346
348
|
if (!model && !modelVersion) {
|
|
347
|
-
throw new UsageError('Name the model to run (
|
|
349
|
+
throw new UsageError('Name the model to run (host hf://org/repo@sha --adapter <key>), or pass --model-version <id>.');
|
|
348
350
|
}
|
|
349
351
|
const body = { providerType: need(flags, 'adapter') };
|
|
350
352
|
if (modelVersion)
|
|
@@ -394,8 +396,14 @@ export function unselectableReason(c) {
|
|
|
394
396
|
return 'nothing can call it: no provider row and no endpoint URL';
|
|
395
397
|
if (c.validationStatus !== 'passed') {
|
|
396
398
|
const err = c.lastValidationError ? `: ${c.lastValidationError}` : '';
|
|
397
|
-
|
|
398
|
-
|
|
399
|
+
if (c.validationStatus === 'failed')
|
|
400
|
+
return `the provider says this model is not available${err}`;
|
|
401
|
+
// A provider's models are usable once the provider's key check passes,
|
|
402
|
+
// which is on the connection's page under Credentials; an endpoint with
|
|
403
|
+
// no provider row is checked on its own.
|
|
404
|
+
if (c.providerId)
|
|
405
|
+
return `waiting for its provider's key check${err} (check its connection under Credentials)`;
|
|
406
|
+
return `not checked yet${err} — run: almyty models validate ${c.id}`;
|
|
399
407
|
}
|
|
400
408
|
return 'the catalog does not consider it selectable';
|
|
401
409
|
}
|
|
@@ -431,7 +439,7 @@ export function formatCardDetail(c) {
|
|
|
431
439
|
if (c.measuredLatencyMs)
|
|
432
440
|
lines.push(` latency p50 ${c.measuredLatencyMs.p50 ?? '?'} ms, p95 ${c.measuredLatencyMs.p95 ?? '?'} ms`);
|
|
433
441
|
if (c.deploymentId)
|
|
434
|
-
lines.push(`
|
|
442
|
+
lines.push(` hosted as ${c.deploymentId}`);
|
|
435
443
|
if (c.modelVersionId)
|
|
436
444
|
lines.push(` version ${c.modelVersionId}`);
|
|
437
445
|
if (!c.selectable)
|
|
@@ -529,7 +537,7 @@ export function formatSync(data) {
|
|
|
529
537
|
}
|
|
530
538
|
}
|
|
531
539
|
if ((data.created?.length ?? 0) === 0 && (data.retired?.length ?? 0) === 0) {
|
|
532
|
-
lines.push(
|
|
540
|
+
lines.push("", "A provider's models are selectable once its key check passes; nothing is needed per model.");
|
|
533
541
|
}
|
|
534
542
|
return lines.join('\n');
|
|
535
543
|
}
|
|
@@ -554,7 +562,7 @@ export function assertStdinIsPiped(flag) {
|
|
|
554
562
|
if (!process.stdin.isTTY)
|
|
555
563
|
return;
|
|
556
564
|
throw new UsageError(`${flag} reads stdin, and stdin is your terminal, so it would wait forever.\n` +
|
|
557
|
-
` Pipe it in: cat config.json | almyty models
|
|
565
|
+
` Pipe it in: cat config.json | almyty models host ... ${flag}`);
|
|
558
566
|
}
|
|
559
567
|
function readStdin() {
|
|
560
568
|
return new Promise((resolve, reject) => {
|
|
@@ -565,50 +573,10 @@ function readStdin() {
|
|
|
565
573
|
process.stdin.on('error', reject);
|
|
566
574
|
});
|
|
567
575
|
}
|
|
568
|
-
function askHidden(label) {
|
|
569
|
-
return new Promise((resolve) => {
|
|
570
|
-
const rl = createInterface({ input: process.stdin, output: process.stdout, terminal: true });
|
|
571
|
-
const anyRl = rl;
|
|
572
|
-
anyRl._writeToOutput = (s) => {
|
|
573
|
-
if (s.includes(label))
|
|
574
|
-
anyRl.output.write(label);
|
|
575
|
-
};
|
|
576
|
-
rl.question(label, (answer) => {
|
|
577
|
-
rl.close();
|
|
578
|
-
process.stdout.write('\n');
|
|
579
|
-
resolve(answer.trim());
|
|
580
|
-
});
|
|
581
|
-
});
|
|
582
|
-
}
|
|
583
|
-
/**
|
|
584
|
-
* The endpoint key, from the safest place it can come from. `--api-key` is
|
|
585
|
-
* refused because argv is world-readable; `-` means stdin, which is what a
|
|
586
|
-
* script should use.
|
|
587
|
-
*/
|
|
588
|
-
async function endpointApiKey(flags) {
|
|
589
|
-
const inline = str(flags, 'api-key');
|
|
590
|
-
if (inline && inline !== '-') {
|
|
591
|
-
throw new UsageError('--api-key puts the key in your shell history and in `ps`.\n' +
|
|
592
|
-
' Leave it off and the key is prompted without echo, or read it from stdin:\n' +
|
|
593
|
-
' --api-key-stdin (also: --api-key -)\n' +
|
|
594
|
-
' An endpoint with no key at all: --api-key ""');
|
|
595
|
-
}
|
|
596
|
-
if (flags['api-key-stdin'] || inline === '-') {
|
|
597
|
-
assertStdinIsPiped('--api-key-stdin');
|
|
598
|
-
return (await readStdin()).trim() || undefined;
|
|
599
|
-
}
|
|
600
|
-
// `--api-key` with no value, or an explicitly empty one: an open endpoint.
|
|
601
|
-
if (flags['api-key'] === true || inline === '')
|
|
602
|
-
return undefined;
|
|
603
|
-
if (!process.stdin.isTTY)
|
|
604
|
-
return undefined; // unattended and none supplied: an open endpoint
|
|
605
|
-
const typed = await askHidden('API key for the endpoint (empty for none): ');
|
|
606
|
-
return typed || undefined;
|
|
607
|
-
}
|
|
608
576
|
const CONFIG_ALTERNATIVES = [
|
|
609
577
|
'--config-file <path> read the JSON object from a file',
|
|
610
578
|
'--config-stdin read the JSON object from stdin',
|
|
611
|
-
'--credential <id> use a
|
|
579
|
+
'--credential <id> use a credential added with `almyty credentials add`',
|
|
612
580
|
];
|
|
613
581
|
/**
|
|
614
582
|
* Adapter configuration, from a file or stdin. Never a secret from argv.
|
|
@@ -618,7 +586,7 @@ const CONFIG_ALTERNATIVES = [
|
|
|
618
586
|
* than sent blind: failing open here would make an unreachable catalog the
|
|
619
587
|
* way to get a secret onto the command line.
|
|
620
588
|
*/
|
|
621
|
-
async function
|
|
589
|
+
async function hostConfig(flags, adapterSchema, schemaKnown) {
|
|
622
590
|
const file = str(flags, 'config-file');
|
|
623
591
|
if (file)
|
|
624
592
|
return parseJsonObject(readFileSync(file, 'utf8'), `--config-file ${file}`);
|
|
@@ -657,13 +625,13 @@ async function main() {
|
|
|
657
625
|
const client = newClient();
|
|
658
626
|
const q = (path, init) => client.request(path, init);
|
|
659
627
|
const post = (path, body) => q(path, { method: 'POST', body: JSON.stringify(body) });
|
|
660
|
-
switch (args.command) {
|
|
628
|
+
switch (resolveCommand(args.command)) {
|
|
661
629
|
case 'list': {
|
|
662
630
|
const res = await q(`/models${listQuery(args.flags)}`);
|
|
663
631
|
out(args, res.data, () => (res.data.length
|
|
664
632
|
? res.data.map(formatCard).join('\n')
|
|
665
633
|
: args.flags.selectable
|
|
666
|
-
?
|
|
634
|
+
? "No selectable models. A provider's models become selectable once its key check passes: connect one on the Models page"
|
|
667
635
|
: 'No model cards yet. Register one (almyty models register) or import a provider\'s list (almyty models sync).'));
|
|
668
636
|
return;
|
|
669
637
|
}
|
|
@@ -687,12 +655,6 @@ async function main() {
|
|
|
687
655
|
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
688
656
|
return;
|
|
689
657
|
}
|
|
690
|
-
case 'register-endpoint': {
|
|
691
|
-
const apiKey = await endpointApiKey(args.flags);
|
|
692
|
-
const res = await post('/models/register-endpoint', registerEndpointBody(args.flags, apiKey));
|
|
693
|
-
out(args, res.data, () => `Registered.\n${formatCard(res.data)}\nRun: almyty models validate ${res.data.id}`);
|
|
694
|
-
return;
|
|
695
|
-
}
|
|
696
658
|
case 'set': {
|
|
697
659
|
const id = needArg(args.positional, 0, 'card id', 'set <id> [--tier t] [--price-in n --price-out n] ...');
|
|
698
660
|
const res = await q(`/models/${id}`, { method: 'PATCH', body: JSON.stringify(setBody(args.flags)) });
|
|
@@ -736,7 +698,7 @@ async function main() {
|
|
|
736
698
|
out(args, res.data, () => (res.data.length ? res.data.map(formatAdapter).join('\n') : 'No adapters registered.'));
|
|
737
699
|
return;
|
|
738
700
|
}
|
|
739
|
-
case '
|
|
701
|
+
case 'host': {
|
|
740
702
|
// The adapter's schema says which config fields are secret, so the
|
|
741
703
|
// check happens before anything is sent.
|
|
742
704
|
const adapterKey = need(args.flags, 'adapter');
|
|
@@ -755,27 +717,27 @@ async function main() {
|
|
|
755
717
|
if (err instanceof UsageError)
|
|
756
718
|
throw err;
|
|
757
719
|
// The catalog could not be read. The API still validates the body,
|
|
758
|
-
// but --config can no longer be screened, so
|
|
720
|
+
// but --config can no longer be screened, so hostConfig refuses it.
|
|
759
721
|
}
|
|
760
|
-
const providerConfig = await
|
|
722
|
+
const providerConfig = await hostConfig(args.flags, adapterSchema, schemaKnown);
|
|
761
723
|
const res = await post('/model-deployments', deployBody(args.flags, args.positional, providerConfig));
|
|
762
724
|
out(args, res.data, () => `Queued. Reconcile picks it up within a couple of minutes.\n${formatDeployment(res.data)}`);
|
|
763
725
|
return;
|
|
764
726
|
}
|
|
765
|
-
case '
|
|
727
|
+
case 'hosted': {
|
|
728
|
+
const id = args.positional[0];
|
|
729
|
+
if (id) {
|
|
730
|
+
const res = await q(`/model-deployments/${id}`);
|
|
731
|
+
out(args, res.data, () => formatDeploymentDetail(res.data));
|
|
732
|
+
return;
|
|
733
|
+
}
|
|
766
734
|
const res = await q('/model-deployments');
|
|
767
|
-
out(args, res.data, () => (res.data.length ? res.data.map(formatDeployment).join('\n') : 'No
|
|
768
|
-
return;
|
|
769
|
-
}
|
|
770
|
-
case 'deployment': {
|
|
771
|
-
const id = needArg(args.positional, 0, 'deployment id', 'deployment <id>');
|
|
772
|
-
const res = await q(`/model-deployments/${id}`);
|
|
773
|
-
out(args, res.data, () => formatDeploymentDetail(res.data));
|
|
735
|
+
out(args, res.data, () => (res.data.length ? res.data.map(formatDeployment).join('\n') : 'No hosted models.'));
|
|
774
736
|
return;
|
|
775
737
|
}
|
|
776
738
|
case 'scale': {
|
|
777
|
-
const id = needArg(args.positional, 0, '
|
|
778
|
-
const raw = needArg(args.positional, 1, 'replica count', 'scale <
|
|
739
|
+
const id = needArg(args.positional, 0, 'hosted model id', 'scale <hostedId> <replicas>');
|
|
740
|
+
const raw = needArg(args.positional, 1, 'replica count', 'scale <hostedId> <replicas>');
|
|
779
741
|
const replicas = Number(raw);
|
|
780
742
|
if (!Number.isInteger(replicas) || replicas < 0)
|
|
781
743
|
throw new UsageError(`replicas must be a whole number of zero or more, got ${raw}`);
|
|
@@ -784,7 +746,7 @@ async function main() {
|
|
|
784
746
|
return;
|
|
785
747
|
}
|
|
786
748
|
case 'teardown': {
|
|
787
|
-
const id = needArg(args.positional, 0, '
|
|
749
|
+
const id = needArg(args.positional, 0, 'hosted model id', 'teardown <hostedId>');
|
|
788
750
|
const res = await post(`/model-deployments/${id}/teardown`, {});
|
|
789
751
|
out(args, res.data, () => formatDeployment(res.data));
|
|
790
752
|
return;
|
|
@@ -795,8 +757,23 @@ async function main() {
|
|
|
795
757
|
process.exit(EXIT.USAGE);
|
|
796
758
|
}
|
|
797
759
|
}
|
|
798
|
-
|
|
799
|
-
|
|
760
|
+
/**
|
|
761
|
+
* True when `argv1` is this module, directly or through the npm bin
|
|
762
|
+
* symlink. The old check matched the path against a regex and bailed out
|
|
763
|
+
* when VITEST was set, so a smoke test that spawned the built CLI from
|
|
764
|
+
* vitest inherited VITEST and got no output at all.
|
|
765
|
+
*/
|
|
766
|
+
export function isEntrypoint(argv1, moduleUrl) {
|
|
767
|
+
if (!argv1)
|
|
768
|
+
return false;
|
|
769
|
+
try {
|
|
770
|
+
return realpathSync(argv1) === realpathSync(fileURLToPath(moduleUrl));
|
|
771
|
+
}
|
|
772
|
+
catch {
|
|
773
|
+
return false;
|
|
774
|
+
}
|
|
775
|
+
}
|
|
776
|
+
if (isEntrypoint(process.argv[1], import.meta.url)) {
|
|
800
777
|
main().catch((err) => {
|
|
801
778
|
console.error(describeError(err, process.env.ALMYTY_URL));
|
|
802
779
|
process.exit(exitCodeFor(err));
|
package/package.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@almyty/models",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.2",
|
|
4
4
|
"publishConfig": {
|
|
5
5
|
"access": "public"
|
|
6
6
|
},
|
|
7
|
-
"description": "Manage the almyty model catalog from your terminal: register cards, validate them,
|
|
7
|
+
"description": "Manage the almyty model catalog from your terminal: register cards, validate them, host weights, watch spend.",
|
|
8
8
|
"type": "module",
|
|
9
9
|
"main": "dist/index.js",
|
|
10
10
|
"bin": {
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
"author": "almyty",
|
|
29
29
|
"license": "Apache-2.0",
|
|
30
30
|
"dependencies": {
|
|
31
|
-
"@almyty/client": "^1.
|
|
31
|
+
"@almyty/client": "^1.5.1"
|
|
32
32
|
},
|
|
33
33
|
"devDependencies": {
|
|
34
34
|
"@types/node": "^25.4.0",
|