@zenera/cli 1.1.3 → 1.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +160 -40
- package/dist/audit.d.ts +5 -2
- package/dist/audit.js +7 -2
- package/dist/catalog.d.ts +111 -0
- package/dist/catalog.js +439 -0
- package/dist/commands/check.js +39 -11
- package/dist/commands/index.d.ts +2 -2
- package/dist/commands/index.js +3 -2
- package/dist/commands/key.js +18 -0
- package/dist/commands/models.d.ts +0 -6
- package/dist/commands/models.js +546 -101
- package/dist/home.d.ts +2 -0
- package/dist/home.js +2 -0
- package/dist/keys.d.ts +9 -1
- package/dist/lib.d.ts +1 -0
- package/dist/lib.js +1 -0
- package/dist/liveness.d.ts +32 -0
- package/dist/liveness.js +194 -5
- package/dist/validate.d.ts +17 -1
- package/dist/validate.js +97 -7
- package/package.json +2 -2
- package/templates/editor/.github/copilot-instructions.md +30 -3
- package/templates/editor/.github/skills/zen-cli/SKILL.md +6 -3
- package/templates/editor/.github/skills/zen-cli/references/check.md +15 -19
- package/templates/editor/.github/skills/zen-cli/references/keys.md +5 -0
- package/templates/editor/.github/skills/zen-cli/references/models.md +108 -0
- package/templates/project/sandbox/Dockerfile +1 -1
|
@@ -1,12 +1,12 @@
|
|
|
1
|
-
# Validating — `zen check`
|
|
1
|
+
# Validating — `zen check`
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
It reads the project without running it, and spends a few tokens asking each
|
|
4
|
+
model to answer once; `--no-models` gives an answer that costs nothing at all.
|
|
5
5
|
|
|
6
6
|
## `zen check`
|
|
7
7
|
|
|
8
8
|
```
|
|
9
|
-
zen check [name|dir] [--project <name|dir>] [--no-sandbox] [--strict] [--quiet]
|
|
9
|
+
zen check [name|dir] [--project <name|dir>] [--no-sandbox] [--no-models] [--strict] [--quiet]
|
|
10
10
|
```
|
|
11
11
|
|
|
12
12
|
Aliases: `validate`, `doctor`.
|
|
@@ -19,6 +19,7 @@ with a code, a location and the fix for it.
|
|
|
19
19
|
| ----------------------- | --------------------------------------------------- |
|
|
20
20
|
| `--project <name\|dir>` | Which project |
|
|
21
21
|
| `--no-sandbox` | Skip building and smoke-testing the container image |
|
|
22
|
+
| `--no-models` | Skip asking each model to answer |
|
|
22
23
|
| `--strict` | Warnings count as failure |
|
|
23
24
|
| `--quiet` | The findings and nothing else |
|
|
24
25
|
|
|
@@ -39,10 +40,16 @@ checked too. A word that is neither is a usage error (exit 2), not a report.
|
|
|
39
40
|
reachable from some agent.
|
|
40
41
|
- Every declared model and embedding resolves to a provider, and that provider
|
|
41
42
|
has a credential on this machine.
|
|
43
|
+
- Unless `--no-models`, every model that has a credential is **asked to answer**
|
|
44
|
+
once. A key that authenticates says nothing about the id it is spent on, so
|
|
45
|
+
this is the only way to catch a misspelt, retired or ungranted model. A refusal
|
|
46
|
+
is an error (`model.refused`, `embedding.refused`); a model that never answered
|
|
47
|
+
is a warning (`.unreachable`), because that is the network's problem and not
|
|
48
|
+
the project's.
|
|
42
49
|
- The sandbox: paths stay inside the project, the Dockerfile and its context
|
|
43
50
|
exist, and — unless `--no-sandbox` — the image **builds** and one command runs
|
|
44
|
-
in it, against a temporary directory rather than your workspace.
|
|
45
|
-
|
|
51
|
+
in it, against a temporary directory rather than your workspace. No container
|
|
52
|
+
engine at all is a warning, not an error.
|
|
46
53
|
|
|
47
54
|
### Findings
|
|
48
55
|
|
|
@@ -59,6 +66,8 @@ skills.no-catalog skills.unreachable skill.unused
|
|
|
59
66
|
assets.missing assets.overbroad sandbox.dockerfile.missing
|
|
60
67
|
sandbox.build sandbox.smoke sandbox.start / .unchecked
|
|
61
68
|
provider.invalid model.none model.unresolvable
|
|
69
|
+
model.refused model.unreachable model.unusable
|
|
70
|
+
embedding.refused embedding.unreachable embedding.unusable
|
|
62
71
|
credential.* service.credential
|
|
63
72
|
```
|
|
64
73
|
|
|
@@ -77,16 +86,3 @@ model with `reasoningEffort` and tools on the default chat-completions API is a
|
|
|
77
86
|
valid configuration that fails at runtime with _"Function tools with
|
|
78
87
|
reasoning_effort are not supported … in /v1/chat/completions"_. Set
|
|
79
88
|
`api: responses` alongside it.
|
|
80
|
-
|
|
81
|
-
## `zen models`
|
|
82
|
-
|
|
83
|
-
```
|
|
84
|
-
zen models [--project <name|dir>]
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
The narrower question: what each agent would actually talk to. It resolves every
|
|
88
|
-
provider, model and embedding the project declares, says which credential each
|
|
89
|
-
one needs and whether it is present, and calls nothing.
|
|
90
|
-
|
|
91
|
-
Reach for `zen models` when a run says a model has no credential and for
|
|
92
|
-
`zen check` when something structural is wrong.
|
|
@@ -97,6 +97,11 @@ the verdict:
|
|
|
97
97
|
- **live** — authenticated. A rate-limited answer counts as live, because it
|
|
98
98
|
proves the credential.
|
|
99
99
|
- **dead** — the provider rejected it. A verdict.
|
|
100
|
+
- **blocked** — the credential was accepted and the **account** refused: an API
|
|
101
|
+
switched off in the project, an empty balance, a model this key was never
|
|
102
|
+
granted. Do not rotate the key; a new one is refused identically. The check
|
|
103
|
+
carries a `fix` — for a disabled Google API, the exact
|
|
104
|
+
`gcloud services enable <api> --project <id>`.
|
|
100
105
|
- **unknown** — the provider could not be asked. Says nothing about the key;
|
|
101
106
|
usually the network.
|
|
102
107
|
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Models — `zen models`
|
|
2
|
+
|
|
3
|
+
```
|
|
4
|
+
zen models <providers|ls|search|show|test|pick> [ref] [options]
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Alias: `model`.
|
|
8
|
+
|
|
9
|
+
Answers **what can I use**. `zen check` answers the other question — whether one
|
|
10
|
+
particular project works — and needs a project to do it. This one needs nothing
|
|
11
|
+
but a credential.
|
|
12
|
+
|
|
13
|
+
## Subcommands
|
|
14
|
+
|
|
15
|
+
| Command | What it does |
|
|
16
|
+
| --------------------------- | ------------------------------------------------------ |
|
|
17
|
+
| `zen models` | Providers, credential source, how many models, how old |
|
|
18
|
+
| `zen models <provider>` | Short for `ls <provider>` |
|
|
19
|
+
| `zen models ls [provider]` | Everything it serves |
|
|
20
|
+
| `zen models search <query>` | Narrow it |
|
|
21
|
+
| `zen models show <ref>` | One model, every field the vendor gave |
|
|
22
|
+
| `zen models test <ref> …` | One real minimal call per ref, and a verdict |
|
|
23
|
+
| `zen models pick` | The first ref that answers, printed on stdout |
|
|
24
|
+
|
|
25
|
+
Filters, on `ls` and `search`: `--chat`, `--embeddings`, `--images`, `--audio`,
|
|
26
|
+
`--tools`, `--vision`, `--free`, `--min-context <n>`, `--provider <name>`,
|
|
27
|
+
`--limit <n>`, `--all`, `--refresh`.
|
|
28
|
+
|
|
29
|
+
`pick` requires `--chat` or `--embedding`, and takes `--provider` and `--limit`.
|
|
30
|
+
|
|
31
|
+
## Where the lists come from
|
|
32
|
+
|
|
33
|
+
The providers themselves, cached for a day in `~/.zenera/neo/catalog`. When a
|
|
34
|
+
provider cannot be asked, the last listing is used and reported as stale; only
|
|
35
|
+
if there was never one does a short built-in list stand in. Every row says which
|
|
36
|
+
it was — `--json` carries `source` per model and `origin` per provider.
|
|
37
|
+
|
|
38
|
+
`--refresh` bypasses the cache. It is the only thing that does.
|
|
39
|
+
|
|
40
|
+
`zen models` on its own does **not** go to the network. `ls` and `search` do.
|
|
41
|
+
|
|
42
|
+
## Testing a model
|
|
43
|
+
|
|
44
|
+
```
|
|
45
|
+
zen models test openai:gpt-4o-mini
|
|
46
|
+
zen models test vertex:gemini-embedding-001 --embedding
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
The role is taken from the flag, else from what the provider says the model is
|
|
50
|
+
for, else from the id. One minimal call: `ok` for a chat model, one short vector
|
|
51
|
+
for an embedder — the embedding's width is reported, which matters because a
|
|
52
|
+
model serving a different number of dimensions is not interchangeable with the
|
|
53
|
+
one an index was built on.
|
|
54
|
+
|
|
55
|
+
Four verdicts, and they want four different actions:
|
|
56
|
+
|
|
57
|
+
| Verdict | Means | Do |
|
|
58
|
+
| ----------- | -------------------------------------------- | -------------------------- |
|
|
59
|
+
| `answers` | it works | nothing |
|
|
60
|
+
| `refused` | the credential was rejected | `zen key check <provider>` |
|
|
61
|
+
| `blocked` | the credential was fine, the account said no | the `fix` printed under it |
|
|
62
|
+
| `no answer` | it could not be reached | try again |
|
|
63
|
+
|
|
64
|
+
Exit `0` when every ref answered, `4` when any did not, `2` for a ref that does
|
|
65
|
+
not parse.
|
|
66
|
+
|
|
67
|
+
## Recovering from a blocked model
|
|
68
|
+
|
|
69
|
+
This is what the command is for.
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
$ zen models test vertex:gemini-embedding-001
|
|
73
|
+
vertex:gemini-embedding-001 blocked Vertex AI API has not been used in project my-proj …
|
|
74
|
+
vertex:gemini-embedding-001: gcloud services enable aiplatform.googleapis.com --project my-proj
|
|
75
|
+
error 1 of 1 did not answer
|
|
76
|
+
find one that does: zen models pick --embedding
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Two ways out. Run the `gcloud` line, or take a different model:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
$ zen models pick --embedding
|
|
83
|
+
openai:text-embedding-3-small
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
`pick` tries a short ordered list one at a time and stops at the first that
|
|
87
|
+
works. Sequential on purpose — the goal is one working ref, not a survey. The
|
|
88
|
+
ref goes to **stdout alone and unstyled**, so it substitutes directly:
|
|
89
|
+
|
|
90
|
+
```sh
|
|
91
|
+
zen rag schema index --embedding "$(zen models pick --embedding)" ./specs/*.yaml
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`zen models pick --embedding --json` gives `{ref, provider, model, role,
|
|
95
|
+
dimensions, ms, tried}` — `tried` lists every candidate and why it was passed
|
|
96
|
+
over, so a caller can see _why_ a provider was skipped rather than only that it
|
|
97
|
+
was.
|
|
98
|
+
|
|
99
|
+
When nothing answers, exit `4` and the table of everything tried.
|
|
100
|
+
|
|
101
|
+
## What it will not do
|
|
102
|
+
|
|
103
|
+
There is no `test --all`. A sweep across every model on the machine is a bill,
|
|
104
|
+
not a diagnostic — name the refs you care about, or use `pick`.
|
|
105
|
+
|
|
106
|
+
There is no ranking. The candidate order in `pick` is cheapest-and-fastest
|
|
107
|
+
first, which is about how quickly an answer arrives, not about which model is
|
|
108
|
+
better.
|
|
@@ -18,4 +18,4 @@ COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
|
|
|
18
18
|
RUN ln -s /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm \
|
|
19
19
|
&& ln -s /usr/local/lib/node_modules/npm/bin/npx-cli.js /usr/local/bin/npx
|
|
20
20
|
|
|
21
|
-
RUN npm install -g @zenera/cli @zenera/rag
|
|
21
|
+
RUN npm install -g @zenera/cli @zenera/rag
|