jevctl 0.1.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -1
- package/NOTICE +4 -0
- package/README.md +54 -388
- package/SECURITY.md +5 -3
- package/dist/cli.js +105 -16
- package/dist/cli.js.map +1 -1
- package/dist/commands/ask.d.ts +5 -1
- package/dist/commands/ask.js +27 -10
- package/dist/commands/ask.js.map +1 -1
- package/dist/commands/auth.d.ts +14 -0
- package/dist/commands/auth.js +158 -0
- package/dist/commands/auth.js.map +1 -0
- package/dist/commands/batch.d.ts +28 -0
- package/dist/commands/batch.js +221 -0
- package/dist/commands/batch.js.map +1 -0
- package/dist/commands/classify.d.ts +34 -0
- package/dist/commands/classify.js +187 -0
- package/dist/commands/classify.js.map +1 -0
- package/dist/commands/compact.d.ts +29 -0
- package/dist/commands/compact.js +146 -0
- package/dist/commands/compact.js.map +1 -0
- package/dist/commands/config.d.ts +2 -1
- package/dist/commands/config.js +27 -10
- package/dist/commands/config.js.map +1 -1
- package/dist/commands/extract.d.ts +18 -0
- package/dist/commands/extract.js +108 -0
- package/dist/commands/extract.js.map +1 -0
- package/dist/commands/find.d.ts +5 -1
- package/dist/commands/find.js +30 -15
- package/dist/commands/find.js.map +1 -1
- package/dist/commands/match.d.ts +18 -0
- package/dist/commands/match.js +118 -0
- package/dist/commands/match.js.map +1 -0
- package/dist/commands/models.js +12 -10
- package/dist/commands/models.js.map +1 -1
- package/dist/commands/rerank.d.ts +15 -0
- package/dist/commands/rerank.js +82 -0
- package/dist/commands/rerank.js.map +1 -0
- package/dist/commands/route.d.ts +19 -0
- package/dist/commands/route.js +120 -0
- package/dist/commands/route.js.map +1 -0
- package/dist/commands/screen.d.ts +5 -1
- package/dist/commands/screen.js +27 -10
- package/dist/commands/screen.js.map +1 -1
- package/dist/commands/verify.d.ts +5 -1
- package/dist/commands/verify.js +23 -12
- package/dist/commands/verify.js.map +1 -1
- package/dist/config.d.ts +66 -6
- package/dist/config.js +53 -2
- package/dist/config.js.map +1 -1
- package/dist/context.d.ts +4 -0
- package/dist/context.js +11 -5
- package/dist/context.js.map +1 -1
- package/dist/core/ask.js +8 -3
- package/dist/core/ask.js.map +1 -1
- package/dist/core/batch.d.ts +42 -0
- package/dist/core/batch.js +93 -0
- package/dist/core/batch.js.map +1 -0
- package/dist/core/classify.d.ts +103 -0
- package/dist/core/classify.js +216 -0
- package/dist/core/classify.js.map +1 -0
- package/dist/core/compact.d.ts +32 -0
- package/dist/core/compact.js +47 -0
- package/dist/core/compact.js.map +1 -0
- package/dist/core/extract.d.ts +58 -0
- package/dist/core/extract.js +214 -0
- package/dist/core/extract.js.map +1 -0
- package/dist/core/find.js +3 -0
- package/dist/core/find.js.map +1 -1
- package/dist/core/match.d.ts +55 -0
- package/dist/core/match.js +114 -0
- package/dist/core/match.js.map +1 -0
- package/dist/core/rerank.d.ts +47 -0
- package/dist/core/rerank.js +60 -0
- package/dist/core/rerank.js.map +1 -0
- package/dist/core/route.d.ts +74 -0
- package/dist/core/route.js +153 -0
- package/dist/core/route.js.map +1 -0
- package/dist/core/transcript.d.ts +15 -0
- package/dist/core/transcript.js +118 -0
- package/dist/core/transcript.js.map +1 -0
- package/dist/credentials.d.ts +30 -0
- package/dist/credentials.js +161 -0
- package/dist/credentials.js.map +1 -0
- package/dist/errors.js.map +1 -1
- package/dist/index.d.ts +10 -0
- package/dist/index.js +10 -0
- package/dist/index.js.map +1 -1
- package/dist/input.js +1 -1
- package/dist/input.js.map +1 -1
- package/dist/lib.d.ts +2 -0
- package/dist/lib.js +9 -0
- package/dist/lib.js.map +1 -1
- package/dist/output.d.ts +41 -3
- package/dist/output.js +131 -6
- package/dist/output.js.map +1 -1
- package/dist/provider.d.ts +8 -0
- package/dist/provider.js +44 -4
- package/dist/provider.js.map +1 -1
- package/dist/vendor/compaction/compact.d.ts +29 -0
- package/dist/vendor/compaction/compact.js +236 -0
- package/dist/vendor/compaction/compact.js.map +1 -0
- package/dist/vendor/compaction/index.d.ts +4 -0
- package/dist/vendor/compaction/index.js +8 -0
- package/dist/vendor/compaction/index.js.map +1 -0
- package/dist/vendor/compaction/request.d.ts +19 -0
- package/dist/vendor/compaction/request.js +53 -0
- package/dist/vendor/compaction/request.js.map +1 -0
- package/dist/vendor/compaction/state.d.ts +28 -0
- package/dist/vendor/compaction/state.js +258 -0
- package/dist/vendor/compaction/state.js.map +1 -0
- package/dist/vendor/compaction/types.d.ts +177 -0
- package/dist/vendor/compaction/types.js +4 -0
- package/dist/vendor/compaction/types.js.map +1 -0
- package/docs/ask.md +52 -0
- package/docs/auth.md +53 -0
- package/docs/batch.md +48 -0
- package/docs/classify.md +55 -0
- package/docs/compact.md +58 -0
- package/docs/config.md +81 -0
- package/docs/extract.md +59 -0
- package/docs/find.md +49 -0
- package/docs/guidelines.md +19 -0
- package/docs/library.md +41 -0
- package/docs/match.md +52 -0
- package/docs/output.md +65 -0
- package/docs/recipes.md +78 -0
- package/docs/rerank.md +44 -0
- package/docs/route.md +57 -0
- package/docs/screen.md +53 -0
- package/docs/troubleshooting.md +17 -0
- package/docs/verify.md +58 -0
- package/package.json +9 -6
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,54 @@ breaking changes to flags or JSON output; they are called out below.
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.2.1] - 2026-09-19
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- `jev auth login`: the key prompt now echoes `*` per keystroke instead of muting all output, so keystrokes are visibly registered. Backspace edits, Ctrl+C aborts cleanly.
|
|
15
|
+
|
|
16
|
+
## [0.2.0] - 2026-09-18
|
|
17
|
+
|
|
18
|
+
### Changed
|
|
19
|
+
|
|
20
|
+
- `jev --help` is regrouped: options under `Output` and `Model and transport`, commands under `Judgments`, `Ranking`, `Pipelines`, and `Account`, each with a one-line summary; headings are bold and names colored on a TTY (`NO_COLOR` honored). Adds an `Examples` block. `jev help <command>` still works and exits 1 on an unknown name.
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
|
|
24
|
+
- `jev classify`: single-label (Choice), multi-label (`--multi`, one Noul per label), and hierarchical (`--taxonomy`, greedy level-by-level) classification with `--other` escape, `--min-confidence`, and `--fail-on review,other,unlabeled`.
|
|
25
|
+
- `jev extract`: regex-candidate extraction with builtin fields (`email`, `phone`, `url`, `amount`, `date`, `percent`, `number`) and custom `name=/regex/:description` fields; Jev selects the span, code normalizes it. No API call when nothing matches.
|
|
26
|
+
- `jev batch <command>`: run `classify`, `screen`, `extract`, `ask`, `verify`, or `find` over plain-line or JSONL input with a concurrency pool; JSONL records in input order, `--output`, `--fail-fast`, exit 1 on row errors and 2 on matched `--fail-on`.
|
|
27
|
+
- `jev rerank`: independent relevance score (Noul) per candidate, sorted, with `--min` keep threshold, `--criteria`, and `--fail-on empty`.
|
|
28
|
+
- `jev match`: same/unclear/different decision per pair from a three-level Score; `--pairs`, `--left/--right` cross product, `--dedupe`; chunked requests up to 200 pairs.
|
|
29
|
+
- `jev route`: handler Choice with a built-in `none` plus speculative typed argument questions (choice/noul/score) per handler in one request; `-H` shorthand or `--handlers-json`; `--fail-on review,unrouted`.
|
|
30
|
+
- `batch` also accepts `rerank` and `route`.
|
|
31
|
+
- `jev compact`: verbatim context compaction for agent transcripts (Claude Code session `.jsonl` or messages JSON). Jev decides per tool call whether the call and its result still matter; nothing is summarized. `--out`, `--goal`, thresholds, `--fail-on low-reduction`. Vendors [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT) under `src/vendor/compaction/`.
|
|
32
|
+
- Claude Code plugin: `session.compact` / `turn.complete` function hook that replaces the built-in compaction summary with the same procedure; `userConfig` options for thresholds and a `compaction` master switch. Requires `CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1`.
|
|
33
|
+
- Config sections `classify`, `extract`, `batch`, `rerank`, `route`.
|
|
34
|
+
- Output formats: `--format json|jsonl|md|csv|tsv` (shortcuts `--json`, `--md`) on every command, `--pluck <path>` to print one value, `JEV_FORMAT` and `config.format` defaults. `batch --format json` collects an array; `md|csv|tsv` print a summary table. Rerank's blank column is now named `Keep`.
|
|
35
|
+
- Docs: README reduced to install, first run, a command index, and shared conventions; one page per command under `docs/`, plus configuration, recipes, guidelines, library, and troubleshooting pages.
|
|
36
|
+
- `jev auth login|status|logout`: store the API key in the macOS Keychain, Linux Secret Service, or a `0600` credentials file instead of exporting it. Keys resolve env first, then the store. `JEV_CREDENTIAL_STORE`, `JEV_CREDENTIALS`, `JEV_NO_STORED_CREDENTIALS`. `jev config` reports the key source.
|
|
37
|
+
- Claude Code plugin (`plugin/`) with the `jev` skill and `/jev:verify`, `/jev:screen`, `/jev:find`, `/jev:ask` commands; installable via `claude plugin marketplace add Nasrallah-AL/jev-cli`.
|
|
38
|
+
|
|
39
|
+
### Changed
|
|
40
|
+
|
|
41
|
+
- `match` results carry `left_text` and `right_text` alongside the ids, and the text table shows `id: text` so string pairs are identifiable.
|
|
42
|
+
|
|
43
|
+
### Fixed
|
|
44
|
+
|
|
45
|
+
- A response with missing, mistyped, or absent answers is now an error (exit 1, `Malformed response from <model>`). Previously `screen` printed `PASS`, `find` reported all candidates at 0.00, and `verify` marked claims `unknown` with exit 0.
|
|
46
|
+
- `find --lines` / `rerank --lines`: ids are now the real 1-based source line numbers (`L7` is line 7 of the file), not the index among non-empty lines.
|
|
47
|
+
- `find`: `--absent` above `--found` is rejected instead of silently producing inconsistent verdicts.
|
|
48
|
+
- `compact`: messages JSON may omit `text` and `toolUses` (a tool-result-only message needs neither).
|
|
49
|
+
- Text and Markdown footers print `no API call made` instead of an empty model and provider when no request was sent.
|
|
50
|
+
- `extract --help` quotes the descriptions-with-spaces examples so they paste into a shell.
|
|
51
|
+
- `extract`: the `number` builtin no longer reads a hyphen glued to a word as a minus sign (`INV-20931` → `20931`) and no longer captures a trailing comma.
|
|
52
|
+
- `config set` rejects keys the schema does not know instead of writing them silently.
|
|
53
|
+
- `--claims @file` containing a JSON object (not an array) is an error rather than one literal claim; a blank claim is rejected before any request.
|
|
54
|
+
- `--pluck` with a path that matches nothing exits 1 instead of printing an empty line (`batch` rows stay lenient).
|
|
55
|
+
- `ask`: a shorthand question id used twice is an error instead of silently overwriting the first question.
|
|
56
|
+
- Keys are masked the same way from every source (`…` plus the last four characters, or `********` for short keys); previously file-stored keys showed their first four characters.
|
|
57
|
+
|
|
10
58
|
## [0.1.0] - 2026-09-18
|
|
11
59
|
|
|
12
60
|
Initial release, published to npm as `jevctl`. The installed command is `jev`.
|
|
@@ -24,5 +72,6 @@ Initial release, published to npm as `jevctl`. The installed command is `jev`.
|
|
|
24
72
|
- Programmatic exports for embedding the same judgments in Node scripts.
|
|
25
73
|
- Test suite: unit tests, provider transport tests with mocked fetch, CLI tests against a local fake TypeSafe API, and optional live e2e tests.
|
|
26
74
|
|
|
27
|
-
[Unreleased]: https://github.com/Nasrallah-AL/jev-cli/compare/v0.
|
|
75
|
+
[Unreleased]: https://github.com/Nasrallah-AL/jev-cli/compare/v0.2.0...HEAD
|
|
76
|
+
[0.2.0]: https://github.com/Nasrallah-AL/jev-cli/compare/v0.1.0...v0.2.0
|
|
28
77
|
[0.1.0]: https://github.com/Nasrallah-AL/jev-cli/releases/tag/v0.1.0
|
package/NOTICE
CHANGED
|
@@ -3,3 +3,7 @@ Copyright (c) 2026 Nasr Shaer
|
|
|
3
3
|
|
|
4
4
|
This product includes code adapted from jev-mcp,
|
|
5
5
|
Copyright (c) 2026 Joey Kudish, licensed under the MIT License.
|
|
6
|
+
|
|
7
|
+
This product includes code adapted from fast-jev-compaction
|
|
8
|
+
(https://github.com/tamaratran/fast-jev-compaction), Copyright (c) 2025,
|
|
9
|
+
licensed under the MIT License. See src/vendor/compaction/ and plugin/hooks/.
|
package/README.md
CHANGED
|
@@ -3,427 +3,93 @@
|
|
|
3
3
|
[](https://github.com/Nasrallah-AL/jev-cli/actions/workflows/ci.yml)
|
|
4
4
|
[](https://www.npmjs.com/package/jevctl)
|
|
5
5
|
[](LICENSE)
|
|
6
|
+
[](https://jevcli.vectorz.app/)
|
|
6
7
|
|
|
7
8
|
Fast, typed AI judgments from the command line, powered by TypeSafe's [Jev](https://docs.typesafe.ai) model.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
Jev does not write text. You give it content and a question with a fixed set of answers, and it returns a probability for each answer in a few hundred milliseconds for a fraction of a cent. `jev` wraps that into commands you can pipe into, script around, and gate CI on.
|
|
10
11
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
| Command | Question it answers | Typical use |
|
|
14
|
-
| --- | --- | --- |
|
|
15
|
-
| `jev verify` | Does this evidence support, contradict, or ignore each claim? | Fact-check a PR description, report, or AI summary against its sources |
|
|
16
|
-
| `jev screen` | Is this text trying to hijack an AI agent? Is it worth reading? | Guardrail before fetched web pages or emails enter an agent's context |
|
|
17
|
-
| `jev find` | Which of these candidates best answers the query? Does any? | Pick the right file, note, or line without building a search index |
|
|
18
|
-
| `jev ask` | Anything with a yes/no, pick-one, or rated answer | Classify, route, score, or extract with your own questions |
|
|
19
|
-
|
|
20
|
-
Every command prints a readable table by default, full JSON with `--json`, and an exit code you can branch on.
|
|
21
|
-
|
|
22
|
-
## Contents
|
|
23
|
-
|
|
24
|
-
- [Install](#install)
|
|
25
|
-
- [Sixty-second start](#sixty-second-start)
|
|
26
|
-
- [Commands](#commands)
|
|
27
|
-
- [Passing input](#passing-input)
|
|
28
|
-
- [Output and exit codes](#output-and-exit-codes)
|
|
29
|
-
- [Configuration](#configuration)
|
|
30
|
-
- [Guidelines for good results](#guidelines-for-good-results)
|
|
31
|
-
- [Recipes](#recipes)
|
|
32
|
-
- [Troubleshooting](#troubleshooting)
|
|
33
|
-
- [Use from Node.js](#use-from-nodejs)
|
|
34
|
-
- [Development](#development)
|
|
12
|
+
Installed from npm as **`jevctl`**. The command is **`jev`**. Website: [jevcli.vectorz.app](https://jevcli.vectorz.app/).
|
|
35
13
|
|
|
36
14
|
## Install
|
|
37
15
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
```bash
|
|
44
|
-
npm install -g jevctl
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
3. Export the key and confirm it is picked up:
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
export TYPESAFE_API_KEY=your_key_here
|
|
51
|
-
jev config
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
`jev config` prints the resolved settings and a masked view of the key. It exits 1 if no credentials are found.
|
|
16
|
+
```bash
|
|
17
|
+
npm install -g jevctl
|
|
18
|
+
jev auth login # paste your TypeSafe key at the hidden prompt; stored in the OS keychain
|
|
19
|
+
jev config # confirms the key is found and shows the effective settings
|
|
20
|
+
```
|
|
55
21
|
|
|
56
|
-
|
|
22
|
+
Needs Node.js 20.12+ and a key from [console.typesafe.ai/settings/keys](https://console.typesafe.ai/settings/keys). No install: `npx jevctl <command>`.
|
|
57
23
|
|
|
58
|
-
##
|
|
24
|
+
## First run
|
|
59
25
|
|
|
60
26
|
```bash
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
--evidence "Every rider must wear an approved helmet. Riders under 18 must wear reflective gear after dark."
|
|
27
|
+
jev verify "Helmets are optional for adults" \
|
|
28
|
+
--evidence "Every rider must wear an approved helmet."
|
|
64
29
|
```
|
|
65
30
|
|
|
66
31
|
```text
|
|
67
32
|
# Verdict Conf Action Claim
|
|
68
33
|
- ------------ ---- ------ -------------------------------
|
|
69
34
|
1 contradicted 1.00 auto Helmets are optional for adults
|
|
70
|
-
2 verified 1.00 auto Reflective gear is mentioned
|
|
71
|
-
|
|
72
|
-
1 verified · 1 contradicted · 0 unsupported · 0 need review (auto-accept ≥ 0.8)
|
|
73
|
-
604 in / 91 out tokens · jev-1.13.0 via typesafe
|
|
74
35
|
```
|
|
75
36
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
```bash
|
|
79
|
-
# 2. Screen a web page before an agent reads it. Exit 2 means "block".
|
|
80
|
-
curl -s https://example.com/pricing | jev screen --purpose "extract pricing tiers"
|
|
81
|
-
|
|
82
|
-
# 3. Find which doc answers a question. No index, no embeddings.
|
|
83
|
-
jev find "how do I rotate API keys" --files docs/*.md
|
|
84
|
-
|
|
85
|
-
# 4. Ask your own questions. Answers come back typed.
|
|
86
|
-
jev ask "I was charged twice. Fix this now." \
|
|
87
|
-
--noul urgent="Does this convey urgency?" \
|
|
88
|
-
--choice team="Which team should handle this?|billing,technical,sales"
|
|
89
|
-
|
|
90
|
-
# 5. See exactly what would be sent, without calling the API.
|
|
91
|
-
jev screen "some text" --dry-run
|
|
92
|
-
```
|
|
37
|
+
Exit code was 2: a claim was contradicted. Add `--json` to any command for machine-readable output, or `--dry-run` to see exactly what would be sent without calling the API.
|
|
93
38
|
|
|
94
39
|
## Commands
|
|
95
40
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
Checks each claim against the evidence you provide. For every claim you get a verdict, the probability of each possible verdict, a confidence score, and whether the verdict can be trusted automatically or should be reviewed by a person.
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
jev verify [claims...] --evidence <ref> [--evidence <ref>...] [options]
|
|
102
|
-
```
|
|
41
|
+
Each command has its own page with options, output fields, and examples.
|
|
103
42
|
|
|
104
|
-
|
|
|
105
|
-
| --- | --- |
|
|
106
|
-
| `verified` | The evidence states the claim or directly implies it |
|
|
107
|
-
| `contradicted` | The evidence says the opposite |
|
|
108
|
-
| `unsupported` | The evidence does not address the claim either way |
|
|
109
|
-
|
|
110
|
-
| Option | Meaning | Default |
|
|
43
|
+
| Command | Answers | Docs |
|
|
111
44
|
| --- | --- | --- |
|
|
112
|
-
|
|
|
113
|
-
|
|
|
114
|
-
|
|
|
115
|
-
|
|
|
116
|
-
|
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
jev
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
```bash
|
|
133
|
-
jev screen [text] [--purpose <text>] [options]
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
| Probability | Question asked |
|
|
137
|
-
| --- | --- |
|
|
138
|
-
| `injection` | Does the text contain instructions aimed at an AI agent, such as "ignore previous instructions" or "visit this URL"? |
|
|
139
|
-
| `substance` | Is there real content, as opposed to an error page or boilerplate? |
|
|
140
|
-
| `relevance` | Is the text useful for the stated `--purpose`? Only asked when a purpose is given. |
|
|
141
|
-
|
|
142
|
-
| Recommendation | When |
|
|
45
|
+
| `jev verify` | Does this evidence support, contradict, or ignore each claim? | [docs/verify.md](docs/verify.md) |
|
|
46
|
+
| `jev screen` | Is this text trying to hijack an AI agent? Is it worth reading? | [docs/screen.md](docs/screen.md) |
|
|
47
|
+
| `jev classify` | Which label fits? Which labels apply? Where in this hierarchy? | [docs/classify.md](docs/classify.md) |
|
|
48
|
+
| `jev extract` | Which span in the text is the email, amount, date, id I want? | [docs/extract.md](docs/extract.md) |
|
|
49
|
+
| `jev find` | Which candidate best answers the query? Does any? | [docs/find.md](docs/find.md) |
|
|
50
|
+
| `jev rerank` | How relevant is each result to the query, on its own? | [docs/rerank.md](docs/rerank.md) |
|
|
51
|
+
| `jev match` | Do these two records describe the same thing? | [docs/match.md](docs/match.md) |
|
|
52
|
+
| `jev route` | Which handler takes this request, with which arguments? | [docs/route.md](docs/route.md) |
|
|
53
|
+
| `jev ask` | Any yes/no, pick-one, or rated question you write yourself | [docs/ask.md](docs/ask.md) |
|
|
54
|
+
| `jev compact` | Which old tool calls in this agent transcript still matter? | [docs/compact.md](docs/compact.md) |
|
|
55
|
+
| `jev batch` | Any of the above, over many rows, with a concurrency pool | [docs/batch.md](docs/batch.md) |
|
|
56
|
+
| `jev auth` | Store the API key in the keychain instead of exporting it | [docs/auth.md](docs/auth.md) |
|
|
57
|
+
| `jev config` | Show or edit settings; check which key source is in use | [docs/config.md](docs/config.md) |
|
|
58
|
+
| `jev models` | List the models your account can use | [docs/config.md#models](docs/config.md#models) |
|
|
59
|
+
|
|
60
|
+
## Conventions
|
|
61
|
+
|
|
62
|
+
Same rules for every command. Details in [docs/output.md](docs/output.md) and [docs/config.md](docs/config.md).
|
|
63
|
+
|
|
64
|
+
| | Rule |
|
|
143
65
|
| --- | --- |
|
|
144
|
-
|
|
|
145
|
-
|
|
|
146
|
-
| `
|
|
147
|
-
|
|
|
148
|
-
|
|
149
|
-
|
|
|
150
|
-
| --- | --- | --- |
|
|
151
|
-
| `[text]` | Text, `@file`, or `-`. Reads stdin when omitted and piped. | stdin |
|
|
152
|
-
| `-p, --purpose <text>` | What the consumer is trying to do. Enables the relevance check and `skip`. | |
|
|
153
|
-
| `--block-at <p>` | Injection probability that triggers `block` | `0.75` |
|
|
154
|
-
| `--review-at <p>` | Injection probability that triggers `review` | `0.25` |
|
|
155
|
-
| `--fail-on <list>` | Exit 2 when the recommendation is one of `block`, `review`, `skip`. Use `none` to always exit 0. | `block` |
|
|
156
|
-
|
|
157
|
-
The recommendation is advisory. `jev` never blocks anything itself. Enforcement is the exit code, or your code reading the JSON.
|
|
158
|
-
|
|
159
|
-
```text
|
|
160
|
-
$ curl -s https://shop.example/sale | jev screen --purpose "summarize the products"
|
|
161
|
-
BLOCK injection probability 0.99 >= block threshold 0.75
|
|
162
|
-
injection 0.99 · substance 0.97 · relevance 0.97
|
|
163
|
-
```
|
|
66
|
+
| Input | Any value is literal text, `@path` for a file, or `-` for stdin (one per command). `@@x` is a literal starting with `@`. |
|
|
67
|
+
| Output | Table by default. `--json` for scripts, `--md` for PR comments, `--format csv\|tsv\|jsonl`, `--pluck <path>` for one value. JSON field names are a stable contract. |
|
|
68
|
+
| Exit codes | `0` ok · `1` usage, config, input, or network error (stderr) · `2` a `--fail-on` condition matched. Code 2 makes `jev` a gate in `&&`, hooks, and CI. |
|
|
69
|
+
| Thresholds | Defaults follow TypeSafe's cookbooks. Sample your data with `--json`, then tune via flags or [config](docs/config.md). |
|
|
70
|
+
| Global flags | `--json` `--md` `--format` `--pluck` `--dry-run` `-m` `-P` `--timeout` `-q` `--no-color`, before or after the subcommand. |
|
|
71
|
+
| Errors | One line on stderr, exit 1. A response that is not one answer per question is `Malformed response`, never a silent pass. |
|
|
164
72
|
|
|
165
|
-
|
|
73
|
+
## Claude Code plugin
|
|
166
74
|
|
|
167
|
-
|
|
75
|
+
The repo ships a plugin that teaches Claude Code when to use `jev`, adds `/jev:*` slash commands, and replaces Claude Code's compaction summary with `jev compact`'s verbatim procedure.
|
|
168
76
|
|
|
169
77
|
```bash
|
|
170
|
-
|
|
78
|
+
claude plugin marketplace add Nasrallah-AL/jev-cli
|
|
79
|
+
claude plugin install jev@jev-cli
|
|
171
80
|
```
|
|
172
81
|
|
|
173
|
-
|
|
174
|
-
| --- | --- | --- |
|
|
175
|
-
| `-f, --files <paths...>` | Each file is a candidate. Its id is the path. | |
|
|
176
|
-
| `-c, --candidates <ref>` | JSON candidates: `["text", ...]`, `[{"id": "...", "text": "..."}]`, or `{"id": "text"}` | |
|
|
177
|
-
| `-l, --lines <ref>` | Each non-empty line of a file or stdin is a candidate with id `L1`, `L2`, ... | |
|
|
178
|
-
| `-k, --top-k <n>` | How many ranked results to show | `5` |
|
|
179
|
-
| `--found <p>` | Exists probability at or above which the verdict is `answered` | `0.7` |
|
|
180
|
-
| `--absent <p>` | Exists probability below which the verdict is `absent`. In between is `partial`. | `0.35` |
|
|
181
|
-
| `--fail-on <list>` | Exit 2 when the exists verdict is `absent` or `partial` | `none` |
|
|
182
|
-
|
|
183
|
-
Limits: 250 candidates per call. Each candidate's text is truncated to 2,000 characters before sending.
|
|
184
|
-
|
|
185
|
-
```text
|
|
186
|
-
$ jev find "how do I rotate API keys" --files docs/*.md -k 2
|
|
187
|
-
answered (exists 0.99) how do I rotate API keys
|
|
188
|
-
|
|
189
|
-
# Prob Id Text
|
|
190
|
-
- ---- --------------- ------------------------------------------------------------
|
|
191
|
-
1 0.99 docs/auth.md To rotate an API key: create a new key in Settings > Keys, …
|
|
192
|
-
2 0.01 docs/billing.md Invoices are issued monthly and can be downloaded as PDF.
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
### `jev ask`
|
|
196
|
-
|
|
197
|
-
The general form. Send any state and any number of questions; get one typed answer per question. Questions run in parallel over the same state, so asking five costs about the same latency as asking one.
|
|
198
|
-
|
|
199
|
-
```bash
|
|
200
|
-
jev ask [state] (--noul ... | --choice ... | --score ... | --questions <ref>) [options]
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
Three question types:
|
|
204
|
-
|
|
205
|
-
| Flag | Answer type | Shorthand format |
|
|
206
|
-
| --- | --- | --- |
|
|
207
|
-
| `--noul` | Probability of "yes", from 0 to 1 | `id=Question text?` |
|
|
208
|
-
| `--choice` | One option from a set, with a probability for each and a confidence | `id=Question text?\|optionA,optionB:description,optionC` |
|
|
209
|
-
| `--score` | A position on an ordered scale, with a probability per level and a confidence | `id=Question text?\|low level,middle level,high level` |
|
|
210
|
-
|
|
211
|
-
| Option | Meaning |
|
|
212
|
-
| --- | --- |
|
|
213
|
-
| `[state]` | Text, `@file`, or `-`. Reads stdin when omitted and piped. |
|
|
214
|
-
| `--state-json` | Treat the state as JSON so you can send an object or array |
|
|
215
|
-
| `-q, --questions <ref>` | A JSON questions map in the [TypeSafe API shape](https://docs.typesafe.ai/api), instead of shorthand flags |
|
|
216
|
-
|
|
217
|
-
Shorthand rules: the `=` separates the id from the question, the first `|` separates the question from its options, and options are comma separated. Write `\|` or `\,` for a literal pipe or comma. Ids may contain letters, digits, `_`, `-`, and `.`.
|
|
218
|
-
|
|
219
|
-
```text
|
|
220
|
-
$ jev ask @ticket.txt --noul urgent="Does this convey urgency?" \
|
|
221
|
-
--choice team="Which team?|billing:refunds and invoices,technical:bugs and outages,sales" \
|
|
222
|
-
--score frustration="How frustrated is the customer?|calm,frustrated,very angry"
|
|
223
|
-
urgent: 0.92
|
|
224
|
-
team: billing conf 0.82 [billing 0.85, technical 0.08, sales 0.07]
|
|
225
|
-
frustration: 1.60 conf 0.71 [0 0.10, 1 0.20, 2 0.70]
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
### `jev models`
|
|
229
|
-
|
|
230
|
-
Lists the models your account can use, with release dates. Requires the TypeSafe provider.
|
|
231
|
-
|
|
232
|
-
### `jev config`
|
|
233
|
-
|
|
234
|
-
Manages the config file and shows the effective settings.
|
|
235
|
-
|
|
236
|
-
| Subcommand | Does |
|
|
237
|
-
| --- | --- |
|
|
238
|
-
| `jev config` or `jev config show` | Print effective settings, masked credentials, and the resolved provider. Exit 1 if none. |
|
|
239
|
-
| `jev config path` | Print the config file location |
|
|
240
|
-
| `jev config init` | Write a file with the defaults |
|
|
241
|
-
| `jev config set <key> <value>` | Set one value, e.g. `jev config set screen.blockAt 0.6` |
|
|
242
|
-
| `jev config unset <key>` | Remove a top-level key |
|
|
243
|
-
| `jev config reset` | Delete the file |
|
|
82
|
+
See [plugin/README.md](plugin/README.md), and [plugin/hooks/README.md](plugin/hooks/README.md) for the compaction hook.
|
|
244
83
|
|
|
245
|
-
##
|
|
246
|
-
|
|
247
|
-
Every place that accepts a value accepts three forms:
|
|
248
|
-
|
|
249
|
-
| You write | jev reads |
|
|
250
|
-
| --- | --- |
|
|
251
|
-
| `some text` | The literal text |
|
|
252
|
-
| `@path/to/file` | The file's contents |
|
|
253
|
-
| `-` | Standard input |
|
|
254
|
-
| `@@text` | The literal text `@text` (escape for values that start with `@`) |
|
|
255
|
-
|
|
256
|
-
Stdin is read once and shared, so only one argument per command can be `-`.
|
|
257
|
-
|
|
258
|
-
Lists (claims) accept one item per line or a JSON array of strings. Items with ids (evidence, candidates) accept a JSON array of strings, an array of `{id, text}` objects, or an object mapping id to text.
|
|
259
|
-
|
|
260
|
-
## Output and exit codes
|
|
261
|
-
|
|
262
|
-
Text output is the default and is meant for humans. Add `--json` for scripts. The JSON always includes `command`, `model`, `provider`, and `usage` (token counts), plus the command's results. Field names are a stable contract and changes are noted in [CHANGELOG.md](CHANGELOG.md).
|
|
263
|
-
|
|
264
|
-
| Exit code | Meaning |
|
|
265
|
-
| --- | --- |
|
|
266
|
-
| `0` | Success, and no `--fail-on` condition matched |
|
|
267
|
-
| `1` | Usage, configuration, input, or network error. Details on stderr. |
|
|
268
|
-
| `2` | The command ran fine, but a `--fail-on` condition matched |
|
|
269
|
-
|
|
270
|
-
Code 2 is what lets `jev` act as a gate in `&&` chains, hooks, and CI steps.
|
|
271
|
-
|
|
272
|
-
### Global options
|
|
273
|
-
|
|
274
|
-
Work before or after the subcommand.
|
|
275
|
-
|
|
276
|
-
| Option | Meaning |
|
|
277
|
-
| --- | --- |
|
|
278
|
-
| `--json` | JSON output. Same as `--format json`. |
|
|
279
|
-
| `-m, --model <name>` | Model to use, e.g. `jev-latest` or a pinned `jev-1.13.0` |
|
|
280
|
-
| `-P, --provider <name>` | `auto`, `typesafe`, `openrouter`, or `cloudflare` |
|
|
281
|
-
| `--timeout <ms>` | Per-request timeout. Default `30000`. |
|
|
282
|
-
| `--dry-run` | Print the exact request that would be sent, then exit 0 without calling the API |
|
|
283
|
-
| `-q, --quiet` | Omit the token usage footer in text output |
|
|
284
|
-
| `--no-color` | Disable colors. `NO_COLOR` and `FORCE_COLOR` environment variables are also honored. |
|
|
285
|
-
|
|
286
|
-
## Configuration
|
|
287
|
-
|
|
288
|
-
Settings resolve in this order, later wins: built-in defaults, config file, environment variables, command-line flags.
|
|
289
|
-
|
|
290
|
-
The config file lives at `$JEV_CONFIG` if set, else `$XDG_CONFIG_HOME/jev/config.json`, else `~/.config/jev/config.json`. It never holds API keys.
|
|
291
|
-
|
|
292
|
-
```json
|
|
293
|
-
{
|
|
294
|
-
"provider": "auto",
|
|
295
|
-
"model": "jev-latest",
|
|
296
|
-
"timeoutMs": 30000,
|
|
297
|
-
"format": "text",
|
|
298
|
-
"verify": { "autoAccept": 0.8 },
|
|
299
|
-
"screen": { "blockAt": 0.75, "reviewAt": 0.25 },
|
|
300
|
-
"find": { "topK": 5, "found": 0.7, "absent": 0.35 }
|
|
301
|
-
}
|
|
302
|
-
```
|
|
303
|
-
|
|
304
|
-
### Environment variables
|
|
305
|
-
|
|
306
|
-
| Variable | Purpose |
|
|
307
|
-
| --- | --- |
|
|
308
|
-
| `TYPESAFE_API_KEY` | TypeSafe API key. Recommended provider; used automatically when set. |
|
|
309
|
-
| `TYPESAFE_BASE_URL` | Alternate TypeSafe endpoint, for proxies or testing |
|
|
310
|
-
| `OPENROUTER_API_KEY` | OpenRouter key (`sk-or-...`). Used when no TypeSafe key is present. |
|
|
311
|
-
| `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` | Cloudflare Workers AI. Used when no other key is present. `JEV_CLOUDFLARE_API_TOKEN` takes precedence over `CLOUDFLARE_API_TOKEN` if you need separate credentials. |
|
|
312
|
-
| `JEV_PROVIDER` | Force `typesafe`, `openrouter`, or `cloudflare` |
|
|
313
|
-
| `JEV_MODEL` | Default model |
|
|
314
|
-
| `JEV_TIMEOUT_MS` | Default timeout |
|
|
315
|
-
| `JEV_FORMAT` | Default output format, `text` or `json` |
|
|
316
|
-
| `JEV_CONFIG` | Config file path |
|
|
317
|
-
| `JEV_DEBUG=1` | Print stack traces on errors |
|
|
318
|
-
|
|
319
|
-
### Providers
|
|
320
|
-
|
|
321
|
-
| Provider | Notes |
|
|
322
|
-
| --- | --- |
|
|
323
|
-
| TypeSafe (direct) | Recommended. Lowest latency, `jev-latest` alias, pinned versions, automatic retries with backoff. |
|
|
324
|
-
| OpenRouter | Alpha endpoint. Serves pinned versions only, so `jev-latest` maps to `typesafe/jev-1.13`. Adds a network hop. |
|
|
325
|
-
| Cloudflare Workers AI | Serves the single `typesafe/jev` alias, no version pinning. Adds a network hop. |
|
|
326
|
-
|
|
327
|
-
## Guidelines for good results
|
|
328
|
-
|
|
329
|
-
**Write claims as single, checkable statements.** "The API returns JSON and supports pagination" is two claims. Split them so each gets its own verdict.
|
|
330
|
-
|
|
331
|
-
**Give the model everything it needs, and only that.** Evidence should contain the passages the claims depend on. Very long evidence dilutes accuracy and costs more. For quote-level checks, locate the relevant passage in code first and send only that.
|
|
332
|
-
|
|
333
|
-
**Treat thresholds as starting points.** The defaults (`0.8` auto-accept, `0.75` block, `0.25` review, `0.7`/`0.35` exists) come from TypeSafe's published cookbooks. Run `jev` over a sample of your own data with `--json`, look at the distributions, and set thresholds that match the cost of a wrong answer in your situation. Pin a model version with `--model jev-1.13.0` once you have tuned them, so an alias update does not silently shift results.
|
|
334
|
-
|
|
335
|
-
**Read confidence correctly.** For `choice` and `score` answers, confidence measures how concentrated the probability distribution is. Low confidence means the options were close, not that the model is wrong. For `noul` answers there is no separate confidence; a value near `0.5` means "as likely yes as no".
|
|
336
|
-
|
|
337
|
-
**Keep policy in your code.** `jev` reports judgments. Whether to block, retry, escalate, or ignore is your decision, expressed through `--fail-on` or by reading the JSON.
|
|
338
|
-
|
|
339
|
-
**Mind what you send.** Everything you pass is sent to the configured provider. Do not include secrets or data you are not permitted to share. `--dry-run` shows the exact payload.
|
|
340
|
-
|
|
341
|
-
**Watch cost with the usage footer.** Each result reports input and output tokens. Only input tokens are billed. Batching several questions into one `jev ask` call is cheaper and faster than several calls.
|
|
342
|
-
|
|
343
|
-
**Prefer direct TypeSafe.** Proxies add latency and lag behind on model versions.
|
|
344
|
-
|
|
345
|
-
## Recipes
|
|
346
|
-
|
|
347
|
-
**Gate a pull request on its own description.** Put each claim from the PR body on its own line in `claims.txt`, then:
|
|
348
|
-
|
|
349
|
-
```bash
|
|
350
|
-
git diff origin/main...HEAD | jev verify --claims @claims.txt --evidence - --fail-on contradicted,unsupported
|
|
351
|
-
```
|
|
352
|
-
|
|
353
|
-
**Guard an agent's web fetches.** Only pass content through if it is safe and relevant:
|
|
354
|
-
|
|
355
|
-
```bash
|
|
356
|
-
page=$(curl -s "$url")
|
|
357
|
-
if printf '%s' "$page" | jev screen --purpose "$task" --fail-on block,review,skip -q; then
|
|
358
|
-
printf '%s' "$page" | my-agent --context -
|
|
359
|
-
fi
|
|
360
|
-
```
|
|
361
|
-
|
|
362
|
-
**Route a support ticket in a shell script.**
|
|
363
|
-
|
|
364
|
-
```bash
|
|
365
|
-
team=$(jev ask @ticket.txt --choice team="Which team?|billing,technical,sales" --json | jq -r .answers.team.choice)
|
|
366
|
-
```
|
|
367
|
-
|
|
368
|
-
**Find the right file, then open it.**
|
|
369
|
-
|
|
370
|
-
```bash
|
|
371
|
-
jev find "where is retry logic configured" --files src/**/*.ts --json | jq -r '.top[0].id' | xargs code
|
|
372
|
-
```
|
|
373
|
-
|
|
374
|
-
**Check that a knowledge base actually answers a question** before handing the top hit to a user:
|
|
375
|
-
|
|
376
|
-
```bash
|
|
377
|
-
jev find "$question" --lines @faq.txt --fail-on absent,partial --json
|
|
378
|
-
```
|
|
379
|
-
|
|
380
|
-
## Troubleshooting
|
|
381
|
-
|
|
382
|
-
| Symptom | Fix |
|
|
383
|
-
| --- | --- |
|
|
384
|
-
| `No credentials found` | Export `TYPESAFE_API_KEY`, then run `jev config` to confirm it is visible to the process |
|
|
385
|
-
| `HTTP 401` | The key is wrong or revoked. Check it at console.typesafe.ai and re-export. |
|
|
386
|
-
| `HTTP 429` | Rate limited. The TypeSafe provider retries automatically; if it persists, slow down or batch questions. |
|
|
387
|
-
| `Request timed out` | Raise `--timeout` or set `JEV_TIMEOUT_MS`. Large evidence takes longer. |
|
|
388
|
-
| `Expected ... on stdin but stdin is a terminal` | You used `-` without piping anything in. Pass text or `@file` instead. |
|
|
389
|
-
| `Too many candidates` | `find` accepts 250 per call. Pre-filter in code or split the set. |
|
|
390
|
-
| Colors in captured output | Add `--no-color` or set `NO_COLOR=1` |
|
|
391
|
-
| Want to see the request | Add `--dry-run` |
|
|
392
|
-
| Want a stack trace | Set `JEV_DEBUG=1` |
|
|
393
|
-
|
|
394
|
-
## Use from Node.js
|
|
395
|
-
|
|
396
|
-
The same functions the CLI uses are exported, so you can embed judgments in a script without shelling out.
|
|
397
|
-
|
|
398
|
-
```ts
|
|
399
|
-
import { createAsk, runVerify, runScreen, runFind } from "jevctl";
|
|
400
|
-
|
|
401
|
-
const ask = createAsk({ provider: "auto", model: "jev-latest", timeoutMs: 30_000 });
|
|
402
|
-
|
|
403
|
-
const verdicts = await runVerify(ask, {
|
|
404
|
-
claims: ["The ordinance mentions reflective gear."],
|
|
405
|
-
evidence: [{ id: "ordinance", text: ordinanceText }],
|
|
406
|
-
autoAccept: 0.8,
|
|
407
|
-
});
|
|
408
|
-
console.log(verdicts.results[0].verdict, verdicts.results[0].confidence);
|
|
409
|
-
|
|
410
|
-
const screened = await runScreen(ask, { text: pageHtml, purpose: "extract prices", blockAt: 0.75, reviewAt: 0.25 });
|
|
411
|
-
if (screened.recommendation.action === "block") throw new Error(screened.recommendation.reason);
|
|
412
|
-
```
|
|
413
|
-
|
|
414
|
-
## Development
|
|
415
|
-
|
|
416
|
-
```bash
|
|
417
|
-
git clone https://github.com/Nasrallah-AL/jev-cli
|
|
418
|
-
cd jev-cli
|
|
419
|
-
npm install
|
|
420
|
-
npm run check # typecheck, lint, and tests. No API key needed.
|
|
421
|
-
npm run build # compiles to dist/
|
|
422
|
-
npm run dev -- screen "hello" --dry-run # run from source
|
|
423
|
-
npm run test:e2e # live API tests. Requires TYPESAFE_API_KEY.
|
|
424
|
-
```
|
|
84
|
+
## More
|
|
425
85
|
|
|
426
|
-
|
|
86
|
+
- [Output formats](docs/output.md): text, json, jsonl, md, csv, tsv, and `--pluck`
|
|
87
|
+
- [Configuration](docs/config.md): config file, environment variables, providers, global flags
|
|
88
|
+
- [Recipes](docs/recipes.md): CI gates, agent guardrails, dedupe, routing, compaction
|
|
89
|
+
- [Guidelines](docs/guidelines.md): writing good claims and labels, reading confidence, cost
|
|
90
|
+
- [Use from Node.js](docs/library.md): the same functions as a library
|
|
91
|
+
- [Troubleshooting](docs/troubleshooting.md)
|
|
92
|
+
- [Contributing](CONTRIBUTING.md), [Security](SECURITY.md), [Changelog](CHANGELOG.md)
|
|
427
93
|
|
|
428
94
|
## License
|
|
429
95
|
|
package/SECURITY.md
CHANGED
|
@@ -13,9 +13,11 @@ an acknowledgement within a few days.
|
|
|
13
13
|
candidates, state) to the configured provider: TypeSafe, OpenRouter, or
|
|
14
14
|
Cloudflare. Do not pass secrets or data you are not allowed to send to those
|
|
15
15
|
services. `--dry-run` shows exactly what would be sent.
|
|
16
|
-
- API keys
|
|
17
|
-
|
|
18
|
-
|
|
16
|
+
- API keys come from environment variables or from `jev auth login`, which
|
|
17
|
+
stores them in the OS keychain (macOS Keychain, Linux Secret Service) or, when
|
|
18
|
+
no keychain is available, in `~/.config/jev/credentials.json` with mode 0600.
|
|
19
|
+
The config file never stores keys. `jev config` and `jev auth status` print
|
|
20
|
+
keys masked.
|
|
19
21
|
- Third-party proxies (OpenRouter, Cloudflare) add a hop; direct TypeSafe is
|
|
20
22
|
the recommended default.
|
|
21
23
|
- Supported versions: the latest published minor release receives fixes.
|