kronk-cli 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +384 -10
- package/package.json +1 -1
- package/src/agent.js +118 -23
- package/src/argv.js +160 -0
- package/src/boot.js +59 -0
- package/src/client.js +109 -4
- package/src/compact.js +22 -5
- package/src/config.js +38 -0
- package/src/index.js +99 -63
- package/src/plan.js +167 -0
- package/src/reasoning.js +118 -0
- package/src/setup.js +535 -0
- package/src/tools.js +73 -13
- package/src/ui.js +28 -1
package/README.md
CHANGED
|
@@ -23,6 +23,9 @@ entirely on your machine. No network, no API key, no per-token cost.
|
|
|
23
23
|
2104→812 tok · 61.3 tok/s · ttft 240ms · 1980 cached
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
+
*A real session transcript, kept here to show the UI format. For numbers you can reproduce
|
|
27
|
+
yourself on your own hardware, see [Performance](#performance).*
|
|
28
|
+
|
|
26
29
|
[](https://github.com/BardiaN/kronk-cli/actions/workflows/ci.yml)
|
|
27
30
|
[](https://github.com/BardiaN/kronk-cli/actions/workflows/security.yml)
|
|
28
31
|
[](https://github.com/BardiaN/kronk-cli/actions/workflows/codeql.yml)
|
|
@@ -50,6 +53,16 @@ workflow run that produced it:
|
|
|
50
53
|
gh attestation verify kronk-cli-*.tgz --repo BardiaN/kronk-cli
|
|
51
54
|
```
|
|
52
55
|
|
|
56
|
+
The same attestation is attached to every release as a file, so it can be checked without
|
|
57
|
+
GitHub's attestations API in the loop — `<tarball>.sigstore.json` for `gh attestation verify`
|
|
58
|
+
and cosign, `<tarball>.intoto.jsonl` for SLSA tooling:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
gh release download v0.1.3 --repo BardiaN/kronk-cli
|
|
62
|
+
gh attestation verify kronk-cli-0.1.3.tgz --repo BardiaN/kronk-cli \
|
|
63
|
+
--bundle kronk-cli-0.1.3.tgz.sigstore.json
|
|
64
|
+
```
|
|
65
|
+
|
|
53
66
|
npm packages carry the same provenance, shown as a **Provenance** panel on the
|
|
54
67
|
[package page](https://www.npmjs.com/package/kronk-cli), and verifiable locally:
|
|
55
68
|
|
|
@@ -311,18 +324,232 @@ kronk-cli "explain src/agent.js" # one shot, prints and exits
|
|
|
311
324
|
git diff | kronk-cli "review this diff" # stdin as extra context
|
|
312
325
|
git log --oneline -20 | kronk-cli # stdin as the whole prompt
|
|
313
326
|
kronk-cli --auto "make the tests pass" # unattended, runs the whole task
|
|
327
|
+
kronk-cli setup # first run: model, profile, restart
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
---
|
|
331
|
+
|
|
332
|
+
## Subcommands
|
|
333
|
+
|
|
334
|
+
### `kronk-cli setup`
|
|
335
|
+
|
|
336
|
+
The three things a fresh install needs — the model on disk, an `/AGENT` profile for it, and a
|
|
337
|
+
server restart so Kronk reads that profile — walked in order, announcing each step. `--dry-run`
|
|
338
|
+
walks the whole thing and prints what it *would* do, which is the safest way to see it:
|
|
339
|
+
|
|
340
|
+
```console
|
|
341
|
+
$ kronk-cli setup --dry-run --model unsloth/Qwen3-0.6B-Q8_0/AGENT
|
|
342
|
+
|
|
343
|
+
kronk-cli setup · dry run, nothing will change
|
|
344
|
+
|
|
345
|
+
1) Checking the Kronk server
|
|
346
|
+
http://localhost:11435/v1 · serving 2 models
|
|
347
|
+
|
|
348
|
+
2) Resolving the target
|
|
349
|
+
profile unsloth/Qwen3-0.6B-Q8_0/AGENT
|
|
350
|
+
catalog unsloth/Qwen3-0.6B-Q8_0
|
|
351
|
+
binary /opt/homebrew/bin/kronk
|
|
352
|
+
|
|
353
|
+
3) Checking whether the model is downloaded
|
|
354
|
+
would run: kronk catalog show unsloth/Qwen3-0.6B-Q8_0 --local
|
|
355
|
+
|
|
356
|
+
4) Downloading the model
|
|
357
|
+
would run: kronk model pull unsloth/Qwen3-0.6B-Q8_0
|
|
358
|
+
|
|
359
|
+
5) Writing the /AGENT profile
|
|
360
|
+
file ~/.kronk/models/model_config.yaml
|
|
361
|
+
This block will be added:
|
|
362
|
+
|
|
363
|
+
unsloth/Qwen3-0.6B-Q8_0/AGENT:
|
|
364
|
+
context-window: 131072
|
|
365
|
+
nseq-max: 2
|
|
366
|
+
chat-template-kwargs:
|
|
367
|
+
preserve_thinking: true
|
|
368
|
+
sampling-parameters:
|
|
369
|
+
max_tokens: 16384
|
|
370
|
+
|
|
371
|
+
would update: ~/.kronk/models/model_config.yaml
|
|
372
|
+
would back up first: ~/.kronk/models/model_config.yaml.bak…
|
|
373
|
+
|
|
374
|
+
6) Restarting Kronk
|
|
375
|
+
model_config.yaml is read only when the server starts, so the new
|
|
376
|
+
profile does nothing until Kronk is restarted.
|
|
377
|
+
would run: kronk server stop
|
|
378
|
+
would run: kronk server start --detach
|
|
379
|
+
|
|
380
|
+
Dry run complete — nothing was written and nothing was started.
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
| Flag | |
|
|
384
|
+
|---|---|
|
|
385
|
+
| `--model <id>` | Set up a model other than the default. A trailing `/AGENT` is a Kronk profile name, not a catalog id, so it is stripped before pulling |
|
|
386
|
+
| `--context <n>` | Override the profile's `context-window`. Default `131072`, capped at the model's native maximum when Kronk reports one |
|
|
387
|
+
| `-y`, `--yes` | Answer every prompt yes. Required in CI |
|
|
388
|
+
| `--dry-run` | Print every action, including the exact YAML and the exact commands, and change nothing |
|
|
389
|
+
|
|
390
|
+
Nothing slow or destructive happens without an answer: the pull, the file write and the restart
|
|
391
|
+
are each confirmed. Declining exits 0 and prints the commands you would run by hand. If the
|
|
392
|
+
`kronk` binary is not on `PATH`, setup prints the whole manual recipe instead of failing with a
|
|
393
|
+
spawn error. Run it twice and the second run does nothing.
|
|
394
|
+
|
|
395
|
+
Piped or unattended input answers the first question only — readline discards lines nobody is
|
|
396
|
+
waiting for — so a question with no answer left is a **no**: setup says `stdin ended, assuming
|
|
397
|
+
no`, prints the command, and exits 0. Scripts and CI want `-y`.
|
|
398
|
+
|
|
399
|
+
**What it writes.** Only the `models:` mapping of `~/.kronk/models/model_config.yaml` — point it
|
|
400
|
+
elsewhere with `KRONK_MODEL_CONFIG`. This project has no YAML parser and will not gain one, so
|
|
401
|
+
the writer is a structural scan with a single job — find the one `models:` key and splice the
|
|
402
|
+
entry beneath it — and it refuses whenever the answer is not obvious:
|
|
403
|
+
|
|
404
|
+
- The file is copied aside before any write, to `model_config.yaml.bak`, and **never over a
|
|
405
|
+
backup that already exists**: `.bak2`, `.bak3`, and so on until a free name is found.
|
|
406
|
+
- Every line it did not add survives byte for byte — comments, blank lines and key order
|
|
407
|
+
included. LF and CRLF files each keep their own endings.
|
|
408
|
+
- Missing file → created with `version: 1`, `models:` and the entry. No `models:` key → both are
|
|
409
|
+
appended. A `models:` key with no children → the entry becomes its first child.
|
|
410
|
+
- **More than one top-level `models:` key** — two concatenated documents, which does happen in
|
|
411
|
+
the wild — is refused outright. It prints the block, names the file, and exits non-zero
|
|
412
|
+
without touching anything.
|
|
413
|
+
- A missing `~/.kronk/models/` means Kronk has never run. Setup says so and stops, rather than
|
|
414
|
+
creating Kronk's data directory on its behalf.
|
|
415
|
+
|
|
416
|
+
The profile it writes is the one documented under
|
|
417
|
+
[Tip: use an `/AGENT` profile](#tip-use-an-agent-profile) — `context-window`, `nseq-max`,
|
|
418
|
+
`preserve_thinking` and `max_tokens`, and deliberately no sampling parameters.
|
|
419
|
+
|
|
420
|
+
---
|
|
421
|
+
|
|
422
|
+
## Startup: the model is loaded before you type
|
|
423
|
+
|
|
424
|
+
Kronk has no load command. It lists a model in `GET /v1/models` as soon as the
|
|
425
|
+
server starts, but the weights only reach VRAM on the first inference request —
|
|
426
|
+
so on a fresh server the first prompt you type pays a 10–30 s cold load, and
|
|
427
|
+
looks like a hang.
|
|
428
|
+
|
|
429
|
+
`kronk-cli` pays it at boot instead. It asks Kronk what is resident, and if the
|
|
430
|
+
selected model is not, sends the cheapest completion there is — one token, no
|
|
431
|
+
reasoning — to trigger admission:
|
|
432
|
+
|
|
433
|
+
```console
|
|
434
|
+
$ kronk-cli
|
|
435
|
+
loaded unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT · 11.4s
|
|
436
|
+
|
|
437
|
+
██ kronk-cli · local agent, no network
|
|
314
438
|
```
|
|
315
439
|
|
|
440
|
+
A model already in the pool is left alone; nothing is sent. If the selected one
|
|
441
|
+
cannot be admitted — it will not fit next to what is already resident — the
|
|
442
|
+
fallback runs the same order the CLI uses when you name nothing: the configured
|
|
443
|
+
default, then the best id Kronk is serving. Each is tried once, and a failure
|
|
444
|
+
says why:
|
|
445
|
+
|
|
446
|
+
```console
|
|
447
|
+
$ kronk-cli -m Qwen3.6-27B
|
|
448
|
+
unsloth/Qwen3.6-27B-Q4_K_M failed to load — 507 /chat/completions — insufficient VRAM
|
|
449
|
+
loaded unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT · 2.0s · fallback
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
If nothing loads, the original pick stands and the first turn reports the real
|
|
453
|
+
error. A warm-up is a convenience, not a gate — it never decides whether the CLI
|
|
454
|
+
starts.
|
|
455
|
+
|
|
456
|
+
Skip it with `--no-warm` or `KRONK_WARM=false` and the first prompt pays the load,
|
|
457
|
+
as before.
|
|
458
|
+
|
|
459
|
+
---
|
|
460
|
+
|
|
461
|
+
## Sampling override warning
|
|
462
|
+
|
|
463
|
+
The same startup lookup that finds the context window also carries the model's own sampling
|
|
464
|
+
recommendations — `general.sampling.temp`, `top_k` and `top_p`, straight from the GGUF — next
|
|
465
|
+
to the effective `sampling-parameters` Kronk is actually applying once any profile has been
|
|
466
|
+
merged in. `kronk-cli` compares the two and, if a profile is overriding what the model ships,
|
|
467
|
+
says so in one grey line under the startup banner:
|
|
468
|
+
|
|
469
|
+
```console
|
|
470
|
+
$ kronk-cli
|
|
471
|
+
██ kronk-cli · local agent, no network
|
|
472
|
+
model unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT
|
|
473
|
+
server http://localhost:11435/v1
|
|
474
|
+
/help for commands · Ctrl-C to interrupt · /exit to quit
|
|
475
|
+
|
|
476
|
+
note profile overrides the model's own sampling: temperature 0.6 (model recommends 1)
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
One line, however many of the three parameters disagree — not one per parameter. It stays
|
|
480
|
+
silent when the values agree, when the model ships no `general.sampling.*` metadata at all, or
|
|
481
|
+
when the model-info lookup fails: this is information, not a failure, and it never blocks,
|
|
482
|
+
prompts, or changes the exit code.
|
|
483
|
+
|
|
484
|
+
**The fix** is the one described in [Tip: use an `/AGENT` profile](#tip-use-an-agent-profile):
|
|
485
|
+
remove `temperature`, `top_k` and `top_p` from the profile's `sampling-parameters` block so
|
|
486
|
+
Kronk applies what the model itself recommends.
|
|
487
|
+
|
|
488
|
+
Like the rest of the banner, the line is part of the interactive REPL's startup — a one-shot
|
|
489
|
+
run (`kronk-cli "prompt"`) prints no banner and prints nothing here either.
|
|
490
|
+
|
|
491
|
+
---
|
|
492
|
+
|
|
493
|
+
## Performance
|
|
494
|
+
|
|
495
|
+
The transcripts above are real sessions, kept to show what the UI looks like — not a benchmark
|
|
496
|
+
table, and nothing here reproduced them until now. `scripts/bench.mjs` does: a fixed prompt run
|
|
497
|
+
twice for generation speed, a long deterministic filler prompt carried across three turns for
|
|
498
|
+
prompt-cache retention, and — only when the target model is not already resident — a cold-load
|
|
499
|
+
timing using the same request `warm()` sends at boot. It is a dev tool, excluded from the
|
|
500
|
+
published package; run it straight from a checkout:
|
|
501
|
+
|
|
502
|
+
```bash
|
|
503
|
+
node scripts/bench.mjs # table
|
|
504
|
+
node scripts/bench.mjs --json # same numbers, one JSON object, diffable between runs
|
|
505
|
+
node scripts/bench.mjs --skip-cold-load
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
It talks to whatever `KRONK_URL` / `KRONK_MODEL` you already have configured, so the numbers
|
|
509
|
+
below are specific to one machine — measure your own rather than trusting these across different
|
|
510
|
+
hardware.
|
|
511
|
+
|
|
512
|
+
Measured 2026-08-23 on an Apple M4 Max, 64 GB RAM (macOS), against
|
|
513
|
+
`unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT`, kronk 1.31.9, llama.cpp b10549 — from
|
|
514
|
+
`node scripts/bench.mjs`:
|
|
515
|
+
|
|
516
|
+
```
|
|
517
|
+
kronk-bench · model unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT · kronk 1.31.9 · llama.cpp b10549
|
|
518
|
+
|
|
519
|
+
1. Generation speed
|
|
520
|
+
run 1: 229 tok · 2.97s · 77.1 tok/s · ttft 133ms
|
|
521
|
+
run 2: 220 tok · 2.82s · 78 tok/s · ttft 115ms
|
|
522
|
+
|
|
523
|
+
2. Prompt cache retention (filler prompt: 5491 tok)
|
|
524
|
+
turn 1: 5518 tok prompt · 5496 tok cached (100%) · 149ms
|
|
525
|
+
turn 2: 5544 tok prompt · 5511 tok cached (99%) · 185ms
|
|
526
|
+
turn 3: 5571 tok prompt · 5537 tok cached (99%) · 160ms
|
|
527
|
+
|
|
528
|
+
3. Cold load
|
|
529
|
+
skipped — unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT is already resident — a cold load number
|
|
530
|
+
here would be misleading
|
|
531
|
+
```
|
|
532
|
+
|
|
533
|
+
That generation speed and cache-retention region matches the transcripts above (60–80 tok/s,
|
|
534
|
+
prompt cache reused above 99% on repeat turns) — this machine's Kronk server is shared with other
|
|
535
|
+
work, so a run under contention lands lower in that range and a quiet run lands higher; that
|
|
536
|
+
variance is expected, which is why the script reports each run rather than a single averaged
|
|
537
|
+
number. The cold-load section only prints a number the one time a run finds the model not
|
|
538
|
+
resident — do not stop or unload a shared server's model just to force that path; skip it, the
|
|
539
|
+
same way this run did.
|
|
540
|
+
|
|
316
541
|
---
|
|
317
542
|
|
|
318
543
|
## Command-line options
|
|
319
544
|
|
|
545
|
+
|
|
320
546
|
| Flag | Default | |
|
|
321
547
|
|---|---|---|
|
|
322
548
|
| `-m`, `--model <id>` | `unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT` | Model to use. A substring is enough; `/AGENT` profiles win ties |
|
|
323
549
|
| `-l`, `--models`, `--list` | — | List the models Kronk is serving, then exit |
|
|
324
550
|
| `--no-context` | off | Skip the startup scan of the working directory |
|
|
325
551
|
| `--no-compact` | off | Never auto-compact; fail when the window fills instead |
|
|
552
|
+
| `--no-warm` | off | Don't preload the model at startup; let the first prompt trigger the load |
|
|
326
553
|
| `--mcp [names]` | off | Attach MCP servers — bare for all, or a comma list |
|
|
327
554
|
| `--mcp-list` | — | Show configured MCP servers and their tools, then exit |
|
|
328
555
|
| `-a`, `--auto` | off | Autonomous: auto-approve tools **and** run until the task is done. Implies `--yes` |
|
|
@@ -330,9 +557,15 @@ kronk-cli --auto "make the tests pass" # unattended, runs the whole task
|
|
|
330
557
|
| `--no-think` | off | Disable the model's reasoning pass server-side. Much faster |
|
|
331
558
|
| `--steps <n>` | unlimited | Cap tool calls per task. `0`, `off`, `none`, `inf`, `unlimited` all mean no cap |
|
|
332
559
|
| `-h`, `--help` | — | Print all options and exit |
|
|
560
|
+
| `--` | — | End option parsing; everything after it is prompt text, dashes and all |
|
|
333
561
|
|
|
334
|
-
Anything not consumed as
|
|
335
|
-
|
|
562
|
+
Anything not consumed as an option becomes the prompt, except a token that looks like an
|
|
563
|
+
option and is not one: `kronk-cli -auto "…"` exits 2 with `unknown option: -auto` and a
|
|
564
|
+
suggestion, rather than folding the typo into the prompt and running with the mode off.
|
|
565
|
+
Prose is untouched — a dash followed by a space is not option-shaped, so
|
|
566
|
+
`kronk-cli "- fix the dashes bug"` still asks the question — and `--` ends option parsing,
|
|
567
|
+
so `kronk-cli -- --explain this` sends `--explain this`. With both an inline prompt and
|
|
568
|
+
piped stdin, the two are concatenated.
|
|
336
569
|
|
|
337
570
|
`Ctrl-C` aborts the in-flight response and the tool loop without killing the session.
|
|
338
571
|
|
|
@@ -381,16 +614,20 @@ The per-turn usage line still prints after each response; this one is the runnin
|
|
|
381
614
|
| `KRONK_URL` | `http://localhost:11435/v1` | Kronk API base |
|
|
382
615
|
| `KRONK_TOKEN` | `kronk` | Any non-empty value while Kronk runs open; a real JWT when protected |
|
|
383
616
|
| `KRONK_MODEL` | `unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT` | Model id |
|
|
617
|
+
| `KRONK_MODEL_CONFIG` | `~/.kronk/models/model_config.yaml` | Kronk's per-model config, the file `kronk-cli setup` writes |
|
|
384
618
|
| `KRONK_MAX_TOKENS` | `8192` | Output cap per response |
|
|
385
619
|
| `KRONK_MAX_STEPS` | unlimited | Cap on tool calls per task |
|
|
386
620
|
| `KRONK_THINKING` | `true` | `false` hides reasoning but still generates it |
|
|
387
621
|
| `KRONK_NO_THINK` | — | `1` disables reasoning server-side |
|
|
622
|
+
| `KRONK_PRESERVE_THINKING` | `true` | `false` stops pinning earlier think blocks in the prompt |
|
|
623
|
+
| `KRONK_REPLAY_REASONING` | autonomous-only | `true`/`false` overrides the default in either direction — see [What reasoning gets sent back](#what-reasoning-gets-sent-back) |
|
|
388
624
|
| `KRONK_TOOL_TIMEOUT` | `900` | Seconds before a shell command is killed |
|
|
389
625
|
| `KRONK_SANDBOX` | `auto` | `auto` confines `bash` when the OS can, `strict` refuses to run it when it cannot, `off` disables it |
|
|
390
626
|
| `KRONK_SANDBOX_ALLOW` | — | Paths to make fully available inside the sandbox, comma or colon separated |
|
|
391
627
|
| `KRONK_SANDBOX_DENY` | — | Extra paths to hide from `bash`, comma or colon separated |
|
|
392
628
|
| `KRONK_DISTILL` | `true` | `false` disables tool-output distillation |
|
|
393
629
|
| `KRONK_DISTILL_AT` | `8000` | Characters of output that trigger distillation |
|
|
630
|
+
| `KRONK_WARM` | `true` | `false` skips the boot-time model preload |
|
|
394
631
|
| `KRONK_AUTO_COMPACT` | `true` | `false` disables automatic compaction |
|
|
395
632
|
| `KRONK_COMPACT_AT` | `0.85` | Fraction of the window that triggers compaction |
|
|
396
633
|
| `NO_COLOR` | — | Any value disables colour |
|
|
@@ -409,7 +646,9 @@ The per-turn usage line still prints after each response; this one is the runnin
|
|
|
409
646
|
"showThinking": false,
|
|
410
647
|
"autoCompact": true,
|
|
411
648
|
"compactAt": 0.85,
|
|
412
|
-
"noThink": true
|
|
649
|
+
"noThink": true,
|
|
650
|
+
"preserveThinking": true,
|
|
651
|
+
"replayReasoning": true
|
|
413
652
|
}
|
|
414
653
|
```
|
|
415
654
|
|
|
@@ -453,7 +692,9 @@ memory it holds.
|
|
|
453
692
|
|
|
454
693
|
### Tip: use an `/AGENT` profile
|
|
455
694
|
|
|
456
|
-
Kronk lets one GGUF serve several runtime configurations.
|
|
695
|
+
Kronk lets one GGUF serve several runtime configurations. `kronk-cli setup` writes this profile
|
|
696
|
+
for you, backs the file up first, and offers the restart — see
|
|
697
|
+
[Subcommands](#kronk-cli-setup). To do it by hand, add this to
|
|
457
698
|
`~/.kronk/models/model_config.yaml` and restart the server:
|
|
458
699
|
|
|
459
700
|
```yaml
|
|
@@ -462,12 +703,101 @@ models:
|
|
|
462
703
|
unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT:
|
|
463
704
|
context-window: 131072
|
|
464
705
|
nseq-max: 2
|
|
706
|
+
chat-template-kwargs:
|
|
707
|
+
preserve_thinking: true
|
|
465
708
|
sampling-parameters:
|
|
466
|
-
|
|
467
|
-
top_k: 20
|
|
468
|
-
top_p: 0.95
|
|
709
|
+
max_tokens: 16384
|
|
469
710
|
```
|
|
470
711
|
|
|
712
|
+
**Do not set `temperature`, `top_k` or `top_p` here.** Most current GGUFs carry the values
|
|
713
|
+
their authors recommend, Kronk reads them, and an explicit block only overrides the model's own
|
|
714
|
+
advice. This model ships `general.sampling.temp: 1`, `top_k: 20`, `top_p: 0.95`; an earlier
|
|
715
|
+
version of this page recommended `temperature: 0.6`, which quietly fought that. Check what your
|
|
716
|
+
model carries with `kronk model show <id> --local`. If a profile does pin one of these,
|
|
717
|
+
`kronk-cli` says so at startup — see [Sampling override warning](#sampling-override-warning).
|
|
718
|
+
|
|
719
|
+
`preserve_thinking` earns its place in an agent profile. The chat template decides per assistant
|
|
720
|
+
message whether to render its `<think>` block, and the decision depends on which user message is
|
|
721
|
+
the most recent real query. During a tool loop that boundary moves, so the same earlier turn
|
|
722
|
+
renders one way now and another way after your next prompt — the prefix changes and the cached
|
|
723
|
+
prompt is thrown away. `preserve_thinking: true` renders those blocks the same way every time, so
|
|
724
|
+
the prefix stays stable and the server keeps the cache.
|
|
725
|
+
|
|
726
|
+
Measured on Kronk 1.31.8, on a 13.4k-token conversation with the profile above: the first turn prefilled in
|
|
727
|
+
10.9 s with nothing cached, and every following turn reported **13,4xx of 13,4xx tokens cached**
|
|
728
|
+
and answered in 1.6 s. *A one-off session, not reproducible from this repo as run — see
|
|
729
|
+
[Performance](#performance) for the current, scripted measurement of the same effect.*
|
|
730
|
+
|
|
731
|
+
**You no longer have to set it.** kronk-cli sends `chat_template_kwargs: {preserve_thinking: true}`
|
|
732
|
+
on every chat request, regardless of what the profile says, so an existing profile that predates
|
|
733
|
+
this line gets the stable prefix anyway. It is sent only when the selected model's chat template
|
|
734
|
+
actually declares the parameter — read from `tokenizer.chat_template` in
|
|
735
|
+
`GET /v1/kronk/models/<id>` at startup — and never guessed at when that lookup fails.
|
|
736
|
+
|
|
737
|
+
It is a trade, not a free win: the retained `<think>` blocks stay in the prompt and cost tokens.
|
|
738
|
+
On a small window you may prefer to pay the prefill instead, so `KRONK_PRESERVE_THINKING=false`
|
|
739
|
+
(or `"preserveThinking": false` in `~/.kronk-cli.json`) turns it off, and the status line says
|
|
740
|
+
`no-preserve` when it is off. With `--no-think` there is no reasoning to preserve and the field is
|
|
741
|
+
not sent at all.
|
|
742
|
+
|
|
743
|
+
### What reasoning gets sent back
|
|
744
|
+
|
|
745
|
+
`preserve_thinking` decides how the template *renders* a think block. What is in that block is a
|
|
746
|
+
separate question, and kronk-cli answers it like this: **the model's reasoning is replayed for the
|
|
747
|
+
current task and dropped for everything before it — by default in `--auto`, and never by default
|
|
748
|
+
in the interactive REPL.**
|
|
749
|
+
|
|
750
|
+
Concretely, an assistant message keeps its `reasoning_content` on the wire while it sits after the
|
|
751
|
+
most recent real user prompt. A tool result is `role: tool`, so it does not end the task — one
|
|
752
|
+
prompt and the whole tool loop it started share the boundary. The moment a new user prompt arrives,
|
|
753
|
+
the previous task's blocks are stripped and render empty. That is the same boundary the chat
|
|
754
|
+
template computes as `ns.last_query_index`.
|
|
755
|
+
|
|
756
|
+
Within a task the model reasons about tool result N before it picks tool N+1, and dropping that
|
|
757
|
+
makes it re-derive its plan from tool output alone at every step. Those tokens are also
|
|
758
|
+
append-only — new on every step, never part of the cached prefix — so replaying them costs nothing
|
|
759
|
+
in cache terms. Reasoning from *earlier* turns is the opposite: it sits in the prefix for the rest
|
|
760
|
+
of the session, grows without bound, and brings automatic compaction forward.
|
|
761
|
+
|
|
762
|
+
It is a trade, and the bill arrives at your *next* prompt: stripping the previous task's blocks at
|
|
763
|
+
the boundary rewrites the prefix that prompt sits after, so the request resets to the cached system
|
|
764
|
+
prompt instead of the cached conversation and re-prefills the rest. Measured on a live server, first
|
|
765
|
+
turn after a second prompt, prompt tokens cached vs. re-prefilled:
|
|
766
|
+
|
|
767
|
+
| | cached | re-prefilled |
|
|
768
|
+
|---|---|---|
|
|
769
|
+
| never replay | 906 | 60 |
|
|
770
|
+
| replay current task | 757 | 516 |
|
|
771
|
+
| replay everything | 1,827 | 367 |
|
|
772
|
+
|
|
773
|
+
`--auto` runs exactly one user prompt through a whole tool loop, so that boundary is never crossed
|
|
774
|
+
in a run: the within-task benefit above is free there, and the reset never happens. The REPL is a
|
|
775
|
+
user typing repeatedly, so every prompt after the first pays it. Measured over five paired sessions
|
|
776
|
+
on Kronk 1.31.9 with `unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M/AGENT` (llama.cpp `b10549`,
|
|
777
|
+
darwin/arm64/metal) with replay forced on for the whole session — the same multi-step task, nine to
|
|
778
|
+
twenty-four tool calls depending on the run, followed by the same follow-up question:
|
|
779
|
+
|
|
780
|
+
| | replay on | replay off |
|
|
781
|
+
|---|---|---|
|
|
782
|
+
| Prompt tokens at end of session | 10,179 / 11,238 / 7,960 / 10,018 / 9,470 | 20,640 / 8,889 / 10,648 / 10,259 / 13,209 |
|
|
783
|
+
| Model turns to finish the first task | 6 / 7 / 2 / 7 / 8 | 9 / 9 / 7 / 7 / 5 |
|
|
784
|
+
| Cached tokens on the first turn after the second prompt | 1,222 every run | the whole prefix every run |
|
|
785
|
+
| Time to first token on that turn | 5.9 / 7.4 / 4.4 / 5.9 / 6.5 s | 1.2 / 0.8 / 0.9 / 0.7 / 1.1 s |
|
|
786
|
+
| Wall clock, three timed pairs | 51 / 63 / 50 s | 68 / 49 / 55 s |
|
|
787
|
+
|
|
788
|
+
So: fewer turns and fewer prompt tokens over a session, paid for with one re-prefill per prompt —
|
|
789
|
+
4.4-7.4 s to first token instead of 0.7-1.2 s once cached. That cost is only worth paying when there
|
|
790
|
+
is exactly one prompt to begin with, which is why the default is autonomous-only rather than on
|
|
791
|
+
everywhere. Sampling is at the model's own `temperature: 1`, so the turn counts above are indicative
|
|
792
|
+
rather than reproducible.
|
|
793
|
+
|
|
794
|
+
`KRONK_REPLAY_REASONING`, or `"replayReasoning"` in `~/.kronk-cli.json`, overrides the default in
|
|
795
|
+
either direction rather than just turning it off: `true` replays reasoning in the REPL too, `false`
|
|
796
|
+
turns it off even in `--auto`. Leaving it unset is what gives you the autonomous-only default. It is
|
|
797
|
+
off regardless of `--auto` when `--no-think` is set — there is no reasoning to replay — and when the
|
|
798
|
+
selected model's chat template does not declare `preserve_thinking`, because such a template does
|
|
799
|
+
not read `reasoning_content` either and would discard the blocks.
|
|
800
|
+
|
|
471
801
|
---
|
|
472
802
|
|
|
473
803
|
## Context window
|
|
@@ -489,7 +819,8 @@ Three places surface it:
|
|
|
489
819
|
18471→57 tok · 69.0 tok/s · ttft 397ms · 18260 cached 18k/131k 14% ▓░░░░░░░░░
|
|
490
820
|
```
|
|
491
821
|
|
|
492
|
-
Grey under 70%, yellow past 70%, red past 90%.
|
|
822
|
+
Grey under 70%, yellow past 70%, red past 90%. *Also a real transcript — see
|
|
823
|
+
[Performance](#performance) for how to reproduce numbers like these.*
|
|
493
824
|
|
|
494
825
|
**`/context`**, on demand:
|
|
495
826
|
|
|
@@ -584,6 +915,10 @@ Disable with `--no-compact` or `KRONK_AUTO_COMPACT=false` if you would rather se
|
|
|
584
915
|
| `search` | — | Regex search via ripgrep, falling back to grep |
|
|
585
916
|
| `write_file` | ✋ | Create or overwrite; shows a diff preview first |
|
|
586
917
|
| `bash` | ✋ | Run a command; shows it first |
|
|
918
|
+
| `set_plan` | — | Record the task checklist; replaces the stored one |
|
|
919
|
+
|
|
920
|
+
`set_plan` writes nothing and runs nothing — it hands the harness a list, which is why it is
|
|
921
|
+
never gated. See [Autonomous mode](#autonomous-mode) for what the harness then does with it.
|
|
587
922
|
|
|
588
923
|
`--yes` and `--auto` skip the prompts. Paths resolve against the session directory and cannot
|
|
589
924
|
escape the launch root — including through a symlink. `bash` additionally runs under an OS
|
|
@@ -806,6 +1141,35 @@ $ kronk-cli --auto "write a CSV stats script, add a node:test, run it, fix what
|
|
|
806
1141
|
|
|
807
1142
|
`Ctrl-C` stops it. Add `--steps N` for unattended runs where nobody is watching.
|
|
808
1143
|
|
|
1144
|
+
**The task checklist.** Long tickets used to end early: the model satisfied one criterion of a
|
|
1145
|
+
dozen, wrote a confident summary, and stopped. The autonomous prompt now tells it to call
|
|
1146
|
+
`set_plan` before its first edit, with one item per acceptance criterion, and to update the list
|
|
1147
|
+
as it goes. The harness holds that list and does two things with it:
|
|
1148
|
+
|
|
1149
|
+
- **Re-states it every round.** The open items, verbatim, and how many of how many are done, are
|
|
1150
|
+
appended to the last tool result of the round that has just run — so the original request is
|
|
1151
|
+
never thousands of tokens behind. Appended, never moved: the conversation only ever grows at
|
|
1152
|
+
the end, which is what keeps Kronk's prompt cache alive across a long run. Earlier rounds keep
|
|
1153
|
+
the snapshot they were given, and each one says it is a point in time; the last is the current
|
|
1154
|
+
one. The list is held outside the message list as well, so [compaction](#compaction) cannot
|
|
1155
|
+
summarise it away.
|
|
1156
|
+
- **Declines the first premature "done".** A reply with no tool calls, while items are still
|
|
1157
|
+
open, gets the open list handed back instead of ending the run. Twice at most: after that the
|
|
1158
|
+
turn ends and the unfinished items are printed in yellow. `--steps` still wins, and `Ctrl-C`
|
|
1159
|
+
still stops everything.
|
|
1160
|
+
|
|
1161
|
+
```console
|
|
1162
|
+
1 ⚙ plan: 4 items
|
|
1163
|
+
· keep the push trigger on master
|
|
1164
|
+
▸ add the concurrency guard
|
|
1165
|
+
· document the change in README.md
|
|
1166
|
+
· run the linter
|
|
1167
|
+
```
|
|
1168
|
+
|
|
1169
|
+
None of this fires without a plan: if the model never calls `set_plan`, a run behaves exactly as
|
|
1170
|
+
it did before, and interactive (non-`--auto`) conversation is never held to a checklist. The
|
|
1171
|
+
model writes the plan; the harness only holds it to it.
|
|
1172
|
+
|
|
809
1173
|
> ⚠️ `--auto` runs shell commands without asking. Use it where `git checkout` can save you.
|
|
810
1174
|
|
|
811
1175
|
---
|
|
@@ -834,6 +1198,7 @@ previous prompt prefix — watch `cached` climb in the usage line.
|
|
|
834
1198
|
| `src/tools.js` | tool definitions, sandbox, shell session |
|
|
835
1199
|
| `src/context.js` | startup scan: git, layout, `AGENTS.md` |
|
|
836
1200
|
| `src/compact.js` | summarizing the conversation when the window fills |
|
|
1201
|
+
| `src/plan.js` | the task checklist and the snapshot the model sees |
|
|
837
1202
|
| `src/mcp.js` | MCP client: stdio + HTTP transports, tool routing |
|
|
838
1203
|
| `src/distill.js` | summarizing large tool output in a throwaway context |
|
|
839
1204
|
| `src/config.js` | precedence of flags, env, config file |
|
|
@@ -847,7 +1212,7 @@ previous prompt prefix — watch `cached` climb in the usage line.
|
|
|
847
1212
|
|---|---|
|
|
848
1213
|
| `Cannot reach Kronk` | `kronk server start --detach` |
|
|
849
1214
|
| `Kronk is running but has no models` | `kronk model pull <id>` |
|
|
850
|
-
| First response takes ~25 s | Cold model load
|
|
1215
|
+
| First response takes ~25 s | Cold model load, and you started with `--no-warm`. Drop the flag, or keep the model warm with `--pool-ttl 1h` on the server |
|
|
851
1216
|
| Long silence before text | The model is reasoning. `--no-think`, or `/thinking` to watch it |
|
|
852
1217
|
| `(model produced no answer)` | Reasoning consumed the whole budget. Raise `KRONK_MAX_TOKENS` or use `--no-think` |
|
|
853
1218
|
| `kronk-cli: command not found` after an nvm switch | Re-run `npm link`, or see [Using nvm?](#using-nvm) |
|
|
@@ -864,6 +1229,15 @@ previous prompt prefix — watch `cached` climb in the usage line.
|
|
|
864
1229
|
|
|
865
1230
|
---
|
|
866
1231
|
|
|
1232
|
+
## Contributing
|
|
1233
|
+
|
|
1234
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for setup and what gets merged quickly. Commits and PR
|
|
1235
|
+
titles follow [Conventional Commits](https://www.conventionalcommits.org/), and releases follow
|
|
1236
|
+
npm [semver](https://semver.org/) — a feature or fix PR leaves `package.json` alone, and a
|
|
1237
|
+
separate release commit bumps the version. Details are in CONTRIBUTING.md.
|
|
1238
|
+
|
|
1239
|
+
---
|
|
1240
|
+
|
|
867
1241
|
## License
|
|
868
1242
|
|
|
869
|
-
|
|
1243
|
+
Apache-2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).
|