@gobing-ai/knowledge-kit 0.0.6 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +21643 -11933
- package/package.json +4 -1
- package/plugins/generations/content-gen/package.json +2 -1
- package/plugins/generations/content-gen/src/index.ts +9 -8
- package/plugins/generations/content-gen/src/storm.ts +12 -4
- package/plugins/generations/voice-gen/package.json +17 -0
- package/plugins/generations/voice-gen/plugin.json +6 -0
- package/plugins/generations/voice-gen/src/concat.ts +218 -0
- package/plugins/generations/voice-gen/src/index.ts +213 -0
- package/plugins/generations/voice-gen/src/voicebox-client.ts +223 -0
- package/plugins/generations/voice-gen/src/voicescript.ts +365 -0
- package/plugins/generations/voice-gen/tsconfig.json +8 -0
- package/plugins/ingestions/karakeep-local/package.json +17 -0
- package/plugins/ingestions/karakeep-local/src/index.ts +31 -26
- package/plugins/ingestions/karakeep-local/tsconfig.json +4 -0
- package/plugins/ingestions/web-search/package.json +4 -1
- package/plugins/ingestions/web-search/src/index.ts +299 -17
- package/plugins/kk/README.md +12 -1
- package/plugins/kk/commands/workflow-run.md +202 -0
- package/plugins/kk/config.example.yaml +34 -0
- package/plugins/kk/scripts/render-md.ts +150 -0
- package/plugins/kk/skills/{kk-judge → content-judge}/SKILL.md +13 -14
- package/plugins/kk/skills/{kk-judge → content-judge}/references/rubrics.md +2 -2
- package/plugins/kk/skills/{kk-judge → content-judge}/references/workflow-integration.md +17 -15
- package/plugins/kk/skills/itc-generating/SKILL.md +147 -0
- package/plugins/kk/skills/itc-generating/references/generic-craft.md +80 -0
- package/plugins/kk/skills/itc-generating/references/platform-english.md +72 -0
- package/plugins/kk/skills/itc-generating/references/platform-wechat.md +60 -0
- package/plugins/kk/skills/itc-generating/references/skill-authoring.md +62 -0
- package/plugins/kk/skills/storm-research/SKILL.md +173 -0
- package/plugins/kk/skills/{kk-topic → topic}/SKILL.md +4 -4
- package/plugins/kk/workflows/judge-gated-publish-example.yaml +101 -0
- package/plugins/kk/workflows/kk-ingest-generate-publish.yaml +72 -0
- package/plugins/kk/workflows/kk-itc.yaml +285 -0
- package/plugins/kk/workflows/kk-solo-podcast.yaml +374 -0
- package/plugins/kk/workflows/kk-storm-research.yaml +184 -0
- package/plugins/kk/workflows/validate-voicescript.ts +226 -0
- package/plugins/publishings/emdash-pub/package.json +17 -0
- package/plugins/publishings/emdash-pub/plugin.json +7 -0
- package/plugins/publishings/emdash-pub/src/index.ts +450 -0
- package/plugins/publishings/emdash-pub/tsconfig.json +4 -0
- package/plugins/publishings/qiita-pub/package.json +2 -1
- package/plugins/publishings/qiita-pub/src/index.ts +9 -9
- package/plugins/publishings/surfdash-pub/package.json +2 -1
- package/plugins/publishings/surfdash-pub/src/index.ts +16 -11
- package/plugins/publishings/zenn-pub/package.json +2 -1
- package/plugins/publishings/zenn-pub/src/index.ts +11 -11
- package/plugins/kk/agents/kk-judge-compliance.md +0 -37
- package/plugins/kk/agents/kk-judge-tech.md +0 -35
- package/plugins/kk/agents/kk-judge-tone.md +0 -37
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: workflow-run
|
|
3
|
+
description: >-
|
|
4
|
+
Run STORM daily research or IT content authoring end to end. Installs the workflow
|
|
5
|
+
YAML on first run (two-root copy rule), creates the config with defaults if missing,
|
|
6
|
+
computes the workspace, and shells spur workflow run against the installed workflow.
|
|
7
|
+
argument-hint: "[name] <topic|--in file> [--dir <path>] [--playbook generic|english|wechat] [--research] [--judge] [--outline <a|b|c>] [--writer itc-generating|topic] [--duration <min>] [--language <code>] [--script-approved] [--fixture] [--force]"
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
Thin runner around the **storm-research**, **itc-generating** / **topic**, and **kk-solo-podcast**
|
|
11
|
+
capabilities. Read `plugins/kk/skills/storm-research/SKILL.md` or
|
|
12
|
+
`plugins/kk/skills/itc-generating/SKILL.md` for craft; solo-podcast craft is inlined in the
|
|
13
|
+
workflow `agent.run` prompt (no fat skill). This command owns the run procedure: config create,
|
|
14
|
+
YAML install (two-root copy rule) into `$HOME/.config/kk/workflows`, workspace computation, and
|
|
15
|
+
the `spur workflow run` of that **runtime** dest. Design-time SSOT remains
|
|
16
|
+
`plugins/kk/workflows/` in the package/repo. No new `kk` CLI noun (ADR-011) — everything here
|
|
17
|
+
is shell + the workflow.
|
|
18
|
+
|
|
19
|
+
**Args:** `$ARGUMENTS`
|
|
20
|
+
|
|
21
|
+
## 1. Parse `$ARGUMENTS`
|
|
22
|
+
|
|
23
|
+
Parse per the frozen rule:
|
|
24
|
+
|
|
25
|
+
- First token is the workflow **name** when it matches `^[a-z0-9][a-z0-9-]*$` and is not `--in`;
|
|
26
|
+
otherwise `name=kk-storm-research` and that token is the topic. Supported profile names:
|
|
27
|
+
`kk-storm-research` (default), `kk-itc`, `kk-solo-podcast`. Any other name after successful copy → exit 1 listing
|
|
28
|
+
`kk-storm-research`, `kk-itc`, `kk-solo-podcast`.
|
|
29
|
+
- `--in <file>` → file mode. `--force` → overwrite an existing dest YAML / existing `brief.md`.
|
|
30
|
+
- Profile `kk-storm-research`: `--fixture` → force `fixture: "true"`.
|
|
31
|
+
- Profile `kk-itc`: `--dir <path>` (default `./<kebab>`), `--playbook generic|english|wechat` (default `generic`),
|
|
32
|
+
`--research` (`true`|`false`, default `false`), `--judge` (`true`|`false`, default `false`),
|
|
33
|
+
`--outline <a|b|c>` (default empty), `--writer itc-generating|topic` (default `itc-generating`).
|
|
34
|
+
`--fixture` passed with `kk-itc` or `kk-solo-podcast` → exit 1 (`fixture is storm-only`).
|
|
35
|
+
- Profile `kk-solo-podcast`: `--dir <path>` (default `$works_dir/<kebab>`), `--outline <a|b|c>`
|
|
36
|
+
(default empty), `--duration <min>` (default `8`), `--language <code>` (default `en`),
|
|
37
|
+
`--script-approved` (sets `script_approved=true`; default off). `--fixture` is storm-only.
|
|
38
|
+
- **XOR:** exactly one of `<topic>` / `--in <file>`. Both set or both empty → exit 1, stderr
|
|
39
|
+
states the XOR rule, no writes.
|
|
40
|
+
|
|
41
|
+
Resolve these before step 2: `NAME`, `TOPIC`, `INPUT_FILE` (empty in sentence mode), `FIXTURE`
|
|
42
|
+
(`true`|`false`), `FORCE` (`true`|`false`), `DIR`, `PLAYBOOK`, `RESEARCH`, `JUDGE`, `OUTLINE`,
|
|
43
|
+
`WRITER`, `DURATION` (`8`), `LANGUAGE` (`en`), `SCRIPT_APPROVED` (`true`|`false`),
|
|
44
|
+
`VOICE_PROFILE` (from `VOICEBOX_DEFAULT_PROFILE`, default empty).
|
|
45
|
+
|
|
46
|
+
## 2. Resolve the config
|
|
47
|
+
|
|
48
|
+
`config_path="${KK_CONFIG:-$HOME/.config/kk/config.yaml}"`.
|
|
49
|
+
|
|
50
|
+
- **Missing file → create-on-first-run** with compiled defaults (never overwrite an existing
|
|
51
|
+
file — no merge, no rewrite):
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
install -d "$(dirname "$config_path")"
|
|
55
|
+
cat > "$config_path" <<'EOF'
|
|
56
|
+
works_dir: "$HOME/.config/kk/works"
|
|
57
|
+
workflows_dir: "$HOME/.config/kk/workflows"
|
|
58
|
+
defaults:
|
|
59
|
+
maxResults: 8
|
|
60
|
+
fixture: false
|
|
61
|
+
EOF
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
- **Unreadable / unparseable / invalid types** (e.g. `maxResults: "eight"`) → exit 1, stderr
|
|
65
|
+
names the key and expected vs actual type; touch nothing.
|
|
66
|
+
- Read the four values with precedence **command flag > env > file > compiled default**:
|
|
67
|
+
|
|
68
|
+
| Value | Env override | Compiled default |
|
|
69
|
+
| --- | --- | --- |
|
|
70
|
+
| `works_dir` | `KK_WORKS_DIR` | `$HOME/.config/kk/works` |
|
|
71
|
+
| `workflows_dir` | `KK_WORKFLOWS_DIR` | `$HOME/.config/kk/workflows` |
|
|
72
|
+
| `defaults.maxResults` | — | `8` |
|
|
73
|
+
| `defaults.fixture` | — | `false` |
|
|
74
|
+
|
|
75
|
+
`--fixture` sets `fixture=true` regardless of the file. `FIRECRAWL_API_KEY` is **never** read
|
|
76
|
+
from config — env-only.
|
|
77
|
+
|
|
78
|
+
## 3. Install the workflow YAML
|
|
79
|
+
|
|
80
|
+
`dest="$workflows_dir/$NAME.yaml"`. Copy from the **first hit** of, in order:
|
|
81
|
+
|
|
82
|
+
1. `$KK_WORKFLOWS_SOURCE/$NAME.yaml`
|
|
83
|
+
2. `plugins/kk/workflows/$NAME.yaml`
|
|
84
|
+
|
|
85
|
+
- `dest` missing → copy from the first hit. Source missing from both roots → exit 1, stderr
|
|
86
|
+
lists the roots searched: `no source workflow for $NAME (searched KK_WORKFLOWS_SOURCE, plugins/kk/workflows)`.
|
|
87
|
+
- `dest` exists + `--force` → overwrite from source.
|
|
88
|
+
- `dest` exists + identical bytes → no-op.
|
|
89
|
+
- `dest` exists + different bytes → **warn and leave** (run proceeds with the user's copy).
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
src=""
|
|
93
|
+
for d in "$KK_WORKFLOWS_SOURCE" "plugins/kk/workflows"; do
|
|
94
|
+
[ -n "$d" ] && [ -f "$d/$NAME.yaml" ] && { src="$d/$NAME.yaml"; break; }
|
|
95
|
+
done
|
|
96
|
+
[ -z "$src" ] && { echo "no source workflow for $NAME (searched KK_WORKFLOWS_SOURCE, plugins/kk/workflows)" >&2; exit 1; }
|
|
97
|
+
if [ ! -f "$dest" ] || [ "$FORCE" = true ]; then
|
|
98
|
+
install -d "$(dirname "$dest")" && cp "$src" "$dest"
|
|
99
|
+
elif ! cmp -s "$src" "$dest"; then
|
|
100
|
+
echo "warning: $dest differs from install source; leaving user copy (pass --force to replace)" >&2
|
|
101
|
+
fi
|
|
102
|
+
# Runtime dest is $workflows_dir (default ~/.config/kk/workflows). Copy YAML sidecars
|
|
103
|
+
# that the machine shells (kk-solo-podcast: validate-voicescript.ts).
|
|
104
|
+
if [ "$NAME" = "kk-solo-podcast" ] && [ -n "$src" ]; then
|
|
105
|
+
side="validate-voicescript.ts"
|
|
106
|
+
sdir=$(dirname "$src")
|
|
107
|
+
ddir=$(dirname "$dest")
|
|
108
|
+
if [ -f "$sdir/$side" ]; then
|
|
109
|
+
if [ ! -f "$ddir/$side" ] || [ "$FORCE" = true ]; then
|
|
110
|
+
cp "$sdir/$side" "$ddir/$side"
|
|
111
|
+
elif ! cmp -s "$sdir/$side" "$ddir/$side"; then
|
|
112
|
+
echo "warning: $ddir/$side differs from install source; leaving user copy (pass --force to replace)" >&2
|
|
113
|
+
fi
|
|
114
|
+
fi
|
|
115
|
+
fi
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## 4. Compute workspace
|
|
119
|
+
|
|
120
|
+
### Profile `kk-storm-research` (0056 `topicId`)
|
|
121
|
+
|
|
122
|
+
`raw` = `TOPIC` (sentence mode) or the first ATX H1 (`/^#\s+(.+)$/m`) else basename without
|
|
123
|
+
extension (`--in` mode). Then:
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
normalized=$(printf '%s' "$raw" | tr '[:upper:]' '[:lower:]' | tr -s '[:space:]' ' ' | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')
|
|
127
|
+
slug=$(printf '%s' "$normalized" | tr -cs 'a-z0-9' '-' | sed 's/^-*//; s/-*$//' | cut -c1-48 | sed 's/-*$//')
|
|
128
|
+
[ -z "$slug" ] && slug="topic"
|
|
129
|
+
hash=$(printf '%s' "$normalized" | shasum -a 256 | cut -c1-8)
|
|
130
|
+
topic_id="${slug}-${hash}"
|
|
131
|
+
work_dir="$works_dir/$topic_id"
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
- `raw` derives empty (no H1 and empty basename) → exit 1 with the derived value.
|
|
135
|
+
- Same `raw` → same workspace; **last write wins**; never fail on an existing works dir.
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
mkdir -p "$work_dir"
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
### Profile `kk-itc` (authoring workspace)
|
|
142
|
+
|
|
143
|
+
`raw` = `TOPIC` or file-mode first ATX H1 else basename without extension.
|
|
144
|
+
`kebab` = lower-case, `[^a-z0-9]+` → `-`, trimmed dashes (no length cap, no sha256 hash).
|
|
145
|
+
`work_dir="${DIR:-./$kebab}"`. Empty derived `kebab` → exit 1.
|
|
146
|
+
|
|
147
|
+
### Profile `kk-solo-podcast`
|
|
148
|
+
|
|
149
|
+
`raw` = `TOPIC` or file-mode first ATX H1 else basename without extension.
|
|
150
|
+
`kebab` = lower-case, `[^a-z0-9]+` → `-`, trimmed dashes (no length cap, no sha256 hash).
|
|
151
|
+
`work_dir="${DIR:-$works_dir/$kebab}"`. Empty derived `kebab` → exit 1.
|
|
152
|
+
|
|
153
|
+
## 5. Run the workflow
|
|
154
|
+
|
|
155
|
+
All values are strings.
|
|
156
|
+
|
|
157
|
+
### Profile `kk-storm-research`
|
|
158
|
+
|
|
159
|
+
Bind vars: `maxResults` and `fixture` from step 2; `plugins_path` defaults to `./plugins`
|
|
160
|
+
in a checkout, else installed package `plugins/` or `KK_PLUGIN_PATH`; `render_script` is
|
|
161
|
+
`plugins/kk/scripts/render-md.ts`.
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
spur workflow run "$dest" --vars \
|
|
165
|
+
"{\"topic\":\"$TOPIC\",\"input_file\":\"$INPUT_FILE\",\"maxResults\":\"$maxResults\",\"fixture\":\"$FIXTURE\",\"work_dir\":\"$work_dir\",\"plugins_path\":\"$plugins_path\",\"render_script\":\"$render_script\"}"
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Profile `kk-itc`
|
|
169
|
+
|
|
170
|
+
Bind vars: `topic`, `input_file`, `work_dir`, `writer`, `playbook`, `research`, `judge`, `outline`,
|
|
171
|
+
`force`, `rubric` (`tech-accuracy`), `verdictFile` (`.spur/run/${vars.__runId}-itc-verdict.json`), `agent`.
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
spur workflow run "$dest" --vars \
|
|
175
|
+
"{\"topic\":\"$TOPIC\",\"input_file\":\"$INPUT_FILE\",\"work_dir\":\"$work_dir\",\"writer\":\"$WRITER\",\"playbook\":\"$PLAYBOOK\",\"research\":\"$RESEARCH\",\"judge\":\"$JUDGE\",\"outline\":\"$OUTLINE\",\"force\":\"$FORCE\",\"rubric\":\"tech-accuracy\",\"verdictFile\":\".spur/run/\${vars.__runId}-itc-verdict.json\",\"agent\":\"$AGENT\"}"
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
### Profile `kk-solo-podcast`
|
|
179
|
+
|
|
180
|
+
Bind vars: `topic`, `input_file`, `work_dir`, `outline`, `script_approved`, `force`,
|
|
181
|
+
`target_duration_min`, `language`, `voice_profile` (from `VOICEBOX_DEFAULT_PROFILE` or empty),
|
|
182
|
+
`validate_script` (`$workflows_dir/validate-voicescript.ts`), `plugins_path` (empty → ADR-012
|
|
183
|
+
default discovery), `agent`. **Always** `spur workflow run "$dest"` — `$dest` is
|
|
184
|
+
`$workflows_dir/kk-solo-podcast.yaml`, never the design-time `plugins/kk/workflows/` path.
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
spur workflow run "$dest" --vars \
|
|
188
|
+
"{\"topic\":\"$TOPIC\",\"input_file\":\"$INPUT_FILE\",\"work_dir\":\"$work_dir\",\"outline\":\"$OUTLINE\",\"script_approved\":\"$SCRIPT_APPROVED\",\"force\":\"$FORCE\",\"target_duration_min\":\"$DURATION\",\"language\":\"$LANGUAGE\",\"voice_profile\":\"$VOICE_PROFILE\",\"validate_script\":\"$workflows_dir/validate-voicescript.ts\",\"plugins_path\":\"\",\"agent\":\"$AGENT\"}"
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
## 6. Report
|
|
192
|
+
|
|
193
|
+
- **`kk-storm-research`**: print resolved `work_dir`, then `$work_dir/content.md` (report sidecar)
|
|
194
|
+
and `$work_dir/content.json` (machine contract). Skim `content.md` citations against `docs.json`
|
|
195
|
+
ids before presenting.
|
|
196
|
+
- **`kk-itc`**: print resolved `work_dir`, then `$work_dir/brief.md`, `$work_dir/2-outline/outline-approved.md`,
|
|
197
|
+
and `$work_dir/3-draft/draft-article.md`. If `judge=true`, also print the verdict path.
|
|
198
|
+
- **`kk-solo-podcast`**: print resolved `work_dir`, then `$work_dir/3-script/voicescript.yaml`,
|
|
199
|
+
`$work_dir/4-audio/content.json`, and `$work_dir/4-audio/content.wav`.
|
|
200
|
+
|
|
201
|
+
If the run ended `failed`, show the failing state (`spur workflow trace <run-id> --json`) and the
|
|
202
|
+
fail-loud recovery line.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
works_dir: "~/.config/kk/works"
|
|
2
|
+
workflows_dir: "~/.config/kk/workflows"
|
|
3
|
+
defaults:
|
|
4
|
+
maxResults: 8
|
|
5
|
+
fixture: false
|
|
6
|
+
|
|
7
|
+
# Plugin credentials — paste values between the quotes.
|
|
8
|
+
# Path: plugins/<collection>/<name>/env — each entry is injected into that plugin's
|
|
9
|
+
# process environment at exec time. A real environment variable always wins
|
|
10
|
+
# over a value here (flag > env > config > compiled default).
|
|
11
|
+
plugins:
|
|
12
|
+
ingestions:
|
|
13
|
+
web-search:
|
|
14
|
+
env:
|
|
15
|
+
FIRECRAWL_API_KEY: "" # firecrawl.dev — required for live research
|
|
16
|
+
generations:
|
|
17
|
+
voice-gen:
|
|
18
|
+
env:
|
|
19
|
+
VOICEBOX_URL: "http://127.0.0.1:17493" # local Voicebox REST origin
|
|
20
|
+
VOICEBOX_DEFAULT_PROFILE: "Robin" # GET /profiles name or id — required for plain-text Docs
|
|
21
|
+
VOICEBOX_TIMEOUT_MS: "600000" # per-generation poll budget (ms)
|
|
22
|
+
VOICEBOX_POLL_MS: "1000" # GET /history/{id} interval
|
|
23
|
+
VOICEBOX_MAX_CHUNK_CHARS: "800" # Voicebox auto-chunk size (100–5000)
|
|
24
|
+
VOICEBOX_CROSSFADE_MS: "50" # Voicebox chunk crossfade (0–500)
|
|
25
|
+
publishings:
|
|
26
|
+
qiita-pub:
|
|
27
|
+
env:
|
|
28
|
+
QIITA_TOKEN: "" # Qiita API v2 token (read_qiita/write_qiita) — required
|
|
29
|
+
zenn-pub:
|
|
30
|
+
env:
|
|
31
|
+
ZENN_REPO: "" # GitHub repo for Zenn CLI deploy — used by zenn-pub
|
|
32
|
+
surfdash-pub:
|
|
33
|
+
env:
|
|
34
|
+
POSTSURFING_BIN: "" # optional — delete if unused; defaults to `postsurfing` on PATH
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Shipped no-checkout runner for the Content→markdown research-report helper.
|
|
3
|
+
* 0059's `render` state shells: `bun <this script> --in <content.json> --out <content.md>`.
|
|
4
|
+
*
|
|
5
|
+
* Self-contained by design: end users install the published `kk` package and cannot
|
|
6
|
+
* resolve `@gobing-ai/kk-core` (private). The render logic below is a COPY of
|
|
7
|
+
* `packages/kk-core/src/render-md.ts`; keep them byte-identical for the same Content.
|
|
8
|
+
*
|
|
9
|
+
* NODE-BUILTIN-ONLY — no workspace or `@gobing-ai/*` imports, and therefore raw
|
|
10
|
+
* `console.error` rather than the ts-utils output seam. This script ships to arbitrary
|
|
11
|
+
* user projects and is shelled as `bun <this script>`; an `@gobing-ai/*` import resolves
|
|
12
|
+
* there only because bun auto-installs it from the network, so it fails under
|
|
13
|
+
* `bun --no-install`, offline, or in an air-gapped environment. Enforced by
|
|
14
|
+
* `shipped-script-isolation` in `.spur/rules/boundary/`.
|
|
15
|
+
*
|
|
16
|
+
* SYNC: packages/kk-core/src/render-md.ts (renderContentMarkdown + helpers)
|
|
17
|
+
*/
|
|
18
|
+
|
|
19
|
+
import { readFile, writeFile } from 'node:fs/promises';
|
|
20
|
+
import { parseArgs } from 'node:util';
|
|
21
|
+
|
|
22
|
+
interface Ref {
|
|
23
|
+
url?: string;
|
|
24
|
+
title?: string;
|
|
25
|
+
cite?: string;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
interface ContentInput {
|
|
29
|
+
body: string;
|
|
30
|
+
title?: string;
|
|
31
|
+
outline?: string;
|
|
32
|
+
references?: Ref[];
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
/** Mirror of `ContentSchema` structural requirements for the fields this render touches. */
|
|
36
|
+
function parseContent(raw: string): ContentInput {
|
|
37
|
+
const value: unknown = JSON.parse(raw);
|
|
38
|
+
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
|
|
39
|
+
throw new Error('content must be an object');
|
|
40
|
+
}
|
|
41
|
+
const c = value as Record<string, unknown>;
|
|
42
|
+
if (typeof c.body !== 'string') {
|
|
43
|
+
throw new Error('body is required and must be a string');
|
|
44
|
+
}
|
|
45
|
+
const out: ContentInput = { body: c.body };
|
|
46
|
+
if (c.title !== undefined) {
|
|
47
|
+
if (typeof c.title !== 'string') {
|
|
48
|
+
throw new Error('title must be a string');
|
|
49
|
+
}
|
|
50
|
+
out.title = c.title;
|
|
51
|
+
}
|
|
52
|
+
if (c.outline !== undefined) {
|
|
53
|
+
if (typeof c.outline !== 'string') {
|
|
54
|
+
throw new Error('outline must be a string');
|
|
55
|
+
}
|
|
56
|
+
out.outline = c.outline;
|
|
57
|
+
}
|
|
58
|
+
if (c.references !== undefined) {
|
|
59
|
+
if (!Array.isArray(c.references)) {
|
|
60
|
+
throw new Error('references must be an array');
|
|
61
|
+
}
|
|
62
|
+
out.references = c.references.map((r, i) => {
|
|
63
|
+
if (r === null || typeof r !== 'object' || Array.isArray(r)) {
|
|
64
|
+
throw new Error(`references[${i}] must be an object`);
|
|
65
|
+
}
|
|
66
|
+
const rr = r as Record<string, unknown>;
|
|
67
|
+
for (const key of ['url', 'title', 'cite'] as const) {
|
|
68
|
+
if (rr[key] !== undefined && typeof rr[key] !== 'string') {
|
|
69
|
+
throw new Error(`references[${i}].${key} must be a string`);
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
return { url: rr.url, title: rr.title, cite: rr.cite } as Ref;
|
|
73
|
+
});
|
|
74
|
+
}
|
|
75
|
+
return out;
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
export function renderContentMarkdown(content: ContentInput): string {
|
|
79
|
+
const heading = content.title?.trim() || 'Untitled';
|
|
80
|
+
const startsWithHeading = content.body.startsWith(`# ${heading}`);
|
|
81
|
+
const lead = startsWithHeading ? '' : `# ${heading}\n\n`;
|
|
82
|
+
const refsBlock = renderReferences(content.references);
|
|
83
|
+
return `${lead}${content.body}${refsBlock}\n`;
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
function renderReferences(references: ContentInput['references']): string {
|
|
87
|
+
if (!references || references.length === 0) {
|
|
88
|
+
return '';
|
|
89
|
+
}
|
|
90
|
+
const lines: string[] = [];
|
|
91
|
+
for (const ref of references) {
|
|
92
|
+
const label = renderReferenceLabel(ref);
|
|
93
|
+
if (label === '') {
|
|
94
|
+
continue;
|
|
95
|
+
}
|
|
96
|
+
lines.push(`- ${label}`);
|
|
97
|
+
}
|
|
98
|
+
if (lines.length === 0) {
|
|
99
|
+
return '';
|
|
100
|
+
}
|
|
101
|
+
return `\n\n## References\n\n${lines.join('\n')}`;
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
function renderReferenceLabel(ref: Ref): string {
|
|
105
|
+
const { url, title, cite } = ref;
|
|
106
|
+
const titleT = title?.trim() ?? '';
|
|
107
|
+
const urlT = url?.trim() ?? '';
|
|
108
|
+
const citeT = cite?.trim() ?? '';
|
|
109
|
+
let label: string;
|
|
110
|
+
if (titleT !== '' && urlT !== '') {
|
|
111
|
+
label = `[${titleT}](${urlT})`;
|
|
112
|
+
} else if (titleT !== '') {
|
|
113
|
+
label = titleT;
|
|
114
|
+
} else if (urlT !== '') {
|
|
115
|
+
label = urlT;
|
|
116
|
+
} else if (citeT !== '') {
|
|
117
|
+
label = `\`${citeT}\``;
|
|
118
|
+
} else {
|
|
119
|
+
return '';
|
|
120
|
+
}
|
|
121
|
+
if (citeT !== '' && label !== `\`${citeT}\``) {
|
|
122
|
+
label = `${label} (${citeT})`;
|
|
123
|
+
}
|
|
124
|
+
return label;
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
async function main(): Promise<number> {
|
|
128
|
+
const { values } = parseArgs({
|
|
129
|
+
options: {
|
|
130
|
+
in: { type: 'string' },
|
|
131
|
+
out: { type: 'string' },
|
|
132
|
+
},
|
|
133
|
+
});
|
|
134
|
+
if (!values.in || !values.out) {
|
|
135
|
+
console.error('Error: Missing required arguments --in and --out');
|
|
136
|
+
return 1;
|
|
137
|
+
}
|
|
138
|
+
try {
|
|
139
|
+
const content = parseContent(await readFile(values.in, 'utf-8'));
|
|
140
|
+
await writeFile(values.out, renderContentMarkdown(content), 'utf-8');
|
|
141
|
+
return 0;
|
|
142
|
+
} catch (err: unknown) {
|
|
143
|
+
console.error(`render-md failed: ${err instanceof Error ? err.message : String(err)}`);
|
|
144
|
+
return 1;
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
|
|
148
|
+
if (import.meta.main) {
|
|
149
|
+
process.exitCode = await main();
|
|
150
|
+
}
|
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
3
|
-
description: This skill should be used when an LLM evaluation gate is needed in a knowledge-kit pipeline — "judge this content", "evaluate against rubric", "quality gate before publish", "check technical accuracy", "review brand tone", "compliance check", or when a spur workflow step of type `agent.run` targets the
|
|
2
|
+
name: content-judge
|
|
3
|
+
description: This skill should be used when an LLM evaluation gate is needed in a knowledge-kit pipeline — "judge this content", "evaluate against rubric", "quality gate before publish", "check technical accuracy", "review brand tone", "compliance check", or when a spur workflow step of type `agent.run` targets the content-judge skill to emit a PASS/FAIL/NEEDS_REVISION verdict. Centralizes evaluation rubrics, prompt templates, and structured verdict emission for generated Content.
|
|
4
4
|
version: 0.1.0
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
#
|
|
7
|
+
# content-judge — LLM-as-judge evaluation skill
|
|
8
8
|
|
|
9
9
|
## Purpose
|
|
10
10
|
|
|
11
11
|
Evaluate generated `Content` against a rubric and emit a structured **verdict** that a spur
|
|
12
12
|
workflow gate can consume. This is the fat skill — the single source of truth for evaluation
|
|
13
|
-
logic, prompt templates, and verdict formatting.
|
|
14
|
-
|
|
13
|
+
logic, prompt templates, and verdict formatting. The `agent.run` prompt (or `--judge` flag on
|
|
14
|
+
`itc-generating`) selects the rubric; there are no persona subagents.
|
|
15
15
|
|
|
16
16
|
LLM-as-judge is **not** a knowledge-kit plugin kind (ADR-007 amdt 2026-08-07). It runs as an
|
|
17
17
|
`agent.run` step in a spur workflow; the verdict is a skill output, not a plugin-kind contract.
|
|
@@ -20,7 +20,7 @@ LLM-as-judge is **not** a knowledge-kit plugin kind (ADR-007 amdt 2026-08-07). I
|
|
|
20
20
|
|
|
21
21
|
- A spur workflow needs a quality gate between `generate` and `publish`.
|
|
22
22
|
- A user asks to "judge", "evaluate", "quality-check", or "gate" generated content.
|
|
23
|
-
- A
|
|
23
|
+
- A workflow or skill names a rubric (`tech-accuracy` / `brand-tone` / `compliance` / `general`).
|
|
24
24
|
|
|
25
25
|
## Verdict output contract
|
|
26
26
|
|
|
@@ -61,8 +61,8 @@ Every evaluation writes a JSON verdict file. The default path is
|
|
|
61
61
|
|
|
62
62
|
1. **Read the Content** from the path given in the evaluation request (the `content.json` written
|
|
63
63
|
by the preceding `kk executor run <generator>` step).
|
|
64
|
-
2. **Select the rubric** —
|
|
65
|
-
`compliance`, `general`)
|
|
64
|
+
2. **Select the rubric** — named in the request (`tech-accuracy`, `brand-tone`,
|
|
65
|
+
`compliance`, `general`). Default: `general`.
|
|
66
66
|
3. **Evaluate each criterion** in the rubric. For each, produce a `pass` / `fail` / `warn` status
|
|
67
67
|
backed by a specific, quotable evidence reference (quote, section, claim).
|
|
68
68
|
4. **Score** — weighted aggregate per the rubric's scoring rules. `score` is 0.0–1.0.
|
|
@@ -107,7 +107,7 @@ Content to evaluate: {{content_path}}
|
|
|
107
107
|
Rubric: {{rubric}} # general | tech-accuracy | brand-tone | compliance
|
|
108
108
|
Output verdict to: {{verdict_path}}
|
|
109
109
|
|
|
110
|
-
Load the rubric definition from plugins/kk/skills/
|
|
110
|
+
Load the rubric definition from plugins/kk/skills/content-judge/references/rubrics.md (section: {{rubric}}).
|
|
111
111
|
Evaluate the content against every criterion in that rubric.
|
|
112
112
|
For each criterion, quote the specific passage and assign pass | fail | warn.
|
|
113
113
|
Decide the verdict: any must-pass fail → FAIL; else score < threshold → NEEDS_REVISION; else PASS.
|
|
@@ -119,13 +119,12 @@ Print the verdict path to stdout as the last line.
|
|
|
119
119
|
|
|
120
120
|
See `references/workflow-integration.md` for the `agent.run` step shape and the shell/`jq` guard
|
|
121
121
|
that reads the verdict. The example workflow is at
|
|
122
|
-
|
|
122
|
+
`plugins/kk/workflows/judge-gated-publish-example.yaml`.
|
|
123
123
|
|
|
124
|
-
##
|
|
124
|
+
## Rubric selection
|
|
125
125
|
|
|
126
|
-
The
|
|
127
|
-
|
|
128
|
-
new persona, add a wrapper that sets the rubric and reuses this skill's procedure verbatim.
|
|
126
|
+
The `agent.run` prompt (or the calling skill) sets `{{rubric}}`. To add a persona, add a rubric
|
|
127
|
+
section in `references/rubrics.md` and name it from the prompt — do not add a subagent wrapper.
|
|
129
128
|
|
|
130
129
|
## Additional resources
|
|
131
130
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Rubrics —
|
|
1
|
+
# Rubrics — judge evaluation criteria
|
|
2
2
|
|
|
3
|
-
Loaded by the
|
|
3
|
+
Loaded by the judge skill at evaluation time. Each rubric defines criteria, weights,
|
|
4
4
|
must-pass flags, and the score thresholds that map to verdicts.
|
|
5
5
|
|
|
6
6
|
## Scoring model
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Workflow integration —
|
|
1
|
+
# Workflow integration — content-judge in a spur workflow
|
|
2
2
|
|
|
3
3
|
Spur's workflow engine executes one node at a time (`type: parallel` is an inert schema field —
|
|
4
4
|
see `docs/03_ARCHITECTURE.md`). The judge step is therefore a single `agent.run` node followed
|
|
@@ -7,16 +7,18 @@ by a `shell` node that guards the transition to `publish`.
|
|
|
7
7
|
## Step shape
|
|
8
8
|
|
|
9
9
|
```yaml
|
|
10
|
-
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
10
|
+
- kind: agent.run
|
|
11
|
+
options:
|
|
12
|
+
role: reviewer
|
|
13
|
+
input: |
|
|
14
|
+
Evaluate the generated content against the {{rubric}} rubric.
|
|
15
|
+
Content: {{content_path}}
|
|
16
|
+
Write the verdict to {{verdict_path}}.
|
|
17
|
+
Load plugins/kk/skills/content-judge/SKILL.md.
|
|
18
|
+
expectFile: "{{verdict_path}}"
|
|
17
19
|
```
|
|
18
20
|
|
|
19
|
-
- `agent.run` spawns an agent that loads the `
|
|
21
|
+
- `agent.run` spawns an agent that loads the `content-judge` skill, performs the evaluation, and writes
|
|
20
22
|
the verdict JSON to `{{verdict_path}}` (typically `.spur/run/verdict-judge.json`).
|
|
21
23
|
- The verdict file is the only durable artifact. The agent's prose output is not the contract.
|
|
22
24
|
|
|
@@ -28,22 +30,22 @@ The guard runs as a `shell` step after `judge`. It reads the verdict and gates t
|
|
|
28
30
|
#!/usr/bin/env sh
|
|
29
31
|
verdict_path="{{verdict_path}}"
|
|
30
32
|
if [ ! -f "$verdict_path" ]; then
|
|
31
|
-
echo "
|
|
33
|
+
echo "judge: verdict file not found at $verdict_path"
|
|
32
34
|
exit 1
|
|
33
35
|
fi
|
|
34
36
|
verdict=$(jq -r '.verdict' "$verdict_path")
|
|
35
37
|
case "$verdict" in
|
|
36
38
|
PASS)
|
|
37
|
-
echo "
|
|
39
|
+
echo "judge: PASS"
|
|
38
40
|
exit 0
|
|
39
41
|
;;
|
|
40
42
|
FAIL|NEEDS_REVISION)
|
|
41
|
-
echo "
|
|
43
|
+
echo "judge: $verdict — see $verdict_path for findings"
|
|
42
44
|
jq -r '.findings[]? | select(.status=="fail") | " FAIL: \(.criterion): \(.detail)"' "$verdict_path"
|
|
43
45
|
exit 1
|
|
44
46
|
;;
|
|
45
47
|
*)
|
|
46
|
-
echo "
|
|
48
|
+
echo "judge: unrecognized verdict '$verdict' in $verdict_path"
|
|
47
49
|
exit 1
|
|
48
50
|
;;
|
|
49
51
|
esac
|
|
@@ -60,7 +62,7 @@ esac
|
|
|
60
62
|
|
|
61
63
|
For a bounded regenerate-retry loop, wire the workflow so that a `NEEDS_REVISION` verdict routes
|
|
62
64
|
back to the `generate` step with the verdict's `feedback` as additional input. Cap retries (the
|
|
63
|
-
example workflow uses 2). This is workflow-level orchestration — `
|
|
65
|
+
example workflow uses 2). This is workflow-level orchestration — `content-judge` itself is stateless
|
|
64
66
|
and re-runnable.
|
|
65
67
|
|
|
66
68
|
```
|
|
@@ -73,5 +75,5 @@ generate ──▶ judge ──PASS──▶ publish
|
|
|
73
75
|
|
|
74
76
|
## Workflow YAML location
|
|
75
77
|
|
|
76
|
-
The runnable example lives at
|
|
78
|
+
The runnable example lives at `plugins/kk/workflows/judge-gated-publish-example.yaml`. It demonstrates
|
|
77
79
|
`generate → judge → guard → publish` with a 2-retry NEEDS_REVISION loop.
|