happyskills 0.46.0 → 0.47.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/package.json +1 -1
- package/src/api/repos.js +4 -0
- package/src/api/telemetry.js +37 -0
- package/src/commands/agents.js +57 -23
- package/src/commands/postlex.js +402 -0
- package/src/commands/postlex.test.js +303 -0
- package/src/commands/search.js +139 -6
- package/src/commands/search.test.js +122 -0
- package/src/constants.js +2 -1
- package/src/index.js +3 -2
- package/src/utils/slug_tokens.js +68 -0
- package/src/utils/slug_tokens.test.js +126 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.47.0] - 2026-05-21
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- Add `--with-rerank` flag to `happyskills search`. When set, the CLI passes `with_rerank_digests=true` to `POST /repos:search` (via `dispatch_search()`) and wraps the `--json` response in the universal envelope shape `{ data, error, next_step }`. The `data` payload gains four new fields alongside the existing `results`: `rerank_digests`, `rerank_system_prompt`, `rerank_prompt_version`, `rerank_response_schema`, plus `formulated_query`. `next_step.action` is one of `rank_digests_inline` (digests came back — agent ranks them and pipes to `postlex`), `clarify` (semantic match_notice fired and clarification budget remains — agent uses its native question mechanism with a calibrated narrowing question), or `present_to_user` (budget exhausted / non-semantic mode — render results). Silently ignored on non-semantic dispatches (slug-shape, scoped, exact). Combining `--with-rerank --exact` exits 2 (`USAGE_ERROR`). Intended for use inside an agentic session — see the `happyskills-help@0.3.0` skill's `references/discovery-protocol.md` for the envelope contract.
|
|
14
|
+
- Add `--clarification-turns-used <N>` companion flag on `search`. Carries the clarification budget (0–2, default 0) across re-searches in the discovery protocol's clarification flow. The CLI clamps to the hard cap of 2 and emits `present_to_user` (instead of `clarify`) once the budget is spent.
|
|
15
|
+
- Add new `happyskills postlex` subcommand — deterministic post-lex finalization for the discovery protocol. Takes the LLM's `ranking[]` JSON plus the original `data[]` array (from stdin via `--ranking -`, or from separate `--ranking <file>` + `--data <file>` paths), applies the canonical 30-line slug-overlap promotion algorithm, and emits a `next_step` envelope (`present_to_user` on success, `clarify` if the post-rerank top results are still weak and budget remains, `retry_rank` with an `input_template` when the ranking fails schema validation). Stateless — the agent carries all cross-call state via `--clarification-turns-used` and `next_step.context`. Human-readable output renders only the LLM-ranked subset (the unranked tail is not shown — the LLM chose not to rank those rows). Refuses to crash on out-of-range `candidate_id` values: invalid entries are dropped with a stderr warning; if every entry drops out, exit 1.
|
|
16
|
+
- Add `cli/src/utils/slug_tokens.js` — STOP_TOKENS + `slug_tokens()` + `slug_token_set()` + `compute_lex_tier()`. Byte-identical mirror of `api/app/utils/slug_tokens.js`. The companion test (`slug_tokens.test.js`) cross-imports the API canonical version and asserts STOP_TOKENS set-equality in both directions — fails CI loud if either side drifts. This is the load-bearing invariant from spec 260521-01 v2 § 6: if the CLI's view of which candidate is "exact" disagrees with the API's slug-boost behavior, the post-lex stage promotes the wrong candidate (or none).
|
|
17
|
+
- Add `cli/src/api/telemetry.js` — fire-and-forget client for `POST /telemetry/discovery`. 2-second `AbortController` timeout; all errors swallowed; never affects exit codes. Called from `search` (events `rerank_started` when emitting `rank_digests_inline`, `clarify_triggered` when emitting `clarify`, `clarify_completed` when re-running after a clarify cycle) and from `postlex` (event `rerank_completed` after the algorithm runs, plus `clarify_triggered` when emitting a post-rerank clarify). Server-side aggregation in CloudWatch Insights drives the two key signals: postlex fire rate (0.5%–5% target) and protocol completion rate (≥85% target — `count(rerank_completed)/count(rerank_started)`).
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
- `search --json` output is wrapped in the universal envelope `{ data, error, next_step }` when (and only when) `--with-rerank` is set. Plain `search --json` (without the flag) keeps its existing `{ data: { … } }` shape for backward compatibility — no consumer of the existing JSON contract sees a change.
|
|
21
|
+
- Argument parser (`parse_args` in `cli/src/index.js`) now treats a literal `-` as a flag value instead of as the prefix of another flag. This is the standard Unix stdin sentinel and lets `happyskills postlex --ranking -` parse correctly. Conservative change: `next.startsWith('-') && next !== '-'` is the new gate, so any flag whose value would previously have been swallowed and replaced with `true` because the next arg happened to be `-` now correctly receives `'-'`. No existing command relied on the previous behavior.
|
|
22
|
+
|
|
23
|
+
## [0.46.1] - 2026-05-20
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
- Fix `happyskills agents add` ignoring two categories of skills that should be mirrored into the newly added agent's folder: **kits** (skills with `type: kit` in the lock file — previously excluded by a copy-pasted "not agent-invocable" filter that conflated *invocation routing* with *filesystem presence*) and **locally-authored skills that were never published** (skills scaffolded via `happyskills init` and kept under source control but not pushed to the registry — they exist on disk in `.agents/skills/<name>/` but have no lock entry, so the lock-based enumeration missed them). The `add` subcommand now enumerates `.agents/skills/<name>/` directories directly, so anything physically present in the canonical location — regardless of whether it's lock-managed, a kit, or a private local-only skill — is symlinked into the new agent's folder.
|
|
27
|
+
- Fix `happyskills agents remove` ignoring the same two categories — it previously read short names from the lock file, so a project containing kits or unpublished skills would leave dangling symlinks in the removed agent's folder. The subcommand now reads canonical names from `.agents/skills/<name>/` so every skill that could have been mirrored can also be unlinked.
|
|
28
|
+
- Fix `happyskills agents list` undercounting the `Linked` column for the same reason — count is now against the canonical on-disk list so kits and unpublished skills are included.
|
|
29
|
+
|
|
10
30
|
## [0.46.0] - 2026-05-20
|
|
11
31
|
|
|
12
32
|
### Added
|
package/package.json
CHANGED
package/src/api/repos.js
CHANGED
|
@@ -145,6 +145,10 @@ const dispatch_search = (query, options = {}) => catch_errors('Search failed', a
|
|
|
145
145
|
workspace_slugs: options.workspace_slug ? options.workspace_slug.split(',').map(s => s.trim()).filter(Boolean) : null,
|
|
146
146
|
scope: options.workspace_slug ? null : (options.scope || null),
|
|
147
147
|
}
|
|
148
|
+
// Spec 260521-01 v2 § 5.1 — opt-in rerank digests for the discovery
|
|
149
|
+
// protocol. Server returns digests + system prompt + json_schema only
|
|
150
|
+
// when this is true AND the dispatcher routes to mode='semantic'.
|
|
151
|
+
if (options.with_rerank_digests) body.with_rerank_digests = true
|
|
148
152
|
const [errors, data] = await client.post('/repos:search', body, { auth: true })
|
|
149
153
|
if (errors) throw errors[errors.length - 1]
|
|
150
154
|
return data
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
// Fire-and-forget telemetry client for the discovery protocol.
|
|
2
|
+
// Spec 260521-01 v2 § 5.3 — POST /telemetry/discovery accepts event-discriminated
|
|
3
|
+
// payloads. The CLI fires these on protocol transitions; the API logs them as
|
|
4
|
+
// structured CloudWatch events. Aggregation lives entirely in CloudWatch Insights.
|
|
5
|
+
//
|
|
6
|
+
// Invariants (spec § 6):
|
|
7
|
+
// - MUST NOT block the CLI's primary work. 2-second timeout, all errors swallowed.
|
|
8
|
+
// - MUST NOT affect exit codes. A network failure here is silently ignored.
|
|
9
|
+
|
|
10
|
+
const { API_URL, CLI_VERSION } = require('../constants')
|
|
11
|
+
|
|
12
|
+
const TELEMETRY_TIMEOUT_MS = 2000
|
|
13
|
+
|
|
14
|
+
// fire_discovery_telemetry — POST a beacon. Returns a promise that always
|
|
15
|
+
// resolves (never rejects) so callers can safely fire-and-forget without try/catch.
|
|
16
|
+
const fire_discovery_telemetry = (payload) => {
|
|
17
|
+
const base = process.env.HAPPYSKILLS_API_URL || API_URL
|
|
18
|
+
const body = JSON.stringify({
|
|
19
|
+
client: `happyskills-cli@${CLI_VERSION}`,
|
|
20
|
+
agent_hint: null,
|
|
21
|
+
...payload,
|
|
22
|
+
})
|
|
23
|
+
|
|
24
|
+
const controller = new AbortController()
|
|
25
|
+
const timer = setTimeout(() => controller.abort(), TELEMETRY_TIMEOUT_MS)
|
|
26
|
+
|
|
27
|
+
return fetch(`${base}/telemetry/discovery`, {
|
|
28
|
+
method: 'POST',
|
|
29
|
+
headers: { 'content-type': 'application/json' },
|
|
30
|
+
body,
|
|
31
|
+
signal: controller.signal,
|
|
32
|
+
})
|
|
33
|
+
.catch(() => {}) // swallow network errors, abort errors, etc.
|
|
34
|
+
.finally(() => clearTimeout(timer))
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
module.exports = { fire_discovery_telemetry }
|
package/src/commands/agents.js
CHANGED
|
@@ -11,7 +11,7 @@ const { find_project_root, lock_root, skills_dir, skill_install_dir, agent_skill
|
|
|
11
11
|
const { print_help, print_json, print_success, print_warn, print_info, print_table, code } = require('../ui/output')
|
|
12
12
|
const { green, dim } = require('../ui/colors')
|
|
13
13
|
const { exit_with_error, UsageError } = require('../utils/errors')
|
|
14
|
-
const { EXIT_CODES
|
|
14
|
+
const { EXIT_CODES } = require('../constants')
|
|
15
15
|
|
|
16
16
|
const HELP_TEXT = `Usage: happyskills agents <subcommand> [args] [options]
|
|
17
17
|
|
|
@@ -63,26 +63,70 @@ const _parse_agent_ids = (raw_args) => {
|
|
|
63
63
|
}
|
|
64
64
|
|
|
65
65
|
/**
|
|
66
|
-
*
|
|
67
|
-
*
|
|
68
|
-
*
|
|
66
|
+
* Enumerate every skill physically present in the canonical .agents/skills/
|
|
67
|
+
* directory for the given scope. Covers all three categories of "skill that
|
|
68
|
+
* lives in this project":
|
|
69
|
+
* 1. Lock-managed skills (installed from the registry)
|
|
70
|
+
* 2. Kits (lock-managed or not — kits have a SKILL.md without frontmatter,
|
|
71
|
+
* so a frontmatter-aware scan would miss them; we enumerate directories
|
|
72
|
+
* directly to catch them)
|
|
73
|
+
* 3. Locally-authored skills that were created via `happyskills init` but
|
|
74
|
+
* never published — they exist on disk but have no lock entry
|
|
69
75
|
*
|
|
70
|
-
*
|
|
71
|
-
|
|
76
|
+
* Returns short directory names (e.g. "deploy-aws"), not owner-qualified names.
|
|
77
|
+
*/
|
|
78
|
+
const _list_canonical_skill_names = async (is_global, project_root) => {
|
|
79
|
+
const base_dir = skills_dir(is_global, project_root)
|
|
80
|
+
try {
|
|
81
|
+
const entries = await fs.promises.readdir(base_dir, { withFileTypes: true })
|
|
82
|
+
return entries
|
|
83
|
+
.filter(e => (e.isDirectory() || e.isSymbolicLink()) && !e.name.startsWith('.'))
|
|
84
|
+
.map(e => e.name)
|
|
85
|
+
} catch {
|
|
86
|
+
return []
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* Build a short-name → display-name map from the lock file so on-disk skills
|
|
92
|
+
* that happen to be lock-managed are shown with their owner-qualified name.
|
|
93
|
+
* Skills not in the lock fall back to their short directory name.
|
|
94
|
+
*/
|
|
95
|
+
const _build_display_name_map = (lock_data) => {
|
|
96
|
+
const lock_skills = get_all_locked_skills(lock_data)
|
|
97
|
+
const map = new Map()
|
|
98
|
+
for (const full_name of Object.keys(lock_skills)) {
|
|
99
|
+
const short = full_name.split('/')[1] || full_name
|
|
100
|
+
map.set(short, full_name)
|
|
101
|
+
}
|
|
102
|
+
return map
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
/**
|
|
106
|
+
* For each skill physically present in the canonical .agents/skills/ directory,
|
|
107
|
+
* decide whether it should be mirrored into the newly added agent. A skill is
|
|
108
|
+
* mirrored when it is currently enabled — i.e., a symlink for it exists in at
|
|
109
|
+
* least one *other* already-configured agent.
|
|
110
|
+
*
|
|
111
|
+
* When no other agent folders exist yet (fresh project bootstrap), every skill
|
|
112
|
+
* present on disk is mirrored.
|
|
113
|
+
*
|
|
114
|
+
* Includes kits (which have SKILL.md without frontmatter — still need agent
|
|
115
|
+
* symlinks for filesystem parity) AND locally-authored skills that were never
|
|
116
|
+
* published (no lock entry, but legitimate project skills).
|
|
72
117
|
*/
|
|
73
118
|
const _select_skills_to_mirror = async (lock_data, is_global, project_root, new_agent) => {
|
|
74
|
-
const
|
|
75
|
-
const
|
|
119
|
+
const display_names = _build_display_name_map(lock_data)
|
|
120
|
+
const candidates = await _list_canonical_skill_names(is_global, project_root)
|
|
76
121
|
|
|
77
|
-
// Find any *other* agents that are already configured in this scope
|
|
78
122
|
const [, all_detected] = await detect_agents({ global: is_global, project_root })
|
|
79
123
|
const other_detected = (all_detected || []).filter(a => a.id !== new_agent.id)
|
|
80
124
|
|
|
81
125
|
const selected = []
|
|
82
126
|
const skipped_disabled = []
|
|
83
127
|
|
|
84
|
-
for (const
|
|
85
|
-
const
|
|
128
|
+
for (const short of candidates) {
|
|
129
|
+
const full_name = display_names.get(short) || short
|
|
86
130
|
|
|
87
131
|
if (other_detected.length === 0) {
|
|
88
132
|
selected.push({ full_name, short })
|
|
@@ -105,7 +149,6 @@ const _add = async (raw_args, args) => {
|
|
|
105
149
|
const base_dir = skills_dir(is_global, project_root)
|
|
106
150
|
|
|
107
151
|
const [, lock_data] = await read_lock(lock_root(is_global, project_root))
|
|
108
|
-
const has_lock = !!(lock_data && lock_data.skills && Object.keys(lock_data.skills).length > 0)
|
|
109
152
|
|
|
110
153
|
const per_agent = []
|
|
111
154
|
|
|
@@ -113,11 +156,6 @@ const _add = async (raw_args, args) => {
|
|
|
113
156
|
const target_root = agent_skills_dir(agent, is_global, project_root)
|
|
114
157
|
await fs.promises.mkdir(target_root, { recursive: true })
|
|
115
158
|
|
|
116
|
-
if (!has_lock) {
|
|
117
|
-
per_agent.push({ agent_id: agent.id, status: 'configured', linked: [], skipped_disabled: [] })
|
|
118
|
-
continue
|
|
119
|
-
}
|
|
120
|
-
|
|
121
159
|
const { selected, skipped_disabled } = await _select_skills_to_mirror(lock_data, is_global, project_root, agent)
|
|
122
160
|
const linked = []
|
|
123
161
|
|
|
@@ -160,9 +198,7 @@ const _remove = async (raw_args, args) => {
|
|
|
160
198
|
const project_root = find_project_root()
|
|
161
199
|
const is_json = args.flags.json || false
|
|
162
200
|
|
|
163
|
-
const
|
|
164
|
-
const all = get_all_locked_skills(lock_data)
|
|
165
|
-
const short_names = Object.keys(all).map(k => k.split('/')[1] || k)
|
|
201
|
+
const short_names = await _list_canonical_skill_names(is_global, project_root)
|
|
166
202
|
|
|
167
203
|
const per_agent = []
|
|
168
204
|
|
|
@@ -235,9 +271,7 @@ const _list = async (args) => {
|
|
|
235
271
|
const [, detected] = await detect_agents({ global: is_global, project_root })
|
|
236
272
|
const detected_ids = new Set((detected || []).map(a => a.id))
|
|
237
273
|
|
|
238
|
-
const
|
|
239
|
-
const all = get_all_locked_skills(lock_data)
|
|
240
|
-
const short_names = Object.keys(all).map(k => k.split('/')[1] || k)
|
|
274
|
+
const short_names = await _list_canonical_skill_names(is_global, project_root)
|
|
241
275
|
|
|
242
276
|
const rows = []
|
|
243
277
|
for (const agent of AGENTS) {
|
|
@@ -0,0 +1,402 @@
|
|
|
1
|
+
// happyskills postlex — apply deterministic post-lex slug-overlap promotion
|
|
2
|
+
// to an LLM-emitted ranking, then emit a `next_step` envelope telling the
|
|
3
|
+
// agent what to do next.
|
|
4
|
+
//
|
|
5
|
+
// Spec 260521-01 v2 § 5.2 — this command is the deterministic safety net
|
|
6
|
+
// for the rerank protocol. Frontier model emits ranking → postlex finalizes
|
|
7
|
+
// → agent renders. The envelope makes the protocol legible to the agent
|
|
8
|
+
// without piling conditional logic into SKILL.md.
|
|
9
|
+
|
|
10
|
+
const { error: { catch_errors, wrap_errors: e } } = require('puffy-core')
|
|
11
|
+
const fs = require('fs')
|
|
12
|
+
const { print_help, print_table, print_json } = require('../ui/output')
|
|
13
|
+
const { bold, dim, cyan, yellow, gray } = require('../ui/colors')
|
|
14
|
+
const { exit_with_error, UsageError, CliError } = require('../utils/errors')
|
|
15
|
+
const { EXIT_CODES } = require('../constants')
|
|
16
|
+
const { slug_token_set, compute_lex_tier } = require('../utils/slug_tokens')
|
|
17
|
+
const { fire_discovery_telemetry } = require('../api/telemetry')
|
|
18
|
+
|
|
19
|
+
const HELP_TEXT = `Usage: happyskills postlex --query <q> --ranking <file|-> [options]
|
|
20
|
+
|
|
21
|
+
Apply deterministic slug-overlap promotion to an LLM-emitted ranking and
|
|
22
|
+
print the final ordering. Intended for use inside the rerank protocol —
|
|
23
|
+
the LLM ranks candidates returned by \`happyskills search --with-rerank\`,
|
|
24
|
+
then pipes the result here for the deterministic finalization step.
|
|
25
|
+
|
|
26
|
+
The command emits a \`next_step\` envelope describing what the agent should
|
|
27
|
+
do next. See references/discovery-protocol.md in the happyskills-help skill
|
|
28
|
+
for the full envelope contract.
|
|
29
|
+
|
|
30
|
+
Required:
|
|
31
|
+
--query <q> The original search query (used for slug-overlap)
|
|
32
|
+
--ranking <file|-> Path to the ranking JSON, or \`-\` for stdin
|
|
33
|
+
|
|
34
|
+
Optional:
|
|
35
|
+
--data <file> Separate data file (when --ranking does not embed it)
|
|
36
|
+
--clarification-turns-used <N>
|
|
37
|
+
Clarification budget already spent (0-2, default 0)
|
|
38
|
+
--original-query <q> Original user query (opaque context from prior step)
|
|
39
|
+
--json Output as JSON (default: human-readable table)
|
|
40
|
+
|
|
41
|
+
Input shape (stdin or --ranking file):
|
|
42
|
+
{ "ranking": [{ "rank": 1, "candidate_id": 5, "rationale": "..." }, ...],
|
|
43
|
+
"data": [{ "name": "...", "workspace_slug": "...", ... }, ...] }
|
|
44
|
+
|
|
45
|
+
Examples:
|
|
46
|
+
echo '{"ranking":[...],"data":[...]}' | happyskills postlex --query "deploy aws" --ranking -
|
|
47
|
+
happyskills postlex --query "deploy aws" --ranking r.json --data d.json --json`
|
|
48
|
+
|
|
49
|
+
// ─── Pure logic — exported for unit testing ────────────────────────────────
|
|
50
|
+
|
|
51
|
+
// Validate the ranking shape. Returns { valid_items, dropped } — invalid
|
|
52
|
+
// entries are dropped with a reason rather than crashing.
|
|
53
|
+
const validate_ranking = (ranking, data) => {
|
|
54
|
+
const valid_items = []
|
|
55
|
+
const dropped = []
|
|
56
|
+
if (!Array.isArray(ranking)) return { valid_items, dropped: [{ reason: 'ranking is not an array' }] }
|
|
57
|
+
if (!Array.isArray(data)) return { valid_items, dropped: [{ reason: 'data is not an array' }] }
|
|
58
|
+
for (const item of ranking) {
|
|
59
|
+
if (!item || typeof item !== 'object') {
|
|
60
|
+
dropped.push({ item, reason: 'not an object' })
|
|
61
|
+
continue
|
|
62
|
+
}
|
|
63
|
+
const cid = item.candidate_id
|
|
64
|
+
if (!Number.isInteger(cid) || cid < 1 || cid > data.length) {
|
|
65
|
+
dropped.push({ candidate_id: cid, reason: 'candidate_id out of range' })
|
|
66
|
+
continue
|
|
67
|
+
}
|
|
68
|
+
const row = data[cid - 1]
|
|
69
|
+
if (!row || typeof row.name !== 'string' || !row.name) {
|
|
70
|
+
dropped.push({ candidate_id: cid, reason: 'data row missing name' })
|
|
71
|
+
continue
|
|
72
|
+
}
|
|
73
|
+
valid_items.push(item)
|
|
74
|
+
}
|
|
75
|
+
return { valid_items, dropped }
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
// The canonical 30-line post-lex algorithm. Pure. Returns an object capturing
|
|
79
|
+
// the reorder + telemetry signal — callers join with `data` to render.
|
|
80
|
+
const apply_postlex = (query, ranking, data) => {
|
|
81
|
+
const q = slug_token_set(query)
|
|
82
|
+
const top10 = ranking.slice(0, 10).map(item => ({
|
|
83
|
+
...item,
|
|
84
|
+
_name: data[item.candidate_id - 1].name,
|
|
85
|
+
}))
|
|
86
|
+
const exact = top10
|
|
87
|
+
.map(r => ({ r, tier: compute_lex_tier(q, slug_token_set(r._name)) }))
|
|
88
|
+
.filter(x => x.tier === 'exact')
|
|
89
|
+
.sort((a, b) => a.r.rank - b.r.rank)
|
|
90
|
+
const exact_match_count_in_window = exact.length
|
|
91
|
+
if (exact.length === 0 || exact[0].r.rank === 1) {
|
|
92
|
+
return { reordered: ranking, promoted: false, promoted_from_rank: null, exact_match_count_in_window }
|
|
93
|
+
}
|
|
94
|
+
const winner_id = exact[0].r.candidate_id
|
|
95
|
+
const winner_idx = ranking.findIndex(x => x.candidate_id === winner_id)
|
|
96
|
+
const promoted_from_rank = ranking[winner_idx]?.rank ?? null
|
|
97
|
+
const reordered = [...ranking]
|
|
98
|
+
const [moved] = reordered.splice(winner_idx, 1)
|
|
99
|
+
reordered.unshift(moved)
|
|
100
|
+
return {
|
|
101
|
+
reordered: reordered.map((x, i) => ({ ...x, rank: i + 1 })),
|
|
102
|
+
promoted: true,
|
|
103
|
+
promoted_from_rank,
|
|
104
|
+
exact_match_count_in_window,
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
// Join the reordered ranking with the data rows. Output is the user-facing
|
|
109
|
+
// `final_ordering` — only the rows the LLM included in its ranking.
|
|
110
|
+
const build_final_ordering = (reordered, data) =>
|
|
111
|
+
reordered.map(item => {
|
|
112
|
+
const row = data[item.candidate_id - 1]
|
|
113
|
+
return {
|
|
114
|
+
rank: item.rank,
|
|
115
|
+
candidate_id: item.candidate_id,
|
|
116
|
+
slug: row.workspace_slug ? `${row.workspace_slug}/${row.name}` : row.name,
|
|
117
|
+
name: row.name,
|
|
118
|
+
workspace_slug: row.workspace_slug || null,
|
|
119
|
+
description: row.description || '',
|
|
120
|
+
version: row.latest_version || row.version || '-',
|
|
121
|
+
match_quality: row.match_quality || null,
|
|
122
|
+
quality_score: row.quality_score != null ? row.quality_score : null,
|
|
123
|
+
star_count: row.star_count || 0,
|
|
124
|
+
rationale: item.rationale || '',
|
|
125
|
+
}
|
|
126
|
+
})
|
|
127
|
+
|
|
128
|
+
const STRONG_TIERS = new Set(['strong', 'good'])
|
|
129
|
+
|
|
130
|
+
// Decide which envelope to emit based on the top-3 of final_ordering and the
|
|
131
|
+
// clarification budget. Returns a `next_step` object (or null when no protocol
|
|
132
|
+
// applies).
|
|
133
|
+
const determine_next_step = (final_ordering, query, clarification_turns_used) => {
|
|
134
|
+
const top3 = final_ordering.slice(0, 3)
|
|
135
|
+
const all_strong = top3.length > 0 && top3.every(r => STRONG_TIERS.has(r.match_quality))
|
|
136
|
+
const turns_used = Math.max(0, Math.min(2, clarification_turns_used | 0))
|
|
137
|
+
const turns_remaining = 2 - turns_used
|
|
138
|
+
|
|
139
|
+
if (all_strong || final_ordering.length === 0) {
|
|
140
|
+
return {
|
|
141
|
+
action: 'present_to_user',
|
|
142
|
+
instructions: 'Render `data.final_ordering` as a table to the user, with skill name, slug, description, and rationale. Include `data.formulated_query` in your preamble.',
|
|
143
|
+
context: null,
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
if (turns_remaining <= 0) {
|
|
148
|
+
return {
|
|
149
|
+
action: 'present_to_user',
|
|
150
|
+
instructions: 'Even after reranking, no top result is a strong match — but the clarification budget (2 turns) is spent. Render `data.final_ordering` honestly: note that no result was a strong match, and present what you have so the user can decide. Do NOT ask another clarifying question.',
|
|
151
|
+
context: null,
|
|
152
|
+
}
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
return {
|
|
156
|
+
action: 'clarify',
|
|
157
|
+
instructions: `After reranking, no top result is a strong match. Ask the user one of \`suggested_questions\` using your agent's question mechanism (e.g. AskUserQuestion in Claude Code), then re-run search with the refined query and add --clarification-turns-used ${turns_used + 1}. The last option is always "Just search anyway" — honor it by re-running with the original query unchanged.`,
|
|
158
|
+
suggested_questions: [
|
|
159
|
+
{
|
|
160
|
+
question: 'After reranking I still don\'t have a strong match. Could you narrow the scope a bit?',
|
|
161
|
+
options: [
|
|
162
|
+
{ label: 'Focus on a specific stack (Node / Python / Go / etc.)', refined_query_hint: 'stack-specific' },
|
|
163
|
+
{ label: 'Focus on a specific platform (AWS / GCP / Vercel / etc.)', refined_query_hint: 'platform-specific' },
|
|
164
|
+
{ label: 'Broaden the search — show me more options', refined_query_hint: 'broader' },
|
|
165
|
+
{ label: 'Just search anyway', refined_query_hint: null },
|
|
166
|
+
],
|
|
167
|
+
},
|
|
168
|
+
],
|
|
169
|
+
max_turns_remaining: turns_remaining,
|
|
170
|
+
context: {
|
|
171
|
+
original_query: query,
|
|
172
|
+
clarification_turns_used: turns_used,
|
|
173
|
+
},
|
|
174
|
+
}
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
// Envelope used when the LLM's ranking is malformed and we need to ask it
|
|
178
|
+
// to re-emit. The CLI exits 0 — the protocol continues via the envelope.
|
|
179
|
+
const build_retry_envelope = (query, reason, clarification_turns_used, retry_count) => ({
|
|
180
|
+
data: null,
|
|
181
|
+
error: {
|
|
182
|
+
code: 'ranking_schema_mismatch',
|
|
183
|
+
message: reason,
|
|
184
|
+
exit_code: 0,
|
|
185
|
+
},
|
|
186
|
+
next_step: {
|
|
187
|
+
action: 'retry_rank',
|
|
188
|
+
instructions: 'Your ranking failed validation. Re-emit a JSON object matching the `rerank_response_schema` from the original search response, then re-pipe to `happyskills postlex --query "<q>" --ranking -`. Max one retry — if it fails again, render the search results without rerank (the API\'s relevance_score order is the fallback).',
|
|
189
|
+
context: {
|
|
190
|
+
original_query: query,
|
|
191
|
+
clarification_turns_used: clarification_turns_used | 0,
|
|
192
|
+
retry_count: (retry_count | 0) + 1,
|
|
193
|
+
},
|
|
194
|
+
},
|
|
195
|
+
})
|
|
196
|
+
|
|
197
|
+
// ─── I/O helpers ───────────────────────────────────────────────────────────
|
|
198
|
+
|
|
199
|
+
const read_stdin_sync = () => {
|
|
200
|
+
try {
|
|
201
|
+
return fs.readFileSync(0, 'utf8')
|
|
202
|
+
} catch {
|
|
203
|
+
return ''
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
const parse_input = (raw_ranking_input, raw_data_input) => {
|
|
208
|
+
// Accept either combined `{ranking, data}` from stdin/one file, or
|
|
209
|
+
// separate `{ranking: ...}`/`{data: ...}` (or bare arrays) from two files.
|
|
210
|
+
const parse_one = (raw, label) => {
|
|
211
|
+
if (raw == null || raw === '') return { value: null, parse_error: `${label} input is empty` }
|
|
212
|
+
try {
|
|
213
|
+
return { value: JSON.parse(raw), parse_error: null }
|
|
214
|
+
} catch (parse_err) {
|
|
215
|
+
return { value: null, parse_error: `${label} is not valid JSON: ${parse_err.message}` }
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
|
|
219
|
+
const ranking_parse = parse_one(raw_ranking_input, 'ranking')
|
|
220
|
+
if (ranking_parse.parse_error) return { ranking: null, data: null, parse_error: ranking_parse.parse_error }
|
|
221
|
+
|
|
222
|
+
const ranking_obj = ranking_parse.value
|
|
223
|
+
|
|
224
|
+
let ranking = null
|
|
225
|
+
let data = null
|
|
226
|
+
|
|
227
|
+
if (Array.isArray(ranking_obj)) {
|
|
228
|
+
ranking = ranking_obj
|
|
229
|
+
} else if (ranking_obj && typeof ranking_obj === 'object') {
|
|
230
|
+
ranking = Array.isArray(ranking_obj.ranking) ? ranking_obj.ranking : null
|
|
231
|
+
if (Array.isArray(ranking_obj.data)) data = ranking_obj.data
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
if (raw_data_input != null) {
|
|
235
|
+
const data_parse = parse_one(raw_data_input, 'data')
|
|
236
|
+
if (data_parse.parse_error) return { ranking, data, parse_error: data_parse.parse_error }
|
|
237
|
+
const data_obj = data_parse.value
|
|
238
|
+
if (Array.isArray(data_obj)) data = data_obj
|
|
239
|
+
else if (data_obj && Array.isArray(data_obj.data)) data = data_obj.data
|
|
240
|
+
else return { ranking, data: null, parse_error: 'data file does not contain a data array' }
|
|
241
|
+
}
|
|
242
|
+
|
|
243
|
+
if (!Array.isArray(ranking)) return { ranking: null, data, parse_error: 'ranking field is missing or not an array' }
|
|
244
|
+
if (!Array.isArray(data)) return { ranking, data: null, parse_error: 'data field is missing or not an array' }
|
|
245
|
+
return { ranking, data, parse_error: null }
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
const read_file = (path) => {
|
|
249
|
+
try {
|
|
250
|
+
return { content: fs.readFileSync(path, 'utf8'), err: null }
|
|
251
|
+
} catch (read_err) {
|
|
252
|
+
return { content: null, err: read_err.message }
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
const MATCH_QUALITY_LABELS = {
|
|
257
|
+
strong: { label: 'Strong match', color: cyan },
|
|
258
|
+
good: { label: 'Good match', color: cyan },
|
|
259
|
+
partial: { label: 'Partial match', color: yellow },
|
|
260
|
+
weak: { label: 'Weak match', color: yellow },
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
const format_human_row = (row) => {
|
|
264
|
+
const match = row.match_quality ? MATCH_QUALITY_LABELS[row.match_quality] : null
|
|
265
|
+
const match_str = match ? match.color(match.label) : ''
|
|
266
|
+
const stars_str = `★ ${row.star_count}`
|
|
267
|
+
const meta_parts = [match_str, stars_str].filter(Boolean).join(' · ')
|
|
268
|
+
|
|
269
|
+
const num = ` ${String(row.rank).padStart(2)}. `
|
|
270
|
+
const name_and_meta = `${bold(row.slug)}${meta_parts ? ` ${dim(meta_parts)}` : ''}`
|
|
271
|
+
const desc = row.description ? ` ${row.description}` : ''
|
|
272
|
+
const rationale = row.rationale ? ` ${dim('Why: ' + row.rationale)}` : ''
|
|
273
|
+
const lines = [`${num}${name_and_meta}`]
|
|
274
|
+
if (desc) lines.push(desc)
|
|
275
|
+
if (rationale) lines.push(rationale)
|
|
276
|
+
return lines.join('\n')
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
// ─── Orchestration ─────────────────────────────────────────────────────────
|
|
280
|
+
|
|
281
|
+
const run = (args) => catch_errors('Postlex failed', async () => {
|
|
282
|
+
if (args.flags._show_help) {
|
|
283
|
+
print_help(HELP_TEXT)
|
|
284
|
+
return process.exit(EXIT_CODES.SUCCESS)
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
const query = args.flags.query
|
|
288
|
+
const ranking_path = args.flags.ranking
|
|
289
|
+
const data_path = args.flags.data
|
|
290
|
+
const clarification_turns_used = parseInt(args.flags['clarification-turns-used'] || '0', 10) || 0
|
|
291
|
+
|
|
292
|
+
if (!query || typeof query !== 'string')
|
|
293
|
+
throw new UsageError('--query is required.')
|
|
294
|
+
if (!ranking_path || typeof ranking_path !== 'string')
|
|
295
|
+
throw new UsageError('--ranking is required (path to file, or `-` for stdin).')
|
|
296
|
+
|
|
297
|
+
// Read inputs
|
|
298
|
+
const raw_ranking = ranking_path === '-'
|
|
299
|
+
? read_stdin_sync()
|
|
300
|
+
: (() => {
|
|
301
|
+
const r = read_file(ranking_path)
|
|
302
|
+
if (r.err) throw new UsageError(`Cannot read --ranking file: ${r.err}`)
|
|
303
|
+
return r.content
|
|
304
|
+
})()
|
|
305
|
+
const raw_data = data_path
|
|
306
|
+
? (() => {
|
|
307
|
+
const r = read_file(data_path)
|
|
308
|
+
if (r.err) throw new UsageError(`Cannot read --data file: ${r.err}`)
|
|
309
|
+
return r.content
|
|
310
|
+
})()
|
|
311
|
+
: null
|
|
312
|
+
|
|
313
|
+
// Parse
|
|
314
|
+
const { ranking, data, parse_error } = parse_input(raw_ranking, raw_data)
|
|
315
|
+
if (parse_error) {
|
|
316
|
+
process.stderr.write(`postlex: ${parse_error}\n`)
|
|
317
|
+
const env = build_retry_envelope(query, parse_error, clarification_turns_used, 0)
|
|
318
|
+
print_json(env)
|
|
319
|
+
return process.exit(EXIT_CODES.SUCCESS)
|
|
320
|
+
}
|
|
321
|
+
|
|
322
|
+
// Validate per-item
|
|
323
|
+
const { valid_items, dropped } = validate_ranking(ranking, data)
|
|
324
|
+
for (const d of dropped) {
|
|
325
|
+
process.stderr.write(`postlex: dropping ranking entry — ${d.reason}${d.candidate_id != null ? ` (candidate_id=${d.candidate_id})` : ''}\n`)
|
|
326
|
+
}
|
|
327
|
+
if (valid_items.length === 0) {
|
|
328
|
+
throw new CliError('No candidates matched the data array — every ranking entry was dropped.', EXIT_CODES.ERROR)
|
|
329
|
+
}
|
|
330
|
+
|
|
331
|
+
// Apply algorithm
|
|
332
|
+
const { reordered, promoted, promoted_from_rank, exact_match_count_in_window } =
|
|
333
|
+
apply_postlex(query, valid_items, data)
|
|
334
|
+
const final_ordering = build_final_ordering(reordered, data)
|
|
335
|
+
|
|
336
|
+
// Decide next_step
|
|
337
|
+
const next_step = determine_next_step(final_ordering, query, clarification_turns_used)
|
|
338
|
+
|
|
339
|
+
// Fire telemetry — rerank_completed first (the rerank step succeeded);
|
|
340
|
+
// clarify_triggered separately if applicable.
|
|
341
|
+
fire_discovery_telemetry({
|
|
342
|
+
event: 'rerank_completed',
|
|
343
|
+
query,
|
|
344
|
+
promoted,
|
|
345
|
+
promoted_from_rank,
|
|
346
|
+
exact_match_count_in_window,
|
|
347
|
+
})
|
|
348
|
+
if (next_step.action === 'clarify') {
|
|
349
|
+
fire_discovery_telemetry({
|
|
350
|
+
event: 'clarify_triggered',
|
|
351
|
+
query,
|
|
352
|
+
reason: 'post_rerank_weak',
|
|
353
|
+
turn_number: clarification_turns_used + 1,
|
|
354
|
+
})
|
|
355
|
+
}
|
|
356
|
+
|
|
357
|
+
// Emit envelope
|
|
358
|
+
const envelope = {
|
|
359
|
+
data: {
|
|
360
|
+
final_ordering,
|
|
361
|
+
postlex_promoted: promoted,
|
|
362
|
+
promoted_from_rank,
|
|
363
|
+
exact_match_count_in_window,
|
|
364
|
+
formulated_query: query,
|
|
365
|
+
},
|
|
366
|
+
error: null,
|
|
367
|
+
next_step,
|
|
368
|
+
}
|
|
369
|
+
|
|
370
|
+
if (args.flags.json) {
|
|
371
|
+
print_json(envelope)
|
|
372
|
+
return process.exit(EXIT_CODES.SUCCESS)
|
|
373
|
+
}
|
|
374
|
+
|
|
375
|
+
// Human-readable output
|
|
376
|
+
console.log(`\n${bold(`Reranked results for: "${query}"`)}\n`)
|
|
377
|
+
if (promoted) {
|
|
378
|
+
console.log(` ${dim(`(post-lex promoted exact slug match from rank ${promoted_from_rank} → 1)`)}\n`)
|
|
379
|
+
}
|
|
380
|
+
final_ordering.forEach((row, i) => {
|
|
381
|
+
console.log(format_human_row(row))
|
|
382
|
+
if (i < final_ordering.length - 1) console.log('')
|
|
383
|
+
})
|
|
384
|
+
if (next_step.action === 'clarify') {
|
|
385
|
+
console.log(`\n ${yellow('No top result is a strong match. Suggested clarification:')}`)
|
|
386
|
+
console.log(` ${dim(next_step.suggested_questions[0].question)}`)
|
|
387
|
+
} else if (next_step.action === 'present_to_user' && !final_ordering.slice(0, 3).every(r => STRONG_TIERS.has(r.match_quality))) {
|
|
388
|
+
console.log(`\n ${yellow('No top result is a strong match (clarification budget spent).')}`)
|
|
389
|
+
}
|
|
390
|
+
console.log(`\n${gray(`Showing ${final_ordering.length} result${final_ordering.length === 1 ? '' : 's'}.`)}\n`)
|
|
391
|
+
}).then(([errors]) => { if (errors) { exit_with_error(errors); return } })
|
|
392
|
+
|
|
393
|
+
module.exports = {
|
|
394
|
+
run,
|
|
395
|
+
// Exported for unit testing — pure functions only.
|
|
396
|
+
validate_ranking,
|
|
397
|
+
apply_postlex,
|
|
398
|
+
build_final_ordering,
|
|
399
|
+
determine_next_step,
|
|
400
|
+
build_retry_envelope,
|
|
401
|
+
parse_input,
|
|
402
|
+
}
|