happyskills 0.47.0 → 0.48.0

package/CHANGELOG.md

@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [0.48.0] - 2026-05-22
+
+### Added
+- Add `--search-output <file>` flag to `happyskills postlex`. When set, postlex reads the full `search --with-rerank --json` response envelope from that file and extracts `data.results` internally — the calling agent's stdin payload shrinks to just `{"ranking": [...]}`. **Recommended path** for agentic callers: the agent never has to construct the `data` array by hand, which eliminates two observed failure modes in production (missing `data` field consuming the one retry budget, and `skill` vs `name` field mismatch dropping every ranking entry). Backward-compatible: the legacy `{"ranking", "data"}` stdin shape and the separate `--data <file>` flag both still work and take precedence only when `--search-output` is absent.
+
+### Fixed
+- Fix `to_smart_json` in `search.js` stripping the bare `name` field from `data.results` rows. The function emitted `skill` (the composite "workspace/name" slug) but consumed the underlying `name` field into the template literal, so downstream consumers of `data.results` — most notably `happyskills postlex` — received rows without a usable `name`. postlex's `validate_ranking` then dropped every ranking entry with `data row missing name`. The function now emits both `name` (raw, for downstream pipeline consumers) AND `skill` (composite, for human-readable display), keeping backward compatibility with anything reading `skill`.
+- Add `star_count` to `to_smart_json` output (in addition to the existing `stars` field) so `postlex`'s human-readable table renderer — which expected `row.star_count` — can find the value on rows that originated from the search response. Same root cause as the `name` issue: the search-output and postlex-input shapes had drifted apart silently.
+- Add `resolve_row_name` + `normalize_data_rows` helpers in `postlex.js` that handle the legacy case where `data.results` rows have `skill` but no `name`. `resolve_row_name` falls back to the last `/`-separated segment of `skill`; `normalize_data_rows` runs the resolution across the array. Idempotent. This is defense-in-depth — anyone passing rows from an older CLI version or a hand-crafted payload no longer trips the validator.
+
+### Changed
+- `parse_input` in `postlex.js` accepts a new optional third argument carrying the raw `--search-output` content. When provided, it's parsed and `data.results` is extracted via `extract_data_array_from_search_output`, which tolerates the canonical envelope shape (`{ data: { results: [...] } }`), the legacy `{ data: [...] }` shape, a bare array, and a defensive double-wrapped `{ data: { data: [...] } }`. When both inline `data` (from the stdin payload) and `--search-output` are provided, the search-output wins.
+- `postlex`'s help text rewritten to lead with the recommended `--search-output` recipe and demote the legacy stdin `{ranking, data}` shape to a "still supported" alternative.
+- When `data` cannot be located from any source (no inline `data`, no `--data` file, no `--search-output`), the error message now points the caller at `--search-output` explicitly so they know which input is missing.
+
+## [0.47.1] - 2026-05-22
+
+### Fixed
+- Fix `search --with-rerank` silently dropping every rerank field from the API response. The HTTP client's default `{ data: ... }` envelope unwrap was discarding `mode`, `match_notice`, `workspace_match`, and (when opted-in) `rerank_digests`, `rerank_system_prompt`, `rerank_prompt_version`, `rerank_response_schema` from `POST /repos:search`. The CLI's envelope emission saw `digests.length === 0` and fell through to "no protocol envelope" → the agent rendered the baseline ranking instead of running the LLM rerank. With this fix, `dispatch_search()` opts out of the auto-unwrap via a new `unwrap: false` option on `client.request()` and the caller now receives the full envelope. Verified end-to-end against production: natural-language query returns `next_step.action == "rank_digests_inline"` with 50 digests + the 1519-char system prompt + the strict json_schema; slug-shape query still emits `next_step: null` (no false-positive protocol fire). The bug had been latent since `0.47.0`'s introduction of `--with-rerank` because the existing rendering code in `search.js` already had a defensive fallback for both response shapes (`Array.isArray(response) ? response : response?.data`), but the rerank path requires the top-level fields specifically.
+- Plain `search --json` (without `--with-rerank`) now correctly carries `data.mode` (e.g. `'semantic'` / `'fuzzy_slug'` / `'fuzzy_scoped'`) in the JSON output. Previously `mode` was always `null` because the client auto-unwrap silently dropped it. The human-readable output also gains a `[mode]` annotation next to the query header, and the yellow `match_notice` warning ("No strong matches found…") now shows up on weak-result searches — both were already coded but never received their inputs.
+
+### Changed
+- `client.request()` (`cli/src/api/client.js`) gains an `unwrap` option (default `true`, matching the existing behavior). Set `unwrap: false` to receive the raw response body instead of the auto-unwrapped `data.data`. Required for endpoints (currently only `POST /repos:search`) that emit metadata fields at the top level of the response alongside the `data` payload. Existing callers are unaffected because the default preserves current behavior.
+
 ## [0.47.0] - 2026-05-21
 
 ### Added

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "happyskills",
-	"version": "0.47.0",
+	"version": "0.48.0",
 	"description": "Package manager for AI agent skills",
 	"license": "SEE LICENSE IN LICENSE",
 	"author": "Nicolas Dao <nic@cloudlesslabs.com> (https://cloudlesslabs.com)",

package/src/api/client.js

@@ -7,7 +7,7 @@ const { load_token } = require('../auth/token_store')
 const get_base_url = () => process.env.HAPPYSKILLS_API_URL || API_URL
 
 const request = (method, path, options = {}) => catch_errors(`API ${method} ${path} failed`, async () => {
-	const { body, auth = true, raw_response = false, headers: extra_headers = {} } = options
+	const { body, auth = true, raw_response = false, unwrap = true, headers: extra_headers = {} } = options
 	const url = `${get_base_url()}${path}`
 	const headers = { ...extra_headers }
 
@@ -64,6 +64,11 @@ const request = (method, path, options = {}) => catch_errors(`API ${method} ${pa
 		throw new ApiError(err_msg, res.status, err_code)
 	}
 
+	// Auto-unwrap `{ data: ... }` envelope by default. Callers that need the
+	// full response (e.g. endpoints that emit top-level metadata fields
+	// alongside `data` — see `dispatch_search` for `/repos:search`'s
+	// `mode` / `rerank_digests` / `match_notice` fields) pass `unwrap: false`.
+	if (!unwrap) return data
 	return data?.data !== undefined ? data.data : data
 })
 

package/src/api/repos.js

@@ -149,7 +149,12 @@ const dispatch_search = (query, options = {}) => catch_errors('Search failed', a
 	// protocol. Server returns digests + system prompt + json_schema only
 	// when this is true AND the dispatcher routes to mode='semantic'.
 	if (options.with_rerank_digests) body.with_rerank_digests = true
-	const [errors, data] = await client.post('/repos:search', body, { auth: true })
+	// `/repos:search` returns top-level metadata alongside `data` —
+	// `mode`, `match_notice`, `workspace_match`, and (when opted-in)
+	// `rerank_digests`, `rerank_system_prompt`, `rerank_prompt_version`,
+	// `rerank_response_schema`. The client's default unwrap would drop these
+	// fields entirely. Opt out so the caller sees the full envelope.
+	const [errors, data] = await client.post('/repos:search', body, { auth: true, unwrap: false })
 	if (errors) throw errors[errors.length - 1]
 	return data
 })

package/src/commands/postlex.js

@@ -32,22 +32,61 @@ Required:
   --ranking <file|->          Path to the ranking JSON, or \`-\` for stdin
 
 Optional:
-  --data <file>               Separate data file (when --ranking does not embed it)
+  --search-output <file>      Path to the full \`search --with-rerank --json\` response
+                              envelope. postlex extracts \`data.results\` from it
+                              internally, so the ranking payload only needs the
+                              ranking array. **Recommended path** for agentic
+                              callers — eliminates the join the agent would
+                              otherwise have to assemble by hand.
+  --data <file>               Legacy: separate data file (when --ranking does not
+                              embed it AND --search-output isn't used)
   --clarification-turns-used <N>
                               Clarification budget already spent (0-2, default 0)
   --original-query <q>        Original user query (opaque context from prior step)
   --json                      Output as JSON (default: human-readable table)
 
-Input shape (stdin or --ranking file):
-  { "ranking": [{ "rank": 1, "candidate_id": 5, "rationale": "..." }, ...],
-    "data":    [{ "name": "...", "workspace_slug": "...", ... }, ...] }
+Recommended shape (v0.48.0+) — agent emits ONLY the ranking, postlex does the join:
+  echo '{"ranking":[{"rank":1,"candidate_id":5,"rationale":"..."}, ...]}' | \\
+    happyskills postlex --query "deploy aws" \\
+                        --search-output /tmp/search-out.json \\
+                        --ranking -
 
-Examples:
+Legacy shape (still supported):
   echo '{"ranking":[...],"data":[...]}' | happyskills postlex --query "deploy aws" --ranking -
   happyskills postlex --query "deploy aws" --ranking r.json --data d.json --json`
 
 // ─── Pure logic — exported for unit testing ────────────────────────────────
 
+// Resolve a candidate's bare name, tolerating multiple field conventions.
+// The API row shape has `name`. The CLI's `to_smart_json` output (as of
+// happyskills@0.48.0) emits both `name` and the composite `skill`
+// ("workspace/name"). Older callers may pass through rows that only have
+// `skill`, so we fall back to extracting the bare name from the slug's last
+// `/`-separated segment. Returns null when no name can be derived.
+const resolve_row_name = (row) => {
+	if (!row || typeof row !== 'object') return null
+	if (typeof row.name === 'string' && row.name) return row.name
+	if (typeof row.skill === 'string' && row.skill) {
+		const parts = row.skill.split('/')
+		const tail = parts[parts.length - 1]
+		if (tail) return tail
+	}
+	return null
+}
+
+// Mutate `data` rows in-place to ensure they have a `name` field.
+// Idempotent. Used before validate_ranking / apply_postlex / build_final_ordering
+// so downstream code can keep its simple `row.name` access pattern.
+const normalize_data_rows = (data) => {
+	if (!Array.isArray(data)) return
+	for (const row of data) {
+		if (row && typeof row === 'object' && !row.name) {
+			const resolved = resolve_row_name(row)
+			if (resolved) row.name = resolved
+		}
+	}
+}
+
 // Validate the ranking shape. Returns { valid_items, dropped } — invalid
 // entries are dropped with a reason rather than crashing.
 const validate_ranking = (ranking, data) => {
@@ -204,9 +243,33 @@ const read_stdin_sync = () => {
 	}
 }
 
-const parse_input = (raw_ranking_input, raw_data_input) => {
-	// Accept either combined `{ranking, data}` from stdin/one file, or
-	// separate `{ranking: ...}`/`{data: ...}` (or bare arrays) from two files.
+// Extract the data array (the rerank candidate set) from a full
+// `happyskills search --with-rerank --json` response envelope. The
+// envelope's shape is `{ data: { query, mode, results: [...], ... },
+// error, next_step }` — we want `envelope.data.results`. Also tolerates a
+// bare array, a `{data: [...]}` shape (legacy callers), and the
+// `{data: {data: [...]}}` shape that appears if someone double-wraps.
+// Returns the array, or null when no array can be located.
+const extract_data_array_from_search_output = (parsed) => {
+	if (Array.isArray(parsed)) return parsed
+	if (!parsed || typeof parsed !== 'object') return null
+	const inner = parsed.data
+	if (Array.isArray(inner)) return inner
+	if (inner && typeof inner === 'object') {
+		if (Array.isArray(inner.results)) return inner.results
+		if (Array.isArray(inner.data)) return inner.data
+	}
+	if (Array.isArray(parsed.results)) return parsed.results
+	return null
+}
+
+const parse_input = (raw_ranking_input, raw_data_input, raw_search_output_input) => {
+	// Accept either:
+	//   - combined `{ranking, data}` from stdin/one file (legacy v0.47.x shape), OR
+	//   - separate ranking + data files (`--ranking <file>` + `--data <file>`), OR
+	//   - just `{ranking: ...}` or a bare ranking array PLUS a `--search-output` file
+	//     containing the full search envelope from which we extract `data.results`
+	//     (v0.48.0+ recommended shape — agent never has to construct `data`).
 	const parse_one = (raw, label) => {
 		if (raw == null || raw === '') return { value: null, parse_error: `${label} input is empty` }
 		try {
@@ -240,8 +303,25 @@ const parse_input = (raw_ranking_input, raw_data_input) => {
 		else return { ranking, data: null, parse_error: 'data file does not contain a data array' }
 	}
 
+	if (raw_search_output_input != null) {
+		const so_parse = parse_one(raw_search_output_input, 'search-output')
+		if (so_parse.parse_error) return { ranking, data, parse_error: so_parse.parse_error }
+		const extracted = extract_data_array_from_search_output(so_parse.value)
+		if (!Array.isArray(extracted)) {
+			return { ranking, data: null, parse_error: 'search-output does not contain a data.results array' }
+		}
+		// search-output is the recommended source — it overrides any inline data.
+		data = extracted
+	}
+
 	if (!Array.isArray(ranking)) return { ranking: null, data, parse_error: 'ranking field is missing or not an array' }
-	if (!Array.isArray(data))    return { ranking, data: null, parse_error: 'data field is missing or not an array' }
+	if (!Array.isArray(data))    return { ranking, data: null, parse_error: 'data field is missing or not an array — provide --search-output <file> with the search response, or include "data" in the stdin payload' }
+
+	// Normalize row names (handles the `skill`-without-`name` case from older
+	// CLI versions or hand-crafted payloads). Idempotent on rows that already
+	// have a `name` field.
+	normalize_data_rows(data)
+
 	return { ranking, data, parse_error: null }
 }
 
@@ -287,6 +367,7 @@ const run = (args) => catch_errors('Postlex failed', async () => {
 	const query                       = args.flags.query
 	const ranking_path                = args.flags.ranking
 	const data_path                   = args.flags.data
+	const search_output_path          = args.flags['search-output']
 	const clarification_turns_used    = parseInt(args.flags['clarification-turns-used'] || '0', 10) || 0
 
 	if (!query || typeof query !== 'string')
@@ -309,9 +390,16 @@ const run = (args) => catch_errors('Postlex failed', async () => {
 			return r.content
 		})()
 		: null
+	const raw_search_output = search_output_path
+		? (() => {
+			const r = read_file(search_output_path)
+			if (r.err) throw new UsageError(`Cannot read --search-output file: ${r.err}`)
+			return r.content
+		})()
+		: null
 
 	// Parse
-	const { ranking, data, parse_error } = parse_input(raw_ranking, raw_data)
+	const { ranking, data, parse_error } = parse_input(raw_ranking, raw_data, raw_search_output)
 	if (parse_error) {
 		process.stderr.write(`postlex: ${parse_error}\n`)
 		const env = build_retry_envelope(query, parse_error, clarification_turns_used, 0)
@@ -399,4 +487,7 @@ module.exports = {
 	determine_next_step,
 	build_retry_envelope,
 	parse_input,
+	resolve_row_name,
+	normalize_data_rows,
+	extract_data_array_from_search_output,
 }

package/src/commands/postlex.test.js

@@ -16,6 +16,9 @@ const {
 	determine_next_step,
 	build_retry_envelope,
 	parse_input,
+	resolve_row_name,
+	normalize_data_rows,
+	extract_data_array_from_search_output,
 } = require('./postlex')
 
 // ─── Test fixtures ────────────────────────────────────────────────────────
@@ -283,6 +286,144 @@ describe('parse_input', () => {
 
 // ─── build_final_ordering ─────────────────────────────────────────────────
 
+// ─── resolve_row_name + normalize_data_rows (v0.48.0) ─────────────────────
+
+describe('resolve_row_name', () => {
+	it('returns row.name when present', () => {
+		assert.equal(resolve_row_name({ name: 'foo', skill: 'acme/foo' }), 'foo')
+	})
+
+	it('falls back to last segment of skill when name is missing', () => {
+		assert.equal(resolve_row_name({ skill: 'acme/deploy-aws' }), 'deploy-aws')
+	})
+
+	it('returns null when neither name nor skill is present', () => {
+		assert.equal(resolve_row_name({ description: 'whatever' }), null)
+		assert.equal(resolve_row_name({}), null)
+		assert.equal(resolve_row_name(null), null)
+	})
+
+	it('returns null when skill is malformed (no slash, just a bare value, returns the value itself)', () => {
+		// "deploy-aws" with no slash is a one-segment slug — last segment IS deploy-aws.
+		assert.equal(resolve_row_name({ skill: 'deploy-aws' }), 'deploy-aws')
+	})
+
+	it('handles empty-string name by falling back to skill', () => {
+		assert.equal(resolve_row_name({ name: '', skill: 'acme/foo' }), 'foo')
+	})
+})
+
+describe('normalize_data_rows', () => {
+	it('adds name to rows that only have skill', () => {
+		const data = [{ skill: 'acme/deploy-aws', workspace_slug: 'acme' }]
+		normalize_data_rows(data)
+		assert.equal(data[0].name, 'deploy-aws')
+	})
+
+	it('leaves rows with existing name untouched', () => {
+		const data = [{ name: 'pre-existing', skill: 'acme/different' }]
+		normalize_data_rows(data)
+		assert.equal(data[0].name, 'pre-existing')
+	})
+
+	it('is idempotent', () => {
+		const data = [{ skill: 'acme/deploy-aws' }]
+		normalize_data_rows(data)
+		normalize_data_rows(data)
+		assert.equal(data[0].name, 'deploy-aws')
+	})
+
+	it('handles non-array input without crashing', () => {
+		normalize_data_rows(null)
+		normalize_data_rows('not an array')
+		normalize_data_rows({})
+		// no assertion — just confirming no throw
+	})
+})
+
+// ─── extract_data_array_from_search_output (v0.48.0) ──────────────────────
+
+describe('extract_data_array_from_search_output', () => {
+	it('extracts data.results from the canonical envelope shape', () => {
+		const env = { data: { query: 'q', mode: 'semantic', results: [{ name: 'foo' }] }, error: null, next_step: null }
+		const r = extract_data_array_from_search_output(env)
+		assert.equal(r.length, 1)
+		assert.equal(r[0].name, 'foo')
+	})
+
+	it('handles legacy {data: [...]} (data is bare array)', () => {
+		const env = { data: [{ name: 'foo' }] }
+		assert.deepEqual(extract_data_array_from_search_output(env), [{ name: 'foo' }])
+	})
+
+	it('handles bare array', () => {
+		const env = [{ name: 'foo' }]
+		assert.deepEqual(extract_data_array_from_search_output(env), env)
+	})
+
+	it('handles double-wrapped {data: {data: [...]}} (defensive)', () => {
+		const env = { data: { data: [{ name: 'foo' }] } }
+		assert.deepEqual(extract_data_array_from_search_output(env), [{ name: 'foo' }])
+	})
+
+	it('returns null when no data array can be found', () => {
+		assert.equal(extract_data_array_from_search_output({}), null)
+		assert.equal(extract_data_array_from_search_output({ data: null }), null)
+		assert.equal(extract_data_array_from_search_output('string'), null)
+		assert.equal(extract_data_array_from_search_output(null), null)
+	})
+})
+
+// ─── parse_input with --search-output path (v0.48.0) ──────────────────────
+
+describe('parse_input with --search-output', () => {
+	it('extracts data.results from a full search envelope passed as the third argument', () => {
+		const ranking = JSON.stringify({ ranking: [{ rank: 1, candidate_id: 1, rationale: 'top' }] })
+		const search_out = JSON.stringify({
+			data: { query: 'q', mode: 'semantic', results: [{ name: 'deploy-aws', workspace_slug: 'acme' }] },
+			error: null,
+			next_step: null,
+		})
+		const r = parse_input(ranking, null, search_out)
+		assert.equal(r.parse_error, null)
+		assert.equal(r.ranking[0].candidate_id, 1)
+		assert.equal(r.data[0].name, 'deploy-aws')
+	})
+
+	it('search-output overrides inline data when both are provided', () => {
+		const combined = JSON.stringify({
+			ranking: [{ rank: 1, candidate_id: 1, rationale: 'x' }],
+			data: [{ name: 'stale-inline-name' }],
+		})
+		const search_out = JSON.stringify({ data: { results: [{ name: 'fresh-from-search-output' }] } })
+		const r = parse_input(combined, null, search_out)
+		assert.equal(r.parse_error, null)
+		assert.equal(r.data[0].name, 'fresh-from-search-output')
+	})
+
+	it('normalizes rows that only have skill (no name) when sourced from search-output', () => {
+		const ranking = JSON.stringify({ ranking: [{ rank: 1, candidate_id: 1, rationale: 'x' }] })
+		// to_smart_json before v0.48.0 emitted `skill` without `name`. Simulate that.
+		const search_out = JSON.stringify({ data: { results: [{ skill: 'acme/legacy-row', workspace_slug: 'acme' }] } })
+		const r = parse_input(ranking, null, search_out)
+		assert.equal(r.parse_error, null)
+		assert.equal(r.data[0].name, 'legacy-row')
+	})
+
+	it('errors when search-output does not contain a data array', () => {
+		const ranking = JSON.stringify({ ranking: [{ rank: 1, candidate_id: 1, rationale: 'x' }] })
+		const search_out = JSON.stringify({ data: { query: 'q' } })
+		const r = parse_input(ranking, null, search_out)
+		assert.match(r.parse_error, /does not contain a data\.results array/)
+	})
+
+	it('errors with an actionable message when data is missing from all sources', () => {
+		const ranking = JSON.stringify({ ranking: [{ rank: 1, candidate_id: 1, rationale: 'x' }] })
+		const r = parse_input(ranking, null, null)
+		assert.match(r.parse_error, /--search-output/)
+	})
+})
+
 describe('build_final_ordering', () => {
 	it('joins ranking with data rows, producing slug + rationale', () => {
 		const data = make_data(['deploy-aws', 'serverless'])

package/src/commands/search.js

@@ -103,12 +103,14 @@ const format_smart_result = (item, index) => {
 
 const to_smart_json = (item) => ({
 	skill: `${item.workspace_slug}/${item.name}`,
+	name: item.name,
 	type: item.type || 'skill',
 	description: item.description || '',
 	version: item.latest_version || item.version || '-',
 	visibility: item.visibility || 'public',
 	workspace_slug: item.workspace_slug,
 	stars: item.star_count || 0,
+	star_count: item.star_count || 0,
 	quality_score: item.quality_score != null ? item.quality_score : null,
 	quality_tier: get_quality_tier_name(item.quality_score),
 	relevance_score: item.relevance_score != null ? item.relevance_score : null,