@ataraxy-labs/sem 0.15.1 → 0.16.0

package/CHANGELOG.md

@@ -2,6 +2,89 @@
 
 All notable changes to sem are documented in this file.
 
+## [Unreleased]
+
+### Performance
+
+- **Token-efficient tool output**: the same answers at a fraction of the tokens the consuming model has to read (and pay for). (1) The context packer stops enumerating noise: related test entities are folded into per-role counts instead of packed as one-line "#[test]" stubs (unless the target itself is a test, when its test neighborhood is the question), bare attribute/comment signatures are skipped, and transitive tiers are capped at 25 entries per role with the remainder counted. What was dropped is stated explicitly in one line ("not packed: +64 direct dependents (64 tests) · sem_impact lists them"), so the signal survives at a fraction of the cost. Measured: `sem context` on a hot sem-core entity 7,272 to 4,113 tokens (-43%, 119 to 48 entries); on a hot weave entity 5,216 to 3,498 (-33%, 146 to 27 entries, the 119 test stubs now one line). (2) The `sem_entities` MCP tool renders compact per-line trees (name · type · lines, children indented, files as group headers) instead of pretty-printed JSON: measured 3.7x fewer tokens on a 115-entity file (5,028 to 1,361). Applies to the MCP path and query modes; CLI `--json` output is unchanged for scripts.
+
+- **Semantic commit index (storage engine layer 2)**: history is now stored as entity deltas. Each commit is semantic-diffed against its first parent exactly once and persisted as entity-change rows in the cache (`commits` + `entity_changes` tables, sha-keyed and branch-agnostic); every later history query is a SQLite lookup plus a diff of only the commits git gained since. Measured on the sem repo: `sem log` over 500 commits drops from 4.46s to **0.03s** on the second query (~150x), with the first query as the one-time indexing pass and each new commit costing one incremental diff. Applies to `sem log` repo analytics (hotspots + co-change pairs) and the MCP `sem_log` tool; per-entity traces are unchanged. Aggregation is shared code between the live git walk and the store (`aggregate_history_analytics`), tested to produce identical hotspot/co-change output, so the two paths cannot drift. Merge-heavy history gets better semantics: each commit is attributed its own first-parent diff instead of a diff against its arbitrary revwalk neighbor, and merge commits contribute no changes of their own (their first-parent diff restates the merged-in commits, which index individually). File-filtered queries index full-repo diffs once and filter at aggregation, so the first filtered query costs more than the old pathspec-scoped walk but every later query on any filter is instant. Cache schema v9; existing caches rebuild automatically on first use.
+
+- **CLI sidecar fast path**: `sem impact` now answers from the resident `sem mcp` server's warm graph via its unix socket before doing any local work — measured **4.5ms** end-to-end on a 158K-LOC repo, versus 22.4ms for the local cold path and 7.7ms for a ripgrep scan of the same repo: the full blast radius (callers, dependencies, depth-bounded transitive impact, affected tests) is now cheaper than a raw text match. Output is byte-identical to the local path (the sidecar ships serialized `EntityInfo`s that the CLI feeds to its existing printers; verified across all modes and `--json`). The fast path is an accelerator, never a requirement: bounded socket timeouts and silent fallback mean no resident server (or `SEM_NO_SIDECAR=1`, `--no-cache`, custom scopes, `.semignore`, `--entity-id`) just runs the normal local path. Server-side, the new sidecar `impact` op classifies affected tests only among the entities the impact BFS actually reached, instead of walking the whole corpus per call (6.8ms → 0.1ms on a 4.7K-entity graph).
+
+### Fixed
+
+- The context packer's "not packed" summary line pluralizes roles correctly ("transitive dependencies", not "dependencys").
+
+- Workspace version bumped to 0.16.0: `ContextResult` gained the public `omitted` field (a breaking change for struct-literal constructors, flagged by cargo-semver-checks), and 0.x semantics put breaking changes in the minor version.
+
+- The docs site deploys through workflow-based GitHub Pages (`.github/workflows/docs-pages.yml`: upload `/docs` verbatim, deploy) instead of the legacy branch-based Jekyll builder, which began failing repo-wide with zero-duration "Page build failed" errors on commits that didn't touch docs — including on direct build requests via the Pages API. The site is pure static HTML, so the legacy builder added nothing but a failure mode; deploys now also skip entirely on commits that don't change `docs/`.
+
+### Changed
+
+- The GitHub Action's PR-comment footer now tells the reader what to do next — "add it to your repo in 2 minutes", linking to the action's install snippet — instead of only naming the tool. Every entity-diff comment is seen by all of a repo's collaborators; the footer is the loop that turns viewers into installs.
+
+### Added
+
+- **`sem repos`** — where your code is stored, in one command. Two inventories side by side: the **cloud account** (authoritative `GET /v1/repos`: every indexed repo with status, entity/file counts, last-indexed time, indexed commit, and any indexing error rendered inline) and **local storage** (every entity cache under the sem cache root with size on disk, entity count, cache kind, and the repo it was built from). `--json` for scripts. Listing the account also reconciles this machine's `~/.sem/repos.json` mirror with server truth — stale entries (a repo registered mid-index stays "pending, 0 entities" forever otherwise) were silently mis-routing the local-vs-cloud decision for impact/context queries. Caches are now stamped with their repo root at save time (`repo_root` in `cache_metadata`); caches built before this show as unlabeled and self-label on their next rebuild.
+
+- Fish shell support, via the `tree-sitter-fish` grammar (gated behind the `lang-fish` feature in `grammar-all`). Extracts functions — including the config.fish pattern of definitions inside a top-level `if status is-interactive` block — and resolves fish call edges (a `command`'s name against repo functions, builtins excluded), so `sem impact` sees which fish functions call which. A `function` defined inside another function stays part of the outer entity's content, matching fish's runtime semantics (inner definitions become global, not lexical children). Previously `.fish` files fell back to generic line-based chunking with an unsupported-language warning. Thanks @thalys for the request (#433).
+
+### Documentation
+
+- **First-principles page on the docs site** (`docs/first-principles.html`, linked from every page's nav): four charts explaining why the recent latency work changes what an agent can afford to do, not just how fast it runs — the scan-vs-index crossover (a text scan pays per byte; residency removes the index's ~800ms hydrate floor, so the constant-time line wins at every repo size), the ~100ms human-perception threshold every new path now sits under, model turns as the real cost unit (3 → 2 → 1 inference turns per structural answer via one-call lookup, then prompt-time prefetch), and tokens per answer (the measured ~15% entity-tree-vs-JSON ratio). Measured numbers come from this changelog; model curves and turn timings are labeled illustrative on the page. Charts are dependency-free inline SVG with hover tooltips and a table view each.
+
+### Performance
+
+- **Content-store cache (storage engine layer 1)**: the entity cache no longer duplicates source text per entity. Each file's text is stored once (zstd) in a `file_contents` table, and any entity whose body is provably a byte slice of it (`content == file[start_byte..end_byte]`, verified at save time) stores NULL content and is re-sliced on load; unprovable entities (no spans, normalized endings) keep content inline. On a 139K-entity corpus (fresh cache both sides) this cut the cache 20% (269MB to 216MB; the content layer itself −58%, 80MB to 24MB inline + 10MB zstd), engaged for 77% of entities. Honest costs: warm full-content loads pay ~0.13s extra for decompress+slice on that corpus (0.38s to 0.51s); topology loads and the MCP server's in-memory hot path are unaffected, and cold build time is unchanged within noise (peak RSS ~−5%). Correctness gates: byte-identical graph vs the previous binary on the full corpus, byte-identical entity content round-trip (including multi-byte unicode and nested entities), and incremental saves keep the file store in sync with entity deletes. Cache schema v8 — existing caches rebuild automatically on first use.
+
+### Added
+
+- **Entity-addressed text search**: `sem_entities` takes a `text` parameter — an exact substring searched across entity bodies in the warm in-memory graph (no file reads). Hits come back addressed by the innermost enclosing entity (`file: entity (Lline): matched text`), ready to chain into `sem_context`/`sem_impact`, in ~20-30ms warm on an 85K-LOC repo. This retires the main remaining reason agents fell back to grep (strings, error messages, config keys); misses say honestly that comments between entities and non-code files are not covered.
+
+### Performance
+
+- Graph build: the scope resolver no longer allocates its debug resolution log (several owned strings per reference, discarded by every production path — only a bench consumed it), and edge dedup is index-based instead of cloning both entity IDs per edge into a hash set. Output is byte-identical (proven edge-for-edge on a 139K-entity build); ~1-3% fewer instructions retired. Groundwork toward #320/#322 — the remaining peak-memory work (entity content sharing, ID interning) is tracked there.
+
+### Added
+
+- **`sem hook prompt-submit`** (hidden plumbing): the prompt-time prefetch, compiled. Reads a Claude Code UserPromptSubmit event, extracts identifier-shaped tokens from the prompt (backticked, snake_case, CamelCase, qualified — never plain words), resolves them against the resident server's socket sidecar, and prints packed entity context for injection. **10ms end-to-end** (was ~40ms as a Python hook — interpreter startup and a git subprocess, both eliminated: repo root is found by walking to `.git` in-process). Silent on conversational prompts, slash commands, unknown names, or when no server is resident.
+
+### Added
+
+- The socket sidecar is unix-only (`cfg(unix)`): Windows builds skip it with a no-op and the prefetch hook falls back silently — the sidecar is an accelerator, never a requirement. (Fixes the Windows build break the sidecar introduced.)
+- **Socket sidecar**: `sem mcp` now exposes the warm in-memory graph on a per-repo unix socket (`~/.sem/sock/<repo-hash>.sock`, one JSON line in, one out). Short-lived local callers — the prompt-prefetch hook, future CLI fast paths — get one-call entity context in single-digit milliseconds instead of paying a fresh process plus SQLite hydrate (~800ms). Stale sockets from dead servers are detected and taken over; the sidecar is a silent accelerator, never a requirement.
+
+### Added
+
+- **One-call lookup**: `sem_context`'s `file_path` is now optional. With only an `entity_name`, the entity is resolved across the whole repo (unique match proceeds; ambiguity returns a compact candidate list with the files; no match returns near-name suggestions) and the body plus callers/callees comes back in a single round-trip — one agent call where grep needs two (search, then read). Measured 26ms wall on a prewarmed server, name-only, unfamiliar repo.
+
+### Performance
+
+- The sem MCP server is now **local-first and prewarmed**. Cloud-first routing on `sem_impact`/`sem_context` cost a network round-trip on every call before the local answer (and carried the same wrong-entity risk gated in the CLI); it is now behind `SEM_MCP_CLOUD=1` until the server resolves name+file strictly. The server also builds the CWD repo's graph in the background at startup, so the agent's first structural query answers from memory. Measured on an 85K-LOC repo: warm `sem_context` runs in under 1ms wall (faster than a ripgrep scan of the same repo), and the first call dropped from 129ms cold to ~0 with prewarm.
+
+### Removed
+
+- Team presence was pulled from the `--badge` package before it shipped as a feature (product call: not a feature for now). The statusline no longer shows teammates and the hook sends nothing anywhere; the dormant server endpoints remain unadvertised.
+
+### Added
+
+- The `--badge` statusline is now **live at trigger time**: a PreToolUse hook flips the badge to an animated spinner with the entity name the moment the agent calls sem (`⊕ sem ⠹ impact validateToken…`), and the completed state (count, latency, savings) lands when the call finishes. The render hot path never touches the network (renders measured at ~20ms).
+
+### Fixed
+
+- The `sem context` / `sem_context` budget packer no longer starves the target while neighbors feast. Previously a target too big for the budget collapsed to its first line (2 tokens) while a single large dependency could consume the entire budget with its full body. The target now degrades gracefully — full body → head-truncated body (docstring, fields, leading code, with an explicit `… truncated: N more lines` marker) using up to ~70% of the budget → bare signature — and no neighbor may cost more tokens than the target itself did (budget/10 floor), oversized neighbors degrading to signatures. On the same query (a large class, budget 2000) the target went from 2 tokens to 1,398 and the answer-relevant attributes are now in the payload.
+
+### Added
+
+- **Entity-level history analytics**: `sem log` with no entity now analyzes recent repo history in one pass and reports **hotspots** (the most-changed code entities, with commit counts, distinct authors, and the last commit that touched each) and **co-change pairs** (entities that repeatedly change in the same commits, with a confidence score — "these two never change apart"). Counts are per commit, code entities only (doc headings, config properties, and lockfile chunks are excluded so the signal is about code), and bulk commits touching >50 entities are excluded from pair-counting to keep quadratic noise out. Same via MCP: `sem_log` without `entity_name`. `--file` scopes to one file; `--json` returns everything. This is the time axis a snapshot dependency graph cannot see: which code churns, and which code moves together.
+
+### Changed
+
+- `sem_impact` MCP results now render as a **blast-radius tree** (`◉` header, one `├─▶` branch per file, real callers first, all-test files sunk to the bottom, nothing elided) — expanding the tool widget is the live graph, no separate viewer process needed. The bundled skill also instructs agents to draw the blast radius as a small ASCII tree directly in their reply when an impact result drives the answer.
+
+- `sem_impact` and `sem_context` MCP results now render as a compact entity tree instead of pretty-printed JSON: dependents/dependencies/transitive impact grouped one line per file, every entity name preserved, with the elapsed time and source in a footer. The same information lands in about 15% of the tokens, and the expanded tool widget in agent UIs reads at a glance (`⊕ entity · file`, `← 29 dependents · 10 files`, `⚡ 70 transitively affected`). Context entries keep their verbatim content under a per-entry header.
+
 ## [0.15.1] - 2026-07-01
 
 ### Added

package/README.md

@@ -51,6 +51,12 @@ Or via Homebrew:
 brew install sem-cli
 ```
 
+Or via winget on Windows:
+
+```powershell
+winget install AtaraxyLabs.sem
+```
+
 Or install the npm wrapper into `node_modules`:
 
 ```bash
@@ -205,6 +211,18 @@ sem log authenticateUser --limit 20
 sem log authenticateUser --json
 ```
 
+With no entity, `sem log` analyzes recent repo history at the entity level:
+**hotspots** (most-changed functions/classes, with author counts) and
+**co-change pairs** (entities that repeatedly change in the same commits —
+"if you touch one, don't forget the other"):
+
+```bash
+sem log                 # repo hotspots + co-change pairs (last 50 commits)
+sem log --limit 200     # deeper history
+sem log --file src/auth.ts   # scoped to one file
+sem log --json          # full data
+```
+
 ### sem entities
 
 List all entities under a file or directory path. No path is the same as `.`.
@@ -276,7 +294,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: Ataraxy-Labs/sem/action@main
+      - uses: Ataraxy-Labs/sem/action@v0.15.1
 ```
 
 No config, no API keys, never fails your build. See [action/](action/) for details.
@@ -318,6 +336,7 @@ It is fully optional and transparent:
 | Swift | `.swift` | functions, classes, protocols, structs, enums, properties |
 | Elixir | `.ex` `.exs` | modules, functions, macros, guards, protocols |
 | Bash | `.sh` | functions |
+| Fish | `.fish` | functions |
 | Lua | `.lua` | functions (global, local, table, and method forms) |
 | HCL/Terraform | `.hcl` `.tf` `.tfvars` | blocks, attributes (qualified names for nested blocks) |
 | Kotlin | `.kt` `.kts` | classes, interfaces, objects, functions, properties, companion objects |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ataraxy-labs/sem",
-  "version": "0.15.1",
+  "version": "0.16.0",
   "description": "npm wrapper for the sem CLI. Downloads the matching release binary and exposes the sem command in node_modules/.bin.",
   "license": "MIT OR Apache-2.0",
   "type": "module",