@nithin554/code-crawler 0.1.62 → 0.1.64

package/skills/code-crawler/SKILL.md

@@ -0,0 +1,180 @@
+---
+name: code-crawler
+description: Answer questions about code relationships in an indexed repository - callers, callees, impact, entry points, routes, DI wiring, dead code, cycles - using the code-crawler MCP server's SQLite property graph. Use when the question is about how symbols connect (who calls this, what breaks if I change it, where does a request enter, is this reachable) rather than about text you can find by searching. Do not crawl a repository just to answer a question about a single file you already have open.
+---
+
+# Code Crawler
+
+A SQLite property graph of a codebase, exposed as 27 MCP tools. The graph is built
+once by a crawl, then queried cheaply many times.
+
+## Getting the tools
+
+**Check first.** If no `code-crawler` tools appear in your tool list, the server is not
+registered and nothing else in this skill will run. Say so instead of falling back to
+`grep` — and ask before editing a client config file, which is the user's machine
+state, not yours.
+
+Installing the *skill* does not install the *server*: `npx skills add nithin554/code-crawler`
+delivers these instructions, and `npx add-mcp @nithin554/code-crawler` — or the config
+below — registers the server that delivers the tools. Both are needed, and they are
+independent.
+
+The server is the npm package `@nithin554/code-crawler` and needs Node.js ≥ 22.12. Every
+stdio MCP client takes the same command/args/env triple:
+
+```json
+{
+  "mcpServers": {
+    "code-crawler": {
+      "command": "npx",
+      "args": ["-y", "@nithin554/code-crawler", "serve"],
+      "env": { "CODECRAWLER_DB": "/absolute/path/to/index.db" }
+    }
+  }
+}
+```
+
+`npx -y` needs no prior install. If the package is already installed globally, use
+`"command": "codecrawler"` with `"args": ["serve"]`. From a source checkout, point
+`"command": "node"` at `dist/index.js`, keeping `"args": ["serve"]`. `serve` is the default
+subcommand, so `args` may omit it.
+
+Writing that JSON by hand is the slow way. `add-mcp` fills it in for whichever client you
+name. Run it from inside the repository you want indexed:
+
+```bash
+npx add-mcp @nithin554/code-crawler --args serve -a cursor
+```
+
+It derives the server name `code-crawler` from the package, so no `--name` is needed. `-a`
+accepts `claude-code`, `cursor`, `vscode`, `claude-desktop`, `codex`, `opencode`, `zed` and
+about twenty more — `npx add-mcp list-agents` prints the list. Without `-g` the entry lives
+in the project (`.mcp.json`, `.cursor/mcp.json`, `.vscode/mcp.json`); `-g` writes user-level
+config. Repeat `--args` and `--env` once per value.
+
+`--env CODECRAWLER_DB` is optional: the server indexes into `<project>/.code-crawler/index.db`
+by default. Pin it — `--env CODECRAWLER_DB=/absolute/path/index.db` — when you want the file
+elsewhere, or when the client's working directory is not the project.
+
+Three things decide whether the graph is usable:
+
+- **`CODECRAWLER_DB`** is the SQLite file holding the graph. The default needs no
+  configuration: the server walks up from its working directory to the nearest project root
+  (`.git`, `package.json`, `pom.xml`, `build.gradle`, `pyproject.toml`, `setup.py`,
+  `requirements.txt`, `go.mod`, `Cargo.toml`) and uses `<project>/.code-crawler/index.db`, so
+  **each repository gets its own file**. With no enclosing project it falls back to
+  `~/.code-crawler/index.db`. That home file is the fallback and nothing more, never an
+  enclosing project, so an unmarked folder under `~` gets its own file rather than sharing
+  it. A database still holds exactly one project — crawling a
+  different path into one is refused before anything is written; give that project its own
+  database (its own registration), or run `clear_index` to replace the existing index.
+- **One server per project.** A process opens one database at startup and a database holds
+  one project, so register the server once per repository (project-scoped config, above)
+  rather than once globally for all of them. `crawl` resolves the same way, so crawling a
+  subdirectory such as `repo/src` shares `repo`'s file instead of growing a second one.
+- **Never share one `CODECRAWLER_DB` across repositories.** Do that and the second project's
+  crawl hits the one-project error above.
+- **Use absolute paths.** Clients launch servers from an unpredictable working directory,
+  so a relative `CODECRAWLER_DB` or a relative project root lands somewhere unexpected —
+  and the default resolution follows the cwd you may not have chosen.
+- **Config is all environment variables**, identical for the CLI and the client:
+  `CODECRAWLER_DB`, `CODECRAWLER_EXCLUDE_DIRS`, `CODECRAWLER_DEFAULT_LIMIT`,
+  `CODECRAWLER_MAX_DEPTH`, `CODECRAWLER_MAX_FILE_BYTES`, `CODECRAWLER_RESPECT_GITIGNORE`,
+  `CODECRAWLER_HELPFUL_ERRORS`. Invalid values are clamped and warned about, never fatal.
+
+Some clients namespace tool names with the server key — `code-crawler-search_symbols`
+rather than `search_symbols`. Same tools, so match on the suffix if the exact name is
+missing.
+
+Once the tools appear, call `index_status` to confirm the server answers, then follow the
+bootstrap below. The CLI shares the database, so `npx -y @nithin554/code-crawler crawl
+/path/to/repo` warms the index ahead of time if you would rather not spend a turn on it.
+
+## When to reach for it
+
+The graph answers **relationship** questions. Text search answers **location** questions.
+Picking the wrong one wastes the most time of anything in this skill.
+
+| Question | Use |
+| --- | --- |
+| Who calls this method? | `get_callers` |
+| What does this method call? | `get_callees` |
+| What breaks if I change this? | `impact_analysis` |
+| Where does a request enter this system? | `find_entry_points`, `find_routes` |
+| Which implementation handles this interface? | `find_implementations` |
+| Is this method reachable at all? | `find_dead_code`, `find_hotspots` |
+| How does this request reach that handler? | `trace_path`, `trace_execution` |
+| What is wired in by DI or annotations? | `find_injections`, `find_annotated`, `get_decorators` |
+| What third-party code is actually used? | `find_external_dependencies` |
+| Where is `ThingFactory` defined? | **plain search** - do not crawl |
+| Show me the body of this function | **plain file read** - do not crawl |
+
+Crawling is not free: it parses every file in the project. Do it when you intend to ask
+several relationship questions, or when one relationship question has no cheaper answer.
+For a single-file question you already have open, reading the file beats crawling the repo.
+
+## Bootstrap
+
+1. `index_status` first. If it reports files and a recent crawl, the index is already
+   warm and you can skip step 2.
+2. `crawl_project` with the project root. It is idempotent and incremental - unchanged
+   files are skipped via a content manifest, so a second crawl is cheap.
+3. Only then run the query tools.
+
+**Never trust a stale index.** Call `index_status` before concluding "there are no
+callers". An empty result from an empty or stale index is indistinguishable from a true
+negative, and that is the single most common way this tool misleads.
+
+Re-index with the `force` option when the working tree changed underneath the index:
+
+- after a rebase, merge, or `git reset` that rewrote history
+- after switching branches with a large diff
+- after `git stash pop`
+
+An incremental crawl keys off file content and mtime, which is enough for ordinary
+editing but not for a history rewrite. `force: true` bypasses the manifest.
+
+## Reading the results
+
+Two fields decide whether a result means what you think: `confidence` and `external`.
+Phantom nodes mark the third trap. See
+[reference/edge-semantics.md](reference/edge-semantics.md) before reasoning about
+"unreachable" or "no callers".
+
+Short version:
+
+- `confidence` is not a probability. It is a band: `1.0` resolved by construction,
+  `0.9`/`0.85` strong static evidence, `0.8` inherited or external, `0.5` heuristics,
+  `0` phantom.
+- `external: true` means the target is outside the project. It is *not* dead code and
+  *not* unresolved.
+- `__phantom__:` / `__reflect__:` / `__inject__:` targets mark something the resolver
+  could not bind. They are inventory, not findings.
+
+## Reference
+
+- [reference/tools.md](reference/tools.md) - all 27 tools, grouped by the question they
+  answer, with the argument that matters most for each.
+- [reference/edge-semantics.md](reference/edge-semantics.md) - `confidence`, `external`,
+  phantom nodes, and the ways a "missing" edge is legitimate.
+
+## Budget discipline
+
+Every list-returning tool accepts `max_tokens` (100-200000) and `cursor`. The response
+then carries a `page` block with `totalCount`, `truncated`, and `nextCursor`.
+
+Start with a small `max_tokens` and widen only if the answer is genuinely incomplete.
+Reading one page of `get_callers` and reporting the shape is almost always better than
+paging through all of them. Prefer `explain_symbol` over chaining `get_node` +
+`get_callers` + `get_callees` + `get_decorators` when you want a digest rather than a
+specific list.
+
+## Do not
+
+- Crawl a monorepo to answer a question about one file.
+- Report "no callers" or "dead code" without checking `confidence` and `external` on the
+  edges you did get.
+- Treat `find_unresolved` as a bug list. It is a coverage report; some entries are
+  dynamic dispatch that cannot be resolved statically.
+- Page through an entire result set to count things. Use `totalCount` from the first page.

package/skills/code-crawler/reference/edge-semantics.md

@@ -0,0 +1,110 @@
+# Edge semantics
+
+Three things cause most misreads of this graph: `confidence`, `external`, and phantom
+nodes. Read this before reporting "no callers", "dead code", or "unresolved".
+
+## Edge kinds
+
+| Kind | Means |
+| --- | --- |
+| `CALLS` | A statically resolved call |
+| `DYNAMIC_DISPATCH` | A call that reaches an implementation through an interface or superclass |
+| `ADVISES` | Around/before/after advice intercepting a method |
+| `DECLARED_IN` | Symbol to its containing type |
+| `CONTAINS` | Type to its members, module to its declarations |
+| `IMPLEMENTS` | Type to the interface it implements |
+| `EXTENDS` | Type to its superclass |
+| `IMPORTS` | File to the module it imports |
+| `ANNOTATED_WITH` | Symbol to an annotation or decorator |
+| `REFERENCES` | A non-call reference, such as a field or type use |
+| `REFLECTS` | A runtime lookup that may reach a symbol |
+| `INJECTS` | A DI-supplied dependency |
+| `ROUTES` | An HTTP route to its handler |
+
+A method reached only by `DYNAMIC_DISPATCH`, `ADVISES`, or `ROUTES` **is still called**.
+When asking "who calls this", look at all inbound edge kinds, not just `CALLS`.
+
+## `confidence` is a band, not a probability
+
+Do not average it, threshold it, or present it as certainty. It is a coarse label telling
+you *what kind of evidence* produced the edge.
+
+| Value | Evidence |
+| --- | --- |
+| `1.0` | Resolved by construction - a constructor or overload matched exactly |
+| `0.9` | Same-module declaration, typed local, or default constructor |
+| `0.85` | Imported from a named module |
+| `0.8` | Inherited from a supertype, or an external target |
+| `0.75` | Call to an imported class resolved through its import |
+| `0.6` | Pointcut `args(...)` matched on arity only |
+| `0.55` | Pointcut primitive matched by static approximation |
+| `0.3` | Empty pointcut - the advice matches everything |
+| `0` | Phantom - nothing was resolved |
+
+Practical rule: `>= 0.75` is safe to report as fact, `0.5`-`0.7` deserves a hedge, and `0`
+carries no information about the target at all.
+
+## `external` means outside the project
+
+An edge with `external: true` points at a node like `ext:axios` or `ext:fastapi`, and its
+`externalModule` names the package.
+
+Consequences that are easy to get wrong:
+
+- An external target is **not** dead code. It is a real dependency you do not index.
+- An external target is **not** an unresolved call. It was resolved, just to the boundary.
+- `find_dead_code` ignores external targets, so a method whose only caller is external can
+  still look dead. Cross-check `find_external_dependencies`.
+
+## Phantom nodes
+
+A phantom target means the resolver saw a call but could not name its destination. Phantoms
+carry `confidence: 0` and `resolved: false`, and their ids are prefixed:
+
+| Prefix | Origin |
+| --- | --- |
+| `__phantom__:` | A call site with no resolvable target |
+| `__reflect__:` | A runtime lookup whose type or method was not found |
+| `__inject__:` | An injection marker that named neither a type nor a package |
+
+Phantoms are **inventory, not findings**. A phantom is a statement about the analyser, not
+about the code. Never report one as "calls something unknown" without reading its `reason` -
+that field says which lookup step failed.
+
+## Why "no callers" can be legitimate and wrong at the same time
+
+Five ways a real caller produces no inbound `CALLS` edge:
+
+1. **The index is stale.** Always `index_status` first. This is the most common cause.
+2. **Dynamic dispatch.** The call goes through an interface; look for
+   `DYNAMIC_DISPATCH` inbound edges and use `find_implementations`.
+3. **Framework invocation.** Routes, listeners, schedulers, and tests are entry points,
+   not calls. Check `find_entry_points` and `find_routes`.
+4. **Reflection.** `Class.forName`, `getattr`, `importlib`, and dynamic `import()` produce
+   `REFLECTS` edges, not `CALLS`. Check `find_reflections`.
+5. **The caller is external.** Only in-project call edges are recorded. Check
+   `find_external_dependencies`.
+
+So the correct phrasing is "no in-project static callers", not "unused".
+
+## Stale indexes and `force`
+
+Incremental crawls compare file content and mtime against a manifest. That covers normal
+editing. It does **not** cover the working tree changing underneath the index:
+
+- after a rebase, merge, or `git reset` that rewrote history
+- after switching branches with a large diff
+- after `git stash pop`
+
+A file whose content changed but whose mtime was reset can be skipped, leaving edges that
+point at declarations that no longer exist. Pass `force: true` to `crawl_project` in those
+cases. If results look impossible, forcing a re-index is cheaper than debugging the graph.
+
+## Token budgeting
+
+List-returning tools accept `max_tokens` and `cursor`, adding a `page` block with
+`totalCount`, `returnedCount`, `truncated`, `estimatedTokens`, and `nextCursor`.
+
+Truncation is reported, never silent: if `truncated` is true the answer is incomplete and
+you should either widen `max_tokens` or say so. Paging never returns an empty page while
+items remain, so a continuation loop terminates.

package/skills/code-crawler/reference/tools.md

@@ -0,0 +1,108 @@
+# Tool reference
+
+All 27 tools, grouped by the question they answer. Run `index_status` before trusting any
+result, and `crawl_project` before that if the index is empty.
+
+## Orientation
+
+Call these to establish what the index contains, not to answer questions about code.
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `crawl_project` | The index is empty, or files changed on disk | `path` (root), `force` |
+| `index_status` | Before every session, and before believing a negative result | none |
+| `get_schema` | You need the node/edge vocabulary or property names | none |
+| `clear_index` | A crawl was rooted at the wrong directory and the graph is unusable | none |
+
+`crawl_project` is incremental by default. Pass `force: true` only after a history rewrite,
+branch switch, or stash pop - see [edge-semantics.md](edge-semantics.md) for why.
+
+## Find a symbol
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `search_symbols` | You have a name fragment but not a node id | `query`, `kind` |
+| `get_node` | You have a node id and want its metadata and source location | `id` |
+| `explain_symbol` | You want one digest instead of four separate calls | `id` or `name` |
+
+`explain_symbol` returns members, decorators, callers, callees, and framework wiring in one
+payload. Reach for it first when exploring an unfamiliar symbol; drop to `get_callers` or
+`get_callees` when you need the full list rather than a summary.
+
+## Relationships
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `get_callers` | "Who calls this?" - direct or transitive | `id`, `depth` |
+| `get_callees` | "What does this call?" - direct or transitive | `id`, `depth` |
+| `trace_path` | "How does execution get from A to B?" | `from`, `to` |
+| `trace_execution` | You need dynamic dispatch and woven advice, not just static calls | `id` |
+| `get_subgraph` | You want the neighbourhood as data for a diagram | `id`, `radius`, `limit` |
+| `export_graph` | You want Mermaid or Graphviz text you can paste | `id`, `format` |
+
+`get_callers`/`get_callees` take `depth` - keep it at 1 unless the question is explicitly
+transitive, because depth 3 on a hub method returns most of the repository.
+
+`get_subgraph` is bounded by `radius`/`limit` rather than `max_tokens`, because its `nodes`
+and `edges` are two views of one slice and cutting one list leaves dangling references.
+
+## Impact and risk
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `impact_analysis` | "What breaks if I change this?" - the one call to prefer | `id` |
+| `find_implementations` | You have an interface and need its implementers | `id` |
+| `find_cycles` | You suspect mutual recursion or a dependency knot | none |
+| `find_dead_code` | You want methods with no incoming edges at all | none |
+| `find_hotspots` | You want the highest fan-in/fan-out methods ranked | `limit` |
+
+`impact_analysis` bundles callers, entry points, routes, overrides, and a risk band. Prefer
+it over assembling the same answer from four calls.
+
+`find_dead_code` reports *no incoming call, dispatch, or advice edge*. Treat that as a
+candidate list: a method invoked purely by reflection or by an external framework will
+appear here. Cross-check with `find_reflections` and `find_entry_points` before deleting
+anything.
+
+## Framework wiring
+
+These tools exist because the call graph alone cannot see how a framework invokes code.
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `find_entry_points` | "Where does this system begin?" | `kind` |
+| `find_routes` | You want HTTP route to handler mappings | `framework` |
+| `find_injections` | You want DI-supplied dependencies | `framework` |
+| `find_annotated` | You want every symbol carrying one annotation | `annotation` |
+| `get_decorators` | You want the decorators on one symbol | `id`, `includeMembers` |
+| `find_aspects` | You want aspects, advice kinds, and intercepted methods | none |
+| `find_reflections` | You want runtime lookups (`Class.forName`, `getattr`, `import()`) | none |
+
+Entry-point kinds include HTTP routes, listeners, schedulers, CLI mains, and tests. If a
+method looks dead but is plausibly framework-invoked, check `find_entry_points` and
+`find_reflections` before concluding it is unreachable.
+
+## Coverage and dependencies
+
+| Tool | Use it when | Key argument |
+| --- | --- | --- |
+| `find_unresolved` | You need to know how much of the graph failed to bind | none |
+| `find_external_dependencies` | You want the third-party packages actually referenced | none |
+
+`find_unresolved` is grouped by failure reason, which is the useful part: a cluster of one
+reason usually means one modelling gap, not many broken call sites.
+
+`find_external_dependencies` lists packages with reference and file counts. Use it to answer
+"do we actually use this dependency?" - an empty result for a package in `package.json` means
+nothing in the indexed code references it.
+
+## Budgeting
+
+Every list-returning tool accepts `max_tokens` and `cursor`; passing either adds a `page`
+block with `totalCount`, `returnedCount`, `truncated`, `estimatedTokens`, and `nextCursor`.
+Omit both and the response shape is unchanged.
+
+Aggregate tools (`impact_analysis`, `explain_symbol`, `export_graph`) and single-object
+tools (`get_node`, `index_status`, `get_schema`) return one object and take no budget.
+
+Use `totalCount` from the first page instead of paging to count.