@g3nomaly/analyze 0.1.2 → 0.1.4

package/README.md

@@ -11,11 +11,11 @@ cd ~/code/my-repo
 bunx @g3nomaly/analyze create_docs
 ```
 
-This writes `analyze_out/index.html` in the repository, and the query commands keep their scan cache in `analyze_out/` as well. Add it to `.gitignore`. The one other file analyze writes is `.analyzeignore`, once, when the repository has none.
+This writes `analyze_out/index.html` in the repository, and the query commands keep their scan cache in `analyze_out/` as well. analyze puts a `.gitignore` holding `*` in `analyze_out/`, so git skips the directory. It writes nothing else in the repository.
 
 ## What it skips
 
-`.analyzeignore` at the repository root lists what the scan skips, on top of `.gitignore`, with the same patterns: `name/` skips a directory anywhere, `/name` only at the root, `*.ext` a file anywhere. analyze creates it with `node_modules/`, `dist/`, `build/`, `out/`, `target/`, `vendor/`, `coverage/`, `data/` and the like when it is absent. Edit it to change what is skipped. `--exclude` adds patterns for one run.
+On top of `.gitignore`, the scan skips `node_modules/`, `dist/`, `build/`, `out/`, `target/`, `vendor/`, `coverage/`, `data/`, `fixtures/`, `__tmp__/`, `__pycache__/`, virtual environments, minified files, `.d.ts` files and npm lock files. To change that list, add a `.analyzeignore` at the repository root. It replaces the built-in list and takes the patterns of `.gitignore`: `name/` skips a directory anywhere, `/name` only at the root, `*.ext` a file anywhere. analyze never writes this file. `--exclude` adds patterns for one run.
 
 `analyze` needs [Bun](https://bun.sh) 1.2.21 or newer.
 
@@ -24,13 +24,13 @@ This writes `analyze_out/index.html` in the repository, and the query commands k
 | Section | Read from |
 |---|---|
 | Skills | Every `SKILL.md` inside the directory of an AI tool (`.claude`, `.codex`, `.cursor`, `.gemini`, `.agents`, `.windsurf`, `.opencode`) and the files beside it. A `SKILL.md` elsewhere is a doc |
-| Docs | Markdown. `glossary.md` is read as terms, `decisions.md` as decisions |
+| Docs | Markdown. `glossary.md` is read as terms, `decisions.md` as decisions, in any case. Any other markdown file is a document, whatever its folder |
 | Code | Classes and functions in 13 languages through tree-sitter, with their doc comments. JSON, TOML and YAML files, OpenAPI documents and JSON Schemas among them |
 | Database | `.sql` files. A file that creates tables becomes a schema diagram, any other a query catalog. No database is ever opened |
 | Tasks | The plans, phases and tasks that task-manager keeps, in `skills.settings.filetreeRoot` of `.g3n.config.json` (default `docs/task-manager`) |
-| Packages | A directory with a `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`, `composer.json`, `.csproj`, `.gemspec` or `Gemfile` is shown as a package: an app, a library or a workspace |
+| Packages | A directory with a `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`, `composer.json`, `.csproj`, `.gemspec`, `Gemfile`, `vcpkg.json` or `CMakeLists.txt` is shown as a package: an app, a library or a workspace. A `CMakeLists.txt` counts when it calls `project(`, builds a library or an executable (`add_library`, `add_executable`, or a macro whose name holds `library` or `executable`), or sits beside an `include/` folder. In a `src/`, `test/`, `tests/`, `cmake/` or `examples/` directory only `project(` counts, so `libs/<lib>/src/CMakeLists.txt` stays part of `libs/<lib>` |
 
-Every page, entry, parameter and field is a node of one graph. Edges connect a type to where it is used, a term to the text that uses it, a decision to the code it affects, a table to its foreign keys and queries, and a task to the code, docs and decisions it names and to the tasks it depends on. One registry in `src/model/registry.ts` defines every kind of node and edge.
+Every page, entry, parameter and field is a node of one graph. Edges connect a type to where it is used, a source file to the files it includes or imports, a function to the functions it calls, a term to the text that uses it, a decision to the code it affects, a table to its foreign keys and queries, and a task to the code, docs and decisions it names and to the tasks it depends on. One registry in `src/model/registry.ts` defines every kind of node and edge.
 
 ## Querying
 
@@ -39,10 +39,25 @@ g3n-analyze search how are routes chosen     # by words and by meaning
 g3n-analyze show choose_route                # one page or entry in full
 g3n-analyze refs Sailing --kind term         # what it links to, and what links to it
 g3n-analyze list --kind task                 # every task with its status
+g3n-analyze graph --level packages --json    # which package uses which, as nodes and edges
+g3n-analyze list --community 3               # every page and entry of one community
 g3n-analyze show TASK-002 --json             # a task, as JSON
 ```
 
-Output is markdown, or JSON with `--json`. `--from analyze_out/index.html` queries a built site without the repository. `g3n-analyze --help` lists every option, node kind, section and edge kind.
+When nothing matches, `search` lists up to three weak matches by meaning instead, under a note and with `weak: true` in the JSON. Rewording the query often finds a better one. Output is markdown, or JSON with `--json`. `--from analyze_out/index.html` queries a built site without the repository. `g3n-analyze --help` lists every option, node kind, section and edge kind.
+
+`graph` exports the whole graph in one call. `--level` picks how much of it: `files`, `entries` and `all` are the levels of the graph page of the site, and `packages` and `communities` sum the edges between packages or communities. `--kind`, `--section`, `--community` and `--edge` narrow the nodes and edges. An edge of a node the level leaves out, such as a call at the `files` level, counts for the node that holds it, such as its file.
+
+### Communities
+
+A **community** is a group of nodes that link to each other more than to the rest, such as the classes and functions of one feature. Each scan splits the graph once, with the Louvain method, over every page and entry; a parameter or field is in the community of its entry, and a folder or package is in none. The community is a value of the node: `graph --level communities` lists them with their ids, `--community <id>` narrows `list`, `search`, `show`, `refs` and `graph` to one, and every node of a `graph` export carries its id. On the graph page of the site, dots can be colored by community and one community can be shown on its own.
+
+### How code links
+
+- **Imports**: C and C++ `#include`, JavaScript and TypeScript imports, and Python imports link to the file of the repository they name. An include resolves next to the including file, then against every file whose path ends with it, the closest winning. A TypeScript import resolves a relative path, a package of the repository by name (`@scope/core`), or a path alias (`@/lib/api`). A header or package from outside the repository links to nothing.
+- **What a file sees**: in C, C++, JavaScript, TypeScript and Python, a file sees itself and the files it imports, directly or through other files. A type name or a call only links to a declaration in a file it sees, so a common name never links across the repository. In the other languages every file is seen, and a type name links to the closest declaration of that name.
+- **Types**: a qualified C++ name such as `geo::Point` only links to a `Point` inside `geo`. A type declared in a C or C++ source file is seen only in that file.
+- **Calls**: a call links when it names one function: by name, through a namespace or type (`geo::distance()`, `Timer.once()`), through `this` or `self`, or through a parameter or field whose declared type is known. A call through a local variable, a module alias such as `helpers.go()`, or any other expression links nowhere, since its type is unknown. A bare call inside a JavaScript, TypeScript or Python method never means a method of its own class. A C++ method declared in a header and defined in a source file is called at its declaration.
 
 ## From hooks
 
@@ -56,7 +71,7 @@ const task = await show({ repo: '/home/me/app', target: 'TASK-002' });
 const callers = await refs({ repo: '/home/me/app', target: 'choose_route', incoming: true, edges: ['names'] });
 ```
 
-`createDocs`, `search`, `show`, `refs` and `list` take the options of the CLI as fields.
+`createDocs`, `search`, `show`, `refs`, `list` and `graph` take the options of the CLI as fields.
 
 ## Development
 
@@ -70,7 +85,7 @@ bun run test:integration   # builds first
 bun run analyze -- --help  # the CLI from source, after one build
 ```
 
-The web primitives in `src/web/components/shadcn/` are installed with the `g3nui` CLI of `@g3nomaly/ui`. The reasons behind the design are in [`decisions.md`](decisions.md).
+The web primitives in `src/web/components/shadcn/` are installed with `@g3nomaly/ui`. The reasons behind the design are in [`decisions.md`](decisions.md).
 
 ## License
 

package/dist/api/bundle-cache.d.ts

@@ -48,14 +48,16 @@ export declare function fingerprintOf(sources: ScanSources): Promise<string>;
  */
 export declare class FileBundleCache implements BundleCache {
     private readonly directory;
+    private readonly prepare;
     private readonly file;
     /**
      * Creates the cache of one scan.
      *
      * @param directory - Cache directory. Created on the first write.
      * @param key - Name of the entry, from `cacheKey`.
+     * @param prepare - Runs before each write, before the cache directory is created, such as `prepareOutputDir` of the repository.
      */
-    constructor(directory: string, key: string);
+    constructor(directory: string, key: string, prepare?: () => Promise<void>);
     /**
      * Reads the entry when its first line is the fingerprint.
      *

package/dist/api/graph.d.ts

@@ -0,0 +1,131 @@
+import { z } from 'zod';
+import { type EdgeKind } from '../model';
+import { type ExportLevel, type GraphExport, type NodeFilter, type Retriever } from '../retrieval';
+/** The level `graph` exports when the caller names none. */
+export declare const DEFAULT_EXPORT_LEVEL: ExportLevel;
+/**
+ * Schema of the `graph` options.
+ *
+ * @property level - How much of the bundle to export. Default: `files`.
+ * @property edges - Only edges of these kinds. Empty or absent keeps every kind.
+ */
+declare const graphSchema: z.ZodPipe<z.ZodObject<{
+    onWarning: z.ZodOptional<z.ZodCustom<import("./options").WarningHandler, import("./options").WarningHandler>>;
+    kinds: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodEnum<{
+        class: "class";
+        decision: "decision";
+        definition: "definition";
+        enum: "enum";
+        field: "field";
+        file: "file";
+        folder: "folder";
+        function: "function";
+        interface: "interface";
+        namespace: "namespace";
+        operation: "operation";
+        package: "package";
+        parameter: "parameter";
+        phase: "phase";
+        plan: "plan";
+        query: "query";
+        struct: "struct";
+        table: "table";
+        task: "task";
+        term: "term";
+        trait: "trait";
+        type: "type";
+    }>>>>;
+    section: z.ZodOptional<z.ZodEnum<{
+        code: "code";
+        database: "database";
+        docs: "docs";
+        skills: "skills";
+        tasks: "tasks";
+    }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
+    repo: z.ZodOptional<z.ZodString>;
+    skillDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
+    docDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
+    excludes: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
+    embeddings: z.ZodOptional<z.ZodBoolean>;
+    from: z.ZodOptional<z.ZodString>;
+    cache: z.ZodOptional<z.ZodBoolean>;
+    level: z.ZodOptional<z.ZodEnum<{
+        all: "all";
+        communities: "communities";
+        entries: "entries";
+        files: "files";
+        packages: "packages";
+    }>>;
+    edges: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodEnum<{
+        affects: "affects";
+        calls: "calls";
+        depends: "depends";
+        extends: "extends";
+        imports: "imports";
+        mentions: "mentions";
+        names: "names";
+        reads: "reads";
+        references: "references";
+        returns: "returns";
+        throws: "throws";
+        type: "type";
+        uses: "uses";
+        writes: "writes";
+    }>>>>;
+}, z.core.$strict>, z.ZodTransform<{
+    source: import("./source").BundleSource;
+    reporter: import("../scanner").Reporter;
+    request: {
+        level: "all" | "communities" | "entries" | "files" | "packages";
+        filter: NodeFilter;
+        edges: readonly ("affects" | "calls" | "depends" | "extends" | "imports" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[];
+    };
+}, {
+    onWarning?: import("./options").WarningHandler | undefined;
+    kinds?: readonly ("class" | "decision" | "definition" | "enum" | "field" | "file" | "folder" | "function" | "interface" | "namespace" | "operation" | "package" | "parameter" | "phase" | "plan" | "query" | "struct" | "table" | "task" | "term" | "trait" | "type")[] | undefined;
+    section?: "code" | "database" | "docs" | "skills" | "tasks" | undefined;
+    communities?: readonly number[] | undefined;
+    repo?: string | undefined;
+    skillDirs?: readonly string[] | undefined;
+    docDirs?: readonly string[] | undefined;
+    excludes?: readonly string[] | undefined;
+    embeddings?: boolean | undefined;
+    from?: string | undefined;
+    cache?: boolean | undefined;
+    level?: "all" | "communities" | "entries" | "files" | "packages" | undefined;
+    edges?: readonly ("affects" | "calls" | "depends" | "extends" | "imports" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[] | undefined;
+}>>;
+/** Options of `graph`: the level, the nodes and edges to keep, and the bundle to read. */
+export type GraphOptions = z.input<typeof graphSchema>;
+/**
+ * One `graph` on a loaded bundle.
+ *
+ * @property level - How much of the bundle to export.
+ * @property filter - Kinds and section of the nodes to keep.
+ * @property edges - Edge kinds to keep. Empty keeps every kind.
+ */
+export interface GraphRequest {
+    readonly level: ExportLevel;
+    readonly filter: NodeFilter;
+    readonly edges: readonly EdgeKind[];
+}
+/**
+ * Exports one level of a loaded bundle.
+ *
+ * @param retriever - The loaded bundle to export.
+ * @param request - The level and what to keep.
+ * @returns The nodes and edges of the level. The CLI prints it with `graph --json`.
+ */
+export declare function graphIn(retriever: Retriever, request: GraphRequest): GraphExport;
+/**
+ * Exports the nodes and edges of a repository or a site in one call, at one level: packages, files, entries or all.
+ *
+ * @param options - The level, the kinds, section and edge kinds to keep, and the repository or site to read.
+ *   Default: the files of the working directory with every edge.
+ * @returns The nodes and edges of the level.
+ * @throws {OptionsError} When the options are not valid.
+ * @throws {SourceError} When the repository or the site cannot be read.
+ */
+export declare function graph(options?: GraphOptions): Promise<GraphExport>;
+export {};

package/dist/api/list.d.ts

@@ -38,6 +38,7 @@ declare const listSchema: z.ZodPipe<z.ZodObject<{
         skills: "skills";
         tasks: "tasks";
     }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
     repo: z.ZodOptional<z.ZodString>;
     skillDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
     docDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
@@ -57,6 +58,7 @@ declare const listSchema: z.ZodPipe<z.ZodObject<{
     onWarning?: import("./options").WarningHandler | undefined;
     kinds?: readonly ("class" | "decision" | "definition" | "enum" | "field" | "file" | "folder" | "function" | "interface" | "namespace" | "operation" | "package" | "parameter" | "phase" | "plan" | "query" | "struct" | "table" | "task" | "term" | "trait" | "type")[] | undefined;
     section?: "code" | "database" | "docs" | "skills" | "tasks" | undefined;
+    communities?: readonly number[] | undefined;
     repo?: string | undefined;
     skillDirs?: readonly string[] | undefined;
     docDirs?: readonly string[] | undefined;

package/dist/api/options.d.ts

@@ -45,8 +45,9 @@ export declare const filterShape: {
         skills: "skills";
         tasks: "tasks";
     }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
 };
-/** The filter options after validation: `kinds` and `section`, both optional. */
+/** The filter options after validation: `kinds`, `section` and `communities`, each optional. */
 type FilterFields = z.output<z.ZodObject<typeof filterShape>>;
 /**
  * Turns validated filter options into the filter the retriever takes.

package/dist/api/refs.d.ts

@@ -41,6 +41,7 @@ declare const refsSchema: z.ZodPipe<z.ZodObject<{
         skills: "skills";
         tasks: "tasks";
     }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
     repo: z.ZodOptional<z.ZodString>;
     skillDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
     docDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
@@ -53,8 +54,10 @@ declare const refsSchema: z.ZodPipe<z.ZodObject<{
     outgoing: z.ZodOptional<z.ZodBoolean>;
     edges: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodEnum<{
         affects: "affects";
+        calls: "calls";
         depends: "depends";
         extends: "extends";
+        imports: "imports";
         mentions: "mentions";
         names: "names";
         reads: "reads";
@@ -73,13 +76,14 @@ declare const refsSchema: z.ZodPipe<z.ZodObject<{
         filter: NodeFilter;
         references: {
             direction: "both" | "in" | "out";
-            kinds: readonly ("affects" | "depends" | "extends" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[];
+            kinds: readonly ("affects" | "calls" | "depends" | "extends" | "imports" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[];
         };
     };
 }, {
     onWarning?: import("./options").WarningHandler | undefined;
     kinds?: readonly ("class" | "decision" | "definition" | "enum" | "field" | "file" | "folder" | "function" | "interface" | "namespace" | "operation" | "package" | "parameter" | "phase" | "plan" | "query" | "struct" | "table" | "task" | "term" | "trait" | "type")[] | undefined;
     section?: "code" | "database" | "docs" | "skills" | "tasks" | undefined;
+    communities?: readonly number[] | undefined;
     repo?: string | undefined;
     skillDirs?: readonly string[] | undefined;
     docDirs?: readonly string[] | undefined;
@@ -90,7 +94,7 @@ declare const refsSchema: z.ZodPipe<z.ZodObject<{
     target: string;
     incoming?: boolean | undefined;
     outgoing?: boolean | undefined;
-    edges?: readonly ("affects" | "depends" | "extends" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[] | undefined;
+    edges?: readonly ("affects" | "calls" | "depends" | "extends" | "imports" | "mentions" | "names" | "reads" | "references" | "returns" | "throws" | "type" | "uses" | "writes")[] | undefined;
 }>>;
 /** Options of `refs`: the node, which links to follow, and the bundle to read. */
 export type RefsOptions = z.input<typeof refsSchema>;

package/dist/api/scan.d.ts

@@ -41,7 +41,7 @@ export type ScanFields = z.output<z.ZodObject<typeof scanShape>>;
  */
 export declare function scanOf(fields: ScanFields): ScanOptions;
 /**
- * Names the site file of a repository. `createDocs` writes it, and nothing else outside the cache.
+ * Names the site file of a repository. `createDocs` writes it, and outside the cache only the `.gitignore` of `analyze_out/`.
  *
  * @param repo - Absolute path of the repository.
  * @returns `<repo>/analyze_out/index.html`.
@@ -54,3 +54,10 @@ export declare function siteFileOf(repo: string): string;
  * @returns `<repo>/analyze_out/cache`.
  */
 export declare function cacheDirOf(repo: string): string;
+/**
+ * @brief Creates `analyze_out/` in a repository, with a `.gitignore` that holds `*` so git skips everything
+ * analyze writes there. Leaves a `.gitignore` that is already there as it is.
+ *
+ * @param repo - Absolute path of the repository.
+ */
+export declare function prepareOutputDir(repo: string): Promise<void>;

package/dist/api/search.d.ts

@@ -42,6 +42,7 @@ declare const searchSchema: z.ZodPipe<z.ZodObject<{
         skills: "skills";
         tasks: "tasks";
     }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
     repo: z.ZodOptional<z.ZodString>;
     skillDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
     docDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
@@ -69,6 +70,7 @@ declare const searchSchema: z.ZodPipe<z.ZodObject<{
     onWarning?: import("./options").WarningHandler | undefined;
     kinds?: readonly ("class" | "decision" | "definition" | "enum" | "field" | "file" | "folder" | "function" | "interface" | "namespace" | "operation" | "package" | "parameter" | "phase" | "plan" | "query" | "struct" | "table" | "task" | "term" | "trait" | "type")[] | undefined;
     section?: "code" | "database" | "docs" | "skills" | "tasks" | undefined;
+    communities?: readonly number[] | undefined;
     repo?: string | undefined;
     skillDirs?: readonly string[] | undefined;
     docDirs?: readonly string[] | undefined;
@@ -102,7 +104,7 @@ export interface SearchRequest {
  * @property query - The words searched for.
  * @property mode - How the words were matched.
  * @property model - Name of the embedding model of the bundle. `null` when the bundle has none.
- * @property hits - Hits, best first.
+ * @property hits - Hits, best first. When nothing matched, the few closest by meaning, each marked `weak`.
  */
 export interface SearchResult {
     readonly query: string;
@@ -111,7 +113,8 @@ export interface SearchResult {
     readonly hits: readonly SearchHit[];
 }
 /**
- * Searches a loaded bundle.
+ * @brief Searches a loaded bundle. When nothing clears the cut-off, it returns the few closest hits by meaning
+ * below it instead, each marked `weak`.
  *
  * @param retriever - The loaded bundle to search.
  * @param request - The words, the mode, the limit and the filter.

package/dist/api/services.d.ts

@@ -6,7 +6,7 @@ export declare const GENERATOR: string;
 /**
  * @brief Wires the services that scan one repository into a builder.
  *
- * Reads the `.analyzeignore` of the repository, creating it when absent, and skips its patterns and `scan.excludes`.
+ * Skips the patterns of the repository's `.analyzeignore`, or the built-in ones when it has none, and `scan.excludes`.
  *
  * @param scan - What to scan. Its `embeddings` flag decides whether the builder embeds the nodes.
  * @param reporter - Receives the warnings of the scan.

package/dist/api/show.d.ts

@@ -38,6 +38,7 @@ declare const showSchema: z.ZodPipe<z.ZodObject<{
         skills: "skills";
         tasks: "tasks";
     }>>;
+    communities: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodNumber>>>;
     repo: z.ZodOptional<z.ZodString>;
     skillDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
     docDirs: z.ZodOptional<z.ZodReadonly<z.ZodArray<z.ZodString>>>;
@@ -57,6 +58,7 @@ declare const showSchema: z.ZodPipe<z.ZodObject<{
     onWarning?: import("./options").WarningHandler | undefined;
     kinds?: readonly ("class" | "decision" | "definition" | "enum" | "field" | "file" | "folder" | "function" | "interface" | "namespace" | "operation" | "package" | "parameter" | "phase" | "plan" | "query" | "struct" | "table" | "task" | "term" | "trait" | "type")[] | undefined;
     section?: "code" | "database" | "docs" | "skills" | "tasks" | undefined;
+    communities?: readonly number[] | undefined;
     repo?: string | undefined;
     skillDirs?: readonly string[] | undefined;
     docDirs?: readonly string[] | undefined;