canopy-engine 0.1.3 → 0.1.4

package/README.md

@@ -4,7 +4,7 @@
 
 Canopy reads a **JavaScript or TypeScript** codebase and draws a map of what it does: where execution starts (routes, event listeners, socket handlers, the app's main file), what each of those calls, in order, and how the parts of a project (services) fit together. The map is built from the code itself.
 
-**JavaScript and TypeScript only** (`.js .jsx .mjs .cjs .ts .tsx .mts .cts`). Other languages (for example Python) and Vue / Svelte files are not read.
+**JavaScript and TypeScript only.** Exactly which files are read is listed in "Which files are analysed" below.
 
 ## Install (needs Node 18 or newer)
 
@@ -21,7 +21,9 @@ npm install -g canopy-engine   # needs the network
 canopy
 ```
 
-(The two lines marked "needs the network" download from the npm registry. If you were given a file instead, `canopy-engine-0.1.3.tgz`, install that: `npm install -g ./canopy-engine-0.1.3.tgz`.)
+Without installing anything, `npx canopy-engine` runs the same program (it downloads it the first time). **Do not type `npx install -g canopy-engine`**: `npx install` is not a command. The install command is `npm install -g canopy-engine`, and then you run `canopy`.
+
+(The two lines marked "needs the network" download from the npm registry. If you were given a file instead, `canopy-engine-0.1.4.tgz`, install that: `npm install -g ./canopy-engine-0.1.4.tgz`.)
 
 **If the `canopy` command clashes with another tool** on your computer, use `canopy-engine` instead. It is the same program under a second name: `canopy-engine --version`, `canopy-engine doctor`, and so on.
 
@@ -33,12 +35,32 @@ canopy
 2. **Looks at your project** - counts the JavaScript / TypeScript files. A folder with none (for example a Python-only project) stops here: "Canopy currently supports JavaScript and TypeScript".
 3. **Prepares `.canopy/`** - the only folder Canopy writes inside your project; it adds `.canopy/` to `.gitignore` (creating the file if needed, never removing a line).
 4. **Asks about the commit hook** - "Update the map on every commit? [Y/n]" (default yes; `-y` skips the question; with no terminal it answers yes). See "Hook on and off" below.
-5. **Builds the map** - the first time takes seconds to minutes depending on size; later runs only re-read what changed. It ends with a short summary: services, workflows, files inside a service, files with parse errors.
+5. **Builds the map** - the first time takes seconds to minutes depending on size (a project of ten thousand files takes some minutes); a later run on an unchanged project answers "up to date" at once, and after an edit it re-reads only the changed files and links the whole project again. It ends with a short summary: services, workflows, files inside a service, files with parse errors.
 6. **Opens the map** - starts a viewer on a free local port (127.0.0.1 only) and opens your browser. Ctrl+C stops it.
 
-Then it prints "Next time just run `canopy`." and the feedback link. Options: `-y` (no questions), `--no-open` (build the map, do not start the viewer; for scripts), `--port=N`, `--for=SECONDS` (stop the viewer by itself), and a folder to look at instead of the current one: `canopy path/to/project`.
+Then it prints "Next time just run `canopy`." and the feedback link. Options: `-y` / `--yes` (no questions; installs the hook), `--no-hook` (no hook, no question), `--no-open` (build the map, do not start the viewer; for scripts), `--port=N`, `--for=SECONDS` (stop the viewer by itself), `--full` (ignore everything remembered and analyse from scratch), `--quiet` (print only problems and the address), `--no-color` (or set `NO_COLOR=1`; also used automatically when the output is not a terminal), and a folder to look at instead of the current one: `canopy path/to/project`.
+
+Supported languages: JavaScript and TypeScript only (see "Which files are analysed" below).
+
+## Which files are analysed
+
+Read from the source of the engine, not guessed:
+
+- **Parsed as JavaScript:** `.js .jsx .mjs .cjs`. **Parsed as TypeScript:** `.ts .mts .cts` (and `.tsx` with the TSX grammar).
+- **Found but not analysed:** declaration files (`.d.ts .d.mts .d.cts`: they only describe types), `*.min.js`, `*.bundle.js`, files over 1 MB and files with a line over 10,000 characters (minified or bundled code), and everything inside `node_modules`, `.git`, `dist`, `build`, `out`, `coverage`, `vendor`, `.next`, `.nuxt`, `.turbo`, `.cache`, `.yarn`, `.pnp`, `bower_components` and anything your `.gitignore` lists. They are counted in the coverage summary.
+- **`.html`** is read only to find the page's `<script src>` (the start of a browser app); its other contents are not analysed.
+- **`.vue` and `.svelte` files are skipped, not parsed**: they are counted ("not read yet: N files in languages Canopy does not support"), they do not fail the run, and their code does not appear on the map.
+- Every other file type (`.py`, `.go`, `.css`, `.json`, ...) is ignored.
+
+## Large projects, memory and what happens when it still fails
+
+- A whole analysis runs in a second Node process that is given about **75% of this computer's memory** (at least 2 GB, at most 8 GB). Set `CANOPY_MAX_MEMORY_MB=6144` (any number from 256) to choose it yourself; an existing `--max-old-space-size` in `NODE_OPTIONS` is respected. `CANOPY_DEBUG_MEM=1` prints the memory use at each stage (to the terminal only).
+- If Canopy still runs out of memory, or the process ends abnormally, it says so in plain words with what to try next (analyse one package or app at a time with `canopy <folder>`, give it more memory, close other programs) and exits with a non-zero code. It never ends silently. `.canopy/last-error.txt` then holds the time, the Canopy version, the stage it was in, the exit code and the memory setting, and nothing else (no code, no secrets).
+- Nothing is lost by a crash: every file already read is remembered, so the next run starts where the last one stopped. A run that was stopped leaves no stale lock behind (a lock whose process has ended is removed, and the run says so).
+
+## What Canopy remembers
 
-**Supported languages: JavaScript and TypeScript only** (`.js .jsx .mjs .cjs .ts .tsx .mts .cts`). Other languages (for example Python) and Vue / Svelte files are not read.
+Inside the project's `.canopy/` folder, on this computer only: the versions of your code it has analysed, which files it has already read, and **which version you last looked at**. That is what lets the Changes view start with "Since you last opened Canopy, 2 days ago (12 commits)" instead of only the last commit. The "last looked at" mark moves only after the page has loaded and stayed open for a minute, or when you stop Canopy normally; a crash does not move it. Delete `.canopy/` to start over; a damaged store is moved aside (`.canopy/state.corrupt-<time>`) and rebuilt, and Canopy says so. If you run `canopy` again in the same project while its viewer is open, it opens the running one instead of starting a second.
 
 ## Updating
 
@@ -100,7 +122,7 @@ The hook starts `canopy sync` in the background after every commit and merge so
 Run `canopy doctor` and send its output with your feedback. Healthy output (the hook line is not critical):
 
 ```
-canopy 0.1.3 doctor (C:\work\my-project)
+canopy 0.1.4 doctor (C:\work\my-project)
 
 OK    Node version >= 18 - v22.12.0
 OK    tree-sitter grammars (wasm) - 5 files
@@ -136,6 +158,10 @@ Exit codes of `canopy`: 0 ok, 1 a problem (the message says what to do), 3 no Ja
 | `output/history.js` | the version list and recent diffs shown in the viewer's Changes panel |
 | `history/*.json.gz` | one compressed snapshot per version (total size capped at 200 MB, `CANOPY_SNAPSHOT_MAX_MB`) |
 | `sync.log` | log of background syncs started by the hook |
+| `meta.json` | the project's random id, when it was first set up, which version you last looked at (checksummed) |
+| `files.json` | size, time and hash of each analysed file, so an unchanged project is recognised without re-reading it |
+| `server.json` | the address of the viewer while it runs (so a second `canopy` reuses it) |
+| `last-error.txt` | only after a crash: time, version, stage, exit code, memory setting |
 
 ## The viewer in five bullets
 
@@ -143,7 +169,8 @@ Exit codes of `canopy`: 0 ok, 1 a problem (the message says what to do), 3 no Ja
 - A workflow opens as a chain of cards: the entry point (a route, a listener, the main file) and the functions it calls, in source order, with branches (`if`, `switch`, `try`) shown side by side.
 - Click a call to open it as its own card; **Trace from root** opens everything beneath the entry point (it pauses at 3,000 rows and offers Continue).
 - The `</>` button on any card shows the exact source lines.
-- The **Changes** panel marks what is NEW or MODIFIED between two stored versions.
+- The **Changes** panel starts with what changed since you last opened Canopy, and can also step through one commit at a time; click a changed function for its line-by-line diff.
+- **Runtime (predicted)** draws a workflow end to end across the services it calls, one band per service.
 
 ## Known limits
 
@@ -155,6 +182,7 @@ Each line is backed by a measurement (the numbers are quoted in the lines below)
 - Flow type annotations are not supported (parse-server: 47 of 370 files have syntax errors).
 - Anonymous exports have no node, so calls into them are not followed.
 - Repositories with 5,000+ files take minutes (strapi, 5,451 files: 219 s the first time).
+- Vue and Svelte files are skipped (see "Which files are analysed").
 
 ## Troubleshooting