prodex 2.0.1 → 2.1.0

package/README.md

@@ -1,239 +1,460 @@
 # Prodex
 
-Focused code-context extraction for large, multi-file projects.
+Prodex packages the code an AI tool, reviewer, or teammate needs to see into readable Markdown or TXT.
 
-Prodex starts from real project entrypoints, follows supported dependencies, adds explicitly included files, and exports a clean Markdown trace. It helps you isolate the files that matter for a feature, review, debug session, documentation pass, handoff, or AI-assisted workflow without manually collecting every related file.
+Use it to snapshot Git changes, collect files by search, save repeatable scopes, attach command output, or gather explicit files — without copy-pasting half your repo into a chat window.
 
-Prodex is intentionally narrow: it creates readable, reproducible context bundles. It is not trying to be a full IDE indexer, static analysis platform, architecture rule engine, or graph visualizer.
+## Installation
 
-## Why Prodex
+```bash
+npm install -g prodex
+```
 
-Large projects make context gathering expensive. A feature can start in one route or component, then spread through shared utilities, types, controllers, services, bindings, and framework conventions. Copying those files by hand is slow, inconsistent, and easy to get wrong.
+Or run it ad hoc without installing:
 
-Prodex gives that workflow a repeatable shape:
+```bash
+npx prodex git --against origin/main
+```
 
-1. Choose one or more entry files or globs.
-2. Trace the dependencies Prodex can resolve.
-3. Add any extra files you explicitly include.
-4. Exclude noisy paths such as generated output, vendor code, or UI primitives.
-5. Export a Markdown bundle with an index and file sections.
+Create a starter config:
 
-The result is a focused project trace you can read, share, review, archive, or hand to another tool.
+```bash
+prodex init
+```
 
-## How It Works
+## Quick Start
 
 ```bash
-prodex run --entry src/index.ts
+# Package all files changed in your current branch vs. origin/main
+prodex git --against origin/main
+
+# Include the diff beside the changed files
+prodex git --changed --include-diff
+
+# Collect every file that references a term or symbol
+prodex grep --query "StripeWebhookHandler"
+
+# Run a saved context selection
+prodex scope -k billing
+
+# Attach test output beside the code
+prodex scope -k auth --cmd "npm test -- auth"
+
+# Collect explicit files without dependency tracing
+prodex pack -i src/config/database.ts,src/models/User.ts,docs/schema.md
+
+# Trace imports from an entry file (JS, TS, PHP)
+prodex trace -t src/api/router.ts
 ```
 
-Prodex resolves the requested entrypoints from your project root, follows supported dependency references, applies include and exclude rules, then writes a versioned Markdown file to `./prodex/` by default.
+By default, commands write a versioned Markdown file to `./prodex/`. You can switch to TXT with `--format txt`. Output filenames include a `-trace` suffix and a timestamp when versioning is enabled (e.g. `billing-trace_260620-0430.md`). The output includes a file index, line ranges, and the collected source content in a single readable document.
+
+## Common Workflows
 
-For example:
+### Review your current branch against origin/main
 
 ```bash
-prodex run --entry resources/js/pages/Dashboard.tsx --include "routes/**/*.php"
+prodex git --against origin/main
 ```
 
-This traces the dashboard entrypoint and adds the matching route files to the exported context.
+Snapshots every file changed between your current HEAD and the merge-base with `origin/main`. Good for pre-PR review or sending a branch summary to an AI assistant.
 
-## Profiles
+### Include the diff beside changed files
 
-Profiles are reusable named context maps stored in `prodex.json`.
+```bash
+prodex git --changed --include-diff
+```
 
-They are one of Prodex's most useful workflow features: instead of rebuilding the same trace commands every time, teams can save important project areas as named profiles such as `dashboard`, `auth`, `billing`, `api`, `admin`, or `checkout`.
+Includes the raw Git diff in the output alongside the collected files. In working-tree mode, two diff sections are added: one for unstaged changes and one for staged changes.
+
+### Collect files by search
 
 ```bash
-prodex run --profile dashboard
-prodex run --profile auth,billing
-prodex run --all
-prodex profiles
+prodex grep --query "legacyPaymentClient"
+prodex grep --any "AuthService,SessionManager"
+prodex grep --regex "use.*Repository"
 ```
 
-Use profiles when a project has recurring review surfaces or ownership areas. A profile can define its own entries, includes, excludes, and optional output name; when no output name is set, the profile key becomes the trace basename. This makes context extraction repeatable across debugging, reviews, documentation, handoffs, and release work. Run one profile, a comma-separated set of profiles, or all profiles with `--all` when you need a broader pass.
+Finds files containing the term or pattern and packages them into one document. Use this when the files you need are spread across the codebase and defined by what they reference, not where they live.
 
-Example:
+### Run a saved scope
 
-```json
-{
-    "version": 4,
-    "$schema": "https://raw.githubusercontent.com/emxhive/prodex/main/schema/prodex.schema.json",
-    "output": {
-        "dir": "prodex",
-        "format": "md",
-        "versioned": true
-    },
-    "entry": [],
-    "include": [],
-    "exclude": ["node_modules/**", "vendor/**", "dist/**"],
-    "resolve": {
-        "aliases": {
-            "@": "resources/js"
-        },
-        "maxDepth": 10,
-        "maxFiles": 200
-    },
-    "profiles": {
-        "dashboard": {
-            "entry": ["resources/js/pages/Dashboard.tsx"],
-            "include": ["routes/**/*.php", "resources/js/types/**/*.d.ts"],
-            "exclude": ["resources/js/components/ui/**"]
-        },
-        "api": {
-            "entry": ["routes/api.php"],
-            "include": ["app/Http/Requests/**/*.php"],
-            "exclude": ["vendor/**"]
-        }
-    }
-}
+```bash
+prodex scope -k billing
+prodex scope -k auth,billing
+prodex scope --all
 ```
 
-## Markdown Output
+Runs a named context selection you configured in `prodex.json`. Scopes are the repeatable version of an ad hoc command — same files, every time, no manual reassembly.
 
-Markdown is the primary Prodex output.
+### Attach command output beside the code
 
-Each generated trace includes:
+```bash
+prodex scope -k auth --cmd "npm test -- auth"
+prodex git --against origin/main --cmd "npm run lint"
+```
 
-- A file index
-- A total file count marker
-- Links to each exported file section
-- Line ranges for the generated sections
-- Syntax-highlighted code fences where possible
-- The collected source content in one readable bundle
+Runs the command after collecting files and embeds its output in the document. The AI sees the failing test or lint result right next to the source that produced it.
 
-Versioned filenames are enabled by default so repeated runs do not overwrite earlier traces.
+### Collect explicit files
 
-## Current Support
+```bash
+prodex pack -i src/config/database.ts,src/models/User.ts,docs/schema.md
+prodex pack -i "app/Http/Controllers/**"
+```
 
-Prodex's broader identity is controlled context extraction, not a single-framework tool.
+Adds files directly by path or glob without dependency tracing. Use this when you know exactly which files you want and do not need imports followed automatically.
 
-Include-only extraction is language-agnostic: any source or text file matched by `--include` or a profile `include` rule can be added to a trace.
+### Trace imports from an entry file
 
-Dependency tracing is resolver-based. Today Prodex traces JavaScript, TypeScript, TSX, declaration files, and PHP entrypoints. Current tracing support includes JS/TS imports, dynamic imports, CommonJS `require`, re-exports, static PHP include/require statements, PHP namespace imports, PSR-4 resolution, and some Laravel binding awareness.
+```bash
+prodex trace -t src/api/router.ts
+prodex pack -e app/Http/Controllers/CheckoutController.php
+```
 
-Unsupported or dynamic relationships may need to be added with `--include` or profile `include` rules. That is expected: Prodex favors a focused, readable trace over pretending to understand every runtime edge in a project.
+Follows import and require statements from the entry file to collect the files it depends on. Supported for JavaScript, TypeScript, TSX, declaration files, and PHP. See [Dependency Tracing](#dependency-tracing) for current language support details.
 
-## Installation
+`prodex trace` resolves targets by exact path, extensionless name, or bare module name — it does not accept glob patterns. Use `prodex pack --entry <glob>` to collect files by glob.
+
+## How Prodex Collects Context
+
+Prodex can collect context from Git changes, search results, saved scopes, explicit files, or traced entrypoints. Command output can be attached to any run.
+
+### Git
+
+`prodex git` reads your repository's Git history or working tree and snapshots the files involved.
+
+Working-tree modes read file contents from disk:
+- `--changed` (default): staged, unstaged, and untracked files
+- `--staged`: staged files only
+- `--unstaged`: unstaged files only
+- `--untracked`: untracked files only
+
+Historical modes read file contents from Git instead of the working tree:
+- `--commit <rev>`: files changed by one commit; contents from that commit
+- `--range <base..head>` or `<base...head>`: files changed between two refs; contents from `head`
+- `--against <base>`: files changed between the merge-base of `<base>` and `HEAD`; contents from `HEAD`
+
+Add `--include-diff` to any git run to include the raw diff output alongside the file contents. In working-tree mode this adds two sections — one for unstaged changes (`Full Diff`) and one for staged changes (`Cached Full Diff`).
+
+### Search
+
+`prodex grep` searches your project for files containing a term or pattern and packages the matching files.
+
+Search modes:
+- `--query <text>`: files containing a fixed string
+- `--any <list>`: files containing any of the terms (OR)
+- `--all <list>`: files containing all of the terms (AND)
+- `--regex <pattern>`: files matching a regular expression
+
+Narrow the search with `--within <path>` or exclude paths with `--skip <path>`. Filter out files whose contents contain a term using `--not <text>`.
+
+### Scopes
+
+Saved named selections in `prodex.json`. See [Saved Scopes](#saved-scopes).
+
+### Pack
+
+`prodex pack` accepts `--include` for direct file collection, `--entry` to trace imports from an entry file, or both together. It is the same tracing engine as `prodex trace` with the option to mix in explicit files in the same run.
+
+### Command Output
+
+`--cmd` runs a shell command after files are collected and embeds the output in the document. See [Command Attachments](#command-attachments).
+
+## Saved Scopes
+
+Scopes are named, reusable context selections you configure once in `prodex.json` and run any time.
+
+A scope can collect files in two ways:
+
+- **By path**: select files and folders you know by name.
+- **By search**: select files by what they contain.
 
 ```bash
-npm install -g prodex
+prodex scope -k dashboard        # run one scope
+prodex scope -k auth,billing     # run multiple
+prodex scope --all               # run all configured scopes
+prodex scope --list              # show available scope keys
 ```
 
-Or run it ad hoc:
+### File scope example
 
-```bash
-npx prodex run --entry src/index.ts
+Selects files by path, with optional dependency tracing from `entry` files:
+
+```json
+{
+  "version": 5,
+  "scopes": {
+    "dashboard": {
+      "entry": ["resources/js/pages/Dashboard.tsx"],
+      "include": ["routes/web.php"],
+      "exclude": ["resources/js/components/ui"]
+    }
+  }
+}
 ```
 
-Create a starter config:
+`entry` traces imports. `include` adds files directly. Both are optional — a scope can use either or both.
 
-```bash
-prodex init
+### Search scope example
+
+Selects files by what they contain:
+
+```json
+{
+  "version": 5,
+  "scopes": {
+    "payments": {
+      "grep": {
+        "query": "StripeWebhookHandler",
+        "within": ["app", "tests"],
+        "skip": ["vendor"]
+      }
+    }
+  }
+}
 ```
 
-## Common Commands
+Search scope options under `grep`:
+- `query`: fixed-string match
+- `any`: list — files containing any term
+- `all`: list — files containing all terms
+- `regex`: regular expression
+- `not`: filter out files whose contents contain these terms
+- `within`: search only these paths
+- `skip`: skip these paths
 
-```bash
-prodex run [root] --entry src/index.ts
-prodex run [root] --entry routes/web.php --include "**/*.d.ts"
-prodex run [root] --profile dashboard
-prodex run [root] --profile dashboard,api
-prodex run [root] --all
-prodex profiles [root]
-prodex migrate [root]
+A scope can collect by path using `entry` and/or `include`, or by search using `grep`. A search scope may also use `include` for extra files, but it cannot define `entry`.
+
+### Scope keys and names
+
+The scope key is the identifier used in CLI selection. The optional `name` field sets the output filename:
+
+```json
+"scopes": {
+  "dashboard": {
+    "name": "frontend-dashboard",
+    "entry": ["resources/js/pages/Dashboard.tsx"]
+  }
+}
 ```
 
-`prodex run` requires the `run` command. Root-only positional command shortcuts are intentionally not supported.
+```bash
+prodex scope -k dashboard   # output filename: frontend-dashboard
+```
 
-## CLI Reference
+If `name` is omitted, the scope key is used as the output filename.
 
-| Flag | Short | Type | Description |
-| --- | --- | --- | --- |
-| `--entry` | `-e` | list | Entry file or glob to trace. Repeatable and comma-aware. |
-| `--include` | `-i` | list | Extra file or glob to add without dependency tracing. Repeatable and comma-aware. |
-| `--exclude` | `-x` | list | File or glob to skip during traversal. Repeatable and comma-aware. |
-| `--profile` | `-p` | list | Named profile to run. Comma-aware and repeatable. |
-| `--all` | `-a` | boolean | Run every configured profile. `--all-profiles` remains available as an explicit alias. |
-| `--name` | `-n` | string | Output basename for this run. |
-| `--format` | `-F` | `md`/`txt` | Output format. Markdown is the default. |
-| `--max-depth` |  | number | Maximum dependency traversal depth. |
-| `--max-files` |  | number | Maximum traced file count. |
+## Command Attachments
 
-Global metadata flags:
+Add `--cmd` to any run to capture shell command output and embed it in the document alongside the collected files.
 
 ```bash
-prodex --version
-prodex run --help
-prodex profiles --help
-prodex migrate --help
+prodex scope -k auth --cmd "npm test -- auth"
+prodex git --against origin/main --cmd "npm run lint"
+prodex pack -e src/index.ts --cmd "npm run build"
 ```
 
+`--cmd` is repeatable. Commands run sequentially after files are collected, and each output is included as a separate section.
+
+By default, a failed command adds a warning but still produces the output file. Use `--fail-on-cmd-error` to exit with an error instead.
+
+Use `--dry-run` to preview which commands would run without executing them.
+
+## Dependency Tracing
+
+`prodex pack --entry` and `prodex trace` follow import statements from an entry file to collect the files it depends on.
+
+**Currently supported:**
+
+- JavaScript and TypeScript: ES module imports, dynamic imports, CommonJS `require`, re-exports
+- TSX and declaration files (`.d.ts`)
+- PHP: static `include`/`require` statements, namespace imports, PSR-4 resolution, some Laravel service binding awareness
+
+**Limitations:**
+
+- Dynamic imports that are computed at runtime may not be followed
+- Framework-injected dependencies (e.g., auto-resolved services, IoC containers) may not be resolved automatically
+- Languages other than JS, TS, TSX, and PHP are not traced — use `--include` or a file scope to add those files directly
+
+When tracing does not follow a relationship, add the file manually with `--include` or in the scope's `include` list. Prodex does not attempt to guess at runtime behaviour.
+
 ## Configuration
 
-Prodex reads `prodex.json` from the project root.
+Prodex reads `prodex.json` from the project root. All fields are optional — Prodex works without a config file.
 
 ```json
 {
-    "version": 4,
-    "$schema": "https://raw.githubusercontent.com/emxhive/prodex/main/schema/prodex.schema.json",
-    "output": {
-        "dir": "prodex",
-        "format": "md",
-        "versioned": true
-    },
-    "entry": ["src/index.ts"],
-    "include": [],
-    "exclude": ["node_modules/**", "vendor/**", "dist/**"],
-    "resolve": {
-        "aliases": {
-            "@": "resources/js"
-        },
-        "maxDepth": 10,
-        "maxFiles": 200
-    },
-    "profiles": {}
+  "version": 5,
+  "$schema": "https://raw.githubusercontent.com/emxhive/prodex/main/schema/prodex.schema.json",
+  "output": {
+    "dir": "prodex",
+    "format": "md",
+    "versioned": true
+  },
+  "exclude": ["node_modules", "vendor", "dist"],
+  "aliases": {
+    "@": "resources/js"
+  },
+  "depth": 2,
+  "scopes": {}
 }
 ```
 
-Naming precedence:
+`exclude`, `include`, and similar path fields accept file paths, folder names, or glob patterns.
 
-1. `--name`
-2. `profile.name`
-3. Profile key, when running a named profile
-4. Automatic name from entries
+**Naming precedence** (highest to lowest):
+
+1. `--name` flag
+2. `scope.name` in config
+3. Scope key, when running a configured scope
+4. Automatic name derived from entries
 5. Internal fallback: `combined`
 
-CLI flags override config values for a run. Profile arrays replace base arrays for that profile run.
+CLI flags override config values for a run. Excludes are additive: root excludes, scope excludes, and CLI excludes are all applied together.
 
-## Migrating Configs
+## CLI Reference
+
+### `git` — Snapshot Git changes
+
+```bash
+prodex git [root] --against <base>
+prodex git [root] --changed
+prodex git [root] --staged
+prodex git [root] --unstaged
+prodex git [root] --untracked
+prodex git [root] --commit <rev>
+prodex git [root] --range <base..head>
+prodex git [root] --range <base...head>
+```
+
+| Flag | Type | Description |
+|---|---|---|
+| `--changed` | boolean | Staged, unstaged, and untracked files (default) |
+| `--staged` | boolean | Staged files only |
+| `--unstaged` | boolean | Unstaged files only |
+| `--untracked` | boolean | Untracked files only |
+| `--commit <rev>` | string | Files changed by one commit; contents from that commit |
+| `--range <spec>` | string | Files changed between two refs; contents from head |
+| `--against <base>` | string | Files changed between merge-base and HEAD |
+| `--include-diff` | boolean | Include full diff output in the document |
+| `--include` / `-i` | list | Extra files to add |
+| `--exclude` / `-x` | list | Files or paths to skip |
+| `--name` / `-n` | string | Output filename |
+| `--format` / `-F` | `md`/`txt` | Output format |
+| `--cmd` | list | Command to run and embed. Repeatable. |
+| `--fail-on-cmd-error` | boolean | Exit nonzero if an attached command fails |
+| `--dry-run` | boolean | Preview without writing output |
+
+### `grep` — Collect files by search
+
+```bash
+prodex grep [root] --query <text>
+prodex grep [root] --any <term,term,...>
+prodex grep [root] --all <term,term,...>
+prodex grep [root] --regex <pattern>
+```
 
-Prodex requires config version 4. If a project has an older `prodex.json`, `prodex run` and `prodex profiles` fail with migration instructions instead of guessing.
+| Flag | Short | Type | Description |
+|---|---|---|---|
+| `--query` | `-q` | string | Fixed-string search |
+| `--any` | | list | Files containing any of the terms (OR) |
+| `--all` | | list | Files containing all of the terms (AND) |
+| `--regex` | `-r` | string | Regular expression search |
+| `--not` | | list | Filter out files whose contents contain these terms |
+| `--within` | | list | Search only inside these paths |
+| `--skip` | | list | Skip these paths during search |
+| `--include` | `-i` | list | Add extra files |
+| `--exclude` | `-x` | list | Skip these files |
+| `--name` | `-n` | string | Output filename |
+| `--format` | `-F` | `md`/`txt` | Output format |
+| `--cmd` | | list | Command to run and embed. Repeatable. |
+| `--fail-on-cmd-error` | | boolean | Exit nonzero if an attached command fails |
+| `--dry-run` | | boolean | Preview without writing output |
+
+### `scope` — Run saved scopes
 
-Preview a migration:
+```bash
+prodex scope [root] -k <key>
+prodex scope [root] -k <key,key,...>
+prodex scope [root] --all
+prodex scope [root] --list
+```
+
+| Flag | Short | Type | Description |
+|---|---|---|---|
+| `--key` | `-k` | list | Scope key to run. Comma-aware and repeatable. |
+| `--all` | `-a` | boolean | Run all configured scopes |
+| `--list` | | boolean | List configured scope keys |
+| `--exclude` | `-x` | list | Additional paths to skip |
+| `--format` | `-F` | `md`/`txt` | Output format |
+| `--cmd` | | list | Command to run and embed. Repeatable. |
+| `--fail-on-cmd-error` | | boolean | Exit nonzero if an attached command fails |
+| `--dry-run` | | boolean | Preview without writing output |
+
+### `pack` — Collect explicit files or combine sources
 
 ```bash
-prodex migrate
+prodex pack [root] -i <glob>
+prodex pack [root] -e <entry> -i <glob>
 ```
 
-Check whether migration is needed:
+| Flag | Short | Type | Description |
+|---|---|---|---|
+| `--entry` | `-e` | list | Entry file to trace imports from. Repeatable and comma-aware. |
+| `--include` | `-i` | list | File or glob to add without tracing. Repeatable and comma-aware. |
+| `--exclude` | `-x` | list | File or path to skip. |
+| `--scope` | `-s` | list | Merge a configured scope's files into this pack. |
+| `--name` | `-n` | string | Output filename |
+| `--format` | `-F` | `md`/`txt` | Output format |
+| `--depth` | | number | Maximum tracing depth |
+| `--cmd` | | list | Command to run and embed. Repeatable. |
+| `--fail-on-cmd-error` | | boolean | Exit nonzero if an attached command fails |
+| `--dry-run` | | boolean | Preview without writing output |
+
+### `trace` — Trace imports from an entry file
 
 ```bash
-prodex migrate --check
+prodex trace [root] -t <file>
 ```
 
-Write the migration:
+| Flag | Short | Type | Description |
+|---|---|---|---|
+| `--target` | `-t` | list | File or module name to trace from. Accepts exact paths, extensionless names, or bare module names — not globs. Repeatable and comma-aware. |
+| `--include` | `-i` | list | Extra files to add |
+| `--exclude` | `-x` | list | Files or paths to skip |
+| `--name` | `-n` | string | Output filename |
+| `--format` | `-F` | `md`/`txt` | Output format |
+| `--depth` | | number | Maximum tracing depth |
+| `--cmd` | | list | Command to run and embed. Repeatable. |
+| `--fail-on-cmd-error` | | boolean | Exit nonzero if an attached command fails |
+| `--dry-run` | | boolean | Preview without writing output |
+
+### Global flags
 
 ```bash
-prodex migrate --write
+prodex --version
+prodex --help
+prodex <command> --help
 ```
 
-`--write` creates a backup before replacing `prodex.json`.
+## Migrating Configs
+
+Prodex requires config version 5. If your project has an older `prodex.json`, Prodex will fail with a prompt to migrate.
+
+```bash
+prodex migrate           # preview the migration
+prodex migrate --check   # exit nonzero if migration is needed
+prodex migrate --write   # apply the migration (creates a backup first)
+```
 
 ## Requirements
 
 - Node.js 22+
-- A project with JS, TS, TSX, declaration-file, or PHP entrypoints for dependency tracing
-- Any file type can be added through include-only extraction
-- Optional `prodex.json` for saved defaults and profiles
+- `prodex.json` is optional; Prodex works without one
+- Dependency tracing requires JS, TS, TSX, or PHP entrypoints
+- Any file type can be collected with `--include`, `prodex grep`, or a saved scope
+- `prodex grep` and search scopes require [ripgrep](https://github.com/BurntSushi/ripgrep) (`rg`) to be installed and available on your PATH
 
 ## License
 

package/dist/app/dependency-source-provider.d.ts

@@ -0,0 +1,2 @@
+import type { ExecutionPlan, SourceCollectionResult } from "../types";
+export declare function collectDependencySources(plan: ExecutionPlan): Promise<SourceCollectionResult>;