copilot-usage-studio 0.2.0 → 0.2.2

package/dist/copilot-usage-studio/browser/styles-7RDDQXQV.css

@@ -0,0 +1 @@
+*{box-sizing:border-box}html,body{margin:0;min-height:100%;font-family:Aptos,Segoe UI Variable,Segoe UI,Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,sans-serif;font-size:14px;line-height:1.45;background:#090d14;color:#d7dde7;text-rendering:optimizeLegibility;-webkit-font-smoothing:antialiased}html[data-theme=light],html[data-theme=light] body{background:#f4f7fb;color:#182536}html[data-theme=dark],html[data-theme=dark] body{background:#090d14;color:#d7dde7}button,input,select{font:inherit}code{border:1px solid color-mix(in srgb,var(--accent, #6f5cff) 18%,var(--line, #d7e1ea));border-radius:5px;padding:1px 5px;background:color-mix(in srgb,var(--accent, #6f5cff) 8%,transparent);color:var(--text-strong, #182536);font-family:ui-monospace,SFMono-Regular,Consolas,Liberation Mono,monospace;font-size:.92em}.data-refresh{display:grid;justify-items:end;min-width:0}.data-refresh-button{display:inline-flex;align-items:center;justify-content:center;gap:6px;min-height:32px;border:1px solid var(--line);border-radius:9px;padding:5px 10px;background:var(--control-bg);color:var(--text-strong);font-size:12px;font-weight:760;cursor:pointer;box-shadow:var(--shadow-soft)}.data-refresh-button:hover:not(:disabled){border-color:var(--accent);color:var(--accent)}.data-refresh-button:disabled{cursor:progress;opacity:.72}.refresh-icon{display:grid;place-items:center;width:16px;height:16px;font-size:17px;line-height:1;transform-origin:50% 50%}.refresh-icon.spinning{animation:refresh-spin .8s linear infinite}.data-refresh-status{max-width:190px;overflow:hidden;color:var(--muted-2);font-size:10px;line-height:1.2;text-overflow:ellipsis;white-space:nowrap}.data-refresh.error .data-refresh-status{color:var(--danger-text)}.host-vscode .data-refresh-status{display:none}@keyframes refresh-spin{to{transform:rotate(360deg)}}@media(max-width:760px){.data-refresh-status{display:none}}@media(max-width:520px){.data-refresh-button>span:last-child{display:none}.data-refresh-button{padding-inline:5px}}

package/docs/local-deployment.md

@@ -2,9 +2,9 @@
 
 The app is intended to run locally, near the VS Code data it reads. It should not require a hosted SaaS backend for the core workflow.
 
-Current release posture: early local developer preview. Version `0.1.0` is published publicly on npm, while clone-and-run remains available for contributors and development builds.
+Current release posture: the VS Code Marketplace extension is the product path. The npm package remains a development and fallback host.
 
-Packaging foundation now available: the scanner exposes an in-memory Node API through `lib/scanner-api.mjs`. The existing scan command uses that same API and only adds argument parsing plus JSON persistence. This is the common core for the next local host, an `npx` command, Electron packaging, and a future thin VS Code extension.
+Packaging foundation now available: the scanner exposes an in-memory Node API through `lib/scanner-api.mjs`. The existing scan command, local runtime, npm host, and VS Code extension preview use the same scanner/runtime core so behavior stays aligned across hosts.
 
 Supported source today: VS Code GitHub Copilot Chat and Agent local storage. Visual Studio, JetBrains IDEs, Copilot CLI, GitHub.com chat, and GitHub billing exports are outside the current importer scope.
 
@@ -76,7 +76,7 @@ npm run test:scripts
 npm run build
 ```
 
-Release copy should describe the app as a local Copilot usage inspector for VS Code. The strongest promise is: "VS Code already logs useful usage data locally; this app makes it understandable for developers who do not have billing-console access."
+Release copy should describe the app as a local Copilot usage inspector for VS Code, now available as a local VSIX preview. The strongest promise is: "VS Code already logs useful usage data locally; this app makes it understandable for developers who do not have billing-console access."
 
 ### Packaged Data Location
 
@@ -92,13 +92,24 @@ The npm tarball contains the compiled UI, scanner, pricing table, and runtime co
 
 GitHub Actions is the release control plane:
 
-- `.github/workflows/ci.yml` runs the full release gate for pull requests and pushes to `main`.
+- `.github/workflows/ci.yml` runs the full release gate and packages a VSIX artifact for every pushed branch, plus pull requests to `main`.
 - `.github/workflows/release.yml` runs automatically for version tags such as `v0.1.1`, with a manual existing-tag mode for repair and backfill.
-- The release workflow verifies that the tag matches `package.json`, publishes the exact tagged source, and creates the matching GitHub Release.
+- The release workflow verifies that the tag matches both package versions, runs the release gate, packages one VSIX, publishes that artifact to the VS Code Marketplace, and attaches the exact VSIX to the matching GitHub Release.
+- Marketplace publication uses GitHub OIDC and Microsoft Entra workload identity federation. No Marketplace token or client secret is stored in GitHub.
 - A failed workflow can be rerun safely: an existing npm version is accepted only when its published `gitHead` matches the exact tagged commit. Conflicting or unverifiable versions are refused.
-- New versions must pass the full release gate before publication. An exact-commit backfill of an already-published historical version skips its old test suite and only repairs the missing GitHub Release.
+- Every release and repair run passes the current release gate before publication.
 
-This keeps GitHub, npm, and the source tag tied to the same commit. Ordinary pushes never publish.
+This keeps GitHub, release artifacts, release notes, and the source tag tied to the same commit. Ordinary pushes never publish.
+
+### Branch Validation
+
+For feature work, push the branch and let CI run:
+
+```bash
+git push -u origin your-branch
+```
+
+The CI run proves the npm app and VS Code extension package from that branch. Download the `copilot-usage-studio-vsix` artifact from the workflow run when you want to test exactly what GitHub built. For a solo-maintainer flow, a pull request is useful for review and history, but branch CI is enough for a pre-merge smoke test.
 
 ### One-Time npm Setup
 
@@ -109,14 +120,34 @@ Commit and push the workflow files first, then configure npm Trusted Publishing
 3. Set repository to `copilot-usage-studio`.
 4. Set workflow filename to `release.yml`.
 5. Allow the `npm publish` action.
-6. Leave the environment blank unless the workflow is later changed to use a protected GitHub environment.
+6. Set the trusted-publisher environment to `marketplace`, matching the protected environment used by the release job.
 
 The workflow uses GitHub's OIDC identity, so no long-lived `NPM_TOKEN` repository secret is required. npm automatically adds provenance for a public package published from a public repository through Trusted Publishing.
 
+### One-Time VS Code Marketplace Setup
+
+1. In Microsoft Entra ID, create an app registration for Marketplace publishing. Record its application (client) ID and directory (tenant) ID. Do not create a client secret.
+2. Add a federated credential to that app registration using the GitHub Actions scenario:
+   - organization: `fortunes-guardian`
+   - repository: `copilot-usage-studio`
+   - entity type: **Environment**
+   - environment: `marketplace`
+3. In GitHub, create an environment named `marketplace`. Restrict deployment to protected version tags when practical.
+4. Add environment variables `AZURE_CLIENT_ID` and `AZURE_TENANT_ID`. These identifiers are not credentials; GitHub's short-lived OIDC assertion is the credential.
+5. After this workflow is merged to `main`, run **Marketplace Identity Check** from GitHub Actions.
+6. Copy the profile ID from the workflow summary. In the Visual Studio Marketplace publisher management page, add that identity ID as a member of publisher `fortunes-guardian` with the **Contributor** role.
+7. Rerun **Marketplace Identity Check** if publisher membership or federation changes.
+
+The release job uses `azure/login` with GitHub OIDC, then publishes with `vsce --azure-credential`. No PAT, client secret, or certificate is created. The workflow publishes the already packaged `tmp/*.vsix`, and that exact file is uploaded to the GitHub Release.
+
+References: [VS Code secure automated publishing](https://code.visualstudio.com/api/working-with-extensions/publishing-extension#secure-automated-publishing-to-visual-studio-marketplace) and [GitHub OIDC with Azure](https://docs.github.com/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure).
+
 ### Publishing a Version
 
 Start from an up-to-date, clean `main` branch. Choose the semantic-version bump that matches the change:
 
+First, make sure `CHANGELOG.md` has a useful human summary under `Unreleased` or under the exact version heading you are about to publish. The GitHub Release body is generated from that curated changelog text, with commits included only as an audit trail.
+
 ```bash
 git switch main
 git pull --ff-only
@@ -125,18 +156,21 @@ npm version patch
 git push origin main --follow-tags
 ```
 
-`npm version patch` updates both `package.json` and `package-lock.json`, creates a version commit, and creates an annotated `vX.Y.Z` tag. Use `npm version minor` for a backward-compatible feature release or `npm version major` for a breaking release.
+`npm version patch` updates `package.json` and `package-lock.json`; the repository's `version` lifecycle hook synchronizes `vscode-extension/package.json` before npm creates the version commit and annotated `vX.Y.Z` tag. Use `npm version minor` for a backward-compatible feature release or `npm version major` for a breaking release.
 
 After the tag is pushed:
 
 1. Watch the **Release** workflow in GitHub Actions.
-2. Confirm the new version appears on npm.
-3. Confirm GitHub created a Release for the same tag.
-4. Run `npx copilot-usage-studio@X.Y.Z --version` from a clean directory as a final smoke test.
+2. Confirm the new version appears in the VS Code Marketplace.
+3. Confirm the new version appears on npm.
+4. Confirm GitHub created a Release for the same tag and attached the same-version VSIX.
+5. Confirm the release notes read well and include the curated highlights.
+6. Install or update from the Marketplace and run one final extension smoke test.
+7. Run `npx copilot-usage-studio@X.Y.Z --version` from a clean directory as a fallback-host smoke test.
 
 If CI or release validation fails, fix the issue on `main` and publish a new version. Do not move or reuse a public release tag, and do not overwrite an npm version; both are immutable release coordinates.
 
-The workflow also has a manual **Run workflow** action with a required existing tag. Use this to repair or backfill the GitHub Release for an exact tag. For the already-published first version, ensure `v0.1.0` exists on GitHub, then run the workflow once with `v0.1.0`; it will verify npm's published `gitHead`, skip the historical release gate and npm publication, and create the missing GitHub Release.
+The workflow also has a manual **Run workflow** action with a required existing tag. Use it to repair a release from that immutable tag. Duplicate npm and Marketplace publication is skipped safely, while the release gate reruns and the GitHub Release notes/artifact are repaired.
 
 ### Manual Emergency Fallback
 
@@ -176,7 +210,7 @@ Then serve the generated `dist/` output with a small local static server.
 
 Why: Angular production output is just static assets plus the generated `public/data/sessions.json` copy. This is a good first packaging target because it avoids an app server and keeps the data model simple.
 
-Current build status: `npm run build` passes. There is still an initial bundle budget warning to clean up later; it does not block the local preview release.
+Current build status: `npm run build` passes. There are still Angular budget warnings for the initial bundle and `src/app/app.css`; they do not block the local preview release.
 
 For day-to-day development, use the in-app **Refresh** action. `npm run refresh:data` remains the explicit command-line scan plus verification path.
 
@@ -190,13 +224,38 @@ npm run runtime
 
 It listens on `http://127.0.0.1:4312/` by default and exposes:
 
-- `GET /api/status`: scan state, timestamps, last error, and session count
+- `GET /api/status`: scan state, timestamps, current progress, recent log lines, last error, and session count
+- `GET /api/logs`: bounded in-memory runtime logs and the local log-file path
 - `GET /api/sessions`: the current normalized in-memory `SessionData`
 - `POST /api/scan`: run a scan, persist the new snapshot, and return it
 - `GET /data/sessions.json`: compatibility route backed by the current in-memory snapshot
 
 The runtime seeds itself from `public/data/sessions.json` when needed, then keeps its live cache under `tmp/local-runtime/`. This avoids triggering an Angular development rebuild every time the user refreshes data. It serves cached data while a scan is running, and a failed scan does not replace the last valid snapshot.
 
+Runtime scans run in a child process. This keeps `/api/status` and `/api/logs` responsive even when a large VS Code profile is being imported or a workspace has unusually heavy local data.
+
+### Startup Diagnostics
+
+If the app appears stuck on startup, there are three places to inspect:
+
+1. The app loading screen shows the current scan step, recent runtime log messages, cached session count, and the runtime log-file path.
+2. The terminal running `npm start` or `npx copilot-usage-studio` prints scan progress such as discovered workspace folders and debug-log folder counts.
+3. The status endpoint can be queried directly:
+
+```bash
+npx copilot-usage-studio status
+```
+
+For packaged `npx` runs, the runtime log file is stored beside the local session cache:
+
+- Windows: `%LOCALAPPDATA%\Copilot Usage Studio\runtime.log`
+- macOS: `~/Library/Application Support/Copilot Usage Studio/runtime.log`
+- Linux: `${XDG_DATA_HOME:-~/.local/share}/copilot-usage-studio/runtime.log`
+
+For repository development with `npm start`, the log is written to `tmp/local-runtime/runtime.log`.
+
+The scanner is intentionally bounded. It scans known VS Code Copilot storage locations plus targeted `.github/instructions`, `.github/skills`, `.github/prompts`, and `.github/hooks` folders for the workspace. Recursive Markdown scans have depth and directory-count caps and skip common dependency/build folders such as `node_modules`, `.git`, `dist`, `build`, `target`, and virtual environments. It should not crawl arbitrary project directories.
+
 To build the production UI and serve the complete local app with one command:
 
 ```bash
@@ -233,9 +292,25 @@ Why: this keeps the project local and scriptable without committing to a desktop
 
 The shared scanner API, runtime, and npm executable are complete. The same runtime can be embedded by Electron later.
 
-## Future Option: VS Code Companion Extension
+## VS Code Companion Extension Preview
+
+A thin VS Code extension now calls the shared scanner from the local extension host and shows the existing UI in a webview. It remains a host, not a fork of the pricing and parsing logic.
+
+Build a local preview VSIX:
+
+```bash
+npm run vscode:package
+```
+
+Install it locally:
+
+```bash
+code --install-extension tmp/copilot-usage-studio-vscode-X.Y.Z.vsix --force
+```
+
+The extension exposes the full app inside VS Code: Usage, Sessions, Memory, Customizations preview, Compare, Insights, and Prices. The npm/browser host remains useful for development and fallback testing, but the extension is the primary product surface.
 
-A thin VS Code extension can call the shared scanner from the local extension host and show the existing UI in a webview. It should remain a host, not a fork of the pricing and parsing logic.
+The VSIX is a separate distribution artifact from the npm package. It is built from the same source tree. After Marketplace release is configured, the VS Code Marketplace should be the normal install route; `npx copilot-usage-studio` remains a development/fallback host.
 
 Why not move the whole product into an extension: keeping the scanner and analysis core editor-independent preserves a path to a desktop app and possible Visual Studio support if Visual Studio exposes equivalent local evidence later.
 

package/docs/pricing.md

@@ -17,10 +17,10 @@ https://docs.github.com/en/copilot/concepts/billing/usage-based-billing-for-orga
 Current version:
 
 ```text
-github-copilot-usage-pricing-2026-06-14
+github-copilot-usage-pricing-2026-07-05
 ```
 
-The source table says prices are per 1 million tokens. The committed snapshot was checked against GitHub Docs on June 14, 2026.
+The source table says prices are per 1 million tokens. The committed snapshot was checked against GitHub Docs on July 5, 2026.
 
 ## Where Pricing Lives
 
@@ -40,7 +40,7 @@ This should be a deliberate maintainer update, not a runtime web fetch.
 
 1. Open GitHub's official model pricing page and record the check date.
 2. Update `data/github-copilot-pricing.json`, including model availability, all token buckets, and request-size tiers.
-3. Keep tier thresholds machine-readable. Apply them per model call using normal plus cached input, never aggregated session tokens.
+3. Keep tier thresholds and raw-model aliases machine-readable. Apply tiers per model call using normal plus cached input, never aggregated session tokens.
 4. Update the pricing version and snapshot date.
 5. Add or update threshold boundary tests and verify that several smaller calls do not accidentally trigger a long-context tier.
 6. Run `npm run refresh:data` to recalculate the local development snapshot.
@@ -169,6 +169,8 @@ The user should be able to inspect the cost inputs directly. If a session looks
 - cached input or cache-write accounting
 - a model fallback because the raw model id was not in the known GitHub table
 
+Some VS Code logs can contain internal raw model ids that are not published as GitHub pricing rows. For example, a raw id such as `oswe-vscode-prime` stays visible in the UI and is estimated with the configured fallback pricing row. The app should not silently rename that id unless a reliable pricing alias is added.
+
 The `GitHub prices` view therefore shows every rate row the app knows about, where it came from, and whether any imported session currently uses it.
 
 The same view also shows the selected Business/Enterprise AI-credit allowance and how the current imported sessions compare with it.

package/docs/scanner-api.md

@@ -42,23 +42,63 @@ This is the intended path for:
 const sessionData = await scanVsCodeSessions({
   roots: ['/custom/Code/User'],
   sqlite: true,
+  includeCustomizations: true,
   generatedAt: new Date(),
+  onProgress: (event) => console.log(event.message),
 });
 ```
 
 - `roots`: optional array of VS Code `User` directories or individual workspace-storage directories. Defaults to stable VS Code and VS Code Insiders paths for the current operating system.
 - `sqlite`: set to `false` to skip optional `state.vscdb` metadata enrichment. Debug-log token and usage import still works.
+- `includeCustomizations`: set to `false` to skip instruction/skill/prompt/hook/agent inventory and evidence matching. The VS Code extension normally leaves this enabled because it exposes the Customizations view.
+- `includeSystemCustomizations`: set to `true` to include customizations contributed by installed VS Code extensions or VS Code application extension folders. The default is `false` so startup focuses on user/repo customizations and avoids expensive evidence matching for system-provided skills.
 - `generatedAt`: optional `Date` or date-compatible value, primarily for deterministic hosts and tests.
 - `usdToEur`: legacy generated-contract conversion value. The product UI is USD-first and callers should normally leave this unset.
+- `onProgress`: optional callback for host diagnostics. It receives scan-stage events such as root discovery, workspace discovery, debug-log folder counts, memory indexing, and completion. Hosts should treat these as user-facing diagnostics, not stable analytics data.
 
 Duplicate roots are normalized and scanned once. Each call receives fresh ingestion diagnostics; counters and warnings do not leak from earlier scans in a long-running process.
 
 Only one scan may run at a time within one Node process. Hosts should debounce filesystem events and await the current scan instead of starting overlapping work.
 
+## Internal Scanner Modules
+
+The public host boundary remains `lib/scanner-api.mjs`, but the scanner implementation is being decomposed so large-profile performance issues are easier to diagnose.
+
+Current internal split:
+
+- `scripts/scanner-traversal.mjs`: VS Code user-data root discovery, workspace-storage entry discovery, bounded recursive file traversal, dependency/build-folder skips, and root normalization.
+- `scripts/scanner-workspace.mjs`: per-storage-entry orchestration for state DB enrichment, customization inventory/evidence, debug-log imports, chat-snapshot imports, memory imports, progress events, and per-entry diagnostics.
+- `scripts/scanner-customization-inventory.mjs`: bounded discovery of user, workspace, repo, and configured Copilot customization files without crawling arbitrary repositories.
+- `scripts/scanner-customization-evidence.mjs`: request-payload and side-file text matching for whether customization content is visible in local model-request logs.
+- `scripts/scanner-memory.mjs`: Copilot memory/plan Markdown import plus memory recall linking from debug-log tool events.
+- `scripts/scanner-session-parser.mjs`: Agent Debug Log and chat-snapshot session parsing, token/cache fields, model breakdowns, request payload summaries, trace events, source usage, and fallback chat estimates.
+- `scripts/scan-vscode-sessions.mjs`: scan lifecycle, diagnostics lifetime, SQLite enrichment, workspace aggregation, final normalized session-data assembly, CLI argument parsing, and JSON persistence.
+
+Traversal is intentionally separate because broad filesystem walking is the highest-risk performance path. The scanner should scan known VS Code Copilot storage plus targeted Copilot customization/memory roots; it must not crawl arbitrary repositories or home directories.
+
+Next decomposition targets:
+
+- pricing/token normalization
+- SQLite/state enrichment helpers
+
+## Incremental scans
+
+The runtime passes the previous snapshot's `generatedAt` value as `incrementalSince` during a normal quick refresh. The scanner still discovers known VS Code storage entries, but it parses only debug-session `main.jsonl` files and chat snapshots modified after that cursor. The runtime merges changed sessions by stable session ID and retains unchanged sessions.
+
+This provides two explicit modes: quick/incremental for normal startup, visibility refresh, and `Refresh Data`; full for recovery/debugging through `Copilot Usage Studio: Full Rescan`.
+
+Incremental scans intentionally do not infer deletions. A full rescan rebuilds the snapshot and is the recovery path for deleted or externally rewritten sources. Memory roots remain indexed during quick scans because memory files are small and can be independently added or removed.
+
+Customization analysis has its own cache boundary. Inventory records include a SHA-256 content fingerprint. If all current fingerprints match the previous analysis, only changed session logs are checked and prior evidence is retained. Any customization content change invalidates that shortcut.
+
+The current delta cursor is timestamp-based because VS Code does not publish a stable scanner API or change journal for Agent Debug Logs. The full-rescan command exists specifically to recover from timestamp anomalies or external file restoration.
+
 ## Local Runtime Host
 
 `lib/local-runtime.mjs` is the first shared API host. It keeps the last valid `SessionData` snapshot in memory, serves it while scans run, and persists successful refreshes.
 
+The local runtime runs the scanner in a child process by default. This keeps the HTTP status/log endpoints responsive while large local VS Code profiles are being imported.
+
 The runtime intentionally keeps scanner and transport concerns separate:
 
 - the scanner knows how to read and normalize VS Code data

package/lib/cli.mjs

@@ -98,7 +98,21 @@ export async function runCli(args = process.argv.slice(2), dependencies = {}) {
   }
   if (options.command === 'status') {
     const fetchStatus = dependencies.fetch ?? globalThis.fetch;
-    const response = await fetchStatus(`http://${options.host}:${options.port}/api/status`);
+    const timeoutMs = Number(dependencies.statusTimeoutMs ?? 5000);
+    const signal = Number.isFinite(timeoutMs) && timeoutMs > 0 && typeof AbortSignal !== 'undefined'
+      ? AbortSignal.timeout(timeoutMs)
+      : undefined;
+    let response;
+    try {
+      response = await fetchStatus(`http://${options.host}:${options.port}/api/status`, { signal });
+    } catch (error) {
+      const timedOut = error instanceof Error && error.name === 'TimeoutError';
+      throw new Error(
+        timedOut
+          ? `Local runtime did not respond on ${options.host}:${options.port} within ${timeoutMs}ms.`
+          : `Could not reach local runtime on ${options.host}:${options.port}.`,
+      );
+    }
     if (!response.ok) {
       throw new Error(`Local runtime returned ${response.status}.`);
     }