copilot-usage-studio 0.1.0 → 0.2.0

package/docs/copilot-memory.md

@@ -0,0 +1,91 @@
+# Copilot Memory
+
+Copilot Usage Studio indexes locally saved GitHub Copilot memories and plans so a developer can see what Copilot has stored across VS Code workspaces.
+
+## Observed storage
+
+Workspace-scoped files:
+
+```text
+<VS Code User>/workspaceStorage/<workspace-id>/GitHub.copilot-chat/memory-tool/memories/
+```
+
+Global files:
+
+```text
+<VS Code User>/globalStorage/github.copilot-chat/memory-tool/memories/
+```
+
+Observed workspace layouts include:
+
+```text
+memories/repo/<topic>.md
+memories/<base64-session-id>/plan.md
+memories/<base64-session-id>/<research>.md
+```
+
+The base64 directory decodes to the same session UUID used by Agent Debug Logs and transcripts. This gives the app a source-backed link between a saved plan or research note and the run that created it.
+
+## Normalized record
+
+Each indexed Markdown file records:
+
+- stable local id derived from its absolute path
+- `kind`: `memory` or `plan`
+- `scope`: `global`, `repository`, `session`, or `workspace`
+- title, excerpt, and full Markdown content
+- workspace name when applicable
+- decoded session id when applicable
+- absolute and relative source paths
+- created and modified timestamps
+- byte, character, and line counts
+
+The scanner reads Markdown files once during the normal local scan. It skips files larger than 1 MiB and caps each memory root at 5,000 files. These limits protect refresh performance without affecting normal observed memory stores, whose files are small.
+
+## Product boundary
+
+Version one is a read-only knowledge library:
+
+- search all saved memory content
+- filter by plan/memory, scope, and workspace
+- read the Markdown source
+- link session-scoped files to imported runs
+- show observed recall frequency, time, session, returned content size, and the following model request
+- open the file or reveal it through the local runtime
+
+The browser does not receive arbitrary filesystem access. Open/reveal actions identify a memory by its scanner-generated id, and the runtime accepts only files present in the current scanned snapshot.
+
+## Observed recall evidence
+
+Agent Debug Logs provide a useful distinction between a memory being available and actually being recalled:
+
+- VS Code injects a memory inventory into model input. The observed inventory names available user, session, and repository memory files, but does not include every file's full contents.
+- When the agent calls the `memory` tool with `command: "view"`, the tool event records the requested memory path and returned content.
+- That tool result is then present in the following model request. The app can therefore say that a specific memory was loaded before a specific model call.
+- The following model call exposes exact total input, cached input, output, and source usage when those fields are present. It does not expose a separate token total or cost for the memory text alone.
+
+In the June 15, 2026 local log sample, 46 JSONL files contained 127 model requests with a memory inventory and 18 explicit memory-tool operations across 11 sessions. Fifteen operations were reads, including repeated reads of the repository memory. These counts establish that recall is observable, but they are only a local sample and not a VS Code API contract.
+
+The recall view shows:
+
+- which memory path was read
+- when and in which session it was read
+- the returned content size
+- the next model call and that call's total token/usage facts
+
+It must not label the entire following request as memory cost or derive an exact memory-only token charge from character count.
+
+## What the app does not claim
+
+A saved memory is evidence that Copilot wrote a local file. A matching `memory view` tool event is evidence that Copilot later recalled it and included the returned content in the request chain.
+
+Therefore the first version does not label memories as useful, stale, harmful, or token-consuming. Those conclusions require a matching Agent Debug Log or request side-file showing that the memory was included in a model request.
+
+Future evidence-backed opportunities:
+
+- flag conflicting or old memories for review
+- compare request input before and after a memory is edited or removed
+- help turn a proven durable memory into repository instructions
+- map repository memories to code/domain areas when paths or content provide a reliable relationship
+
+Editing and deletion remain later work. The files are technically accessible, but VS Code's indexing and lifecycle contract is not documented strongly enough to make destructive management a first-release feature.

package/docs/local-deployment.md

@@ -2,7 +2,7 @@
 
 The app is intended to run locally, near the VS Code data it reads. It should not require a hosted SaaS backend for the core workflow.
 
-Current release posture: early local developer preview. The npm package is prepared and locally verified; clone-and-run remains available until version `0.1.0` is published to the registry.
+Current release posture: early local developer preview. Version `0.1.0` is published publicly on npm, while clone-and-run remains available for contributors and development builds.
 
 Packaging foundation now available: the scanner exposes an in-memory Node API through `lib/scanner-api.mjs`. The existing scan command uses that same API and only adds argument parsing plus JSON persistence. This is the common core for the next local host, an `npx` command, Electron packaging, and a future thin VS Code extension.
 
@@ -33,7 +33,7 @@ http://127.0.0.1:4200/
 
 Why: this is the fastest loop while the UI and scanner are changing. `npm start` launches the Angular dev server and the local runtime together. The runtime serves the last valid snapshot immediately, performs a background startup scan, and powers the in-app refresh action.
 
-## First Public Release Path
+## Install From npm
 
 The npm package is configured for a one-command local launch:
 
@@ -41,7 +41,7 @@ The npm package is configured for a one-command local launch:
 npx copilot-usage-studio
 ```
 
-This command will become available after the package owner publishes version `0.1.0` to npm. It starts the packaged production UI and scanner runtime on `http://127.0.0.1:4312/` by default.
+This starts the packaged production UI and scanner runtime on `http://127.0.0.1:4312/` by default. It downloads the latest published npm version rather than the current GitHub `main` branch.
 
 Useful packaged commands:
 
@@ -54,10 +54,10 @@ npx copilot-usage-studio --help
 
 The CLI requires Node.js 22.5 or newer because optional VS Code `state.vscdb` enrichment uses Node's built-in SQLite support.
 
-Before registry publication, use the repository flow:
+For development or the current GitHub source, use the repository flow:
 
 ```bash
-git clone <repo-url>
+git clone https://github.com/fortunes-guardian/copilot-usage-studio.git
 cd copilot-usage-studio
 npm install
 npm start
@@ -76,7 +76,7 @@ npm run test:scripts
 npm run build
 ```
 
-Release copy should describe the app as a local Copilot usage debugger for VS Code. The strongest promise is: "VS Code already logs useful usage data locally; this app makes it understandable for developers who do not have billing-console access."
+Release copy should describe the app as a local Copilot usage inspector for VS Code. The strongest promise is: "VS Code already logs useful usage data locally; this app makes it understandable for developers who do not have billing-console access."
 
 ### Packaged Data Location
 
@@ -88,7 +88,57 @@ The packaged CLI stores its current normalized snapshot outside the npm installa
 
 The npm tarball contains the compiled UI, scanner, pricing table, and runtime code. It must never contain `public/data/sessions.json` or any other developer's imported session data.
 
-### Publishing
+## Automated Releases
+
+GitHub Actions is the release control plane:
+
+- `.github/workflows/ci.yml` runs the full release gate for pull requests and pushes to `main`.
+- `.github/workflows/release.yml` runs automatically for version tags such as `v0.1.1`, with a manual existing-tag mode for repair and backfill.
+- The release workflow verifies that the tag matches `package.json`, publishes the exact tagged source, and creates the matching GitHub Release.
+- A failed workflow can be rerun safely: an existing npm version is accepted only when its published `gitHead` matches the exact tagged commit. Conflicting or unverifiable versions are refused.
+- New versions must pass the full release gate before publication. An exact-commit backfill of an already-published historical version skips its old test suite and only repairs the missing GitHub Release.
+
+This keeps GitHub, npm, and the source tag tied to the same commit. Ordinary pushes never publish.
+
+### One-Time npm Setup
+
+Commit and push the workflow files first, then configure npm Trusted Publishing for the existing `copilot-usage-studio` package:
+
+1. Open the package settings on npm and add a GitHub Actions trusted publisher.
+2. Set organization or user to `fortunes-guardian`.
+3. Set repository to `copilot-usage-studio`.
+4. Set workflow filename to `release.yml`.
+5. Allow the `npm publish` action.
+6. Leave the environment blank unless the workflow is later changed to use a protected GitHub environment.
+
+The workflow uses GitHub's OIDC identity, so no long-lived `NPM_TOKEN` repository secret is required. npm automatically adds provenance for a public package published from a public repository through Trusted Publishing.
+
+### Publishing a Version
+
+Start from an up-to-date, clean `main` branch. Choose the semantic-version bump that matches the change:
+
+```bash
+git switch main
+git pull --ff-only
+npm run release:check
+npm version patch
+git push origin main --follow-tags
+```
+
+`npm version patch` updates both `package.json` and `package-lock.json`, creates a version commit, and creates an annotated `vX.Y.Z` tag. Use `npm version minor` for a backward-compatible feature release or `npm version major` for a breaking release.
+
+After the tag is pushed:
+
+1. Watch the **Release** workflow in GitHub Actions.
+2. Confirm the new version appears on npm.
+3. Confirm GitHub created a Release for the same tag.
+4. Run `npx copilot-usage-studio@X.Y.Z --version` from a clean directory as a final smoke test.
+
+If CI or release validation fails, fix the issue on `main` and publish a new version. Do not move or reuse a public release tag, and do not overwrite an npm version; both are immutable release coordinates.
+
+The workflow also has a manual **Run workflow** action with a required existing tag. Use this to repair or backfill the GitHub Release for an exact tag. For the already-published first version, ensure `v0.1.0` exists on GitHub, then run the workflow once with `v0.1.0`; it will verify npm's published `gitHead`, skip the historical release gate and npm publication, and create the missing GitHub Release.
+
+### Manual Emergency Fallback
 
 Run the full package gate:
 
@@ -102,7 +152,7 @@ Inspect the tarball list, confirm no local session data or absolute paths are pr
 npm publish --access public
 ```
 
-Publishing is an external release action and should not be automated from ordinary development commands.
+Use this only if Trusted Publishing or GitHub Actions is unavailable. The normal path is the tag-driven workflow above.
 
 If the browser shows updated markup with stale component styles, stop the dev server and restart it with a cache reset:
 

package/docs/scanner-api.md

@@ -26,7 +26,7 @@ import { scanVsCodeSessions } from './lib/scanner-api.mjs';
 const sessionData = await scanVsCodeSessions();
 ```
 
-The result is the normalized `SessionData` document consumed by the Angular UI. No file is created or changed.
+The result is the normalized `SessionData` document consumed by the Angular UI. It contains imported `sessions` plus a sibling `memories` collection for observed Copilot Markdown memories and plans. No file is created or changed.
 
 This is the intended path for:
 
@@ -96,7 +96,7 @@ There is one parser and pricing implementation. Future hosts should not copy sca
 
 ## Privacy And Source Rules
 
-The API reads local VS Code Copilot storage and can encounter prompts, file paths, repository context, tool schemas, and tool results. Hosts must keep results local unless a user explicitly exports them.
+The API reads local VS Code Copilot storage and can encounter prompts, file paths, repository context, tool schemas, tool results, saved memories, and saved plans. Hosts must keep results local unless a user explicitly exports them.
 
 The API does not change evidence rules:
 
@@ -105,5 +105,6 @@ The API does not change evidence rules:
 - cached input remains separate from normal input and output
 - chat snapshots remain fallback data
 - SQLite is metadata enrichment, not a billing source
+- saved memory files are indexed as local artifacts; their presence does not prove they were recalled into a model request
 
-See [data-ingestion.md](data-ingestion.md), [debug-log-schema.md](debug-log-schema.md), and [pricing.md](pricing.md) for those contracts.
+See [data-ingestion.md](data-ingestion.md), [copilot-memory.md](copilot-memory.md), [debug-log-schema.md](debug-log-schema.md), and [pricing.md](pricing.md) for those contracts.

package/lib/local-runtime.mjs

@@ -1,6 +1,7 @@
+import { spawn } from 'node:child_process';
 import { createReadStream, existsSync, readFileSync, statSync } from 'node:fs';
 import { createServer } from 'node:http';
-import { extname, join, normalize, resolve, sep } from 'node:path';
+import { dirname, extname, join, normalize, resolve, sep } from 'node:path';
 
 import { scanVsCodeSessions, writeSessionData } from './scanner-api.mjs';
 
@@ -29,7 +30,9 @@ export function createLocalRuntime(options = {}) {
   const scanOptions = options.scanOptions ?? {};
   const scanner = options.scanner ?? scanVsCodeSessions;
   const writer = options.writer ?? writeSessionData;
+  const openPath = options.openPath ?? openLocalPath;
   const logger = options.logger ?? console;
+  const backendOnly = options.backendOnly === true;
 
   let sessionData = readCachedSessionData(dataFile, logger)
     ?? (seedDataFile ? readCachedSessionData(seedDataFile, logger) : null);
@@ -46,6 +49,7 @@ export function createLocalRuntime(options = {}) {
       scanning: Boolean(activeScan),
       hasData: Boolean(sessionData),
       sessionCount: sessionData?.sessions?.length ?? 0,
+      memoryCount: sessionData?.memories?.length ?? 0,
       generatedAt: sessionData?.generatedAt ?? '',
       lastScanStartedAt,
       lastScanCompletedAt,
@@ -123,6 +127,19 @@ export function createLocalRuntime(options = {}) {
         }
       }
 
+      const memoryAction = url.pathname.match(/^\/api\/memories\/([a-f0-9]{24})\/open$/);
+      if (request.method === 'POST' && memoryAction) {
+        const memory = sessionData?.memories?.find((candidate) => candidate.id === memoryAction[1]);
+        if (!memory) {
+          return sendJson(response, 404, { error: 'Memory file not found in the current scan.' });
+        }
+
+        const body = await readJsonBody(request);
+        const action = body?.action === 'reveal' ? 'reveal' : 'open';
+        await openPath(memory.sourcePath, action);
+        return sendJson(response, 200, { ok: true, action });
+      }
+
       if (request.method !== 'GET' && request.method !== 'HEAD') {
         return sendJson(response, 405, { error: 'Method not allowed.' });
       }
@@ -144,7 +161,11 @@ export function createLocalRuntime(options = {}) {
       server.listen(port, host, () => {
         server.off('error', rejectListen);
         const address = server.address();
-        logger.log(`Copilot Usage Studio local runtime: http://${host}:${address.port}/`);
+        logger.log(
+          backendOnly
+            ? `Copilot Usage Studio backend API: http://${host}:${address.port}/ (used by the dev server)`
+            : `Copilot Usage Studio local app: http://${host}:${address.port}/`,
+        );
         resolveListen(address);
 
         if (options.scanOnStart !== false) {
@@ -167,6 +188,60 @@ export function createLocalRuntime(options = {}) {
   }
 }
 
+function readJsonBody(request) {
+  return new Promise((resolveBody, rejectBody) => {
+    const chunks = [];
+    let size = 0;
+    request.on('data', (chunk) => {
+      size += chunk.length;
+      if (size > 16 * 1024) {
+        rejectBody(new Error('Request body is too large.'));
+        request.destroy();
+        return;
+      }
+      chunks.push(chunk);
+    });
+    request.on('end', () => {
+      if (!chunks.length) {
+        resolveBody({});
+        return;
+      }
+      try {
+        resolveBody(JSON.parse(Buffer.concat(chunks).toString('utf8')));
+      } catch {
+        rejectBody(new Error('Request body must be valid JSON.'));
+      }
+    });
+    request.on('error', rejectBody);
+  });
+}
+
+function openLocalPath(path, action) {
+  const target = resolve(path);
+  let command;
+  let args;
+
+  if (process.platform === 'win32') {
+    command = 'explorer.exe';
+    args = action === 'reveal' ? ['/select,', target] : [target];
+  } else if (process.platform === 'darwin') {
+    command = 'open';
+    args = action === 'reveal' ? ['-R', target] : [target];
+  } else {
+    command = 'xdg-open';
+    args = [action === 'reveal' ? dirname(target) : target];
+  }
+
+  return new Promise((resolveOpen, rejectOpen) => {
+    const child = spawn(command, args, { detached: true, stdio: 'ignore', windowsHide: true });
+    child.once('error', rejectOpen);
+    child.once('spawn', () => {
+      child.unref();
+      resolveOpen();
+    });
+  });
+}
+
 function readCachedSessionData(dataFile, logger) {
   if (!existsSync(dataFile)) {
     return null;

package/package.json

@@ -1,54 +1,55 @@
-{
-  "name": "copilot-usage-studio",
-  "version": "0.1.0",
-  "description": "Local VS Code GitHub Copilot usage, cost, and session insights.",
-  "type": "module",
-  "bin": {
-    "copilot-usage-studio": "bin/copilot-usage-studio.mjs"
-  },
-  "files": [
-    "CHANGELOG.md",
-    "bin/",
-    "data/github-copilot-pricing.json",
-    "dist/copilot-usage-studio/browser/",
-    "docs/local-deployment.md",
-    "docs/pricing.md",
-    "docs/scanner-api.md",
-    "lib/",
-    "scripts/pricing-utils.mjs",
-    "scripts/scan-vscode-sessions.mjs"
-  ],
-  "scripts": {
-    "ng": "ng",
-    "start": "node scripts/start-local-dev.mjs",
-    "start:angular": "ng serve --proxy-config proxy.conf.json",
-    "start:clean": "npm run clean:ng-cache && node scripts/start-local-dev.mjs",
-    "runtime": "node scripts/local-runtime.mjs",
-    "preview:local": "npm run build && node scripts/local-runtime.mjs",
-    "cli": "node bin/copilot-usage-studio.mjs",
-    "prebuild": "node scripts/clean-dist.mjs",
-    "build": "ng build",
-    "watch": "ng build --watch --configuration development",
-    "test": "ng test",
-    "test:scripts": "node --test scripts/*.test.mjs",
-    "clean:ng-cache": "ng cache clean",
-    "refresh:data": "npm run scan && npm run verify:data",
-    "scan": "node scripts/scan-vscode-sessions.mjs",
-    "verify:data": "node scripts/verify-session-data.mjs",
-    "schema:audit": "node scripts/audit-vscode-schema.mjs",
-    "schema:accept": "node scripts/audit-vscode-schema.mjs --accept",
-    "verify:package": "node scripts/verify-package.mjs",
-    "release:check": "npm run test:scripts && npm test -- --watch=false && npm run build && npm run verify:package",
-    "prepack": "npm run build"
-  },
-  "exports": {
-    ".": "./lib/cli.mjs",
-    "./scanner": "./lib/scanner-api.mjs",
-    "./local-runtime": "./lib/local-runtime.mjs"
-  },
-  "engines": {
-    "node": ">=22.5.0"
-  },
+{
+  "name": "copilot-usage-studio",
+  "version": "0.2.0",
+  "description": "Local VS Code GitHub Copilot usage, cost, and session insights.",
+  "type": "module",
+  "bin": {
+    "copilot-usage-studio": "bin/copilot-usage-studio.mjs"
+  },
+  "files": [
+    "CHANGELOG.md",
+    "bin/",
+    "data/github-copilot-pricing.json",
+    "dist/copilot-usage-studio/browser/",
+    "docs/local-deployment.md",
+    "docs/copilot-memory.md",
+    "docs/pricing.md",
+    "docs/scanner-api.md",
+    "lib/",
+    "scripts/pricing-utils.mjs",
+    "scripts/scan-vscode-sessions.mjs"
+  ],
+  "scripts": {
+    "ng": "ng",
+    "start": "node scripts/start-local-dev.mjs",
+    "start:angular": "ng serve --proxy-config proxy.conf.json",
+    "start:clean": "npm run clean:ng-cache && node scripts/start-local-dev.mjs",
+    "runtime": "node scripts/local-runtime.mjs",
+    "preview:local": "npm run build && node scripts/local-runtime.mjs",
+    "cli": "node bin/copilot-usage-studio.mjs",
+    "prebuild": "node scripts/clean-dist.mjs",
+    "build": "ng build",
+    "watch": "ng build --watch --configuration development",
+    "test": "ng test",
+    "test:scripts": "node --test scripts/*.test.mjs",
+    "clean:ng-cache": "ng cache clean",
+    "refresh:data": "npm run scan && npm run verify:data",
+    "scan": "node scripts/scan-vscode-sessions.mjs",
+    "verify:data": "node scripts/verify-session-data.mjs",
+    "schema:audit": "node scripts/audit-vscode-schema.mjs",
+    "schema:accept": "node scripts/audit-vscode-schema.mjs --accept",
+    "verify:package": "node scripts/verify-package.mjs",
+    "release:check": "npm run test:scripts && npm test -- --watch=false && npm run build && npm run verify:package",
+    "prepack": "npm run build"
+  },
+  "exports": {
+    ".": "./lib/cli.mjs",
+    "./scanner": "./lib/scanner-api.mjs",
+    "./local-runtime": "./lib/local-runtime.mjs"
+  },
+  "engines": {
+    "node": ">=22.5.0"
+  },
   "repository": {
     "type": "git",
     "url": "git+https://github.com/fortunes-guardian/copilot-usage-studio.git"
@@ -57,30 +58,30 @@
     "url": "https://github.com/fortunes-guardian/copilot-usage-studio/issues"
   },
   "homepage": "https://github.com/fortunes-guardian/copilot-usage-studio#readme",
-  "keywords": [
-    "github-copilot",
-    "vscode",
-    "ai-usage",
-    "token-cost",
-    "developer-tools"
-  ],
-  "license": "MIT",
-  "packageManager": "npm@11.11.0",
-  "devDependencies": {
-    "@angular/build": "^21.2.9",
-    "@angular/cli": "^21.2.9",
-    "@angular/common": "^21.2.0",
-    "@angular/compiler": "^21.2.0",
-    "@angular/compiler-cli": "^21.2.0",
-    "@angular/core": "^21.2.0",
-    "@angular/forms": "^21.2.0",
-    "@angular/platform-browser": "^21.2.0",
-    "@angular/router": "^21.2.0",
-    "jsdom": "^28.0.0",
-    "prettier": "^3.8.1",
-    "rxjs": "~7.8.0",
-    "tslib": "^2.3.0",
-    "typescript": "~5.9.2",
-    "vitest": "^4.0.8"
-  }
-}
+  "keywords": [
+    "github-copilot",
+    "vscode",
+    "ai-usage",
+    "token-cost",
+    "developer-tools"
+  ],
+  "license": "MIT",
+  "packageManager": "npm@11.11.0",
+  "devDependencies": {
+    "@angular/build": "^21.2.9",
+    "@angular/cli": "^21.2.9",
+    "@angular/common": "^21.2.0",
+    "@angular/compiler": "^21.2.0",
+    "@angular/compiler-cli": "^21.2.0",
+    "@angular/core": "^21.2.0",
+    "@angular/forms": "^21.2.0",
+    "@angular/platform-browser": "^21.2.0",
+    "@angular/router": "^21.2.0",
+    "jsdom": "^28.0.0",
+    "prettier": "^3.8.1",
+    "rxjs": "~7.8.0",
+    "tslib": "^2.3.0",
+    "typescript": "~5.9.2",
+    "vitest": "^4.0.8"
+  }
+}