@hamak/smart-data-dico 1.8.2 → 1.9.0

package/README.md

@@ -321,6 +321,89 @@ Set via `PROFILE` environment variable:
 | **team** | Basic/JWT | Remote | Small team sharing a repo |
 | **server** | Auth0/SSO | Remote | Organization-wide deployment |
 
+## AI Assistance
+
+The dictionary ships with three AI-aware surfaces, all optional. They share one set of credentials stored at `~/.dico-app/dico-app.json` (mode 0600 — never checked into git):
+
+| Surface | What it is | Where to find it |
+|---|---|---|
+| **In-app AI chat panel** | Sidebar chat that grounds against the live dictionary — list packages, describe entities, generate docs, propose edits | Toolbar button **AI Assistant** in the running web app |
+| **Slash commands** | Built-in chat shortcuts (`/list`, `/quality`, `/describe`, `/create`, `/relate`, `/export`, `/diagram`, `/help`) + your own saved prompts | Type `/` in the chat composer |
+| **MCP server** (`dico-mcp`) | A [Model Context Protocol](https://modelcontextprotocol.io) stdio server that exposes the same operations to external clients — Claude Desktop, Cursor, Roo Code, Claude Code | `npx @hamak/smart-data-dico-mcp` or `node bin/dico-mcp.js` from source |
+
+### Configure the in-app chat (one-time)
+
+1. Start the app (Option A, B, or C above).
+2. Open **Settings → AI** (or click the AI Assistant button → ⚙ icon).
+3. Pick a **provider** (`anthropic`, `openai`, or `openai-compatible`), paste an API key, and choose a **model** (e.g. `claude-opus-4-7`, `gpt-4o`, or any model your `openai-compatible` endpoint exposes).
+4. Save. The config writes to `~/.dico-app/dico-app.json` with mode `0600`; no env vars are required.
+
+Alternatively, if you set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` before launching the server, the provider is auto-detected for the first session — but the **Settings dialog is the source of truth**; saving overrides env-var defaults.
+
+Conversations and saved prompts live under `~/.dico-app/storage/` as JSON files — portable, diffable, deletable.
+
+### Slash commands
+
+| Command | What it does |
+|---|---|
+| `/help` | List all available slash commands |
+| `/list` | List every package and its entity count |
+| `/describe` | Describe the current entity in detail (uses page context) |
+| `/quality` | Quality review of the current package (severity-grouped findings) |
+| `/create` | Skeleton: create a new entity with suggested attributes |
+| `/relate` | Skeleton: create a relationship between two entities |
+| `/export` | Generate Markdown documentation for the current package |
+| `/diagram` | Navigate to the organization diagram |
+
+The `pageContext` placeholder is filled automatically from the current route (entity / package / etc.). User-saved prompts (via **Prompts** tab in the chat panel) appear in the same slash-command picker.
+
+### MCP server (`dico-mcp`)
+
+The MCP server reuses the same backend services as the web app — it's a different **transport** for the same operations, not a duplicate code path. You can run the web UI and the MCP server side-by-side against the same project folder.
+
+**Tools exposed:** `listPackages`, `listEntities`, `getEntityDetails`, `createEntity`, `createRelationship`, `listStereotypes`, `listRoutes`.
+
+**Launch (from npm):**
+
+```bash
+# stdio server — talks JSON-RPC over stdin/stdout
+npx @hamak/smart-data-dico --data-dir /path/to/your/project   # web UI
+# (the MCP entrypoint is bin/dico-mcp.js inside the package)
+node "$(npm root -g)/@hamak/smart-data-dico/bin/dico-mcp.js" --data-dir /path/to/your/project
+```
+
+**Register with Claude Desktop** — edit `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows):
+
+```json
+{
+  "mcpServers": {
+    "smart-data-dico": {
+      "command": "npx",
+      "args": ["-y", "@hamak/smart-data-dico", "dico-mcp", "--data-dir", "/absolute/path/to/your/project"]
+    }
+  }
+}
+```
+
+Or, if installed from source, point at the bin script directly:
+
+```json
+{
+  "command": "node",
+  "args": ["/absolute/path/to/smart-data-dico/bin/dico-mcp.js", "--data-dir", "/absolute/path/to/your/project"]
+}
+```
+
+**Register with Cursor** — `.cursor/mcp.json` (project-level) or `~/.cursor/mcp.json` (global): identical shape.
+
+**Register with Roo Code** — `.roo/mcp.json`: identical shape.
+
+The MCP process speaks JSON-RPC on stdio and stays attached to its parent; all logging goes to stderr so it doesn't corrupt the stream. Git auto-commit honours the same `GIT_AUTO_COMMIT` env var as the web backend.
+
+### Connect external MCP servers *to* the in-app chat
+
+The flip side of the above: the in-app chat can also consume external MCP servers (Filesystem, GitHub, etc.). Go to **Settings → MCP** for a curated registry — pick a server, paste any required tokens, save. The chat will then surface those server's tools alongside the built-in ones.
+
 ## Technologies
 
 | Layer | Stack |
@@ -331,3 +414,65 @@ Set via `PROFILE` environment variable:
 | Visualization | Cytoscape.js (dagre + fcose layouts) |
 | Auth | JWT + Auth0 (mock mode for dev) |
 | Deployment | Docker (multi-stage build) |
+
+## Documentation
+
+| Doc | What it covers |
+|-----|----------------|
+| [Format reference](docs/format-reference.md) | Authoritative spec for the on-disk format — project layout, `dico.config.json`, entities, validation, constraints, relationships, stereotypes, rules, cases, actions, state machines |
+| [User guide](docs/user-guide.md) | Task-oriented walkthrough of the app |
+| [API reference](docs/api-reference.md) | REST endpoints (live Swagger UI at `/api-docs` when the backend is running) |
+| [Deployment](docs/deployment.md) | Desktop vs. server modes, file layout, configuration |
+| [Migration plan](docs/migration-plan.md) | @hamak/app-framework migration notes |
+| [ADRs](docs/adr/) | Architecture decision records |
+
+### Claude Code skill
+
+The `docs/` folder doubles as a [Claude Code](https://claude.com/claude-code) skill
+(`docs/SKILL.md` + the format reference) for authoring and validating dictionary files in
+any project. Installing it means copying `docs/` into a skills directory as `smart-data-dico`.
+Claude then loads it automatically whenever you work in a folder containing `dico.config.json`.
+
+**From a local checkout of this repo:**
+
+```bash
+npm run install:skill                          # → ~/.claude/skills/smart-data-dico
+# or into a specific project:
+scripts/install-skill.sh /path/to/project/.claude/skills
+```
+
+**From another repo / machine (no checkout):** pull just `docs/` into your skills directory
+with [`degit`](https://github.com/Rich-Harris/degit) — no auth needed:
+
+```bash
+# global → ~/.claude/skills/smart-data-dico
+npx degit amah/smart-data-dico/docs ~/.claude/skills/smart-data-dico
+```
+
+Or, without Node, via the release tarball:
+
+```bash
+tmp=$(mktemp -d) && curl -fsSL https://codeload.github.com/amah/smart-data-dico/tar.gz/refs/heads/main | tar -xz -C "$tmp" \
+  && rm -rf ~/.claude/skills/smart-data-dico \
+  && cp -R "$tmp"/*/docs ~/.claude/skills/smart-data-dico && rm -rf "$tmp"
+```
+
+Swap the destination for `<project>/.claude/skills/smart-data-dico` to scope it to one project.
+Re-run either command to update.
+
+**Behind a proxy:** `degit` reads the lowercase `https_proxy` env var (not the CLI), while
+`curl` takes an explicit `--proxy` flag — the curl form is usually the more reliable in
+locked-down networks:
+
+```bash
+# degit
+https_proxy=http://proxy.example.com:8080 npx degit amah/smart-data-dico/docs ~/.claude/skills/smart-data-dico
+
+# curl
+tmp=$(mktemp -d) && curl -fsSL --proxy http://proxy.example.com:8080 https://codeload.github.com/amah/smart-data-dico/tar.gz/refs/heads/main | tar -xz -C "$tmp" \
+  && rm -rf ~/.claude/skills/smart-data-dico \
+  && cp -R "$tmp"/*/docs ~/.claude/skills/smart-data-dico && rm -rf "$tmp"
+```
+
+If the proxy does TLS interception, point Node/curl at your CA bundle (`NODE_EXTRA_CA_CERTS=…`
+for degit, `--cacert …` for curl).

package/backend/dist/server.mjs

@@ -37667,6 +37667,18 @@ function normalizeRelationshipEnds(rel) {
     }
   ];
 }
+function normalizeRelationship(rel) {
+  const hasEnds = !!(rel.ends && rel.ends.length >= 2);
+  const hasLegacy = !!(rel.source && rel.target);
+  if (!hasEnds && !hasLegacy) return null;
+  const [a, b] = normalizeRelationshipEnds(rel);
+  return {
+    ...rel,
+    ends: hasEnds ? rel.ends : [a, b],
+    source: rel.source ?? { entity: a.entity, cardinality: a.cardinality, name: a.role, referenceAttributes: a.referenceAttributes },
+    target: rel.target ?? { entity: b.entity, cardinality: b.cardinality, name: b.role, referenceAttributes: b.referenceAttributes }
+  };
+}
 function validateEntity(entity) {
   const validator = new import_jsonschema.Validator();
   const result = validator.validate(entity, entitySchema);
@@ -37721,12 +37733,13 @@ var init_EntitySchema = __esm({
       AttributeType3["UUID"] = "uuid";
       return AttributeType3;
     })(AttributeType || {});
-    Cardinality = /* @__PURE__ */ ((Cardinality4) => {
-      Cardinality4["ONE"] = "one";
-      Cardinality4["MANY"] = "many";
-      return Cardinality4;
+    Cardinality = /* @__PURE__ */ ((Cardinality3) => {
+      Cardinality3["ONE"] = "one";
+      Cardinality3["MANY"] = "many";
+      return Cardinality3;
     })(Cardinality || {});
     __name(normalizeRelationshipEnds, "normalizeRelationshipEnds");
+    __name(normalizeRelationship, "normalizeRelationship");
     entitySchema = {
       type: "object",
       required: ["uuid", "name", "attributes"],
@@ -37891,11 +37904,27 @@ var init_StorageBackendToken = __esm({
     STORAGE_BACKEND_TOKEN = /* @__PURE__ */ Symbol("STORAGE_BACKEND");
     DICTIONARY_QUERY_TOKEN = /* @__PURE__ */ Symbol("DICTIONARY_QUERY");
     StorageRegistry = class {
+      constructor() {
+        this.changeListeners = [];
+      }
       static {
         __name(this, "StorageRegistry");
       }
+      /**
+       * Subscribe to backend swaps/resets. Caches keyed on backend contents
+       * (e.g. the loadPackage cache) register here so they clear when the
+       * backend changes — chiefly so tests that install a fresh in-memory
+       * backend per case don't read stale cached data.
+       */
+      onBackendChange(fn) {
+        this.changeListeners.push(fn);
+      }
+      notifyChange() {
+        for (const fn of this.changeListeners) fn();
+      }
       setBackend(b) {
         this.backend = b;
+        this.notifyChange();
       }
       getBackend() {
         if (!this.backend) throw new Error("STORAGE_BACKEND not registered. server.ts must call storageRegistry.setBackend() at startup.");
@@ -37911,6 +37940,7 @@ var init_StorageBackendToken = __esm({
       reset() {
         this.backend = void 0;
         this.query = void 0;
+        this.notifyChange();
       }
     };
     storageRegistry = new StorageRegistry();
@@ -44170,6 +44200,7 @@ __export(fileOperations_exports, {
   getPackagePath: () => getPackagePath,
   getReservedPackageFiles: () => getReservedPackageFiles,
   getSchemaPackagePath: () => getSchemaPackagePath,
+  invalidatePackageCache: () => invalidatePackageCache,
   listAllDictionaries: () => listAllDictionaries,
   listAllEntities: () => listAllEntities,
   listAllEntityRuleFiles: () => listAllEntityRuleFiles,
@@ -44507,6 +44538,8 @@ async function writeSectionsToStorage(p, sections) {
   if (sections.cases.length > 0) payload.cases = sections.cases;
   if (sections.actions.length > 0) payload.actions = sections.actions;
   if (sections.stateMachines.length > 0) payload.stateMachines = sections.stateMachines;
+  const pkgSeg = String(p).replace(/^\/+/, "").split("/")[0];
+  if (pkgSeg) invalidatePackageCache(pkgSeg);
   if (Object.keys(payload).length === 0) {
     await deleteIfExists(p);
     return;
@@ -44519,7 +44552,13 @@ async function listPackageYamlFilePaths(packageName) {
   const entries = await getStorage().list(WS, dir);
   return entries.filter((e) => !e.isDirectory).map((e) => e.name).filter((f) => f.endsWith(".yaml") && !RESERVED_PACKAGE_FILES.has(f)).sort().map((f) => pathOf(`${packageName}/${f}`));
 }
+function invalidatePackageCache(packageName) {
+  if (packageName === void 0) _packageCache.clear();
+  else _packageCache.delete(packageName);
+}
 async function loadPackage(packageName) {
+  const cached2 = _packageCache.get(packageName);
+  if (cached2 && cached2.expires > Date.now()) return cached2.model;
   const files = await listPackageYamlFilePaths(packageName);
   const parsed = await Promise.all(
     files.map(async (p) => ({
@@ -44527,7 +44566,9 @@ async function loadPackage(packageName) {
       sections: await parseSectionsFromStorage(p, String(p))
     }))
   );
-  return mergePackageSections(packageName, parsed);
+  const model = mergePackageSections(packageName, parsed);
+  _packageCache.set(packageName, { model, expires: Date.now() + PACKAGE_CACHE_TTL_MS });
+  return model;
 }
 function getSchemaPackagePath() {
   return path4.join(getDataDir(), ".dico", "schemas");
@@ -44684,7 +44725,13 @@ async function readRelationshipsFile(packagePath) {
   try {
     const packageName = path4.basename(packagePath);
     const pkg = await loadPackage(packageName);
-    return pkg.relationships;
+    const normalized = [];
+    for (const rel of pkg.relationships) {
+      const n = normalizeRelationship(rel);
+      if (n) normalized.push(n);
+      else logger.warn(`Skipping malformed relationship ${rel.uuid} in package '${packageName}': no ends and no source/target`);
+    }
+    return normalized;
   } catch (error48) {
     logger.error(`Error reading relationships file: ${error48}`);
     return [];
@@ -45244,7 +45291,7 @@ async function deleteStateMachine(uuid3) {
     return { ok: false };
   }
 }
-var import_yaml, WS, getDataDir, RESERVED_DIRS, RESERVED_PACKAGE_FILES, VALIDATION_FIELD_NAMES, gitServiceInstance;
+var import_yaml, WS, getDataDir, RESERVED_DIRS, RESERVED_PACKAGE_FILES, VALIDATION_FIELD_NAMES, gitServiceInstance, PACKAGE_CACHE_TTL_MS, _packageCache;
 var init_fileOperations = __esm({
   "backend/src/utils/fileOperations.ts"() {
     "use strict";
@@ -45289,6 +45336,10 @@ var init_fileOperations = __esm({
     __name(getReservedPackageFiles, "getReservedPackageFiles");
     __name(writeSectionsToStorage, "writeSectionsToStorage");
     __name(listPackageYamlFilePaths, "listPackageYamlFilePaths");
+    PACKAGE_CACHE_TTL_MS = 2e3;
+    _packageCache = /* @__PURE__ */ new Map();
+    __name(invalidatePackageCache, "invalidatePackageCache");
+    storageRegistry.onBackendChange(() => invalidatePackageCache());
     __name(loadPackage, "loadPackage");
     __name(getSchemaPackagePath, "getSchemaPackagePath");
     __name(listSchemaPackageYamlFilePaths, "listSchemaPackageYamlFilePaths");
@@ -45629,21 +45680,18 @@ var init_dictionaryService = __esm({
         try {
           if (id.startsWith("microservices/")) {
             const microservice = id.replace("microservices/", "");
-            const entityNames = await listMicroserviceEntities(microservice);
+            const pkg = await loadPackage(microservice);
             const entries = [];
-            for (const entityName of entityNames) {
-              const entity = await readEntityFile(microservice, entityName);
-              if (entity) {
-                for (const attr of entity.attributes || []) {
-                  entries.push({
-                    id: `${entity.uuid || ""}_${attr.name}`,
-                    name: attr.name,
-                    description: attr.description || "",
-                    type: attr.type || "string",
-                    format: attr.validation?.format,
-                    required: attr.required || false
-                  });
-                }
+            for (const entity of pkg.entities) {
+              for (const attr of entity.attributes || []) {
+                entries.push({
+                  id: `${entity.uuid || ""}_${attr.name}`,
+                  name: attr.name,
+                  description: attr.description || "",
+                  type: attr.type || "string",
+                  format: attr.validation?.format,
+                  required: attr.required || false
+                });
               }
             }
             return entries;
@@ -45791,9 +45839,8 @@ var init_dictionaryService = __esm({
           const microservices = await listMicroservices();
           const result = [];
           for (const microservice of microservices) {
-            const entityNames = await listMicroserviceEntities(microservice);
-            for (const entityName of entityNames) {
-              const entity = await readEntityFile(microservice, entityName);
+            const pkg = await loadPackage(microservice);
+            for (const entity of pkg.entities) {
               if (!entity) continue;
               if (filters.name && !entity.name.toLowerCase().includes(filters.name.toLowerCase())) {
                 continue;
@@ -45882,13 +45929,8 @@ var init_dictionaryService = __esm({
         }
       }
       async getServiceEntities(service) {
-        const entityNames = await listMicroserviceEntities(service);
-        const entities = [];
-        for (const name21 of entityNames) {
-          const entity = await readEntityFile(service, name21);
-          if (entity) entities.push(entity);
-        }
-        return entities;
+        const pkg = await loadPackage(service);
+        return pkg.entities;
       }
     };
     dictionaryService = new DictionaryService();
@@ -47321,29 +47363,13 @@ var init_serviceService = __esm({
         }
       }
       async getServiceEntities(service) {
-        logger.info(`Getting entities for service: ${service}`);
         const startTime = process.hrtime();
         try {
-          const listStartTime = process.hrtime();
-          const entityNames = await listMicroserviceEntities(service);
-          const listEndTime = process.hrtime(listStartTime);
-          const listTimeMs = Number((listEndTime[0] * 1e3 + listEndTime[1] / 1e6).toFixed(2));
-          logger.info(`Listed ${entityNames.length} entity names for service ${service} in ${listTimeMs}ms`);
-          const entities = [];
-          const readStartTime = process.hrtime();
-          for (const entityName of entityNames) {
-            const entity = await readEntityFile(service, entityName);
-            if (entity) {
-              entities.push(entity);
-            }
-          }
-          const readEndTime = process.hrtime(readStartTime);
-          const readTimeMs = Number((readEndTime[0] * 1e3 + readEndTime[1] / 1e6).toFixed(2));
-          logger.info(`Read ${entities.length} entity files for service ${service} in ${readTimeMs}ms`);
+          const pkg = await loadPackage(service);
           const endTime = process.hrtime(startTime);
           const totalTimeMs = Number((endTime[0] * 1e3 + endTime[1] / 1e6).toFixed(2));
-          logger.info(`Total time to get entities for service ${service}: ${totalTimeMs}ms (list: ${listTimeMs}ms, read: ${readTimeMs}ms)`);
-          return entities;
+          logger.info(`Got ${pkg.entities.length} entities for service ${service} in ${totalTimeMs}ms`);
+          return pkg.entities;
         } catch (error48) {
           logger.error(`Error getting service entities: ${error48}`);
           return [];
@@ -47970,6 +47996,17 @@ async function getGitService2() {
     return null;
   }
 }
+function describeError(e) {
+  if (e instanceof Error) return e.message;
+  try {
+    return JSON.stringify(e);
+  } catch {
+    return String(e);
+  }
+}
+function isNotAGitRepo(detail) {
+  return /not a git repo|could not find .*git repo|NotGitRepo|NOT_A_REPO|fatal: not a git/i.test(detail);
+}
 var gitServiceInstance2, VersionService, versionService;
 var init_versionService = __esm({
   "backend/src/services/versionService.ts"() {
@@ -47978,6 +48015,8 @@ var init_versionService = __esm({
     init_config();
     gitServiceInstance2 = null;
     __name(getGitService2, "getGitService");
+    __name(describeError, "describeError");
+    __name(isNotAGitRepo, "isNotAGitRepo");
     VersionService = class {
       static {
         __name(this, "VersionService");
@@ -47996,7 +48035,12 @@ var init_versionService = __esm({
           const ddFiles = (status.files || []).map((f) => f.path).filter((p) => p.startsWith(ddPrefix));
           return { clean: ddFiles.length === 0, files: ddFiles };
         } catch (error48) {
-          logger.warn(`getWorkingTreeStatus failed: ${error48}`);
+          const detail = describeError(error48);
+          if (isNotAGitRepo(detail)) {
+            logger.debug(`getWorkingTreeStatus: project is not a git repository \u2014 treating as clean (${detail})`);
+          } else {
+            logger.warn(`getWorkingTreeStatus failed: ${detail}`);
+          }
           return { clean: true, files: [] };
         }
       }
@@ -148774,7 +148818,8 @@ var init_UuidIndex = __esm({
         }
         this.rebuildInFlight = true;
         try {
-          const { listPackages: listPackages2 } = await Promise.resolve().then(() => (init_fileOperations(), fileOperations_exports));
+          const { listPackages: listPackages2, invalidatePackageCache: invalidatePackageCache2 } = await Promise.resolve().then(() => (init_fileOperations(), fileOperations_exports));
+          invalidatePackageCache2();
           const top = await listPackages2();
           const allPackages = [];
           for (const pkg of top) {
@@ -148899,6 +148944,8 @@ var init_UuidIndex = __esm({
         while (this.rebuildInFlight) {
           await new Promise((r) => setTimeout(r, 0));
         }
+        const { invalidatePackageCache: invalidatePackageCache2 } = await Promise.resolve().then(() => (init_fileOperations(), fileOperations_exports));
+        invalidatePackageCache2(packageName);
         const packagePrefix = `packages/${packageName}/entities/`;
         const previouslyMappedUuids = /* @__PURE__ */ new Set();
         for (const [uuid3, lp] of this.uuidToPath) {
@@ -160474,9 +160521,9 @@ var QualityService = class {
     const services = service ? [service] : await listMicroservices();
     const packages = [];
     for (const svc of services) {
-      const entityNames = await listMicroserviceEntities(svc);
+      const pkg = await loadPackage(svc);
       const entities = [];
-      let relEntityUuids = /* @__PURE__ */ new Set();
+      const relEntityUuids = /* @__PURE__ */ new Set();
       try {
         const rels = await readRelationshipsFile(getPackagePath(svc));
         for (const rel of rels) {
@@ -160485,9 +160532,7 @@ var QualityService = class {
         }
       } catch {
       }
-      for (const rawName of entityNames) {
-        const name21 = rawName.includes("_") ? rawName.split("_").slice(1).join("_") : rawName;
-        const entity = await readEntityFile(svc, name21);
+      for (const entity of pkg.entities) {
         if (!entity) continue;
         const descFilled = !!entity.description;
         const totalAttrs = entity.attributes.length;
@@ -162295,6 +162340,13 @@ async function mountFrameworkRoutes() {
         workspaceRoots
       });
       enricherRegistry2.register(gitEnricher);
+      app.use("/api/git/dictionaries/status", (_req, res, next) => {
+        if (!fs5.existsSync(path13.join(config.dataDir, ".git"))) {
+          res.json({ files: [], hasUncommittedChanges: false });
+          return;
+        }
+        next();
+      });
       const gitRoutes = gitModule.createGitRoutes({ gitService, debug: !config.isProduction });
       app.use("/api/git", gitRoutes);
       app.use("/api/git", gitModule.gitErrorHandler);