openclew 0.2.1 → 0.3.0

package/README.md

@@ -1,3 +1,7 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/openclew/openclew/main/assets/logo.png" alt="openclew" width="200">
+</p>
+
 # openclew
 
 > Long Life Memory for LLMs
@@ -114,7 +118,7 @@ The index auto-regenerates on every commit. Never edit it manually.
 1. Create `doc/` and `doc/log/`
 2. Copy templates from [`templates/`](templates/) (refdoc.md, log.md)
 3. Add the openclew block to your instruction file (see `doc/_USING_OPENCLEW.md` after init for the exact format)
-4. Copy [`hooks/generate-index.py`](hooks/generate-index.py) and wire it as a pre-commit hook
+4. Run `openclew index` to generate `doc/_INDEX.md` (or wire it as a pre-commit hook)
 
 </details>
 

package/UPGRADING.md

@@ -0,0 +1,167 @@
+# Upgrading openclew
+
+openclew evolves. When the format changes, your existing docs still work — parsers
+are backward-compatible. But new features expect the current format.
+
+`openclew migrate` bridges the gap.
+
+---
+
+## Quick version
+
+```bash
+npm install -g openclew@latest   # or: npx openclew@latest
+openclew status                  # shows legacy doc count
+openclew migrate                 # dry-run: what would change
+openclew migrate --write         # apply
+git diff                         # review
+git add doc/ && git commit -m "chore: migrate docs to openclew format"
+```
+
+---
+
+## When to upgrade
+
+After updating openclew, run `openclew status`. If it reports legacy docs,
+run `openclew migrate` to see what needs changing.
+
+You can also check directly:
+
+```bash
+openclew migrate
+# → 12 to migrate, 45 already current, 0 errors (57 total)
+```
+
+No output = nothing to do.
+
+---
+
+## How it works
+
+`migrate` converts docs from older formats to the current openclew format.
+It is **safe by default**:
+
+- **Dry-run first** — shows what would change without touching files
+- **`--write` to apply** — only modifies files when you explicitly ask
+- **Git-friendly** — files are tracked, `git diff` shows exactly what changed
+- **Idempotent** — running it twice produces the same result
+- **Skips current docs** — only touches files that need updating
+
+### What it converts
+
+| Before | After |
+|--------|-------|
+| `R.AlphA.Doc@7.0.0` (line 1) | `openclew@0.3.0 · created: ... · type: ... · ...` |
+| `subject: Title` (plain L1) | `**subject:** Title` (bold L1) |
+| `summary: ...` | `**doc_brief:** ...` |
+| `# 📋 L1 · Métadonnées` | _(removed — metadata is on line 1)_ |
+| `# 📝 L2 · Résumé` | `# L2 - Summary` |
+| `# 🔧 L3 · Détails` | `# L3 - Details` |
+| YAML frontmatter (`---`) | Replaced by line 1 + L1 block |
+| `status: Vivant` | `status: Active` |
+| `status: Terminé` | `status: Done` |
+
+### What it does NOT change
+
+- **L2/L3 body content** — only headers are normalized, your content is untouched
+- **Sub-headers** — `## Objective`, `## Key points` etc. are preserved as-is
+- **`related_docs` paths** — kept on line 1 but not repointed if you move files
+- **Files already in openclew format** — skipped entirely
+- **`_INDEX.md`** — auto-generated, never touched
+
+---
+
+## Step by step
+
+### 1. Update openclew
+
+```bash
+npm install -g openclew@latest
+```
+
+### 2. Check your docs
+
+```bash
+openclew status
+```
+
+Look for the "Legacy format" line. If it says 0, you're done.
+
+### 3. Preview changes
+
+```bash
+openclew migrate
+```
+
+This lists every file that would be converted. No files are modified.
+
+### 4. Apply
+
+```bash
+openclew migrate --write
+```
+
+Each converted file is printed with `✓`.
+
+### 5. Review and commit
+
+```bash
+git diff                    # inspect the changes
+openclew index              # regenerate the index
+git add doc/ && git commit -m "chore: migrate docs to openclew format"
+```
+
+### 6. Verify
+
+```bash
+openclew status             # should show 0 legacy docs
+openclew migrate            # should show "0 to migrate"
+```
+
+---
+
+## After migrating
+
+- **New docs** you create with `openclew add ref` / `openclew add log` already
+  use the current format. No action needed.
+- **Docs created by AI agents** will follow the format they see in your codebase.
+  Once your existing docs are migrated, agents will generate in the new format.
+- **Empty `doc_brief`** — some old docs may have no brief after migration.
+  Run `openclew status` to find them and fill them in.
+
+---
+
+## Version-specific notes
+
+### → 0.4.0 (format migration)
+
+First migration release. Converts from the legacy format (YAML frontmatter,
+plain `key: value` L1, emoji headers) to the openclew format (condensed line 1,
+bold L1 fields, clean headers).
+
+**Scope**: line 1 + L1 block + L2/L3 main headers.
+
+**Not in scope**: sub-header emojis, `related_docs` repointing, recursive
+`doc/` subdirectory scanning (refdocs must be in `doc/_*.md`, not `doc/ref/`).
+
+---
+
+## Limitations
+
+### Flat `doc/` structure required
+
+All openclew tools (search, index, status, migrate) scan `doc/_*.md` for refdocs
+and `doc/log/*.md` for logs. Subdirectories like `doc/ref/` are not scanned yet.
+
+If you plan to reorganize into subdirectories, wait for recursive scan support
+(tracked in the openclew roadmap).
+
+### `related_docs` are not repointed
+
+If a doc references `related_docs: [doc/_AUTH.md]` and you rename that file,
+the reference breaks. `migrate` preserves paths as-is — manual update required.
+
+### Parsers are backward-compatible
+
+Even without migrating, your docs are still readable by openclew tools.
+Migration improves consistency and enables new features, but is not blocking.

package/bin/openclew.js

@@ -9,39 +9,67 @@ const USAGE = `
 openclew — Long Life Memory for LLMs
 
 Usage:
-  openclew init                    Set up openclew in the current project
-  openclew new <title>             Create a refdoc (evolves with the project)
-  openclew log <title>             Create a session log (frozen facts)
-  openclew checkout                End-of-session summary + log creation
-  openclew index                   Regenerate doc/_INDEX.md
-  openclew help                    Show this help
+  openclew init                    Set up openclew in your project
+  openclew add ref <title>         Create a refdoc (evolves with the project)
+  openclew add log <title>         Create a session log (frozen facts)
+  openclew search <query>          Search docs by keyword
+  openclew checkout                End-of-session summary
 
-Options:
-  --no-hook                        Skip pre-commit hook installation (init)
-  --no-inject                      Skip instruction file injection (init)
+Run 'openclew help --all' for advanced commands.
+More at: https://github.com/openclew/openclew
+`.trim();
 
-Getting started:
-  npx openclew init                1. Set up doc/ + guide + examples + git hook
-  # Edit doc/_ARCHITECTURE.md      2. Replace the example with your project's architecture
-  openclew new "API design"        3. Create your own refdocs
-  git commit                       4. Index auto-regenerates on commit
+const USAGE_ALL = `
+openclew — Long Life Memory for LLMs
 
-Docs have 3 levels: L1 (metadata) → L2 (summary) → L3 (details).
-Agents read L1 to decide what's relevant, then L2 for context.
-More at: https://github.com/openclew/openclew
+Usage:
+  openclew init                    Set up openclew in your project
+  openclew add ref <title>         Create a refdoc (evolves with the project)
+  openclew add log <title>         Create a session log (frozen facts)
+  openclew search <query>          Search docs by keyword
+  openclew checkout                End-of-session summary
+
+Advanced:
+  openclew status                  Documentation health dashboard
+  openclew index                   Regenerate doc/_INDEX.md
+  openclew mcp                     Start MCP server (stdio JSON-RPC)
+
+Options (init):
+  --no-hook                        Skip pre-commit hook installation
+  --no-inject                      Skip instruction file injection
 `.trim();
 
 if (!command || command === "help" || command === "--help" || command === "-h") {
-  console.log(USAGE);
+  const showAll = args.includes("--all");
+  console.log(showAll ? USAGE_ALL : USAGE);
+  process.exit(0);
+}
+
+// Handle "add ref" / "add log" subcommands
+if (command === "add") {
+  const sub = args[1];
+  if (sub === "ref") {
+    require("../lib/new-doc");
+  } else if (sub === "log") {
+    require("../lib/new-log");
+  } else {
+    console.error(`Unknown type: ${sub || "(none)"}`);
+    console.error('Usage: openclew add ref <title>  or  openclew add log <title>');
+    process.exit(1);
+  }
   process.exit(0);
 }
 
+// Legacy aliases
 const commands = {
   init: () => require("../lib/init"),
   new: () => require("../lib/new-doc"),
   log: () => require("../lib/new-log"),
   checkout: () => require("../lib/checkout"),
+  search: () => require("../lib/search"),
+  status: () => require("../lib/status"),
   index: () => require("../lib/index-gen"),
+  mcp: () => require("../lib/mcp-server"),
 };
 
 if (!commands[command]) {

package/lib/index-gen.js

@@ -1,40 +1,115 @@
 /**
  * openclew index — regenerate doc/_INDEX.md
  *
- * Wraps hooks/generate-index.py. Falls back to a JS implementation
- * if Python is not available.
+ * Pure JS implementation. Reuses parsers from search.js (SSOT).
+ * Zero dependencies — Node 16+ standard library only.
  */
 
-const { execSync } = require("child_process");
 const fs = require("fs");
 const path = require("path");
+const { collectDocs } = require("./search");
 
-const docDir = path.join(process.cwd(), "doc");
+/**
+ * Generate _INDEX.md content from parsed docs.
+ *
+ * @param {string} docDir - Absolute path to doc/ directory
+ * @returns {string} Generated index content
+ */
+function generateIndex(docDir) {
+  const docs = collectDocs(docDir);
+  const refdocs = docs.filter((d) => d.kind === "refdoc");
+  const logs = docs.filter((d) => d.kind === "log");
+
+  const now = new Date();
+  const timestamp = `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, "0")}-${String(now.getDate()).padStart(2, "0")} ${String(now.getHours()).padStart(2, "0")}:${String(now.getMinutes()).padStart(2, "0")}`;
+
+  const lines = [
+    "# Project Knowledge Index",
+    "",
+    `> Auto-generated by [openclew](https://github.com/openclew/openclew) on ${timestamp}.`,
+    "> Do not edit manually — rebuilt from L1 metadata on every commit.",
+    "",
+  ];
+
+  // Refdocs section
+  lines.push("## Refdocs");
+  lines.push("");
+  if (refdocs.length) {
+    lines.push("| Document | Subject | Status | Category |");
+    lines.push("|----------|---------|--------|----------|");
+    for (const doc of refdocs) {
+      const name = path.basename(doc.filepath);
+      const subject = doc.meta.subject || "—";
+      const status = doc.meta.status || "—";
+      const category = doc.meta.category || "—";
+      const relPath = path.relative(path.dirname(docDir), doc.filepath);
+      lines.push(`| [${name}](${relPath}) | ${subject} | ${status} | ${category} |`);
+    }
+  } else {
+    lines.push("_No refdocs yet. Create one with `npx openclew add ref \"Title\"`._");
+  }
+  lines.push("");
+
+  // Logs section (last 20)
+  const displayLogs = logs.slice(0, 20);
+  lines.push("## Recent logs");
+  lines.push("");
+  if (displayLogs.length) {
+    lines.push("| Date | Subject | Status | Category |");
+    lines.push("|------|---------|--------|----------|");
+    for (const doc of displayLogs) {
+      const date = doc.meta.date || path.basename(doc.filepath).slice(0, 10);
+      const subject = doc.meta.subject || "—";
+      const status = doc.meta.status || "—";
+      const category = doc.meta.category || "—";
+      const relPath = path.relative(path.dirname(docDir), doc.filepath);
+      lines.push(`| ${date} | [${subject}](${relPath}) | ${status} | ${category} |`);
+    }
+    if (logs.length > 20) {
+      lines.push("");
+      lines.push(`_${logs.length - 20} older logs not shown._`);
+    }
+  } else {
+    lines.push("_No logs yet. Create one with `npx openclew add log \"Title\"`._");
+  }
+  lines.push("");
 
-if (!fs.existsSync(docDir)) {
-  console.error("No doc/ directory found. Run 'openclew init' first.");
-  process.exit(1);
+  // Stats
+  lines.push("---");
+  lines.push(`**${refdocs.length}** refdocs, **${logs.length}** logs.`);
+  lines.push("");
+
+  return lines.join("\n");
 }
 
-// Try local generate-index.py first
-const localScript = path.join(docDir, "generate-index.py");
-const packageScript = path.join(__dirname, "..", "hooks", "generate-index.py");
-const script = fs.existsSync(localScript) ? localScript : packageScript;
-
-if (fs.existsSync(script)) {
-  try {
-    const output = execSync(`python3 "${script}" "${docDir}"`, {
-      encoding: "utf-8",
-    });
-    console.log(output.trim());
-    process.exit(0);
-  } catch {
-    console.error(
-      "python3 not available. Install Python 3.8+ or regenerate manually."
-    );
+/**
+ * Write _INDEX.md to disk.
+ *
+ * @param {string} docDir - Absolute path to doc/ directory
+ * @returns {{ refdocs: number, logs: number }} Counts
+ */
+function writeIndex(docDir) {
+  const content = generateIndex(docDir);
+  const indexPath = path.join(docDir, "_INDEX.md");
+  fs.writeFileSync(indexPath, content, "utf-8");
+
+  const docs = collectDocs(docDir);
+  const refdocs = docs.filter((d) => d.kind === "refdoc").length;
+  const logs = docs.filter((d) => d.kind === "log").length;
+  return { refdocs, logs };
+}
+
+// CLI runner
+if (require.main === module || process.argv.includes("index")) {
+  const docDir = process.argv[2] || path.join(process.cwd(), "doc");
+
+  if (!fs.existsSync(docDir)) {
+    console.error("No doc/ directory found. Run 'openclew init' first.");
     process.exit(1);
   }
+
+  const { refdocs, logs } = writeIndex(docDir);
+  console.log(`Generated doc/_INDEX.md (${refdocs} refdocs, ${logs} logs)`);
 }
 
-console.error("generate-index.py not found. Run 'openclew init' first.");
-process.exit(1);
+module.exports = { generateIndex, writeIndex };

package/lib/init.js

@@ -15,7 +15,7 @@ const readline = require("readline");
 const { detectInstructionFiles, findAgentsMdCaseInsensitive } = require("./detect");
 const { inject, isAlreadyInjected } = require("./inject");
 const { writeConfig } = require("./config");
-const { guideContent, exampleRefdocContent, exampleLogContent, today } = require("./templates");
+const { guideContent, frameworkIntegrationContent, exampleRefdocContent, exampleLogContent, today } = require("./templates");
 
 const PROJECT_ROOT = process.cwd();
 const DOC_DIR = path.join(PROJECT_ROOT, "doc");
@@ -126,8 +126,8 @@ function installPreCommitHook() {
   }
 
   const preCommitPath = path.join(hooksDir, "pre-commit");
-  const indexScript = `if [ -f doc/generate-index.py ]; then
-  python3 doc/generate-index.py doc 2>/dev/null || echo "openclew: index generation failed"
+  const indexScript = `if command -v npx >/dev/null 2>&1; then
+  npx --yes openclew index 2>/dev/null || echo "openclew: index generation failed"
   git add doc/_INDEX.md 2>/dev/null
 fi`;
 
@@ -168,26 +168,19 @@ function updateGitignore() {
   }
 }
 
-function copyGenerateIndex() {
-  const src = path.join(__dirname, "..", "hooks", "generate-index.py");
-  const dst = path.join(DOC_DIR, "generate-index.py");
-
-  if (fs.existsSync(dst)) {
-    console.log("  doc/generate-index.py already exists");
-    return false;
-  }
-
-  if (fs.existsSync(src)) {
-    fs.copyFileSync(src, dst);
-    console.log("  Copied generate-index.py to doc/");
+function cleanupLegacyPython() {
+  // Remove legacy generate-index.py if present (replaced by JS-native index-gen)
+  const legacyScript = path.join(DOC_DIR, "generate-index.py");
+  if (fs.existsSync(legacyScript)) {
+    fs.unlinkSync(legacyScript);
+    console.log("  Removed legacy doc/generate-index.py (now JS-native)");
     return true;
   }
-
-  console.log("  generate-index.py not found in package — skipping");
+  console.log("  No legacy Python script to clean up");
   return false;
 }
 
-function createDocs() {
+function createDocs(entryPointPath) {
   // Guide — always created
   const guidePath = path.join(DOC_DIR, "_USING_OPENCLEW.md");
   if (!fs.existsSync(guidePath)) {
@@ -197,11 +190,30 @@ function createDocs() {
     console.log("  doc/_USING_OPENCLEW.md already exists");
   }
 
-  // Example refdoc
+  // Framework integration guide
+  const frameworkPath = path.join(DOC_DIR, "_OPENCLEW_FRAMEWORK_INTEGRATION.md");
+  if (!fs.existsSync(frameworkPath)) {
+    fs.writeFileSync(frameworkPath, frameworkIntegrationContent(), "utf-8");
+    console.log("  Created doc/_OPENCLEW_FRAMEWORK_INTEGRATION.md (framework integration guide)");
+  } else {
+    console.log("  doc/_OPENCLEW_FRAMEWORK_INTEGRATION.md already exists");
+  }
+
+  // Architecture refdoc — seeded from existing instruction file if available
   const examplePath = path.join(DOC_DIR, "_ARCHITECTURE.md");
   if (!fs.existsSync(examplePath)) {
-    fs.writeFileSync(examplePath, exampleRefdocContent(), "utf-8");
-    console.log("  Created doc/_ARCHITECTURE.md (example refdoc)");
+    let existingInstructions = null;
+    if (entryPointPath && fs.existsSync(entryPointPath)) {
+      try {
+        existingInstructions = fs.readFileSync(entryPointPath, "utf-8");
+      } catch {}
+    }
+    fs.writeFileSync(examplePath, exampleRefdocContent(existingInstructions), "utf-8");
+    if (existingInstructions) {
+      console.log("  Created doc/_ARCHITECTURE.md (seeded from instruction file)");
+    } else {
+      console.log("  Created doc/_ARCHITECTURE.md (template)");
+    }
   } else {
     console.log("  doc/_ARCHITECTURE.md already exists");
   }
@@ -217,15 +229,14 @@ function createDocs() {
 }
 
 function runIndexGenerator() {
-  const indexScript = path.join(DOC_DIR, "generate-index.py");
-  if (!fs.existsSync(indexScript)) return;
+  if (!fs.existsSync(DOC_DIR)) return;
 
   try {
-    const { execSync } = require("child_process");
-    execSync(`python3 "${indexScript}" "${DOC_DIR}"`, { stdio: "pipe" });
-    console.log("  Generated doc/_INDEX.md");
-  } catch {
-    console.log("  Could not generate index (python3 not available)");
+    const { writeIndex } = require("./index-gen");
+    const { refdocs, logs } = writeIndex(DOC_DIR);
+    console.log(`  Generated doc/_INDEX.md (${refdocs} refdocs, ${logs} logs)`);
+  } catch (err) {
+    console.log(`  Could not generate index: ${err.message}`);
   }
 }
 
@@ -240,9 +251,9 @@ async function main() {
   console.log("\n2. Gitignore");
   updateGitignore();
 
-  // Step 3: Copy index generator
+  // Step 3: Cleanup legacy Python (if upgrading from older version)
   console.log("\n3. Index generator");
-  copyGenerateIndex();
+  cleanupLegacyPython();
 
   // Step 4: Entry point
   console.log("\n4. Entry point");
@@ -273,7 +284,7 @@ async function main() {
 
   // Step 6: Docs
   console.log("\n6. Docs");
-  createDocs();
+  createDocs(entryPoint ? entryPoint.fullPath : null);
 
   // Step 7: Generate index
   console.log("\n7. Index");

package/lib/inject.js

@@ -7,17 +7,26 @@ const fs = require("fs");
 const OPENCLEW_BLOCK = `
 ## Project knowledge (openclew)
 
-This file is the **entry point** for project documentation.
+This project uses \`doc/\` as its knowledge base. Before starting any task:
 
-**Doc-first rule:** before any task, read \`doc/_INDEX.md\` to find docs related to the task. Read them before exploring code.
+1. **Read \`doc/_INDEX.md\`** — it lists all available docs with a one-line summary each
+2. **Pick your reference doc(s)** — choose one or more docs relevant to what you're about to do
+3. **Read them** (L1 for relevance, L2 for context) — then start working
+4. **No matching doc?** — propose creating a refdoc with \`npx openclew new "Title"\` before starting
 
-Two types of docs in \`doc/\`:
-- **Refdocs** (\`doc/_*.md\`) — evolve with the project (architecture, conventions, decisions)
-- **Logs** (\`doc/log/YYYY-MM-DD_*.md\`) — frozen facts from a session, never modified after
+If a doc contains placeholder comments (\`<!-- ... -->\`), fill them in based on what you observe in the code. This is expected — the docs are meant to be written by you.
 
-Each doc has 3 levels: **L1** (metadata — read first to decide relevance) → **L2** (summary) → **L3** (full details, only when needed).
+Two types of docs:
+- **Refdocs** (\`doc/_*.md\`) — architecture, conventions, decisions (evolve over time)
+- **Logs** (\`doc/log/YYYY-MM-DD_*.md\`) — frozen facts from past sessions
 
-**Creating docs:** when a decision, convention, or significant event needs to be captured, create the file directly following the format in \`doc/_USING_OPENCLEW.md\`.
+Each doc has 3 levels: **L1** (subject + brief — 1 line) → **L2** (summary) → **L3** (full details, only when needed).
+
+**Session commands** (user asks in chat, you run):
+- "checkout" → \`npx openclew checkout\` (end-of-session summary + log)
+- "new doc about X" → \`npx openclew new "X"\` (create refdoc)
+- "search X" → \`npx openclew search "X"\` (search docs)
+- "doc status" → \`npx openclew status\` (health dashboard)
 `.trim();
 
 const MARKER_START = "<!-- openclew_START -->";