openclew 0.2.0 → 0.3.0

package/README.md

@@ -1,3 +1,7 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/openclew/openclew/main/assets/logo.png" alt="openclew" width="200">
+</p>
+
 # openclew
 
 > Long Life Memory for LLMs
@@ -70,152 +74,54 @@ L1 answers "should I read this?" L2 answers "what do I need to know?" L3 is ther
 
 | Type | Location | Role | Mutability |
 |------|----------|------|------------|
-| **Living** | `doc/_SUBJECT.md` | Living knowledge (architecture, conventions, decisions) | Updated over time |
+| **Refdoc** | `doc/_SUBJECT.md` | Reference knowledge (architecture, conventions, decisions) | Updated over time |
 | **Log** | `doc/log/YYYY-MM-DD_subject.md` | Frozen facts (what happened, what was decided) | Never modified |
 
-**Living docs** are your project's brain — they evolve as the project evolves.
+**Refdocs** are your project's brain — they evolve as the project evolves.
 **Logs** are your project's journal — immutable records of what happened and why.
 
-Together, they form the thread. The living docs tell you where you are. The logs tell you how you got here.
+Together, they form the thread. The refdocs tell you where you are. The logs tell you how you got here.
 
 ---
 
-## Quick start (5 minutes)
+## Quick start (2 minutes)
 
-### 1. Create the structure
+### 1. Install
 
 ```bash
-mkdir -p doc/log
+npx openclew init
 ```
 
-### 2. Copy the templates
-
-Download from [`templates/`](templates/) or create manually:
-
-<details>
-<summary><b>templates/living.md</b> — for living knowledge</summary>
-
-```markdown
-<!-- L1_START -->
-# L1 - Metadata
-type: Reference | Architecture | Guide | Analysis
-subject: Short title (< 60 chars)
-created: YYYY-MM-DD
-updated: YYYY-MM-DD
-short_story: 1-2 sentences. What this doc covers and what it concludes.
-status: Active | Stable | Archived
-category: Main domain (e.g. Auth, API, Database, UI...)
-keywords: [tag1, tag2, tag3]
-<!-- L1_END -->
-
----
-
-<!-- L2_START -->
-# L2 - Summary
-
-## Objective
-<!-- Why this document exists -->
+This:
+- Creates `doc/` with a guide, an example doc, and an example log
+- Detects your instruction file (CLAUDE.md, .cursorrules, AGENTS.md...)
+- Injects a block that teaches your agent about the doc structure
+- Installs a pre-commit hook that auto-generates `doc/_INDEX.md`
 
-## Key points
-<!-- 3-5 essential takeaways -->
+### 2. Start a session with your agent
 
-## Solution
-<!-- Recommended approach or pattern -->
-<!-- L2_END -->
-
----
+Ask it:
 
-<!-- L3_START -->
-# L3 - Details
+> Read doc/_USING_OPENCLEW.md and document our architecture.
 
-<!-- Full technical content: examples, code, references... -->
+Your agent reads the guide, understands the L1/L2/L3 format, and creates `doc/_ARCHITECTURE.md` with your project's actual architecture.
 
-## Changelog
+### 3. There is no step 3
 
-| Date | Change |
-|------|--------|
-| YYYY-MM-DD | Initial creation |
-<!-- L3_END -->
-```
+Next session, your agent reads the index, finds the doc, has the context. No re-explanation needed. As your project evolves, your agent creates and updates docs during sessions — refdocs for ongoing knowledge, logs for frozen facts.
 
-</details>
+The index auto-regenerates on every commit. Never edit it manually.
 
 <details>
-<summary><b>templates/log.md</b> — for frozen facts</summary>
-
-```markdown
-<!-- L1_START -->
-# L1 - Metadata
-date: YYYY-MM-DD
-type: Bug | Feature | Refactor | Doc | Deploy
-subject: Short title (< 60 chars)
-short_story: 1-2 sentences. What happened and what was the outcome.
-status: Done | In progress | Abandoned
-category: Main domain
-keywords: [tag1, tag2, tag3]
-<!-- L1_END -->
-
----
-
-<!-- L2_START -->
-# L2 - Summary
-
-## Problem
-<!-- What was observed -->
-
-## Solution
-<!-- How it was resolved -->
-<!-- L2_END -->
-
----
+<summary><b>Manual setup</b> — if you prefer not to use the CLI</summary>
 
-<!-- L3_START -->
-# L3 - Details
-
-<!-- Technical details: code changes, debugging steps, references... -->
-<!-- L3_END -->
-```
+1. Create `doc/` and `doc/log/`
+2. Copy templates from [`templates/`](templates/) (refdoc.md, log.md)
+3. Add the openclew block to your instruction file (see `doc/_USING_OPENCLEW.md` after init for the exact format)
+4. Run `openclew index` to generate `doc/_INDEX.md` (or wire it as a pre-commit hook)
 
 </details>
 
-### 3. Write your first doc
-
-```bash
-cp templates/living.md doc/_ARCHITECTURE.md
-```
-
-Edit it — describe your project's architecture. Fill in L1 (metadata), L2 (summary), skip L3 if you don't need it yet.
-
-### 4. Point your agent to it
-
-Add this to your `CLAUDE.md`, `.cursorrules`, or `AGENTS.md`:
-
-```markdown
-## Project knowledge
-
-Documentation lives in `doc/`. Each doc has 3 levels (L1/L2/L3).
-- Read L1 first to decide if you need more
-- Living docs: `doc/_*.md` (living knowledge, updated)
-- Logs: `doc/log/YYYY-MM-DD_*.md` (frozen facts, never modified)
-- Index: `doc/_INDEX.md` (auto-generated, start here)
-```
-
-### 5. Auto-generate the index (optional)
-
-Copy [`hooks/generate-index.py`](hooks/generate-index.py) to your project and add it as a pre-commit hook:
-
-```bash
-# Option A: git hook
-cp hooks/generate-index.py .git/hooks/generate-index.py
-echo 'python .git/hooks/generate-index.py && git add doc/_INDEX.md' >> .git/hooks/pre-commit
-chmod +x .git/hooks/pre-commit
-
-# Option B: pre-commit framework
-# See hooks/README.md for .pre-commit-config.yaml setup
-```
-
-The index auto-regenerates on every commit. Never edit it manually.
-
 ---
 
 ## How it works in practice
@@ -263,7 +169,7 @@ doc/
 - **Shared knowledge** — Same docs for humans and AI. One source, multiple readers.
 - **SSOT** (Single Source of Truth) — Each piece of information lives in one place.
 - **Logs are immutable** — Once written, never modified. Frozen facts.
-- **Living docs evolve** — They evolve as the project evolves.
+- **Refdocs evolve** — They evolve as the project evolves.
 - **Index is auto-generated** — Never edit `_INDEX.md` manually.
 
 ---

package/UPGRADING.md

@@ -0,0 +1,167 @@
+# Upgrading openclew
+
+openclew evolves. When the format changes, your existing docs still work — parsers
+are backward-compatible. But new features expect the current format.
+
+`openclew migrate` bridges the gap.
+
+---
+
+## Quick version
+
+```bash
+npm install -g openclew@latest   # or: npx openclew@latest
+openclew status                  # shows legacy doc count
+openclew migrate                 # dry-run: what would change
+openclew migrate --write         # apply
+git diff                         # review
+git add doc/ && git commit -m "chore: migrate docs to openclew format"
+```
+
+---
+
+## When to upgrade
+
+After updating openclew, run `openclew status`. If it reports legacy docs,
+run `openclew migrate` to see what needs changing.
+
+You can also check directly:
+
+```bash
+openclew migrate
+# → 12 to migrate, 45 already current, 0 errors (57 total)
+```
+
+No output = nothing to do.
+
+---
+
+## How it works
+
+`migrate` converts docs from older formats to the current openclew format.
+It is **safe by default**:
+
+- **Dry-run first** — shows what would change without touching files
+- **`--write` to apply** — only modifies files when you explicitly ask
+- **Git-friendly** — files are tracked, `git diff` shows exactly what changed
+- **Idempotent** — running it twice produces the same result
+- **Skips current docs** — only touches files that need updating
+
+### What it converts
+
+| Before | After |
+|--------|-------|
+| `R.AlphA.Doc@7.0.0` (line 1) | `openclew@0.3.0 · created: ... · type: ... · ...` |
+| `subject: Title` (plain L1) | `**subject:** Title` (bold L1) |
+| `summary: ...` | `**doc_brief:** ...` |
+| `# 📋 L1 · Métadonnées` | _(removed — metadata is on line 1)_ |
+| `# 📝 L2 · Résumé` | `# L2 - Summary` |
+| `# 🔧 L3 · Détails` | `# L3 - Details` |
+| YAML frontmatter (`---`) | Replaced by line 1 + L1 block |
+| `status: Vivant` | `status: Active` |
+| `status: Terminé` | `status: Done` |
+
+### What it does NOT change
+
+- **L2/L3 body content** — only headers are normalized, your content is untouched
+- **Sub-headers** — `## Objective`, `## Key points` etc. are preserved as-is
+- **`related_docs` paths** — kept on line 1 but not repointed if you move files
+- **Files already in openclew format** — skipped entirely
+- **`_INDEX.md`** — auto-generated, never touched
+
+---
+
+## Step by step
+
+### 1. Update openclew
+
+```bash
+npm install -g openclew@latest
+```
+
+### 2. Check your docs
+
+```bash
+openclew status
+```
+
+Look for the "Legacy format" line. If it says 0, you're done.
+
+### 3. Preview changes
+
+```bash
+openclew migrate
+```
+
+This lists every file that would be converted. No files are modified.
+
+### 4. Apply
+
+```bash
+openclew migrate --write
+```
+
+Each converted file is printed with `✓`.
+
+### 5. Review and commit
+
+```bash
+git diff                    # inspect the changes
+openclew index              # regenerate the index
+git add doc/ && git commit -m "chore: migrate docs to openclew format"
+```
+
+### 6. Verify
+
+```bash
+openclew status             # should show 0 legacy docs
+openclew migrate            # should show "0 to migrate"
+```
+
+---
+
+## After migrating
+
+- **New docs** you create with `openclew add ref` / `openclew add log` already
+  use the current format. No action needed.
+- **Docs created by AI agents** will follow the format they see in your codebase.
+  Once your existing docs are migrated, agents will generate in the new format.
+- **Empty `doc_brief`** — some old docs may have no brief after migration.
+  Run `openclew status` to find them and fill them in.
+
+---
+
+## Version-specific notes
+
+### → 0.4.0 (format migration)
+
+First migration release. Converts from the legacy format (YAML frontmatter,
+plain `key: value` L1, emoji headers) to the openclew format (condensed line 1,
+bold L1 fields, clean headers).
+
+**Scope**: line 1 + L1 block + L2/L3 main headers.
+
+**Not in scope**: sub-header emojis, `related_docs` repointing, recursive
+`doc/` subdirectory scanning (refdocs must be in `doc/_*.md`, not `doc/ref/`).
+
+---
+
+## Limitations
+
+### Flat `doc/` structure required
+
+All openclew tools (search, index, status, migrate) scan `doc/_*.md` for refdocs
+and `doc/log/*.md` for logs. Subdirectories like `doc/ref/` are not scanned yet.
+
+If you plan to reorganize into subdirectories, wait for recursive scan support
+(tracked in the openclew roadmap).
+
+### `related_docs` are not repointed
+
+If a doc references `related_docs: [doc/_AUTH.md]` and you rename that file,
+the reference breaks. `migrate` preserves paths as-is — manual update required.
+
+### Parsers are backward-compatible
+
+Even without migrating, your docs are still readable by openclew tools.
+Migration improves consistency and enables new features, but is not blocking.

package/bin/openclew.js

@@ -9,39 +9,67 @@ const USAGE = `
 openclew — Long Life Memory for LLMs
 
 Usage:
-  openclew init                    Set up openclew in the current project
-  openclew new <title>             Create a living doc (evolves with the project)
-  openclew log <title>             Create a session log (frozen facts)
-  openclew checkout                End-of-session summary + log creation
-  openclew index                   Regenerate doc/_INDEX.md
-  openclew help                    Show this help
+  openclew init                    Set up openclew in your project
+  openclew add ref <title>         Create a refdoc (evolves with the project)
+  openclew add log <title>         Create a session log (frozen facts)
+  openclew search <query>          Search docs by keyword
+  openclew checkout                End-of-session summary
 
-Options:
-  --no-hook                        Skip pre-commit hook installation (init)
-  --no-inject                      Skip instruction file injection (init)
+Run 'openclew help --all' for advanced commands.
+More at: https://github.com/openclew/openclew
+`.trim();
 
-Getting started:
-  npx openclew init                1. Set up doc/ + guide + examples + git hook
-  # Edit doc/_ARCHITECTURE.md      2. Replace the example with your project's architecture
-  openclew new "API design"        3. Create your own living docs
-  git commit                       4. Index auto-regenerates on commit
+const USAGE_ALL = `
+openclew — Long Life Memory for LLMs
 
-Docs have 3 levels: L1 (metadata) → L2 (summary) → L3 (details).
-Agents read L1 to decide what's relevant, then L2 for context.
-More at: https://github.com/openclew/openclew
+Usage:
+  openclew init                    Set up openclew in your project
+  openclew add ref <title>         Create a refdoc (evolves with the project)
+  openclew add log <title>         Create a session log (frozen facts)
+  openclew search <query>          Search docs by keyword
+  openclew checkout                End-of-session summary
+
+Advanced:
+  openclew status                  Documentation health dashboard
+  openclew index                   Regenerate doc/_INDEX.md
+  openclew mcp                     Start MCP server (stdio JSON-RPC)
+
+Options (init):
+  --no-hook                        Skip pre-commit hook installation
+  --no-inject                      Skip instruction file injection
 `.trim();
 
 if (!command || command === "help" || command === "--help" || command === "-h") {
-  console.log(USAGE);
+  const showAll = args.includes("--all");
+  console.log(showAll ? USAGE_ALL : USAGE);
+  process.exit(0);
+}
+
+// Handle "add ref" / "add log" subcommands
+if (command === "add") {
+  const sub = args[1];
+  if (sub === "ref") {
+    require("../lib/new-doc");
+  } else if (sub === "log") {
+    require("../lib/new-log");
+  } else {
+    console.error(`Unknown type: ${sub || "(none)"}`);
+    console.error('Usage: openclew add ref <title>  or  openclew add log <title>');
+    process.exit(1);
+  }
   process.exit(0);
 }
 
+// Legacy aliases
 const commands = {
   init: () => require("../lib/init"),
   new: () => require("../lib/new-doc"),
   log: () => require("../lib/new-log"),
   checkout: () => require("../lib/checkout"),
+  search: () => require("../lib/search"),
+  status: () => require("../lib/status"),
   index: () => require("../lib/index-gen"),
+  mcp: () => require("../lib/mcp-server"),
 };
 
 if (!commands[command]) {

package/lib/checkout.js

@@ -10,9 +10,18 @@
 const fs = require("fs");
 const path = require("path");
 const { execSync } = require("child_process");
-const { logContent, slugifyLog, today } = require("./templates");
+const { slugifyLog, today } = require("./templates");
 const { readConfig } = require("./config");
 
+function ocVersion() {
+  try {
+    const pkg = require(path.join(__dirname, "..", "package.json"));
+    return pkg.version;
+  } catch {
+    return "0.0.0";
+  }
+}
+
 const PROJECT_ROOT = process.cwd();
 const DOC_DIR = path.join(PROJECT_ROOT, "doc");
 const LOG_DIR = path.join(DOC_DIR, "log");
@@ -53,12 +62,12 @@ function collectGitActivity() {
     ? fs.readdirSync(LOG_DIR).filter((f) => f.startsWith(date))
     : [];
 
-  // Living docs
-  const livingDocs = fs.existsSync(DOC_DIR)
+  // Refdocs
+  const refdocs = fs.existsSync(DOC_DIR)
     ? fs.readdirSync(DOC_DIR).filter((f) => f.startsWith("_") && f !== "_INDEX.md" && f.endsWith(".md"))
     : [];
 
-  return { date, commits, uncommitted, files, existingLogs, livingDocs };
+  return { date, commits, uncommitted, files, existingLogs, refdocs };
 }
 
 function extractActions(commits) {
@@ -98,7 +107,7 @@ function typeLabel(type) {
 }
 
 function displaySummary(activity) {
-  const { date, commits, uncommitted, existingLogs, livingDocs } = activity;
+  const { date, commits, uncommitted, existingLogs, refdocs } = activity;
   const actions = extractActions(commits);
 
   console.log(`\nopenclew checkout — ${date}\n`);
@@ -152,10 +161,10 @@ function displaySummary(activity) {
   }
   console.log("");
 
-  // Living docs reminder
-  if (livingDocs.length > 0) {
-    console.log("  📚 Living docs — check if any need updating:");
-    for (const doc of livingDocs) {
+  // Refdocs reminder
+  if (refdocs.length > 0) {
+    console.log("  📚 Refdocs — check if any need updating:");
+    for (const doc of refdocs) {
       console.log(`     ${doc}`);
     }
     console.log("");
@@ -189,15 +198,14 @@ function generateSessionLog(activity, actions) {
     .map((a) => `- ${typeLabel(a.type)}: ${a.desc} (${a.hash})`)
     .join("\n");
 
-  const content = `<!-- L1_START -->
-# L1 - Metadata
-date: ${date}
-type: ${actions.length === 1 ? actions[0].type === "fix" ? "Bug" : "Feature" : "Feature"}
-subject: ${sessionTitle}
-short_story: ${actions.map((a) => a.desc).join(". ")}.
-status: Done
-category:
-keywords: ${keywordsStr}
+  const ver = ocVersion();
+  const logType = actions.length === 1 ? actions[0].type === "fix" ? "Bug" : "Feature" : "Feature";
+  const content = `openclew@${ver} · date: ${date} · type: ${logType} · status: Done · category: · keywords: ${keywordsStr}
+
+<!-- L1_START -->
+**subject:** ${sessionTitle}
+
+**doc_brief:** ${actions.map((a) => a.desc).join(". ")}.
 <!-- L1_END -->
 
 ---

package/lib/index-gen.js

@@ -1,40 +1,115 @@
 /**
  * openclew index — regenerate doc/_INDEX.md
  *
- * Wraps hooks/generate-index.py. Falls back to a JS implementation
- * if Python is not available.
+ * Pure JS implementation. Reuses parsers from search.js (SSOT).
+ * Zero dependencies — Node 16+ standard library only.
  */
 
-const { execSync } = require("child_process");
 const fs = require("fs");
 const path = require("path");
+const { collectDocs } = require("./search");
 
-const docDir = path.join(process.cwd(), "doc");
+/**
+ * Generate _INDEX.md content from parsed docs.
+ *
+ * @param {string} docDir - Absolute path to doc/ directory
+ * @returns {string} Generated index content
+ */
+function generateIndex(docDir) {
+  const docs = collectDocs(docDir);
+  const refdocs = docs.filter((d) => d.kind === "refdoc");
+  const logs = docs.filter((d) => d.kind === "log");
+
+  const now = new Date();
+  const timestamp = `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, "0")}-${String(now.getDate()).padStart(2, "0")} ${String(now.getHours()).padStart(2, "0")}:${String(now.getMinutes()).padStart(2, "0")}`;
+
+  const lines = [
+    "# Project Knowledge Index",
+    "",
+    `> Auto-generated by [openclew](https://github.com/openclew/openclew) on ${timestamp}.`,
+    "> Do not edit manually — rebuilt from L1 metadata on every commit.",
+    "",
+  ];
+
+  // Refdocs section
+  lines.push("## Refdocs");
+  lines.push("");
+  if (refdocs.length) {
+    lines.push("| Document | Subject | Status | Category |");
+    lines.push("|----------|---------|--------|----------|");
+    for (const doc of refdocs) {
+      const name = path.basename(doc.filepath);
+      const subject = doc.meta.subject || "—";
+      const status = doc.meta.status || "—";
+      const category = doc.meta.category || "—";
+      const relPath = path.relative(path.dirname(docDir), doc.filepath);
+      lines.push(`| [${name}](${relPath}) | ${subject} | ${status} | ${category} |`);
+    }
+  } else {
+    lines.push("_No refdocs yet. Create one with `npx openclew add ref \"Title\"`._");
+  }
+  lines.push("");
+
+  // Logs section (last 20)
+  const displayLogs = logs.slice(0, 20);
+  lines.push("## Recent logs");
+  lines.push("");
+  if (displayLogs.length) {
+    lines.push("| Date | Subject | Status | Category |");
+    lines.push("|------|---------|--------|----------|");
+    for (const doc of displayLogs) {
+      const date = doc.meta.date || path.basename(doc.filepath).slice(0, 10);
+      const subject = doc.meta.subject || "—";
+      const status = doc.meta.status || "—";
+      const category = doc.meta.category || "—";
+      const relPath = path.relative(path.dirname(docDir), doc.filepath);
+      lines.push(`| ${date} | [${subject}](${relPath}) | ${status} | ${category} |`);
+    }
+    if (logs.length > 20) {
+      lines.push("");
+      lines.push(`_${logs.length - 20} older logs not shown._`);
+    }
+  } else {
+    lines.push("_No logs yet. Create one with `npx openclew add log \"Title\"`._");
+  }
+  lines.push("");
 
-if (!fs.existsSync(docDir)) {
-  console.error("No doc/ directory found. Run 'openclew init' first.");
-  process.exit(1);
+  // Stats
+  lines.push("---");
+  lines.push(`**${refdocs.length}** refdocs, **${logs.length}** logs.`);
+  lines.push("");
+
+  return lines.join("\n");
 }
 
-// Try local generate-index.py first
-const localScript = path.join(docDir, "generate-index.py");
-const packageScript = path.join(__dirname, "..", "hooks", "generate-index.py");
-const script = fs.existsSync(localScript) ? localScript : packageScript;
-
-if (fs.existsSync(script)) {
-  try {
-    const output = execSync(`python3 "${script}" "${docDir}"`, {
-      encoding: "utf-8",
-    });
-    console.log(output.trim());
-    process.exit(0);
-  } catch {
-    console.error(
-      "python3 not available. Install Python 3.8+ or regenerate manually."
-    );
+/**
+ * Write _INDEX.md to disk.
+ *
+ * @param {string} docDir - Absolute path to doc/ directory
+ * @returns {{ refdocs: number, logs: number }} Counts
+ */
+function writeIndex(docDir) {
+  const content = generateIndex(docDir);
+  const indexPath = path.join(docDir, "_INDEX.md");
+  fs.writeFileSync(indexPath, content, "utf-8");
+
+  const docs = collectDocs(docDir);
+  const refdocs = docs.filter((d) => d.kind === "refdoc").length;
+  const logs = docs.filter((d) => d.kind === "log").length;
+  return { refdocs, logs };
+}
+
+// CLI runner
+if (require.main === module || process.argv.includes("index")) {
+  const docDir = process.argv[2] || path.join(process.cwd(), "doc");
+
+  if (!fs.existsSync(docDir)) {
+    console.error("No doc/ directory found. Run 'openclew init' first.");
     process.exit(1);
   }
+
+  const { refdocs, logs } = writeIndex(docDir);
+  console.log(`Generated doc/_INDEX.md (${refdocs} refdocs, ${logs} logs)`);
 }
 
-console.error("generate-index.py not found. Run 'openclew init' first.");
-process.exit(1);
+module.exports = { generateIndex, writeIndex };