openclew 0.2.0 → 0.3.0

package/templates/FORMAT.md

@@ -0,0 +1,299 @@
+# openclew document format
+
+> SSOT — this file defines the canonical format for all openclew documents.
+> Templates (`refdoc.md`, `log.md`), embedded templates (`lib/templates.js`), and parsers (`generate-index.py`, `lib/search.js`) must conform to this spec.
+
+---
+
+## File structure
+
+Every openclew document is a single Markdown file with 4 sections:
+
+```
+┌─────────────────────────────────────────────────┐
+│  Line 1 — Metadata line                         │
+│  Condensed key-value pairs for indexing          │
+├─────────────────────────────────────────────────┤
+│  L1 — Subject + Brief                           │
+│  ~40 tokens. "Should I read this?"              │
+├─────────────────────────────────────────────────┤
+│  L2 — Summary                                   │
+│  Essential context. Enough for most decisions.   │
+├─────────────────────────────────────────────────┤
+│  L3 — Details                                   │
+│  Full technical content. Deep-dive only.         │
+└─────────────────────────────────────────────────┘
+```
+
+---
+
+## Line 1 — Metadata
+
+A single line of condensed key-value pairs separated by ` · `. Always the first line of the file, no blank line before it.
+
+### Refdoc format
+
+```
+openclew@VERSION · created: YYYY-MM-DD · updated: YYYY-MM-DD · type: TYPE · status: STATUS · category: CATEGORY · keywords: [tag1, tag2]
+```
+
+### Log format
+
+```
+openclew@VERSION · date: YYYY-MM-DD · type: TYPE · status: STATUS · category: CATEGORY · keywords: [tag1, tag2]
+```
+
+### Fields
+
+| Field | Refdoc | Log | Description |
+|-------|:------:|:---:|-------------|
+| `openclew@VERSION` | ✅ | ✅ | Package version that created the doc |
+| `created` | ✅ | — | Creation date |
+| `updated` | ✅ | — | Last update date |
+| `date` | — | ✅ | Session date |
+| `type` | ✅ | ✅ | Document type (see below) |
+| `status` | ✅ | ✅ | Document status (see below) |
+| `category` | ✅ | ✅ | Main domain (free text) |
+| `keywords` | ✅ | ✅ | Tags for search `[tag1, tag2]` |
+
+### Types
+
+| Type | Usage |
+|------|-------|
+| `Reference` | Durable knowledge (architecture, conventions, decisions) |
+| `Architecture` | Structural design document |
+| `Guide` | How-to, onboarding, process explanation |
+| `Analysis` | Investigation, comparison, study |
+| `Bug` | Bug investigation and fix |
+| `Feature` | New functionality |
+| `Refactor` | Code restructuring |
+| `Doc` | Documentation-only change |
+| `Deploy` | Deployment or release |
+
+### Statuses
+
+| Status | Refdoc | Log | Description |
+|--------|:------:|:---:|-------------|
+| `Active` | ✅ | — | Living document, actively maintained |
+| `Stable` | ✅ | — | Mature, rarely updated |
+| `Archived` | ✅ | — | No longer relevant, kept for history |
+| `In progress` | — | ✅ | Work ongoing |
+| `Done` | — | ✅ | Work completed |
+| `Abandoned` | — | ✅ | Work stopped, approach not viable |
+
+---
+
+## L1 — Subject + Brief
+
+Between `<!-- L1_START -->` and `<!-- L1_END -->` markers. Two fields only:
+
+```markdown
+<!-- L1_START -->
+**subject:** Short title (< 60 chars)
+
+**doc_brief:** 1-2 sentences of **assertions** (what is true), not narration (what was done). Must be enough to decide if you need to read further.
+<!-- L1_END -->
+```
+
+### Rules
+
+- **subject** is the document's title. Keep it short and scannable.
+- **doc_brief** answers: "Should I read this?" Write **assertions** — state what is true, decided, or concluded. Don't narrate what was done. Quick test: remove "we investigated" / "we compared" — if the sentence collapses, rewrite it.
+  - Bad: "Investigated memory leak in worker pool. Profiled with pprof, tested several fixes."
+  - Good: "Worker pool leaked memory via unclosed response bodies in retry path. Fixed with deferred close. Steady at 120MB under load."
+- Both fields use `**bold_key:**` syntax (not YAML `key: value`).
+- No other fields in L1. All metadata lives on line 1.
+
+---
+
+## L2 — Summary
+
+Between `<!-- L2_START -->` and `<!-- L2_END -->` markers. Starts with `# L2 - Summary`.
+
+```markdown
+<!-- L2_START -->
+# L2 - Summary
+
+## Objective
+<!-- Why this document exists / why this work was done -->
+
+## Key points
+<!-- 3-5 essential takeaways -->
+
+## Solution
+<!-- Recommended approach or what was implemented -->
+
+## Watch out
+<!-- Pitfalls, edge cases, limitations -->
+<!-- L2_END -->
+```
+
+### Rules
+
+- Must fit on one screen (~40 lines of ~80 chars).
+- No emojis in headers.
+- Sections are suggested, not mandatory — adapt to the content.
+- Refdocs typically use: Objective, Key points, Solution, Watch out.
+- Logs typically use: Objective, Problem, Solution, Watch out.
+
+---
+
+## L3 — Details
+
+Between `<!-- L3_START -->` and `<!-- L3_END -->` markers. Starts with `# L3 - Details`.
+
+```markdown
+<!-- L3_START -->
+# L3 - Details
+
+<!-- Full technical content: code examples, diagrams, deep dives... -->
+
+---
+
+## Changelog
+
+| Date | Change |
+|------|--------|
+| YYYY-MM-DD | Initial creation |
+<!-- L3_END -->
+```
+
+### Rules
+
+- Exhaustive — include everything that might be needed later.
+- Code examples, before/after, commands, links.
+- Changelog table at the end (refdocs only — logs are immutable).
+
+---
+
+## Naming conventions
+
+| Type | Location | Naming |
+|------|----------|--------|
+| Refdoc | `doc/_SUBJECT.md` | UPPER_SNAKE_CASE, prefixed `_` |
+| Log | `doc/log/YYYY-MM-DD_subject.md` | lowercase-with-hyphens, dated |
+| Index | `doc/_INDEX.md` | Auto-generated, never edit manually |
+
+---
+
+## Parser compatibility
+
+The `generate-index.py` parser extracts:
+1. **Line 1 metadata**: splits on ` · `, parses `key: value` pairs
+2. **L1 fields**: regex for `**subject:**` and `**doc_brief:**` between L1 markers
+
+**Legacy fallback**: if no `**subject:**` is found in L1, the parser falls back to plain `key: value` lines (for docs created before the `**bold:**` convention).
+
+**Line 1 prefix**: must start with `openclew@`. Files starting with other prefixes (e.g. `R.AlphA.Doc@`) are not parsed by the openclew toolchain but remain valid Markdown.
+
+---
+
+## Complete examples
+
+### Refdoc example
+
+```markdown
+openclew@0.2.1 · created: 2026-03-07 · updated: 2026-03-20 · type: Reference · status: Active · category: Auth · keywords: [JWT, sessions, Redis]
+
+<!-- L1_START -->
+**subject:** Authentication architecture
+
+**doc_brief:** JWT-based auth with refresh tokens. Sessions stored in Redis with 15-min expiry. Google OAuth as sole provider. Token refresh handled client-side.
+<!-- L1_END -->
+
+---
+
+<!-- L2_START -->
+# L2 - Summary
+
+## Objective
+Document the auth architecture so agents and new devs can work on auth-related code without re-investigating.
+
+## Key points
+- JWT access tokens (15 min) + refresh tokens (7 days)
+- Redis for session storage (not Postgres — latency matters)
+- Google OAuth only (no email/password)
+- Refresh handled client-side via interceptor
+
+## Watch out
+- Refresh endpoint has no rate limiting yet — track issue #42
+- Test tokens in .env.test, never commit real secrets
+<!-- L2_END -->
+
+---
+
+<!-- L3_START -->
+# L3 - Details
+
+## Token flow
+Client → /auth/google → JWT + refresh token
+Client → /auth/refresh → new JWT (if refresh valid)
+
+## Key files
+- src/routes/auth.ts — OAuth + refresh endpoints
+- src/middleware/auth.ts — JWT verification
+- src/services/session.ts — Redis session management
+
+---
+
+## Changelog
+
+| Date | Change |
+|------|--------|
+| 2026-03-20 | Added rate limiting note |
+| 2026-03-07 | Initial creation |
+<!-- L3_END -->
+```
+
+### Log example
+
+```markdown
+openclew@0.2.1 · date: 2026-03-15 · type: Bug · status: Done · category: Auth · keywords: [token, refresh, race condition]
+
+<!-- L1_START -->
+**subject:** Fix token refresh race condition
+
+**doc_brief:** Two concurrent requests could trigger double refresh, invalidating both tokens. Fixed with a mutex in the interceptor. No more 401 cascades.
+<!-- L1_END -->
+
+---
+
+<!-- L2_START -->
+# L2 - Summary
+
+## Objective
+Fix intermittent 401 errors when multiple API calls happen during token refresh.
+
+## Problem
+Two concurrent requests both detect an expired token and both call /auth/refresh. The second call invalidates the first's new token, causing a 401 cascade.
+
+## Solution
+Added a refresh mutex in the HTTP interceptor. First request triggers refresh, others wait for the result. Single refresh call per expiry cycle.
+<!-- L2_END -->
+
+---
+
+<!-- L3_START -->
+# L3 - Details
+
+## Root cause
+The interceptor checked `isTokenExpired()` synchronously but `refreshToken()` was async. Between the check and the refresh response, other requests would also see the expired token and trigger their own refresh.
+
+## Fix (src/lib/http-client.ts)
+```ts
+let refreshPromise = null;
+
+async function ensureValidToken() {
+  if (!isTokenExpired()) return;
+  if (!refreshPromise) {
+    refreshPromise = refreshToken().finally(() => { refreshPromise = null; });
+  }
+  await refreshPromise;
+}
+```
+
+## Testing
+- Added test: 10 concurrent requests during token expiry → exactly 1 refresh call
+- Manual test: rapid navigation between pages during expiry window
+<!-- L3_END -->
+```

package/templates/log.md

@@ -1,12 +1,9 @@
+openclew@VERSION · date: YYYY-MM-DD · type: Bug | Feature | Refactor | Doc | Deploy · status: Done | In progress | Abandoned · category: Main domain · keywords: [tag1, tag2, tag3]
+
 <!-- L1_START -->
-# L1 - Metadata
-date: YYYY-MM-DD
-type: Bug | Feature | Refactor | Doc | Deploy
-subject: Short title (< 60 chars)
-short_story: 1-2 sentences. What happened and what was the outcome. Include conclusions, not just process.
-status: Done | In progress | Abandoned
-category: Main domain (e.g. Auth, API, Database, UI...)
-keywords: [tag1, tag2, tag3]
+**subject:** Short title (< 60 chars)
+
+**doc_brief:** 1-2 sentences of assertions (what is true/decided), not narration (what was done). Remove "we did X" — state the result.
 <!-- L1_END -->
 
 ---

package/templates/onboarding/flow.md

@@ -0,0 +1,59 @@
+# Onboarding openclew — Flow
+
+## Principe
+
+L'onboarding openclew guide un utilisateur pour structurer la documentation de son projet.
+Le résultat minimum = un `doc/_INDEX.md` (point d'entrée) lisible par les agents IA.
+
+## Détection
+
+| Signal | Résultat |
+|--------|----------|
+| `doc/_INDEX.md` existe | Projet configuré → pas d'onboarding |
+| `.openclew.json` existe | Projet configuré → pas d'onboarding |
+| `doc/_*.md` existe (sans _INDEX) | Projet partiel → proposer complétion |
+| Rien | Projet vierge → proposer setup complet |
+
+## Étapes
+
+### 1. Détection automatique (client)
+
+Au lancement, le client IDE scanne le workspace :
+- Cherche `doc/_INDEX.md` ou `.openclew.json`
+- Si absent → affiche un CTA "Configurer la documentation projet"
+- Si présent → flow normal (enrichissement L1/L2)
+
+### 2. Scaffold (bot-guided)
+
+Quand l'utilisateur accepte, le bot :
+1. Crée `doc/` si absent
+2. Scanne les fichiers existants (README, code, config) pour comprendre le projet
+3. Génère `doc/_INDEX.md` à partir du template `scaffold_index.md`
+4. Propose un premier refdoc (`doc/_ARCHITECTURE.md` ou similaire) basé sur ce qu'il a détecté
+
+### 3. Validation
+
+Le bot montre le résultat et demande confirmation. L'utilisateur peut :
+- Accepter tel quel
+- Demander des modifications
+- Ajouter d'autres refdocs
+
+### 4. Complétion
+
+L'onboarding est marqué `completed` pour ce workspace. Le knowledge provider prend le relais normalement.
+
+## Déclenchement
+
+| Méthode | Quand |
+|---------|-------|
+| Automatique | Premier message dans un projet sans openclew |
+| Manuel | Slash command `/setup` |
+| Re-déclenchement | `/setup` fonctionne même si déjà complété (pour ajouter des docs) |
+
+## Prompt bot
+
+Le bot reçoit un contexte enrichi qui l'instruit de :
+1. Lister les fichiers du projet (via `list_files`)
+2. Identifier le langage principal, le framework, la structure
+3. Créer `doc/_INDEX.md` avec les métadonnées détectées
+4. Proposer 1-3 refdocs pertinents selon le type de projet

package/templates/onboarding/scaffold_index.md

@@ -0,0 +1,31 @@
+openclew@1.0.0 · created: {{DATE}} · updated: {{DATE}} · type: Reference · status: Active · category: Index · keywords: [index, documentation, entry-point]
+
+<!-- L1_START -->
+**subject:** {{PROJECT_NAME}} — Documentation index
+
+**doc_brief:** Point d'entrée documentation du projet {{PROJECT_NAME}}. Liste les refdocs disponibles avec leur sujet et statut.
+<!-- L1_END -->
+
+---
+
+<!-- L2_START -->
+# L2 - Summary
+
+## Project
+
+- **Name**: {{PROJECT_NAME}}
+- **Stack**: {{STACK}}
+- **Description**: {{DESCRIPTION}}
+
+## Refdocs
+
+| Document | Subject | Status |
+|----------|---------|--------|
+{{REFDOCS_TABLE}}
+
+## Logs
+
+Session logs in `doc/log/`. Most recent:
+
+{{RECENT_LOGS}}
+<!-- L2_END -->

package/templates/{living.md → refdoc.md}

@@ -1,13 +1,9 @@
+openclew@VERSION · created: YYYY-MM-DD · updated: YYYY-MM-DD · type: Reference | Architecture | Guide | Analysis · status: Active | Stable | Archived · category: Main domain · keywords: [tag1, tag2, tag3]
+
 <!-- L1_START -->
-# L1 - Metadata
-type: Reference | Architecture | Guide | Analysis
-subject: Short title (< 60 chars)
-created: YYYY-MM-DD
-updated: YYYY-MM-DD
-short_story: 1-2 sentences. What this doc covers and what it concludes. Must be enough to decide if you need to read further.
-status: Active | Stable | Archived
-category: Main domain (e.g. Auth, API, Database, UI...)
-keywords: [tag1, tag2, tag3]
+**subject:** Short title (< 60 chars)
+
+**doc_brief:** 1-2 sentences of assertions (what is true/decided), not narration (what was done). Must be enough to decide if you need to read further.
 <!-- L1_END -->
 
 ---

package/hooks/generate-index.py

@@ -1,157 +0,0 @@
-#!/usr/bin/env python3
-"""
-openclew index generator.
-
-Scans doc/_*.md (living docs) and doc/log/*.md (logs),
-parses L1 metadata blocks, and generates doc/_INDEX.md.
-
-Usage:
-    python generate-index.py              # from project root
-    python generate-index.py /path/to/doc # custom doc directory
-
-Idempotent: running twice produces the same output.
-Zero dependencies: Python 3.8+ standard library only.
-"""
-
-import os
-import re
-import sys
-from datetime import datetime
-from pathlib import Path
-
-
-def find_doc_dir():
-    """Find the doc/ directory."""
-    if len(sys.argv) > 1:
-        doc_dir = Path(sys.argv[1])
-    else:
-        doc_dir = Path("doc")
-
-    if not doc_dir.is_dir():
-        print(f"No '{doc_dir}' directory found. Nothing to index.")
-        sys.exit(0)
-
-    return doc_dir
-
-
-def parse_l1(filepath):
-    """Extract L1 metadata from a file."""
-    try:
-        content = filepath.read_text(encoding="utf-8")
-    except (OSError, UnicodeDecodeError):
-        return None
-
-    match = re.search(
-        r"<!--\s*L1_START\s*-->(.+?)<!--\s*L1_END\s*-->",
-        content,
-        re.DOTALL,
-    )
-    if not match:
-        return None
-
-    block = match.group(1)
-    meta = {}
-    for line in block.splitlines():
-        line = line.strip()
-        if line.startswith("#") or not line:
-            continue
-        if ":" in line:
-            key, _, value = line.partition(":")
-            meta[key.strip().lower()] = value.strip()
-
-    return meta
-
-
-def collect_docs(doc_dir):
-    """Collect living docs and logs with their L1 metadata."""
-    living_docs = []
-    logs = []
-
-    # Living docs: doc/_*.md
-    for f in sorted(doc_dir.glob("_*.md")):
-        if f.name == "_INDEX.md":
-            continue
-        meta = parse_l1(f)
-        if meta:
-            living_docs.append((f, meta))
-
-    # Log docs: doc/log/*.md
-    log_dir = doc_dir / "log"
-    if log_dir.is_dir():
-        for f in sorted(log_dir.glob("*.md"), reverse=True):
-            meta = parse_l1(f)
-            if meta:
-                logs.append((f, meta))
-
-    return living_docs, logs
-
-
-def generate_index(doc_dir, living_docs, logs):
-    """Generate _INDEX.md content."""
-    now = datetime.now().strftime("%Y-%m-%d %H:%M")
-    lines = [
-        f"# Project Knowledge Index",
-        f"",
-        f"> Auto-generated by [openclew](https://github.com/openclew/openclew) on {now}.",
-        f"> Do not edit manually — rebuilt from L1 metadata on every commit.",
-        f"",
-    ]
-
-    # Living docs section
-    lines.append("## Living docs")
-    lines.append("")
-    if living_docs:
-        lines.append("| Document | Subject | Status | Category |")
-        lines.append("|----------|---------|--------|----------|")
-        for f, meta in living_docs:
-            name = f.name
-            subject = meta.get("subject", "—")
-            status = meta.get("status", "—")
-            category = meta.get("category", "—")
-            rel_path = f.relative_to(doc_dir.parent)
-            lines.append(f"| [{name}]({rel_path}) | {subject} | {status} | {category} |")
-    else:
-        lines.append("_No living docs yet. Create one with `templates/living.md`._")
-    lines.append("")
-
-    # Logs section (last 20)
-    lines.append("## Recent logs")
-    lines.append("")
-    display_logs = logs[:20]
-    if display_logs:
-        lines.append("| Date | Subject | Status | Category |")
-        lines.append("|------|---------|--------|----------|")
-        for f, meta in display_logs:
-            date = meta.get("date", f.stem[:10])
-            subject = meta.get("subject", "—")
-            status = meta.get("status", "—")
-            category = meta.get("category", "—")
-            rel_path = f.relative_to(doc_dir.parent)
-            lines.append(f"| {date} | [{subject}]({rel_path}) | {status} | {category} |")
-        if len(logs) > 20:
-            lines.append(f"")
-            lines.append(f"_{len(logs) - 20} older logs not shown._")
-    else:
-        lines.append("_No logs yet. Create one with `templates/log.md`._")
-    lines.append("")
-
-    # Stats
-    lines.append("---")
-    lines.append(f"**{len(living_docs)}** living docs, **{len(logs)}** logs.")
-    lines.append("")
-
-    return "\n".join(lines)
-
-
-def main():
-    doc_dir = find_doc_dir()
-    living_docs, logs = collect_docs(doc_dir)
-    index_content = generate_index(doc_dir, living_docs, logs)
-
-    index_path = doc_dir / "_INDEX.md"
-    index_path.write_text(index_content, encoding="utf-8")
-    print(f"Generated {index_path} ({len(living_docs)} living docs, {len(logs)} logs)")
-
-
-if __name__ == "__main__":
-    main()