ruvnet-kb-first 5.0.0

package/phases/10-documentation.md

@@ -0,0 +1,613 @@
+# Phase 10: Documentation & Versioning
+
+Updated: 2026-01-02 00:20:00 EST | Version 1.0.0
+Created: 2026-01-02 00:20:00 EST
+
+## Purpose
+
+Ensure the application has complete, accurate documentation and proper versioning before production deployment. Good documentation is not optional—it's a quality requirement.
+
+---
+
+## Prerequisites
+
+- Phase 9 complete (security audit passed)
+- Application fully functional
+- All tests passing
+
+---
+
+## Why Documentation Matters for KB-First Apps
+
+KB-First applications are **knowledge systems**. If the application itself isn't well-documented:
+- Users won't trust the expert knowledge it provides
+- Maintainers won't understand how KB enforcement works
+- Contributors won't know how to add to the KB
+- Operators won't know how to monitor or troubleshoot
+
+**The documentation IS part of the product.**
+
+---
+
+## Sub-Phases
+
+| Sub-Phase | Name | Purpose |
+|-----------|------|---------|
+| 10.1 | README Complete | Project overview, quick start, usage |
+| 10.2 | API Documentation | All endpoints documented |
+| 10.3 | KB Schema Documentation | Knowledge structure documented |
+| 10.4 | Architecture Docs | System design, decisions, diagrams |
+| 10.5 | Operator Guide | Deployment, monitoring, troubleshooting |
+| 10.6 | Versioning Setup | SemVer, changelog, release process |
+
+---
+
+## 10.1 README Complete
+
+The README is the first thing users see. It must answer:
+
+1. **What is this?** (1-2 sentences)
+2. **Why should I use it?** (key benefits)
+3. **How do I get started?** (quick start)
+4. **How do I use it?** (basic usage)
+5. **Where do I get help?** (links to docs, issues)
+
+### README Template
+
+```markdown
+# [Project Name]
+
+[One-line description of what this application does]
+
+## Features
+
+- Feature 1: [Brief description]
+- Feature 2: [Brief description]
+- Feature 3: [Brief description]
+
+## Quick Start
+
+\`\`\`bash
+# Prerequisites
+- Docker
+- Node.js 18+
+
+# Installation
+git clone [repo]
+cd [project]
+npm install
+
+# Start
+docker-compose up -d
+npm run start
+\`\`\`
+
+## Usage
+
+[Basic usage examples]
+
+## Documentation
+
+- [API Reference](docs/api.md)
+- [Architecture](docs/architecture.md)
+- [Operator Guide](docs/operator-guide.md)
+- [Contributing](CONTRIBUTING.md)
+
+## License
+
+[License name] - See [LICENSE](LICENSE) for details.
+```
+
+### Verification
+
+```bash
+#!/bin/bash
+# scripts/10.1-readme-check.sh
+
+echo "=== 10.1 README Completeness ==="
+
+REQUIRED_SECTIONS=(
+  "# "           # Title
+  "## Features"  # or similar
+  "## Quick Start\|## Installation\|## Getting Started"
+  "## Usage"
+  "## Documentation\|## Docs"
+  "## License"
+)
+
+PASS=0
+FAIL=0
+
+for section in "${REQUIRED_SECTIONS[@]}"; do
+  if grep -qE "$section" README.md 2>/dev/null; then
+    echo "✅ Found: $section"
+    PASS=$((PASS + 1))
+  else
+    echo "❌ Missing: $section"
+    FAIL=$((FAIL + 1))
+  fi
+done
+
+echo ""
+echo "Results: $PASS passed, $FAIL failed"
+[ $FAIL -eq 0 ] && exit 0 || exit 1
+```
+
+---
+
+## 10.2 API Documentation
+
+Every API endpoint must be documented with:
+
+1. **Endpoint** (method, path)
+2. **Description** (what it does)
+3. **Authentication** (required or not)
+4. **Request** (parameters, body schema)
+5. **Response** (success and error schemas)
+6. **Example** (curl or code snippet)
+
+### OpenAPI/Swagger
+
+Generate from code or write manually:
+
+```yaml
+# openapi.yaml
+openapi: 3.0.0
+info:
+  title: [App Name] API
+  version: 1.0.0
+  description: API for [description]
+
+paths:
+  /api/search:
+    post:
+      summary: Search the knowledge base
+      description: Semantic search across KB nodes
+      security:
+        - bearerAuth: []
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+                - query
+              properties:
+                query:
+                  type: string
+                  description: Search query
+                  example: "withdrawal rate retirement"
+                namespace:
+                  type: string
+                  description: KB namespace to search
+                limit:
+                  type: integer
+                  default: 10
+                  maximum: 100
+      responses:
+        '200':
+          description: Search results
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  results:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/KBNode'
+                  kbSources:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/KBSource'
+        '401':
+          description: Unauthorized
+        '429':
+          description: Rate limit exceeded
+
+components:
+  schemas:
+    KBNode:
+      type: object
+      properties:
+        id:
+          type: string
+        title:
+          type: string
+        content:
+          type: string
+        source_expert:
+          type: string
+        confidence:
+          type: number
+          minimum: 0
+          maximum: 1
+    KBSource:
+      type: object
+      properties:
+        title:
+          type: string
+        expert:
+          type: string
+        confidence:
+          type: number
+  securitySchemes:
+    bearerAuth:
+      type: http
+      scheme: bearer
+```
+
+### Generate Docs from OpenAPI
+
+```bash
+# Install Swagger UI
+npm install swagger-ui-express
+
+# Or generate static docs
+npx redoc-cli bundle openapi.yaml -o docs/api.html
+```
+
+### Verification
+
+```bash
+#!/bin/bash
+# scripts/10.2-api-docs.sh
+
+echo "=== 10.2 API Documentation ==="
+
+PASS=0
+FAIL=0
+
+# Check for OpenAPI spec
+if [ -f "openapi.yaml" ] || [ -f "openapi.json" ] || [ -f "swagger.yaml" ]; then
+  echo "✅ OpenAPI specification found"
+  PASS=$((PASS + 1))
+else
+  echo "❌ No OpenAPI specification found"
+  FAIL=$((FAIL + 1))
+fi
+
+# Check for API docs
+if [ -f "docs/api.md" ] || [ -f "docs/api.html" ] || [ -d "docs/api" ]; then
+  echo "✅ API documentation found"
+  PASS=$((PASS + 1))
+else
+  echo "❌ No API documentation found"
+  FAIL=$((FAIL + 1))
+fi
+
+# Check that all routes are documented
+ROUTES=$(grep -rh "app\.\(get\|post\|put\|delete\|patch\)" src/ 2>/dev/null | wc -l)
+if [ "$ROUTES" -gt 0 ]; then
+  echo "ℹ️ Found $ROUTES route definitions - ensure all are documented"
+fi
+
+echo ""
+echo "Results: $PASS passed, $FAIL failed"
+[ $FAIL -eq 0 ] && exit 0 || exit 1
+```
+
+---
+
+## 10.3 KB Schema Documentation
+
+Document the knowledge base structure:
+
+### KB Schema Document
+
+```markdown
+# Knowledge Base Schema
+
+## Overview
+
+This knowledge base contains [domain] expertise from [X] experts.
+
+## Namespaces
+
+| Namespace | Description | Node Count |
+|-----------|-------------|------------|
+| [namespace1] | [description] | [count] |
+| [namespace2] | [description] | [count] |
+
+## Node Structure
+
+Each KB node contains:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| id | UUID | Unique identifier |
+| title | string | Node title (searchable) |
+| content | text | Full content |
+| path | ltree | Hierarchical path |
+| source_expert | string | Expert attribution |
+| confidence | float | Confidence score (0-1) |
+| embedding | vector(384) | Semantic embedding |
+| namespace | string | Isolation namespace |
+| created_at | timestamp | Creation time |
+| updated_at | timestamp | Last update |
+
+## Topic Hierarchy
+
+[Tree diagram or list showing KB structure]
+
+## Expert Sources
+
+| Expert | Domain | Node Count | Avg Confidence |
+|--------|--------|------------|----------------|
+| [name] | [domain] | [count] | [avg] |
+
+## Gap Tracking
+
+Unanswered queries are logged to `kb_gaps` for KB improvement.
+
+| Field | Description |
+|-------|-------------|
+| query | The unanswered query |
+| context | Additional context (JSON) |
+| created_at | When the gap was detected |
+```
+
+---
+
+## 10.4 Architecture Documentation
+
+### Required Diagrams
+
+1. **System Context** - How the app fits in the larger ecosystem
+2. **Container Diagram** - Major components (app, DB, external services)
+3. **Component Diagram** - Internal structure
+4. **Data Flow** - How data moves through the system
+
+### Architecture Decision Records (ADRs)
+
+Document key decisions:
+
+```markdown
+# ADR-001: Use PostgreSQL with pgvector for KB Storage
+
+## Status
+Accepted
+
+## Context
+We need persistent storage for the knowledge base with semantic search capabilities.
+
+## Decision
+Use PostgreSQL with the pgvector extension for vector storage and similarity search.
+
+## Consequences
+- **Positive:** Single database for all data, mature ecosystem, strong consistency
+- **Negative:** Requires vector extension, may need tuning for large KBs
+- **Neutral:** Team already familiar with PostgreSQL
+```
+
+### Architecture Document Template
+
+```markdown
+# Architecture Documentation
+
+## System Overview
+
+[High-level description]
+
+## Components
+
+### Knowledge Base Layer
+- PostgreSQL with pgvector
+- kb_nodes table with embeddings
+- HNSW index for fast search
+
+### Intelligence Layer
+- GNN for decision modeling (if applicable)
+- Attention routing (if applicable)
+- SONA learning (if applicable)
+
+### Application Layer
+- Node.js/Express API
+- React frontend
+- KB client library
+
+### Infrastructure
+- Docker containers
+- [Cloud provider] hosting
+- [CDN] for static assets
+
+## Data Flow
+
+[Diagram or description]
+
+## Security Model
+
+[Authentication, authorization, encryption]
+
+## Scalability
+
+[How the system scales]
+```
+
+---
+
+## 10.5 Operator Guide
+
+For teams deploying and maintaining the application:
+
+```markdown
+# Operator Guide
+
+## Prerequisites
+
+- Docker 20+
+- 4GB RAM minimum
+- 20GB disk space
+
+## Deployment
+
+### Environment Variables
+
+| Variable | Required | Description | Example |
+|----------|----------|-------------|---------|
+| DATABASE_URL | Yes | PostgreSQL connection | postgres://... |
+| JWT_SECRET | Yes | JWT signing key | [random 256-bit] |
+| ALLOWED_ORIGINS | Yes | CORS origins | https://app.com |
+
+### Docker Deployment
+
+\`\`\`bash
+docker-compose up -d
+\`\`\`
+
+### Kubernetes Deployment
+
+[Helm chart or kubectl instructions]
+
+## Monitoring
+
+### Health Checks
+
+- `/health` - Basic health
+- `/health/db` - Database connectivity
+- `/health/kb` - KB accessibility
+
+### Metrics
+
+Prometheus metrics available at `/metrics`:
+- `kb_search_duration_seconds` - Search latency
+- `kb_gap_total` - Gap count
+- `kb_node_count` - Total KB nodes
+
+### Alerts
+
+[Recommended alerting thresholds]
+
+## Troubleshooting
+
+### Common Issues
+
+#### "KB unavailable" at startup
+1. Check DATABASE_URL is correct
+2. Verify PostgreSQL is running
+3. Check network connectivity
+
+#### Slow searches
+1. Check HNSW index exists
+2. Monitor query explain plans
+3. Consider increasing work_mem
+
+## Backup & Recovery
+
+### Backup
+
+\`\`\`bash
+pg_dump $DATABASE_URL > backup.sql
+\`\`\`
+
+### Recovery
+
+\`\`\`bash
+psql $DATABASE_URL < backup.sql
+\`\`\`
+
+## Updates
+
+[Process for updating the application]
+```
+
+---
+
+## 10.6 Versioning Setup
+
+### Semantic Versioning
+
+Use [SemVer](https://semver.org/):
+- **MAJOR** (X.0.0): Breaking changes
+- **MINOR** (0.X.0): New features, backward compatible
+- **PATCH** (0.0.X): Bug fixes, backward compatible
+
+### package.json Version
+
+```json
+{
+  "name": "your-app",
+  "version": "1.0.0"
+}
+```
+
+### CHANGELOG.md
+
+```markdown
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+- [New feature]
+
+### Changed
+- [Changed feature]
+
+### Fixed
+- [Bug fix]
+
+## [1.0.0] - 2026-01-02
+
+### Added
+- Initial release
+- KB-First architecture implementation
+- Phase 0-11 build process
+- Security hardening
+- Complete documentation
+
+[Unreleased]: https://github.com/user/repo/compare/v1.0.0...HEAD
+[1.0.0]: https://github.com/user/repo/releases/tag/v1.0.0
+```
+
+### Version Bump Script
+
+```bash
+#!/bin/bash
+# scripts/bump-version.sh
+
+VERSION_TYPE=${1:-patch}  # major, minor, or patch
+
+# Bump version
+npm version $VERSION_TYPE --no-git-tag-version
+
+# Get new version
+NEW_VERSION=$(node -p "require('./package.json').version")
+
+echo "Bumped to version $NEW_VERSION"
+echo ""
+echo "Next steps:"
+echo "  1. Update CHANGELOG.md"
+echo "  2. Commit: git commit -am 'Release v$NEW_VERSION'"
+echo "  3. Tag: git tag v$NEW_VERSION"
+echo "  4. Push: git push && git push --tags"
+```
+
+---
+
+## Quality Gate Checklist
+
+Before proceeding to Phase 11, verify:
+
+- [ ] README is complete (10.1 script passes)
+- [ ] API documentation exists (OpenAPI spec or equivalent)
+- [ ] KB schema is documented
+- [ ] Architecture documentation exists with diagrams
+- [ ] Operator guide is complete
+- [ ] CHANGELOG.md exists and is current
+- [ ] Version is set correctly in package.json
+- [ ] All documentation has been reviewed for accuracy
+
+---
+
+## Exit Criteria
+
+All documentation is complete and accurate. Versioning is properly configured.
+
+**Proceed to Phase 11: Deployment Planning**