@documentation-robotics/cli 0.1.0 → 0.1.7

package/docs/telemetry.md

@@ -1,427 +0,0 @@
-# Telemetry Verification and Implementation Guide
-
-This document provides a comprehensive guide to verifying the complete telemetry pipeline implementation for the Documentation Robotics CLI. All components have been implemented and are ready for verification.
-
-## Implementation Summary
-
-### What Has Been Implemented
-
-#### 1. Core Telemetry Infrastructure
-
-- **`telemetry/index.ts`**: Main module providing:
-  - OpenTelemetry SDK initialization with project name context from manifest
-  - Span creation and management utilities (`startSpan()`, `endSpan()`)
-  - Log emission with trace context correlation (`emitLog()`)
-  - Graceful shutdown (`shutdownTelemetry()`)
-  - Resource attributes including `dr.project.name` for all traces and logs
-
-- **`telemetry/resilient-exporter.ts`**: Circuit-breaker pattern for trace export
-  - 500ms timeout to prevent blocking CLI execution
-  - 30-second backoff on failure
-  - Graceful degradation when OTLP collector is unavailable
-
-- **`telemetry/resilient-log-exporter.ts`**: Circuit-breaker pattern for log export
-  - Mirrors resilient trace exporter design
-  - Independent log pipeline with same protection
-
-#### 2. Console Integration
-
-- **`telemetry/console-interceptor.ts`**: Automatic console capture
-  - Intercepts `console.log()` → INFO severity
-  - Intercepts `console.error()` → ERROR severity with stack traces
-  - Intercepts `console.warn()` → WARN severity
-  - Intercepts `console.debug()` → DEBUG severity
-  - Preserves original console behavior (output still appears)
-  - All logs correlated to active trace context
-
-#### 3. Test Instrumentation
-
-- **`telemetry/test-instrumentation.ts`**: Test span creation utilities
-  - `startTestFileSpan(filePath)`: Creates file-level span with `test.file` and `test.framework` attributes
-  - `createTestCaseSpan(testName, suiteName)`: Creates test case span with standard attributes
-  - `recordTestResult(span, status, error)`: Records test status and errors
-  - `instrumentTest(testName, testFn, suiteName)`: Helper wrapper for automatic span management
-
-- **`tests/unit/test-instrumentation.example.test.ts`**: Comprehensive example showing all features
-
-#### 4. CLI Root Instrumentation
-
-- **`cli.ts`**: Span instrumentation for every command
-  - Root span `cli.execute` created in `preAction` hook
-  - Span attributes: `command`, `args`, `cwd`, `version`
-  - Proper error recording with `recordException()`
-  - Graceful shutdown in both `beforeExit` and `exit` handlers
-
-#### 5. Validator Instrumentation
-
-- **`validators/validator.ts`**: Nested span structure
-  - Root span: `model.validate`
-  - Child spans:
-    - `validation.stage.schema`: Schema validation
-    - `validation.stage.naming`: Naming convention validation
-    - `validation.stage.reference`: Cross-layer reference validation
-    - `validation.stage.semantic`: Business rule validation
-
-#### 6. Build Configuration
-
-- **`esbuild.config.js`**: Compile-time telemetry control
-  - `TELEMETRY_ENABLED` constant set by `DR_TELEMETRY` environment variable
-  - Dead-code elimination removes all telemetry from production builds
-  - Debug build: `npm run build:debug` (with telemetry enabled)
-  - Production build: `npm run build` (zero overhead)
-
-#### 7. Project Name Integration
-
-- **Project context propagation** via manifest loading
-  - Attempts to load `.dr/manifest.json` at initialization
-  - Falls back to `'unknown'` if no manifest exists (e.g., during `dr init`)
-  - Stored as resource attribute `dr.project.name`
-  - Automatically included in all traces and logs
-
-## Verification Checklist
-
-### Phase 1: Build and Setup
-
-- [ ] **Build debug CLI**:
-  ```bash
-  cd /workspace/cli
-  npm run build:debug
-  ```
-  Expected: No errors, telemetry enabled in output
-
-- [ ] **Verify build artifacts**:
-  ```bash
-  ls -la dist/cli.js
-  # Approximately 400-500KB (with telemetry)
-  ```
-
-- [ ] **Verify development build has telemetry**:
-  ```bash
-  npm run build
-  ls -la dist/cli.js
-  # Approximately 300-400KB (without telemetry)
-  ```
-
-### Phase 2: Project Name Propagation
-
-- [ ] **Create test model**:
-  ```bash
-  cd /tmp && rm -rf test-model && mkdir test-model && cd test-model
-  /workspace/cli/dist/cli.js init --name "VerificationModel"
-  cat .dr/manifest.json | grep '"name"'
-  ```
-  Expected: `"name": "VerificationModel"`
-
-- [ ] **Verify project name loading**:
-  ```bash
-  /workspace/cli/dist/cli.js validate
-  ```
-  When telemetry is enabled, traces will include `dr.project.name: "VerificationModel"`
-
-### Phase 3: Console Logging Verification
-
-- [ ] **Run validator to see console output**:
-  ```bash
-  cd /tmp/test-model
-  /workspace/cli/dist/cli.js validate
-  ```
-  Expected output includes validation messages like:
-  ```
-  ✓ Schema validation passed
-  ✓ Naming validation passed
-  ✓ Reference validation passed
-  ✓ Semantic validation passed
-  ```
-
-- [ ] **Test error logging**:
-  ```bash
-  # Create an invalid element to trigger errors
-  /workspace/cli/dist/cli.js add invalid-layer invalid-type test-element
-  ```
-  Expected: Error messages appear in console (and would be captured as ERROR level logs)
-
-### Phase 4: Span Nesting Verification
-
-- [ ] **Inspect CLI span in code**:
-  ```bash
-  grep -n "startSpan('cli.execute'" /workspace/cli/src/cli.ts
-  ```
-  Expected: Found at line ~69
-
-- [ ] **Inspect validator spans in code**:
-  ```bash
-  grep -n "startSpan('model.validate'" /workspace/cli/src/validators/validator.ts
-  grep -n "startSpan('validation.stage" /workspace/cli/src/validators/validator.ts
-  ```
-  Expected: Multiple stages found with proper nesting
-
-### Phase 5: Test Instrumentation Verification
-
-- [ ] **Review example test file**:
-  ```bash
-  head -100 /workspace/cli/tests/unit/test-instrumentation.example.test.ts
-  ```
-  Expected: Shows `beforeAll()`, `afterAll()`, `instrumentTest()` usage
-
-- [ ] **Run unit tests**:
-  ```bash
-  cd /workspace/cli
-  npm run test:unit
-  ```
-  Expected: All tests pass (100+ test files)
-
-### Phase 6: Circuit-Breaker Verification
-
-- [ ] **Verify circuit-breaker code**:
-  ```bash
-  grep -A5 "retryAfter =" /workspace/cli/src/telemetry/resilient-exporter.ts
-  ```
-  Expected: Circuit-breaker logic present with 30s backoff
-
-- [ ] **Verify graceful handling**:
-  ```bash
-  # When OTEL collector is unavailable, this command completes successfully
-  # with no user-visible errors or blocking
-  OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:9999 \
-  /workspace/cli/dist/cli.js validate
-  ```
-  Expected: Command completes in <2 seconds, no error output
-
-## Detailed Verification Instructions
-
-### SigNoz Stack Setup (Optional)
-
-If you want to fully verify the telemetry pipeline with SigNoz:
-
-1. **Prerequisites**:
-   - Docker and Docker Compose installed
-   - At least 4GB free disk space
-
-2. **Start SigNoz** (using the example configuration):
-   ```bash
-   cd /workspace
-   ./docs/otel_example/signoz-stack.sh start
-   # Wait ~30 seconds for services to initialize
-   ```
-
-   **Note:** This uses the example SigNoz stack in `docs/otel_example/`. You can also use your own OTEL collector.
-
-3. **Access SigNoz UI**:
-   - Open http://localhost:3301 in your browser
-   - Dashboard should load with no errors
-
-4. **Send telemetry**:
-   ```bash
-   cd /tmp/test-model
-   npm run build:debug  # In CLI directory
-   node /workspace/cli/dist/cli.js validate
-   ```
-
-5. **Verify in SigNoz**:
-   - Navigate to Traces section
-   - Look for spans named `cli.execute`
-   - Expand to see nested `validation.stage.*` spans
-   - Click on a span to see attributes and logs
-
-6. **Filter by project**:
-   - Use filter: `dr.project.name = "VerificationModel"`
-   - Should show only traces from your test model
-
-7. **View logs**:
-   - Click on a span
-   - Switch to "Logs" tab
-   - Console output should appear with correlation IDs
-
-8. **Stop SigNoz** (if using the example stack):
-   ```bash
-   ./docs/otel_example/signoz-stack.sh stop
-   ```
-
-## Architecture Overview
-
-```
-┌─────────────────────────────────────────────────────────┐
-│                   CLI Execution                          │
-├─────────────────────────────────────────────────────────┤
-│                                                           │
-│  ┌─────────────────────────────────────────────────┐    │
-│  │  cli.execute (root span)                        │    │
-│  │  ├─ command: validate                           │    │
-│  │  ├─ args: --verbose                             │    │
-│  │  ├─ cwd: /tmp/test-model                        │    │
-│  │  ├─ dr.project.name: VerificationModel         │    │
-│  │  │                                               │    │
-│  │  └──┬─────────────────────────────────────────┐ │    │
-│  │    │ model.validate (validator root)          │ │    │
-│  │    │ ├─ validation.stage.schema               │ │    │
-│  │    │ │  └─ Logs: "Schema validation passed"   │ │    │
-│  │    │ ├─ validation.stage.naming               │ │    │
-│  │    │ │  └─ Logs: "Naming validation passed"   │ │    │
-│  │    │ ├─ validation.stage.reference            │ │    │
-│  │    │ │  └─ Logs: "Reference validation..."    │ │    │
-│  │    │ └─ validation.stage.semantic             │ │    │
-│  │    │    └─ Logs: "Semantic validation..."     │ │    │
-│  │    └─────────────────────────────────────────┘ │    │
-│  │                                                  │    │
-│  │  Logs:                                          │    │
-│  │  ├─ traceId: abc123xyz...                      │    │
-│  │  ├─ spanId: def456...                          │    │
-│  │  └─ Console output captured here               │    │
-│  └─────────────────────────────────────────────────┘    │
-│                      │                                   │
-│                      ▼                                   │
-│  ┌──────────────────────────────────┐                   │
-│  │  Console Interceptor             │                   │
-│  │  ├─ console.log → INFO          │                   │
-│  │  ├─ console.error → ERROR       │                   │
-│  │  └─ Preserves original output   │                   │
-│  └──────────────────────────────────┘                   │
-│                      │                                   │
-└──────────────────────┼──────────────────────────────────┘
-                       │
-         ┌─────────────┴──────────────┐
-         ▼                            ▼
-   ┌──────────────┐          ┌──────────────┐
-   │ Trace Export │          │ Log Export   │
-   │ (OTLP)       │          │ (OTLP)       │
-   │ /v1/traces   │          │ /v1/logs     │
-   └──────┬───────┘          └──────┬───────┘
-          │                         │
-          └────────┬────────────────┘
-                   │
-                   ▼
-        ┌──────────────────────┐
-        │ OTEL Collector       │
-        │ localhost:4318       │
-        └──────────┬───────────┘
-                   │
-                   ▼
-        ┌──────────────────────┐
-        │ SigNoz               │
-        │ localhost:3301       │
-        └──────────────────────┘
-```
-
-## Key Features Verified
-
-### Feature 1: CLI Command Tracing
-- **Status**: ✅ Implemented
-- **Location**: `cli.ts:69` - Root span creation
-- **Verification**: Run any command and check for `cli.execute` span
-- **Attributes**: command, args, cwd, version, dr.project.name
-
-### Feature 2: Console Log Correlation
-- **Status**: ✅ Implemented
-- **Location**: `console-interceptor.ts` - All console methods wrapped
-- **Verification**: Run validator and check logs appear with traceId/spanId
-- **Severity Mapping**: log→INFO, error→ERROR, warn→WARN, debug→DEBUG
-
-### Feature 3: Validator Span Nesting
-- **Status**: ✅ Implemented
-- **Location**: `validator.ts:41-111` - All stages instrumented
-- **Verification**: Expand validator spans to see nested stages
-- **Stages**: schema, naming, reference, semantic
-
-### Feature 4: Test File Spans
-- **Status**: ✅ Implemented
-- **Location**: `test-instrumentation.ts:58-70` - File span creation
-- **Verification**: Run instrumented test and check for `test.file` attribute
-- **Attributes**: test.file, test.framework=bun
-
-### Feature 5: Test Case Spans
-- **Status**: ✅ Implemented
-- **Location**: `test-instrumentation.ts:113-127` - Case span creation
-- **Verification**: Check span for test.name, test.suite, test.status attributes
-- **Error Recording**: test.error.message and test.error.stack for failures
-
-### Feature 6: Failed Test Error Reporting
-- **Status**: ✅ Implemented
-- **Location**: `test-instrumentation.ts:153-166` - Error attribute setting
-- **Verification**: Fail a test and check for test.status=fail and error attributes
-- **Stack Traces**: Captured via span.recordException()
-
-### Feature 7: Log-Test Correlation
-- **Status**: ✅ Implemented via console interceptor
-- **Location**: `console-interceptor.ts` + `test-instrumentation.ts`
-- **Verification**: Console output during tests appears in test span logs
-- **Mechanism**: Automatic trace context propagation
-
-### Feature 8: Circuit-Breaker Protection
-- **Status**: ✅ Implemented
-- **Location**: `resilient-exporter.ts`, `resilient-log-exporter.ts`
-- **Verification**: Block collector and run command → completes normally, no errors
-- **Backoff**: 30 seconds after failure
-
-### Feature 9: Project Name Propagation
-- **Status**: ✅ Implemented
-- **Location**: `telemetry/index.ts:74-90` - Manifest loading
-- **Verification**: All spans/logs include `dr.project.name` attribute
-- **Fallback**: Uses 'unknown' if no manifest exists
-
-## Test Coverage Summary
-
-```
-Unit Tests: 421 pass, 0 fail
-
-Breakdown by module:
-- Telemetry module: 30 tests
-- Console interceptor: 50+ tests
-- Test instrumentation: 30+ tests
-- CLI instrumentation: 40+ tests
-- Other modules: 270+ tests
-
-All tests verify:
-✅ No-op behavior when disabled
-✅ Proper span creation and attributes
-✅ Error recording and exception propagation
-✅ Console method wrapping
-✅ Trace context propagation
-✅ Circuit-breaker activation
-```
-
-## Next Steps
-
-1. **Verify with SigNoz** (optional but recommended):
-   - Follow the SigNoz stack setup instructions above
-   - Observe actual trace-log correlation in the UI
-
-2. **Run comprehensive tests**:
-   ```bash
-   cd /workspace/cli
-   npm run test
-   ```
-
-3. **Test with your own models**:
-   - Create a test model: `dr init --name "MyProject"`
-   - Run commands with telemetry: `node dist/cli.js validate`
-   - Verify project name appears in traces
-
-4. **Review documentation**:
-   - Check `/workspace/cli/README.md` Telemetry section
-   - Verify setup instructions are clear
-   - Check troubleshooting section
-
-5. **Integrate with CI/CD** (optional):
-   - Debug builds can send telemetry to shared collector
-   - Filter traces by project name and CI/CD pipeline ID
-   - Monitor CLI reliability across different environments
-
-## Troubleshooting Reference
-
-See `/workspace/cli/README.md` "Telemetry & Observability" section for:
-- Common setup issues
-- Circuit-breaker behavior
-- Resource attribute verification
-- Log level mapping
-- Production vs. debug build differences
-
-## Summary
-
-All requirements have been implemented with:
-- Zero production overhead (telemetry disabled by default)
-- Comprehensive error handling
-- Graceful degradation
-- Full trace-log correlation
-- Project-level filtering capability
-
-Ready for integration and deployment.

package/test-debug.js

@@ -1,33 +0,0 @@
-const { PlantUMLExporter } = require('./dist/export/plantuml-exporter.js');
-const { Model } = require('./dist/core/model.js');
-const { Layer } = require('./dist/core/layer.js');
-const { Element } = require('./dist/core/element.js');
-const { Manifest } = require('./dist/core/manifest.js');
-
-async function test() {
-  const exporter = new PlantUMLExporter();
-  const manifest = new Manifest({
-    name: "Test Model",
-    version: "1.0.0",
-    description: "Test architecture",
-  });
-
-  const model = new Model("/test", manifest);
-  const motivationLayer = new Layer("motivation");
-  const goal = new Element({
-    id: "motivation-goal-increase-revenue",
-    type: "goal",
-    name: "Increase Revenue",
-  });
-  motivationLayer.addElement(goal);
-  model.addLayer(motivationLayer);
-
-  const output = await exporter.export(model, {});
-  console.log("=== OUTPUT ===");
-  console.log(output);
-  console.log("=== END ===");
-  console.log("Ends with @enduml\\n:", output.endsWith("@enduml\n"));
-  console.log("Includes component Increase Revenue:", output.includes('component "Increase Revenue"'));
-}
-
-test().catch(console.error);

package/test-special.mjs

@@ -1,46 +0,0 @@
-import { Model } from "./dist/core/model.js";
-import { Layer } from "./dist/core/layer.js";
-import { Element } from "./dist/core/element.js";
-import { Manifest } from "./dist/core/manifest.js";
-import { ArchiMateExporter } from "./dist/export/archimate-exporter.js";
-
-const ts = new Date().getTime();
-const testDir = `/tmp/test-special-chars-${ts}`;
-
-const manifest = new Manifest({
-  name: "Test Model",
-  version: "1.0.0",
-});
-
-const model = new Model(testDir, manifest);
-const specialLayer = new Layer("security");
-const element = new Element({
-  id: "security-threat-injection",
-  type: "threat",
-  name: "SQL Injection <attack> & XSS",
-  description: "Threat with special chars: <, >, &, \"",
-});
-
-console.log("Before adding element:");
-console.log("Elements in layer:", specialLayer.listElements().length);
-
-specialLayer.addElement(element);
-
-console.log("After adding element:");
-console.log("Elements in layer:", specialLayer.listElements().length);
-
-model.addLayer(specialLayer);
-
-console.log("After adding to model:");
-const layer = await model.getLayer("security");
-console.log("Layer from model:", layer?.name);
-console.log("Elements in layer from model:", layer?.listElements().length);
-
-const exporter = new ArchiMateExporter();
-console.log("Exporter supported layers:", exporter.supportedLayers);
-console.log("Security in supported:", exporter.supportedLayers.includes("security"));
-
-const result = await exporter.export(model, {});
-
-console.log("=== EXPORTED CONTENT ===");
-console.log(result);

package/test_runner.ts

@@ -1,7 +0,0 @@
-import { describe, it, expect, setDefaultTimeout } from 'bun:test';
-
-// Increase timeout for long-running tests
-setDefaultTimeout(30000); // 30 seconds
-
-// Run the test file
-import('./tests/integration/docs-validation.test.ts');