@bluefly/openstandardagents 0.4.4 → 0.4.6

package/spec/v0.3/conformance/README.md

@@ -1,263 +0,0 @@
-# OSSA Conformance Testing
-
-Conformance testing validates that OSSA agent manifests meet specific requirements defined in conformance profiles.
-
-## Overview
-
-The conformance suite provides:
-- **Profiles**: Predefined sets of requirements for different use cases
-- **Feature Detection**: Automatic detection of features in manifests
-- **Scoring**: Weighted scoring based on required and optional features
-- **Constraint Validation**: Enforce specific value constraints
-- **Recommendations**: Actionable guidance for improving conformance
-
-## Available Profiles
-
-### Baseline Profile
-**ID**: `baseline`
-**Purpose**: Minimal OSSA compliance for basic agent manifests
-
-**Required Features** (70% weight):
-- `apiVersion`: Must follow `ossa/vX.Y.Z` format
-- `kind`: Must be Agent, AgentTeam, or AgentWorkflow
-- `metadata.name`: Agent name
-- `spec.type`: Agent type (service, workflow, etc.)
-- `spec.identity.id`: Unique agent identifier
-
-**Optional Features** (30% weight):
-- `metadata.description`: Agent description
-- `metadata.labels`: Metadata labels
-- `spec.capabilities`: Agent capabilities
-- `spec.llm`: LLM configuration
-- `spec.instructions`: Agent instructions
-
-**Pass Threshold**: 70%
-
-### Enterprise Profile
-**ID**: `enterprise`
-**Purpose**: Production-ready requirements for enterprise deployments
-
-Extends baseline profile with additional requirements:
-
-**Required Features** (60% weight):
-- All baseline required features
-- `metadata.version`: Semantic version
-- `spec.llm.provider` & `spec.llm.model`: LLM configuration
-- `spec.autonomy.level` & `spec.autonomy.approval_required`: Autonomy settings
-- `spec.constraints.cost` & `spec.constraints.performance`: Resource limits
-- `spec.observability.*`: Tracing, metrics, logging
-
-**Optional Features** (40% weight):
-- `spec.security.*`: Authentication, authorization, data privacy
-- `spec.resilience.*`: Retry policies, timeouts, circuit breakers
-- `spec.deployment.*`: Environment and scaling configuration
-
-**Pass Threshold**: 80%
-
-### GitLab Kagent Profile
-**ID**: `gitlab-kagent`
-**Purpose**: Requirements for GitLab Kagent extension compatibility
-
-Extends enterprise profile with Kagent-specific requirements:
-
-**Required Features** (50% weight):
-- All enterprise required features
-- `extensions.kagent.enabled`: Kagent extension enabled
-- `extensions.kagent.gitlab_integration`: GitLab integration config
-- `extensions.kagent.kubernetes_native`: Kubernetes native support
-
-**Optional Features** (50% weight):
-- `extensions.kagent.gitlab_integration.*`: CI/CD, issues, MRs, pipelines
-- `extensions.kagent.kubernetes_native.*`: Deployment, scaling, monitoring
-- `extensions.kagent.observability.*`: Traces and metrics
-- `extensions.kagent.security.*`: RBAC and secrets management
-
-**Pass Threshold**: 75%
-
-## Usage
-
-### CLI Commands
-
-#### Run Conformance Test
-```bash
-# Test against baseline profile
-ossa conformance run manifest.yaml --profile baseline
-
-# Test against enterprise profile with verbose output
-ossa conformance run manifest.yaml --profile enterprise --verbose
-
-# Test with strict validation (fail on validation errors)
-ossa conformance run manifest.yaml --profile enterprise --strict
-
-# JSON output for CI/CD integration
-ossa conformance run manifest.yaml --profile baseline --output json
-```
-
-#### List Available Profiles
-```bash
-ossa conformance list
-```
-
-#### Show Profile Details
-```bash
-ossa conformance profile baseline
-ossa conformance profile enterprise --output json
-```
-
-### Programmatic Usage
-
-```typescript
-import { ConformanceService } from '@ossa/openstandardagents';
-import { container } from '@ossa/openstandardagents/di-container';
-
-const conformanceService = container.get(ConformanceService);
-
-// Run conformance test
-const result = await conformanceService.runConformanceTest(
-  manifest,
-  'enterprise',
-  false // strict mode
-);
-
-// Generate report
-const report = await conformanceService.generateReport(
-  manifest,
-  'baseline'
-);
-
-// Batch test multiple manifests
-const results = await conformanceService.batchTest(
-  manifests,
-  'enterprise'
-);
-
-// Get summary statistics
-const stats = conformanceService.getSummaryStatistics(results);
-```
-
-## Test Fixtures
-
-Test fixtures are organized by profile:
-
-```
-spec/v0.3/conformance/tests/
-├── baseline/
-│   ├── valid/
-│   │   ├── minimal-agent.yaml
-│   │   └── basic-agent.yaml
-│   └── invalid/
-│       ├── missing-kind.yaml
-│       └── missing-identity.yaml
-├── enterprise/
-│   ├── valid/
-│   │   ├── production-agent.yaml
-│   │   └── enterprise-agent.yaml
-└── gitlab-kagent/
-    └── valid/
-        ├── kagent-agent.yaml
-        └── kagent-full.yaml
-```
-
-## Scoring Algorithm
-
-```
-score = (required_present / required_total) * required_weight +
-        (optional_present / optional_total) * optional_weight
-```
-
-- **Required features** have higher weight (typically 60-70%)
-- **Optional features** have lower weight (typically 30-40%)
-- **Pass threshold** varies by profile (70-80%)
-- **Constraint violations** fail the test regardless of score
-
-## CI/CD Integration
-
-### GitLab CI
-
-The conformance suite includes GitLab CI jobs:
-
-```yaml
-# .gitlab-ci.yml
-include:
-  - local: .gitlab/ci/conformance-jobs.yml
-
-# Runs on MR and main branches
-conformance:baseline:
-  script:
-    - ossa conformance run manifest.yaml --profile baseline
-```
-
-See `.gitlab/ci/conformance-jobs.yml` for complete CI configuration.
-
-### GitHub Actions
-
-```yaml
-- name: Run conformance tests
-  run: |
-    npm run ossa -- conformance run manifest.yaml --profile enterprise --output json
-```
-
-## Creating Custom Profiles
-
-Create a JSON file in `spec/v0.3/conformance/profiles/`:
-
-```json
-{
-  "id": "custom",
-  "name": "Custom Profile",
-  "version": "0.3.5",
-  "description": "Custom conformance requirements",
-  "extends": "baseline",
-  "required": {
-    "features": ["apiVersion", "kind"],
-    "weight": 0.7
-  },
-  "optional": {
-    "features": ["metadata.description"],
-    "weight": 0.3
-  },
-  "constraints": {
-    "apiVersion": {
-      "pattern": "^ossa/v[0-9]+\\.[0-9]+\\.[0-9]+$",
-      "description": "Must follow ossa/vX.Y.Z format"
-    }
-  },
-  "scoring": {
-    "pass_threshold": 0.75,
-    "warn_threshold": 0.85
-  }
-}
-```
-
-## Best Practices
-
-1. **Start with baseline**: Ensure baseline conformance before testing higher profiles
-2. **Use verbose mode**: Get detailed feature detection results and recommendations
-3. **Enable strict mode**: Fail fast on validation errors in production pipelines
-4. **Review recommendations**: Follow actionable guidance to improve conformance
-5. **Test early**: Run conformance tests during development, not just in CI
-6. **Track scores**: Monitor conformance scores over time to ensure quality
-
-## Troubleshooting
-
-### Score too low
-- Use `--verbose` flag to see missing features
-- Check recommendations section for guidance
-- Review profile requirements with `ossa conformance profile <name>`
-
-### Constraint violations
-- Check expected vs actual values in error messages
-- Review constraint descriptions in profile
-- Validate manifest structure with `ossa validate`
-
-### Profile not found
-- List available profiles: `ossa conformance list`
-- Check profile ID matches exactly (case-sensitive)
-- Ensure profiles directory exists and is readable
-
-## Resources
-
-- [OSSA Specification](../../README.md)
-- [Example Manifests](../../examples/)
-- [Validation Guide](../../../docs/validation.md)
-- [CI/CD Integration](../.gitlab/ci/conformance-jobs.yml)

package/spec/v0.3/conformance/profiles/baseline.json

@@ -1,27 +0,0 @@
-{
-  "$schema": "https://openstandardagents.org/schemas/conformance-profile.json",
-  "id": "baseline",
-  "name": "Baseline",
-  "version": "0.3.5",
-  "description": "Minimal OSSA v0.3 compliance - Linux Foundation adoption ready",
-  "required": {
-    "features": [
-      "metadata.name",
-      "metadata.version",
-      "spec.identity.id",
-      "spec.role"
-    ],
-    "weight": 100
-  },
-  "optional": {
-    "features": [
-      "metadata.labels",
-      "metadata.annotations"
-    ],
-    "weight": 0
-  },
-  "scoring": {
-    "pass_threshold": 100,
-    "warn_threshold": 80
-  }
-}

package/spec/v0.3/conformance/profiles/enterprise.json

@@ -1,33 +0,0 @@
-{
-  "$schema": "https://openstandardagents.org/schemas/conformance-profile.json",
-  "id": "enterprise",
-  "name": "Enterprise",
-  "extends": "baseline",
-  "version": "0.3.5",
-  "description": "Production-ready features for enterprise deployment",
-  "required": {
-    "features": [
-      "metadata.name",
-      "metadata.version",
-      "spec.identity.id",
-      "spec.role",
-      "spec.completion",
-      "spec.checkpointing",
-      "spec.monitoring",
-      "spec.security"
-    ],
-    "weight": 70
-  },
-  "optional": {
-    "features": [
-      "spec.extensions.moe",
-      "spec.extensions.feedback_loops",
-      "spec.extensions.bat"
-    ],
-    "weight": 30
-  },
-  "scoring": {
-    "pass_threshold": 85,
-    "warn_threshold": 70
-  }
-}

package/spec/v0.3/conformance/profiles/gitlab-kagent.json

@@ -1,36 +0,0 @@
-{
-  "$schema": "https://openstandardagents.org/schemas/conformance-profile.json",
-  "id": "gitlab-kagent",
-  "name": "GitLab Kagent",
-  "extends": "enterprise",
-  "version": "0.3.5",
-  "description": "GitLab Kagent extension compliance",
-  "required": {
-    "features": [
-      "metadata.name",
-      "metadata.version",
-      "spec.identity.id",
-      "spec.role",
-      "spec.completion",
-      "spec.checkpointing",
-      "spec.monitoring",
-      "spec.security",
-      "spec.extensions.kagent",
-      "spec.extensions.kagent.kubernetes.namespace",
-      "spec.extensions.kagent.kubernetes.resourceLimits"
-    ],
-    "weight": 80
-  },
-  "optional": {
-    "features": [
-      "spec.extensions.moe",
-      "spec.extensions.feedback_loops",
-      "spec.extensions.bat"
-    ],
-    "weight": 20
-  },
-  "scoring": {
-    "pass_threshold": 90,
-    "warn_threshold": 75
-  }
-}

package/spec/v0.3/conformance/tests/baseline/invalid/missing-identity.yaml

@@ -1,6 +0,0 @@
-apiVersion: ossa/v0.3.5
-kind: Agent
-metadata:
-  name: invalid-agent-no-identity
-spec:
-  type: service

package/spec/v0.3/conformance/tests/baseline/invalid/missing-kind.yaml

@@ -1,7 +0,0 @@
-apiVersion: ossa/v0.3.5
-metadata:
-  name: invalid-agent-no-kind
-spec:
-  type: service
-  identity:
-    id: invalid-agent

package/spec/v0.3/conformance/tests/baseline/valid/basic-agent.yaml

@@ -1,22 +0,0 @@
-apiVersion: ossa/v0.3.5
-kind: Agent
-metadata:
-  name: basic-test-agent
-  version: "1.0.0"
-  description: Basic agent with optional features for baseline testing
-  labels:
-    environment: test
-    category: conformance
-spec:
-  role: assistant
-  type: service
-  identity:
-    id: basic-test-agent
-    version: "1.0.0"
-  instructions: "You are a test agent for conformance testing"
-  capabilities:
-    - name: echo
-      description: Echo back user input
-  llm:
-    provider: openai
-    model: gpt-4

package/spec/v0.3/conformance/tests/baseline/valid/minimal-agent.yaml

@@ -1,15 +0,0 @@
-apiVersion: ossa/v0.3.5
-kind: Agent
-metadata:
-  name: minimal-test-agent
-  version: "1.0.0"
-  description: Minimal valid agent for baseline conformance testing
-spec:
-  role: assistant
-  type: service
-  identity:
-    id: minimal-test-agent
-    version: "1.0.0"
-  capabilities:
-    - name: test_capability
-      description: Test capability

package/spec/v0.3/conformance/tests/enterprise/valid/enterprise-agent.yaml

@@ -1,88 +0,0 @@
-apiVersion: ossa/v0.3.5
-kind: Agent
-metadata:
-  name: enterprise-test-agent
-  description: Enterprise agent with full configuration
-  version: "2.1.0"
-  labels:
-    environment: staging
-    tier: enterprise
-  annotations:
-    owner: platform-team
-    compliance: SOC2
-spec:
-  role: assistant
-  type: service
-  identity:
-    id: enterprise-test-agent
-    version: "2.1.0"
-  completion:
-    criteria:
-      - type: task_completed
-        description: Task is successfully completed
-    timeout_ms: 300000
-  checkpointing:
-    enabled: true
-    interval_ms: 30000
-    strategy: incremental
-  monitoring:
-    metrics:
-      enabled: true
-      interval_ms: 60000
-    alerts:
-      enabled: true
-  security:
-    authentication:
-      required: true
-      method: api_key
-    authorization:
-      rbac: true
-  capabilities:
-    - name: analysis
-      description: Analyze data
-  llm:
-    provider: openai
-    model: gpt-4-turbo
-  autonomy:
-    level: semi_autonomous
-    approval_required: false
-  constraints:
-    cost:
-      max_cost_per_task: 1.0
-      currency: USD
-    performance:
-      max_execution_time_ms: 60000
-  observability:
-    tracing:
-      enabled: true
-    metrics:
-      enabled: true
-    logging:
-      enabled: true
-  extensions:
-    moe:
-      enabled: true
-      routing_strategy: least_cost
-      models:
-        - provider: openai
-          model: gpt-4o-mini
-          use_case: simple_queries
-        - provider: openai
-          model: gpt-4-turbo
-          use_case: complex_analysis
-    bat:
-      enabled: true
-      criteria:
-        - type: cost
-          weight: 0.4
-        - type: performance
-          weight: 0.3
-        - type: quality
-          weight: 0.3
-    feedback_loops:
-      enabled: true
-      mechanisms:
-        - type: user_feedback
-          collection_method: inline
-        - type: performance_metrics
-          interval_ms: 60000

package/spec/v0.3/conformance/tests/enterprise/valid/production-agent.yaml

@@ -1,112 +0,0 @@
-apiVersion: ossa/v0.3.5
-kind: Agent
-metadata:
-  name: production-test-agent
-  description: Production-ready agent for enterprise conformance testing
-  version: "1.0.0"
-  labels:
-    environment: production
-    tier: enterprise
-    category: conformance
-spec:
-  role: assistant
-  type: service
-  identity:
-    id: production-test-agent
-    version: "1.0.0"
-  completion:
-    criteria:
-      - type: success_criteria
-        description: Task completed successfully
-    timeout_ms: 180000
-  checkpointing:
-    enabled: true
-    interval_ms: 60000
-    strategy: full
-  monitoring:
-    metrics:
-      enabled: true
-      interval_ms: 30000
-    alerts:
-      enabled: true
-      channels: ["email", "slack"]
-  capabilities:
-    - name: data_processing
-      description: Process and analyze data
-    - name: reporting
-      description: Generate reports
-  llm:
-    provider: anthropic
-    model: claude-3-5-sonnet-20241022
-    temperature: 0.7
-  autonomy:
-    level: monitored
-    approval_required: true
-  constraints:
-    cost:
-      max_cost_per_task: 0.50
-      currency: USD
-    performance:
-      max_execution_time_ms: 30000
-      max_memory_mb: 512
-  observability:
-    tracing:
-      enabled: true
-      provider: opentelemetry
-    metrics:
-      enabled: true
-      provider: prometheus
-    logging:
-      enabled: true
-      level: info
-  security:
-    authentication:
-      required: true
-      methods:
-        - oauth2
-    authorization:
-      enabled: true
-      model: rbac
-    data_privacy:
-      enabled: true
-      pii_handling: redact
-  resilience:
-    retry_policy:
-      max_attempts: 3
-      backoff_multiplier: 2
-    timeout_ms: 30000
-    circuit_breaker:
-      enabled: true
-      failure_threshold: 5
-  deployment:
-    environment: production
-    scaling:
-      min_instances: 2
-      max_instances: 10
-  extensions:
-    moe:
-      enabled: true
-      routing_strategy: least_cost
-      models:
-        - provider: anthropic
-          model: claude-3-5-haiku-20241022
-          use_case: simple_queries
-        - provider: anthropic
-          model: claude-3-5-sonnet-20241022
-          use_case: complex_analysis
-    bat:
-      enabled: true
-      criteria:
-        - type: cost
-          weight: 0.4
-        - type: performance
-          weight: 0.3
-        - type: quality
-          weight: 0.3
-    feedback_loops:
-      enabled: true
-      mechanisms:
-        - type: user_feedback
-          collection_method: inline
-        - type: performance_metrics
-          interval_ms: 60000