claude-flow-novice 2.18.6 → 2.18.7

package/claude-assets/skills/cfn-test-framework/lib/runner/SKILL.md

@@ -1,288 +0,0 @@
-# CFN Test Runner Skill
-
-**Version:** 1.0.0
-**Status:** Production
-**Purpose:** Unified test execution with benchmarking and regression detection
-
----
-
-## Overview
-
-Provides comprehensive test execution across all CFN test suites:
-- **Hello World Tests** (Layer 0-7)
-- **CFN Loop E2E Tests** (9 integration tests)
-- **Benchmark Tracking** (SQLite storage)
-- **Regression Detection** (automated alerts)
-
-**Key Features:**
-- Single command test execution
-- Historical benchmark comparison
-- Performance regression alerts
-- Git-aware baseline tracking
-- Parallel test execution
-- Comprehensive reporting
-
----
-
-## Architecture
-
-### Skill Components
-
-```
-.claude/skills/cfn-test-runner/
-├── SKILL.md                    # This file
-├── run-all-tests.sh            # Main test runner
-├── store-benchmarks.sh         # SQLite benchmark storage
-├── detect-regressions.sh       # Regression analysis
-├── init-benchmark-db.sh        # Database initialization
-└── generate-report.sh          # Test report generation
-```
-
----
-
-## Usage
-
-### Run All Tests
-
-```bash
-./.claude/skills/cfn-test-runner/run-all-tests.sh \
-  --suite all \
-  --benchmark \
-  --detect-regressions
-```
-
-### Run Specific Suite
-
-```bash
-# Hello World only
-./.claude/skills/cfn-test-runner/run-all-tests.sh --suite hello-world
-
-# CFN E2E only
-./.claude/skills/cfn-test-runner/run-all-tests.sh --suite cfn-e2e
-
-# Both
-./.claude/skills/cfn-test-runner/run-all-tests.sh --suite all
-```
-
-### With Regression Detection
-
-```bash
-./.claude/skills/cfn-test-runner/run-all-tests.sh \
-  --suite all \
-  --benchmark \
-  --detect-regressions \
-  --threshold 0.10
-```
-
----
-
-## Parameters
-
-### run-all-tests.sh
-
-| Parameter | Required | Description | Default |
-|-----------|----------|-------------|---------|
-| `--suite` | No | Test suite: `all`, `hello-world`, `cfn-e2e` | `all` |
-| `--benchmark` | No | Store results in SQLite | `false` |
-| `--detect-regressions` | No | Run regression analysis | `false` |
-| `--threshold` | No | Regression threshold (%) | `0.10` |
-| `--parallel` | No | Run tests in parallel | `false` |
-| `--output` | No | Output format: `text`, `json`, `html` | `text` |
-
----
-
-## Test Suites
-
-### Hello World (Layer 0-7)
-
-**Purpose:** Validate agent spawning patterns and coordination
-
-| Layer | Test | Duration | Critical |
-|-------|------|----------|----------|
-| 0 | Tool Validation | 60s | ✅ |
-| 1-4 | [Future] | - | - |
-| 5 | Coordinator Spawning | 120s | ✅ |
-| 6 | Review Handoff | 180s | ✅ |
-| 7 | Error Retry | 150s | ✅ |
-
-**Location:** `tests/hello-world/`
-
-### CFN Loop E2E (9 Tests)
-
-**Purpose:** Validate full CFN Loop workflow
-
-| Test | Component | Duration | Critical |
-|------|-----------|----------|----------|
-| 1 | Coordinator → Orchestrator | 30s | ✅ |
-| 2 | Loop 3 → Gate Check | 60s | ✅ |
-| 3 | Gate Pass → Loop 2 | 30s | ⚠️ |
-| 4 | Loop 2 → Product Owner | 90s | ✅ |
-| 5 | Product Owner Decision | 60s | ✅ |
-| 6 | Iteration Cycle | 120s | ✅ |
-| 7 | Redis Key Structure | 10s | ✅ |
-| 8 | Error Recovery | 60s | ⚠️ |
-| 9 | Cleanup | 10s | ⚠️ |
-
-**Location:** `tests/cfn-v3/test-e2e-cfn-loop.sh`
-
----
-
-## Benchmark Storage
-
-### SQLite Schema
-
-```sql
-test_suites          -- Suite definitions
-test_runs            -- Execution records
-test_cases           -- Individual tests
-test_results         -- Per-test outcomes
-performance_metrics  -- Performance data
-regression_alerts    -- Detected regressions
-```
-
-**Database:** `.artifacts/test-benchmarks.db`
-
-### Stored Metrics
-
-**Test-Level:**
-- Duration (ms)
-- Status (passed/failed/skipped)
-- Assertion count
-- Error messages
-
-**Suite-Level:**
-- Total duration (seconds)
-- Success rate (%)
-- Pass/fail/skip counts
-- Git commit/branch
-
-**Performance:**
-- Agent spawn time
-- Redis operation latency
-- File I/O duration
-- Memory usage
-
----
-
-## Regression Detection
-
-### Automatic Alerts
-
-```bash
-# Critical: Test started failing
-alert_type: test_failure
-severity: critical
-message: "TEST 5: Product Owner Decision FAILED (was passing)"
-
-# Warning: Performance degraded
-alert_type: performance_regression
-severity: warning
-message: "Loop 3 spawn time increased 25% (baseline: 5s, current: 6.25s)"
-
-# Info: Success rate dropped
-alert_type: success_rate_drop
-severity: info
-message: "Suite success rate: 88% → 77% (threshold: 10%)"
-```
-
-### Threshold Configuration
-
-```bash
-# Default: 10% regression threshold
---threshold 0.10
-
-# Strict: 5% threshold
---threshold 0.05
-
-# Relaxed: 20% threshold
---threshold 0.20
-```
-
----
-
-## Output Formats
-
-### Text (Console)
-
-```
-==========================================
-CFN Test Suite Results
-==========================================
-
-Suite: Hello World
-  Layer 0: ✅ PASSED (58.3s)
-  Layer 5: ✅ PASSED (115.7s)
-  Layer 6: ✅ PASSED (172.4s)
-  Layer 7: ✅ PASSED (148.2s)
-
-Suite: CFN E2E
-  TEST 1: ✅ PASSED (28.1s)
-  TEST 2: ✅ PASSED (54.6s)
-  TEST 3: ⚠️  SKIPPED
-  TEST 4: ✅ PASSED (87.3s)
-  TEST 5: ✅ PASSED (59.2s)
-  TEST 6: ✅ PASSED (118.9s)
-  TEST 7: ✅ PASSED (9.4s)
-
-Total: 11 tests, 9 passed, 0 failed, 2 skipped
-Duration: 851.1s
-Success Rate: 81.8%
-
-Regressions Detected: 0
-==========================================
-```
-
-### JSON (API/CI)
-
-```json
-{
-  "timestamp": "2025-11-04T02:00:00Z",
-  "git_commit": "abc123",
-  "git_branch": "main",
-  "suites": {
-    "hello-world": {
-      "total": 4,
-      "passed": 4,
-      "failed": 0,
-      "skipped": 0,
-      "duration": 494.6,
-      "success_rate": 1.0
-    },
-    "cfn-e2e": {
-      "total": 9,
-      "passed": 7,
-      "failed": 0,
-      "skipped": 2,
-      "duration": 356.5,
-      "success_rate": 0.778
-    }
-  },
-  "regressions": [],
-  "baseline_comparison": {
-    "duration_change": "+2.3%",
-    "success_rate_change": "+0.0%"
-  }
-}
-```
-
-### HTML (Report)
-
-Generates interactive HTML report with:
-- Trend graphs (Chart.js)
-- Regression highlights
-- Test history
-- Performance metrics
-
----
-
-## Baseline Management
-
-### Establish Baseline
-
-```bash
-# First run establishes baseline
-./.claude/skills/cfn-test-runner/run-all-tests.sh --suite all --benchmark
-
-# Mark as baseline
-sqlite3 .artifacts/test-benchmarks.db << EOF
-UPDATE test_runs SET environment = 'baseline' WHERE id = (SELECT MAX(id) FROM test_runs);

package/claude-assets/skills/cfn-test-framework/lib/runner/detect-regressions.sh

@@ -1,58 +0,0 @@
-#!/bin/bash
-# Detect test regressions
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
-
-# Load parameterized query library (SQL injection prevention)
-source "$PROJECT_ROOT/.claude/skills/bootstrap/sqlite-params.sh"
-
-DB_FILE="$PROJECT_ROOT/.artifacts/test-benchmarks.db"
-
-THRESHOLD=0.10
-
-while [[ $# -gt 0 ]]; do
-  case $1 in
-    --threshold) THRESHOLD="$2"; shift 2 ;;
-    *) shift ;;
-  esac
-done
-
-# Get latest run
-LATEST_RUN=$(sqlite_select "$DB_FILE" "SELECT id FROM test_runs ORDER BY run_timestamp DESC LIMIT 1")
-
-if [ -z "$LATEST_RUN" ]; then
-  echo "No test runs found"
-  exit 0
-fi
-
-# Get baseline (average of last 10 runs excluding latest)
-BASELINE_SUCCESS_RATE=$(sqlite_select "$DB_FILE" "
-  SELECT AVG(success_rate) FROM (
-    SELECT success_rate FROM test_runs
-    WHERE id != ?1
-    ORDER BY run_timestamp DESC
-    LIMIT 10
-  )
-" "$LATEST_RUN")
-
-LATEST_SUCCESS_RATE=$(sqlite_select "$DB_FILE" "SELECT success_rate FROM test_runs WHERE id = ?1" "$LATEST_RUN")
-
-# Check for regression
-REGRESSION=$(awk "BEGIN {print ($BASELINE_SUCCESS_RATE - $LATEST_SUCCESS_RATE) > $THRESHOLD}")
-
-if [ "$REGRESSION" = "1" ]; then
-  DIFF=$(awk "BEGIN {printf \"%.1f\", ($BASELINE_SUCCESS_RATE - $LATEST_SUCCESS_RATE) * 100}")
-  MESSAGE="Success rate dropped ${DIFF}% (baseline: ${BASELINE_SUCCESS_RATE}, current: ${LATEST_SUCCESS_RATE})"
-
-  sqlite_exec "$DB_FILE" \
-    "INSERT INTO regression_alerts (run_id, alert_type, severity, message) VALUES (?1, ?2, ?3, ?4)" \
-    "$LATEST_RUN" "success_rate_drop" "warning" "$MESSAGE"
-
-  echo "⚠️  Regression detected: Success rate dropped ${DIFF}%"
-  exit 1
-fi
-
-echo "✅ No regressions detected"
-exit 0

package/claude-assets/skills/cfn-test-framework/lib/runner/detect-regressions.sh.backup-1763392821

@@ -1,55 +0,0 @@
-#!/bin/bash
-# Detect test regressions
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
-DB_FILE="$PROJECT_ROOT/.artifacts/test-benchmarks.db"
-
-THRESHOLD=0.10
-
-while [[ $# -gt 0 ]]; do
-  case $1 in
-    --threshold) THRESHOLD="$2"; shift 2 ;;
-    *) shift ;;
-  esac
-done
-
-# Get latest run
-LATEST_RUN=$(sqlite3 "$DB_FILE" "SELECT id FROM test_runs ORDER BY run_timestamp DESC LIMIT 1")
-
-if [ -z "$LATEST_RUN" ]; then
-  echo "No test runs found"
-  exit 0
-fi
-
-# Get baseline (average of last 10 runs excluding latest)
-BASELINE_SUCCESS_RATE=$(sqlite3 "$DB_FILE" "
-  SELECT AVG(success_rate) FROM (
-    SELECT success_rate FROM test_runs 
-    WHERE id != $LATEST_RUN 
-    ORDER BY run_timestamp DESC 
-    LIMIT 10
-  )
-")
-
-LATEST_SUCCESS_RATE=$(sqlite3 "$DB_FILE" "SELECT success_rate FROM test_runs WHERE id = $LATEST_RUN")
-
-# Check for regression
-REGRESSION=$(awk "BEGIN {print ($BASELINE_SUCCESS_RATE - $LATEST_SUCCESS_RATE) > $THRESHOLD}")
-
-if [ "$REGRESSION" = "1" ]; then
-  DIFF=$(awk "BEGIN {printf \"%.1f\", ($BASELINE_SUCCESS_RATE - $LATEST_SUCCESS_RATE) * 100}")
-  
-  sqlite3 "$DB_FILE" << EOFSQL
-INSERT INTO regression_alerts (run_id, alert_type, severity, message)
-VALUES ($LATEST_RUN, 'success_rate_drop', 'warning', 
-        'Success rate dropped ${DIFF}% (baseline: ${BASELINE_SUCCESS_RATE}, current: ${LATEST_SUCCESS_RATE})');
-EOFSQL
-  
-  echo "⚠️  Regression detected: Success rate dropped ${DIFF}%"
-  exit 1
-fi
-
-echo "✅ No regressions detected"
-exit 0

package/claude-assets/skills/cfn-test-framework/lib/runner/init-benchmark-db.sh

@@ -1,48 +0,0 @@
-#!/bin/bash
-# Initialize benchmark database
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
-DB_FILE="$PROJECT_ROOT/.artifacts/test-benchmarks.db"
-
-mkdir -p "$PROJECT_ROOT/.artifacts"
-
-sqlite3 "$DB_FILE" << 'EOFSQL'
-CREATE TABLE IF NOT EXISTS test_suites (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  name TEXT NOT NULL UNIQUE,
-  description TEXT,
-  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-);
-
-CREATE TABLE IF NOT EXISTS test_runs (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  suite_id INTEGER NOT NULL,
-  run_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-  git_commit TEXT,
-  git_branch TEXT,
-  total_tests INTEGER NOT NULL,
-  passed INTEGER NOT NULL,
-  failed INTEGER NOT NULL,
-  skipped INTEGER NOT NULL,
-  duration_seconds REAL NOT NULL,
-  success_rate REAL,
-  FOREIGN KEY (suite_id) REFERENCES test_suites(id)
-);
-
-CREATE TABLE IF NOT EXISTS regression_alerts (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  run_id INTEGER NOT NULL,
-  alert_type TEXT NOT NULL,
-  severity TEXT NOT NULL,
-  message TEXT NOT NULL,
-  acknowledged INTEGER DEFAULT 0,
-  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-  FOREIGN KEY (run_id) REFERENCES test_runs(id)
-);
-
-CREATE INDEX IF NOT EXISTS idx_runs_timestamp ON test_runs(run_timestamp DESC);
-EOFSQL
-
-echo "✅ Benchmark database initialized: $DB_FILE"

package/claude-assets/skills/cfn-test-framework/lib/runner/run-all-tests.sh

@@ -1,222 +0,0 @@
-#!/bin/bash
-# CFN Unified Test Runner
-# Version: 1.0.0
-# Purpose: Run all test suites with benchmarking and regression detection
-
-set -euo pipefail
-
-# Colors
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[1;33m'
-BLUE='\033[0;34m'
-NC='\033[0m'
-
-# Configuration
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
-DB_FILE="$PROJECT_ROOT/.artifacts/test-benchmarks.db"
-
-# Parameters
-SUITE="all"
-BENCHMARK=false
-DETECT_REGRESSIONS=false
-THRESHOLD=0.10
-PARALLEL=false
-OUTPUT_FORMAT="text"
-
-# Parse arguments
-while [[ $# -gt 0 ]]; do
-  case $1 in
-    --suite)
-      SUITE="$2"
-      shift 2
-      ;;
-    --benchmark)
-      BENCHMARK=true
-      shift
-      ;;
-    --detect-regressions)
-      DETECT_REGRESSIONS=true
-      shift
-      ;;
-    --threshold)
-      THRESHOLD="$2"
-      shift 2
-      ;;
-    --parallel)
-      PARALLEL=true
-      shift
-      ;;
-    --output)
-      OUTPUT_FORMAT="$2"
-      shift 2
-      ;;
-    *)
-      echo "Unknown parameter: $1"
-      exit 1
-      ;;
-  esac
-done
-
-# Initialize benchmark database if needed
-if [ "$BENCHMARK" = true ] && [ ! -f "$DB_FILE" ]; then
-  echo -e "${BLUE}[INFO]${NC} Initializing benchmark database..."
-  "$SCRIPT_DIR/init-benchmark-db.sh"
-fi
-
-# Get git info
-GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
-GIT_BRANCH=$(git branch --show-current 2>/dev/null || echo "unknown")
-
-# Start test run
-START_TIME=$(date +%s)
-TOTAL_TESTS=0
-PASSED_TESTS=0
-FAILED_TESTS=0
-SKIPPED_TESTS=0
-
-echo -e "${GREEN}=========================================="
-echo "CFN Test Suite Runner"
-echo -e "==========================================${NC}"
-echo "Suite: $SUITE"
-echo "Benchmark: $BENCHMARK"
-echo "Detect Regressions: $DETECT_REGRESSIONS"
-echo "Git: $GIT_BRANCH @ $GIT_COMMIT"
-echo ""
-
-# Run Hello World tests
-if [ "$SUITE" = "all" ] || [ "$SUITE" = "hello-world" ]; then
-  echo -e "${BLUE}[INFO]${NC} Running Hello World tests..."
-  
-  HW_START=$(date +%s)
-  HW_PASSED=0
-  HW_FAILED=0
-  HW_SKIPPED=0
-  
-  # Layer 0
-  if timeout 90 node "$PROJECT_ROOT/tests/hello-world/layer0-tool-validation.js" > /tmp/hw-layer0.log 2>&1; then
-    echo -e "${GREEN}✅ Layer 0: PASSED${NC}"
-    ((HW_PASSED++))
-  else
-    echo -e "${RED}❌ Layer 0: FAILED${NC}"
-    ((HW_FAILED++))
-  fi
-  
-  # Layer 5
-  if timeout 180 node "$PROJECT_ROOT/tests/hello-world/layer5-coordinator-spawning.js" > /tmp/hw-layer5.log 2>&1; then
-    echo -e "${GREEN}✅ Layer 5: PASSED${NC}"
-    ((HW_PASSED++))
-  else
-    echo -e "${RED}❌ Layer 5: FAILED${NC}"
-    ((HW_FAILED++))
-  fi
-  
-  # Layer 6
-  if timeout 240 node "$PROJECT_ROOT/tests/hello-world/layer6-coordinator-review.js" > /tmp/hw-layer6.log 2>&1; then
-    echo -e "${GREEN}✅ Layer 6: PASSED${NC}"
-    ((HW_PASSED++))
-  else
-    echo -e "${RED}❌ Layer 6: FAILED${NC}"
-    ((HW_FAILED++))
-  fi
-  
-  # Layer 7
-  if timeout 200 node "$PROJECT_ROOT/tests/hello-world/layer7-coordinator-error-retry.js" > /tmp/hw-layer7.log 2>&1; then
-    echo -e "${GREEN}✅ Layer 7: PASSED${NC}"
-    ((HW_PASSED++))
-  else
-    echo -e "${RED}❌ Layer 7: FAILED${NC}"
-    ((HW_FAILED++))
-  fi
-  
-  HW_END=$(date +%s)
-  HW_DURATION=$((HW_END - HW_START))
-  
-  echo -e "${BLUE}Hello World: ${HW_PASSED} passed, ${HW_FAILED} failed, ${HW_SKIPPED} skipped (${HW_DURATION}s)${NC}"
-  echo ""
-  
-  TOTAL_TESTS=$((TOTAL_TESTS + HW_PASSED + HW_FAILED + HW_SKIPPED))
-  PASSED_TESTS=$((PASSED_TESTS + HW_PASSED))
-  FAILED_TESTS=$((FAILED_TESTS + HW_FAILED))
-  SKIPPED_TESTS=$((SKIPPED_TESTS + HW_SKIPPED))
-fi
-
-# Run CFN E2E tests
-if [ "$SUITE" = "all" ] || [ "$SUITE" = "cfn-e2e" ]; then
-  echo -e "${BLUE}[INFO]${NC} Running CFN E2E tests..."
-  
-  E2E_START=$(date +%s)
-  
-  # Run E2E test and capture results
-  if timeout 600 bash "$PROJECT_ROOT/tests/cfn-v3/test-e2e-cfn-loop.sh" > /tmp/e2e-output.log 2>&1; then
-    E2E_EXIT=0
-  else
-    E2E_EXIT=$?
-  fi
-  
-  # Parse results
-  E2E_PASSED=$(grep -c "PASS" /tmp/e2e-output.log || echo 0)
-  E2E_FAILED=$(grep -c "FAIL" /tmp/e2e-output.log || echo 0)
-  E2E_SKIPPED=$(grep -c "SKIPPED" /tmp/e2e-output.log || echo 0)
-  
-  if [ $E2E_EXIT -eq 0 ]; then
-    echo -e "${GREEN}✅ CFN E2E: PASSED${NC}"
-  else
-    echo -e "${RED}❌ CFN E2E: FAILED (exit $E2E_EXIT)${NC}"
-  fi
-  
-  E2E_END=$(date +%s)
-  E2E_DURATION=$((E2E_END - E2E_START))
-  
-  echo -e "${BLUE}CFN E2E: ${E2E_PASSED} passed, ${E2E_FAILED} failed, ${E2E_SKIPPED} skipped (${E2E_DURATION}s)${NC}"
-  echo ""
-  
-  TOTAL_TESTS=$((TOTAL_TESTS + E2E_PASSED + E2E_FAILED + E2E_SKIPPED))
-  PASSED_TESTS=$((PASSED_TESTS + E2E_PASSED))
-  FAILED_TESTS=$((FAILED_TESTS + E2E_FAILED))
-  SKIPPED_TESTS=$((SKIPPED_TESTS + E2E_SKIPPED))
-fi
-
-END_TIME=$(date +%s)
-TOTAL_DURATION=$((END_TIME - START_TIME))
-SUCCESS_RATE=$(awk "BEGIN {printf \"%.1f\", ($PASSED_TESTS / $TOTAL_TESTS) * 100}")
-
-# Summary
-echo -e "${GREEN}=========================================="
-echo "Test Summary"
-echo -e "==========================================${NC}"
-echo "Total: $TOTAL_TESTS tests"
-echo "Passed: $PASSED_TESTS"
-echo "Failed: $FAILED_TESTS"
-echo "Skipped: $SKIPPED_TESTS"
-echo "Duration: ${TOTAL_DURATION}s"
-echo "Success Rate: ${SUCCESS_RATE}%"
-echo ""
-
-# Store benchmarks
-if [ "$BENCHMARK" = true ]; then
-  echo -e "${BLUE}[INFO]${NC} Storing benchmarks..."
-  "$SCRIPT_DIR/store-benchmarks.sh" \
-    --suite "$SUITE" \
-    --total "$TOTAL_TESTS" \
-    --passed "$PASSED_TESTS" \
-    --failed "$FAILED_TESTS" \
-    --skipped "$SKIPPED_TESTS" \
-    --duration "$TOTAL_DURATION" \
-    --commit "$GIT_COMMIT" \
-    --branch "$GIT_BRANCH"
-fi
-
-# Detect regressions
-if [ "$DETECT_REGRESSIONS" = true ]; then
-  echo -e "${BLUE}[INFO]${NC} Detecting regressions..."
-  if ! "$SCRIPT_DIR/detect-regressions.sh" --threshold "$THRESHOLD"; then
-    echo -e "${YELLOW}⚠️  Regressions detected!${NC}"
-  fi
-fi
-
-# Exit with failure if tests failed
-if [ $FAILED_TESTS -gt 0 ]; then
-  exit 1
-fi