@cubis/foundry 0.3.83 → 0.3.84

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/pentest-skill/evals/assertions.md

@@ -1,25 +0,0 @@
-# Eval Assertions — Penetration Testing Skill
-
-## Eval 1: Pentest Engagement Planning
-
-**ID:** `pentest-engagement-planning`
-
-| # | Assertion Type | Expected Value | Rationale |
-|---|---------------|----------------|-----------|
-| 1 | content_matches_regex | `authoriz\|permission\|rules of engagement\|scope\|legal` | The skill must acknowledge or verify authorization before proceeding with any planning, enforcing the auth gate. |
-| 2 | content_matches_regex | `reconnaissance\|recon\|OSINT\|enumerat` | The plan must include a reconnaissance phase as the first technical phase per PTES methodology. |
-| 3 | content_matches_regex | `nmap\|Burp\|nikto\|nuclei\|sqlmap\|scanner` | The plan must recommend specific, industry-standard penetration testing tools appropriate to the scope. |
-| 4 | content_matches_regex | `out.of.scope\|production database\|exclusion` | The plan must explicitly acknowledge scope exclusions to prevent unauthorized testing. |
-| 5 | content_matches_regex | `report\|finding\|deliverable\|documentation` | The plan must include a reporting and documentation phase as a core deliverable. |
-
-## Eval 2: Vulnerability Report Writing
-
-**ID:** `pentest-vulnerability-report`
-
-| # | Assertion Type | Expected Value | Rationale |
-|---|---------------|----------------|-----------|
-| 1 | content_matches_regex | `Broken Access Control\|IDOR\|authorization\|privilege\|A01` | Must correctly classify the finding as a Broken Access Control issue (OWASP A01). |
-| 2 | content_matches_regex | `Critical\|High\|[89]\\.[0-9]\|CVSS` | Must rate the finding as Critical or High given the exposure of 50,000 user records with password hashes. |
-| 3 | content_matches_regex | `role.based\|RBAC\|authorization check\|middleware\|guard` | Must recommend implementing role-based access control as the primary remediation. |
-| 4 | content_matches_regex | `50.000\|user records\|password hash\|PII\|data exposure` | Must describe the business impact including the scale of data exposure to convey urgency. |
-| 5 | content_matches_regex | `steps to reproduce\|reproduction\|proof\|evidence\|request\|response` | Must include reproduction steps or evidence documentation for the finding to be actionable. |

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/pentest-skill/evals/evals.json

@@ -1,68 +0,0 @@
-[
-  {
-    "id": "pentest-engagement-planning",
-    "title": "Plan a penetration test for an authorized web application engagement",
-    "description": "Given an authorized engagement scope, the skill should produce a structured pentest plan including reconnaissance strategy, tool selection, and rules of engagement — after confirming authorization.",
-    "prompt": "I have written authorization to perform a penetration test on our company's e-commerce platform at shop.example.com. The scope includes the web application, REST API at api.example.com, and the staging database. Production database is out of scope. Testing window is weekdays 6 PM - 6 AM EST. Plan the engagement.",
-    "assertions": [
-      {
-        "type": "content_matches_regex",
-        "value": "authoriz|permission|rules of engagement|scope|legal",
-        "description": "Must acknowledge or verify authorization before proceeding"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "reconnaissance|recon|OSINT|enumerat",
-        "description": "Must include a reconnaissance phase in the plan"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "nmap|Burp|nikto|nuclei|sqlmap|scanner",
-        "description": "Must recommend specific penetration testing tools"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "out.of.scope|production database|exclusion",
-        "description": "Must acknowledge scope exclusions (production database)"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "report|finding|deliverable|documentation",
-        "description": "Must include reporting and documentation as part of the plan"
-      }
-    ]
-  },
-  {
-    "id": "pentest-vulnerability-report",
-    "title": "Write a professional vulnerability report for a pentest finding",
-    "description": "Given exploitation details, the skill should produce a professional pentest finding report with evidence, severity, and remediation.",
-    "prompt": "During our authorized penetration test, I found that the /api/admin/users endpoint returns all user records including password hashes when accessed with any valid JWT token, regardless of the user's role. The endpoint does not check for admin role. I was able to retrieve 50,000 user records including bcrypt password hashes, email addresses, and phone numbers. Write the finding report.",
-    "assertions": [
-      {
-        "type": "content_matches_regex",
-        "value": "Broken Access Control|IDOR|authorization|privilege|A01",
-        "description": "Must classify as a Broken Access Control or authorization vulnerability"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "Critical|High|[89]\\.[0-9]|CVSS",
-        "description": "Must rate as Critical or High severity given the data exposure"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "role.based|RBAC|authorization check|middleware|guard",
-        "description": "Must recommend implementing role-based access control as remediation"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "50.000|user records|password hash|PII|data exposure",
-        "description": "Must describe the business impact including the scale of data exposure"
-      },
-      {
-        "type": "content_matches_regex",
-        "value": "steps to reproduce|reproduction|proof|evidence|request|response",
-        "description": "Must include steps to reproduce or evidence documentation"
-      }
-    ]
-  }
-]

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/performance-testing/SKILL.md

@@ -1,122 +0,0 @@
----
-name: performance-testing
-description: "Performance testing and benchmarking covering load testing, stress testing, profiling, and bottleneck identification. Use when evaluating system performance or detecting regressions."
----
-# Performance Testing and Benchmarking
-
-## Purpose
-
-Provide a systematic methodology for designing, executing, and interpreting performance tests that reveal throughput limits, latency characteristics, and resource bottlenecks. This skill ensures performance is measured objectively, regressions are detected early, and optimization efforts target the actual bottleneck rather than guesswork.
-
-## When to Use
-
-- Designing load test scenarios for a new feature or service.
-- Establishing baseline performance metrics before optimization.
-- Running stress tests to find the breaking point of a system.
-- Profiling application code to identify CPU, memory, or I/O hotspots.
-- Investigating a reported performance regression.
-- Integrating performance benchmarks into CI/CD pipelines.
-- Capacity planning for expected traffic growth.
-- Validating that an optimization actually improved performance.
-
-## Instructions
-
-1. **Define performance objectives with SLOs** -- Establish quantitative targets (p50 < 200ms, p99 < 1s, throughput > 500 rps) before testing. Without targets, results are just numbers with no pass/fail criteria.
-
-2. **Characterize the workload profile** -- Identify realistic user patterns, request distributions, and data volumes. Synthetic workloads that don't match production produce misleading results.
-
-3. **Establish a baseline before changing anything** -- Record current performance metrics under controlled conditions. A baseline is the only way to measure whether a change helped, hurt, or had no effect.
-
-4. **Design load tests with a ramp-up pattern** -- Gradually increase concurrent users or request rate rather than slamming full load instantly. Ramp-ups reveal the load level where degradation begins.
-
-5. **Run stress tests beyond expected capacity** -- Push the system past its limits to discover failure modes, resource exhaustion points, and recovery behavior. Knowing the ceiling prevents surprise outages.
-
-6. **Use representative test data** -- Populate databases with production-scale data volumes and distributions. Performance on empty tables or small datasets hides query plan issues and cache behavior.
-
-7. **Profile before optimizing** -- Use CPU profilers, memory profilers, and tracing tools to find actual hotspots. Optimizing without profiling wastes effort on code paths that don't matter.
-
-8. **Measure latency at meaningful percentiles** -- Report p50, p90, p95, and p99 latency, not just averages. Averages hide tail latency that affects real users.
-
-9. **Isolate the variable under test** -- Change one thing at a time and measure the effect. Simultaneous changes make it impossible to attribute improvement or degradation to a specific cause.
-
-10. **Monitor resource utilization during tests** -- Track CPU, memory, disk I/O, network, and connection pools alongside request metrics. Resource saturation explains why throughput plateaus or latency spikes.
-
-11. **Account for warm-up effects** -- Discard initial measurements affected by JIT compilation, cache population, and connection pool initialization. Cold-start metrics don't represent steady-state performance.
-
-12. **Automate benchmarks in CI/CD** -- Run a standardized benchmark suite on every commit or pull request. Automated benchmarks catch regressions before they reach production.
-
-13. **Compare results statistically** -- Use multiple runs, confidence intervals, and statistical tests rather than single-run comparisons. A single run can be an outlier that leads to wrong conclusions.
-
-14. **Test under realistic network conditions** -- Introduce representative latency, bandwidth limits, and packet loss for distributed systems. LAN-only testing hides performance issues that appear in production.
-
-15. **Document findings with reproducible methodology** -- Record tool versions, configuration, environment specs, and exact commands. Undocumented benchmarks cannot be reproduced or trusted.
-
-16. **Feed results back into capacity planning** -- Use test data to model resource requirements for projected growth. Performance testing without capacity planning leaves the team reactive instead of prepared.
-
-## Output Format
-
-```markdown
-## Performance Test Report
-
-### Objectives
-| SLO Metric       | Target       | Result       | Status  |
-|------------------|-------------|-------------|---------|
-| p50 Latency      | < 200ms     | ...         | PASS/FAIL |
-| p99 Latency      | < 1000ms    | ...         | PASS/FAIL |
-| Throughput        | > 500 rps   | ...         | PASS/FAIL |
-| Error Rate        | < 0.1%      | ...         | PASS/FAIL |
-
-### Load Profile
-- **Tool:** k6 / Artillery / JMeter / Locust
-- **Duration:** <total time>
-- **Ramp-up:** <pattern>
-- **Peak Concurrent Users:** <number>
-
-### Latency Distribution
-| Percentile | Latency (ms) |
-|-----------|-------------|
-| p50       | ...         |
-| p90       | ...         |
-| p95       | ...         |
-| p99       | ...         |
-
-### Resource Utilization
-| Resource    | Peak     | Average  | Saturation? |
-|------------|---------|---------|-------------|
-| CPU        | ...     | ...     | Yes/No      |
-| Memory     | ...     | ...     | Yes/No      |
-| Disk I/O   | ...     | ...     | Yes/No      |
-| Network    | ...     | ...     | Yes/No      |
-
-### Bottleneck Analysis
-| Bottleneck       | Evidence              | Recommendation        |
-|-----------------|----------------------|----------------------|
-| ...             | ...                  | ...                  |
-
-### Methodology
-- **Environment:** <specs>
-- **Data Volume:** <description>
-- **Commands:** <exact commands to reproduce>
-```
-
-## References
-
-| Topic                | File                                  | Load When                                        |
-|----------------------|---------------------------------------|-------------------------------------------------|
-| Load Testing         | `references/load-testing.md`          | Designing and running load test scenarios        |
-| Profiling            | `references/profiling.md`             | Using CPU, memory, and I/O profilers             |
-| Benchmarking         | `references/benchmarking.md`          | Writing and interpreting microbenchmarks          |
-| Bottleneck Analysis  | `references/bottleneck-analysis.md`   | Identifying and resolving performance hotspots   |
-| CI Integration       | `references/ci-integration.md`        | Automating performance tests in pipelines        |
-
-## Gemini Platform Notes
-
-- Workflow and agent routes are compiled into `.gemini/commands/*.toml` TOML command files.
-- Commands use `{{args}}` for user input, `!{shell command}` for shell output, `@{file}` for file content.
-- Specialists are internal postures (modes of reasoning), not spawned subagent processes.
-- Gemini does not support `context: fork` — all skill execution is inline within the current session.
-- Skills are loaded via MCP when the Cubis Foundry MCP server is configured. Local `.agents/skills/` paths serve as hints.
-- User arguments are passed as natural language in the activation prompt.
-- Rules file: `.gemini/GEMINI.md`.
-- Reference files are loaded relative to the skill directory under `.agents/skills/<skill-id>/`.
-- MCP skill tools (`skill_search`, `skill_get`, `skill_validate`, `skill_get_reference`) are available when MCP is connected.

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/performance-testing/evals/assertions.md

@@ -1,43 +0,0 @@
-# Performance Testing -- Eval Assertions
-
-## Eval 001: Load Test Scenario Design for E-Commerce Checkout
-
-| # | Assertion Type | Value | Rationale |
-|---|---------------|-------|-----------|
-| 1 | contains | `ramp` | The response must define a ramp-up pattern that gradually increases load from normal (50 users) to flash-sale levels (500 users), revealing the degradation threshold. |
-| 2 | contains | `SLO` | The response must define specific Service Level Objectives with quantitative thresholds (e.g., p95 < 500ms, error rate < 1%) as pass/fail criteria. |
-| 3 | contains | `percentile` | The response must measure latency at meaningful percentiles (p50, p90, p95, p99) rather than relying on averages that hide tail latency. |
-| 4 | contains | `CPU` | The response must include resource monitoring for CPU, memory, or other infrastructure metrics alongside request-level measurements. |
-| 5 | contains | `scenario` | The response must model realistic user scenarios with sequential steps (browse, cart, checkout, payment) and think times between actions. |
-
-### What a Passing Response Looks Like
-
-A passing response designs a load test that:
-- Models the checkout flow as a multi-step scenario: browse -> add to cart -> checkout -> payment.
-- Defines a ramp-up from 50 to 500 concurrent users over a reasonable period (e.g., 5-10 minutes).
-- Sets SLOs for p95 latency, throughput, and error rate with specific numeric targets.
-- Reports latency at p50, p90, p95, and p99 percentiles.
-- Monitors server-side CPU, memory, and connection pool usage during the test.
-- Includes think times between steps to simulate real user behavior.
-
----
-
-## Eval 002: Performance Regression Detection in CI
-
-| # | Assertion Type | Value | Rationale |
-|---|---------------|-------|-----------|
-| 1 | contains | `baseline` | The response must describe a baseline management strategy for storing historical benchmark results and comparing against them. |
-| 2 | contains | `threshold` | The response must define regression thresholds (e.g., >10% degradation) that trigger build failures to prevent slow merges. |
-| 3 | contains | `CI` | The response must integrate into the CI pipeline (GitHub Actions or equivalent) to run automatically on every PR. |
-| 4 | contains | `compar` | The response must describe how current benchmark results are compared against stored baselines using statistical or threshold methods. |
-| 5 | contains | `alert` | The response must include alerting or notification when a regression is detected so the team can act immediately. |
-
-### What a Passing Response Looks Like
-
-A passing response designs a regression detection system that:
-- Stores baseline benchmark results in a persistent location (artifact storage, git, or database).
-- Runs a standardized benchmark suite on every PR in GitHub Actions.
-- Compares PR results against the baseline using a configurable threshold (e.g., 10% degradation = failure).
-- Fails the build and posts a comment on the PR when regression is detected.
-- Updates the baseline when a PR merges to main, keeping the reference current.
-- Handles statistical noise through multiple runs or confidence intervals.

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/performance-testing/evals/evals.json

@@ -1,68 +0,0 @@
-[
-  {
-    "id": "performance-testing-eval-001",
-    "name": "Load test scenario design for e-commerce checkout",
-    "description": "Evaluate the skill's ability to design a comprehensive load test scenario for an e-commerce checkout flow, including workload modeling, ramp-up strategy, SLOs, and resource monitoring.",
-    "prompt": "Our e-commerce site expects a flash sale that will drive 10x normal traffic. Design a load test scenario for the checkout flow (browse -> add to cart -> checkout -> payment). Include the workload model, ramp-up pattern, SLO definitions, resource monitoring plan, and the exact k6 or equivalent test structure. Normal traffic is 50 concurrent users with 200ms average response time.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "ramp",
-        "description": "Should define a ramp-up pattern that gradually increases load to 10x (500 concurrent users)"
-      },
-      {
-        "type": "contains",
-        "value": "SLO",
-        "description": "Should define specific SLOs with quantitative thresholds for latency and error rate"
-      },
-      {
-        "type": "contains",
-        "value": "percentile",
-        "description": "Should measure latency at meaningful percentiles (p50, p90, p95, p99), not just averages"
-      },
-      {
-        "type": "contains",
-        "value": "CPU",
-        "description": "Should include resource monitoring for CPU, memory, or other infrastructure metrics"
-      },
-      {
-        "type": "contains",
-        "value": "scenario",
-        "description": "Should model realistic user scenarios with think times and step sequences"
-      }
-    ]
-  },
-  {
-    "id": "performance-testing-eval-002",
-    "name": "Performance regression detection in CI",
-    "description": "Evaluate the skill's ability to design a CI-integrated performance regression detection system that catches degradations before they reach production.",
-    "prompt": "Our API's p95 latency has been creeping up over the last month but nobody noticed until users complained. Design a performance regression detection system integrated into our GitHub Actions CI pipeline. It should run benchmarks on every PR, compare against baselines, and fail the build if performance degrades beyond a threshold. Include the baseline management strategy and alerting approach.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "baseline",
-        "description": "Should describe a baseline management strategy for storing and comparing benchmark results"
-      },
-      {
-        "type": "contains",
-        "value": "threshold",
-        "description": "Should define regression thresholds that trigger build failures (e.g., 10% degradation)"
-      },
-      {
-        "type": "contains",
-        "value": "CI",
-        "description": "Should integrate directly into CI pipeline to run automatically on PRs"
-      },
-      {
-        "type": "contains",
-        "value": "compar",
-        "description": "Should compare current results against stored baselines using statistical or threshold methods"
-      },
-      {
-        "type": "contains",
-        "value": "alert",
-        "description": "Should include alerting or notification when regression is detected"
-      }
-    ]
-  }
-]

package/workflows/workflows/agent-environment-setup/platforms/gemini/skills/performance-testing/examples/01-load-test-scenario.md

@@ -1,160 +0,0 @@
-# Example: Load Testing an API Gateway with k6
-
-## Context
-
-An API gateway handles user authentication and proxies requests to downstream services. A marketing campaign is expected to drive 5x normal traffic. The team needs to validate the system can handle the spike.
-
-## User Prompt
-
-> Our API gateway normally handles 200 requests per second. We expect a 5x spike next week from a marketing email blast. Design a k6 load test for the /api/auth/login and /api/users/me endpoints. Show me the full test script, SLOs, and how to interpret the results.
-
-## Skill Response
-
-### Performance Objectives (SLOs)
-
-| Metric       | Target (Normal) | Target (5x Load) | Failure Threshold |
-|-------------|----------------|-------------------|-------------------|
-| p50 Latency  | < 100ms        | < 200ms           | > 500ms           |
-| p95 Latency  | < 300ms        | < 800ms           | > 2000ms          |
-| p99 Latency  | < 500ms        | < 1500ms          | > 3000ms          |
-| Throughput   | 200 rps        | 1000 rps          | < 800 rps         |
-| Error Rate   | < 0.1%         | < 0.5%            | > 1%              |
-
-### k6 Test Script
-
-```javascript
-// load-tests/api-gateway-load.js
-import http from 'k6/http';
-import { check, sleep, group } from 'k6';
-import { Rate, Trend } from 'k6/metrics';
-
-// Custom metrics
-const loginLatency = new Trend('login_latency', true);
-const profileLatency = new Trend('profile_latency', true);
-const errorRate = new Rate('errors');
-
-// Test configuration: ramp up to 5x, hold, then ramp down
-export const options = {
-  stages: [
-    { duration: '2m', target: 50 },    // Warm-up to baseline
-    { duration: '3m', target: 50 },    // Hold at baseline (200 rps)
-    { duration: '5m', target: 250 },   // Ramp to 5x load (1000 rps)
-    { duration: '5m', target: 250 },   // Hold at peak
-    { duration: '2m', target: 0 },     // Ramp down (recovery test)
-  ],
-  thresholds: {
-    'login_latency': ['p(95)<800', 'p(99)<1500'],
-    'profile_latency': ['p(95)<800', 'p(99)<1500'],
-    'errors': ['rate<0.005'],           // < 0.5% error rate
-    'http_req_duration': ['p(95)<800'],
-  },
-};
-
-const BASE_URL = __ENV.BASE_URL || 'https://staging-api.example.com';
-
-// Test data: pre-generated user credentials
-const users = JSON.parse(open('./test-data/users.json'));
-
-export default function () {
-  const user = users[Math.floor(Math.random() * users.length)];
-
-  group('Login Flow', () => {
-    // Step 1: Authenticate
-    const loginRes = http.post(
-      `${BASE_URL}/api/auth/login`,
-      JSON.stringify({
-        email: user.email,
-        password: user.password,
-      }),
-      { headers: { 'Content-Type': 'application/json' } }
-    );
-
-    loginLatency.add(loginRes.timings.duration);
-
-    const loginOk = check(loginRes, {
-      'login status 200': (r) => r.status === 200,
-      'login has token': (r) => r.json('token') !== undefined,
-      'login under 800ms': (r) => r.timings.duration < 800,
-    });
-
-    if (!loginOk) {
-      errorRate.add(1);
-      return; // Skip profile fetch if login failed
-    }
-    errorRate.add(0);
-
-    const token = loginRes.json('token');
-
-    // Think time: simulate user reading the dashboard
-    sleep(Math.random() * 2 + 1); // 1-3 seconds
-
-    // Step 2: Fetch user profile
-    const profileRes = http.get(`${BASE_URL}/api/users/me`, {
-      headers: { Authorization: `Bearer ${token}` },
-    });
-
-    profileLatency.add(profileRes.timings.duration);
-
-    const profileOk = check(profileRes, {
-      'profile status 200': (r) => r.status === 200,
-      'profile has email': (r) => r.json('email') !== undefined,
-      'profile under 800ms': (r) => r.timings.duration < 800,
-    });
-
-    if (!profileOk) errorRate.add(1);
-    else errorRate.add(0);
-  });
-
-  // Think time between iterations
-  sleep(Math.random() * 3 + 2); // 2-5 seconds
-}
-```
-
-### Running the Test
-
-```bash
-# Against staging environment
-k6 run --out json=results.json \
-  -e BASE_URL=https://staging-api.example.com \
-  load-tests/api-gateway-load.js
-
-# With Grafana Cloud k6 for dashboard visualization
-k6 cloud load-tests/api-gateway-load.js
-```
-
-### Interpreting Results
-
-```
-     scenarios: (100.00%) 1 scenario, 250 max VUs, 17m30s max duration
-     ...
-     ✓ login status 200........: 99.7%  ✓ 48231  ✗ 145
-     ✓ login has token..........: 99.7%  ✓ 48231  ✗ 145
-     ✗ login under 800ms........: 94.2%  ✓ 45432  ✗ 2799
-       ↳ INVESTIGATION NEEDED: 5.8% of logins exceeded 800ms
-
-     login_latency...........: avg=187ms  p(50)=142ms  p(95)=623ms  p(99)=1289ms
-     profile_latency.........: avg=95ms   p(50)=72ms   p(95)=312ms  p(99)=687ms
-     errors..................: 0.30%  ✓ 290  ✗ 96172
-
-     ✓ p(95) login_latency < 800ms
-     ✓ p(99) login_latency < 1500ms
-     ✓ error rate < 0.5%
-```
-
-### Analysis
-
-| Finding | Severity | Evidence | Recommendation |
-|---------|----------|----------|----------------|
-| Login p95 at 623ms (target: 800ms) | Medium | Only 177ms headroom | Profile login endpoint; consider connection pool tuning |
-| 5.8% of logins exceeded 800ms | High | Concentrated during ramp-up phase | Investigate cold-start behavior; pre-warm connection pools |
-| Profile endpoint well within SLOs | Low | p99 at 687ms vs 1500ms target | No action needed |
-| Error rate 0.3% at peak | Low | Within 0.5% threshold | Monitor during real event |
-
-### Resource Monitoring Checklist
-
-During the test, the following should be tracked on the server dashboard:
-- CPU utilization per pod/instance
-- Memory usage and garbage collection pauses
-- Database connection pool usage (active vs. idle)
-- Network bandwidth and connection count
-- Downstream service response times