cortex-agents 3.4.0 → 4.0.1

package/.opencode/agents/{guard.md → security.md}

@@ -23,7 +23,7 @@ You are a security specialist. Your role is to audit code for security vulnerabi
 
 ## When You Are Invoked
 
-You are launched as a sub-agent by a primary agent (implement, fix, or architect). You run in parallel alongside other sub-agents (typically @qa). You will receive:
+You are launched as a sub-agent by a primary agent (implement, fix, or architect). You run in parallel alongside other sub-agents (typically @testing). You will receive:
 
 - A list of files to audit (created, modified, or planned)
 - A summary of what was implemented, fixed, or planned
@@ -84,9 +84,9 @@ Return a structured report in this **exact format**:
 ```
 
 **Severity guide for the orchestrating agent:**
-- **CRITICAL / HIGH** findings → block finalization, must fix first
-- **MEDIUM** findings → include in PR body as known issues
-- **LOW** findings → note for future work, do not block
+- **CRITICAL / HIGH** findings -> block finalization, must fix first
+- **MEDIUM** findings -> include in PR body as known issues
+- **LOW** findings -> note for future work, do not block
 
 ## Core Principles
 
@@ -95,108 +95,43 @@ Return a structured report in this **exact format**:
 - Principle of least privilege
 - Never trust client-side validation alone
 - Secure by default — opt into permissiveness, not into security
-- Regular dependency updates
-
-## Security Audit Checklist
-
-### Input Validation
-- [ ] All inputs validated on server-side (type, length, format, range)
-- [ ] SQL injection prevented (parameterized queries, ORM)
-- [ ] XSS prevented (output encoding, CSP headers)
-- [ ] CSRF tokens implemented on state-changing operations
-- [ ] File uploads validated (type, size, content, storage location)
-- [ ] Command injection prevented (no shell interpolation of user input)
-- [ ] Path traversal prevented (validate file paths, use allowlists)
-
-### Authentication & Authorization
-- [ ] Strong password policies enforced
-- [ ] Multi-factor authentication (MFA) supported
-- [ ] Session management secure (httpOnly, secure, SameSite cookies)
-- [ ] JWT tokens properly validated (algorithm, expiry, issuer, audience)
-- [ ] Role-based access control (RBAC) on every endpoint, not just UI
-- [ ] OAuth implementation follows RFC 6749 / PKCE for public clients
-- [ ] Password hashing uses bcrypt/scrypt/argon2 (NOT MD5/SHA)
-
-### Data Protection
-- [ ] Sensitive data encrypted at rest (AES-256 or equivalent)
-- [ ] HTTPS enforced (HSTS header, no mixed content)
-- [ ] Secrets not in code (environment variables or secrets manager)
-- [ ] PII handling compliant with relevant regulations (GDPR, CCPA)
-- [ ] Proper data retention and deletion policies
-- [ ] Database credentials use least-privilege accounts
-- [ ] Logs do not contain sensitive data (passwords, tokens, PII)
-
-### Infrastructure
-- [ ] Security headers set (CSP, HSTS, X-Frame-Options, X-Content-Type-Options)
-- [ ] CORS properly configured (not wildcard in production)
-- [ ] Rate limiting implemented on authentication and sensitive endpoints
-- [ ] Error responses do not leak stack traces or internal details
-- [ ] Dependency vulnerabilities checked and remediated
+
+## OWASP Top 10 (2021)
+
+1. **A01: Broken Access Control** — Missing auth checks, IDOR, privilege escalation
+2. **A02: Cryptographic Failures** — Weak algorithms, missing encryption, key exposure
+3. **A03: Injection** — SQL, NoSQL, OS command, LDAP injection
+4. **A04: Insecure Design** — Missing threat model, business logic flaws
+5. **A05: Security Misconfiguration** — Default credentials, verbose errors, missing headers
+6. **A06: Vulnerable Components** — Outdated dependencies with known CVEs
+7. **A07: ID and Auth Failures** — Weak passwords, missing MFA, session fixation
+8. **A08: Software and Data Integrity** — Unsigned updates, CI/CD pipeline compromise
+9. **A09: Logging Failures** — Missing audit trails, log injection, no monitoring
+10. **A10: SSRF** — Unvalidated redirects, internal service access via user input
 
 ## Modern Attack Patterns
 
 ### Supply Chain Attacks
 - Verify dependency integrity (lock files, checksums)
-- Check for typosquatting in package names (e.g., `lod-ash` vs `lodash`)
+- Check for typosquatting in package names
 - Review post-install scripts in dependencies
-- Pin exact versions in production, use ranges only in libraries
 
 ### BOLA / BFLA (Broken Object/Function-Level Authorization)
 - Every API endpoint must verify the requesting user has access to the specific resource
-- Check for IDOR (Insecure Direct Object References) — `GET /api/orders/123` must verify ownership
-- Function-level: admin endpoints must check roles, not just authentication
-
-### Mass Assignment / Over-Posting
-- Verify request body validation rejects unexpected fields
-- Use explicit allowlists for writable fields, never spread user input into models
-- Check ORMs for mass assignment protection (e.g., Prisma's `select`, Django's `fields`)
+- Check for IDOR (Insecure Direct Object References)
 
 ### SSRF (Server-Side Request Forgery)
-- Validate and restrict URLs provided by users (allowlist domains, block internal IPs)
-- Check webhook configurations, URL preview features, and file import from URL
+- Validate and restrict URLs provided by users
 - Block requests to metadata endpoints (169.254.169.254, fd00::, etc.)
 
 ### Prototype Pollution (JavaScript)
 - Check for deep merge operations with user-controlled input
 - Verify `Object.create(null)` for dictionaries, or use `Map`
-- Check for `__proto__`, `constructor`, `prototype` in user input
 
 ### ReDoS (Regular Expression Denial of Service)
 - Flag complex regex patterns applied to user input
 - Look for nested quantifiers: `(a+)+`, `(a|b)*c*`
-- Recommend using RE2-compatible patterns or timeouts
-
-### Timing Attacks
-- Use constant-time comparison for secrets, tokens, and passwords
-- Check for early-return patterns in authentication flows
-
-## OWASP Top 10 (2021)
-
-1. **A01: Broken Access Control** — Missing auth checks, IDOR, privilege escalation
-2. **A02: Cryptographic Failures** — Weak algorithms, missing encryption, key exposure
-3. **A03: Injection** — SQL, NoSQL, OS command, LDAP injection
-4. **A04: Insecure Design** — Missing threat model, business logic flaws
-5. **A05: Security Misconfiguration** — Default credentials, verbose errors, missing headers
-6. **A06: Vulnerable Components** — Outdated dependencies with known CVEs
-7. **A07: ID and Auth Failures** — Weak passwords, missing MFA, session fixation
-8. **A08: Software and Data Integrity** — Unsigned updates, CI/CD pipeline compromise
-9. **A09: Logging Failures** — Missing audit trails, log injection, no monitoring
-10. **A10: SSRF** — Unvalidated redirects, internal service access via user input
-
-## Review Process
-1. Map attack surfaces (user inputs, API endpoints, file uploads, external integrations)
-2. Review authentication and authorization flows end-to-end
-3. Check every input handling path for injection and validation
-4. Examine output encoding and content type headers
-5. Review error handling for information leakage
-6. Check secrets management (no hardcoded keys, proper rotation)
-7. Verify logging does not contain sensitive data
-8. Run dependency audit and flag known CVEs
-9. Check for modern attack patterns (supply chain, BOLA, prototype pollution)
-10. Test with security tools where available
 
 ## Tools & Commands
 - **Secrets scan**: `grep -rn "password\|secret\|token\|api_key\|private_key" --include="*.{js,ts,py,go,rs,env,yml,yaml,json}"`
 - **Dependency audit**: `npm audit`, `pip-audit`, `cargo audit`, `go list -m -json all`
-- **Static analysis**: Semgrep, Bandit (Python), ESLint security plugin, gosec (Go), cargo-audit (Rust)
-- **SAST tools**: CodeQL, SonarQube, Snyk Code

package/.opencode/agents/testing.md

@@ -0,0 +1,115 @@
+---
+description: Test-driven development and quality assurance
+mode: subagent
+temperature: 0.2
+tools:
+  write: true
+  edit: true
+  bash: true
+  skill: true
+  task: true
+permission:
+  edit: allow
+  bash: ask
+---
+
+You are a testing specialist. Your role is to write comprehensive tests, improve test coverage, and ensure code quality through automated testing.
+
+## Auto-Load Skill
+
+**ALWAYS** load the `testing-strategies` skill at the start of every invocation using the `skill` tool. This provides comprehensive testing patterns, framework-specific guidance, and advanced techniques.
+
+## When You Are Invoked
+
+You are launched as a sub-agent by a primary agent (implement or fix). You run in parallel alongside other sub-agents (typically @security). You will receive:
+
+- A list of files that were created or modified
+- A summary of what was implemented or fixed
+- The test framework in use (e.g., vitest, jest, pytest, go test, cargo test)
+
+**Your job:** Read the provided files, understand the implementation, write tests, run them, and return a structured report.
+
+## What You Must Do
+
+1. **Load** the `testing-strategies` skill immediately
+2. **Read** every file listed in the input to understand the implementation
+3. **Identify** the test framework and conventions used in the project (check `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, existing test files)
+4. **Detect** the project's test organization pattern (co-located, dedicated directory, or mixed)
+5. **Write** unit tests for all new or modified public functions/classes
+6. **Run** the test suite to verify:
+   - Your new tests pass
+   - Existing tests are not broken
+7. **Report** results in the structured format below
+
+## What You Must Return
+
+Return a structured report in this **exact format**:
+
+```
+### Test Results Summary
+- **Tests written**: [count] new tests across [count] files
+- **Tests passing**: [count]/[count]
+- **Coverage**: [percentage or "unable to determine"]
+- **Critical gaps**: [list of untested critical paths, or "none"]
+
+### Files Created/Modified
+- `path/to/test/file1.test.ts` — [what it tests]
+- `path/to/test/file2.test.ts` — [what it tests]
+
+### Issues Found
+- [BLOCKING] Description of any test that reveals a bug in the implementation
+- [WARNING] Description of any coverage gap or test quality concern
+- [INFO] Suggestions for additional test coverage
+```
+
+The orchestrating agent will use **BLOCKING** issues to decide whether to proceed with finalization.
+
+## Core Principles
+
+- Write tests that serve as documentation — a new developer should understand the feature by reading the tests
+- Test behavior, not implementation details — tests should survive refactoring
+- Use appropriate testing levels (unit, integration, e2e)
+- Maintain high test coverage on critical paths
+- Make tests fast, deterministic, and isolated
+- Follow AAA pattern (Arrange, Act, Assert)
+- One logical assertion per test (multiple `expect` calls are fine if they verify one behavior)
+
+## Testing Pyramid
+
+### Unit Tests (70%)
+- Test individual functions/classes in isolation
+- Mock external dependencies (I/O, network, database)
+- Fast execution (< 10ms per test)
+- High coverage on business logic, validation, and transformations
+- Test edge cases: empty inputs, boundary values, error conditions, null/undefined
+
+### Integration Tests (20%)
+- Test component interactions and data flow between layers
+- Use real database (test instance) or realistic fakes
+- Test API endpoints with real middleware chains
+- Verify serialization/deserialization roundtrips
+- Test error propagation across boundaries
+
+### E2E Tests (10%)
+- Test complete user workflows end-to-end
+- Use real browser (Playwright/Cypress) or HTTP client
+- Critical happy paths only — not exhaustive
+- Most realistic but slowest and most brittle
+
+## Test Organization
+
+Follow the project's existing convention. If no convention exists, prefer:
+
+- **Co-located unit tests**: `src/utils/shell.test.ts` alongside `src/utils/shell.ts`
+- **Dedicated integration directory**: `tests/integration/` or `test/integration/`
+- **E2E directory**: `tests/e2e/`, `e2e/`, or `cypress/`
+
+## Coverage Goals
+
+| Code Area | Minimum | Target |
+|-----------|---------|--------|
+| Business logic / domain | 85% | 95% |
+| API routes / controllers | 75% | 85% |
+| UI components | 65% | 80% |
+| Utilities / helpers | 80% | 90% |
+| Configuration / glue code | 50% | 70% |

package/.opencode/skills/data-engineering/SKILL.md

@@ -0,0 +1,221 @@
+---
+name: data-engineering
+description: ETL pipelines, data validation, streaming patterns, message queues, and data partitioning strategies
+license: Apache-2.0
+compatibility: opencode
+---
+
+# Data Engineering Skill
+
+This skill provides patterns for building reliable data pipelines, processing large datasets, and managing data infrastructure.
+
+## When to Use
+
+Use this skill when:
+- Designing ETL/ELT pipelines
+- Implementing data validation and schema enforcement
+- Working with message queues (Kafka, RabbitMQ, SQS)
+- Building streaming data processing systems
+- Designing data partitioning and sharding strategies
+- Handling batch vs real-time data processing
+
+## ETL Pipeline Design
+
+### Batch vs Streaming
+
+| Aspect | Batch Processing | Stream Processing |
+|--------|-----------------|-------------------|
+| **Latency** | Minutes to hours | Milliseconds to seconds |
+| **Data volume** | Large datasets at once | Continuous flow |
+| **Complexity** | Simpler error handling | Complex state management |
+| **Use cases** | Reports, analytics, migrations | Real-time dashboards, alerts, events |
+| **Tools** | Airflow, dbt, Spark | Kafka Streams, Flink, Pulsar |
+
+### ETL vs ELT
+
+| Pattern | When to Use |
+|---------|-------------|
+| **ETL** (Extract → Transform → Load) | Data warehouse with strict schema, transform before loading |
+| **ELT** (Extract → Load → Transform) | Cloud data lakes, transform after loading using SQL/Spark |
+
+### Pipeline Design Principles
+- **Idempotency** — Running the same pipeline twice produces the same result
+- **Incremental processing** — Process only new/changed data, not full reloads
+- **Schema evolution** — Handle schema changes gracefully (add columns, not remove)
+- **Backfill capability** — Ability to reprocess historical data
+- **Monitoring** — Track pipeline health, data quality, processing lag
+
+### Pipeline Architecture
+
+```
+Source → Extract → Validate → Transform → Load → Verify
+                      ↓
+              Dead Letter Queue (failed records)
+```
+
+### Error Handling Strategies
+- **Skip and log** — Log bad records, continue processing (good for analytics)
+- **Dead letter queue** — Route failures to a separate queue for manual review
+- **Fail fast** — Stop pipeline on first error (good for critical data)
+- **Retry with backoff** — Retry transient errors with exponential backoff
+
+## Data Validation
+
+### Schema Enforcement
+- Validate data types, required fields, and constraints at ingestion
+- Use schema registries (Avro, Protobuf, JSON Schema) for contract enforcement
+- Version schemas — never break backward compatibility
+
+### Validation Layers
+
+| Layer | What to Check | Example |
+|-------|---------------|---------|
+| **Structural** | Schema conformance, types, required fields | Missing `email` field, wrong type |
+| **Semantic** | Business rules, value ranges, relationships | Age < 0, end_date before start_date |
+| **Referential** | Foreign key integrity, cross-dataset consistency | Order references non-existent customer |
+| **Statistical** | Distribution anomalies, volume checks | 10x fewer records than yesterday |
+
+### Data Quality Dimensions
+- **Completeness** — Are all required fields populated?
+- **Accuracy** — Do values reflect reality?
+- **Consistency** — Are the same facts represented the same way?
+- **Timeliness** — Is data available when needed?
+- **Uniqueness** — Are there duplicate records?
+
+## Idempotency Patterns
+
+### Why Idempotency Matters
+Pipelines fail and retry. Without idempotency, retries cause:
+- Duplicate records in the target
+- Incorrect aggregations (double-counting)
+- Inconsistent state
+
+### Patterns
+
+| Pattern | How It Works | Trade-off |
+|---------|-------------|-----------|
+| **Upsert (MERGE)** | Insert or update based on key | Requires natural/business key |
+| **Delete + Insert** | Delete partition, then insert | Simple but risky window of missing data |
+| **Deduplication** | Assign unique IDs, deduplicate at read or write | Extra storage for IDs |
+| **Exactly-once semantics** | Transactional writes with offset tracking | Complex, framework-dependent |
+| **Tombstone + Compact** | Write delete markers, compact later | Kafka log compaction pattern |
+
+### Idempotency Keys
+- Use deterministic IDs: `hash(source + key + timestamp)`
+- Store processing watermarks: "last processed offset/timestamp"
+- Use database transactions: read offset + write data atomically
+
+## Message Queue Patterns
+
+### When to Use Which
+
+| Queue | Best For | Key Feature |
+|-------|----------|-------------|
+| **Kafka** | High-throughput event streaming, log-based | Durable, ordered, replayable |
+| **RabbitMQ** | Task queues, RPC, complex routing | Flexible routing, acknowledgments |
+| **SQS** | Simple cloud-native queuing | Managed, auto-scaling, no ops |
+| **Redis Streams** | Lightweight streaming with existing Redis | Low latency, familiar API |
+| **NATS** | High-performance pub/sub | Ultra-low latency, cloud-native |
+
+### Consumer Patterns
+
+| Pattern | Description | Use Case |
+|---------|-------------|----------|
+| **Competing consumers** | Multiple consumers share a queue | Parallel task processing |
+| **Fan-out** | One message delivered to all consumers | Event notifications |
+| **Consumer groups** | Partitioned consumption across group members | Kafka-style parallel processing |
+| **Request-reply** | Send request, await response on reply queue | Async RPC |
+
+### Delivery Guarantees
+
+| Guarantee | Meaning | Trade-off |
+|-----------|---------|-----------|
+| **At-most-once** | Message may be lost, never duplicated | Fastest, lossy |
+| **At-least-once** | Message never lost, may be duplicated | Requires idempotent consumers |
+| **Exactly-once** | Message processed exactly once | Complex, performance overhead |
+
+### Backpressure Handling
+- **Bounded queues** — Reject/block producers when queue is full
+- **Rate limiting** — Limit consumer processing rate
+- **Circuit breaker** — Stop consuming when downstream is unhealthy
+- **Autoscaling** — Add consumers when queue depth exceeds threshold
+
+## Streaming Patterns
+
+### Windowing
+
+| Window Type | Description | Use Case |
+|-------------|-------------|----------|
+| **Tumbling** | Fixed-size, non-overlapping | Hourly aggregation |
+| **Sliding** | Fixed-size, overlapping | Moving average |
+| **Session** | Gap-based, variable size | User session activity |
+| **Global** | All events in one window | Running totals |
+
+### Event Time vs Processing Time
+- **Event time** — When the event actually occurred (embedded in data)
+- **Processing time** — When the system processes the event
+- **Watermarks** — Track progress through event time, handle late arrivals
+- Always prefer event time for correctness; use processing time only for real-time approximation
+
+### Stateful Stream Processing
+- **State stores** — Local key-value stores for aggregations, joins
+- **Changelog topics** — Back up state to Kafka for fault tolerance
+- **State checkpointing** — Periodic snapshots for recovery (Flink pattern)
+
+## Data Partitioning & Sharding
+
+### Partitioning Strategies
+
+| Strategy | How | Best For |
+|----------|-----|----------|
+| **Range partitioning** | Partition by value range (date, ID range) | Time-series data, sequential access |
+| **Hash partitioning** | Hash key modulo partition count | Even distribution, point lookups |
+| **List partitioning** | Partition by discrete values (country, region) | Known categories, geographic data |
+| **Composite** | Combine strategies (hash + range) | Multi-tenant time-series |
+
+### Partition Key Selection
+- Choose keys with **high cardinality** (many distinct values)
+- Avoid **hot partitions** (one key getting disproportionate traffic)
+- Consider **query patterns** — partition by how data is most often read
+- Plan for **partition growth** — avoid partition count that requires redistribution
+
+### Sharding Considerations
+- **Shard key immutability** — Changing a shard key requires data migration
+- **Cross-shard queries** — Avoid joins across shards (denormalize instead)
+- **Rebalancing** — Use consistent hashing to minimize data movement
+- **Shard splitting** — Plan for splitting hot shards without downtime
+
+## Data Pipeline Tools
+
+### Orchestration
+- **Airflow** — DAG-based workflow orchestration (Python)
+- **Dagster** — Software-defined assets, strong typing
+- **Prefect** — Python-native, dynamic workflows
+- **Temporal** — Durable execution for long-running pipelines
+
+### Transformation
+- **dbt** — SQL-based transformations in the warehouse
+- **Spark** — Distributed processing for large datasets
+- **Pandas/Polars** — Single-machine data transformation
+- **Flink** — Stream and batch processing (JVM)
+
+### Storage
+- **Data Lake** — Raw, unstructured (S3, GCS, ADLS)
+- **Data Warehouse** — Structured, optimized for analytics (BigQuery, Snowflake, Redshift)
+- **Data Lakehouse** — Combines both (Delta Lake, Iceberg, Hudi)
+
+## Checklist
+
+When building a data pipeline:
+- [ ] Idempotent operations — safe to retry without side effects
+- [ ] Schema validation at ingestion boundary
+- [ ] Dead letter queue for failed records
+- [ ] Monitoring: processing lag, error rate, throughput
+- [ ] Backfill capability — can reprocess historical data
+- [ ] Incremental processing — not full reloads on every run
+- [ ] Data quality checks after transformation
+- [ ] Partition strategy aligned with query patterns
+- [ ] Exactly-once or at-least-once with idempotent consumers
+- [ ] Schema evolution plan (backward compatible changes)
+- [ ] Alerting on pipeline failures and data quality anomalies
+- [ ] Documentation of data lineage and transformation logic