@harness-engineering/cli 1.23.2 → 1.24.1

package/dist/agents/skills/codex/db-third-normal-form/SKILL.md

@@ -0,0 +1,210 @@
+# Third Normal Form (3NF)
+
+> "Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key." -- Codd's memorable definition of full normalization through 3NF.
+
+## When to Use
+
+- Removing redundant data that survives after 2NF decomposition
+- Fixing update anomalies caused by transitive dependencies
+- Deciding whether 3NF or BCNF is the right target for your schema
+- Extracting lookup tables from columns that depend on other non-key columns
+- Auditing OLTP schemas for over- or under-normalization
+
+## Instructions
+
+Third Normal Form builds on Second Normal Form. A table is in 3NF when it is in 2NF and no non-key column depends on another non-key column -- eliminating transitive dependencies.
+
+### Transitive Dependency
+
+A transitive dependency exists when: `A -> B -> C`, where A is the primary key, B is a non-key column, and C is another non-key column that depends on B rather than directly on A.
+
+### Worked Example 1: Employees with Department Data
+
+**BAD -- transitive dependency through department_id:**
+
+```sql
+CREATE TABLE employees (
+  id                SERIAL PRIMARY KEY,
+  name              TEXT NOT NULL,
+  department_id     INT NOT NULL,
+  department_name   TEXT NOT NULL,     -- depends on department_id, not employee id
+  department_budget NUMERIC(12,2)      -- depends on department_id, not employee id
+);
+```
+
+The dependency chain is: `employee_id -> department_id -> department_name, department_budget`. Changing a department's name requires updating every employee row in that department.
+
+**GOOD -- extract the transitively dependent columns:**
+
+```sql
+CREATE TABLE departments (
+  id     INT PRIMARY KEY,
+  name   TEXT NOT NULL,
+  budget NUMERIC(12,2) NOT NULL CHECK (budget >= 0)
+);
+
+CREATE TABLE employees (
+  id            SERIAL PRIMARY KEY,
+  name          TEXT NOT NULL,
+  department_id INT NOT NULL REFERENCES departments(id)
+);
+```
+
+Now department data lives in one place. Changing a department name is a single-row update.
+
+### Worked Example 2: Orders with Customer Data
+
+**BAD -- customer details embedded in orders:**
+
+```sql
+CREATE TABLE orders (
+  id             SERIAL PRIMARY KEY,
+  order_date     DATE NOT NULL,
+  customer_id    INT NOT NULL,
+  customer_name  TEXT NOT NULL,      -- depends on customer_id
+  customer_email TEXT NOT NULL,      -- depends on customer_id
+  total          NUMERIC(10,2)
+);
+```
+
+**GOOD -- customers extracted:**
+
+```sql
+CREATE TABLE customers (
+  id    INT PRIMARY KEY,
+  name  TEXT NOT NULL,
+  email TEXT NOT NULL UNIQUE
+);
+
+CREATE TABLE orders (
+  id          SERIAL PRIMARY KEY,
+  order_date  DATE NOT NULL,
+  customer_id INT NOT NULL REFERENCES customers(id),
+  total       NUMERIC(10,2) NOT NULL
+);
+```
+
+### When 3NF Is Sufficient
+
+For most OLTP applications, 3NF is the target. Higher normal forms (BCNF, 4NF, 5NF) solve edge cases that rarely appear in practice:
+
+- **BCNF** eliminates all redundancy from candidate key overlap. It differs from 3NF only when a table has multiple overlapping candidate keys -- uncommon in typical application schemas.
+- **4NF and 5NF** address multi-valued and join dependencies -- corner cases that arise in highly specialized data models.
+
+**The practical heuristic:** Normalize to 3NF during initial design. Only go to BCNF if you encounter a specific anomaly that 3NF does not resolve. Most developers never need beyond 3NF.
+
+### BCNF Comparison
+
+3NF allows some redundancy when a non-key attribute is part of a candidate key. BCNF eliminates it completely, but can make some queries impossible without additional joins.
+
+```sql
+-- 3NF allows this (student_id, course_id -> instructor, but instructor -> course_id)
+-- BCNF would require further decomposition
+CREATE TABLE teaching_assignments (
+  student_id    INT,
+  course_id     INT,
+  instructor_id INT,
+  PRIMARY KEY (student_id, course_id)
+);
+```
+
+In practice, the BCNF decomposition here creates two tables that cannot be rejoined without loss -- a rare case where 3NF is the better practical choice.
+
+### The Lookup Table Pattern
+
+The most common 3NF extraction is the lookup table for status codes, country codes, and category types:
+
+```sql
+-- Before: status as a string repeated in every row
+-- orders.status = 'pending', 'shipped', 'delivered', 'cancelled'
+
+-- After: lookup table with referential integrity
+CREATE TABLE order_statuses (
+  id   SERIAL PRIMARY KEY,
+  name TEXT NOT NULL UNIQUE
+);
+
+CREATE TABLE orders (
+  id        SERIAL PRIMARY KEY,
+  status_id INT NOT NULL REFERENCES order_statuses(id),
+  total     NUMERIC(10,2)
+);
+```
+
+### Anti-Patterns
+
+1. **Over-normalizing lookup data that never changes independently.** Boolean-like values (`is_active`) or binary states (`enabled/disabled`) do not need a lookup table. A `BOOLEAN` column or a `CHECK` constraint is simpler and equally correct.
+
+2. **Single-column lookup tables for trivial enums.** If the lookup table has only an `id` and a `name` column, and the values are a small fixed set, consider using PostgreSQL's `ENUM` type or a `CHECK` constraint instead:
+
+```sql
+-- Simpler than a lookup table for a fixed set
+CREATE TABLE orders (
+  id     SERIAL PRIMARY KEY,
+  status TEXT NOT NULL CHECK (status IN ('pending', 'shipped', 'delivered', 'cancelled'))
+);
+```
+
+3. **Ignoring transitive dependencies because "the JOIN is slow."** Fix the query plan, do not denormalize during initial design. Denormalization is a performance optimization applied after measurement, not a shortcut during schema design.
+
+### PostgreSQL Specifics
+
+PostgreSQL's `ENUM` type provides a middle ground between a CHECK constraint and a lookup table:
+
+```sql
+CREATE TYPE order_status AS ENUM ('pending', 'shipped', 'delivered', 'cancelled');
+
+CREATE TABLE orders (
+  id     SERIAL PRIMARY KEY,
+  status order_status NOT NULL DEFAULT 'pending'
+);
+```
+
+ENUMs are stored as integers internally (fast) but display as strings (readable). The tradeoff: adding values requires `ALTER TYPE ... ADD VALUE`, which cannot be rolled back in a transaction before PostgreSQL 12.
+
+## Details
+
+### OLTP vs OLAP Tradeoff
+
+3NF is ideal for OLTP (transactional) workloads where data integrity and write efficiency matter. For OLAP (analytical) workloads, denormalized star schemas or materialized views are typically more appropriate because they optimize for read-heavy aggregate queries rather than write correctness.
+
+### Identifying Transitive Dependencies
+
+For each non-key column, ask: "Does this column depend on the primary key directly, or does it depend on another non-key column that in turn depends on the key?"
+
+Systematic approach:
+
+1. List all non-key columns.
+2. For each pair of non-key columns (A, B), ask: "If I know A, can I determine B without knowing the primary key?"
+3. If yes, B transitively depends on the key through A. Extract A and B into a new table with A as the key.
+
+### MySQL Callout
+
+MySQL does not support PostgreSQL-style `ENUM` modifications as easily. Adding values to a MySQL `ENUM` requires an `ALTER TABLE` that may lock the table for the duration of the change on large tables. For evolving value sets, a lookup table is safer in MySQL than an ENUM.
+
+### Real-World Case Study: Multi-Tenant SaaS Platform
+
+A project management SaaS stored `organization_name`, `organization_plan`, and `organization_max_users` on every `project` row. With 500 organizations averaging 200 projects each, changing an organization's plan required updating 200 rows. After extracting `organizations` as a separate table, plan upgrades became instant single-row updates, and a billing audit query that previously scanned 100K project rows to count plans by type could instead scan 500 organization rows.
+
+## Source
+
+- [PostgreSQL Constraints](https://www.postgresql.org/docs/current/ddl-constraints.html)
+- Codd, E.F. "Further Normalization of the Data Base Relational Model" (1971)
+- Kent, W. "A Simple Guide to Five Normal Forms in Relational Database Theory" (1983)
+
+## Process
+
+1. Read the transitive dependency rules and examples in this document.
+2. For each table in 2NF, check whether any non-key column depends on another non-key column rather than directly on the primary key.
+3. Extract transitively dependent column groups into their own tables with appropriate foreign key references.
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** db-first-normal-form, db-second-normal-form, db-denormalization
+
+## Success Criteria
+
+- All transitive dependencies in the schema have been identified and eliminated by extracting them into separate tables.
+- The lookup table pattern is applied where appropriate, without over-normalizing trivial values.

package/dist/agents/skills/codex/db-third-normal-form/skill.yaml

@@ -0,0 +1,38 @@
+name: db-third-normal-form
+version: '1.0.0'
+description: Eliminating transitive dependencies and knowing when 3NF is sufficient for Third Normal Form (3NF)
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - db-first-normal-form
+  - db-second-normal-form
+  - db-denormalization
+stack_signals:
+  - postgresql
+  - mysql
+  - database
+keywords:
+  - normalization
+  - third-normal-form
+  - 3NF
+  - transitive-dependency
+  - boyce-codd
+  - BCNF
+metadata:
+  author: community
+  upstream: postgresql.org/docs/current/ddl-constraints.html
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/codex/db-time-series/SKILL.md

@@ -0,0 +1,196 @@
+# Time-Series Data
+
+> Designing append-heavy tables for metrics, events, and logs with time-based partitioning, retention policies, and efficient aggregation.
+
+## When to Use
+
+- IoT sensor data (temperature, pressure, GPS coordinates)
+- Application metrics and observability data (request latency, error rates)
+- Financial tick data and market prices
+- Server logs, access logs, audit events
+- Any workload that is append-heavy with time-ordered queries and eventual data expiry
+
+## Instructions
+
+### Key Concepts
+
+Time-series workloads share four characteristics:
+
+1. **Append-only or append-mostly** -- rows are inserted, rarely updated or deleted
+2. **Time-range queries** -- WHERE clauses filter by time window
+3. **Recency bias** -- recent data is queried far more than old data
+4. **Retention period** -- data expires after days, months, or years
+
+**Schema design with partitioning:**
+
+```sql
+CREATE TABLE metrics (
+  time        timestamptz NOT NULL,
+  device_id   int NOT NULL,
+  temperature float,
+  humidity    float
+) PARTITION BY RANGE (time);
+
+-- Create monthly partitions
+CREATE TABLE metrics_2024_01 PARTITION OF metrics
+  FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');
+CREATE TABLE metrics_2024_02 PARTITION OF metrics
+  FOR VALUES FROM ('2024-02-01') TO ('2024-03-01');
+```
+
+**Key design choices:**
+
+- **Partition interval:** Match query granularity. Daily partitions for dashboards, monthly for reports.
+- **Chunk size:** Target each partition at 25% of available memory for cache efficiency.
+- **Index strategy:** BRIN index on time (much smaller than B-tree for sequential data). B-tree composite on `(device_id, time)` for per-device queries.
+- **Retention:** `DROP TABLE metrics_2024_01;` is instant. `DELETE WHERE time < X` creates dead tuples requiring vacuum -- avoid this.
+
+### Worked Example
+
+IoT sensor monitoring platform:
+
+```sql
+-- BRIN index on time (100x smaller than B-tree for sequential inserts)
+CREATE INDEX idx_metrics_time_brin ON metrics USING brin (time);
+
+-- Composite B-tree for per-device queries
+CREATE INDEX idx_metrics_device_time ON metrics (device_id, time);
+```
+
+**Time-range query with partition pruning:**
+
+```sql
+EXPLAIN ANALYZE
+SELECT device_id, avg(temperature), max(humidity)
+FROM metrics
+WHERE time BETWEEN '2024-03-01' AND '2024-03-31'
+GROUP BY device_id;
+```
+
+The query plan shows `Partitions removed: 10` -- only the March partition is scanned. Without partitioning, the entire table would be read.
+
+**Retention by dropping old partitions:**
+
+```sql
+-- Instant operation, no dead tuples, no vacuum needed
+DROP TABLE metrics_2023_01;
+```
+
+Compare with DELETE-based retention: `DELETE FROM metrics WHERE time < '2023-02-01'` on a 500M-row table generates millions of dead tuples, triggers autovacuum storms, and takes hours.
+
+**Continuous aggregation for dashboards:**
+
+```sql
+CREATE MATERIALIZED VIEW hourly_metrics AS
+SELECT
+  date_trunc('hour', time) AS hour,
+  device_id,
+  avg(temperature) AS avg_temp,
+  max(temperature) AS max_temp,
+  count(*) AS readings
+FROM metrics
+GROUP BY 1, 2;
+
+CREATE UNIQUE INDEX ON hourly_metrics (hour, device_id);
+
+-- Refresh periodically
+REFRESH MATERIALIZED VIEW CONCURRENTLY hourly_metrics;
+```
+
+### Anti-Patterns
+
+1. **Single unpartitioned table for time-series.** Leads to bloat, vacuum pressure, and slow range queries as the table grows past 100M rows.
+2. **DELETE for retention instead of DROP PARTITION.** Creates dead tuples, triggers vacuum overhead, and can take hours on large tables.
+3. **B-tree index on time column for sequential append data.** BRIN indexes are 100x smaller and provide comparable query performance for time-range scans on physically ordered data.
+4. **Not aligning partition boundaries with query patterns.** Monthly partitions with daily queries waste I/O. Daily partitions with yearly queries create thousands of partitions.
+5. **Updating time-series rows.** Breaks the append-only assumption, causes HOT update failures on BRIN-indexed tables, and degrades partition pruning effectiveness.
+
+### PostgreSQL Specifics
+
+**Declarative partitioning** (`PARTITION BY RANGE`) is the foundation. PostgreSQL 11+ supports partition pruning at plan time and execution time.
+
+**BRIN indexes** are ideal for time columns because data is physically ordered by insertion time:
+
+```sql
+CREATE INDEX ON metrics USING brin (time) WITH (pages_per_range = 32);
+```
+
+The `pages_per_range` parameter controls granularity -- smaller values give tighter range summaries at the cost of a larger index.
+
+**pg_partman extension** automates partition creation and retention:
+
+```sql
+SELECT partman.create_parent(
+  'public.metrics', 'time', 'native', 'monthly'
+);
+-- Automatically creates future partitions and drops old ones
+```
+
+**Materialized views** with `REFRESH CONCURRENTLY` for zero-downtime dashboard aggregation. Requires a unique index on the materialized view.
+
+## Details
+
+### Advanced Topics
+
+**TimescaleDB hypertables** automatically chunk data by time, eliminating manual partition management:
+
+```sql
+SELECT create_hypertable('metrics', 'time');
+```
+
+TimescaleDB adds compression (columnar, 90%+ size reduction on old chunks), continuous aggregates (real-time aggregation combining materialized and recent data), and `time_bucket()` for flexible time grouping.
+
+**Two-step aggregation:** Pre-aggregate in a materialized view (hourly), then re-aggregate at query time (daily, weekly). This reduces the data scanned for dashboard queries by 100-1000x.
+
+**Write-ahead log tuning** for high-throughput ingestion:
+
+- `synchronous_commit = off` for metrics where losing a few seconds of data on crash is acceptable
+- `wal_level = minimal` if replication is not needed
+- Batch inserts with `COPY` instead of individual INSERTs (10-50x throughput improvement)
+
+### Engine Differences
+
+MySQL 8.0 supports `PARTITION BY RANGE` but with key differences:
+
+- MySQL lacks BRIN indexes -- use B-tree on time column (larger but functional)
+- MySQL partitioning has a limit of 8192 partitions per table
+- MySQL lacks materialized views entirely -- use summary tables with scheduled events (`CREATE EVENT`) or application-level refresh
+- MySQL does not have a TimescaleDB equivalent -- consider ClickHouse or InfluxDB for high-volume time-series workloads on MySQL stacks
+
+For MySQL, the partition management pattern:
+
+```sql
+ALTER TABLE metrics DROP PARTITION p_2023_01;  -- instant retention
+ALTER TABLE metrics ADD PARTITION (
+  PARTITION p_2024_04 VALUES LESS THAN ('2024-05-01')
+);
+```
+
+### Real-World Case Studies
+
+**Fleet management platform ingesting 500K sensor readings/minute from 100K vehicles.** Partitioned by day (30 partitions retained), BRIN index on time, composite B-tree on `(vehicle_id, time)`. Write throughput: 500K rows/minute sustained using batched COPY. Query latency: "last 24h for vehicle X" in 4ms. Retention: daily cron drops partitions older than 30 days -- instant, no vacuum. Storage: 2TB raw data per month, reduced to 200GB with TimescaleDB compression on chunks older than 7 days.
+
+## Source
+
+- [TimescaleDB Documentation](https://docs.timescale.com/use-timescale/latest/hypertables/)
+- [PostgreSQL Partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)
+- [PostgreSQL BRIN Indexes](https://www.postgresql.org/docs/current/brin-intro.html)
+
+## Process
+
+1. Read the key concepts to understand time-series characteristics and partitioning strategy.
+2. Apply time-based partitioning with BRIN indexes and partition-drop retention.
+3. Verify with EXPLAIN ANALYZE that partition pruning is active and that BRIN index scans are used for time-range queries.
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** db-temporal-data, db-audit-trail, db-btree-index, db-composite-index
+
+## Success Criteria
+
+- Time-series tables use time-based partitioning with BRIN indexes.
+- Retention is implemented via partition drop, not DELETE.
+- Aggregation uses materialized views or continuous aggregates for dashboard queries.
+- EXPLAIN ANALYZE confirms partition pruning and BRIN index usage.

package/dist/agents/skills/codex/db-time-series/skill.yaml

@@ -0,0 +1,41 @@
+name: db-time-series
+version: "1.0.0"
+description: Append-only tables, time-based partitioning, retention policies, and TimescaleDB patterns
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - db-temporal-data
+  - db-audit-trail
+  - db-btree-index
+  - db-composite-index
+stack_signals:
+  - postgresql
+  - mysql
+  - database
+keywords:
+  - time-series
+  - append-only
+  - partitioning
+  - retention
+  - TimescaleDB
+  - hypertable
+  - downsampling
+  - continuous-aggregate
+metadata:
+  author: community
+  upstream: "docs.timescale.com/use-timescale/latest/hypertables/"
+state:
+  persistent: false
+  files: []
+depends_on: []