thanh-kit 2.5.0 → 2.5.2

package/templates/skills/databases/stacks/bigquery.md

@@ -0,0 +1,231 @@
+# BigQuery Rules
+
+Guidelines for designing schemas specific to Google BigQuery.
+
+---
+
+## Data Types
+
+| Use case | Type | Notes |
+|----------|------|-------|
+| Integer | `INT64` | |
+| Decimal | `NUMERIC` or `BIGNUMERIC` | |
+| Float | `FLOAT64` | |
+| Boolean | `BOOL` | |
+| String | `STRING` | |
+| Timestamp | `TIMESTAMP` | |
+| Date | `DATE` | |
+| Time | `TIME` | |
+| DateTime | `DATETIME` | No timezone |
+| JSON | `JSON` | |
+| Bytes | `BYTES` | |
+| Nested | `STRUCT` | Record type |
+| Repeated | `ARRAY` | |
+
+---
+
+## Description (NO metadata tables needed)
+
+BigQuery supports description via `OPTIONS`:
+
+### Table description
+```sql
+CREATE TABLE orders (
+    ...
+) OPTIONS(description='Table storing customer orders');
+```
+
+### Column descriptions
+```sql
+CREATE TABLE orders (
+    id INT64 NOT NULL OPTIONS(description='Primary key'),
+    order_number STRING NOT NULL OPTIONS(description='Unique order code'),
+    status STRING NOT NULL OPTIONS(description='pending|confirmed|shipped|cancelled'),
+    total_amount NUMERIC OPTIONS(description='Total value')
+);
+```
+
+### Query descriptions
+```sql
+SELECT column_name, description
+FROM `project.dataset.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`
+WHERE table_name = 'orders';
+```
+
+---
+
+## Partitioning & Clustering
+
+BigQuery **does not have traditional indexes**. Instead, use partitioning and clustering:
+
+### Partitioning (required for large tables)
+
+```sql
+-- Partition by date column
+CREATE TABLE fact_orders (
+    order_date DATE,
+    customer_id INT64,
+    order_id INT64,
+    amount NUMERIC
+)
+PARTITION BY order_date;
+
+-- Partition by timestamp (daily)
+CREATE TABLE events (
+    event_time TIMESTAMP,
+    ...
+)
+PARTITION BY DATE(event_time);
+
+-- Integer range partitioning
+CREATE TABLE logs (
+    log_id INT64,
+    ...
+)
+PARTITION BY RANGE_BUCKET(log_id, GENERATE_ARRAY(0, 1000000, 10000));
+```
+
+### Clustering (optimize filter/group)
+
+```sql
+CREATE TABLE fact_orders (
+    order_date DATE,
+    customer_id INT64,
+    product_id INT64,
+    store_id INT64,
+    amount NUMERIC
+)
+PARTITION BY order_date
+CLUSTER BY customer_id, product_id;
+-- Can cluster up to 4 columns
+```
+
+### When to use what?
+- **Partition**: Filter by date range → reduces data scan
+- **Cluster**: Filter/Group by specific columns → data is organized closer together
+
+---
+
+## Nested & Repeated Fields (Denormalization)
+
+BigQuery encourages denormalization with STRUCT and ARRAY:
+
+```sql
+CREATE TABLE orders (
+    order_id INT64,
+    order_date DATE,
+    customer STRUCT<
+        id INT64,
+        name STRING,
+        email STRING
+    >,
+    items ARRAY<STRUCT<
+        product_id INT64,
+        product_name STRING,
+        quantity INT64,
+        unit_price NUMERIC
+    >>
+);
+
+-- Query nested fields
+SELECT
+    order_id,
+    customer.name,
+    item.product_name,
+    item.quantity
+FROM orders, UNNEST(items) AS item;
+```
+
+---
+
+## Best Practices
+
+### Query optimization
+- **Avoid `SELECT *`** - only select needed columns
+- **Filter early** - use partition column in WHERE
+- **Avoid cross-join** with large tables
+
+### Table design
+- Partition tables > 1GB
+- Cluster by frequently filtered/grouped columns
+- Denormalize with STRUCT/ARRAY when appropriate
+- Avoid too many small tables
+
+### Cost control
+- Partition pruning: always filter by partition column
+- Use `--dry_run` to estimate cost
+- Set up query quotas
+
+---
+
+## Example DDL
+
+```sql
+CREATE TABLE `project.dataset.fact_orders` (
+    -- Keys
+    order_id INT64 NOT NULL OPTIONS(description='PK from source'),
+    order_date DATE NOT NULL OPTIONS(description='Order date'),
+
+    -- Dimensions (denormalized)
+    customer_id INT64 NOT NULL,
+    customer_name STRING,
+    customer_segment STRING OPTIONS(description='VIP|Regular|New'),
+
+    -- Measures
+    item_count INT64 NOT NULL,
+    gross_amount NUMERIC NOT NULL,
+    discount_amount NUMERIC DEFAULT 0,
+    net_amount NUMERIC NOT NULL,
+
+    -- Nested items
+    items ARRAY<STRUCT<
+        product_id INT64,
+        product_name STRING,
+        quantity INT64,
+        unit_price NUMERIC
+    >> OPTIONS(description='Product details in order'),
+
+    -- Metadata
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP(),
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP()
+)
+PARTITION BY order_date
+CLUSTER BY customer_id
+OPTIONS(
+    description='Fact table for orders, partitioned by date, clustered by customer'
+);
+```
+
+---
+
+## Scheduled Queries / Materialized Views
+
+### Materialized View
+```sql
+CREATE MATERIALIZED VIEW `project.dataset.mv_daily_sales`
+PARTITION BY report_date
+CLUSTER BY store_id
+AS
+SELECT
+    DATE(order_date) as report_date,
+    store_id,
+    COUNT(*) as order_count,
+    SUM(net_amount) as revenue
+FROM `project.dataset.fact_orders`
+GROUP BY 1, 2;
+```
+
+### Scheduled Query (ETL)
+Create via BigQuery Console or API with schedule expression.
+
+---
+
+## Checklist
+
+- [ ] Partition large tables by date column
+- [ ] Cluster by frequently filtered/grouped columns (max 4)
+- [ ] `OPTIONS(description=...)` for tables and columns
+- [ ] Denormalize with STRUCT/ARRAY when appropriate
+- [ ] Avoid `SELECT *` and cross-join large tables
+- [ ] Filter by partition column in queries
+- [ ] Consider Materialized Views for frequently used aggregations

package/templates/skills/databases/stacks/d1_cloudflare.md

@@ -0,0 +1,137 @@
+# D1 Cloudflare Rules
+
+D1 is a SQLite-based database running on Cloudflare edge. Inherits from [sqlite.md](sqlite.md) with its own specifics.
+
+---
+
+## Important Limitations
+
+| Limit | Value |
+|-------|-------|
+| Max database size | **10 GB** |
+| Concurrent queries | Single-threaded (1 query/time) |
+| Batch operations | No more than 100k rows/query |
+| Max tables | No hard limit, but recommend < 100 |
+
+**Throughput**: ~1000 queries/s if avg query = 1ms. Query 100ms = only 10 queries/s.
+
+---
+
+## Data Types
+
+Same as SQLite:
+
+| Storage Class | Use case |
+|---------------|----------|
+| `INTEGER` | PK, FK, counts, booleans |
+| `REAL` | Floats, decimals |
+| `TEXT` | Strings, dates, JSON |
+| `BLOB` | Binary |
+
+**Date/Time**: Store as `TEXT` with ISO format: `datetime('now')`
+
+---
+
+## Metadata Tables (REQUIRED)
+
+D1 has no native comments. Need to create metadata tables:
+
+```sql
+CREATE TABLE metadata_tables (
+    table_name TEXT PRIMARY KEY,
+    description TEXT NOT NULL,
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE TABLE metadata_columns (
+    table_name TEXT NOT NULL,
+    column_name TEXT NOT NULL,
+    description TEXT NOT NULL,
+    data_type TEXT,
+    PRIMARY KEY (table_name, column_name),
+    FOREIGN KEY (table_name) REFERENCES metadata_tables(table_name) ON DELETE CASCADE
+);
+
+-- After CREATE TABLE
+INSERT INTO metadata_tables VALUES ('orders', 'Orders table', datetime('now'));
+INSERT INTO metadata_columns VALUES
+    ('orders', 'id', 'PK', 'INTEGER'),
+    ('orders', 'status', 'pending|confirmed|shipped', 'TEXT');
+```
+
+---
+
+## Foreign Keys
+
+```sql
+-- Enable FK (ON by default in D1, but should be explicit)
+PRAGMA foreign_keys = ON;
+
+CREATE TABLE order_items (
+    id INTEGER PRIMARY KEY,
+    order_id INTEGER NOT NULL,
+    FOREIGN KEY (order_id) REFERENCES orders(id) ON DELETE CASCADE
+);
+CREATE INDEX idx_order_items_order ON order_items(order_id);
+```
+
+---
+
+## Best Practices
+
+### Performance
+- **Use indexes** - since single-threaded, queries must be fast
+- **Batch writes** to reduce latency
+- **Avoid large transactions** (>100k rows)
+- **Pagination** with LIMIT/OFFSET or cursor-based
+
+### Monitoring
+- Track query duration
+- Monitor database size
+- Alert when approaching 10GB limit
+
+---
+
+## Example DDL
+
+```sql
+-- Enable FK
+PRAGMA foreign_keys = ON;
+
+CREATE TABLE orders (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    order_number TEXT NOT NULL UNIQUE,
+    user_id INTEGER NOT NULL,
+    status TEXT DEFAULT 'pending' CHECK (status IN ('pending', 'confirmed', 'shipped', 'cancelled')),
+    total_amount REAL DEFAULT 0,
+    created_at TEXT DEFAULT (datetime('now')),
+    updated_at TEXT DEFAULT (datetime('now')),
+    FOREIGN KEY (user_id) REFERENCES users(id)
+);
+
+-- Indexes
+CREATE INDEX idx_orders_user ON orders(user_id);
+CREATE INDEX idx_orders_status ON orders(status);
+CREATE INDEX idx_orders_updated ON orders(updated_at);
+
+-- Metadata
+INSERT INTO metadata_tables (table_name, description)
+VALUES ('orders', 'Orders table');
+
+INSERT INTO metadata_columns (table_name, column_name, description, data_type)
+VALUES
+    ('orders', 'id', 'PK', 'INTEGER'),
+    ('orders', 'order_number', 'Unique order code', 'TEXT'),
+    ('orders', 'status', 'pending|confirmed|shipped|cancelled', 'TEXT');
+```
+
+---
+
+## Checklist
+
+- [ ] Database size < 10GB
+- [ ] Metadata tables for documentation
+- [ ] Index for every query (single-threaded!)
+- [ ] FK has index
+- [ ] Batch operations < 100k rows
+- [ ] Monitor size and query performance

package/templates/skills/databases/stacks/mysql.md

@@ -0,0 +1,216 @@
+# MySQL Rules
+
+Guidelines for designing schemas specific to MySQL / MariaDB.
+
+---
+
+## Data Types
+
+| Use case | Type | Notes |
+|----------|------|-------|
+| Primary key | `BIGINT UNSIGNED AUTO_INCREMENT` | For large tables |
+| Primary key (small) | `INT UNSIGNED AUTO_INCREMENT` | Tables < 2 billion rows |
+| Foreign key | Match with PK type | |
+| Money/Price | `DECIMAL(18,2)` | **Do NOT use FLOAT/DOUBLE** |
+| Quantity | `INT` or `DECIMAL(18,4)` | |
+| Boolean | `TINYINT(1)` or `BOOLEAN` | |
+| Short text | `VARCHAR(n)` | n ≤ 255 for indexing |
+| Long text | `TEXT` | Cannot index directly |
+| JSON | `JSON` | MySQL 5.7+ |
+| Timestamp | `DATETIME` or `TIMESTAMP` | |
+| Date | `DATE` | |
+| UUID | `CHAR(36)` or `BINARY(16)` | |
+
+### TIMESTAMP vs DATETIME
+- `TIMESTAMP`: Auto-converts timezone, range 1970-2038
+- `DATETIME`: No timezone conversion, range 1000-9999
+
+---
+
+## Comments (NO metadata tables needed)
+
+MySQL supports comments directly in DDL:
+
+### Table comment
+```sql
+CREATE TABLE orders (
+    ...
+) COMMENT='Table storing customer orders';
+```
+
+### Column comment
+```sql
+CREATE TABLE orders (
+    id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT
+        COMMENT 'Primary key, auto increment',
+    order_number VARCHAR(50) NOT NULL
+        COMMENT 'Unique order code, format: ORD-YYYYMMDD-XXXXX',
+    status VARCHAR(32) NOT NULL DEFAULT 'pending'
+        COMMENT 'Status: pending|confirmed|shipped|cancelled',
+    total_amount DECIMAL(18,2) NOT NULL
+        COMMENT 'Total value including tax'
+);
+```
+
+### Query comments
+```sql
+-- Table comment
+SELECT TABLE_COMMENT FROM INFORMATION_SCHEMA.TABLES
+WHERE TABLE_SCHEMA = 'your_db' AND TABLE_NAME = 'orders';
+
+-- Column comments
+SELECT COLUMN_NAME, COLUMN_COMMENT FROM INFORMATION_SCHEMA.COLUMNS
+WHERE TABLE_SCHEMA = 'your_db' AND TABLE_NAME = 'orders';
+```
+
+---
+
+## Index Types
+
+### B-Tree (default)
+```sql
+CREATE INDEX idx_orders_status ON orders(status);
+CREATE INDEX idx_orders_user_status_created ON orders(user_id, status, created_at DESC);
+CREATE UNIQUE INDEX idx_orders_number ON orders(order_number);
+```
+
+### FULLTEXT (text search)
+```sql
+CREATE FULLTEXT INDEX idx_products_search ON products(name, description);
+
+-- Query
+SELECT * FROM products
+WHERE MATCH(name, description) AGAINST('gaming laptop' IN NATURAL LANGUAGE MODE);
+```
+
+### Notes
+- **Partial index**: Not natively supported in MySQL
+- Index prefix for TEXT: `CREATE INDEX idx ON table(col(255))`
+
+---
+
+## Auto Increment
+
+```sql
+CREATE TABLE users (
+    id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT
+);
+
+-- Set starting value
+ALTER TABLE users AUTO_INCREMENT = 1000;
+
+-- Get last inserted ID
+SELECT LAST_INSERT_ID();
+```
+
+---
+
+## Auto update timestamp
+
+```sql
+created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
+```
+
+---
+
+## ENUM vs VARCHAR
+
+### Use ENUM when values are fixed and rarely change
+```sql
+status ENUM('pending', 'confirmed', 'shipped', 'cancelled') NOT NULL DEFAULT 'pending'
+```
+
+### Use VARCHAR + CHECK when flexibility needed
+```sql
+status VARCHAR(32) NOT NULL DEFAULT 'pending',
+CONSTRAINT chk_orders_status CHECK (status IN ('pending', 'confirmed', 'shipped'))
+```
+
+---
+
+## Foreign Keys
+
+```sql
+CREATE TABLE order_items (
+    id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
+    order_id BIGINT UNSIGNED NOT NULL,
+    product_id BIGINT UNSIGNED NOT NULL,
+
+    CONSTRAINT fk_orderitems_order
+        FOREIGN KEY (order_id) REFERENCES orders(id)
+        ON DELETE CASCADE ON UPDATE CASCADE,
+
+    CONSTRAINT fk_orderitems_product
+        FOREIGN KEY (product_id) REFERENCES products(id)
+        ON DELETE RESTRICT
+);
+
+-- IMPORTANT: Index for FK
+CREATE INDEX idx_orderitems_order ON order_items(order_id);
+CREATE INDEX idx_orderitems_product ON order_items(product_id);
+```
+
+**Note:** FK only works with **InnoDB** engine.
+
+---
+
+## Partitioning
+
+### Range partitioning (by date)
+```sql
+CREATE TABLE fact_orders (
+    id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
+    order_date DATE NOT NULL,
+    ...
+    PRIMARY KEY (id, order_date)
+) PARTITION BY RANGE (YEAR(order_date)) (
+    PARTITION p2023 VALUES LESS THAN (2024),
+    PARTITION p2024 VALUES LESS THAN (2025),
+    PARTITION pmax VALUES LESS THAN MAXVALUE
+);
+```
+
+---
+
+## Example DDL
+
+```sql
+CREATE TABLE orders (
+    id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT
+        COMMENT 'Primary key',
+    order_number VARCHAR(50) NOT NULL
+        COMMENT 'Unique order code',
+    user_id BIGINT UNSIGNED NOT NULL
+        COMMENT 'FK to users.id',
+    status ENUM('pending', 'confirmed', 'shipped', 'cancelled', 'completed')
+        NOT NULL DEFAULT 'pending'
+        COMMENT 'Order status',
+    subtotal DECIMAL(18,2) NOT NULL DEFAULT 0,
+    discount_amount DECIMAL(18,2) NOT NULL DEFAULT 0,
+    total_amount DECIMAL(18,2) NOT NULL DEFAULT 0,
+    created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+
+    UNIQUE KEY uk_orders_number (order_number),
+    KEY idx_orders_user (user_id),
+    KEY idx_orders_status (status),
+    KEY idx_orders_user_status_created (user_id, status, created_at DESC),
+
+    CONSTRAINT fk_orders_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE RESTRICT
+) ENGINE=InnoDB
+  DEFAULT CHARSET=utf8mb4
+  COLLATE=utf8mb4_unicode_ci
+  COMMENT='Table storing order information';
+```
+
+---
+
+## Checklist
+
+- [ ] Engine: `InnoDB` (required for FK)
+- [ ] Charset: `utf8mb4` + `utf8mb4_unicode_ci`
+- [ ] Comments for tables and columns
+- [ ] All FKs have indexes
+- [ ] Use `DECIMAL` for money, not `FLOAT`
+- [ ] `ON UPDATE CURRENT_TIMESTAMP` for `updated_at`