opencode-skills-antigravity 1.0.11 → 1.0.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/aegisops-ai/SKILL.md +127 -0
- package/bundled-skills/docs/integrations/jetski-cortex.md +3 -3
- package/bundled-skills/docs/integrations/jetski-gemini-loader/README.md +1 -1
- package/bundled-skills/docs/maintainers/repo-growth-seo.md +3 -3
- package/bundled-skills/docs/maintainers/skills-update-guide.md +1 -1
- package/bundled-skills/docs/users/bundles.md +1 -1
- package/bundled-skills/docs/users/claude-code-skills.md +1 -1
- package/bundled-skills/docs/users/gemini-cli-skills.md +1 -1
- package/bundled-skills/docs/users/getting-started.md +1 -1
- package/bundled-skills/docs/users/kiro-integration.md +1 -1
- package/bundled-skills/docs/users/usage.md +4 -4
- package/bundled-skills/docs/users/visual-guide.md +4 -4
- package/bundled-skills/snowflake-development/SKILL.md +228 -0
- package/bundled-skills/wordpress/SKILL.md +281 -4
- package/bundled-skills/wordpress-penetration-testing/SKILL.md +106 -1
- package/bundled-skills/wordpress-plugin-development/SKILL.md +296 -3
- package/bundled-skills/wordpress-theme-development/SKILL.md +316 -3
- package/bundled-skills/wordpress-woocommerce-development/SKILL.md +442 -2
- package/bundled-skills/xvary-stock-research/LICENSE +21 -0
- package/bundled-skills/xvary-stock-research/SKILL.md +103 -0
- package/bundled-skills/xvary-stock-research/assets/nvda-deep-dive-hero.png +0 -0
- package/bundled-skills/xvary-stock-research/assets/nvda-deep-dive-scenarios.png +0 -0
- package/bundled-skills/xvary-stock-research/assets/nvda-deep-dive-thesis.png +0 -0
- package/bundled-skills/xvary-stock-research/assets/social-preview.png +0 -0
- package/bundled-skills/xvary-stock-research/examples/nvda-analysis.md +60 -0
- package/bundled-skills/xvary-stock-research/references/edgar-guide.md +53 -0
- package/bundled-skills/xvary-stock-research/references/methodology.md +153 -0
- package/bundled-skills/xvary-stock-research/references/scoring.md +111 -0
- package/bundled-skills/xvary-stock-research/tests/test_edgar.py +90 -0
- package/bundled-skills/xvary-stock-research/tests/test_market.py +113 -0
- package/bundled-skills/xvary-stock-research/tools/edgar.py +495 -0
- package/bundled-skills/xvary-stock-research/tools/market.py +302 -0
- package/package.json +1 -1
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: aegisops-ai
|
|
3
|
+
description: "Autonomous DevSecOps & FinOps Guardrails.
|
|
4
|
+
Orchestrates Gemini 3 Flash to audit Linux Kernel patches,
|
|
5
|
+
Terraform cost drifts, and K8s compliance."
|
|
6
|
+
risk: safe
|
|
7
|
+
source: community
|
|
8
|
+
author: Champbreed
|
|
9
|
+
date_added: "2026-03-24"
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# /aegisops-ai — Autonomous Governance Orchestrator
|
|
13
|
+
|
|
14
|
+
AegisOps-AI is a professional-grade "Living Pipeline"
|
|
15
|
+
that integrates advanced AI reasoning directly into
|
|
16
|
+
the SDLC. It acts as an intelligent gatekeeper for
|
|
17
|
+
systems-level security, cloud infrastructure costs,
|
|
18
|
+
and Kubernetes compliance.
|
|
19
|
+
|
|
20
|
+
## Goal
|
|
21
|
+
|
|
22
|
+
To automate high-stakes security and financial audits by:
|
|
23
|
+
1. Identifying logic-based vulnerabilities (UAF, Stale
|
|
24
|
+
State) in Linux Kernel patches.
|
|
25
|
+
2. Detecting massive "Silent Disaster" cost drifts in
|
|
26
|
+
Terraform plans.
|
|
27
|
+
3. Translating natural language security intent into
|
|
28
|
+
hardened K8s manifests.
|
|
29
|
+
|
|
30
|
+
## When to Use
|
|
31
|
+
|
|
32
|
+
- **Kernel Patch Review:** Auditing raw C-based Git diffs for memory safety.
|
|
33
|
+
- **Pre-Apply IaC Audit:** Analyzing `terraform plan` outputs to prevent bill spikes.
|
|
34
|
+
- **Cluster Hardening:** Generating "Least Privilege" securityContexts for deployments.
|
|
35
|
+
- **CI/CD Quality Gating:** Blocking non-compliant merges via GitHub Actions.
|
|
36
|
+
|
|
37
|
+
## When Not to Use
|
|
38
|
+
|
|
39
|
+
- **Web App Logic:** Do not use for standard web vulnerabilities (XSS, SQLi); use dedicated SAST scanners.
|
|
40
|
+
- **Non-C Memory Analysis:** The patch analyzer is optimized for C-logic; avoid using it for high-level languages like Python or JS.
|
|
41
|
+
- **Direct Resource Mutation:** This is an *auditor*, not a deployment tool. It does not execute `terraform apply` or `kubectl apply`.
|
|
42
|
+
- **Post-Mortem Analysis:** For analyzing *why* a previous AI session failed, use `/analyze-project` instead.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
## 🤖 Generative AI Integration
|
|
46
|
+
|
|
47
|
+
AegisOps-AI leverages the **Google GenAI SDK** to implement a "Reasoning Path" for autonomous security and financial audits:
|
|
48
|
+
|
|
49
|
+
* **Neural Patch Analysis:** Performs semantic code reviews of Linux Kernel patches, moving beyond simple pattern matching to understand complex memory state logic.
|
|
50
|
+
* **Intelligent Cost Synthesis:** Processes raw Terraform plan diffs through a financial reasoning model to detect high-risk resource escalations and "silent" fiscal drifts.
|
|
51
|
+
* **Natural Language Policy Mapping:** Translates human security intent into syntactically correct, hardened Kubernetes `securityContext` configurations.
|
|
52
|
+
|
|
53
|
+
## 🧭 Core Modules
|
|
54
|
+
|
|
55
|
+
### 1. 🐧 Kernel Patch Reviewer (`patch_analyzer.py`)
|
|
56
|
+
|
|
57
|
+
* **Problem:** Manual review of Linux Kernel memory safety is time-consuming and prone to human error.
|
|
58
|
+
* **Solution:** Gemini 3 performs a "Deep Reasoning" audit on raw Git diffs to detect critical memory corruption vulnerabilities (UAF, Stale State) in seconds.
|
|
59
|
+
* **Key Output:** `analysis_results.json`
|
|
60
|
+
|
|
61
|
+
### 2. 💰 FinOps & Cloud Auditor (`cost_auditor.py`)
|
|
62
|
+
|
|
63
|
+
* **Problem:** Infrastructure-as-Code (IaC) changes can lead to accidental "Silent Disasters" and massive cloud bill spikes.
|
|
64
|
+
* **Solution:** Analyzes `terraform plan` output to identify cost anomalies—such as accidental upgrades from `t3.micro` to high-performance GPU instances.
|
|
65
|
+
* **Key Output:** `infrastructure_audit_report.json`
|
|
66
|
+
|
|
67
|
+
### 3. ☸️ K8s Policy Hardener (`k8s_policy_generator.py`)
|
|
68
|
+
|
|
69
|
+
* **Problem:** Implementing "Least Privilege" security contexts in Kubernetes is complex and often neglected.
|
|
70
|
+
* **Solution:** Translates natural language security requirements into production-ready, hardened YAML manifests (Read-only root FS, Non-root enforcement, etc.).
|
|
71
|
+
* **Key Output:** `hardened_deployment.yaml`
|
|
72
|
+
|
|
73
|
+
## 🛠️ Setup & Environment
|
|
74
|
+
|
|
75
|
+
### 1. Clone the Repository
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
git clone https://github.com/Champbreed/AegisOps-AI.git
|
|
79
|
+
cd AegisOps-AI
|
|
80
|
+
```
|
|
81
|
+
## 2. Setup
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
python3 -m venv venv
|
|
85
|
+
source venv/bin/activate
|
|
86
|
+
pip install google-genai python-dotenv
|
|
87
|
+
```
|
|
88
|
+
### 3. API Configuration
|
|
89
|
+
|
|
90
|
+
Create a `.env` file in the root directory to securely
|
|
91
|
+
store your credentials:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
echo "GEMINI_API_KEY='your_api_key_here'" > .env
|
|
95
|
+
```
|
|
96
|
+
## 🏁 Operational Dashboard
|
|
97
|
+
|
|
98
|
+
To execute the full suite of agents in sequence and generate all security reports:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
python3 main.py
|
|
102
|
+
```
|
|
103
|
+
### Pattern: Over-Privileged Container
|
|
104
|
+
|
|
105
|
+
* **Indicators:** `allowPrivilegeEscalation: true` or root user execution.
|
|
106
|
+
* **Investigation:** Pass security intent (e.g., "non-root only") to the K8s Hardener module.
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## 💡 Best Practices
|
|
111
|
+
|
|
112
|
+
* **Context is King:** Provide at least 5 lines of context around Git diffs for more accurate neural reasoning.
|
|
113
|
+
* **Continuous Gating:** Run the FinOps auditor before every infrastructure change, not after.
|
|
114
|
+
* **Manual Sign-off:** Use AI findings as a high-fidelity signal, but maintain human-in-the-loop for kernel-level merges.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## 🔒 Security & Safety Notes
|
|
119
|
+
|
|
120
|
+
* **Key Management:** Use CI/CD secrets for `GEMINI_API_KEY` in production.
|
|
121
|
+
* **Least Privilege:** Test "Hardened" manifests in staging first to ensure no functional regressions.
|
|
122
|
+
|
|
123
|
+
## Links
|
|
124
|
+
|
|
125
|
+
+ - **Repository**: https://github.com/Champbreed/AegisOps-AI
|
|
126
|
+
+ - **Documentation**: https://github.com/Champbreed/AegisOps-AI#readme
|
|
127
|
+
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Jetski/Cortex + Gemini Integration Guide
|
|
3
|
-
description: "Come usare antigravity-awesome-skills con Jetski/Cortex evitando l’overflow di contesto con 1.
|
|
3
|
+
description: "Come usare antigravity-awesome-skills con Jetski/Cortex evitando l’overflow di contesto con 1.311+ skill."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Jetski/Cortex + Gemini: integrazione sicura con 1.
|
|
6
|
+
# Jetski/Cortex + Gemini: integrazione sicura con 1.311+ skill
|
|
7
7
|
|
|
8
8
|
Questa guida mostra come integrare il repository `antigravity-awesome-skills` con un agente basato su **Jetski/Cortex + Gemini** (o framework simili) **senza superare il context window** del modello.
|
|
9
9
|
|
|
@@ -23,7 +23,7 @@ Non bisogna mai:
|
|
|
23
23
|
- concatenare il contenuto di tutte le `SKILL.md` in un singolo system prompt;
|
|
24
24
|
- reiniettare l’intera libreria per **ogni** richiesta.
|
|
25
25
|
|
|
26
|
-
Con oltre 1.
|
|
26
|
+
Con oltre 1.311 skill, questo approccio riempie il context window prima ancora di aggiungere i messaggi dell’utente, causando l’errore di truncation.
|
|
27
27
|
|
|
28
28
|
---
|
|
29
29
|
|
|
@@ -20,7 +20,7 @@ This example shows one way to integrate **antigravity-awesome-skills** with a Je
|
|
|
20
20
|
- How to enforce a **maximum number of skills per turn** via `maxSkillsPerTurn`.
|
|
21
21
|
- How to choose whether to **truncate or error** when too many skills are requested via `overflowBehavior`.
|
|
22
22
|
|
|
23
|
-
This pattern avoids context overflow when you have 1,
|
|
23
|
+
This pattern avoids context overflow when you have 1,311+ skills installed.
|
|
24
24
|
|
|
25
25
|
---
|
|
26
26
|
|
|
@@ -6,7 +6,7 @@ This document keeps the repository's GitHub-facing discovery copy aligned with t
|
|
|
6
6
|
|
|
7
7
|
Preferred positioning:
|
|
8
8
|
|
|
9
|
-
> Installable GitHub library of 1,
|
|
9
|
+
> Installable GitHub library of 1,311+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and other AI coding assistants.
|
|
10
10
|
|
|
11
11
|
Key framing:
|
|
12
12
|
|
|
@@ -20,7 +20,7 @@ Key framing:
|
|
|
20
20
|
|
|
21
21
|
Preferred description:
|
|
22
22
|
|
|
23
|
-
> Installable GitHub library of 1,
|
|
23
|
+
> Installable GitHub library of 1,311+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.
|
|
24
24
|
|
|
25
25
|
Preferred homepage:
|
|
26
26
|
|
|
@@ -28,7 +28,7 @@ Preferred homepage:
|
|
|
28
28
|
|
|
29
29
|
Preferred social preview:
|
|
30
30
|
|
|
31
|
-
- use a clean preview image that says `1,
|
|
31
|
+
- use a clean preview image that says `1,311+ Agentic Skills`;
|
|
32
32
|
- mention Claude Code, Cursor, Codex CLI, and Gemini CLI;
|
|
33
33
|
- avoid dense text and tiny logos that disappear in social cards.
|
|
34
34
|
|
|
@@ -69,7 +69,7 @@ For manual updates, you need:
|
|
|
69
69
|
The update process refreshes:
|
|
70
70
|
- Skills index (`skills_index.json`)
|
|
71
71
|
- Web app skills data (`apps\web-app\public\skills.json`)
|
|
72
|
-
- All 1,
|
|
72
|
+
- All 1,311+ skills from the skills directory
|
|
73
73
|
|
|
74
74
|
## When to Update
|
|
75
75
|
|
|
@@ -6,7 +6,7 @@ Antigravity Awesome Skills gives Claude Code users an installable library of `SK
|
|
|
6
6
|
|
|
7
7
|
## Why use this repo for Claude Code
|
|
8
8
|
|
|
9
|
-
- It includes 1,
|
|
9
|
+
- It includes 1,311+ skills instead of a narrow single-domain starter pack.
|
|
10
10
|
- It supports the standard `.claude/skills/` path and the Claude Code plugin marketplace flow.
|
|
11
11
|
- It includes onboarding docs, bundles, and workflows so new users do not need to guess where to begin.
|
|
12
12
|
- It covers both everyday engineering tasks and specialized work like security reviews, infrastructure, product planning, and documentation.
|
|
@@ -8,7 +8,7 @@ Antigravity Awesome Skills supports Gemini CLI through the `.gemini/skills/` pat
|
|
|
8
8
|
|
|
9
9
|
- It installs directly into the expected Gemini skills path.
|
|
10
10
|
- It includes both core software engineering skills and deeper agent/LLM-oriented skills.
|
|
11
|
-
- It helps new users get started with bundles and workflows rather than forcing a cold start from 1,
|
|
11
|
+
- It helps new users get started with bundles and workflows rather than forcing a cold start from 1,311+ files.
|
|
12
12
|
- It is useful whether you want a broad internal skill library or a single repo to test many workflows quickly.
|
|
13
13
|
|
|
14
14
|
## Install Gemini CLI Skills
|
|
@@ -18,7 +18,7 @@ Kiro is AWS's agentic AI IDE that combines:
|
|
|
18
18
|
|
|
19
19
|
Kiro's agentic capabilities are enhanced by skills that provide:
|
|
20
20
|
|
|
21
|
-
- **Domain expertise** across 1,
|
|
21
|
+
- **Domain expertise** across 1,311+ specialized areas
|
|
22
22
|
- **Best practices** from Anthropic, OpenAI, Google, Microsoft, and AWS
|
|
23
23
|
- **Workflow automation** for common development tasks
|
|
24
24
|
- **AWS-specific patterns** for serverless, infrastructure, and cloud architecture
|
|
@@ -12,7 +12,7 @@ Great question! Here's what just happened and what to do next:
|
|
|
12
12
|
|
|
13
13
|
When you ran `npx antigravity-awesome-skills` or cloned the repository, you:
|
|
14
14
|
|
|
15
|
-
✅ **Downloaded 1,
|
|
15
|
+
✅ **Downloaded 1,311+ skill files** to your computer (default: `~/.gemini/antigravity/skills/`; or a custom path like `~/.agent/skills/` if you used `--path`)
|
|
16
16
|
✅ **Made them available** to your AI assistant
|
|
17
17
|
❌ **Did NOT enable them all automatically** (they're just sitting there, waiting)
|
|
18
18
|
|
|
@@ -32,7 +32,7 @@ Bundles are **recommended lists** of skills grouped by role. They help you decid
|
|
|
32
32
|
|
|
33
33
|
**Analogy:**
|
|
34
34
|
|
|
35
|
-
- You installed a toolbox with 1,
|
|
35
|
+
- You installed a toolbox with 1,311+ tools (✅ done)
|
|
36
36
|
- Bundles are like **labeled organizer trays** saying: "If you're a carpenter, start with these 10 tools"
|
|
37
37
|
- You don't install bundles—you **pick skills from them**
|
|
38
38
|
|
|
@@ -192,7 +192,7 @@ Let's actually use a skill right now. Follow these steps:
|
|
|
192
192
|
|
|
193
193
|
## Step 5: Picking Your First Skills (Practical Advice)
|
|
194
194
|
|
|
195
|
-
Don't try to use all 1,
|
|
195
|
+
Don't try to use all 1,311+ skills at once. Here's a sensible approach:
|
|
196
196
|
|
|
197
197
|
If you want a tool-specific starting point before choosing skills, use:
|
|
198
198
|
|
|
@@ -323,7 +323,7 @@ Usually no, but if your AI doesn't recognize a skill:
|
|
|
323
323
|
|
|
324
324
|
### "Can I load all skills into the model at once?"
|
|
325
325
|
|
|
326
|
-
No. Even though you have 1,
|
|
326
|
+
No. Even though you have 1,311+ skills installed locally, you should **not** concatenate every `SKILL.md` into a single system prompt or context block.
|
|
327
327
|
|
|
328
328
|
The intended pattern is:
|
|
329
329
|
|
|
@@ -34,7 +34,7 @@ antigravity-awesome-skills/
|
|
|
34
34
|
├── 📄 CONTRIBUTING.md ← Contributor workflow
|
|
35
35
|
├── 📄 CATALOG.md ← Full generated catalog
|
|
36
36
|
│
|
|
37
|
-
├── 📁 skills/ ← 1,
|
|
37
|
+
├── 📁 skills/ ← 1,311+ skills live here
|
|
38
38
|
│ │
|
|
39
39
|
│ ├── 📁 brainstorming/
|
|
40
40
|
│ │ └── 📄 SKILL.md ← Skill definition
|
|
@@ -47,7 +47,7 @@ antigravity-awesome-skills/
|
|
|
47
47
|
│ │ └── 📁 2d-games/
|
|
48
48
|
│ │ └── 📄 SKILL.md ← Nested skills also supported
|
|
49
49
|
│ │
|
|
50
|
-
│ └── ... (1,
|
|
50
|
+
│ └── ... (1,311+ total)
|
|
51
51
|
│
|
|
52
52
|
├── 📁 apps/
|
|
53
53
|
│ └── 📁 web-app/ ← Interactive browser
|
|
@@ -100,7 +100,7 @@ antigravity-awesome-skills/
|
|
|
100
100
|
|
|
101
101
|
```
|
|
102
102
|
┌─────────────────────────┐
|
|
103
|
-
│ 1,
|
|
103
|
+
│ 1,311+ SKILLS │
|
|
104
104
|
└────────────┬────────────┘
|
|
105
105
|
│
|
|
106
106
|
┌────────────────────────┼────────────────────────┐
|
|
@@ -201,7 +201,7 @@ If you want a workspace-style manual install instead, cloning into `.agent/skill
|
|
|
201
201
|
│ ├── 📁 brainstorming/ │
|
|
202
202
|
│ ├── 📁 stripe-integration/ │
|
|
203
203
|
│ ├── 📁 react-best-practices/ │
|
|
204
|
-
│ └── ... (1,
|
|
204
|
+
│ └── ... (1,311+ total) │
|
|
205
205
|
└─────────────────────────────────────────┘
|
|
206
206
|
```
|
|
207
207
|
|
|
@@ -0,0 +1,228 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: snowflake-development
|
|
3
|
+
description: "Comprehensive Snowflake development assistant covering SQL best practices, data pipeline design (Dynamic Tables, Streams, Tasks, Snowpipe), Cortex AI functions, Cortex Agents, Snowpark Python, dbt integration, performance tuning, and security hardening."
|
|
4
|
+
category: data-engineering
|
|
5
|
+
risk: safe
|
|
6
|
+
source: community
|
|
7
|
+
date_added: "2026-03-24"
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Snowflake Development
|
|
11
|
+
|
|
12
|
+
You are a Snowflake development expert. Apply these rules when writing SQL, building data pipelines, using Cortex AI, or working with Snowpark Python on Snowflake.
|
|
13
|
+
|
|
14
|
+
## SQL Best Practices
|
|
15
|
+
|
|
16
|
+
### Naming and Style
|
|
17
|
+
|
|
18
|
+
- Use `snake_case` for all identifiers. Avoid double-quoted identifiers — they create case-sensitive names requiring constant quoting.
|
|
19
|
+
- Use CTEs (`WITH` clauses) over nested subqueries.
|
|
20
|
+
- Use `CREATE OR REPLACE` for idempotent DDL.
|
|
21
|
+
- Use explicit column lists — never `SELECT *` in production (Snowflake's columnar storage scans only referenced columns).
|
|
22
|
+
|
|
23
|
+
### Stored Procedures — Colon Prefix Rule
|
|
24
|
+
|
|
25
|
+
In SQL stored procedures (BEGIN...END blocks), variables and parameters **must** use the colon `:` prefix inside SQL statements. Without it, Snowflake raises "invalid identifier" errors.
|
|
26
|
+
|
|
27
|
+
BAD:
|
|
28
|
+
```sql
|
|
29
|
+
CREATE PROCEDURE my_proc(p_id INT) RETURNS STRING LANGUAGE SQL AS
|
|
30
|
+
BEGIN
|
|
31
|
+
LET result STRING;
|
|
32
|
+
SELECT name INTO result FROM users WHERE id = p_id;
|
|
33
|
+
RETURN result;
|
|
34
|
+
END;
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
GOOD:
|
|
38
|
+
```sql
|
|
39
|
+
CREATE PROCEDURE my_proc(p_id INT) RETURNS STRING LANGUAGE SQL AS
|
|
40
|
+
BEGIN
|
|
41
|
+
LET result STRING;
|
|
42
|
+
SELECT name INTO :result FROM users WHERE id = :p_id;
|
|
43
|
+
RETURN result;
|
|
44
|
+
END;
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### Semi-Structured Data
|
|
48
|
+
|
|
49
|
+
- VARIANT, OBJECT, ARRAY for JSON/Avro/Parquet/ORC.
|
|
50
|
+
- Access nested fields: `src:customer.name::STRING`. Always cast: `src:price::NUMBER(10,2)`.
|
|
51
|
+
- VARIANT null vs SQL NULL: JSON `null` is stored as `"null"`. Use `STRIP_NULL_VALUE = TRUE` on load.
|
|
52
|
+
- Flatten arrays: `SELECT f.value:name::STRING FROM my_table, LATERAL FLATTEN(input => src:items) f;`
|
|
53
|
+
|
|
54
|
+
### MERGE for Upserts
|
|
55
|
+
|
|
56
|
+
```sql
|
|
57
|
+
MERGE INTO target t USING source s ON t.id = s.id
|
|
58
|
+
WHEN MATCHED THEN UPDATE SET t.name = s.name, t.updated_at = CURRENT_TIMESTAMP()
|
|
59
|
+
WHEN NOT MATCHED THEN INSERT (id, name, updated_at) VALUES (s.id, s.name, CURRENT_TIMESTAMP());
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Data Pipelines
|
|
63
|
+
|
|
64
|
+
### Choosing Your Approach
|
|
65
|
+
|
|
66
|
+
| Approach | When to Use |
|
|
67
|
+
|----------|-------------|
|
|
68
|
+
| Dynamic Tables | Declarative transformations. **Default choice.** Define the query, Snowflake handles refresh. |
|
|
69
|
+
| Streams + Tasks | Imperative CDC. Use for procedural logic, stored procedure calls. |
|
|
70
|
+
| Snowpipe | Continuous file loading from S3/GCS/Azure. |
|
|
71
|
+
|
|
72
|
+
### Dynamic Tables
|
|
73
|
+
|
|
74
|
+
```sql
|
|
75
|
+
CREATE OR REPLACE DYNAMIC TABLE cleaned_events
|
|
76
|
+
TARGET_LAG = '5 minutes'
|
|
77
|
+
WAREHOUSE = transform_wh
|
|
78
|
+
AS
|
|
79
|
+
SELECT event_id, event_type, user_id, event_timestamp
|
|
80
|
+
FROM raw_events
|
|
81
|
+
WHERE event_type IS NOT NULL;
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Key rules:
|
|
85
|
+
- Set `TARGET_LAG` progressively: tighter at top, looser at bottom.
|
|
86
|
+
- Incremental DTs **cannot** depend on Full refresh DTs.
|
|
87
|
+
- `SELECT *` breaks on schema changes — use explicit column lists.
|
|
88
|
+
- Change tracking must stay enabled on base tables.
|
|
89
|
+
- Views cannot sit between two Dynamic Tables.
|
|
90
|
+
|
|
91
|
+
### Streams and Tasks
|
|
92
|
+
|
|
93
|
+
```sql
|
|
94
|
+
CREATE OR REPLACE STREAM raw_stream ON TABLE raw_events;
|
|
95
|
+
|
|
96
|
+
CREATE OR REPLACE TASK process_events
|
|
97
|
+
WAREHOUSE = transform_wh
|
|
98
|
+
SCHEDULE = 'USING CRON 0 */1 * * * America/Los_Angeles'
|
|
99
|
+
WHEN SYSTEM$STREAM_HAS_DATA('raw_stream')
|
|
100
|
+
AS INSERT INTO cleaned_events SELECT ... FROM raw_stream;
|
|
101
|
+
|
|
102
|
+
-- Tasks start SUSPENDED — you MUST resume them
|
|
103
|
+
ALTER TASK process_events RESUME;
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Cortex AI
|
|
107
|
+
|
|
108
|
+
### Function Reference
|
|
109
|
+
|
|
110
|
+
| Function | Purpose |
|
|
111
|
+
|----------|---------|
|
|
112
|
+
| `AI_COMPLETE` | LLM completion (text, images, documents) |
|
|
113
|
+
| `AI_CLASSIFY` | Classify into categories (up to 500 labels) |
|
|
114
|
+
| `AI_FILTER` | Boolean filter on text/images |
|
|
115
|
+
| `AI_EXTRACT` | Structured extraction from text/images/documents |
|
|
116
|
+
| `AI_SENTIMENT` | Sentiment score (-1 to 1) |
|
|
117
|
+
| `AI_PARSE_DOCUMENT` | OCR or layout extraction |
|
|
118
|
+
| `AI_REDACT` | PII removal |
|
|
119
|
+
|
|
120
|
+
**Deprecated (do NOT use):** `COMPLETE`, `CLASSIFY_TEXT`, `EXTRACT_ANSWER`, `PARSE_DOCUMENT`, `SUMMARIZE`, `TRANSLATE`, `SENTIMENT`, `EMBED_TEXT_768`.
|
|
121
|
+
|
|
122
|
+
### TO_FILE — Common Error Source
|
|
123
|
+
|
|
124
|
+
Stage path and filename are **SEPARATE** arguments:
|
|
125
|
+
|
|
126
|
+
```sql
|
|
127
|
+
-- BAD: TO_FILE('@stage/file.pdf')
|
|
128
|
+
-- GOOD:
|
|
129
|
+
TO_FILE('@db.schema.mystage', 'invoice.pdf')
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
### Use AI_CLASSIFY for Classification (Not AI_COMPLETE)
|
|
133
|
+
|
|
134
|
+
```sql
|
|
135
|
+
SELECT AI_CLASSIFY(ticket_text,
|
|
136
|
+
['billing', 'technical', 'account']):labels[0]::VARCHAR AS category
|
|
137
|
+
FROM tickets;
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
### Cortex Agents
|
|
141
|
+
|
|
142
|
+
```sql
|
|
143
|
+
CREATE OR REPLACE AGENT my_db.my_schema.sales_agent
|
|
144
|
+
FROM SPECIFICATION $spec$
|
|
145
|
+
{
|
|
146
|
+
"models": {"orchestration": "auto"},
|
|
147
|
+
"instructions": {
|
|
148
|
+
"orchestration": "You are SalesBot...",
|
|
149
|
+
"response": "Be concise."
|
|
150
|
+
},
|
|
151
|
+
"tools": [{"tool_spec": {"type": "cortex_analyst_text_to_sql", "name": "Sales", "description": "Queries sales..."}}],
|
|
152
|
+
"tool_resources": {"Sales": {"semantic_model_file": "@stage/model.yaml"}}
|
|
153
|
+
}
|
|
154
|
+
$spec$;
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Agent rules:
|
|
158
|
+
- Use `$spec$` delimiter (not `$$`).
|
|
159
|
+
- `models` must be an object, not an array.
|
|
160
|
+
- `tool_resources` is a separate top-level object, not nested inside tools.
|
|
161
|
+
- Do NOT include empty/null values in edit specs — clears existing values.
|
|
162
|
+
- Tool descriptions are the #1 quality factor.
|
|
163
|
+
- Never modify production agents directly — clone first.
|
|
164
|
+
|
|
165
|
+
## Snowpark Python
|
|
166
|
+
|
|
167
|
+
```python
|
|
168
|
+
from snowflake.snowpark import Session
|
|
169
|
+
import os
|
|
170
|
+
|
|
171
|
+
session = Session.builder.configs({
|
|
172
|
+
"account": os.environ["SNOWFLAKE_ACCOUNT"],
|
|
173
|
+
"user": os.environ["SNOWFLAKE_USER"],
|
|
174
|
+
"password": os.environ["SNOWFLAKE_PASSWORD"],
|
|
175
|
+
"role": "my_role", "warehouse": "my_wh",
|
|
176
|
+
"database": "my_db", "schema": "my_schema"
|
|
177
|
+
}).create()
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
- Never hardcode credentials.
|
|
181
|
+
- DataFrames are lazy — executed on `collect()`/`show()`.
|
|
182
|
+
- Do NOT use `collect()` on large DataFrames — process server-side.
|
|
183
|
+
- Use **vectorized UDFs** (10-100x faster) for batch/ML workloads instead of scalar UDFs.
|
|
184
|
+
|
|
185
|
+
## dbt on Snowflake
|
|
186
|
+
|
|
187
|
+
Dynamic table materialization (streaming/near-real-time marts):
|
|
188
|
+
```sql
|
|
189
|
+
{{ config(materialized='dynamic_table', snowflake_warehouse='transforming', target_lag='1 hour') }}
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Incremental materialization (large fact tables):
|
|
193
|
+
```sql
|
|
194
|
+
{{ config(materialized='incremental', unique_key='event_id') }}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Snowflake-specific configs (combine with any materialization):
|
|
198
|
+
```sql
|
|
199
|
+
{{ config(transient=true, copy_grants=true, query_tag='team_daily') }}
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
- Do NOT use `{{ this }}` without `{% if is_incremental() %}` guard.
|
|
203
|
+
- Use `dynamic_table` materialization for streaming/near-real-time marts.
|
|
204
|
+
|
|
205
|
+
## Performance
|
|
206
|
+
|
|
207
|
+
- **Cluster keys**: Only multi-TB tables, on WHERE/JOIN/GROUP BY columns.
|
|
208
|
+
- **Search Optimization**: `ALTER TABLE t ADD SEARCH OPTIMIZATION ON EQUALITY(col);`
|
|
209
|
+
- **Warehouse sizing**: Start X-Small, scale up. `AUTO_SUSPEND = 60`, `AUTO_RESUME = TRUE`.
|
|
210
|
+
- **Separate warehouses** per workload.
|
|
211
|
+
- Estimate AI costs first: `SELECT SUM(AI_COUNT_TOKENS('claude-4-sonnet', text)) FROM table;`
|
|
212
|
+
|
|
213
|
+
## Security
|
|
214
|
+
|
|
215
|
+
- Follow least-privilege RBAC. Use database roles for object-level grants.
|
|
216
|
+
- Audit ACCOUNTADMIN regularly: `SHOW GRANTS OF ROLE ACCOUNTADMIN;`
|
|
217
|
+
- Use network policies for IP allowlisting.
|
|
218
|
+
- Use masking policies for PII columns and row access policies for multi-tenant isolation.
|
|
219
|
+
|
|
220
|
+
## Common Error Patterns
|
|
221
|
+
|
|
222
|
+
| Error | Cause | Fix |
|
|
223
|
+
|-------|-------|-----|
|
|
224
|
+
| "Object does not exist" | Wrong context or missing grants | Fully qualify names, check grants |
|
|
225
|
+
| "Invalid identifier" in proc | Missing colon prefix | Use `:variable_name` |
|
|
226
|
+
| "Numeric value not recognized" | VARIANT not cast | `src:field::NUMBER(10,2)` |
|
|
227
|
+
| Task not running | Forgot to resume | `ALTER TASK ... RESUME` |
|
|
228
|
+
| DT refresh failing | Schema change or tracking disabled | Use explicit columns, check change tracking |
|