everything-gemini 1.2.0 โ 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/ISSUE_TEMPLATE/bug_report.md +29 -0
- package/.github/ISSUE_TEMPLATE/feature_request.md +27 -0
- package/.github/ISSUE_TEMPLATE/subagent_proposal.md +29 -0
- package/.github/workflows/ci.yml +72 -1
- package/.github/workflows/release.yml +17 -2
- package/CHANGELOG.md +155 -0
- package/CONTRIBUTING.md +85 -4
- package/README.md +134 -19
- package/action.yml +35 -0
- package/agents/browser-qa.md +1 -1
- package/agents/database-migrator.md +26 -0
- package/agents/documentation-writer.md +27 -0
- package/agents/incident-responder.md +34 -0
- package/agents/performance-tuner.md +27 -0
- package/agents/release-engineer.md +34 -0
- package/bin/everything-gemini +1517 -122
- package/docs/COMMUNITY_SUBMISSIONS.md +88 -0
- package/docs/LAUNCH_KIT.md +138 -13
- package/docs/architecture/progressive-disclosure.md +3 -3
- package/docs/index.html +401 -40
- package/docs/launch-kit.md +163 -0
- package/examples/cli-workflows/demo_workflow.sh +15 -7
- package/examples/python-sdk/README.md +28 -5
- package/examples/python-sdk/documentation_audit_pipeline.py +83 -0
- package/examples/python-sdk/incident_triage_pipeline.py +170 -0
- package/examples/python-sdk/multi_agent_pipeline.py +95 -0
- package/examples/python-sdk/release_readiness_audit.py +144 -0
- package/hooks/secret-scrubber/hooks.json +16 -0
- package/hooks/secret-scrubber/scripts/scrub-secrets.sh +62 -0
- package/mcp/README.md +20 -0
- package/mcp/mcp_config.example.json +35 -0
- package/package.json +3 -1
- package/plugins/core/rules/core-practices.md +63 -4
- package/plugins/python-ai/rules/ai-agent-patterns.md +67 -4
- package/plugins/security-shield/rules/zero-trust.md +60 -4
- package/plugins/web-fullstack/rules/frontend-standards.md +74 -4
- package/rules/paradigms/security-hardened.md +53 -10
- package/rules/paradigms/tdd.md +65 -8
- package/scripts/install.ps1 +14 -0
- package/scripts/install.sh +15 -1
- package/scripts/validate.sh +103 -12
- package/skills/apache-kafka-streams/SKILL.md +152 -0
- package/skills/argo-workflows/SKILL.md +151 -0
- package/skills/axum-web-service/SKILL.md +159 -0
- package/skills/cilium-ebpf-networking/SKILL.md +145 -0
- package/skills/clickhouse-analytics/SKILL.md +128 -0
- package/skills/dbt-transformation/SKILL.md +179 -0
- package/skills/debezium-cdc/SKILL.md +166 -0
- package/skills/duckdb-local-olap/SKILL.md +134 -0
- package/skills/fastapi-production/SKILL.md +157 -0
- package/skills/grpc-connect-rpc/SKILL.md +176 -0
- package/skills/langgraph-orchestrator/SKILL.md +144 -0
- package/skills/litellm-proxy-gateway/SKILL.md +154 -0
- package/skills/nats-jetstream/SKILL.md +122 -0
- package/skills/open-telemetry-tracing/SKILL.md +151 -0
- package/skills/playwright-visual-regression/SKILL.md +124 -0
- package/skills/redis-cache-patterns/SKILL.md +159 -0
- package/skills/svelte-kit-architect/SKILL.md +190 -0
- package/skills/temporal-workflows/SKILL.md +158 -0
- package/skills/trpc-react-query/SKILL.md +169 -0
- package/skills/turborepo-monorepo/SKILL.md +199 -0
- package/skills/vllm-inference/SKILL.md +144 -0
- package/skills/wasm-rust-optimizer/SKILL.md +130 -0
- package/templates/full-power/hooks.json +14 -0
- package/templates/full-power/mcp_config.json +43 -0
- package/templates/recommended/hooks.json +14 -0
- package/tests/cli.test.js +383 -0
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: langgraph-orchestrator
|
|
3
|
+
description: >-
|
|
4
|
+
Designs and implements stateful multi-agent systems, cyclic graphs, persistent checkpoints, human-in-the-loop pauses, and streaming orchestration using LangGraph (Python/TypeScript).
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# LangGraph Multi-Agent Orchestration & Stateful Graphs
|
|
8
|
+
|
|
9
|
+
This skill guides the design, construction, and deployment of complex multi-agent architectures using LangGraph. It provides deterministic patterns for cyclic routing, persistent graph checkpointing, and human-in-the-loop (HITL) approval gates.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## โก Core Concepts
|
|
14
|
+
|
|
15
|
+
1. **State Graph (`StateGraph`)**: Typed dictionary or Pydantic state passed through graph nodes.
|
|
16
|
+
2. **Nodes**: Discrete execution functions that receive current state and return state updates.
|
|
17
|
+
3. **Edges & Conditional Edges**: Transition rules between nodes (deterministic or LLM-routed).
|
|
18
|
+
4. **Checkpointers (`BaseCheckpointSaver`)**: Persists graph state across turns and enables time-travel debugging.
|
|
19
|
+
5. **Human-in-the-Loop (`interrupt_before`)**: Pauses execution graph at critical checkpoints for human intervention.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## ๐๏ธ Production Implementation (Python)
|
|
24
|
+
|
|
25
|
+
### 1. State Definition & Reducers (`state.py`)
|
|
26
|
+
|
|
27
|
+
Define immutable state schemas with explicit reducer operators:
|
|
28
|
+
|
|
29
|
+
```python
|
|
30
|
+
from typing import Annotated, Sequence, TypedDict
|
|
31
|
+
from langchain_core.messages import BaseMessage
|
|
32
|
+
from langgraph.graph.message import add_messages
|
|
33
|
+
|
|
34
|
+
class AgentState(TypedDict):
|
|
35
|
+
# add_messages reducer appends new messages rather than overwriting
|
|
36
|
+
messages: Annotated[Sequence[BaseMessage], add_messages]
|
|
37
|
+
sender: str
|
|
38
|
+
iteration_count: int
|
|
39
|
+
is_approved: bool
|
|
40
|
+
context_data: dict
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
### 2. Multi-Agent Nodes & Graph Assembly (`graph.py`)
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
from langgraph.graph import StateGraph, END, START
|
|
49
|
+
from langgraph.checkpoint.memory import MemorySaver
|
|
50
|
+
from langchain_core.messages import AIMessage, HumanMessage
|
|
51
|
+
|
|
52
|
+
def researcher_node(state: AgentState) -> dict:
|
|
53
|
+
"""Researches user inquiry and extracts factual grounding."""
|
|
54
|
+
# LLM invocation logic here
|
|
55
|
+
result = "Research findings: latency reduced by 40% via caching."
|
|
56
|
+
return {
|
|
57
|
+
"messages": [AIMessage(content=result, name="researcher")],
|
|
58
|
+
"sender": "researcher",
|
|
59
|
+
"iteration_count": state.get("iteration_count", 0) + 1,
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
def reviewer_node(state: AgentState) -> dict:
|
|
63
|
+
"""Validates research quality and enforces compliance."""
|
|
64
|
+
last_msg = state["messages"][-1].content
|
|
65
|
+
approved = "latency" in last_msg.lower()
|
|
66
|
+
return {
|
|
67
|
+
"messages": [AIMessage(content="Review complete.", name="reviewer")],
|
|
68
|
+
"sender": "reviewer",
|
|
69
|
+
"is_approved": approved,
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
def human_approval_node(state: AgentState) -> dict:
|
|
73
|
+
"""Node executed after human approval gate."""
|
|
74
|
+
return {"messages": [AIMessage(content="Action executed after sign-off.", name="human_gate")]}
|
|
75
|
+
|
|
76
|
+
def router_condition(state: AgentState) -> str:
|
|
77
|
+
"""Evaluate whether to loop or terminate."""
|
|
78
|
+
if state.get("is_approved", False):
|
|
79
|
+
return "human_approval"
|
|
80
|
+
if state.get("iteration_count", 0) >= 3:
|
|
81
|
+
return "abort"
|
|
82
|
+
return "researcher"
|
|
83
|
+
|
|
84
|
+
# Build State Graph
|
|
85
|
+
workflow = StateGraph(AgentState)
|
|
86
|
+
|
|
87
|
+
# Add Nodes
|
|
88
|
+
workflow.add_node("researcher", researcher_node)
|
|
89
|
+
workflow.add_node("reviewer", reviewer_node)
|
|
90
|
+
workflow.add_node("human_approval", human_approval_node)
|
|
91
|
+
|
|
92
|
+
# Add Edges
|
|
93
|
+
workflow.add_edge(START, "researcher")
|
|
94
|
+
workflow.add_edge("researcher", "reviewer")
|
|
95
|
+
workflow.add_conditional_edges(
|
|
96
|
+
"reviewer",
|
|
97
|
+
router_condition,
|
|
98
|
+
{
|
|
99
|
+
"human_approval": "human_approval",
|
|
100
|
+
"researcher": "researcher",
|
|
101
|
+
"abort": END,
|
|
102
|
+
}
|
|
103
|
+
)
|
|
104
|
+
workflow.add_edge("human_approval", END)
|
|
105
|
+
|
|
106
|
+
# Compile with Persistent Checkpointer and Human-in-the-Loop Interruption
|
|
107
|
+
checkpointer = MemorySaver()
|
|
108
|
+
app = workflow.compile(
|
|
109
|
+
checkpointer=checkpointer,
|
|
110
|
+
interrupt_before=["human_approval"]
|
|
111
|
+
)
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
### 3. Execution & Human-in-the-Loop Resumption
|
|
117
|
+
|
|
118
|
+
```python
|
|
119
|
+
thread_config = {"configurable": {"thread_id": "session-prod-1024"}}
|
|
120
|
+
|
|
121
|
+
# Run until human interrupt
|
|
122
|
+
inputs = {"messages": [HumanMessage(content="Optimize database latency.")], "iteration_count": 0}
|
|
123
|
+
for event in app.stream(inputs, config=thread_config):
|
|
124
|
+
print(f"Step: {event}")
|
|
125
|
+
|
|
126
|
+
# Inspect current snapshot before approval
|
|
127
|
+
snapshot = app.get_state(thread_config)
|
|
128
|
+
print(f"Next Node Pending Approval: {snapshot.next}")
|
|
129
|
+
|
|
130
|
+
# Resume graph after human confirmation
|
|
131
|
+
for event in app.stream(None, config=thread_config):
|
|
132
|
+
print(f"Post-approval Step: {event}")
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## ๐ก๏ธ Best Practices & Anti-Patterns
|
|
138
|
+
|
|
139
|
+
| Category | โ Anti-Pattern | โ
Production Best Practice |
|
|
140
|
+
| :--- | :--- | :--- |
|
|
141
|
+
| **State Mutation** | Modifying state objects in-place | Return clean partial state dictionaries with reducer keys |
|
|
142
|
+
| **Cycles** | Unbounded recursive loops | Enforce explicit `iteration_count` or recursion limit (`recursion_limit=25`) |
|
|
143
|
+
| **Checkpoints** | Storing giant binary blobs in state | Keep state lean; persist media in object stores (GCS/S3) and store URIs |
|
|
144
|
+
| **HITL** | Hardcoded polling loops | Use native `interrupt_before` / `interrupt_after` with thread configurations |
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: litellm-proxy-gateway
|
|
3
|
+
description: >-
|
|
4
|
+
Enterprise LLM proxy gateway architecture using LiteLLM, multi-provider routing (Gemini, Claude, OpenAI, vLLM), virtual keys, budget tracking, fallback cascading, and rate-limiting. Use when centralizing LLM API access, managing cost quotas, or implementing resilient failover strategies.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# LiteLLM Proxy Gateway Architecture
|
|
8
|
+
|
|
9
|
+
A production runbook for centralizing, securing, and load-balancing LLM inference across multi-provider deployments (Google Gemini, Anthropic, OpenAI, local vLLM) using **LiteLLM Proxy**.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## ๐ฏ Architectural Principles
|
|
14
|
+
|
|
15
|
+
1. **Unified API Gateway**: Exposes a standard OpenAI-compatible API (`/v1/chat/completions`, `/v1/embeddings`) for 100+ LLM providers.
|
|
16
|
+
2. **Resilient Fallback Cascades**: Automatically catches rate limits (`429`) or provider outages (`5xx`) and retries the prompt on configured fallback models.
|
|
17
|
+
3. **Budget Controls & Virtual Keys**: Issues team-specific API keys with enforce-able spending caps, RPM/TPM rate limits, and model access whitelists.
|
|
18
|
+
4. **Token Usage & Cost Attribution**: Centralizes Prometheus metrics, OTEL telemetry, and audit logs per team, project, or key.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## ๐ Gateway Configuration (`config.yaml`)
|
|
23
|
+
|
|
24
|
+
```yaml
|
|
25
|
+
model_list:
|
|
26
|
+
# Primary: Gemini 2.5 Flash
|
|
27
|
+
- model_name: production-fast
|
|
28
|
+
litellm_params:
|
|
29
|
+
model: gemini/gemini-2.5-flash
|
|
30
|
+
api_key: "os.environ/GEMINI_API_KEY"
|
|
31
|
+
rpm: 1000
|
|
32
|
+
tpm: 1000000
|
|
33
|
+
|
|
34
|
+
# Fallback: OpenAI GPT-4o-mini
|
|
35
|
+
- model_name: production-fast-fallback
|
|
36
|
+
litellm_params:
|
|
37
|
+
model: gpt-4o-mini
|
|
38
|
+
api_key: "os.environ/OPENAI_API_KEY"
|
|
39
|
+
rpm: 500
|
|
40
|
+
|
|
41
|
+
# High-Reasoning: Gemini 2.5 Pro with Claude 3.5 Sonnet Fallback
|
|
42
|
+
- model_name: high-reasoning
|
|
43
|
+
litellm_params:
|
|
44
|
+
model: gemini/gemini-2.5-pro
|
|
45
|
+
api_key: "os.environ/GEMINI_API_KEY"
|
|
46
|
+
|
|
47
|
+
- model_name: high-reasoning-fallback
|
|
48
|
+
litellm_params:
|
|
49
|
+
model: anthropic/claude-3-5-sonnet-20241022
|
|
50
|
+
api_key: "os.environ/ANTHROPIC_API_KEY"
|
|
51
|
+
|
|
52
|
+
router_settings:
|
|
53
|
+
routing_strategy: "latency-based-routing" # options: simple-shuffle, least-busy, latency-based-routing
|
|
54
|
+
redis_host: "redis.internal"
|
|
55
|
+
redis_port: 6379
|
|
56
|
+
redis_password: "os.environ/REDIS_PASSWORD"
|
|
57
|
+
num_retries: 3
|
|
58
|
+
timeout: 45
|
|
59
|
+
fallbacks:
|
|
60
|
+
- production-fast: ["production-fast-fallback"]
|
|
61
|
+
- high-reasoning: ["high-reasoning-fallback"]
|
|
62
|
+
|
|
63
|
+
general_settings:
|
|
64
|
+
master_key: "os.environ/LITELLM_MASTER_KEY"
|
|
65
|
+
database_url: "os.environ/DATABASE_URL" # PostgreSQL for virtual keys & spend logs
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## ๐ณ Docker Production Deployment
|
|
71
|
+
|
|
72
|
+
```yaml
|
|
73
|
+
# docker-compose.yml
|
|
74
|
+
version: '3.8'
|
|
75
|
+
|
|
76
|
+
services:
|
|
77
|
+
litellm:
|
|
78
|
+
image: ghcr.io/berriai/litellm:main-latest
|
|
79
|
+
ports:
|
|
80
|
+
- "4000:4000"
|
|
81
|
+
volumes:
|
|
82
|
+
- ./config.yaml:/app/config.yaml
|
|
83
|
+
environment:
|
|
84
|
+
- GEMINI_API_KEY=${GEMINI_API_KEY}
|
|
85
|
+
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
|
86
|
+
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
|
|
87
|
+
- LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
|
|
88
|
+
- DATABASE_URL=postgresql://litellm:${DB_PASS}@postgres:5432/litellm_db
|
|
89
|
+
command: ["--config", "/app/config.yaml", "--port", "4000", "--workers", "4"]
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## ๐ Virtual Key Management (cURL / REST API)
|
|
95
|
+
|
|
96
|
+
Generate isolated, budget-constrained virtual keys for distinct microservices or teams:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
curl -X POST 'http://localhost:4000/key/generate' \
|
|
100
|
+
-H 'Authorization: Bearer sk-master-key-xyz' \
|
|
101
|
+
-H 'Content-Type: application/json' \
|
|
102
|
+
-d '{
|
|
103
|
+
"models": ["production-fast", "high-reasoning"],
|
|
104
|
+
"max_budget": 500.0,
|
|
105
|
+
"budget_duration": "30d",
|
|
106
|
+
"rpm_limit": 120,
|
|
107
|
+
"metadata": {
|
|
108
|
+
"team": "analytics-agents",
|
|
109
|
+
"environment": "production"
|
|
110
|
+
}
|
|
111
|
+
}'
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## ๐ป Client Invocation (Python / OpenAI SDK)
|
|
117
|
+
|
|
118
|
+
Applications interact with LiteLLM identically to native OpenAI endpoints:
|
|
119
|
+
|
|
120
|
+
```python
|
|
121
|
+
from openai import OpenAI
|
|
122
|
+
|
|
123
|
+
# Connect to internal LiteLLM Proxy Gateway
|
|
124
|
+
client = OpenAI(
|
|
125
|
+
base_url="http://localhost:4000/v1",
|
|
126
|
+
api_key="sk-team-analytics-virtual-key",
|
|
127
|
+
)
|
|
128
|
+
|
|
129
|
+
response = client.chat.completions.create(
|
|
130
|
+
model="production-fast", # Automatically routes to Gemini 2.5 Flash, falls back to GPT-4o-mini
|
|
131
|
+
messages=[
|
|
132
|
+
{"role": "system", "content": "You are a cloud infrastructure specialist."},
|
|
133
|
+
{"role": "user", "content": "Explain eBPF socket redirection performance."}
|
|
134
|
+
],
|
|
135
|
+
temperature=0.3,
|
|
136
|
+
)
|
|
137
|
+
|
|
138
|
+
print(response.choices[0].message.content)
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## ๐ก๏ธ Production Health & Monitoring
|
|
144
|
+
|
|
145
|
+
1. **Check Gateway Health**:
|
|
146
|
+
```bash
|
|
147
|
+
curl http://localhost:4000/health
|
|
148
|
+
```
|
|
149
|
+
2. **Prometheus Metrics**:
|
|
150
|
+
- `litellm_deployment_latency_per_output_token`: Model latency breakdowns.
|
|
151
|
+
- `litellm_deployment_spend`: Real-time cumulative spend per model and key.
|
|
152
|
+
- `litellm_proxy_failed_requests`: Rate of fallback activations.
|
|
153
|
+
3. **Automatic Circuit Breaking**:
|
|
154
|
+
- If a provider triggers 5 consecutive `500` or `503` errors, the router pauses traffic to that upstream for 60 seconds.
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: nats-jetstream
|
|
3
|
+
description: >-
|
|
4
|
+
Architects, publishes, and consumes ultra-low latency event streams with NATS JetStream, configuring stream retention, message deduplication, durable consumer groups, and KV buckets.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# NATS JetStream Architecture & High-Speed Event Streaming
|
|
8
|
+
|
|
9
|
+
This skill guides building resilient, cloud-native event-driven architectures with NATS JetStream, delivering high throughput, sub-millisecond latencies, and at-least-once delivery semantics.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## โก Core Concepts: Core NATS vs. JetStream
|
|
14
|
+
|
|
15
|
+
| Feature | Core NATS | NATS JetStream |
|
|
16
|
+
| :--- | :--- | :--- |
|
|
17
|
+
| **Delivery Model** | At-most-once (Fire & Forget) | At-least-once / Exactly-once (via deduplication) |
|
|
18
|
+
| **Persistence** | In-memory only (ephemeral) | File / Memory persistent disk storage |
|
|
19
|
+
| **Consumer State**| None (real-time listeners only)| Durable cursors with acknowledgments & replays |
|
|
20
|
+
| **Flow Control** | TCP backpressure only | Consumer ack wait, max deliver limits, backoffs |
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## ๐๏ธ Stream Definition & Deduplication (TypeScript)
|
|
25
|
+
|
|
26
|
+
Configure streams with subject patterns and deduplication windows:
|
|
27
|
+
|
|
28
|
+
```typescript
|
|
29
|
+
import { connect, JSONCodec, RetentionPolicy, StorageType } from 'nats';
|
|
30
|
+
|
|
31
|
+
const nc = await connect({ servers: process.env.NATS_URL || 'localhost:4222' });
|
|
32
|
+
const jsm = await nc.jetstreamManager();
|
|
33
|
+
const jc = JSONCodec();
|
|
34
|
+
|
|
35
|
+
// Create or update stream
|
|
36
|
+
await jsm.streams.add({
|
|
37
|
+
name: 'ORDERS',
|
|
38
|
+
subjects: ['orders.>'],
|
|
39
|
+
storage: StorageType.File,
|
|
40
|
+
retention: RetentionPolicy.Limits,
|
|
41
|
+
max_age: 7 * 24 * 60 * 60 * 1e9, // 7 days in nanoseconds
|
|
42
|
+
duplicate_window: 120 * 1e9, // 2 minutes deduplication window
|
|
43
|
+
});
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## ๐ค Publishing with Message Deduplication
|
|
49
|
+
|
|
50
|
+
Provide a unique `Nats-Msg-Id` header to guarantee idempotent delivery even during network retries:
|
|
51
|
+
|
|
52
|
+
```typescript
|
|
53
|
+
import { headers } from 'nats';
|
|
54
|
+
|
|
55
|
+
const js = nc.jetstream();
|
|
56
|
+
|
|
57
|
+
export async function publishOrderEvent(orderId: string, payload: Record<string, any>) {
|
|
58
|
+
const h = headers();
|
|
59
|
+
// Server drops duplicates published with the same ID within the duplicate_window
|
|
60
|
+
h.append('Nats-Msg-Id', `order-${orderId}-${payload.status}`);
|
|
61
|
+
|
|
62
|
+
const ack = await js.publish(`orders.${payload.region}.${orderId}`, jc.encode(payload), {
|
|
63
|
+
headers: h,
|
|
64
|
+
});
|
|
65
|
+
|
|
66
|
+
return ack.seq;
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## ๐ฅ Durable Pull Consumers (High-Throughput Worker Pools)
|
|
73
|
+
|
|
74
|
+
Pull consumers give workers complete control over processing speed and batch sizes:
|
|
75
|
+
|
|
76
|
+
```typescript
|
|
77
|
+
import { AckPolicy } from 'nats';
|
|
78
|
+
|
|
79
|
+
export async function startOrderWorker() {
|
|
80
|
+
const js = nc.jetstream();
|
|
81
|
+
|
|
82
|
+
// Create or bind durable consumer
|
|
83
|
+
const consumer = await js.consumers.get('ORDERS', 'order-fulfillment-workers');
|
|
84
|
+
|
|
85
|
+
while (true) {
|
|
86
|
+
// Pull up to 25 messages, waiting at most 2 seconds
|
|
87
|
+
const batch = await consumer.fetch({ max_messages: 25, expires: 2000 });
|
|
88
|
+
|
|
89
|
+
for await (const msg of batch) {
|
|
90
|
+
try {
|
|
91
|
+
const orderData = jc.decode(msg.data);
|
|
92
|
+
console.log(`Processing Order seq ${msg.seq}:`, orderData);
|
|
93
|
+
|
|
94
|
+
// Acknowledge successful processing
|
|
95
|
+
msg.ack();
|
|
96
|
+
} catch (err) {
|
|
97
|
+
// Negative acknowledgment: triggers redelivery based on backoff
|
|
98
|
+
msg.nak();
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## ๐๏ธ NATS Key-Value (KV) Buckets
|
|
108
|
+
|
|
109
|
+
Use JetStream KV buckets for distributed application configuration and state:
|
|
110
|
+
|
|
111
|
+
```typescript
|
|
112
|
+
const kv = await js.views.kv('app-configuration');
|
|
113
|
+
|
|
114
|
+
// Set configuration
|
|
115
|
+
await kv.put('features.dark_mode', jc.encode({ enabled: true }));
|
|
116
|
+
|
|
117
|
+
// Watch for real-time configuration changes across all pods
|
|
118
|
+
const watcher = await kv.watch({ key: 'features.*' });
|
|
119
|
+
for await (const entry of watcher) {
|
|
120
|
+
console.log(`Config updated: ${entry.key} =`, jc.decode(entry.value));
|
|
121
|
+
}
|
|
122
|
+
```
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: open-telemetry-tracing
|
|
3
|
+
description: >-
|
|
4
|
+
Architects and instruments distributed tracing, OTLP exporters, W3C traceparent propagation, custom span attributes, and metric correlations using OpenTelemetry (Node.js/Python).
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# OpenTelemetry Distributed Tracing & Observability
|
|
8
|
+
|
|
9
|
+
This skill guides the implementation of production-grade distributed tracing using the OpenTelemetry (OTel) standard, providing end-to-end request visibility across microservices, async event buses, and AI agent reasoning loops.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## โก The Observability Triad in OpenTelemetry
|
|
14
|
+
|
|
15
|
+
1. **Traces**: Directed acyclic graphs of spans tracking request lifecycles across process boundaries.
|
|
16
|
+
2. **Context Propagation**: Standardized W3C `traceparent` HTTP headers (`00-<trace_id>-<span_id>-<flags>`).
|
|
17
|
+
3. **Semantic Conventions**: Standardized attribute keys (`http.status_code`, `db.system`, `gen_ai.request.model`).
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## ๐๏ธ Production Implementation (Node.js / TypeScript)
|
|
22
|
+
|
|
23
|
+
### 1. SDK Initialization (`instrumentation.ts`)
|
|
24
|
+
|
|
25
|
+
Must execute before any other application modules are imported:
|
|
26
|
+
|
|
27
|
+
```typescript
|
|
28
|
+
import { NodeSDK } from '@opentelemetry/sdk-node';
|
|
29
|
+
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
|
|
30
|
+
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
|
|
31
|
+
import { Resource } from '@opentelemetry/resources';
|
|
32
|
+
import { SEMRESATTRS_SERVICE_NAME, SEMRESATTRS_SERVICE_VERSION } from '@opentelemetry/semantic-conventions';
|
|
33
|
+
|
|
34
|
+
const sdk = new NodeSDK({
|
|
35
|
+
resource: new Resource({
|
|
36
|
+
[SEMRESATTRS_SERVICE_NAME]: 'agentic-gateway',
|
|
37
|
+
[SEMRESATTRS_SERVICE_VERSION]: '1.7.0',
|
|
38
|
+
'deployment.environment': process.env.NODE_ENV || 'production',
|
|
39
|
+
}),
|
|
40
|
+
traceExporter: new OTLPTraceExporter({
|
|
41
|
+
url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://otel-collector:4318/v1/traces',
|
|
42
|
+
headers: {},
|
|
43
|
+
}),
|
|
44
|
+
instrumentations: [
|
|
45
|
+
getNodeAutoInstrumentations({
|
|
46
|
+
'@opentelemetry/instrumentation-fs': { enabled: false }, // Prevent span explosion
|
|
47
|
+
}),
|
|
48
|
+
],
|
|
49
|
+
});
|
|
50
|
+
|
|
51
|
+
sdk.start();
|
|
52
|
+
|
|
53
|
+
process.on('SIGTERM', () => {
|
|
54
|
+
sdk.shutdown()
|
|
55
|
+
.then(() => console.log('Tracing terminated'))
|
|
56
|
+
.finally(() => process.exit(0));
|
|
57
|
+
});
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
### 2. Custom Business & Agent Spans (`tracer.ts`)
|
|
63
|
+
|
|
64
|
+
```typescript
|
|
65
|
+
import { trace, SpanStatusCode, propagation, context } from '@opentelemetry/api';
|
|
66
|
+
|
|
67
|
+
const tracer = trace.getTracer('agent-core', '1.7.0');
|
|
68
|
+
|
|
69
|
+
export async function executeAgentReasoningStep<T>(
|
|
70
|
+
stepName: string,
|
|
71
|
+
modelName: string,
|
|
72
|
+
fn: () => Promise<T>
|
|
73
|
+
): Promise<T> {
|
|
74
|
+
return tracer.startActiveSpan(
|
|
75
|
+
`agent.reasoning.${stepName}`,
|
|
76
|
+
{
|
|
77
|
+
attributes: {
|
|
78
|
+
'gen_ai.system': 'google-gemini',
|
|
79
|
+
'gen_ai.request.model': modelName,
|
|
80
|
+
'agent.step': stepName,
|
|
81
|
+
},
|
|
82
|
+
},
|
|
83
|
+
async (span) => {
|
|
84
|
+
try {
|
|
85
|
+
span.addEvent('llm_prompt_dispatched', { timestamp: Date.now() });
|
|
86
|
+
const result = await fn();
|
|
87
|
+
span.setStatus({ code: SpanStatusCode.OK });
|
|
88
|
+
return result;
|
|
89
|
+
} catch (error: any) {
|
|
90
|
+
span.recordException(error);
|
|
91
|
+
span.setStatus({
|
|
92
|
+
code: SpanStatusCode.ERROR,
|
|
93
|
+
message: error?.message || 'Reasoning execution failed',
|
|
94
|
+
});
|
|
95
|
+
throw error;
|
|
96
|
+
} finally {
|
|
97
|
+
span.end();
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
);
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
/** Propagate traceparent to downstream HTTP calls */
|
|
104
|
+
export function injectTraceHeaders(targetHeaders: Record<string, string>): void {
|
|
105
|
+
propagation.inject(context.active(), targetHeaders);
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## ๐ Python Implementation (`telemetry.py`)
|
|
112
|
+
|
|
113
|
+
```python
|
|
114
|
+
from opentelemetry import trace
|
|
115
|
+
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
|
|
116
|
+
from opentelemetry.sdk.trace import TracerProvider
|
|
117
|
+
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
|
118
|
+
from opentelemetry.sdk.resources import Resource
|
|
119
|
+
from opentelemetry.trace import Status, StatusCode
|
|
120
|
+
|
|
121
|
+
provider = TracerProvider(
|
|
122
|
+
resource=Resource.create({"service.name": "python-worker", "service.version": "1.7.0"})
|
|
123
|
+
)
|
|
124
|
+
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://otel-collector:4318/v1/traces"))
|
|
125
|
+
provider.add_span_processor(processor)
|
|
126
|
+
trace.set_tracer_provider(provider)
|
|
127
|
+
|
|
128
|
+
tracer = trace.get_tracer("worker-agent", "1.7.0")
|
|
129
|
+
|
|
130
|
+
def run_workflow_step(task_id: str):
|
|
131
|
+
with tracer.start_as_current_span("worker.process_task") as span:
|
|
132
|
+
span.set_attribute("task.id", task_id)
|
|
133
|
+
try:
|
|
134
|
+
# Task logic here
|
|
135
|
+
span.set_status(Status(StatusCode.OK))
|
|
136
|
+
except Exception as exc:
|
|
137
|
+
span.record_exception(exc)
|
|
138
|
+
span.set_status(Status(StatusCode.ERROR, str(exc)))
|
|
139
|
+
raise
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## ๐ก๏ธ Best Practices & Guardrails
|
|
145
|
+
|
|
146
|
+
| Category | โ Anti-Pattern | โ
Production Best Practice |
|
|
147
|
+
| :--- | :--- | :--- |
|
|
148
|
+
| **Span Lifecycle** | Forgetting to invoke `span.end()` in error blocks | Wrap with `try / catch / finally` and call `span.end()` |
|
|
149
|
+
| **High Cardinality** | Injecting user emails, passwords, or raw tokens into attributes | Use hashed IDs (`user_hash`), sanitize PII, follow semantic conventions |
|
|
150
|
+
| **Context Leaks** | Starting detached spans without active parent context | Use `tracer.startActiveSpan()` to maintain parent-child hierarchy |
|
|
151
|
+
| **Over-instrumentation** | Tracing every internal loop iteration or file read | Instrument service boundaries, external HTTP/DB calls, and workflow steps |
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: playwright-visual-regression
|
|
3
|
+
description: >-
|
|
4
|
+
Implements automated visual regression testing, pixelmatch snapshot diffing, cross-browser viewport matrices, dynamic element masking, and containerized baseline stability using Playwright.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Playwright Visual Regression Testing & Snapshot Diffing
|
|
8
|
+
|
|
9
|
+
This skill provides production standards for catching unintended UI regressions, CSS layout shifts, and visual anomalies across viewports using Playwright's native screenshot comparison engine.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## โก The Golden Rules of Visual Testing
|
|
14
|
+
|
|
15
|
+
1. **Deterministic Environment**: Always generate reference snapshots inside the same containerized Linux environment (`mcr.microsoft.com/playwright`) to avoid font rendering divergence between macOS and CI runners.
|
|
16
|
+
2. **Mask Dynamic Elements**: Mask timestamps, avatars, animations, and non-deterministic text using Playwright's `mask` option.
|
|
17
|
+
3. **Wait for Font & Network Idle**: Ensure web fonts and critical API payloads are loaded before capturing snapshots.
|
|
18
|
+
4. **Strict Diff Thresholds**: Use `maxDiffPixelRatio: 0.01` (1% tolerance) to balance anti-aliasing tolerance against regression detection.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## ๐๏ธ Production Playwright Configuration (`playwright.config.ts`)
|
|
23
|
+
|
|
24
|
+
```typescript
|
|
25
|
+
import { defineConfig, devices } from '@playwright/test';
|
|
26
|
+
|
|
27
|
+
export default defineConfig({
|
|
28
|
+
testDir: './tests/visual',
|
|
29
|
+
snapshotPathTemplate: '{testDir}/__snapshots__/{testFilePath}/{arg}{ext}',
|
|
30
|
+
expect: {
|
|
31
|
+
toHaveScreenshot: {
|
|
32
|
+
maxDiffPixelRatio: 0.01,
|
|
33
|
+
animations: 'disabled',
|
|
34
|
+
threshold: 0.2, // Pixel color sensitivity (0.0 to 1.0)
|
|
35
|
+
},
|
|
36
|
+
},
|
|
37
|
+
use: {
|
|
38
|
+
baseURL: process.env.BASE_URL || 'http://localhost:3000',
|
|
39
|
+
trace: 'on-first-retry',
|
|
40
|
+
screenshot: 'only-on-failure',
|
|
41
|
+
},
|
|
42
|
+
projects: [
|
|
43
|
+
{
|
|
44
|
+
name: 'Desktop Chrome',
|
|
45
|
+
use: { ...devices['Desktop Chrome'], viewport: { width: 1440, height: 900 } },
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
name: 'Mobile Safari',
|
|
49
|
+
use: { ...devices['iPhone 14'], viewport: { width: 390, height: 844 } },
|
|
50
|
+
},
|
|
51
|
+
{
|
|
52
|
+
name: 'Tablet iPad',
|
|
53
|
+
use: { ...devices['iPad Pro 11'], viewport: { width: 834, height: 1194 } },
|
|
54
|
+
},
|
|
55
|
+
],
|
|
56
|
+
});
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## ๐งช Visual Test Implementation (`tests/visual/dashboard.spec.ts`)
|
|
62
|
+
|
|
63
|
+
```typescript
|
|
64
|
+
import { test, expect } from '@playwright/test';
|
|
65
|
+
|
|
66
|
+
test.describe('Dashboard Visual Regression Suite', () => {
|
|
67
|
+
test.beforeEach(async ({ page }) => {
|
|
68
|
+
// Navigate and stabilize network
|
|
69
|
+
await page.goto('/dashboard', { waitUntil: 'networkidle' });
|
|
70
|
+
|
|
71
|
+
// Wait for custom web fonts to be completely rendered
|
|
72
|
+
await page.evaluate(() => document.fonts.ready);
|
|
73
|
+
});
|
|
74
|
+
|
|
75
|
+
test('full dashboard page layout matches baseline', async ({ page }) => {
|
|
76
|
+
// Dynamic locators that vary between test runs
|
|
77
|
+
const dynamicTimestamp = page.locator('[data-testid="last-sync-time"]');
|
|
78
|
+
const userAvatar = page.locator('[data-testid="user-avatar"]');
|
|
79
|
+
|
|
80
|
+
await expect(page).toHaveScreenshot('dashboard-main-view.png', {
|
|
81
|
+
fullPage: true,
|
|
82
|
+
mask: [dynamicTimestamp, userAvatar],
|
|
83
|
+
maskColor: '#1e293b',
|
|
84
|
+
});
|
|
85
|
+
});
|
|
86
|
+
|
|
87
|
+
test('component-level snapshot for modal dialog', async ({ page }) => {
|
|
88
|
+
await page.click('[data-testid="open-settings-modal"]');
|
|
89
|
+
const modal = page.locator('[role="dialog"]');
|
|
90
|
+
|
|
91
|
+
await expect(modal).toBeVisible();
|
|
92
|
+
await expect(modal).toHaveScreenshot('settings-modal.png');
|
|
93
|
+
});
|
|
94
|
+
});
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## ๐ณ Containerized Snapshot Workflow (CI Stability)
|
|
100
|
+
|
|
101
|
+
Run snapshots inside the official Playwright Docker container to prevent cross-platform font rendering diffs:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
# 1. Update baseline snapshots inside Docker container
|
|
105
|
+
docker run --rm --network host -v $(pwd):/work/ -w /work/ \
|
|
106
|
+
mcr.microsoft.com/playwright:v1.45.0-jammy \
|
|
107
|
+
npx playwright test --update-snapshots
|
|
108
|
+
|
|
109
|
+
# 2. Run visual validation in CI
|
|
110
|
+
docker run --rm --network host -v $(pwd):/work/ -w /work/ \
|
|
111
|
+
mcr.microsoft.com/playwright:v1.45.0-jammy \
|
|
112
|
+
npx playwright test
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## ๐ Checklist & Anti-Patterns
|
|
118
|
+
|
|
119
|
+
| Category | โ Anti-Pattern | โ
Production Best Practice |
|
|
120
|
+
| :--- | :--- | :--- |
|
|
121
|
+
| **Animation** | Taking snapshots with active CSS transitions | Set `animations: 'disabled'` in config |
|
|
122
|
+
| **Flakiness** | Relying on `page.waitForTimeout(3000)` | Wait on `networkidle`, `document.fonts.ready`, and locator visibility |
|
|
123
|
+
| **Scope** | Capturing entire page for minor widget changes | Use element locators: `expect(locator).toHaveScreenshot()` |
|
|
124
|
+
| **OS Mismatch** | Committing macOS snapshots to run in Ubuntu CI | Generate reference snapshots with Docker container |
|