@geraldmaron/construct 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +69 -0
- package/LICENSE +21 -0
- package/README.md +344 -0
- package/agents/prompts/cx-accessibility.md +29 -0
- package/agents/prompts/cx-ai-engineer.md +31 -0
- package/agents/prompts/cx-architect.md +40 -0
- package/agents/prompts/cx-business-strategist.md +25 -0
- package/agents/prompts/cx-data-analyst.md +30 -0
- package/agents/prompts/cx-data-engineer.md +32 -0
- package/agents/prompts/cx-debugger.md +27 -0
- package/agents/prompts/cx-designer.md +33 -0
- package/agents/prompts/cx-devil-advocate.md +36 -0
- package/agents/prompts/cx-docs-keeper.md +50 -0
- package/agents/prompts/cx-engineer.md +36 -0
- package/agents/prompts/cx-evaluator.md +26 -0
- package/agents/prompts/cx-explorer.md +38 -0
- package/agents/prompts/cx-legal-compliance.md +25 -0
- package/agents/prompts/cx-operations.md +28 -0
- package/agents/prompts/cx-orchestrator.md +30 -0
- package/agents/prompts/cx-platform-engineer.md +25 -0
- package/agents/prompts/cx-product-manager.md +30 -0
- package/agents/prompts/cx-qa.md +42 -0
- package/agents/prompts/cx-rd-lead.md +25 -0
- package/agents/prompts/cx-release-manager.md +31 -0
- package/agents/prompts/cx-researcher.md +31 -0
- package/agents/prompts/cx-reviewer.md +36 -0
- package/agents/prompts/cx-security.md +37 -0
- package/agents/prompts/cx-sre.md +26 -0
- package/agents/prompts/cx-test-automation.md +35 -0
- package/agents/prompts/cx-trace-reviewer.md +76 -0
- package/agents/prompts/cx-ux-researcher.md +30 -0
- package/agents/registry.json +879 -0
- package/agents/teams.json +94 -0
- package/bin/construct +613 -0
- package/commands/build/feature.md +21 -0
- package/commands/build/fix.md +20 -0
- package/commands/design/access.md +19 -0
- package/commands/design/flow.md +16 -0
- package/commands/design/ui.md +18 -0
- package/commands/measure/experiment.md +18 -0
- package/commands/measure/metrics.md +18 -0
- package/commands/measure/results.md +18 -0
- package/commands/plan/api.md +19 -0
- package/commands/plan/challenge.md +18 -0
- package/commands/plan/decide.md +20 -0
- package/commands/plan/feature.md +49 -0
- package/commands/plan/requirements.md +24 -0
- package/commands/remember/context.md +26 -0
- package/commands/remember/handoff.md +19 -0
- package/commands/remember/runbook.md +20 -0
- package/commands/review/code.md +21 -0
- package/commands/review/quality.md +19 -0
- package/commands/review/security.md +16 -0
- package/commands/ship/ready.md +19 -0
- package/commands/ship/release.md +19 -0
- package/commands/ship/status.md +18 -0
- package/commands/understand/docs.md +22 -0
- package/commands/understand/research.md +22 -0
- package/commands/understand/this.md +23 -0
- package/commands/understand/why.md +18 -0
- package/commands/work/clean.md +32 -0
- package/commands/work/drive.md +69 -0
- package/commands/work/optimize-prompts.md +46 -0
- package/commands/work/parallel-review.md +40 -0
- package/db/migrations/001_init.sql +38 -0
- package/langfuse/docker-compose.yml +82 -0
- package/lib/audit-skills.mjs +118 -0
- package/lib/auto-docs.mjs +293 -0
- package/lib/cli-commands.mjs +409 -0
- package/lib/codex-config.mjs +114 -0
- package/lib/comment-lint.mjs +191 -0
- package/lib/completions.mjs +170 -0
- package/lib/context-state.mjs +99 -0
- package/lib/cost.mjs +213 -0
- package/lib/distill.mjs +403 -0
- package/lib/efficiency.mjs +139 -0
- package/lib/env-config.mjs +53 -0
- package/lib/eval-harness.mjs +59 -0
- package/lib/features.mjs +209 -0
- package/lib/headhunt.mjs +597 -0
- package/lib/hooks/adaptive-lint.mjs +112 -0
- package/lib/hooks/agent-tracker.mjs +50 -0
- package/lib/hooks/bootstrap-guard.mjs +90 -0
- package/lib/hooks/comment-lint.mjs +26 -0
- package/lib/hooks/config-protection.mjs +52 -0
- package/lib/hooks/console-warn.mjs +43 -0
- package/lib/hooks/context-window-recovery.mjs +90 -0
- package/lib/hooks/continuation-enforcer.mjs +72 -0
- package/lib/hooks/dep-audit.mjs +155 -0
- package/lib/hooks/drive-guard.mjs +89 -0
- package/lib/hooks/edit-accumulator.mjs +36 -0
- package/lib/hooks/edit-error-recovery.mjs +46 -0
- package/lib/hooks/edit-guard.mjs +109 -0
- package/lib/hooks/env-check.mjs +80 -0
- package/lib/hooks/guard-bash.mjs +83 -0
- package/lib/hooks/mcp-audit.mjs +57 -0
- package/lib/hooks/mcp-health-check.mjs +51 -0
- package/lib/hooks/mcp-task-scope.mjs +47 -0
- package/lib/hooks/model-fallback.mjs +105 -0
- package/lib/hooks/pre-compact.mjs +212 -0
- package/lib/hooks/pre-push-gate.mjs +129 -0
- package/lib/hooks/read-tracker.mjs +129 -0
- package/lib/hooks/registry-sync.mjs +23 -0
- package/lib/hooks/scan-secrets.mjs +76 -0
- package/lib/hooks/session-start.mjs +95 -0
- package/lib/hooks/stop-notify.mjs +191 -0
- package/lib/hooks/stop-typecheck.mjs +83 -0
- package/lib/hooks/task-completed-guard.mjs +43 -0
- package/lib/hooks/teammate-idle-guard.mjs +54 -0
- package/lib/hooks/workflow-guard.mjs +62 -0
- package/lib/host-capabilities.mjs +123 -0
- package/lib/init-docs.mjs +382 -0
- package/lib/mcp/server.mjs +1123 -0
- package/lib/mcp-catalog.json +243 -0
- package/lib/mcp-manager.mjs +433 -0
- package/lib/mcp-platform-config.mjs +104 -0
- package/lib/model-router.mjs +810 -0
- package/lib/opencode-config.mjs +44 -0
- package/lib/opencode-runtime-plugin.mjs +909 -0
- package/lib/opencode-telemetry.mjs +433 -0
- package/lib/orchestration-policy.mjs +258 -0
- package/lib/prompt-composer.mjs +196 -0
- package/lib/prompt-metadata.mjs +68 -0
- package/lib/review.mjs +429 -0
- package/lib/role-preload.mjs +52 -0
- package/lib/schemas/decision.json +50 -0
- package/lib/schemas/implementation.json +51 -0
- package/lib/schemas/review-report.json +32 -0
- package/lib/schemas/test-report.json +43 -0
- package/lib/server/index.mjs +334 -0
- package/lib/server/static/app.js +841 -0
- package/lib/server/static/index.html +820 -0
- package/lib/service-manager.mjs +225 -0
- package/lib/setup.mjs +319 -0
- package/lib/status.mjs +707 -0
- package/lib/storage/backend.mjs +44 -0
- package/lib/storage/embeddings.mjs +70 -0
- package/lib/storage/hybrid-query.mjs +184 -0
- package/lib/storage/sql-store.mjs +55 -0
- package/lib/storage/state-source.mjs +101 -0
- package/lib/storage/sync.mjs +164 -0
- package/lib/storage/vector-store.mjs +59 -0
- package/lib/telemetry/backends/langfuse.mjs +58 -0
- package/lib/telemetry/backends/noop.mjs +16 -0
- package/lib/telemetry/langfuse-ingest.mjs +108 -0
- package/lib/telemetry/langfuse-model-sync.mjs +270 -0
- package/lib/telemetry/team-rollup.mjs +143 -0
- package/lib/toolkit-env.mjs +30 -0
- package/lib/validator.mjs +181 -0
- package/lib/workflow-state.mjs +794 -0
- package/package.json +53 -0
- package/personas/construct.md +94 -0
- package/platforms/claude/CLAUDE.md +30 -0
- package/platforms/claude/settings.template.json +472 -0
- package/platforms/opencode/config.template.json +65 -0
- package/platforms/opencode/plugins/construct-fallback.js +5 -0
- package/platforms/opencode/sync-config.mjs +69 -0
- package/rules/common/agents.md +55 -0
- package/rules/common/code-review.md +129 -0
- package/rules/common/coding-style.md +95 -0
- package/rules/common/comments.md +139 -0
- package/rules/common/cx-agent-routing.md +61 -0
- package/rules/common/cx-skill-routing.md +11 -0
- package/rules/common/development-workflow.md +49 -0
- package/rules/common/git-workflow.md +29 -0
- package/rules/common/hooks.md +35 -0
- package/rules/common/patterns.md +36 -0
- package/rules/common/performance.md +71 -0
- package/rules/common/security.md +34 -0
- package/rules/common/testing.md +62 -0
- package/rules/golang/coding-style.md +37 -0
- package/rules/golang/hooks.md +22 -0
- package/rules/golang/patterns.md +50 -0
- package/rules/golang/security.md +39 -0
- package/rules/golang/testing.md +36 -0
- package/rules/python/coding-style.md +47 -0
- package/rules/python/hooks.md +24 -0
- package/rules/python/patterns.md +44 -0
- package/rules/python/security.md +35 -0
- package/rules/python/testing.md +43 -0
- package/rules/swift/coding-style.md +52 -0
- package/rules/swift/hooks.md +25 -0
- package/rules/swift/patterns.md +71 -0
- package/rules/swift/security.md +38 -0
- package/rules/swift/testing.md +50 -0
- package/rules/typescript/coding-style.md +204 -0
- package/rules/typescript/hooks.md +27 -0
- package/rules/typescript/patterns.md +57 -0
- package/rules/typescript/security.md +33 -0
- package/rules/typescript/testing.md +23 -0
- package/rules/web/coding-style.md +101 -0
- package/rules/web/design-quality.md +68 -0
- package/rules/web/hooks.md +125 -0
- package/rules/web/patterns.md +84 -0
- package/rules/web/performance.md +69 -0
- package/rules/web/security.md +62 -0
- package/rules/web/testing.md +60 -0
- package/rules/zh/README.md +113 -0
- package/rules/zh/agents.md +55 -0
- package/rules/zh/code-review.md +129 -0
- package/rules/zh/coding-style.md +53 -0
- package/rules/zh/development-workflow.md +49 -0
- package/rules/zh/git-workflow.md +29 -0
- package/rules/zh/hooks.md +35 -0
- package/rules/zh/patterns.md +36 -0
- package/rules/zh/performance.md +60 -0
- package/rules/zh/security.md +34 -0
- package/rules/zh/testing.md +34 -0
- package/skills/ai/agent-dev.md +102 -0
- package/skills/ai/llm-security.md +105 -0
- package/skills/ai/ml-ops.md +169 -0
- package/skills/ai/orchestration-workflow.md +87 -0
- package/skills/ai/prompt-and-eval.md +114 -0
- package/skills/ai/prompt-optimizer.md +114 -0
- package/skills/ai/rag-system.md +104 -0
- package/skills/architecture/api-design.md +100 -0
- package/skills/architecture/caching.md +101 -0
- package/skills/architecture/cloud-native.md +97 -0
- package/skills/architecture/message-queue.md +100 -0
- package/skills/architecture/security-arch.md +105 -0
- package/skills/development/cpp.md +127 -0
- package/skills/development/go.md +129 -0
- package/skills/development/java.md +134 -0
- package/skills/development/kotlin.md +140 -0
- package/skills/development/mobile-crossplatform.md +144 -0
- package/skills/development/python.md +109 -0
- package/skills/development/rust.md +127 -0
- package/skills/development/shell.md +127 -0
- package/skills/development/swift.md +129 -0
- package/skills/development/typescript.md +129 -0
- package/skills/devops/ci-cd.md +108 -0
- package/skills/devops/containerization.md +106 -0
- package/skills/devops/cost-optimization.md +105 -0
- package/skills/devops/data-engineering.md +181 -0
- package/skills/devops/database.md +95 -0
- package/skills/devops/dependency-management.md +91 -0
- package/skills/devops/devsecops.md +96 -0
- package/skills/devops/git-workflow.md +100 -0
- package/skills/devops/incident-response.md +167 -0
- package/skills/devops/monorepo.md +87 -0
- package/skills/devops/observability.md +103 -0
- package/skills/devops/performance.md +104 -0
- package/skills/devops/testing.md +104 -0
- package/skills/docs/adr-workflow.md +32 -0
- package/skills/docs/backlog-proposal-workflow.md +19 -0
- package/skills/docs/customer-profile-workflow.md +22 -0
- package/skills/docs/evidence-ingest-workflow.md +23 -0
- package/skills/docs/init-docs.md +160 -0
- package/skills/docs/init-project.md +50 -0
- package/skills/docs/prd-workflow.md +57 -0
- package/skills/docs/prfaq-workflow.md +25 -0
- package/skills/docs/product-intelligence-review.md +22 -0
- package/skills/docs/product-intelligence-workflow.md +62 -0
- package/skills/docs/product-signal-workflow.md +27 -0
- package/skills/docs/research-workflow.md +28 -0
- package/skills/docs/runbook-workflow.md +24 -0
- package/skills/exploration/repo-map.md +297 -0
- package/skills/frameworks/django.md +159 -0
- package/skills/frameworks/nextjs.md +148 -0
- package/skills/frameworks/react.md +132 -0
- package/skills/frameworks/spring-boot.md +165 -0
- package/skills/frontend-design/accessibility.md +153 -0
- package/skills/frontend-design/component-patterns.md +111 -0
- package/skills/frontend-design/engineering.md +122 -0
- package/skills/frontend-design/state-management.md +118 -0
- package/skills/frontend-design/ui-aesthetics.md +102 -0
- package/skills/frontend-design/ux-principles.md +126 -0
- package/skills/quality-gates/review-work.md +90 -0
- package/skills/quality-gates/verify-change.md +94 -0
- package/skills/quality-gates/verify-module.md +96 -0
- package/skills/quality-gates/verify-quality.md +93 -0
- package/skills/quality-gates/verify-security.md +95 -0
- package/skills/roles/architect.ai-systems.md +37 -0
- package/skills/roles/architect.data.md +37 -0
- package/skills/roles/architect.enterprise.md +37 -0
- package/skills/roles/architect.integration.md +37 -0
- package/skills/roles/architect.md +68 -0
- package/skills/roles/architect.platform.md +37 -0
- package/skills/roles/data-analyst.experiment.md +37 -0
- package/skills/roles/data-analyst.md +68 -0
- package/skills/roles/data-analyst.product-intelligence.md +37 -0
- package/skills/roles/data-analyst.product.md +37 -0
- package/skills/roles/data-analyst.telemetry.md +37 -0
- package/skills/roles/data-engineer.pipeline.md +37 -0
- package/skills/roles/data-engineer.vector-retrieval.md +37 -0
- package/skills/roles/data-engineer.warehouse.md +37 -0
- package/skills/roles/debugger.md +68 -0
- package/skills/roles/designer.accessibility.md +43 -0
- package/skills/roles/designer.md +68 -0
- package/skills/roles/engineer.ai.md +49 -0
- package/skills/roles/engineer.data.md +49 -0
- package/skills/roles/engineer.md +85 -0
- package/skills/roles/engineer.platform.md +49 -0
- package/skills/roles/operator.docs.md +43 -0
- package/skills/roles/operator.md +68 -0
- package/skills/roles/operator.release.md +43 -0
- package/skills/roles/operator.sre.md +43 -0
- package/skills/roles/orchestrator.md +66 -0
- package/skills/roles/product-manager.ai-product.md +37 -0
- package/skills/roles/product-manager.business-strategy.md +43 -0
- package/skills/roles/product-manager.enterprise.md +37 -0
- package/skills/roles/product-manager.growth.md +37 -0
- package/skills/roles/product-manager.md +67 -0
- package/skills/roles/product-manager.platform.md +37 -0
- package/skills/roles/product-manager.product.md +37 -0
- package/skills/roles/qa.ai-eval.md +37 -0
- package/skills/roles/qa.api-contract.md +37 -0
- package/skills/roles/qa.data-pipeline.md +37 -0
- package/skills/roles/qa.md +68 -0
- package/skills/roles/qa.test-automation.md +49 -0
- package/skills/roles/qa.web-ui.md +37 -0
- package/skills/roles/researcher.explorer.md +43 -0
- package/skills/roles/researcher.md +68 -0
- package/skills/roles/researcher.ux.md +43 -0
- package/skills/roles/reviewer.devil-advocate.md +43 -0
- package/skills/roles/reviewer.evaluator.md +43 -0
- package/skills/roles/reviewer.md +68 -0
- package/skills/roles/reviewer.trace.md +43 -0
- package/skills/roles/security.ai.md +37 -0
- package/skills/roles/security.appsec.md +37 -0
- package/skills/roles/security.cloud.md +37 -0
- package/skills/roles/security.legal-compliance.md +49 -0
- package/skills/roles/security.md +68 -0
- package/skills/roles/security.privacy.md +37 -0
- package/skills/roles/security.supply-chain.md +37 -0
- package/skills/routing.md +139 -0
- package/skills/security/blue-team.md +83 -0
- package/skills/security/code-audit.md +94 -0
- package/skills/security/pentest.md +84 -0
- package/skills/security/red-team.md +81 -0
- package/skills/security/threat-intel.md +89 -0
- package/skills/security/vuln-research.md +87 -0
- package/skills/utility/clean-code.md +165 -0
- package/sync-agents.mjs +899 -0
- package/templates/docs/adr.md +27 -0
- package/templates/docs/backlog-proposal.md +26 -0
- package/templates/docs/customer-profile.md +34 -0
- package/templates/docs/evidence-brief.md +37 -0
- package/templates/docs/incident-report.md +46 -0
- package/templates/docs/memo.md +27 -0
- package/templates/docs/meta-prd.md +56 -0
- package/templates/docs/one-pager.md +24 -0
- package/templates/docs/prd-business.md +61 -0
- package/templates/docs/prd-platform.md +62 -0
- package/templates/docs/prd.md +47 -0
- package/templates/docs/prfaq.md +31 -0
- package/templates/docs/product-intelligence-report.md +31 -0
- package/templates/docs/research-brief.md +29 -0
- package/templates/docs/rfc-platform.md +60 -0
- package/templates/docs/rfc.md +46 -0
- package/templates/docs/runbook.md +33 -0
- package/templates/docs/signal-brief.md +25 -0
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/cost-optimization.md — Cloud Cost Optimization — Use this skill when reducing cloud spend, right-sizing resources, or implementin
|
|
3
|
+
|
|
4
|
+
Use this skill when reducing cloud spend, right-sizing resources, or implementing FinOps practices. ## Cost Visibility
|
|
5
|
+
-->
|
|
6
|
+
# Cloud Cost Optimization
|
|
7
|
+
|
|
8
|
+
Use this skill when reducing cloud spend, right-sizing resources, or implementing FinOps practices.
|
|
9
|
+
|
|
10
|
+
## Cost Visibility
|
|
11
|
+
|
|
12
|
+
### Tagging Strategy
|
|
13
|
+
- Mandatory tags: team, service, environment, cost-center
|
|
14
|
+
- Enforce tagging via policy-as-code; block untagged resource creation
|
|
15
|
+
- Use consistent naming across providers
|
|
16
|
+
- Tag ephemeral resources (CI runners, dev environments) for cleanup automation
|
|
17
|
+
|
|
18
|
+
### Cost Allocation
|
|
19
|
+
- Map spend to teams, services, and features
|
|
20
|
+
- Separate shared infrastructure costs (networking, logging, security) and distribute fairly
|
|
21
|
+
- Track unit economics: cost per request, cost per user, cost per transaction
|
|
22
|
+
- Report weekly to team leads; monthly to leadership
|
|
23
|
+
|
|
24
|
+
### Tooling
|
|
25
|
+
- Cloud provider native: AWS Cost Explorer, GCP Billing, Azure Cost Management
|
|
26
|
+
- Third-party: Infracost, Kubecost, CloudHealth, Spot.io
|
|
27
|
+
- FinOps dashboards visible to engineering teams, not just finance
|
|
28
|
+
|
|
29
|
+
## Right-Sizing
|
|
30
|
+
|
|
31
|
+
### Compute
|
|
32
|
+
- Analyze CPU and memory utilization over 2+ weeks
|
|
33
|
+
- Target 60-70% average utilization for steady-state
|
|
34
|
+
- Downsize instances with <30% average utilization
|
|
35
|
+
- Use burstable instances (T-series, e2-micro) for variable workloads
|
|
36
|
+
- Auto-scale based on demand; avoid over-provisioning for peaks
|
|
37
|
+
|
|
38
|
+
### Database
|
|
39
|
+
- Match instance class to actual query load
|
|
40
|
+
- Use read replicas only when read traffic justifies the cost
|
|
41
|
+
- Consider serverless options (Aurora Serverless, DynamoDB on-demand) for variable loads
|
|
42
|
+
- Archive cold data to cheaper storage tiers
|
|
43
|
+
- Right-size connection pools to reduce idle connections
|
|
44
|
+
|
|
45
|
+
### Storage
|
|
46
|
+
- Lifecycle policies: move to infrequent access after 30 days, archive after 90, delete after retention
|
|
47
|
+
- Compress data before storage
|
|
48
|
+
- Deduplicate where applicable
|
|
49
|
+
- Delete orphaned volumes, snapshots, and unused images
|
|
50
|
+
- Monitor storage growth; alert on unexpected spikes
|
|
51
|
+
|
|
52
|
+
## Commitment Strategies
|
|
53
|
+
|
|
54
|
+
### Reserved Instances / Savings Plans
|
|
55
|
+
- Commit to 1-year for stable baseline workloads
|
|
56
|
+
- Cover 60-70% of steady-state with commitments
|
|
57
|
+
- Leave headroom for growth and experimentation
|
|
58
|
+
- Review utilization monthly; exchange underused reservations
|
|
59
|
+
- Use convertible reservations when instance types may change
|
|
60
|
+
|
|
61
|
+
### Spot/Preemptible Instances
|
|
62
|
+
- Use for fault-tolerant, stateless workloads: CI/CD, batch processing, dev environments
|
|
63
|
+
- Implement graceful handling of interruptions
|
|
64
|
+
- Diversify across instance types and availability zones
|
|
65
|
+
- Savings: 60-90% compared to on-demand
|
|
66
|
+
|
|
67
|
+
## Architecture Cost Patterns
|
|
68
|
+
|
|
69
|
+
### Scale to Zero
|
|
70
|
+
- Serverless functions for infrequent or variable workloads
|
|
71
|
+
- Scale Kubernetes deployments to zero during off-hours
|
|
72
|
+
- Shut down development environments outside business hours
|
|
73
|
+
- Use spot instances for non-production environments
|
|
74
|
+
|
|
75
|
+
### Efficient Data Transfer
|
|
76
|
+
- Minimize cross-region and cross-AZ traffic
|
|
77
|
+
- Use VPC endpoints for AWS service communication
|
|
78
|
+
- Compress payloads between services
|
|
79
|
+
- Cache at the edge to reduce origin traffic
|
|
80
|
+
- Consolidate API calls; avoid chatty microservice communication
|
|
81
|
+
|
|
82
|
+
### Storage Tiering
|
|
83
|
+
- Hot: SSD, high IOPS for active data
|
|
84
|
+
- Warm: standard storage for recent data
|
|
85
|
+
- Cold: infrequent access for archived data
|
|
86
|
+
- Glacier/Archive: long-term compliance retention
|
|
87
|
+
- Automate transitions with lifecycle policies
|
|
88
|
+
|
|
89
|
+
## Waste Elimination
|
|
90
|
+
|
|
91
|
+
- Idle resources: instances running with no traffic
|
|
92
|
+
- Orphaned resources: unattached volumes, unused elastic IPs, stale snapshots
|
|
93
|
+
- Over-provisioned resources: 4 CPUs allocated, 0.5 used
|
|
94
|
+
- Redundant data: multiple copies without retention policy
|
|
95
|
+
- Unused features: load balancers, NAT gateways, reserved capacity not utilized
|
|
96
|
+
|
|
97
|
+
## FinOps Process
|
|
98
|
+
|
|
99
|
+
1. **Inform**: visibility into who spends what
|
|
100
|
+
2. **Optimize**: right-size, commit, eliminate waste
|
|
101
|
+
3. **Operate**: continuous monitoring, budget alerts, anomaly detection
|
|
102
|
+
- Set budget alerts at 80% and 100% of monthly target
|
|
103
|
+
- Anomaly detection for unexpected spend spikes
|
|
104
|
+
- Review optimization recommendations weekly
|
|
105
|
+
- Track cost efficiency trends month-over-month
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/data-engineering.md — Data Engineering — 1. **Idempotency**: pipelines can run multiple times without duplicating data
|
|
3
|
+
|
|
4
|
+
## Core Principles 1. **Idempotency**: pipelines can run multiple times without duplicating data
|
|
5
|
+
-->
|
|
6
|
+
# Data Engineering
|
|
7
|
+
|
|
8
|
+
## Core Principles
|
|
9
|
+
|
|
10
|
+
1. **Idempotency**: pipelines can run multiple times without duplicating data
|
|
11
|
+
2. **Observability**: every pipeline run logs row counts, latency, and anomalies
|
|
12
|
+
3. **Data contracts**: producers and consumers agree on schema and SLAs
|
|
13
|
+
4. **Lineage**: know where every column came from and what it feeds
|
|
14
|
+
|
|
15
|
+
## Pipeline Patterns
|
|
16
|
+
|
|
17
|
+
### ELT over ETL
|
|
18
|
+
|
|
19
|
+
Modern data platforms (BigQuery, Snowflake, Redshift) are powerful enough to transform inside the warehouse. Prefer **ELT** (Extract → Load raw → Transform in warehouse):
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
Source → Airbyte/Fivetran → Raw tables → dbt models → Mart/API
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
**ETL** (transform before loading) is still appropriate for:
|
|
26
|
+
- PII masking before data leaves the source system
|
|
27
|
+
- Extremely large aggregations that reduce volume significantly
|
|
28
|
+
- Real-time streaming with latency constraints
|
|
29
|
+
|
|
30
|
+
### Batch pipelines
|
|
31
|
+
|
|
32
|
+
Use **Apache Airflow** or **Prefect** for orchestration:
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
@dag(schedule="@daily", start_date=pendulum.datetime(2024, 1, 1), catchup=False)
|
|
36
|
+
def orders_pipeline():
|
|
37
|
+
raw = extract_orders()
|
|
38
|
+
validated = validate_orders(raw)
|
|
39
|
+
loaded = load_to_warehouse(validated)
|
|
40
|
+
transform_dbt_models(loaded)
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
- `catchup=False` prevents backfilling missed runs on deploy
|
|
44
|
+
- Idempotent tasks: use `INSERT OVERWRITE` / `MERGE` instead of `INSERT`
|
|
45
|
+
- Partition by date and process one partition at a time
|
|
46
|
+
|
|
47
|
+
### Streaming pipelines
|
|
48
|
+
|
|
49
|
+
Use **Apache Kafka** + **Flink** or **Spark Structured Streaming** for sub-minute latency:
|
|
50
|
+
|
|
51
|
+
```python
|
|
52
|
+
# Flink example (PyFlink)
|
|
53
|
+
env = StreamExecutionEnvironment.get_execution_environment()
|
|
54
|
+
stream = env \
|
|
55
|
+
.from_source(kafka_source, WatermarkStrategy.no_watermarks(), "Kafka") \
|
|
56
|
+
.map(parse_event) \
|
|
57
|
+
.filter(lambda e: e.type == "order_placed") \
|
|
58
|
+
.sink_to(warehouse_sink)
|
|
59
|
+
env.execute("Orders Streaming Pipeline")
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Watermarks define event-time handling for late-arriving events — configure based on source latency SLA.
|
|
63
|
+
|
|
64
|
+
## Data Modeling
|
|
65
|
+
|
|
66
|
+
### Kimball dimensional model (warehouses)
|
|
67
|
+
|
|
68
|
+
```sql
|
|
69
|
+
-- Fact table: measurements/events at grain
|
|
70
|
+
CREATE TABLE fct_orders (
|
|
71
|
+
order_id BIGINT PRIMARY KEY,
|
|
72
|
+
customer_sk BIGINT REFERENCES dim_customer(sk),
|
|
73
|
+
product_sk BIGINT REFERENCES dim_product(sk),
|
|
74
|
+
ordered_at TIMESTAMP,
|
|
75
|
+
amount_usd NUMERIC(12,2)
|
|
76
|
+
);
|
|
77
|
+
|
|
78
|
+
-- Dimension: slowly changing attribute context
|
|
79
|
+
CREATE TABLE dim_customer (
|
|
80
|
+
sk BIGINT PRIMARY KEY, -- surrogate key
|
|
81
|
+
customer_id VARCHAR NOT NULL, -- natural key
|
|
82
|
+
name VARCHAR NOT NULL,
|
|
83
|
+
tier VARCHAR,
|
|
84
|
+
valid_from TIMESTAMP NOT NULL,
|
|
85
|
+
valid_to TIMESTAMP, -- NULL = current
|
|
86
|
+
is_current BOOLEAN DEFAULT TRUE
|
|
87
|
+
);
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
- Facts are narrow (IDs + measures)
|
|
91
|
+
- Dimensions are wide (all attributes about an entity)
|
|
92
|
+
- Slowly Changing Dimension Type 2 (SCD2) tracks historical state via `valid_from`/`valid_to`
|
|
93
|
+
|
|
94
|
+
### dbt for transformations
|
|
95
|
+
|
|
96
|
+
```sql
|
|
97
|
+
-- models/marts/fct_orders.sql
|
|
98
|
+
WITH source AS (
|
|
99
|
+
SELECT * FROM {{ ref('stg_orders') }}
|
|
100
|
+
),
|
|
101
|
+
validated AS (
|
|
102
|
+
SELECT * FROM source WHERE amount_usd > 0
|
|
103
|
+
)
|
|
104
|
+
SELECT
|
|
105
|
+
order_id,
|
|
106
|
+
customer_id,
|
|
107
|
+
{{ dbt_utils.generate_surrogate_key(['customer_id']) }} AS customer_sk,
|
|
108
|
+
ordered_at,
|
|
109
|
+
amount_usd
|
|
110
|
+
FROM validated
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
- `ref()` builds the DAG automatically
|
|
114
|
+
- `source()` for raw tables with freshness assertions
|
|
115
|
+
- Add `dbt test` assertions: `not_null`, `unique`, `accepted_values`, `relationships`
|
|
116
|
+
- `dbt docs generate && dbt docs serve` for lineage UI
|
|
117
|
+
|
|
118
|
+
## Data Quality
|
|
119
|
+
|
|
120
|
+
### At ingestion
|
|
121
|
+
|
|
122
|
+
- Schema validation: reject records that don't match expected types/nullability
|
|
123
|
+
- Volume checks: alert if row count deviates > 20% from trailing 7-day average
|
|
124
|
+
- Freshness: alert if source hasn't updated within its SLA window
|
|
125
|
+
|
|
126
|
+
### At transformation
|
|
127
|
+
|
|
128
|
+
```yaml
|
|
129
|
+
# dbt tests in schema.yml
|
|
130
|
+
models:
|
|
131
|
+
- name: fct_orders
|
|
132
|
+
columns:
|
|
133
|
+
- name: order_id
|
|
134
|
+
tests: [unique, not_null]
|
|
135
|
+
- name: amount_usd
|
|
136
|
+
tests:
|
|
137
|
+
- not_null
|
|
138
|
+
- dbt_utils.accepted_range:
|
|
139
|
+
min_value: 0
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Great Expectations (standalone)
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
context.sources.add_pandas_filesystem(name="orders", base_directory="data/")
|
|
146
|
+
validator = context.get_validator(batch_request=..., expectation_suite_name="orders")
|
|
147
|
+
validator.expect_column_values_to_not_be_null("order_id")
|
|
148
|
+
validator.expect_column_values_to_be_between("amount_usd", min_value=0)
|
|
149
|
+
validator.save_expectation_suite()
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
## Data Contracts
|
|
153
|
+
|
|
154
|
+
Define and enforce schema agreements between producers and consumers:
|
|
155
|
+
|
|
156
|
+
```yaml
|
|
157
|
+
# contract: orders-v1.yaml
|
|
158
|
+
name: orders
|
|
159
|
+
version: 1
|
|
160
|
+
schema:
|
|
161
|
+
- name: order_id
|
|
162
|
+
type: string
|
|
163
|
+
nullable: false
|
|
164
|
+
- name: amount_usd
|
|
165
|
+
type: number
|
|
166
|
+
nullable: false
|
|
167
|
+
sla:
|
|
168
|
+
freshness: 1h
|
|
169
|
+
availability: 99.5%
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Tools: **Soda Core**, **DataHub**, **OpenLineage** for lineage tracking.
|
|
173
|
+
|
|
174
|
+
## Common Pitfalls
|
|
175
|
+
|
|
176
|
+
- **No idempotency**: `INSERT` without `ON CONFLICT` or partition-replace duplicates rows on reruns
|
|
177
|
+
- **Wide tables**: adding columns to fact tables without versioning breaks downstream models
|
|
178
|
+
- **Missing backfill strategy**: pipelines without `catchup` support can't recover historical data gaps
|
|
179
|
+
- **Unmonitored late data**: streaming jobs without watermarks silently drop late events
|
|
180
|
+
- **Schema evolution without contracts**: upstream changes break downstream models silently
|
|
181
|
+
- **Full table scans**: no partition pruning on fact tables causes escalating costs
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/database.md — Database Patterns — Use this skill when designing schemas, writing migrations, optimizing queries, o
|
|
3
|
+
|
|
4
|
+
Use this skill when designing schemas, writing migrations, optimizing queries, or selecting databases. ## Schema Design
|
|
5
|
+
-->
|
|
6
|
+
# Database Patterns
|
|
7
|
+
|
|
8
|
+
Use this skill when designing schemas, writing migrations, optimizing queries, or selecting databases.
|
|
9
|
+
|
|
10
|
+
## Schema Design
|
|
11
|
+
|
|
12
|
+
### Relational
|
|
13
|
+
- Normalize to 3NF by default; denormalize deliberately for read performance
|
|
14
|
+
- Every table has a primary key; prefer UUID or ULID for distributed systems
|
|
15
|
+
- Foreign keys enforced at the database level
|
|
16
|
+
- Use appropriate column types; avoid stringly-typed data
|
|
17
|
+
- Add `created_at` and `updated_at` timestamps to all entities
|
|
18
|
+
- Soft delete (`deleted_at`) when audit trail matters
|
|
19
|
+
|
|
20
|
+
### Document (NoSQL)
|
|
21
|
+
- Model for access patterns, not entity relationships
|
|
22
|
+
- Embed data that is read together
|
|
23
|
+
- Reference data that changes independently or is shared
|
|
24
|
+
- Avoid deeply nested documents (>3 levels)
|
|
25
|
+
- Define a schema even when the database does not enforce one
|
|
26
|
+
|
|
27
|
+
## Indexing
|
|
28
|
+
|
|
29
|
+
### When to Index
|
|
30
|
+
- Columns in WHERE, JOIN, ORDER BY, and GROUP BY clauses
|
|
31
|
+
- Foreign key columns (often missed)
|
|
32
|
+
- Columns used in unique constraints
|
|
33
|
+
|
|
34
|
+
### Index Types
|
|
35
|
+
- B-tree: default, good for equality and range queries
|
|
36
|
+
- Hash: equality only, faster for point lookups
|
|
37
|
+
- GIN/GiST: full-text search, JSONB, arrays, geometric data
|
|
38
|
+
- Partial index: index only rows matching a condition
|
|
39
|
+
- Composite index: column order matters; most selective first
|
|
40
|
+
|
|
41
|
+
### Anti-Patterns
|
|
42
|
+
- Indexing every column (write overhead, storage waste)
|
|
43
|
+
- Unused indexes (monitor with `pg_stat_user_indexes` or equivalent)
|
|
44
|
+
- Missing index on foreign keys (causes slow joins and cascading deletes)
|
|
45
|
+
|
|
46
|
+
## Query Optimization
|
|
47
|
+
|
|
48
|
+
- Always EXPLAIN queries before and after optimization
|
|
49
|
+
- Avoid `SELECT *`; specify only needed columns
|
|
50
|
+
- Use pagination: LIMIT + OFFSET for small datasets, cursor-based for large
|
|
51
|
+
- Avoid N+1 queries: use JOINs, batch fetches, or DataLoader pattern
|
|
52
|
+
- Use CTEs for readability; materialize only when the planner needs help
|
|
53
|
+
- Parameterize queries; never interpolate user input
|
|
54
|
+
|
|
55
|
+
## Migrations
|
|
56
|
+
|
|
57
|
+
### Principles
|
|
58
|
+
- Every schema change is a versioned migration
|
|
59
|
+
- Migrations are forward-only in production; rollback via new forward migration
|
|
60
|
+
- Test migrations against a copy of production data
|
|
61
|
+
- Separate schema changes from data migrations
|
|
62
|
+
- Deploy additive schema changes before code changes
|
|
63
|
+
|
|
64
|
+
### Safe Practices
|
|
65
|
+
- Add columns as nullable or with defaults; never add NOT NULL without a default
|
|
66
|
+
- Add indexes concurrently (`CREATE INDEX CONCURRENTLY` in PostgreSQL)
|
|
67
|
+
- Rename via add-new, migrate-data, remove-old (three deployments)
|
|
68
|
+
- Drop columns only after all code references are removed
|
|
69
|
+
- Lock-free migrations for high-traffic tables
|
|
70
|
+
|
|
71
|
+
## Connection Management
|
|
72
|
+
|
|
73
|
+
- Use connection pooling (PgBouncer, HikariCP, or ORM-level)
|
|
74
|
+
- Set pool size based on: CPU cores x 2 + disk spindles (for PostgreSQL)
|
|
75
|
+
- Set connection timeout and idle timeout
|
|
76
|
+
- Monitor active, idle, and waiting connections
|
|
77
|
+
- Close connections cleanly; handle connection loss gracefully
|
|
78
|
+
|
|
79
|
+
## Backup and Recovery
|
|
80
|
+
|
|
81
|
+
- Automated daily backups with point-in-time recovery
|
|
82
|
+
- Test restore procedure quarterly
|
|
83
|
+
- Backup retention: minimum 30 days
|
|
84
|
+
- Encrypt backups at rest and in transit
|
|
85
|
+
- Store backups in a different region or provider
|
|
86
|
+
|
|
87
|
+
## Monitoring
|
|
88
|
+
|
|
89
|
+
- Slow query log with threshold (100ms typical)
|
|
90
|
+
- Connection pool utilization
|
|
91
|
+
- Replication lag (for read replicas)
|
|
92
|
+
- Table and index bloat
|
|
93
|
+
- Disk usage growth rate
|
|
94
|
+
- Lock contention and deadlock frequency
|
|
95
|
+
- Alert on: replication lag >10s, connection pool >80%, slow query spike
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/dependency-management.md — Dependency Management — Use this skill when managing package upgrades, resolving CVEs, or establishing l
|
|
3
|
+
|
|
4
|
+
Use this skill when managing package upgrades, resolving CVEs, or establishing lock file and versioning policy. ## Lock File Hygiene
|
|
5
|
+
-->
|
|
6
|
+
# Dependency Management
|
|
7
|
+
|
|
8
|
+
Use this skill when managing package upgrades, resolving CVEs, or establishing lock file and versioning policy.
|
|
9
|
+
|
|
10
|
+
## Lock File Hygiene
|
|
11
|
+
|
|
12
|
+
- Always commit lock files: `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `go.sum`, `Cargo.lock`.
|
|
13
|
+
- Never delete and regenerate a lock file without reviewing the full diff; regeneration silently upgrades all transitive dependencies.
|
|
14
|
+
- Review lock file diffs in PRs for unexpected transitive version changes — these are where supply chain attacks hide.
|
|
15
|
+
- Do not add lock files to `.gitignore`; they are the reproducibility contract for the build.
|
|
16
|
+
- A PR that changes only the lock file (no `package.json` change) warrants careful scrutiny.
|
|
17
|
+
|
|
18
|
+
## Safe Upgrade Strategies
|
|
19
|
+
|
|
20
|
+
| Tool | Trigger | Grouping | Auto-merge Policy |
|
|
21
|
+
|---|---|---|---|
|
|
22
|
+
| Dependabot | Daily/weekly schedule | One PR per package by default; group with `groups:` config | Patch: auto-merge if CI passes. Minor: batch weekly, manual review. Major: manual always. |
|
|
23
|
+
| Renovate | Configurable schedule | Group by ecosystem, semver range, or custom regex via `packageRules` | Auto-merge patch and minor with passing CI; use `stabilityDays: 3` to avoid merging on day of release |
|
|
24
|
+
|
|
25
|
+
Prefer Renovate for complex repos — its `rangeStrategy`, `grouping`, `stabilityDays`, and `automerge` controls are more expressive than Dependabot's.
|
|
26
|
+
|
|
27
|
+
Always require CI to pass before any auto-merge. Never auto-merge major version bumps.
|
|
28
|
+
|
|
29
|
+
## Transitive CVE Resolution
|
|
30
|
+
|
|
31
|
+
Run `npm audit --json` and parse the output. Each finding includes `severity`, `via` (the dependency chain), and `fixAvailable`.
|
|
32
|
+
|
|
33
|
+
Force a transitive version without upgrading the direct dependency:
|
|
34
|
+
|
|
35
|
+
In `package.json` (npm/pnpm):
|
|
36
|
+
```json
|
|
37
|
+
"overrides": {
|
|
38
|
+
"vulnerable-transitive-dep": ">=2.3.1"
|
|
39
|
+
}
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
In `package.json` (yarn):
|
|
43
|
+
```json
|
|
44
|
+
"resolutions": {
|
|
45
|
+
"vulnerable-transitive-dep": "2.3.1"
|
|
46
|
+
}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Risks of `npm audit fix --force`: it may install a semver-incompatible version and break the direct dependency. Always run your test suite after applying forced fixes. Prefer manual overrides over `--force` for production packages.
|
|
50
|
+
|
|
51
|
+
Verify the fix took effect: run `npm ls vulnerable-transitive-dep` and confirm the version in the tree.
|
|
52
|
+
|
|
53
|
+
## Version Pinning Policy
|
|
54
|
+
|
|
55
|
+
| Context | Policy | Reason |
|
|
56
|
+
|---|---|---|
|
|
57
|
+
| Production app | Pin exact versions (`"react": "18.2.0"`) | Reproducible builds; no surprise upgrades in prod |
|
|
58
|
+
| Library / published package | `^` for minor, `~` for patch | Consumers need flexibility; too-tight pins create version conflicts |
|
|
59
|
+
| Dev dependencies | `^` acceptable | Test/build tools rarely cause production regressions |
|
|
60
|
+
| Security-critical deps (auth, crypto, TLS) | Pin exact regardless of context | Breaking changes in security libs have outsize impact |
|
|
61
|
+
| CI tool versions (actions, runners) | Pin to SHA or exact tag | Unpinned actions are a supply chain attack surface |
|
|
62
|
+
|
|
63
|
+
## Vendoring Decisions
|
|
64
|
+
|
|
65
|
+
Vendor when:
|
|
66
|
+
- Builds must work offline or in air-gapped environments.
|
|
67
|
+
- Supply chain risk is critical and you need full control of the source.
|
|
68
|
+
- You have a stability SLA that cannot tolerate upstream changes.
|
|
69
|
+
|
|
70
|
+
Go: `go mod vendor` copies all dependencies into `vendor/`; pass `-mod=vendor` to enforce it.
|
|
71
|
+
|
|
72
|
+
pnpm strict mode: avoid `--shamefully-hoist`; it recreates the flat `node_modules` structure that allows phantom dependency bugs. Use strict mode by default and resolve missing hoisted deps explicitly.
|
|
73
|
+
|
|
74
|
+
Trade-offs: vendoring increases repo size and creates a maintenance burden to keep vendored copies patched. Weigh against the reproducibility and supply chain benefits for your threat model.
|
|
75
|
+
|
|
76
|
+
## Audit Workflows
|
|
77
|
+
|
|
78
|
+
| Ecosystem | Command | CI Integration |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| Node.js | `npm audit --audit-level=high` | Run on every PR; fail on HIGH+ |
|
|
81
|
+
| Python | `pip-audit` | Run after `pip install`; fail on CRITICAL |
|
|
82
|
+
| Rust | `cargo audit` | Run in CI; fail on CRITICAL |
|
|
83
|
+
| Go | `govulncheck ./...` | Run in CI; fail on CRITICAL |
|
|
84
|
+
|
|
85
|
+
Severity thresholds:
|
|
86
|
+
- CRITICAL: block CI, block merge, fix immediately.
|
|
87
|
+
- HIGH: block merge; fix before shipping.
|
|
88
|
+
- MEDIUM: open a tracked issue; fix within the current sprint.
|
|
89
|
+
- LOW: log; fix in a batch dependency upgrade PR.
|
|
90
|
+
|
|
91
|
+
Automate audit on every PR, not just on a schedule. A scheduled audit catches vulnerabilities too late relative to when the dependency was introduced.
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/devsecops.md — DevSecOps — Use this skill when integrating security into CI/CD pipelines, managing supply c
|
|
3
|
+
|
|
4
|
+
Use this skill when integrating security into CI/CD pipelines, managing supply chain risk, or automating security scanning. ## Pipeline Security Stages
|
|
5
|
+
-->
|
|
6
|
+
# DevSecOps
|
|
7
|
+
|
|
8
|
+
Use this skill when integrating security into CI/CD pipelines, managing supply chain risk, or automating security scanning.
|
|
9
|
+
|
|
10
|
+
## Pipeline Security Stages
|
|
11
|
+
|
|
12
|
+
### Pre-Commit
|
|
13
|
+
- Secret scanning: detect credentials before they reach the repo
|
|
14
|
+
- Tools: gitleaks, truffleHog, git-secrets
|
|
15
|
+
- Pre-commit hooks for fast local feedback
|
|
16
|
+
- Block commits containing patterns matching API keys, tokens, certificates
|
|
17
|
+
|
|
18
|
+
### Build
|
|
19
|
+
- SAST: static application security testing on source code
|
|
20
|
+
- Dependency scanning: check for known CVEs in direct and transitive dependencies
|
|
21
|
+
- License scanning: flag incompatible or unapproved licenses
|
|
22
|
+
- Container image scanning: vulnerabilities in base images and installed packages
|
|
23
|
+
- Fail the build on Critical findings; warn on High
|
|
24
|
+
|
|
25
|
+
### Test
|
|
26
|
+
- DAST: dynamic scanning against a running instance in CI
|
|
27
|
+
- API security testing: fuzz endpoints, test auth bypass
|
|
28
|
+
- Infrastructure-as-code scanning: misconfigurations in Terraform, CloudFormation, Helm
|
|
29
|
+
- Compliance-as-code: CIS benchmarks, custom policies
|
|
30
|
+
|
|
31
|
+
### Deploy
|
|
32
|
+
- Image signing and verification (cosign, Notary)
|
|
33
|
+
- Admission controllers enforce signed images only
|
|
34
|
+
- Runtime security policies (Falco, AppArmor, seccomp)
|
|
35
|
+
- Secrets injected from vault at deploy time, not baked into images
|
|
36
|
+
|
|
37
|
+
### Runtime
|
|
38
|
+
- Runtime application self-protection (RASP) where applicable
|
|
39
|
+
- Container runtime monitoring
|
|
40
|
+
- Network policy enforcement
|
|
41
|
+
- Continuous compliance scanning
|
|
42
|
+
|
|
43
|
+
## Supply Chain Security
|
|
44
|
+
|
|
45
|
+
### Dependency Management
|
|
46
|
+
- Pin exact versions in lock files
|
|
47
|
+
- Verify checksums of downloaded packages
|
|
48
|
+
- Use private registries or mirrors for critical dependencies
|
|
49
|
+
- Audit new dependencies before adding: maintainer reputation, download count, known issues
|
|
50
|
+
- Automate dependency updates with PR-based tools (Dependabot, Renovate)
|
|
51
|
+
|
|
52
|
+
### SBOM (Software Bill of Materials)
|
|
53
|
+
- Generate SBOM in CycloneDX or SPDX format
|
|
54
|
+
- Include in build artifacts
|
|
55
|
+
- Track components across all deployed services
|
|
56
|
+
- Automate SBOM comparison against vulnerability databases
|
|
57
|
+
|
|
58
|
+
### Container Security
|
|
59
|
+
- Use minimal base images (distroless, Alpine, scratch)
|
|
60
|
+
- Pin base image by digest, not tag
|
|
61
|
+
- Scan images at build and before deployment
|
|
62
|
+
- No secrets in image layers; use runtime injection
|
|
63
|
+
- Run containers as non-root with read-only filesystem where possible
|
|
64
|
+
|
|
65
|
+
## Secret Management in CI/CD
|
|
66
|
+
|
|
67
|
+
- Store secrets in the CI/CD platform's secret store or external vault
|
|
68
|
+
- Never print secrets in build logs
|
|
69
|
+
- Mask secret values in CI output
|
|
70
|
+
- Rotate CI/CD secrets on a schedule
|
|
71
|
+
- Audit secret access: who used which secret, when
|
|
72
|
+
|
|
73
|
+
## Policy as Code
|
|
74
|
+
|
|
75
|
+
- Define security policies in a machine-enforceable format
|
|
76
|
+
- OPA/Rego for Kubernetes admission control
|
|
77
|
+
- Sentinel or OPA for Terraform plan validation
|
|
78
|
+
- Custom policies for organization-specific requirements
|
|
79
|
+
- Version policies alongside code; review changes via PR
|
|
80
|
+
|
|
81
|
+
## Metrics
|
|
82
|
+
|
|
83
|
+
- Mean time to remediate (MTTR) by severity
|
|
84
|
+
- Percentage of builds with zero Critical/High findings
|
|
85
|
+
- Dependency freshness: age of oldest dependency
|
|
86
|
+
- SBOM coverage: percentage of services with current SBOM
|
|
87
|
+
- Secret rotation compliance rate
|
|
88
|
+
- Policy violation trend over time
|
|
89
|
+
|
|
90
|
+
## Incident Response Integration
|
|
91
|
+
|
|
92
|
+
- Security findings automatically create tickets in the issue tracker
|
|
93
|
+
- Critical findings trigger immediate notification
|
|
94
|
+
- Runbook for each finding category
|
|
95
|
+
- Post-incident review includes pipeline gap analysis
|
|
96
|
+
- Update pipeline rules based on incident learnings
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
skills/devops/git-workflow.md — Git Workflow — Use this skill when establishing branching strategies, commit conventions, or co
|
|
3
|
+
|
|
4
|
+
Use this skill when establishing branching strategies, commit conventions, or code integration practices. ## Branching Strategies
|
|
5
|
+
-->
|
|
6
|
+
# Git Workflow
|
|
7
|
+
|
|
8
|
+
Use this skill when establishing branching strategies, commit conventions, or code integration practices.
|
|
9
|
+
|
|
10
|
+
## Branching Strategies
|
|
11
|
+
|
|
12
|
+
### Trunk-Based Development
|
|
13
|
+
- Main branch is always deployable
|
|
14
|
+
- Short-lived feature branches (1-2 days max)
|
|
15
|
+
- Feature flags for incomplete work
|
|
16
|
+
- Best for: teams with strong CI/CD and automated testing
|
|
17
|
+
- Requires: comprehensive test suites, fast builds
|
|
18
|
+
|
|
19
|
+
### GitHub Flow
|
|
20
|
+
- `main` is always deployable
|
|
21
|
+
- Feature branches created from `main`
|
|
22
|
+
- Pull request for review; merge to `main` after approval
|
|
23
|
+
- Deploy from `main` after merge
|
|
24
|
+
- Best for: most teams, SaaS products
|
|
25
|
+
|
|
26
|
+
### GitFlow
|
|
27
|
+
- `main` for production, `develop` for integration
|
|
28
|
+
- Feature branches from `develop`, release branches for stabilization
|
|
29
|
+
- Hotfix branches from `main` for urgent fixes
|
|
30
|
+
- Best for: projects with scheduled releases
|
|
31
|
+
- Overhead is high; avoid unless release cadence demands it
|
|
32
|
+
|
|
33
|
+
## Commit Conventions
|
|
34
|
+
|
|
35
|
+
### Format
|
|
36
|
+
```
|
|
37
|
+
<type>(<scope>): <description>
|
|
38
|
+
|
|
39
|
+
<optional body>
|
|
40
|
+
|
|
41
|
+
<optional footer>
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Types
|
|
45
|
+
- `feat`: new feature
|
|
46
|
+
- `fix`: bug fix
|
|
47
|
+
- `refactor`: code change that neither fixes nor adds
|
|
48
|
+
- `docs`: documentation only
|
|
49
|
+
- `test`: adding or updating tests
|
|
50
|
+
- `chore`: build, CI, dependency updates
|
|
51
|
+
- `perf`: performance improvement
|
|
52
|
+
- `ci`: CI/CD configuration changes
|
|
53
|
+
|
|
54
|
+
### Rules
|
|
55
|
+
- Subject line: imperative mood, <72 characters
|
|
56
|
+
- Body: explain what and why, not how
|
|
57
|
+
- Footer: reference issues (`Closes #123`) or note breaking changes
|
|
58
|
+
- One logical change per commit
|
|
59
|
+
- Atomic commits: each commit should build and pass tests
|
|
60
|
+
|
|
61
|
+
## Pull Request Process
|
|
62
|
+
|
|
63
|
+
1. Branch from the latest base branch
|
|
64
|
+
2. Keep PRs small: <400 lines changed when possible
|
|
65
|
+
3. Self-review before requesting others
|
|
66
|
+
4. Write a description: what changed, why, how to test
|
|
67
|
+
5. Link related issues
|
|
68
|
+
6. All CI checks must pass before merge
|
|
69
|
+
7. At least one approval required
|
|
70
|
+
8. Squash merge for clean history; merge commit when individual commits matter
|
|
71
|
+
|
|
72
|
+
## Code Review Guidelines
|
|
73
|
+
|
|
74
|
+
- Review within 24 hours of request
|
|
75
|
+
- Focus on: correctness, security, maintainability, test coverage
|
|
76
|
+
- Use severity labels: Critical, High, Medium, Low
|
|
77
|
+
- Approve with minor comments; block only for Critical/High
|
|
78
|
+
- Praise good patterns; review is not only for finding problems
|
|
79
|
+
|
|
80
|
+
## Merge Conflict Resolution
|
|
81
|
+
|
|
82
|
+
- Rebase feature branch on latest base branch frequently
|
|
83
|
+
- Resolve conflicts locally, not in the merge UI
|
|
84
|
+
- Re-run tests after conflict resolution
|
|
85
|
+
- When in doubt, ask the original author
|
|
86
|
+
|
|
87
|
+
## Release Tagging
|
|
88
|
+
|
|
89
|
+
- Semantic versioning: MAJOR.MINOR.PATCH
|
|
90
|
+
- Tag releases on `main` after deployment verification
|
|
91
|
+
- Generate changelog from conventional commits
|
|
92
|
+
- Sign tags for authenticity when required
|
|
93
|
+
|
|
94
|
+
## Repository Hygiene
|
|
95
|
+
|
|
96
|
+
- Delete merged branches automatically
|
|
97
|
+
- Protect `main`: require PR, require CI, no force push
|
|
98
|
+
- `.gitignore` covers build artifacts, dependencies, environment files
|
|
99
|
+
- No large binaries in the repo; use Git LFS or external storage
|
|
100
|
+
- Commit messages reference issue trackers for traceability
|