arkaos 2.0.0 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +100 -74
- package/VERSION +1 -1
- package/bin/arkaos +1 -1
- package/config/constitution.yaml +4 -0
- package/config/hooks/user-prompt-submit-v2.sh +20 -38
- package/core/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/agents/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/agents/__pycache__/loader.cpython-313.pyc +0 -0
- package/core/agents/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/agents/__pycache__/validator.cpython-313.pyc +0 -0
- package/core/budget/__init__.py +6 -0
- package/core/budget/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/budget/__pycache__/manager.cpython-313.pyc +0 -0
- package/core/budget/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/budget/manager.py +193 -0
- package/core/budget/schema.py +82 -0
- package/core/conclave/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/advisor_db.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/display.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/matcher.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/persistence.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/profiler.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/prompts.cpython-313.pyc +0 -0
- package/core/conclave/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/governance/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/governance/__pycache__/constitution.cpython-313.pyc +0 -0
- package/core/obsidian/__init__.py +6 -0
- package/core/obsidian/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/obsidian/__pycache__/templates.cpython-313.pyc +0 -0
- package/core/obsidian/__pycache__/writer.cpython-313.pyc +0 -0
- package/core/obsidian/templates.py +76 -0
- package/core/obsidian/writer.py +148 -0
- package/core/orchestration/__init__.py +6 -0
- package/core/orchestration/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/orchestration/__pycache__/patterns.cpython-313.pyc +0 -0
- package/core/orchestration/__pycache__/protocol.cpython-313.pyc +0 -0
- package/core/orchestration/patterns.py +136 -0
- package/core/orchestration/protocol.py +96 -0
- package/core/registry/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/registry/__pycache__/generator.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/base.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/claude_code.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/codex_cli.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/cursor.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/gemini_cli.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/registry.cpython-313.pyc +0 -0
- package/core/runtime/__pycache__/subagent.cpython-313.pyc +0 -0
- package/core/specs/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/specs/__pycache__/manager.cpython-313.pyc +0 -0
- package/core/specs/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/squads/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/squads/__pycache__/loader.cpython-313.pyc +0 -0
- package/core/squads/__pycache__/registry.cpython-313.pyc +0 -0
- package/core/squads/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/synapse/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/synapse/__pycache__/cache.cpython-313.pyc +0 -0
- package/core/synapse/__pycache__/engine.cpython-313.pyc +0 -0
- package/core/synapse/__pycache__/layers.cpython-313.pyc +0 -0
- package/core/tasks/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/tasks/__pycache__/manager.cpython-313.pyc +0 -0
- package/core/tasks/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/tasks/schema.py +6 -0
- package/core/workflow/__pycache__/__init__.cpython-313.pyc +0 -0
- package/core/workflow/__pycache__/engine.cpython-313.pyc +0 -0
- package/core/workflow/__pycache__/loader.cpython-313.pyc +0 -0
- package/core/workflow/__pycache__/schema.cpython-313.pyc +0 -0
- package/core/workflow/engine.py +44 -0
- package/core/workflow/schema.py +1 -0
- package/departments/dev/skills/agent-design/SKILL.md +4 -0
- package/departments/dev/skills/agent-design/references/architecture-patterns.md +223 -0
- package/departments/dev/skills/ai-security/SKILL.md +4 -0
- package/departments/dev/skills/ai-security/references/prompt-injection-catalog.md +230 -0
- package/departments/dev/skills/ci-cd-pipeline/SKILL.md +4 -0
- package/departments/dev/skills/ci-cd-pipeline/references/github-actions-patterns.md +202 -0
- package/departments/dev/skills/db-schema/SKILL.md +4 -0
- package/departments/dev/skills/db-schema/references/indexing-strategy.md +197 -0
- package/departments/dev/skills/dependency-audit/SKILL.md +4 -0
- package/departments/dev/skills/dependency-audit/references/license-matrix.md +191 -0
- package/departments/dev/skills/incident/SKILL.md +4 -0
- package/departments/dev/skills/incident/references/severity-playbook.md +221 -0
- package/departments/dev/skills/observability/SKILL.md +4 -0
- package/departments/dev/skills/observability/references/slo-design.md +200 -0
- package/departments/dev/skills/rag-architect/SKILL.md +5 -0
- package/departments/dev/skills/rag-architect/references/chunking-strategies.md +129 -0
- package/departments/dev/skills/rag-architect/references/evaluation-guide.md +158 -0
- package/departments/dev/skills/red-team/SKILL.md +4 -0
- package/departments/dev/skills/red-team/references/mitre-attack-web.md +165 -0
- package/departments/dev/skills/security-audit/SKILL.md +4 -0
- package/departments/dev/skills/security-audit/references/owasp-2025-deep.md +409 -0
- package/departments/dev/skills/security-compliance/SKILL.md +117 -0
- package/departments/finance/skills/ciso-advisor/SKILL.md +4 -0
- package/departments/finance/skills/ciso-advisor/references/compliance-roadmap.md +172 -0
- package/departments/marketing/skills/programmatic-seo/SKILL.md +4 -0
- package/departments/marketing/skills/programmatic-seo/references/template-playbooks.md +289 -0
- package/departments/ops/skills/gdpr-compliance/SKILL.md +104 -0
- package/departments/ops/skills/iso27001/SKILL.md +113 -0
- package/departments/ops/skills/quality-management/SKILL.md +118 -0
- package/departments/ops/skills/risk-management/SKILL.md +120 -0
- package/departments/ops/skills/soc2-compliance/SKILL.md +120 -0
- package/departments/strategy/skills/cto-advisor/SKILL.md +4 -0
- package/departments/strategy/skills/cto-advisor/references/build-vs-buy-framework.md +190 -0
- package/installer/cli.js +13 -2
- package/installer/index.js +1 -2
- package/installer/migrate.js +123 -0
- package/installer/update.js +28 -15
- package/package.json +1 -1
- package/pyproject.toml +1 -1
- package/core/agents/__pycache__/registry_gen.cpython-313.pyc +0 -0
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
# GitHub Actions Best Practices — Deep Reference
|
|
2
|
+
|
|
3
|
+
> Companion to `ci-cd-pipeline/SKILL.md`. Patterns, pitfalls, and production-ready snippets.
|
|
4
|
+
|
|
5
|
+
## Caching Strategies by Ecosystem
|
|
6
|
+
|
|
7
|
+
| Ecosystem | Action | Cache Key | Cache Path | Restore Key Fallback |
|
|
8
|
+
|-----------|--------|-----------|------------|---------------------|
|
|
9
|
+
| Node (npm) | `actions/setup-node@v4` | `node-${{ runner.os }}-${{ hashFiles('package-lock.json') }}` | `~/.npm` | `node-${{ runner.os }}-` |
|
|
10
|
+
| Node (pnpm) | `pnpm/action-setup@v4` | `pnpm-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}` | `~/.pnpm-store` | `pnpm-${{ runner.os }}-` |
|
|
11
|
+
| Node (yarn) | `actions/setup-node@v4` | `yarn-${{ runner.os }}-${{ hashFiles('yarn.lock') }}` | `~/.yarn/cache` | `yarn-${{ runner.os }}-` |
|
|
12
|
+
| PHP | `shivammathur/setup-php@v2` | `php-${{ runner.os }}-${{ hashFiles('composer.lock') }}` | `vendor/` | `php-${{ runner.os }}-` |
|
|
13
|
+
| Python | `actions/setup-python@v5` | `pip-${{ runner.os }}-${{ hashFiles('requirements*.txt') }}` | `~/.cache/pip` | `pip-${{ runner.os }}-` |
|
|
14
|
+
| Go | `actions/setup-go@v5` | `go-${{ runner.os }}-${{ hashFiles('go.sum') }}` | `~/go/pkg/mod` | `go-${{ runner.os }}-` |
|
|
15
|
+
| Rust | `actions/cache@v4` | `cargo-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}` | `~/.cargo/registry, target/` | `cargo-${{ runner.os }}-` |
|
|
16
|
+
| Docker | `docker/build-push-action@v5` | Registry cache or `actions/cache` | `/tmp/.buildx-cache` | Use `mode=max` for layer reuse |
|
|
17
|
+
|
|
18
|
+
### Cache Size Limits
|
|
19
|
+
|
|
20
|
+
- Max 10 GB per repository (LRU eviction after 7 days unused)
|
|
21
|
+
- Single cache entry max: 10 GB
|
|
22
|
+
- If builds are slow, check cache hit rate in Actions UI
|
|
23
|
+
|
|
24
|
+
## Matrix Builds
|
|
25
|
+
|
|
26
|
+
### When to Use Matrix
|
|
27
|
+
|
|
28
|
+
| Situation | Matrix? | Why |
|
|
29
|
+
|-----------|---------|-----|
|
|
30
|
+
| Library supporting multiple runtimes | Yes | Must verify compatibility |
|
|
31
|
+
| Application with fixed runtime | No | One version in production, test that |
|
|
32
|
+
| Cross-platform CLI tool | Yes | OS-specific behavior matters |
|
|
33
|
+
| PR from feature branch | Minimal | Full matrix on main only (saves minutes) |
|
|
34
|
+
|
|
35
|
+
### Smart Matrix Pattern
|
|
36
|
+
|
|
37
|
+
```yaml
|
|
38
|
+
strategy:
|
|
39
|
+
fail-fast: true
|
|
40
|
+
matrix:
|
|
41
|
+
include:
|
|
42
|
+
- os: ubuntu-latest
|
|
43
|
+
node: 20
|
|
44
|
+
full-suite: true # Only run slow tests on primary
|
|
45
|
+
- os: ubuntu-latest
|
|
46
|
+
node: 18
|
|
47
|
+
full-suite: false
|
|
48
|
+
- os: windows-latest
|
|
49
|
+
node: 20
|
|
50
|
+
full-suite: false
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### Conditional Matrix Expansion
|
|
54
|
+
|
|
55
|
+
```yaml
|
|
56
|
+
# Full matrix on main, minimal on PRs
|
|
57
|
+
jobs:
|
|
58
|
+
test:
|
|
59
|
+
strategy:
|
|
60
|
+
matrix:
|
|
61
|
+
node: ${{ github.ref == 'refs/heads/main' && fromJson('[18, 20, 22]') || fromJson('[20]') }}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Reusable Workflows
|
|
65
|
+
|
|
66
|
+
### When to Extract
|
|
67
|
+
|
|
68
|
+
- Same CI logic in 3+ repositories
|
|
69
|
+
- Org-wide policy (security scanning, deployment gates)
|
|
70
|
+
- Shared infrastructure steps (Docker build, cloud deploy)
|
|
71
|
+
|
|
72
|
+
### Caller Pattern
|
|
73
|
+
|
|
74
|
+
```yaml
|
|
75
|
+
# .github/workflows/ci.yml (caller)
|
|
76
|
+
jobs:
|
|
77
|
+
ci:
|
|
78
|
+
uses: org/shared-workflows/.github/workflows/node-ci.yml@v1
|
|
79
|
+
with:
|
|
80
|
+
node-version: 20
|
|
81
|
+
run-e2e: true
|
|
82
|
+
secrets:
|
|
83
|
+
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
### Versioning Reusable Workflows
|
|
87
|
+
|
|
88
|
+
| Strategy | Stability | Flexibility |
|
|
89
|
+
|----------|-----------|------------|
|
|
90
|
+
| `@v1` (major tag) | High | Auto-get patches |
|
|
91
|
+
| `@v1.2.3` (exact) | Highest | Must bump manually |
|
|
92
|
+
| `@main` | Low | Always latest (dangerous) |
|
|
93
|
+
| `@sha` | Highest | Immutable, hard to read |
|
|
94
|
+
|
|
95
|
+
**Recommendation:** Use `@v1` (major tag), pin to SHA in security-critical pipelines.
|
|
96
|
+
|
|
97
|
+
## Secret Management
|
|
98
|
+
|
|
99
|
+
### Secret Scoping
|
|
100
|
+
|
|
101
|
+
| Scope | Access | Use Case |
|
|
102
|
+
|-------|--------|----------|
|
|
103
|
+
| Repository secret | This repo only | App-specific keys |
|
|
104
|
+
| Environment secret | Gated by environment rules | Production credentials |
|
|
105
|
+
| Organization secret | All/selected repos | Shared registry tokens |
|
|
106
|
+
|
|
107
|
+
### Secret Hygiene Checklist
|
|
108
|
+
|
|
109
|
+
- [ ] Never echo secrets in logs (`add-mask` for dynamic values)
|
|
110
|
+
- [ ] Use environment protection rules for production secrets
|
|
111
|
+
- [ ] Rotate secrets on a schedule (90 days max)
|
|
112
|
+
- [ ] Scope secrets to specific environments, not repo-wide
|
|
113
|
+
- [ ] Use OIDC instead of long-lived credentials for cloud providers
|
|
114
|
+
|
|
115
|
+
### OIDC for Cloud Providers (No Stored Secrets)
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
permissions:
|
|
119
|
+
id-token: write
|
|
120
|
+
contents: read
|
|
121
|
+
|
|
122
|
+
steps:
|
|
123
|
+
- uses: aws-actions/configure-aws-credentials@v4
|
|
124
|
+
with:
|
|
125
|
+
role-to-assume: arn:aws:iam::123456789:role/deploy
|
|
126
|
+
aws-region: eu-west-1
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Supported: AWS, GCP, Azure, HashiCorp Vault, Cloudflare.
|
|
130
|
+
|
|
131
|
+
## Deployment Environments
|
|
132
|
+
|
|
133
|
+
### Environment Protection Rules
|
|
134
|
+
|
|
135
|
+
| Rule | Purpose | When |
|
|
136
|
+
|------|---------|------|
|
|
137
|
+
| Required reviewers | Human approval gate | Production deploys |
|
|
138
|
+
| Wait timer | Cooldown between stages | Canary + production |
|
|
139
|
+
| Branch restriction | Only main can deploy | Prevent feature-branch deploys |
|
|
140
|
+
| Custom deployment rules | Third-party gates (Datadog, PagerDuty) | Status checks before deploy |
|
|
141
|
+
|
|
142
|
+
### Deployment Strategy Pattern
|
|
143
|
+
|
|
144
|
+
```yaml
|
|
145
|
+
deploy-staging:
|
|
146
|
+
environment: staging
|
|
147
|
+
needs: [test, build]
|
|
148
|
+
# Auto-deploy, no approval needed
|
|
149
|
+
|
|
150
|
+
deploy-production:
|
|
151
|
+
environment: production # Has required reviewers
|
|
152
|
+
needs: deploy-staging
|
|
153
|
+
if: github.ref == 'refs/heads/main'
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
## Common Pitfalls and Fixes
|
|
157
|
+
|
|
158
|
+
| Pitfall | Impact | Fix |
|
|
159
|
+
|---------|--------|-----|
|
|
160
|
+
| `actions/checkout` without `fetch-depth: 0` | Shallow clone breaks git history tools | Set `fetch-depth: 0` for changelog/versioning |
|
|
161
|
+
| Cache key without OS prefix | Cross-OS cache corruption | Always include `runner.os` in key |
|
|
162
|
+
| `pull_request_target` with checkout of PR code | Critical security: runs with write access on untrusted code | Use `pull_request` event instead |
|
|
163
|
+
| No `concurrency` group | Parallel deploys to same environment | Add `concurrency: { group: deploy-${{ github.ref }} }` |
|
|
164
|
+
| Artifact upload without retention | Storage bloat, hitting limits | Set `retention-days: 7` (or less) |
|
|
165
|
+
| `continue-on-error: true` hiding failures | Silent breakage | Use only for optional/informational steps |
|
|
166
|
+
| Running on `push` to all branches | Wasted minutes | Restrict to `main`, `develop`, use `pull_request` for branches |
|
|
167
|
+
| Hardcoded action versions (`@master`) | Breaking changes without notice | Pin to `@v1` or SHA |
|
|
168
|
+
|
|
169
|
+
## Workflow Performance Optimization
|
|
170
|
+
|
|
171
|
+
| Technique | Savings | Complexity |
|
|
172
|
+
|-----------|---------|------------|
|
|
173
|
+
| Dependency caching | 30-60% of install time | Low |
|
|
174
|
+
| Path-based triggers | Skip irrelevant jobs | Low |
|
|
175
|
+
| Parallel jobs (no `needs`) | Total time = slowest job | Low |
|
|
176
|
+
| Larger runners (org-hosted) | 2-4x faster builds | Medium |
|
|
177
|
+
| Docker layer caching | 50-80% of build time | Medium |
|
|
178
|
+
| Composite actions for shared steps | DRY, faster authoring | Medium |
|
|
179
|
+
| Self-hosted runners | No queue wait, persistent cache | High |
|
|
180
|
+
|
|
181
|
+
### Path-Based Triggers
|
|
182
|
+
|
|
183
|
+
```yaml
|
|
184
|
+
on:
|
|
185
|
+
push:
|
|
186
|
+
paths:
|
|
187
|
+
- 'src/**'
|
|
188
|
+
- 'tests/**'
|
|
189
|
+
- 'package.json'
|
|
190
|
+
paths-ignore:
|
|
191
|
+
- 'docs/**'
|
|
192
|
+
- '*.md'
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
## Security Hardening
|
|
196
|
+
|
|
197
|
+
- [ ] Pin all third-party actions to full SHA (not tag)
|
|
198
|
+
- [ ] Use `permissions` block with minimal scopes
|
|
199
|
+
- [ ] Never use `pull_request_target` with PR code checkout
|
|
200
|
+
- [ ] Enable `CODEOWNERS` for `.github/workflows/`
|
|
201
|
+
- [ ] Audit third-party actions before adoption
|
|
202
|
+
- [ ] Use `concurrency` to prevent parallel deploys
|
|
@@ -128,3 +128,7 @@ Surface these issues WITHOUT being asked:
|
|
|
128
128
|
### ERD
|
|
129
129
|
<Mermaid diagram>
|
|
130
130
|
```
|
|
131
|
+
|
|
132
|
+
## References
|
|
133
|
+
|
|
134
|
+
- [indexing-strategy.md](references/indexing-strategy.md) — B-tree vs hash vs GIN vs GiST, composite index ordering, partial indexes, covering indexes, EXPLAIN ANALYZE interpretation
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# Database Indexing Strategy — Deep Reference
|
|
2
|
+
|
|
3
|
+
> Companion to `db-schema/SKILL.md`. Index types, ordering rules, anti-patterns, and EXPLAIN interpretation.
|
|
4
|
+
|
|
5
|
+
## Index Types Comparison
|
|
6
|
+
|
|
7
|
+
| Type | Engine | Best For | Not For | Supports |
|
|
8
|
+
|------|--------|----------|---------|----------|
|
|
9
|
+
| **B-tree** | All | Range queries, sorting, equality, LIKE 'prefix%' | Full-text, array containment | `<, <=, =, >=, >, BETWEEN, IN, IS NULL, LIKE 'x%'` |
|
|
10
|
+
| **Hash** | PostgreSQL | Exact equality only | Range, sorting, partial match | `=` only |
|
|
11
|
+
| **GIN** | PostgreSQL | JSONB containment, arrays, full-text | Single-value equality, range | `@>, ?, ?&, ?|, @@, to_tsvector` |
|
|
12
|
+
| **GiST** | PostgreSQL | Geometric, range types, full-text (ranking) | Exact equality at scale | `&&, @>, <@, <<, >>`, nearest-neighbor |
|
|
13
|
+
| **BRIN** | PostgreSQL | Physically ordered data (timestamps, sequences) | Random access, updates | `<, <=, =, >=, >` (block-level) |
|
|
14
|
+
| **Clustered** | MySQL (InnoDB) | Primary key lookups, range scans | Only one per table | Table data physically ordered by PK |
|
|
15
|
+
| **Full-text** | MySQL/PostgreSQL | Natural language search | Exact match, sorting | `MATCH AGAINST` / `@@` |
|
|
16
|
+
|
|
17
|
+
## B-tree: The Default Workhorse
|
|
18
|
+
|
|
19
|
+
### When B-tree Wins
|
|
20
|
+
- WHERE with `=, <, >, <=, >=, BETWEEN, IN, IS NULL`
|
|
21
|
+
- ORDER BY (avoids sort operation)
|
|
22
|
+
- LIKE with fixed prefix (`LIKE 'abc%'`)
|
|
23
|
+
- GROUP BY on indexed columns
|
|
24
|
+
|
|
25
|
+
### When B-tree Loses
|
|
26
|
+
- LIKE with leading wildcard (`LIKE '%abc'`) -- cannot use index
|
|
27
|
+
- Array containment (`@>`) -- use GIN
|
|
28
|
+
- JSON path queries -- use GIN on JSONB
|
|
29
|
+
- Full-text search -- use GIN/GiST with tsvector
|
|
30
|
+
|
|
31
|
+
## Composite Index Ordering Rules
|
|
32
|
+
|
|
33
|
+
**The Left-Prefix Rule:** A composite index on `(A, B, C)` can serve queries on `(A)`, `(A, B)`, or `(A, B, C)` -- but NOT `(B)`, `(C)`, or `(B, C)`.
|
|
34
|
+
|
|
35
|
+
### Column Order Decision Tree
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
1. Put equality columns FIRST (WHERE status = 'active')
|
|
39
|
+
2. Put range/inequality column NEXT (WHERE created_at > '2024-01-01')
|
|
40
|
+
3. Put ORDER BY columns LAST (ORDER BY created_at DESC)
|
|
41
|
+
4. Only ONE range column benefits from the index
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Worked Example
|
|
45
|
+
|
|
46
|
+
```sql
|
|
47
|
+
-- Query:
|
|
48
|
+
SELECT * FROM orders
|
|
49
|
+
WHERE org_id = 5 AND status = 'shipped' AND created_at > '2024-01-01'
|
|
50
|
+
ORDER BY created_at DESC;
|
|
51
|
+
|
|
52
|
+
-- Optimal index:
|
|
53
|
+
CREATE INDEX idx_orders_lookup ON orders(org_id, status, created_at DESC);
|
|
54
|
+
-- equality(1) equality(2) range+sort(3)
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Common Ordering Mistakes
|
|
58
|
+
|
|
59
|
+
| Mistake | Why It Fails | Fix |
|
|
60
|
+
|---------|-------------|-----|
|
|
61
|
+
| Range column before equality | Index scan stops at first range | Equality columns first |
|
|
62
|
+
| ORDER BY column before WHERE | Cannot skip to ORDER BY | WHERE columns first |
|
|
63
|
+
| Too many columns (>4) | Diminishing returns, write overhead | Profile actual queries |
|
|
64
|
+
|
|
65
|
+
## Partial Indexes
|
|
66
|
+
|
|
67
|
+
Create an index on a subset of rows. Smaller index, faster scans.
|
|
68
|
+
|
|
69
|
+
```sql
|
|
70
|
+
-- Only index active records (90% of queries hit active)
|
|
71
|
+
CREATE INDEX idx_tasks_active ON tasks(project_id, priority)
|
|
72
|
+
WHERE deleted_at IS NULL;
|
|
73
|
+
|
|
74
|
+
-- Only index unprocessed jobs
|
|
75
|
+
CREATE INDEX idx_jobs_pending ON jobs(queue, created_at)
|
|
76
|
+
WHERE status = 'pending';
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### When to Use Partial Indexes
|
|
80
|
+
|
|
81
|
+
| Scenario | Benefit |
|
|
82
|
+
|----------|---------|
|
|
83
|
+
| Soft-deleted tables (query active only) | Index is 10-50% smaller |
|
|
84
|
+
| Status columns (query one status predominantly) | Skip irrelevant rows |
|
|
85
|
+
| Boolean flags (query TRUE only, small % of table) | Dramatic size reduction |
|
|
86
|
+
| Multi-tenant (one tenant is 80% of queries) | Focused index for hot tenant |
|
|
87
|
+
|
|
88
|
+
## Covering Indexes (Index-Only Scans)
|
|
89
|
+
|
|
90
|
+
Include all columns a query needs so the database never touches the table.
|
|
91
|
+
|
|
92
|
+
```sql
|
|
93
|
+
-- PostgreSQL: INCLUDE clause
|
|
94
|
+
CREATE INDEX idx_orders_covering ON orders(customer_id, status)
|
|
95
|
+
INCLUDE (total, created_at);
|
|
96
|
+
|
|
97
|
+
-- MySQL: all needed columns in the index
|
|
98
|
+
CREATE INDEX idx_orders_covering ON orders(customer_id, status, total, created_at);
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
### Covering Index Checklist
|
|
102
|
+
|
|
103
|
+
- [ ] Query SELECT list is fully covered by index
|
|
104
|
+
- [ ] WHERE clause columns are in the index key
|
|
105
|
+
- [ ] ORDER BY columns are in the index key
|
|
106
|
+
- [ ] INCLUDE columns are for SELECT only (not filterable)
|
|
107
|
+
|
|
108
|
+
## When NOT to Index
|
|
109
|
+
|
|
110
|
+
| Situation | Why |
|
|
111
|
+
|-----------|-----|
|
|
112
|
+
| Table has < 1,000 rows | Full scan is faster than index lookup |
|
|
113
|
+
| Column with < 5 distinct values on large table | Low selectivity, planner ignores index |
|
|
114
|
+
| Write-heavy table with rare reads | Index maintenance cost > read benefit |
|
|
115
|
+
| Columns only used in SELECT (not WHERE/ORDER) | Index does not help (unless covering) |
|
|
116
|
+
| Duplicate of existing composite prefix | `(A, B)` already covers `(A)` queries |
|
|
117
|
+
| Expression not matching index | `WHERE LOWER(email)` does not use index on `email` |
|
|
118
|
+
|
|
119
|
+
### Low Selectivity Exception
|
|
120
|
+
|
|
121
|
+
If a boolean/status column is in a composite index with a high-selectivity column, it CAN help:
|
|
122
|
+
```sql
|
|
123
|
+
-- Low selectivity alone (useless): INDEX(is_active)
|
|
124
|
+
-- High selectivity in composite (useful): INDEX(org_id, is_active)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
## EXPLAIN ANALYZE Interpretation
|
|
128
|
+
|
|
129
|
+
### Key Fields to Check
|
|
130
|
+
|
|
131
|
+
| Field | Good | Bad | Action |
|
|
132
|
+
|-------|------|-----|--------|
|
|
133
|
+
| **Seq Scan** | Small tables (<1K rows) | Large tables (>10K rows) | Add index on filter columns |
|
|
134
|
+
| **Index Scan** | Expected on filtered queries | N/A | Ideal for selective queries |
|
|
135
|
+
| **Index Only Scan** | Best case (no table access) | N/A | Covering index working |
|
|
136
|
+
| **Bitmap Heap Scan** | Multiple index conditions | N/A | Normal for OR/multi-index |
|
|
137
|
+
| **Sort** | Small result set | `Sort Method: external merge` | Add ORDER BY to index |
|
|
138
|
+
| **Hash Join** | Equal-size tables | Huge hash table (memory) | Check join column indexes |
|
|
139
|
+
| **Nested Loop** | Small outer, indexed inner | Large outer without index | Add index on inner join column |
|
|
140
|
+
| **Rows** (estimated vs actual) | Close match | 10x+ difference | Run ANALYZE, check statistics |
|
|
141
|
+
|
|
142
|
+
### Reading EXPLAIN Output
|
|
143
|
+
|
|
144
|
+
```
|
|
145
|
+
Execution time breakdown:
|
|
146
|
+
- actual time=X..Y X = time to first row, Y = time to all rows
|
|
147
|
+
- rows=N Actual rows returned by this node
|
|
148
|
+
- loops=N How many times this node executed
|
|
149
|
+
- Buffers: shared hit Reads from cache (good)
|
|
150
|
+
- Buffers: shared read Reads from disk (slow)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
### Red Flags in EXPLAIN
|
|
154
|
+
|
|
155
|
+
| Pattern | Problem | Fix |
|
|
156
|
+
|---------|---------|-----|
|
|
157
|
+
| `Seq Scan` on table > 10K rows | Missing index | Add index on WHERE columns |
|
|
158
|
+
| `Sort Method: external merge Disk` | Sort spills to disk | Increase `work_mem` or add index |
|
|
159
|
+
| `Rows Removed by Filter: 99000` (of 100K) | Non-selective scan | Add partial index or composite |
|
|
160
|
+
| `actual rows=100000` vs `rows=1` | Stale statistics | `ANALYZE table_name` |
|
|
161
|
+
| `Nested Loop` with `Seq Scan` inner | Missing join index | Index inner table's join column |
|
|
162
|
+
|
|
163
|
+
## Index Maintenance
|
|
164
|
+
|
|
165
|
+
### Regular Tasks
|
|
166
|
+
|
|
167
|
+
| Task | Frequency | Command (PostgreSQL) |
|
|
168
|
+
|------|-----------|---------------------|
|
|
169
|
+
| Check bloat | Weekly | `SELECT * FROM pg_stat_user_indexes WHERE idx_scan = 0` |
|
|
170
|
+
| Remove unused indexes | Monthly | Drop indexes with 0 scans over 30+ days |
|
|
171
|
+
| Reindex bloated indexes | Quarterly | `REINDEX INDEX CONCURRENTLY idx_name` |
|
|
172
|
+
| Update statistics | After bulk loads | `ANALYZE table_name` |
|
|
173
|
+
| Check index size vs table | Quarterly | Total index size should be < 2x table size |
|
|
174
|
+
|
|
175
|
+
### Identifying Unused Indexes
|
|
176
|
+
|
|
177
|
+
```sql
|
|
178
|
+
SELECT schemaname, relname, indexrelname, idx_scan, pg_size_pretty(pg_relation_size(indexrelid))
|
|
179
|
+
FROM pg_stat_user_indexes
|
|
180
|
+
WHERE idx_scan = 0 AND indexrelid NOT IN (
|
|
181
|
+
SELECT conindid FROM pg_constraint WHERE contype IN ('p', 'u')
|
|
182
|
+
)
|
|
183
|
+
ORDER BY pg_relation_size(indexrelid) DESC;
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
## MySQL vs PostgreSQL Index Differences
|
|
187
|
+
|
|
188
|
+
| Feature | PostgreSQL | MySQL (InnoDB) |
|
|
189
|
+
|---------|-----------|---------------|
|
|
190
|
+
| Covering indexes | INCLUDE clause | All columns in index key |
|
|
191
|
+
| Partial indexes | Yes (WHERE clause) | No native support |
|
|
192
|
+
| Expression indexes | Yes (`CREATE INDEX ON (LOWER(email))`) | Generated columns + index |
|
|
193
|
+
| GIN/GiST | Yes | No |
|
|
194
|
+
| BRIN | Yes | No |
|
|
195
|
+
| Clustered index | Optional (CLUSTER) | Always on PK (mandatory) |
|
|
196
|
+
| Index-only scan | Requires recent VACUUM | Automatic via clustered index |
|
|
197
|
+
| Concurrent creation | `CREATE INDEX CONCURRENTLY` | `ALGORITHM=INPLACE` (limited) |
|
|
@@ -116,3 +116,7 @@ Surface these issues WITHOUT being asked:
|
|
|
116
116
|
- Outdated: {count} ({major} major updates pending)
|
|
117
117
|
- Recommendation: {SAFE TO DEPLOY | FIX BEFORE DEPLOY | BLOCK DEPLOY}
|
|
118
118
|
```
|
|
119
|
+
|
|
120
|
+
## References
|
|
121
|
+
|
|
122
|
+
- [license-matrix.md](references/license-matrix.md) — Open source license compatibility matrix, copyleft obligations, commercial use implications, and dual-licensing strategies
|
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
# Open Source License Compatibility Matrix — Deep Reference
|
|
2
|
+
|
|
3
|
+
> Companion to `dependency-audit/SKILL.md`. License obligations, compatibility rules, and commercial implications.
|
|
4
|
+
|
|
5
|
+
## License Classification
|
|
6
|
+
|
|
7
|
+
| License | Type | OSI Approved | Copyleft Strength |
|
|
8
|
+
|---------|------|-------------|-------------------|
|
|
9
|
+
| MIT | Permissive | Yes | None |
|
|
10
|
+
| ISC | Permissive | Yes | None |
|
|
11
|
+
| BSD-2-Clause | Permissive | Yes | None |
|
|
12
|
+
| BSD-3-Clause | Permissive | Yes | None |
|
|
13
|
+
| Apache-2.0 | Permissive | Yes | None (patent grant) |
|
|
14
|
+
| MPL-2.0 | Weak copyleft | Yes | File-level |
|
|
15
|
+
| LGPL-2.1 | Weak copyleft | Yes | Library-level |
|
|
16
|
+
| LGPL-3.0 | Weak copyleft | Yes | Library-level |
|
|
17
|
+
| GPL-2.0 | Strong copyleft | Yes | Entire work |
|
|
18
|
+
| GPL-3.0 | Strong copyleft | Yes | Entire work |
|
|
19
|
+
| AGPL-3.0 | Network copyleft | Yes | Entire work + network use |
|
|
20
|
+
| SSPL | Source-available | No | Entire service stack |
|
|
21
|
+
| BSL 1.1 | Source-available | No | Time-delayed open source |
|
|
22
|
+
| Proprietary | Proprietary | No | Full restriction |
|
|
23
|
+
|
|
24
|
+
## Compatibility Matrix
|
|
25
|
+
|
|
26
|
+
Can you combine code under these licenses in the SAME project?
|
|
27
|
+
|
|
28
|
+
| Dependency License | Your Project: MIT | Your Project: Apache-2.0 | Your Project: GPL-3.0 | Your Project: Proprietary |
|
|
29
|
+
|-------------------|:-:|:-:|:-:|:-:|
|
|
30
|
+
| MIT | OK | OK | OK | OK |
|
|
31
|
+
| BSD-2/3 | OK | OK | OK | OK |
|
|
32
|
+
| Apache-2.0 | OK | OK | OK | OK |
|
|
33
|
+
| MPL-2.0 | OK (keep MPL files) | OK (keep MPL files) | OK | OK (keep MPL files) |
|
|
34
|
+
| LGPL-2.1/3.0 | OK (dynamic link) | OK (dynamic link) | OK | OK (dynamic link only) |
|
|
35
|
+
| GPL-2.0 | NO | NO | OK (if "or later") | NO |
|
|
36
|
+
| GPL-3.0 | NO | NO | OK | NO |
|
|
37
|
+
| AGPL-3.0 | NO | NO | OK (becomes AGPL) | NO |
|
|
38
|
+
|
|
39
|
+
**Key:** OK = compatible. NO = license violation.
|
|
40
|
+
|
|
41
|
+
## Obligations by License
|
|
42
|
+
|
|
43
|
+
### Permissive Licenses (MIT, BSD, ISC, Apache-2.0)
|
|
44
|
+
|
|
45
|
+
| Obligation | MIT | BSD-2 | BSD-3 | Apache-2.0 |
|
|
46
|
+
|------------|:---:|:-----:|:-----:|:----------:|
|
|
47
|
+
| Include copyright notice | Yes | Yes | Yes | Yes |
|
|
48
|
+
| Include license text | Yes | Yes | Yes | Yes |
|
|
49
|
+
| State changes | No | No | No | Yes |
|
|
50
|
+
| Patent grant | No | No | No | Yes |
|
|
51
|
+
| No endorsement clause | No | No | Yes | No |
|
|
52
|
+
| NOTICE file preservation | No | No | No | Yes |
|
|
53
|
+
|
|
54
|
+
### Copyleft Licenses (MPL, LGPL, GPL, AGPL)
|
|
55
|
+
|
|
56
|
+
| Obligation | MPL-2.0 | LGPL-3.0 | GPL-3.0 | AGPL-3.0 |
|
|
57
|
+
|------------|:-------:|:--------:|:-------:|:--------:|
|
|
58
|
+
| Source code of modified files | Yes | Yes | Yes | Yes |
|
|
59
|
+
| Source code of entire work | No | No | Yes | Yes |
|
|
60
|
+
| Network use triggers copyleft | No | No | No | Yes |
|
|
61
|
+
| Allow reverse engineering | No | Yes | Yes | Yes |
|
|
62
|
+
| Provide installation info | No | No | Yes | Yes |
|
|
63
|
+
| Anti-tivoization clause | No | No | Yes | Yes |
|
|
64
|
+
|
|
65
|
+
## Commercial Use Decision Tree
|
|
66
|
+
|
|
67
|
+
```
|
|
68
|
+
START: Is the dependency's license identified?
|
|
69
|
+
NO --> STOP. Do not use. Unknown license = all rights reserved.
|
|
70
|
+
YES --> Is it permissive (MIT, BSD, ISC, Apache)?
|
|
71
|
+
YES --> SAFE for commercial use. Include attribution.
|
|
72
|
+
NO --> Is it weak copyleft (MPL, LGPL)?
|
|
73
|
+
YES --> SAFE if:
|
|
74
|
+
MPL: Keep modified MPL files open-source
|
|
75
|
+
LGPL: Dynamic linking only (no static linking in proprietary)
|
|
76
|
+
NO --> Is it strong copyleft (GPL)?
|
|
77
|
+
YES --> NOT SAFE for proprietary. Your entire project becomes GPL.
|
|
78
|
+
Exception: GPL with Classpath Exception (like OpenJDK)
|
|
79
|
+
NO --> Is it network copyleft (AGPL)?
|
|
80
|
+
YES --> NOT SAFE. Even SaaS use triggers copyleft.
|
|
81
|
+
NO --> Is it source-available (SSPL, BSL)?
|
|
82
|
+
YES --> Review specific terms. Usually NOT safe for competing services.
|
|
83
|
+
NO --> Legal review required.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Common Ecosystem License Risks
|
|
87
|
+
|
|
88
|
+
### Node.js (npm)
|
|
89
|
+
|
|
90
|
+
| Risk Pattern | Example | Action |
|
|
91
|
+
|-------------|---------|--------|
|
|
92
|
+
| Transitive GPL dependency | `node-sass` (deprecated, had GPL deps) | Audit full tree with `license-checker` |
|
|
93
|
+
| License field missing in package.json | Small utilities | Check source repo manually |
|
|
94
|
+
| "SEE LICENSE IN" custom file | Some enterprise libs | Read the actual file |
|
|
95
|
+
| Dual license with one GPL option | Some databases | Verify you are using the permissive option |
|
|
96
|
+
|
|
97
|
+
### PHP (Composer)
|
|
98
|
+
|
|
99
|
+
| Risk Pattern | Example | Action |
|
|
100
|
+
|-------------|---------|--------|
|
|
101
|
+
| Laravel itself is MIT | Framework code | Safe |
|
|
102
|
+
| Packages wrapping GPL tools | ImageMagick bindings | Check if wrapper license differs from tool |
|
|
103
|
+
| WordPress ecosystem (GPL-2.0+) | Themes, plugins | All derivatives must be GPL |
|
|
104
|
+
|
|
105
|
+
### Python (pip)
|
|
106
|
+
|
|
107
|
+
| Risk Pattern | Example | Action |
|
|
108
|
+
|-------------|---------|--------|
|
|
109
|
+
| PSF license (Python itself) | Standard lib | Safe, permissive |
|
|
110
|
+
| LGPL in scientific computing | Some NumPy deps historically | Dynamic linking usually fine |
|
|
111
|
+
| GPL in CLI tools used via subprocess | `pandoc` (GPL) | Subprocess call != linking (generally safe) |
|
|
112
|
+
|
|
113
|
+
## Dual Licensing Strategies
|
|
114
|
+
|
|
115
|
+
| Strategy | How It Works | Example |
|
|
116
|
+
|----------|-------------|---------|
|
|
117
|
+
| Open core | AGPL for community, proprietary for enterprise | MongoDB (was AGPL, now SSPL) |
|
|
118
|
+
| GPL + commercial | GPL for open use, paid license for proprietary embedding | MySQL, Qt |
|
|
119
|
+
| MIT + paid support | MIT code, paid for SLA/support/features | Many SaaS tools |
|
|
120
|
+
| BSL time-delay | Source-available now, becomes open source after N years | CockroachDB, Sentry |
|
|
121
|
+
|
|
122
|
+
## License Audit Automation
|
|
123
|
+
|
|
124
|
+
### npm
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
# List all licenses
|
|
128
|
+
npx license-checker --summary
|
|
129
|
+
|
|
130
|
+
# Fail CI on copyleft
|
|
131
|
+
npx license-checker --failOn "GPL-2.0;GPL-3.0;AGPL-3.0"
|
|
132
|
+
|
|
133
|
+
# Export for legal review
|
|
134
|
+
npx license-checker --csv --out licenses.csv
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### Composer (PHP)
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
# List licenses
|
|
141
|
+
composer licenses
|
|
142
|
+
|
|
143
|
+
# Detailed with dependencies
|
|
144
|
+
composer licenses --format=json
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
### pip (Python)
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
pip-licenses --format=table --with-urls
|
|
151
|
+
pip-licenses --fail-on="GPL-2.0;GPL-3.0;AGPL-3.0"
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
## License Change Risks
|
|
155
|
+
|
|
156
|
+
| Event | Risk | Action |
|
|
157
|
+
|-------|------|--------|
|
|
158
|
+
| Maintainer relicenses (MIT -> BSL) | Future versions restricted | Pin to last MIT version |
|
|
159
|
+
| CLA-based project changes license | All contributor code affected | Monitor project governance |
|
|
160
|
+
| Acquisition changes license | New owner may restrict | Evaluate fork options |
|
|
161
|
+
| License removed from package | Defaults to all-rights-reserved | Contact maintainer, find alternative |
|
|
162
|
+
|
|
163
|
+
### Notable License Changes (Precedents)
|
|
164
|
+
|
|
165
|
+
| Project | From | To | Year | Impact |
|
|
166
|
+
|---------|------|----|------|--------|
|
|
167
|
+
| Elasticsearch | Apache-2.0 | SSPL | 2021 | AWS forked as OpenSearch |
|
|
168
|
+
| Redis modules | BSD | SSPL | 2024 | Community forks (Valkey) |
|
|
169
|
+
| HashiCorp (Terraform) | MPL-2.0 | BSL 1.1 | 2023 | OpenTofu fork |
|
|
170
|
+
| MongoDB | AGPL-3.0 | SSPL | 2018 | Cloud providers stopped offering |
|
|
171
|
+
|
|
172
|
+
## NOTICE File Template
|
|
173
|
+
|
|
174
|
+
When using Apache-2.0 licensed code, preserve and extend the NOTICE file:
|
|
175
|
+
|
|
176
|
+
```
|
|
177
|
+
This product includes software developed by [Original Author].
|
|
178
|
+
Licensed under the Apache License, Version 2.0.
|
|
179
|
+
|
|
180
|
+
Modifications by [Your Company], [Year].
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
## Quick Reference: "Can I Use This?"
|
|
184
|
+
|
|
185
|
+
| Your Project Type | Safe Licenses | Caution | Blocked |
|
|
186
|
+
|-------------------|--------------|---------|---------|
|
|
187
|
+
| Proprietary SaaS | MIT, BSD, ISC, Apache-2.0 | MPL, LGPL (check linking) | GPL, AGPL, SSPL |
|
|
188
|
+
| Proprietary desktop app | MIT, BSD, ISC, Apache-2.0 | LGPL (dynamic link only) | GPL, AGPL |
|
|
189
|
+
| Open source (MIT) | MIT, BSD, ISC, Apache-2.0 | MPL | GPL, AGPL (would change your license) |
|
|
190
|
+
| Open source (GPL-3.0) | All OSI-approved | SSPL, BSL | Proprietary, GPL-2.0-only |
|
|
191
|
+
| Internal tool (never distributed) | All (copyleft not triggered) | AGPL (network use triggers) | SSPL (service use triggers) |
|
|
@@ -123,3 +123,7 @@ Surface these issues WITHOUT being asked:
|
|
|
123
123
|
1. [Key takeaway]
|
|
124
124
|
2. [Key takeaway]
|
|
125
125
|
```
|
|
126
|
+
|
|
127
|
+
## References
|
|
128
|
+
|
|
129
|
+
- [severity-playbook.md](references/severity-playbook.md) — SEV1-4 definitions, escalation paths, communication templates, and PIR checklist
|