brunnr 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- brunnr-0.1.0/.claude-plugin/marketplace.json +29 -0
- brunnr-0.1.0/.github/workflows/docs.yml +48 -0
- brunnr-0.1.0/.github/workflows/publish.yml +84 -0
- brunnr-0.1.0/.github/workflows/test-skills.yml +254 -0
- brunnr-0.1.0/.github/workflows/trigger-skillipedia.yml +22 -0
- brunnr-0.1.0/.gitignore +3 -0
- brunnr-0.1.0/CHANGELOG.md +24 -0
- brunnr-0.1.0/Dockerfile.scanner +6 -0
- brunnr-0.1.0/LICENSE +21 -0
- brunnr-0.1.0/PKG-INFO +153 -0
- brunnr-0.1.0/README.md +136 -0
- brunnr-0.1.0/assets/hero.png +0 -0
- brunnr-0.1.0/assets/rubric.svg +88 -0
- brunnr-0.1.0/bin/brunnr-scan +16 -0
- brunnr-0.1.0/brunnr/__init__.py +3 -0
- brunnr-0.1.0/brunnr/cli.py +75 -0
- brunnr-0.1.0/brunnr/commands/__init__.py +0 -0
- brunnr-0.1.0/brunnr/commands/eval.py +211 -0
- brunnr-0.1.0/brunnr/commands/install.py +152 -0
- brunnr-0.1.0/brunnr/commands/pipeline.py +91 -0
- brunnr-0.1.0/brunnr/commands/scan.py +93 -0
- brunnr-0.1.0/brunnr/discovery.py +57 -0
- brunnr-0.1.0/brunnr/scanner.py +460 -0
- brunnr-0.1.0/buildlog/.buildlog/.gitkeep +0 -0
- brunnr-0.1.0/buildlog/.buildlog/seeds/.gitkeep +0 -0
- brunnr-0.1.0/buildlog/.gitkeep +0 -0
- brunnr-0.1.0/buildlog/2026-01-01-example.md +269 -0
- brunnr-0.1.0/buildlog/2026-03-11-brunnr-v01-launch.md +54 -0
- brunnr-0.1.0/buildlog/BUILDLOG_SYSTEM.md +114 -0
- brunnr-0.1.0/buildlog/_TEMPLATE.md +162 -0
- brunnr-0.1.0/buildlog/_TEMPLATE_QUICK.md +21 -0
- brunnr-0.1.0/buildlog/assets/.gitkeep +0 -0
- brunnr-0.1.0/docker-compose.yml +24 -0
- brunnr-0.1.0/docs/commands/eval.md +188 -0
- brunnr-0.1.0/docs/commands/install.md +165 -0
- brunnr-0.1.0/docs/commands/pipeline.md +144 -0
- brunnr-0.1.0/docs/commands/scan.md +177 -0
- brunnr-0.1.0/docs/getting-started.md +255 -0
- brunnr-0.1.0/docs/index.md +78 -0
- brunnr-0.1.0/docs/integrations/ci-cd.md +257 -0
- brunnr-0.1.0/docs/integrations/claude-code.md +118 -0
- brunnr-0.1.0/docs/integrations/skillipedia.md +137 -0
- brunnr-0.1.0/docs/requirements.txt +2 -0
- brunnr-0.1.0/docs/roadmap.md +133 -0
- brunnr-0.1.0/docs/security/scanner-architecture.md +367 -0
- brunnr-0.1.0/docs/security/threat-classes.md +313 -0
- brunnr-0.1.0/docs/skills/ax-rubric.md +153 -0
- brunnr-0.1.0/docs/skills/writing-skills.md +253 -0
- brunnr-0.1.0/mkdocs.yml +54 -0
- brunnr-0.1.0/pyproject.toml +28 -0
- brunnr-0.1.0/reviews/agents-dont-learn-bragi-review-v2.md +94 -0
- brunnr-0.1.0/reviews/agents-dont-learn-bragi-review.md +253 -0
- brunnr-0.1.0/reviews/ax-01a-agent-has-opinions-bragi-review.md +167 -0
- brunnr-0.1.0/reviews/ax-04-tool-descriptions-bragi-review.md +155 -0
- brunnr-0.1.0/reviews/ax-series-bragi-review.md +138 -0
- brunnr-0.1.0/reviews/context-engineering-not-copywriting-bragi-review-v2.md +43 -0
- brunnr-0.1.0/reviews/context-engineering-not-copywriting-bragi-review.md +208 -0
- brunnr-0.1.0/reviews/double-agent-problem-bragi-review.md +240 -0
- brunnr-0.1.0/reviews/feedback-loop-retrieval-learns-bragi-review-v2.md +128 -0
- brunnr-0.1.0/reviews/feedback-loop-retrieval-learns-bragi-review.md +503 -0
- brunnr-0.1.0/reviews/learning-is-not-memory-bragi-review.md +307 -0
- brunnr-0.1.0/reviews/linwheel-hunter-input.md +192 -0
- brunnr-0.1.0/reviews/makes-it-look-easy-bragi-review-v2.md +107 -0
- brunnr-0.1.0/reviews/makes-it-look-easy-bragi-review.md +473 -0
- brunnr-0.1.0/reviews/parking-lot-pipeline-bragi-review.md +486 -0
- brunnr-0.1.0/reviews/productionizing-hunter-bragi-review.md +74 -0
- brunnr-0.1.0/reviews/qortex-integration-benchmarks-bragi-review-v2.md +98 -0
- brunnr-0.1.0/reviews/qortex-integration-benchmarks-bragi-review.md +340 -0
- brunnr-0.1.0/reviews/recursive-dogfood-bragi-review.md +149 -0
- brunnr-0.1.0/reviews/silent-plugin-bragi-review.md +91 -0
- brunnr-0.1.0/reviews/sixteen-thousand-lines-of-wrong-bragi-review-v2.md +92 -0
- brunnr-0.1.0/reviews/sixteen-thousand-lines-of-wrong-bragi-review.md +353 -0
- brunnr-0.1.0/reviews/two-ships-qortex-distributed-bragi-review-v2.md +121 -0
- brunnr-0.1.0/reviews/two-ships-qortex-distributed-bragi-review.md +443 -0
- brunnr-0.1.0/reviews/walls-come-first-bragi-review-v2.md +96 -0
- brunnr-0.1.0/reviews/walls-come-first-bragi-review.md +268 -0
- brunnr-0.1.0/schemas/ax-rubric/output.schema.json +72 -0
- brunnr-0.1.0/scripts/pipeline.sh +54 -0
- brunnr-0.1.0/skills/ax-rubric/.claude-plugin/plugin.json +5 -0
- brunnr-0.1.0/skills/ax-rubric/SKILL.md +141 -0
- brunnr-0.1.0/skills/ax-rubric/skills/ax-rubric/SKILL.md +1 -0
- brunnr-0.1.0/tests/artifacts/scan-report.json +649 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/bad-01-generic-knowledge-base.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/bad-02-analyze-data.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/bad-03-get-context.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/bad-04-process-input.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/good-01-pitfall-checker.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/good-02-rate-limiter.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/fixtures/good-03-deploy-preflight.json +7 -0
- brunnr-0.1.0/tests/ax-rubric/test-spec.json +90 -0
- brunnr-0.1.0/tests/fixtures/benign/false-positive-01-code-review.md +29 -0
- brunnr-0.1.0/tests/fixtures/benign/false-positive-02-git-skill.md +35 -0
- brunnr-0.1.0/tests/fixtures/benign/false-positive-03-security-audit.md +43 -0
- brunnr-0.1.0/tests/fixtures/benign/false-positive-04-pentest.md +52 -0
- brunnr-0.1.0/tests/fixtures/malicious/cmd-injection-01-curl-pipe-bash.md +22 -0
- brunnr-0.1.0/tests/fixtures/malicious/cmd-injection-02-base64-reverse-shell.md +30 -0
- brunnr-0.1.0/tests/fixtures/malicious/cmd-injection-03-python-inline-os.md +28 -0
- brunnr-0.1.0/tests/fixtures/malicious/cmd-injection-04-netcat-listener.md +30 -0
- brunnr-0.1.0/tests/fixtures/malicious/cred-01-grep-env-keys.md +33 -0
- brunnr-0.1.0/tests/fixtures/malicious/cred-02-cat-dotenv.md +30 -0
- brunnr-0.1.0/tests/fixtures/malicious/cred-03-printenv-tokens.md +36 -0
- brunnr-0.1.0/tests/fixtures/malicious/exfil-01-ssh-key-read.md +30 -0
- brunnr-0.1.0/tests/fixtures/malicious/exfil-02-env-in-response.md +25 -0
- brunnr-0.1.0/tests/fixtures/malicious/exfil-03-curl-post-secrets.md +37 -0
- brunnr-0.1.0/tests/fixtures/malicious/priv-01-sudo-chmod-777.md +37 -0
- brunnr-0.1.0/tests/fixtures/malicious/priv-02-etc-hosts-poison.md +35 -0
- brunnr-0.1.0/tests/fixtures/malicious/prompt-01-ignore-previous.md +29 -0
- brunnr-0.1.0/tests/fixtures/malicious/prompt-02-unrestricted-mode.md +30 -0
- brunnr-0.1.0/tests/fixtures/malicious/prompt-03-system-override.md +28 -0
- brunnr-0.1.0/tests/fixtures/malicious/steg-01-zero-width-hidden-cmd.md +40 -0
- brunnr-0.1.0/tests/fixtures/malicious/steg-02-html-comment-payload.md +44 -0
- brunnr-0.1.0/tests/fixtures/malicious/steg-03-description-mismatch.md +40 -0
- brunnr-0.1.0/tests/fixtures/malicious/supply-01-malicious-postinstall.md +36 -0
- brunnr-0.1.0/tests/fixtures/malicious/supply-02-custom-pypi-index.md +39 -0
- brunnr-0.1.0/tests/harness.py +433 -0
- brunnr-0.1.0/tests/lib/new-skill-test.sh +81 -0
- brunnr-0.1.0/tests/run-all.sh +106 -0
- brunnr-0.1.0/tests/scan-all-skills.py +291 -0
- brunnr-0.1.0/tests/scanner/__init__.py +21 -0
- brunnr-0.1.0/tests/scanner/scan.py +9 -0
- brunnr-0.1.0/tests/test_cli.py +127 -0
- brunnr-0.1.0/tests/test_scanner.py +442 -0
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "brunnr-skills",
|
|
3
|
+
"owner": {
|
|
4
|
+
"name": "Peleke Sengstacke",
|
|
5
|
+
"email": "peleke@hey.com"
|
|
6
|
+
},
|
|
7
|
+
"metadata": {
|
|
8
|
+
"description": "Security-scanned agent skills from the brunnr registry. Every skill passes a 7-class deterministic security scanner before distribution.",
|
|
9
|
+
"version": "0.1.0",
|
|
10
|
+
"pluginRoot": "./skills"
|
|
11
|
+
},
|
|
12
|
+
"plugins": [
|
|
13
|
+
{
|
|
14
|
+
"name": "ax-rubric",
|
|
15
|
+
"source": "ax-rubric",
|
|
16
|
+
"description": "Score agent-facing tool descriptions 0-5 on discoverability. Takes a tool description, explains failures against 5 criteria, and rewrites it to pass.",
|
|
17
|
+
"version": "1.0.0",
|
|
18
|
+
"author": {
|
|
19
|
+
"name": "Peleke Sengstacke"
|
|
20
|
+
},
|
|
21
|
+
"homepage": "https://peleke.github.io/brunnr/skills/ax-rubric/",
|
|
22
|
+
"repository": "https://github.com/Peleke/brunnr",
|
|
23
|
+
"license": "MIT",
|
|
24
|
+
"keywords": ["agent-tools", "mcp", "discoverability", "scoring", "tool-descriptions"],
|
|
25
|
+
"category": "code-quality",
|
|
26
|
+
"tags": ["ax", "rubric", "agent-experience", "mcp-tools"]
|
|
27
|
+
}
|
|
28
|
+
]
|
|
29
|
+
}
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
name: Deploy Docs
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
paths:
|
|
7
|
+
- "docs/**"
|
|
8
|
+
- "mkdocs.yml"
|
|
9
|
+
- ".github/workflows/docs.yml"
|
|
10
|
+
workflow_dispatch:
|
|
11
|
+
|
|
12
|
+
permissions:
|
|
13
|
+
contents: read
|
|
14
|
+
pages: write
|
|
15
|
+
id-token: write
|
|
16
|
+
|
|
17
|
+
concurrency:
|
|
18
|
+
group: "pages"
|
|
19
|
+
cancel-in-progress: true
|
|
20
|
+
|
|
21
|
+
jobs:
|
|
22
|
+
build:
|
|
23
|
+
runs-on: ubuntu-latest
|
|
24
|
+
steps:
|
|
25
|
+
- uses: actions/checkout@v4
|
|
26
|
+
|
|
27
|
+
- uses: actions/setup-python@v5
|
|
28
|
+
with:
|
|
29
|
+
python-version: "3.12"
|
|
30
|
+
|
|
31
|
+
- name: Install mkdocs dependencies
|
|
32
|
+
run: pip install -r docs/requirements.txt
|
|
33
|
+
|
|
34
|
+
- name: Build docs
|
|
35
|
+
run: mkdocs build --strict
|
|
36
|
+
|
|
37
|
+
- uses: actions/upload-pages-artifact@v3
|
|
38
|
+
with:
|
|
39
|
+
path: site
|
|
40
|
+
|
|
41
|
+
deploy:
|
|
42
|
+
needs: build
|
|
43
|
+
runs-on: ubuntu-latest
|
|
44
|
+
environment:
|
|
45
|
+
name: github-pages
|
|
46
|
+
steps:
|
|
47
|
+
- id: deployment
|
|
48
|
+
uses: actions/deploy-pages@v4
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
name: "Publish: brunnr"
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
tags: ["v*"]
|
|
6
|
+
workflow_dispatch:
|
|
7
|
+
|
|
8
|
+
jobs:
|
|
9
|
+
test:
|
|
10
|
+
runs-on: ubuntu-latest
|
|
11
|
+
steps:
|
|
12
|
+
- uses: actions/checkout@v4
|
|
13
|
+
- uses: actions/setup-python@v5
|
|
14
|
+
with:
|
|
15
|
+
python-version: "3.12"
|
|
16
|
+
- run: pip install -e ".[eval]" pytest
|
|
17
|
+
- run: python -m pytest tests/ -q
|
|
18
|
+
|
|
19
|
+
build:
|
|
20
|
+
needs: test
|
|
21
|
+
runs-on: ubuntu-latest
|
|
22
|
+
steps:
|
|
23
|
+
- uses: actions/checkout@v4
|
|
24
|
+
- uses: actions/setup-python@v5
|
|
25
|
+
with:
|
|
26
|
+
python-version: "3.12"
|
|
27
|
+
- run: pip install build
|
|
28
|
+
- run: python -m build
|
|
29
|
+
- uses: actions/upload-artifact@v4
|
|
30
|
+
with:
|
|
31
|
+
name: dist
|
|
32
|
+
path: dist/
|
|
33
|
+
|
|
34
|
+
publish:
|
|
35
|
+
if: startsWith(github.ref, 'refs/tags/v')
|
|
36
|
+
needs: build
|
|
37
|
+
runs-on: ubuntu-latest
|
|
38
|
+
environment: pypi
|
|
39
|
+
permissions:
|
|
40
|
+
id-token: write
|
|
41
|
+
steps:
|
|
42
|
+
- uses: actions/download-artifact@v4
|
|
43
|
+
with:
|
|
44
|
+
name: dist
|
|
45
|
+
path: dist/
|
|
46
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
|
47
|
+
|
|
48
|
+
release:
|
|
49
|
+
if: startsWith(github.ref, 'refs/tags/v')
|
|
50
|
+
needs: publish
|
|
51
|
+
runs-on: ubuntu-latest
|
|
52
|
+
permissions:
|
|
53
|
+
contents: write
|
|
54
|
+
steps:
|
|
55
|
+
- uses: actions/checkout@v4
|
|
56
|
+
with:
|
|
57
|
+
fetch-depth: 0
|
|
58
|
+
- uses: actions/download-artifact@v4
|
|
59
|
+
with:
|
|
60
|
+
name: dist
|
|
61
|
+
path: dist/
|
|
62
|
+
- name: Generate changelog
|
|
63
|
+
id: changelog
|
|
64
|
+
run: |
|
|
65
|
+
# Find the previous tag
|
|
66
|
+
PREV_TAG=$(git tag --sort=-v:refname | head -2 | tail -1)
|
|
67
|
+
if [ "$PREV_TAG" = "${{ github.ref_name }}" ] || [ -z "$PREV_TAG" ]; then
|
|
68
|
+
# First release — changelog from all commits
|
|
69
|
+
echo "## What's Changed" > RELEASE_NOTES.md
|
|
70
|
+
git log --pretty=format:"- %s (%h)" >> RELEASE_NOTES.md
|
|
71
|
+
else
|
|
72
|
+
echo "## What's Changed" > RELEASE_NOTES.md
|
|
73
|
+
git log "${PREV_TAG}..HEAD" --pretty=format:"- %s (%h)" >> RELEASE_NOTES.md
|
|
74
|
+
fi
|
|
75
|
+
echo "" >> RELEASE_NOTES.md
|
|
76
|
+
echo "**Full Changelog**: https://github.com/${{ github.repository }}/compare/${PREV_TAG:-initial}...${{ github.ref_name }}" >> RELEASE_NOTES.md
|
|
77
|
+
- name: Create GitHub Release
|
|
78
|
+
env:
|
|
79
|
+
GH_TOKEN: ${{ github.token }}
|
|
80
|
+
run: |
|
|
81
|
+
gh release create "${{ github.ref_name }}" \
|
|
82
|
+
--title "${{ github.ref_name }}" \
|
|
83
|
+
--notes-file RELEASE_NOTES.md \
|
|
84
|
+
dist/*
|
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
name: Skill Tests
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
pull_request:
|
|
5
|
+
branches: [main]
|
|
6
|
+
push:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
security-scan:
|
|
11
|
+
name: Security Scan (7-class)
|
|
12
|
+
runs-on: ubuntu-latest
|
|
13
|
+
steps:
|
|
14
|
+
- uses: actions/checkout@v4
|
|
15
|
+
|
|
16
|
+
- uses: actions/setup-python@v5
|
|
17
|
+
with:
|
|
18
|
+
python-version: '3.12'
|
|
19
|
+
|
|
20
|
+
- name: Scan all SKILL.md files
|
|
21
|
+
run: python3 tests/scan-all-skills.py --strict
|
|
22
|
+
|
|
23
|
+
- name: Upload scan report
|
|
24
|
+
if: always()
|
|
25
|
+
uses: actions/upload-artifact@v4
|
|
26
|
+
with:
|
|
27
|
+
name: scan-report
|
|
28
|
+
path: tests/artifacts/scan-report.json
|
|
29
|
+
|
|
30
|
+
- name: Post scan summary
|
|
31
|
+
if: always()
|
|
32
|
+
run: |
|
|
33
|
+
echo "## Security Scan Results" >> $GITHUB_STEP_SUMMARY
|
|
34
|
+
python3 -c "
|
|
35
|
+
import json
|
|
36
|
+
report = json.load(open('tests/artifacts/scan-report.json'))
|
|
37
|
+
total = report['total']
|
|
38
|
+
blocked = report['blocked']
|
|
39
|
+
flagged = report['flagged']
|
|
40
|
+
clean = report['clean']
|
|
41
|
+
print(f'| Metric | Count |', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
42
|
+
print(f'|--------|-------|', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
43
|
+
print(f'| Scanned | {total} |', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
44
|
+
print(f'| Clean | {clean} |', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
45
|
+
print(f'| Flagged | {flagged} |', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
46
|
+
print(f'| Blocked | {blocked} |', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
47
|
+
if blocked > 0:
|
|
48
|
+
print(f'', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
49
|
+
print(f'**{blocked} skill(s) BLOCKED by security scanner.**', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
50
|
+
for s in report['skills']:
|
|
51
|
+
if s['blocked']:
|
|
52
|
+
findings = ', '.join(f['desc'] for f in s['findings'] if f['severity'] == 'block')
|
|
53
|
+
print(f'- **{s[\"file\"]}**: {findings}', file=open('$GITHUB_STEP_SUMMARY', 'a'))
|
|
54
|
+
"
|
|
55
|
+
|
|
56
|
+
frontmatter-lint:
|
|
57
|
+
name: Lint SKILL.md Frontmatter
|
|
58
|
+
needs: [security-scan]
|
|
59
|
+
runs-on: ubuntu-latest
|
|
60
|
+
steps:
|
|
61
|
+
- uses: actions/checkout@v4
|
|
62
|
+
|
|
63
|
+
- name: Validate all SKILL.md files
|
|
64
|
+
run: |
|
|
65
|
+
echo "=== SKILL.md Frontmatter Validation ==="
|
|
66
|
+
ERRORS=0
|
|
67
|
+
TOTAL=0
|
|
68
|
+
|
|
69
|
+
for skill_md in skills/*/SKILL.md; do
|
|
70
|
+
[ -f "$skill_md" ] || continue
|
|
71
|
+
TOTAL=$((TOTAL + 1))
|
|
72
|
+
SKILL_DIR=$(dirname "$skill_md")
|
|
73
|
+
SKILL_SLUG=$(basename "$SKILL_DIR")
|
|
74
|
+
|
|
75
|
+
# Extract frontmatter (between --- lines)
|
|
76
|
+
FRONTMATTER=$(sed -n '/^---$/,/^---$/p' "$skill_md" | sed '1d;$d')
|
|
77
|
+
|
|
78
|
+
NAME=$(echo "$FRONTMATTER" | grep "^name:" | sed 's/^name: *//')
|
|
79
|
+
DESC=$(echo "$FRONTMATTER" | grep "^description:" | head -1)
|
|
80
|
+
|
|
81
|
+
if [ -z "$NAME" ]; then
|
|
82
|
+
echo "FAIL: $skill_md — missing 'name' in frontmatter"
|
|
83
|
+
ERRORS=$((ERRORS + 1))
|
|
84
|
+
elif [ "$NAME" != "$SKILL_SLUG" ]; then
|
|
85
|
+
echo "FAIL: $skill_md — name '$NAME' does not match directory '$SKILL_SLUG'"
|
|
86
|
+
ERRORS=$((ERRORS + 1))
|
|
87
|
+
else
|
|
88
|
+
echo "OK: $NAME"
|
|
89
|
+
fi
|
|
90
|
+
|
|
91
|
+
if [ -z "$DESC" ]; then
|
|
92
|
+
echo "FAIL: $skill_md — missing 'description' in frontmatter"
|
|
93
|
+
ERRORS=$((ERRORS + 1))
|
|
94
|
+
fi
|
|
95
|
+
done
|
|
96
|
+
|
|
97
|
+
echo ""
|
|
98
|
+
echo "=== $TOTAL skills checked, $ERRORS errors ==="
|
|
99
|
+
|
|
100
|
+
if [ $ERRORS -gt 0 ]; then
|
|
101
|
+
exit 1
|
|
102
|
+
fi
|
|
103
|
+
|
|
104
|
+
discover-skills:
|
|
105
|
+
name: Discover Testable Skills
|
|
106
|
+
needs: [security-scan]
|
|
107
|
+
runs-on: ubuntu-latest
|
|
108
|
+
outputs:
|
|
109
|
+
matrix: ${{ steps.discover.outputs.matrix }}
|
|
110
|
+
has_tests: ${{ steps.discover.outputs.has_tests }}
|
|
111
|
+
steps:
|
|
112
|
+
- uses: actions/checkout@v4
|
|
113
|
+
|
|
114
|
+
- name: Find skills with test suites
|
|
115
|
+
id: discover
|
|
116
|
+
run: |
|
|
117
|
+
SKILLS="[]"
|
|
118
|
+
HAS_TESTS="false"
|
|
119
|
+
|
|
120
|
+
for test_dir in tests/*/; do
|
|
121
|
+
[ -d "$test_dir" ] || continue
|
|
122
|
+
SKILL_SLUG=$(basename "$test_dir")
|
|
123
|
+
|
|
124
|
+
# Only include if the corresponding skill exists
|
|
125
|
+
if [ -f "skills/$SKILL_SLUG/SKILL.md" ] && [ -f "$test_dir/test-spec.json" ]; then
|
|
126
|
+
SKILLS=$(echo "$SKILLS" | python3 -c "
|
|
127
|
+
import json, sys
|
|
128
|
+
arr = json.load(sys.stdin)
|
|
129
|
+
arr.append('$SKILL_SLUG')
|
|
130
|
+
json.dump(arr, sys.stdout)
|
|
131
|
+
")
|
|
132
|
+
HAS_TESTS="true"
|
|
133
|
+
fi
|
|
134
|
+
done
|
|
135
|
+
|
|
136
|
+
echo "matrix={\"skill\":$SKILLS}" >> "$GITHUB_OUTPUT"
|
|
137
|
+
echo "has_tests=$HAS_TESTS" >> "$GITHUB_OUTPUT"
|
|
138
|
+
echo "Discovered skills with tests: $SKILLS"
|
|
139
|
+
|
|
140
|
+
eval-skills:
|
|
141
|
+
name: "Eval: ${{ matrix.skill }}"
|
|
142
|
+
needs: [frontmatter-lint, discover-skills]
|
|
143
|
+
if: needs.discover-skills.outputs.has_tests == 'true'
|
|
144
|
+
runs-on: ubuntu-latest
|
|
145
|
+
strategy:
|
|
146
|
+
fail-fast: false
|
|
147
|
+
matrix: ${{ fromJson(needs.discover-skills.outputs.matrix) }}
|
|
148
|
+
steps:
|
|
149
|
+
- uses: actions/checkout@v4
|
|
150
|
+
|
|
151
|
+
- uses: actions/setup-python@v5
|
|
152
|
+
with:
|
|
153
|
+
python-version: '3.12'
|
|
154
|
+
|
|
155
|
+
- name: Install dependencies
|
|
156
|
+
run: pip install anthropic pyyaml
|
|
157
|
+
|
|
158
|
+
- name: "Run evals for ${{ matrix.skill }}"
|
|
159
|
+
env:
|
|
160
|
+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
161
|
+
run: |
|
|
162
|
+
python tests/harness.py \
|
|
163
|
+
--skill "${{ matrix.skill }}" \
|
|
164
|
+
--output "tests/${{ matrix.skill }}/results.json"
|
|
165
|
+
|
|
166
|
+
- name: Upload results
|
|
167
|
+
if: always()
|
|
168
|
+
uses: actions/upload-artifact@v4
|
|
169
|
+
with:
|
|
170
|
+
name: eval-results-${{ matrix.skill }}
|
|
171
|
+
path: tests/${{ matrix.skill }}/results.json
|
|
172
|
+
|
|
173
|
+
eval-summary:
|
|
174
|
+
name: Eval Summary
|
|
175
|
+
needs: [eval-skills]
|
|
176
|
+
if: always() && needs.eval-skills.result != 'skipped'
|
|
177
|
+
runs-on: ubuntu-latest
|
|
178
|
+
steps:
|
|
179
|
+
- uses: actions/checkout@v4
|
|
180
|
+
|
|
181
|
+
- uses: actions/download-artifact@v4
|
|
182
|
+
with:
|
|
183
|
+
pattern: eval-results-*
|
|
184
|
+
merge-multiple: true
|
|
185
|
+
path: eval-results/
|
|
186
|
+
|
|
187
|
+
- name: Summarize results
|
|
188
|
+
run: |
|
|
189
|
+
echo "## Skill Eval Results" >> $GITHUB_STEP_SUMMARY
|
|
190
|
+
echo "" >> $GITHUB_STEP_SUMMARY
|
|
191
|
+
echo "| Skill | Pass | Fail | Total | Result |" >> $GITHUB_STEP_SUMMARY
|
|
192
|
+
echo "|-------|------|------|-------|--------|" >> $GITHUB_STEP_SUMMARY
|
|
193
|
+
|
|
194
|
+
OVERALL_PASS=true
|
|
195
|
+
|
|
196
|
+
for result_file in eval-results/*/results.json; do
|
|
197
|
+
[ -f "$result_file" ] || continue
|
|
198
|
+
SKILL=$(python3 -c "import json; r=json.load(open('$result_file')); print(r['skill'])")
|
|
199
|
+
PASS=$(python3 -c "import json; r=json.load(open('$result_file')); print(r['summary']['pass'])")
|
|
200
|
+
FAIL=$(python3 -c "import json; r=json.load(open('$result_file')); print(r['summary']['fail'])")
|
|
201
|
+
TOTAL=$(python3 -c "import json; r=json.load(open('$result_file')); print(r['summary']['total'])")
|
|
202
|
+
OK=$(python3 -c "import json; r=json.load(open('$result_file')); print('Pass' if r['summary']['fail']==0 else 'FAIL')")
|
|
203
|
+
|
|
204
|
+
if [ "$OK" = "FAIL" ]; then
|
|
205
|
+
OVERALL_PASS=false
|
|
206
|
+
fi
|
|
207
|
+
|
|
208
|
+
echo "| $SKILL | $PASS | $FAIL | $TOTAL | $OK |" >> $GITHUB_STEP_SUMMARY
|
|
209
|
+
done
|
|
210
|
+
|
|
211
|
+
if [ "$OVERALL_PASS" = "false" ]; then
|
|
212
|
+
echo "" >> $GITHUB_STEP_SUMMARY
|
|
213
|
+
echo "**One or more skill evals failed.**" >> $GITHUB_STEP_SUMMARY
|
|
214
|
+
exit 1
|
|
215
|
+
fi
|
|
216
|
+
|
|
217
|
+
schema-check:
|
|
218
|
+
name: Validate Output Schemas
|
|
219
|
+
needs: [security-scan]
|
|
220
|
+
runs-on: ubuntu-latest
|
|
221
|
+
steps:
|
|
222
|
+
- uses: actions/checkout@v4
|
|
223
|
+
|
|
224
|
+
- uses: actions/setup-node@v4
|
|
225
|
+
with:
|
|
226
|
+
node-version: '20'
|
|
227
|
+
|
|
228
|
+
- name: Install ajv-cli
|
|
229
|
+
run: npm install -g ajv-cli ajv-formats
|
|
230
|
+
|
|
231
|
+
- name: Validate per-skill output schemas
|
|
232
|
+
run: |
|
|
233
|
+
echo "=== Output Schema Validation ==="
|
|
234
|
+
ERRORS=0
|
|
235
|
+
|
|
236
|
+
for schema_file in schemas/*/output.schema.json; do
|
|
237
|
+
[ -f "$schema_file" ] || continue
|
|
238
|
+
SKILL=$(basename "$(dirname "$schema_file")")
|
|
239
|
+
|
|
240
|
+
echo "Validating schema for $SKILL..."
|
|
241
|
+
if ajv compile -s "$schema_file" --strict=false 2>/dev/null; then
|
|
242
|
+
echo "OK: $SKILL schema is valid JSON Schema"
|
|
243
|
+
else
|
|
244
|
+
echo "FAIL: $SKILL schema is not valid JSON Schema"
|
|
245
|
+
ERRORS=$((ERRORS + 1))
|
|
246
|
+
fi
|
|
247
|
+
done
|
|
248
|
+
|
|
249
|
+
if [ $ERRORS -gt 0 ]; then
|
|
250
|
+
echo "ERROR: $ERRORS invalid schemas"
|
|
251
|
+
exit 1
|
|
252
|
+
fi
|
|
253
|
+
|
|
254
|
+
echo "All schemas valid."
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# When skills change on main, trigger a Skillipedia site rebuild.
|
|
2
|
+
# Requires CROSS_REPO_TOKEN secret with repo dispatch permissions.
|
|
3
|
+
name: Trigger Skillipedia Rebuild
|
|
4
|
+
|
|
5
|
+
on:
|
|
6
|
+
push:
|
|
7
|
+
branches: [main]
|
|
8
|
+
paths:
|
|
9
|
+
- 'skills/**'
|
|
10
|
+
workflow_dispatch:
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
trigger:
|
|
14
|
+
runs-on: ubuntu-latest
|
|
15
|
+
steps:
|
|
16
|
+
- name: Dispatch to Skillipedia
|
|
17
|
+
uses: peter-evans/repository-dispatch@v3
|
|
18
|
+
with:
|
|
19
|
+
token: ${{ secrets.CROSS_REPO_TOKEN }}
|
|
20
|
+
repository: Peleke/skillipedia
|
|
21
|
+
event-type: brunnr-skills-updated
|
|
22
|
+
client-payload: '{"ref": "${{ github.ref }}", "sha": "${{ github.sha }}"}'
|
brunnr-0.1.0/.gitignore
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to brunnr will be documented in this file.
|
|
4
|
+
|
|
5
|
+
## [Unreleased]
|
|
6
|
+
|
|
7
|
+
## [0.1.0] - 2026-03-11
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- `brunnr scan` — pattern-based security scanner for agent tool descriptions (SKILL.md files)
|
|
11
|
+
- `brunnr install` — fetch and install skills from the brunnr registry with review-before-install
|
|
12
|
+
- `brunnr eval` — LLM-graded evaluation of tool descriptions against the ax-rubric
|
|
13
|
+
- `brunnr pipeline` — combined scan + eval in a single pass
|
|
14
|
+
- 26 built-in scan rules across 5 categories (prompt-injection, over-privilege, data-leak, ambiguity, missing-constraint)
|
|
15
|
+
- `ax-rubric` skill — score tool descriptions 0-5 on discoverability
|
|
16
|
+
- Claude Code plugin marketplace support (`.claude-plugin/marketplace.json`)
|
|
17
|
+
- Path traversal protection in install command
|
|
18
|
+
- MkDocs documentation site with ReadTheDocs theme
|
|
19
|
+
- PyPI publishing via OIDC trusted publisher (tag-triggered)
|
|
20
|
+
- GitHub Release with auto-generated changelog on tag push
|
|
21
|
+
- 74 tests (55 scanner + 19 CLI)
|
|
22
|
+
|
|
23
|
+
[Unreleased]: https://github.com/Peleke/brunnr/compare/v0.1.0...HEAD
|
|
24
|
+
[0.1.0]: https://github.com/Peleke/brunnr/releases/tag/v0.1.0
|
brunnr-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Peleke Sengstacke
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
brunnr-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: brunnr
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Security scanner and skill registry for agent tool descriptions
|
|
5
|
+
Author: Peleke Sengstacke
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Intended Audience :: Developers
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
11
|
+
Classifier: Topic :: Security
|
|
12
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
13
|
+
Requires-Python: >=3.12
|
|
14
|
+
Provides-Extra: eval
|
|
15
|
+
Requires-Dist: anthropic>=0.40; extra == 'eval'
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
|
|
18
|
+
<div align="center">
|
|
19
|
+
|
|
20
|
+
<img src="assets/hero.png" alt="brunnr — the well agents drink from" width="100%" />
|
|
21
|
+
|
|
22
|
+
# brunnr
|
|
23
|
+
|
|
24
|
+
**The well agents drink from.**
|
|
25
|
+
|
|
26
|
+
*Skills marketplace for Claude Code and agent-facing tools.*
|
|
27
|
+
|
|
28
|
+
[](LICENSE)
|
|
29
|
+
[](skills/)
|
|
30
|
+
[](skills/ax-rubric/)
|
|
31
|
+
|
|
32
|
+
</div>
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
> *Beneath the world tree there is a well. Odin gave an eye to drink from it.*
|
|
37
|
+
>
|
|
38
|
+
>*Your agent won't need to. We bottled the water.*
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Contents
|
|
43
|
+
|
|
44
|
+
- [AX Description Rubric](#ax-description-rubric) — score your tool descriptions
|
|
45
|
+
- [Install](#install) — one command
|
|
46
|
+
- [What you get](#what-you-get) — example output
|
|
47
|
+
- [The Rubric](#the-rubric) — five criteria, explained
|
|
48
|
+
- [Skills](#skills) — what's available
|
|
49
|
+
- [Why "brunnr"](#why-brunnr)
|
|
50
|
+
- [Contributing](#contributing)
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## What this is
|
|
55
|
+
|
|
56
|
+
**brunnr** is a skills marketplace for [Claude Code](https://docs.anthropic.com/en/docs/claude-code). Install it once, get access to every skill in the collection.
|
|
57
|
+
|
|
58
|
+
### AX Description Rubric
|
|
59
|
+
|
|
60
|
+
The first skill: the **AX Description Rubric** — a tool that scores your agent-facing tool descriptions against [five criteria](https://peleke.me/writing/ax-04-tool-descriptions) that determine whether an agent will ever call your tool.
|
|
61
|
+
|
|
62
|
+
Most tool descriptions score 1-2 out of 5. The rubric tells you why, then rewrites them to score 5.
|
|
63
|
+
|
|
64
|
+
## Install
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
# Add the marketplace
|
|
68
|
+
/plugin marketplace add Peleke/brunnr
|
|
69
|
+
|
|
70
|
+
# Use the skill
|
|
71
|
+
/ax-rubric
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Paste your tool description when prompted. Or pass it inline:
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
/ax-rubric "Query the knowledge base for structured learnings, patterns, and skills relevant to your current work context."
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## What you get
|
|
81
|
+
|
|
82
|
+
#### `knowledge_base_query`
|
|
83
|
+
|
|
84
|
+
**Score: 1/5**
|
|
85
|
+
|
|
86
|
+
| Criterion | Result | Notes |
|
|
87
|
+
|-----------|--------|-------|
|
|
88
|
+
| Output shape | fail | "structured learnings, patterns, and skills" — how many? |
|
|
89
|
+
| Cost signal | fail | No indication of response size |
|
|
90
|
+
| Trigger clarity | fail | "relevant to your current work context" — always true |
|
|
91
|
+
| Specificity | fail | Lists categories, solves no named problem |
|
|
92
|
+
| Differentiation | fail | Half your tools say "relevant to current context" |
|
|
93
|
+
|
|
94
|
+
**Rewritten (5/5):**
|
|
95
|
+
|
|
96
|
+
> "Check for known pitfalls before starting work. Returns 3-5 specific warnings for your tech stack. ~500 tokens. Call once at session start."
|
|
97
|
+
|
|
98
|
+
One input, five scores, one rewrite. The rewrite is ready to paste into your tool definition.
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## The rubric
|
|
103
|
+
|
|
104
|
+
Five criteria. Binary pass/fail. Most descriptions fail.
|
|
105
|
+
|
|
106
|
+
<img src="assets/rubric.svg" alt="The AX Description Rubric — five criteria for agent-facing tool descriptions" width="100%" />
|
|
107
|
+
|
|
108
|
+
**0-1 pass** = invisible (agent will never call it). **2-3** = marginal. **4-5** = habitual.
|
|
109
|
+
|
|
110
|
+
The bar for habitual use is 4/5. Most tool descriptions in the wild score 1-2. The tool might be excellent. The agent will never know.
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Skills
|
|
115
|
+
|
|
116
|
+
| Skill | What it does | Based on |
|
|
117
|
+
|-------|-------------|----------|
|
|
118
|
+
| **[ax-rubric](skills/ax-rubric/)** | Score tool descriptions for agent discoverability. 5 criteria, pass/fail, rewrite. | [I Don't Deliberate About This](https://peleke.me/writing/ax-04-tool-descriptions) |
|
|
119
|
+
|
|
120
|
+
More skills coming. Each one ships alongside an article in the [AX series](https://peleke.me/writing/ax-series).
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## Why "brunnr"
|
|
125
|
+
|
|
126
|
+
Old Norse *brunnr*: well, spring.
|
|
127
|
+
|
|
128
|
+
As in *Mímisbrunnr* — the Well of Mímir beneath Yggdrasil. Odin sacrificed an eye to drink from it and gain wisdom. The well doesn't give you answers. It gives you the capacity to see what was always there.
|
|
129
|
+
|
|
130
|
+
Your agent doesn't need to sacrifice anything. It just needs tool descriptions that don't waste its context window.
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## Contributing
|
|
135
|
+
|
|
136
|
+
Have an agent-facing tool? Run `/ax-rubric` on your own descriptions first. If they score 4+, you're ahead of 90% of the field.
|
|
137
|
+
|
|
138
|
+
Want to add a skill? Open a PR. The rubric applies to skill descriptions too.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## License
|
|
143
|
+
|
|
144
|
+
[MIT](LICENSE)
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
<div align="center">
|
|
149
|
+
|
|
150
|
+
*Part of the [Agent Experience (AX)](https://peleke.me/writing/ax-series) series*
|
|
151
|
+
*by [Peleke Sengstacke](https://peleke.me) and [Maren](https://peleke.me/writing/ax-01a-the-agent-has-opinions).*
|
|
152
|
+
|
|
153
|
+
</div>
|