skillscheck 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- skillscheck-0.1.0/.gitignore +3 -0
- skillscheck-0.1.0/LICENSE +21 -0
- skillscheck-0.1.0/LOG.md +277 -0
- skillscheck-0.1.0/Makefile +25 -0
- skillscheck-0.1.0/PKG-INFO +60 -0
- skillscheck-0.1.0/PLAN.md +283 -0
- skillscheck-0.1.0/README.md +31 -0
- skillscheck-0.1.0/TODO.md +42 -0
- skillscheck-0.1.0/main.py +4 -0
- skillscheck-0.1.0/pyproject.toml +57 -0
- skillscheck-0.1.0/skillscheck/__init__.py +0 -0
- skillscheck-0.1.0/skillscheck/agents/__init__.py +106 -0
- skillscheck-0.1.0/skillscheck/agents/_util.py +91 -0
- skillscheck-0.1.0/skillscheck/agents/claude.py +383 -0
- skillscheck-0.1.0/skillscheck/agents/codex.py +348 -0
- skillscheck-0.1.0/skillscheck/agents/copilot.py +50 -0
- skillscheck-0.1.0/skillscheck/agents/cursor.py +63 -0
- skillscheck-0.1.0/skillscheck/agents/gemini.py +87 -0
- skillscheck-0.1.0/skillscheck/agents/roo.py +122 -0
- skillscheck-0.1.0/skillscheck/agents/swival.py +66 -0
- skillscheck-0.1.0/skillscheck/agents/windsurf.py +48 -0
- skillscheck-0.1.0/skillscheck/checks/__init__.py +0 -0
- skillscheck-0.1.0/skillscheck/checks/disclosure.py +80 -0
- skillscheck-0.1.0/skillscheck/checks/quality.py +201 -0
- skillscheck-0.1.0/skillscheck/checks/spec.py +548 -0
- skillscheck-0.1.0/skillscheck/cli.py +113 -0
- skillscheck-0.1.0/skillscheck/mdutil.py +81 -0
- skillscheck-0.1.0/skillscheck/models.py +116 -0
- skillscheck-0.1.0/skillscheck/parser.py +99 -0
- skillscheck-0.1.0/skillscheck/tokenutil.py +25 -0
- skillscheck-0.1.0/skillscheck/validator.py +62 -0
- skillscheck-0.1.0/tests/conftest.py +20 -0
- skillscheck-0.1.0/tests/fixtures/allowed-tools-list/SKILL.md +9 -0
- skillscheck-0.1.0/tests/fixtures/bad-frontmatter/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/bad-name-consecutive/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/bad-name-hyphen-start/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/bad-name-mismatch/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/bad-name-uppercase/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/broken-link/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/description-too-long/SKILL.md +6 -0
- skillscheck-0.1.0/tests/fixtures/missing-description/SKILL.md +5 -0
- skillscheck-0.1.0/tests/fixtures/unknown-fields/SKILL.md +8 -0
- skillscheck-0.1.0/tests/fixtures/valid-full/SKILL.md +20 -0
- skillscheck-0.1.0/tests/fixtures/valid-full/references/guide.md +3 -0
- skillscheck-0.1.0/tests/fixtures/valid-minimal/SKILL.md +6 -0
- skillscheck-0.1.0/tests/test_agents.py +1707 -0
- skillscheck-0.1.0/tests/test_disclosure.py +113 -0
- skillscheck-0.1.0/tests/test_parser.py +115 -0
- skillscheck-0.1.0/tests/test_quality.py +233 -0
- skillscheck-0.1.0/tests/test_spec.py +555 -0
- skillscheck-0.1.0/tests/test_validator.py +314 -0
- skillscheck-0.1.0/uv.lock +458 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Frank Denis
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
skillscheck-0.1.0/LOG.md
ADDED
|
@@ -0,0 +1,277 @@
|
|
|
1
|
+
# LOG
|
|
2
|
+
|
|
3
|
+
## 2026-03-11: Agent Skills Specification Research
|
|
4
|
+
|
|
5
|
+
**User asked:** Research which AI coding agents support the Agent Skills specification from agentskills.io, including what aspects they support, agent-specific extensions/deviations, and validation tools.
|
|
6
|
+
|
|
7
|
+
**What I did:** Searched the web and fetched documentation from agentskills.io, GitHub repos (anthropics/skills, agentskills/agentskills), and official docs for Claude Code, OpenAI Codex, Gemini CLI, VS Code Copilot, Cursor, Windsurf, Roo Code, Kiro, Zed, Antigravity, Cline, and Qwen Code. Compiled detailed findings about spec support, frontmatter fields, discovery paths, agent-specific extensions, and validation tooling across 12+ agents.
|
|
8
|
+
|
|
9
|
+
## 2026-03-11: Plan creation
|
|
10
|
+
|
|
11
|
+
**User asked:** Write a plan for a Python app (uv) that validates a skills directory against the agentskills.io spec and agent expectations.
|
|
12
|
+
|
|
13
|
+
**What I did:**
|
|
14
|
+
- Fetched the complete Agent Skills specification from agentskills.io/specification
|
|
15
|
+
- Explored ~/src/fastly-agent-skills (6 skills, existing validate.sh, claude-plugin/, gemini-extension.json)
|
|
16
|
+
- Researched agent compatibility matrix (12+ agents)
|
|
17
|
+
- Created PLAN.md with 4 check categories: spec compliance (frontmatter fields, naming, directory structure), quality (description, hygiene, links), agent compatibility (Claude Code plugin, Gemini extension, Codex, Copilot, Cursor, Windsurf, Roo Code), and progressive disclosure (token budgets, reference sizing)
|
|
18
|
+
- Defined project architecture, CLI interface, output format, and implementation order
|
|
19
|
+
|
|
20
|
+
## 2026-03-11: Plan revision from review feedback
|
|
21
|
+
|
|
22
|
+
**User asked:** Addressed 5 findings (2 high, 3 medium) from plan review.
|
|
23
|
+
|
|
24
|
+
**What I did:**
|
|
25
|
+
- Finding 1 (high): Removed directory allowlist warning. The spec allows "any additional files or directories" — `scripts/`, `references/`, `assets/` are recognized as having special semantics, not enforced as the only allowed dirs.
|
|
26
|
+
- Finding 2 (high): Replaced flat agent checklist with a plugin-style adapter architecture. Only Claude Code and Gemini CLI have concrete, documented rules → v1 adapters. Other agents (Codex, Copilot, Cursor, Windsurf, Roo Code) deferred to "future adapters" section with criteria for inclusion.
|
|
27
|
+
- Finding 3 (medium): Added two-pass schema model. Base spec pass recognizes official fields. Agent extension pass recognizes per-adapter fields. Unknown fields get info-level notes (not warnings) since the spec doesn't forbid extras. `allowed-tools` now accepts string or list.
|
|
28
|
+
- Finding 4 (medium): Replaced vague "markdown parses without errors" with concrete checks: frontmatter boundary validation, relative link extraction via regex, target existence verification. No markdown parser dependency.
|
|
29
|
+
- Finding 5 (medium): Reordered implementation to build test fixtures of intentionally valid/invalid skills first (step 2), before any checks. `fastly-agent-skills` becomes one integration target (step 8), not the calibration source.
|
|
30
|
+
|
|
31
|
+
## 2026-03-11: Second plan revision (3 findings + open question)
|
|
32
|
+
|
|
33
|
+
**User asked:** Address 3 medium findings and one open question about exit codes.
|
|
34
|
+
|
|
35
|
+
**What I did:**
|
|
36
|
+
- Finding 1: `allowed-tools` list form now emits info note ("not portable base-spec syntax") when no active adapter authorizes it, instead of silently accepting both forms equally.
|
|
37
|
+
- Finding 2: Link extraction narrowed to `[text](path)` and `` syntax only. Bare `path/file.md` scanning removed — too many false positives from prose/code. Fenced code blocks, inline code, and URLs are excluded.
|
|
38
|
+
- Finding 3: Secret detection scoped to high-confidence filename patterns (`.env`, `.pem`, `.key`, `credentials.json`) and explicit token regexes (AWS `AKIA*`, GitHub `ghp_`/`gho_`/`ghs_`, PEM private key headers). Broad keyword scanning deferred to dedicated tools.
|
|
39
|
+
- Open question: Added severity/exit-code table (error→1, warning→0, info→0; `--strict` promotes warnings to exit 1, info stays 0). Added JSON output structure with `skills` and `agents` as separate top-level keys.
|
|
40
|
+
|
|
41
|
+
## 2026-03-11: Comprehensive test suite
|
|
42
|
+
|
|
43
|
+
**User asked:** Create test fixtures and tests for all modules in the skillscheck project.
|
|
44
|
+
|
|
45
|
+
**What I did:**
|
|
46
|
+
- Created 12 static fixture directories under `tests/fixtures/` with SKILL.md files covering valid skills (minimal, full with references) and invalid skills (missing SKILL.md, uppercase name, hyphen-start name, consecutive hyphens, name-dir mismatch, missing description, description too long, bad frontmatter YAML, broken links, allowed-tools as list, unknown fields)
|
|
47
|
+
- Created 6 test modules (124 tests total): `test_parser.py` (frontmatter splitting, skill parsing, discovery), `test_spec.py` (presence, name, description, optional fields, unknown fields, body, cross-skill checks), `test_quality.py` (description quality, file hygiene, link validation), `test_disclosure.py` (reference sizing, nesting), `test_agents.py` (Claude/Gemini adapters, adapter registry, cross-agent checks), `test_validator.py` (end-to-end validation, agent integration, result structure)
|
|
48
|
+
- Fixed two bugs found during testing: (1) missing `Level` import in `skillscheck/agents/__init__.py` causing NameError in `cross_agent_check`, (2) `_check_consistency` in `skillscheck/agents/claude.py` crashing on non-dict JSON (AttributeError on list/string)
|
|
49
|
+
- Added pytest as dev dependency and hatchling as build backend to `pyproject.toml`
|
|
50
|
+
- All 124 tests pass
|
|
51
|
+
|
|
52
|
+
## 2026-03-11: Implementation complete
|
|
53
|
+
|
|
54
|
+
**User asked:** Build the application per the plan.
|
|
55
|
+
|
|
56
|
+
**What I did:**
|
|
57
|
+
- Implemented all modules: `models.py` (Diagnostic/SkillInfo/ValidationResult), `parser.py` (frontmatter splitting, skill discovery), `checks/spec.py` (1a-1g), `checks/quality.py` (2a-2c), `checks/disclosure.py` (4a-4c), `agents/claude.py` (3a), `agents/gemini.py` (3b), `agents/__init__.py` (adapter registry + cross-agent checks), `validator.py` (orchestrator), `cli.py` (click CLI with text/JSON output)
|
|
58
|
+
- Integration test against ~/src/fastly-agent-skills: 6 skills validated, 0 errors, 3 warnings (2 secret token patterns in logging docs, 1 Claude plugin name mismatch), 2 info notes (token count, Gemini no context)
|
|
59
|
+
- Exit codes work as designed: default exit 0 with warnings, `--strict` exit 1
|
|
60
|
+
- 124 tests pass in 0.10s
|
|
61
|
+
|
|
62
|
+
## 2026-03-11: Fix 4 findings (2 high, 2 medium)
|
|
63
|
+
|
|
64
|
+
**User asked:** Fix adapter contract gaps, JSON output shape, allowed-tools list-form check, and code stripping.
|
|
65
|
+
|
|
66
|
+
**What I did:**
|
|
67
|
+
- Finding 1 (high, adapters): Claude adapter now validates version, description, author (must be object with name), keywords (must be string array), marketplace metadata.description, owner.name, plugin source, and description/version consistency. Gemini adapter now validates version and description fields.
|
|
68
|
+
- Finding 2 (high, JSON shape): Introduced `SkillDiagnostics` dataclass with `spec`, `quality`, `disclosure` buckets. `ValidationResult.skills` is now `dict[str, SkillDiagnostics]`. JSON output emits `{"skills":{"name":{"spec":[...],"quality":[...]}}` so CI can distinguish spec-invalid from quality/disclosure findings.
|
|
69
|
+
- Finding 3 (medium, allowed-tools): Added `allows_tools_list_syntax()` to adapter protocol. Info note now fires when no active adapter explicitly authorizes list syntax (Claude and Gemini both return False), instead of checking `bool(extension_fields)`.
|
|
70
|
+
- Finding 4 (medium, code stripping): Extracted `mdutil.py` with `strip_code()` that removes fenced blocks, indented code blocks (4+ spaces/tab after blank line), AND inline code spans. Both `quality.py` and `disclosure.py` now use `mdutil.extract_local_link_targets()`.
|
|
71
|
+
- Tests: 143 pass (up from 124). Added 19 new tests for expanded adapter validation, category buckets, and allowed-tools list-form with Claude active.
|
|
72
|
+
|
|
73
|
+
## 2026-03-11: Fix 3 more findings (2 high, 1 medium)
|
|
74
|
+
|
|
75
|
+
**User asked:** Fix `--agents all` false errors, plugin source path validation, and under-validated Claude fields.
|
|
76
|
+
|
|
77
|
+
**What I did:**
|
|
78
|
+
- Finding 1 (high): `--agents all` now filters to detected adapters (`a.detect(repo_root)`) instead of returning all adapters unconditionally. A Claude-only repo no longer gets Gemini missing-file errors.
|
|
79
|
+
- Finding 2 (high): Marketplace plugin `source` paths are now resolved against repo root and validated as existing directories. `"source": "./missing-skills"` now emits `3a.marketplace-json.plugin-source-missing` error. Moved `root` parameter from unused `_check_plugin_json` to `_check_marketplace_json` where it's needed.
|
|
80
|
+
- Finding 3 (medium): `plugin.json` now warns when `author` or `keywords` are missing (not just when present with wrong type). Author non-object and keywords non-list are now errors, not warnings. `marketplace.json` now flags non-object `metadata` (error: `3a.marketplace-json.metadata-type`) and non-object `owner` (error: `3a.marketplace-json.owner-type`) instead of silently skipping them.
|
|
81
|
+
- Tests: 150 pass (up from 143). Added 7 new tests for missing author/keywords, non-object metadata/owner, source path resolution, and `--agents all` detection filtering.
|
|
82
|
+
|
|
83
|
+
## 2026-03-11: /simplify code review cleanup
|
|
84
|
+
|
|
85
|
+
**User asked:** Run /simplify to review code for reuse, quality, and efficiency.
|
|
86
|
+
|
|
87
|
+
**What I did:** Three review agents identified findings across reuse, quality, and efficiency. Fixed all valid issues:
|
|
88
|
+
- Removed unused imports: `SkillDiagnostics` from cli.py, `Path` from checks/spec.py
|
|
89
|
+
- Used `CATEGORIES` constant in `SkillDiagnostics.to_dict()` instead of hardcoded field names
|
|
90
|
+
- Merged identical `None`/`["all"]` branches in `get_adapters`
|
|
91
|
+
- Deduplicated `SPEC_URL` — moved to `models.py`, imported by spec.py, quality.py, disclosure.py
|
|
92
|
+
- Renamed `_ensure_skill` to `ensure_skill` (public API used by validator.py)
|
|
93
|
+
- Extracted `load_json_object` helper into `agents/_util.py` to deduplicate JSON load/validate/type-check pattern across Claude and Gemini adapters
|
|
94
|
+
- Refactored `cross_agent_check` to use a loop over config files and a loop over fields, eliminating copy-paste
|
|
95
|
+
- Merged two `os.walk` passes in `discover_skills` into one
|
|
96
|
+
- Fixed redundant fence-close condition in `mdutil._strip_fenced_blocks`
|
|
97
|
+
- Used `stat().st_size` instead of reading entire files for token estimation in disclosure.py
|
|
98
|
+
- All 152 tests pass
|
|
99
|
+
|
|
100
|
+
## 2026-03-11: Humanize code and create TODO
|
|
101
|
+
|
|
102
|
+
**User asked:** Make the code look like written by a human, remove redundant comments, create a TODO list.
|
|
103
|
+
|
|
104
|
+
**What I did:**
|
|
105
|
+
- Removed ~20 docstrings that just restated function names (e.g. `"""Parse a skill directory, extracting SKILL.md frontmatter and body."""` on `parse_skill()`)
|
|
106
|
+
- Simplified formulaic module docstrings (e.g. `"""Main validation orchestrator."""` → describes the module's actual purpose)
|
|
107
|
+
- Kept docstrings that carry genuine information (return tuple docs, discovery logic, frontmatter split semantics)
|
|
108
|
+
- Created TODO.md with items grouped by category: agent adapters, checks, secret detection, CLI features, packaging/CI
|
|
109
|
+
- All 152 tests still pass
|
|
110
|
+
|
|
111
|
+
## 2026-03-11: Codex adapter
|
|
112
|
+
|
|
113
|
+
**User asked:** Address the next TODO item (Codex adapter).
|
|
114
|
+
|
|
115
|
+
**What I did:**
|
|
116
|
+
- Researched OpenAI Codex's skills support by reading the openai/codex repo source: skill loader, model structs, TypeScript schemas, and example skills (babysit-pr, test-tui)
|
|
117
|
+
- Codex discovers skills under `.codex/skills/` and `.agents/skills/`. Each skill can have an optional `agents/openai.yaml` sidecar with `interface`, `dependencies`, `policy`, and `permissions` sections
|
|
118
|
+
- Implemented `CodexAdapter` in `skillscheck/agents/codex.py` (check ID prefix: `3d`):
|
|
119
|
+
- Detection: `.codex/` directory exists
|
|
120
|
+
- Per-skill validation of `agents/openai.yaml` when present: YAML parse, top-level field allowlist, interface field types, dependencies.tools structure (type/value required, known types: env_var/cli/mcp), policy.allow_implicit_invocation type check
|
|
121
|
+
- No Codex-specific frontmatter extensions (empty `known_frontmatter_fields`)
|
|
122
|
+
- Registered adapter in `ALL_ADAPTERS` in `agents/__init__.py`
|
|
123
|
+
- Added 23 new tests covering detection, valid/invalid YAML, type errors, unknown fields, dependencies, and policy
|
|
124
|
+
- Updated existing `test_explicit_all` and added `test_auto_detect_codex`
|
|
125
|
+
- All 175 tests pass (up from 152)
|
|
126
|
+
|
|
127
|
+
## 2026-03-11: Codex adapter fixes (2 medium findings)
|
|
128
|
+
|
|
129
|
+
**User asked:** Fix two gaps: (1) `permissions` accepted in schema but never validated, (2) detection missed `.agents/` directory.
|
|
130
|
+
|
|
131
|
+
**What I did:**
|
|
132
|
+
- Finding 1: Added `_check_permissions()` that validates the `permissions` section of `openai.yaml` — must be a mapping, known sub-keys (`network`, `file_system`, `macos`), each sub-key type-checked (network/macos must be mappings, file_system.read/write must be lists)
|
|
133
|
+
- Finding 2: `detect()` now checks for `.codex/` or `.agents/` directory, matching both paths Codex uses for skill discovery
|
|
134
|
+
- Added 10 new tests: 8 permissions tests (not-mapping, unknown fields, network/fs/macos type checks, valid) and 2 detection tests (.agents dir, both dirs)
|
|
135
|
+
- All 185 tests pass (up from 175)
|
|
136
|
+
|
|
137
|
+
## 2026-03-11: Copilot adapter research
|
|
138
|
+
|
|
139
|
+
**User asked:** Research how VS Code Copilot / GitHub Copilot handles the Agent Skills specification from agentskills.io, covering discovery paths, sidecar files, frontmatter extensions, configuration files, and allowed-tools handling.
|
|
140
|
+
|
|
141
|
+
**What I did:**
|
|
142
|
+
- Fetched the agentskills.io specification, client implementation guide, and what-are-skills pages
|
|
143
|
+
- Fetched VS Code Copilot docs: customization overview, agent-skills, prompt-files, custom-instructions, custom-agents, copilot-settings reference, concepts
|
|
144
|
+
- Fetched GitHub Copilot docs: repository custom instructions, building extensions
|
|
145
|
+
- Read the agentskills/agentskills skills-ref reference library source code (validator.py, parser.py, models.py)
|
|
146
|
+
- Compiled comprehensive findings about Copilot's skill discovery, configuration settings, frontmatter extensions, file formats, and cross-client conventions
|
|
147
|
+
|
|
148
|
+
## 2026-03-11: Copilot adapter implementation
|
|
149
|
+
|
|
150
|
+
**User asked:** Address the next TODO item (Copilot adapter).
|
|
151
|
+
|
|
152
|
+
**What I did:**
|
|
153
|
+
- Copilot discovers skills from `.github/skills/` (native), plus configurable paths via `chat.agentSkillsLocations`. No sidecar config files (unlike Codex's `agents/openai.yaml`)
|
|
154
|
+
- Copilot extends base spec frontmatter with three fields: `argument-hint` (string), `user-invocable` (boolean), `disable-model-invocation` (boolean)
|
|
155
|
+
- Implemented `CopilotAdapter` in `skillscheck/agents/copilot.py` (check ID prefix: `3e`):
|
|
156
|
+
- Detection: `.github/skills/` directory exists
|
|
157
|
+
- Per-skill frontmatter validation: boolean fields type-checked, string fields type-checked
|
|
158
|
+
- Known frontmatter fields reported so spec checker doesn't flag them as unknown
|
|
159
|
+
- Registered adapter in `ALL_ADAPTERS` in `agents/__init__.py`
|
|
160
|
+
- Added 13 new tests: detection (present, absent, .github without skills), frontmatter field types, valid skills, multiple errors, multiple skills, auto-detect
|
|
161
|
+
- Updated `test_explicit_all` and added `test_auto_detect_copilot`
|
|
162
|
+
- All 198 tests pass (up from 185)
|
|
163
|
+
|
|
164
|
+
## 2026-03-11: Fix Copilot adapter (1 high, 1 medium)
|
|
165
|
+
|
|
166
|
+
**User asked:** Fix two findings: (1) `_check_frontmatter` crashes on `None` frontmatter, (2) adapter validates all skills instead of only Copilot skills under `.github/skills/`.
|
|
167
|
+
|
|
168
|
+
**What I did:**
|
|
169
|
+
- Finding 1 (high): `_check_frontmatter` now uses `skill.frontmatter or {}` instead of bare `skill.frontmatter`, matching the pattern used by `spec.py` and `quality.py`. Skills with parse errors no longer crash.
|
|
170
|
+
- Finding 2 (medium): `check()` now filters skills by `dir_path` prefix — only skills under `<repo_root>/.github/skills/` are checked. Non-Copilot skills elsewhere in the repo are skipped.
|
|
171
|
+
- Added 3 new tests: `test_none_frontmatter_no_crash`, `test_non_copilot_skill_ignored`, `test_copilot_skill_checked_non_copilot_skipped`
|
|
172
|
+
- Updated `_make_copilot_skill` helper to place skills under `.github/skills/` instead of repo root
|
|
173
|
+
- All 201 tests pass (up from 198)
|
|
174
|
+
|
|
175
|
+
## 2026-03-11: Cursor adapter
|
|
176
|
+
|
|
177
|
+
**User asked:** Implement the next TODO item (Cursor adapter).
|
|
178
|
+
|
|
179
|
+
**What I did:**
|
|
180
|
+
- Researched Cursor's Agent Skills support: discovers skills from `.cursor/skills/` (project) and `.agents/skills/` (cross-client), with `.cursor/` directory as detection signal
|
|
181
|
+
- Cursor extends base spec frontmatter with `disable-model-invocation` (boolean) — same field Copilot uses, prevents auto-invocation by the agent
|
|
182
|
+
- No sidecar config files (unlike Codex's `agents/openai.yaml` or Claude's `plugin.json`)
|
|
183
|
+
- Implemented `CursorAdapter` in `skillscheck/agents/cursor.py` (check ID prefix: `3f`):
|
|
184
|
+
- Detection: `.cursor/` directory exists
|
|
185
|
+
- Per-skill frontmatter validation for skills under `.cursor/skills/` or `.agents/skills/`
|
|
186
|
+
- Legacy `.cursorrules` file detection — emits deprecation warning pointing to `.cursor/rules/`
|
|
187
|
+
- Registered adapter in `ALL_ADAPTERS` in `agents/__init__.py`
|
|
188
|
+
- Added 16 new tests: detection (present, absent, dir without skills subdir), frontmatter type checks, None frontmatter safety, `.agents/skills/` path support, non-Cursor skill filtering, `.cursorrules` deprecation warning, auto-detect integration
|
|
189
|
+
- Updated `test_explicit_all` to include Cursor setup
|
|
190
|
+
- All 217 tests pass (up from 201)
|
|
191
|
+
|
|
192
|
+
## 2026-03-11: Implement all TODO checks and secret detection
|
|
193
|
+
|
|
194
|
+
**User asked:** Implement all checks from the TODO list.
|
|
195
|
+
|
|
196
|
+
**What I did:**
|
|
197
|
+
- **Semver validation** (`1c.version.format`): Added `version` to `BASE_SPEC_FIELDS` and validates against semver regex (MAJOR.MINOR.PATCH with optional pre-release/build metadata) when present. Warning-level.
|
|
198
|
+
- **SPDX license validation** (`1c.license.spdx`): Uses `packaging.licenses.canonicalize_license_expression()` to validate SPDX identifiers. Warning-level.
|
|
199
|
+
- **Duplicate descriptions** (`1g.duplicate-description`): Extended `check_cross_skill` to track descriptions (case-insensitive) and warn when two skills share identical description text.
|
|
200
|
+
- **Placeholder descriptions** (`1b.description.placeholder`): Added 10 regex patterns matching common placeholders ("TODO", "A skill that", "This skill", "...", "FIXME", "TBD", "PLACEHOLDER", "Description goes here", "Enter description", "Replace this"). Warning-level.
|
|
201
|
+
- **Body heading check** (`1e.body.no-heading`): Checks if SKILL.md body starts with a markdown heading (`#`). Info-level note if not.
|
|
202
|
+
- **Stale allowed-tools** (`1c.allowed-tools.unknown-tool`): Maintains a set of known tool names across agents (Claude Code, Codex, etc.) plus `mcp__*` prefix matching. Info-level note for unrecognized tools. Works with both string and list forms.
|
|
203
|
+
- **Secret detection additions**: Added GitLab PAT (`glpat-`), Slack tokens (`xoxb-`, `xoxp-`, `xapp-`), base64-encoded private keys (`LS0tLS1CRUdJTi`), and `.env.local`/`.env.production`/`.env.staging`/`.env.development` filename patterns.
|
|
204
|
+
- Added 30 new tests covering all new checks
|
|
205
|
+
- All 248 tests pass (up from 218)
|
|
206
|
+
|
|
207
|
+
## 2026-03-11: Fix 3 spec-model findings (2 high, 1 medium)
|
|
208
|
+
|
|
209
|
+
**User asked:** Fix three findings where the tool contradicted the spec it cites.
|
|
210
|
+
|
|
211
|
+
**What I did:**
|
|
212
|
+
- Finding 1 (high, license): Removed SPDX-only validation of `license` field. The spec says "license name or reference to a bundled license file" and gives `Proprietary. LICENSE.txt has complete terms` as its own example — freeform strings are valid. No validation beyond type-checking is appropriate.
|
|
213
|
+
- Finding 2 (high, allowed-tools): `_is_known_tool()` now strips parenthesized arguments before lookup (`Bash(git:*)` → `Bash`). The spec's own example `allowed-tools: Bash(git:*) Bash(jq:*) Read` previously produced false `unknown-tool` notes.
|
|
214
|
+
- Finding 3 (medium, version): Removed `version` from `BASE_SPEC_FIELDS` and dropped the semver check. The spec lists only 6 top-level fields (name, description, license, compatibility, metadata, allowed-tools). The spec example puts version under `metadata`, not as a top-level field. Top-level `version:` is now correctly flagged as an unknown field by the two-pass model.
|
|
215
|
+
- Replaced `TestVersionValidation` (5 tests) and `TestLicenseValidation` (3 tests) with `TestVersionField` (2 tests verifying unknown-field behavior) and `TestLicenseField` (3 tests verifying freeform acceptance). Added `test_parenthesized_tool_args_recognized`.
|
|
216
|
+
- All 246 tests pass (down from 248 — net -2 from removing 8 wrong tests and adding 6 correct ones)
|
|
217
|
+
|
|
218
|
+
## 2026-03-11: Windsurf adapter
|
|
219
|
+
|
|
220
|
+
**User asked:** Implement the Windsurf adapter.
|
|
221
|
+
|
|
222
|
+
**What I did:**
|
|
223
|
+
- Researched Windsurf's Agent Skills support: discovers skills from `.windsurf/skills/` (project) and `.agents/skills/` (cross-client), with `.windsurf/` directory as detection signal
|
|
224
|
+
- No Windsurf-specific frontmatter extensions beyond the base spec. No sidecar config files.
|
|
225
|
+
- Legacy `.windsurfrules` file (pre-Wave 8) deprecated in favor of `.windsurf/rules/`
|
|
226
|
+
- Implemented `WindsurfAdapter` in `skillscheck/agents/windsurf.py` (check ID prefix: `3g`):
|
|
227
|
+
- Detection: `.windsurf/` directory exists
|
|
228
|
+
- `.windsurfrules` deprecation warning
|
|
229
|
+
- Empty `known_frontmatter_fields` (no extensions)
|
|
230
|
+
- Registered adapter in `ALL_ADAPTERS` in `agents/__init__.py`
|
|
231
|
+
- Added 8 new tests: detection (present, absent), empty frontmatter fields, no list tools syntax, clean repo, `.windsurfrules` deprecation warning, auto-detect integration
|
|
232
|
+
- Updated `test_explicit_all` to include Windsurf setup
|
|
233
|
+
- All 254 tests pass (up from 246)
|
|
234
|
+
|
|
235
|
+
## 2026-03-11: Roo Code skills research
|
|
236
|
+
|
|
237
|
+
**User asked:** Research how Roo Code handles skills: discovery paths, SKILL.md frontmatter extensions, sidecar config files, deprecated paths, and detection markers.
|
|
238
|
+
|
|
239
|
+
**What I did:**
|
|
240
|
+
- Fetched Roo Code official docs: docs.roocode.com/features/skills, /features/custom-modes, /features/custom-instructions, /advanced-usage/available-tools/skill
|
|
241
|
+
- Fetched DeepWiki analysis of Roo Code skill configuration
|
|
242
|
+
- Fetched Kilo Code (Roo Code fork) technical explanation gist
|
|
243
|
+
- Read actual source code from GitHub via `gh api`: `src/services/skills/SkillsManager.ts` (719 lines), `src/shared/skills.ts`, `packages/types/src/skills.ts`, `src/core/prompts/sections/skills.ts`, `src/core/tools/SkillTool.ts`, `src/core/prompts/tools/native-tools/skill.ts`, `src/services/skills/skillInvocation.ts`, `src/services/roo-config/index.ts`, `src/core/prompts/sections/custom-instructions.ts`
|
|
244
|
+
- Compiled comprehensive findings covering all 5 questions
|
|
245
|
+
|
|
246
|
+
## 2026-03-11: Roo Code and Swival adapters
|
|
247
|
+
|
|
248
|
+
**User asked:** Implement the Roo Code adapter and the Swival adapter.
|
|
249
|
+
|
|
250
|
+
**What I did:**
|
|
251
|
+
- Implemented `RooAdapter` in `skillscheck/agents/roo.py` (check ID prefix: `3h`):
|
|
252
|
+
- Detection: `.roo/` directory, `.roomodes` file, or `.roorules` file
|
|
253
|
+
- Frontmatter extensions: `modeSlugs` (list of strings) and `mode` (string, deprecated)
|
|
254
|
+
- Validates `modeSlugs` must be list of strings, `mode` must be string
|
|
255
|
+
- Info-level note when deprecated `mode` field is used (recommends `modeSlugs`)
|
|
256
|
+
- Deprecation warnings for `.roorules` and `.clinerules` legacy files
|
|
257
|
+
- Skills checked under `.roo/skills/` and `.agents/skills/` paths
|
|
258
|
+
- Implemented `SwivalAdapter` in `skillscheck/agents/swival.py` (check ID prefix: `3i`):
|
|
259
|
+
- Detection: `swival.toml` file or `.swival/` directory
|
|
260
|
+
- No Swival-specific frontmatter extensions
|
|
261
|
+
- Warns when description exceeds Swival's 1024-char limit
|
|
262
|
+
- Warns when body exceeds Swival's 20,000-char limit
|
|
263
|
+
- Skills checked under `skills/` path
|
|
264
|
+
- Registered both adapters in `ALL_ADAPTERS` in `agents/__init__.py`
|
|
265
|
+
- Added 33 new tests: Roo (19 tests — detection, frontmatter types, deprecated fields, path filtering, auto-detect) and Swival (14 tests — detection, limits, path filtering, auto-detect)
|
|
266
|
+
- Updated `test_explicit_all` to include both new adapters
|
|
267
|
+
- All 287 tests pass (up from 254)
|
|
268
|
+
|
|
269
|
+
## 2026-03-11: Replace token estimation with tiktoken
|
|
270
|
+
|
|
271
|
+
**User asked:** Use tiktoken for proper token counting instead of the `chars // 4` heuristic.
|
|
272
|
+
|
|
273
|
+
**What I did:**
|
|
274
|
+
- Added `tiktoken>=0.7` as a project dependency
|
|
275
|
+
- Created `skillscheck/tokenutil.py` with `count_tokens(text)` and `estimate_file_tokens(path)` using the `cl100k_base` encoding (encoder shared across calls via module-level cache)
|
|
276
|
+
- Updated `checks/spec.py` (body token estimation) and `checks/disclosure.py` (reference file sizing) to use the new functions instead of `len(text) // 4`
|
|
277
|
+
- All 287 tests still pass — existing test data was constructed to be above/below thresholds regardless of counting method
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
.PHONY: all install test lint format check clean dist
|
|
2
|
+
|
|
3
|
+
all: check test
|
|
4
|
+
|
|
5
|
+
install:
|
|
6
|
+
uv sync
|
|
7
|
+
|
|
8
|
+
test:
|
|
9
|
+
uv run python -m pytest tests/ -v
|
|
10
|
+
|
|
11
|
+
lint:
|
|
12
|
+
uv run ruff check skillscheck/ tests/
|
|
13
|
+
|
|
14
|
+
format:
|
|
15
|
+
uv run ruff format skillscheck/ tests/
|
|
16
|
+
|
|
17
|
+
check: lint
|
|
18
|
+
uv run ruff format --check skillscheck/ tests/
|
|
19
|
+
|
|
20
|
+
clean:
|
|
21
|
+
rm -rf dist/ __pycache__ skillscheck/__pycache__ tests/__pycache__ .pytest_cache
|
|
22
|
+
find . -name '*.pyc' -delete
|
|
23
|
+
|
|
24
|
+
dist: clean
|
|
25
|
+
uv build
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: skillscheck
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Validate agent skills against the agentskills.io specification
|
|
5
|
+
Project-URL: Repository, https://github.com/swival/skillscheck
|
|
6
|
+
Project-URL: Issues, https://github.com/swival/skillscheck/issues
|
|
7
|
+
Author-email: Frank Denis <github@pureftpd.org>
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: agent,ai,cli,linter,skills,validation
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Operating System :: OS Independent
|
|
16
|
+
Classifier: Programming Language :: Python :: 3
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Topic :: Software Development
|
|
21
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
22
|
+
Classifier: Topic :: Software Development :: Testing
|
|
23
|
+
Classifier: Typing :: Typed
|
|
24
|
+
Requires-Python: >=3.11
|
|
25
|
+
Requires-Dist: click>=8.0
|
|
26
|
+
Requires-Dist: pyyaml>=6.0
|
|
27
|
+
Requires-Dist: tiktoken>=0.7
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
|
|
30
|
+
# skillscheck - a linter for SKILL.md files
|
|
31
|
+
|
|
32
|
+
A CLI tool that validates agent skill directories against the [agentskills.io specification](https://agentskills.io/specification) and checks compatibility with AI coding agents (Claude Code, Gemini CLI, Codex, Copilot, Cursor, Roo Code, Windsurf, Swival).
|
|
33
|
+
|
|
34
|
+
## Installation / Usage
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
uvx skillscheck /path/to/skills-repo
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Options
|
|
41
|
+
|
|
42
|
+
- `--format json`: JSON output for CI pipelines
|
|
43
|
+
- `--strict`: treat warnings as errors (exit 1)
|
|
44
|
+
- `--agents claude,gemini,codex,copilot,cursor,roo,windsurf,swival`: run specific agent adapter checks (auto-detects if omitted, or `all`)
|
|
45
|
+
- `--check spec,quality,disclosure,agents`: run specific check categories
|
|
46
|
+
|
|
47
|
+
### Check categories
|
|
48
|
+
|
|
49
|
+
- `spec`: core specification compliance (frontmatter fields, naming, directory structure)
|
|
50
|
+
- `quality`: description quality, file hygiene, broken links, secret detection
|
|
51
|
+
- `disclosure`: progressive disclosure (reference file sizing, nesting depth)
|
|
52
|
+
- `agents`: agent-specific config validation (Claude plugin.json, Gemini extension.json, Codex openai.yaml, Copilot/Cursor/Roo/Windsurf/Swival conventions)
|
|
53
|
+
|
|
54
|
+
### Exit codes
|
|
55
|
+
|
|
56
|
+
| Condition | Exit code |
|
|
57
|
+
| :----------------------- | :-------- |
|
|
58
|
+
| No errors | 0 |
|
|
59
|
+
| Errors found | 1 |
|
|
60
|
+
| `--strict` with warnings | 1 |
|