@podo/skill-validator 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Giedrius Jaloveckas
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,287 @@
-# Temporary Holding Version
+# Skill Validator CLI
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Offline Node 22+ validator for directories containing `SKILL.md`. Uses the
+website's deterministic rules, security heuristics, static package checks, and
+token estimates. No network requests, AI calls, or execution of skill scripts.
+
+**Public npm package:** `@podo/skill-validator`. Stable installations should pin
+`0.1.0`; verify its registry publication before installing it.
+The package does not require Cloudsmith or Nexus. Direct npm downloads from the
+target Jenkins agent still need a network/policy check. The source repository
+does not need to move to distribute this package.
+
+## Try locally
+
+From the repository root:
+
+```sh
+npm --prefix cli ci
+npm --prefix cli run build
+node cli/dist/cli.js validate cli/test/fixtures/skills/csv-summary
+node cli/dist/cli.js --help
+```
+
+Build an installable tarball with `npm pack` from `cli/`, then install that
+tarball in another project with `npm install -D /absolute/path/to/the.tgz`.
+Both `skill-validator` and `skillval` invoke the same command.
+
+## Install from public npm
+
+**After the stable release is published and verified**, pin its exact version:
+
+```sh
+npm install --save-dev --save-exact --ignore-scripts \
+  @podo/skill-validator@0.1.0 \
+  --registry=https://registry.npmjs.org/
+./node_modules/.bin/skill-validator validate skills
+```
+
+Commit the manifest and lockfile for reproducible installs. Public consumers need
+no npm credentials. An existing `@podo:registry` mapping takes precedence over the
+default registry; remove an obsolete mapping or explicitly set
+`--@podo:registry=https://registry.npmjs.org/` for this command.
+
+The public package and its bundled first-party validator engine are MIT-licensed;
+see [`LICENSE`](LICENSE). Bundled dependency licenses are preserved in
+[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). This does not change the license
+of the rest of the website repository.
+
+## Commands
+
+After installing the package locally:
+
+```sh
+skill-validator validate skills
+skill-validator skills/foo/SKILL.md          # validate is the default command
+skill-validator doctor                     # install/config/discovery diagnostics
+skill-validator rules list                 # web-only rules are marked [web/live only]
+skill-validator rules explain B4
+skill-validator prompt skills/foo           # offline repair prompt, to stdout
+```
+
+With no paths, discovery starts at `skills/` in the current directory if it
+exists, otherwise at the current directory. Paths must stay within the nearest
+Git repository root (or the current directory when outside Git).
+
+### Reports and flags
+
+```sh
+skill-validator validate skills --format json --out report.json
+skill-validator validate skills --format junit,json,markdown --report-dir reports
+skill-validator validate skills --fail-on critical --quiet
+```
+
+- `--format`: comma-separated `text,json,junit,markdown`; default `text`.
+- `--report-dir`: required for multiple formats. Fixed filenames are
+  `report.txt`, `report.json`, `junit.xml`, and `report.md`.
+- `--out`: a single-format destination; `-` means stdout. Mutually exclusive
+  with `--report-dir`. Outputs must stay inside the repository and outside skill
+  directories, so reports cannot overwrite skill source.
+- `--fail-on`: `critical`, `high`, `suggestion`, `info`, or `none`; default `high`.
+- `--quiet`: counts and gate line only in text output; machine formats stay intact.
+- `--config`: explicit config path, otherwise `.skill-validator.json` at repo root.
+
+File output creates parent directories and writes atomically. Its short summary
+goes to stderr; stdout JSON/XML is not mixed with logs. Text is uncolored. Text
+and Markdown cap output at 20 findings per skill; JSON and JUnit are complete.
+Destinations inside any skill are refused, including unselected or ignored skills
+and filesystem case aliases. Failed writes remove their owned temporary file and
+leave any existing destination intact.
+
+Exit codes:
+
+- `0`: no findings meet the configured threshold.
+- `1`: findings fail the gate.
+- `2`: invalid invocation, config, missing skills, incomplete discovery,
+  unreadable or refused input, or output I/O failure. A closed stdout/stderr pipe
+  exits 2 without an unhandled stack trace and without forcing pending output to
+  be truncated.
+- `3`: internal CLI error; prints the version, Node version, and the npm package
+  page (the source repository is private, so there is no issue tracker link). Set
+  `SKILL_VALIDATOR_DEBUG=1` for a stack trace, which may contain sensitive data.
+
+Zero skills is never a successful validation. `doctor` can report zero skills
+without failing because it is diagnostic, not a gate.
+
+## Configuration
+
+The default gate is `high`. A team can initially choose `critical` with reviewed
+severity overrides, then tighten the gate after checking real skills:
+
+```json
+{
+  "failOn": "critical",
+  "ruleSeverities": {
+    "TT-003": "high",
+    "DE-002": "suggestion"
+  },
+  "ignore": ["drafts/**"]
+}
+```
+
+Flags override config. Unknown keys, rule IDs, and severity values are errors,
+as are overrides for web/live-only rules (AI-assisted review, URL/registry
+liveness, HP-001): the CLI never reports them, so an override cannot take effect.
+Overrides change visible findings and preserve `originalSeverity` in JSON;
+they do not hide rules. First-party API integrations can legitimately read tokens
+and make requests, which is why the example recalibrates those security rules.
+
+`ignore` accepts repo-relative glob patterns and affects **skill discovery only**.
+Each pattern is limited to 1,024 characters and 128 glob operators; invalid or
+unevaluable patterns are configuration errors, not internal crashes.
+It cannot hide files inside a discovered skill. `.git` and `node_modules` are
+excluded at the root of a skill only; nested copies (for example
+`scripts/node_modules`) are scanned and count toward the per-skill limits below.
+Discovery itself never descends into `.git` or `node_modules`. Protect CI configuration with CODEOWNERS/review requirements:
+a PR-controlled threshold can otherwise weaken its own gate.
+
+## Scope and safety
+
+- Discovery descends at most four directories below each input root. If any
+  eligible directory (not ignored, not `.git`/`node_modules`) lies below that
+  limit, the run exits 2 and names it, even when other skills were found, so a
+  gate never passes on a partly searched tree. Pass a narrower root, or add
+  directories without skills to `ignore`. `doctor` reports them without failing.
+  Overlapping inputs are deduplicated, including explicit nested inputs. Nested
+  skills stay in their parent bundle so rule A0 can report them. A skill's contents
+  are loaded as a complete bundle rather than subject to the discovery depth limit.
+- Symlinks and nonregular skill files are refused. Per-skill limits are 2,000 files,
+  100 MiB total file size, and 5 MiB per text file. Discovery and bundle traversal
+  each cap directories at 10,000; discovery counts unique directories, not repeated
+  overlapping visits. Bundle nesting is capped at 64.
+- Archives are not CLI inputs. Binary files contribute metadata and size, not
+  decoded content. Skills and inputs are never executed.
+- JavaScript `.js`, `.mjs`, and `.cjs` files are decoded and security-scanned with
+  the same executable-risk weighting. Malformed frontmatter mappings and eval
+  document/case shapes produce findings instead of passing silently or crashing.
+- These checks bound ordinary validation; they are not a sandbox against a
+  separate process maliciously changing directories during a scan.
+- Security patterns currently report the first matching occurrence per rule/file.
+  A passing result is not proof a skill is safe or effective.
+
+The website additionally performs liveness and optional AI review, so its complete
+report can differ. CLI reports explicitly list `ai-review` and `liveness` as not
+run. Static known-hallucinated-package rule HP-002 still runs offline.
+
+### Report contract and privacy
+
+JSON has `schemaVersion: 1`, relative paths, stable finding fingerprints, and an
+explicit `gate`. The published package includes
+[`schema/report.schema.json`](schema/report.schema.json). Fingerprints normalize
+line endings and include diagnostic text so separate eval cases stay distinct.
+They exclude line numbers and severity; wording changes can change an identifier.
+They are identifiers, not a baseline feature.
+Legal POSIX names such as colons and newlines retain their identity. A relative
+path that could be read as a Windows drive-absolute or rooted path (a drive letter
+and colon followed by a separator or nothing, or a leading backslash) is prefixed
+with `./`; the prefixed path resolves to the same file. Absolute and `..` paths
+never appear in reports.
+
+JUnit includes one failing testcase per blocking finding and an `@summary` case
+per skill. A testcase is named by rule ID; when a rule blocks more than once in
+a skill, the name becomes `<ruleId>:<fingerprint>` so cases stay distinct.
+Advisory findings are neither failures nor fake skipped tests. Counts match the
+gate; timestamps and durations are omitted for stable output.
+XML attribute encoding preserves tabs, newlines, and carriage returns in paths,
+so distinct files do not merge into the same Jenkins testcase identity.
+
+Security snippets are omitted from reports and from repair prompts, and common
+credential patterns are redacted from both. This is best-effort, not complete
+secret detection. Prompts still quote other source content (for example
+descriptions and suggested replacements), and reports include skill paths and
+finding text. Treat both as potentially sensitive; use appropriate artifact
+access/retention. Text and Markdown output show control characters in paths as
+visible `\uXXXX` escapes; JSON keeps the exact path.
+
+## Jenkins advisory example
+
+**After publishing and verifying direct registry access:** add a separate stage
+to the existing pipeline and preserve all existing checks. The example assumes
+Node 22+ is already available and skills live under `plugins/`; adapt this path
+to the consuming repository. It uses an exact package version without adding
+application dependencies to that repository. A consumer with a package manifest
+can instead commit a pinned devDependency/lockfile and use `npm ci --ignore-scripts`.
+
+```groovy
+stage('Skill Validator (advisory)') {
+    steps {
+        // Remove stale artifacts before invocation, including crash paths.
+        dir('skill-validator-report') { deleteDir() }
+        script {
+            def status = sh(returnStatus: true, script: '''
+                npm exec --yes --ignore-scripts \
+                  --registry=https://registry.npmjs.org/ \
+                  --@podo:registry=https://registry.npmjs.org/ \
+                  --package=@podo/skill-validator@0.1.0 -- \
+                  skill-validator validate plugins --fail-on none \
+                  --format junit,json,markdown \
+                  --report-dir skill-validator-report --quiet
+            ''')
+            if (status != 0) {
+                error("skill-validator input/tool failure (${status}); see log")
+            }
+        }
+    }
+    post {
+        always {
+            archiveArtifacts artifacts: 'skill-validator-report/*', allowEmptyArchive: true
+            junit testResults: 'skill-validator-report/junit.xml', allowEmptyResults: false
+        }
+    }
+}
+```
+
+`--fail-on none` keeps findings advisory; invalid input, failed downloads, and tool
+errors remain blocking. Missing JUnit is an error, not an empty successful run.
+Protect policy files and preserve the marketplace's existing validation stages.
+This example has not yet been executed on a Jenkins agent; it does not establish
+permission to bypass any employer-required registry proxy.
+
+## Development
+
+From `cli/`, run `npm ci`, `npm run typecheck`, and `npm test`. The tests build
+the bundle, validate report schemas/JUnit, exercise the exit-code and filesystem
+contracts, and install a packed tarball without runtime dependencies.
+Release-orchestration tests simulate identity/registry/upload responses while
+using real npm packing; they never upload a package or request credentials.
+Also run root typecheck, tests, and build when changing shared engine code.
+The test suite uses Python 3's standard library for XML round-trips and isolated
+OS-level write/pipe fault injection; Python is not a package runtime dependency.
+The repository CI workflow (`.github/workflows/ci.yml`) runs these checks for
+the application, CLI, and server across its OS/Node matrix. It never publishes
+packages or deploys the app.
+
+## Publishing
+
+The release script targets **public npm**, not an internal registry, and verifies
+the signed-in account is `podo`. Sign in from your own terminal:
+
+```sh
+npm login --registry=https://registry.npmjs.org/
+npm whoami --registry=https://registry.npmjs.org/
+```
+
+Use npm's browser/2FA flow; never paste credentials into chat or tracked files.
+The script uses existing npm authentication without copying or printing secrets.
+Public publishing requires the account's applicable 2FA or npm-approved automation
+authentication. A successful dry-run is not proof of publishing permission.
+
+- `scripts/publish-cli.sh dry-run`: installs build dependencies without lifecycle
+  scripts, typechecks, tests, and packs an audited tarball. No authentication,
+  registry availability request, or upload; allowed before committing.
+- `scripts/publish-cli.sh test`: requires a clean committed tree and a unique
+  prerelease version, then publishes the exact checked tarball with `canary`.
+- `scripts/publish-cli.sh release`: additionally requires a stable version and an
+  exact matching `cli-vX.Y.Z` tag on HEAD, then publishes with `latest`.
+
+Both publishing modes pin default **and scoped** registries, check identity and
+version availability, and stop on registry errors. An existing exact version is
+not overwritten; use a new version for changed bytes. The packed allowlist is
+only the executable, schema, package metadata, README, license, and dependency
+notices. Inspect the artifact for sensitive material before the first upload.
+
+CLI tags are independent of web `vX.Y.Z` tags and do not require a Playground
+deploy. After each release, verify its public access and integrity,
+install it in a clean directory without credentials, and run it on a real Jenkins
+agent before enabling consumer integration. Publication does not establish
+Jenkins compatibility: that remains a separate acceptance check.

package/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,58 @@
+# Bundled dependency notices
+
+The CLI bundles the following third-party software. These notices apply to that
+software; the CLI and its bundled first-party validator engine use the MIT
+license in `LICENSE`. No dependency code has been intentionally modified; the
+build bundles and transforms it for Node.js.
+
+## js-yaml
+
+Source: https://github.com/nodeca/js-yaml
+
+(The MIT License)
+
+Copyright (C) 2011-2015 by Vitaly Puzrin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+## picomatch
+
+Source: https://github.com/micromatch/picomatch
+
+The MIT License (MIT)
+
+Copyright (c) 2017-present, Jon Schlinkert.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.