skill-claims 1.0.0__tar.gz

skill_claims-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jasveer Singh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

skill_claims-1.0.0/PKG-INFO

@@ -0,0 +1,432 @@
+Metadata-Version: 2.4
+Name: skill-claims
+Version: 1.0.0
+Summary: Receipt an agent skill's claims against its own code — every claim SUPPORTED, UNEVIDENCED or CONTRADICTED, with file-and-line evidence.
+Author: Jasveer Singh
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/jasveerai88/skill-claims
+Project-URL: Repository, https://github.com/jasveerai88/skill-claims
+Project-URL: Issues, https://github.com/jasveerai88/skill-claims/issues
+Project-URL: Changelog, https://github.com/jasveerai88/skill-claims/blob/main/CHANGELOG.md
+Keywords: agent-skills,claude-code,ai-agents,honesty,audit,static-analysis,cli
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+<h1 align="center">skill-claims</h1>
+
+<p align="center"><strong>Your agent skill says "offline". Receipt it — every claim SUPPORTED, UNEVIDENCED, or CONTRADICTED, with file-and-line evidence.</strong></p>
+
+<p align="center">
+  <img alt="tests: 48 passing" src="https://img.shields.io/badge/tests-48%20passing-brightgreen">
+  <img alt="python 3.10 | 3.11 | 3.12" src="https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue">
+  <img alt="version 1.0.0" src="https://img.shields.io/badge/version-1.0.0-orange">
+  <img alt="dependencies: none" src="https://img.shields.io/badge/dependencies-0-brightgreen">
+  <img alt="license: MIT" src="https://img.shields.io/badge/license-MIT-blue">
+</p>
+
+<p align="center"><img alt="skill-claims demo: an 'offline' formatter's receipt flips CONTRADICTED on three claims" src="docs/demo.gif" width="880"></p>
+
+A skill's description is a promise. The loader trusts it, the
+marketplace lists it, and you install on it. The measurement of
+how often the promise holds is not reassuring:
+*Behavioral Integrity Verification for AI Agent Skills*
+(arXiv:2605.11770, checked 2026-11-06) compared declared vs
+actual behaviour across **49,943 OpenClaw-registry skills and
+found 80.0% deviate from their declared behaviour — and 81.1%
+of the deviations were developer oversight, only 18.9%
+adversarial.** The honesty problem is mostly not malice. It is
+drift, copy-paste, and a description nobody re-read after the
+code changed.
+
+Security scanners ask a different question — *is it
+dangerous?* — and the few that touch claims at all bury the
+answer inside an LLM judgement or a 0–100 security score.
+skill-claims asks only the honesty question, and answers it
+the boring way: extract what the package **claims** (from its
+own description and docs, by a deterministic pattern library),
+extract what the package **does** (from its bundled scripts,
+by static pattern tables), and print a per-claim receipt.
+No LLM. No score. No network. Nothing uploaded.
+
+**Static and read-only — it executes nothing.** Claims never
+imports, runs, or evaluates anything in the package; every
+file is read as text and judged as evidence. Its own source
+imports no network or subprocess module, and the test suite
+proves the read-only claim: a fixture script that would write
+a marker file if it ran never runs, and every fixture tree
+comes out byte-identical after an audit.
+
+## The honesty pair (and the trio)
+
+Two different questions get asked before a skill is trusted,
+and they deserve two different tools:
+
+- **[skill-vet](https://github.com/jasveerai88/skill-vet) —
+  is it dangerous?** Stealing, spying, hidden sending, each
+  proven as a combination of evidence.
+- **skill-claims — is it honest?** Does the package do what
+  its description claims — and only that?
+
+And after the agent runs:
+**[done-claim-auditor](https://github.com/jasveerai88/done-claim-auditor) —
+did the agent do what *it* claimed?** The same
+claim-vs-evidence logic pointed at session transcripts.
+Claims is built to run beside skill-vet as a pre-install
+gate: vet first for danger, then claims for honesty. That
+is exactly how it is used in its author's own engine —
+every skill adopted there passes both before install.
+
+## Quick start
+
+```bash
+pipx install git+https://github.com/jasveerai88/skill-claims              # or: pip install git+https://github.com/jasveerai88/skill-claims
+skill-claims audit ~/Downloads/some-skill
+```
+
+One command, one receipt, an exit code you can gate on —
+**1** = at least one claim CONTRADICTED, **0** = no claim
+contradicted, **2** = input error. Add `--json` for a
+machine-readable receipt. From a source checkout,
+`python3 src/skill_claims.py audit <package>` does the same
+thing. Python 3.10+, standard library only, no dependencies.
+
+## What it receipts
+
+Claims are read from the frontmatter `description`, the
+SKILL.md body, and a root README.md. Behaviour is read from
+every bundled script. The verdicts:
+
+| Claim type | Example phrasing | SUPPORTED when | CONTRADICTED when |
+|---|---|---|---|
+| **Capability** | "Formats markdown tables" | The claim's key words appear in the package's code | *Never* — static analysis cannot prove a feature absent, so a capability with no code evidence is UNEVIDENCED |
+| **Offline** | "Works fully offline, no network access" | No network call in any scanned script | Any network call (`urllib`, `requests`, `socket`, `curl`, …), cited file + line |
+| **Sends nothing** | "Nothing leaves your machine" | No network call in any scanned script | Any network call — every connection sends at least a request |
+| **Read scope** | "Only reads the files you pass it" | No credential read, no read outside the package | A credential-store read, or a read of a path outside the package |
+| **Credentials** | "Never touches your credentials" | No credential-store or env-secret read | Any credential-store path (`.aws`, `.ssh`, …) or env-secret read |
+| **No writes** | "Writes nothing" | No file write in any scanned script | Any write (`open(..., 'w')`, `.write(`, `shutil.copy`, …) |
+| **No writes (scoped)** | "Never write inside the project directory" | No writes at all | A write provably aimed at the package's own location (`__file__`-relative). Writes to runtime-computed paths are UNEVIDENCED — static evidence can't place them inside the named scope |
+| **Write scope** | "Never writes outside the project" | Writes exist but all stay inside scope, or no writes at all | A write to a home / system / outside path |
+| **No shell** | "Never runs shell commands" | No subprocess / shell use; no shell scripts | `subprocess`, `os.system`, `shell=True`, or a bundled `.sh` |
+| **No deletion** | "Never deletes anything" | No deletion in any scanned script | `os.remove`, `shutil.rmtree`, `.unlink(`, `rm -rf`, … |
+| **Environment** | "Does not read environment variables" | No env read in any scanned script | `os.environ`, `getenv(`, `process.env`, … |
+| **Quality / absolute** | "Guaranteed 100% accurate" | — | — (always UNEVIDENCED: no static evidence can test a quality claim, and the receipt says so instead of pretending) |
+
+Behaviour that **no claim covers** is listed at the end of
+every receipt too — a skill that never claims to be offline
+but calls the network is not contradicted, it is simply on
+the record.
+
+[Quick start](#quick-start) · [What it receipts](#what-it-receipts) ·
+[How it compares](#how-it-compares) · [Reference](#reference)
+
+## How it compares
+
+| | **skill-claims** | NVIDIA SkillSpector | SkillAuditor.dev | PyPI skill-auditor |
+|---|---|---|---|---|
+| The question | Is it **honest** — does it do what it claims? | Is it **dangerous**? (security) | Is it safe, proven in a sandbox? | Is it safe to install? (security) |
+| Claim-vs-behaviour | The whole product: a per-claim receipt | One part of an optional LLM "Deep Mode" | Part of the hosted verdict | One rule (`description-mismatch`) inside a scanner |
+| Method | Deterministic static patterns — same package, same receipt, forever | Pattern scan + LLM judgement | Hosted sandbox simulation, verdict recorded onchain | LLM-assisted scanner |
+| Output | Per-claim SUPPORTED / UNEVIDENCED / CONTRADICTED with file + line | A 0–100 security score | A hosted verdict | Scanner findings |
+| Runs where | Your machine, offline, stdlib only | Your machine (LLM mode needs a model) | Their service | Your machine + an LLM |
+| Score? | **No score, by design** — a receipt, not a rating | Yes | Verdict | Findings |
+
+*(Competitor descriptions from their own public pages,
+checked 2026-11-06. SkillSpector is a strong security
+scanner — it is listed here because it is the closest
+neighbour, and because "dangerous?" and "honest?" are
+different questions that both deserve asking.)*
+
+## Honest limits (the short version)
+
+Static patterns are evidence, not proof. A package that
+assembles its behaviour at runtime, downloads its real
+logic later, hides behaviour in a bundled binary, or buries
+it in obfuscated code can look consistent — every receipt
+prints this. A SUPPORTED verdict means *the static evidence
+backs the claim*, not that the skill is proven honest. The
+full limits are [below](#honest-limits).
+
+---
+
+## Reference
+
+**Contents:** [The lying formatter](#the-lying-formatter--the-demo-fixture) ·
+[The honest skill](#the-honest-skill--the-other-direction) ·
+[For agents](#for-agents) · [Tests](#tests) ·
+[Honest limits](#honest-limits)
+
+### The lying formatter — the demo fixture
+
+`examples/lying-formatter/` is a constructed package: a
+markdown formatter whose description promises full offline
+working and reads limited to the files you pass, whose body
+promises it never touches your credentials — and whose
+`scripts/telemetry.py` helper reads `~/.aws/credentials`
+and POSTs it with `urllib`. Every endpoint is an `.invalid`
+name — fictional by design. Verbatim output:
+
+```
+$ skill-claims audit examples/lying-formatter
+skill-claims v1.0.0 — audit · package lying-formatter · 3 files · 4 claims
+Frontmatter name: lying-formatter
+Read-only: it edits nothing, writes nothing, and never executes anything in the package — every file is analysed as static text.
+Verdict rule: CONTRADICTED needs hard evidence: a claim is only contradicted when the package's own code does the thing the claim rules out, cited file + line. SUPPORTED = the evidence the claim needs was found, or the ruled-out behaviour was searched for across every scanned script and not found. UNEVIDENCED = the claim was made, but static evidence can neither back it nor break it. A capability claim is never CONTRADICTED. Exit 1 only when at least one claim is CONTRADICTED; exit 2 is an input error.
+Limits: Static evidence only: skill-claims reads the package's text and never runs it. Behaviour that appears only at runtime — later downloads, payloads assembled from pieces, obfuscated or encoded code, bundled binaries — is out of scope and can make a dishonest skill look consistent. SUPPORTED means the static evidence backs the claim, not that the skill is proven honest.
+
+CLAIMS (4):
+[SUPPORTED] capability — "Formats markdown tables and tidies headings in the files you pass it"
+  stated: SKILL.md · line 3
+  evidence for: scripts/format.py · line 2 — 'format' appears here
+  evidence for: scripts/format.py · line 2 — 'markdown' appears here
+  evidence for: scripts/format.py · line 2 — 'tables' appears here
+  basis: 5 of the claim's 5 key words (formats, headings, markdown, tables, tidies) appear in the package's code — the vocabulary of the feature the claim names is present in the implementation itself, which is the static evidence a capability claim can be receipted on.
+[CONTRADICTED] offline — "Works fully offline with no network access"
+  stated: SKILL.md · line 3
+  evidence against: scripts/telemetry.py · line 20 — a urllib request call
+  evidence against: scripts/telemetry.py · line 22 — a urllib request call
+  basis: the claim rules out network use, but the package's own code makes network calls — the claim and the code cannot both be true.
+[CONTRADICTED] read scope — "only reads the files you pass it"
+  stated: SKILL.md · line 3
+  evidence against: scripts/telemetry.py · line 11 — an AWS credentials/config path
+  evidence against: scripts/telemetry.py · line 11 — a read of a path outside the package
+  basis: the claim limits reads to the files the user passes (or the package's own scope), but the package's own code reads beyond that.
+[CONTRADICTED] credentials — "Never touches your credentials"
+  stated: SKILL.md · line 21
+  evidence against: scripts/telemetry.py · line 11 — an AWS credentials/config path
+  basis: the claim rules out touching credentials, but the package's own code reads credential material.
+
+BEHAVIOUR FOUND (2 script(s) scanned):
+  network calls — 2 hit(s); first: scripts/telemetry.py · line 20 — a urllib request call
+  credential-store / env-secret reads — 1 hit(s); first: scripts/telemetry.py · line 11 — an AWS credentials/config path
+  environment-variable reads — none found
+  reads of paths outside the package — 1 hit(s); first: scripts/telemetry.py · line 11 — a read of a path outside the package
+  file writes — none found
+  writes to paths outside the package — none found
+  shell / subprocess use — none found
+  deletions — none found
+
+Summary: 4 claims · 1 supported · 0 unevidenced · 3 contradicted
+Verdict: CONTRADICTED — 3 claim(s) are contradicted by the package's own code.
+(exit code 1)
+```
+
+Note what is *not* contradicted: the capability claim. The
+formatter really does format tables — the receipt says so,
+on the same page as the three broken promises. A receipt
+that only lists failures would be a scanner, not a receipt.
+
+### The honest skill — the other direction
+
+`examples/honest-skill/` makes five claims — a capability,
+offline working, no writes, reads limited to named files,
+no deletion — and its one script does exactly that and no
+more:
+
+```
+$ skill-claims audit examples/honest-skill
+skill-claims v1.0.0 — audit · package honest-skill · 2 files · 5 claims
+Frontmatter name: honest-skill
+Read-only: it edits nothing, writes nothing, and never executes anything in the package — every file is analysed as static text.
+Verdict rule: CONTRADICTED needs hard evidence: a claim is only contradicted when the package's own code does the thing the claim rules out, cited file + line. SUPPORTED = the evidence the claim needs was found, or the ruled-out behaviour was searched for across every scanned script and not found. UNEVIDENCED = the claim was made, but static evidence can neither back it nor break it. A capability claim is never CONTRADICTED. Exit 1 only when at least one claim is CONTRADICTED; exit 2 is an input error.
+Limits: Static evidence only: skill-claims reads the package's text and never runs it. Behaviour that appears only at runtime — later downloads, payloads assembled from pieces, obfuscated or encoded code, bundled binaries — is out of scope and can make a dishonest skill look consistent. SUPPORTED means the static evidence backs the claim, not that the skill is proven honest.
+
+CLAIMS (5):
+[SUPPORTED] capability — "Formats markdown tables in the files you pass it"
+  stated: SKILL.md · line 3
+  evidence for: scripts/format.py · line 2 — 'format' appears here
+  evidence for: scripts/format.py · line 2 — 'markdown' appears here
+  evidence for: scripts/format.py · line 2 — 'tables' appears here
+  basis: 3 of the claim's 3 key words (formats, markdown, tables) appear in the package's code — the vocabulary of the feature the claim names is present in the implementation itself, which is the static evidence a capability claim can be receipted on.
+[SUPPORTED] offline — "Works fully offline"
+  stated: SKILL.md · line 3; SKILL.md · line 19
+  basis: no network call was found in any of the 1 scanned script(s) — the behaviour the claim rules out is absent from the static evidence.
+[SUPPORTED] no writes — "writes nothing"
+  stated: SKILL.md · line 3; SKILL.md · line 10
+  basis: no file write was found in any of the 1 scanned script(s).
+[SUPPORTED] read scope — "reads only the files you name"
+  stated: SKILL.md · line 3
+  basis: no credential-store read and no read of a path outside the package was found in any of the 1 scanned script(s).
+[SUPPORTED] no deletion — "It never deletes anything"
+  stated: SKILL.md · line 20
+  basis: no deletion was found in any of the 1 scanned script(s).
+
+BEHAVIOUR FOUND (1 script(s) scanned):
+  network calls — none found
+  credential-store / env-secret reads — none found
+  environment-variable reads — none found
+  reads of paths outside the package — none found
+  file writes — none found
+  writes to paths outside the package — none found
+  shell / subprocess use — none found
+  deletions — none found
+
+Summary: 5 claims · 5 supported · 0 unevidenced · 0 contradicted
+Verdict: SUPPORTED — every claim the package makes is backed by the static evidence.
+(exit code 0)
+```
+
+### For agents
+
+`--json` emits one object: the package record, the full
+file inventory (`path`, `bytes`, `kind`, `binary`,
+`scanned`, `symlink_out`), the claims (each with `type`,
+`text`, every `stated_at` file + line, `verdict`, `basis`,
+and `evidence` hits with file + line), the behaviour map
+(per class: `count` + `hits`), `scripts_scanned`,
+`unclaimed_behaviour`, summary counts that recount exactly
+against the claims, the overall verdict, the verdict rule,
+and the limits statement. Input errors — including argument
+errors — are a single JSON object with an `error` key and
+exit 2. [SKILL.md](SKILL.md) states the full claim table,
+the verdict rules, and the JSON contract — and makes
+skill-claims itself installable as a skill: point your
+agent at this repo's SKILL.md and it will receipt skills
+before installing them.
+
+### Tests
+
+```
+python3 -m unittest discover -s tests
+```
+
+48 tests: both fixtures pinned to hand-worked receipts;
+every claim type extracted and judged in both directions
+(the behaviour present flips the claim to CONTRADICTED;
+absent, the same claim is SUPPORTED); the UNEVIDENCED
+paths (a capability with no code evidence, a quality
+claim, a package whose binary content cannot be scanned,
+a scoped write claim whose targets are runtime paths);
+unclaimed behaviour listed without being contradicted;
+the error contract (missing folder, no SKILL.md, empty or
+unparseable SKILL.md — clean `{"error": ...}`, exit 2,
+never a traceback, in text and `--json` modes); regression
+pins from receipts run against real packages before
+release (a directory-tree diagram is not a claim, vendored
+`.venv` code is not the package's own, a noun-phrase
+fragment is not a capability); and the safety proofs —
+nothing in a package is ever executed, fixture trees are
+byte-identical after runs, output is byte-identical
+across runs, and the source's imports are pinned to a
+stdlib set with no network or subprocess module.
+
+### Honest limits
+
+- **Static evidence only — dynamic behaviour is out of
+  scope.** Claims reads the package's text; it never runs
+  it. Behaviour that appears only at runtime — code
+  downloaded and executed later, payloads assembled from
+  innocuous pieces, obfuscated or encoded logic, and
+  bundled binaries — cannot be receipted, and a dishonest
+  skill can hide behind exactly that. Every receipt
+  prints this.
+- **CONTRADICTED is reserved for hard evidence.** A claim
+  flips only when the package's own code does the ruled-out
+  thing. Suspicion, style, and similarity never flip a
+  claim — they produce no verdict at all.
+- **Capability claims are never CONTRADICTED.** Static
+  analysis cannot prove a feature absent — a capability
+  implemented in an unusual way may earn UNEVIDENCED
+  instead of SUPPORTED. The receipt states the key words
+  it looked for, so a human can overrule it in seconds.
+- **Capability evidence is vocabulary-level.** SUPPORTED
+  for a capability means the feature's key words appear in
+  the code — the strongest static signal available without
+  running the skill, and a weaker one than the
+  contradiction evidence. The basis line says exactly
+  which words were found.
+- **Claim extraction is a pattern library.** It reads the
+  frontmatter description, the SKILL.md body, and a root
+  README.md, and recognises the phrasings in the claim
+  table. A claim phrased outside the library is not
+  extracted — and therefore not receipted. Claims of the
+  same type are receipted together (first phrasing quoted,
+  every stating place listed). Capability claims are read
+  from the description only — the declared capability of
+  record; behavioural claims are read from all three
+  documents. In the documents, only **prose** counts:
+  fenced code blocks, indented code, table rows, and
+  directory-tree diagrams are illustrations, not claims.
+- **Vendored code is not the package's own code.**
+  Dependency trees (`node_modules`, `.venv`, `venv`,
+  `site-packages`) are excluded from the walk: a claim is
+  receipted against what the author wrote, not against a
+  bundled library's behaviour. A skill that hides its own
+  code inside a folder with one of those names is out of
+  scope for the receipt — the exclusion is stated here
+  rather than silently applied.
+- **Scoped claims get scoped verdicts.** "Never write
+  inside the project directory" forbids a named place,
+  not writing itself: it is contradicted only by a write
+  provably aimed at the package's own location, and
+  writes to runtime-computed paths leave it UNEVIDENCED,
+  because static evidence cannot place them.
+- **Behaviour is extracted from scripts.** Prose
+  instructions in a SKILL.md that tell the *agent* to do
+  something are claims' territory, not behaviour evidence,
+  in v1.0.0. Full-line comments are ignored for behaviour
+  (a comment is a statement, not an action). Code-shaped
+  content that cannot be scanned (binaries, outward
+  symlinks, oversized scripts) downgrades affected claims
+  to UNEVIDENCED rather than letting them pass as
+  SUPPORTED.
+- **Frontmatter is required.** The declared description
+  is the claim of record, so a SKILL.md with no `---`
+  block — or one the stated subset parser cannot read —
+  is a clean input error (exit 2), not a receipt.
+- **Pattern tables are a snapshot.** New client
+  libraries, credential stores, and phrasings appear
+  constantly; the tables are dated by the release, not
+  treated as complete.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). The evidence
+discipline is the product: a claim is contradicted only
+on hard evidence, and contributions are held to that.
+Found a package skill-claims misjudges — a claim it
+misses, a verdict the evidence doesn't support? Open an
+issue with the package (or a minimal fixture); that is
+exactly the feedback that improves the pattern library.
+
+## Security
+
+skill-claims is itself an audit tool, so its own posture
+is simple: it has no network code, no subprocess code, no
+dependencies, and it never executes what it scans. See
+[SECURITY.md](SECURITY.md) for how to report a problem.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Related
+
+- [skill-vet](https://github.com/jasveerai88/skill-vet)
+  — the other half of the pre-install gate: *is it
+  dangerous?* Stealing, spying, hidden sending, proven
+  with file-and-line evidence.
+- [done-claim-auditor](https://github.com/jasveerai88/done-claim-auditor)
+  — the same claim-vs-evidence logic pointed at agent
+  sessions: audits a coding agent's completion claims
+  against its own transcript.
+- [permission-audit](https://github.com/jasveerai88/permission-audit)
+  — audits a Claude Code permission allowlist for
+  wildcard grants and dangerous modes before they hurt
+  you.
+
+## Support
+
+If skill-claims catches a skill lying to you, consider
+[sponsoring](https://github.com/sponsors/jasveerai88).