accuracy-claim-kit 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,14 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.1.2] - 2026-11-20
4
+
5
+ Documentation only. The package code and API are the same as 0.1.1.
6
+
7
+ ### Changed
8
+
9
+ - The README says the kit returns refused when required details are missing, without a claim about how often that happens.
10
+ - A shorter package description that npm shows in full.
11
+
3
12
  ## [0.1.1] - 2026-11-18
4
13
 
5
14
  Wording fix. No verdict, threshold or API change.
@@ -13,7 +22,7 @@ First release. MIT licensed.
13
22
 
14
23
  ### What it does
15
24
 
16
- - `assessAccuracyClaim` judges a published accuracy or error-rate figure against the counts behind it and returns one of four verdicts: `supported`, `overstated`, `contradicted` or `refused`. Every result has reason codes, one plain sentence per code, and a cautious suggested statement that begins `On this sample` and says the interval covers sampling error only.
25
+ - `assessAccuracyClaim` judges a published accuracy or error-rate figure against the counts behind it and returns one of four verdicts: `supported`, `overstated`, `contradicted` or `refused`. Every result has reason codes, one plain sentence per code, and a suggested statement based on the result. If a sample gets an interval, the statement explains that the range covers sampling error only.
17
26
  - `refused` means "cannot tell", never "false". It outranks every other verdict, lists every failed precondition and returns no interval. Preconditions: counts disclosed, human or deterministic labels, random or stratified or named whole-set sampling, `n` at or above `minN` (default 30), strata that do not clearly differ, known retractions included, and optional gates for agreement, age and scope.
18
27
  - Sample claims are compared with the Wilson interval (95% by default). A figure equal to the observed rate to its printed precision is `supported` (`matches-observed-rate`). Bounds such as `99%+` and `<1%`, and absolute claims of 0% or 100%, are never excused that way.
19
28
  - Claims about a whole fixed set (`sampling: 'complete'` or `'held-out-split'`, with `benchmark.name` and wording that names the set) are compared with the exact count, with no interval.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Does this accuracy or error-rate number have the evidence to carry it? You give the kit the counts behind a published figure, who labeled the items and how they were chosen. It says whether those counts can carry the figure, and writes a cautious statement the evidence supports, as plain text you can publish instead.
4
4
 
5
- It is for people who publish a figure from their own evaluation: an AI-eval author, a team with a model or detector in production, a maintainer writing a benchmark README or a methodology page. Most real claims fail on a missing declaration (nobody wrote down who labeled, how items were picked, or what was measured), so expect `refused` often and read it as "write this down", not "false".
5
+ It is for people who publish a figure from their own evaluation: an AI-eval author, a team with a model or detector in production, a maintainer writing a benchmark README or a methodology page. The kit returns `refused` if required details are missing, such as who labeled the items, how they were chosen or what was measured. That verdict tells you what is missing.
6
6
 
7
7
  The kit checks evidence. It does not certify that a figure is true or that a system is accurate, and it makes no legal or regulatory determination. A `supported` verdict means the claim is no more favorable than the counts you supplied, or is an honest report of the rate you observed. See "What it does not do" and [METHOD.md](https://github.com/lkopietz3-byte/accuracy-claim-kit/blob/main/METHOD.md), which gives the reason for every rule and default.
8
8
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "accuracy-claim-kit",
3
- "version": "0.1.1",
4
- "description": "Does this accuracy or error-rate number have the evidence to carry it? Wilson intervals, inter-rater agreement (percent agreement, Cohen's kappa, Gwet's AC1) and a refuse-or-judge verdict on a claimed figure, computed from caller-supplied counts. Zero-dependency, offline TypeScript.",
3
+ "version": "0.1.2",
4
+ "description": "Check whether the counts and methods you supply support an accuracy or error-rate claim. Get a suggested statement and an uncertainty range when the sample allows it.",
5
5
  "keywords": [
6
6
  "accuracy",
7
7
  "error-rate",