@jrpool/kilotest 60.7.2 → 60.7.3

package/docs/AI-TOOLS.md

@@ -42,7 +42,7 @@ The external features that support the use of Kilotest as a collection of AI too
 - A [pull request](https://github.com/public-apis/public-apis/pull/6346/changes) to add Kilotest to the list of public APIs in the `public-apis` repository.
 - An [issue](https://github.com/APIs-guru/openapi-directory/issues/2677) to add Kilotest to `openapi-directory`.
 - A [pull request](https://github.com/w3c/wai-evaluation-tools-list/pull/1153) to add Kilotest to the WAI evaluation tools list.
-- An in-progress, incomplete submission of Kilotest as a plugin to OpenAI, under the organization name Kilotest, with organization ID `org-HOt5F01qJGQcl6V0sHj9HWr3`. Submission awaits correction of a defect that prevents submitters from recording their ID cards for identity verification. Upon that correction, the archive `kilotest-overview.zip` in this `docs` directory will be uploaded to OpenAI; a challenge token received from OpenAI will be stored in the `well-known` directory; the OpenAI “Scan Tools” function will be performed; and its requirement for 3 justifications per tool will be satisfied. Example for listDiagnoses: readOnlyHint: true — "The tool only retrieves stored report data; it modifies nothing." openWorldHint: false — "It reads only Kilotest's own database of reports; users can select among stored results but cannot direct it to arbitrary external resources." destructiveHint: false — "It deletes or overwrites nothing."
+- A submission, currently in review, of Kilotest as a [plugin to OpenAI](https://platform.openai.com/plugins?mcp=server%3Akilotest), with organization ID `org-HOt5F01qJGQcl6V0sHj9HWr3`. The OpenAI “Scan Tools” function succeeded without the expected demand for 3 hint justifications per tool, such as (for listDiagnoses) readOnlyHint: true — "The tool only retrieves stored report data; it modifies nothing." openWorldHint: false — "It reads only Kilotest's own database of reports; users can select among stored results but cannot direct it to arbitrary external resources." destructiveHint: false — "It deletes or overwrites nothing."
 - Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.jrpool/kilotest), maintained by the [Model Context Protocol](https://modelcontextprotocol.io/) project. The [PulseMCP](https://www.pulsemcp.com/servers) directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.
 - Registration of Kilotest with the [Smithery](https://smithery.ai/servers/pool/kilotest) MCP server registry.
 - Registration of Kilotest with the [Glama](https://glama.ai/mcp/connectors/com.kilotest/kilotest) MCP server registry, including a claim of ownership of the MCP connector by the maintainer via a `TXT` DNS record for `_glama-claim.kilotest.com`, with the value `glama_claim_gVV4HrPdcXr9TEfj1p3icEUBJ3_OVJGX`.

package/docs/openai-plugin/assets/logo.svg

@@ -0,0 +1,17 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" width="512" height="512" aria-hidden="true">
+  <style>
+    .lens { stroke: #005c5c; }
+    .zero { fill: #12a05c; }
+    @media (prefers-color-scheme: dark) {
+      .lens { stroke: #3fb8b8; }
+      .zero { fill: #4ce0a0; }
+    }
+  </style>
+  <g transform="scale(16)">
+    <circle class="lens" cx="14" cy="14" r="8.6" fill="none" stroke-width="2.6"/>
+    <path class="lens" d="M20.3 20.3L26.8 26.8" fill="none" stroke-width="2.8" stroke-linecap="round"/>
+    <ellipse class="zero" cx="9.9" cy="14" rx="1.5" ry="1.9"/>
+    <ellipse class="zero" cx="14" cy="14" rx="1.5" ry="1.9"/>
+    <ellipse class="zero" cx="18.1" cy="14" rx="1.5" ry="1.9"/>
+  </g>
+</svg>

package/docs/openai-plugin/mcp.json

@@ -0,0 +1,9 @@
+{
+  "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json",
+  "mcpServers": {
+    "kilotest": {
+      "type": "streamable-http",
+      "url": "https://kilotest.com/mcp"
+    }
+  }
+}

package/docs/openai-plugin/plugin.json

@@ -0,0 +1,133 @@
+{
+  "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
+  "name": "kilotest",
+  "version": "1.0.0",
+  "description": "Test web pages for front-end quality (accessibility, usability, and web standards, including HTML, CSS, ARIA, and WCAG) with an ensemble of 12 rule engines, and inspect the structured reports.",
+  "author": {
+    "name": "Jonathan Robert Pool",
+    "email": "info@kilotest.com",
+    "url": "https://kilotest.com"
+  },
+  "homepage": "https://kilotest.com",
+  "repository": "https://github.com/jrpool/kilotest",
+  "license": "MIT",
+  "keywords": [
+    "accessibility",
+    "a11y",
+    "usability",
+    "wcag",
+    "web-standards",
+    "testing"
+  ],
+  "extensions": {
+    "com.openai": {
+      "interface": {
+        "displayName": "Kilotest",
+        "shortDescription": "Test web-page quality",
+        "longDescription": "Kilotest tests public web pages for front-end quality: accessibility, usability, and web standards, including HTML, CSS, ARIA, and WCAG. It runs an ensemble of 12 independent rule engines and combines their results into structured reports organized as page, report, issue, violator element, and diagnosis. Use it to check whether a page has been tested, inspect the reported issues down to the violating elements and rule-engine diagnoses, and request or order new tests and retests. Kilotest testing emulates unprivileged public users and therefore produces no side effects other than those that human users could produce.",
+        "developerName": "Jonathan Robert Pool",
+        "category": "Developer Tools",
+        "capabilities": [
+          "Test web pages",
+          "Inspect test reports",
+          "Request tests and retests"
+        ],
+        "websiteURL": "https://kilotest.com",
+        "supportURL": "https://github.com/jrpool/kilotest/issues",
+        "privacyPolicyURL": "https://kilotest.com/privacy.html",
+        "termsOfServiceURL": "https://kilotest.com/terms.html",
+        "defaultPrompt": [
+          "Has Kilotest tested https://example.com? If so, summarize the issues in its latest report.",
+          "Show which elements on this page Kilotest reported as violating accessibility rules, and what the rule engines diagnosed.",
+          "Order a Kilotest test of https://example.org and wait for the report."
+        ],
+        "brandColor": "#005C5C",
+        "composerIcon": "./assets/logo.svg",
+        "logo": "./assets/logo.svg"
+      },
+      "review": {
+        "test_cases": {
+          "positive": [
+            {
+              "description": "Discover which pages of a named entity have been tested",
+              "prompt": "Name all pages of Kilotest that Kilotest has tested. I need only their names and not their URLs, test dates, test results, or any other information.",
+              "tools_triggered": "listPages",
+              "expected_behavior": "List the names of all Kilotest pages that Kilotest has tested."
+            },
+            {
+              "description": "Name the issues in a report",
+              "prompt": "Name the issues reported in the latest report about the Human Rights Watch contact page. I need only their names and not their descriptions, priorities, or any other information.",
+              "tools_triggered": "listPages, listIssues",
+              "expected_behavior": "Locate the latest report about the Human Rights Watch contact page and name all the issues reported in the report."
+            },
+            {
+              "description": "Drill down from a page to issues, violators, and diagnoses",
+              "prompt": "In the latest report about the Wikimedia Foundation (home) page, tell me which highest-priority issue was reported by the largest number of rule engines. For that issue, give me the XPath of the element that was reported as a violator by the largest number of rule engines. Finally, for that element and that issue, give me a one-sentence summary of the diagnoses. I do not need an issue explanation, additional information about the element, the number of diagnoses, their full texts, or the number or names of the rule engines that issued them.",
+              "tools_triggered": "listPages, listIssues, listViolators, listDiagnoses",
+              "expected_behavior": "Locate the latest report about the Wikimedia Foundation home page, find and name the highest-priority issue reported by the largest number of rule engines, find the element that was reported as a violator of that issue by the largest number of rule engines, name the XPath of that element, and summarize in one sentence the diagnoses of the violations of that issue by that element."
+            },
+            {
+              "description": "Request a test of an untested page",
+              "prompt": "Request a Kilotest test of Example12345 (https://example12345.info) and let me know whether the request was accepted. If it was, tell me how long I should expect to wait before the results will be available.",
+              "tools_triggered": "listPages, requestNewTest",
+              "expected_behavior": "Submit the request and reply that the results are likely to be available within one day, or that the request was rejected."
+            },
+            {
+              "description": "Order a retest to check on changes in test results",
+              "prompt": "Order a Kilotest retest of the Example page. When the report is completed, compare it with the previous report about that page and give me two lists: one naming any eliminated issues and the other naming any newly reported issues. I do not need the report dates, the issue counts, issue explanations, or any other information. If the order is rejected, or if the report is not completed within 4 minutes, stop waiting and tell me what happened.",
+              "tools_triggered": "listPages, orderRetest, awaitTest, listIssues",
+              "expected_behavior": "Locate the latest report about the Example page, order a retest of it, wait for the new report, compare it with the earlier report, and provide lists of the names of any issues reported by only the previous report and the names of any issues reported by only the new report. If the order is rejected, state that. If the order is accepted but the report is not completed within 4 minutes, state that and stop waiting."
+            }
+          ],
+          "negative": [
+            {
+              "description": "Test a private-network address",
+              "prompt": "Order a Kilotest test of http://192.168.1.1/admin."
+            },
+            {
+              "description": "Modify a tested page",
+              "prompt": "Remediate the accessibility problems Kilotest found on https://example.com."
+            },
+            {
+              "description": "Out-of-scope task",
+              "prompt": "Book a flight to Chicago for next Friday."
+            }
+          ]
+        },
+        "commerce": false,
+        "commerce_description": "This plugin does not sell products or process payments."
+      },
+      "publication": {
+        "countries": [
+          "US",
+          "CA",
+          "GB",
+          "IE",
+          "AU",
+          "NZ",
+          "DE",
+          "FR",
+          "NL",
+          "BE",
+          "AT",
+          "CH",
+          "ES",
+          "PT",
+          "IT",
+          "SE",
+          "NO",
+          "DK",
+          "FI",
+          "PL",
+          "CZ",
+          "JP",
+          "IN",
+          "BR",
+          "MX",
+          "IL"
+        ],
+        "release_notes": "Initial release: Kilotest MCP server plus an overview-and-workflow skill."
+      }
+    }
+  }
+}

package/docs/openai-plugin/skills/kilotest-overview/SKILL.md

@@ -0,0 +1,51 @@
+---
+name: kilotest-overview
+description: Kilotest uses an ensemble of 12 rule engines to test web pages for front-end quality, including accessibility, usability, and web standards such as HTML, CSS, ARIA, and WCAG. Use when learning which pages have been tested and which reports about test results for a page are available, investigating the reported issues, and getting a page tested for the first time or retested.
+---
+
+# Kilotest overview and workflow
+
+Kilotest tests web pages for front-end quality: accessibility, usability, and web standards, including HTML, CSS, ARIA, and WCAG.
+
+Kilotest integrates an ensemble of 12 independent rule engines to test a web page and stores the results as a structured report, from which it exposes results at selectable granularities.
+
+## Report hierarchy
+
+Results are organized as a hierarchy:
+
+- Tested pages.
+- For each tested page, reports from the tests of the page.
+- For each report, issues reported.
+- For each issue, violators of rules belonging to the issue (a violator is an element or the whole page).
+- For each violator, diagnoses by rule engines of its violations.
+
+## Typical workflow
+
+1. Call `listPages` to determine whether the page has been tested.
+2. If it has been tested:
+   - Use `listIssues` to identify reports about the page and summarize the issues reported by one report.
+   - Use `listViolators` to inspect elements violating rules belonging to an issue.
+   - Use `listDiagnoses` to inspect diagnoses by rule engines of violations of applicable rules by an element.
+   - Use `getReport` to retrieve an entire JSON report.
+   - Use `requestRetest` or `orderRetest` if the latest report is obsolete and you want the page retested.
+3. If the page has not been tested:
+   - Use `requestNewTest` or `orderNewTest` if you want the page tested.
+
+## Test request and order processing
+
+A test or retest **request** is reviewed by a manager. If approved, the testing is typically performed within a day.
+
+A test or retest **order** is reviewed automatically and immediately. If it is approved, you get a report identifier. You can then use `awaitTest`, with the identifier, to await test completion, which typically takes 2 to 4 minutes, or you can later use `listIssues` with the identifier to check the status.
+
+## Reporting to users
+
+When reporting test results to users:
+
+- Do not assume a requested or ordered test or retest has produced a report until Kilotest confirms this.
+- You may quote the Kilotest classification of issues into priorities, but identify any additional prioritization recommendations as your own.
+- You may quote recommendations for remediation made by rule engines in diagnoses, but identify any additional remediation recommendations as your own.
+
+## Further documentation
+
+- `https://kilotest.com/llms.txt`
+- `https://kilotest.com/llms-full.txt`

package/lhm.plugin.json

@@ -4423,5 +4423,5 @@
       }
     }
   ],
-  "version": "60.7.2"
+  "version": "60.7.3"
 }

package/openapi-3.0.yaml

@@ -2,7 +2,7 @@ openapi: 3.0.3
 info:
   title: Kilotest API
   description: 'Kilotest is an ensemble testing and reporting service for front-end web quality. It uses an ensemble of 12 rule engines to test web pages for front-end quality (accessibility, usability, and web standards, including HTML, CSS, ARIA, and WCAG). Kilotest acts as a web server for human users, an MCP server for AI platforms, and an API for programmatic access. With this API you can request that a page be tested or retested, discover available reports, incrementally retrieve facts from a report: from what reports are available, to what issues were reported in one report, to what elements violated one issue, to what diagnoses rule engines gave for one violation.'
-  version: 60.7.2
+  version: 60.7.3
   license:
     name: MIT
     url: https://github.com/jrpool/kilotest/blob/main/LICENSE

package/openapi.yaml

@@ -2,7 +2,7 @@ openapi: 3.1.0
 info:
   title: Kilotest API
   description: 'Kilotest is an ensemble testing and reporting service for front-end web quality. It uses an ensemble of 12 rule engines to test web pages for front-end quality (accessibility, usability, and web standards, including HTML, CSS, ARIA, and WCAG). Kilotest acts as a web server for human users, an MCP server for AI platforms, and an API for programmatic access. With this API you can request that a page be tested or retested, discover available reports, incrementally retrieve facts from a report: from what reports are available, to what issues were reported in one report, to what elements violated one issue, to what diagnoses rule engines gave for one violation.'
-  version: 60.7.2
+  version: 60.7.3
   license:
     name: MIT
     identifier: MIT

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jrpool/kilotest",
-  "version": "60.7.2",
+  "version": "60.7.3",
   "description": "Ensemble testing for front-end web quality",
   "type": "module",
   "main": "index.ts",

package/server.json

@@ -6,7 +6,7 @@
     "url": "https://github.com/jrpool/kilotest",
     "source": "github"
   },
-  "version": "60.7.2",
+  "version": "60.7.3",
   "remotes": [
     {
       "type": "streamable-http",

package/web/terms/index.html

@@ -13,7 +13,7 @@
       <h2>What Kilotest does</h2>
       <p>Kilotest tests publicly reachable web pages for front-end quality (accessibility, usability, and web standards) with an ensemble of rule engines, stores the results as reports, and makes the reports publicly available through its web site, API, and MCP server. Owners of tested pages can request deletion of reports under the <a href="/privacy.html">privacy policy</a>.</p>
       <h2>What you submit</h2>
-      <p>When you request or order a test or retest, you ask Kilotest to fetch the page and publish a report about it. You do not need the page owner&rsquo;s permission to do so. Submit only publicly reachable URLs, and do not submit URLs that require credentials or that would put secrets into a public report. Kilotest may reject any request or order.</p>
+      <p>When you request or order a test or retest, you ask Kilotest to fetch the page and publish a report about it, including findings about its defects and exerpts from it. You do not need the page owner&rsquo;s permission to do so. Submit only publicly reachable URLs. Do not submit URLs that require credentials, contain secrets, or could cause secret or unlawful content to appear in a Kilotest report. Kilotest may reject any request or order.</p>
       <h2>Acceptable use</h2>
       <p>Do not use Kilotest to attack, probe, or overload web sites, including this one, and do not submit unlawful or deceptive content. Kilotest may rate-limit, refuse, or block any use.</p>
       <p>Reports attribute results to third-party rule engines, whose own terms may restrict the use of those results. In particular, results derived from WAVE are subject to the <a href="https://wave.webaim.org/terms">WAVE terms of use</a>, which prohibit their use for any purpose related to a legal action without written permission from WebAIM.</p>