@jrpool/kilotest 46.0.11 → 46.0.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -8,7 +8,8 @@
8
8
  "Exec(find)",
9
9
  "Exec(git show)",
10
10
  "Exec(git commit)",
11
- "Exec(git check-ignore)"
11
+ "Exec(git check-ignore)",
12
+ "Exec(git remote)"
12
13
  ]
13
14
  }
14
15
  }
package/README.md CHANGED
@@ -1,10 +1,15 @@
1
1
  # Kilotest
2
2
 
3
+ [![npm version](https://img.shields.io/npm/v/@jrpool/kilotest?label=npm)](https://www.npmjs.com/package/@jrpool/kilotest)
4
+ [![license](https://img.shields.io/github/license/jrpool/kilotest?label=license)](LICENSE)
5
+ [![MCP Registry](https://img.shields.io/badge/MCP_Registry-active-success)](https://registry.modelcontextprotocol.io/v0/servers?search=kilotest)
6
+ [![GitHub stars](https://img.shields.io/github/stars/jrpool/kilotest?label=stars)](https://github.com/jrpool/kilotest/stargazers)
7
+
3
8
  An ensemble testing and reporting service for front-end web quality
4
9
 
5
10
  ## Features
6
11
 
7
- This application uses an ensemble of 10 rule engines to test public web pages for front-end quality (i.e. accessibility, usability, and standards conformity).
12
+ This application uses an ensemble of 12 rule engines to test public web pages for front-end quality (i.e. accessibility, usability, and standards conformity).
8
13
 
9
14
  The testing paradigm employed by Kilotest is discussed in these papers:
10
15
 
@@ -24,7 +29,7 @@ An active production instance of Kilotest may require multiple testing agents to
24
29
 
25
30
  In the steps below, hosts `T` and `K` may be the same host or two different hosts. Host `T` can be a Debian stable, Ubuntu LTS, Windows, or macOS host. Host `K` can be any server host that can run the latest LTS version of Node.js. If hosts `T` and `K` differ, then they must be open to `https` traffic and host `K` must permit `https` requests from host `T`.
26
31
 
27
- 1. Clone the [Testaro project](https://github.com/jrpool/testaro) into a new directory on host `T`.
32
+ 1. Clone the [Testaro project](https://github.com/YRA-Tech/testaro) into a new directory on host `T`.
28
33
  1. In that directory, install the Testaro dependencies: `npm install`.
29
34
  1. Update the Testaro dependencies and rebuild: `npm run deps`.
30
35
  1. Clone the Kilotest repository into a new directory on host `K`.
@@ -43,20 +48,21 @@ Contributions are welcome! You can use GitHub issues to initiate discussions and
43
48
 
44
49
  ## Making Kilotest a service
45
50
 
46
- See the `SERVICE.md` file for instructions on how to make Kilotest a service.
51
+ See the [`SERVICE.md`](docs/SERVICE.md) file for instructions on how to make Kilotest a service.
47
52
 
48
53
  ## Using Kilotest as an AI tool
49
54
 
50
55
  Kilotest is not only a web application but also an API and an MCP server. AI platforms and models can use Kilotest via those interfaces.
51
56
 
52
- The URL for models using the MCP server is `http://kilotest.com/mcp`. The MCP server is listed at these services:
57
+ The URL for models using the MCP server is `https://kilotest.com/mcp`. The MCP server is listed at these services:
53
58
 
54
59
  - [Smithery](https://smithery.ai/servers/pool/kilotest)
55
60
  - [Glama](https://glama.ai/mcp/connectors/com.kilotest/kilotest)
56
61
 
57
62
  There is additional documentation at:
58
63
 
59
- - [AI-TOOLS.md](AI-TOOLS.md)
60
- - [llms.txt](llms.txt)
61
- - [llms-full.txt](llms-full.txt)
62
- - [QAI](https://kilotest.com/qai)
64
+ - [QAI](https://kilotest.com/qai): a tutorial for allowing AI assistants to use Kilotest
65
+ - [AI-TOOLS.md](docs/AI-TOOLS.md): how Kilotest implements the MCP protocol
66
+ - [llms.txt](llms.txt): basic documentation for LLMs
67
+ - [llms-full.txt](llms-full.txt): comprehensive documentation for LLMs
68
+ - [lhm.plugin.json](lhm.plugin.json): documentation for registration with the LobeHub MCP Market
package/api/util.js CHANGED
@@ -35,7 +35,7 @@ exports.getResponseMetadata = () => ({
35
35
  exports.getToolsFacts = () => ({
36
36
  'name': 'Kilotest',
37
37
  'description': {
38
- 'what Kilotest does': 'Kilotest tools generate and make available findings about the front-end quality (i.e. accessibility, usability, and standards conformity) of web pages. A Kilotest job generates findings by using Testaro to test a page against more than a thousand rules defined by an ensemble of ten rule engines. Testaro produces a report of the job. The report describes violations of the rules. Kilotest uses Testilo to enhance the report with a classification of the rule violations into about 300 issues. Kilotest makes facts about the issues and the violations retrievable at four levels of granularity.',
38
+ 'what Kilotest does': 'Kilotest tools generate and make available findings about the front-end quality (i.e. accessibility, usability, and standards conformity) of web pages. A Kilotest job generates findings by using Testaro to test a page against more than a thousand rules defined by an ensemble of twelve rule engines. Testaro produces a report of the job. The report describes violations of the rules. Kilotest uses Testilo to enhance the report with a classification of the rule violations into about 300 issues. Kilotest makes facts about the issues and the violations retrievable at four levels of granularity.',
39
39
  'how to retrieve findings': {
40
40
  'level 1': 'Use the listReports tool to get a list of available reports.',
41
41
  'level 2': 'Use the listIssues tool to get a list of issues in one report.',
package/docs/AI-TOOLS.md CHANGED
@@ -4,56 +4,70 @@
4
4
 
5
5
  Until 2026 Kilotest was intended, and implemented, as a web application performing a service for human users.
6
6
 
7
- Beginning in May 2026, it [became evident](https://github.com/jrpool/kilotest/issues/2) that Kilotest could also act as a provider of tools for use by language models. Language models are asked for help in all domains, including the domain of software quality. When asked about the front-end quality (accessibility, usability, and standards conformity) of specific web pages, models gave answers without the use of tools. The answers were at best fragmentary but often speculative and fabricated. If Kilotest provided its functionalities to language models, models could give more inexpensive, comprehensive, factual, authoritative, and grounded answers. Every reported defect could be documented and ascribed to one or more specific rule engines in the Kilotest ensemble.
7
+ Beginning in May 2026, it [became evident](https://github.com/jrpool/kilotest/issues/2) that Kilotest could operate as a set of tools for language models to allow them to improve their answers to questions about the front-end quality (accessibility, usability, and standards conformity) of specific web pages.
8
8
 
9
- Given the potential of Kilotest to serve language models and the expected continued growth in the share of questions that are directed to AI platforms that use language models, a decision was made to **make Kilotest discoverable, usable, and, where appropriate, used as a provider of tools for language models**.
9
+ A decision was made to **make Kilotest discoverable, usable, and, where appropriate, used as a provider of tools for language models**.
10
10
 
11
11
  ## Terms
12
12
 
13
- - **Language model**: A model (e.g., Claude Haiku 4.5, Kimi K2.6, GPT-5.4, Gemini 3.1 Pro) that can consume and generate text, images, or other content.
13
+ - **Language model**: A model (e.g., Claude Haiku 4.5, Kimi K2.7, GPT-5.4, Gemini 3.1 Pro) that can consume and generate text, images, or other content.
14
14
  - **AI platform**: A platform (e.g., Claude Desktop, Perplexity, ChatGPT, Gemini 3.5 Flash) that gives human users access to the services of language models and connects language models to productivity resources.
15
15
  - **tool**: A specialized productivity resource providing a capability that a language model needs but does not have. In the OpenAPI specification, the term _operation_ is used to mean tool.
16
16
  - **connector**: A service that allows AI platforms to enable their language models to discover, evaluate, and use tools.
17
17
 
18
- ## Internal additions
18
+ ## Internal features
19
19
 
20
- The internal changes that have been made in the Kilotest codebase to support the use of Kilotest as a collection of tools are:
20
+ The internal features that make Kilotest a collection of tools for language models are:
21
21
 
22
- - An API, consisting of:
23
- - Additions to `index.js`.
24
- - Additions to `util.js`.
25
- - An `api` directory containing:
26
- - Eight tool modules.
27
- - A `util.js` module with shared functions used by tool modules.
28
- - A `routes.js` file.
29
- - A `schemas.js` file.
30
- - A `testSuite.js` module for testing the API.
22
+ - An API, with specific functional and utility modules in the `api` directory.
23
+ - Tests of the functionalities of the API.
31
24
  - A [`JSON-LD`](https://json-ld.org/) script in the `index.html` file, providing structured data about the Kilotest API.
32
- - Additions to the `env.example` file.
25
+ - Environment variables in the `env.example` file.
33
26
  - An `llms.txt` file and an `llms-full.txt` file, documenting the use of Kilotest by language models, conforming to the [llms-txt](https://llmstxt.org/) specification.
34
- - An `openapi.yaml` file, documenting the Kilotest API, conforming to the [OpenAPI specification](https://spec.openapis.org/oas/v3.1.0).
27
+ - A generated `openapi.yaml` file, documenting the Kilotest API, conforming to the [OpenAPI specification](https://spec.openapis.org/oas/v3.1.0).
35
28
  - An `mcp.js` file, providing an MCP server for Kilotest.
29
+ - A `server.json` file, conforming to the [MCP server schema](https://modelcontextprotocol.io/), and a GitHub Actions workflow (`publish-mcp.yml`) that authenticates via GitHub OIDC and publishes that file to the official MCP Registry whenever `server.json` or `api/version.js` changes on the `main` branch.
30
+ - `mcp`, `mcp-server`, and `modelcontextprotocol` keywords in `package.json`, supporting discovery of the `@jrpool/kilotest` npm package by tools and humans searching the npm registry for MCP servers.
31
+ - An `lhm.plugin.json` file, conforming to the [LobeHub plugin manifest format](https://market.lobehub.com/s/publish-mcp/references/manifest), declaring Kilotest as a plugin in the [LobeHub MCP Market](https://lobehub.com/mcp). Publishing and updating are non-automated (`lhm plugin publish`/`update`), because LobeHub requires browser-based OAuth and does not support machine-to-machine publishing.
36
32
  - A `sitemap.xml` file.
37
- - Additions to the `README.md` file.
33
+ - Documentation in the `README.md` file.
38
34
  - This `AI-TOOLS.md` file.
39
35
  - A [tutorial on authorizing language models to use Kilotest](https://kilotest.com/qai/).
40
36
 
41
- ## External actions
37
+ ## External features
42
38
 
43
- The external actions that have been taken to support the use of Kilotest as a collection of AI tools are:
39
+ The external features that support the use of Kilotest as a collection of AI tools are:
44
40
 
45
41
  - A [pull request](https://github.com/public-apis/public-apis/pull/6346/changes) to add Kilotest to the list of public APIs in the `public-apis` repository.
46
42
  - An [issue](https://github.com/APIs-guru/openapi-directory/issues/2677) to add Kilotest to `openapi-directory`.
47
43
  - A [pull request](https://github.com/w3c/wai-evaluation-tools-list/pull/1153) to add Kilotest to the WAI evaluation tools list.
44
+ - Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io/v0/servers?search=kilotest) (`io.github.jrpool/kilotest`), maintained by the [Model Context Protocol](https://modelcontextprotocol.io/) project. The [PulseMCP](https://www.pulsemcp.com/servers) directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.
48
45
  - Registration of Kilotest with the [Smithery](https://smithery.ai/servers/pool/kilotest) MCP server registry.
49
46
  - Registration of Kilotest with the [Glama](https://glama.ai/mcp/connectors/com.kilotest/kilotest) MCP server registry.
47
+ - a [pull request](https://github.com/TensorBlock/awesome-mcp-servers/pull/2221) (on 2026-09-07) to add Kilotest to the [Awesome MCP Servers](https://github.com/TensorBlock/awesome-mcp-servers) list.
50
48
  - Registration of Kilotest with the [RapidAPI](https://rapidapi.com/jrpool/api/kilotest/playground/apiendpoint_0f03577a-ff9a-472a-a0ed-533bd198981a) Hub.
49
+ - Registration of Kilotest with the [LobeHub MCP Market](https://lobehub.com/mcp) (identifier: `jrpool-kilotest`), enabled with an `lhm.plugin.json` file.
51
50
  - Deployment of an MCP server in HTTP mode on the Kilotest service host.
52
51
  - Configuration of Claude Desktop on the local development host and the `claude.ai` web application to connect Claude Desktop models to the Kilotest MCP server. The configuration was performed in the UI of each platform with the addition of Kilotest as a _connector_. The user used the `Customize/Connectors/Add connector/Add custom connector` interface, providing these data before activating the `Add` button:
53
52
 
54
53
  - Name: Kilotest
55
54
  - Remote MCP server URL: `https://kilotest.com/mcp`
56
55
 
56
+ - MCP-related topics added to the [repository details](https://github.com/jrpool/kilotest).
57
+ - An [MCP Registry badge](https://registry.modelcontextprotocol.io/v0/servers?search=kilotest) added to the `README.md` file.
58
+ - A link to the MCP server on the `README.md` page of the [`testaro-issues`](https://github.com/jrpool/testaro-issues) repository.
59
+ - Registration of Kilotest and its sitemap at [Google Search Console](https://search.google.com/search-console?resource_id=sc-domain:kilotest.com) and [Bing Webmaster Tools](https://www.bing.com/webmasters/home?siteUrl=https://kilotest.com/).
60
+
61
+ ## Required maintenance
62
+
63
+ Whenever the `/api/version.js` file is updated:
64
+
65
+ - Update the `server.json` file (version field).
66
+ - Update the `lhm.plugin.json` file (version field).
67
+ - Regenerate `openapi.yaml` (`npm run generate:openapi`).
68
+ - Create a pull request and merge to `main` (triggers `publish-mcp.yml` automatically for the MCP Registry).
69
+ - Execute `lhm plugin update --dir .` (for LobeHub).
70
+
57
71
  ## Use cases
58
72
 
59
73
  The rationale for Kilotest as a collection of tools for language models is set forth in the `llms-full.txt` file and is not repeated here.
@@ -71,148 +85,17 @@ Among these use cases, case 3 would make it feasible for the user to tell an AI
71
85
 
72
86
  Use cases 1, 2, 4, 5, and 6 exemplify a widespread expectation and demand for AI platform capability. The commonality is: “I have a question; answer it.” If Kilotest can be employed as an expert for AI platforms in relevant cases, platforms will be more successful in satisfying that demand. At present this is a difficult problem because of platform limitations and a lack of standardization.
73
87
 
74
- ## Implementation strategy
75
-
76
- Connector and tool discovery and utilization have been only partly standardized. Major differences in protocols exist among model and platform families. Therefore, small testable increments of improvement in the connector and tool functionality of Kilotest can best be defined by model and platform family and by use-case characteristics.
77
-
78
- One benefit of such incrementalism is that, after the first increment succeeds, it becomes possible to make and test external changes publicizing the fact that a particular platform-model combination delivers unprecedentedly inexpensive, comprehensive, and truthful answers in a particular class of use cases to questions about front-end web quality.
79
-
80
- Another benefit is that subsequent increments can be defined incrementally rather than in advance. Lessons learned from the work on each increment can inform the choice of what to work on next.
81
-
82
- ## Implementation history
83
-
84
- The Kilotest project has been attempting to solve the just-described problem. A chronology of this work follows.
85
-
86
- ### Increment 1
87
-
88
- In the first increment, the objective was to make Anthropic Claude models use Kilotest to help them answer questions from developers using the Claude Desktop application for use case 3, where the code in question is already deployed as a public web page.
89
-
90
- A locally installed MCP server, `@ivotoby/openapi-mcp-server`, was used. It turned out to be incompatible with Claude Desktop, so even though connections to Kilotest were made and calls to its tools were submitted the responses did not arrive.
91
-
92
- The model behavior motivated an architectural change in the MCP server and improvements in the naming and description of the Kilotest tools before the next increment.
93
-
94
- ### Increment 2
95
-
96
- Increment 2 repeated the exercise of increment 1 with the `claude.ai` web application instead of Claude Desktop, and with the improvements indicated by increment 1 made.
97
-
98
- With Kilotest added as a custom connector, the user asked the platform, “I want to know whether the home page of the nonprofit organization named ‘Open Secrets’ has quality defects, such as with accissibility [_sic_] and usability.”
99
-
100
- It was discovered that Claude Haiku 4.5 with Extended capability used Kilotest tools, but Claude Haiku 4.5 without Extended capability did not.
101
-
102
- When Kilotest was used, it was used appropriately for the task, but the planning was suboptimal, because the model initially stated an intention to recommend the web page for testing instead of first checking for an existing available report on it.
103
-
104
- The output to the user was a mixture of Kilotest results with knowledge and judgments of the model. Kilotest results were presented with some selective pruning, interpretation, and advice on what to do.
105
-
106
- The trial in increment 2 suggested some opportunities for further improvements:
107
-
108
- - Clearer instructions about the sequencing of tool calls
109
- - Addition of instructions to attribute the interpretive additions by the model to the model or its other sources rather than to Kilotest.
110
-
111
- ### Increment 3
112
-
113
- After the above-described improvements were made, a similar exercise, with the home page of a company named “CMT Services”, was performed again in Claude Desktop. In this case Claude Haiku 4.5 succeeded in recognizing the relevance of Kilotest and used its tools, though imperfectly. Instead of using Kilotest to check for an existing report on the page, the model searched for it in a cached file with:
114
-
115
- ```bash
116
- Bash
117
- jq '.[] | .tested_web_page.description' /Users/pool/.claude/projects/-Users-pool-Documents/18b3821c-b60d-4f63-bf3f-542830ed3bc4/tool-results/toolu_012qMKW66d88vWjp8JKj5qNn.json 2>/dev/null | grep -i "cmt" || echo "CMT Services not found in tested pages"
118
- ```
119
-
120
- Claude Haiku then incorrectly concluded: `CMT Services hasn't been tested yet. Let me search for their website so I can recommend it for quality testing.` The model asked the user for the URL and said:
121
-
122
- ```text
123
- Once you provide the URL, I'll:
124
-
125
- Recommend their home page for quality testing with Kilotest
126
- Retrieve the quality report showing accessibility and usability defects
127
- Provide you with a detailed analysis of any issues found
128
- ```
129
-
130
- The user provided the URL, and Claude Haiku then submitted a recommendation to Kilotest to test the page. The submission was successful and, as expected, yielded an error message: `{"error":"A report about the page is already available"}`.
131
-
132
- The model drew the correct conclusion from this and (contradicting itself) announced: `Great news—there's already a quality report for CMT Services! Let me retrieve it for you.` The model did so and gave the user an appropriate rendition of the results, modulo attribution. The results were introduced with `The Kilotest analysis found 51 quality defects on the CMT Services home page, affecting 163 HTML elements. Here's a breakdown:`. Some mildly interpretive additions, such as calling “high-priority” issues “critical”, were made without attribution, making it easy for a user to infer that they were made by Kilotest tools rather than the model.
133
-
134
- The same trial was repeated, with a different website, using Claude Sonnet 4.6 with Low effort. This model immediately checked for and found an already available report and used it in the production of output to the user. The output was somewhat less interpretive than that of Claude Haiku.
135
-
136
- Increment 3 showed that, after the naming, instructions, and connector configuration were improved, the relatively inexpensive Claude models use Kilotest where appropriate.
137
-
138
- The misbehavior of Claude Haiku suggested that further improvements in the instructions may improve the utilization of Kilotest, namely:
139
-
140
- - a warning not to use cached lists of available reports
141
- - stronger advice to avoid implicitly attributing interpretations, judgments, and advice to Kilotest.
142
-
143
- ### Increment 4
144
-
145
- The first trial in which the user asked about a page **without** an available report repeated the conditions of Increment 3, except for the web page in question and the model. The model here was Claude Haiku in Extended mode, an option available with `claude.ai` but not Claude Desktop.
146
-
147
- The user asked: “I want to know whether the home page of Milgard Windows and Doors, at `www.milgard.com`, has quality defects, such as with accessibility and usability.”
148
-
149
- The model immediately:
150
-
151
- - found Kilotest
152
- - checked for an available report about the page
153
- - found no report
154
- - submitted a recommendation to test the page
155
-
156
- The recommendation was successfully received by the Kilotest service.
157
-
158
- During the workflow the model kept the user informed. After the last step, the model summarized the type of report that the user could expect and advised the user to check directly with the Kilotest UI at `https://kilotest.com/targets` in 24 to 48 hours for the results.
159
-
160
- The behavior of Claude Haiku in Extended mode was nearly perfect. The naming of the page in the recommendation was “Milgard Windows and Doors home page”, whereas “Milgard Windows and Doors” would have followed the existing naming pattern. This deviation suggests adding a specific instruction to name home pages without anything more than the name of the organization.
161
-
162
- The most notable fact was that Claude Haiku converted a long workflow to a short one by ending the output with a recommendation to the user to get the test results by self-service. The instructions to models presumed that the user and the model would jointly decide to maintain their relationship until the testing is completed and the model would then interpret the results for the user. Instead, Claude Haiku had enough imagination to suggest a viable workflow not even hinted at by the instructions.
163
-
164
- This result suggested that the instructions should be revised to describe, as a valid option, a short workflow ending with a recommendation to the user to get the test results by self-service after the model submits a testing recommendation.
165
-
166
- ### Increment 5
167
-
168
- In the next increment the AI platform was changed to [Perplexity Pro](https://www.perplexity.ai/) on the web. With the platform UI, two configuration steps were taken:
169
-
170
- 1. Kilotest was added as a custom connector, and the list of connectors thereafter included Kilotest.
171
- 1. In the input section of the chat interface, with the “Add files or tools” button (visually `+`), Kilotest was added as a connector. Thereafter, a “Kilotest” button appeared in the input section to confirm the availability of that connector.
172
-
173
- The first model used was GPT-5.4.
174
-
175
- The prompt was: “I want to know whether the home page of the nonprofit organization named ‘Center for Democracy and Technology’ has quality defects, such as with accessibility and usability.”
176
-
177
- GPT-5.4 responded by correctly using two Kilotest tools, first to check for an available report and then to inspect the results of the report about the specified web page. The model then appropriately itemized and summarized findings from the report, without any substantial interpretive additions.
178
-
179
- The model was then changed to Sonar 2, created by Perplexity itself.
180
-
181
- The results were similar, with one exception: The answer from Sonar 2 included advice and judgments (“most urgent fixes”, “especially notable”), without any attribution, so a user might wrongly infer that these were generated by Kilotest tools.
182
-
183
- For both models, the same exercise was also attempted with the second configuration omitted, i.e. ••without** Kilotest being a named connector in the input section of the chat interface. In both cases, the models created their own unaided assessments and failed to mention or use the Kilotest tools.
184
-
185
- The [Perplexity documentation](https://www.perplexity.ai/help-center/en/articles/13915507-adding-custom-remote-connectors) states that as of now the installed Perplexity application for macOS does not support remote connectors.
186
-
187
- ### Increment 6
188
-
189
- Once the compatibility of Kilotest as a connector with AI platforms of two different vendors had been confirmed, the focus of work returned to completion of the missing endpoints of the API. The first was an endpoint that would let a model efficiently search for an available report about a particular web page.
190
-
191
- A new `summarizeQualityOfMatchingWebPages` tool provided inputs for fragments of a description and of a URL hostname of a web page and responds with summary data about all and only the available reports of pages that match at least one of those fragments, where matching means either including or being included by the provided fragment, case-insensitively.
192
-
193
- ### Increment 7
194
-
195
- Further experimentation made it clear that Increment 6 had been a detour. The problem with Increment 6 was that it does not improve efficiency by eliminating the need to get a list of all reports. Models must get a list of all reports in order to be confident that the answer to the question “Is a report available yet?” is correct. Any algorithm that matches a page description and a URL with those of existing reports is error-prone, organization names change, website designs change, URLs change, and organizations are related to one another. Typically, a desired report does not yet exist. In that case, the model needs the full list of reports in order to understand the naming convention for pages and to adhere to that convention when submitting a request to have the page of interest tested.
196
-
197
- For this reason, the tool developed in increment 6 was deleted.
198
-
199
- ### Increment 8
88
+ ## Future work
200
89
 
201
- The next increment reorganized the API by placing API-related code into a new `api` directory.
90
+ To-dos recommended by Claude Sonnet 5 Medium on Devin:
202
91
 
203
- ### Increment 9
92
+ - **No content/backlink outreach.** Registrations are passive listings; there's no blog post, Show HN, or subreddit post (r/modelcontextprotocol, r/accessibility, r/webdev) building external backlinks and social-discovery signal, which the doc's own "difficult problem…lack of standardization" framing suggests is exactly the gap left for the **human**-driven parts of discoverability.
204
93
 
205
- The collection of tools was enlarged to a total of eight tools:
94
+ - **Other directories not yet targeted.** The [mcp.so](https://mcp.so/submit) directory offers a free tier but steers submitters toward a $39 paid listing; deferred as a commercial decision. The Docker MCP Catalog is not a fit because it requires stdio transport and a Dockerfile, and Kilotest is streamable-http.
206
95
 
207
- - listReports
208
- - listIssues
209
- - listViolators
210
- - listDiagnoses
211
- - getReport
212
- - requestRetest
213
- - requestTest
214
- - requestFeature
96
+ - **Client-onboarding friction isn't documented.** Some MCP clients need the `mcp-remote` npx bridge to use a remote streamable-HTTP server like Kilotest's. Ready-made Claude Desktop/Cursor/Windsurf config instructions added to QAI would convert "discovered" into "used" faster, since case 3 of your use cases assumes a technically savvy discoverer but not all users are.
215
97
 
216
- ## Increment 10
98
+ Other future work:
217
99
 
218
- In the next increment, API code was refactored and the tools were made more consistent with one another.
100
+ - Add to QAI, or replace much of the current QAI content with, an [Add Custom Connector link](https://claude.ai/customize/connectors?modal=add-custom-connector&connectorName=Kilotest&connectorUrl=https%3A%2F%2Fkilotest.com%2Fmcp).
101
+ - Consider subscribing to a `Claude.ai` Team plan and submitting Kilotest to the [Claude Connectors Directory](https://claude.com/docs/connectors/building/submission). Details provided by GLM-5.2 High: “If accepted, Kilotest would get a permanent listing URL like `https://claude.ai/directory/connectors/kilotest` with a Connect button, and would appear in Claude’s in-product browse/search and ‘Suggested Connectors’. This is the highest-value listing for your use cases, because it puts Kilotest directly in front of Claude users who have not explicitly searched for it.”
package/index.html CHANGED
@@ -18,7 +18,7 @@
18
18
  "dateModified": "2026-08-26",
19
19
  "applicationCategory": "Front-end quality (i.e. web accessibility, usability, and standards conformity) testing and reporting service",
20
20
  "isAccessibleForFree": true,
21
- "description": "Testing of web pages for front-end quality (i.e. web accessibility, usability, and standards conformity) with an ensemble of 10 rule engines and provision of results at selectable levels of detail.",
21
+ "description": "Testing of web pages for front-end quality (i.e. web accessibility, usability, and standards conformity) with an ensemble of 12 rule engines and provision of results at selectable levels of detail.",
22
22
  "softwareHelp": {
23
23
  "@type": "CreativeWork",
24
24
  "url": "https://kilotest.com/tutorial.html"
@@ -28,7 +28,7 @@
28
28
  "WCAG 2.2 conformance evaluation",
29
29
  "HTML5 conformance evaluation",
30
30
  "CSS3 conformance evaluation",
31
- "Ensemble of 10 rule engines performing more than a thousand tests",
31
+ "Ensemble of 12 rule engines performing more than a thousand tests",
32
32
  "Multi-page, single-page, single-issue, and single-element reporting",
33
33
  "Priority-ranked issue lists",
34
34
  "Public API, connector, and tools for language-model integration"
@@ -67,7 +67,7 @@
67
67
  <p>Choosing among the <a href="https://www.w3.org/WAI/test-evaluate/tools/list/">many automated rule engines</a> is hard, because they <a href="https://arxiv.org/pdf/2304.07591">vary in what and how they test</a>.</p>
68
68
  <p><strong>Ensemble testing</strong>, instead of choosing only one, combines <strong>multiple rule engines</strong>. It can discover more issues than single-rule-engine testing, and the rule engines can check one another&rsquo;s quality.</p>
69
69
  <h2>The rule engines</h2>
70
- <p>The Kilotest ensemble is <a href="https://github.com/jrpool/testaro?tab=readme-ov-file#dependencies">10 rule engines</a> developed by teams in Australia, Canada, Denmark, Luxembourg, Portugal, Spain, Switzerland, the United Kingdom, and the United States.</p>
70
+ <p>The Kilotest ensemble is <a href="https://github.com/YRA-Tech/testaro?tab=readme-ov-file#dependencies">12 rule engines</a> developed by teams in Australia, Canada, Denmark, Luxembourg, Portugal, Spain, Switzerland, the United Kingdom, and the United States.</p>
71
71
  <p>The rule engines perform about 1,300 tests in total. <a href="https://medium.com/cvs-health-tech-blog/how-to-run-a-thousand-accessibility-tests-63692ad120c3">Open-source software</a> developed initially at CVS Health drives the rule engines, tries to weed out invalid results, and distills the findings into about 380 <q>issues</q>.</p>
72
72
  <h2>Features</h2>
73
73
  <p>With Kilotest, you can get results of already performed tests and propose web pages for testing. (Immediate testing on request may become available in a future version.)</p>