@mknrt/autotests-overkill 1.2.2 → 1.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -15,7 +15,7 @@ Runtime requirement:
15
15
  - Knowledge-store-backed retrieval for asset search, area context, impact analysis, and blueprint support
16
16
  - Optional `fis_platform` backend knowledge through an evidence-driven `ApiDiscoveryCatalog`, `api-interaction` documents, and `backend-endpoint` contracts
17
17
  - Optional ReportPortal and GitLab adapters with graceful artifacts-first fallback
18
- - Codex plugin packaging via `.codex-plugin/plugin.json` and `.mcp.json`
18
+ - Codex MCP setup for npm package usage and local checkout development
19
19
 
20
20
  ## Repository layout
21
21
 
@@ -48,7 +48,7 @@ Repository-local development:
48
48
  ## Daily workflow
49
49
 
50
50
  1. Refresh local knowledge with `npm run index`.
51
- 2. Start the MCP server through the checked-in MCP manifest or by running `npx autotests-overkill mcp`.
51
+ 2. Start the MCP server through the consumer repository's `.codex/config.toml`, or run `npx autotests-overkill mcp` directly for manual checks.
52
52
  3. Use `find_frontend_contract`, `find_existing_test_assets`, and `generate_spec_blueprint` for a new feature draft.
53
53
  4. Use `review_test_draft` to critique a draft for reuse, selector stability, setup drift, and likely gap coverage before editing `autotests2`.
54
54
  5. Use `find_backend_test_context` when a request flow needs backend validation, params, async completion, or contract-drift evidence.
@@ -132,12 +132,22 @@ Every tool returns the same normalized shape:
132
132
  - `confidence`
133
133
  - `warnings`
134
134
 
135
- ## Codex plugin packaging
135
+ ## Codex MCP setup
136
136
 
137
137
  This repository can be consumed either from a Git checkout or as a published npm package:
138
138
 
139
- - `.codex-plugin/plugin.json` exposes the plugin metadata
140
- - `.mcp.json` registers the local MCP server command for checkout-based usage
139
+ - `.mcp.json` registers a local checkout MCP command only
141
140
  - `skills/` is auto-discovered as the bundled skill set
142
141
 
143
- See [consumer-integration.md](/C:/gitrep/autotests-overkill/docs/consumer-integration.md), [operator-cookbook.md](/C:/gitrep/autotests-overkill/docs/operator-cookbook.md), and [plugin-packaging.md](/C:/gitrep/autotests-overkill/docs/plugin-packaging.md) for setup details.
142
+ For npm package consumers such as `autotests2`, prefer a project-local Codex config:
143
+
144
+ ```toml
145
+ [mcp_servers.autotests-overkill]
146
+ command = "node"
147
+ args = ["./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js"]
148
+ enabled = true
149
+ startup_timeout_sec = 20
150
+ tool_timeout_sec = 120
151
+ ```
152
+
153
+ See [consumer-integration.md](docs/consumer-integration.md), [operator-cookbook.md](docs/operator-cookbook.md), and [plugin-packaging.md](docs/plugin-packaging.md) for setup details.
@@ -1,3 +1,4 @@
1
+ import path from 'node:path';
1
2
  import { createRuntimeAppContext } from '../appContext.js';
2
3
  import { analyzeDiffImpact } from '../domain/analyzeDiffImpact.js';
3
4
  import { analyzeTestGaps } from '../domain/analyzeTestGaps.js';
@@ -35,7 +36,7 @@ export async function evaluateAcceptanceMode(mode, args, context) {
35
36
  };
36
37
  }
37
38
  if (mode === 'impact') {
38
- const changedFile = args[0] ?? 'C:/gitrep/caseplatform-web/vcm/jqVCM/widget/caption/CaptionSingleton.ts';
39
+ const changedFile = args[0] ?? path.join(context.config.repos.caseplatformWeb, 'vcm/jqVCM/widget/caption/CaptionSingleton.ts');
39
40
  const impact = await analyzeDiffImpact({ changedFiles: [changedFile] }, context);
40
41
  return {
41
42
  mode,
@@ -9,7 +9,7 @@ Connect this package to `autotests2`, `caseplatform-web`, and CI artifacts witho
9
9
  - Local clone of `autotests2`
10
10
  - Local clone of `caseplatform-web`
11
11
  - Local Cypress artifacts or a synced CI artifact directory
12
- - Node.js 20+
12
+ - Node.js 22.5+
13
13
 
14
14
  ## Setup
15
15
 
@@ -31,29 +31,28 @@ Connect this package to `autotests2`, `caseplatform-web`, and CI artifacts witho
31
31
  ## Recommended operator flow
32
32
 
33
33
  1. Run `npx autotests-overkill index` after fresh pulls or artifact sync.
34
- 2. Start the MCP server through a consumer-local MCP manifest or run `npx autotests-overkill mcp` directly.
34
+ 2. Start the MCP server through a consumer-local Codex config or run `npx autotests-overkill mcp` directly for manual checks.
35
35
  3. Use the bundled skills or direct MCP tools to gather context.
36
36
  4. Review the generated blueprint, gap analysis, or failure triage.
37
37
  5. Apply test edits manually in `autotests2` only after review.
38
38
 
39
- ## Consumer-local MCP example
40
-
41
- Create `.mcp.json` in the consumer repository:
42
-
43
- ```json
44
- {
45
- "mcpServers": {
46
- "autotests-overkill": {
47
- "command": "node",
48
- "args": [
49
- "./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js"
50
- ],
51
- "cwd": "."
52
- }
53
- }
54
- }
39
+ ## Consumer-local Codex MCP example
40
+
41
+ Create `.codex/config.toml` in the consumer repository:
42
+
43
+ ```toml
44
+ [mcp_servers.autotests-overkill]
45
+ command = "node"
46
+ args = ["./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js"]
47
+ enabled = true
48
+ startup_timeout_sec = 20
49
+ tool_timeout_sec = 120
55
50
  ```
56
51
 
52
+ Do not set `required = true` until the server has been verified locally. A required MCP server blocks Codex session startup if the package, Node.js runtime, or `overkill.config.json` is not ready.
53
+
54
+ The command is intentionally relative to the consumer repository. It works in Windows or WSL as long as dependencies were installed in the same environment where Codex starts the server.
55
+
57
56
  Because the package is published publicly to npm, consumers do not need private registry setup to install it.
58
57
 
59
58
  ## CI integration notes
@@ -37,7 +37,7 @@
37
37
 
38
38
  1. Start with `skills/overkill-onboard/SKILL.md`.
39
39
  2. Confirm that `overkill.config.json` points to real repo paths.
40
- 3. Let the client start MCP through `.mcp.json` and use the acceptance commands to prove the installation against local data.
40
+ 3. Let the client start MCP through the consumer repository's `.codex/config.toml` and use the acceptance commands to prove the installation against local data.
41
41
 
42
42
  ## Useful verification commands
43
43
 
@@ -1,9 +1,8 @@
1
- # Codex Plugin Packaging
1
+ # Codex MCP Packaging
2
2
 
3
- ## Included plugin files
3
+ ## Included MCP files
4
4
 
5
- - `.codex-plugin/plugin.json` defines the plugin metadata
6
- - `.mcp.json` registers the local MCP server command
5
+ - `.mcp.json` registers the local checkout MCP server command
7
6
  - `skills/` contains the bundled skill set used by Codex
8
7
 
9
8
  ## Local installation shape
@@ -13,10 +12,18 @@ The repository is self-contained for local checkout-based Codex usage:
13
12
  1. Keep the repository checked out locally.
14
13
  2. Ensure `npm install` has been run once.
15
14
  3. Create `overkill.config.json`.
16
- 4. Let Codex discover `.codex-plugin/plugin.json`.
17
- 5. Let Codex start the MCP server through `.mcp.json`.
15
+ 4. Let Codex start the MCP server through `.mcp.json` when using the checkout directly.
18
16
 
19
- For npm-installed usage inside a consumer repository, prefer a consumer-local `.mcp.json` that points to `./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js`.
17
+ For npm-installed usage inside a consumer repository, prefer a consumer-local `.codex/config.toml`:
18
+
19
+ ```toml
20
+ [mcp_servers.autotests-overkill]
21
+ command = "node"
22
+ args = ["./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js"]
23
+ enabled = true
24
+ startup_timeout_sec = 20
25
+ tool_timeout_sec = 120
26
+ ```
20
27
 
21
28
  ## MCP registration
22
29
 
@@ -24,7 +31,7 @@ The local MCP definition starts the server through:
24
31
 
25
32
  - command: `node`
26
33
  - args: `./node_modules/tsx/dist/cli.mjs`, `src/cli/runMcpServer.ts`
27
- - cwd: `C:\gitrep\autotests-overkill`
34
+ - cwd: `.`
28
35
 
29
36
  The server reads `overkill.config.json` from the repository root and keeps all writes inside `cacheDir`.
30
37
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mknrt/autotests-overkill",
3
- "version": "1.2.2",
3
+ "version": "1.2.3",
4
4
  "private": false,
5
5
  "type": "module",
6
6
  "description": "AI-assisted testing intelligence platform for autotests2, caseplatform-web, and CI artifacts",
@@ -9,8 +9,9 @@ Start Here:
9
9
  4. Run `npm run build`.
10
10
  5. Run `npm run index`.
11
11
  6. Run `npm run mcp:smoke`.
12
- 7. Use the checked-in `.mcp.json` manifest for client integration instead of `npm run mcp`, because stdio MCP servers must keep `stdout` clean.
13
- 7. Run the acceptance commands against local repos.
12
+ 7. For npm package consumers, configure the consumer repository's `.codex/config.toml` to run `node ./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js`.
13
+ 8. Use the checked-in `.mcp.json` only for local checkout development.
14
+ 9. Run the acceptance commands against local repos.
14
15
 
15
16
  Key Idea:
16
17
  - The platform analyzes and drafts; it does not silently edit consumer repos.
@@ -1,251 +0,0 @@
1
- # Audit Iteration 1
2
-
3
- Date: 2026-04-20
4
-
5
- ## Goal
6
-
7
- Evaluate the current state of `autotests-overkill` as a strong assistant for:
8
-
9
- - test design
10
- - test drafting
11
- - test debugging
12
- - coverage and impact analysis
13
-
14
- This first iteration focuses on architectural quality, product readiness, verification coverage, portability, and obvious drift risks.
15
-
16
- ## Scope
17
-
18
- Reviewed:
19
-
20
- - product docs and positioning
21
- - MCP and CLI entrypoints
22
- - domain and indexing architecture
23
- - bundled skills
24
- - package and plugin manifests
25
- - current verification commands
26
- - representative tests
27
-
28
- Key files reviewed:
29
-
30
- - `README.md`
31
- - `package.json`
32
- - `.codex-plugin/plugin.json`
33
- - `.mcp.json`
34
- - `src/mcp/registerTools.ts`
35
- - `src/mcp/server.ts`
36
- - `src/indexer/refreshPipeline.ts`
37
- - `src/contracts/toolOutput.ts`
38
- - `src/contracts/confidence.ts`
39
- - `src/cli/runAcceptance.ts`
40
- - `docs/architecture/overview.md`
41
-
42
- ## Verified State
43
-
44
- The following command was run during the audit:
45
-
46
- ```bash
47
- npm run verify
48
- ```
49
-
50
- Observed result:
51
-
52
- - TypeScript lint passed
53
- - Vitest suite passed
54
- - Build passed
55
- - `75` tests passed across `40` test files
56
-
57
- This means the repository is currently in a healthy technical state at the baseline verification level.
58
-
59
- ## Strong Areas
60
-
61
- ### 1. Clear architecture
62
-
63
- The project already has a strong boundary model:
64
-
65
- - connectors gather evidence
66
- - indexers normalize and persist it
67
- - domain services compose outputs
68
- - MCP exposes narrow, composable tools
69
-
70
- This is one of the strongest parts of the repository because it matches the actual product goal: helping an agent reason about tests without silently editing consumer repositories.
71
-
72
- Relevant files:
73
-
74
- - `docs/architecture/overview.md`
75
- - `src/indexer/refreshPipeline.ts`
76
- - `src/mcp/registerTools.ts`
77
-
78
- ### 2. Strong normalized tool contract
79
-
80
- The unified output shape is a major strength:
81
-
82
- - `summary`
83
- - `evidence`
84
- - `recommended_actions`
85
- - `repo_paths`
86
- - `confidence`
87
- - `warnings`
88
-
89
- This makes the project much more reliable as an LLM-facing tool platform because every MCP tool returns a predictable structure.
90
-
91
- Relevant files:
92
-
93
- - `src/contracts/toolOutput.ts`
94
- - `src/contracts/confidence.ts`
95
- - `src/domain/shared.ts`
96
-
97
- ### 3. Good product stance
98
-
99
- The platform consistently keeps the right operating boundary:
100
-
101
- - analyze
102
- - explain
103
- - draft
104
- - suggest
105
- - do not silently mutate consumer repositories
106
-
107
- That principle is visible in README, skills, and domain behavior. This is important because it makes the tool safer and more trustworthy in real engineering workflows.
108
-
109
- Relevant files:
110
-
111
- - `README.md`
112
- - `skills/overkill-generate-test/SKILL.md`
113
- - `skills/overkill-onboard/SKILL.md`
114
-
115
- ### 4. Better-than-average verification mindset
116
-
117
- The project does not stop at unit-level checks. It already includes:
118
-
119
- - docs checks
120
- - packaging checks
121
- - MCP handshake coverage
122
- - acceptance-style checks for coverage, blueprint, failure triage, and impact analysis
123
-
124
- That is a strong base for a tool whose value is mostly in workflow quality and output usefulness.
125
-
126
- Relevant files:
127
-
128
- - `tests/mcp/stdioHandshake.test.ts`
129
- - `tests/docs/pluginManifest.test.ts`
130
- - `tests/docs/packageManifest.test.ts`
131
- - `tests/cli/runAcceptance.test.ts`
132
-
133
- ## Weak Areas
134
-
135
- ### 1. Release metadata drift
136
-
137
- There is already visible drift between package, plugin, and MCP metadata.
138
-
139
- Observed examples:
140
-
141
- - `package.json` version: `1.1.5`
142
- - `.codex-plugin/plugin.json` version: `1.1.0`
143
- - `src/mcp/server.ts` version: `1.1.0`
144
-
145
- There is also identity drift in plugin metadata:
146
-
147
- - plugin manifest still points to `openai.com`
148
- - plugin manifest still points to `https://github.com/openai/plugins`
149
- - plugin manifest still uses `OpenAI Codex` as project author/developer identity
150
-
151
- Why this matters:
152
-
153
- - packaging looks less trustworthy
154
- - plugin identity is partially incorrect
155
- - release information can become misleading for users and tooling
156
-
157
- Relevant files:
158
-
159
- - `package.json`
160
- - `.codex-plugin/plugin.json`
161
- - `src/mcp/server.ts`
162
-
163
- ### 2. Poor portability
164
-
165
- The project currently contains machine-specific assumptions that reduce reliability outside the current workstation.
166
-
167
- Observed examples:
168
-
169
- - `.mcp.json` contains hardcoded `cwd: C:\\gitrep\\autotests-overkill`
170
- - docs repeat that exact local path
171
- - tests assert that exact path
172
- - test helpers assume `C:/gitrep/autotests2` and `C:/gitrep/caseplatform-web`
173
-
174
- Why this matters:
175
-
176
- - local checkout usage is brittle
177
- - plugin packaging is less reusable
178
- - tests are tightly coupled to one environment shape
179
- - onboarding quality drops sharply outside the author machine
180
-
181
- Relevant files:
182
-
183
- - `.mcp.json`
184
- - `docs/plugin-packaging.md`
185
- - `tests/docs/pluginManifest.test.ts`
186
- - `tests/helpers/testContext.ts`
187
-
188
- ### 3. Verify command does not fully protect the product promise
189
-
190
- The docs position `mcp:smoke` and acceptance commands as important proof of installation and behavior, but `npm run verify` currently includes only:
191
-
192
- - lint
193
- - test
194
- - build
195
-
196
- This is not broken, but it does leave a gap between:
197
-
198
- - what the product says matters
199
- - what the main verification gate actually enforces
200
-
201
- Why this matters:
202
-
203
- - release confidence depends on social discipline instead of automation
204
- - MCP/package workflow regressions may survive longer than necessary
205
- - product-level trust is slightly weaker than the current architecture deserves
206
-
207
- Relevant files:
208
-
209
- - `README.md`
210
- - `docs/operator-cookbook.md`
211
- - `package.json`
212
-
213
- ## Assessment
214
-
215
- This repository is already a strong first-generation foundation for an AI-assisted testing platform. Its strongest qualities are:
216
-
217
- - disciplined architecture
218
- - evidence-first tool outputs
219
- - strong problem framing
220
- - good initial verification culture
221
-
222
- Its most important current risks are not algorithmic quality failures or test instability. The higher-priority risks are:
223
-
224
- - release/metadata drift
225
- - portability problems
226
- - incomplete automation around product-level verification
227
-
228
- In other words: the core is promising, but the outer delivery shell is weaker than the inner design.
229
-
230
- ## Recommended Next Iteration
231
-
232
- Recommended focus for the next improvement cycle:
233
-
234
- `operability / release integrity`
235
-
236
- That iteration should prioritize:
237
-
238
- 1. Synchronizing package, plugin, and MCP versions and identity metadata
239
- 2. Reducing machine-specific assumptions in MCP packaging and test setup
240
- 3. Adding anti-drift checks so metadata divergence is caught automatically
241
- 4. Strengthening the verification gate around packaging and MCP operability
242
-
243
- ## Short Conclusion
244
-
245
- Iteration 1 verdict:
246
-
247
- - the project is technically healthy
248
- - the architecture is strong
249
- - the product direction is correct
250
- - the most valuable next work is not feature expansion
251
- - the most valuable next work is making the platform more portable, more release-safe, and harder to drift out of sync