@copilotkit/aimock 1.15.1 → 1.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +8 -0
- package/README.md +1 -1
- package/dist/config-loader.d.cts.map +1 -1
- package/dist/config-loader.d.ts.map +1 -1
- package/dist/fixture-loader.cjs +19 -4
- package/dist/fixture-loader.cjs.map +1 -1
- package/dist/fixture-loader.d.cts.map +1 -1
- package/dist/fixture-loader.d.ts.map +1 -1
- package/dist/fixture-loader.js +19 -4
- package/dist/fixture-loader.js.map +1 -1
- package/dist/journal.cjs +1 -1
- package/dist/journal.cjs.map +1 -1
- package/dist/journal.d.cts.map +1 -1
- package/dist/journal.d.ts.map +1 -1
- package/dist/journal.js +1 -1
- package/dist/journal.js.map +1 -1
- package/dist/llmock.cjs +6 -0
- package/dist/llmock.cjs.map +1 -1
- package/dist/llmock.d.cts +1 -0
- package/dist/llmock.d.cts.map +1 -1
- package/dist/llmock.d.ts +1 -0
- package/dist/llmock.d.ts.map +1 -1
- package/dist/llmock.js +6 -0
- package/dist/llmock.js.map +1 -1
- package/dist/router.cjs +6 -0
- package/dist/router.cjs.map +1 -1
- package/dist/router.js +6 -0
- package/dist/router.js.map +1 -1
- package/dist/types.d.cts +4 -0
- package/dist/types.d.cts.map +1 -1
- package/dist/types.d.ts +4 -0
- package/dist/types.d.ts.map +1 -1
- package/package.json +1 -1
- package/skills/write-fixtures/SKILL.md +75 -49
|
@@ -19,25 +19,50 @@ aimock is a zero-dependency mock infrastructure for AI apps. Fixture-driven. Mul
|
|
|
19
19
|
|
|
20
20
|
## Match Field Reference
|
|
21
21
|
|
|
22
|
-
| Field | Type | Matches Against
|
|
23
|
-
| ---------------- | ----------------------------------------- |
|
|
24
|
-
| `userMessage` | `string` | Substring of last `role: "user"` message text
|
|
25
|
-
| `userMessage` | `RegExp` | Pattern test on last `role: "user"` message text
|
|
26
|
-
| `inputText` | `string` | Substring of embedding input text (concatenated if multiple inputs)
|
|
27
|
-
| `inputText` | `RegExp` | Pattern test on embedding input text
|
|
28
|
-
| `toolName` | `string` | Exact match on any tool in request's `tools[]` array (by `function.name`)
|
|
29
|
-
| `toolCallId` | `string` | Exact match on `tool_call_id` of last `role: "tool"` message
|
|
30
|
-
| `model` | `string` | Exact match on `req.model`
|
|
31
|
-
| `model` | `RegExp` | Pattern test on `req.model`
|
|
32
|
-
| `responseFormat` | `string` | Exact match on `req.response_format.type` (`"json_object"`, `"json_schema"`)
|
|
33
|
-
| `sequenceIndex` | `number` | Matches only when this fixture's match count equals the given index (0-based)
|
|
34
|
-
| `
|
|
35
|
-
| `
|
|
22
|
+
| Field | Type | Matches Against |
|
|
23
|
+
| ---------------- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
24
|
+
| `userMessage` | `string` | Substring of last `role: "user"` message text |
|
|
25
|
+
| `userMessage` | `RegExp` | Pattern test on last `role: "user"` message text |
|
|
26
|
+
| `inputText` | `string` | Substring of embedding input text (concatenated if multiple inputs) |
|
|
27
|
+
| `inputText` | `RegExp` | Pattern test on embedding input text |
|
|
28
|
+
| `toolName` | `string` | Exact match on any tool in request's `tools[]` array (by `function.name`) |
|
|
29
|
+
| `toolCallId` | `string` | Exact match on `tool_call_id` of last `role: "tool"` message |
|
|
30
|
+
| `model` | `string` | Exact match on `req.model` |
|
|
31
|
+
| `model` | `RegExp` | Pattern test on `req.model` |
|
|
32
|
+
| `responseFormat` | `string` | Exact match on `req.response_format.type` (`"json_object"`, `"json_schema"`) |
|
|
33
|
+
| `sequenceIndex` | `number` | Matches only when this fixture's match count equals the given index (0-based) |
|
|
34
|
+
| `turnIndex` | `number` | Stateless conversation-depth matching. Counts `role: "assistant"` messages in the request; matches when that count equals the value. `turnIndex: 0` = first turn (no prior assistant messages). Use instead of `sequenceIndex` for shared/deployed instances where stateful counters break under concurrency |
|
|
35
|
+
| `hasToolResult` | `boolean` | Stateless tool-message presence matching. `true` matches when any `role: "tool"` message exists in the request; `false` matches when none exist. Provider-consistent across all aimock handlers (OpenAI, Claude, Gemini, Bedrock, Ollama, Cohere) |
|
|
36
|
+
| `endpoint` | `string` | Restrict to endpoint type: `"chat"`, `"image"`, `"speech"`, `"transcription"`, `"video"`, `"embedding"` |
|
|
37
|
+
| `predicate` | `(req: ChatCompletionRequest) => boolean` | Custom function — full access to request |
|
|
36
38
|
|
|
37
39
|
**AND logic**: all specified fields must match. Empty match `{}` = catch-all.
|
|
38
40
|
|
|
39
41
|
Multi-part content (e.g., `[{type: "text", text: "hello"}]`) is automatically extracted — `userMessage` matching works regardless of content format.
|
|
40
42
|
|
|
43
|
+
### When to Use Each Multi-turn Matching Approach
|
|
44
|
+
|
|
45
|
+
| Approach | Stateless? | Best For |
|
|
46
|
+
| --------------- | ---------- | --------------------------------------------------------------------------------------------------------- |
|
|
47
|
+
| `turnIndex` | Yes | Shared/deployed instances; matches on conversation depth (count of assistant messages in request) |
|
|
48
|
+
| `hasToolResult` | Yes | Simplest option for 2-step tool flows — boolean: are there tool results in the request? |
|
|
49
|
+
| `sequenceIndex` | No | Single-client unit tests with repeated identical requests (server-side counter, breaks under concurrency) |
|
|
50
|
+
| `toolCallId` | Yes | Matching specific tool result IDs in the conversation history |
|
|
51
|
+
|
|
52
|
+
**Prefer stateless approaches** (`turnIndex`, `hasToolResult`) for shared aimock instances (deployed via Docker, used by multiple test runners). Use `sequenceIndex` only in isolated single-client unit tests where the counter won't be corrupted by concurrent requests.
|
|
53
|
+
|
|
54
|
+
### Multi-turn fixture examples
|
|
55
|
+
|
|
56
|
+
```jsonc
|
|
57
|
+
// 2-step HITL with turnIndex
|
|
58
|
+
{"match": {"userMessage": "trip to mars", "turnIndex": 0}, "response": {"toolCalls": [{"id": "call_001", "name": "generate_steps", "arguments": "{}"}]}}
|
|
59
|
+
{"match": {"userMessage": "trip to mars", "turnIndex": 1}, "response": {"content": "Great choices! Proceeding."}}
|
|
60
|
+
|
|
61
|
+
// Same thing with hasToolResult (simpler for 2-step)
|
|
62
|
+
{"match": {"userMessage": "trip to mars", "hasToolResult": false}, "response": {"toolCalls": [{"id": "call_001", "name": "generate_steps", "arguments": "{}"}]}}
|
|
63
|
+
{"match": {"userMessage": "trip to mars", "hasToolResult": true}, "response": {"content": "Great choices!"}}
|
|
64
|
+
```
|
|
65
|
+
|
|
41
66
|
## Response Types
|
|
42
67
|
|
|
43
68
|
### Text
|
|
@@ -653,41 +678,42 @@ const mock = await LLMock.create({ port: 0 }); // creates + starts in one call
|
|
|
653
678
|
|
|
654
679
|
## API Quick Reference
|
|
655
680
|
|
|
656
|
-
| Method
|
|
657
|
-
|
|
|
658
|
-
| `addFixture(f)`
|
|
659
|
-
| `addFixtures(f[])`
|
|
660
|
-
| `prependFixture(f)`
|
|
661
|
-
| `clearFixtures()`
|
|
662
|
-
| `getFixtures()`
|
|
663
|
-
| `on(match, response, opts?)`
|
|
664
|
-
| `onMessage(pattern, response, opts?)`
|
|
665
|
-
| `onEmbedding(pattern, response, opts?)`
|
|
666
|
-
| `onJsonOutput(pattern, json, opts?)`
|
|
667
|
-
| `onToolCall(name, response, opts?)`
|
|
668
|
-
| `onToolResult(id, response, opts?)`
|
|
669
|
-
| `
|
|
670
|
-
| `
|
|
671
|
-
| `
|
|
672
|
-
| `
|
|
673
|
-
| `
|
|
674
|
-
| `
|
|
675
|
-
| `
|
|
676
|
-
| `
|
|
677
|
-
| `
|
|
678
|
-
| `
|
|
679
|
-
| `
|
|
680
|
-
| `
|
|
681
|
-
| `
|
|
682
|
-
| `
|
|
683
|
-
| `
|
|
684
|
-
| `
|
|
685
|
-
| `
|
|
686
|
-
| `
|
|
687
|
-
| `
|
|
688
|
-
| `
|
|
689
|
-
| `
|
|
690
|
-
| `
|
|
681
|
+
| Method | Purpose |
|
|
682
|
+
| ---------------------------------------- | ------------------------------------------- |
|
|
683
|
+
| `addFixture(f)` | Append fixture (last priority) |
|
|
684
|
+
| `addFixtures(f[])` | Append multiple |
|
|
685
|
+
| `prependFixture(f)` | Insert at front (highest priority) |
|
|
686
|
+
| `clearFixtures()` | Remove all fixtures |
|
|
687
|
+
| `getFixtures()` | Read current fixture list |
|
|
688
|
+
| `on(match, response, opts?)` | Shorthand for `addFixture` |
|
|
689
|
+
| `onMessage(pattern, response, opts?)` | Match by user message |
|
|
690
|
+
| `onEmbedding(pattern, response, opts?)` | Match by embedding input text |
|
|
691
|
+
| `onJsonOutput(pattern, json, opts?)` | Match by user message with `responseFormat` |
|
|
692
|
+
| `onToolCall(name, response, opts?)` | Match by tool name in `tools[]` |
|
|
693
|
+
| `onToolResult(id, response, opts?)` | Match by `tool_call_id` |
|
|
694
|
+
| `onTurn(turn, pattern, response, opts?)` | Match by turn index + user message |
|
|
695
|
+
| `nextRequestError(status, body?)` | One-shot error, auto-removes |
|
|
696
|
+
| `loadFixtureFile(path)` | Load JSON fixture file |
|
|
697
|
+
| `loadFixtureDir(path)` | Load all JSON files in directory |
|
|
698
|
+
| `start()` | Start server, returns URL |
|
|
699
|
+
| `stop()` | Stop server |
|
|
700
|
+
| `reset()` | Clear fixtures + journal + match counts |
|
|
701
|
+
| `resetMatchCounts()` | Clear sequence match counts only |
|
|
702
|
+
| `getRequests()` | All journal entries |
|
|
703
|
+
| `getLastRequest()` | Most recent journal entry |
|
|
704
|
+
| `clearRequests()` | Clear journal only |
|
|
705
|
+
| `setChaos(opts)` | Set server-level chaos rates |
|
|
706
|
+
| `clearChaos()` | Remove server-level chaos |
|
|
707
|
+
| `onSearch(pattern, results)` | Match search requests by query |
|
|
708
|
+
| `onRerank(pattern, results)` | Match rerank requests by query |
|
|
709
|
+
| `onModerate(pattern, result)` | Match moderation requests by input |
|
|
710
|
+
| `onImage(pattern, response)` | Match image generation by prompt |
|
|
711
|
+
| `onSpeech(pattern, response)` | Match TTS by input text |
|
|
712
|
+
| `onTranscription(response)` | Match audio transcription |
|
|
713
|
+
| `onVideo(pattern, response)` | Match video generation by prompt |
|
|
714
|
+
| `mount(path, handler)` | Mount a Mountable (VectorMock, etc.) |
|
|
715
|
+
| `url` / `baseUrl` | Server URL (throws if not started) |
|
|
716
|
+
| `port` | Server port number |
|
|
691
717
|
|
|
692
718
|
Sequential responses use `on()` with `sequenceIndex` in the match — there is no dedicated convenience method.
|
|
693
719
|
|