ai-stream-auditor 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +25 -1
- package/CODE_OF_CONDUCT.md +17 -0
- package/CONTRIBUTING.md +10 -0
- package/README.md +43 -2
- package/SECURITY.md +13 -0
- package/examples/audit-anthropic.mjs +20 -0
- package/examples/audit-openai.mjs +14 -0
- package/package.json +49 -10
package/CHANGELOG.md
CHANGED
|
@@ -4,7 +4,31 @@ All notable changes follow [Keep a Changelog](https://keepachangelog.com/en/1.1.
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
-
## [1.0.
|
|
7
|
+
## [1.0.3] - 2026-10-21
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- Exercise credential-free truncated and complete `MessageStream` and
|
|
12
|
+
`BetaMessageStream` lifecycles from the official Anthropic TypeScript SDK,
|
|
13
|
+
including partial text and tool-input EOF, on the SDK's supported Node
|
|
14
|
+
runtimes while preserving a dependency-free Node 18 core check.
|
|
15
|
+
- Verify both official SDK stream helpers preserve a transport failure as the
|
|
16
|
+
cause of the `AnthropicError` from `done()` when it occurs between iterator
|
|
17
|
+
reads and that the auditor records the failure as `transport_error` rather
|
|
18
|
+
than a successful completion.
|
|
19
|
+
- Ship and execute a runnable Anthropic Messages lifecycle example.
|
|
20
|
+
- Include the security, contribution, and conduct policies in the package
|
|
21
|
+
tarball.
|
|
22
|
+
|
|
23
|
+
## [1.0.2] - 2026-10-10
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- Ship the runnable OpenAI Responses lifecycle example in the npm tarball.
|
|
28
|
+
- Execute the example as part of the aggregate package check.
|
|
29
|
+
- Document how lifecycle auditing composes with async-iterator deadlines.
|
|
30
|
+
|
|
31
|
+
## [1.0.1] - 2026-10-06
|
|
8
32
|
|
|
9
33
|
### Fixed
|
|
10
34
|
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Contributor Covenant Code of Conduct
|
|
2
|
+
|
|
3
|
+
## Our pledge
|
|
4
|
+
|
|
5
|
+
We pledge to make participation in this project a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, experience level, education, socioeconomic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
|
|
6
|
+
|
|
7
|
+
## Our standards
|
|
8
|
+
|
|
9
|
+
Positive behavior includes empathy, respectful feedback, accepting responsibility, and focusing on what benefits the community. Unacceptable behavior includes harassment, sexualized conduct, insults, threats, trolling, or publishing another person's private information.
|
|
10
|
+
|
|
11
|
+
## Enforcement
|
|
12
|
+
|
|
13
|
+
Report unacceptable behavior privately through the security contact or repository owner's public contact details. Maintainers may remove, edit, or reject contributions and may temporarily or permanently ban contributors for behavior they deem inappropriate.
|
|
14
|
+
|
|
15
|
+
## Attribution
|
|
16
|
+
|
|
17
|
+
Adapted from the Contributor Covenant, version 2.1: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
1. Open an issue for new dialects or public API changes.
|
|
4
|
+
2. Create a focused branch from the current default branch.
|
|
5
|
+
3. Run `npm ci` and `npm run check` on Node.js 18 or newer.
|
|
6
|
+
4. Base protocol behavior on current primary provider documentation and link it in the change.
|
|
7
|
+
5. Add valid fixtures plus truncation, reordering, duplication, and error mutations.
|
|
8
|
+
6. Update the README and changelog for behavioral changes.
|
|
9
|
+
|
|
10
|
+
Never submit real prompts, generated private content, API keys, request headers, or customer event logs. Contributions must be original or appropriately licensed and are distributed under the MIT license.
|
package/README.md
CHANGED
|
@@ -15,6 +15,7 @@ The package's checks are based on the providers' documented event flows and moti
|
|
|
15
15
|
- [OpenAI streaming Responses guide](https://developers.openai.com/api/docs/guides/streaming-responses)
|
|
16
16
|
- [OpenAI Node issue #2046: mid-stream errors silently swallowed during iteration](https://github.com/openai/openai-node/issues/2046)
|
|
17
17
|
- [Anthropic streaming Messages documentation](https://platform.claude.com/docs/en/build-with-claude/streaming)
|
|
18
|
+
- [Anthropic TypeScript SDK issue #1264: `done()` resolves after EOF without `message_stop`](https://github.com/anthropics/anthropic-sdk-typescript/issues/1264)
|
|
18
19
|
- [AxonHub issue #2092: synthetic end marker hides incomplete Responses stream](https://github.com/looplj/axonhub/issues/2092)
|
|
19
20
|
|
|
20
21
|
This project is independent and is not an official OpenAI or Anthropic package.
|
|
@@ -55,6 +56,23 @@ for await (const event of stream) auditor.push(event);
|
|
|
55
56
|
const report = auditor.finish();
|
|
56
57
|
```
|
|
57
58
|
|
|
59
|
+
The integration suite feeds credential-free synthetic streams through the
|
|
60
|
+
official Anthropic TypeScript SDK and then through this auditor. It covers both
|
|
61
|
+
`MessageStream` and `BetaMessageStream`, with EOF after `message_start`, during
|
|
62
|
+
partial text, and during partial tool input. Every truncated case is reported
|
|
63
|
+
as `transport_closed` even when the SDK's `done()` promise resolves, while a
|
|
64
|
+
stream containing `message_stop` remains successful. This guards the exact
|
|
65
|
+
EOF boundary and EOF cases from the suggested regression matrix described in
|
|
66
|
+
[Anthropic SDK issue #1264](https://github.com/anthropics/anthropic-sdk-typescript/issues/1264).
|
|
67
|
+
|
|
68
|
+
The same integration boundary covers a transport failure between iterator
|
|
69
|
+
reads, where current SDK helpers can report iterator completion while
|
|
70
|
+
`done()` rejects with an `AnthropicError` whose cause is the original error.
|
|
71
|
+
For both helpers the suite verifies the cause is preserved and the auditor
|
|
72
|
+
reports `outcome: 'error'` with
|
|
73
|
+
`finishCause: 'transport_error'`, matching
|
|
74
|
+
[Anthropic SDK issue #1247](https://github.com/anthropics/anthropic-sdk-typescript/issues/1247).
|
|
75
|
+
|
|
58
76
|
## The three questions in every report
|
|
59
77
|
|
|
60
78
|
```ts
|
|
@@ -226,12 +244,35 @@ The auditor does not retain raw event payloads or generated text by default beca
|
|
|
226
244
|
|
|
227
245
|
See [SECURITY.md](SECURITY.md) for vulnerability reporting.
|
|
228
246
|
|
|
247
|
+
## Runnable example and related tool
|
|
248
|
+
|
|
249
|
+
[`examples/audit-openai.mjs`](examples/audit-openai.mjs) and
|
|
250
|
+
[`examples/audit-anthropic.mjs`](examples/audit-anthropic.mjs) feed minimal
|
|
251
|
+
valid provider lifecycles through the auditor. Both are executed by
|
|
252
|
+
`npm run check` and ship in the npm tarball.
|
|
253
|
+
|
|
254
|
+
Use [`async-iterator-watchdog`](https://www.npmjs.com/package/async-iterator-watchdog)
|
|
255
|
+
when you also need active first-progress, idle, or total deadlines around the
|
|
256
|
+
source iterator. The watchdog enforces time; this package validates protocol
|
|
257
|
+
lifecycle and outcome.
|
|
258
|
+
|
|
229
259
|
## Development
|
|
230
260
|
|
|
261
|
+
Node.js 20 or newer runs the complete suite, including the dev-only official
|
|
262
|
+
Anthropic SDK integration:
|
|
263
|
+
|
|
231
264
|
```sh
|
|
232
|
-
npm ci
|
|
265
|
+
npm ci --ignore-scripts
|
|
233
266
|
npm run check
|
|
234
|
-
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
Node.js 18 remains supported by the package runtime. Because the current
|
|
270
|
+
Anthropic SDK dev dependency requires Node.js 20 or newer, use the
|
|
271
|
+
dependency-free core and package-surface check on Node.js 18:
|
|
272
|
+
|
|
273
|
+
```sh
|
|
274
|
+
npm ci --omit=dev --ignore-scripts
|
|
275
|
+
npm run check:core
|
|
235
276
|
```
|
|
236
277
|
|
|
237
278
|
## License
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
## Supported versions
|
|
4
|
+
|
|
5
|
+
Security fixes are provided for the latest major version.
|
|
6
|
+
|
|
7
|
+
## Reporting
|
|
8
|
+
|
|
9
|
+
Do not open a public issue for an undisclosed vulnerability. Use GitHub private vulnerability reporting. If unavailable, contact the maintainer through the address on the public GitHub profile with the affected version, reproduction, impact, and mitigation. Expect acknowledgement within seven days.
|
|
10
|
+
|
|
11
|
+
## Scope
|
|
12
|
+
|
|
13
|
+
This package processes caller-supplied JavaScript objects without executing provider content. It does not parse transports or enforce output safety. Reports intentionally omit raw payloads, but lifecycle keys and error/status metadata can still be sensitive; callers are responsible for log access and retention controls.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
import { createStreamAuditor } from 'ai-stream-auditor';
|
|
2
|
+
|
|
3
|
+
const events = [
|
|
4
|
+
{ type: 'message_start', message: { stop_reason: null } },
|
|
5
|
+
{ type: 'content_block_start', index: 0, content_block: { type: 'text', text: '' } },
|
|
6
|
+
{ type: 'content_block_delta', index: 0, delta: { type: 'text_delta', text: 'hello' } },
|
|
7
|
+
{ type: 'content_block_stop', index: 0 },
|
|
8
|
+
{ type: 'message_delta', delta: { stop_reason: 'end_turn' } },
|
|
9
|
+
{ type: 'message_stop' },
|
|
10
|
+
];
|
|
11
|
+
|
|
12
|
+
const auditor = createStreamAuditor('anthropic-messages');
|
|
13
|
+
for (const event of events) auditor.push(event);
|
|
14
|
+
|
|
15
|
+
const report = auditor.finish();
|
|
16
|
+
console.log(JSON.stringify({
|
|
17
|
+
protocolComplete: report.protocolComplete,
|
|
18
|
+
successful: report.successful,
|
|
19
|
+
outcome: report.outcome,
|
|
20
|
+
}));
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
import { createStreamAuditor } from 'ai-stream-auditor';
|
|
2
|
+
|
|
3
|
+
const auditor = createStreamAuditor('openai-responses');
|
|
4
|
+
|
|
5
|
+
// Feed decoded SDK events as they arrive. The auditor does not parse SSE bytes.
|
|
6
|
+
for (const event of [
|
|
7
|
+
{ type: 'response.created', sequence_number: 0 },
|
|
8
|
+
{ type: 'response.completed', sequence_number: 1, response: { status: 'completed' } },
|
|
9
|
+
]) {
|
|
10
|
+
auditor.push(event);
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
const report = auditor.finish({ cause: 'iterator_end' });
|
|
14
|
+
console.log(report);
|
package/package.json
CHANGED
|
@@ -1,26 +1,65 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ai-stream-auditor",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.3",
|
|
4
4
|
"description": "Audit decoded OpenAI Responses and Anthropic Messages event lifecycles for silent or malformed stream endings.",
|
|
5
|
-
"keywords": [
|
|
5
|
+
"keywords": [
|
|
6
|
+
"ai",
|
|
7
|
+
"streaming",
|
|
8
|
+
"openai",
|
|
9
|
+
"anthropic",
|
|
10
|
+
"llm",
|
|
11
|
+
"diagnostics",
|
|
12
|
+
"testing"
|
|
13
|
+
],
|
|
6
14
|
"license": "MIT",
|
|
7
15
|
"author": "Huzaifa Asif",
|
|
8
|
-
"repository": {
|
|
9
|
-
|
|
16
|
+
"repository": {
|
|
17
|
+
"type": "git",
|
|
18
|
+
"url": "git+https://github.com/Huzaifa-Asif/ai-stream-auditor.git"
|
|
19
|
+
},
|
|
20
|
+
"bugs": {
|
|
21
|
+
"url": "https://github.com/Huzaifa-Asif/ai-stream-auditor/issues"
|
|
22
|
+
},
|
|
10
23
|
"homepage": "https://github.com/Huzaifa-Asif/ai-stream-auditor#readme",
|
|
11
24
|
"type": "module",
|
|
12
25
|
"main": "./src/index.cjs",
|
|
13
26
|
"module": "./src/index.js",
|
|
14
27
|
"types": "./src/index.d.ts",
|
|
15
|
-
"exports": {
|
|
16
|
-
|
|
17
|
-
|
|
28
|
+
"exports": {
|
|
29
|
+
".": {
|
|
30
|
+
"types": "./src/index.d.ts",
|
|
31
|
+
"import": "./src/index.js",
|
|
32
|
+
"require": "./src/index.cjs"
|
|
33
|
+
}
|
|
34
|
+
},
|
|
35
|
+
"files": [
|
|
36
|
+
"src",
|
|
37
|
+
"examples",
|
|
38
|
+
"README.md",
|
|
39
|
+
"LICENSE",
|
|
40
|
+
"CHANGELOG.md",
|
|
41
|
+
"CODE_OF_CONDUCT.md",
|
|
42
|
+
"CONTRIBUTING.md",
|
|
43
|
+
"SECURITY.md"
|
|
44
|
+
],
|
|
45
|
+
"engines": {
|
|
46
|
+
"node": ">=18"
|
|
47
|
+
},
|
|
18
48
|
"sideEffects": false,
|
|
19
49
|
"scripts": {
|
|
20
50
|
"lint": "node scripts/lint.mjs",
|
|
21
|
-
"test": "
|
|
51
|
+
"test": "npm run test:core && npm run test:integration",
|
|
52
|
+
"test:core": "node --test test/audit.test.js test/regressions.test.js",
|
|
53
|
+
"test:integration": "node --test test/anthropic-sdk.integration.test.js",
|
|
22
54
|
"test:coverage": "node --experimental-test-coverage --test",
|
|
23
|
-
"
|
|
55
|
+
"test:coverage:core": "node --experimental-test-coverage --test test/audit.test.js test/regressions.test.js",
|
|
56
|
+
"check:core": "npm run lint && npm run test:coverage:core && node examples/audit-openai.mjs && node examples/audit-anthropic.mjs && npm pack --dry-run",
|
|
57
|
+
"check": "npm run lint && npm run test:coverage && node examples/audit-openai.mjs && node examples/audit-anthropic.mjs && npm pack --dry-run"
|
|
58
|
+
},
|
|
59
|
+
"publishConfig": {
|
|
60
|
+
"access": "public"
|
|
24
61
|
},
|
|
25
|
-
"
|
|
62
|
+
"devDependencies": {
|
|
63
|
+
"@anthropic-ai/sdk": "0.135.0"
|
|
64
|
+
}
|
|
26
65
|
}
|