ai-capability-framework 1.0.0-rc.3 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +97 -103
- package/CODE_OF_CONDUCT.md +11 -13
- package/CONTRIBUTING.md +40 -43
- package/GOVERNANCE.md +34 -35
- package/README.md +99 -63
- package/ROADMAP.md +12 -12
- package/SECURITY.md +15 -15
- package/dist/audit/ledger.d.ts.map +1 -1
- package/dist/audit/ledger.js +2 -1
- package/dist/audit/records.d.ts.map +1 -1
- package/dist/audit/records.js +2 -1
- package/dist/audit/types.d.ts +4 -1
- package/dist/audit/types.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.d.ts +2 -7
- package/dist/aws/dynamodb-stores.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.js +56 -5
- package/dist/aws/testing.d.ts.map +1 -1
- package/dist/aws/testing.js +9 -0
- package/dist/cli.d.ts +6 -6
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +77 -7
- package/dist/evals-live/index.d.ts +2 -2
- package/dist/evals-live/index.d.ts.map +1 -1
- package/dist/evals-live/index.js +1 -1
- package/dist/evals-live/live-eval-runner.d.ts +49 -1
- package/dist/evals-live/live-eval-runner.d.ts.map +1 -1
- package/dist/evals-live/live-eval-runner.js +246 -4
- package/dist/evals-live/types.d.ts +62 -9
- package/dist/evals-live/types.d.ts.map +1 -1
- package/dist/index.d.ts +0 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +0 -1
- package/dist/runtime/actions.d.ts +2 -0
- package/dist/runtime/actions.d.ts.map +1 -1
- package/dist/runtime/actions.js +200 -55
- package/dist/runtime/index.d.ts +1 -1
- package/dist/runtime/index.d.ts.map +1 -1
- package/dist/runtime/stores.d.ts +2 -7
- package/dist/runtime/stores.d.ts.map +1 -1
- package/dist/runtime/stores.js +12 -0
- package/dist/runtime/types.d.ts +25 -6
- package/dist/runtime/types.d.ts.map +1 -1
- package/docs/action-lifecycle.md +38 -32
- package/docs/adapter-roadmap.md +51 -49
- package/docs/adapters.md +14 -15
- package/docs/ai-sdk-runtime.md +36 -39
- package/docs/anthropic-runtime.md +29 -32
- package/docs/api/public-api-policy.md +59 -0
- package/docs/api.md +354 -371
- package/docs/architecture/current-state.md +28 -31
- package/docs/audit/index.md +29 -29
- package/docs/aws/cloudwatch-telemetry.md +5 -7
- package/docs/aws/dynamodb-single-table.md +8 -9
- package/docs/aws/kms-redaction.md +5 -6
- package/docs/aws/overview.md +5 -5
- package/docs/aws/production-reference.md +13 -14
- package/docs/aws/step-functions-approval.md +7 -9
- package/docs/aws-runtime.md +24 -25
- package/docs/capability-manifests.md +17 -21
- package/docs/control-plane/overview.md +2 -2
- package/docs/control-plane.md +24 -27
- package/docs/controls/index.md +19 -22
- package/docs/core/capability-manifests.md +4 -5
- package/docs/core/entity-manifests.md +5 -5
- package/docs/eval-runner.md +29 -31
- package/docs/evalops.md +13 -14
- package/docs/evals/golden-tests.md +4 -5
- package/docs/evals/overview.md +3 -3
- package/docs/evals/replay-and-trace-to-golden.md +19 -20
- package/docs/evals.md +19 -20
- package/docs/evidence.md +22 -23
- package/docs/gemini-runtime.md +31 -33
- package/docs/getting-started/anthropic-quickstart.md +61 -0
- package/docs/getting-started/concepts.md +9 -10
- package/docs/getting-started/gemini-quickstart.md +62 -0
- package/docs/getting-started/installation.md +3 -3
- package/docs/getting-started/openai-quickstart.md +61 -0
- package/docs/getting-started/provider-neutral-quickstart.md +106 -0
- package/docs/getting-started/quickstart.md +2 -2
- package/docs/getting-started.md +23 -20
- package/docs/glossary.md +31 -48
- package/docs/governance/gate.md +28 -30
- package/docs/governance/impact-and-compatibility.md +2 -3
- package/docs/governance/index.md +53 -54
- package/docs/governance/lifecycle.md +4 -5
- package/docs/governance/overview.md +2 -3
- package/docs/governance/risk-compiler.md +4 -5
- package/docs/host-responsibilities.md +6 -8
- package/docs/implementation-notes/baseline-gaps.md +15 -18
- package/docs/index.md +21 -6
- package/docs/interoperability.md +9 -9
- package/docs/langchain-runtime.md +27 -31
- package/docs/live-evals.md +68 -32
- package/docs/mcp-server-runtime.md +36 -40
- package/docs/memory.md +35 -36
- package/docs/migration-0.1-to-1.0.md +14 -9
- package/docs/observability/overview.md +2 -3
- package/docs/observability-runtime.md +35 -39
- package/docs/openai-responses.md +36 -37
- package/docs/openai-runtime.md +41 -46
- package/docs/openai-walkthrough.md +24 -28
- package/docs/policy-broker.md +9 -10
- package/docs/provenance.md +30 -29
- package/docs/providers/anthropic.md +5 -5
- package/docs/providers/choose-a-runtime.md +37 -0
- package/docs/providers/conformance.md +14 -15
- package/docs/providers/gemini.md +5 -5
- package/docs/providers/mcp.md +2 -3
- package/docs/providers/openai.md +2 -2
- package/docs/providers/semantic-kernel.md +2 -2
- package/docs/providers.md +43 -38
- package/docs/public/license-exceptions.md +187 -0
- package/docs/public/npm-release-preflight.md +112 -0
- package/docs/public-framework/compatibility-policy.md +12 -13
- package/docs/public-framework/deprecation-policy.md +8 -8
- package/docs/public-framework/final-certification-matrix.md +104 -0
- package/docs/public-framework/license-decision.md +20 -0
- package/docs/public-framework/release-process.md +46 -16
- package/docs/public-framework/security-disclosure.md +4 -5
- package/docs/public-framework/v1-certification.md +102 -15
- package/docs/release.md +200 -122
- package/docs/runtime/action-lifecycle.md +3 -3
- package/docs/runtime/policy-broker.md +4 -4
- package/docs/runtime/runtime-overview.md +4 -5
- package/docs/runtime/tool-result-envelope.md +7 -7
- package/docs/runtime.md +90 -93
- package/docs/security/overview.md +4 -4
- package/docs/security/security-packs.md +24 -26
- package/docs/security/trust-taint-redaction.md +42 -47
- package/docs/semantic-kernel-runtime.md +35 -37
- package/docs/spec.md +95 -101
- package/docs/start-here.md +18 -20
- package/examples/01-basic-read-capability/README.md +3 -4
- package/examples/02-prepare-approve-commit/README.md +6 -6
- package/examples/03-multi-provider-tools/README.md +2 -2
- package/examples/04-mcp-server/README.md +4 -5
- package/examples/05-langchain-langgraph-bridge/README.md +4 -5
- package/examples/06-vercel-ai-sdk-bridge/README.md +5 -7
- package/examples/07-policy-broker-custom-auth/README.md +4 -4
- package/examples/08-aws-step-functions-approval/README.md +4 -5
- package/examples/09-security-packs-promptfoo/README.md +4 -4
- package/examples/10-trace-to-golden/README.md +2 -2
- package/examples/11-control-plane/README.md +4 -5
- package/examples/aws/README.md +5 -8
- package/examples/control-plane/README.md +8 -9
- package/examples/control-plane/fixtures/control-plane.seed.json +8 -32
- package/examples/control-plane/public/app.js +57 -18
- package/examples/control-plane/public/index.html +15 -6
- package/examples/control-plane/server.mjs +15 -3
- package/examples/eval-results/public.results.passing.json +6 -21
- package/examples/providers/ai-sdk-next/README.md +11 -8
- package/examples/providers/anthropic-claude/README.md +5 -6
- package/examples/providers/gemini/README.md +6 -6
- package/examples/providers/langchain-agent/README.md +3 -4
- package/examples/providers/langgraph-tool-node/README.md +7 -7
- package/examples/providers/mcp/README.md +8 -8
- package/examples/providers/provider-conformance/README.md +3 -3
- package/examples/providers/semantic-kernel-mcp/README.md +6 -6
- package/examples/providers/semantic-kernel-openapi/README.md +6 -7
- package/examples/runtime-support-billing/README.md +8 -9
- package/examples/runtime-support-billing/support-billing-runtime.mjs +8 -4
- package/examples/scheduling/capabilities/scheduling.availability.get.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.prepare.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.send.yaml +2 -2
- package/examples/scheduling/entities/AvailabilitySlot.yaml +2 -2
- package/examples/scheduling/entities/MeetingInvite.yaml +2 -2
- package/examples/scheduling/evals/scheduling.invite.prepare.valid.yaml +3 -1
- package/examples/semantic-kernel-mcp/README.md +11 -12
- package/examples/semantic-kernel-openapi/README.md +8 -8
- package/examples/support/capabilities/support.refund.commit_case.yaml +3 -3
- package/examples/support/capabilities/support.refund.prepare_case.yaml +2 -2
- package/examples/support/capabilities/support.ticket.get.yaml +2 -2
- package/examples/support/entities/Ticket.yaml +0 -1
- package/examples/support/eval-results/support.results.passing.json +3 -12
- package/examples/support/evals/support.refund.prepare_case.approval_required.yaml +3 -1
- package/examples/support/evals/support.refund.prepare_case.valid.yaml +3 -2
- package/examples/support/memory/support.agent.preferences.json +2 -6
- package/examples/support/openai/context.support_agent.json +1 -4
- package/examples/support/provenance/support.refund.summary.provenance.json +2 -6
- package/examples/support/replay/support.refund.approval_required.trace.json +4 -14
- package/package.json +22 -5
- package/schemas/audit/action-record.schema.json +4 -1
- package/schemas/aws/approval-task.schema.json +8 -1
- package/schemas/conformance/conformance-case.schema.json +18 -2
- package/schemas/conformance/conformance-report.schema.json +17 -2
- package/schemas/conformance/provider-target-matrix.schema.json +35 -4
- package/schemas/controls/control-decision.schema.json +7 -1
- package/schemas/eval-case.schema.json +8 -1
- package/schemas/eval-result.schema.json +8 -1
- package/schemas/evidence/evidence-pack.schema.json +34 -5
- package/schemas/governance/compatibility-diff.schema.json +8 -1
- package/schemas/governance/lifecycle-transition.schema.json +10 -1
- package/schemas/governance/risk-compilation-result.schema.json +9 -1
- package/schemas/provenance/generated-content-provenance.schema.json +3 -1
- package/schemas/replay/replay-result.schema.json +16 -2
- package/schemas/replay/replay-trace.schema.json +11 -2
- package/schemas/security/context-segment.schema.json +21 -12
- package/schemas/security/redaction-policy.schema.json +4 -1
- package/schemas/security-packs/coverage-report.schema.json +17 -2
- package/schemas/security-packs/security-case-suite.schema.json +11 -1
- package/schemas/tool-result-envelope.schema.json +9 -1
- package/security-packs/approval_bypass.yaml +12 -2
- package/security-packs/capability_spoofing.yaml +17 -3
- package/security-packs/cost_amplification.yaml +21 -4
- package/security-packs/cross_tenant_access.yaml +16 -3
- package/security-packs/insecure_output_rendering.yaml +15 -4
- package/security-packs/mcp_tool_abuse.yaml +17 -3
- package/security-packs/memory_scope_violation.yaml +19 -4
- package/security-packs/prompt_injection_direct.yaml +19 -4
- package/security-packs/prompt_injection_indirect.yaml +18 -3
- package/security-packs/provider_payload_exposure.yaml +26 -4
- package/security-packs/retrieval_poisoning.yaml +5 -2
- package/security-packs/schema_confusion.yaml +17 -3
- package/security-packs/sensitive_data_disclosure.yaml +22 -5
- package/security-packs/tool_exfiltration.yaml +20 -4
- package/security-packs/tool_result_poisoning.yaml +19 -4
- package/security-packs/unsafe_commit_attempt.yaml +6 -3
package/docs/openai-runtime.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# OpenAI Responses Runtime
|
|
2
2
|
|
|
3
|
-
The OpenAI Responses runtime is an optional subpath for host applications that
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
3
|
+
The OpenAI Responses runtime is an optional subpath for host applications that want AICF
|
|
4
|
+
to run a bounded Responses tool loop. It builds runtime context, routes model-safe read
|
|
5
|
+
and prepare capabilities, calls a caller-provided OpenAI Responses client, executes
|
|
6
|
+
function calls through the R2 `AicfToolExecutor`, returns model-safe tool envelopes, and
|
|
7
|
+
produces final text.
|
|
8
8
|
|
|
9
9
|
Import it separately:
|
|
10
10
|
|
|
@@ -15,20 +15,19 @@ import {
|
|
|
15
15
|
} from "ai-capability-framework/openai";
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
-
The package root and `ai-capability-framework/runtime` imports do not import the
|
|
19
|
-
|
|
18
|
+
The package root and `ai-capability-framework/runtime` imports do not import the OpenAI
|
|
19
|
+
runtime code.
|
|
20
20
|
|
|
21
21
|
For a first pass through the support/refund example, start with the
|
|
22
|
-
[OpenAI walkthrough](openai-walkthrough.md). This page is the runtime API
|
|
23
|
-
|
|
24
|
-
client.
|
|
22
|
+
[OpenAI walkthrough](openai-walkthrough.md). This page is the runtime API reference for
|
|
23
|
+
host applications that want to provide a real OpenAI-compatible client.
|
|
25
24
|
|
|
26
25
|
## Boundary
|
|
27
26
|
|
|
28
|
-
The runtime loop is non-streaming and Responses-only. It does not expose commit
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
27
|
+
The runtime loop is non-streaming and Responses-only. It does not expose commit tools to
|
|
28
|
+
the model, execute side effects through model tool calls, persist production state,
|
|
29
|
+
collect approvals, write durable audit logs, add observability systems, or require the
|
|
30
|
+
OpenAI SDK for Core or runtime imports.
|
|
32
31
|
|
|
33
32
|
Host applications remain responsible for:
|
|
34
33
|
|
|
@@ -57,9 +56,8 @@ npm install openai
|
|
|
57
56
|
const client = await createDefaultOpenAIResponsesClient();
|
|
58
57
|
```
|
|
59
58
|
|
|
60
|
-
If the SDK is unavailable, `createDefaultOpenAIResponsesClient()` throws an
|
|
61
|
-
|
|
62
|
-
import `openai`.
|
|
59
|
+
If the SDK is unavailable, `createDefaultOpenAIResponsesClient()` throws an actionable
|
|
60
|
+
optional-dependency error. This is the only API that attempts to import `openai`.
|
|
63
61
|
|
|
64
62
|
## Running A Request
|
|
65
63
|
|
|
@@ -85,16 +83,16 @@ Defaults:
|
|
|
85
83
|
- `parallel_tool_calls = false`
|
|
86
84
|
- streaming is out of scope
|
|
87
85
|
|
|
88
|
-
The runner builds context, routes capabilities, converts the routed slice into
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
`function_call_output` items. Tool output is serialized with runtime
|
|
92
|
-
|
|
86
|
+
The runner builds context, routes capabilities, converts the routed slice into OpenAI
|
|
87
|
+
function tools, sends a non-streaming `responses.create` request, executes returned
|
|
88
|
+
`function_call` items through `AicfToolExecutor`, and continues with
|
|
89
|
+
`function_call_output` items. Tool output is serialized with runtime model-safe envelope
|
|
90
|
+
helpers.
|
|
93
91
|
|
|
94
|
-
If an OpenAI function call omits `call_id`, the runner fails safely rather than
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
92
|
+
If an OpenAI function call omits `call_id`, the runner fails safely rather than guessing
|
|
93
|
+
how to pair output. Unknown tool names and invalid arguments become model-safe
|
|
94
|
+
unavailable or validation-error envelopes. Provider failures return `provider_error`
|
|
95
|
+
without raw provider payloads.
|
|
98
96
|
|
|
99
97
|
## Result Shape
|
|
100
98
|
|
|
@@ -111,8 +109,8 @@ model-safe unavailable or validation-error envelopes. Provider failures return
|
|
|
111
109
|
- lightweight runtime events
|
|
112
110
|
- safe errors
|
|
113
111
|
|
|
114
|
-
Runtime events are summaries only. They must not be treated as trace storage and
|
|
115
|
-
|
|
112
|
+
Runtime events are summaries only. They must not be treated as trace storage and do not
|
|
113
|
+
include raw provider payloads, prompts, secrets, or diagnostics.
|
|
116
114
|
|
|
117
115
|
Pass `traceSink` and `contentCapture` to emit sanitized events to the optional
|
|
118
116
|
observability subpath:
|
|
@@ -125,29 +123,26 @@ const result = await runOpenAIResponses({
|
|
|
125
123
|
});
|
|
126
124
|
```
|
|
127
125
|
|
|
128
|
-
See [observability runtime](observability-runtime.md) for trace sinks and
|
|
129
|
-
|
|
126
|
+
See [observability runtime](observability-runtime.md) for trace sinks and redaction
|
|
127
|
+
behavior.
|
|
130
128
|
|
|
131
129
|
## Agents SDK Bridge
|
|
132
130
|
|
|
133
131
|
The OpenAI subpath also exports `buildAgentsSdkTools()` and
|
|
134
|
-
`createDefaultAgentsSdkBridgeFactory()` for host applications that already use
|
|
135
|
-
|
|
132
|
+
`createDefaultAgentsSdkBridgeFactory()` for host applications that already use the
|
|
133
|
+
OpenAI Agents SDK.
|
|
136
134
|
|
|
137
|
-
The bridge creates function-tool definitions backed by `AicfToolExecutor`. It
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
Commit tools are not generated.
|
|
135
|
+
The bridge creates function-tool definitions backed by `AicfToolExecutor`. It does not
|
|
136
|
+
create an Agent, start an Agents SDK run, call a model, or make the Agents SDK approval
|
|
137
|
+
system the source of truth. Approval-required AICF prepare results are returned as
|
|
138
|
+
model-safe envelopes with prepared-action references. Commit tools are not generated.
|
|
142
139
|
|
|
143
140
|
## Testing
|
|
144
141
|
|
|
145
142
|
Mock tests use `MockOpenAIResponsesClient`:
|
|
146
143
|
|
|
147
144
|
```ts
|
|
148
|
-
const client = new MockOpenAIResponsesClient([
|
|
149
|
-
mockTextResponse("Done.")
|
|
150
|
-
]);
|
|
145
|
+
const client = new MockOpenAIResponsesClient([mockTextResponse("Done.")]);
|
|
151
146
|
```
|
|
152
147
|
|
|
153
148
|
Run mock tests without API keys:
|
|
@@ -170,14 +165,14 @@ Then run:
|
|
|
170
165
|
npm run test:openai:live
|
|
171
166
|
```
|
|
172
167
|
|
|
173
|
-
Live tests should stay harmless and synthetic. Do not store raw provider
|
|
174
|
-
|
|
168
|
+
Live tests should stay harmless and synthetic. Do not store raw provider payloads,
|
|
169
|
+
traces, prompts, or customer data in the repository.
|
|
175
170
|
|
|
176
171
|
Live eval suites use this same runtime path with mock clients by default. See
|
|
177
172
|
[live evals](live-evals.md).
|
|
178
173
|
|
|
179
|
-
For a no-model runtime flow that uses the same context, router, executor, and
|
|
180
|
-
|
|
174
|
+
For a no-model runtime flow that uses the same context, router, executor, and action
|
|
175
|
+
lifecycle primitives, run:
|
|
181
176
|
|
|
182
177
|
```bash
|
|
183
178
|
npm run build
|
|
@@ -186,8 +181,8 @@ node examples/runtime-support-billing/run-mock.mjs
|
|
|
186
181
|
|
|
187
182
|
## Reference Basis
|
|
188
183
|
|
|
189
|
-
The runtime uses the Responses `responses.create` request shape and function
|
|
190
|
-
|
|
184
|
+
The runtime uses the Responses `responses.create` request shape and function calling
|
|
185
|
+
continuation pattern documented by OpenAI:
|
|
191
186
|
|
|
192
187
|
- [Responses create API](https://platform.openai.com/docs/api-reference/responses/create?api-mode=responses)
|
|
193
188
|
- [Function calling guide](https://platform.openai.com/docs/guides/function-calling?api-mode=responses)
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
# OpenAI Walkthrough
|
|
2
2
|
|
|
3
|
-
This walkthrough uses the public support/refund example to show how AICF works
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
This walkthrough uses the public support/refund example to show how AICF works with
|
|
4
|
+
OpenAI. The default path is API-key-free: it exports tools, shows the tool call and
|
|
5
|
+
repair shapes, and uses the mock runtime example. Live OpenAI usage is optional and
|
|
6
|
+
requires a caller-provided client.
|
|
7
7
|
|
|
8
8
|
## The Scenario
|
|
9
9
|
|
|
@@ -19,14 +19,13 @@ The support example has three relevant capabilities:
|
|
|
19
19
|
- `support.refund.prepare_case`: prepare a refund case for approval.
|
|
20
20
|
- `support.refund.commit_case`: commit the refund after host approval.
|
|
21
21
|
|
|
22
|
-
The first two can be model-facing tools. The commit capability is host
|
|
23
|
-
|
|
22
|
+
The first two can be model-facing tools. The commit capability is host controlled and is
|
|
23
|
+
not exposed to the model by default.
|
|
24
24
|
|
|
25
25
|
## 1. Capability Manifests Define The Contract
|
|
26
26
|
|
|
27
|
-
A capability manifest describes the model-facing purpose, the input and output
|
|
28
|
-
|
|
29
|
-
eval links.
|
|
27
|
+
A capability manifest describes the model-facing purpose, the input and output schemas,
|
|
28
|
+
the risk tier, required permissions, policy gates, lifecycle flags, and eval links.
|
|
30
29
|
|
|
31
30
|
Small excerpt:
|
|
32
31
|
|
|
@@ -41,8 +40,7 @@ policy:
|
|
|
41
40
|
approval_required: true
|
|
42
41
|
```
|
|
43
42
|
|
|
44
|
-
AICF validates these manifests before any provider tool export or runtime
|
|
45
|
-
execution.
|
|
43
|
+
AICF validates these manifests before any provider tool export or runtime execution.
|
|
46
44
|
|
|
47
45
|
## 2. AICF Exports OpenAI Function Tools
|
|
48
46
|
|
|
@@ -77,8 +75,8 @@ Small excerpt:
|
|
|
77
75
|
}
|
|
78
76
|
```
|
|
79
77
|
|
|
80
|
-
The binding map is important. AICF never guesses a capability ID from a provider
|
|
81
|
-
|
|
78
|
+
The binding map is important. AICF never guesses a capability ID from a provider tool
|
|
79
|
+
name; it maps tool calls back through generated bindings.
|
|
82
80
|
|
|
83
81
|
## 3. OpenAI Returns A Function Call
|
|
84
82
|
|
|
@@ -93,14 +91,13 @@ A model might return a Responses function call like this:
|
|
|
93
91
|
}
|
|
94
92
|
```
|
|
95
93
|
|
|
96
|
-
AICF parses the provider call, resolves the binding to `support.ticket.get`,
|
|
97
|
-
|
|
98
|
-
`input_schema`.
|
|
94
|
+
AICF parses the provider call, resolves the binding to `support.ticket.get`, parses JSON
|
|
95
|
+
arguments, and validates those arguments against the original AICF `input_schema`.
|
|
99
96
|
|
|
100
97
|
## 4. Invalid Calls Become Repair Envelopes
|
|
101
98
|
|
|
102
|
-
If the model calls an unknown tool or sends incorrect arguments, AICF fails
|
|
103
|
-
|
|
99
|
+
If the model calls an unknown tool or sends incorrect arguments, AICF fails closed. The
|
|
100
|
+
app can return a model-safe repair envelope as the tool output.
|
|
104
101
|
|
|
105
102
|
Example validation failure:
|
|
106
103
|
|
|
@@ -128,9 +125,9 @@ OpenAI continuation shape:
|
|
|
128
125
|
}
|
|
129
126
|
```
|
|
130
127
|
|
|
131
|
-
The model can then repair its next call. The app may also decide to stop, ask
|
|
132
|
-
|
|
133
|
-
|
|
128
|
+
The model can then repair its next call. The app may also decide to stop, ask the user
|
|
129
|
+
for clarification, or escalate to a stronger model. AICF only provides the safe envelope
|
|
130
|
+
and deterministic validation result.
|
|
134
131
|
|
|
135
132
|
## 5. Prepare Can Require Approval
|
|
136
133
|
|
|
@@ -149,8 +146,7 @@ envelope:
|
|
|
149
146
|
}
|
|
150
147
|
```
|
|
151
148
|
|
|
152
|
-
This tells the model that work has paused. It must not claim the refund was
|
|
153
|
-
issued.
|
|
149
|
+
This tells the model that work has paused. It must not claim the refund was issued.
|
|
154
150
|
|
|
155
151
|
## 6. Commit Is Host-Controlled
|
|
156
152
|
|
|
@@ -180,16 +176,16 @@ Expected excerpt:
|
|
|
180
176
|
|
|
181
177
|
## 7. Optional Live OpenAI Runtime
|
|
182
178
|
|
|
183
|
-
The optional `ai-capability-framework/openai` subpath can run a bounded
|
|
184
|
-
|
|
179
|
+
The optional `ai-capability-framework/openai` subpath can run a bounded non-streaming
|
|
180
|
+
Responses loop with a caller-provided compatible client:
|
|
185
181
|
|
|
186
182
|
```ts
|
|
187
183
|
import { runOpenAIResponses } from "ai-capability-framework/openai";
|
|
188
184
|
```
|
|
189
185
|
|
|
190
|
-
AICF does not require the OpenAI SDK for root or runtime imports. Live usage is
|
|
191
|
-
|
|
192
|
-
|
|
186
|
+
AICF does not require the OpenAI SDK for root or runtime imports. Live usage is opt-in,
|
|
187
|
+
and host applications remain responsible for provider credentials, model selection,
|
|
188
|
+
auth, side effects, approvals, storage, and audit.
|
|
193
189
|
|
|
194
190
|
## What To Read Next
|
|
195
191
|
|
package/docs/policy-broker.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Policy Broker
|
|
2
2
|
|
|
3
|
-
`DefaultPolicyBroker` combines Core manifest decisions with runtime context. It
|
|
4
|
-
|
|
3
|
+
`DefaultPolicyBroker` combines Core manifest decisions with runtime context. It is
|
|
4
|
+
deterministic and fail-closed.
|
|
5
5
|
|
|
6
6
|
Import it from:
|
|
7
7
|
|
|
@@ -15,13 +15,12 @@ Policy evaluation receives:
|
|
|
15
15
|
|
|
16
16
|
- the loaded capability manifest;
|
|
17
17
|
- operation: `select`, `prepare`, or `commit`;
|
|
18
|
-
- runtime context with subject, account, tenant, autonomy, risk, facts, and
|
|
19
|
-
metadata;
|
|
18
|
+
- runtime context with subject, account, tenant, autonomy, risk, facts, and metadata;
|
|
20
19
|
- args, approval, prepared-action, idempotency, and built context when relevant.
|
|
21
20
|
|
|
22
21
|
Host applications remain responsible for real authorization, account state,
|
|
23
|
-
entitlements, facts, and approval verification. AICF evaluates the information
|
|
24
|
-
|
|
22
|
+
entitlements, facts, and approval verification. AICF evaluates the information the host
|
|
23
|
+
supplies.
|
|
25
24
|
|
|
26
25
|
## Default Behavior
|
|
27
26
|
|
|
@@ -34,8 +33,8 @@ The broker wraps Core `decideCapability()` and adds runtime checks for:
|
|
|
34
33
|
- missing, rejected, expired, or mismatched approvals;
|
|
35
34
|
- missing idempotency keys when required by the capability.
|
|
36
35
|
|
|
37
|
-
Ambiguous state denies or pauses. The broker does not guess missing facts or
|
|
38
|
-
|
|
36
|
+
Ambiguous state denies or pauses. The broker does not guess missing facts or repair
|
|
37
|
+
incomplete context.
|
|
39
38
|
|
|
40
39
|
## Host Hooks
|
|
41
40
|
|
|
@@ -45,5 +44,5 @@ An optional host policy hook may return a stricter decision:
|
|
|
45
44
|
- `approval_required` can override `allowed`;
|
|
46
45
|
- `allowed` cannot override an AICF denial.
|
|
47
46
|
|
|
48
|
-
Thrown hooks fail closed. Hook diagnostics must stay safe for logs and must not
|
|
49
|
-
|
|
47
|
+
Thrown hooks fail closed. Hook diagnostics must stay safe for logs and must not include
|
|
48
|
+
secrets, raw prompts, provider payloads, or customer data.
|
package/docs/provenance.md
CHANGED
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
# Content Provenance Hooks
|
|
2
2
|
|
|
3
|
-
The provenance subpath creates public-safe metadata for generated content that
|
|
4
|
-
|
|
5
|
-
media.
|
|
3
|
+
The provenance subpath creates public-safe metadata for generated content that may
|
|
4
|
+
become customer-facing text, documents, images, audio, video, or other media.
|
|
6
5
|
|
|
7
|
-
It does not sign content, implement C2PA, process files, call providers, store
|
|
8
|
-
|
|
9
|
-
|
|
6
|
+
It does not sign content, implement C2PA, process files, call providers, store records,
|
|
7
|
+
or claim authenticity. Host applications own real signing, document embedding, CMS/media
|
|
8
|
+
pipelines, approval identity, retention, and legal review.
|
|
10
9
|
|
|
11
10
|
## TypeScript
|
|
12
11
|
|
|
@@ -27,12 +26,15 @@ const provenance = createGeneratedContentProvenance({
|
|
|
27
26
|
{ capabilityId: "support.ticket.get", operation: "read" },
|
|
28
27
|
{ capabilityId: "support.refund.prepare_case", operation: "prepare" }
|
|
29
28
|
],
|
|
30
|
-
sourceRefs: [
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
29
|
+
sourceRefs: [
|
|
30
|
+
{
|
|
31
|
+
contentHash:
|
|
32
|
+
"sha256:1111111111111111111111111111111111111111111111111111111111111111",
|
|
33
|
+
sourceId: "support.ticket.synthetic.ref",
|
|
34
|
+
sourceType: "app_record",
|
|
35
|
+
trust: "app_data"
|
|
36
|
+
}
|
|
37
|
+
],
|
|
36
38
|
approvalRefs: ["approval.support.refund.synthetic"],
|
|
37
39
|
createdAt: new Date().toISOString()
|
|
38
40
|
});
|
|
@@ -44,20 +46,20 @@ const validation = validateGeneratedContentProvenance(provenance);
|
|
|
44
46
|
|
|
45
47
|
`GeneratedContentProvenance` records:
|
|
46
48
|
|
|
47
|
-
- content ID, content type, creation time, and whether content was model,
|
|
48
|
-
|
|
49
|
+
- content ID, content type, creation time, and whether content was model, model-assisted
|
|
50
|
+
human, or human-approved model output;
|
|
49
51
|
- provider, model, capability, approval, and trace references;
|
|
50
52
|
- source references with hashes or public-safe IDs.
|
|
51
53
|
|
|
52
|
-
The record is intentionally a sidecar contract. It contains refs, hashes,
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
54
|
+
The record is intentionally a sidecar contract. It contains refs, hashes, statuses, and
|
|
55
|
+
metadata only. It must not include report body text, raw prompts, provider
|
|
56
|
+
request/response payloads, transcripts, secrets, stack traces, unredacted
|
|
57
|
+
tenant/account/user IDs, or sensitive source content.
|
|
56
58
|
|
|
57
59
|
## Adapter Hooks
|
|
58
60
|
|
|
59
|
-
`runProvenanceAdapterHook()` lets a host pass validated metadata to a signing,
|
|
60
|
-
|
|
61
|
+
`runProvenanceAdapterHook()` lets a host pass validated metadata to a signing, CMS,
|
|
62
|
+
document, or media pipeline adapter:
|
|
61
63
|
|
|
62
64
|
```ts
|
|
63
65
|
const result = await runProvenanceAdapterHook(provenance, async (record) => ({
|
|
@@ -71,20 +73,19 @@ const result = await runProvenanceAdapterHook(provenance, async (record) => ({
|
|
|
71
73
|
}));
|
|
72
74
|
```
|
|
73
75
|
|
|
74
|
-
The hook receives only validated, public-safe provenance metadata. If a hook
|
|
75
|
-
|
|
76
|
-
|
|
76
|
+
The hook receives only validated, public-safe provenance metadata. If a hook throws or
|
|
77
|
+
returns unsafe metadata, AICF returns a safe failed result without stack traces or raw
|
|
78
|
+
error text.
|
|
77
79
|
|
|
78
80
|
## Validation
|
|
79
81
|
|
|
80
|
-
Validation is fail-closed. It rejects missing required references, invalid
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
hash but do not.
|
|
82
|
+
Validation is fail-closed. It rejects missing required references, invalid timestamps,
|
|
83
|
+
unknown content types, unsafe raw-content-looking fields, provider payload markers,
|
|
84
|
+
credential-looking keys, and source refs that should include a hash but do not.
|
|
84
85
|
|
|
85
86
|
The public example at
|
|
86
|
-
`examples/support/provenance/support.refund.summary.provenance.json` shows a
|
|
87
|
-
|
|
87
|
+
`examples/support/provenance/support.refund.summary.provenance.json` shows a synthetic
|
|
88
|
+
support/refund document sidecar with hashes and refs only.
|
|
88
89
|
|
|
89
90
|
## Relationship To Other AICF Surfaces
|
|
90
91
|
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# Anthropic Claude
|
|
2
2
|
|
|
3
|
-
The Anthropic provider subpath exposes routed AICF read/prepare capabilities as
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
The Anthropic provider subpath exposes routed AICF read/prepare capabilities as Claude
|
|
4
|
+
Messages tools and can run a bounded mock-tested tool-use loop with a caller-provided
|
|
5
|
+
client.
|
|
6
6
|
|
|
7
7
|
Read:
|
|
8
8
|
|
|
9
9
|
- [Anthropic Claude runtime](../anthropic-runtime.md)
|
|
10
10
|
|
|
11
|
-
Default tests use mock clients. Live Claude tests are opt-in and require
|
|
12
|
-
|
|
11
|
+
Default tests use mock clients. Live Claude tests are opt-in and require explicit
|
|
12
|
+
environment variables.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Choose a Runtime
|
|
2
|
+
|
|
3
|
+
Start with provider-neutral AICF contracts, then pick the smallest adapter layer your
|
|
4
|
+
host application needs.
|
|
5
|
+
|
|
6
|
+
| Path | Use When | Default Test Mode |
|
|
7
|
+
| ------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------ |
|
|
8
|
+
| Core contracts | You need manifests, validation, deterministic decisions, and evals without model calls. | `npm run validate`, `npm run conformance` |
|
|
9
|
+
| OpenAI Responses | You want a bounded OpenAI tool loop over routed read/prepare capabilities. | `npm run test:openai:mock` |
|
|
10
|
+
| Anthropic Claude | You want Claude Messages tool use over routed read/prepare capabilities. | `npm run test:anthropic:mock` |
|
|
11
|
+
| Google Gemini | You want GenerateContent function calling over routed read/prepare capabilities. | `npm run test:gemini:mock` |
|
|
12
|
+
| Vercel AI SDK | You want AI SDK tools around host-supplied `generateText` or `streamText`. | `npm run test:ai-sdk:mock` |
|
|
13
|
+
| LangChain/LangGraph | You want LangChain tools or a host-supplied LangGraph `ToolNode`. | `npm run test:langchain:mock` |
|
|
14
|
+
| MCP | You want MCP descriptors or an AICF-backed MCP server surface. | `npm run test:mcp-provider`, `npm run test:mcp-server` |
|
|
15
|
+
| Semantic Kernel | You want MCP guidance or OpenAPI plugin metadata for Semantic Kernel hosts. | `npm run test:semantic-kernel` |
|
|
16
|
+
|
|
17
|
+
## Selection Rules
|
|
18
|
+
|
|
19
|
+
- If you are learning AICF, start with the
|
|
20
|
+
[provider-neutral quickstart](../getting-started/provider-neutral-quickstart.md).
|
|
21
|
+
- If you already have an OpenAI app, use the
|
|
22
|
+
[OpenAI quickstart](../getting-started/openai-quickstart.md).
|
|
23
|
+
- If you need a non-OpenAI direct runtime, use the
|
|
24
|
+
[Anthropic quickstart](../getting-started/anthropic-quickstart.md) or
|
|
25
|
+
[Gemini quickstart](../getting-started/gemini-quickstart.md).
|
|
26
|
+
- If an agent framework owns the model loop, use the AI SDK, LangChain, MCP, or Semantic
|
|
27
|
+
Kernel bridge and keep AICF as the validation and lifecycle authority.
|
|
28
|
+
|
|
29
|
+
## Boundary
|
|
30
|
+
|
|
31
|
+
All runtime choices share the same safety boundary:
|
|
32
|
+
|
|
33
|
+
- provider descriptors expose routed read/prepare capabilities only;
|
|
34
|
+
- commit remains host-controlled through the action lifecycle;
|
|
35
|
+
- provider SDK validation does not replace AICF validation;
|
|
36
|
+
- raw provider payloads and raw prompts are not stored by default;
|
|
37
|
+
- live calls require explicit environment variables and host-owned credentials.
|
|
@@ -4,9 +4,9 @@ AICF conformance checks prove that one routed capability slice can be exported
|
|
|
4
4
|
consistently across provider and framework targets. The canonical import path is
|
|
5
5
|
`ai-capability-framework/conformance`.
|
|
6
6
|
|
|
7
|
-
The conformance harness is descriptor/mock-only. It does not call live models,
|
|
8
|
-
|
|
9
|
-
|
|
7
|
+
The conformance harness is descriptor/mock-only. It does not call live models, use
|
|
8
|
+
credentials, execute provider SDKs, store provider payloads, or expose commit
|
|
9
|
+
capabilities to models.
|
|
10
10
|
|
|
11
11
|
## Install And Import
|
|
12
12
|
|
|
@@ -22,8 +22,8 @@ import {
|
|
|
22
22
|
} from "ai-capability-framework/conformance";
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
-
The older `ai-capability-framework/providers/conformance` path remains available
|
|
26
|
-
|
|
25
|
+
The older `ai-capability-framework/providers/conformance` path remains available as a
|
|
26
|
+
compatibility alias.
|
|
27
27
|
|
|
28
28
|
## Targets
|
|
29
29
|
|
|
@@ -38,8 +38,8 @@ The canonical matrix covers:
|
|
|
38
38
|
- Semantic Kernel through MCP compatibility;
|
|
39
39
|
- Semantic Kernel through OpenAPI plugin metadata.
|
|
40
40
|
|
|
41
|
-
`semantic-kernel` is accepted as an alias for `semantic-kernel-openapi`.
|
|
42
|
-
|
|
41
|
+
`semantic-kernel` is accepted as an alias for `semantic-kernel-openapi`. `vercel-ai-sdk`
|
|
42
|
+
and `vercel_ai_sdk` are accepted as aliases for `ai-sdk`.
|
|
43
43
|
|
|
44
44
|
## What Gets Checked
|
|
45
45
|
|
|
@@ -56,9 +56,8 @@ The report scores each target across these dimensions:
|
|
|
56
56
|
- provider error normalization;
|
|
57
57
|
- loop or streaming boundary metadata.
|
|
58
58
|
|
|
59
|
-
Failures are structured by provider, capability, case, and dimension. The report
|
|
60
|
-
|
|
61
|
-
callers.
|
|
59
|
+
Failures are structured by provider, capability, case, and dimension. The report also
|
|
60
|
+
keeps the older `passed`, `counts`, and `results` fields for existing callers.
|
|
62
61
|
|
|
63
62
|
## CLI
|
|
64
63
|
|
|
@@ -84,12 +83,12 @@ aicf providers conformance examples --format text
|
|
|
84
83
|
|
|
85
84
|
## Boundary
|
|
86
85
|
|
|
87
|
-
Host applications remain responsible for model calls, provider SDK setup,
|
|
88
|
-
|
|
89
|
-
|
|
86
|
+
Host applications remain responsible for model calls, provider SDK setup, runtime
|
|
87
|
+
context, real authorization, approval collection, idempotency, side effects, audit
|
|
88
|
+
persistence, and optional live tests.
|
|
90
89
|
|
|
91
|
-
Conformance uses public manifests, synthetic contexts, and mock/canonical calls.
|
|
92
|
-
|
|
90
|
+
Conformance uses public manifests, synthetic contexts, and mock/canonical calls. Live
|
|
91
|
+
provider behavior belongs in provider-specific opt-in tests.
|
|
93
92
|
|
|
94
93
|
## Checks
|
|
95
94
|
|
package/docs/providers/gemini.md
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# Google Gemini
|
|
2
2
|
|
|
3
|
-
The Gemini provider subpath exposes routed AICF read/prepare capabilities as
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
The Gemini provider subpath exposes routed AICF read/prepare capabilities as Gemini
|
|
4
|
+
function declarations and can run a bounded mock-tested GenerateContent loop with a
|
|
5
|
+
caller-provided client.
|
|
6
6
|
|
|
7
7
|
Read:
|
|
8
8
|
|
|
9
9
|
- [Google Gemini runtime](../gemini-runtime.md)
|
|
10
10
|
|
|
11
|
-
Default tests use mock clients. Live Gemini tests are opt-in and require
|
|
12
|
-
|
|
11
|
+
Default tests use mock clients. Live Gemini tests are opt-in and require explicit
|
|
12
|
+
environment variables.
|
package/docs/providers/mcp.md
CHANGED
|
@@ -1,8 +1,7 @@
|
|
|
1
1
|
# Model Context Protocol
|
|
2
2
|
|
|
3
|
-
AICF supports MCP descriptors and an optional MCP server runtime. MCP clients
|
|
4
|
-
|
|
5
|
-
boundaries.
|
|
3
|
+
AICF supports MCP descriptors and an optional MCP server runtime. MCP clients receive
|
|
4
|
+
routed read/prepare tools backed by AICF validation and executor boundaries.
|
|
6
5
|
|
|
7
6
|
Read:
|
|
8
7
|
|
package/docs/providers/openai.md
CHANGED
|
@@ -11,5 +11,5 @@ Start here:
|
|
|
11
11
|
- [OpenAI Responses descriptor adapter](../openai-responses.md)
|
|
12
12
|
- [OpenAI Responses runtime](../openai-runtime.md)
|
|
13
13
|
|
|
14
|
-
Default examples do not call OpenAI. Live use requires a caller-provided client
|
|
15
|
-
|
|
14
|
+
Default examples do not call OpenAI. Live use requires a caller-provided client or
|
|
15
|
+
optional SDK setup.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Semantic Kernel
|
|
2
2
|
|
|
3
|
-
AICF supports Semantic Kernel-compatible hosts through MCP guidance and OpenAPI
|
|
4
|
-
|
|
3
|
+
AICF supports Semantic Kernel-compatible hosts through MCP guidance and OpenAPI plugin
|
|
4
|
+
metadata export. It does not provide a native Semantic Kernel runtime.
|
|
5
5
|
|
|
6
6
|
Read:
|
|
7
7
|
|