ai-capability-framework 1.0.0-rc.3 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +97 -103
- package/CODE_OF_CONDUCT.md +11 -13
- package/CONTRIBUTING.md +40 -43
- package/GOVERNANCE.md +34 -35
- package/README.md +99 -63
- package/ROADMAP.md +12 -12
- package/SECURITY.md +15 -15
- package/dist/audit/ledger.d.ts.map +1 -1
- package/dist/audit/ledger.js +2 -1
- package/dist/audit/records.d.ts.map +1 -1
- package/dist/audit/records.js +2 -1
- package/dist/audit/types.d.ts +4 -1
- package/dist/audit/types.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.d.ts +2 -7
- package/dist/aws/dynamodb-stores.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.js +56 -5
- package/dist/aws/testing.d.ts.map +1 -1
- package/dist/aws/testing.js +9 -0
- package/dist/cli.d.ts +6 -6
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +77 -7
- package/dist/evals-live/index.d.ts +2 -2
- package/dist/evals-live/index.d.ts.map +1 -1
- package/dist/evals-live/index.js +1 -1
- package/dist/evals-live/live-eval-runner.d.ts +49 -1
- package/dist/evals-live/live-eval-runner.d.ts.map +1 -1
- package/dist/evals-live/live-eval-runner.js +246 -4
- package/dist/evals-live/types.d.ts +62 -9
- package/dist/evals-live/types.d.ts.map +1 -1
- package/dist/index.d.ts +0 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +0 -1
- package/dist/runtime/actions.d.ts +2 -0
- package/dist/runtime/actions.d.ts.map +1 -1
- package/dist/runtime/actions.js +200 -55
- package/dist/runtime/index.d.ts +1 -1
- package/dist/runtime/index.d.ts.map +1 -1
- package/dist/runtime/stores.d.ts +2 -7
- package/dist/runtime/stores.d.ts.map +1 -1
- package/dist/runtime/stores.js +12 -0
- package/dist/runtime/types.d.ts +25 -6
- package/dist/runtime/types.d.ts.map +1 -1
- package/docs/action-lifecycle.md +38 -32
- package/docs/adapter-roadmap.md +51 -49
- package/docs/adapters.md +14 -15
- package/docs/ai-sdk-runtime.md +36 -39
- package/docs/anthropic-runtime.md +29 -32
- package/docs/api/public-api-policy.md +59 -0
- package/docs/api.md +354 -371
- package/docs/architecture/current-state.md +28 -31
- package/docs/audit/index.md +29 -29
- package/docs/aws/cloudwatch-telemetry.md +5 -7
- package/docs/aws/dynamodb-single-table.md +8 -9
- package/docs/aws/kms-redaction.md +5 -6
- package/docs/aws/overview.md +5 -5
- package/docs/aws/production-reference.md +13 -14
- package/docs/aws/step-functions-approval.md +7 -9
- package/docs/aws-runtime.md +24 -25
- package/docs/capability-manifests.md +17 -21
- package/docs/control-plane/overview.md +2 -2
- package/docs/control-plane.md +24 -27
- package/docs/controls/index.md +19 -22
- package/docs/core/capability-manifests.md +4 -5
- package/docs/core/entity-manifests.md +5 -5
- package/docs/eval-runner.md +29 -31
- package/docs/evalops.md +13 -14
- package/docs/evals/golden-tests.md +4 -5
- package/docs/evals/overview.md +3 -3
- package/docs/evals/replay-and-trace-to-golden.md +19 -20
- package/docs/evals.md +19 -20
- package/docs/evidence.md +22 -23
- package/docs/gemini-runtime.md +31 -33
- package/docs/getting-started/anthropic-quickstart.md +61 -0
- package/docs/getting-started/concepts.md +9 -10
- package/docs/getting-started/gemini-quickstart.md +62 -0
- package/docs/getting-started/installation.md +3 -3
- package/docs/getting-started/openai-quickstart.md +61 -0
- package/docs/getting-started/provider-neutral-quickstart.md +106 -0
- package/docs/getting-started/quickstart.md +2 -2
- package/docs/getting-started.md +23 -20
- package/docs/glossary.md +31 -48
- package/docs/governance/gate.md +28 -30
- package/docs/governance/impact-and-compatibility.md +2 -3
- package/docs/governance/index.md +53 -54
- package/docs/governance/lifecycle.md +4 -5
- package/docs/governance/overview.md +2 -3
- package/docs/governance/risk-compiler.md +4 -5
- package/docs/host-responsibilities.md +6 -8
- package/docs/implementation-notes/baseline-gaps.md +15 -18
- package/docs/index.md +21 -6
- package/docs/interoperability.md +9 -9
- package/docs/langchain-runtime.md +27 -31
- package/docs/live-evals.md +68 -32
- package/docs/mcp-server-runtime.md +36 -40
- package/docs/memory.md +35 -36
- package/docs/migration-0.1-to-1.0.md +14 -9
- package/docs/observability/overview.md +2 -3
- package/docs/observability-runtime.md +35 -39
- package/docs/openai-responses.md +36 -37
- package/docs/openai-runtime.md +41 -46
- package/docs/openai-walkthrough.md +24 -28
- package/docs/policy-broker.md +9 -10
- package/docs/provenance.md +30 -29
- package/docs/providers/anthropic.md +5 -5
- package/docs/providers/choose-a-runtime.md +37 -0
- package/docs/providers/conformance.md +14 -15
- package/docs/providers/gemini.md +5 -5
- package/docs/providers/mcp.md +2 -3
- package/docs/providers/openai.md +2 -2
- package/docs/providers/semantic-kernel.md +2 -2
- package/docs/providers.md +43 -38
- package/docs/public/license-exceptions.md +187 -0
- package/docs/public/npm-release-preflight.md +112 -0
- package/docs/public-framework/compatibility-policy.md +12 -13
- package/docs/public-framework/deprecation-policy.md +8 -8
- package/docs/public-framework/final-certification-matrix.md +104 -0
- package/docs/public-framework/license-decision.md +20 -0
- package/docs/public-framework/release-process.md +46 -16
- package/docs/public-framework/security-disclosure.md +4 -5
- package/docs/public-framework/v1-certification.md +102 -15
- package/docs/release.md +200 -122
- package/docs/runtime/action-lifecycle.md +3 -3
- package/docs/runtime/policy-broker.md +4 -4
- package/docs/runtime/runtime-overview.md +4 -5
- package/docs/runtime/tool-result-envelope.md +7 -7
- package/docs/runtime.md +90 -93
- package/docs/security/overview.md +4 -4
- package/docs/security/security-packs.md +24 -26
- package/docs/security/trust-taint-redaction.md +42 -47
- package/docs/semantic-kernel-runtime.md +35 -37
- package/docs/spec.md +95 -101
- package/docs/start-here.md +18 -20
- package/examples/01-basic-read-capability/README.md +3 -4
- package/examples/02-prepare-approve-commit/README.md +6 -6
- package/examples/03-multi-provider-tools/README.md +2 -2
- package/examples/04-mcp-server/README.md +4 -5
- package/examples/05-langchain-langgraph-bridge/README.md +4 -5
- package/examples/06-vercel-ai-sdk-bridge/README.md +5 -7
- package/examples/07-policy-broker-custom-auth/README.md +4 -4
- package/examples/08-aws-step-functions-approval/README.md +4 -5
- package/examples/09-security-packs-promptfoo/README.md +4 -4
- package/examples/10-trace-to-golden/README.md +2 -2
- package/examples/11-control-plane/README.md +4 -5
- package/examples/aws/README.md +5 -8
- package/examples/control-plane/README.md +8 -9
- package/examples/control-plane/fixtures/control-plane.seed.json +8 -32
- package/examples/control-plane/public/app.js +57 -18
- package/examples/control-plane/public/index.html +15 -6
- package/examples/control-plane/server.mjs +15 -3
- package/examples/eval-results/public.results.passing.json +6 -21
- package/examples/providers/ai-sdk-next/README.md +11 -8
- package/examples/providers/anthropic-claude/README.md +5 -6
- package/examples/providers/gemini/README.md +6 -6
- package/examples/providers/langchain-agent/README.md +3 -4
- package/examples/providers/langgraph-tool-node/README.md +7 -7
- package/examples/providers/mcp/README.md +8 -8
- package/examples/providers/provider-conformance/README.md +3 -3
- package/examples/providers/semantic-kernel-mcp/README.md +6 -6
- package/examples/providers/semantic-kernel-openapi/README.md +6 -7
- package/examples/runtime-support-billing/README.md +8 -9
- package/examples/runtime-support-billing/support-billing-runtime.mjs +8 -4
- package/examples/scheduling/capabilities/scheduling.availability.get.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.prepare.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.send.yaml +2 -2
- package/examples/scheduling/entities/AvailabilitySlot.yaml +2 -2
- package/examples/scheduling/entities/MeetingInvite.yaml +2 -2
- package/examples/scheduling/evals/scheduling.invite.prepare.valid.yaml +3 -1
- package/examples/semantic-kernel-mcp/README.md +11 -12
- package/examples/semantic-kernel-openapi/README.md +8 -8
- package/examples/support/capabilities/support.refund.commit_case.yaml +3 -3
- package/examples/support/capabilities/support.refund.prepare_case.yaml +2 -2
- package/examples/support/capabilities/support.ticket.get.yaml +2 -2
- package/examples/support/entities/Ticket.yaml +0 -1
- package/examples/support/eval-results/support.results.passing.json +3 -12
- package/examples/support/evals/support.refund.prepare_case.approval_required.yaml +3 -1
- package/examples/support/evals/support.refund.prepare_case.valid.yaml +3 -2
- package/examples/support/memory/support.agent.preferences.json +2 -6
- package/examples/support/openai/context.support_agent.json +1 -4
- package/examples/support/provenance/support.refund.summary.provenance.json +2 -6
- package/examples/support/replay/support.refund.approval_required.trace.json +4 -14
- package/package.json +22 -5
- package/schemas/audit/action-record.schema.json +4 -1
- package/schemas/aws/approval-task.schema.json +8 -1
- package/schemas/conformance/conformance-case.schema.json +18 -2
- package/schemas/conformance/conformance-report.schema.json +17 -2
- package/schemas/conformance/provider-target-matrix.schema.json +35 -4
- package/schemas/controls/control-decision.schema.json +7 -1
- package/schemas/eval-case.schema.json +8 -1
- package/schemas/eval-result.schema.json +8 -1
- package/schemas/evidence/evidence-pack.schema.json +34 -5
- package/schemas/governance/compatibility-diff.schema.json +8 -1
- package/schemas/governance/lifecycle-transition.schema.json +10 -1
- package/schemas/governance/risk-compilation-result.schema.json +9 -1
- package/schemas/provenance/generated-content-provenance.schema.json +3 -1
- package/schemas/replay/replay-result.schema.json +16 -2
- package/schemas/replay/replay-trace.schema.json +11 -2
- package/schemas/security/context-segment.schema.json +21 -12
- package/schemas/security/redaction-policy.schema.json +4 -1
- package/schemas/security-packs/coverage-report.schema.json +17 -2
- package/schemas/security-packs/security-case-suite.schema.json +11 -1
- package/schemas/tool-result-envelope.schema.json +9 -1
- package/security-packs/approval_bypass.yaml +12 -2
- package/security-packs/capability_spoofing.yaml +17 -3
- package/security-packs/cost_amplification.yaml +21 -4
- package/security-packs/cross_tenant_access.yaml +16 -3
- package/security-packs/insecure_output_rendering.yaml +15 -4
- package/security-packs/mcp_tool_abuse.yaml +17 -3
- package/security-packs/memory_scope_violation.yaml +19 -4
- package/security-packs/prompt_injection_direct.yaml +19 -4
- package/security-packs/prompt_injection_indirect.yaml +18 -3
- package/security-packs/provider_payload_exposure.yaml +26 -4
- package/security-packs/retrieval_poisoning.yaml +5 -2
- package/security-packs/schema_confusion.yaml +17 -3
- package/security-packs/sensitive_data_disclosure.yaml +22 -5
- package/security-packs/tool_exfiltration.yaml +20 -4
- package/security-packs/tool_result_poisoning.yaml +19 -4
- package/security-packs/unsafe_commit_attempt.yaml +6 -3
package/docs/runtime.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Runtime Contracts
|
|
2
2
|
|
|
3
3
|
AICF Runtime is an additive TypeScript subpath for host applications that want
|
|
4
|
-
deterministic runtime context, capability routing, policy brokering, handler
|
|
5
|
-
|
|
4
|
+
deterministic runtime context, capability routing, policy brokering, handler execution,
|
|
5
|
+
action lifecycle control, and model-safe tool result envelopes.
|
|
6
6
|
|
|
7
7
|
Import runtime APIs from:
|
|
8
8
|
|
|
@@ -14,22 +14,20 @@ import {
|
|
|
14
14
|
} from "ai-capability-framework/runtime";
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
-
The runtime subpath does not call models, depend on provider SDKs, or expose
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
retention, and provider calls.
|
|
17
|
+
The runtime subpath does not call models, depend on provider SDKs, or expose commit
|
|
18
|
+
through model tool execution. It can execute host-provided read and prepare handlers,
|
|
19
|
+
and it can commit only through the lifecycle manager after a stored prepared action,
|
|
20
|
+
approval, and idempotency checks. Host applications still own real auth, payment/account
|
|
21
|
+
state, durable storage, approval workflows, audit retention, and provider calls.
|
|
23
22
|
|
|
24
|
-
Optional runtime controls live in `ai-capability-framework/controls`. A host can
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
23
|
+
Optional runtime controls live in `ai-capability-framework/controls`. A host can pass a
|
|
24
|
+
controls evaluator into routing, execution, lifecycle, or provider runtime calls to
|
|
25
|
+
apply kill switches, read-only mode, force approval, circuit breakers, and per-run
|
|
26
|
+
budgets. The reference stores are in-memory/local only.
|
|
28
27
|
|
|
29
|
-
The optional OpenAI Responses runtime lives in
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
caller-provided OpenAI-compatible clients.
|
|
28
|
+
The optional OpenAI Responses runtime lives in `ai-capability-framework/openai`, not
|
|
29
|
+
this runtime subpath. It uses these runtime contracts to run a bounded non-streaming
|
|
30
|
+
model/tool loop with caller-provided OpenAI-compatible clients.
|
|
33
31
|
|
|
34
32
|
For a concrete support/refund flow that shows routing, tool export, validation,
|
|
35
33
|
approval-required prepare, and host-controlled commit, read the
|
|
@@ -37,13 +35,13 @@ approval-required prepare, and host-controlled commit, read the
|
|
|
37
35
|
|
|
38
36
|
## Runtime Context
|
|
39
37
|
|
|
40
|
-
Runtime decisions use an `AicfRuntimeContext` with run and request IDs,
|
|
41
|
-
|
|
42
|
-
|
|
38
|
+
Runtime decisions use an `AicfRuntimeContext` with run and request IDs, environment,
|
|
39
|
+
subject identity, account and tenant identity, workflow hints, autonomy and risk limits,
|
|
40
|
+
host facts, and metadata.
|
|
43
41
|
|
|
44
|
-
Production contexts must include user, account, tenant, run, and request
|
|
45
|
-
|
|
46
|
-
|
|
42
|
+
Production contexts must include user, account, tenant, run, and request identifiers.
|
|
43
|
+
Missing required identity or request fields fail safely before policy or routing
|
|
44
|
+
decisions continue.
|
|
47
45
|
|
|
48
46
|
Use `AicfAuthPlatformAdapter` to connect AICF to a host auth/account platform.
|
|
49
47
|
`StaticAuthPlatformAdapter` is provided only for tests and examples.
|
|
@@ -54,109 +52,108 @@ Use `AicfAuthPlatformAdapter` to connect AICF to a host auth/account platform.
|
|
|
54
52
|
context items into deterministic `modelContextText`.
|
|
55
53
|
|
|
56
54
|
The builder labels user text as untrusted, includes approved application context
|
|
57
|
-
separately, applies `DefaultRedactionPolicy` before formatting, enforces max item
|
|
58
|
-
|
|
55
|
+
separately, applies `DefaultRedactionPolicy` before formatting, enforces max item and
|
|
56
|
+
character budgets, and rejects raw attachment contents.
|
|
59
57
|
|
|
60
|
-
Default redaction removes values under obviously sensitive keys such as
|
|
61
|
-
|
|
62
|
-
|
|
58
|
+
Default redaction removes values under obviously sensitive keys such as passwords,
|
|
59
|
+
tokens, secrets, API keys, cookies, sessions, card numbers, CVV values, and private
|
|
60
|
+
keys. It intentionally does not claim broad PII detection.
|
|
63
61
|
|
|
64
62
|
For provider-boundary and trace-boundary trust labels, taint metadata, data
|
|
65
63
|
classifications, and retention checks, use the optional
|
|
66
|
-
`ai-capability-framework/security` subpath. Runtime context items can be mapped
|
|
67
|
-
|
|
64
|
+
`ai-capability-framework/security` subpath. Runtime context items can be mapped to
|
|
65
|
+
security context segments without changing existing runtime behavior.
|
|
68
66
|
|
|
69
|
-
For host-owned memory or user preferences, use
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
67
|
+
For host-owned memory or user preferences, use `ai-capability-framework/memory` before
|
|
68
|
+
adding memory summaries to host context. It filters by use case, scope, consent, expiry,
|
|
69
|
+
and sensitivity, then converts allowed records to runtime context items. It does not
|
|
70
|
+
retrieve or store memory.
|
|
73
71
|
|
|
74
72
|
## Capability Router
|
|
75
73
|
|
|
76
|
-
`DefaultCapabilityRouter` builds a compact runtime capability slice without
|
|
77
|
-
|
|
74
|
+
`DefaultCapabilityRouter` builds a compact runtime capability slice without calling a
|
|
75
|
+
model or embedding service.
|
|
78
76
|
|
|
79
|
-
It filters deterministically by capability status, risk ceiling, permissions,
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
exposed by default.
|
|
77
|
+
It filters deterministically by capability status, risk ceiling, permissions, required
|
|
78
|
+
user/tenant context, explicit include/exclude lists, domain, capability type, and
|
|
79
|
+
restricted side effects. Commit, external-send, money-moving, permission-changing,
|
|
80
|
+
destructive, and workflow-triggering capabilities are not exposed by default.
|
|
84
81
|
|
|
85
|
-
The router scores remaining capabilities with lexical matching over public
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
operations.
|
|
82
|
+
The router scores remaining capabilities with lexical matching over public capability
|
|
83
|
+
metadata and workflow hints, then sorts by score, risk, and capability ID. Model-facing
|
|
84
|
+
slices expose only `select` and `prepare` operations.
|
|
89
85
|
|
|
90
|
-
When controls are supplied, routing excludes denied capabilities and capabilities
|
|
91
|
-
|
|
86
|
+
When controls are supplied, routing excludes denied capabilities and capabilities that
|
|
87
|
+
are incompatible with active read-only controls.
|
|
92
88
|
|
|
93
|
-
Use `formatCapabilitySliceForModel()` to render concise model-facing capability
|
|
94
|
-
|
|
95
|
-
schemas.
|
|
89
|
+
Use `formatCapabilitySliceForModel()` to render concise model-facing capability guidance
|
|
90
|
+
without private diagnostics, internal policy rules, or full output schemas.
|
|
96
91
|
|
|
97
92
|
## Policy Broker
|
|
98
93
|
|
|
99
|
-
`DefaultPolicyBroker` wraps the Core `decideCapability()` control plane and adds
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
94
|
+
`DefaultPolicyBroker` wraps the Core `decideCapability()` control plane and adds runtime
|
|
95
|
+
context checks. It fails closed for missing identity, missing permissions, risk/autonomy
|
|
96
|
+
violations, rejected or expired approvals, missing idempotency, and thrown host policy
|
|
97
|
+
hooks.
|
|
103
98
|
|
|
104
|
-
Host policy hooks may make a decision stricter by denying or requiring approval.
|
|
105
|
-
|
|
99
|
+
Host policy hooks may make a decision stricter by denying or requiring approval. They
|
|
100
|
+
cannot override an AICF denial into an allowed decision.
|
|
106
101
|
|
|
107
102
|
## Handler Registry And Executor
|
|
108
103
|
|
|
109
|
-
`AicfHandlerRegistry` stores host-provided capability handlers. Duplicate
|
|
110
|
-
|
|
111
|
-
|
|
104
|
+
`AicfHandlerRegistry` stores host-provided capability handlers. Duplicate handler IDs
|
|
105
|
+
fail during registration, and optional manifest-registry validation rejects handlers for
|
|
106
|
+
unknown capabilities.
|
|
112
107
|
|
|
113
108
|
`AicfToolExecutor` executes `read` and `prepare` requests only. It validates the
|
|
114
|
-
capability, lifecycle, input schema, policy, and handler presence before calling
|
|
115
|
-
|
|
116
|
-
|
|
109
|
+
capability, lifecycle, input schema, policy, and handler presence before calling a
|
|
110
|
+
handler. Handler output is validated against the capability `output_schema`. Missing
|
|
111
|
+
handlers return safe unavailable envelopes by default.
|
|
117
112
|
|
|
118
|
-
The executor rejects `commit` even if a malformed request attempts to pass it.
|
|
119
|
-
|
|
113
|
+
The executor rejects `commit` even if a malformed request attempts to pass it. Commit is
|
|
114
|
+
host-controlled through `AicfActionLifecycleManager`.
|
|
120
115
|
|
|
121
116
|
## Action Lifecycle And Stores
|
|
122
117
|
|
|
123
|
-
`AicfActionLifecycleManager` prepares actions, records approvals, and commits
|
|
124
|
-
|
|
125
|
-
|
|
118
|
+
`AicfActionLifecycleManager` prepares actions, records approvals, and commits stored
|
|
119
|
+
prepared actions. Commit uses stored prepared-action state plus an approval decision and
|
|
120
|
+
idempotency key; it does not use model-provided raw args. Optional verification checks
|
|
121
|
+
committed actions through a registered verify handler. Verification records
|
|
122
|
+
`verificationStatus` separately, so a failed verification does not rewrite a completed
|
|
123
|
+
side effect from `committed` to `failed`.
|
|
126
124
|
|
|
127
|
-
The package includes in-memory prepared-action, approval, idempotency, and audit
|
|
128
|
-
|
|
129
|
-
|
|
125
|
+
The package includes in-memory prepared-action, approval, idempotency, and audit stores
|
|
126
|
+
for tests, local demos, and reference implementations. They are not production-durable
|
|
127
|
+
stores.
|
|
130
128
|
|
|
131
|
-
Optional AWS reference stores are available from
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
129
|
+
Optional AWS reference stores are available from `ai-capability-framework/aws` for host
|
|
130
|
+
applications that want DynamoDB-backed prepared actions, approvals, idempotency
|
|
131
|
+
reservations, and audit summaries. They are adapters around the same runtime interfaces;
|
|
132
|
+
they do not provision AWS resources, own auth, or make AWS part of the runtime subpath
|
|
133
|
+
import.
|
|
136
134
|
|
|
137
|
-
Audit events are structured summaries for attempts, denials, approval-required
|
|
138
|
-
|
|
139
|
-
|
|
135
|
+
Audit events are structured summaries for attempts, denials, approval-required pauses,
|
|
136
|
+
successes, and failures. They must not contain raw traces, provider payloads, secrets,
|
|
137
|
+
or customer data.
|
|
140
138
|
|
|
141
|
-
For canonical evidence records, pass an optional
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
Without a configured ledger, runtime behavior is unchanged.
|
|
139
|
+
For canonical evidence records, pass an optional `ai-capability-framework/audit` ledger
|
|
140
|
+
to `AicfToolExecutor` and `AicfActionLifecycleManager`. The ledger writes schema-valid
|
|
141
|
+
policy decision, action, approval, and idempotency records with redacted refs and hashed
|
|
142
|
+
inputs. Without a configured ledger, runtime behavior is unchanged.
|
|
146
143
|
|
|
147
144
|
See [action lifecycle](action-lifecycle.md) for the prepare, approval, commit,
|
|
148
|
-
idempotency, and audit flow. See [policy broker](policy-broker.md) for runtime
|
|
149
|
-
|
|
145
|
+
idempotency, and audit flow. See [policy broker](policy-broker.md) for runtime policy
|
|
146
|
+
evaluation and host hook behavior.
|
|
150
147
|
|
|
151
148
|
## Runtime Envelopes
|
|
152
149
|
|
|
153
|
-
Runtime tool results use `createToolEnvelope()`,
|
|
154
|
-
`
|
|
150
|
+
Runtime tool results use `createToolEnvelope()`, `toModelSafeToolEnvelope()`, and
|
|
151
|
+
`serializeToolEnvelopeForModel()`.
|
|
155
152
|
|
|
156
|
-
Runtime envelopes include run and request IDs, capability ID/version, operation,
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
153
|
+
Runtime envelopes include run and request IDs, capability ID/version, operation, status,
|
|
154
|
+
optional policy/action summaries, safe user messages, and optional diagnostics.
|
|
155
|
+
Diagnostics are stripped from model-facing output by default and are never included for
|
|
156
|
+
production model output through the default helper.
|
|
160
157
|
|
|
161
158
|
## Public Example
|
|
162
159
|
|
|
@@ -167,6 +164,6 @@ npm run build
|
|
|
167
164
|
node examples/runtime-support-billing/run-mock.mjs
|
|
168
165
|
```
|
|
169
166
|
|
|
170
|
-
It loads the support manifests, routes read and prepare capabilities, executes a
|
|
171
|
-
|
|
172
|
-
|
|
167
|
+
It loads the support manifests, routes read and prepare capabilities, executes a ticket
|
|
168
|
+
read, prepares a refund case that requires approval, records a host approval, and
|
|
169
|
+
commits only through `AicfActionLifecycleManager`.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Security Overview
|
|
2
2
|
|
|
3
|
-
AICF security features help keep model-accessible capabilities constrained,
|
|
4
|
-
|
|
3
|
+
AICF security features help keep model-accessible capabilities constrained, validated,
|
|
4
|
+
and reviewable.
|
|
5
5
|
|
|
6
6
|
Key guides:
|
|
7
7
|
|
|
@@ -10,5 +10,5 @@ Key guides:
|
|
|
10
10
|
- [Governed memory](../memory.md)
|
|
11
11
|
- [Content provenance hooks](../provenance.md)
|
|
12
12
|
|
|
13
|
-
AICF does not replace host DLP, auth, privacy review, incident response, or
|
|
14
|
-
|
|
13
|
+
AICF does not replace host DLP, auth, privacy review, incident response, or security
|
|
14
|
+
operations.
|
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
# Capability-Aware Security Packs
|
|
2
2
|
|
|
3
|
-
Security packs are public-safe test templates for common AI capability risks.
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
Security packs are public-safe test templates for common AI capability risks. They help
|
|
4
|
+
teams generate deterministic eval cases and Promptfoo red-team configs from the
|
|
5
|
+
capabilities already described in AICF manifests.
|
|
6
6
|
|
|
7
|
-
Security packs are implementation aids only. They are not certification,
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
7
|
+
Security packs are implementation aids only. They are not certification, compliance
|
|
8
|
+
attestation, legal advice, an audit opinion, or a security guarantee. Host applications
|
|
9
|
+
own real red-team execution, evidence retention, policy review, incident response, and
|
|
10
|
+
production controls.
|
|
11
11
|
|
|
12
12
|
## What They Cover
|
|
13
13
|
|
|
@@ -29,8 +29,8 @@ The built-in catalog includes packs for:
|
|
|
29
29
|
- retrieval poisoning
|
|
30
30
|
- memory scope violation
|
|
31
31
|
|
|
32
|
-
The catalog lives in public YAML files under `security-packs/`. The TypeScript
|
|
33
|
-
|
|
32
|
+
The catalog lives in public YAML files under `security-packs/`. The TypeScript API
|
|
33
|
+
exposes the same catalog from:
|
|
34
34
|
|
|
35
35
|
```ts
|
|
36
36
|
import {
|
|
@@ -71,15 +71,14 @@ Waivers must include `pack_id`, `reason`, `reviewer`, and `reviewed_at`.
|
|
|
71
71
|
|
|
72
72
|
`assessSecurityPackCoverage()` recommends packs from manifest metadata:
|
|
73
73
|
|
|
74
|
-
- high and critical risk capabilities require assigned/generated packs or a
|
|
75
|
-
|
|
76
|
-
-
|
|
77
|
-
|
|
78
|
-
provider-exposed capabilities receive additional recommendations;
|
|
74
|
+
- high and critical risk capabilities require assigned/generated packs or a waiver;
|
|
75
|
+
- money movement, commit, irreversible, external-message, workflow, permission-changing,
|
|
76
|
+
tenant-scoped, retrieval, memory, MCP, and provider-exposed capabilities receive
|
|
77
|
+
additional recommendations;
|
|
79
78
|
- missing required packs are reported as coverage failures.
|
|
80
79
|
|
|
81
|
-
The coverage report is deterministic and local. It does not inspect raw traces,
|
|
82
|
-
|
|
80
|
+
The coverage report is deterministic and local. It does not inspect raw traces, call
|
|
81
|
+
providers, or infer production security posture.
|
|
83
82
|
|
|
84
83
|
## Generate Cases
|
|
85
84
|
|
|
@@ -91,9 +90,9 @@ aicf security generate examples \
|
|
|
91
90
|
--out ./security-cases.approval-bypass.yaml
|
|
92
91
|
```
|
|
93
92
|
|
|
94
|
-
The output is a synthetic case suite. Case text uses public capability IDs and
|
|
95
|
-
|
|
96
|
-
|
|
93
|
+
The output is a synthetic case suite. Case text uses public capability IDs and generic
|
|
94
|
+
prompts only; it does not include raw prompts, provider payloads, credentials, customer
|
|
95
|
+
records, or private diagnostics.
|
|
97
96
|
|
|
98
97
|
TypeScript:
|
|
99
98
|
|
|
@@ -103,8 +102,8 @@ const suite = generateSecurityCases(registry, {
|
|
|
103
102
|
});
|
|
104
103
|
```
|
|
105
104
|
|
|
106
|
-
Generated cases can be reviewed and converted into ordinary AICF eval manifests
|
|
107
|
-
|
|
105
|
+
Generated cases can be reviewed and converted into ordinary AICF eval manifests or used
|
|
106
|
+
as host-owned test inputs.
|
|
108
107
|
|
|
109
108
|
## Promptfoo Export
|
|
110
109
|
|
|
@@ -117,8 +116,8 @@ aicf security export-promptfoo examples \
|
|
|
117
116
|
```
|
|
118
117
|
|
|
119
118
|
The generated YAML defaults to the API-key-free `echo` provider and includes a
|
|
120
|
-
placeholder target endpoint. Replace the provider and target in a host-owned
|
|
121
|
-
|
|
119
|
+
placeholder target endpoint. Replace the provider and target in a host-owned test
|
|
120
|
+
environment before running Promptfoo.
|
|
122
121
|
|
|
123
122
|
The export includes assertions for:
|
|
124
123
|
|
|
@@ -140,6 +139,5 @@ aicf security generate examples --pack approval_bypass --out ./cases.json --form
|
|
|
140
139
|
aicf security export-promptfoo examples --out ./promptfooconfig.aicf-security.yaml
|
|
141
140
|
```
|
|
142
141
|
|
|
143
|
-
Commands validate manifests first. Unknown pack IDs, invalid manifests, missing
|
|
144
|
-
|
|
145
|
-
|
|
142
|
+
Commands validate manifests first. Unknown pack IDs, invalid manifests, missing output
|
|
143
|
+
paths, and empty generated suites exit nonzero.
|
|
@@ -11,52 +11,48 @@ import {
|
|
|
11
11
|
} from "ai-capability-framework/security";
|
|
12
12
|
```
|
|
13
13
|
|
|
14
|
-
It does not replace host DLP, privacy review, auth, policy engines,
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
retention defaults.
|
|
14
|
+
It does not replace host DLP, privacy review, auth, policy engines, observability
|
|
15
|
+
systems, or data retention infrastructure. It gives AICF users a small public contract
|
|
16
|
+
for labelling context, keeping untrusted data separate from instructions, redacting
|
|
17
|
+
sensitive classes, and evaluating conservative retention defaults.
|
|
19
18
|
|
|
20
19
|
## Trust Segments
|
|
21
20
|
|
|
22
|
-
`ContextSegment` records what a piece of context is, where it came from, and
|
|
23
|
-
|
|
21
|
+
`ContextSegment` records what a piece of context is, where it came from, and whether it
|
|
22
|
+
is allowed to carry instructions.
|
|
24
23
|
|
|
25
|
-
Only `system_instruction`, `developer_instruction`, and `app_policy` segments
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
trusted instructions.
|
|
24
|
+
Only `system_instruction`, `developer_instruction`, and `app_policy` segments may set
|
|
25
|
+
`instructionsAllowed: true`. User input, uploaded files, retrieved documents, tool
|
|
26
|
+
results, external API results, and model output are data segments. They can inform the
|
|
27
|
+
model or host application, but they are not trusted instructions.
|
|
30
28
|
|
|
31
29
|
## Taint And Provenance
|
|
32
30
|
|
|
33
|
-
`SourceRef` records public-safe provenance such as source ID, source type,
|
|
34
|
-
|
|
31
|
+
`SourceRef` records public-safe provenance such as source ID, source type, trust label,
|
|
32
|
+
optional URI, content hash, freshness, and retrieval time.
|
|
35
33
|
|
|
36
|
-
Generated customer-facing content can use
|
|
37
|
-
`
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
content metadata.
|
|
34
|
+
Generated customer-facing content can use `ai-capability-framework/provenance` to create
|
|
35
|
+
sidecar metadata that references these `SourceRef` values. The provenance subpath keeps
|
|
36
|
+
raw prompts, provider payloads, transcripts, secrets, and sensitive source content out
|
|
37
|
+
of generated content metadata.
|
|
41
38
|
|
|
42
|
-
`markTainted()` records that content is not trusted for tool input until the
|
|
43
|
-
|
|
44
|
-
|
|
39
|
+
`markTainted()` records that content is not trusted for tool input until the host
|
|
40
|
+
validates it. This is especially important for model output and tool results that might
|
|
41
|
+
contain hostile text such as instructions to ignore policy.
|
|
45
42
|
|
|
46
43
|
## Redaction
|
|
47
44
|
|
|
48
|
-
`redactForProvider()` applies policy before content crosses a provider
|
|
49
|
-
|
|
50
|
-
observability sinks.
|
|
45
|
+
`redactForProvider()` applies policy before content crosses a provider boundary.
|
|
46
|
+
`redactForTrace()` applies policy before content enters traces or observability sinks.
|
|
51
47
|
|
|
52
48
|
Default behavior is conservative:
|
|
53
49
|
|
|
54
50
|
- `credential_material` is denied.
|
|
55
|
-
- PII, payment, financial, health, legal, and security-sensitive content is
|
|
56
|
-
|
|
51
|
+
- PII, payment, financial, health, legal, and security-sensitive content is redacted
|
|
52
|
+
from traces.
|
|
57
53
|
- raw prompts, raw provider payloads, and raw traces are denied by default.
|
|
58
|
-
- provider-boundary rules can be scoped by provider ID, capability ID,
|
|
59
|
-
|
|
54
|
+
- provider-boundary rules can be scoped by provider ID, capability ID, operation, trust
|
|
55
|
+
label, data classification, and path.
|
|
60
56
|
|
|
61
57
|
## Retention
|
|
62
58
|
|
|
@@ -69,32 +65,31 @@ Default behavior is conservative:
|
|
|
69
65
|
- eval dataset retention: `365` days
|
|
70
66
|
- raw content in eval datasets: `false`
|
|
71
67
|
|
|
72
|
-
`evaluateRetentionPolicy()` returns allow/deny reasons and warnings. Unsafe
|
|
73
|
-
|
|
74
|
-
|
|
68
|
+
`evaluateRetentionPolicy()` returns allow/deny reasons and warnings. Unsafe diagnostic
|
|
69
|
+
raw-content modes must be explicit and should stay out of public examples and release
|
|
70
|
+
artifacts.
|
|
75
71
|
|
|
76
72
|
## Runtime And Observability
|
|
77
73
|
|
|
78
|
-
Runtime `AicfContextItem` values can be converted into security
|
|
79
|
-
|
|
74
|
+
Runtime `AicfContextItem` values can be converted into security `ContextSegment` values
|
|
75
|
+
with `contextItemToSegment()`.
|
|
80
76
|
|
|
81
77
|
Host-owned memory and preferences can be governed through
|
|
82
|
-
`ai-capability-framework/memory` before they become runtime context. Selected
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
instructions.
|
|
78
|
+
`ai-capability-framework/memory` before they become runtime context. Selected memory
|
|
79
|
+
records convert into `ContextSegment` values with `instructionsAllowed: false`, so
|
|
80
|
+
memory summaries cannot become hidden instructions.
|
|
86
81
|
|
|
87
|
-
Observability trace sanitization continues to support `contentCapture` modes
|
|
88
|
-
`
|
|
89
|
-
|
|
90
|
-
|
|
82
|
+
Observability trace sanitization continues to support `contentCapture` modes `none`,
|
|
83
|
+
`metadata`, and `redacted_content`. When trace attributes include classification
|
|
84
|
+
metadata, the security redaction defaults are applied before the event is collected or
|
|
85
|
+
forwarded.
|
|
91
86
|
|
|
92
87
|
## Security Packs
|
|
93
88
|
|
|
94
|
-
Capability-aware security packs live in
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
89
|
+
Capability-aware security packs live in `ai-capability-framework/security-packs`. They
|
|
90
|
+
generate public-safe test cases and Promptfoo red-team configs for risks such as prompt
|
|
91
|
+
injection, approval bypass, unsafe commit attempts, provider payload exposure, and
|
|
92
|
+
cross-tenant access. They do not replace host security review or runtime redaction
|
|
93
|
+
policy.
|
|
99
94
|
|
|
100
95
|
See [Capability-aware security packs](security-packs.md).
|
|
@@ -1,20 +1,19 @@
|
|
|
1
1
|
# Semantic Kernel Compatibility
|
|
2
2
|
|
|
3
|
-
AICF supports Semantic Kernel through compatibility exports, not through a
|
|
4
|
-
|
|
3
|
+
AICF supports Semantic Kernel through compatibility exports, not through a native
|
|
4
|
+
Semantic Kernel runtime package.
|
|
5
5
|
|
|
6
6
|
The recommended path is MCP: expose a routed AICF capability slice through
|
|
7
|
-
`ai-capability-framework/mcp-server`, then import that MCP server as a Semantic
|
|
8
|
-
|
|
7
|
+
`ai-capability-framework/mcp-server`, then import that MCP server as a Semantic Kernel
|
|
8
|
+
plugin when the host runtime supports MCP plugins.
|
|
9
9
|
|
|
10
|
-
The fallback path is OpenAPI: generate an OpenAPI 3.1 descriptor for a
|
|
11
|
-
|
|
12
|
-
OpenAPI plugin.
|
|
10
|
+
The fallback path is OpenAPI: generate an OpenAPI 3.1 descriptor for a host-owned HTTP
|
|
11
|
+
executor route, then import that descriptor as a Semantic Kernel OpenAPI plugin.
|
|
13
12
|
|
|
14
13
|
## Import
|
|
15
14
|
|
|
16
|
-
Install AICF in hosts that generate Semantic Kernel compatibility metadata. No
|
|
17
|
-
|
|
15
|
+
Install AICF in hosts that generate Semantic Kernel compatibility metadata. No Semantic
|
|
16
|
+
Kernel npm dependency is required by AICF.
|
|
18
17
|
|
|
19
18
|
```bash
|
|
20
19
|
npm install ai-capability-framework
|
|
@@ -28,27 +27,27 @@ import {
|
|
|
28
27
|
} from "ai-capability-framework/providers/semantic-kernel";
|
|
29
28
|
```
|
|
30
29
|
|
|
31
|
-
This subpath has no Semantic Kernel dependency and performs no model calls,
|
|
32
|
-
|
|
30
|
+
This subpath has no Semantic Kernel dependency and performs no model calls, handler
|
|
31
|
+
execution, storage, approval runtime, or side effects.
|
|
33
32
|
|
|
34
|
-
Minimal usage is either: expose a routed AICF MCP server and import it as a
|
|
35
|
-
|
|
36
|
-
|
|
33
|
+
Minimal usage is either: expose a routed AICF MCP server and import it as a Semantic
|
|
34
|
+
Kernel MCP plugin, or generate OpenAPI metadata for a host-owned executor route and
|
|
35
|
+
import that descriptor as an OpenAPI plugin.
|
|
37
36
|
|
|
38
37
|
## MCP Path
|
|
39
38
|
|
|
40
|
-
Use `AicfMcpServer` to list and call routed read/prepare capabilities through
|
|
41
|
-
|
|
42
|
-
|
|
39
|
+
Use `AicfMcpServer` to list and call routed read/prepare capabilities through the AICF
|
|
40
|
+
runtime. Semantic Kernel imports the host MCP server; AICF remains the policy,
|
|
41
|
+
validation, lifecycle, approval, idempotency, and envelope authority.
|
|
43
42
|
|
|
44
|
-
Do not expose commit capabilities through MCP. Commit remains host-controlled
|
|
45
|
-
|
|
46
|
-
|
|
43
|
+
Do not expose commit capabilities through MCP. Commit remains host-controlled through
|
|
44
|
+
`AicfActionLifecycleManager` after a stored prepared action, approval, and idempotency
|
|
45
|
+
check.
|
|
47
46
|
|
|
48
47
|
## OpenAPI Export
|
|
49
48
|
|
|
50
|
-
`exportSemanticKernelOpenApiPlugin()` returns a descriptor-only OpenAPI 3.1
|
|
51
|
-
|
|
49
|
+
`exportSemanticKernelOpenApiPlugin()` returns a descriptor-only OpenAPI 3.1 document. It
|
|
50
|
+
creates one operation per exported read/prepare capability:
|
|
52
51
|
|
|
53
52
|
```text
|
|
54
53
|
POST /aicf/capabilities/{providerToolName}/execute
|
|
@@ -57,33 +56,32 @@ POST /aicf/capabilities/{providerToolName}/execute
|
|
|
57
56
|
Each request body contains:
|
|
58
57
|
|
|
59
58
|
- `args`: the capability input object;
|
|
60
|
-
- `runtime_context_ref`: an opaque host reference used to resolve authenticated
|
|
61
|
-
|
|
59
|
+
- `runtime_context_ref`: an opaque host reference used to resolve authenticated runtime
|
|
60
|
+
context server-side.
|
|
62
61
|
|
|
63
|
-
The generated document intentionally does not expose auth, permission, tenant,
|
|
64
|
-
|
|
65
|
-
|
|
62
|
+
The generated document intentionally does not expose auth, permission, tenant, account,
|
|
63
|
+
approval, idempotency, or audit internals. The host executor route must resolve those
|
|
64
|
+
values and enforce them before invoking AICF runtime objects.
|
|
66
65
|
|
|
67
66
|
## Plugin Metadata
|
|
68
67
|
|
|
69
|
-
`exportSemanticKernelPluginMetadata()` returns a small metadata object with the
|
|
70
|
-
|
|
71
|
-
|
|
68
|
+
`exportSemanticKernelPluginMetadata()` returns a small metadata object with the plugin
|
|
69
|
+
name, function summaries, MCP recommendation, OpenAPI import hints, and warnings for
|
|
70
|
+
automatic invocation.
|
|
72
71
|
|
|
73
72
|
## Boundary
|
|
74
73
|
|
|
75
|
-
- AICF does not import `semantic-kernel` or an unofficial TypeScript Semantic
|
|
76
|
-
|
|
74
|
+
- AICF does not import `semantic-kernel` or an unofficial TypeScript Semantic Kernel
|
|
75
|
+
runtime.
|
|
77
76
|
- AICF does not start an HTTP server for the generated OpenAPI descriptor.
|
|
78
77
|
- AICF does not call models or provider SDKs in this subpath.
|
|
79
78
|
- AICF does not expose commit capabilities to Semantic Kernel.
|
|
80
|
-
- Host applications own auth, account and tenant authority, approvals,
|
|
81
|
-
|
|
79
|
+
- Host applications own auth, account and tenant authority, approvals, idempotency, side
|
|
80
|
+
effects, and audit persistence.
|
|
82
81
|
- Treat approval-required envelopes as pauses, not completed actions.
|
|
83
82
|
|
|
84
|
-
Known limitations: no native Semantic Kernel runtime, no hosted HTTP executor,
|
|
85
|
-
|
|
86
|
-
path.
|
|
83
|
+
Known limitations: no native Semantic Kernel runtime, no hosted HTTP executor, no model
|
|
84
|
+
calls, no production auth, no approval UI, and no model-exposed commit path.
|
|
87
85
|
|
|
88
86
|
## Checks
|
|
89
87
|
|