ai-capability-framework 1.0.0-rc.3 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (217) hide show
  1. package/CHANGELOG.md +97 -103
  2. package/CODE_OF_CONDUCT.md +11 -13
  3. package/CONTRIBUTING.md +40 -43
  4. package/GOVERNANCE.md +34 -35
  5. package/README.md +99 -63
  6. package/ROADMAP.md +12 -12
  7. package/SECURITY.md +15 -15
  8. package/dist/audit/ledger.d.ts.map +1 -1
  9. package/dist/audit/ledger.js +2 -1
  10. package/dist/audit/records.d.ts.map +1 -1
  11. package/dist/audit/records.js +2 -1
  12. package/dist/audit/types.d.ts +4 -1
  13. package/dist/audit/types.d.ts.map +1 -1
  14. package/dist/aws/dynamodb-stores.d.ts +2 -7
  15. package/dist/aws/dynamodb-stores.d.ts.map +1 -1
  16. package/dist/aws/dynamodb-stores.js +56 -5
  17. package/dist/aws/testing.d.ts.map +1 -1
  18. package/dist/aws/testing.js +9 -0
  19. package/dist/cli.d.ts +6 -6
  20. package/dist/cli.d.ts.map +1 -1
  21. package/dist/cli.js +77 -7
  22. package/dist/evals-live/index.d.ts +2 -2
  23. package/dist/evals-live/index.d.ts.map +1 -1
  24. package/dist/evals-live/index.js +1 -1
  25. package/dist/evals-live/live-eval-runner.d.ts +49 -1
  26. package/dist/evals-live/live-eval-runner.d.ts.map +1 -1
  27. package/dist/evals-live/live-eval-runner.js +246 -4
  28. package/dist/evals-live/types.d.ts +62 -9
  29. package/dist/evals-live/types.d.ts.map +1 -1
  30. package/dist/index.d.ts +0 -1
  31. package/dist/index.d.ts.map +1 -1
  32. package/dist/index.js +0 -1
  33. package/dist/runtime/actions.d.ts +2 -0
  34. package/dist/runtime/actions.d.ts.map +1 -1
  35. package/dist/runtime/actions.js +200 -55
  36. package/dist/runtime/index.d.ts +1 -1
  37. package/dist/runtime/index.d.ts.map +1 -1
  38. package/dist/runtime/stores.d.ts +2 -7
  39. package/dist/runtime/stores.d.ts.map +1 -1
  40. package/dist/runtime/stores.js +12 -0
  41. package/dist/runtime/types.d.ts +25 -6
  42. package/dist/runtime/types.d.ts.map +1 -1
  43. package/docs/action-lifecycle.md +38 -32
  44. package/docs/adapter-roadmap.md +51 -49
  45. package/docs/adapters.md +14 -15
  46. package/docs/ai-sdk-runtime.md +36 -39
  47. package/docs/anthropic-runtime.md +29 -32
  48. package/docs/api/public-api-policy.md +59 -0
  49. package/docs/api.md +354 -371
  50. package/docs/architecture/current-state.md +28 -31
  51. package/docs/audit/index.md +29 -29
  52. package/docs/aws/cloudwatch-telemetry.md +5 -7
  53. package/docs/aws/dynamodb-single-table.md +8 -9
  54. package/docs/aws/kms-redaction.md +5 -6
  55. package/docs/aws/overview.md +5 -5
  56. package/docs/aws/production-reference.md +13 -14
  57. package/docs/aws/step-functions-approval.md +7 -9
  58. package/docs/aws-runtime.md +24 -25
  59. package/docs/capability-manifests.md +17 -21
  60. package/docs/control-plane/overview.md +2 -2
  61. package/docs/control-plane.md +24 -27
  62. package/docs/controls/index.md +19 -22
  63. package/docs/core/capability-manifests.md +4 -5
  64. package/docs/core/entity-manifests.md +5 -5
  65. package/docs/eval-runner.md +29 -31
  66. package/docs/evalops.md +13 -14
  67. package/docs/evals/golden-tests.md +4 -5
  68. package/docs/evals/overview.md +3 -3
  69. package/docs/evals/replay-and-trace-to-golden.md +19 -20
  70. package/docs/evals.md +19 -20
  71. package/docs/evidence.md +22 -23
  72. package/docs/gemini-runtime.md +31 -33
  73. package/docs/getting-started/anthropic-quickstart.md +61 -0
  74. package/docs/getting-started/concepts.md +9 -10
  75. package/docs/getting-started/gemini-quickstart.md +62 -0
  76. package/docs/getting-started/installation.md +3 -3
  77. package/docs/getting-started/openai-quickstart.md +61 -0
  78. package/docs/getting-started/provider-neutral-quickstart.md +106 -0
  79. package/docs/getting-started/quickstart.md +2 -2
  80. package/docs/getting-started.md +23 -20
  81. package/docs/glossary.md +31 -48
  82. package/docs/governance/gate.md +28 -30
  83. package/docs/governance/impact-and-compatibility.md +2 -3
  84. package/docs/governance/index.md +53 -54
  85. package/docs/governance/lifecycle.md +4 -5
  86. package/docs/governance/overview.md +2 -3
  87. package/docs/governance/risk-compiler.md +4 -5
  88. package/docs/host-responsibilities.md +6 -8
  89. package/docs/implementation-notes/baseline-gaps.md +15 -18
  90. package/docs/index.md +21 -6
  91. package/docs/interoperability.md +9 -9
  92. package/docs/langchain-runtime.md +27 -31
  93. package/docs/live-evals.md +68 -32
  94. package/docs/mcp-server-runtime.md +36 -40
  95. package/docs/memory.md +35 -36
  96. package/docs/migration-0.1-to-1.0.md +14 -9
  97. package/docs/observability/overview.md +2 -3
  98. package/docs/observability-runtime.md +35 -39
  99. package/docs/openai-responses.md +36 -37
  100. package/docs/openai-runtime.md +41 -46
  101. package/docs/openai-walkthrough.md +24 -28
  102. package/docs/policy-broker.md +9 -10
  103. package/docs/provenance.md +30 -29
  104. package/docs/providers/anthropic.md +5 -5
  105. package/docs/providers/choose-a-runtime.md +37 -0
  106. package/docs/providers/conformance.md +14 -15
  107. package/docs/providers/gemini.md +5 -5
  108. package/docs/providers/mcp.md +2 -3
  109. package/docs/providers/openai.md +2 -2
  110. package/docs/providers/semantic-kernel.md +2 -2
  111. package/docs/providers.md +43 -38
  112. package/docs/public/license-exceptions.md +187 -0
  113. package/docs/public/npm-release-preflight.md +112 -0
  114. package/docs/public-framework/compatibility-policy.md +12 -13
  115. package/docs/public-framework/deprecation-policy.md +8 -8
  116. package/docs/public-framework/final-certification-matrix.md +104 -0
  117. package/docs/public-framework/license-decision.md +20 -0
  118. package/docs/public-framework/release-process.md +46 -16
  119. package/docs/public-framework/security-disclosure.md +4 -5
  120. package/docs/public-framework/v1-certification.md +102 -15
  121. package/docs/release.md +200 -122
  122. package/docs/runtime/action-lifecycle.md +3 -3
  123. package/docs/runtime/policy-broker.md +4 -4
  124. package/docs/runtime/runtime-overview.md +4 -5
  125. package/docs/runtime/tool-result-envelope.md +7 -7
  126. package/docs/runtime.md +90 -93
  127. package/docs/security/overview.md +4 -4
  128. package/docs/security/security-packs.md +24 -26
  129. package/docs/security/trust-taint-redaction.md +42 -47
  130. package/docs/semantic-kernel-runtime.md +35 -37
  131. package/docs/spec.md +95 -101
  132. package/docs/start-here.md +18 -20
  133. package/examples/01-basic-read-capability/README.md +3 -4
  134. package/examples/02-prepare-approve-commit/README.md +6 -6
  135. package/examples/03-multi-provider-tools/README.md +2 -2
  136. package/examples/04-mcp-server/README.md +4 -5
  137. package/examples/05-langchain-langgraph-bridge/README.md +4 -5
  138. package/examples/06-vercel-ai-sdk-bridge/README.md +5 -7
  139. package/examples/07-policy-broker-custom-auth/README.md +4 -4
  140. package/examples/08-aws-step-functions-approval/README.md +4 -5
  141. package/examples/09-security-packs-promptfoo/README.md +4 -4
  142. package/examples/10-trace-to-golden/README.md +2 -2
  143. package/examples/11-control-plane/README.md +4 -5
  144. package/examples/aws/README.md +5 -8
  145. package/examples/control-plane/README.md +8 -9
  146. package/examples/control-plane/fixtures/control-plane.seed.json +8 -32
  147. package/examples/control-plane/public/app.js +57 -18
  148. package/examples/control-plane/public/index.html +15 -6
  149. package/examples/control-plane/server.mjs +15 -3
  150. package/examples/eval-results/public.results.passing.json +6 -21
  151. package/examples/providers/ai-sdk-next/README.md +11 -8
  152. package/examples/providers/anthropic-claude/README.md +5 -6
  153. package/examples/providers/gemini/README.md +6 -6
  154. package/examples/providers/langchain-agent/README.md +3 -4
  155. package/examples/providers/langgraph-tool-node/README.md +7 -7
  156. package/examples/providers/mcp/README.md +8 -8
  157. package/examples/providers/provider-conformance/README.md +3 -3
  158. package/examples/providers/semantic-kernel-mcp/README.md +6 -6
  159. package/examples/providers/semantic-kernel-openapi/README.md +6 -7
  160. package/examples/runtime-support-billing/README.md +8 -9
  161. package/examples/runtime-support-billing/support-billing-runtime.mjs +8 -4
  162. package/examples/scheduling/capabilities/scheduling.availability.get.yaml +2 -2
  163. package/examples/scheduling/capabilities/scheduling.invite.prepare.yaml +2 -2
  164. package/examples/scheduling/capabilities/scheduling.invite.send.yaml +2 -2
  165. package/examples/scheduling/entities/AvailabilitySlot.yaml +2 -2
  166. package/examples/scheduling/entities/MeetingInvite.yaml +2 -2
  167. package/examples/scheduling/evals/scheduling.invite.prepare.valid.yaml +3 -1
  168. package/examples/semantic-kernel-mcp/README.md +11 -12
  169. package/examples/semantic-kernel-openapi/README.md +8 -8
  170. package/examples/support/capabilities/support.refund.commit_case.yaml +3 -3
  171. package/examples/support/capabilities/support.refund.prepare_case.yaml +2 -2
  172. package/examples/support/capabilities/support.ticket.get.yaml +2 -2
  173. package/examples/support/entities/Ticket.yaml +0 -1
  174. package/examples/support/eval-results/support.results.passing.json +3 -12
  175. package/examples/support/evals/support.refund.prepare_case.approval_required.yaml +3 -1
  176. package/examples/support/evals/support.refund.prepare_case.valid.yaml +3 -2
  177. package/examples/support/memory/support.agent.preferences.json +2 -6
  178. package/examples/support/openai/context.support_agent.json +1 -4
  179. package/examples/support/provenance/support.refund.summary.provenance.json +2 -6
  180. package/examples/support/replay/support.refund.approval_required.trace.json +4 -14
  181. package/package.json +22 -5
  182. package/schemas/audit/action-record.schema.json +4 -1
  183. package/schemas/aws/approval-task.schema.json +8 -1
  184. package/schemas/conformance/conformance-case.schema.json +18 -2
  185. package/schemas/conformance/conformance-report.schema.json +17 -2
  186. package/schemas/conformance/provider-target-matrix.schema.json +35 -4
  187. package/schemas/controls/control-decision.schema.json +7 -1
  188. package/schemas/eval-case.schema.json +8 -1
  189. package/schemas/eval-result.schema.json +8 -1
  190. package/schemas/evidence/evidence-pack.schema.json +34 -5
  191. package/schemas/governance/compatibility-diff.schema.json +8 -1
  192. package/schemas/governance/lifecycle-transition.schema.json +10 -1
  193. package/schemas/governance/risk-compilation-result.schema.json +9 -1
  194. package/schemas/provenance/generated-content-provenance.schema.json +3 -1
  195. package/schemas/replay/replay-result.schema.json +16 -2
  196. package/schemas/replay/replay-trace.schema.json +11 -2
  197. package/schemas/security/context-segment.schema.json +21 -12
  198. package/schemas/security/redaction-policy.schema.json +4 -1
  199. package/schemas/security-packs/coverage-report.schema.json +17 -2
  200. package/schemas/security-packs/security-case-suite.schema.json +11 -1
  201. package/schemas/tool-result-envelope.schema.json +9 -1
  202. package/security-packs/approval_bypass.yaml +12 -2
  203. package/security-packs/capability_spoofing.yaml +17 -3
  204. package/security-packs/cost_amplification.yaml +21 -4
  205. package/security-packs/cross_tenant_access.yaml +16 -3
  206. package/security-packs/insecure_output_rendering.yaml +15 -4
  207. package/security-packs/mcp_tool_abuse.yaml +17 -3
  208. package/security-packs/memory_scope_violation.yaml +19 -4
  209. package/security-packs/prompt_injection_direct.yaml +19 -4
  210. package/security-packs/prompt_injection_indirect.yaml +18 -3
  211. package/security-packs/provider_payload_exposure.yaml +26 -4
  212. package/security-packs/retrieval_poisoning.yaml +5 -2
  213. package/security-packs/schema_confusion.yaml +17 -3
  214. package/security-packs/sensitive_data_disclosure.yaml +22 -5
  215. package/security-packs/tool_exfiltration.yaml +20 -4
  216. package/security-packs/tool_result_poisoning.yaml +19 -4
  217. package/security-packs/unsafe_commit_attempt.yaml +6 -3
@@ -1,34 +1,33 @@
1
1
  # AICF Governance
2
2
 
3
- AICF governance is a pure contract and analysis layer. It helps maintainers
4
- review capability lifecycle changes, infer minimum risk, compare versions, and
5
- understand what else a capability touches before it is exposed to AI systems.
3
+ AICF governance is a pure contract and analysis layer. It helps maintainers review
4
+ capability lifecycle changes, infer minimum risk, compare versions, and understand what
5
+ else a capability touches before it is exposed to AI systems.
6
6
 
7
- Governance APIs do not call models, execute handlers, mutate manifests, write
8
- stores, or replace your policy engine. They return structured decisions that a
9
- host app, CI job, or release process can act on.
7
+ Governance APIs do not call models, execute handlers, mutate manifests, write stores, or
8
+ replace your policy engine. They return structured decisions that a host app, CI job, or
9
+ release process can act on.
10
10
 
11
- Runtime controls are separate from governance analysis. Governance can report
12
- active kill switches or circuit breakers as promotion blockers; the optional
11
+ Runtime controls are separate from governance analysis. Governance can report active
12
+ kill switches or circuit breakers as promotion blockers; the optional
13
13
  `ai-capability-framework/controls` subpath evaluates those controls at runtime.
14
14
 
15
- Governed memory is also separate. The optional
16
- `ai-capability-framework/memory` subpath checks host-owned memory summaries for
17
- use-case, scope, expiry, consent, and sensitivity before they become model
18
- context. It does not store or recall memory.
15
+ Governed memory is also separate. The optional `ai-capability-framework/memory` subpath
16
+ checks host-owned memory summaries for use-case, scope, expiry, consent, and sensitivity
17
+ before they become model context. It does not store or recall memory.
19
18
 
20
- Capability-aware security packs are also separate runtime/test artifacts, but
21
- their assignment metadata lives under `extensions.governance.security_packs`.
22
- Explicit waivers live under `extensions.governance.security_pack_waivers` and
23
- must include `pack_id`, `reason`, `reviewer`, and `reviewed_at`. See
19
+ Capability-aware security packs are also separate runtime/test artifacts, but their
20
+ assignment metadata lives under `extensions.governance.security_packs`. Explicit waivers
21
+ live under `extensions.governance.security_pack_waivers` and must include `pack_id`,
22
+ `reason`, `reviewer`, and `reviewed_at`. See
24
23
  [security packs](../security/security-packs.md).
25
24
 
26
25
  ## Governance Gate
27
26
 
28
- `aicf gate` runs the release-oriented checks together: manifest and fixture
29
- validation, semantic invariants, risk, lifecycle posture, optional compatibility
30
- baseline checks, impact analysis, eval coverage, security-pack coverage,
31
- configured provider conformance, and public artifact hygiene.
27
+ `aicf gate` runs the release-oriented checks together: manifest and fixture validation,
28
+ semantic invariants, risk, lifecycle posture, optional compatibility baseline checks,
29
+ impact analysis, eval coverage, security-pack coverage, configured provider conformance,
30
+ and public artifact hygiene.
32
31
 
33
32
  ```bash
34
33
  aicf gate examples --env production --format json
@@ -38,31 +37,31 @@ See [governance gate](gate.md) for config, defaults, and CI exit codes.
38
37
 
39
38
  ## Control Plane
40
39
 
41
- `ai-capability-framework/control-plane` provides an optional self-hostable API
42
- and local reference UI for reviewing governance outputs, policy decision
43
- records, action/approval queues, runtime controls, eval/security/conformance
44
- status, and redacted replay metadata.
40
+ `ai-capability-framework/control-plane` provides an optional self-hostable API and local
41
+ reference UI for reviewing governance outputs, policy decision records, action/approval
42
+ queues, runtime controls, eval/security/conformance status, and redacted replay
43
+ metadata.
45
44
 
46
- The control plane is not a hosted service and does not provide production auth,
47
- durable storage, workflow orchestration, model calls, provider SDK execution, or
48
- side-effect execution. See [control plane](../control-plane.md).
45
+ The control plane is not a hosted service and does not provide production auth, durable
46
+ storage, workflow orchestration, model calls, provider SDK execution, or side-effect
47
+ execution. See [control plane](../control-plane.md).
49
48
 
50
49
  ## Lifecycle
51
50
 
52
- Governance lifecycle status is separate from the current manifest `status`
53
- field. When a command needs a starting status and `--from` is not supplied,
54
- AICF maps current manifest status like this:
51
+ Governance lifecycle status is separate from the current manifest `status` field. When a
52
+ command needs a starting status and `--from` is not supplied, AICF maps current manifest
53
+ status like this:
55
54
 
56
55
  | Manifest status | Governance status |
57
- | --- | --- |
58
- | `draft` | `draft` |
59
- | `experimental` | `review` |
60
- | `active` | `production` |
61
- | `deprecated` | `deprecated` |
62
- | `disabled` | `disabled` |
56
+ | --------------- | ----------------- |
57
+ | `draft` | `draft` |
58
+ | `experimental` | `review` |
59
+ | `active` | `production` |
60
+ | `deprecated` | `deprecated` |
61
+ | `disabled` | `disabled` |
63
62
 
64
- Supported governance statuses are `draft`, `review`, `approved`, `canary`,
65
- `production`, `deprecated`, `disabled`, and `removed`.
63
+ Supported governance statuses are `draft`, `review`, `approved`, `canary`, `production`,
64
+ `deprecated`, `disabled`, and `removed`.
66
65
 
67
66
  Example:
68
67
 
@@ -74,26 +73,26 @@ aicf governance lifecycle examples \
74
73
  --reason "Ready for controlled review"
75
74
  ```
76
75
 
77
- The result includes `allowed`, blocking reasons, warnings, and required actions.
78
- No files are changed.
76
+ The result includes `allowed`, blocking reasons, warnings, and required actions. No
77
+ files are changed.
79
78
 
80
79
  ## Risk Compilation
81
80
 
82
- Risk compilation compares the declared `risk_tier` with an inferred minimum risk
83
- from side effects, lifecycle metadata, auth scope, entity classifications, and
84
- policy controls.
81
+ Risk compilation compares the declared `risk_tier` with an inferred minimum risk from
82
+ side effects, lifecycle metadata, auth scope, entity classifications, and policy
83
+ controls.
85
84
 
86
85
  ```bash
87
86
  aicf governance risk examples --capability support.refund.prepare_case
88
87
  ```
89
88
 
90
- Missing safety controls are blocking. Catalogue-quality gaps, such as missing
91
- usage guidance, remain warnings unless `--strict` is used.
89
+ Missing safety controls are blocking. Catalogue-quality gaps, such as missing usage
90
+ guidance, remain warnings unless `--strict` is used.
92
91
 
93
92
  ## Compatibility
94
93
 
95
- Compatibility compares two versions of a capability manifest and classifies the
96
- change as `compatible`, `requires_minor`, or `breaking`.
94
+ Compatibility compares two versions of a capability manifest and classifies the change
95
+ as `compatible`, `requires_minor`, or `breaking`.
97
96
 
98
97
  ```bash
99
98
  aicf governance compatibility \
@@ -101,21 +100,21 @@ aicf governance compatibility \
101
100
  --after examples/support/capabilities/support.refund.prepare_case.yaml
102
101
  ```
103
102
 
104
- Breaking changes include adding required input fields, removing inputs, lowering
105
- risk, lowering approval requirements, adding side effects, or removing
106
- idempotency/audit controls.
103
+ Breaking changes include adding required input fields, removing inputs, lowering risk,
104
+ lowering approval requirements, adding side effects, or removing idempotency/audit
105
+ controls.
107
106
 
108
107
  ## Impact
109
108
 
110
- Impact analysis reports directly affected capabilities, entities, evals,
111
- provider exports, policy references, and coverage gaps from the loaded registry.
109
+ Impact analysis reports directly affected capabilities, entities, evals, provider
110
+ exports, policy references, and coverage gaps from the loaded registry.
112
111
 
113
112
  ```bash
114
113
  aicf governance impact examples --capability support.refund.prepare_case
115
114
  ```
116
115
 
117
- F1 impact analysis is registry-local. Trace, tenant, or production-store impact
118
- can be supplied by later host integrations, but no database is required.
116
+ F1 impact analysis is registry-local. Trace, tenant, or production-store impact can be
117
+ supplied by later host integrations, but no database is required.
119
118
 
120
119
  ## TypeScript
121
120
 
@@ -1,8 +1,7 @@
1
1
  # Lifecycle
2
2
 
3
- Lifecycle checks decide whether a capability can move between governance
4
- statuses such as draft, review, approved, canary, production, deprecated,
5
- disabled, and removed.
3
+ Lifecycle checks decide whether a capability can move between governance statuses such
4
+ as draft, review, approved, canary, production, deprecated, disabled, and removed.
6
5
 
7
6
  Use:
8
7
 
@@ -10,5 +9,5 @@ Use:
10
9
  node dist/cli.js governance lifecycle examples --capability support.refund.prepare_case --to production
11
10
  ```
12
11
 
13
- Lifecycle decisions are structured recommendations. AICF does not mutate
14
- manifests or deploy capabilities.
12
+ Lifecycle decisions are structured recommendations. AICF does not mutate manifests or
13
+ deploy capabilities.
@@ -1,8 +1,7 @@
1
1
  # Governance Overview
2
2
 
3
- AICF governance checks lifecycle transitions, inferred risk, compatibility,
4
- impact, eval coverage, security-pack coverage, provider conformance, and public
5
- artifact hygiene.
3
+ AICF governance checks lifecycle transitions, inferred risk, compatibility, impact, eval
4
+ coverage, security-pack coverage, provider conformance, and public artifact hygiene.
6
5
 
7
6
  Start with:
8
7
 
@@ -1,8 +1,7 @@
1
1
  # Risk Compiler
2
2
 
3
- The risk compiler compares declared risk to inferred minimum risk from side
4
- effects, lifecycle, policy, idempotency, audit, auth scoping, and entity
5
- metadata.
3
+ The risk compiler compares declared risk to inferred minimum risk from side effects,
4
+ lifecycle, policy, idempotency, audit, auth scoping, and entity metadata.
6
5
 
7
6
  Use:
8
7
 
@@ -10,5 +9,5 @@ Use:
10
9
  node dist/cli.js governance risk examples --format text
11
10
  ```
12
11
 
13
- Safety and runtime correctness issues are errors. Quality or migration gaps are
14
- warnings unless they affect safety.
12
+ Safety and runtime correctness issues are errors. Quality or migration gaps are warnings
13
+ unless they affect safety.
@@ -1,7 +1,6 @@
1
1
  # Host Responsibilities
2
2
 
3
- AICF defines contracts and deterministic checks. Host applications own runtime
4
- behavior.
3
+ AICF defines contracts and deterministic checks. Host applications own runtime behavior.
5
4
 
6
5
  Host applications MUST provide or enforce:
7
6
 
@@ -10,14 +9,13 @@ Host applications MUST provide or enforce:
10
9
  - policy facts used by `deny_if` and `approval_required_if`;
11
10
  - approval collection, approval verification, and approval records;
12
11
  - handler registries and capability execution;
13
- - side effects such as writes, sends, refunds, workflow starts, and permission
14
- changes;
12
+ - side effects such as writes, sends, refunds, workflow starts, and permission changes;
15
13
  - idempotency storage, replay protection, and conflict handling;
16
14
  - durable action state, verification state, and audit logs;
17
15
  - trace redaction and provider payload retention policies.
18
16
 
19
- AICF decision results and audit previews are inputs to host control logic. They
20
- are not proof that a real user is authorized and are not durable audit records.
17
+ AICF decision results and audit previews are inputs to host control logic. They are not
18
+ proof that a real user is authorized and are not durable audit records.
21
19
 
22
- Public examples in this repository are synthetic. Do not replace host security
23
- controls with example permissions, IDs, or policy facts.
20
+ Public examples in this repository are synthetic. Do not replace host security controls
21
+ with example permissions, IDs, or policy facts.
@@ -1,26 +1,23 @@
1
1
  # F0 Baseline Gaps
2
2
 
3
- This note records the F0 baseline reconciliation result for the public
4
- repository. It is intentionally limited to public implementation facts.
3
+ This note records the F0 baseline reconciliation result for the public repository. It is
4
+ intentionally limited to public implementation facts.
5
5
 
6
6
  ## Findings
7
7
 
8
- - Core repairs are present: schemas, manifest loading, registry construction,
9
- capability decisions, adapter exports, tool-result envelopes, and eval scoring
10
- are covered by deterministic tests.
11
- - Runtime APIs are present under the runtime subpath and provide context,
12
- routing, policy, handler execution, action lifecycle, in-memory stores, audit,
13
- redaction, and envelope helpers.
14
- - Provider APIs are present under optional subpaths for OpenAI, Anthropic,
15
- Gemini, AI SDK, LangChain, MCP, Semantic Kernel, AWS, observability, live eval,
16
- Promptfoo, Langfuse, and MCP server integrations.
17
- - Release hygiene exists through package contents checks, public artifact
18
- checks, workspace hygiene checks, source archive checks, and clean install
19
- smoke tests.
20
- - The explicit `npm run typecheck` command has been added for F0 command
21
- compatibility.
22
- - The current public architecture is documented in
23
- `docs/architecture/current-state.md`.
8
+ - Core repairs are present: schemas, manifest loading, registry construction, capability
9
+ decisions, adapter exports, tool-result envelopes, and eval scoring are covered by
10
+ deterministic tests.
11
+ - Runtime APIs are present under the runtime subpath and provide context, routing,
12
+ policy, handler execution, action lifecycle, in-memory stores, audit, redaction, and
13
+ envelope helpers.
14
+ - Provider APIs are present under optional subpaths for OpenAI, Anthropic, Gemini, AI
15
+ SDK, LangChain, MCP, Semantic Kernel, AWS, observability, live eval, Promptfoo,
16
+ Langfuse, and MCP server integrations.
17
+ - Release hygiene exists through package contents checks, public artifact checks,
18
+ workspace hygiene checks, source archive checks, and clean install smoke tests.
19
+ - The explicit `npm run typecheck` command has been added for F0 command compatibility.
20
+ - The current public architecture is documented in `docs/architecture/current-state.md`.
24
21
 
25
22
  ## Result
26
23
 
package/docs/index.md CHANGED
@@ -1,20 +1,29 @@
1
1
  # AICF Documentation
2
2
 
3
- AI Capability Framework is a governed capability layer for AI-accessible
4
- application functionality. AICF is not an agent framework.
3
+ AI Capability Framework (AICF) is a provider-agnostic governed capability layer for
4
+ AI-accessible application functionality. AICF is not an agent framework.
5
5
 
6
6
  Models propose; applications validate, authorize, execute, and audit.
7
7
 
8
8
  ## Start
9
9
 
10
10
  1. [Installation](getting-started/installation.md)
11
- 2. [Quickstart](getting-started/quickstart.md)
12
- 3. [Concepts](getting-started/concepts.md)
13
- 4. [OpenAI walkthrough](openai-walkthrough.md)
14
- 5. [Glossary](glossary.md)
11
+ 2. [Provider-neutral quickstart](getting-started/provider-neutral-quickstart.md)
12
+ 3. [Quickstart](getting-started/quickstart.md)
13
+ 4. [Concepts](getting-started/concepts.md)
14
+ 5. [OpenAI quickstart](getting-started/openai-quickstart.md)
15
+ 6. [Anthropic quickstart](getting-started/anthropic-quickstart.md)
16
+ 7. [Gemini quickstart](getting-started/gemini-quickstart.md)
17
+ 8. [OpenAI walkthrough](openai-walkthrough.md)
18
+ 9. [Glossary](glossary.md)
15
19
 
16
20
  ## Build The Capability Layer
17
21
 
22
+ - Core contracts define public manifests, schemas, validation, and deterministic
23
+ no-execution decisions.
24
+ - Runtime, providers, governance, evals, security, observability, the control plane, and
25
+ agent skills are optional surfaces around those contracts.
26
+
18
27
  - [Capability manifests](core/capability-manifests.md)
19
28
  - [Entity manifests](core/entity-manifests.md)
20
29
  - [Runtime overview](runtime/runtime-overview.md)
@@ -25,6 +34,7 @@ Models propose; applications validate, authorize, execute, and audit.
25
34
  ## Connect Providers
26
35
 
27
36
  - [Provider overview](providers.md)
37
+ - [Choose a runtime](providers/choose-a-runtime.md)
28
38
  - [OpenAI](providers/openai.md)
29
39
  - [Anthropic Claude](providers/anthropic.md)
30
40
  - [Google Gemini](providers/gemini.md)
@@ -54,7 +64,12 @@ Models propose; applications validate, authorize, execute, and audit.
54
64
  - [Control plane](control-plane/overview.md)
55
65
  - [Release process](public-framework/release-process.md)
56
66
  - [Final v1 certification](public-framework/v1-certification.md)
67
+ - [Final certification matrix](public-framework/final-certification-matrix.md)
68
+ - [npm release preflight](public/npm-release-preflight.md)
69
+ - [License decision](public-framework/license-decision.md)
70
+ - [Dependency license exceptions](public/license-exceptions.md)
57
71
  - [Compatibility policy](public-framework/compatibility-policy.md)
58
72
  - [Deprecation policy](public-framework/deprecation-policy.md)
59
73
  - [Security disclosure](public-framework/security-disclosure.md)
60
74
  - [API reference](api.md)
75
+ - [Public API policy](api/public-api-policy.md)
@@ -1,7 +1,7 @@
1
1
  # Interoperability
2
2
 
3
- AICF describes AI-facing application capabilities. Other frameworks may execute
4
- or orchestrate those capabilities.
3
+ AICF describes AI-facing application capabilities. Other frameworks may execute or
4
+ orchestrate those capabilities.
5
5
 
6
6
  Use AICF as the portable contract layer for:
7
7
 
@@ -22,11 +22,11 @@ Use host or adjacent runtime frameworks for:
22
22
  - durable audit pipelines;
23
23
  - production observability.
24
24
 
25
- When integrating with another runtime, keep the boundary explicit: AICF can
26
- validate and decide whether a capability request is allowed, approval-required,
27
- or denied, but the runtime must still enforce real auth, produce facts, execute
28
- or refuse handlers, and persist records.
25
+ When integrating with another runtime, keep the boundary explicit: AICF can validate and
26
+ decide whether a capability request is allowed, approval-required, or denied, but the
27
+ runtime must still enforce real auth, produce facts, execute or refuse handlers, and
28
+ persist records.
29
29
 
30
- See [the adapter guide](adapters.md) for implemented provider/runtime adapters
31
- and [the adapter status page](adapter-roadmap.md) for reference links and
32
- priority rationale.
30
+ See [the adapter guide](adapters.md) for implemented provider/runtime adapters and
31
+ [the adapter status page](adapter-roadmap.md) for reference links and priority
32
+ rationale.
@@ -1,13 +1,13 @@
1
1
  # LangChain And LangGraph Bridge
2
2
 
3
- The LangChain bridge is an optional provider adapter for applications that
4
- already use LangChain.js or LangGraph.js. It translates a routed AICF capability
5
- slice into LangChain tools, executes tool calls through `AicfToolExecutor`, and
6
- returns serialized model-safe AICF runtime envelopes.
3
+ The LangChain bridge is an optional provider adapter for applications that already use
4
+ LangChain.js or LangGraph.js. It translates a routed AICF capability slice into
5
+ LangChain tools, executes tool calls through `AicfToolExecutor`, and returns serialized
6
+ model-safe AICF runtime envelopes.
7
7
 
8
- It does not install model provider integrations, choose a chat model, expose
9
- commit tools, persist state, log raw provider payloads, or make root/runtime
10
- imports depend on LangChain.
8
+ It does not install model provider integrations, choose a chat model, expose commit
9
+ tools, persist state, log raw provider payloads, or make root/runtime imports depend on
10
+ LangChain.
11
11
 
12
12
  ## Import
13
13
 
@@ -27,47 +27,43 @@ import {
27
27
  ```
28
28
 
29
29
  `createDefaultLangChainToolFactory()` dynamically imports the optional
30
- `@langchain/core/tools` package. Host applications may instead pass a
31
- compatible `tool` factory.
30
+ `@langchain/core/tools` package. Host applications may instead pass a compatible `tool`
31
+ factory.
32
32
 
33
33
  ## Tool Bridge
34
34
 
35
- `buildLangChainTools()` builds a normal LangChain tool array from an
36
- already-routed AICF capability slice. Each generated tool uses a provider-safe
37
- name, model-safe description, schema, and function that delegates to
38
- `executeProviderToolCall()`.
35
+ `buildLangChainTools()` builds a normal LangChain tool array from an already-routed AICF
36
+ capability slice. Each generated tool uses a provider-safe name, model-safe description,
37
+ schema, and function that delegates to `executeProviderToolCall()`.
39
38
 
40
- LangChain schema validation is advisory. AICF still performs canonical input
41
- validation, policy checks, lifecycle checks, approval handling, and model-safe
42
- envelope formatting.
39
+ LangChain schema validation is advisory. AICF still performs canonical input validation,
40
+ policy checks, lifecycle checks, approval handling, and model-safe envelope formatting.
43
41
 
44
42
  Minimal usage is: validate manifests, build a routed runtime slice, register
45
- read/prepare handlers, create or pass a LangChain tool factory, then give the
46
- generated tools to the host-owned agent or graph.
43
+ read/prepare handlers, create or pass a LangChain tool factory, then give the generated
44
+ tools to the host-owned agent or graph.
47
45
 
48
46
  ## LangGraph ToolNode
49
47
 
50
48
  `buildLangGraphToolNode()` is a thin wrapper around a host-supplied `ToolNode`
51
- constructor. AICF does not import `@langchain/langgraph` by default. The helper
52
- creates the same AICF-backed tools and passes them to the supplied constructor.
49
+ constructor. AICF does not import `@langchain/langgraph` by default. The helper creates
50
+ the same AICF-backed tools and passes them to the supplied constructor.
53
51
 
54
52
  ## Boundary
55
53
 
56
- - AICF capability manifests, policy, lifecycle, and runtime envelopes remain the
57
- source of truth.
58
- - Commit, destructive, money-moving, and external-send capabilities are not
59
- exported by default and are not executable through LangChain tool calls.
54
+ - AICF capability manifests, policy, lifecycle, and runtime envelopes remain the source
55
+ of truth.
56
+ - Commit, destructive, money-moving, and external-send capabilities are not exported by
57
+ default and are not executable through LangChain tool calls.
60
58
  - Provider packages such as `@langchain/openai`, `@langchain/anthropic`, and
61
- `@langchain/google-genai` are host application dependencies, not AICF
62
- dependencies.
63
- - Raw prompts, provider request/response payloads, traces, credentials, and
64
- private diagnostics are not captured by default.
59
+ `@langchain/google-genai` are host application dependencies, not AICF dependencies.
60
+ - Raw prompts, provider request/response payloads, traces, credentials, and private
61
+ diagnostics are not captured by default.
65
62
  - Live tests are opt-in with `RUN_LIVE_LANGCHAIN=1` and may use fake
66
63
  LangChain-compatible tools instead of a real provider model.
67
64
 
68
- Known limitations: no provider-specific LangChain integrations, no model
69
- selection, no production storage, no automatic approval UI, and no model-exposed
70
- commit path.
65
+ Known limitations: no provider-specific LangChain integrations, no model selection, no
66
+ production storage, no automatic approval UI, and no model-exposed commit path.
71
67
 
72
68
  ## Checks
73
69
 
@@ -1,61 +1,98 @@
1
1
  # Live Evals
2
2
 
3
- Live evals are optional runtime checks that execute synthetic eval cases through
4
- the OpenAI Responses runtime with a caller-provided client. Import them from:
3
+ Live evals are optional runtime checks that execute synthetic eval cases through a
4
+ caller-provided provider runner. They are provider-neutral by default and keep OpenAI as
5
+ one convenience adapter. Import them from:
5
6
 
6
7
  ```ts
7
8
  import {
9
+ createOpenAILiveEvalRunner,
8
10
  evaluateGate,
9
11
  runLiveEvalSuite
10
12
  } from "ai-capability-framework/evals-live";
11
13
  ```
12
14
 
13
- Default tests use mock clients. No live model call runs in normal CI.
14
- The public runtime support/billing example is separate from live evals and runs
15
- without model credentials.
15
+ Default tests use mock clients. No live model call runs in normal CI. The public runtime
16
+ support/billing example is separate from live evals and runs without model credentials.
16
17
 
17
18
  ## Deterministic Fixtures Vs Live Evals
18
19
 
19
- The Core eval runner scores public candidate-result fixtures without API keys.
20
- Live evals run the same kind of behavior through the runtime path first, then
21
- summarize the result into candidate fields such as selected capabilities, tool
22
- calls, policy status, action state, committed capabilities, and final text.
20
+ The Core eval runner scores public candidate-result fixtures without API keys. Live
21
+ evals run the same kind of behavior through the runtime path first, then summarize the
22
+ result into candidate fields such as selected capabilities, tool calls, policy status,
23
+ action state, committed capabilities, and final text.
23
24
 
24
- The live runner reuses deterministic scorer semantics where possible. It does
25
- not add LLM-as-judge scoring in this phase.
25
+ The live runner reuses deterministic scorer semantics where possible. It does not add
26
+ LLM-as-judge scoring in this phase.
26
27
 
27
28
  ## Running A Suite
28
29
 
29
30
  ```ts
31
+ const runner = createOpenAILiveEvalRunner({
32
+ client: openAIClient,
33
+ model: "gpt-4.1-mini"
34
+ });
35
+
30
36
  const results = await runLiveEvalSuite({
31
37
  cases,
32
38
  contextBuilderFactory: () => contextBuilder,
33
39
  executor,
34
- model: "gpt-4.1-mini",
35
- openAIClient,
36
40
  registry,
37
- router
41
+ router,
42
+ runner
38
43
  });
39
44
 
40
45
  const gate = evaluateGate(results);
41
46
  ```
42
47
 
43
- The default gate fails when any eval errors, a safety/policy scorer fails, or
44
- the average score is below `0.90`.
48
+ The default gate fails when any eval errors, a safety/policy scorer fails, or the
49
+ average score is below `0.90`.
50
+
51
+ For source-compatible OpenAI callers, `runLiveEvalSuite({ openAIClient, model, ... })`
52
+ still works. New code should pass an explicit runner. Convenience runner factories are
53
+ available for OpenAI Responses, Anthropic Messages, Gemini GenerateContent, and Vercel
54
+ AI SDK `generateText`.
55
+
56
+ Non-OpenAI example:
57
+
58
+ ```ts
59
+ import { createAnthropicLiveEvalRunner } from "ai-capability-framework/evals-live";
60
+
61
+ const runner = createAnthropicLiveEvalRunner({
62
+ client: anthropicClient,
63
+ model: "claude-3-5-sonnet-latest"
64
+ });
65
+ ```
66
+
67
+ LangChain/LangGraph and MCP hosts can implement `AicfLiveEvalRunner` directly around
68
+ their own runtime loop. AICF does not expose commit tools to any live eval runner.
69
+
70
+ ## CLI Live Evals
71
+
72
+ The CLI is for intentional live checks only:
73
+
74
+ ```bash
75
+ RUN_REAL_AICF_LIVE_EVALS=1 \
76
+ OPENAI_API_KEY=... \
77
+ aicf eval-live examples --cases cases.json --provider openai --model gpt-4.1-mini
78
+ ```
79
+
80
+ Supported CLI providers are `openai`, `anthropic`, and `gemini`. The AI SDK path is
81
+ available through TypeScript because host applications must supply `generateText` and a
82
+ configured model object. The CLI refuses to run live evals unless
83
+ `RUN_REAL_AICF_LIVE_EVALS=1` is set.
45
84
 
46
85
  ## Trace To Eval
47
86
 
48
- `createEvalCaseFromTrace()` creates a draft public-safe eval case from an
49
- OpenAI runtime result. It excludes raw user text and raw model output by
50
- default. Include model output only when the host has reviewed it for public
51
- safety.
87
+ `createEvalCaseFromTrace()` creates a draft public-safe eval case from an OpenAI runtime
88
+ result. It excludes raw user text and raw model output by default. Include model output
89
+ only when the host has reviewed it for public safety.
52
90
 
53
91
  ## Promptfoo Export
54
92
 
55
93
  Promptfoo support is available from `ai-capability-framework/promptfoo`.
56
- `exportPromptfooSuite()` generates files only; it does not run Promptfoo. The
57
- default provider is `echo` so generated suites can be inspected without API
58
- keys.
94
+ `exportPromptfooSuite()` generates files only; it does not run Promptfoo. The default
95
+ provider is `echo` so generated suites can be inspected without API keys.
59
96
 
60
97
  The CLI command writes generated files when explicitly requested:
61
98
 
@@ -63,18 +100,17 @@ The CLI command writes generated files when explicitly requested:
63
100
  aicf export promptfoo examples --out ./promptfoo
64
101
  ```
65
102
 
66
- `importPromptfooResults()` maps basic Promptfoo JSON results back into live eval
67
- result summaries.
103
+ `importPromptfooResults()` maps basic Promptfoo JSON results back into live eval result
104
+ summaries.
68
105
 
69
106
  Capability-aware security packs have a focused Promptfoo export under
70
107
  `ai-capability-framework/security-packs`. That export generates red-team config
71
- templates with `echo` as the default provider and placeholders for host runtime
72
- targets. AICF does not run Promptfoo or call providers when exporting those
73
- files.
108
+ templates with `echo` as the default provider and placeholders for host runtime targets.
109
+ AICF does not run Promptfoo or call providers when exporting those files.
74
110
 
75
111
  ## EvalOps Exports
76
112
 
77
- `ai-capability-framework/evalops` provides dependency-free JSON exporters and
78
- importers for Braintrust-style and OpenAI-eval-style datasets/results. These
79
- helpers transform public-safe eval cases and summarized results only. They do
80
- not upload data, call external services, or require provider SDKs.
113
+ `ai-capability-framework/evalops` provides dependency-free JSON exporters and importers
114
+ for Braintrust-style and OpenAI-eval-style datasets/results. These helpers transform
115
+ public-safe eval cases and summarized results only. They do not upload data, call
116
+ external services, or require provider SDKs.