ai-capability-framework 1.0.0-rc.3 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +97 -103
- package/CODE_OF_CONDUCT.md +11 -13
- package/CONTRIBUTING.md +40 -43
- package/GOVERNANCE.md +34 -35
- package/README.md +99 -63
- package/ROADMAP.md +12 -12
- package/SECURITY.md +15 -15
- package/dist/audit/ledger.d.ts.map +1 -1
- package/dist/audit/ledger.js +2 -1
- package/dist/audit/records.d.ts.map +1 -1
- package/dist/audit/records.js +2 -1
- package/dist/audit/types.d.ts +4 -1
- package/dist/audit/types.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.d.ts +2 -7
- package/dist/aws/dynamodb-stores.d.ts.map +1 -1
- package/dist/aws/dynamodb-stores.js +56 -5
- package/dist/aws/testing.d.ts.map +1 -1
- package/dist/aws/testing.js +9 -0
- package/dist/cli.d.ts +6 -6
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +77 -7
- package/dist/evals-live/index.d.ts +2 -2
- package/dist/evals-live/index.d.ts.map +1 -1
- package/dist/evals-live/index.js +1 -1
- package/dist/evals-live/live-eval-runner.d.ts +49 -1
- package/dist/evals-live/live-eval-runner.d.ts.map +1 -1
- package/dist/evals-live/live-eval-runner.js +246 -4
- package/dist/evals-live/types.d.ts +62 -9
- package/dist/evals-live/types.d.ts.map +1 -1
- package/dist/index.d.ts +0 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +0 -1
- package/dist/runtime/actions.d.ts +2 -0
- package/dist/runtime/actions.d.ts.map +1 -1
- package/dist/runtime/actions.js +200 -55
- package/dist/runtime/index.d.ts +1 -1
- package/dist/runtime/index.d.ts.map +1 -1
- package/dist/runtime/stores.d.ts +2 -7
- package/dist/runtime/stores.d.ts.map +1 -1
- package/dist/runtime/stores.js +12 -0
- package/dist/runtime/types.d.ts +25 -6
- package/dist/runtime/types.d.ts.map +1 -1
- package/docs/action-lifecycle.md +38 -32
- package/docs/adapter-roadmap.md +51 -49
- package/docs/adapters.md +14 -15
- package/docs/ai-sdk-runtime.md +36 -39
- package/docs/anthropic-runtime.md +29 -32
- package/docs/api/public-api-policy.md +59 -0
- package/docs/api.md +354 -371
- package/docs/architecture/current-state.md +28 -31
- package/docs/audit/index.md +29 -29
- package/docs/aws/cloudwatch-telemetry.md +5 -7
- package/docs/aws/dynamodb-single-table.md +8 -9
- package/docs/aws/kms-redaction.md +5 -6
- package/docs/aws/overview.md +5 -5
- package/docs/aws/production-reference.md +13 -14
- package/docs/aws/step-functions-approval.md +7 -9
- package/docs/aws-runtime.md +24 -25
- package/docs/capability-manifests.md +17 -21
- package/docs/control-plane/overview.md +2 -2
- package/docs/control-plane.md +24 -27
- package/docs/controls/index.md +19 -22
- package/docs/core/capability-manifests.md +4 -5
- package/docs/core/entity-manifests.md +5 -5
- package/docs/eval-runner.md +29 -31
- package/docs/evalops.md +13 -14
- package/docs/evals/golden-tests.md +4 -5
- package/docs/evals/overview.md +3 -3
- package/docs/evals/replay-and-trace-to-golden.md +19 -20
- package/docs/evals.md +19 -20
- package/docs/evidence.md +22 -23
- package/docs/gemini-runtime.md +31 -33
- package/docs/getting-started/anthropic-quickstart.md +61 -0
- package/docs/getting-started/concepts.md +9 -10
- package/docs/getting-started/gemini-quickstart.md +62 -0
- package/docs/getting-started/installation.md +3 -3
- package/docs/getting-started/openai-quickstart.md +61 -0
- package/docs/getting-started/provider-neutral-quickstart.md +106 -0
- package/docs/getting-started/quickstart.md +2 -2
- package/docs/getting-started.md +23 -20
- package/docs/glossary.md +31 -48
- package/docs/governance/gate.md +28 -30
- package/docs/governance/impact-and-compatibility.md +2 -3
- package/docs/governance/index.md +53 -54
- package/docs/governance/lifecycle.md +4 -5
- package/docs/governance/overview.md +2 -3
- package/docs/governance/risk-compiler.md +4 -5
- package/docs/host-responsibilities.md +6 -8
- package/docs/implementation-notes/baseline-gaps.md +15 -18
- package/docs/index.md +21 -6
- package/docs/interoperability.md +9 -9
- package/docs/langchain-runtime.md +27 -31
- package/docs/live-evals.md +68 -32
- package/docs/mcp-server-runtime.md +36 -40
- package/docs/memory.md +35 -36
- package/docs/migration-0.1-to-1.0.md +14 -9
- package/docs/observability/overview.md +2 -3
- package/docs/observability-runtime.md +35 -39
- package/docs/openai-responses.md +36 -37
- package/docs/openai-runtime.md +41 -46
- package/docs/openai-walkthrough.md +24 -28
- package/docs/policy-broker.md +9 -10
- package/docs/provenance.md +30 -29
- package/docs/providers/anthropic.md +5 -5
- package/docs/providers/choose-a-runtime.md +37 -0
- package/docs/providers/conformance.md +14 -15
- package/docs/providers/gemini.md +5 -5
- package/docs/providers/mcp.md +2 -3
- package/docs/providers/openai.md +2 -2
- package/docs/providers/semantic-kernel.md +2 -2
- package/docs/providers.md +43 -38
- package/docs/public/license-exceptions.md +187 -0
- package/docs/public/npm-release-preflight.md +112 -0
- package/docs/public-framework/compatibility-policy.md +12 -13
- package/docs/public-framework/deprecation-policy.md +8 -8
- package/docs/public-framework/final-certification-matrix.md +104 -0
- package/docs/public-framework/license-decision.md +20 -0
- package/docs/public-framework/release-process.md +46 -16
- package/docs/public-framework/security-disclosure.md +4 -5
- package/docs/public-framework/v1-certification.md +102 -15
- package/docs/release.md +200 -122
- package/docs/runtime/action-lifecycle.md +3 -3
- package/docs/runtime/policy-broker.md +4 -4
- package/docs/runtime/runtime-overview.md +4 -5
- package/docs/runtime/tool-result-envelope.md +7 -7
- package/docs/runtime.md +90 -93
- package/docs/security/overview.md +4 -4
- package/docs/security/security-packs.md +24 -26
- package/docs/security/trust-taint-redaction.md +42 -47
- package/docs/semantic-kernel-runtime.md +35 -37
- package/docs/spec.md +95 -101
- package/docs/start-here.md +18 -20
- package/examples/01-basic-read-capability/README.md +3 -4
- package/examples/02-prepare-approve-commit/README.md +6 -6
- package/examples/03-multi-provider-tools/README.md +2 -2
- package/examples/04-mcp-server/README.md +4 -5
- package/examples/05-langchain-langgraph-bridge/README.md +4 -5
- package/examples/06-vercel-ai-sdk-bridge/README.md +5 -7
- package/examples/07-policy-broker-custom-auth/README.md +4 -4
- package/examples/08-aws-step-functions-approval/README.md +4 -5
- package/examples/09-security-packs-promptfoo/README.md +4 -4
- package/examples/10-trace-to-golden/README.md +2 -2
- package/examples/11-control-plane/README.md +4 -5
- package/examples/aws/README.md +5 -8
- package/examples/control-plane/README.md +8 -9
- package/examples/control-plane/fixtures/control-plane.seed.json +8 -32
- package/examples/control-plane/public/app.js +57 -18
- package/examples/control-plane/public/index.html +15 -6
- package/examples/control-plane/server.mjs +15 -3
- package/examples/eval-results/public.results.passing.json +6 -21
- package/examples/providers/ai-sdk-next/README.md +11 -8
- package/examples/providers/anthropic-claude/README.md +5 -6
- package/examples/providers/gemini/README.md +6 -6
- package/examples/providers/langchain-agent/README.md +3 -4
- package/examples/providers/langgraph-tool-node/README.md +7 -7
- package/examples/providers/mcp/README.md +8 -8
- package/examples/providers/provider-conformance/README.md +3 -3
- package/examples/providers/semantic-kernel-mcp/README.md +6 -6
- package/examples/providers/semantic-kernel-openapi/README.md +6 -7
- package/examples/runtime-support-billing/README.md +8 -9
- package/examples/runtime-support-billing/support-billing-runtime.mjs +8 -4
- package/examples/scheduling/capabilities/scheduling.availability.get.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.prepare.yaml +2 -2
- package/examples/scheduling/capabilities/scheduling.invite.send.yaml +2 -2
- package/examples/scheduling/entities/AvailabilitySlot.yaml +2 -2
- package/examples/scheduling/entities/MeetingInvite.yaml +2 -2
- package/examples/scheduling/evals/scheduling.invite.prepare.valid.yaml +3 -1
- package/examples/semantic-kernel-mcp/README.md +11 -12
- package/examples/semantic-kernel-openapi/README.md +8 -8
- package/examples/support/capabilities/support.refund.commit_case.yaml +3 -3
- package/examples/support/capabilities/support.refund.prepare_case.yaml +2 -2
- package/examples/support/capabilities/support.ticket.get.yaml +2 -2
- package/examples/support/entities/Ticket.yaml +0 -1
- package/examples/support/eval-results/support.results.passing.json +3 -12
- package/examples/support/evals/support.refund.prepare_case.approval_required.yaml +3 -1
- package/examples/support/evals/support.refund.prepare_case.valid.yaml +3 -2
- package/examples/support/memory/support.agent.preferences.json +2 -6
- package/examples/support/openai/context.support_agent.json +1 -4
- package/examples/support/provenance/support.refund.summary.provenance.json +2 -6
- package/examples/support/replay/support.refund.approval_required.trace.json +4 -14
- package/package.json +22 -5
- package/schemas/audit/action-record.schema.json +4 -1
- package/schemas/aws/approval-task.schema.json +8 -1
- package/schemas/conformance/conformance-case.schema.json +18 -2
- package/schemas/conformance/conformance-report.schema.json +17 -2
- package/schemas/conformance/provider-target-matrix.schema.json +35 -4
- package/schemas/controls/control-decision.schema.json +7 -1
- package/schemas/eval-case.schema.json +8 -1
- package/schemas/eval-result.schema.json +8 -1
- package/schemas/evidence/evidence-pack.schema.json +34 -5
- package/schemas/governance/compatibility-diff.schema.json +8 -1
- package/schemas/governance/lifecycle-transition.schema.json +10 -1
- package/schemas/governance/risk-compilation-result.schema.json +9 -1
- package/schemas/provenance/generated-content-provenance.schema.json +3 -1
- package/schemas/replay/replay-result.schema.json +16 -2
- package/schemas/replay/replay-trace.schema.json +11 -2
- package/schemas/security/context-segment.schema.json +21 -12
- package/schemas/security/redaction-policy.schema.json +4 -1
- package/schemas/security-packs/coverage-report.schema.json +17 -2
- package/schemas/security-packs/security-case-suite.schema.json +11 -1
- package/schemas/tool-result-envelope.schema.json +9 -1
- package/security-packs/approval_bypass.yaml +12 -2
- package/security-packs/capability_spoofing.yaml +17 -3
- package/security-packs/cost_amplification.yaml +21 -4
- package/security-packs/cross_tenant_access.yaml +16 -3
- package/security-packs/insecure_output_rendering.yaml +15 -4
- package/security-packs/mcp_tool_abuse.yaml +17 -3
- package/security-packs/memory_scope_violation.yaml +19 -4
- package/security-packs/prompt_injection_direct.yaml +19 -4
- package/security-packs/prompt_injection_indirect.yaml +18 -3
- package/security-packs/provider_payload_exposure.yaml +26 -4
- package/security-packs/retrieval_poisoning.yaml +5 -2
- package/security-packs/schema_confusion.yaml +17 -3
- package/security-packs/sensitive_data_disclosure.yaml +22 -5
- package/security-packs/tool_exfiltration.yaml +20 -4
- package/security-packs/tool_result_poisoning.yaml +19 -4
- package/security-packs/unsafe_commit_attempt.yaml +6 -3
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# Anthropic Quickstart
|
|
2
|
+
|
|
3
|
+
Use this guide when your host app wants Claude Messages tools backed by AICF. The
|
|
4
|
+
default path uses mock clients and descriptor checks. Live calls are explicitly opt-in.
|
|
5
|
+
|
|
6
|
+
## Install
|
|
7
|
+
|
|
8
|
+
Core AICF does not require the Anthropic SDK:
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
npm install ai-capability-framework
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Only apps that construct a default Claude client need the optional SDK:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npm install @anthropic-ai/sdk
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Provider imports stay isolated:
|
|
21
|
+
|
|
22
|
+
```ts
|
|
23
|
+
import { runAnthropicMessages } from "ai-capability-framework/providers/anthropic";
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Mock-Tested Flow
|
|
27
|
+
|
|
28
|
+
Run the Anthropic mock runtime tests:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
npm run test:anthropic:mock
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Expected output excerpt:
|
|
35
|
+
|
|
36
|
+
```text
|
|
37
|
+
Test Files ... passed
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The Anthropic adapter converts routed AICF read/prepare capabilities into Claude tool
|
|
41
|
+
definitions and maps `tool_use` blocks back through AICF bindings.
|
|
42
|
+
|
|
43
|
+
## Live Opt-In
|
|
44
|
+
|
|
45
|
+
Live tests are skipped unless you explicitly opt in:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
RUN_LIVE_ANTHROPIC=1 ANTHROPIC_API_KEY=... AICF_ANTHROPIC_MODEL=claude-sonnet-4-5 \
|
|
49
|
+
npm run test:anthropic:live
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
For live evals:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
RUN_REAL_AICF_LIVE_EVALS=1 ANTHROPIC_API_KEY=... \
|
|
56
|
+
node dist/cli.js eval-live examples --cases cases.json --provider anthropic --model claude-sonnet-4-5
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
AICF remains the validation, policy, lifecycle, approval, idempotency, and model-safe
|
|
60
|
+
envelope authority. Anthropic tool definitions are an adapter projection, not trusted
|
|
61
|
+
execution.
|
|
@@ -1,22 +1,21 @@
|
|
|
1
1
|
# Concepts
|
|
2
2
|
|
|
3
|
-
AICF is not an agent framework. It is a governed capability layer for
|
|
4
|
-
|
|
3
|
+
AICF is not an agent framework. It is a governed capability layer for AI-accessible
|
|
4
|
+
application functionality.
|
|
5
5
|
|
|
6
6
|
Models propose; applications validate, authorize, execute, and audit.
|
|
7
7
|
|
|
8
8
|
Core concepts:
|
|
9
9
|
|
|
10
10
|
- A capability is a typed, versioned operation your app may expose to AI.
|
|
11
|
-
- A manifest describes the capability, schema, risk, lifecycle, policy, and
|
|
12
|
-
|
|
11
|
+
- A manifest describes the capability, schema, risk, lifecycle, policy, and eval
|
|
12
|
+
coverage.
|
|
13
13
|
- A registry is the validated set of capability, entity, and eval manifests.
|
|
14
|
-
- A routed slice is the small set of read/prepare capabilities exposed for one
|
|
15
|
-
request.
|
|
14
|
+
- A routed slice is the small set of read/prepare capabilities exposed for one request.
|
|
16
15
|
- Provider adapters turn a slice into provider-specific tool descriptors.
|
|
17
|
-
- Runtime validation maps model tool calls back to capability IDs and validates
|
|
18
|
-
|
|
19
|
-
- Commit stays host-controlled through prepared actions, approvals,
|
|
20
|
-
|
|
16
|
+
- Runtime validation maps model tool calls back to capability IDs and validates args
|
|
17
|
+
before any handler runs.
|
|
18
|
+
- Commit stays host-controlled through prepared actions, approvals, idempotency, audit
|
|
19
|
+
records, and optional controls.
|
|
21
20
|
|
|
22
21
|
See the [Glossary](../glossary.md) for short definitions.
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# Gemini Quickstart
|
|
2
|
+
|
|
3
|
+
Use this guide when your host app wants Google Gemini GenerateContent function calling
|
|
4
|
+
backed by AICF. The default path uses mock clients and descriptor checks. Live calls are
|
|
5
|
+
explicitly opt-in.
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
Core AICF does not require the Google GenAI SDK:
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npm install ai-capability-framework
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Only apps that construct a default Gemini client need the optional SDK:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npm install @google/genai
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Provider imports stay isolated:
|
|
22
|
+
|
|
23
|
+
```ts
|
|
24
|
+
import { runGeminiGenerateContent } from "ai-capability-framework/providers/gemini";
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Mock-Tested Flow
|
|
28
|
+
|
|
29
|
+
Run the Gemini mock runtime tests:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
npm run test:gemini:mock
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Expected output excerpt:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
Test Files ... passed
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
The Gemini adapter converts routed AICF read/prepare capabilities into function
|
|
42
|
+
declarations and maps `functionCall` parts back through AICF bindings.
|
|
43
|
+
|
|
44
|
+
## Live Opt-In
|
|
45
|
+
|
|
46
|
+
Live tests are skipped unless you explicitly opt in:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
RUN_LIVE_GEMINI=1 GEMINI_API_KEY=... AICF_GEMINI_MODEL=gemini-2.5-flash \
|
|
50
|
+
npm run test:gemini:live
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
For live evals:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
RUN_REAL_AICF_LIVE_EVALS=1 GEMINI_API_KEY=... \
|
|
57
|
+
node dist/cli.js eval-live examples --cases cases.json --provider gemini --model gemini-2.5-flash
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
AICF remains the validation, policy, lifecycle, approval, idempotency, and model-safe
|
|
61
|
+
envelope authority. Gemini function declarations are an adapter projection, not the
|
|
62
|
+
source of truth.
|
|
@@ -14,8 +14,8 @@ npm run build
|
|
|
14
14
|
npm run validate
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
-
The default examples use fake data and do not call live model providers. Live
|
|
18
|
-
|
|
19
|
-
|
|
17
|
+
The default examples use fake data and do not call live model providers. Live provider
|
|
18
|
+
tests are opt-in and require explicit environment variables documented in the provider
|
|
19
|
+
guides.
|
|
20
20
|
|
|
21
21
|
Next: [Quickstart](quickstart.md).
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# OpenAI Quickstart
|
|
2
|
+
|
|
3
|
+
Use this guide when your host app wants OpenAI Responses tools backed by AICF. The
|
|
4
|
+
default commands are mock or descriptor checks. Live calls are opt-in.
|
|
5
|
+
|
|
6
|
+
## Install
|
|
7
|
+
|
|
8
|
+
Core AICF does not require the OpenAI SDK:
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
npm install ai-capability-framework
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Only apps that create a default OpenAI client need the optional SDK:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npm install openai
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Root and runtime imports remain lightweight:
|
|
21
|
+
|
|
22
|
+
```ts
|
|
23
|
+
import { loadRegistryFromPath } from "ai-capability-framework";
|
|
24
|
+
import { runOpenAIResponses } from "ai-capability-framework/openai";
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Mock-Tested Flow
|
|
28
|
+
|
|
29
|
+
Run the existing mock runtime tests:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
npm run test:openai:mock
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Expected output excerpt:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
Test Files ... passed
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
The OpenAI runtime exports only routed read/prepare tools. Commit tools remain
|
|
42
|
+
host-controlled through AICF action lifecycle APIs.
|
|
43
|
+
|
|
44
|
+
## Live Opt-In
|
|
45
|
+
|
|
46
|
+
Live tests are skipped unless you explicitly opt in:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
RUN_REAL_OPENAI=1 OPENAI_API_KEY=... AICF_OPENAI_MODEL=gpt-4.1-mini npm run test:openai:live
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
For live evals, the CLI has its own explicit guard:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
RUN_REAL_AICF_LIVE_EVALS=1 OPENAI_API_KEY=... \
|
|
56
|
+
node dist/cli.js eval-live examples --cases cases.json --provider openai --model gpt-4.1-mini
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
AICF remains the validation, policy, lifecycle, approval, idempotency, and model-safe
|
|
60
|
+
envelope authority. OpenAI tool schemas are an adapter projection, not the source of
|
|
61
|
+
truth.
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# Provider-Neutral Quickstart
|
|
2
|
+
|
|
3
|
+
Use this path first if you want to understand AICF without choosing a model provider. It
|
|
4
|
+
uses the public support/refund examples and does not require API keys, provider SDKs,
|
|
5
|
+
live calls, or credentials.
|
|
6
|
+
|
|
7
|
+
## What You Will Prove
|
|
8
|
+
|
|
9
|
+
- Capability manifests can describe app functionality without provider-specific schemas.
|
|
10
|
+
- AICF can validate the registry and inspect the capability set.
|
|
11
|
+
- Deterministic decisions and evals run without calling a model.
|
|
12
|
+
- Provider adapters are optional layers over the same AICF contracts.
|
|
13
|
+
|
|
14
|
+
## 1. Install And Build
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npm ci
|
|
18
|
+
npm run build
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Expected output excerpt:
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
added ... packages
|
|
25
|
+
tsc -p tsconfig.json
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## 2. Inspect A Capability Manifest
|
|
29
|
+
|
|
30
|
+
Open `examples/support/capabilities/support.ticket.get.yaml`. It defines a read-only
|
|
31
|
+
support ticket capability with input/output schemas, permissions, risk, status, and
|
|
32
|
+
lifecycle metadata.
|
|
33
|
+
|
|
34
|
+
The important point is that the manifest is provider-neutral. It is not an OpenAI,
|
|
35
|
+
Anthropic, Gemini, MCP, LangChain, or AI SDK schema.
|
|
36
|
+
|
|
37
|
+
## 3. Validate Public Examples
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
npm run validate
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Expected output excerpt:
|
|
44
|
+
|
|
45
|
+
```text
|
|
46
|
+
Validated 16 manifest(s) and 18 fixture(s).
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## 4. Inspect The Registry
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
node dist/cli.js inspect examples
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Expected output excerpt:
|
|
56
|
+
|
|
57
|
+
```text
|
|
58
|
+
Capabilities:
|
|
59
|
+
- support.ticket.get
|
|
60
|
+
- support.refund.prepare_case
|
|
61
|
+
- support.refund.commit_case
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## 5. Run A Deterministic Decision
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
node dist/cli.js decide examples --request examples/support/decisions/support.ticket.get.select.json
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Expected output excerpt:
|
|
71
|
+
|
|
72
|
+
```text
|
|
73
|
+
allowed: true
|
|
74
|
+
capability: support.ticket.get
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
This proves the policy/selection path without a model. The request uses a synthetic
|
|
78
|
+
user, tenant, permissions, and arguments.
|
|
79
|
+
|
|
80
|
+
## 6. Run Deterministic Evals
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
node dist/cli.js eval examples --results examples/eval-results/public.results.passing.json
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Expected output excerpt:
|
|
87
|
+
|
|
88
|
+
```text
|
|
89
|
+
Eval suite passed
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
The eval runner checks expected selected capabilities, tool calls, arguments,
|
|
93
|
+
approval-required behavior, and no-commit boundaries from public fixtures. It does not
|
|
94
|
+
call a provider.
|
|
95
|
+
|
|
96
|
+
## 7. Export Provider Descriptors Later
|
|
97
|
+
|
|
98
|
+
When you are ready to connect a provider, choose a runtime from
|
|
99
|
+
[Choose a Runtime](../providers/choose-a-runtime.md). The provider adapter will export a
|
|
100
|
+
routed AICF capability slice, but AICF validation and policy remain the authority.
|
|
101
|
+
|
|
102
|
+
Useful next steps:
|
|
103
|
+
|
|
104
|
+
- [OpenAI quickstart](openai-quickstart.md)
|
|
105
|
+
- [Anthropic quickstart](anthropic-quickstart.md)
|
|
106
|
+
- [Gemini quickstart](gemini-quickstart.md)
|
|
@@ -25,8 +25,8 @@ What this proves:
|
|
|
25
25
|
|
|
26
26
|
- manifests and fixtures validate;
|
|
27
27
|
- read/prepare tools can be exported without exposing commit tools;
|
|
28
|
-
- the runtime can read, prepare, require approval, and commit through host
|
|
29
|
-
|
|
28
|
+
- the runtime can read, prepare, require approval, and commit through host lifecycle
|
|
29
|
+
APIs;
|
|
30
30
|
- evals prove expected behavior without calling a model.
|
|
31
31
|
|
|
32
32
|
Full command explanations are in [Start here](../start-here.md).
|
package/docs/getting-started.md
CHANGED
|
@@ -1,29 +1,31 @@
|
|
|
1
1
|
# Getting Started
|
|
2
2
|
|
|
3
3
|
If you are new to AICF, start with [Start here](start-here.md). It uses the
|
|
4
|
-
support/refund example and includes exact commands plus expected output
|
|
5
|
-
excerpts.
|
|
4
|
+
support/refund example and includes exact commands plus expected output excerpts.
|
|
6
5
|
|
|
7
|
-
Use this page as a short checklist when you are ready to model your own
|
|
8
|
-
capability.
|
|
6
|
+
Use this page as a short checklist when you are ready to model your own capability.
|
|
9
7
|
|
|
10
8
|
## Checklist
|
|
11
9
|
|
|
12
10
|
1. Read the [glossary](glossary.md) so the main terms are clear.
|
|
13
|
-
2. Run the
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
5.
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
11
|
+
2. Run the no-key
|
|
12
|
+
[provider-neutral quickstart](getting-started/provider-neutral-quickstart.md).
|
|
13
|
+
3. Run the public support/refund walkthrough in [Start here](start-here.md).
|
|
14
|
+
4. Choose a provider/runtime path with
|
|
15
|
+
[Choose a Runtime](providers/choose-a-runtime.md).
|
|
16
|
+
5. Use the provider quickstart that matches your host app:
|
|
17
|
+
[OpenAI](getting-started/openai-quickstart.md),
|
|
18
|
+
[Anthropic](getting-started/anthropic-quickstart.md), or
|
|
19
|
+
[Gemini](getting-started/gemini-quickstart.md).
|
|
20
|
+
6. Pick one application behavior the model may request.
|
|
21
|
+
7. Write a capability manifest with strict input and output schemas.
|
|
22
|
+
8. Mark risk, side effects, authorization, lifecycle, and approval rules explicitly.
|
|
23
|
+
9. Add synthetic eval cases for selection, arguments, refusal, approval, and no-commit
|
|
24
|
+
boundaries.
|
|
25
|
+
10. Run `npm run validate`.
|
|
26
|
+
11. Export the relevant provider tools from a routed context.
|
|
27
|
+
12. Keep real auth, side effects, approvals, durable storage, and audit in your host
|
|
28
|
+
application.
|
|
27
29
|
|
|
28
30
|
## Useful Commands
|
|
29
31
|
|
|
@@ -31,9 +33,10 @@ capability.
|
|
|
31
33
|
npm run validate
|
|
32
34
|
npm run build
|
|
33
35
|
node dist/cli.js inspect examples
|
|
36
|
+
node dist/cli.js decide examples --request examples/support/decisions/support.ticket.get.select.json
|
|
34
37
|
node dist/cli.js openai-tools examples --context examples/support/openai/context.support_agent.json
|
|
35
38
|
node dist/cli.js eval examples --results examples/eval-results/public.results.passing.json
|
|
36
39
|
```
|
|
37
40
|
|
|
38
|
-
AICF validation and evals are deterministic. They do not call models, execute
|
|
39
|
-
|
|
41
|
+
AICF validation and evals are deterministic. They do not call models, execute side
|
|
42
|
+
effects, or require provider credentials.
|
package/docs/glossary.md
CHANGED
|
@@ -1,71 +1,54 @@
|
|
|
1
1
|
# Glossary
|
|
2
2
|
|
|
3
|
-
Capability:
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
and eval links.
|
|
3
|
+
Capability: An application action or read operation that a model may request. A
|
|
4
|
+
capability has an ID, input/output schemas, risk metadata, lifecycle rules, policy
|
|
5
|
+
gates, and eval links.
|
|
7
6
|
|
|
8
|
-
Manifest:
|
|
9
|
-
|
|
10
|
-
eval, decision, context, and result manifests are validated before use.
|
|
7
|
+
Manifest: A public YAML or JSON file that describes an AICF contract. Capability,
|
|
8
|
+
entity, eval, decision, context, and result manifests are validated before use.
|
|
11
9
|
|
|
12
|
-
Registry:
|
|
13
|
-
The validated in-memory collection of manifests. AICF uses the registry for
|
|
10
|
+
Registry: The validated in-memory collection of manifests. AICF uses the registry for
|
|
14
11
|
routing, provider export, decisions, execution checks, and eval scoring.
|
|
15
12
|
|
|
16
|
-
Slice:
|
|
17
|
-
|
|
18
|
-
the model tool list focused and exclude unsafe or irrelevant capabilities.
|
|
13
|
+
Slice: A small filtered set of capabilities for one request or workflow. Slices keep the
|
|
14
|
+
model tool list focused and exclude unsafe or irrelevant capabilities.
|
|
19
15
|
|
|
20
|
-
Adapter:
|
|
21
|
-
Code that converts AICF capabilities into another provider or framework tool
|
|
16
|
+
Adapter: Code that converts AICF capabilities into another provider or framework tool
|
|
22
17
|
shape. Descriptor adapters do not call models or execute handlers.
|
|
23
18
|
|
|
24
|
-
Provider Runtime:
|
|
25
|
-
|
|
26
|
-
Anthropic, or Gemini. It still routes through AICF validation, policy, and
|
|
19
|
+
Provider Runtime: An optional bounded model/tool loop for a specific provider, such as
|
|
20
|
+
OpenAI, Anthropic, or Gemini. It still routes through AICF validation, policy, and
|
|
27
21
|
runtime envelopes.
|
|
28
22
|
|
|
29
|
-
Handler:
|
|
30
|
-
Host application code registered for a capability. Handlers read data or
|
|
23
|
+
Handler: Host application code registered for a capability. Handlers read data or
|
|
31
24
|
prepare actions; commit handlers are invoked only by the lifecycle manager.
|
|
32
25
|
|
|
33
|
-
Executor:
|
|
34
|
-
|
|
35
|
-
calls the registered handler, validates output, and returns a model-safe
|
|
36
|
-
envelope.
|
|
26
|
+
Executor: The runtime component that validates a read or prepare request, checks policy,
|
|
27
|
+
calls the registered handler, validates output, and returns a model-safe envelope.
|
|
37
28
|
|
|
38
|
-
Lifecycle Manager:
|
|
39
|
-
|
|
40
|
-
idempotency, commits stored prepared actions, and emits audit summaries.
|
|
29
|
+
Lifecycle Manager: The runtime component that prepares actions, records approvals,
|
|
30
|
+
checks idempotency, commits stored prepared actions, and emits audit summaries.
|
|
41
31
|
|
|
42
|
-
Prepared Action:
|
|
43
|
-
|
|
44
|
-
validated arguments and summary needed for approval and later commit.
|
|
32
|
+
Prepared Action: A stored action preview created before side effects happen. It contains
|
|
33
|
+
the validated arguments and summary needed for approval and later commit.
|
|
45
34
|
|
|
46
|
-
Approval:
|
|
47
|
-
A host-controlled decision that allows or rejects a prepared action. AICF can
|
|
35
|
+
Approval: A host-controlled decision that allows or rejects a prepared action. AICF can
|
|
48
36
|
record and check approvals, but the host owns the approval UI and real authority.
|
|
49
37
|
|
|
50
|
-
Idempotency:
|
|
51
|
-
|
|
52
|
-
return an existing result instead of repeating a side effect.
|
|
38
|
+
Idempotency: A key-based protection against duplicate commits. It lets the lifecycle
|
|
39
|
+
manager return an existing result instead of repeating a side effect.
|
|
53
40
|
|
|
54
|
-
Envelope:
|
|
55
|
-
|
|
56
|
-
success, validation errors, denials, approval-required pauses, unavailable
|
|
41
|
+
Envelope: A JSON-serializable result wrapper returned to the model or host. Envelopes
|
|
42
|
+
make success, validation errors, denials, approval-required pauses, unavailable
|
|
57
43
|
handlers, and failures explicit.
|
|
58
44
|
|
|
59
|
-
Eval:
|
|
60
|
-
|
|
61
|
-
evals do not call models; they score candidate result fixtures.
|
|
45
|
+
Eval: A deterministic test case that describes expected model/tool behavior. AICF evals
|
|
46
|
+
do not call models; they score candidate result fixtures.
|
|
62
47
|
|
|
63
|
-
Candidate Result:
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
text without raw provider payloads.
|
|
48
|
+
Candidate Result: A public-safe summary of observed behavior for an eval case. It
|
|
49
|
+
records selected capabilities, tool calls, policy decisions, action state, refusal, and
|
|
50
|
+
response text without raw provider payloads.
|
|
67
51
|
|
|
68
|
-
Conformance:
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
errors, and no commit exposure.
|
|
52
|
+
Conformance: A descriptor/mock check that one routed capability slice maps consistently
|
|
53
|
+
across providers and frameworks. It verifies names, bindings, schemas, safe errors, and
|
|
54
|
+
no commit exposure.
|
package/docs/governance/gate.md
CHANGED
|
@@ -1,15 +1,13 @@
|
|
|
1
1
|
# Governance Gate
|
|
2
2
|
|
|
3
3
|
`aicf gate` is a CI-friendly command that runs the public AICF checks in one
|
|
4
|
-
deterministic pass. It coordinates validation, semantic invariants, governance
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
hygiene.
|
|
4
|
+
deterministic pass. It coordinates validation, semantic invariants, governance risk,
|
|
5
|
+
lifecycle posture, optional compatibility baselines, impact analysis, eval coverage,
|
|
6
|
+
security-pack coverage, provider conformance, and public artifact hygiene.
|
|
8
7
|
|
|
9
8
|
The gate does not call models, run live provider SDKs, mutate manifests, execute
|
|
10
|
-
handlers, write production stores, or expose commit tools to models.
|
|
11
|
-
|
|
12
|
-
checking only.
|
|
9
|
+
handlers, write production stores, or expose commit tools to models. It does not call
|
|
10
|
+
live integrations; provider conformance is descriptor/mock checking only.
|
|
13
11
|
|
|
14
12
|
## Basic Usage
|
|
15
13
|
|
|
@@ -26,17 +24,17 @@ node dist/cli.js gate examples --env production --format json
|
|
|
26
24
|
|
|
27
25
|
Exit codes are stable:
|
|
28
26
|
|
|
29
|
-
| Code | Meaning
|
|
30
|
-
|
|
|
31
|
-
| `0`
|
|
32
|
-
| `1`
|
|
33
|
-
| `2`
|
|
34
|
-
| `3`
|
|
35
|
-
| `4`
|
|
36
|
-
| `5`
|
|
27
|
+
| Code | Meaning |
|
|
28
|
+
| ---- | --------------------------------------------------------------- |
|
|
29
|
+
| `0` | Gate passed. |
|
|
30
|
+
| `1` | Validation or gate check failed. |
|
|
31
|
+
| `2` | Usage or configuration error. |
|
|
32
|
+
| `3` | Reserved for missing optional dependencies. |
|
|
33
|
+
| `4` | Reserved for disabled live integrations or missing credentials. |
|
|
34
|
+
| `5` | Unexpected internal error. |
|
|
37
35
|
|
|
38
|
-
F8 does not run live integrations, so codes `3` and `4` are reserved for future
|
|
39
|
-
|
|
36
|
+
F8 does not run live integrations, so codes `3` and `4` are reserved for future optional
|
|
37
|
+
paths or errors surfaced by existing optional helpers.
|
|
40
38
|
|
|
41
39
|
## Configuration
|
|
42
40
|
|
|
@@ -70,9 +68,9 @@ gates:
|
|
|
70
68
|
artifact_hygiene: true
|
|
71
69
|
```
|
|
72
70
|
|
|
73
|
-
Enabled providers are descriptor/mock conformance targets. Provider aliases use
|
|
74
|
-
|
|
75
|
-
|
|
71
|
+
Enabled providers are descriptor/mock conformance targets. Provider aliases use the same
|
|
72
|
+
normalization as provider conformance, for example `vercel-ai-sdk` maps to `ai-sdk` and
|
|
73
|
+
`semantic-kernel` maps to `semantic-kernel-openapi`.
|
|
76
74
|
|
|
77
75
|
## Production Defaults
|
|
78
76
|
|
|
@@ -80,14 +78,14 @@ When no config exists, the production gate uses conservative defaults:
|
|
|
80
78
|
|
|
81
79
|
- warnings do not fail the gate unless `--fail-on-warnings` is supplied;
|
|
82
80
|
- medium, high, and critical capabilities require eval coverage;
|
|
83
|
-
- high and critical capabilities require assigned, generated, or waived
|
|
84
|
-
|
|
81
|
+
- high and critical capabilities require assigned, generated, or waived security-pack
|
|
82
|
+
coverage;
|
|
85
83
|
- configured providers must pass conformance;
|
|
86
84
|
- deprecated capabilities are blocked;
|
|
87
85
|
- public artifact hygiene is enabled.
|
|
88
86
|
|
|
89
|
-
Compatibility checks run only when `--baseline <path>` or
|
|
90
|
-
|
|
87
|
+
Compatibility checks run only when `--baseline <path>` or `compatibility.baseline_root`
|
|
88
|
+
is provided.
|
|
91
89
|
|
|
92
90
|
## CLI Options
|
|
93
91
|
|
|
@@ -101,10 +99,10 @@ aicf gate <manifest-root> --env <name> \
|
|
|
101
99
|
[--no-artifact-hygiene]
|
|
102
100
|
```
|
|
103
101
|
|
|
104
|
-
Use `--no-artifact-hygiene` only for narrow local debugging. Release checks
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
102
|
+
Use `--no-artifact-hygiene` only for narrow local debugging. Release checks should keep
|
|
103
|
+
artifact hygiene enabled so private paths, traces, prompts, raw provider payloads,
|
|
104
|
+
archives, Office/PDF artifacts, credential-looking files, and local-only material are
|
|
105
|
+
rejected.
|
|
108
106
|
|
|
109
107
|
## TypeScript
|
|
110
108
|
|
|
@@ -133,5 +131,5 @@ node dist/cli.js gate examples --env production --format json > gate-report.json
|
|
|
133
131
|
node dist/cli.js evidence export examples --out evidence-pack.json --gate-report gate-report.json
|
|
134
132
|
```
|
|
135
133
|
|
|
136
|
-
The evidence pack records gate, eval, security, and conformance gaps explicitly
|
|
137
|
-
|
|
134
|
+
The evidence pack records gate, eval, security, and conformance gaps explicitly when
|
|
135
|
+
reports are not supplied.
|
|
@@ -1,9 +1,8 @@
|
|
|
1
1
|
# Impact And Compatibility
|
|
2
2
|
|
|
3
3
|
Compatibility checks compare two capability manifests and classify changes as
|
|
4
|
-
compatible, minor, or breaking. Impact analysis reports direct registry
|
|
5
|
-
|
|
6
|
-
provider export relevance.
|
|
4
|
+
compatible, minor, or breaking. Impact analysis reports direct registry coverage across
|
|
5
|
+
entities, evals, prepare/commit links, policy references, and provider export relevance.
|
|
7
6
|
|
|
8
7
|
Use:
|
|
9
8
|
|