darwin-agentic-cloud 0.1.0__tar.gz → 3.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- darwin_agentic_cloud-3.0.0/CHANGELOG.md +108 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/CONTRIBUTING.md +76 -0
- darwin_agentic_cloud-3.0.0/DARWIN_AGENTIC_CLOUD_PHASE_2_PROGRESS_2of3.md +273 -0
- darwin_agentic_cloud-3.0.0/DARWIN_AGENTIC_CLOUD_PHASE_2_SPEC.md +400 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/Dockerfile +8 -0
- darwin_agentic_cloud-3.0.0/LAUNCH_POST_v3.0.0.md +96 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/PKG-INFO +30 -36
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/QUICKSTART.md +41 -62
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/README.md +26 -35
- darwin_agentic_cloud-3.0.0/darwin/__init__.py +32 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/__init__.py +1 -1
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/admin_cli.py +275 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/class_keys.py +434 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/cli.py +1481 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/mcp_server.py +393 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/router.py +364 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/runtime_v02.py +253 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/schemas/__init__.py +1 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/schemas/attestation_v0_2.json +229 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/server.py +502 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/__init__.py +41 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/aws_batch.py +870 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/aws_batch_event.py +320 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/aws_lambda.py +657 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/aws_lambda_event.py +277 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/aws_pricing.py +199 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/base.py +525 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/identity.py +448 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/local_docker.py +306 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/substrate/modal.py +473 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/templates/demo.html +614 -0
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/templates/demo_attestation.json +51 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/templates/docs.html +3 -2
- darwin_agentic_cloud-3.0.0/darwin/agenticcloud/ui.py +687 -0
- darwin_agentic_cloud-3.0.0/docker-entrypoint.sh +66 -0
- darwin_agentic_cloud-3.0.0/docs/AGENT_INVENTORY.md +172 -0
- darwin_agentic_cloud-3.0.0/examples/hello.py +1 -0
- darwin_agentic_cloud-3.0.0/infra/__init__.py +0 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/Dockerfile.batch-runner-python +29 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/Dockerfile.runner-node +14 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/Dockerfile.runner-python +18 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/__init__.py +0 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/batch_deploy.py +701 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/batch_runner.py +353 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/deploy.py +485 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/runner.mjs +157 -0
- darwin_agentic_cloud-3.0.0/infra/aws_runner/runner.py +256 -0
- darwin_agentic_cloud-3.0.0/infra/bake_demo_attestation.py +92 -0
- darwin_agentic_cloud-3.0.0/infra/batch_smoketest.py +125 -0
- darwin_agentic_cloud-3.0.0/infra/modal_smoketest.py +36 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/pyproject.toml +9 -1
- darwin_agentic_cloud-3.0.0/tests/fixtures/ec2_ondemand_m5xlarge.json +7 -0
- darwin_agentic_cloud-3.0.0/tests/fixtures/ec2_spot_history_m5xlarge.json +40 -0
- darwin_agentic_cloud-3.0.0/tests/runtime/__init__.py +0 -0
- darwin_agentic_cloud-3.0.0/tests/runtime/test_router.py +337 -0
- darwin_agentic_cloud-3.0.0/tests/runtime/test_runtime_v02.py +187 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_aws_batch.py +415 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_aws_lambda.py +777 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_aws_pricing.py +277 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_base.py +597 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_identity.py +617 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_local_docker.py +565 -0
- darwin_agentic_cloud-3.0.0/tests/substrate/test_modal.py +396 -0
- darwin_agentic_cloud-3.0.0/tests/test_admin_cli.py +410 -0
- darwin_agentic_cloud-3.0.0/tests/test_class_keys.py +331 -0
- darwin_agentic_cloud-3.0.0/tests/test_server_signing.py +387 -0
- darwin_agentic_cloud-3.0.0/tests/test_ui.py +268 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/uv.lock +736 -1
- darwin_agentic_cloud-0.1.0/CHANGELOG.md +0 -55
- darwin_agentic_cloud-0.1.0/darwin/__init__.py +0 -10
- darwin_agentic_cloud-0.1.0/darwin/agenticcloud/cli.py +0 -502
- darwin_agentic_cloud-0.1.0/darwin/agenticcloud/mcp_server.py +0 -303
- darwin_agentic_cloud-0.1.0/darwin/agenticcloud/server.py +0 -230
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/.dockerignore +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/.gitignore +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/CODE_OF_CONDUCT.md +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/LICENSE +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/Makefile +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/SECURITY.md +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/attestation.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/cost.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/hashing.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/runtime.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/sandbox.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/signing.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/storage.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/darwin/agenticcloud/types.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/examples/example_workloads/hello.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/fly.toml +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/tests/__init__.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/tests/test_attestation.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/tests/test_cost.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/tests/test_smoke.py +0 -0
- {darwin_agentic_cloud-0.1.0 → darwin_agentic_cloud-3.0.0}/tests/test_storage.py +0 -0
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [3.0.0] — 2026-05-27
|
|
11
|
+
|
|
12
|
+
The verifiable-compute toll-booth. This release introduces the v0.2 attestation schema,
|
|
13
|
+
the multi-substrate router, and a clean seven-verb public CLI/MCP surface that mirrors
|
|
14
|
+
the Apple-grade `darwin.run()` agent API.
|
|
15
|
+
|
|
16
|
+
### Highlights
|
|
17
|
+
|
|
18
|
+
- **`from darwin import run` — one-line agent API.** Auto-discovers available substrates,
|
|
19
|
+
routes to the cheapest one by default, executes workload, returns a cryptographically
|
|
20
|
+
signed v0.2 attestation.
|
|
21
|
+
- **v0.2 attestation schema** at `darwin.cloud/agenticcloud/attestation/v0.2`. Adds the
|
|
22
|
+
`value_added_service` block (cost-cap enforcement, routing decision, identity-signing
|
|
23
|
+
metadata) on top of the substrate execution receipt.
|
|
24
|
+
- **Multi-substrate routing.** Workloads can run on `local-docker`, `aws-lambda`
|
|
25
|
+
(4 regions), `modal`, or `aws-batch-ec2-spot-v0-us-east-1`. The router picks by cost,
|
|
26
|
+
by explicit substrate, or by capability. Auto-discovery checks credentials and
|
|
27
|
+
daemon availability at call time.
|
|
28
|
+
- **AWS Batch substrate adapter.** Real EC2 Spot pricing via `DescribeSpotPriceHistory`,
|
|
29
|
+
60-second minimum billing, soft-fail on pricing API unreachability. Substrate identity
|
|
30
|
+
signed by the hosted Darwin class key.
|
|
31
|
+
- **Engraved-certificate panel.** v0.2 attestations render as a Treasury-style
|
|
32
|
+
certificate: double-line borders, gold seals (`✦`), diamond dividers, motto
|
|
33
|
+
(`SECURITAS · STABILITAS · SIGNUM`), and full-center alignment. The cert reads as
|
|
34
|
+
the receipt agents actually need.
|
|
35
|
+
- **HTML demo page** at `/demo` with a live signed attestation, downloadable JSON
|
|
36
|
+
artifact, and three-step verification instructions.
|
|
37
|
+
- **Public v0.2 JSON Schema** (Draft 07) at `/.well-known/schemas/attestation/v0.2`
|
|
38
|
+
for self-describing attestations.
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- `darwin/__init__.py` re-exports `run`, `Runtime`, `CostCapExceeded` for ergonomic
|
|
43
|
+
imports.
|
|
44
|
+
- `darwin.agenticcloud.runtime_v02.Runtime` — auto-discovering runtime that constructs
|
|
45
|
+
the v0.2 attestation, validates against the schema registry, and signs the outer
|
|
46
|
+
envelope.
|
|
47
|
+
- `darwin.agenticcloud.router` — substrate discovery + routing policies
|
|
48
|
+
(`pick_by_cost`, `pick_by_substrate`, `pick_by_capability`). Short-name resolution
|
|
49
|
+
(`aws-batch` → `aws-batch-ec2-spot-v0-us-east-1` using `AWS_REGION`).
|
|
50
|
+
- `darwin.agenticcloud.substrate.aws_batch` — full AWS Batch (EC2 Spot) substrate
|
|
51
|
+
adapter with idempotent `boto3` deploy orchestrator (`infra/aws_runner/batch_deploy.py`).
|
|
52
|
+
- Seven top-level CLI verbs: `run`, `verify`, `price`, `list`, `sign`, `try`, `who`.
|
|
53
|
+
- Seven MCP tools mirroring the CLI: `darwin_run`, `darwin_verify`, `darwin_price`,
|
|
54
|
+
`darwin_list`, `darwin_who`, `darwin_history`, `darwin_stats`.
|
|
55
|
+
- `infra/bake_demo_attestation.py` — bake-time generator for the hosted demo page.
|
|
56
|
+
- Pre-flight cost-cap enforcement (was in [Unreleased]). Workloads whose preflight
|
|
57
|
+
estimate exceeds `cost_cap_usd` are rejected before any sandbox is launched;
|
|
58
|
+
the rejection is signed.
|
|
59
|
+
- 79 new tests across substrate adapters, router, runtime, pricing client, schema
|
|
60
|
+
validation. Total project tests: 388 passing.
|
|
61
|
+
|
|
62
|
+
### Changed
|
|
63
|
+
|
|
64
|
+
- `darwin run` is now the v0.2 entry point. Accepts code-as-string OR file path
|
|
65
|
+
(heuristic: if the argument exists as a file, read it; otherwise execute inline).
|
|
66
|
+
- Legacy v0.1 `run` preserved at `darwin attest run-v01`.
|
|
67
|
+
- `darwin substrates demo` now executes real substrates by default; `--mock`
|
|
68
|
+
preserves the prior synthetic behavior.
|
|
69
|
+
- Attestation panel renderer redesigned around the engraved-certificate aesthetic.
|
|
70
|
+
- `SUBSTRATE_KEYLIST_URL` updated to `https://darwin-agentic-cloud.fly.dev` until
|
|
71
|
+
DNS for `darwin.cloud` is provisioned.
|
|
72
|
+
- MCP tool surface rewritten from `dac_*` prefix to `darwin_*`. Old tool names
|
|
73
|
+
removed (breaking change for MCP client configs).
|
|
74
|
+
|
|
75
|
+
### Removed
|
|
76
|
+
|
|
77
|
+
- v0.1-only MCP tools (`dac_run_python`, `dac_run_node`, etc.). Their behavior is
|
|
78
|
+
preserved via the new `darwin_run` tool with optional `language` parameter.
|
|
79
|
+
|
|
80
|
+
## [0.1.0] — initial alpha
|
|
81
|
+
|
|
82
|
+
### Added
|
|
83
|
+
|
|
84
|
+
- **Attestation layer.** Ed25519-signed, canonical-JSON-hashed, tamper-evident
|
|
85
|
+
attestations binding workload spec, execution result, substrate identity,
|
|
86
|
+
cost, and signer identity.
|
|
87
|
+
- **Docker sandbox.** Memory limits, CPU limits, pids limit, no network,
|
|
88
|
+
read-only-friendly tmpfs, no-new-privileges, non-root user (65534:65534),
|
|
89
|
+
all capabilities dropped. Python and Node.js runtimes.
|
|
90
|
+
- **Runtime orchestrator.** Single entry point: `Runtime().run(spec)`
|
|
91
|
+
returns a `SignedAttestation`.
|
|
92
|
+
- **CLI** (`dac`): `run`, `attest verify`, `attest show`, `keys init`,
|
|
93
|
+
`keys show`, `serve`, `mcp serve`, `version`.
|
|
94
|
+
- **HTTP server** (FastAPI): `GET /healthz`, `GET /v0/identity`,
|
|
95
|
+
`POST /v0/run`, `POST /v0/attestations/verify`. Auto-generated OpenAPI
|
|
96
|
+
docs at `/docs`.
|
|
97
|
+
- **MCP server**: stdio transport, four tools (`dac_run_python`,
|
|
98
|
+
`dac_run_node`, `dac_verify_attestation`, `dac_identity`). Verified
|
|
99
|
+
end-to-end with Claude Desktop.
|
|
100
|
+
- Apache 2.0 license, contributing guide with DCO sign-off, security
|
|
101
|
+
policy, code of conduct, issue/PR templates.
|
|
102
|
+
- 16 attestation tests proving tamper-evidence under every tampering
|
|
103
|
+
vector (stdout, cost, workload spec, signer key id, public key,
|
|
104
|
+
signature), JSON round-trip, deepcopy survival, and malformed input
|
|
105
|
+
rejection.
|
|
106
|
+
|
|
107
|
+
[Unreleased]: https://github.com/vje013/darwin-agentic-cloud/compare/v0.1.0...HEAD
|
|
108
|
+
[0.1.0]: https://github.com/vje013/darwin-agentic-cloud/releases/tag/v0.1.0
|
|
@@ -110,6 +110,82 @@ Do **not** open public issues for security vulnerabilities. See [SECURITY.md](SE
|
|
|
110
110
|
- Documentation and examples
|
|
111
111
|
- Performance benchmarks
|
|
112
112
|
|
|
113
|
+
## For Autonomous Agents
|
|
114
|
+
|
|
115
|
+
This section is for AI coding agents (Copilot, Sweep, Devin, Aider, Codex CLI, etc.) contributing to Darwin Agentic Cloud. Human contributors should follow the standard flow above.
|
|
116
|
+
|
|
117
|
+
### DCO sign-off
|
|
118
|
+
|
|
119
|
+
All commits — including those from agents — **must** include DCO sign-off. Use the `-s` flag:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
git commit -s -m "feat: add foo"
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Agents that cannot sign off interactively should include the `Signed-off-by:` trailer in the commit message body:
|
|
126
|
+
|
|
127
|
+
```text
|
|
128
|
+
feat(sandbox): add gVisor support
|
|
129
|
+
|
|
130
|
+
Signed-off-by: Agent Name <agent@example.com>
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
### Branch naming
|
|
134
|
+
|
|
135
|
+
Agent branches must follow the pattern:
|
|
136
|
+
|
|
137
|
+
```text
|
|
138
|
+
<agent>/<short-description>
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Examples:
|
|
142
|
+
- `copilot/fix-attestation-hash`
|
|
143
|
+
- `sweep/add-bun-runtime`
|
|
144
|
+
- `devin/otel-logging`
|
|
145
|
+
- `aider/benchmark-harness`
|
|
146
|
+
|
|
147
|
+
### PR template
|
|
148
|
+
|
|
149
|
+
Agent-opened PRs must include:
|
|
150
|
+
|
|
151
|
+
1. **Title:** Conventional Commit style (e.g., `feat(sandbox): add Bun runtime`)
|
|
152
|
+
2. **Body:**
|
|
153
|
+
- What problem this solves (reference the issue number)
|
|
154
|
+
- What changed (brief summary)
|
|
155
|
+
- How to test (commands to run)
|
|
156
|
+
3. **Labels:** Add relevant labels (`agent-contributed`, plus area labels)
|
|
157
|
+
4. **Linked issue:** Reference the triggering issue with `Closes #N` or `Fixes #N`
|
|
158
|
+
|
|
159
|
+
### CI requirements
|
|
160
|
+
|
|
161
|
+
All PRs must pass before merge:
|
|
162
|
+
|
|
163
|
+
- `make lint` — Ruff linting (zero warnings)
|
|
164
|
+
- `make typecheck` — mypy strict mode
|
|
165
|
+
- `make test-unit` — Unit tests (no Docker required)
|
|
166
|
+
- `make test-integration` — Integration tests (Docker required, runs in CI)
|
|
167
|
+
|
|
168
|
+
If CI fails, agents should attempt to fix up to 3 times. After 3 failures, add the `needs-human` label.
|
|
169
|
+
|
|
170
|
+
### Mention syntax reference
|
|
171
|
+
|
|
172
|
+
| Agent | How to invoke |
|
|
173
|
+
|-------|---------------|
|
|
174
|
+
| Copilot Coding Agent | `@copilot` in issue/PR comments |
|
|
175
|
+
| Sweep | Issue title prefix `sweep:` or `/sweep` comment |
|
|
176
|
+
| Devin | `@devin-ai-integration` in comments |
|
|
177
|
+
| PR-Agent | `/review`, `/improve`, `/describe` in PR comments |
|
|
178
|
+
|
|
179
|
+
For full details on each agent's capabilities, see [`docs/AGENT_INVENTORY.md`](docs/AGENT_INVENTORY.md).
|
|
180
|
+
|
|
181
|
+
### Good-first-issue guidelines for agents
|
|
182
|
+
|
|
183
|
+
Issues labeled `good first issue` + `agent-friendly` are designed for autonomous agents:
|
|
184
|
+
- Self-contained with clear acceptance criteria
|
|
185
|
+
- Include pointers to relevant source files
|
|
186
|
+
- Specify expected test coverage
|
|
187
|
+
- Reference any relevant RFCs or design docs
|
|
188
|
+
|
|
113
189
|
## License
|
|
114
190
|
|
|
115
191
|
By contributing, you agree that your contributions will be licensed under the [Apache License 2.0](LICENSE).
|
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
# Darwin Agentic Cloud --- Build Specification (Phase 2, 2/3 progress)
|
|
2
|
+
|
|
3
|
+
Second-third checkpoint document for Phase 2 of Darwin Agentic Cloud. Picks up where DARWIN_AGENTIC_CLOUD_PHASE_2_SPEC.md left off.
|
|
4
|
+
|
|
5
|
+
## 1. What This Checkpoint Covers
|
|
6
|
+
|
|
7
|
+
The first-half checkpoint locked Phase 2 commitments, the substrate ABC, the local-docker port, the aws-lambda adapter, the hosted-signer plumbing, and the Fly ceremony for local-docker-v0. The build continued into the actual AWS deployment, the four-region class-key rollout, the Modal substrate adapter, and the test-suite expansion.
|
|
8
|
+
|
|
9
|
+
## 2. Connection to Prior Work
|
|
10
|
+
|
|
11
|
+
Phase 1 ended with v2.0.0 on PyPI: single-substrate (local-docker-v0) Python package, v0.1 attestations, Fly hosted demo, 55 tests passing. First-half of Phase 2 ended with substrate ABC, identity layer, two reference adapters (local-docker and aws-lambda), hosted-signer plumbing, entrypoint bootstrap script, and the public keylist endpoint serving the first class key.
|
|
12
|
+
|
|
13
|
+
This 2/3 stretch did not break any of that. Phase 1 abstractions (WorkloadSpec, Substrate, Attestation, Runtime) remained unchanged. The v0.1 attestation path still works for callers that have not migrated. The original local-docker-v0 class key was not rotated. The 55 Phase 1 tests continued to pass through every PR.
|
|
14
|
+
|
|
15
|
+
## 3. What Connects to What is Next
|
|
16
|
+
|
|
17
|
+
The remaining Phase 2 work has hard dependencies on what landed in this stretch:
|
|
18
|
+
|
|
19
|
+
- Akash substrate (next) follows the same adapter pattern as aws_lambda.py and modal.py
|
|
20
|
+
- Router (runtime/router.py) calls each substrate preflight and compares CostEstimate.cost_usd_max
|
|
21
|
+
- Runtime rewire takes the current Phase 1 runtime path and parameterizes it on a Substrate instance
|
|
22
|
+
- CLI --substrate flag uses the substrate ID strings defined in ALLOWED_SUBSTRATES
|
|
23
|
+
- Hosted demo upgrade invokes the deployed AWS Lambda runners via the substrate path
|
|
24
|
+
- Ship v3.0.0 packages everything together
|
|
25
|
+
|
|
26
|
+
The 2/3 work made the remaining work mechanical rather than architectural.
|
|
27
|
+
|
|
28
|
+
## 4. AWS Lambda Runner: Deployed Live
|
|
29
|
+
|
|
30
|
+
### 4.1 Runner Code
|
|
31
|
+
|
|
32
|
+
infra/aws_runner/runner.py (Python) and infra/aws_runner/runner.mjs (Node) are the Lambda handlers. They share a schema contract:
|
|
33
|
+
|
|
34
|
+
- Event schema URI: darwin.cloud/event/aws-lambda-runner/v1
|
|
35
|
+
- Shared module: darwin.agenticcloud.substrate.aws_lambda_event
|
|
36
|
+
- Validation: both handlers re-implement the validator independently --- the schema is the contract, not the import
|
|
37
|
+
|
|
38
|
+
Both handlers were smoke-tested locally before deploy. The same workload produced the same output_hash (74c738815abf786c...) on both Python and Node runners. The schema contract holds across languages.
|
|
39
|
+
|
|
40
|
+
### 4.2 Deploy Orchestrator
|
|
41
|
+
|
|
42
|
+
infra/aws_runner/deploy.py is a 485-line idempotent boto3 script. The choice to use plain boto3 rather than CDK saved a tooling dependency and matched the rest of the codebase.
|
|
43
|
+
|
|
44
|
+
Resources created per region:
|
|
45
|
+
|
|
46
|
+
- ECR repository (Python): darwin-runner-python
|
|
47
|
+
- ECR repository (Node): darwin-runner-node
|
|
48
|
+
- Lambda function (Python): darwin-runner-python-REGION
|
|
49
|
+
- Lambda function (Node): darwin-runner-node-REGION
|
|
50
|
+
- IAM role: darwin-runner-execution-role (created once globally, reused)
|
|
51
|
+
|
|
52
|
+
### 4.3 Live Deployment
|
|
53
|
+
|
|
54
|
+
Deploy ran against AWS account 529088294890 across four regions. All four completed successfully:
|
|
55
|
+
|
|
56
|
+
- us-east-1: Python sha256:e0647a8e..., Node sha256:6be06af1..., Active
|
|
57
|
+
- us-west-2: Python sha256:e0647a8e..., Node sha256:6be06af1..., Active
|
|
58
|
+
- eu-west-1: Python sha256:e0647a8e..., Node sha256:6be06af1..., Active
|
|
59
|
+
- ap-northeast-1: Python sha256:e0647a8e..., Node sha256:6be06af1..., Active
|
|
60
|
+
|
|
61
|
+
Image digests are identical across all four regions. Auditors can verify the same code runs in every region.
|
|
62
|
+
|
|
63
|
+
### 4.4 Real Costs
|
|
64
|
+
|
|
65
|
+
- Lambda function idle: 0 dollars
|
|
66
|
+
- Per invocation in us-east-1, us-west-2: 0.0000167 dollars per GB-second
|
|
67
|
+
- Per invocation in eu-west-1: 0.0000183 dollars per GB-second
|
|
68
|
+
- Per invocation in ap-northeast-1: 0.0000200 dollars per GB-second
|
|
69
|
+
- Lambda request fee: 0.0000002 dollars per request
|
|
70
|
+
- ECR storage: ~0.05 dollars per repo per month
|
|
71
|
+
- IAM role: 0 dollars
|
|
72
|
+
|
|
73
|
+
Total idle cost across 4 regions: ~0.40 dollars per month (just ECR storage).
|
|
74
|
+
|
|
75
|
+
## 5. AWS Lambda Class Keys: Generated and Live
|
|
76
|
+
|
|
77
|
+
Four aws-lambda-REGION class keys were generated, uploaded to Fly, and the entrypoint script updated:
|
|
78
|
+
|
|
79
|
+
- aws-lambda-us-east-1: dac-class-aws-lambda-us-east-1-4104384a41b714cc
|
|
80
|
+
- aws-lambda-us-west-2: dac-class-aws-lambda-us-west-2-0ff057e0ad590ce3
|
|
81
|
+
- aws-lambda-eu-west-1: dac-class-aws-lambda-eu-west-1-90dab9b6c9eddfed
|
|
82
|
+
- aws-lambda-ap-northeast-1: dac-class-aws-lambda-ap-northeast-1-7b19728bbbe00ae5
|
|
83
|
+
|
|
84
|
+
After Fly redeployed with the updated entrypoint, the public keylist endpoint reflects 5 active keys: the original local-docker-v0 key plus the four aws-lambda regions. Five class keys, five substrates, publicly verifiable. Anyone can fetch them and verify any attestation hosted Darwin produces for those substrates.
|
|
85
|
+
|
|
86
|
+
The private parts of all five keys exist only on Fly. Local PEM files were generated, uploaded, and deleted immediately.
|
|
87
|
+
|
|
88
|
+
## 6. Modal Substrate: Coded and Verified
|
|
89
|
+
|
|
90
|
+
darwin/agenticcloud/substrate/modal.py is the third substrate adapter. It uses the ephemeral-sandbox model: each workload spawns a fresh modal.Sandbox.create(), runs to completion, tears down.
|
|
91
|
+
|
|
92
|
+
### 6.1 Architecture Decision
|
|
93
|
+
|
|
94
|
+
The choice between pre-deployed runner (mirroring aws-lambda) and ephemeral sandbox was made deliberately for ephemeral:
|
|
95
|
+
|
|
96
|
+
- Modal sandbox product was built for "run untrusted code." Darwin use case.
|
|
97
|
+
- No deploy ceremony to repeat. pip install + Modal token = ready.
|
|
98
|
+
- Cleaner threat model: zero state bleed between workloads.
|
|
99
|
+
- Latency cost (3-8s cold start vs ~100ms warm) is acceptable for v3.0.0.
|
|
100
|
+
|
|
101
|
+
### 6.2 Adapter Details
|
|
102
|
+
|
|
103
|
+
- Substrate ID: modal-v0
|
|
104
|
+
- Substrate version: 0.1.0
|
|
105
|
+
- Evidence schema URI: darwin.cloud/evidence/modal/v1
|
|
106
|
+
- Wholesale cost rate: 0.000131 dollars per wall-second (0.125 CPU cores)
|
|
107
|
+
- Cold-start overhead: ~5 seconds, included in preflight estimate
|
|
108
|
+
- Max wall time: 900 seconds
|
|
109
|
+
- Supported languages: python, node
|
|
110
|
+
- Default Python image: python:3.12-slim
|
|
111
|
+
- Default Node image: node:20-slim
|
|
112
|
+
|
|
113
|
+
### 6.3 Real-Modal Verification
|
|
114
|
+
|
|
115
|
+
infra/modal_smoketest.py ran the adapter against a real Modal account. One sandbox spawned, one workload executed, one attestation evidence dict produced. Real run:
|
|
116
|
+
|
|
117
|
+
- sandbox_id: sb-TLfkXYsE2cjWNGfZsrfaJo
|
|
118
|
+
- task_id: modal-task-a83afda3b6dc
|
|
119
|
+
- image_tag: python:3.12-slim
|
|
120
|
+
- exit_code: 0
|
|
121
|
+
- container_status: ok
|
|
122
|
+
- wall_time_sec: 2.615875005722046
|
|
123
|
+
- stdout_hash: sha256:5c779f64bc689227bb1b90db557c3325a65e05eccd22bf6581c3967449bead08
|
|
124
|
+
- cost_usd: 0.000343
|
|
125
|
+
|
|
126
|
+
First time a substrate other than local-docker has run a real workload against a real cloud account and produced a real attestation evidence dict. Three substrate types (local Docker, AWS Lambda, Modal) all produce evidence conforming to v0.2 schema. Toll-booth promise is real, not aspirational.
|
|
127
|
+
|
|
128
|
+
## 7. Test Suite Growth
|
|
129
|
+
|
|
130
|
+
First-half count: 278 tests. 2/3 count: 309 tests (+31 from Modal). Every test continued to pass through every PR merge.
|
|
131
|
+
|
|
132
|
+
Two test suites needed updates to accommodate the expanded ALLOWED_SUBSTRATES allowlist (test_class_keys.py and test_admin_cli.py). Both updated to use e2b-v0 as the not-allowlisted fixture.
|
|
133
|
+
|
|
134
|
+
## 8. Architecture Decisions Locked in This Stretch
|
|
135
|
+
|
|
136
|
+
- boto3 deploy orchestrator over CDK (avoid new tooling dep, matches codebase)
|
|
137
|
+
- Two Lambda functions per region (cleaner isolation, smaller images, separate audit)
|
|
138
|
+
- Same darwin-runner image content across all 4 regions (verifiable consistency)
|
|
139
|
+
- Modal ephemeral sandbox over pre-deployed runner (stronger isolation, Modal product fit)
|
|
140
|
+
- e2b-v0 as test "not allowlisted" fixture (stable choice)
|
|
141
|
+
- Both languages share aws-lambda-runner event schema (schema is the contract)
|
|
142
|
+
- Class-key storage on Fly via secrets + entrypoint materialization (private material never on disk after upload)
|
|
143
|
+
|
|
144
|
+
## 9. Environment Issues Encountered
|
|
145
|
+
|
|
146
|
+
### 9.1 Editable Install Drops
|
|
147
|
+
|
|
148
|
+
The darwin-agentic-cloud editable install (uv pip install -e .) repeatedly went silent and stopped being importable after roughly every other tool run. Workaround: rerun uv pip install -e . immediately before any pytest invocation. Root cause not yet diagnosed. Should be fixed before Phase 3.
|
|
149
|
+
|
|
150
|
+
### 9.2 Docker Push Broken-Pipe Failures
|
|
151
|
+
|
|
152
|
+
Pushing the AWS Lambda Python base image (b79219105021) to ECR in us-west-2 failed with write: broken pipe on Docker Desktop HTTP proxy. Same byte layer failed every retry until Docker Desktop was restarted. Manual docker push after restart succeeded. Did not recur for the other regions.
|
|
153
|
+
|
|
154
|
+
### 9.3 IAM Eventual Consistency
|
|
155
|
+
|
|
156
|
+
Account 529088294890 chronicle-developer IAM user initially returned AccessDeniedException on lambda:ListFunctions despite holding AdministratorAccess. After ~30 seconds of propagation, all Lambda calls returned 200. No actual SCP existed.
|
|
157
|
+
|
|
158
|
+
### 9.4 CI Workflow Skips
|
|
159
|
+
|
|
160
|
+
PR #26 and PR #28 initially showed no CI checks. An empty git commit --allow-empty -s -m "ci: retrigger workflows" woke them up.
|
|
161
|
+
|
|
162
|
+
### 9.5 Branch Rebases Required Mid-Stretch
|
|
163
|
+
|
|
164
|
+
PR #26 was opened on a branch behind main. After PR #27 merged with ruff-formatted test_server_signing.py, PR #26 CI failed on the same file because it was not yet rebased. Resolved with git fetch origin main + git rebase + git push --force-with-lease.
|
|
165
|
+
|
|
166
|
+
## 10. Cumulative State
|
|
167
|
+
|
|
168
|
+
### 10.1 Substrates
|
|
169
|
+
|
|
170
|
+
| Substrate ID | Adapter | Class key | Deployed | Verified e2e |
|
|
171
|
+
|---|---|---|---|---|
|
|
172
|
+
| local-docker-v0 | merged | live | customer laptop | yes (Phase 1) |
|
|
173
|
+
| aws-lambda-us-east-1 | merged | live | yes | runtime rewire pending |
|
|
174
|
+
| aws-lambda-us-west-2 | merged | live | yes | runtime rewire pending |
|
|
175
|
+
| aws-lambda-eu-west-1 | merged | live | yes | runtime rewire pending |
|
|
176
|
+
| aws-lambda-ap-northeast-1 | merged | live | yes | runtime rewire pending |
|
|
177
|
+
| modal-v0 | merged | not yet | n/a (ephemeral) | yes (real account) |
|
|
178
|
+
| akash-v0 | next | not yet | not yet | not yet |
|
|
179
|
+
|
|
180
|
+
### 10.2 PRs Merged This Stretch
|
|
181
|
+
|
|
182
|
+
- #25: Class-key bootstrap tooling for Fly deploy ceremony
|
|
183
|
+
- #26: AWS Lambda runner deploy + 4-region launch + Phase 2 spec + v3.0.0 launch post + Modal substrate
|
|
184
|
+
- #27: Expand ALLOWED_SUBSTRATES allowlist for v3.0.0
|
|
185
|
+
- #28: Materialize aws-lambda class keys at container start
|
|
186
|
+
|
|
187
|
+
### 10.3 Files Added/Modified This Stretch
|
|
188
|
+
|
|
189
|
+
- infra/aws_runner/runner.py (Python Lambda handler)
|
|
190
|
+
- infra/aws_runner/runner.mjs (Node Lambda handler)
|
|
191
|
+
- infra/aws_runner/Dockerfile.runner-python
|
|
192
|
+
- infra/aws_runner/Dockerfile.runner-node
|
|
193
|
+
- infra/aws_runner/deploy.py (boto3 deploy orchestrator)
|
|
194
|
+
- infra/modal_smoketest.py (real-Modal smoke test)
|
|
195
|
+
- darwin/agenticcloud/substrate/modal.py (Modal ephemeral-sandbox adapter)
|
|
196
|
+
- tests/substrate/test_modal.py (31 tests)
|
|
197
|
+
- darwin/agenticcloud/class_keys.py (ALLOWED_SUBSTRATES expanded to 7)
|
|
198
|
+
- docker-entrypoint.sh (4 new materialize_key calls)
|
|
199
|
+
- tests/test_class_keys.py, tests/test_admin_cli.py, tests/test_server_signing.py
|
|
200
|
+
- examples/hello.py (sample workload)
|
|
201
|
+
- pyproject.toml (add boto3 dep)
|
|
202
|
+
- DARWIN_AGENTIC_CLOUD_PHASE_2_SPEC.md (first-half spec)
|
|
203
|
+
- LAUNCH_POST_v3.0.0.md (draft launch post)
|
|
204
|
+
|
|
205
|
+
### 10.4 Live Public Endpoints
|
|
206
|
+
|
|
207
|
+
- https://darwin-agentic-cloud.fly.dev/healthz (liveness, 200 OK)
|
|
208
|
+
- https://darwin-agentic-cloud.fly.dev/.well-known/substrate-keys.json (5 keys, 200 OK)
|
|
209
|
+
- https://darwin-agentic-cloud.fly.dev/v0/sign-substrate-identity (rate-limited hosted signer)
|
|
210
|
+
- https://darwin-agentic-cloud.fly.dev/v0/identity (operator identity)
|
|
211
|
+
|
|
212
|
+
## 11. Unit Costs vs Unit Prices
|
|
213
|
+
|
|
214
|
+
### 11.1 Wholesale costs
|
|
215
|
+
|
|
216
|
+
- local-docker-v0: 0.0001 dollars per wall-second (synthetic, real cost is 0)
|
|
217
|
+
- aws-lambda US: 0.0000167 dollars per GB-second + 0.0000002 dollars per request
|
|
218
|
+
- aws-lambda EU: 0.0000183 dollars per GB-second
|
|
219
|
+
- aws-lambda AP: 0.0000200 dollars per GB-second
|
|
220
|
+
- modal-v0: 0.000131 dollars per wall-second
|
|
221
|
+
- akash-v0: TBD (cheap tier)
|
|
222
|
+
|
|
223
|
+
### 11.2 Customer prices
|
|
224
|
+
|
|
225
|
+
- Free CLI/SDK tier: 0 dollars per month, no toll
|
|
226
|
+
- Individual tier: 99 dollars per month, 0.001 dollars per execution above included
|
|
227
|
+
- Team tier: 999 dollars per month, 0.0007 dollars per execution above included
|
|
228
|
+
- Enterprise tier: 5K to 50K dollars per month, 0.0003 to 0.0008 per execution
|
|
229
|
+
- Marketplace (Phase 8): 15 to 30 percent take rate
|
|
230
|
+
|
|
231
|
+
### 11.3 Margin model
|
|
232
|
+
|
|
233
|
+
- Subscription: 90 percent plus gross margin (SaaS)
|
|
234
|
+
- Per-execution toll: 50 to 250 times wholesale (drives margin at volume)
|
|
235
|
+
- Compute: passthrough, near-zero margin
|
|
236
|
+
- Free tier: zero cost to Darwin (self-hosted)
|
|
237
|
+
|
|
238
|
+
Margin comes from receipt-issuance standard and convenience tier, not compute markup. Faithful to original toll-booth thesis.
|
|
239
|
+
|
|
240
|
+
## 12. Open Compute for Agents Fidelity Audit
|
|
241
|
+
|
|
242
|
+
| Test | Status | Evidence |
|
|
243
|
+
|---|---|---|
|
|
244
|
+
| Is the schema open? | yes | v0.2 schema documented on disk, RFC-0001 submission planned |
|
|
245
|
+
| Is verification free? | yes | public keylist endpoint returns 1.2 KB JSON without auth |
|
|
246
|
+
| Is the substrate ABC open? | yes | Python ABC with 4 methods, anyone can write an adapter |
|
|
247
|
+
|
|
248
|
+
Architectural choices reinforce the vision:
|
|
249
|
+
|
|
250
|
+
- Two-signature attestation (neither party can fake the receipt alone)
|
|
251
|
+
- Hosted signer is open-source (anyone can run their own keylist server)
|
|
252
|
+
- Multi-substrate from day one (same receipt on AWS, Modal, local Docker, all live now)
|
|
253
|
+
- Reference implementations live alongside the ABC
|
|
254
|
+
|
|
255
|
+
The build is faithful to the original open-compute thesis. No drift.
|
|
256
|
+
|
|
257
|
+
## 13. What is Next
|
|
258
|
+
|
|
259
|
+
Remaining Phase 2 work in order:
|
|
260
|
+
|
|
261
|
+
5. substrate/akash.py adapter + tests + provider lease ceremony
|
|
262
|
+
6. runtime/router.py with pick_by_cost() policy
|
|
263
|
+
7. Runtime rewire: Runtime.run() calls substrate ABC, v0.2 attestations end-to-end
|
|
264
|
+
8. CLI darwin run --substrate ID flag
|
|
265
|
+
9. MCP tools substrate parameter
|
|
266
|
+
10. Hosted demo upgraded to real execution via substrate path
|
|
267
|
+
11. Ship v3.0.0 (version bump, CHANGELOG, git tag, PyPI release, Fly redeploy, launch post)
|
|
268
|
+
|
|
269
|
+
Each step is mechanical given what landed in this stretch.
|
|
270
|
+
|
|
271
|
+
## 14. Closing Note
|
|
272
|
+
|
|
273
|
+
This 2/3 checkpoint represents real working infrastructure, not aspirational documentation. Five live class keys. Eight live AWS Lambda functions. One real Modal sandbox execution with a real attestation evidence dict. 309 tests passing. Two specification documents on disk. The open-compute-for-agents promise has been instantiated against three substrate categories. Remaining work is wiring it together and shipping.
|