genesis-agi 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- genesis_agi-0.1.0/LICENSE +21 -0
- genesis_agi-0.1.0/PKG-INFO +331 -0
- genesis_agi-0.1.0/README.md +303 -0
- genesis_agi-0.1.0/genesis/__main__.py +5 -0
- genesis_agi-0.1.0/genesis/agents/__init__.py +5 -0
- genesis_agi-0.1.0/genesis/agents/roles.py +54 -0
- genesis_agi-0.1.0/genesis/cli.py +72 -0
- genesis_agi-0.1.0/genesis/core/__init__.py +38 -0
- genesis_agi-0.1.0/genesis/core/coordinator.py +61 -0
- genesis_agi-0.1.0/genesis/core/events.py +91 -0
- genesis_agi-0.1.0/genesis/core/models.py +195 -0
- genesis_agi-0.1.0/genesis/core/permissions.py +60 -0
- genesis_agi-0.1.0/genesis/environment/__init__.py +5 -0
- genesis_agi-0.1.0/genesis/environment/runner.py +88 -0
- genesis_agi-0.1.0/genesis/evaluation/__init__.py +6 -0
- genesis_agi-0.1.0/genesis/evaluation/benchmark.py +33 -0
- genesis_agi-0.1.0/genesis/evaluation/evaluator.py +89 -0
- genesis_agi-0.1.0/genesis/evolution/__init__.py +5 -0
- genesis_agi-0.1.0/genesis/evolution/capability_builder.py +47 -0
- genesis_agi-0.1.0/genesis/memory/__init__.py +0 -0
- genesis_agi-0.1.0/genesis/providers/__init__.py +9 -0
- genesis_agi-0.1.0/genesis/providers/base.py +21 -0
- genesis_agi-0.1.0/genesis/providers/huggingface_api.py +26 -0
- genesis_agi-0.1.0/genesis/providers/huggingface_local.py +49 -0
- genesis_agi-0.1.0/genesis/providers/mock.py +17 -0
- genesis_agi-0.1.0/genesis/providers/model_onboarding.py +86 -0
- genesis_agi-0.1.0/genesis/storage/__init__.py +6 -0
- genesis_agi-0.1.0/genesis/storage/artifacts.py +19 -0
- genesis_agi-0.1.0/genesis/storage/database.py +79 -0
- genesis_agi-0.1.0/genesis/ui/__init__.py +5 -0
- genesis_agi-0.1.0/genesis/ui/server.py +111 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/PKG-INFO +331 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/SOURCES.txt +47 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/dependency_links.txt +1 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/entry_points.txt +2 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/requires.txt +10 -0
- genesis_agi-0.1.0/genesis_agi.egg-info/top_level.txt +1 -0
- genesis_agi-0.1.0/pyproject.toml +50 -0
- genesis_agi-0.1.0/setup.cfg +4 -0
- genesis_agi-0.1.0/tests/test_capabilities.py +21 -0
- genesis_agi-0.1.0/tests/test_end_to_end.py +10 -0
- genesis_agi-0.1.0/tests/test_events.py +49 -0
- genesis_agi-0.1.0/tests/test_huggingface_model.py +17 -0
- genesis_agi-0.1.0/tests/test_model_onboarding.py +17 -0
- genesis_agi-0.1.0/tests/test_models.py +69 -0
- genesis_agi-0.1.0/tests/test_permissions.py +26 -0
- genesis_agi-0.1.0/tests/test_smoke.py +6 -0
- genesis_agi-0.1.0/tests/test_speed.py +25 -0
- genesis_agi-0.1.0/tests/test_ui.py +10 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 GENESIS contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,331 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: genesis-agi
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: A measurable framework for AGI-oriented multi-agent experiments
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Project-URL: Homepage, https://github.com/osamahdesk/GENESIS
|
|
7
|
+
Project-URL: Repository, https://github.com/osamahdesk/GENESIS
|
|
8
|
+
Project-URL: Documentation, https://github.com/osamahdesk/GENESIS/tree/main/docs
|
|
9
|
+
Project-URL: Issues, https://github.com/osamahdesk/GENESIS/issues
|
|
10
|
+
Keywords: ai,agi,agents,multi-agent,experiments,evaluation,huggingface
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Science/Research
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
16
|
+
Requires-Python: >=3.11
|
|
17
|
+
Description-Content-Type: text/markdown
|
|
18
|
+
License-File: LICENSE
|
|
19
|
+
Provides-Extra: dev
|
|
20
|
+
Requires-Dist: pytest>=8.0; extra == "dev"
|
|
21
|
+
Requires-Dist: ruff>=0.6; extra == "dev"
|
|
22
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
23
|
+
Requires-Dist: twine>=5.0; extra == "dev"
|
|
24
|
+
Provides-Extra: hf
|
|
25
|
+
Requires-Dist: transformers>=4.40; extra == "hf"
|
|
26
|
+
Requires-Dist: torch>=2.0; extra == "hf"
|
|
27
|
+
Dynamic: license-file
|
|
28
|
+
|
|
29
|
+
# GENESIS
|
|
30
|
+
|
|
31
|
+
> **A reproducible, local-first framework for measurable multi-agent AI experiments.**
|
|
32
|
+
|
|
33
|
+
GENESIS is an open-source research framework for studying whether structured collaboration between specialized AI roles can produce better problem-solving workflows under independent evaluation.
|
|
34
|
+
|
|
35
|
+
It is intentionally not marketed as AGI, consciousness, or unrestricted autonomous intelligence. The project starts with a smaller and testable question:
|
|
36
|
+
|
|
37
|
+
> **Can an AI workflow propose, build, test, critique, remember, and compare solutions while preserving enough evidence to explain why one strategy was accepted?**
|
|
38
|
+
|
|
39
|
+

|
|
40
|
+
|
|
41
|
+
## Why GENESIS exists
|
|
42
|
+
|
|
43
|
+
Many multi-agent demos show several models talking to one another and then call the conversation progress. GENESIS uses a stricter standard. A conversation is only a proposal layer. The important object is the **experiment**: a versioned task, strategy, artifact, execution, evaluation, critique, and decision that another person can inspect.
|
|
44
|
+
|
|
45
|
+
The project therefore treats every claimed improvement as a scientific claim. It must be compared with a frozen baseline, measured under the same conditions, evaluated independently, and preserved together with failures and resource usage.
|
|
46
|
+
|
|
47
|
+
## Current status
|
|
48
|
+
|
|
49
|
+
The repository currently contains a runnable **v0.1 development core**:
|
|
50
|
+
|
|
51
|
+
| Capability | Current state |
|
|
52
|
+
|---|---|
|
|
53
|
+
| Typed task, strategy, budget, artifact, metric, and experiment models | Implemented |
|
|
54
|
+
| Strict experiment lifecycle and domain events | Implemented |
|
|
55
|
+
| Offline deterministic model provider | Implemented |
|
|
56
|
+
| SQLite experiment database | Implemented |
|
|
57
|
+
| Content-addressed artifact references | Implemented |
|
|
58
|
+
| Restricted local Python runner with timeouts | Implemented |
|
|
59
|
+
| Independent benchmark evaluator | Implemented |
|
|
60
|
+
| Researcher, Builder, and Critic roles | Implemented |
|
|
61
|
+
| End-to-end Coordinator | Implemented |
|
|
62
|
+
| CLI commands | Implemented |
|
|
63
|
+
| Local GitHub-style dashboard | Implemented |
|
|
64
|
+
| Included local Hugging Face checkpoint and provider | Implemented as an optional integration |
|
|
65
|
+
| Strategy evolution and dynamic roles | Deferred until the baseline is scientifically measured |
|
|
66
|
+
|
|
67
|
+
The current example is deliberately deterministic. It establishes a reproducible baseline before external model variability is introduced.
|
|
68
|
+
|
|
69
|
+
## The complete workflow
|
|
70
|
+
|
|
71
|
+

|
|
72
|
+
|
|
73
|
+
The runtime sequence is:
|
|
74
|
+
|
|
75
|
+
1. A `Task` defines the problem and benchmark version.
|
|
76
|
+
2. A `Strategy` defines the workflow used to solve it.
|
|
77
|
+
3. The `Researcher` produces a structured hypothesis and plan.
|
|
78
|
+
4. The `Builder` produces an artifact.
|
|
79
|
+
5. The artifact is executed in a bounded local runner.
|
|
80
|
+
6. The independent evaluator runs the benchmark cases outside the builder's control.
|
|
81
|
+
7. The `Critic` analyzes failures without deciding the score.
|
|
82
|
+
8. SQLite stores the experiment, events, metrics, and status history.
|
|
83
|
+
9. GENESIS records a terminal decision such as `verified`, `rejected`, `failed`, or `budget_exceeded`.
|
|
84
|
+
|
|
85
|
+

|
|
86
|
+
|
|
87
|
+
## A first experiment
|
|
88
|
+
|
|
89
|
+
The included example task is `python-sum-squares-v1`.
|
|
90
|
+
|
|
91
|
+
### Input
|
|
92
|
+
|
|
93
|
+
The candidate program receives a JSON integer through standard input:
|
|
94
|
+
|
|
95
|
+
```json
|
|
96
|
+
7
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
The task asks it to print the sum of squares from `1` through `n`, also as JSON:
|
|
100
|
+
|
|
101
|
+
```json
|
|
102
|
+
140
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
The benchmark contains development, validation, and hidden cases. The initial deterministic baseline is evaluated on seven cases.
|
|
106
|
+
|
|
107
|
+
### Output
|
|
108
|
+
|
|
109
|
+
A successful run produces a structured report similar to:
|
|
110
|
+
|
|
111
|
+
```json
|
|
112
|
+
{
|
|
113
|
+
"status": "verified",
|
|
114
|
+
"score": 1.0,
|
|
115
|
+
"passed": 7,
|
|
116
|
+
"total": 7,
|
|
117
|
+
"critique": {
|
|
118
|
+
"failure_count": 0,
|
|
119
|
+
"summary": "No failures detected."
|
|
120
|
+
},
|
|
121
|
+
"artifact": "solution.py"
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+

|
|
126
|
+
|
|
127
|
+
This result means that the current baseline passed the current example benchmark. It does **not** mean that the system became generally intelligent or that a model improved itself. The next scientific comparison is a model-backed candidate against this frozen baseline under equal budgets.
|
|
128
|
+
|
|
129
|
+
## Performance engineering
|
|
130
|
+
|
|
131
|
+
GENESIS treats speed as a measured property. The evaluator now supports per-case isolated execution, accelerated batch execution in one isolated worker with a fresh namespace per case, and hash-keyed result caching. The local model provider also uses evaluation mode, inference mode, and key/value caching during generation.
|
|
132
|
+
|
|
133
|
+
On the included seven-case benchmark, the measured results were:
|
|
134
|
+
|
|
135
|
+
| Path | Wall time | Result |
|
|
136
|
+
|---|---:|---|
|
|
137
|
+
| Per-case isolated evaluation | 0.327261 s | 7/7, score 1.0 |
|
|
138
|
+
| Accelerated batch evaluation | 0.047891 s | 7/7, score 1.0 |
|
|
139
|
+
| In-memory repeated evaluation | 0.000051 s | Cache hit, score 1.0 |
|
|
140
|
+
|
|
141
|
+
That run measured a **6.83×** batch improvement and approximately **939×** for an in-memory repeated evaluation. These are workload-specific measurements, not universal promises. End-to-end CLI time also includes interpreter startup, SQLite writes, artifact hashing, and output rendering.
|
|
142
|
+
|
|
143
|
+
Reproduce the measurements with:
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
python tools/benchmark_speed.py
|
|
147
|
+
python tools/benchmark_cli_cache.py
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
The isolated path remains available for hostile or untrusted artifacts; performance optimizations never bypass evaluation or the safety boundary.
|
|
151
|
+
|
|
152
|
+
See the full example in [`docs/example-result.md`](docs/example-result.md).
|
|
153
|
+
|
|
154
|
+
## Quick start
|
|
155
|
+
|
|
156
|
+
GENESIS is designed to work offline for its first run.
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
git clone https://github.com/osamahdesk/GENESIS.git
|
|
160
|
+
cd GENESIS
|
|
161
|
+
python3 -m venv .venv
|
|
162
|
+
source .venv/bin/activate
|
|
163
|
+
pip install -e ".[dev]"
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Initialize the local workspace:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
genesis init
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Run the complete offline baseline experiment:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
genesis run
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Inspect stored experiments:
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
genesis status
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Inspect a specific experiment:
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
genesis experiment EXP-000001
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
## The local dashboard
|
|
191
|
+
|
|
192
|
+
The CLI is the primary interface for automation and reproducibility. The dashboard is a lightweight visual layer for humans who want a simpler overview.
|
|
193
|
+
|
|
194
|
+
Run an experiment first, then start the dashboard:
|
|
195
|
+
|
|
196
|
+
```bash
|
|
197
|
+
genesis run
|
|
198
|
+
genesis dashboard
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Open:
|
|
202
|
+
|
|
203
|
+
```text
|
|
204
|
+
http://127.0.0.1:8765
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
The dashboard shows the latest status, score, test count, workflow, experiment identity, evidence snapshot, and a short hint explaining the next action. Highlighted cards and workflow steps are clickable: a small teaching bubble explains what the selected element means and how it connects to the experiment. A Permission Center makes the deny-by-default boundary visible. It uses the same local SQLite records as the CLI and does not require an external service.
|
|
208
|
+
|
|
209
|
+
The Permission Center also exposes reversible ON/OFF controls for `Transformers`, `PyTorch`, `Hugging Face`, `AI Builder`, and `Model Training`. A toggle changes the experiment policy; it does not silently install packages, execute generated code, or grant host access. When `AI Builder` is enabled, GENESIS can create a reviewable model-project proposal inside the sandbox. The proposal must be evaluated before any future execution or promotion.
|
|
210
|
+
|
|
211
|
+
## Design principles
|
|
212
|
+
|
|
213
|
+
### Evidence over conversation
|
|
214
|
+
|
|
215
|
+
An agent's explanation is a hypothesis. A measured evaluation is evidence. The evaluator is independent from the artifact-generation role.
|
|
216
|
+
|
|
217
|
+
### Reproducibility by default
|
|
218
|
+
|
|
219
|
+
Experiments preserve task identity, strategy identity, artifact hashes, status events, model metadata, configuration, metrics, and timestamps. Future releases will export a complete experiment bundle.
|
|
220
|
+
|
|
221
|
+
### Safe defaults
|
|
222
|
+
|
|
223
|
+
The default mode is offline. Network access is not silently enabled. Generated code runs in a bounded temporary workspace with a timeout and a sanitized environment. This reduces risk but is not a perfect security boundary; users must not execute untrusted code with permissions they would not grant to an unknown program.
|
|
224
|
+
|
|
225
|
+
### Provider independence
|
|
226
|
+
|
|
227
|
+
The core domain models do not depend on OpenAI, Anthropic, Hugging Face, or any single vendor. Provider adapters are deliberately separated from tasks, experiments, evaluation, storage, and orchestration.
|
|
228
|
+
|
|
229
|
+
### Small increments
|
|
230
|
+
|
|
231
|
+
GENESIS is developed one daily work file at a time. Each file defines its scope, acceptance criteria, tests, blockers, and final status. Later features are not silently pulled into an earlier day.
|
|
232
|
+
|
|
233
|
+
## Repository structure
|
|
234
|
+
|
|
235
|
+
```text
|
|
236
|
+
GENESIS/
|
|
237
|
+
├── README.md
|
|
238
|
+
├── pyproject.toml
|
|
239
|
+
├── configs/
|
|
240
|
+
│ └── default.yaml
|
|
241
|
+
├── docs/
|
|
242
|
+
│ ├── vision.md
|
|
243
|
+
│ ├── v0.1-spec.md
|
|
244
|
+
│ ├── architecture.md
|
|
245
|
+
│ ├── evaluation.md
|
|
246
|
+
│ ├── security.md
|
|
247
|
+
│ ├── roadmap.md
|
|
248
|
+
│ ├── work-tree.md
|
|
249
|
+
│ ├── example-result.md
|
|
250
|
+
│ ├── images/
|
|
251
|
+
│ └── daily/
|
|
252
|
+
├── genesis/
|
|
253
|
+
│ ├── agents/
|
|
254
|
+
│ ├── core/
|
|
255
|
+
│ ├── environment/
|
|
256
|
+
│ ├── evaluation/
|
|
257
|
+
│ ├── providers/
|
|
258
|
+
│ ├── storage/
|
|
259
|
+
│ └── ui/
|
|
260
|
+
├── benchmarks/
|
|
261
|
+
├── examples/
|
|
262
|
+
└── tests/
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
## Architecture overview
|
|
266
|
+
|
|
267
|
+
The Coordinator is the boundary between user intent and experiment execution. It invokes specialized roles, applies the lifecycle state machine, records domain events, and delegates scoring to the independent evaluator.
|
|
268
|
+
|
|
269
|
+
The safety boundary is intentionally outside the parts that future evolution may modify. A later strategy or role candidate may be tested in an isolated experiment, but it cannot silently replace the evaluator, hidden-test policy, approval mechanism, or host permissions.
|
|
270
|
+
|
|
271
|
+
## Evaluation policy
|
|
272
|
+
|
|
273
|
+
A fair comparison uses the same task set, model, model parameters, retry rules, time budget, execution policy, and output format for the baseline and candidate. A single run is not enough to establish a scientific improvement. Future reports will support repeated trials, spread, cost, latency, hidden-task performance, and ablation studies.
|
|
274
|
+
|
|
275
|
+
The intended baseline ladder is:
|
|
276
|
+
|
|
277
|
+
```text
|
|
278
|
+
Single model → Single model + self-critique
|
|
279
|
+
→ Researcher + Builder
|
|
280
|
+
→ Researcher + Builder + Critic + Repair
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
The purpose is to measure which component contributes to an observed change rather than assuming that more agents automatically means better performance.
|
|
284
|
+
|
|
285
|
+
## Roadmap
|
|
286
|
+
|
|
287
|
+
The approved roadmap is phased:
|
|
288
|
+
|
|
289
|
+
- **Foundation:** schemas, events, configuration, providers, and storage.
|
|
290
|
+
- **First experiment:** evaluator, runner, roles, Coordinator, and CLI.
|
|
291
|
+
- **Scientific integrity:** hidden splits, repetitions, cost metrics, anti-cheating checks, and reproducibility bundles.
|
|
292
|
+
- **Strategy evolution:** frozen parents, mutation, selection, failure memory, and regression gates.
|
|
293
|
+
- **Future research:** dynamic roles, tool experiments, capability-gap analysis, and carefully controlled external research.
|
|
294
|
+
|
|
295
|
+
The detailed sequence is in [`docs/roadmap.md`](docs/roadmap.md), and each day is tracked in [`docs/daily/day-index.md`](docs/daily/day-index.md).
|
|
296
|
+
|
|
297
|
+
## Tests
|
|
298
|
+
|
|
299
|
+
Install development dependencies and run:
|
|
300
|
+
|
|
301
|
+
```bash
|
|
302
|
+
python -m pytest
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
The current development core includes unit, lifecycle, evaluation, end-to-end, and dashboard tests. A clean test run is a release requirement, not an optional polish step.
|
|
306
|
+
|
|
307
|
+
## Hugging Face integration
|
|
308
|
+
|
|
309
|
+
Phase 2 includes the small [`sshleifer/tiny-gpt2`](models/sshleifer-tiny-gpt2) checkpoint from Hugging Face. The files are committed with a `SHA256SUMS` manifest, and [`HuggingFaceLocalProvider`](genesis/providers/huggingface_local.py) loads them from disk through the same provider boundary used by the mock provider.
|
|
310
|
+
|
|
311
|
+
The integration is optional because PyTorch and Transformers are heavyweight dependencies compared with the deterministic core. Install it only when you want to run the local model:
|
|
312
|
+
|
|
313
|
+
```bash
|
|
314
|
+
pip install -e '.[hf]'
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
The default test suite and offline baseline still work without the `hf` extra. This separation lets GENESIS use a real model while preserving clean, fast, reproducible development for the rest of the project. The model is intentionally a tiny integration checkpoint, not a claim of production-quality generation.
|
|
318
|
+
|
|
319
|
+
## Teacher model onboarding
|
|
320
|
+
|
|
321
|
+
Users do not need to know the Hub workflow in advance. The dashboard provides a Teacher Model center that recommends catalog entries using detected machine memory, accepts an exact `owner/model` identifier, accepts a Hugging Face URL, or accepts an existing local model directory. The selection is saved as `teacher.json` for later experiments.
|
|
322
|
+
|
|
323
|
+
When the Hugging Face capability and read-only network permission are enabled, the dashboard can query the public Hub API on explicit request. Results include model ID, downloads, and likes, with a `Use` action for selecting a teacher. The teacher produces reviewable candidates inside the sandbox; it does not silently clone itself, replace GENESIS, install arbitrary code, or bypass evaluation.
|
|
324
|
+
|
|
325
|
+
## Contributing
|
|
326
|
+
|
|
327
|
+
Start by reading [`docs/v0.1-spec.md`](docs/v0.1-spec.md), then choose the next incomplete daily work file. Keep changes small, add tests, preserve the offline path, and document any architecture change. Do not publish claims of improvement without the benchmark evidence that supports them.
|
|
328
|
+
|
|
329
|
+
## License
|
|
330
|
+
|
|
331
|
+
MIT. See [`LICENSE`](LICENSE).
|
|
@@ -0,0 +1,303 @@
|
|
|
1
|
+
# GENESIS
|
|
2
|
+
|
|
3
|
+
> **A reproducible, local-first framework for measurable multi-agent AI experiments.**
|
|
4
|
+
|
|
5
|
+
GENESIS is an open-source research framework for studying whether structured collaboration between specialized AI roles can produce better problem-solving workflows under independent evaluation.
|
|
6
|
+
|
|
7
|
+
It is intentionally not marketed as AGI, consciousness, or unrestricted autonomous intelligence. The project starts with a smaller and testable question:
|
|
8
|
+
|
|
9
|
+
> **Can an AI workflow propose, build, test, critique, remember, and compare solutions while preserving enough evidence to explain why one strategy was accepted?**
|
|
10
|
+
|
|
11
|
+

|
|
12
|
+
|
|
13
|
+
## Why GENESIS exists
|
|
14
|
+
|
|
15
|
+
Many multi-agent demos show several models talking to one another and then call the conversation progress. GENESIS uses a stricter standard. A conversation is only a proposal layer. The important object is the **experiment**: a versioned task, strategy, artifact, execution, evaluation, critique, and decision that another person can inspect.
|
|
16
|
+
|
|
17
|
+
The project therefore treats every claimed improvement as a scientific claim. It must be compared with a frozen baseline, measured under the same conditions, evaluated independently, and preserved together with failures and resource usage.
|
|
18
|
+
|
|
19
|
+
## Current status
|
|
20
|
+
|
|
21
|
+
The repository currently contains a runnable **v0.1 development core**:
|
|
22
|
+
|
|
23
|
+
| Capability | Current state |
|
|
24
|
+
|---|---|
|
|
25
|
+
| Typed task, strategy, budget, artifact, metric, and experiment models | Implemented |
|
|
26
|
+
| Strict experiment lifecycle and domain events | Implemented |
|
|
27
|
+
| Offline deterministic model provider | Implemented |
|
|
28
|
+
| SQLite experiment database | Implemented |
|
|
29
|
+
| Content-addressed artifact references | Implemented |
|
|
30
|
+
| Restricted local Python runner with timeouts | Implemented |
|
|
31
|
+
| Independent benchmark evaluator | Implemented |
|
|
32
|
+
| Researcher, Builder, and Critic roles | Implemented |
|
|
33
|
+
| End-to-end Coordinator | Implemented |
|
|
34
|
+
| CLI commands | Implemented |
|
|
35
|
+
| Local GitHub-style dashboard | Implemented |
|
|
36
|
+
| Included local Hugging Face checkpoint and provider | Implemented as an optional integration |
|
|
37
|
+
| Strategy evolution and dynamic roles | Deferred until the baseline is scientifically measured |
|
|
38
|
+
|
|
39
|
+
The current example is deliberately deterministic. It establishes a reproducible baseline before external model variability is introduced.
|
|
40
|
+
|
|
41
|
+
## The complete workflow
|
|
42
|
+
|
|
43
|
+

|
|
44
|
+
|
|
45
|
+
The runtime sequence is:
|
|
46
|
+
|
|
47
|
+
1. A `Task` defines the problem and benchmark version.
|
|
48
|
+
2. A `Strategy` defines the workflow used to solve it.
|
|
49
|
+
3. The `Researcher` produces a structured hypothesis and plan.
|
|
50
|
+
4. The `Builder` produces an artifact.
|
|
51
|
+
5. The artifact is executed in a bounded local runner.
|
|
52
|
+
6. The independent evaluator runs the benchmark cases outside the builder's control.
|
|
53
|
+
7. The `Critic` analyzes failures without deciding the score.
|
|
54
|
+
8. SQLite stores the experiment, events, metrics, and status history.
|
|
55
|
+
9. GENESIS records a terminal decision such as `verified`, `rejected`, `failed`, or `budget_exceeded`.
|
|
56
|
+
|
|
57
|
+

|
|
58
|
+
|
|
59
|
+
## A first experiment
|
|
60
|
+
|
|
61
|
+
The included example task is `python-sum-squares-v1`.
|
|
62
|
+
|
|
63
|
+
### Input
|
|
64
|
+
|
|
65
|
+
The candidate program receives a JSON integer through standard input:
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
7
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
The task asks it to print the sum of squares from `1` through `n`, also as JSON:
|
|
72
|
+
|
|
73
|
+
```json
|
|
74
|
+
140
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
The benchmark contains development, validation, and hidden cases. The initial deterministic baseline is evaluated on seven cases.
|
|
78
|
+
|
|
79
|
+
### Output
|
|
80
|
+
|
|
81
|
+
A successful run produces a structured report similar to:
|
|
82
|
+
|
|
83
|
+
```json
|
|
84
|
+
{
|
|
85
|
+
"status": "verified",
|
|
86
|
+
"score": 1.0,
|
|
87
|
+
"passed": 7,
|
|
88
|
+
"total": 7,
|
|
89
|
+
"critique": {
|
|
90
|
+
"failure_count": 0,
|
|
91
|
+
"summary": "No failures detected."
|
|
92
|
+
},
|
|
93
|
+
"artifact": "solution.py"
|
|
94
|
+
}
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+

|
|
98
|
+
|
|
99
|
+
This result means that the current baseline passed the current example benchmark. It does **not** mean that the system became generally intelligent or that a model improved itself. The next scientific comparison is a model-backed candidate against this frozen baseline under equal budgets.
|
|
100
|
+
|
|
101
|
+
## Performance engineering
|
|
102
|
+
|
|
103
|
+
GENESIS treats speed as a measured property. The evaluator now supports per-case isolated execution, accelerated batch execution in one isolated worker with a fresh namespace per case, and hash-keyed result caching. The local model provider also uses evaluation mode, inference mode, and key/value caching during generation.
|
|
104
|
+
|
|
105
|
+
On the included seven-case benchmark, the measured results were:
|
|
106
|
+
|
|
107
|
+
| Path | Wall time | Result |
|
|
108
|
+
|---|---:|---|
|
|
109
|
+
| Per-case isolated evaluation | 0.327261 s | 7/7, score 1.0 |
|
|
110
|
+
| Accelerated batch evaluation | 0.047891 s | 7/7, score 1.0 |
|
|
111
|
+
| In-memory repeated evaluation | 0.000051 s | Cache hit, score 1.0 |
|
|
112
|
+
|
|
113
|
+
That run measured a **6.83×** batch improvement and approximately **939×** for an in-memory repeated evaluation. These are workload-specific measurements, not universal promises. End-to-end CLI time also includes interpreter startup, SQLite writes, artifact hashing, and output rendering.
|
|
114
|
+
|
|
115
|
+
Reproduce the measurements with:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
python tools/benchmark_speed.py
|
|
119
|
+
python tools/benchmark_cli_cache.py
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
The isolated path remains available for hostile or untrusted artifacts; performance optimizations never bypass evaluation or the safety boundary.
|
|
123
|
+
|
|
124
|
+
See the full example in [`docs/example-result.md`](docs/example-result.md).
|
|
125
|
+
|
|
126
|
+
## Quick start
|
|
127
|
+
|
|
128
|
+
GENESIS is designed to work offline for its first run.
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
git clone https://github.com/osamahdesk/GENESIS.git
|
|
132
|
+
cd GENESIS
|
|
133
|
+
python3 -m venv .venv
|
|
134
|
+
source .venv/bin/activate
|
|
135
|
+
pip install -e ".[dev]"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Initialize the local workspace:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
genesis init
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Run the complete offline baseline experiment:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
genesis run
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Inspect stored experiments:
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
genesis status
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Inspect a specific experiment:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
genesis experiment EXP-000001
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
## The local dashboard
|
|
163
|
+
|
|
164
|
+
The CLI is the primary interface for automation and reproducibility. The dashboard is a lightweight visual layer for humans who want a simpler overview.
|
|
165
|
+
|
|
166
|
+
Run an experiment first, then start the dashboard:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
genesis run
|
|
170
|
+
genesis dashboard
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Open:
|
|
174
|
+
|
|
175
|
+
```text
|
|
176
|
+
http://127.0.0.1:8765
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
The dashboard shows the latest status, score, test count, workflow, experiment identity, evidence snapshot, and a short hint explaining the next action. Highlighted cards and workflow steps are clickable: a small teaching bubble explains what the selected element means and how it connects to the experiment. A Permission Center makes the deny-by-default boundary visible. It uses the same local SQLite records as the CLI and does not require an external service.
|
|
180
|
+
|
|
181
|
+
The Permission Center also exposes reversible ON/OFF controls for `Transformers`, `PyTorch`, `Hugging Face`, `AI Builder`, and `Model Training`. A toggle changes the experiment policy; it does not silently install packages, execute generated code, or grant host access. When `AI Builder` is enabled, GENESIS can create a reviewable model-project proposal inside the sandbox. The proposal must be evaluated before any future execution or promotion.
|
|
182
|
+
|
|
183
|
+
## Design principles
|
|
184
|
+
|
|
185
|
+
### Evidence over conversation
|
|
186
|
+
|
|
187
|
+
An agent's explanation is a hypothesis. A measured evaluation is evidence. The evaluator is independent from the artifact-generation role.
|
|
188
|
+
|
|
189
|
+
### Reproducibility by default
|
|
190
|
+
|
|
191
|
+
Experiments preserve task identity, strategy identity, artifact hashes, status events, model metadata, configuration, metrics, and timestamps. Future releases will export a complete experiment bundle.
|
|
192
|
+
|
|
193
|
+
### Safe defaults
|
|
194
|
+
|
|
195
|
+
The default mode is offline. Network access is not silently enabled. Generated code runs in a bounded temporary workspace with a timeout and a sanitized environment. This reduces risk but is not a perfect security boundary; users must not execute untrusted code with permissions they would not grant to an unknown program.
|
|
196
|
+
|
|
197
|
+
### Provider independence
|
|
198
|
+
|
|
199
|
+
The core domain models do not depend on OpenAI, Anthropic, Hugging Face, or any single vendor. Provider adapters are deliberately separated from tasks, experiments, evaluation, storage, and orchestration.
|
|
200
|
+
|
|
201
|
+
### Small increments
|
|
202
|
+
|
|
203
|
+
GENESIS is developed one daily work file at a time. Each file defines its scope, acceptance criteria, tests, blockers, and final status. Later features are not silently pulled into an earlier day.
|
|
204
|
+
|
|
205
|
+
## Repository structure
|
|
206
|
+
|
|
207
|
+
```text
|
|
208
|
+
GENESIS/
|
|
209
|
+
├── README.md
|
|
210
|
+
├── pyproject.toml
|
|
211
|
+
├── configs/
|
|
212
|
+
│ └── default.yaml
|
|
213
|
+
├── docs/
|
|
214
|
+
│ ├── vision.md
|
|
215
|
+
│ ├── v0.1-spec.md
|
|
216
|
+
│ ├── architecture.md
|
|
217
|
+
│ ├── evaluation.md
|
|
218
|
+
│ ├── security.md
|
|
219
|
+
│ ├── roadmap.md
|
|
220
|
+
│ ├── work-tree.md
|
|
221
|
+
│ ├── example-result.md
|
|
222
|
+
│ ├── images/
|
|
223
|
+
│ └── daily/
|
|
224
|
+
├── genesis/
|
|
225
|
+
│ ├── agents/
|
|
226
|
+
│ ├── core/
|
|
227
|
+
│ ├── environment/
|
|
228
|
+
│ ├── evaluation/
|
|
229
|
+
│ ├── providers/
|
|
230
|
+
│ ├── storage/
|
|
231
|
+
│ └── ui/
|
|
232
|
+
├── benchmarks/
|
|
233
|
+
├── examples/
|
|
234
|
+
└── tests/
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
## Architecture overview
|
|
238
|
+
|
|
239
|
+
The Coordinator is the boundary between user intent and experiment execution. It invokes specialized roles, applies the lifecycle state machine, records domain events, and delegates scoring to the independent evaluator.
|
|
240
|
+
|
|
241
|
+
The safety boundary is intentionally outside the parts that future evolution may modify. A later strategy or role candidate may be tested in an isolated experiment, but it cannot silently replace the evaluator, hidden-test policy, approval mechanism, or host permissions.
|
|
242
|
+
|
|
243
|
+
## Evaluation policy
|
|
244
|
+
|
|
245
|
+
A fair comparison uses the same task set, model, model parameters, retry rules, time budget, execution policy, and output format for the baseline and candidate. A single run is not enough to establish a scientific improvement. Future reports will support repeated trials, spread, cost, latency, hidden-task performance, and ablation studies.
|
|
246
|
+
|
|
247
|
+
The intended baseline ladder is:
|
|
248
|
+
|
|
249
|
+
```text
|
|
250
|
+
Single model → Single model + self-critique
|
|
251
|
+
→ Researcher + Builder
|
|
252
|
+
→ Researcher + Builder + Critic + Repair
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
The purpose is to measure which component contributes to an observed change rather than assuming that more agents automatically means better performance.
|
|
256
|
+
|
|
257
|
+
## Roadmap
|
|
258
|
+
|
|
259
|
+
The approved roadmap is phased:
|
|
260
|
+
|
|
261
|
+
- **Foundation:** schemas, events, configuration, providers, and storage.
|
|
262
|
+
- **First experiment:** evaluator, runner, roles, Coordinator, and CLI.
|
|
263
|
+
- **Scientific integrity:** hidden splits, repetitions, cost metrics, anti-cheating checks, and reproducibility bundles.
|
|
264
|
+
- **Strategy evolution:** frozen parents, mutation, selection, failure memory, and regression gates.
|
|
265
|
+
- **Future research:** dynamic roles, tool experiments, capability-gap analysis, and carefully controlled external research.
|
|
266
|
+
|
|
267
|
+
The detailed sequence is in [`docs/roadmap.md`](docs/roadmap.md), and each day is tracked in [`docs/daily/day-index.md`](docs/daily/day-index.md).
|
|
268
|
+
|
|
269
|
+
## Tests
|
|
270
|
+
|
|
271
|
+
Install development dependencies and run:
|
|
272
|
+
|
|
273
|
+
```bash
|
|
274
|
+
python -m pytest
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
The current development core includes unit, lifecycle, evaluation, end-to-end, and dashboard tests. A clean test run is a release requirement, not an optional polish step.
|
|
278
|
+
|
|
279
|
+
## Hugging Face integration
|
|
280
|
+
|
|
281
|
+
Phase 2 includes the small [`sshleifer/tiny-gpt2`](models/sshleifer-tiny-gpt2) checkpoint from Hugging Face. The files are committed with a `SHA256SUMS` manifest, and [`HuggingFaceLocalProvider`](genesis/providers/huggingface_local.py) loads them from disk through the same provider boundary used by the mock provider.
|
|
282
|
+
|
|
283
|
+
The integration is optional because PyTorch and Transformers are heavyweight dependencies compared with the deterministic core. Install it only when you want to run the local model:
|
|
284
|
+
|
|
285
|
+
```bash
|
|
286
|
+
pip install -e '.[hf]'
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
The default test suite and offline baseline still work without the `hf` extra. This separation lets GENESIS use a real model while preserving clean, fast, reproducible development for the rest of the project. The model is intentionally a tiny integration checkpoint, not a claim of production-quality generation.
|
|
290
|
+
|
|
291
|
+
## Teacher model onboarding
|
|
292
|
+
|
|
293
|
+
Users do not need to know the Hub workflow in advance. The dashboard provides a Teacher Model center that recommends catalog entries using detected machine memory, accepts an exact `owner/model` identifier, accepts a Hugging Face URL, or accepts an existing local model directory. The selection is saved as `teacher.json` for later experiments.
|
|
294
|
+
|
|
295
|
+
When the Hugging Face capability and read-only network permission are enabled, the dashboard can query the public Hub API on explicit request. Results include model ID, downloads, and likes, with a `Use` action for selecting a teacher. The teacher produces reviewable candidates inside the sandbox; it does not silently clone itself, replace GENESIS, install arbitrary code, or bypass evaluation.
|
|
296
|
+
|
|
297
|
+
## Contributing
|
|
298
|
+
|
|
299
|
+
Start by reading [`docs/v0.1-spec.md`](docs/v0.1-spec.md), then choose the next incomplete daily work file. Keep changes small, add tests, preserve the offline path, and document any architecture change. Do not publish claims of improvement without the benchmark evidence that supports them.
|
|
300
|
+
|
|
301
|
+
## License
|
|
302
|
+
|
|
303
|
+
MIT. See [`LICENSE`](LICENSE).
|