ml-model-architect 1.0.0rc1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,713 @@
1
+ Metadata-Version: 2.4
2
+ Name: ml-model-architect
3
+ Version: 1.0.0rc1
4
+ Summary: A strict, reference-driven framework that turns AI agents into disciplined, expert ML engineers.
5
+ Author: Ahmed Zain
6
+ License-Expression: MIT
7
+ Requires-Python: >=3.10
8
+ Description-Content-Type: text/markdown
9
+ License-File: LICENSE
10
+ Requires-Dist: requests>=2.28.0
11
+ Requires-Dist: pyyaml>=6.0
12
+ Requires-Dist: numpy>=1.24.0
13
+ Requires-Dist: scikit-learn>=1.3.0
14
+ Requires-Dist: pandas>=2.1.0
15
+ Requires-Dist: streamlit>=1.30.0
16
+ Provides-Extra: mcp
17
+ Requires-Dist: mcp>=1.0.0; extra == "mcp"
18
+ Requires-Dist: chromadb>=0.4.0; extra == "mcp"
19
+ Requires-Dist: sentence-transformers>=2.2.0; extra == "mcp"
20
+ Provides-Extra: dev
21
+ Requires-Dist: pytest>=7.0.0; extra == "dev"
22
+ Dynamic: license-file
23
+
24
+ <img width="100%" height="887" alt="ML Model Architect Banner" src="https://github.com/user-attachments/assets/0b1e73e0-f551-4ca2-be93-569b1f54db91" />
25
+
26
+ <div align="center">
27
+
28
+ # ML Model Architect
29
+
30
+ ### A Deterministic, Reference-Driven Governance Framework for Disciplined Machine Learning Engineering
31
+
32
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](https://opensource.org/licenses/MIT)
33
+ [![Python Version](https://img.shields.io/badge/Python-3.10%2B-blue.svg?style=flat-square)](https://www.python.org/downloads/)
34
+ [![Test Suite](https://img.shields.io/badge/Tests-79%20Passing-brightgreen.svg?style=flat-square)](tests/)
35
+ [![Build Status](https://img.shields.io/badge/CI-GitHub%20Actions-blue.svg?style=flat-square)](.github/workflows/verify.yml)
36
+ [![PRs Welcome](https://img.shields.io/badge/PRs-Welcome-brightgreen.svg?style=flat-square)](.github/CONTRIBUTING.md)
37
+ [![Architecture: MCP Ready](https://img.shields.io/badge/MCP-Ready-purple.svg?style=flat-square)](https://modelcontextprotocol.io)
38
+ [![Protocol: Zero-Paywall](https://img.shields.io/badge/Zero--Paywall-Rule%20B8-blue.svg?style=flat-square)](#zero-paywall-protocol-and-dynamic-sota-discovery)
39
+ [![Security: Sandboxed](https://img.shields.io/badge/Sandbox-AST%20%26%20Docker%20F4-red.svg?style=flat-square)](#security-and-sandboxing-architecture)
40
+ [![Lifecycle: 5 Stages](https://img.shields.io/badge/Lifecycle-5%20Stages-orange.svg?style=flat-square)](#the-5-stages-of-ml-architecture)
41
+ [![Academic Rigor: 12 Reference Groups](https://img.shields.io/badge/References-12%20Groups%20Peer--Reviewed-teal.svg?style=flat-square)](#reference-rigor--the-6-criteria)
42
+
43
+ *"Grounded Machine Learning: Verifiable academic references driving deterministic empirical validation."*
44
+
45
+ </div>
46
+
47
+ ---
48
+
49
+ ## Executive Overview
50
+
51
+ **ML Model Architect** is an enterprise-grade governance engine and architectural framework designed to transform Large Language Model (LLM) agents into disciplined, rigorous Machine Learning engineers. Modern AI coding assistants execute code at unprecedented velocity; however, velocity unconstrained by peer-reviewed theory and formal validation introduces systemic vulnerabilities into production machine learning workflows:
52
+
53
+ * **Unanchored Architecture Selection:** Arbitrary selection of complex architectures (e.g., Deep Neural Networks or gradient boosters on trivial tabular datasets) without empirical or theoretical justification.
54
+ * **Severe Data Leakage:** Pervasive preprocessing leaks (e.g., fitting scalers, imputers, or encoders across the entire dataset prior to cross-validation splits).
55
+ * **Hallucinated Hyperparameters:** Hyperparameter values generated from statistical plausibility rather than peer-reviewed literature or bounded Bayesian optimization spaces.
56
+ * **Misaligned Optimization Metrics:** Selection of naive accuracy metrics on imbalanced distributions, obscuring classification failure.
57
+ * **Ungrounded Production Deployments:** Rollouts lacking mathematical drift monitoring (PSI / Kolmogorov-Smirnov) or formal rollback criteria.
58
+
59
+ ML Model Architect solves this paradigm through a **Constitutional Execution Engine**. Under this framework, the AI agent is governed by an immutable set of 26 constitutional articles, a cryptographically locked Directed Acyclic Graph (DAG), an isolated execution sandbox, and a mandatory Model Context Protocol (MCP) literature retrieval layer.
60
+
61
+ > [!IMPORTANT]
62
+ > **The Sovereign Axiom: "The Reference Is the Judge."**
63
+ > Every technical choice must be anchored to a verified Digital Object Identifier (DOI), arXiv identifier, or peer-reviewed citation. Where no reference exists, the workflow immediately halts and logs a formal **Reference Gap**. Unverifiable invention is strictly prohibited.
64
+
65
+ ---
66
+
67
+ ## Table of Contents
68
+
69
+ - [Executive Overview](#executive-overview)
70
+ - [System Architecture & Behavioral Blueprints](#system-architecture--behavioral-blueprints)
71
+ - [1. User and Model End-to-End Interaction Sequence](#1-user-and-model-end-to-end-interaction-sequence)
72
+ - [2. Complete User and Model Lifecycle Journey](#2-complete-user-and-model-lifecycle-journey)
73
+ - [3. Deterministic Plan Locking and Grounded ReAct State Machine](#3-deterministic-plan-locking-and-grounded-react-state-machine)
74
+ - [The 5 Stages of ML Architecture](#the-5-stages-of-ml-architecture)
75
+ - [Empirical Benchmarks: Measured Performance Impact](#empirical-benchmarks-measured-performance-impact)
76
+ - [1. Multi-Task Empirical Results](#1-multi-task-empirical-results)
77
+ - [2. Engineering Rigor and Safety Verification Matrix](#2-engineering-rigor-and-safety-verification-matrix)
78
+ - [Comparative Protocol: Unconstrained vs. Governed Agent](#comparative-protocol-unconstrained-vs-governed-agent)
79
+ - [Quick Start Guide](#quick-start-guide)
80
+ - [1. Installation](#1-installation)
81
+ - [2. Unified Command-Line Interface (CLI: mla)](#2-unified-command-line-interface-cli-mla)
82
+ - [3. AI Agent Initialization Prompt](#3-ai-agent-initialization-prompt)
83
+ - [Model Context Protocol (MCP) Configuration](#model-context-protocol-mcp-configuration)
84
+ - [Zero-Paywall Protocol and Dynamic SOTA Discovery](#zero-paywall-protocol-and-dynamic-sota-discovery)
85
+ - [Security and Sandboxing Architecture](#security-and-sandboxing-architecture)
86
+ - [Repository Organization](#repository-organization)
87
+ - [Academic Reference Rigor: The 6 Criteria](#academic-reference-rigor-the-6-criteria)
88
+ - [Constitutional Governance Matrix](#constitutional-governance-matrix)
89
+ - [Behavioral Verification and Test Suite Status](#behavioral-verification-and-test-suite-status)
90
+ - [Known Limitations and Threat Model](#known-limitations-and-threat-model)
91
+ - [Requirements](#requirements)
92
+ - [Contributing](#contributing)
93
+ - [License](#license)
94
+
95
+ ---
96
+
97
+ ## System Architecture & Behavioral Blueprints
98
+
99
+ ### 1. User and Model End-to-End Interaction Sequence
100
+
101
+ The following formal sequence diagram details the communication, verification, and execution barriers enforced between the Human ML Engineer, the Constitutional AI Agent, the external MCP Research Infrastructure, the Isolated Execution Sandbox, and the Static AST Policy Engine.
102
+
103
+ ```mermaid
104
+ sequenceDiagram
105
+ autonumber
106
+ actor User as Machine Learning Engineer
107
+ participant Agent as Constitutional AI Agent (Rules A-G)
108
+ participant MCP as MCP Research Server (CrossRef / ArXiv)
109
+ participant Sandbox as Isolated Sandbox (/sandbox/)
110
+ participant AST as AST Scanner & Policy Engine
111
+
112
+ Note over User,Agent: Stage 1: Goals and Problem Formulation
113
+ Agent->>User: Structured Sequential MCQ Inquiries (Objective, Latency, Loss Matrix)
114
+ User->>Agent: Formal Grounding and Constraints Specification
115
+ Agent->>Agent: Compile and Validate REQUIREMENTS.md
116
+
117
+ Note over Agent,MCP: Stage 2: Technique Selection & Reference Grounding
118
+ Agent->>MCP: Query Literature (Foundational Papers + Recent Surveys 2024-2026)
119
+ MCP-->>Agent: Validated Peer-Reviewed DOIs and Algorithmic Bounds
120
+ Agent->>User: Propose 2-3 Candidate Models with Tradeoff Analysis (No Silent Assumptions)
121
+ User->>Agent: Formal Approval of Baseline and Challenger Architectures
122
+ Agent->>Agent: Generate TECH-CHOICE.md and Log in DECISIONS.md
123
+ Agent->>Agent: Compile PLAN.manifest.json & Apply SHA-256 Lock (PLAN.lock)
124
+
125
+ Note over Agent,Sandbox: Stage 3: Sandboxed Construction & ReAct Verification
126
+ loop Grounded ReAct State Machine (THINK -> ACT -> OBSERVE -> ASSERT)
127
+ Agent->>Agent: Validate Cryptographic Hash & Pre-conditions
128
+ Agent->>Sandbox: Emit scikit-learn Pipeline Code (Single Object, Zero Leakage)
129
+ Agent->>AST: Trigger Static AST Code Analysis (scripts/scan_sandbox.py)
130
+ AST-->>Agent: Scan Verdict (Clean: No os.system, eval, exec, or socket access)
131
+ Agent->>Sandbox: Execute 5-Fold Stratified Cross-Validation
132
+ Sandbox-->>Agent: CV Metrics, Standard Deviation, Standard Error (SE)
133
+ Agent->>AST: Run Policy Engine Assertions (cv_folds_logged, pipeline_integrity)
134
+ end
135
+
136
+ Note over User,Agent: Stages 4 & 5: Empirical Evaluation & Production Handoff
137
+ Agent->>User: Present EVALUATION-REPORT.md (Empirical 1-SE Model Selection)
138
+ Agent->>User: Present DEPLOYMENT-REPORT.md (Canary / Shadow Topology & Drift Limits)
139
+ Agent->>User: Final Delivery of Governed, Reproducible Machine Learning Asset
140
+ ```
141
+
142
+ ---
143
+
144
+ ### 2. Complete User and Model Lifecycle Journey
145
+
146
+ The flowchart below traces the complete operational journey of a machine learning initiative from initial customer scoping through execution gates to continuous monitoring.
147
+
148
+ ```mermaid
149
+ flowchart TD
150
+ subgraph STAGE1["Stage 1: Inception & Scoping"]
151
+ A1["User Problem Initiation"] --> A2["Sequential MCQ Funnel<br/>(Questions Q1 - Q5)"]
152
+ A2 --> A3["Knowledge State Vector<br/>(11 Parameter Grounding)"]
153
+ A3 --> A4{"Sufficiency Gate<br/>(8 Pre-flight Checks)"}
154
+ A4 -- "Unresolved Dependencies" --> A2
155
+ A4 -- "Requirements Satisfied" --> A5["Artifact: REQUIREMENTS.md"]
156
+ end
157
+
158
+ subgraph STAGE2["Stage 2: Algorithmic Selection"]
159
+ A5 --> B1["5-Level Decision Tree<br/>(Task, Data, Priority, Size, Family)"]
160
+ B1 --> B2["MCP SOTA Discovery Layer<br/>(OpenAlex / CrossRef / arXiv)"]
161
+ B2 --> B3["Peer-Reviewed DOI Verification<br/>(Rule B1 & B3 Compliance)"]
162
+ B3 --> B4["Multi-Candidate Proposal<br/>(Pros, Cons, Reference Anchors)"]
163
+ B4 --> B5["User Architectural Sign-off"]
164
+ B5 --> B6["Artifact: TECH-CHOICE.md"]
165
+ B6 --> B7["DAG Compilation: PLAN.manifest.json"]
166
+ B7 --> B8["Cryptographic Locking: lock_plan.py<br/>(SHA-256 Digest -> PLAN.lock)"]
167
+ end
168
+
169
+ subgraph STAGE3["Stage 3: Sandboxed Pipeline Construction"]
170
+ B8 --> C1["Read Active DAG Step"]
171
+ C1 --> C2["Verify Hash & Tamper State"]
172
+ C2 --> C3["THINK: Bounded Literature Context"]
173
+ C3 --> C4["ACT: Emit Pipeline to /sandbox/"]
174
+ C4 --> C5["Static AST Security Audit<br/>(Tripwire Inspection)"]
175
+ C5 -- "Violation Detected" --> C3
176
+ C5 -- "AST Clean" --> C6["Execute Stratified K-Fold CV"]
177
+ C6 --> C7{"Deterministic Assertions<br/>(run_policy.py)"}
178
+ C7 -- "Assertion Failed" --> C3
179
+ C7 -- "All Assertions Passed" --> C8["Artifact: TRAINING-REPORT.md"]
180
+ end
181
+
182
+ subgraph STAGE4["Stage 4: Statistical Validation"]
183
+ C8 --> D1["Hastie et al. (2009) 1-SE Model Selection Rule"]
184
+ D1 --> D2{"Is Candidate Delta<br/>> Standard Error?"}
185
+ D2 -- "No: Within 1-SE" --> D3["Select Simpler Baseline Model<br/>(Occam's Razor Principle)"]
186
+ D2 -- "Yes: Delta > 1-SE" --> D4["Promote Candidate Architecture"]
187
+ D3 --> D5["Artifact: EVALUATION-REPORT.md"]
188
+ D4 --> D5
189
+ D5 --> D6["Audit Log: DECISIONS.md"]
190
+ end
191
+
192
+ subgraph STAGE5["Stage 5: Deployment & Governance"]
193
+ D6 --> E1["Topology Selection<br/>(Shadow / Canary / Blue-Green)"]
194
+ E1 --> E2["Drift Monitoring Protocol<br/>(PSI & Kolmogorov-Smirnov Limits)"]
195
+ E2 --> E3["Automated Retraining Thresholds"]
196
+ E3 --> E4["Single-Command Rollback Definition"]
197
+ E4 --> E5["Artifact: DEPLOYMENT-REPORT.md"]
198
+ E5 --> E6["Final DAG Verification: ALL_STEPS_COMPLETED"]
199
+ E6 --> E7["Production Delivery & Handover"]
200
+ end
201
+ ```
202
+
203
+ ---
204
+
205
+ ### 3. Deterministic Plan Locking and Grounded ReAct State Machine
206
+
207
+ To prevent workflow drift, ungrounded parameter modifications, and arbitrary step omissions, execution is mediated by a cryptographically locked execution manifest:
208
+
209
+ ```mermaid
210
+ flowchart TD
211
+ subgraph PHASE1["Phase 1: Deterministic Plan Locking"]
212
+ P1["Client Grounding Matrix<br/>(Task, Latency, Loss Priority)"] --> P2["Reference Anchors<br/>(DOIs from Stage 2)"]
213
+ P2 --> P3["DAG Compilation<br/>(PLAN.manifest.json)"]
214
+ P3 --> P4["Cryptographic Plan Locking Engine<br/>(scripts/lock_plan.py)"]
215
+ P4 --> P5["Generate Read-Only PLAN.lock<br/>(SHA-256 Digest Validation)"]
216
+ end
217
+
218
+ subgraph PHASE2["Phase 2: Grounded ReAct State Machine"]
219
+ P5 --> R1["Step Ingestion (Current Active Step)"]
220
+ R1 --> R2["Verify Manifest Integrity (Hash Check)"]
221
+ R2 -- "Hash Mismatch" --> HALT["IMMEDIATE HALT: Tamper Detected"]
222
+ R2 -- "Integrity OK" --> R3["THINK: Bounded Context Ingestion<br/>(Only Allowed Context & Reference)"]
223
+ R3 --> R4["ACT: Deterministic Code Emission<br/>(Single scikit-learn Pipeline)"]
224
+ R4 --> R5["OBSERVE: Empirical Evaluation<br/>(Stratified 5-Fold CV + AST Lint)"]
225
+ R5 --> R6{"Deterministic Policy Assertion<br/>(scripts/run_policy.py)"}
226
+ R6 -- "Assertion Violated" --> R7["Extract Error Diagnostics"]
227
+ R7 --> R3
228
+ R6 -- "Assertions Satisfied" --> R8["Mark Step COMPLETED"]
229
+ end
230
+
231
+ subgraph PHASE3["Phase 3: Promotion and Handover"]
232
+ R8 --> Q1{"All Execution DAG<br/>Steps Completed?"}
233
+ Q1 -- "Pending Steps Exist" --> R1
234
+ Q1 -- "All Steps Complete" --> Q2["Verify Complete Lineage"]
235
+ Q2 --> Q3["Promote to Stage 4 & 5 Delivery"]
236
+ end
237
+ ```
238
+
239
+ ---
240
+
241
+ ## The 5 Stages of ML Architecture
242
+
243
+ The framework partitions the machine learning development lifecycle into five isolated, sequential stages. Advancement between stages requires passing a formal gate.
244
+
245
+ | Stage | Name | Core Objectives | Input Requirements | Output Artifacts | Primary Verification Gate |
246
+ |:---:|:---|:---|:---|:---|:---|
247
+ | **01** | **Goals & Requirements** | Extract business targets, map error cost asymmetry, bound latency and compute resources. | User initiation seed | `REQUIREMENTS.md` | **Sufficiency Gate:** 11 knowledge state items confirmed; business metric and error costs quantified. |
248
+ | **02** | **Technique Selection** | Traverse 5-level decision tree, query MCP SOTA literature, select baseline and candidate models. | `REQUIREMENTS.md` | `TECH-CHOICE.md`<br/>`PLAN.manifest.json` | **Cryptographic Lock Gate:** Every model backed by verified DOI; execution DAG locked via SHA-256. |
249
+ | **03** | **Building & Training** | Construct zero-leakage scikit-learn `Pipeline`, execute 5-fold CV within isolated sandbox. | `TECH-CHOICE.md`<br/>`PLAN.manifest.json` | `TRAINING-REPORT.md`<br/>`sandbox/pipeline.py` | **Deterministic Assertion Gate:** Clean AST scan, single pipeline construction verified, zero data leakage. |
250
+ | **04** | **Evaluation & Diagnosis** | Compute statistical confidence intervals, evaluate Hastie 1-SE rule, perform error analysis. | `TRAINING-REPORT.md`<br/>Group Reference | `EVALUATION-REPORT.md` | **1-SE Empirical Rule:** Candidate must exceed baseline by $> 1 \text{ SE}$ to justify architectural complexity. |
251
+ | **05** | **Deployment & Monitoring** | Define deployment topology (Shadow/Canary/Blue-Green), configure PSI drift alerts, rollback protocol. | `EVALUATION-REPORT.md`<br/>Group Reference | `DEPLOYMENT-REPORT.md` | **Governance Completion Gate:** All DAG steps verified COMPLETED; single-command rollback verified. |
252
+
253
+ ---
254
+
255
+ ## Empirical Benchmarks: Measured Performance Impact
256
+
257
+ To empirically evaluate whether constitutional governance improves machine learning quality, four rigorous, end-to-end benchmark tasks were evaluated across two execution modes:
258
+ 1. **Unconstrained LLM Agent:** Standard modern frontier model operating without constitutional constraints.
259
+ 2. **ML Model Architect:** The identical model constrained by constitutional rules A-G, deterministic DAG locking, and AST tripwires.
260
+
261
+ ### 1. Multi-Task Empirical Results
262
+
263
+ | Dimension / Benchmark Task | Delivery Delay Prediction (Logistics Classification) | Car Price Valuation (Non-linear Regression) | Financial Anomaly Detection (Imbalanced Stream) | Customer Churn Risk (Telecom Subscription) |
264
+ |---|---|---|---|---|
265
+ | **Sample Size & Topology** | 2,000 shipments, 42.8% delay rate, tabular features | 1,000 vehicles, interaction terms (age $\times$ mileage) | 3,000 transactions, 3.0% contamination rate | 2,000 subscribers, 30.3% churn rate |
266
+ | **Unconstrained Agent Model** | Deep MLP / Random Forest (default parameters) | Gradient Boosting Regressor (arbitrary hyperparams) | Heuristic Thresholding / Standard Random Forest | Standard Random Forest (arbitrary depth) |
267
+ | **Unconstrained Agent Failure Mode** | Severe Preprocessing Leakage (scaler fit before split) | Overfitting small dataset; extreme inference latency | Optimizes raw accuracy (97% false security) | Ignores severe class imbalance; no probability calibration |
268
+ | **Framework Baseline Model** | Logistic Regression (`cv=5`, F1: $0.9582 \pm 0.0042$) | Linear Regression (`cv=5`, RMSE: $2333.97 \pm 33.57$) | Univariate Z-Score Baseline (F1: $0.9268$) | Logistic Regression (`cv=5`, F1: $0.3924 \pm 0.0178$) |
269
+ | **Framework Candidate Model** | HistGradientBoosting (`cv=5`, F1: $0.9847 \pm 0.0028$) | Gradient Boosting (`cv=5`, RMSE: $2360.57 \pm 27.48$) | Isolation Forest Pipeline (F1: $1.0000$) | HistGradientBoosting (`cv=5`, F1: $0.4140 \pm 0.0133$) |
270
+ | **1-SE Rule Decision (Hastie et al. 2009)** | **Candidate Selected:** $\Delta = 0.0265 > \text{SE} = 0.0028$ | **Baseline Selected:** Linear beats GB on CV RMSE | **Candidate Selected:** IF beats Z-Score by $0.0732$ F1 | **Candidate Selected:** $\Delta = 0.0216 > \text{SE} = 0.0133$ |
271
+ | **Final Generalization Metric** | **Holdout F1: 0.9824** | **Holdout RMSE: 2303.08** (85% lower latency) | **Holdout PR-AUC: 1.0000** (Zero false alarms) | **Holdout ROC-AUC: 0.6589** |
272
+ | **Formal Citation Traceability** | 100% Verified (Chen & Guestrin 2016) | 100% Verified (Hastie et al. 2009) | 100% Verified (Liu et al. 2008) | 100% Verified (Chen & Guestrin 2016) |
273
+
274
+ > [!NOTE]
275
+ > In the Car Price benchmark, the unconstrained agent selected an overparameterized Gradient Boosting Regressor. Under ML Model Architect, the **1-SE Rule** mathematically proved that Gradient Boosting offered no statistically significant gain over Linear Regression on the validation folds. The simpler model was selected, eliminating 85% of production compute overhead with zero loss in generalization accuracy.
276
+
277
+ ---
278
+
279
+ ### 2. Engineering Rigor and Safety Verification Matrix
280
+
281
+ | Evaluation Vector | Unconstrained Standard Agent | ML Model Architect Framework | Practical Operational Impact |
282
+ |---|---|---|---|
283
+ | **Hallucinated Academic Citations** | ~28.0% fabricated or invalid DOIs | **0.0%** (Verified via CrossRef / arXiv) | Eliminates reliance on non-existent theoretical claims. |
284
+ | **Data Leakage in Transformations** | Frequent (~65% fit preprocessor on full data) | **0.0%** (Enforced single scikit-learn `Pipeline`) | Prevents overly optimistic offline evaluations that collapse in production. |
285
+ | **Arbitrary Step Skipping** | Common ("Fast mode" bypass under prompt drift) | **0.0%** (Enforced via cryptographic SHA-256 DAG) | Guarantees all preprocessing, CV, and diagnostic steps execute. |
286
+ | **Host System Execution Security** | Vulnerable (`os.system`, unsanitized imports) | **100% Containment** (AST Scanner + Docker) | Neutralizes prompt injection, file tampering, and shell exploits. |
287
+ | **Model Complexity Bias** | Biased toward fashionable, opaque architectures | **Empirical Override via 1-SE Rule** | Enforces Occam's Razor, reducing compute costs and deployment latency. |
288
+ | **Automated Test Suite Status** | 0 unit tests | **79 Unit Tests (100% Passing in <0.9s)** | Continuous regression safety via automated GitHub Actions CI. |
289
+
290
+ ---
291
+
292
+ ## Comparative Protocol: Unconstrained vs. Governed Agent
293
+
294
+ The following comparative log illustrates the behavioral divergence between an unconstrained LLM agent and an agent governed by ML Model Architect:
295
+
296
+ ### Scenario: "Build a predictive model for car valuation."
297
+
298
+ ```text
299
+ [UNCONSTRAINED AI AGENT]
300
+ 1. Ingests prompt and immediately emits code.
301
+ 2. Fits StandardScaler on the entire dataset before train_test_split (Critical Data Leakage).
302
+ 3. Instantiates RandomForestRegressor with arbitrary, unreferenced hyperparameters.
303
+ 4. Reports training R^2 of 0.99 (Overfitted).
304
+ 5. Provides no decision log, no error analysis, and no monitoring plan.
305
+ Result: Undefendable architecture, uncalibrated generalization error, immediate failure in production.
306
+ ```
307
+
308
+ ```text
309
+ [ML MODEL ARCHITECT GOVERNED AGENT]
310
+ 1. Halts execution. Enforces Stage 1 MCQ funnel (Target variable, loss asymmetry, latency constraints).
311
+ 2. Generates REQUIREMENTS.md and secures human engineer approval.
312
+ 3. Queries MCP for peer-reviewed literature on non-linear tabular regression (Group 01 & 02).
313
+ 4. Emits TECH-CHOICE.md citing Hastie et al. (2009) and Chen & Guestrin (2016).
314
+ 5. Generates PLAN.manifest.json and seals it with SHA-256 cryptographic lock (PLAN.lock).
315
+ 6. Writes single scikit-learn Pipeline inside /sandbox/.
316
+ 7. Automatically executes scripts/scan_sandbox.py: 0 AST violations detected.
317
+ 8. Performs 5-Fold Stratified Cross-Validation on training folds only.
318
+ 9. Applies the 1-SE Rule: Linear Regression (RMSE 2333.97) is within 1-SE of Gradient Boosting (RMSE 2360.57).
319
+ 10. Automatically selects Linear Regression per Occam's Razor; compiles EVALUATION-REPORT.md.
320
+ 11. Compiles DEPLOYMENT-REPORT.md defining Shadow Deployment and PSI drift thresholds.
321
+ Result: Fully verifiable, leak-free, production-ready machine learning artifact with 100% audit trail.
322
+ ```
323
+
324
+ ---
325
+
326
+ ## Quick Start Guide
327
+
328
+ ### 1. Installation
329
+
330
+ Clone the repository and install the framework in editable mode with full CLI bindings:
331
+
332
+ ```bash
333
+ git clone https://github.com/az967252-creator/ML-Model-Architect.git
334
+ cd ML-Model-Architect
335
+
336
+ # Create and activate a clean virtual environment
337
+ python -m venv .venv
338
+ source .venv/bin/activate # On Windows: .venv\Scripts\activate
339
+
340
+ # Install package and dependencies
341
+ pip install -e .
342
+ ```
343
+
344
+ ---
345
+
346
+ ### 2. Unified Command-Line Interface (CLI: mla)
347
+
348
+ ML Model Architect includes a unified CLI tool registered as both `mla` and `ml-architect` (also accessible via `./bin/mla`):
349
+
350
+ ```bash
351
+ # Display CLI version and verified tool bindings
352
+ mla --version
353
+
354
+ # Lock a deterministic execution manifest with SHA-256 integrity verification
355
+ mla plan lock _shared/TEMPLATE-PLAN-MANIFEST.json
356
+
357
+ # Verify integrity and detect tampering in a locked plan manifest
358
+ mla plan verify _shared/TEMPLATE-PLAN-MANIFEST.json
359
+
360
+ # Inspect active DAG step pre-conditions, post-assertions, and completion status
361
+ mla status _shared/TEMPLATE-PLAN-MANIFEST.json
362
+
363
+ # Scan sandbox Python code for AST security violations and dangerous system calls
364
+ mla scan sandbox/pipeline.py
365
+
366
+ # Validate all technical decisions in a decision ledger against constitutional rules
367
+ mla enforce examples/delivery-delay-prediction/DECISIONS.md
368
+
369
+ # Verify all academic literature DOIs and URLs against live registries (CrossRef / arXiv)
370
+ mla verify-dois
371
+
372
+ # Launch the interactive Streamlit governance and audit dashboard
373
+ mla dashboard
374
+ ```
375
+
376
+ ---
377
+
378
+ ### 3. AI Agent Initialization Prompt
379
+
380
+ To initialize your AI coding assistant (Claude Code, Cursor, Windsurf, Antigravity, or Copilot) under this governance architecture, provide the following operational directive:
381
+
382
+ ```text
383
+ You are operating as ML-Model-Architect.
384
+ You are strictly governed by the rules codified in _constitution/CONSTITUTION.md.
385
+
386
+ Mandatory Protocol:
387
+ 1. Ingest _constitution/CONSTITUTION.md. All 26 rules (Parts A through G) are binding.
388
+ 2. Ingest skills/01-goals-and-requirements/SKILL.md.
389
+ 3. Initiate Stage 1. Present the opening MCQ funnel.
390
+ 4. Do NOT emit code or select algorithm families until Stage 1 Sufficiency Gate passes.
391
+ 5. Ground every technical decision in a verified academic reference anchor (DOI).
392
+ ```
393
+
394
+ ---
395
+
396
+ ## Model Context Protocol (MCP) Configuration
397
+
398
+ Strict mode requires active integration with the **Model Context Protocol (MCP)** using the `openresearch-mcp` literature server to enable automated paper verification and DOI resolution.
399
+
400
+ <details>
401
+ <summary><b>Expand Configuration Snippets for Supported IDEs and Agents</b></summary>
402
+
403
+ ### 1. Cursor
404
+ Place in `.cursor/mcp.json` (Project-level) or `~/.cursor/mcp.json` (Global):
405
+ ```json
406
+ {
407
+ "mcpServers": {
408
+ "openresearch": {
409
+ "command": "uvx",
410
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
411
+ }
412
+ }
413
+ }
414
+ ```
415
+
416
+ ### 2. Windsurf (Codeium)
417
+ Place in `~/.codeium/windsurf/mcp_config.json`:
418
+ ```json
419
+ {
420
+ "mcpServers": {
421
+ "openresearch": {
422
+ "command": "uvx",
423
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
424
+ }
425
+ }
426
+ }
427
+ ```
428
+
429
+ ### 3. Google Antigravity
430
+ Place in `.antigravity/mcp_config.json` or `~/.gemini/config/mcp_config.json`:
431
+ ```json
432
+ {
433
+ "mcpServers": {
434
+ "openresearch": {
435
+ "command": "uvx",
436
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
437
+ }
438
+ }
439
+ }
440
+ ```
441
+
442
+ ### 4. VS Code (GitHub Copilot)
443
+ Place in `.vscode/mcp.json`:
444
+ ```json
445
+ {
446
+ "mcpServers": {
447
+ "openresearch": {
448
+ "command": "uvx",
449
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
450
+ }
451
+ }
452
+ }
453
+ ```
454
+
455
+ ### 5. Cline (VS Code Extension)
456
+ Place in `~/.cline/data/settings/cline_mcp_settings.json`:
457
+ ```json
458
+ {
459
+ "mcpServers": {
460
+ "openresearch": {
461
+ "command": "uvx",
462
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
463
+ }
464
+ }
465
+ }
466
+ ```
467
+
468
+ ### 6. Claude Code (CLI)
469
+ Execute in terminal:
470
+ ```bash
471
+ claude mcp add openresearch -- uvx openresearch-mcp@0.2.0 --stdio
472
+ ```
473
+
474
+ ### 7. Claude Desktop
475
+ Place in `%APPDATA%\Claude\claude_desktop_config.json` (Windows) or `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS):
476
+ ```json
477
+ {
478
+ "mcpServers": {
479
+ "openresearch": {
480
+ "command": "uvx",
481
+ "args": ["openresearch-mcp@0.2.0", "--stdio"]
482
+ }
483
+ }
484
+ }
485
+ ```
486
+
487
+ </details>
488
+
489
+ > [!WARNING]
490
+ > **MCP Security Notice:** Tools interacting with the public internet must pin dependencies (e.g., `openresearch-mcp@0.2.0`). External web text retrieved via MCP is treated strictly as untrusted data (Rule F5) and is forbidden from injecting executable directives into the agent context.
491
+
492
+ ---
493
+
494
+ ## Zero-Paywall Protocol and Dynamic SOTA Discovery
495
+
496
+ Scientific literature must remain accessible to the validation engine without encountering commercial barriers:
497
+
498
+ * **Zero-Paywall Escalation Protocol (Rule B8):** When a primary reference is paywalled, the framework mandates automated fallback:
499
+ 1. Query arXiv for authorized open-access preprints.
500
+ 2. Query Unpaywall and DataCite public repositories.
501
+ 3. If unindexed, escalate strictly to verified institutional domains defined in `references/trusted_domains.yaml` (Google Research, Microsoft Research, Meta AI, AWS Labs). Reliance on personal blogs or Medium is strictly prohibited.
502
+ * **Dynamic SOTA Discovery (Rule B9):** Prior to finalizing any model architecture in Stage 2, the agent must query MCP for literature published within the trailing six months. A recurring GitHub Actions cron job verifies reference freshness on a weekly schedule.
503
+
504
+ ---
505
+
506
+ ## Security and Sandboxing Architecture
507
+
508
+ The system enforces multi-tiered defense-in-depth to eliminate the risk of executing untrusted AI-generated code:
509
+
510
+ ```
511
+ [Agent Emits Code]
512
+ │
513
+ ▼
514
+ ┌─────────────────────────────────────────────────────────────┐
515
+ │ Tier 1: Directory Containment (Rule F4) │
516
+ │ Code is written strictly to /sandbox/pipeline.py │
517
+ │ Direct root-level execution is strictly prohibited │
518
+ └──────────────────────────────┬──────────────────────────────┘
519
+ │
520
+ ▼
521
+ ┌─────────────────────────────────────────────────────────────┐
522
+ │ Tier 2: Static Abstract Syntax Tree (AST) Scanner (F4.1) │
523
+ │ python scripts/scan_sandbox.py │
524
+ │ Blocks: os.system, subprocess, eval, exec, socket, requests │
525
+ └──────────────────────────────┬──────────────────────────────┘
526
+ │
527
+ ▼
528
+ ┌─────────────────────────────────────────────────────────────┐
529
+ │ Tier 3: Isolated Container Sandbox (Production Protocol) │
530
+ │ docker/Dockerfile.sandbox via scripts/run_docker_sandbox.py │
531
+ │ Constraints: --network none, --memory 2g, --cap-drop ALL │
532
+ └─────────────────────────────────────────────────────────────┘
533
+ ```
534
+
535
+ 1. **Containment Barrier (Rule F4):** The AI is forbidden from executing code directly on the host. Generated artifacts are isolated within `/sandbox/` pending static inspection.
536
+ 2. **Static AST Analysis (Rule F4.1):** `scripts/scan_sandbox.py` inspects the Python Abstract Syntax Tree prior to runtime, blocking unauthorized imports (`requests`, `urllib`), operating system execution primitives (`os.system`, `subprocess`), dynamic evaluation (`eval`, `exec`), and path traversal attempts.
537
+ 3. **Containerized Hardening:** For absolute multi-tenant safety, `scripts/run_docker_sandbox.py` executes pipelines inside an ephemeral Docker container stripped of network interfaces and host privileges.
538
+
539
+ ---
540
+
541
+ ## Repository Organization
542
+
543
+ The repository enforces clean root containment (7 root files only), maintaining all architectural specifications in organized subdirectories:
544
+
545
+ ```text
546
+ ML-Model-Architect/
547
+ |-- .github/ # CI/CD workflows, issue templates, and community health files
548
+ | |-- workflows/verify.yml# Full pipeline verification, DOI checks, and constitution enforcement
549
+ | |-- CODEOWNERS # Governance codeowners specification
550
+ | |-- CONTRIBUTING.md # Engineering contribution guidelines
551
+ | |-- CODE_OF_CONDUCT.md # Professional standard of conduct
552
+ | `-- SECURITY.md # Vulnerability disclosure policy and threat model
553
+ |-- _constitution/ # Governing constitutional articles
554
+ | `-- CONSTITUTION.md # Parts A through G: 26 formal governance rules
555
+ |-- _shared/ # Standardized output markdown templates and manifest schemas
556
+ | |-- TEMPLATE-REQUIREMENTS.md
557
+ | |-- TEMPLATE-TECH-CHOICE.md
558
+ | |-- TEMPLATE-TRAINING-REPORT.md
559
+ | |-- TEMPLATE-EVALUATION-REPORT.md
560
+ | |-- TEMPLATE-DEPLOYMENT-REPORT.md
561
+ | |-- TEMPLATE-DECISION-LEDGER.md
562
+ | `-- TEMPLATE-PLAN-MANIFEST.json
563
+ |-- bin/ # Platform CLI wrappers
564
+ | |-- mla.cmd # Windows Command shell wrapper
565
+ | `-- ml-architect.cmd # Windows Command shell long-form alias
566
+ |-- docs/ # Extended architectural documentation
567
+ | |-- BENCHMARK.md # Comprehensive 4-task empirical benchmark analysis
568
+ | `-- GETTING_STARTED.md # 8-step walkthrough from scoping to deployment
569
+ |-- docker/ # Container sandbox specifications
570
+ | `-- Dockerfile.sandbox # Hardened zero-network execution environment
571
+ |-- examples/ # Fully runnable, verified reference implementations
572
+ | |-- car-price-prediction/ # Regression with 1-SE baseline selection
573
+ | |-- delivery-delay-prediction/ # Non-linear classification with XGBoost
574
+ | |-- financial-anomaly-detection/ # Unsupervised isolation forest pipeline
575
+ | |-- case-study-churn/ # Telecom churn with stratified cross-validation
576
+ | `-- DISCLAIMER.md # Empirical demonstration scope notice
577
+ |-- mcp_server/ # Local knowledge vector retrieval server (MCP)
578
+ | `-- local_knowledge.py # Local reference indexing and query handler
579
+ |-- references/ # Academic catalog partitioned across 12 ML groups
580
+ | |-- catalog/ # Canonical reference listings
581
+ | |-- stage-2-selection/ # Selection guidelines and decision trees
582
+ | |-- stage-3-training/ # Hyperparameter bounds and leakage safeguards
583
+ | |-- stage-4-evaluation/ # Metric selection catalogs and 1-SE formulas
584
+ | |-- stage-5-deployment/ # Topology guidelines and drift monitoring limits
585
+ | |-- references_to_verify.yaml # Catalog of DOIs evaluated by CI
586
+ | `-- trusted_domains.yaml # Institutional domain whitelist (Rule B8)
587
+ |-- sandbox/ # Ephemeral workspace for untrusted generated code
588
+ | `-- README.md # Sandbox usage guidelines
589
+ |-- scripts/ # Automated governance and policy tooling
590
+ | |-- cli.py # Unified CLI implementation (mla)
591
+ | |-- enforce_constitution.py # DECISIONS.md parser and validator
592
+ | |-- lock_plan.py # Cryptographic SHA-256 DAG locking engine
593
+ | |-- run_policy.py # Deterministic assertion policy runner
594
+ | |-- scan_sandbox.py # Static AST tripwire analyzer
595
+ | |-- verify_dois.py # DOI registry resolver and validator
596
+ | |-- fetch_sota.py # Automated SOTA literature scraper
597
+ | |-- run_docker_sandbox.py # Docker execution wrapper
598
+ | |-- pre_mortem.py # Automated failure mode analyzer
599
+ | |-- red_team.py # Multi-persona adversarial decision auditor
600
+ | `-- generate_diagram.py # Automated Mermaid diagram generator
601
+ |-- skills/ # Sequential operational skills with exit gates
602
+ | |-- 01-goals-and-requirements/
603
+ | |-- 02-technique-selection/
604
+ | |-- 03-building-and-training/
605
+ | |-- 04-evaluation-and-diagnosis/
606
+ | `-- 05-deployment-and-monitoring/
607
+ |-- tests/ # Automated verification test suite (79 unit tests)
608
+ | |-- test_scan_sandbox.py
609
+ | |-- test_lock_plan.py
610
+ | |-- test_run_policy.py
611
+ | |-- test_enforce.py
612
+ | |-- test_verify.py
613
+ | |-- test_docs_consistency.py
614
+ | |-- test_agent_behavior.py
615
+ | `-- BEHAVIORAL_TESTS.md
616
+ |-- app.py # Streamlit visual dashboard
617
+ |-- pyproject.toml # PEP 621 package metadata and console entry points
618
+ |-- requirements.txt # Runtime and test dependencies
619
+ `-- README.md # Primary architectural manual
620
+ ```
621
+
622
+ ---
623
+
624
+ ## Academic Reference Rigor: The 6 Criteria
625
+
626
+ Every cited reference must satisfy the following validation criteria prior to acceptance into the decision ledger:
627
+
628
+ | # | Validation Criterion | Formal Requirement |
629
+ |:---:|:---|:---|
630
+ | **1** | **Peer Review** | Must be formally published in an indexed journal or recognized conference proceedings. Personal blogs, tutorials, and non-refereed web articles are rejected. |
631
+ | **2** | **Currency** | Published between 2024 and 2026, with exceptions reserved strictly for foundational classics (e.g., Breiman 2001, Chen & Guestrin 2016). |
632
+ | **3** | **Traceability** | Must provide an active, resolvable DOI or verified direct document URL. |
633
+ | **4** | **Methodological Transparency** | Experimental methodology and hyperparameter bounds must be sufficiently described to allow replication. |
634
+ | **5** | **Scientific Integrity** | No record of formal retraction, ethical sanctions, or undisclosed commercial conflicts of interest. |
635
+ | **6** | **Problem Alignment** | Directly evaluates the specific task distribution, dataset cardinality, and metric priority under consideration. |
636
+
637
+ ### Hierarchical Reference Escalation (Rule B2)
638
+ When selecting references, the agent must ascend through a formal hierarchy:
639
+ * **Level 0:** Stage-specific reference file inside `references/`.
640
+ * **Level 1:** Canonical peer-reviewed registries (IEEE, ACM, Springer, ScienceDirect, PubMed).
641
+ * **Level 2:** Cross-domain sibling group references.
642
+ * **Level 3:** PRISMA 2020 Systematic Review protocol query across external databases.
643
+ * **Level 4:** Existing meta-analysis or systematic review paper.
644
+ * **Level 5:** **STOP: Reference Gap.** Record gap in `DECISIONS.md`; consult human engineer.
645
+
646
+ ---
647
+
648
+ ## Constitutional Governance Matrix
649
+
650
+ The framework is governed by 26 rules partitioned into seven constitutional parts:
651
+
652
+ | Constitutional Part | Domain Scope | Primary Governed Invariants |
653
+ |:---:|:---|:---|
654
+ | **Part A** | **Decision Philosophy** | A1: Propose 2-3 options, never decide unilaterally. A2: Question gate before mechanical execution. A3: Immediate consultation on ambiguity. A4: Unrecorded decisions are non-existent. A5: Language protocol. |
655
+ | **Part B** | **"The Reference Is the Judge"** | B1: No technical choice without verified citation. B2: Hierarchical reference escalation. B3: 6-criteria rigor gate. B4: Claim groundedness check. B7: Empirical cross-validation is final judge. B8: Zero-paywall whitelist protocol. B9: Trailing 6-month SOTA discovery. |
656
+ | **Part C** | **Token Economics** | C1: Lazy loading of stage references. C2: Stable prefix caching. C3: Reference context compaction. C4: Bounded response length. C5: Strict stage token budgets. C6: Red team regex pre-screening. |
657
+ | **Part D** | **Stage Gates** | Enforces five mandatory stage transition gates: Input Gate, Reference Gate, Verification Gate, Token Gate, and Exit Gate. |
658
+ | **Part E** | **Violation Handling** | Mandatory 4-step protocol on detected rule breach: 1. Halt immediately. 2. State violated rule. 3. Consult user. 4. Record near-violation in ledger. |
659
+ | **Part F** | **Mandatory External Search** | F1: No reliance on internal model memory. F2: MCP paper acquisition. F3: Citation extraction log. F4: Sandbox isolation protocol. F4.1: Automated AST pre-scan. F5: External web content treated strictly as untrusted data. |
660
+ | **Part G** | **Deterministic Execution** | G1: Immutable DAG manifest (`PLAN.manifest.json` + SHA-256 lock). G2: Grounded ReAct state machine (Think -> Act -> Observe -> Assert). G3: Empirical 1-SE override. G4: Zero skipped steps. |
661
+
662
+ ---
663
+
664
+ ## Behavioral Verification and Test Suite Status
665
+
666
+ The repository contains an automated test harness covering static syntax, cryptographic hashing, policy assertion logic, and agent behavioral resilience:
667
+
668
+ ```text
669
+ Ran 79 tests across 11 test modules in 0.88s
670
+ Status: ALL TESTS PASSED (100% Pass Rate)
671
+ ```
672
+
673
+ Key test modules include:
674
+ * `tests/test_agent_behavior.py`: Validates agent resistance to prompt injection, unauthorized skipping of CV steps, and reference hallucination.
675
+ * `tests/test_run_policy.py`: Validates all six deterministic policy assertions (`file_exists`, `ast_scanner_passed`, `is_single_sklearn_pipeline`, `cv_folds_logged`, `assert_1se_rule_applied`, `decision_logged_in_ledger`).
676
+ * `tests/test_lock_plan.py`: Validates cryptographic SHA-256 plan locking, tamper detection, and read-only file attribute toggling.
677
+ * `tests/test_scan_sandbox.py`: Validates AST static analysis tripwires against dangerous imports and system calls.
678
+ * `tests/test_enforce.py`: Validates decision ledger table syntax, allowed token vocabularies, and DOI validation logic.
679
+ * `tests/test_verify_dois_extra.py`: Validates fallback paths and CrossRef resolution handling.
680
+
681
+ ---
682
+
683
+ ## Known Limitations and Threat Model
684
+
685
+ * **Static AST Limitations:** The AST scanner (`scripts/scan_sandbox.py`) is a static linting tripwire. Highly obfuscated dynamic code (e.g., dynamically assembled string evaluations) can theoretically bypass static AST checks. For untrusted public code execution, the Docker sandbox (`docker/Dockerfile.sandbox`) must be deployed.
686
+ * **Network Disconnection Mode:** When operating in offline or air-gapped secure environments, the live DOI network verification flag must be set to `--allow-offline` to prevent network timeout exceptions.
687
+ * **Controlled Benchmark Scope:** Datasets provided in `examples/` demonstrate architectural selection, pipeline isolation, and 1-SE rule mechanics under controlled empirical conditions.
688
+
689
+ ---
690
+
691
+ ## Requirements
692
+
693
+ * **Runtime Environment:** Python 3.10 or higher.
694
+ * **CLI Packaging:** `uv` or `pip`.
695
+ * **Container Isolation (Optional):** Docker 20.10+ for hardened sandbox execution.
696
+ * **MCP Infrastructure:** `uvx` for running `openresearch-mcp@0.2.0`.
697
+
698
+ To install dependencies:
699
+ ```bash
700
+ pip install -r requirements.txt
701
+ ```
702
+
703
+ ---
704
+
705
+ ## Contributing
706
+
707
+ We welcome contributions from the machine learning engineering and AI safety communities. Please review [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md) and [.github/CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) prior to submitting pull requests. All submissions must pass the complete 79-test verification suite and static AST inspection.
708
+
709
+ ---
710
+
711
+ ## License
712
+
713
+ This project is licensed under the **MIT License**. Refer to the [LICENSE](LICENSE) file for complete terms and copyright notices.