myai-builder 0.1.0__tar.gz

myai_builder-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

myai_builder-0.1.0/PKG-INFO

@@ -0,0 +1,428 @@
+Metadata-Version: 2.4
+Name: myai-builder
+Version: 0.1.0
+Summary: The Local-First Autonomous AI Model Builder & Standalone Runtime Packager
+Author: Kiran Sai
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/kiransai-62/myai
+Project-URL: Documentation, https://github.com/kiransai-62/myai#readme
+Project-URL: Repository, https://github.com/kiransai-62/myai.git
+Project-URL: Issues, https://github.com/kiransai-62/myai/issues
+Keywords: ai,llm,fine-tuning,lora,qlora,local-ai,quantization,gguf,ollama
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Build Tools
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typer>=0.12.3
+Requires-Dist: rich>=13.7.1
+Requires-Dist: PyYAML>=6.0.1
+Requires-Dist: psutil>=5.9.8
+Provides-Extra: models
+Requires-Dist: huggingface_hub>=0.23.0; extra == "models"
+Provides-Extra: train
+Requires-Dist: torch>=2.3.0; extra == "train"
+Requires-Dist: transformers>=4.43.0; extra == "train"
+Requires-Dist: peft>=0.11.0; extra == "train"
+Requires-Dist: bitsandbytes>=0.43.0; extra == "train"
+Requires-Dist: accelerate>=0.30.0; extra == "train"
+Provides-Extra: retrieval
+Requires-Dist: sentence-transformers>=3.0.0; extra == "retrieval"
+Provides-Extra: knowledge
+Requires-Dist: numpy>=1.24.0; extra == "knowledge"
+Provides-Extra: serving
+Requires-Dist: fastapi>=0.111.0; extra == "serving"
+Requires-Dist: uvicorn>=0.30.0; extra == "serving"
+Requires-Dist: pydantic>=2.7.0; extra == "serving"
+Requires-Dist: requests>=2.31.0; extra == "serving"
+Requires-Dist: httpx>=0.27.0; extra == "serving"
+Provides-Extra: serve
+Requires-Dist: fastapi>=0.111.0; extra == "serve"
+Requires-Dist: uvicorn>=0.30.0; extra == "serve"
+Requires-Dist: pydantic>=2.7.0; extra == "serve"
+Requires-Dist: requests>=2.31.0; extra == "serve"
+Requires-Dist: httpx>=0.27.0; extra == "serve"
+Provides-Extra: eval
+Requires-Dist: textstat>=0.7.3; extra == "eval"
+Requires-Dist: nltk>=3.8.1; extra == "eval"
+Requires-Dist: rouge-score>=0.1.2; extra == "eval"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.2.0; extra == "dev"
+Requires-Dist: httpx>=0.27.0; extra == "dev"
+Dynamic: license-file
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/kiransai-62/myai/main/assets/myai-logo.png" alt="MYAI Logo - Build · Train · Evolve" width="340">
+</p>
+
+<h1 align="center">MYAI</h1>
+
+<p align="center">
+  <strong>The Local-First Autonomous AI Model Builder & Standalone Runtime Packager.</strong><br>
+  <em>BUILD · TRAIN · EVOLVE</em>
+</p>
+
+<p align="center">
+  <a href="#-installation">Installation</a> &middot;
+  <a href="#-quickstart-in-3-commands">Quickstart</a> &middot;
+  <a href="#-why-myai">Why MYAI?</a> &middot;
+  <a href="#-key-capabilities">Capabilities</a> &middot;
+  <a href="#-hardware-tiers--supported-models">Hardware & Models</a> &middot;
+  <a href="#-step-by-step-workflow-guide">Workflow Guide</a> &middot;
+  <a href="#-standalone-runtime-export">Standalone Runtime</a> &middot;
+  <a href="#-complete-cli-reference">CLI Reference</a> &middot;
+  <a href="#-security--containment-policies">Security & Privacy</a>
+</p>
+
+<p align="center">
+  <a href="https://github.com/kiransai-62/myai/actions/workflows/test.yml"><img src="https://github.com/kiransai-62/myai/actions/workflows/test.yml/badge.svg" alt="CI Tests"></a>
+  <a href="https://github.com/kiransai-62/myai"><img src="https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white" alt="Python 3.10+"></a>
+  <a href="https://github.com/kiransai-62/myai/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-green.svg" alt="License"></a>
+  <a href="https://github.com/kiransai-62/myai/blob/main/docs/hardware_catalog.md"><img src="https://img.shields.io/badge/hardware-tier%20matrix-blueviolet" alt="Hardware Intelligence"></a>
+  <a href="#-post-training-preference-alignment"><img src="https://img.shields.io/badge/alignment-DPO%20%7C%20ORPO%20%7C%20SimPO%20%7C%20KTO-orange" alt="Alignment"></a>
+  <a href="#-security--containment-policies"><img src="https://img.shields.io/badge/export%20gate-verified%20containment-brightgreen" alt="Security Gate"></a>
+  <img src="https://img.shields.io/badge/privacy-local--first%20%26%20private-success" alt="Local First Privacy">
+</p>
+
+---
+
+Tell MYAI what AI you want. MYAI analyzes your **goal**, your **hardware**, and your **data** — then helps build, evaluate, optimize, and package your custom model into a standalone, portable runtime with its own Web & CLI Chat UI, or exports it to GGUF format for Ollama and `llama.cpp`.
+
+> **No cloud GPUs required. Local data privacy. No framework lock-in.**
+
+---
+
+## 📦 Installation
+
+### From PyPI (Recommended)
+
+```bash
+# Basic installation
+pip install myai-builder
+
+# Recommended: includes test suite, local serving API, and evaluation metrics
+pip install "myai-builder[serving,eval]"
+
+# Full installation: adds PyTorch, LoRA/QLoRA training & Hugging Face models
+pip install "myai-builder[dev,serving,eval,train,models,retrieval]"
+```
+
+### From Source (Development)
+
+```bash
+git clone https://github.com/kiransai-62/myai.git
+cd myai
+pip install -e ".[dev,serving,eval,train,models,retrieval]"
+```
+
+### Dependency Groups
+
+| Group | Key Dependencies | Purpose |
+| :--- | :--- | :--- |
+| `dev` | `pytest`, `httpx` | Running test suites and security audit checks |
+| `serving` / `serve` | `fastapi`, `uvicorn`, `pydantic`, `requests`, `httpx` | Local-first REST and SSE streaming inference server (`myai serve`) |
+| `eval` | `textstat`, `nltk`, `rouge-score` | Quality scoring, linguistic readability, and evaluation metrics |
+| `train` | `torch`, `transformers`, `peft`, `bitsandbytes`, `accelerate` | LoRA / QLoRA fine-tuning and layer streaming |
+| `models` | `huggingface_hub` | Base model downloading and registry access |
+| `retrieval` | `sentence-transformers` | Semantic search and embedding-based knowledge indexes |
+
+---
+
+## ⚡ Quickstart in 3 Commands
+
+```bash
+# 1. Initialize in current directory (or create a subfolder with 'myai init <name>')
+myai init .
+
+# 2. Add your dataset in-place (JSONL, JSON, CSV, TXT, Parquet)
+myai data add <path/to-data>      # e.g., ./coaching_data.jsonl
+
+# 3. Autopilot: Goal → Hardware & Data → Train → Eval → Optimize → Export
+myai auto --export
+```
+
+### 🖥️ Autopilot Workflow Overview
+
+```text
+Goal Specification
+       ↓
+Hardware & Data Analysis
+       ↓
+Model Recommendation
+       ↓
+Fine-Tuning & Alignment
+       ↓
+Evaluation & Verification
+       ↓
+Hyperparameter Optimization
+       ↓
+Security & Containment Gate
+       ↓
+Standalone Export & Deployment
+```
+
+---
+
+## 💡 Why MYAI?
+
+Building, evaluating, and packaging fine-tuned LLMs has traditionally been fragmented across ad-hoc scripts, CUDA out-of-memory errors, disconnected evaluation tools, and bulky serving setups. MYAI unifies the entire stack into an **autonomous, local-first platform**:
+
+| Feature | Description |
+| :--- | :--- |
+| 🔒 **Local-First & Private** | Compute, tokenization, training, and evaluation run on your local machine with no external cloud telemetry. |
+| 🌊 **Memory-Efficient Training** | Adaptive layer streaming enables fine-tuning on budget and laptop GPUs. |
+| 🖥️ **Hardware-Aware Intelligence** | Analyzes system CPU, RAM, and GPU/VRAM to match feasible models and context windows. |
+| 🎯 **Post-Training Alignment** | Preference optimization supporting **DPO, ORPO, SimPO, and KTO**. |
+| 🧠 **Automated Task Verification** | Evaluates model outputs against task criteria and reference datasets. |
+| 🛡️ **Regression Quality Gate** | Bundled offline test suites (format compliance, tool calling, arithmetic, safety) issuing automated SHIP / DON'T-SHIP verdicts. |
+| 📦 **Standalone Runtime Exports** | Self-contained ZIP packages containing a built-in Web Chat UI, GGUF format for Ollama, and merged weight checkpoints. |
+
+---
+
+## 🌟 Key Capabilities
+
+```mermaid
+graph TD
+    A["🎯 Goal Definition (Task, Domain, Context)"] --> B["🖥️ Hardware & Resource Profiling"]
+    B --> C["🧹 Dataset Intelligence (Reference Mode & Token Analysis)"]
+    C --> D["⚖️ Model Recommendation & Resource Feasibility"]
+    D --> E["⚙️ Adaptive Training Strategy"]
+    E --> F["🏗️ Fine-Tuning & Preference Alignment (LoRA / QLoRA / DPO / SimPO)"]
+    F --> G["🏆 Goal-Aligned Evaluation & Leaderboard"]
+    G --> H["🔧 Automated Optimization Loop"]
+    H --> I["🛡️ Quality & Regression Gate"]
+    I --> J["📦 Security Containment & Export Gate"]
+    J --> K["🚀 Standalone Web Chat ZIP & GGUF (Ollama)"]
+```
+
+### 1. 🎯 Goal-Aligned Planning
+
+Define your AI's task and domain (`chat`, `code`, `domain-qa`, `summarization`, `extraction`, `reasoning`). MYAI tailors evaluation criteria so performance is measured directly against your specific objective.
+
+### 2. 🧹 Dataset Intelligence & Strict Reference Mode
+
+* **Strict Reference Mode**: Original data files are **never modified in-place**.
+* **PII & Secret Scrubbing**: Automated detection and redaction of sensitive strings (emails, phone numbers, API keys).
+* **Deduplication**: Identifies and removes duplicate and near-duplicate samples.
+* **Leakage Detection**: Isolates train and validation sets before training begins.
+
+### 3. 🔬 Token & Context Analysis
+
+* Computes token counts and sequence distributions across supported model families (Llama, Qwen, SmolLM).
+* Flags context length mismatches and memory considerations prior to training.
+
+### 4. 🌊 Memory-Efficient Training
+
+Enables fine-tuning on resource-constrained GPUs via memory-optimized layer streaming.
+
+### 5. 🧬 Post-Training Preference Alignment
+
+Fine-tune beyond standard supervised training with modern preference alignment algorithms:
+
+* **DPO** (Direct Preference Optimization)
+* **ORPO** (Odds Ratio Preference Optimization — reference-model-free)
+* **SimPO** (Simple Preference Optimization — length-normalized margin)
+* **KTO** (Kahneman-Tversky Optimization — binary feedback)
+
+### 6. 🛡️ Quality Gate & Containment Verification
+
+* Validates fine-tuned checkpoints against regression suites before release (`myai ship`).
+* Enforces containment checks (excludes internal source files, `.git`, `.env`, raw datasets, and traversal paths) before packaging.
+
+---
+
+## 🖥️ Hardware Tiers & Supported Models
+
+### Hardware-Aware Feasibility
+
+MYAI evaluates your local hardware capacity (CPU, system RAM, and GPU VRAM) to recommend suitable models and training configurations:
+
+* **Capacity-Matched Recommendations**: Recommends models based on your hardware, data, task, and deployment requirements.
+* **Headroom Feasibility**: Checks available memory headroom across operating context lengths.
+* **Clear Readiness Verdicts**: Provides clear guidance on whether a model is recommended, compatible, or requires reduced context/quantization.
+
+### Compute Tiers Overview
+
+| Hardware Tier | Memory Profile | Example Hardware | Supported Models |
+| :--- | :--- | :--- | :--- |
+| **Tier T0 (CPU Only)** | 8GB–32GB Host RAM | Intel Core / AMD Ryzen / Apple Silicon | SmolLM2 (135M–1.7B), Qwen 2.5 (0.5B–1.5B) |
+| **Tier T1 (Low VRAM)** | **4GB–6GB VRAM** | GTX 1650, RTX 3050 Laptop | SmolLM2, Qwen 2.5 (1.5B/3B), Gemma 3 (1B/4B), 8B (Streaming) |
+| **Tier T2 (Mid VRAM)** | **8GB–16GB VRAM** | RTX 3060, RTX 4070, Apple M-Series | Llama 3.1 (8B), Qwen 2.5 (7B/14B), Phi-4 (14B), Ministral (8B) |
+| **Tier T3 (High VRAM)** | **24GB+ VRAM** | RTX 3090, RTX 4090, A100, H100 | Mistral Small (24B), Qwen 2.5 (32B), Llama 3.1 (70B) |
+
+### Supported Model Families
+
+MYAI supports leading open-weight model architectures spanning **0.1B to 70B+** parameters:
+
+| Model Family | Representative Sizes | Architecture | Quantization Formats | Primary Strengths |
+| :--- | :--- | :--- | :---: | :--- |
+| **SmolLM2** | 135M, 360M, 1.7B | Dense | FP16, INT8, Q4_K_M | Ultra-lightweight on-device assistants, edge devices |
+| **Qwen 2.5 / 3** | 0.5B, 1.5B, 3B, 7B, 14B, 32B | Dense & MoE | FP16, FP8, AWQ, GPTQ, Q4_K_M | Multilingual instruction following, coding, reasoning |
+| **Llama 3.1 / 3.2** | 1B, 3B, 8B, 70B | Dense | FP16, BF16, Q4_K_M, INT8 | General reasoning, tool calling, instruction tuning |
+| **Gemma 3** | 270M, 1B, 4B, 12B | Dense | FP16, BF16, Q4_K_M | Mathematical reasoning, high-efficiency generation |
+| **Mistral / Ministral** | 3B, 8B, 14B, 24B | Dense & MoE | FP16, FP8, AWQ, Q4_K_M | Code generation, reasoning, efficient MoE |
+| **Phi-4** | 3.8B (Mini), 14B | Dense | FP16, BF16, Q4_K_M | Advanced STEM reasoning, logic, and extraction |
+| **DeepSeek (R1 Distill)** | 7B, 32B | Dense | FP16, Q4_K_M, AWQ | Deep analytical reasoning, math, and code synthesis |
+
+---
+
+## 🧭 Step-by-Step Workflow Guide
+
+### 1. Initialize Project & Goal
+
+```bash
+myai init fitness-coach --task domain-qa --domain fitness --context balanced
+cd fitness-coach
+```
+
+### 2. Register & Clean Data
+
+```bash
+# Register dataset source in Reference Mode and inspect token distribution
+myai data add ./coaching_data.jsonl
+
+# Clean dataset, redact PII, deduplicate, and create holdout validation set
+myai data clean --fuzzy --val-split 0.1
+```
+
+### 3. Model Recommendation & System Check
+
+```bash
+# Verify local hardware availability
+myai system check
+
+# Get goal- and hardware-aware base model recommendation
+myai recommend
+```
+
+### 4. Fine-Tuning & Alignment
+
+```bash
+# Standard QLoRA fine-tuning
+myai train --epochs 3 --lr 2e-4
+
+# Fine-tune with memory streaming on low-VRAM hardware
+myai train --stream-layers
+
+# Direct preference alignment (SimPO, DPO, ORPO, or KTO)
+myai train --task simpo
+```
+
+### 5. Automated Verification & Quality Gate
+
+```bash
+# Generate task-specific verifiers from reference data
+myai reward synth ./data/references.jsonl -o reward.py
+
+# Execute regression gate before release
+myai ship
+```
+
+### 6. Export Standalone Package
+
+```bash
+# Export standalone Web Chat ZIP
+myai export
+
+# Export GGUF format for Ollama / llama.cpp
+myai export --format gguf --quant q4_k_m
+
+# Merge adapter weights into base model checkpoint
+myai merge
+```
+
+---
+
+## 🚀 Standalone Runtime Export
+
+The exported `.zip` contains a self-contained runtime that can run independently without requiring the full MYAI development CLI:
+
+```text
+fitness-coach.myai.zip
+├── model/                  # Model / adapter weights
+├── tokenizer/              # Tokenizer config and vocabulary
+├── metadata.json           # Model provenance, base repo, and goal profile
+├── evaluation.json         # Evaluation metrics and gate verification status
+├── loader.py               # Lightweight inference loader
+└── chat/
+    ├── app.py              # Lightweight web server (built-in http.server)
+    ├── ui.py               # Terminal fallback chat interface
+    ├── config.json         # UI configuration & styling
+    └── web/
+        └── index.html      # Responsive Web Chat interface
+```
+
+### Launch Web Chat Interface
+
+```bash
+python chat/app.py
+```
+
+* Runs on Python's standard library `http.server` with zero external web framework dependencies.
+* Responsive, dark-mode browser interface.
+* Includes real-time streaming, parameter controls, token counters, and message history.
+
+---
+
+## 🛡️ Security & Containment Policies
+
+Artifacts produced by `myai export` follow strict automated packaging and containment checks:
+
+| Policy Area | Verification Scope | Description |
+| :--- | :--- | :--- |
+| **Integrity** | Package Structure | Verifies archive integrity, weight files, and tokenizer configurations. |
+| **Provenance** | Manifest Audit | Records base model origin, training configurations, and holdout evaluation metrics. |
+| **Runtime Isolation** | Portable Delivery | Bundles standalone `loader.py` and lightweight Web Chat runtime. |
+| **Containment** | Source Isolation | Excludes development source files and `.git/` history from release packages. |
+| **Data Privacy** | Secret & Data Scrubbing | Checks for sensitive credentials (`sk-`, `ghp_`, `hf_`, `AKIA`) and excludes raw training files. |
+| **Path Security** | Traversal Protection | Prevents absolute host paths and relative directory traversals (`../`). |
+
+---
+
+## 🛠️ Complete CLI Reference
+
+| Command Group | Command | Description |
+| :--- | :--- | :--- |
+| **Project** | `myai init [name]` | Initialize project workspace with interactive Goal Profile interview |
+| | `myai status` | Inspect project lifecycle state and recommended next steps |
+| | `myai system check` | Probe local CPU, RAM, GPU, VRAM, and compute tier |
+| | `myai system benchmark` | Benchmark live hardware compute and memory throughput |
+| **Autopilot** | `myai auto [--export]` | **Autonomous Pipeline**: Goal → Hardware & Data → Train → Eval → Optimize → Export |
+| **Data** | `myai data add <path>` | Register local datasets in Reference Mode and analyze token distributions |
+| | `myai data tokenize` | Inspect token counts, sequence lengths, and context fit |
+| | `myai data clean` | Clean, deduplicate, scrub sensitive data, and split train/val sets |
+| | `myai data list` / `info` | View registered datasets, sample counts, and quality metrics |
+| **Models** | `myai model list` | Browse supported model catalog (Dense, MoE, size, VRAM requirements) |
+| | `myai model use <id>` | Select active base model for the project |
+| | `myai recommend` | Hardware- and goal-aware recommendation with multi-factor suitability |
+| **Training** | `myai train` | Train model with live loss curves and progress telemetry |
+| | `myai train --stream-layers` | Train on resource-constrained GPUs using memory streaming |
+| | `myai train --task <method>` | Train preference alignment (**DPO, ORPO, SimPO, KTO**) |
+| **Evaluation** | `myai evaluate` | Run offline task evaluation benchmarks and accuracy metrics |
+| | `myai leaderboard` | Show experiment leaderboard ranked by goal-weighted composite score |
+| **Alignment** | `myai reward synth` | Generate task-specific verifiers from reference datasets |
+| | `myai ship` | Run regression gate and test suites for release verdict |
+| | `myai merge` | Merge adapter weights into base model checkpoint |
+| **Tracking** | `myai runs list` / `info <id>` | List historical training runs and metric summaries |
+| | `myai runs best` | View experiment leaderboard and current release candidate |
+| | `myai optimize` | Automated retrain and compare hyperparameter optimization loop |
+| **Knowledge** | `myai index build <path>` | Index local documents into semantic embeddings for RAG retrieval |
+| | `myai index list` / `info` | Inspect active knowledge index chunks and embedding dimensions |
+| **Export** | `myai export [--format]` | Package as standalone Web App ZIP, **GGUF (Ollama)**, or Merged weights |
+| **Serving** | `myai serve` / `myai ask` | Serve local model with Knowledge Gate RAG protection |
+| **System** | `myai update [--check]` | Check for latest releases on PyPI and upgrade MYAI |
+| | `myai uninstall [--purge-data]` | Uninstall MYAI package and optionally purge `~/.myai` data |
+
+---
+
+## 📜 License
+
+Distributed under the **Apache 2.0** License. See [LICENSE](https://github.com/kiransai-62/myai/blob/main/LICENSE) for details.