eigencore 0.4.0__tar.gz

eigencore-0.4.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,27 @@
+---
+name: Bug report
+about: Something EigenCore does wrong
+labels: bug
+---
+
+**What happened**
+
+**What you expected**
+
+**How to reproduce**
+Command(s) and the smallest input that shows it:
+
+```
+eigencore ...
+```
+
+**Environment**
+- OS and version:
+- Python version:
+- EigenCore commit or version:
+- llama-cpp-python version and how it was installed:
+- CPU, RAM, GPU (if any):
+- Model file and quant (name, size):
+
+**Output**
+Paste the error or `eigencore profile` output (remove anything private).

eigencore-0.4.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Security vulnerability
+    url: https://github.com/Khubaib7-del/EigenCore/security/policy
+    about: Please report security issues privately, as described in SECURITY.md.

eigencore-0.4.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,16 @@
+---
+name: Feature or optimization proposal
+about: Suggest a feature, or an optimization with a plan to measure it
+labels: enhancement
+---
+
+**Problem it solves**
+
+**Proposal**
+
+**Status label it would change** (PLANNED → SHIPPED, REPORTED-ONLY → SHIPPED, ...)
+
+**How we'd know it works**
+For a performance change: the benchmark, hardware and quality check that would prove it. For a UI change: the terminals and modes it must work in (Rich, `--plain`, Windows Terminal, SSH).
+
+**Alternatives considered**

eigencore-0.4.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,14 @@
+## What this changes
+
+## Why
+
+## Status labels affected
+<!-- e.g. "Speculative decoding: REPORTED-ONLY → SHIPPED (prompt lookup only)" or "none" -->
+
+## Evidence
+- [ ] Tests added or updated
+- [ ] `pytest`, `ruff check` and `ruff format --check` pass locally
+- [ ] Performance claims, if any, include benchmark JSON and the named hardware
+- [ ] User-facing text uses SHIPPED / REPORTED-ONLY / PLANNED correctly
+- [ ] CHANGELOG.md updated under "Unreleased"
+- [ ] No binary files added

eigencore-0.4.0/.github/workflows/ci.yml

@@ -0,0 +1,52 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python-version: ["3.10", "3.12"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]" --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+
+      - name: Lint
+        run: ruff check src/ tests/
+
+      - name: Test
+        run: pytest tests/ -v --tb=short
+
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install ruff
+        run: pip install "ruff>=0.4.0,<0.15"
+
+      - name: Check formatting
+        run: ruff format --check src/ tests/
+
+      - name: Check imports
+        run: ruff check --select I src/ tests/

eigencore-0.4.0/.github/workflows/release.yml

@@ -0,0 +1,62 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: write
+  id-token: write  # PyPI trusted publishing (OIDC) — no API token stored in GitHub
+
+jobs:
+  pypi:
+    name: Publish to PyPI
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build sdist + wheel
+        run: |
+          python -m pip install --upgrade pip build
+          python -m build
+
+      - name: Publish to PyPI (trusted publishing)
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          # Requires the GitHub OIDC publisher configured once at
+          # https://pypi.org/manage/project/eigencore/publishing/ (owner action).
+          print-hash: true
+
+  release:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Generate changelog
+        id: changelog
+        run: |
+          PREV_TAG=$(git tag --sort=-v:refname | head -2 | tail -1)
+          if [ -z "$PREV_TAG" ]; then
+            CHANGES=$(git log --pretty=format:"- %s" HEAD)
+          else
+            CHANGES=$(git log --pretty=format:"- %s" "${PREV_TAG}..HEAD")
+          fi
+          echo "changes<<EOF" >> "$GITHUB_OUTPUT"
+          echo "$CHANGES" >> "$GITHUB_OUTPUT"
+          echo "EOF" >> "$GITHUB_OUTPUT"
+
+      - name: Create GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          generate_release_notes: false
+          body: |
+            ## What's Changed
+            ${{ steps.changelog.outputs.changes }}

eigencore-0.4.0/.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.eggs/
+*.gguf
+.venv/
+.env
+*.log
+.ruff_cache/
+.pytest_cache/

eigencore-0.4.0/CHANGELOG.md

@@ -0,0 +1,86 @@
+# Changelog
+
+All notable changes to EigenCore. Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow the git tags. Dates are commit dates.
+
+## [Unreleased]
+
+## [0.4.0] - 2026-11-11
+
+### Added
+- PyPI release pipeline: `release.yml` now publishes sdist+wheel to PyPI on `v*` tags via trusted publishing (OIDC, `pypa/gh-action-pypi-publish`, no stored tokens). After tagging `v0.4.0` and configuring the GitHub OIDC publisher at pypi.org, `pip install eigencore` works. README quick start updated accordingly (prebuilt llama.cpp wheel index documented for Windows/CPU).
+
+### Added
+- Stage E slice 3: activation-sparsity experiment scaffolding (REPORTED-ONLY, not wired into inference). `engine/sparsity_experiment.py`: per-layer activation sparsity measurement, sparse-mask plan that only applies above a minimum sparsity, and the kill criterion (measured quality delta > allowed → killed). Rationale recorded: on bandwidth-bound CPU decode the compute saving is small, so the kill criterion matters. No claim without owner runs.
+- Stage E slice 2: layer-pruning experiment scaffolding (REPORTED-ONLY, not wired into inference). `engine/pruning.py` computes ShortGPT Block Influence per layer from boundary hidden states, builds a prune plan that protects head/tail layers, and enforces the kill criterion (measured quality delta > allowed → killed, no pruning). No quality/performance claim without the owner's calibration + perplexity runs.
+- Stage E slice 1: Q4_K reference kernel (`kernels/q4k.py`) — full GGUF super-block dequant (fp16 d/dmin, packed 6-bit sc/min, 4-bit quants, `d*sc*q - dmin*m`) plus grouped quantized dot, on `BackendProtocol`/`ScalarBackend`. **Bit-exact vs `gguf.quants.dequantize` for Q4_K** on random blocks. Reference only; no performance claim, not wired into inference.
+- Stage D slice 3: adaptive terminal chrome + first-run onboarding. New `tui/theme.py` capability probe (`--plain`, non-TTY, `NO_COLOR`, `TERM=dumb` → plain; width tiers wide ≥100 / narrow; header shortens on narrow). New `tui/onboarding.py`: first-run model picker grouped by fit verdict (MoE shows resident vs read-per-token GiB), SSD-paging warning before load when the file exceeds free RAM, config persisted to `~/.eigencore/config.json` (`onboarded`) so it runs once; skipped entirely when non-interactive. Speed in the picker is labeled ESTIMATE.
+- Stage D slice 2: policy-backed approvals. New `agent.policy.ApprovalManager` (ASK / SESSION / DENY, session-scoped per tool name, audit-logged). `ToolExecutor(approver=..., ask_fn=...)` consults it after policy passes but before running; no callback attached means deny. Chat UI renders an approval card (exact redacted args, y/s/n) via `ChatUI.render_approval_card`, plus `/approvals`. Enforcement stays in the executor, never in prompt text or rendering.
+- Stage D slice 1: tool-handler process isolation hardened with POSIX rlimits on the child worker (`RLIMIT_CPU`, `RLIMIT_AS`, `RLIMIT_FSIZE`; `RLIMIT_NPROC` opt-in only, default off because it breaks library imports in the child). Configurable via `ToolExecutor(max_cpu_seconds=, max_memory_mb=, max_file_size_mb=, max_processes=)`; skipped safely on non-POSIX. Kill-on-timeout unchanged.
+- Stage C slice 4: prompt-lookup decoding behind `--pld` on `run`/`chat` (default OFF until the owner's §9 item-9 measurements decide). `engine.prompt_lookup.longest_prompt_continuation` finds the longest prompt-verbatim continuation; the engine emits it directly and swallows the covered backend tokens to keep the stream aligned; copied chars reported via `engine.last_pld_copied_chars`. No speedup claimed.
+- Stage C slice 3: GPU/VRAM fit and layer offload. `profile_hardware()` detects NVIDIA VRAM via `nvidia-smi` (never raises; zeros when absent) and `HardwareProfile` gains `gpu_name`/`total_vram_gb`/`available_vram_gb`, shown in `profile`. `fit.gpu_layers_for_vram()` computes how many layers fit in VRAM after a reserve and the KV cache; `estimate_fit(..., available_vram_gb=...)` reports `gpu_layers`/`gpu_weights_gb` with a note. `InferenceEngine(..., n_gpu_layers=...)` replaces the hardcoded 0; `eigencore run`/`chat` gained `--gpu-layers` and auto-recommend offload when a GPU is detected and the flag is omitted (0 still means CPU-only).
+- Stage C slice 2: KV-cache type and flash-attention options. `InferenceEngine(..., kv_type="fp16"|"q8_0", flash_attn=...)` maps to llama.cpp `type_kv` (0/1); `eigencore run` and `eigencore chat` gained `--kv [fp16|q8_0]` and `--no-flash`. `q8_0` halves KV-cache memory; the fit estimator already accounts for it via `kv_quant`.
+- MoE-aware fit (Stage C slice 1): `ModelGeometry.expert_count`/`expert_used_count` parsed from GGUF (`{arch}.expert_count*`); `ModelSpec.experts_total`/`experts_active` with `is_moe`; registry gains `qwen3-30b-a3b` (Qwen3-30B-A3B, 128 experts / 8 active, Q4_K_M ≈ 18.6 GiB). `estimate_fit` now reports `active_weights_gb`, `resident`, and a `decode_ceiling_tok_s` = bandwidth ÷ active bytes-per-token (ceiling, not a measurement); non-resident models get an explicit mmap/SSD-paging warning. CLI shows MoE active params, resident/paging status, and the ceiling. No dense-model SSD streaming built, per the review.
+
+### Fixed
+- Chat TUI: model output is sanitized (ANSI/OSC/CSI stripped, newline/tab kept) and Rich-escaped; Ctrl-C mid-stream keeps partial history without regenerating; stream errors keep the partial response and never double-print.
+- Benchmark harness: completion tokens counted via the backend tokenizer (chars/4 fallback only when no tokenizer), and the summary reports cold (first run per case) vs warm runs separately instead of one mixed median.
+- Agent safety: `ToolExecutor` is now deny-by-default (defaults to `default_policy()` — READ-only, workspace-confined — when no policy is passed; `Forge.agent()`/`ReActAgent` inherit this). Path confinement uses explicitly declared `ToolParam(is_path=True)` parameters (no more name heuristics) and handlers receive resolved workspace-confined absolute paths. Timeouts now run handlers in a spawned process that is terminated/killed on timeout. Successful string output is credential-redacted. Tests cover `/etc/passwd` denial and child-process kill.
+- Q4_0 reference dequantizer: corrected to the ggml strided nibble layout (byte j low nibble → element j, high nibble → element j+16) and now also accepts raw 18-byte blocks; Q8_0 accepts 34-byte blocks. Verified bit-exact against `gguf.quants.dequantize` for Q4_0 and Q8_0.
+- Fit estimator: all sizes in GiB end to end; weights use actual GGUF file size when available; KV cache computed only from architecture-aware GGUF geometry (`{arch}.block_count` / `embedding_length` / `attention.head_count*`), otherwise verdict is `UNKNOWN` instead of a 15–30×-low heuristic; `max_context_for_budget` returns 0 without geometry. CLI `profile`/`models`/`chat` pass real GGUF metadata when the model is downloaded. Profiler GiB/GB mix fixed and 25% of usable RAM reserved for KV, so recommended context no longer pins at 2048 below ~38 GiB. Added `gguf` dependency and `eigencore.fit.read_gguf_model_info`.
+
+### Documentation
+- Added a full review: truth audit, 30–40B memory and bandwidth arithmetic, ranked issues, prioritized plan and owner benchmark list (`docs/review/`).
+- Brand v2: flat logo, small mark, favicon, lockups, wordmarks and banners as SVG in `assets/brand/` (PNG/ICO exports to follow); v2 design handbook.
+- Brand v2 fix: eigen-axis realigned to a clean 45° dotted line inside the die, evenly spaced dots, no trace crossings; middle trace now ends on the axis.
+- Added MIT `LICENSE` (© 2026 Khubaib Nazeer).
+- Added terminal UI status, technology decision and v2 screen mock-ups (`docs/design/TUI.md`, `docs/design/tui/`).
+- Added presentation materials: slides v2 text with change list, and a study guide (`docs/presentation/`).
+- README rewritten to install from source (not on PyPI), with a SHIPPED / REPORTED-ONLY / PLANNED status table and known bugs.
+- Review notes added to research and positioning docs whose claims overstate the code.
+- Added CONTRIBUTING, CODE_OF_CONDUCT, SECURITY, issue and PR templates, and this changelog.
+
+### Known issues (found by the review, not yet fixed)
+- Fit estimator KV fallback underestimates by ~15–30×, and the CLI always uses it.
+- Profiler GiB/GB mix pins recommended context at 2048 below ~38 GB RAM.
+- Q4_0 reference dequantizer and its test use the wrong nibble order.
+- Tool policy is opt-in in the agent API; executor timeout does not stop handlers; path checks depend on parameter names.
+- Chat crashes on Rich markup in model output and passes terminal escape sequences.
+- Speculative verifier is not a correct rejection sampler.
+
+## [Post-0.3.0 on main] - 2026-07-18 to 2026-11-08
+Not tagged; package version is still 0.3.0.
+
+### Added
+- Phase 1 benchmark harness with quality cases and JSON reports.
+- Conservative model-fit estimator with KV-cache and context budgeting.
+- Deny-unless-allow-listed tool policy engine with path confinement and audit log.
+- Streaming chat UI with plain mode, slash commands and fit-aware header.
+- Backend protocol and scalar reference kernels with tests.
+- Brand kit and animated banner.
+
+### Changed
+- Optimization statistics labelled as reported-only; none is wired into the backend.
+- CI pins ruff below 0.15.
+
+### Fixed
+- `generate`/`stream` routed through chat completion.
+
+## [0.3.0] - 2026-07-18
+### Added
+- Agent framework: tool system, GBNF grammar builder, ReAct loop, wired into the Forge API, with tests.
+- `benchmark` command; Phase 2 optimizers attached to the inference engine (reporting only).
+### Changed
+- README rewritten.
+
+## [0.2.0] - 2026-07-17
+### Added
+- Hardware profiler with CPUID detection and model registry.
+- Inference engine over llama.cpp with hardware-aware configuration.
+- Task router, attention-aware context manager, QLoRA training components, sparsity analysis tools.
+- CLI and Forge API; architecture, research and roadmap docs.
+- Layer-skipping scheduler, speculative decoding module and sparsity predictor (not wired into inference).
+- GitHub Actions CI and release workflow.
+
+## [0.1.0] - 2026-07-17
+### Added
+- Project skeleton with `pyproject.toml` and README.

eigencore-0.4.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,32 @@
+# Code of conduct
+
+EigenCore follows the spirit of the [Contributor Covenant, version 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/). This is a short, project-sized version of it.
+
+## Our pledge
+
+We want participation in EigenCore to be harassment-free for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, colour, religion, or sexual identity and orientation.
+
+## Expected behaviour
+
+- Be kind and assume good intent.
+- Critique code and claims, not people. Disagreement backed by evidence is welcome.
+- Accept correction gracefully, including about numbers you published.
+- Help newcomers; this is a learning-friendly project.
+
+## Unacceptable behaviour
+
+- Harassment, insults, slurs, sexualised language or imagery, and personal attacks.
+- Publishing someone's private information without permission.
+- Sustained disruption of issues, discussions or reviews.
+
+## Scope
+
+This applies in all project spaces (issues, pull requests, discussions) and when representing the project elsewhere.
+
+## Reporting and enforcement
+
+Report incidents privately to the maintainer (contact listed in `pyproject.toml`). Reports are handled confidentially. The maintainer may warn, temporarily restrict, or ban participants, depending on severity, following the Contributor Covenant's enforcement guidelines.
+
+## Attribution
+
+Adapted from the Contributor Covenant, version 2.1, https://www.contributor-covenant.org/version/2/1/code_of_conduct/.

eigencore-0.4.0/CONTRIBUTING.md

@@ -0,0 +1,44 @@
+# Contributing to EigenCore
+
+Thanks for helping. EigenCore is a small, pre-alpha project, so this guide is short.
+
+## Before you start
+
+- Read the [review and plan](docs/review/FINAL_REPORT.md); section 8 lists the work in priority order.
+- For anything larger than a small fix, open an issue first so we can agree on the approach.
+
+## Setup
+
+```bash
+git clone https://github.com/Khubaib7-del/EigenCore.git
+cd EigenCore
+python -m pip install -e ".[dev]"
+pytest tests/ -v
+ruff check src/ tests/
+ruff format --check src/ tests/
+```
+
+CI runs on Ubuntu and Windows with Python 3.10 and 3.12. Ruff is pinned below 0.15 on purpose.
+
+## Rules that keep the project honest
+
+1. **Label claims.** Anything user-facing (README, docs, CLI or TUI output) is SHIPPED, REPORTED-ONLY or PLANNED. Don't describe a plan as a feature.
+2. **No unmeasured numbers.** No "x faster", "speedup" or tok/s unless it was measured, with the machine, model, quant, context, threads and backend version recorded. Arithmetic ceilings are labelled "estimate".
+3. **Correctness before speed.** New kernels or optimizations need a reference comparison (for kernels: gguf-py / ggml), a fallback, and a benchmark through `eigencore benchmark` before they can be enabled by default.
+4. **Safety stays below the UI.** Tool permissions are enforced in the policy layer, never only in prompts or rendering.
+5. **No binary files** (images, decks, models, archives) in pull requests unless a maintainer asks.
+
+## Pull requests
+
+- One concern per PR, with tests.
+- Add a line under "Unreleased" in [CHANGELOG.md](CHANGELOG.md).
+- Use [Conventional Commit](https://www.conventionalcommits.org/) prefixes (`feat:`, `fix:`, `docs:`, `test:`, `ci:`), as the history already does.
+- Fill in the PR template, including which status label the change affects.
+
+## Benchmarks
+
+If your change claims a performance effect, attach the JSON from `eigencore benchmark --report run.json` (or `llama-bench -o json`) and name the hardware. Maintainers may ask for a run on the reference laptop before merging.
+
+## Conduct
+
+By participating you agree to the [code of conduct](CODE_OF_CONDUCT.md).

eigencore-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Khubaib Nazeer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

eigencore-0.4.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.5
+Name: eigencore
+Version: 0.4.0
+Summary: CPU-first LLM intelligence runtime — hardware-aware inference, training, and agent orchestration
+Author-email: Khubaib Nazeer <khubaibnazeer8@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: ai,cpu,framework,inference,llm
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: click>=8.0.0
+Requires-Dist: gguf>=0.10.0
+Requires-Dist: huggingface-hub>=0.20.0
+Requires-Dist: llama-cpp-python>=0.3.0
+Requires-Dist: psutil>=5.9.0
+Requires-Dist: rich>=13.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff<0.15,>=0.4.0; extra == 'dev'
+Provides-Extra: train
+Requires-Dist: bitsandbytes>=0.43.0; extra == 'train'
+Requires-Dist: peft>=0.10.0; extra == 'train'
+Requires-Dist: torch>=2.0.0; extra == 'train'
+Requires-Dist: transformers>=4.40.0; extra == 'train'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="assets/brand/eigencore-banner-animated.svg" alt="EigenCore — local models, on your own silicon" width="100%">
+</p>
+
+<h1 align="center">EigenCore</h1>
+<p align="center">A CPU-first Python runtime for local language models, hardware-aware inference, and agent building blocks.</p>
+
+<p align="center">
+  <img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white">
+  <img alt="Version 0.3.0" src="https://img.shields.io/badge/version-0.3.0-56A9F5">
+  <img alt="License MIT" src="https://img.shields.io/badge/license-MIT-72B7A8">
+</p>
+
+EigenCore profiles a machine, estimates whether a local model fits, and runs inference through `llama-cpp-python` (llama.cpp). It also ships a benchmark harness, a terminal chat, and agent building blocks with a tool policy layer. The transformer math is llama.cpp's; EigenCore owns the judgement around it.
+
+> **Reality check:** a dense 30–40B model at 4-bit is roughly 18–24 GB of weights, more than 8–16 GB of RAM before any KV cache. It can only run by paging from SSD, and decode is then expected to be well under 1 token/s (estimate). Even when a model fits, CPU decode is memory-bandwidth-bound: tok/s ≤ bandwidth ÷ weight bytes. The realistic route to 30B-class models on a 16 GB laptop is a Mixture-of-Experts model, which is planned and not yet measured. See the [review](docs/review/FINAL_REPORT.md) for the arithmetic. No speed figure in this repository is a measurement until a named-hardware run is published.
+
+## Quick start
+
+Install from PyPI:
+
+```bash
+python -m pip install eigencore
+# on Windows/CPU without prebuilt llama.cpp wheels, add the prebuilt index for a fast install:
+# python -m pip install eigencore --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+# pipx alternative: pipx install eigencore
+# from source (development): git clone ... && python -m pip install -e ".[dev]"
+
+eigencore profile
+eigencore models
+eigencore download tinyllama-1.1b
+eigencore run --prompt "Explain binary search in one paragraph"
+eigencore chat            # add --plain for scripts or minimal terminals
+```
+
+Use `eigencore --help` for every command. `llama-cpp-python` may need a backend-specific install; see its documentation before assuming GPU acceleration (EigenCore currently loads models CPU-only).
+
+## Status
+
+Labels: **SHIPPED** runs on `main` and is tested · **REPORTED-ONLY** computes plans or numbers that never reach llama.cpp · **PLANNED** design docs only.
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Inference through llama-cpp-python | SHIPPED | `engine/inference.py`; CPU-only (`n_gpu_layers=0`) |
+| Hardware profile | SHIPPED | ISA, RAM, bandwidth probe (reads low). GiB/GB mix fixed; 25% of usable RAM reserved for KV so recommended context no longer pins at 2048 under ~38 GB |
+| Model-fit estimator | SHIPPED | Weights (real GGUF file size when downloaded) + reserve + KV from architecture-aware GGUF geometry + margin; GiB end to end. Without geometry the verdict is `UNKNOWN` — no heuristic. CLI passes real GGUF metadata when the model file exists |
+| Benchmark harness | SHIPPED | `eigencore benchmark --report run.json`; tokens counted via backend tokenizer, cold/warm reported separately; no published runs yet |
+| Terminal chat | SHIPPED | Streaming, `--plain`/NO_COLOR/TERM=dumb auto-detection, width-tiered header, slash commands incl. `/approvals`, sanitized output, safe Ctrl-C. First-run onboarding (fit-grouped model picker + SSD-paging warning, runs once via config) SHIPPED. Full-screen takeover deliberately not done (line mode per TUI.md) |
+| Tool policy | SHIPPED, deny-by-default | `ToolExecutor` defaults to `default_policy()` (READ-only, workspace-confined) even when no policy is passed. Path params are declared (`is_path=True`) and resolved to confined absolute paths before handlers. Timeouts run handlers in a spawned process that is killed on timeout. Handlers still run in-process otherwise — not an OS sandbox |
+| Tool policy | SHIPPED, deny-by-default | Same as above, plus POSIX rlimits on the child worker and approval cards (`ApprovalManager` + `/approvals` in chat; y/s/n with redacted args, enforced in the executor). Full OS sandbox (cgroups/namespaces) PLANNED |
+| Reference kernels | SHIPPED (scalar, NumPy) | RMSNorm, Q8_0/Q4_0 (ggml layout, bit-exact vs gguf-py) and Q4_K super-blocks (bit-exact), Q8_0/Q4_K dots. Not used by inference; SIMD/C-intrinsics lib PLANNED (Stage E) |
+| Layer skipping | REPORTED-ONLY | Planner and estimates only |
+| Layer pruning (ShortGPT BI) | REPORTED-ONLY (experiment scaffolding) | `engine/pruning.py`: BI scores, head/tail-protected prune plan, kill criterion on quality delta. Not wired into inference; needs owner calibration + perplexity runs |
+| Activation sparsity | REPORTED-ONLY | Predictor never receives activations; `eigencore analyze` measures output-logit sparsity, not neurons |
+| Activation-sparsity experiment | REPORTED-ONLY (scaffolding) | `engine/sparsity_experiment.py`: mask plan + kill criterion. Dense-model sparsity is a poor near-term bet on bandwidth-bound CPUs (per review); MoE-native routing is the real path |
+| Speculative decoding | REPORTED-ONLY | Module not wired into generation; verifier now implements correct rejection sampling (distribution-preserving, tested) but no speedup is measured or claimed |
+| MoE-aware fit + SSD-paging warning + decode ceiling on active weights | SHIPPED | `qwen3-30b-a3b` registered; `estimate_fit` reports active/resident/ceiling |
+| KV cache type (`--kv fp16|q8_0`) + flash attention (`--no-flash`) on `run`/`chat` | SHIPPED | Engine passes `type_kv`/`flash_attn` to llama.cpp; q8_0 halves KV memory. GPU offload, prompt-lookup decoding PLANNED |
+| GPU/VRAM detection + layer offload (`--gpu-layers`, auto-recommend on GPU machines) | SHIPPED | `nvidia-smi` VRAM probe (safe when absent); `gpu_layers_for_vram` fits layers after reserve+KV; `n_gpu_layers` wired into llama.cpp. Prompt-lookup decoding PLANNED |
+| Prompt-lookup decoding (`--pld`, default OFF) | SHIPPED (flag) | Copies prompt-verbatim spans and swallows covered backend tokens; copied chars counted. Default stays OFF until owner benchmark §9 item 9 |
+| SIMD / native kernels | PLANNED | Research track; decode is bandwidth-bound |
+| Training (LoRA/QLoRA) | Code present, not evaluated | Optional `[train]` extra |
+
+## Architecture
+
+```mermaid
+flowchart LR
+  U[User] --> CLI[Click CLI]
+  U --> API[Forge Python API]
+  CLI --> HAL[Hardware profile and model registry]
+  API --> HAL
+  HAL --> ROUTE[Task and model routing]
+  ROUTE --> CTX[Context manager]
+  CTX --> ENGINE[InferenceEngine]
+  ENGINE --> BACKEND[llama-cpp-python backend]
+  API --> AGENT[ReAct agent]
+  AGENT --> REG[Tool registry]
+  REG --> EXEC[ToolExecutor]
+  EXEC -. handlers run in process today .-> HOST[Python process]
+  TRAIN[Optional training components] --> ADAPTER[LoRA adapter output]
+  HAL -. proposed: verified fit estimates .-> UX[Future adaptive TUI]
+```
+
+This is a module map, not a process-isolation diagram. In particular, the current executor’s validation and timeout are not an OS sandbox or an approval workflow.
+
+## Development
+
+```bash
+python -m pip install -e ".[dev]"
+pytest tests/ -v
+ruff check src/ tests/
+ruff format --check src/ tests/
+```
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Report security issues privately as described in [SECURITY.md](SECURITY.md). This project follows a [code of conduct](CODE_OF_CONDUCT.md).
+
+## Documentation
+
+- [Review and plan (2026-11)](docs/review/FINAL_REPORT.md): truth audit, 30–40B arithmetic, prioritized plan, benchmarks to run.
+- [Design handbook](docs/design/DESIGN_HANDBOOK.md): v2 identity, palette, terminal UI, approval flow.
+- [Terminal UI status and v2 screens](docs/design/TUI.md).
+- [Brand assets](assets/brand/README.md).
+- [System architecture](docs/architecture/system-overview.md) and [local inference roadmap](docs/architecture/local-inference-roadmap.md).
+- [Terminal chat design](docs/design/terminal-chat.md).
+- [Documentation index](docs/INDEX.md).
+- [Changelog](CHANGELOG.md).
+
+## Project status
+
+Pre-alpha. APIs and behaviour may change. Feature presence is not a performance claim.
+
+## License
+
+[MIT](LICENSE) © 2026 Khubaib Nazeer.

eigencore-0.4.0/README.md

@@ -0,0 +1,114 @@
+<p align="center">
+  <img src="assets/brand/eigencore-banner-animated.svg" alt="EigenCore — local models, on your own silicon" width="100%">
+</p>
+
+<h1 align="center">EigenCore</h1>
+<p align="center">A CPU-first Python runtime for local language models, hardware-aware inference, and agent building blocks.</p>
+
+<p align="center">
+  <img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white">
+  <img alt="Version 0.3.0" src="https://img.shields.io/badge/version-0.3.0-56A9F5">
+  <img alt="License MIT" src="https://img.shields.io/badge/license-MIT-72B7A8">
+</p>
+
+EigenCore profiles a machine, estimates whether a local model fits, and runs inference through `llama-cpp-python` (llama.cpp). It also ships a benchmark harness, a terminal chat, and agent building blocks with a tool policy layer. The transformer math is llama.cpp's; EigenCore owns the judgement around it.
+
+> **Reality check:** a dense 30–40B model at 4-bit is roughly 18–24 GB of weights, more than 8–16 GB of RAM before any KV cache. It can only run by paging from SSD, and decode is then expected to be well under 1 token/s (estimate). Even when a model fits, CPU decode is memory-bandwidth-bound: tok/s ≤ bandwidth ÷ weight bytes. The realistic route to 30B-class models on a 16 GB laptop is a Mixture-of-Experts model, which is planned and not yet measured. See the [review](docs/review/FINAL_REPORT.md) for the arithmetic. No speed figure in this repository is a measurement until a named-hardware run is published.
+
+## Quick start
+
+Install from PyPI:
+
+```bash
+python -m pip install eigencore
+# on Windows/CPU without prebuilt llama.cpp wheels, add the prebuilt index for a fast install:
+# python -m pip install eigencore --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+# pipx alternative: pipx install eigencore
+# from source (development): git clone ... && python -m pip install -e ".[dev]"
+
+eigencore profile
+eigencore models
+eigencore download tinyllama-1.1b
+eigencore run --prompt "Explain binary search in one paragraph"
+eigencore chat            # add --plain for scripts or minimal terminals
+```
+
+Use `eigencore --help` for every command. `llama-cpp-python` may need a backend-specific install; see its documentation before assuming GPU acceleration (EigenCore currently loads models CPU-only).
+
+## Status
+
+Labels: **SHIPPED** runs on `main` and is tested · **REPORTED-ONLY** computes plans or numbers that never reach llama.cpp · **PLANNED** design docs only.
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Inference through llama-cpp-python | SHIPPED | `engine/inference.py`; CPU-only (`n_gpu_layers=0`) |
+| Hardware profile | SHIPPED | ISA, RAM, bandwidth probe (reads low). GiB/GB mix fixed; 25% of usable RAM reserved for KV so recommended context no longer pins at 2048 under ~38 GB |
+| Model-fit estimator | SHIPPED | Weights (real GGUF file size when downloaded) + reserve + KV from architecture-aware GGUF geometry + margin; GiB end to end. Without geometry the verdict is `UNKNOWN` — no heuristic. CLI passes real GGUF metadata when the model file exists |
+| Benchmark harness | SHIPPED | `eigencore benchmark --report run.json`; tokens counted via backend tokenizer, cold/warm reported separately; no published runs yet |
+| Terminal chat | SHIPPED | Streaming, `--plain`/NO_COLOR/TERM=dumb auto-detection, width-tiered header, slash commands incl. `/approvals`, sanitized output, safe Ctrl-C. First-run onboarding (fit-grouped model picker + SSD-paging warning, runs once via config) SHIPPED. Full-screen takeover deliberately not done (line mode per TUI.md) |
+| Tool policy | SHIPPED, deny-by-default | `ToolExecutor` defaults to `default_policy()` (READ-only, workspace-confined) even when no policy is passed. Path params are declared (`is_path=True`) and resolved to confined absolute paths before handlers. Timeouts run handlers in a spawned process that is killed on timeout. Handlers still run in-process otherwise — not an OS sandbox |
+| Tool policy | SHIPPED, deny-by-default | Same as above, plus POSIX rlimits on the child worker and approval cards (`ApprovalManager` + `/approvals` in chat; y/s/n with redacted args, enforced in the executor). Full OS sandbox (cgroups/namespaces) PLANNED |
+| Reference kernels | SHIPPED (scalar, NumPy) | RMSNorm, Q8_0/Q4_0 (ggml layout, bit-exact vs gguf-py) and Q4_K super-blocks (bit-exact), Q8_0/Q4_K dots. Not used by inference; SIMD/C-intrinsics lib PLANNED (Stage E) |
+| Layer skipping | REPORTED-ONLY | Planner and estimates only |
+| Layer pruning (ShortGPT BI) | REPORTED-ONLY (experiment scaffolding) | `engine/pruning.py`: BI scores, head/tail-protected prune plan, kill criterion on quality delta. Not wired into inference; needs owner calibration + perplexity runs |
+| Activation sparsity | REPORTED-ONLY | Predictor never receives activations; `eigencore analyze` measures output-logit sparsity, not neurons |
+| Activation-sparsity experiment | REPORTED-ONLY (scaffolding) | `engine/sparsity_experiment.py`: mask plan + kill criterion. Dense-model sparsity is a poor near-term bet on bandwidth-bound CPUs (per review); MoE-native routing is the real path |
+| Speculative decoding | REPORTED-ONLY | Module not wired into generation; verifier now implements correct rejection sampling (distribution-preserving, tested) but no speedup is measured or claimed |
+| MoE-aware fit + SSD-paging warning + decode ceiling on active weights | SHIPPED | `qwen3-30b-a3b` registered; `estimate_fit` reports active/resident/ceiling |
+| KV cache type (`--kv fp16|q8_0`) + flash attention (`--no-flash`) on `run`/`chat` | SHIPPED | Engine passes `type_kv`/`flash_attn` to llama.cpp; q8_0 halves KV memory. GPU offload, prompt-lookup decoding PLANNED |
+| GPU/VRAM detection + layer offload (`--gpu-layers`, auto-recommend on GPU machines) | SHIPPED | `nvidia-smi` VRAM probe (safe when absent); `gpu_layers_for_vram` fits layers after reserve+KV; `n_gpu_layers` wired into llama.cpp. Prompt-lookup decoding PLANNED |
+| Prompt-lookup decoding (`--pld`, default OFF) | SHIPPED (flag) | Copies prompt-verbatim spans and swallows covered backend tokens; copied chars counted. Default stays OFF until owner benchmark §9 item 9 |
+| SIMD / native kernels | PLANNED | Research track; decode is bandwidth-bound |
+| Training (LoRA/QLoRA) | Code present, not evaluated | Optional `[train]` extra |
+
+## Architecture
+
+```mermaid
+flowchart LR
+  U[User] --> CLI[Click CLI]
+  U --> API[Forge Python API]
+  CLI --> HAL[Hardware profile and model registry]
+  API --> HAL
+  HAL --> ROUTE[Task and model routing]
+  ROUTE --> CTX[Context manager]
+  CTX --> ENGINE[InferenceEngine]
+  ENGINE --> BACKEND[llama-cpp-python backend]
+  API --> AGENT[ReAct agent]
+  AGENT --> REG[Tool registry]
+  REG --> EXEC[ToolExecutor]
+  EXEC -. handlers run in process today .-> HOST[Python process]
+  TRAIN[Optional training components] --> ADAPTER[LoRA adapter output]
+  HAL -. proposed: verified fit estimates .-> UX[Future adaptive TUI]
+```
+
+This is a module map, not a process-isolation diagram. In particular, the current executor’s validation and timeout are not an OS sandbox or an approval workflow.
+
+## Development
+
+```bash
+python -m pip install -e ".[dev]"
+pytest tests/ -v
+ruff check src/ tests/
+ruff format --check src/ tests/
+```
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Report security issues privately as described in [SECURITY.md](SECURITY.md). This project follows a [code of conduct](CODE_OF_CONDUCT.md).
+
+## Documentation
+
+- [Review and plan (2026-11)](docs/review/FINAL_REPORT.md): truth audit, 30–40B arithmetic, prioritized plan, benchmarks to run.
+- [Design handbook](docs/design/DESIGN_HANDBOOK.md): v2 identity, palette, terminal UI, approval flow.
+- [Terminal UI status and v2 screens](docs/design/TUI.md).
+- [Brand assets](assets/brand/README.md).
+- [System architecture](docs/architecture/system-overview.md) and [local inference roadmap](docs/architecture/local-inference-roadmap.md).
+- [Terminal chat design](docs/design/terminal-chat.md).
+- [Documentation index](docs/INDEX.md).
+- [Changelog](CHANGELOG.md).
+
+## Project status
+
+Pre-alpha. APIs and behaviour may change. Feature presence is not a performance claim.
+
+## License
+
+[MIT](LICENSE) © 2026 Khubaib Nazeer.