nested-learning 0.2.0__tar.gz

nested_learning-0.2.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,2 @@
+blank_issues_enabled: false
+contact_links: []

nested_learning-0.2.0/.github/ISSUE_TEMPLATE/eval_request.md

@@ -0,0 +1,27 @@
+---
+name: Evaluation request
+about: Propose a new benchmark or diagnostic to add
+title: "[Eval] "
+labels: ["evaluation", "needs-triage"]
+assignees: []
+---
+
+## Motivation
+Why is this evaluation important for HOPE/TITAN reproduction?
+
+## Task details
+- Dataset / benchmark:
+- Metric(s):
+- Expected runtime / hardware:
+
+## Environment target
+- OS:
+- Python:
+- Torch:
+- Preferred backend (`cpu` / `cuda` / `mps` / `rocm`):
+
+## Implementation sketch
+Outline scripts/flags needed (e.g., extend `scripts/eval/zeroshot.py`).
+
+## Acceptance criteria
+Describe what needs to be captured (JSON fields, plots, etc.).

nested_learning-0.2.0/.github/ISSUE_TEMPLATE/faithfulness_gap.md

@@ -0,0 +1,30 @@
+---
+name: Faithfulness gap
+about: Report deviations vs. the Nested Learning / HOPE specs
+title: "[Faithfulness] "
+labels: ["faithfulness", "needs-triage"]
+assignees: []
+---
+
+## Summary
+Describe the suspected deviation (cite paper section/equation).
+
+## Evidence
+- Config(s) / checkpoints affected
+- Logs / screenshots / metrics
+- Steps to reproduce
+
+## Environment
+- OS:
+- Python:
+- Torch:
+- Backend (`cpu` / `cuda` / `mps` / `rocm`):
+- GPU/accelerator model (if any):
+
+If using ROCm: this project currently treats ROCm support as best-effort. Include HIP/ROCm version and exact torch build.
+
+## Expected behavior
+What should happen according to the paper?
+
+## Additional context
+Add any extra notes, e.g., suggested fix or related PRs.

nested_learning-0.2.0/.github/ISSUE_TEMPLATE/perf_regression.md

@@ -0,0 +1,32 @@
+---
+name: Performance regression
+about: Report a training / eval performance drop vs. baseline
+title: "[Perf] "
+labels: ["performance", "needs-triage"]
+assignees: []
+---
+
+## Summary
+Describe the regression and the baseline you’re comparing against.
+
+## Baseline
+- Config / checkpoint:
+- Metrics (loss / ppl / eval scores):
+
+## Repro steps
+Exact commands with overrides, plus hardware details.
+
+## Environment
+- OS:
+- Python:
+- Torch:
+- Backend (`cpu` / `cuda` / `mps` / `rocm`):
+- GPU/accelerator model (if any):
+
+If using ROCm: this project currently treats ROCm support as best-effort. Include HIP/ROCm version and exact torch build.
+
+## Logs / artifacts
+Attach relevant logs, W&B links, or JSON eval files.
+
+## Suspected cause
+Optional theory or related commits/PRs.

nested_learning-0.2.0/.github/workflows/ci.yml

@@ -0,0 +1,197 @@
+name: CI
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+    branches: ["main"]
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Sync dependencies
+        run: uv sync --all-extras --dev
+
+      - name: Ruff
+        run: uv run ruff check .
+
+      - name: Mypy
+        run: uv run mypy src
+
+      - name: Verify docs path references
+        run: uv run python scripts/checks/verify_docs_refs.py
+
+      - name: Verify README critical commands
+        run: bash scripts/checks/check_readme_commands.sh
+
+      - name: Guard tracked file sizes / artifact extensions
+        run: bash scripts/checks/check_git_tracked_sizes.sh
+
+      - name: Verify scripts/data help exits cleanly
+        run: bash scripts/checks/check_data_script_help.sh
+
+      - name: Pytest
+        run: uv run pytest
+
+  cross-platform-smoke:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python-version: ["3.10", "3.12"]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Sync dependencies
+        run: uv sync --dev
+
+      - name: CLI help + doctor + smoke
+        run: |
+          uv run nl --help
+          uv run nl doctor --json
+          uv run nl smoke --config-name pilot_smoke --device cpu --batch-size 1 --seq-len 8
+          uv run python -m nested_learning --help
+
+  wheel-install-smoke:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Build wheel
+        run: uv build
+
+      - name: Install wheel in isolated venv
+        run: |
+          python -m venv /tmp/wheel-smoke
+          /tmp/wheel-smoke/bin/python -m pip install --upgrade pip
+          /tmp/wheel-smoke/bin/python -m pip install dist/*.whl
+
+      - name: Verify wheel entrypoints outside repo configs
+        run: |
+          /tmp/wheel-smoke/bin/python -m nested_learning --help
+          /tmp/wheel-smoke/bin/python -m nested_learning doctor --json
+          /tmp/wheel-smoke/bin/python - <<'PY'
+          import subprocess
+          import sys
+          import tempfile
+
+          tmp = tempfile.mkdtemp(prefix="nl-wheel-smoke-")
+          cmd = [
+              sys.executable,
+              "-m",
+              "nested_learning",
+              "smoke",
+              "--config-name",
+              "pilot_smoke",
+              "--device",
+              "cpu",
+              "--batch-size",
+              "1",
+              "--seq-len",
+              "8",
+          ]
+          subprocess.run(cmd, cwd=tmp, check=True)
+          PY
+
+  cpu-ddp-smoke:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Sync dependencies
+        run: uv sync --all-extras --dev
+
+      - name: CPU DDP smoke (gloo backend)
+        run: bash scripts/run_cpu_ddp_smoke.sh
+
+  passkey-smoke:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Sync dependencies
+        run: uv sync --all-extras --dev
+
+      - name: Run synthetic passkey memorization test
+        run: bash scripts/tests/run_passkey_smoke.sh
+
+  fidelity-subset:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Sync dependencies
+        run: uv sync --all-extras --dev
+
+      - name: Run fidelity subset + compliance report
+        run: bash scripts/checks/run_fidelity_ci_subset.sh

nested_learning-0.2.0/.github/workflows/release.yml

@@ -0,0 +1,79 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: write
+  id-token: write
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Build source and wheel distributions
+        run: uv build
+
+      - name: Twine check
+        run: uvx twine check dist/*
+
+      - name: Upload dist artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/*
+
+  publish-testpypi:
+    if: contains(github.ref_name, 'rc')
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: testpypi
+      url: https://test.pypi.org/p/nested-learning
+    steps:
+      - name: Download dist artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist
+
+      - name: Publish to TestPyPI via Trusted Publishing
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          repository-url: https://test.pypi.org/legacy/
+          packages-dir: dist/
+
+  publish-pypi:
+    if: ${{ !contains(github.ref_name, 'rc') }}
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/nested-learning
+    steps:
+      - name: Download dist artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist
+
+      - name: Publish to PyPI via Trusted Publishing
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: dist/
+

nested_learning-0.2.0/.github/workflows/security.yml

@@ -0,0 +1,45 @@
+name: Security
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+    branches: ["main"]
+  schedule:
+    - cron: "0 6 * * 1"
+
+jobs:
+  dependency-audit:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "0.9.8"
+
+      - name: Export requirements
+        run: uv export --all-extras --dev --format requirements-txt --output-file /tmp/requirements.txt
+
+      - name: pip-audit
+        run: uvx pip-audit -r /tmp/requirements.txt
+        continue-on-error: true
+
+  secret-scan:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Gitleaks scan
+        uses: gitleaks/gitleaks-action@v2
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+

nested_learning-0.2.0/.gitignore

@@ -0,0 +1,40 @@
+# Environment / tooling
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# Local artifacts
+logs/
+artifacts/
+/data/
+outputs/
+checkpoints/
+*.pt
+train.log
+train_dist.log
+ref_repos/
+configs/_tmp*
+git.env
+docs/POSTS.md
+docs/EX_*.md
+docs/CHECK_2_PLANNING_MODEL_REQUEST.md
+docs/CHECK_2_PLANNING_MODEL_RESPONSE.md
+docs/planner_check2_attachments.zip
+docs/tmp/
+docs_tmp/
+wandb/
+eval/*_ci.json
+
+# Local paper scans / scratch references (keep tracked references separate)
+google_papers/*_arXiv_v1.pdf
+google_papers/*_arXiv_v1/
+google_papers/Nested_Learning_Full_Paper.pdf
+google_papers/Nested_Learning_Full_Paper/
+
+# Editors
+.DS_Store
+.idea/
+.vscode/

nested_learning-0.2.0/CHANGELOG.md

@@ -0,0 +1,59 @@
+# Changelog
+
+All notable changes to this project will be documented here. The format loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and uses semantic versioning once tagged releases begin.
+
+## [Unreleased]
+### Added
+- Optional attention KV-cache path for continuous streaming inference (`init_attention_cache`, `attention_cache`, `return_attention_cache`) across HOPE/TITAN/Transformer blocks.
+- Boundary-target online chunking mode (`train.online_boundary_targets`) and optional training-time attention-cache carry (`train.online_carry_attention_cache`) for stronger chunk-boundary semantics.
+- Evaluation streaming-state utilities (`src/nested_learning/eval_state.py`) plus continual-eval controls (`--eval-state-mode`, `--eval-use-fast-state`, `--eval-use-attention-cache`).
+- Compliance report automation (`scripts/checks/compliance_report.py`) with CI subset + mechanism smoke integration.
+- Flash/SDPA-backed self-attention path with safe fallbacks, unlocking PyTorch 2.9 SDPA kernels by default.
+- Hydra toggles for bf16 autocast (`train.mixed_precision.*`), `torch.compile` (`train.compile.*`), and fused optimizers.
+- Muon + AdamW hybrid optimizer option exposed via `optim.type=muon`, routing ≥2D matrices through `torch.optim.Muon`.
+- Test-time memorization flags (`--memorize*`) documented in README + `docs/guide.md`, matching TITAN eval behavior.
+- Automation helpers: `scripts/run_e2e_smoke.sh` documented in Quickstart, plus new `scripts/run_cpu_ddp_smoke.sh` for CPU-only DDP/gloo smoke coverage.
+- Streaming contract doc (`docs/STREAMING_CONTRACT.md`) defining sequence/segment/chunk semantics and fast-state lifecycle.
+- Cadence verification utility (`scripts/checks/verify_update_cadence.py`) with synthetic tests and release-checklist integration.
+- Fidelity CI subset runner (`scripts/checks/run_fidelity_ci_subset.sh`) and mechanism-auditing smoke runner (`scripts/run_mechanism_audit_smoke.sh`).
+- Progress/status docs for P7 execution (`docs/PLAN_PROGRESS_P7.md`, `docs/IMPLEMENTATION_STATUS.md`).
+- Bug-report reproducibility checklist (`docs/BUG_REPORT_CHECKLIST.md`).
+- Boundary-state training-loop regression coverage (`tests/test_boundary_state_training_loop.py`) plus eval-loader/metadata roundtrip coverage (`tests/test_checkpoint_metadata_and_eval_loaders.py`).
+- `scripts/checks/check_data_script_help.sh` to guarantee `scripts/data/* --help` exits cleanly; wired into CI.
+- Markdown anchor verification in `scripts/checks/verify_docs_refs.py` with dedicated unit coverage.
+
+### Changed
+- README / compliance / streaming docs now reflect boundary-target mode, optional KV-cache carry, and explicit scope boundaries.
+- CPU DDP smoke now includes strict-mode fail-fast verification.
+- Repository license metadata now matches the shipped Apache-2.0 text; badges updated accordingly.
+- README and guide refreshed with performance knobs, optimizer guidance, and memorization instructions so release consumers have a single source of truth.
+- Release checklist tracks the new CPU DDP smoke script to keep packaging instructions aligned with available tooling.
+- Training loop strict-mode guardrails: `train.strict_streaming_contract` now fail-fasts on known semantics violations (DDP feature downgrades, shared-batch fast-state, non paper-defined variant in strict mode).
+- CMS telemetry now includes cadence metrics (`updates_applied`, `tokens_flushed`, `pending_tokens`, `gate_hits`) to make update-frequency behavior auditable.
+- Paper-auditing preset now explicitly enables strict streaming contract checks.
+- `configs/pilot_paper_faithful.yaml` now explicitly sets `train.online_updates=true` and tests verify no implicit algorithm-mode fallback.
+- Boundary-state mode now emits an explicit startup warning code (`experimental_boundary_state_mode`) and validates cache/chunk constraints early.
+- Checkpoint metadata now records algorithm/online flags (`algorithm_mode`, `online_updates`, `online_boundary_targets`, `online_carry_attention_cache`, `use_fast_state`), and release manifest includes those flags.
+- Data split fallback policy is deterministic across data scripts (`train -> validation -> test -> first available`) with explicit available-splits logging.
+
+### Upcoming
+- GitHub Actions workflow covering `ruff`, `mypy`, and `pytest`.
+- End-to-end release dry-run ahead of the `v0.1.0` tag.
+
+## [0.1.0] - 2025-11-09
+### Added
+- PyTorch **2.9.0** / torchvision **0.24.0** environment managed via `uv` with reproducible `pyproject.toml` + `uv.lock`.
+- HOPE block implementation (attention → TITAN memory → CMS + deep optimizers) with configurable level clocks and self-modifier wiring.
+- Hydrated Hydra config tree for pilot, mid, target, and CPU-only smoke runs plus DDP/FSDP/DeepSpeed entrypoints.
+- Data tooling: tokenizer trainer, corpus filtering, mixture processing, and `scripts/data/run_sample.sh` shortcut emitting stats under `data/mixtures/`.
+- Evaluation suite: zero-shot benchmark CLI (PIQA/HellaSwag/WinoGrande/ARC/BoolQ/SIQA), Needle-in-a-Haystack generator, continual-learning forgetting analyzer.
+- Sample artifacts (`artifacts/examples/pilot_dummy.pt`, `logs/pilot_smoke.json`, `logs/mid_smoke.json`) for reproducing eval commands without lengthy training.
+- Documentation set (`docs/stage1_plan.md`, `docs/stage2_plan.md`, `docs/data_pipeline.md`, `docs/guide.md`) outlining architecture, scaling strategy, and onboarding.
+
+### Changed
+- README rewritten with badges, quickstart commands, and references to the new guide + release checklist.
+- Logging defaults clarified (`logging.backend=json|wandb`), with instructions for saving structured metrics under `logs/`.
+
+### Known gaps
+- Release automation and CI are tracked in `docs/release_plan.md`.
+- Scaling guidance for >100 B token corpora pending additional storage + GPU availability.