nhge 0.1.0__tar.gz

nhge-0.1.0/PKG-INFO

@@ -0,0 +1,200 @@
+Metadata-Version: 2.4
+Name: nhge
+Version: 0.1.0
+Summary: Neuro-Harmonic Graph Engine — iterative harmonic resonance ML architecture
+Author: NHGE Project
+License: MIT
+Project-URL: Homepage, https://github.com/mwala400/nhge
+Project-URL: Repository, https://github.com/mwala400/nhge
+Project-URL: Issues, https://github.com/mwala400/nhge/issues
+Keywords: machine learning,deep learning,graph neural network,harmonic,transformer alternative,NLP,NHGE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=2.0.0
+Requires-Dist: numpy>=1.24.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+
+# Neuro-Harmonic Graph Engine (NHGE)
+#IDEA INITIATED BY SIR H.A.Mwala (Full name HEKIMA A. MWALA)
+#TANZANIAN
+A novel machine learning architecture that replaces transformer parallelism
+with **iterative harmonic resonance** over a dynamic graph structure.
+
+---
+
+## Key Idea
+
+Transformers compute attention in one parallel pass over all tokens simultaneously.
+**NHGE iterates** — tokens are graph nodes that resonate with each other, updating
+their states step by step until the graph reaches harmonic convergence.
+
+| Property            | Transformer                  | NHGE                              |
+|---------------------|------------------------------|-----------------------------------|
+| Processing          | Parallel attention           | Iterative harmonic resonance      |
+| Depth               | Fixed layers                 | Dynamic (stops at convergence)    |
+| Edge weights        | Attention scores             | Similarity × phase coherence      |
+| Memory              | O(N²) attention matrix       | O(N²) but iterative, not stacked  |
+| Simple inputs       | Same cost as complex         | Converges faster → cheaper        |
+
+---
+
+## Architecture
+
+```
+Input tokens
+     ↓
+[Embedding + positional + phase initialisation]
+     ↓
+┌─────────────────────────────────────────┐
+│  Harmonic iteration loop (max T steps)  │
+│                                         │
+│  HarmonicEdgeLayer                      │
+│    edge_w = softmax(Q·K/√d) × cos(Δθ)  │
+│                                         │
+│  HarmonicNodeUpdate                     │
+│    h_v ← h_v + FFN(h_v + Σ w·h_u)      │
+│                                         │
+│  PhaseUpdate                            │
+│    θ ← θ + α·tanh(W·h)·π               │
+│                                         │
+│  Convergence: ||h_t − h_{t−1}|| < ε    │
+└─────────────────────────────────────────┘
+     ↓
+[Graph readout: mean / CLS / attention pool]
+     ↓
+Output logits
+```
+
+---
+
+## Files
+
+| File                 | Purpose                                         |
+|----------------------|-------------------------------------------------|
+| `nhge_model.py`      | Core NHGE architecture (all layers + full model)|
+| `nhge_tokenizer.py`  | Word / char / subword tokenizer                 |
+| `nhge_trainer.py`    | Training engine with warmup LR, AMP, checkpoints|
+| `nhge_inference.py`  | Generation, classification, embeddings          |
+| `demo.py`            | Runnable end-to-end demo (no GPU needed)        |
+
+---
+
+## Quick start
+
+```python
+from nhge_model     import nhge_small
+from nhge_tokenizer import NHGETokenizer
+from nhge_inference import NHGEInference
+
+# 1. Build tokenizer
+tok = NHGETokenizer(mode="word")
+tok.build_vocab(your_texts, min_freq=2)
+
+# 2. Build model
+model = nhge_small(vocab_size=tok.vocab_size, num_classes=2)
+
+# 3. Inference
+inf = NHGEInference(model, tok, device="cuda")
+results = inf.classify(["Your input text here"], label_names=["neg", "pos"])
+
+# 4. Generate
+text = inf.generate("The harmonic graph", max_new_tokens=50)
+```
+
+---
+
+## Training
+
+```python
+from nhge_trainer import NHGETrainer, TokenDataset
+from torch.utils.data import DataLoader
+
+dataset = TokenDataset(encoded_tokens, labels=class_labels, max_len=128)
+loader  = DataLoader(dataset, batch_size=32, shuffle=True)
+
+trainer = NHGETrainer(model, loader, val_loader=val_loader,
+                      task="cls", lr=3e-4, device="cuda")
+history = trainer.train(epochs=20)
+```
+
+---
+
+## Model sizes
+
+| Name          | d_model | n_heads | n_layers | max_iter | ~Params |
+|---------------|---------|---------|----------|----------|---------|
+| `nhge_small`  | 128     | 4       | 2        | 6        | ~3M     |
+| `nhge_base`   | 512     | 8       | 4        | 8        | ~85M    |
+| `nhge_large`  | 1024    | 16      | 6        | 10       | ~340M   |
+
+---
+
+## Requirements
+
+```
+torch >= 2.0
+```
+
+No other dependencies required for core functionality.
+
+---
+
+## Run the demo
+
+```bash
+python demo.py
+```
+
+---
+
+## Theory
+
+The harmonic edge weight combines two signals:
+1. **Semantic similarity** — scaled dot product of Q and K projections
+2. **Phase coherence** — cos(θ_i − θ_j) between learned phase angles
+
+Nodes in phase amplify each other's signals. Out-of-phase nodes cancel.
+This mimics oscillatory dynamics in biological neural networks, where
+synchronised firing encodes binding of related concepts.
+
+The phase angles are updated each iteration via a damped gradient:
+```
+θ ← θ + α · tanh(W·h) · π
+```
+This allows the network to "tune" its resonance frequencies as information
+propagates — a form of learned synchronisation.
+
+---
+
+## Licence
+
+MIT — use freely, extend openly, credit the NHGE project.
+
+
+MIT License
+
+Copyright (c) 2026 H.A. Mwala
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.

nhge-0.1.0/README.md

@@ -0,0 +1,170 @@
+# Neuro-Harmonic Graph Engine (NHGE)
+#IDEA INITIATED BY SIR H.A.Mwala (Full name HEKIMA A. MWALA)
+#TANZANIAN
+A novel machine learning architecture that replaces transformer parallelism
+with **iterative harmonic resonance** over a dynamic graph structure.
+
+---
+
+## Key Idea
+
+Transformers compute attention in one parallel pass over all tokens simultaneously.
+**NHGE iterates** — tokens are graph nodes that resonate with each other, updating
+their states step by step until the graph reaches harmonic convergence.
+
+| Property            | Transformer                  | NHGE                              |
+|---------------------|------------------------------|-----------------------------------|
+| Processing          | Parallel attention           | Iterative harmonic resonance      |
+| Depth               | Fixed layers                 | Dynamic (stops at convergence)    |
+| Edge weights        | Attention scores             | Similarity × phase coherence      |
+| Memory              | O(N²) attention matrix       | O(N²) but iterative, not stacked  |
+| Simple inputs       | Same cost as complex         | Converges faster → cheaper        |
+
+---
+
+## Architecture
+
+```
+Input tokens
+     ↓
+[Embedding + positional + phase initialisation]
+     ↓
+┌─────────────────────────────────────────┐
+│  Harmonic iteration loop (max T steps)  │
+│                                         │
+│  HarmonicEdgeLayer                      │
+│    edge_w = softmax(Q·K/√d) × cos(Δθ)  │
+│                                         │
+│  HarmonicNodeUpdate                     │
+│    h_v ← h_v + FFN(h_v + Σ w·h_u)      │
+│                                         │
+│  PhaseUpdate                            │
+│    θ ← θ + α·tanh(W·h)·π               │
+│                                         │
+│  Convergence: ||h_t − h_{t−1}|| < ε    │
+└─────────────────────────────────────────┘
+     ↓
+[Graph readout: mean / CLS / attention pool]
+     ↓
+Output logits
+```
+
+---
+
+## Files
+
+| File                 | Purpose                                         |
+|----------------------|-------------------------------------------------|
+| `nhge_model.py`      | Core NHGE architecture (all layers + full model)|
+| `nhge_tokenizer.py`  | Word / char / subword tokenizer                 |
+| `nhge_trainer.py`    | Training engine with warmup LR, AMP, checkpoints|
+| `nhge_inference.py`  | Generation, classification, embeddings          |
+| `demo.py`            | Runnable end-to-end demo (no GPU needed)        |
+
+---
+
+## Quick start
+
+```python
+from nhge_model     import nhge_small
+from nhge_tokenizer import NHGETokenizer
+from nhge_inference import NHGEInference
+
+# 1. Build tokenizer
+tok = NHGETokenizer(mode="word")
+tok.build_vocab(your_texts, min_freq=2)
+
+# 2. Build model
+model = nhge_small(vocab_size=tok.vocab_size, num_classes=2)
+
+# 3. Inference
+inf = NHGEInference(model, tok, device="cuda")
+results = inf.classify(["Your input text here"], label_names=["neg", "pos"])
+
+# 4. Generate
+text = inf.generate("The harmonic graph", max_new_tokens=50)
+```
+
+---
+
+## Training
+
+```python
+from nhge_trainer import NHGETrainer, TokenDataset
+from torch.utils.data import DataLoader
+
+dataset = TokenDataset(encoded_tokens, labels=class_labels, max_len=128)
+loader  = DataLoader(dataset, batch_size=32, shuffle=True)
+
+trainer = NHGETrainer(model, loader, val_loader=val_loader,
+                      task="cls", lr=3e-4, device="cuda")
+history = trainer.train(epochs=20)
+```
+
+---
+
+## Model sizes
+
+| Name          | d_model | n_heads | n_layers | max_iter | ~Params |
+|---------------|---------|---------|----------|----------|---------|
+| `nhge_small`  | 128     | 4       | 2        | 6        | ~3M     |
+| `nhge_base`   | 512     | 8       | 4        | 8        | ~85M    |
+| `nhge_large`  | 1024    | 16      | 6        | 10       | ~340M   |
+
+---
+
+## Requirements
+
+```
+torch >= 2.0
+```
+
+No other dependencies required for core functionality.
+
+---
+
+## Run the demo
+
+```bash
+python demo.py
+```
+
+---
+
+## Theory
+
+The harmonic edge weight combines two signals:
+1. **Semantic similarity** — scaled dot product of Q and K projections
+2. **Phase coherence** — cos(θ_i − θ_j) between learned phase angles
+
+Nodes in phase amplify each other's signals. Out-of-phase nodes cancel.
+This mimics oscillatory dynamics in biological neural networks, where
+synchronised firing encodes binding of related concepts.
+
+The phase angles are updated each iteration via a damped gradient:
+```
+θ ← θ + α · tanh(W·h) · π
+```
+This allows the network to "tune" its resonance frequencies as information
+propagates — a form of learned synchronisation.
+
+---
+
+## Licence
+
+MIT — use freely, extend openly, credit the NHGE project.
+
+
+MIT License
+
+Copyright (c) 2026 H.A. Mwala
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.

nhge-0.1.0/nhge/__init__.py

@@ -0,0 +1,51 @@
+"""
+NHGE — Neuro-Harmonic Graph Engine
+====================================
+A next-generation ML architecture that replaces transformer parallelism
+with iterative harmonic resonance over a dynamic graph structure.
+
+Quick start:
+    from nhge import NHGE, nhge_small, nhge_base, nhge_large
+    from nhge import NHGETokenizer, NHGETrainer, NHGEInference
+"""
+
+__version__ = "0.1.0"
+__author__ = "NHGE Project"
+__license__ = "MIT"
+
+from nhge.nhge_model import (
+    NHGE,
+    NHGEBlock,
+    HarmonicEdgeLayer,
+    HarmonicNodeUpdate,
+    PhaseUpdate,
+    nhge_small,
+    nhge_base,
+    nhge_large,
+)
+
+from nhge.nhge_tokenizer import NHGETokenizer
+
+from nhge.nhge_trainer import NHGETrainer, TokenDataset, WarmupCosineScheduler
+
+from nhge.nhge_inference import NHGEInference
+
+__all__ = [
+    # Model
+    "NHGE",
+    "NHGEBlock",
+    "HarmonicEdgeLayer",
+    "HarmonicNodeUpdate",
+    "PhaseUpdate",
+    "nhge_small",
+    "nhge_base",
+    "nhge_large",
+    # Tokenizer
+    "NHGETokenizer",
+    # Training
+    "NHGETrainer",
+    "TokenDataset",
+    "WarmupCosineScheduler",
+    # Inference
+    "NHGEInference",
+]

nhge-0.1.0/nhge/demo.py

@@ -0,0 +1,162 @@
+"""
+NHGE Quick-Start Demo
+======================
+Run this file to see NHGE working end-to-end:
+  1. Build a tiny vocabulary from sample text
+  2. Construct an NHGE-small model
+  3. Do a forward pass and inspect convergence behaviour
+  4. Run a mock classification task
+  5. Print harmonic state diagnostics
+
+No GPU required — runs on CPU in seconds.
+"""
+
+import torch
+import sys
+import os
+
+# Allow imports from the same directory
+sys.path.insert(0, os.path.dirname(__file__))
+
+try:
+    from nhge.nhge_model import NHGE, nhge_small
+    from nhge.nhge_tokenizer import NHGETokenizer
+    from nhge.nhge_inference import NHGEInference
+except ImportError:
+    from nhge_model import NHGE, nhge_small
+    from nhge_tokenizer import NHGETokenizer
+    from nhge_inference import NHGEInference
+
+
+# ------------------------------------------------------------------
+# 1. Sample corpus
+# ------------------------------------------------------------------
+
+CORPUS = [
+    "the neuro harmonic graph engine processes tokens as graph nodes",
+    "harmonic resonance allows information to propagate iteratively",
+    "unlike transformers nhge does not require parallel attention",
+    "each node updates its state based on neighbouring node phases",
+    "convergence is detected dynamically reducing unnecessary computation",
+    "the model adapts the number of iterations to input complexity",
+    "simple inputs converge quickly complex ones require more iterations",
+    "phase alignment between nodes encodes semantic similarity",
+    "the harmonic edge weight combines similarity and phase coherence",
+    "this architecture is an advancement over the transformer model",
+]
+
+LABELS = [0, 0, 1, 0, 0, 0, 0, 0, 0, 1]    # 0=architecture, 1=comparison
+LABEL_NAMES = ["architecture", "comparison"]
+
+
+# ------------------------------------------------------------------
+# 2. Build tokenizer
+# ------------------------------------------------------------------
+
+print("=" * 60)
+print("NEURO-HARMONIC GRAPH ENGINE — Demo")
+print("=" * 60)
+print()
+
+tok = NHGETokenizer(mode="word")
+tok.build_vocab(CORPUS, min_freq=1, max_vocab=500)
+print(f"Vocab size: {tok.vocab_size}")
+print()
+
+
+# ------------------------------------------------------------------
+# 3. Build NHGE-small
+# ------------------------------------------------------------------
+
+model = nhge_small(
+    vocab_size=tok.vocab_size,
+    num_classes=2,      # binary classification demo
+    readout="attention",
+    epsilon=1e-4,
+    dropout=0.0,        # no dropout for inference demo
+)
+
+total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
+print(f"Model parameters: {total_params:,}")
+print()
+
+
+# ------------------------------------------------------------------
+# 4. Single forward pass — inspect convergence
+# ------------------------------------------------------------------
+
+print("--- Single forward pass ---")
+enc = tok.batch_encode(CORPUS[:3], max_length=24, add_cls=True)
+ids  = torch.tensor(enc["input_ids"],  dtype=torch.long)
+mask = torch.tensor(enc["attention_mask"], dtype=torch.bool)
+
+with torch.no_grad():
+    out = model(ids, mask, lm_mode=False, return_iterations=True)
+
+print(f"Logits shape  : {out['logits'].shape}")
+print(f"Iterations ran: {out['iterations']}")
+print(f"Deltas per iter: {[f'{d:.5f}' for d in out.get('deltas', [])]}")
+print()
+
+
+# ------------------------------------------------------------------
+# 5. Classification demo
+# ------------------------------------------------------------------
+
+print("--- Classification demo ---")
+inf = NHGEInference(model, tok, device="cpu")
+
+results = inf.classify(
+    ["harmonic resonance drives the graph update",
+     "nhge replaces the transformer in language models"],
+    max_length=24,
+    label_names=LABEL_NAMES,
+)
+for i, r in enumerate(results):
+    probs = [f"{p:.3f}" for p in r["scores"]]
+    print(f"  [{i}] pred={r['name']:<14} scores={probs}  iters={r['iters']}")
+print()
+
+
+# ------------------------------------------------------------------
+# 6. Harmonic state introspection
+# ------------------------------------------------------------------
+
+print("--- Harmonic state introspection ---")
+state = inf.harmonic_state(
+    "phase alignment between nodes encodes semantic similarity",
+    max_length=16,
+)
+print(f"  Tokens    : {state['tokens']}")
+print(f"  Iterations: {state['iterations']}")
+print(f"  Deltas    : {[f'{d:.5f}' for d in state['deltas']]}")
+print(f"  Node norms: {[f'{n:.3f}' for n in state['h_norm']]}")
+print()
+
+
+# ------------------------------------------------------------------
+# 7. Embedding similarity
+# ------------------------------------------------------------------
+
+print("--- Embedding similarity ---")
+embs = inf.embed(
+    ["harmonic graph iteration",
+     "neuro harmonic resonance",
+     "unrelated random words here"],
+    max_length=12,
+)
+def cosine(a, b):
+    return (a @ b / (a.norm() * b.norm())).item()
+
+print(f"  sim(0,1) = {cosine(embs[0], embs[1]):.4f}  ← expect higher (related)")
+print(f"  sim(0,2) = {cosine(embs[0], embs[2]):.4f}  ← expect lower  (unrelated)")
+print()
+
+print("=" * 60)
+print("NHGE demo complete — model is working correctly.")
+print("Next steps:")
+print("  1. Build a real tokenizer with nhge_tokenizer.NHGETokenizer.build_vocab()")
+print("  2. Prepare DataLoaders using nhge_trainer.TokenDataset")
+print("  3. Train with nhge_trainer.NHGETrainer(model, train_loader, ...)")
+print("  4. Generate text with nhge_inference.NHGEInference.generate()")
+print("=" * 60)