natumpy 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- natumpy-1.0.0/LICENSE +21 -0
- natumpy-1.0.0/PKG-INFO +34 -0
- natumpy-1.0.0/README.md +156 -0
- natumpy-1.0.0/natumpy/__init__.py +37 -0
- natumpy-1.0.0/natumpy/layers.py +59 -0
- natumpy-1.0.0/natumpy/model.py +26 -0
- natumpy-1.0.0/natumpy/nat_bindings.cpp +115 -0
- natumpy-1.0.0/natumpy/nat_engine.cpp +213 -0
- natumpy-1.0.0/natumpy/nat_engine.hpp +78 -0
- natumpy-1.0.0/natumpy/text.py +63 -0
- natumpy-1.0.0/natumpy/utils.py +13 -0
- natumpy-1.0.0/natumpy.egg-info/PKG-INFO +34 -0
- natumpy-1.0.0/natumpy.egg-info/SOURCES.txt +17 -0
- natumpy-1.0.0/natumpy.egg-info/dependency_links.txt +1 -0
- natumpy-1.0.0/natumpy.egg-info/requires.txt +1 -0
- natumpy-1.0.0/natumpy.egg-info/top_level.txt +1 -0
- natumpy-1.0.0/pyproject.toml +4 -0
- natumpy-1.0.0/setup.cfg +4 -0
- natumpy-1.0.0/setup.py +55 -0
natumpy-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Eternals
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
natumpy-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: natumpy
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Nawa Causal Engine: Complex-Valued Reservoir Computing based on Harmonic Resonance.
|
|
5
|
+
Home-page: https://github.com/arclaav/natumpy
|
|
6
|
+
Author: Eternals
|
|
7
|
+
Author-email: admin@defacer.cloud-ip.cc
|
|
8
|
+
Keywords: ai,reservoir-computing,esn,complex-valued,neural-network,c++,nawa
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Intended Audience :: Science/Research
|
|
11
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
12
|
+
Classifier: Programming Language :: C++
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Operating System :: OS Independent
|
|
16
|
+
Requires-Python: >=3.8
|
|
17
|
+
License-File: LICENSE
|
|
18
|
+
Requires-Dist: numpy>=1.20.0
|
|
19
|
+
Dynamic: author
|
|
20
|
+
Dynamic: author-email
|
|
21
|
+
Dynamic: classifier
|
|
22
|
+
Dynamic: description
|
|
23
|
+
Dynamic: home-page
|
|
24
|
+
Dynamic: keywords
|
|
25
|
+
Dynamic: license-file
|
|
26
|
+
Dynamic: requires-dist
|
|
27
|
+
Dynamic: requires-python
|
|
28
|
+
Dynamic: summary
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
Natumpy is a deterministic AI engine that uses complex-valued signals (phase & amplitude)
|
|
32
|
+
to model information as interference patterns, distinct from probabilistic Transformers.
|
|
33
|
+
Powered by a high-performance C++ backend implementing the Nawa (9-Sphere) topology.
|
|
34
|
+
|
natumpy-1.0.0/README.md
ADDED
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
[](https://opensource.org/licenses/MIT)
|
|
2
|
+
[](https://www.python.org/downloads/)
|
|
3
|
+
[](https://github.com/eternals/natumpy)
|
|
4
|
+
[]()
|
|
5
|
+
|
|
6
|
+
> **"God does not play dice." — Albert Einstein**
|
|
7
|
+
>
|
|
8
|
+
> **"Neither does Natumpy." — Eternals**
|
|
9
|
+
|
|
10
|
+
**Natumpy** adalah framework kecerdasan buatan berbasis **Fisika Gelombang (Harmonic Resonance)** dan **Reservoir Computing**.
|
|
11
|
+
Berbeda dengan arsitektur Transformer (GPT/LLM) yang mengandalkan probabilitas statistik (dadu) dan memori *KV-Cache* yang boros, Natumpy menggunakan prinsip deterministik aljabar linear dan sinyal kompleks (fase & amplitudo) untuk memodelkan kognisi yang "Cair" (*Liquid*).
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Mengapa Begitu?
|
|
16
|
+
|
|
17
|
+
Dominasi Transformer saat ini memiliki kelemahan fundamental: **Inefisiensi Energi & Halusinasi**. Natumpy hadir sebagai antitesis dengan pendekatan biomimetik.
|
|
18
|
+
|
|
19
|
+
| Fitur | Transformer (GPT-4) | **Natumpy (Nawa)** |
|
|
20
|
+
| :--- | :--- | :--- |
|
|
21
|
+
| **Filosofi** | Probabilitas Statistik | **Resonansi Fisika** |
|
|
22
|
+
| **Logika** | "Menebak" kata selanjutnya | **"Mengalir"** ke ekuilibrium |
|
|
23
|
+
| **Kompleksitas** | $O(N^2)$ (Berat di teks panjang) | **$O(N)$ (Linear / Sangat Cepat)** |
|
|
24
|
+
| **Representasi** | Bilangan Real (Skalar) | **Bilangan Kompleks (Vektor Fasa)** |
|
|
25
|
+
| **Memori** | KV-Cache (Boros RAM) | **Holographic State (Efisien)** |
|
|
26
|
+
| **Training** | Backpropagation (Butuh GPU Sultan) | **Ridge Regression (Bisa di Laptop)** |
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Arsitektur "NAWA" (9 Spheres)
|
|
31
|
+
|
|
32
|
+
Natumpy dibangun di atas struktur **C++ High-Performance** yang mengimplementasikan **Harmonic Hierarchy**:
|
|
33
|
+
|
|
34
|
+
### 1. **NAT (Liquid State):** Neuron tidak statis. Bobotnya bersifat "cair" (*liquid time-constant*), bereaksi terhadap input secara real-time dan memiliki inersia memori.
|
|
35
|
+
### 2. **NAWA (9 Lapisan Waktu):** Otak Natumpy terdiri dari 9 bola resonansi yang berjalan pada frekuensi waktu yang berbeda ($2^0$ hingga $2^8$).
|
|
36
|
+
* **Layer 0 (Refleks):** Detak cepat (1ms). Menangkap detail input sensorik.
|
|
37
|
+
* **Layer 4 (Kognisi):** Detak sedang. Memahami kata dan konsep.
|
|
38
|
+
* **Layer 8 (Kesadaran):** Detak lambat (256ms). Menyimpan konteks global dan jati diri, mencegah halusinasi jangka pendek.
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
# Instalasi
|
|
45
|
+
|
|
46
|
+
Pastikan Anda memiliki compiler C++17 (GCC/Clang/MSVC) dan Python 3.8+.
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
# Clone repository
|
|
50
|
+
git clone https://github.com/eternalys/natumpy.git
|
|
51
|
+
cd natumpy
|
|
52
|
+
```
|
|
53
|
+
```
|
|
54
|
+
# Install (Compiling C++ Core secara otomatis)
|
|
55
|
+
pip install .
|
|
56
|
+
```
|
|
57
|
+
# Memulai (Quick Start)
|
|
58
|
+
## 1. Hello Resonance
|
|
59
|
+
Melihat bagaimana Natumpy merespons sinyal input di level sinyal (Gelombang).
|
|
60
|
+
```
|
|
61
|
+
import natumpy as nt
|
|
62
|
+
import numpy as np
|
|
63
|
+
|
|
64
|
+
# Inisialisasi Engine 2048 Dimensi
|
|
65
|
+
engine = nt.create_engine(2048)
|
|
66
|
+
|
|
67
|
+
# Buat Sinyal Input (Vektor Kompleks)
|
|
68
|
+
# Input Real (0.5) dan Imaginer (0.1)
|
|
69
|
+
in_r = np.full(2048, 0.5)
|
|
70
|
+
in_i = np.full(2048, 0.1)
|
|
71
|
+
|
|
72
|
+
# Step Fisika (Berpikir 1 langkah)
|
|
73
|
+
engine.step(in_r, in_i)
|
|
74
|
+
|
|
75
|
+
# Intip isi otak (Layer 0)
|
|
76
|
+
state = nt.get_state_complex(engine, layer_idx=0)
|
|
77
|
+
print(f"Energi Otak: {np.mean(np.abs(state)):.4f}")
|
|
78
|
+
```
|
|
79
|
+
## 2. Membangun Chatbot (Reservoir Computing)
|
|
80
|
+
Pola desain Natumpy memisahkan Otak (Reservoir) dan Mulut (Readout).
|
|
81
|
+
* Otak (C++) bertugas menghasilkan pola gelombang rumit di dimensi tinggi (Chaos).
|
|
82
|
+
* Mulut (Python) dilatih sederhana untuk menerjemahkan gelombang itu menjadi kata-kata.
|
|
83
|
+
```
|
|
84
|
+
import natumpy as nat
|
|
85
|
+
import numpy as np
|
|
86
|
+
|
|
87
|
+
# --- SETUP ---
|
|
88
|
+
DIM = 2048
|
|
89
|
+
otak = nat.ReservoirLayer(DIM)
|
|
90
|
+
mulut = nat.ReadoutLayer(input_dim=DIM*4, output_dim=DIM*2) # 4 Layer Features
|
|
91
|
+
tokenizer = nat.text.ResonantTokenizer(DIM)
|
|
92
|
+
|
|
93
|
+
TEXT = "Natumpy adalah kecerdasan buatan masa depan yang efisien."
|
|
94
|
+
|
|
95
|
+
# --- FASE 1: MENDENGAR (Rekam Gelombang Otak) ---
|
|
96
|
+
X_states = []
|
|
97
|
+
Y_targets = []
|
|
98
|
+
|
|
99
|
+
# Encode Teks ke Gelombang
|
|
100
|
+
inputs_r, inputs_i = tokenizer.encode(TEXT)
|
|
101
|
+
targets_r, targets_i = tokenizer.encode(TEXT[1:] + " ") # Prediksi huruf selanjutnya
|
|
102
|
+
|
|
103
|
+
print("Sedang membaca...")
|
|
104
|
+
for t in range(len(inputs_r)):
|
|
105
|
+
# 1. Masukkan suara ke otak
|
|
106
|
+
state = otak.forward(inputs_r[t], inputs_i[t])
|
|
107
|
+
X_states.append(state)
|
|
108
|
+
|
|
109
|
+
# 2. Siapkan target jawaban
|
|
110
|
+
target_vec = np.concatenate([targets_r[t], targets_i[t]])
|
|
111
|
+
Y_targets.append(target_vec)
|
|
112
|
+
|
|
113
|
+
# --- FASE 2: BELAJAR BICARA (Linear Algebra) ---
|
|
114
|
+
# Tidak ada epoch ribuan kali. Satu kali hitung (Exact Solution).
|
|
115
|
+
mulut.fit(np.array(X_states), np.array(Y_targets))
|
|
116
|
+
|
|
117
|
+
# --- FASE 3: GENERASI ---
|
|
118
|
+
print("Output:", end=" ")
|
|
119
|
+
# Seed awal
|
|
120
|
+
pr, pi = tokenizer.encode("Natumpy")
|
|
121
|
+
for i in range(len(pr)): otak.forward(pr[i], pi[i])
|
|
122
|
+
|
|
123
|
+
# Biarkan dia bicara sendiri (Feedback Loop)
|
|
124
|
+
curr_r, curr_i = pr[-1], pi[-1]
|
|
125
|
+
for _ in range(50):
|
|
126
|
+
# Ambil state -> Terjemahkan lewat mulut -> Decode jadi huruf
|
|
127
|
+
state = otak.get_state()
|
|
128
|
+
pred_vec = mulut.predict(state)
|
|
129
|
+
|
|
130
|
+
pred_r = pred_vec[:DIM]
|
|
131
|
+
pred_i = pred_vec[DIM:]
|
|
132
|
+
char = tokenizer.decode_sequence([pred_r], [pred_i])
|
|
133
|
+
|
|
134
|
+
print(char, end="", flush=True)
|
|
135
|
+
|
|
136
|
+
# Suara masuk kembali ke telinga (Auto-Regressive)
|
|
137
|
+
idx = tokenizer.decode(pred_r, pred_i)
|
|
138
|
+
next_r = tokenizer.embeddings_r[idx]
|
|
139
|
+
next_i = tokenizer.embeddings_i[idx]
|
|
140
|
+
otak.forward(next_r, next_i)
|
|
141
|
+
```
|
|
142
|
+
# Komponen Library
|
|
143
|
+
* natumpy.natcore: C++ Binding (Jantung Fisika). Jangan dimodifikasi kecuali Anda paham Aljabar Linear Kompleks.
|
|
144
|
+
* natumpy.layers: Abstraksi Python (ReservoirLayer, ReadoutLayer). Memudahkan pembuatan model.
|
|
145
|
+
* natumpy.text: Resonant Tokenizer. Mengubah teks menjadi gelombang tanpa kamus kata (Byte-level processing).
|
|
146
|
+
* natumpy.model: Wadah untuk menyusun arsitektur Deep Learning (Stacking Layers).
|
|
147
|
+
# Kontribusi
|
|
148
|
+
Proyek ini adalah eksperimen ilmiah terbuka. Kami mencari kontributor di bidang:
|
|
149
|
+
* Geometric Algebra: Untuk upgrade v9.0 (Multivectors / Clifford Algebra).
|
|
150
|
+
* Optimization: Implementasi CUDA/OpenMP untuk C++ Core agar mendukung jutaan neuron.
|
|
151
|
+
* Linguistics: Memperbaiki topologi semantik pada Tokenizer.
|
|
152
|
+
# Lisensi
|
|
153
|
+
MIT License
|
|
154
|
+
Copyright (c) 2026 Eternals.
|
|
155
|
+
|
|
156
|
+
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
import numpy as np
|
|
2
|
+
from . import natcore
|
|
3
|
+
from . import layers
|
|
4
|
+
from . import model
|
|
5
|
+
from . import utils
|
|
6
|
+
|
|
7
|
+
__version__ = "8.0.0"
|
|
8
|
+
|
|
9
|
+
NawaEngine = natcore.NawaSystem
|
|
10
|
+
|
|
11
|
+
Model = model.Model
|
|
12
|
+
BaseLayer = layers.BaseLayer
|
|
13
|
+
HolographicLayer = layers.HolographicLayer
|
|
14
|
+
CausalLayer = layers.CausalLayer
|
|
15
|
+
|
|
16
|
+
create_orthogonal_vector = utils.create_orthogonal_vector
|
|
17
|
+
cosine_similarity = utils.cosine_similarity
|
|
18
|
+
|
|
19
|
+
def create_engine(size):
|
|
20
|
+
return NawaEngine(int(size))
|
|
21
|
+
|
|
22
|
+
def get_state_complex(engine, layer_idx=0):
|
|
23
|
+
real, imag = engine.get_state(int(layer_idx))
|
|
24
|
+
return real + 1j * imag
|
|
25
|
+
|
|
26
|
+
def train_causality(engine, layer_idx, cause_state, effect_state):
|
|
27
|
+
c_r = np.real(cause_state).astype(np.float64)
|
|
28
|
+
c_i = np.imag(cause_state).astype(np.float64)
|
|
29
|
+
e_r = np.real(effect_state).astype(np.float64)
|
|
30
|
+
e_i = np.imag(effect_state).astype(np.float64)
|
|
31
|
+
|
|
32
|
+
engine.learn(int(layer_idx), c_r, c_i, e_r, e_i)
|
|
33
|
+
|
|
34
|
+
def think_next(engine, input_vec):
|
|
35
|
+
ir = np.real(input_vec).astype(np.float64)
|
|
36
|
+
ii = np.imag(input_vec).astype(np.float64)
|
|
37
|
+
engine.step(ir, ii)
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
import numpy as np
|
|
2
|
+
from . import natcore
|
|
3
|
+
|
|
4
|
+
class BaseLayer:
|
|
5
|
+
def __init__(self, dim, name="Layer"):
|
|
6
|
+
self.dim = dim
|
|
7
|
+
self.name = name
|
|
8
|
+
self.engine = natcore.NawaSystem(dim)
|
|
9
|
+
|
|
10
|
+
def get_state(self):
|
|
11
|
+
r0, i0 = self.engine.get_state(0)
|
|
12
|
+
r4, i4 = self.engine.get_state(4)
|
|
13
|
+
return np.concatenate([r0, i0, r4, i4])
|
|
14
|
+
|
|
15
|
+
def step(self, r, i):
|
|
16
|
+
self.engine.step(r, i)
|
|
17
|
+
|
|
18
|
+
class ReservoirLayer(BaseLayer):
|
|
19
|
+
def forward(self, input_r, input_i):
|
|
20
|
+
self.engine.step(input_r, input_i)
|
|
21
|
+
return self.get_state()
|
|
22
|
+
|
|
23
|
+
def save(self, filename):
|
|
24
|
+
self.engine.save(filename)
|
|
25
|
+
|
|
26
|
+
def load(self, filename):
|
|
27
|
+
self.engine.load(filename)
|
|
28
|
+
|
|
29
|
+
class ReadoutLayer:
|
|
30
|
+
def __init__(self, input_dim, output_dim, alpha=1.0):
|
|
31
|
+
self.input_dim = input_dim
|
|
32
|
+
self.output_dim = output_dim
|
|
33
|
+
self.alpha = alpha
|
|
34
|
+
self.W_out = None
|
|
35
|
+
|
|
36
|
+
def fit(self, X_states, Y_targets):
|
|
37
|
+
print(f" [Readout] Melatih W_out ({X_states.shape} >> {Y_targets.shape})...")
|
|
38
|
+
|
|
39
|
+
A = X_states.T @ X_states + self.alpha * np.eye(X_states.shape[1])
|
|
40
|
+
B = X_states.T @ Y_targets
|
|
41
|
+
|
|
42
|
+
try:
|
|
43
|
+
self.W_out = np.linalg.solve(A, B)
|
|
44
|
+
print(" [Readout] Solusi Exact ditemukan.")
|
|
45
|
+
except np.linalg.LinAlgError:
|
|
46
|
+
print(" [Readout] Matrix Singular. Menggunakan PseudoInverse.")
|
|
47
|
+
self.W_out = np.linalg.pinv(A) @ B
|
|
48
|
+
|
|
49
|
+
def predict(self, state_vec):
|
|
50
|
+
if self.W_out is None:
|
|
51
|
+
raise ValueError("Readout belum dilatih, Panggil .fit() dulu.")
|
|
52
|
+
return state_vec @ self.W_out
|
|
53
|
+
|
|
54
|
+
def save(self, filename):
|
|
55
|
+
np.save(filename, self.W_out)
|
|
56
|
+
|
|
57
|
+
def load(self, filename):
|
|
58
|
+
self.W_out = np.load(filename)
|
|
59
|
+
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import numpy as np
|
|
2
|
+
|
|
3
|
+
class Model:
|
|
4
|
+
def __init__(self):
|
|
5
|
+
self.layers = {}
|
|
6
|
+
|
|
7
|
+
def add_layer(self, name, layer):
|
|
8
|
+
self.layers[name] = layer
|
|
9
|
+
|
|
10
|
+
def compile(self, beta=10.0, dt=0.2):
|
|
11
|
+
for name, layer in self.layers.items():
|
|
12
|
+
layer.set_config(beta=beta, dt=dt)
|
|
13
|
+
print(f" [Natumpy] Layer '{name}' configured.")
|
|
14
|
+
|
|
15
|
+
def summary(self):
|
|
16
|
+
print("\n🧠 NATUMPY MODEL ARCHITECTURE")
|
|
17
|
+
print("="*50)
|
|
18
|
+
for name, layer in self.layers.items():
|
|
19
|
+
t_name = layer.__class__.__name__
|
|
20
|
+
dim = layer.dim
|
|
21
|
+
if hasattr(layer.engine, 'memory_count'):
|
|
22
|
+
mem = layer.engine.memory_count
|
|
23
|
+
rule = layer.engine.rule_count
|
|
24
|
+
print(f" - {name:<12} | Type: {t_name:<16} | Dim: {dim} | Mem: {mem} | Rules: {rule}")
|
|
25
|
+
print("="*50 + "\n")
|
|
26
|
+
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
#define PY_SSIZE_T_CLEAN
|
|
2
|
+
#include <Python.h>
|
|
3
|
+
#include "nat_engine.hpp"
|
|
4
|
+
|
|
5
|
+
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
|
|
6
|
+
#include <numpy/arrayobject.h>
|
|
7
|
+
|
|
8
|
+
typedef struct {
|
|
9
|
+
PyObject_HEAD
|
|
10
|
+
NawaSystem* sys;
|
|
11
|
+
} PyNawaSystem;
|
|
12
|
+
|
|
13
|
+
static void PyNawaSystem_dealloc(PyNawaSystem* self) {
|
|
14
|
+
if (self->sys) delete self->sys;
|
|
15
|
+
Py_TYPE(self)->tp_free((PyObject*)self);
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
static PyObject* PyNawaSystem_new(PyTypeObject* type, PyObject* args, PyObject* kwds) {
|
|
19
|
+
PyNawaSystem* self = (PyNawaSystem*)type->tp_alloc(type, 0);
|
|
20
|
+
int dim = 512;
|
|
21
|
+
if (self && PyArg_ParseTuple(args, "|i", &dim)) {
|
|
22
|
+
self->sys = new NawaSystem((size_t)dim);
|
|
23
|
+
}
|
|
24
|
+
return (PyObject*)self;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
static bool py_to_vector(PyObject* obj, std::vector<Scalar>& vec) {
|
|
28
|
+
PyObject* arr = PyArray_ContiguousFromObject(obj, NPY_DOUBLE, 1, 1);
|
|
29
|
+
if (!arr) return false;
|
|
30
|
+
double* data = (double*)PyArray_DATA((PyArrayObject*)arr);
|
|
31
|
+
vec.assign(data, data + PyArray_DIM((PyArrayObject*)arr, 0));
|
|
32
|
+
Py_DECREF(arr);
|
|
33
|
+
return true;
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
static PyObject* PyNawaSystem_step(PyNawaSystem* self, PyObject* args) {
|
|
37
|
+
PyObject *ir, *ii;
|
|
38
|
+
if (!PyArg_ParseTuple(args, "OO", &ir, &ii)) return NULL;
|
|
39
|
+
std::vector<Scalar> vr, vi;
|
|
40
|
+
if (!py_to_vector(ir, vr) || !py_to_vector(ii, vi)) return NULL;
|
|
41
|
+
|
|
42
|
+
self->sys->step(vr, vi);
|
|
43
|
+
Py_RETURN_NONE;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
static PyObject* PyNawaSystem_save(PyNawaSystem* self, PyObject* args) {
|
|
47
|
+
const char* f; if(!PyArg_ParseTuple(args, "s", &f)) return NULL;
|
|
48
|
+
self->sys->save(std::string(f)); Py_RETURN_NONE;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
static PyObject* PyNawaSystem_load(PyNawaSystem* self, PyObject* args) {
|
|
52
|
+
const char* f; if(!PyArg_ParseTuple(args, "s", &f)) return NULL;
|
|
53
|
+
self->sys->load(std::string(f)); Py_RETURN_NONE;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
static PyObject* PyNawaSystem_get_state(PyNawaSystem* self, PyObject* args) {
|
|
57
|
+
int layer;
|
|
58
|
+
if (!PyArg_ParseTuple(args, "i", &layer)) return NULL;
|
|
59
|
+
|
|
60
|
+
auto r = self->sys->get_layer_r(layer);
|
|
61
|
+
auto i = self->sys->get_layer_i(layer);
|
|
62
|
+
|
|
63
|
+
npy_intp dims[1] = { (npy_intp)r.size() };
|
|
64
|
+
PyObject* ar = PyArray_SimpleNewFromData(1, dims, NPY_DOUBLE, r.data());
|
|
65
|
+
PyObject* ai = PyArray_SimpleNewFromData(1, dims, NPY_DOUBLE, i.data());
|
|
66
|
+
|
|
67
|
+
PyObject* safe_r = PyArray_NewCopy((PyArrayObject*)ar, NPY_ANYORDER);
|
|
68
|
+
PyObject* safe_i = PyArray_NewCopy((PyArrayObject*)ai, NPY_ANYORDER);
|
|
69
|
+
Py_DECREF(ar); Py_DECREF(ai);
|
|
70
|
+
|
|
71
|
+
return Py_BuildValue("(OO)", safe_r, safe_i);
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
static PyObject* PyNawaSystem_learn(PyNawaSystem* self, PyObject* args) {
|
|
75
|
+
int layer;
|
|
76
|
+
PyObject *ir, *ii, *or_, *oi;
|
|
77
|
+
if (!PyArg_ParseTuple(args, "iOOOO", &layer, &ir, &ii, &or_, &oi)) return NULL;
|
|
78
|
+
std::vector<Scalar> vir, vii, vor, voi;
|
|
79
|
+
if(!py_to_vector(ir, vir) || !py_to_vector(ii, vii) || !py_to_vector(or_, vor) || !py_to_vector(oi, voi)) return NULL;
|
|
80
|
+
|
|
81
|
+
self->sys->learn(layer, vir, vii, vor, voi);
|
|
82
|
+
|
|
83
|
+
Py_RETURN_NONE;
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
static PyMethodDef PyNawaSystem_methods[] = {
|
|
87
|
+
{"step", (PyCFunction)PyNawaSystem_step, METH_VARARGS, ""},
|
|
88
|
+
{"save", (PyCFunction)PyNawaSystem_save, METH_VARARGS, ""},
|
|
89
|
+
{"load", (PyCFunction)PyNawaSystem_load, METH_VARARGS, ""},
|
|
90
|
+
{"get_state", (PyCFunction)PyNawaSystem_get_state, METH_VARARGS, ""},
|
|
91
|
+
{"learn", (PyCFunction)PyNawaSystem_learn, METH_VARARGS, ""},
|
|
92
|
+
{NULL}
|
|
93
|
+
};
|
|
94
|
+
|
|
95
|
+
static PyTypeObject PyNawaSystemType = {
|
|
96
|
+
PyVarObject_HEAD_INIT(NULL, 0)
|
|
97
|
+
.tp_name = "natcore.NawaSystem",
|
|
98
|
+
.tp_basicsize = sizeof(PyNawaSystem),
|
|
99
|
+
.tp_dealloc = (destructor)PyNawaSystem_dealloc,
|
|
100
|
+
.tp_flags = Py_TPFLAGS_DEFAULT,
|
|
101
|
+
.tp_methods = PyNawaSystem_methods,
|
|
102
|
+
.tp_new = PyNawaSystem_new,
|
|
103
|
+
};
|
|
104
|
+
|
|
105
|
+
static struct PyModuleDef natcoremodule = { PyModuleDef_HEAD_INIT, "natcore", "Nawa Harmonic Engine", -1, NULL };
|
|
106
|
+
PyMODINIT_FUNC PyInit_natcore(void) {
|
|
107
|
+
import_array();
|
|
108
|
+
if (PyType_Ready(&PyNawaSystemType) < 0) return NULL;
|
|
109
|
+
PyObject* m = PyModule_Create(&natcoremodule);
|
|
110
|
+
if (!m) return NULL;
|
|
111
|
+
Py_INCREF(&PyNawaSystemType);
|
|
112
|
+
PyModule_AddObject(m, "NawaSystem", (PyObject*)&PyNawaSystemType);
|
|
113
|
+
return m;
|
|
114
|
+
}
|
|
115
|
+
|
|
@@ -0,0 +1,213 @@
|
|
|
1
|
+
#include "nat_engine.hpp"
|
|
2
|
+
#include <cmath>
|
|
3
|
+
#include <algorithm>
|
|
4
|
+
#include <numeric>
|
|
5
|
+
#include <vector>
|
|
6
|
+
#include <random>
|
|
7
|
+
#include <iostream>
|
|
8
|
+
#include <fstream>
|
|
9
|
+
|
|
10
|
+
inline Scalar sigmoid(Scalar x) { return 1.0 / (1.0 + std::exp(-x)); }
|
|
11
|
+
inline Scalar tanh_activation(Scalar x) { return std::tanh(x); }
|
|
12
|
+
|
|
13
|
+
void NawaMemory::init(size_t dimension) {
|
|
14
|
+
dim = dimension;
|
|
15
|
+
W_recurrent_r.resize(dim * dim, 0.0);
|
|
16
|
+
W_recurrent_i.resize(dim * dim, 0.0);
|
|
17
|
+
|
|
18
|
+
W_gate_r.resize(dim); W_gate_i.resize(dim);
|
|
19
|
+
W_input_r.resize(dim); W_input_i.resize(dim);
|
|
20
|
+
W_feedback_r.resize(dim); W_feedback_i.resize(dim);
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
void NawaMemory::save(std::ostream& out) {
|
|
24
|
+
out.write((char*)&dim, sizeof(size_t));
|
|
25
|
+
out.write((char*)W_recurrent_r.data(), dim*dim*sizeof(Scalar));
|
|
26
|
+
out.write((char*)W_recurrent_i.data(), dim*dim*sizeof(Scalar));
|
|
27
|
+
out.write((char*)W_gate_r.data(), dim*sizeof(Scalar)); out.write((char*)W_gate_i.data(), dim*sizeof(Scalar));
|
|
28
|
+
out.write((char*)W_input_r.data(), dim*sizeof(Scalar)); out.write((char*)W_input_i.data(), dim*sizeof(Scalar));
|
|
29
|
+
out.write((char*)W_feedback_r.data(), dim*sizeof(Scalar)); out.write((char*)W_feedback_i.data(), dim*sizeof(Scalar));
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
void NawaMemory::load(std::istream& in) {
|
|
33
|
+
size_t loaded_dim;
|
|
34
|
+
in.read((char*)&loaded_dim, sizeof(size_t));
|
|
35
|
+
if(loaded_dim != dim) return;
|
|
36
|
+
in.read((char*)W_recurrent_r.data(), dim*dim*sizeof(Scalar));
|
|
37
|
+
in.read((char*)W_recurrent_i.data(), dim*dim*sizeof(Scalar));
|
|
38
|
+
in.read((char*)W_gate_r.data(), dim*sizeof(Scalar)); in.read((char*)W_gate_i.data(), dim*sizeof(Scalar));
|
|
39
|
+
in.read((char*)W_input_r.data(), dim*sizeof(Scalar)); in.read((char*)W_input_i.data(), dim*sizeof(Scalar));
|
|
40
|
+
in.read((char*)W_feedback_r.data(), dim*sizeof(Scalar)); in.read((char*)W_feedback_i.data(), dim*sizeof(Scalar));
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
NawaSphere::NawaSphere(size_t dimension, int id, int ratio) {
|
|
44
|
+
layer_id = id; time_ratio = ratio;
|
|
45
|
+
state.resize(dimension); memory.init(dimension);
|
|
46
|
+
buf_input_r.resize(dimension, 0.0); buf_input_i.resize(dimension, 0.0);
|
|
47
|
+
buf_feedback_r.resize(dimension, 0.0); buf_feedback_i.resize(dimension, 0.0);
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
void NawaSphere::init_weights(int seed) {
|
|
51
|
+
std::mt19937 gen(seed);
|
|
52
|
+
std::normal_distribution<Scalar> d(0.0, 0.05);
|
|
53
|
+
|
|
54
|
+
for(size_t i=0; i<state.size; ++i) {
|
|
55
|
+
for(size_t j=0; j<state.size; ++j) {
|
|
56
|
+
size_t idx = i * state.size + j;
|
|
57
|
+
if (i == j) {
|
|
58
|
+
memory.W_recurrent_r[idx] = 1.0;
|
|
59
|
+
memory.W_recurrent_i[idx] = 0.0;
|
|
60
|
+
} else {
|
|
61
|
+
memory.W_recurrent_r[idx] = d(gen) * 0.1;
|
|
62
|
+
memory.W_recurrent_i[idx] = d(gen) * 0.1;
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
for(size_t i=0; i<state.size; ++i) {
|
|
68
|
+
memory.W_gate_r[i] = d(gen) + 1.0;
|
|
69
|
+
memory.W_gate_i[i] = d(gen) * 0.05;
|
|
70
|
+
|
|
71
|
+
memory.W_input_r[i] = 1.0 + d(gen)*0.1;
|
|
72
|
+
memory.W_input_i[i] = d(gen)*0.1;
|
|
73
|
+
|
|
74
|
+
memory.W_feedback_r[i] = 0.3 + d(gen)*0.1; memory.W_feedback_i[i] = d(gen)*0.1;
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
void NawaSphere::set_input(const std::vector<Scalar>& r, const std::vector<Scalar>& i) {
|
|
79
|
+
if(r.size()==state.size) { buf_input_r=r; buf_input_i=i; }
|
|
80
|
+
}
|
|
81
|
+
void NawaSphere::set_feedback(const std::vector<Scalar>& r, const std::vector<Scalar>& i) {
|
|
82
|
+
if(r.size()==state.size) { buf_feedback_r=r; buf_feedback_i=i; }
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
void NawaSphere::step_integrate() {
|
|
86
|
+
size_t N = state.size;
|
|
87
|
+
std::vector<Scalar> next_r(N), next_i(N);
|
|
88
|
+
|
|
89
|
+
for(size_t i=0; i<N; ++i) {
|
|
90
|
+
Scalar sum_r = 0.0;
|
|
91
|
+
Scalar sum_i = 0.0;
|
|
92
|
+
for(size_t j=0; j<N; ++j) {
|
|
93
|
+
size_t idx = i * N + j;
|
|
94
|
+
Scalar w_r = memory.W_recurrent_r[idx];
|
|
95
|
+
Scalar w_i = memory.W_recurrent_i[idx];
|
|
96
|
+
Scalar s_r = state.q_r[j];
|
|
97
|
+
Scalar s_i = state.q_i[j];
|
|
98
|
+
|
|
99
|
+
sum_r += (w_r * s_r - w_i * s_i);
|
|
100
|
+
sum_i += (w_r * s_i + w_i * s_r);
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
Scalar in_r = buf_input_r[i]*memory.W_input_r[i] - buf_input_i[i]*memory.W_input_i[i];
|
|
104
|
+
Scalar in_i = buf_input_r[i]*memory.W_input_i[i] + buf_input_i[i]*memory.W_input_r[i];
|
|
105
|
+
|
|
106
|
+
Scalar fb_r = buf_feedback_r[i]*memory.W_feedback_r[i] - buf_feedback_i[i]*memory.W_feedback_i[i];
|
|
107
|
+
Scalar fb_i = buf_feedback_r[i]*memory.W_feedback_i[i] + buf_feedback_i[i]*memory.W_feedback_r[i];
|
|
108
|
+
|
|
109
|
+
Scalar total_r = sum_r + in_r + fb_r;
|
|
110
|
+
Scalar total_i = sum_i + in_i + fb_i;
|
|
111
|
+
|
|
112
|
+
Scalar gate = sigmoid((memory.W_gate_r[i]*total_r - memory.W_gate_i[i]*total_i));
|
|
113
|
+
|
|
114
|
+
next_r[i] = (1.0 - gate) * state.q_r[i] + gate * total_r;
|
|
115
|
+
next_i[i] = (1.0 - gate) * state.q_i[i] + gate * total_i;
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
state.q_r = next_r;
|
|
119
|
+
state.q_i = next_i;
|
|
120
|
+
|
|
121
|
+
normalize();
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
void NawaSphere::normalize() {
|
|
125
|
+
for(size_t i=0; i<state.size; ++i) {
|
|
126
|
+
Scalar mag = std::sqrt(state.q_r[i]*state.q_r[i] + state.q_i[i]*state.q_i[i]);
|
|
127
|
+
if (mag > 1e-9) {
|
|
128
|
+
Scalar new_mag = std::tanh(mag);
|
|
129
|
+
Scalar scale = new_mag / mag;
|
|
130
|
+
state.q_r[i] *= scale;
|
|
131
|
+
state.q_i[i] *= scale;
|
|
132
|
+
} else {
|
|
133
|
+
state.q_r[i] = 0.0; state.q_i[i] = 0.0;
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
NawaSystem::NawaSystem(size_t dimension) {
|
|
139
|
+
base_dim = dimension;
|
|
140
|
+
global_tick = 0;
|
|
141
|
+
int ratio = 1;
|
|
142
|
+
for(int i=0; i<NAWA_LAYERS; ++i) {
|
|
143
|
+
spheres.push_back(std::make_unique<NawaSphere>(dimension, i, ratio));
|
|
144
|
+
spheres.back()->init_weights(42+i);
|
|
145
|
+
ratio *= 2;
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
void NawaSystem::step(const std::vector<Scalar>& ir, const std::vector<Scalar>& ii) {
|
|
150
|
+
global_tick++;
|
|
151
|
+
spheres[0]->set_input(ir, ii);
|
|
152
|
+
for(int i=0; i<NAWA_LAYERS; ++i) {
|
|
153
|
+
if(global_tick % spheres[i]->time_ratio == 0) {
|
|
154
|
+
if(i > 0) spheres[i]->set_input(spheres[i-1]->state.q_r, spheres[i-1]->state.q_i);
|
|
155
|
+
if(i < NAWA_LAYERS-1) spheres[i]->set_feedback(spheres[i+1]->state.q_r, spheres[i+1]->state.q_i);
|
|
156
|
+
spheres[i]->step_integrate();
|
|
157
|
+
}
|
|
158
|
+
}
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
void NawaSystem::save(const std::string& filename) {
|
|
162
|
+
std::ofstream out(filename, std::ios::binary);
|
|
163
|
+
if(out) for(auto& s : spheres) s->memory.save(out);
|
|
164
|
+
}
|
|
165
|
+
void NawaSystem::load(const std::string& filename) {
|
|
166
|
+
std::ifstream in(filename, std::ios::binary);
|
|
167
|
+
if(in) for(auto& s : spheres) s->memory.load(in);
|
|
168
|
+
}
|
|
169
|
+
std::vector<Scalar> NawaSystem::get_layer_r(int idx) {
|
|
170
|
+
if(idx>=0 && idx<NAWA_LAYERS) return spheres[idx]->state.q_r;
|
|
171
|
+
return {};
|
|
172
|
+
}
|
|
173
|
+
std::vector<Scalar> NawaSystem::get_layer_i(int idx) {
|
|
174
|
+
if(idx>=0 && idx<NAWA_LAYERS) return spheres[idx]->state.q_i;
|
|
175
|
+
return {};
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
void NawaSystem::learn(int idx,
|
|
179
|
+
const std::vector<Scalar>& in_r, const std::vector<Scalar>& in_i,
|
|
180
|
+
const std::vector<Scalar>& error_r, const std::vector<Scalar>& error_i) {
|
|
181
|
+
if (idx < 0 || idx >= NAWA_LAYERS) return;
|
|
182
|
+
|
|
183
|
+
NawaSphere* s = spheres[idx].get();
|
|
184
|
+
size_t N = s->state.size;
|
|
185
|
+
Scalar lr = 0.05;
|
|
186
|
+
|
|
187
|
+
for(size_t i=0; i<N; ++i) {
|
|
188
|
+
Scalar err_real = error_r[i];
|
|
189
|
+
Scalar err_imag = error_i[i];
|
|
190
|
+
|
|
191
|
+
for(size_t j=0; j<N; ++j) {
|
|
192
|
+
size_t widx = i * N + j;
|
|
193
|
+
Scalar inp_real = in_r[j];
|
|
194
|
+
Scalar inp_imag = in_i[j];
|
|
195
|
+
|
|
196
|
+
Scalar grad_r = err_real * inp_real + err_imag * inp_imag;
|
|
197
|
+
Scalar grad_i = err_imag * inp_real - err_real * inp_imag;
|
|
198
|
+
|
|
199
|
+
s->memory.W_recurrent_r[widx] += lr * grad_r;
|
|
200
|
+
s->memory.W_recurrent_i[widx] += lr * grad_i;
|
|
201
|
+
|
|
202
|
+
s->memory.W_recurrent_r[widx] *= 0.9999;
|
|
203
|
+
s->memory.W_recurrent_i[widx] *= 0.9999;
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
Scalar grad_in_r = err_real * in_r[i] + err_imag * in_i[i];
|
|
207
|
+
Scalar grad_in_i = err_imag * in_r[i] - err_real * in_i[i];
|
|
208
|
+
|
|
209
|
+
s->memory.W_input_r[i] += lr * grad_in_r;
|
|
210
|
+
s->memory.W_input_i[i] += lr * grad_in_i;
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
#pragma once
|
|
2
|
+
#include <vector>
|
|
3
|
+
#include <cmath>
|
|
4
|
+
#include <complex>
|
|
5
|
+
#include <algorithm>
|
|
6
|
+
#include <iostream>
|
|
7
|
+
#include <fstream>
|
|
8
|
+
#include <random>
|
|
9
|
+
#include <memory>
|
|
10
|
+
|
|
11
|
+
#define NAWA_LAYERS 9
|
|
12
|
+
|
|
13
|
+
using Scalar = double;
|
|
14
|
+
|
|
15
|
+
struct NawaState {
|
|
16
|
+
size_t size = 0;
|
|
17
|
+
std::vector<Scalar> q_r;
|
|
18
|
+
std::vector<Scalar> q_i;
|
|
19
|
+
|
|
20
|
+
void resize(size_t n) {
|
|
21
|
+
size = n;
|
|
22
|
+
q_r.resize(n, 0.0); q_i.resize(n, 0.0);
|
|
23
|
+
}
|
|
24
|
+
};
|
|
25
|
+
|
|
26
|
+
struct NawaMemory {
|
|
27
|
+
size_t dim = 0;
|
|
28
|
+
|
|
29
|
+
std::vector<Scalar> W_recurrent_r;
|
|
30
|
+
std::vector<Scalar> W_recurrent_i;
|
|
31
|
+
|
|
32
|
+
std::vector<Scalar> W_input_r, W_input_i;
|
|
33
|
+
std::vector<Scalar> W_gate_r, W_gate_i;
|
|
34
|
+
std::vector<Scalar> W_feedback_r, W_feedback_i;
|
|
35
|
+
|
|
36
|
+
void init(size_t dimension);
|
|
37
|
+
void save(std::ostream& out);
|
|
38
|
+
void load(std::istream& in);
|
|
39
|
+
};
|
|
40
|
+
|
|
41
|
+
class NawaSphere {
|
|
42
|
+
public:
|
|
43
|
+
int layer_id;
|
|
44
|
+
int time_ratio;
|
|
45
|
+
|
|
46
|
+
NawaState state;
|
|
47
|
+
NawaMemory memory;
|
|
48
|
+
|
|
49
|
+
std::vector<Scalar> buf_input_r, buf_input_i;
|
|
50
|
+
std::vector<Scalar> buf_feedback_r, buf_feedback_i;
|
|
51
|
+
|
|
52
|
+
NawaSphere(size_t dimension, int id, int ratio);
|
|
53
|
+
|
|
54
|
+
void init_weights(int seed);
|
|
55
|
+
void step_integrate();
|
|
56
|
+
void set_input(const std::vector<Scalar>& r, const std::vector<Scalar>& i);
|
|
57
|
+
void set_feedback(const std::vector<Scalar>& r, const std::vector<Scalar>& i);
|
|
58
|
+
void normalize();
|
|
59
|
+
};
|
|
60
|
+
|
|
61
|
+
class NawaSystem {
|
|
62
|
+
private:
|
|
63
|
+
std::vector<std::unique_ptr<NawaSphere>> spheres;
|
|
64
|
+
size_t global_tick = 0;
|
|
65
|
+
size_t base_dim = 0;
|
|
66
|
+
|
|
67
|
+
public:
|
|
68
|
+
NawaSystem(size_t dimension);
|
|
69
|
+
void step(const std::vector<Scalar>& input_r, const std::vector<Scalar>& input_i);
|
|
70
|
+
void save(const std::string& filename);
|
|
71
|
+
void load(const std::string& filename);
|
|
72
|
+
std::vector<Scalar> get_layer_r(int idx);
|
|
73
|
+
std::vector<Scalar> get_layer_i(int idx);
|
|
74
|
+
void learn(int layer_idx,
|
|
75
|
+
const std::vector<Scalar>& in_r, const std::vector<Scalar>& in_i,
|
|
76
|
+
const std::vector<Scalar>& out_r, const std::vector<Scalar>& out_i);
|
|
77
|
+
};
|
|
78
|
+
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
import numpy as np
|
|
2
|
+
import pickle
|
|
3
|
+
import os
|
|
4
|
+
|
|
5
|
+
class ResonantTokenizer:
|
|
6
|
+
def __init__(self, dim=128):
|
|
7
|
+
self.dim = dim
|
|
8
|
+
self.vocab_size = 256
|
|
9
|
+
|
|
10
|
+
rng = np.random.default_rng(42)
|
|
11
|
+
|
|
12
|
+
r = rng.normal(0, 1, (self.vocab_size, dim))
|
|
13
|
+
i = rng.normal(0, 1, (self.vocab_size, dim))
|
|
14
|
+
|
|
15
|
+
mag = np.sqrt(r**2 + i**2) + 1e-9
|
|
16
|
+
self.embeddings_r = r / mag
|
|
17
|
+
self.embeddings_i = i / mag
|
|
18
|
+
|
|
19
|
+
def encode(self, text):
|
|
20
|
+
if isinstance(text, str):
|
|
21
|
+
bytes_data = text.encode('utf-8')
|
|
22
|
+
else:
|
|
23
|
+
bytes_data = text
|
|
24
|
+
|
|
25
|
+
indices = np.array(list(bytes_data), dtype=int)
|
|
26
|
+
|
|
27
|
+
seq_r = self.embeddings_r[indices]
|
|
28
|
+
seq_i = self.embeddings_i[indices]
|
|
29
|
+
|
|
30
|
+
return seq_r, seq_i
|
|
31
|
+
|
|
32
|
+
def decode(self, vec_r, vec_i):
|
|
33
|
+
scores = (vec_r @ self.embeddings_r.T) + (vec_i @ self.embeddings_i.T)
|
|
34
|
+
best_idx = np.argmax(scores)
|
|
35
|
+
return best_idx
|
|
36
|
+
|
|
37
|
+
def decode_sequence(self, seq_r, seq_i):
|
|
38
|
+
bytes_list = []
|
|
39
|
+
for i in range(len(seq_r)):
|
|
40
|
+
idx = self.decode(seq_r[i], seq_i[i])
|
|
41
|
+
bytes_list.append(idx)
|
|
42
|
+
|
|
43
|
+
try:
|
|
44
|
+
return bytes(bytes_list).decode('utf-8', errors='replace')
|
|
45
|
+
except:
|
|
46
|
+
return "<binary>"
|
|
47
|
+
|
|
48
|
+
def save(self, filename):
|
|
49
|
+
with open(filename, 'wb') as f:
|
|
50
|
+
pickle.dump({
|
|
51
|
+
'dim': self.dim,
|
|
52
|
+
'er': self.embeddings_r,
|
|
53
|
+
'ei': self.embeddings_i
|
|
54
|
+
}, f)
|
|
55
|
+
|
|
56
|
+
def load(self, filename):
|
|
57
|
+
if not os.path.exists(filename): return
|
|
58
|
+
with open(filename, 'rb') as f:
|
|
59
|
+
data = pickle.load(f)
|
|
60
|
+
self.dim = data['dim']
|
|
61
|
+
self.embeddings_r = data['er']
|
|
62
|
+
self.embeddings_i = data['ei']
|
|
63
|
+
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
import numpy as np
|
|
2
|
+
|
|
3
|
+
def create_orthogonal_vector(dim, seed):
|
|
4
|
+
np.random.seed(seed)
|
|
5
|
+
r = np.random.randn(dim)
|
|
6
|
+
i = np.random.randn(dim)
|
|
7
|
+
mag = np.sqrt(r**2 + i**2) + 1e-9
|
|
8
|
+
return (r/mag) + 1j * (i/mag)
|
|
9
|
+
|
|
10
|
+
def cosine_similarity(v1, v2):
|
|
11
|
+
dot = np.real(np.sum(v1 * np.conj(v2)))
|
|
12
|
+
return dot
|
|
13
|
+
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: natumpy
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Nawa Causal Engine: Complex-Valued Reservoir Computing based on Harmonic Resonance.
|
|
5
|
+
Home-page: https://github.com/arclaav/natumpy
|
|
6
|
+
Author: Eternals
|
|
7
|
+
Author-email: admin@defacer.cloud-ip.cc
|
|
8
|
+
Keywords: ai,reservoir-computing,esn,complex-valued,neural-network,c++,nawa
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Intended Audience :: Science/Research
|
|
11
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
12
|
+
Classifier: Programming Language :: C++
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Operating System :: OS Independent
|
|
16
|
+
Requires-Python: >=3.8
|
|
17
|
+
License-File: LICENSE
|
|
18
|
+
Requires-Dist: numpy>=1.20.0
|
|
19
|
+
Dynamic: author
|
|
20
|
+
Dynamic: author-email
|
|
21
|
+
Dynamic: classifier
|
|
22
|
+
Dynamic: description
|
|
23
|
+
Dynamic: home-page
|
|
24
|
+
Dynamic: keywords
|
|
25
|
+
Dynamic: license-file
|
|
26
|
+
Dynamic: requires-dist
|
|
27
|
+
Dynamic: requires-python
|
|
28
|
+
Dynamic: summary
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
Natumpy is a deterministic AI engine that uses complex-valued signals (phase & amplitude)
|
|
32
|
+
to model information as interference patterns, distinct from probabilistic Transformers.
|
|
33
|
+
Powered by a high-performance C++ backend implementing the Nawa (9-Sphere) topology.
|
|
34
|
+
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
setup.py
|
|
5
|
+
natumpy/__init__.py
|
|
6
|
+
natumpy/layers.py
|
|
7
|
+
natumpy/model.py
|
|
8
|
+
natumpy/nat_bindings.cpp
|
|
9
|
+
natumpy/nat_engine.cpp
|
|
10
|
+
natumpy/nat_engine.hpp
|
|
11
|
+
natumpy/text.py
|
|
12
|
+
natumpy/utils.py
|
|
13
|
+
natumpy.egg-info/PKG-INFO
|
|
14
|
+
natumpy.egg-info/SOURCES.txt
|
|
15
|
+
natumpy.egg-info/dependency_links.txt
|
|
16
|
+
natumpy.egg-info/requires.txt
|
|
17
|
+
natumpy.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
numpy>=1.20.0
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
natumpy
|
natumpy-1.0.0/setup.cfg
ADDED
natumpy-1.0.0/setup.py
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
from setuptools import setup, Extension, find_packages
|
|
2
|
+
import sys
|
|
3
|
+
|
|
4
|
+
try:
|
|
5
|
+
import numpy
|
|
6
|
+
np_include = [numpy.get_include()]
|
|
7
|
+
except ImportError:
|
|
8
|
+
np_include = []
|
|
9
|
+
|
|
10
|
+
extra_compile_args = ['-std=c++17', '-O3', '-Wall', '-march=native', '-ffast-math']
|
|
11
|
+
|
|
12
|
+
if sys.platform == 'darwin':
|
|
13
|
+
extra_compile_args.append('-stdlib=libc++')
|
|
14
|
+
|
|
15
|
+
module = Extension(
|
|
16
|
+
'natumpy.natcore',
|
|
17
|
+
sources=[
|
|
18
|
+
'natumpy/nat_bindings.cpp',
|
|
19
|
+
'natumpy/nat_engine.cpp'
|
|
20
|
+
],
|
|
21
|
+
depends=['natumpy/nat_engine.hpp'],
|
|
22
|
+
include_dirs=np_include,
|
|
23
|
+
extra_compile_args=extra_compile_args,
|
|
24
|
+
language='c++'
|
|
25
|
+
)
|
|
26
|
+
|
|
27
|
+
setup(
|
|
28
|
+
name='natumpy',
|
|
29
|
+
version='1.0.0',
|
|
30
|
+
packages=find_packages(),
|
|
31
|
+
ext_modules=[module],
|
|
32
|
+
python_requires='>=3.8',
|
|
33
|
+
install_requires=['numpy>=1.20.0'],
|
|
34
|
+
|
|
35
|
+
description='Nawa Causal Engine: Complex-Valued Reservoir Computing based on Harmonic Resonance.',
|
|
36
|
+
long_description="""
|
|
37
|
+
Natumpy is a deterministic AI engine that uses complex-valued signals (phase & amplitude)
|
|
38
|
+
to model information as interference patterns, distinct from probabilistic Transformers.
|
|
39
|
+
Powered by a high-performance C++ backend implementing the Nawa (9-Sphere) topology.
|
|
40
|
+
""",
|
|
41
|
+
author='Eternals',
|
|
42
|
+
author_email='admin@defacer.cloud-ip.cc',
|
|
43
|
+
url='https://github.com/arclaav/natumpy',
|
|
44
|
+
|
|
45
|
+
keywords=['ai', 'reservoir-computing', 'esn', 'complex-valued', 'neural-network', 'c++', 'nawa'],
|
|
46
|
+
classifiers=[
|
|
47
|
+
'Development Status :: 4 - Beta',
|
|
48
|
+
'Intended Audience :: Science/Research',
|
|
49
|
+
'Topic :: Scientific/Engineering :: Artificial Intelligence',
|
|
50
|
+
'Programming Language :: C++',
|
|
51
|
+
'Programming Language :: Python :: 3',
|
|
52
|
+
'License :: OSI Approved :: MIT License',
|
|
53
|
+
'Operating System :: OS Independent',
|
|
54
|
+
],
|
|
55
|
+
)
|