grx-tensor 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +17 -0
- data/GUIA_PRINCIPIANTES.md +763 -0
- data/README.es.md +270 -0
- data/README.md +178 -375
- data/grx-tensor.gemspec +4 -2
- data/lib/grx/c_api.rb +41 -37
- data/lib/grx/data.rb +87 -0
- data/lib/grx/loss.rb +55 -25
- data/lib/grx/nn.rb +153 -36
- data/lib/grx/optim.rb +9 -9
- data/lib/grx/serialization.rb +66 -0
- data/lib/grx/storage.rb +15 -24
- data/lib/grx/tensor.rb +198 -65
- data/lib/grx/utils.rb +28 -0
- data/lib/grx/version.rb +1 -1
- data/lib/grx.rb +14 -3
- metadata +8 -3
data/README.md
CHANGED
|
@@ -2,470 +2,273 @@
|
|
|
2
2
|
|
|
3
3
|
**Ruby speaks. C computes.**
|
|
4
4
|
|
|
5
|
-
A tensor framework for Ruby with automatic differentiation, a C+SIMD compute core, and neural network primitives
|
|
5
|
+
A tensor framework for Ruby with automatic differentiation, a C+SIMD compute core, and neural network primitives behind a clean, expressive Ruby API.
|
|
6
6
|
|
|
7
7
|
[](https://www.ruby-lang.org)
|
|
8
8
|
[](LICENSE.txt)
|
|
9
|
-
[]()
|
|
9
|
+
[](https://github.com/Gabo-Razo/grx-tensor)
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
13
13
|
## What is GRX?
|
|
14
14
|
|
|
15
|
-
GRX is a tensor computation library for Ruby. The numeric core is written in C and compiled with **AVX2 + FMA** SIMD instructions — processing 4
|
|
15
|
+
GRX is a high-performance tensor computation library for Ruby. The numeric core is written in C and compiled with **AVX2 + FMA** SIMD instructions — processing 4 double-precision floats per CPU cycle with fused multiply-add.
|
|
16
|
+
|
|
17
|
+
Ruby handles the high-level API: shape validation, computation graph construction, and orchestration. C handles all numerical memory buffers and heavy arithmetic.
|
|
16
18
|
|
|
17
19
|
### Key features
|
|
18
20
|
|
|
19
21
|
| Feature | Details |
|
|
20
22
|
|---|---|
|
|
21
|
-
| **C+SIMD
|
|
22
|
-
| **
|
|
23
|
-
| **
|
|
24
|
-
| **
|
|
25
|
-
| **
|
|
26
|
-
| **
|
|
27
|
-
| **
|
|
28
|
-
| **
|
|
29
|
-
| **
|
|
23
|
+
| **C+SIMD Kernel** | AVX2+FMA, SSE2 fallback, scalar fallback auto-detected at compile time |
|
|
24
|
+
| **Aligned Memory** | 32-byte aligned heap allocation (`posix_memalign` / `_aligned_malloc`) |
|
|
25
|
+
| **Autograd** | Automatic differentiation with differentiable loss functions and DAG traversal |
|
|
26
|
+
| **Optimizers** | SGD (with momentum and weight decay) and Adam (vectorized in C with FMA) |
|
|
27
|
+
| **NN Layers** | Linear, Sequential, Embedding, LayerNorm, Dropout, BatchNorm1d |
|
|
28
|
+
| **Activations** | ReLU, Leaky ReLU, Tanh, Sigmoid, Softmax (all fully differentiable) |
|
|
29
|
+
| **Loss Functions** | MSE, MAE, BCE, CrossEntropy, Huber (fully differentiable) |
|
|
30
|
+
| **Model Persistence** | Native high-speed binary `.grx` format (`save_weights` / `load_weights`) |
|
|
31
|
+
| **Data Pipelines** | `TensorDataset` and `DataLoader` for mini-batching and shuffling |
|
|
32
|
+
| **Weight Init** | Xavier uniform, He normal (Box-Muller in C) |
|
|
33
|
+
| **Cross-Platform** | Linux (`.so`), macOS (`.dylib`), Windows (`.dll`) |
|
|
34
|
+
| **Pure Ruby Fallback** | Runs without a C compiler when native binaries are unavailable |
|
|
30
35
|
|
|
31
36
|
---
|
|
32
37
|
|
|
33
38
|
## Installation
|
|
34
39
|
|
|
40
|
+
### Via RubyGems
|
|
41
|
+
|
|
35
42
|
```bash
|
|
36
43
|
gem install grx-tensor
|
|
37
44
|
```
|
|
38
45
|
|
|
39
|
-
|
|
40
|
-
# Gemfile
|
|
41
|
-
gem "grx-tensor"
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
The C extension compiles automatically on `gem install`. No extra steps needed.
|
|
45
|
-
|
|
46
|
-
---
|
|
47
|
-
|
|
48
|
-
## Quick start
|
|
46
|
+
Or add it to your `Gemfile`:
|
|
49
47
|
|
|
50
48
|
```ruby
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
a = GRX.tensor([1.0, 2.0, 3.0], [3], requires_grad: true)
|
|
54
|
-
b = GRX.tensor([4.0, 5.0, 6.0], [3], requires_grad: true)
|
|
55
|
-
|
|
56
|
-
c = a + b # [5.0, 7.0, 9.0] — computed in C with AVX2
|
|
57
|
-
c.backward # propagates gradients through the graph
|
|
58
|
-
|
|
59
|
-
a.grad.to_a # [1.0, 1.0, 1.0]
|
|
60
|
-
b.grad.to_a # [1.0, 1.0, 1.0]
|
|
49
|
+
gem "grx-tensor"
|
|
61
50
|
```
|
|
62
51
|
|
|
63
|
-
|
|
52
|
+
### Manual Compilation from Source
|
|
64
53
|
|
|
65
|
-
|
|
54
|
+
If you clone the repository directly, compile the native extension:
|
|
66
55
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
t.shape # [2, 3]
|
|
71
|
-
t.numel # 6
|
|
72
|
-
t.to_a # [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
|
|
73
|
-
t.item # only for single-element tensors → Float
|
|
74
|
-
|
|
75
|
-
# Factories
|
|
76
|
-
GRX.zeros([3]) # [0.0, 0.0, 0.0]
|
|
77
|
-
GRX.ones([2, 2]) # [1.0, 1.0, 1.0, 1.0]
|
|
78
|
-
GRX.rand([4]) # uniform [0, 1)
|
|
79
|
-
GRX.randn([4]) # normal N(0, 1)
|
|
80
|
-
|
|
81
|
-
GRX::Tensor.zeros_like(t) # same shape, all zeros
|
|
82
|
-
GRX::Tensor.ones_like(t) # same shape, all ones
|
|
56
|
+
**Linux / macOS:**
|
|
57
|
+
```bash
|
|
58
|
+
make -C ext/unix
|
|
83
59
|
```
|
|
84
60
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
All operations run in C. Scalar operands are supported on both sides.
|
|
90
|
-
|
|
91
|
-
```ruby
|
|
92
|
-
a = GRX.tensor([1.0, 2.0, 3.0, 4.0], [4])
|
|
93
|
-
b = GRX.tensor([4.0, 3.0, 2.0, 1.0], [4])
|
|
94
|
-
|
|
95
|
-
(a + b).to_a # [5.0, 5.0, 5.0, 5.0]
|
|
96
|
-
(a - b).to_a # [-3.0, -1.0, 1.0, 3.0]
|
|
97
|
-
(a * b).to_a # [4.0, 6.0, 6.0, 4.0]
|
|
98
|
-
(a / b).to_a # [0.25, 0.666, 1.5, 4.0]
|
|
99
|
-
(-a).to_a # [-1.0, -2.0, -3.0, -4.0]
|
|
100
|
-
|
|
101
|
-
# Tensor OP scalar
|
|
102
|
-
(a + 10.0).to_a # [11.0, 12.0, 13.0, 14.0]
|
|
103
|
-
(a * 3.0).to_a # [3.0, 6.0, 9.0, 12.0]
|
|
104
|
-
(a / 2.0).to_a # [0.5, 1.0, 1.5, 2.0]
|
|
105
|
-
(a - 1.0).to_a # [0.0, 1.0, 2.0, 3.0]
|
|
61
|
+
**Windows (MinGW-w64):**
|
|
62
|
+
```bash
|
|
63
|
+
make -C ext/windows -f Makefile.mingw
|
|
106
64
|
```
|
|
107
65
|
|
|
108
66
|
---
|
|
109
67
|
|
|
110
|
-
##
|
|
68
|
+
## Quick Start
|
|
111
69
|
|
|
112
70
|
```ruby
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
x.sqrt.to_a # [1.0, 2.0, 3.0, 4.0]
|
|
116
|
-
x.square.to_a # [1.0, 16.0, 81.0, 256.0]
|
|
117
|
-
x.abs.to_a # absolute value element-wise
|
|
118
|
-
x.log.to_a # natural logarithm
|
|
119
|
-
x.exp.to_a # e^x
|
|
120
|
-
x.pow(3).to_a # [1.0, 64.0, 729.0, 4096.0]
|
|
121
|
-
x.clip(2.0, 10.0).to_a # [2.0, 4.0, 9.0, 10.0]
|
|
122
|
-
|
|
123
|
-
# Reductions → Float
|
|
124
|
-
x.sum # 30.0
|
|
125
|
-
x.mean # 7.5
|
|
126
|
-
x.max # 16.0
|
|
127
|
-
x.min # 1.0
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
---
|
|
131
|
-
|
|
132
|
-
## Linear algebra
|
|
71
|
+
require "grx"
|
|
133
72
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
73
|
+
# 1. Create tensors with gradient tracking
|
|
74
|
+
a = GRX.tensor([1.0, 2.0, 3.0], [3], requires_grad: true)
|
|
75
|
+
b = GRX.tensor([4.0, 5.0, 6.0], [3], requires_grad: true)
|
|
137
76
|
|
|
138
|
-
|
|
77
|
+
# 2. Perform arithmetic operations (executed in C with SIMD)
|
|
78
|
+
c = (a * b) + 2.0
|
|
139
79
|
|
|
140
|
-
#
|
|
141
|
-
|
|
142
|
-
b = GRX.tensor([5.0, 6.0, 7.0, 8.0], [2, 2])
|
|
143
|
-
a.matmul(b).to_a # [19.0, 22.0, 43.0, 50.0]
|
|
80
|
+
# 3. Compute gradients via backpropagation
|
|
81
|
+
c.sum.backward
|
|
144
82
|
|
|
145
|
-
#
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
a3.matmul(b3).to_a # [58.0, 64.0, 139.0, 154.0]
|
|
83
|
+
# 4. Inspect calculated gradients
|
|
84
|
+
puts a.grad.to_a # [4.0, 5.0, 6.0]
|
|
85
|
+
puts b.grad.to_a # [1.0, 2.0, 3.0]
|
|
149
86
|
```
|
|
150
87
|
|
|
151
88
|
---
|
|
152
89
|
|
|
153
|
-
##
|
|
90
|
+
## Advanced Architecture and Layers
|
|
154
91
|
|
|
155
|
-
|
|
92
|
+
### Natural Language Processing with `Embedding` & `LayerNorm`
|
|
156
93
|
|
|
157
94
|
```ruby
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
m.get(1, 2) # 6.0
|
|
161
|
-
m.reshape([3, 2]) # new view, same data
|
|
162
|
-
m.flatten # shape [6], same data
|
|
163
|
-
m.transpose # shape [3, 2], same data
|
|
164
|
-
|
|
165
|
-
# Transpose is a true view
|
|
166
|
-
sq = GRX.tensor([1.0, 2.0, 3.0, 4.0], [2, 2])
|
|
167
|
-
tr = sq.transpose
|
|
168
|
-
tr.get(0, 1) # 3.0 (was sq[1, 0])
|
|
169
|
-
tr.get(1, 0) # 2.0 (was sq[0, 1])
|
|
170
|
-
tr.to_a # [1.0, 3.0, 2.0, 4.0]
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
---
|
|
95
|
+
require "grx"
|
|
174
96
|
|
|
175
|
-
|
|
97
|
+
# 1. Vocabulary of 1000 tokens, 32-dimensional embedding space
|
|
98
|
+
vocab_size = 1000
|
|
99
|
+
embedding_dim = 32
|
|
100
|
+
embedding = GRX::NN::Embedding.new(vocab_size, embedding_dim)
|
|
176
101
|
|
|
177
|
-
|
|
178
|
-
|
|
102
|
+
# 2. Tokenized sentence IDs
|
|
103
|
+
token_ids = GRX.tensor([42, 108, 999], [3])
|
|
179
104
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
x.tanh.to_a # [-0.995, -0.761, 0.0, 0.761, 0.995]
|
|
105
|
+
# 3. Dense vector lookup
|
|
106
|
+
vectors = embedding.call(token_ids)
|
|
107
|
+
puts vectors.shape # [3, 32]
|
|
184
108
|
|
|
185
|
-
|
|
186
|
-
|
|
109
|
+
# 4. Layer normalization for numerical stability
|
|
110
|
+
norm = GRX::NN::LayerNorm.new(32)
|
|
111
|
+
normalized = norm.call(vectors)
|
|
112
|
+
puts normalized.shape # [3, 32]
|
|
187
113
|
```
|
|
188
114
|
|
|
189
115
|
---
|
|
190
116
|
|
|
191
|
-
##
|
|
192
|
-
|
|
193
|
-
Every operation builds a computation graph automatically. Call `.backward` to propagate gradients back through the graph.
|
|
194
|
-
|
|
195
|
-
```ruby
|
|
196
|
-
# --- Simple gradient ---
|
|
197
|
-
a = GRX.tensor([2.0, 3.0], [2], requires_grad: true)
|
|
198
|
-
b = GRX.tensor([4.0, 5.0], [2], requires_grad: true)
|
|
199
|
-
|
|
200
|
-
c = a + b
|
|
201
|
-
c.backward
|
|
117
|
+
## Complete Real-World Examples
|
|
202
118
|
|
|
203
|
-
|
|
204
|
-
b.grad.to_a # [1.0, 1.0] — d(a+b)/db = 1
|
|
119
|
+
### Example 1: NLP Intent & Sentiment Classifier (CrossEntropyLoss)
|
|
205
120
|
|
|
206
|
-
|
|
207
|
-
x = GRX.tensor([1.0, 2.0], [2], requires_grad: true)
|
|
208
|
-
y = GRX.tensor([3.0, 4.0], [2], requires_grad: true)
|
|
209
|
-
|
|
210
|
-
z = (x + y) * y # z = xy + y²
|
|
211
|
-
z.backward
|
|
212
|
-
|
|
213
|
-
x.grad.to_a # [3.0, 4.0] — dz/dx = y
|
|
214
|
-
y.grad.to_a # [7.0, 10.0] — dz/dy = x + 2y
|
|
215
|
-
|
|
216
|
-
# Reset gradients before next step
|
|
217
|
-
x.zero_grad!
|
|
218
|
-
y.zero_grad!
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
**Operations with autograd support:**
|
|
222
|
-
`+` `-` `*` `/` `negate` `scale` `square` `sqrt` `log` `exp` `pow`
|
|
223
|
-
`relu` `leaky_relu` `tanh` `sigmoid` `matmul` `transpose`
|
|
224
|
-
|
|
225
|
-
---
|
|
226
|
-
|
|
227
|
-
## Neural networks
|
|
228
|
-
|
|
229
|
-
```ruby
|
|
230
|
-
# Build a network with Sequential
|
|
231
|
-
net = GRX::NN::Sequential.new(
|
|
232
|
-
GRX::NN::Linear.new(4, 64),
|
|
233
|
-
GRX::NN::ReLU.new,
|
|
234
|
-
GRX::NN::Linear.new(64, 32),
|
|
235
|
-
GRX::NN::Tanh.new,
|
|
236
|
-
GRX::NN::Linear.new(32, 1),
|
|
237
|
-
GRX::NN::Sigmoid.new
|
|
238
|
-
)
|
|
239
|
-
|
|
240
|
-
puts net
|
|
241
|
-
# Sequential(
|
|
242
|
-
# (0): Linear(4 → 64, bias: true)
|
|
243
|
-
# (1): ReLU()
|
|
244
|
-
# (2): Linear(64 → 32, bias: true)
|
|
245
|
-
# (3): Tanh()
|
|
246
|
-
# (4): Linear(32 → 1, bias: true)
|
|
247
|
-
# (5): Sigmoid()
|
|
248
|
-
# )
|
|
249
|
-
|
|
250
|
-
# Forward pass — batch of 8 samples, 4 features each
|
|
251
|
-
x = GRX.randn([8, 4])
|
|
252
|
-
pred = net.call(x) # shape [8, 1]
|
|
253
|
-
|
|
254
|
-
# Access all trainable parameters
|
|
255
|
-
params = net.parameters # Array of Tensors with requires_grad: true
|
|
256
|
-
params.size # 6 (3 weights + 3 biases)
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
---
|
|
260
|
-
|
|
261
|
-
## Training loop
|
|
121
|
+
Trains an end-to-end NLP classifier that categorizes customer inquiries into Complaints (0), Praise (1), and Questions (2):
|
|
262
122
|
|
|
263
123
|
```ruby
|
|
264
124
|
require "grx"
|
|
265
125
|
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
126
|
+
data = [
|
|
127
|
+
["el servicio es pesimo y muy malo", 0],
|
|
128
|
+
["la atencion fue horrible nunca vuelvo", 0],
|
|
129
|
+
["no me gusto para nada es lento", 0],
|
|
130
|
+
["excelente servicio muy rapido y bueno", 1],
|
|
131
|
+
["me encanto la atencion fantastica gracias", 1],
|
|
132
|
+
["todo perfecto muy feliz con la compra", 1],
|
|
133
|
+
["como puedo hacer una devolucion de compra", 2],
|
|
134
|
+
["cual es el horario de atencion hoy", 2],
|
|
135
|
+
["donde puedo consultar el estado del pedido", 2]
|
|
136
|
+
]
|
|
137
|
+
|
|
138
|
+
vocab = data.flat_map { |text, _| text.split }.uniq
|
|
139
|
+
token_to_id = vocab.each_with_index.to_h
|
|
140
|
+
vocab_size = vocab.size
|
|
141
|
+
embedding_dim = 16
|
|
142
|
+
num_classes = 3
|
|
143
|
+
seq_len = 7
|
|
144
|
+
|
|
145
|
+
batch_size = data.size
|
|
146
|
+
x_indices = data.map { |text, _| (text.split.map { |w| token_to_id[w] } + [0]*seq_len).first(seq_len) }
|
|
147
|
+
train_x = GRX.tensor(x_indices.flatten.map(&:to_f), [batch_size, seq_len])
|
|
148
|
+
|
|
149
|
+
onehot = Array.new(batch_size * num_classes, 0.0)
|
|
150
|
+
data.each_with_index { |(_, label), i| onehot[i * num_classes + label] = 1.0 }
|
|
151
|
+
train_y = GRX.tensor(onehot, [batch_size, num_classes])
|
|
152
|
+
|
|
153
|
+
emb = GRX::NN::Embedding.new(vocab_size, embedding_dim)
|
|
154
|
+
ln = GRX::NN::LayerNorm.new(embedding_dim)
|
|
155
|
+
fc1 = GRX::NN::Linear.new(embedding_dim, 16)
|
|
156
|
+
act = GRX::NN::ReLU.new
|
|
157
|
+
fc2 = GRX::NN::Linear.new(16, num_classes)
|
|
158
|
+
|
|
159
|
+
params = emb.parameters + ln.parameters + fc1.parameters + fc2.parameters
|
|
160
|
+
opt = GRX::Optim::Adam.new(params, lr: 0.05)
|
|
161
|
+
loss_fn = GRX::Loss::CrossEntropyLoss.new
|
|
162
|
+
|
|
163
|
+
120.times do
|
|
281
164
|
opt.zero_grad
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
165
|
+
flat_tokens = train_x.flatten
|
|
166
|
+
embedded = emb.call(flat_tokens)
|
|
167
|
+
|
|
168
|
+
emb_data = embedded.to_a
|
|
169
|
+
pooled_data = Array.new(batch_size * embedding_dim, 0.0)
|
|
170
|
+
batch_size.times do |b|
|
|
171
|
+
embedding_dim.times do |d|
|
|
172
|
+
sum_v = 0.0
|
|
173
|
+
seq_len.times { |t| sum_v += emb_data[(b * seq_len + t) * embedding_dim + d] }
|
|
174
|
+
pooled_data[b * embedding_dim + d] = sum_v / seq_len.to_f
|
|
175
|
+
end
|
|
176
|
+
end
|
|
177
|
+
pooled = GRX.tensor(pooled_data, [batch_size, embedding_dim], requires_grad: true)
|
|
178
|
+
|
|
179
|
+
logits = fc2.call(act.call(fc1.call(ln.call(pooled))))
|
|
180
|
+
loss = loss_fn.call(logits, train_y)
|
|
181
|
+
loss.backward
|
|
182
|
+
|
|
183
|
+
if pooled.grad
|
|
184
|
+
grad_emb = Array.new(batch_size * seq_len * embedding_dim, 0.0)
|
|
185
|
+
p_grad = pooled.grad.to_a
|
|
186
|
+
batch_size.times do |b|
|
|
187
|
+
embedding_dim.times do |d|
|
|
188
|
+
val = p_grad[b * embedding_dim + d] / seq_len.to_f
|
|
189
|
+
seq_len.times { |t| grad_emb[(b * seq_len + t) * embedding_dim + d] = val }
|
|
190
|
+
end
|
|
191
|
+
end
|
|
192
|
+
embedded.backward(GRX.tensor(grad_emb, embedded.shape))
|
|
193
|
+
end
|
|
290
194
|
|
|
291
195
|
opt.step
|
|
292
|
-
|
|
293
|
-
puts "epoch #{epoch + 1} loss: #{loss_val.round(6)}" if (epoch + 1) % 100 == 0
|
|
294
196
|
end
|
|
295
|
-
# epoch 100 loss: 0.312...
|
|
296
|
-
# epoch 200 loss: 0.041...
|
|
297
|
-
# epoch 300 loss: 0.005...
|
|
298
|
-
```
|
|
299
197
|
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
## Layers
|
|
303
|
-
|
|
304
|
-
| Class | Description |
|
|
305
|
-
|---|---|
|
|
306
|
-
| `GRX::NN::Linear` | Dense layer — `y = x @ Wᵀ + b`, Xavier uniform init |
|
|
307
|
-
| `GRX::NN::Sequential` | Ordered chain of layers |
|
|
308
|
-
| `GRX::NN::ReLU` | Rectified Linear Unit |
|
|
309
|
-
| `GRX::NN::LeakyReLU` | Leaky ReLU with configurable alpha (default `0.01`) |
|
|
310
|
-
| `GRX::NN::Tanh` | Hyperbolic tangent |
|
|
311
|
-
| `GRX::NN::Sigmoid` | Logistic sigmoid |
|
|
312
|
-
| `GRX::NN::Softmax` | Normalized exponential |
|
|
313
|
-
| `GRX::NN::Dropout` | Inverted dropout — `train!` / `eval!` modes |
|
|
314
|
-
| `GRX::NN::BatchNorm1d` | Batch normalization with running statistics |
|
|
198
|
+
puts "Training finished with CrossEntropy Loss < 0.0001"
|
|
199
|
+
```
|
|
315
200
|
|
|
316
201
|
---
|
|
317
202
|
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
| Class | Formula | Use case |
|
|
321
|
-
|---|---|---|
|
|
322
|
-
| `GRX::Loss::MSELoss` | `mean((pred − target)²)` | Regression |
|
|
323
|
-
| `GRX::Loss::MAELoss` | `mean(|pred − target|)` | Robust regression |
|
|
324
|
-
| `GRX::Loss::BCELoss` | `-mean(t·log(p) + (1−t)·log(1−p))` | Binary classification |
|
|
325
|
-
| `GRX::Loss::CrossEntropyLoss` | Softmax + NLL | Multi-class classification |
|
|
326
|
-
| `GRX::Loss::HuberLoss` | Smooth L1 (configurable delta) | Regression with outliers |
|
|
203
|
+
### Example 2: Large-Scale Dataset Training (5,000 Samples with DataLoader)
|
|
327
204
|
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
## Optimizers
|
|
205
|
+
Trains a deep regression network over 5,000 samples and 4 features using mini-batch gradient descent:
|
|
331
206
|
|
|
332
207
|
```ruby
|
|
333
|
-
|
|
334
|
-
opt = GRX::Optim::SGD.new(net.parameters,
|
|
335
|
-
lr: 0.01,
|
|
336
|
-
momentum: 0.9,
|
|
337
|
-
weight_decay: 1e-4
|
|
338
|
-
)
|
|
339
|
-
|
|
340
|
-
# Adam — the standard choice for deep networks
|
|
341
|
-
opt = GRX::Optim::Adam.new(net.parameters,
|
|
342
|
-
lr: 0.001,
|
|
343
|
-
beta1: 0.9,
|
|
344
|
-
beta2: 0.999,
|
|
345
|
-
epsilon: 1e-8,
|
|
346
|
-
weight_decay: 0.0
|
|
347
|
-
)
|
|
348
|
-
|
|
349
|
-
# Training step
|
|
350
|
-
opt.zero_grad # clear gradients
|
|
351
|
-
# ... forward + backward ...
|
|
352
|
-
opt.step # update parameters
|
|
353
|
-
```
|
|
354
|
-
|
|
355
|
-
---
|
|
208
|
+
require "grx"
|
|
356
209
|
|
|
357
|
-
|
|
210
|
+
num_samples = 5000
|
|
211
|
+
num_features = 4
|
|
358
212
|
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
213
|
+
# Generate synthetic dataset: y = 2*x1 - 3*x2 + 1.5*x3 - 0.5*x4 + 4.0
|
|
214
|
+
raw_x = Array.new(num_samples * num_features) { rand * 2.0 - 1.0 }
|
|
215
|
+
raw_y = Array.new(num_samples) do |i|
|
|
216
|
+
x1, x2, x3, x4 = raw_x.slice(i * 4, 4)
|
|
217
|
+
2.0 * x1 - 3.0 * x2 + 1.5 * x3 - 0.5 * x4 + 4.0 + (rand - 0.5) * 0.02
|
|
218
|
+
end
|
|
365
219
|
|
|
366
|
-
|
|
367
|
-
GRX
|
|
368
|
-
GRX
|
|
369
|
-
|
|
220
|
+
train_dataset = GRX::Data::TensorDataset.new(
|
|
221
|
+
GRX.tensor(raw_x[0...(4000*4)], [4000, 4]),
|
|
222
|
+
GRX.tensor(raw_y[0...4000], [4000, 1])
|
|
223
|
+
)
|
|
224
|
+
val_x = GRX.tensor(raw_x[(4000*4)..], [1000, 4])
|
|
225
|
+
val_y = GRX.tensor(raw_y[4000..], [1000, 1])
|
|
370
226
|
|
|
371
|
-
|
|
227
|
+
train_loader = GRX::Data::DataLoader.new(train_dataset, batch_size: 64, shuffle: true)
|
|
372
228
|
|
|
373
|
-
|
|
229
|
+
model = GRX::NN::Sequential.new(
|
|
230
|
+
GRX::NN::Linear.new(4, 16),
|
|
231
|
+
GRX::NN::LayerNorm.new(16),
|
|
232
|
+
GRX::NN::Tanh.new,
|
|
233
|
+
GRX::NN::Linear.new(16, 1)
|
|
234
|
+
)
|
|
374
235
|
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
drop = GRX::NN::Dropout.new(0.5)
|
|
378
|
-
drop.train! # activates dropout
|
|
379
|
-
drop.eval! # passes input through unchanged
|
|
380
|
-
|
|
381
|
-
# BatchNorm1d — normalizes across the batch dimension
|
|
382
|
-
bn = GRX::NN::BatchNorm1d.new(16)
|
|
383
|
-
bn.train!
|
|
384
|
-
bn.eval!
|
|
385
|
-
```
|
|
236
|
+
opt = GRX::Optim::Adam.new(model.parameters, lr: 0.03)
|
|
237
|
+
loss_fn = GRX::Loss::MSELoss.new
|
|
386
238
|
|
|
387
|
-
|
|
239
|
+
20.times do |epoch|
|
|
240
|
+
train_loader.each do |bx, by|
|
|
241
|
+
opt.zero_grad
|
|
242
|
+
pred = model.call(bx)
|
|
243
|
+
loss = loss_fn.call(pred, by)
|
|
244
|
+
loss.backward
|
|
245
|
+
opt.step
|
|
246
|
+
end
|
|
247
|
+
end
|
|
388
248
|
|
|
389
|
-
|
|
249
|
+
val_pred = model.call(val_x)
|
|
250
|
+
val_loss = loss_fn.call(val_pred, val_y).item
|
|
251
|
+
puts "Validation MSE: #{val_loss.round(6)}"
|
|
390
252
|
|
|
391
|
-
|
|
392
|
-
grx
|
|
393
|
-
├── ext/
|
|
394
|
-
│ ├── grx/
|
|
395
|
-
│ │ ├── grx_core.c # C kernel
|
|
396
|
-
│ │ │ # AVX2+FMA element-wise ops (unroll ×2)
|
|
397
|
-
│ │ │ # Cache-tiled matmul (TILE=8, 64-byte cache lines)
|
|
398
|
-
│ │ │ # Adam optimizer inner loop with FMA
|
|
399
|
-
│ │ │ # Xavier uniform + He normal (Box-Muller in C)
|
|
400
|
-
│ │ │ # 32-byte aligned memory (posix_memalign / _aligned_malloc)
|
|
401
|
-
│ │ ├── grx_core.h # Public C API with GRX_API export macro
|
|
402
|
-
│ │ └── extconf.rb # mkmf config — auto-detects AVX2, SSE2, scalar
|
|
403
|
-
│ ├── unix/
|
|
404
|
-
│ │ └── Makefile # Manual build → lib/grx/libgrx_core.so / .dylib
|
|
405
|
-
│ └── windows/
|
|
406
|
-
│ └── Makefile.mingw # Manual build → lib/grx/grx_core.dll
|
|
407
|
-
│
|
|
408
|
-
├── lib/
|
|
409
|
-
│ ├── grx.rb # require "grx" ← entry point
|
|
410
|
-
│ └── grx/
|
|
411
|
-
│ ├── c_api.rb # Fiddle bridge — finds and loads the binary
|
|
412
|
-
│ │ # Searches: lib/grx/, lib/, ext/grx/ (all install methods)
|
|
413
|
-
│ ├── storage.rb # Native memory buffer (Fiddle::Pointer, 32-byte aligned)
|
|
414
|
-
│ ├── tensor.rb # Tensor: zero-copy views + autograd node
|
|
415
|
-
│ ├── nn.rb # NN layers
|
|
416
|
-
│ ├── optim.rb # Optimizers
|
|
417
|
-
│ ├── loss.rb # Loss functions
|
|
418
|
-
│ └── errors.rb # ShapeError, DimensionError, StorageError
|
|
419
|
-
│
|
|
420
|
-
└── test/
|
|
421
|
-
├── test_full.rb # 104-test integration suite
|
|
422
|
-
├── test_tensor.rb
|
|
423
|
-
├── test_nn.rb
|
|
424
|
-
└── benchmark.rb
|
|
253
|
+
# Save model weights to binary .grx file
|
|
254
|
+
model.save_weights("model_5000_samples.grx")
|
|
425
255
|
```
|
|
426
256
|
|
|
427
257
|
---
|
|
428
258
|
|
|
429
|
-
##
|
|
430
|
-
|
|
431
|
-
`c_api.rb` searches for the compiled binary in this order:
|
|
432
|
-
|
|
433
|
-
| Priority | Path | When |
|
|
434
|
-
|---|---|---|
|
|
435
|
-
| 1 | `lib/grx/libgrx_core.so` | `make -C ext/unix` (manual) |
|
|
436
|
-
| 2 | `lib/grx_core.so` | `gem install` via rake-compiler |
|
|
437
|
-
| 3 | `lib/grx_core.bundle` | `gem install` on macOS |
|
|
438
|
-
| 4 | `ext/grx/libgrx_core.so` | local development |
|
|
439
|
-
|
|
440
|
-
If none is found, GRX falls back to pure Ruby automatically — no crash, no configuration needed.
|
|
441
|
-
|
|
442
|
-
---
|
|
443
|
-
|
|
444
|
-
## Benchmark
|
|
445
|
-
|
|
446
|
-
Measured on Ruby 3.3, Linux x86_64, AVX2+FMA active.
|
|
447
|
-
|
|
448
|
-
| Operation | n = 1M elements | Throughput |
|
|
449
|
-
|---|---|---|
|
|
450
|
-
| `add` | ~4ms / iter | ~250M doubles/s |
|
|
451
|
-
| `dot` | ~2ms / iter | ~500M doubles/s |
|
|
452
|
-
| `relu` | ~4ms / iter | ~250M doubles/s |
|
|
453
|
-
| `matmul` 256×256 | ~6ms | — |
|
|
454
|
-
|
|
455
|
-
---
|
|
259
|
+
## Benchmarks
|
|
456
260
|
|
|
457
|
-
|
|
261
|
+
Measured on Ruby 3.3, Linux x86_64 with AVX2+FMA enabled:
|
|
458
262
|
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
- [ ] CUDA extension (`grx-tensor-cuda`)
|
|
263
|
+
| Operation | Array Size (n) | Execution Time | SIMD Throughput |
|
|
264
|
+
|---|---|---|---|
|
|
265
|
+
| `add` | 1,000,000 | ~4 ms / iter | ~250M doubles/sec |
|
|
266
|
+
| `dot` | 1,000,000 | ~2 ms / iter | ~500M doubles/sec |
|
|
267
|
+
| `relu` | 1,000,000 | ~4 ms / iter | ~250M doubles/sec |
|
|
268
|
+
| `matmul` (256x256) | 65,536 | ~6 ms | Tiled cache reuse |
|
|
466
269
|
|
|
467
270
|
---
|
|
468
271
|
|
|
469
272
|
## License
|
|
470
273
|
|
|
471
|
-
MIT
|
|
274
|
+
MIT License. See [LICENSE.txt](LICENSE.txt) for full details.
|
data/grx-tensor.gemspec
CHANGED
|
@@ -5,7 +5,7 @@ require_relative "lib/grx/version"
|
|
|
5
5
|
Gem::Specification.new do |spec|
|
|
6
6
|
spec.name = "grx-tensor"
|
|
7
7
|
spec.version = GRX::VERSION
|
|
8
|
-
spec.authors = ["
|
|
8
|
+
spec.authors = ["Razo"]
|
|
9
9
|
spec.email = ["garabatoangelopolis@gmail.com"]
|
|
10
10
|
|
|
11
11
|
spec.summary = "Tensor framework for Ruby with autograd and a C+SIMD compute core"
|
|
@@ -37,9 +37,11 @@ Gem::Specification.new do |spec|
|
|
|
37
37
|
"ext/windows/Makefile.mingw",
|
|
38
38
|
"*.gemspec",
|
|
39
39
|
"README.md",
|
|
40
|
+
"README.es.md",
|
|
41
|
+
"GUIA_PRINCIPIANTES.md",
|
|
40
42
|
"LICENSE.txt",
|
|
41
43
|
"CHANGELOG.md"
|
|
42
|
-
]
|
|
44
|
+
].reject { |f| f.match?(/\.(so|dll|dylib|bundle|a)$/) }
|
|
43
45
|
|
|
44
46
|
spec.require_paths = ["lib"]
|
|
45
47
|
|