aptx-neuron 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,151 @@
1
+ Metadata-Version: 2.4
2
+ Name: aptx_neuron
3
+ Version: 0.0.1
4
+ Summary: A PyTorch implementation of the APTx Neuron.
5
+ Home-page: https://github.com/mr-ravin/aptx_neuron
6
+ Author: Ravin Kumar
7
+ Author-email: mr.ravin_kumar@hotmail.com
8
+ License: MIT
9
+ Keywords: APTx Neuron,unified neuron,neuron,activation function,deep learning,pytorch,neural network,perceptronmachine learning,artificial intelligence,AI,ML,DL,torch
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Operating System :: OS Independent
13
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Intended Audience :: Science/Research
16
+ Requires-Python: >=3.7
17
+ Description-Content-Type: text/markdown
18
+ Requires-Dist: torch>=1.8.0
19
+ Dynamic: author
20
+ Dynamic: author-email
21
+ Dynamic: classifier
22
+ Dynamic: description
23
+ Dynamic: description-content-type
24
+ Dynamic: home-page
25
+ Dynamic: keywords
26
+ Dynamic: license
27
+ Dynamic: requires-dist
28
+ Dynamic: requires-python
29
+ Dynamic: summary
30
+
31
+ ## APTx Neuron
32
+ This repository offers a Python code for the PyTorch implementation of the APTx Neuron and experimentation on MNIST dataset, as introduced in the paper "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation".
33
+
34
+ **Paper Title**: APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
35
+
36
+ **Author**: [Ravin Kumar](https://mr-ravin.github.io)
37
+
38
+ **Sources**:
39
+ - [Arxiv.org](https://arxiv.org/abs/2507.14270)
40
+ - [Research Gate](https://www.researchgate.net/publication/393889376_APTx_Neuron_A_Unified_Trainable_Neuron_Architecture_Integrating_Activation_and_Computation)
41
+
42
+ #### Github Repositories:
43
+ - **APTx Neuron** (Pytorch + PyPI Package): [APTx Neuron](https://github.com/mr-ravin/aptx_neuron)
44
+ - **APTx Activation Function** (Pytorch + PyPI Package): [APTx Activation Function](https://github.com/mr-ravin/aptx_activation)
45
+ - **Experimentation Results with MNIST** (APTx Neuron): [MNIST Experimentation Code](https://github.com/mr-ravin/APTxNeuron)
46
+
47
+ #### Cite Paper as:
48
+ ```
49
+ Kumar, Ravin. "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation." arXiv preprint arXiv:2507.14270 (2025).
50
+ ```
51
+ Or,
52
+ ```
53
+ @article{kumar2025aptx,
54
+ title={APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation},
55
+ author={Kumar, Ravin},
56
+ journal={arXiv preprint arXiv:2507.14270},
57
+ year={2025}
58
+ }
59
+ ```
60
+
61
+ ---
62
+ ### APTx Neuron
63
+ <b>Abstract</b>: We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the [APTx activation function](https://arxiv.org/abs/2209.06119), thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i) + \delta$, where all parameters $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to 96.69\% test accuracy within 11 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and computational efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it.
64
+
65
+ The APTx Neuron is a novel computational unit that unifies linear transformation and non-linear activation into a single, expressive formulation. Inspired by the parametric APTx activation function, this neuron architecture removes the strict separation between computation and activation, allowing both to be learned as a cohesive entity. It is designed to enhance representational flexibility while reducing architectural redundancy.
66
+
67
+ #### Mathematical Formulation
68
+
69
+ Traditionally, a neuron computes the output as:
70
+
71
+ $y = \phi\left( \sum_{i=1}^{n} w_i x_i + b \right)$
72
+
73
+ where:
74
+ - $x_i$ are the inputs,
75
+ - $w_i$ are the weights,
76
+ - $b$ is the bias,
77
+ - and $\phi$ is an activation function such as ReLU, Swish, or Mish.
78
+
79
+
80
+ The APTx Neuron merges these components into a unified trainable expression as:
81
+
82
+ $y = \sum_{i=1}^{n} \left[ (\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i \right] + \delta$
83
+
84
+ where:
85
+ - $x_i$ is the $i$-th input feature,
86
+ - $\alpha_i$, $\beta_i$, and $\gamma_i$ are trainable parameters for each input,
87
+ - $\delta$ is a trainable scalar bias.
88
+
89
+ This equation allows the neuron to modulate each input through a learned, per-dimension non-linearity and scaling operation. The term $(\alpha_i + \tanh(\beta_i x_i))$ introduces adaptive gating, and $\gamma_i x_i$ provides multiplicative control.
90
+
91
+ ---
92
+ ## 📥 Installation
93
+ ```bash
94
+ pip install aptx_neuron
95
+ ```
96
+ or,
97
+
98
+ ```bash
99
+ pip install git+https://github.com/mr-ravin/aptx_neuron.git
100
+ ```
101
+ ----
102
+
103
+ ## Usage
104
+ <b>1</b>. APTx Neuron-based Layer with all $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
105
+
106
+ The setting `is_alpha_trainable = True` keeps $\alpha_i$ trainable. Each APTx neuron will have $(3n + 1)$ trainable parameters, where $n$ is input dimension. Note: The default value of `is_alpha_trainable` is `True`.
107
+
108
+ ```
109
+ import aptx_neuron
110
+ input_dim = 8 # assuming input vector to be of dimension 8.
111
+ output_dim = 1 # assuming output dimension equals 1.
112
+
113
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=True)
114
+ ```
115
+
116
+ <b>2</b>. APTx Neuron-based Layer with $\alpha_i=1$ (not trainable); While $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
117
+
118
+ The setting `is_alpha_trainable = False` makes $\alpha_i$ fixed (non-trainable). Each APTx neuron will then have $(2n + 1)$ trainable parameters, thus reducing memory and training time per epoch. Here, $n$ is input dimension.
119
+
120
+ ```
121
+ import aptx_neuron
122
+ input_dim = 8 # assuming input vector to be of dimension 8.
123
+ output_dim = 1 # assuming output dimension equals 1.
124
+
125
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=False) # α_i is fixed (not trainable)
126
+ ```
127
+
128
+ ----
129
+ #### Conclusion
130
+ This work introduced the APTx Neuron, a unified, fully trainable neural unit that integrates linear transformation and non-linear activation into a single expression, extending the APTx activation function. By learning per-input parameters $\alpha_i$, $\beta_i$, and $\gamma_i$ for each input $x_i$, and a shared bias term $\delta$ within a neuron, the APTx Neuron removes the need for separate activation layers and enables fine-grained input transformation. The APTx Neuron generalizes traditional neurons and activations, offering greater representational power. Our experiments show that a fully connected APTx Neuron-based feedforward neural network achieves 96.69% test accuracy on the MNIST dataset within 11 epochs using approximately 332K trainable parameters, demonstrating rapid convergence and high efficiency. This design lays the groundwork for extending APTx Neurons to CNNs and transformers, paving the way for more compact and adaptive deep learning architectures.
131
+
132
+ ----
133
+
134
+ ### 📜 Copyright License
135
+ ```python
136
+ Copyright (c) 2025 Ravin Kumar
137
+ Website: https://mr-ravin.github.io
138
+
139
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation
140
+ files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy,
141
+ modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the
142
+ Software is furnished to do so, subject to the following conditions:
143
+
144
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
145
+ Software.
146
+
147
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
148
+ WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
149
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
150
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
151
+ ```
@@ -0,0 +1,121 @@
1
+ ## APTx Neuron
2
+ This repository offers a Python code for the PyTorch implementation of the APTx Neuron and experimentation on MNIST dataset, as introduced in the paper "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation".
3
+
4
+ **Paper Title**: APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
5
+
6
+ **Author**: [Ravin Kumar](https://mr-ravin.github.io)
7
+
8
+ **Sources**:
9
+ - [Arxiv.org](https://arxiv.org/abs/2507.14270)
10
+ - [Research Gate](https://www.researchgate.net/publication/393889376_APTx_Neuron_A_Unified_Trainable_Neuron_Architecture_Integrating_Activation_and_Computation)
11
+
12
+ #### Github Repositories:
13
+ - **APTx Neuron** (Pytorch + PyPI Package): [APTx Neuron](https://github.com/mr-ravin/aptx_neuron)
14
+ - **APTx Activation Function** (Pytorch + PyPI Package): [APTx Activation Function](https://github.com/mr-ravin/aptx_activation)
15
+ - **Experimentation Results with MNIST** (APTx Neuron): [MNIST Experimentation Code](https://github.com/mr-ravin/APTxNeuron)
16
+
17
+ #### Cite Paper as:
18
+ ```
19
+ Kumar, Ravin. "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation." arXiv preprint arXiv:2507.14270 (2025).
20
+ ```
21
+ Or,
22
+ ```
23
+ @article{kumar2025aptx,
24
+ title={APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation},
25
+ author={Kumar, Ravin},
26
+ journal={arXiv preprint arXiv:2507.14270},
27
+ year={2025}
28
+ }
29
+ ```
30
+
31
+ ---
32
+ ### APTx Neuron
33
+ <b>Abstract</b>: We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the [APTx activation function](https://arxiv.org/abs/2209.06119), thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i) + \delta$, where all parameters $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to 96.69\% test accuracy within 11 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and computational efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it.
34
+
35
+ The APTx Neuron is a novel computational unit that unifies linear transformation and non-linear activation into a single, expressive formulation. Inspired by the parametric APTx activation function, this neuron architecture removes the strict separation between computation and activation, allowing both to be learned as a cohesive entity. It is designed to enhance representational flexibility while reducing architectural redundancy.
36
+
37
+ #### Mathematical Formulation
38
+
39
+ Traditionally, a neuron computes the output as:
40
+
41
+ $y = \phi\left( \sum_{i=1}^{n} w_i x_i + b \right)$
42
+
43
+ where:
44
+ - $x_i$ are the inputs,
45
+ - $w_i$ are the weights,
46
+ - $b$ is the bias,
47
+ - and $\phi$ is an activation function such as ReLU, Swish, or Mish.
48
+
49
+
50
+ The APTx Neuron merges these components into a unified trainable expression as:
51
+
52
+ $y = \sum_{i=1}^{n} \left[ (\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i \right] + \delta$
53
+
54
+ where:
55
+ - $x_i$ is the $i$-th input feature,
56
+ - $\alpha_i$, $\beta_i$, and $\gamma_i$ are trainable parameters for each input,
57
+ - $\delta$ is a trainable scalar bias.
58
+
59
+ This equation allows the neuron to modulate each input through a learned, per-dimension non-linearity and scaling operation. The term $(\alpha_i + \tanh(\beta_i x_i))$ introduces adaptive gating, and $\gamma_i x_i$ provides multiplicative control.
60
+
61
+ ---
62
+ ## 📥 Installation
63
+ ```bash
64
+ pip install aptx_neuron
65
+ ```
66
+ or,
67
+
68
+ ```bash
69
+ pip install git+https://github.com/mr-ravin/aptx_neuron.git
70
+ ```
71
+ ----
72
+
73
+ ## Usage
74
+ <b>1</b>. APTx Neuron-based Layer with all $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
75
+
76
+ The setting `is_alpha_trainable = True` keeps $\alpha_i$ trainable. Each APTx neuron will have $(3n + 1)$ trainable parameters, where $n$ is input dimension. Note: The default value of `is_alpha_trainable` is `True`.
77
+
78
+ ```
79
+ import aptx_neuron
80
+ input_dim = 8 # assuming input vector to be of dimension 8.
81
+ output_dim = 1 # assuming output dimension equals 1.
82
+
83
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=True)
84
+ ```
85
+
86
+ <b>2</b>. APTx Neuron-based Layer with $\alpha_i=1$ (not trainable); While $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
87
+
88
+ The setting `is_alpha_trainable = False` makes $\alpha_i$ fixed (non-trainable). Each APTx neuron will then have $(2n + 1)$ trainable parameters, thus reducing memory and training time per epoch. Here, $n$ is input dimension.
89
+
90
+ ```
91
+ import aptx_neuron
92
+ input_dim = 8 # assuming input vector to be of dimension 8.
93
+ output_dim = 1 # assuming output dimension equals 1.
94
+
95
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=False) # α_i is fixed (not trainable)
96
+ ```
97
+
98
+ ----
99
+ #### Conclusion
100
+ This work introduced the APTx Neuron, a unified, fully trainable neural unit that integrates linear transformation and non-linear activation into a single expression, extending the APTx activation function. By learning per-input parameters $\alpha_i$, $\beta_i$, and $\gamma_i$ for each input $x_i$, and a shared bias term $\delta$ within a neuron, the APTx Neuron removes the need for separate activation layers and enables fine-grained input transformation. The APTx Neuron generalizes traditional neurons and activations, offering greater representational power. Our experiments show that a fully connected APTx Neuron-based feedforward neural network achieves 96.69% test accuracy on the MNIST dataset within 11 epochs using approximately 332K trainable parameters, demonstrating rapid convergence and high efficiency. This design lays the groundwork for extending APTx Neurons to CNNs and transformers, paving the way for more compact and adaptive deep learning architectures.
101
+
102
+ ----
103
+
104
+ ### 📜 Copyright License
105
+ ```python
106
+ Copyright (c) 2025 Ravin Kumar
107
+ Website: https://mr-ravin.github.io
108
+
109
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation
110
+ files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy,
111
+ modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the
112
+ Software is furnished to do so, subject to the following conditions:
113
+
114
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
115
+ Software.
116
+
117
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
118
+ WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
119
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
120
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
121
+ ```
@@ -0,0 +1,35 @@
1
+ import torch
2
+ import torch.nn as nn
3
+
4
+ __version__ = '0.0.1'
5
+
6
+ # -----------------------------------
7
+ # APTx Neuron
8
+ # -----------------------------------
9
+ class aptx_neuron(nn.Module):
10
+ def __init__(self, input_dim, is_alpha_trainable=True):
11
+ super(aptx_neuron, self).__init__()
12
+ if is_alpha_trainable:
13
+ self.alpha = nn.Parameter(torch.randn(input_dim)) # To reduce trainable parameters from 3n + 1 to 2n + 1 (where n is the input dimension), replace with: self.alpha = torch.ones(input_dim) # (fix α_i = 1 to make it non-trainable)
14
+ else:
15
+ self.alpha = torch.ones(input_dim)
16
+ self.beta = nn.Parameter(torch.randn(input_dim))
17
+ self.gamma = nn.Parameter(torch.randn(input_dim))
18
+ self.delta = nn.Parameter(torch.zeros(1))
19
+
20
+ def forward(self, x): # x: [batch_size, input_dim]
21
+ nonlinear = (self.alpha + torch.tanh(self.beta * x)) * self.gamma * x
22
+ y = nonlinear.sum(dim=1, keepdim=True) + self.delta
23
+ return y
24
+
25
+ # -----------------------------------
26
+ # APTx Layer (Multiple Neurons)
27
+ # -----------------------------------
28
+ class aptx_layer(nn.Module):
29
+ def __init__(self, input_dim, output_dim, is_alpha_trainable=True):
30
+ super(aptx_layer, self).__init__()
31
+ self.neurons = nn.ModuleList([aptx_neuron(input_dim, is_alpha_trainable) for _ in range(output_dim)])
32
+
33
+ def forward(self, x): # x: [batch_size, input_dim]
34
+ outputs = [neuron(x) for neuron in self.neurons] # list of [batch_size, 1]
35
+ return torch.cat(outputs, dim=1) # [batch_size, output_dim]
@@ -0,0 +1,151 @@
1
+ Metadata-Version: 2.4
2
+ Name: aptx_neuron
3
+ Version: 0.0.1
4
+ Summary: A PyTorch implementation of the APTx Neuron.
5
+ Home-page: https://github.com/mr-ravin/aptx_neuron
6
+ Author: Ravin Kumar
7
+ Author-email: mr.ravin_kumar@hotmail.com
8
+ License: MIT
9
+ Keywords: APTx Neuron,unified neuron,neuron,activation function,deep learning,pytorch,neural network,perceptronmachine learning,artificial intelligence,AI,ML,DL,torch
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Operating System :: OS Independent
13
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Intended Audience :: Science/Research
16
+ Requires-Python: >=3.7
17
+ Description-Content-Type: text/markdown
18
+ Requires-Dist: torch>=1.8.0
19
+ Dynamic: author
20
+ Dynamic: author-email
21
+ Dynamic: classifier
22
+ Dynamic: description
23
+ Dynamic: description-content-type
24
+ Dynamic: home-page
25
+ Dynamic: keywords
26
+ Dynamic: license
27
+ Dynamic: requires-dist
28
+ Dynamic: requires-python
29
+ Dynamic: summary
30
+
31
+ ## APTx Neuron
32
+ This repository offers a Python code for the PyTorch implementation of the APTx Neuron and experimentation on MNIST dataset, as introduced in the paper "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation".
33
+
34
+ **Paper Title**: APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
35
+
36
+ **Author**: [Ravin Kumar](https://mr-ravin.github.io)
37
+
38
+ **Sources**:
39
+ - [Arxiv.org](https://arxiv.org/abs/2507.14270)
40
+ - [Research Gate](https://www.researchgate.net/publication/393889376_APTx_Neuron_A_Unified_Trainable_Neuron_Architecture_Integrating_Activation_and_Computation)
41
+
42
+ #### Github Repositories:
43
+ - **APTx Neuron** (Pytorch + PyPI Package): [APTx Neuron](https://github.com/mr-ravin/aptx_neuron)
44
+ - **APTx Activation Function** (Pytorch + PyPI Package): [APTx Activation Function](https://github.com/mr-ravin/aptx_activation)
45
+ - **Experimentation Results with MNIST** (APTx Neuron): [MNIST Experimentation Code](https://github.com/mr-ravin/APTxNeuron)
46
+
47
+ #### Cite Paper as:
48
+ ```
49
+ Kumar, Ravin. "APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation." arXiv preprint arXiv:2507.14270 (2025).
50
+ ```
51
+ Or,
52
+ ```
53
+ @article{kumar2025aptx,
54
+ title={APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation},
55
+ author={Kumar, Ravin},
56
+ journal={arXiv preprint arXiv:2507.14270},
57
+ year={2025}
58
+ }
59
+ ```
60
+
61
+ ---
62
+ ### APTx Neuron
63
+ <b>Abstract</b>: We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the [APTx activation function](https://arxiv.org/abs/2209.06119), thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i) + \delta$, where all parameters $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to 96.69\% test accuracy within 11 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and computational efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it.
64
+
65
+ The APTx Neuron is a novel computational unit that unifies linear transformation and non-linear activation into a single, expressive formulation. Inspired by the parametric APTx activation function, this neuron architecture removes the strict separation between computation and activation, allowing both to be learned as a cohesive entity. It is designed to enhance representational flexibility while reducing architectural redundancy.
66
+
67
+ #### Mathematical Formulation
68
+
69
+ Traditionally, a neuron computes the output as:
70
+
71
+ $y = \phi\left( \sum_{i=1}^{n} w_i x_i + b \right)$
72
+
73
+ where:
74
+ - $x_i$ are the inputs,
75
+ - $w_i$ are the weights,
76
+ - $b$ is the bias,
77
+ - and $\phi$ is an activation function such as ReLU, Swish, or Mish.
78
+
79
+
80
+ The APTx Neuron merges these components into a unified trainable expression as:
81
+
82
+ $y = \sum_{i=1}^{n} \left[ (\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i \right] + \delta$
83
+
84
+ where:
85
+ - $x_i$ is the $i$-th input feature,
86
+ - $\alpha_i$, $\beta_i$, and $\gamma_i$ are trainable parameters for each input,
87
+ - $\delta$ is a trainable scalar bias.
88
+
89
+ This equation allows the neuron to modulate each input through a learned, per-dimension non-linearity and scaling operation. The term $(\alpha_i + \tanh(\beta_i x_i))$ introduces adaptive gating, and $\gamma_i x_i$ provides multiplicative control.
90
+
91
+ ---
92
+ ## 📥 Installation
93
+ ```bash
94
+ pip install aptx_neuron
95
+ ```
96
+ or,
97
+
98
+ ```bash
99
+ pip install git+https://github.com/mr-ravin/aptx_neuron.git
100
+ ```
101
+ ----
102
+
103
+ ## Usage
104
+ <b>1</b>. APTx Neuron-based Layer with all $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
105
+
106
+ The setting `is_alpha_trainable = True` keeps $\alpha_i$ trainable. Each APTx neuron will have $(3n + 1)$ trainable parameters, where $n$ is input dimension. Note: The default value of `is_alpha_trainable` is `True`.
107
+
108
+ ```
109
+ import aptx_neuron
110
+ input_dim = 8 # assuming input vector to be of dimension 8.
111
+ output_dim = 1 # assuming output dimension equals 1.
112
+
113
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=True)
114
+ ```
115
+
116
+ <b>2</b>. APTx Neuron-based Layer with $\alpha_i=1$ (not trainable); While $\beta_i$, $\gamma_i$, and $\delta$ as trainable:
117
+
118
+ The setting `is_alpha_trainable = False` makes $\alpha_i$ fixed (non-trainable). Each APTx neuron will then have $(2n + 1)$ trainable parameters, thus reducing memory and training time per epoch. Here, $n$ is input dimension.
119
+
120
+ ```
121
+ import aptx_neuron
122
+ input_dim = 8 # assuming input vector to be of dimension 8.
123
+ output_dim = 1 # assuming output dimension equals 1.
124
+
125
+ aptx_neuron_layer = aptx_neuron.aptx_layer(input_dim=input_dim, output_dim=output_dim, is_alpha_trainable=False) # α_i is fixed (not trainable)
126
+ ```
127
+
128
+ ----
129
+ #### Conclusion
130
+ This work introduced the APTx Neuron, a unified, fully trainable neural unit that integrates linear transformation and non-linear activation into a single expression, extending the APTx activation function. By learning per-input parameters $\alpha_i$, $\beta_i$, and $\gamma_i$ for each input $x_i$, and a shared bias term $\delta$ within a neuron, the APTx Neuron removes the need for separate activation layers and enables fine-grained input transformation. The APTx Neuron generalizes traditional neurons and activations, offering greater representational power. Our experiments show that a fully connected APTx Neuron-based feedforward neural network achieves 96.69% test accuracy on the MNIST dataset within 11 epochs using approximately 332K trainable parameters, demonstrating rapid convergence and high efficiency. This design lays the groundwork for extending APTx Neurons to CNNs and transformers, paving the way for more compact and adaptive deep learning architectures.
131
+
132
+ ----
133
+
134
+ ### 📜 Copyright License
135
+ ```python
136
+ Copyright (c) 2025 Ravin Kumar
137
+ Website: https://mr-ravin.github.io
138
+
139
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation
140
+ files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy,
141
+ modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the
142
+ Software is furnished to do so, subject to the following conditions:
143
+
144
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
145
+ Software.
146
+
147
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
148
+ WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
149
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
150
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
151
+ ```
@@ -0,0 +1,9 @@
1
+ README.md
2
+ setup.cfg
3
+ setup.py
4
+ aptx_neuron/__init__.py
5
+ aptx_neuron.egg-info/PKG-INFO
6
+ aptx_neuron.egg-info/SOURCES.txt
7
+ aptx_neuron.egg-info/dependency_links.txt
8
+ aptx_neuron.egg-info/requires.txt
9
+ aptx_neuron.egg-info/top_level.txt
@@ -0,0 +1 @@
1
+ torch>=1.8.0
@@ -0,0 +1 @@
1
+ aptx_neuron
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,36 @@
1
+ import os
2
+ from setuptools import setup, find_packages
3
+
4
+ # Read long description safely
5
+ with open("README.md", "r", encoding="utf-8") as f:
6
+ long_description = f.read() if os.path.exists("README.md") else ""
7
+
8
+ setup(
9
+ name="aptx_neuron",
10
+ version="0.0.1",
11
+ author="Ravin Kumar",
12
+ author_email="mr.ravin_kumar@hotmail.com",
13
+ description="A PyTorch implementation of the APTx Neuron.",
14
+ long_description=long_description,
15
+ long_description_content_type="text/markdown",
16
+ url="https://github.com/mr-ravin/aptx_neuron",
17
+ packages=find_packages(),
18
+ install_requires=[
19
+ "torch>=1.8.0"
20
+ ],
21
+ python_requires=">=3.7",
22
+ classifiers=[
23
+ "Programming Language :: Python :: 3",
24
+ "License :: OSI Approved :: MIT License",
25
+ "Operating System :: OS Independent",
26
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
27
+ "Intended Audience :: Developers",
28
+ "Intended Audience :: Science/Research"
29
+ ],
30
+ keywords=[
31
+ "APTx Neuron", "unified neuron", "neuron", "activation function", "deep learning", "pytorch", "neural network", "perceptron"
32
+ "machine learning", "artificial intelligence", "AI", "ML", "DL", "torch"
33
+ ],
34
+ license="MIT",
35
+ include_package_data=True
36
+ )