smallgbm 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
smallgbm-1.0.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Emelyanov Ilya
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,2 @@
1
+ include README.md
2
+ include LICENSE
@@ -0,0 +1,147 @@
1
+ Metadata-Version: 2.4
2
+ Name: smallgbm
3
+ Version: 1.0.0
4
+ Summary: Gradient boosting optimized for small datasets (n < 1000)
5
+ Home-page: https://github.com/nsdmlk/SmallGBM
6
+ Author: Emelyanov Ilya
7
+ Author-email: Nsdmlk@yandex.ru
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: License :: OSI Approved :: MIT License
10
+ Classifier: Operating System :: OS Independent
11
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
12
+ Classifier: Intended Audience :: Science/Research
13
+ Requires-Python: >=3.8
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: numpy>=1.20.0
17
+ Requires-Dist: scikit-learn>=1.0.0
18
+ Dynamic: author
19
+ Dynamic: author-email
20
+ Dynamic: classifier
21
+ Dynamic: description
22
+ Dynamic: description-content-type
23
+ Dynamic: home-page
24
+ Dynamic: license-file
25
+ Dynamic: requires-dist
26
+ Dynamic: requires-python
27
+ Dynamic: summary
28
+
29
+
30
+ # SmallGBM
31
+
32
+ <p align="center">
33
+ <b>Gradient boosting that actually works on small data.</b>
34
+ </p>
35
+
36
+ <p align="center">
37
+ <img src="https://img.shields.io/badge/version-1.0.0-blue" alt="version">
38
+ <img src="https://img.shields.io/badge/python-3.8+-green" alt="python">
39
+ <img src="https://img.shields.io/badge/license-MIT-brightgreen" alt="license">
40
+ <img src="https://img.shields.io/badge/pip%20install-smallgbm-orange" alt="pip">
41
+ </p>
42
+
43
+ ---
44
+
45
+ ## Why SmallGBM?
46
+
47
+ XGBoost and LightGBM are built for scale. They shine on thousands of rows. But when you only have **50, 100, or 500 samples**, their default hyperparameters fail — overfitting, instability, unpredictable results.
48
+
49
+ **SmallGBM** is designed from the ground up for datasets with fewer than 1000 samples.
50
+
51
+ | Feature | SmallGBM | XGBoost | LightGBM |
52
+ | ---------------------------- | -------- | ------- | -------- |
53
+ | Bayesian leaf weights | ✅ | ❌ | ❌ |
54
+ | Adaptive regularization | ✅ | ❌ | ❌ |
55
+ | No bootstrap (uses all data) | ✅ | ❌ | ❌ |
56
+ | Stable under label noise | ✅ | ❌ | ❌ |
57
+ | scikit-learn compatible | ✅ | ✅ | ✅ |
58
+
59
+ ---
60
+
61
+ ## Noise Stability
62
+
63
+ SmallGBM degrades gracefully when labels are noisy — unlike XGBoost and LightGBM which drop sharply.
64
+
65
+ <p align="center">
66
+ <img src="docs/noise_comparison.png" width="600" alt="Noise stability comparison">
67
+ </p>
68
+
69
+ *At 20% label noise, SmallGBM is the best performer. Bayesian regularization keeps it stable when others collapse.*
70
+
71
+ ---
72
+
73
+ ## Learning Curve
74
+
75
+ Clear, predictable improvement as data grows. Reliable performance starts at **n ≈ 40**.
76
+
77
+ <p align="center">
78
+ <img src="docs/learning_curve.png" width="600" alt="Learning curve">
79
+ </p>
80
+
81
+ *No sudden jumps, no catastrophic failures. A safe choice when data is limited.*
82
+
83
+ ---
84
+
85
+ ## Installation
86
+
87
+ ```bash
88
+ pip install smallgbm
89
+ ```
90
+
91
+ ## Quickstart
92
+
93
+ ```python
94
+ from smallgbm import SmallGBMClassifier
95
+
96
+ model = SmallGBMClassifier()
97
+ model.fit(X_train, y_train)
98
+ proba = model.predict_proba(X_test)
99
+ ```
100
+
101
+ ## API
102
+
103
+ ### SmallGBMClassifier
104
+
105
+ | Parameter | Default | Description |
106
+ | -------------------- | ------- | ------------------------- |
107
+ | `n_estimators` | 50 | Number of boosting rounds |
108
+ | `max_depth` | 3 | Maximum tree depth |
109
+ | `min_samples_leaf` | 3 | Minimum samples per leaf |
110
+ | `learning_rate` | 0.1 | Shrinkage factor |
111
+ | `sigma_prior` | 0.5 | Bayesian prior strength |
112
+
113
+ ### SmallGBMRegressor
114
+
115
+ Same parameters, for regression tasks.
116
+
117
+ ```python
118
+ from smallgbm import SmallGBMRegressor
119
+
120
+ model = SmallGBMRegressor()
121
+ model.fit(X_train, y_train)
122
+ preds = model.predict(X_test)
123
+ ```
124
+
125
+ ---
126
+
127
+ ## Research
128
+
129
+ Full characterisation notebook with 5 experiments: `benchmark_final.ipynb`
130
+
131
+ - Noise stability analysis
132
+ - Prior sensitivity
133
+ - Sample size curve
134
+ - Regression performance
135
+ - Class imbalance tolerance
136
+
137
+ ---
138
+
139
+ ## License
140
+
141
+ MIT © [Emelyanov Ilya](https://github.com/nsdmlk) 2026
142
+
143
+ ---
144
+
145
+ <p align="center">
146
+ <sub>Built with ❤️ for the small data community</sub>
147
+ </p>
@@ -0,0 +1,119 @@
1
+
2
+ # SmallGBM
3
+
4
+ <p align="center">
5
+ <b>Gradient boosting that actually works on small data.</b>
6
+ </p>
7
+
8
+ <p align="center">
9
+ <img src="https://img.shields.io/badge/version-1.0.0-blue" alt="version">
10
+ <img src="https://img.shields.io/badge/python-3.8+-green" alt="python">
11
+ <img src="https://img.shields.io/badge/license-MIT-brightgreen" alt="license">
12
+ <img src="https://img.shields.io/badge/pip%20install-smallgbm-orange" alt="pip">
13
+ </p>
14
+
15
+ ---
16
+
17
+ ## Why SmallGBM?
18
+
19
+ XGBoost and LightGBM are built for scale. They shine on thousands of rows. But when you only have **50, 100, or 500 samples**, their default hyperparameters fail — overfitting, instability, unpredictable results.
20
+
21
+ **SmallGBM** is designed from the ground up for datasets with fewer than 1000 samples.
22
+
23
+ | Feature | SmallGBM | XGBoost | LightGBM |
24
+ | ---------------------------- | -------- | ------- | -------- |
25
+ | Bayesian leaf weights | ✅ | ❌ | ❌ |
26
+ | Adaptive regularization | ✅ | ❌ | ❌ |
27
+ | No bootstrap (uses all data) | ✅ | ❌ | ❌ |
28
+ | Stable under label noise | ✅ | ❌ | ❌ |
29
+ | scikit-learn compatible | ✅ | ✅ | ✅ |
30
+
31
+ ---
32
+
33
+ ## Noise Stability
34
+
35
+ SmallGBM degrades gracefully when labels are noisy — unlike XGBoost and LightGBM which drop sharply.
36
+
37
+ <p align="center">
38
+ <img src="docs/noise_comparison.png" width="600" alt="Noise stability comparison">
39
+ </p>
40
+
41
+ *At 20% label noise, SmallGBM is the best performer. Bayesian regularization keeps it stable when others collapse.*
42
+
43
+ ---
44
+
45
+ ## Learning Curve
46
+
47
+ Clear, predictable improvement as data grows. Reliable performance starts at **n ≈ 40**.
48
+
49
+ <p align="center">
50
+ <img src="docs/learning_curve.png" width="600" alt="Learning curve">
51
+ </p>
52
+
53
+ *No sudden jumps, no catastrophic failures. A safe choice when data is limited.*
54
+
55
+ ---
56
+
57
+ ## Installation
58
+
59
+ ```bash
60
+ pip install smallgbm
61
+ ```
62
+
63
+ ## Quickstart
64
+
65
+ ```python
66
+ from smallgbm import SmallGBMClassifier
67
+
68
+ model = SmallGBMClassifier()
69
+ model.fit(X_train, y_train)
70
+ proba = model.predict_proba(X_test)
71
+ ```
72
+
73
+ ## API
74
+
75
+ ### SmallGBMClassifier
76
+
77
+ | Parameter | Default | Description |
78
+ | -------------------- | ------- | ------------------------- |
79
+ | `n_estimators` | 50 | Number of boosting rounds |
80
+ | `max_depth` | 3 | Maximum tree depth |
81
+ | `min_samples_leaf` | 3 | Minimum samples per leaf |
82
+ | `learning_rate` | 0.1 | Shrinkage factor |
83
+ | `sigma_prior` | 0.5 | Bayesian prior strength |
84
+
85
+ ### SmallGBMRegressor
86
+
87
+ Same parameters, for regression tasks.
88
+
89
+ ```python
90
+ from smallgbm import SmallGBMRegressor
91
+
92
+ model = SmallGBMRegressor()
93
+ model.fit(X_train, y_train)
94
+ preds = model.predict(X_test)
95
+ ```
96
+
97
+ ---
98
+
99
+ ## Research
100
+
101
+ Full characterisation notebook with 5 experiments: `benchmark_final.ipynb`
102
+
103
+ - Noise stability analysis
104
+ - Prior sensitivity
105
+ - Sample size curve
106
+ - Regression performance
107
+ - Class imbalance tolerance
108
+
109
+ ---
110
+
111
+ ## License
112
+
113
+ MIT © [Emelyanov Ilya](https://github.com/nsdmlk) 2026
114
+
115
+ ---
116
+
117
+ <p align="center">
118
+ <sub>Built with ❤️ for the small data community</sub>
119
+ </p>
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,28 @@
1
+ from setuptools import setup, find_packages
2
+
3
+ with open("README.md", "r", encoding="utf-8") as fh:
4
+ long_description = fh.read()
5
+
6
+ setup(
7
+ name="smallgbm",
8
+ version="1.0.0",
9
+ author="Emelyanov Ilya",
10
+ author_email="Nsdmlk@yandex.ru",
11
+ description="Gradient boosting optimized for small datasets (n < 1000)",
12
+ long_description=long_description,
13
+ long_description_content_type="text/markdown",
14
+ url="https://github.com/nsdmlk/SmallGBM",
15
+ packages=find_packages(),
16
+ classifiers=[
17
+ "Programming Language :: Python :: 3",
18
+ "License :: OSI Approved :: MIT License",
19
+ "Operating System :: OS Independent",
20
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
21
+ "Intended Audience :: Science/Research",
22
+ ],
23
+ python_requires=">=3.8",
24
+ install_requires=[
25
+ "numpy>=1.20.0",
26
+ "scikit-learn>=1.0.0",
27
+ ],
28
+ )
@@ -0,0 +1,2 @@
1
+ # smallgbm/__init__.py
2
+ from .smallgbm import SmallGBMClassifier, SmallGBMRegressor
@@ -0,0 +1,170 @@
1
+ import numpy as np
2
+ from sklearn.base import BaseEstimator, ClassifierMixin, RegressorMixin
3
+ from .tree import BayesianDecisionTree
4
+
5
+
6
+ class SmallGBMClassifier(BaseEstimator, ClassifierMixin):
7
+ def __init__(self, n_estimators=50, max_depth=3, min_samples_leaf=3,
8
+ learning_rate=0.1, sigma_prior=0.5, adaptive_prior=False,
9
+ dynamic_depth=False, weighted_residuals=False, soft_bootstrap=False):
10
+ self.n_estimators = n_estimators
11
+ self.max_depth = max_depth
12
+ self.min_samples_leaf = min_samples_leaf
13
+ self.learning_rate = learning_rate
14
+ self.sigma_prior = sigma_prior
15
+ self.adaptive_prior = adaptive_prior
16
+ self.dynamic_depth = dynamic_depth
17
+ self.weighted_residuals = weighted_residuals
18
+ self.soft_bootstrap = soft_bootstrap
19
+
20
+ def _log_odds(self, y):
21
+ pos = np.sum(y == 1)
22
+ neg = np.sum(y == 0)
23
+ return np.log((pos + 1e-10) / (neg + 1e-10))
24
+
25
+ def _sigmoid(self, x):
26
+ return 1 / (1 + np.exp(-x))
27
+
28
+ def fit(self, X, y):
29
+ X = np.array(X)
30
+ y = np.array(y)
31
+ n_samples = len(y)
32
+
33
+ init = self._log_odds(y)
34
+ self.base_score_ = init
35
+ self._trees = []
36
+
37
+ current_pred = np.full(y.shape, init)
38
+
39
+ for i in range(self.n_estimators):
40
+ proba = self._sigmoid(current_pred)
41
+ residuals = y - proba
42
+
43
+ # 2. Weighted residuals: weight = proba * (1 - proba)
44
+ if self.weighted_residuals:
45
+ confidence = proba * (1 - proba)
46
+ confidence = np.clip(confidence, 1e-10, None)
47
+ sample_weights = confidence / confidence.sum() * n_samples
48
+ else:
49
+ sample_weights = np.ones(n_samples)
50
+
51
+ # 4. Soft bootstrap: weighted sampling without replacement bias
52
+ if self.soft_bootstrap:
53
+ # Use all data but weight the residuals by sample_weights
54
+ weighted_residuals = residuals * sample_weights
55
+ else:
56
+ weighted_residuals = residuals
57
+
58
+ # 3. Dynamic depth: deeper early, shallower later
59
+ if self.dynamic_depth:
60
+ progress = i / self.n_estimators
61
+ current_max_depth = max(1, int(self.max_depth * (1 - progress * 0.5)))
62
+ else:
63
+ current_max_depth = self.max_depth
64
+
65
+ # 1. Adaptive sigma_prior: 1 / sqrt(n) if not specified
66
+ if self.adaptive_prior and self.sigma_prior is None:
67
+ sigma_prior = 1.0 / np.sqrt(n_samples)
68
+ elif self.sigma_prior is not None:
69
+ sigma_prior = self.sigma_prior
70
+ else:
71
+ sigma_prior = 1.0
72
+
73
+ tree = BayesianDecisionTree(
74
+ max_depth=current_max_depth,
75
+ min_samples_leaf=self.min_samples_leaf,
76
+ sigma_prior=sigma_prior,
77
+ n_splits=20
78
+ )
79
+ tree.fit(X, weighted_residuals)
80
+
81
+ update = tree.predict(X)
82
+ current_pred += self.learning_rate * update
83
+ self._trees.append(tree)
84
+
85
+ return self
86
+
87
+ def predict_proba(self, X):
88
+ X = np.array(X)
89
+ current_pred = np.full(X.shape[0], self.base_score_)
90
+ for tree in self._trees:
91
+ current_pred += self.learning_rate * tree.predict(X)
92
+ proba_pos = self._sigmoid(current_pred)
93
+ return np.column_stack([1 - proba_pos, proba_pos])
94
+
95
+ def predict(self, X):
96
+ proba = self.predict_proba(X)[:, 1]
97
+ return (proba > 0.5).astype(int)
98
+
99
+ class SmallGBMRegressor(BaseEstimator, RegressorMixin):
100
+ def __init__(self, n_estimators=50, max_depth=3, min_samples_leaf=3,
101
+ learning_rate=0.1, sigma_prior=0.5, adaptive_prior=False,
102
+ dynamic_depth=False, weighted_residuals=False, soft_bootstrap=False):
103
+ self.n_estimators = n_estimators
104
+ self.max_depth = max_depth
105
+ self.min_samples_leaf = min_samples_leaf
106
+ self.learning_rate = learning_rate
107
+ self.sigma_prior = sigma_prior
108
+ self.adaptive_prior = adaptive_prior
109
+ self.dynamic_depth = dynamic_depth
110
+ self.weighted_residuals = weighted_residuals
111
+ self.soft_bootstrap = soft_bootstrap
112
+
113
+ def fit(self, X, y):
114
+ X = np.array(X)
115
+ y = np.array(y).astype(float)
116
+ n_samples = len(y)
117
+
118
+ # Initial prediction: mean of y
119
+ init = np.mean(y)
120
+ self.base_score_ = init
121
+ self._trees = []
122
+
123
+ current_pred = np.full(y.shape, init)
124
+
125
+ for i in range(self.n_estimators):
126
+ residuals = y - current_pred # MSE residuals
127
+
128
+ if self.weighted_residuals:
129
+ confidence = np.abs(residuals)
130
+ confidence = np.clip(confidence, 1e-10, None)
131
+ sample_weights = confidence / confidence.sum() * n_samples
132
+ else:
133
+ sample_weights = np.ones(n_samples)
134
+
135
+ if self.soft_bootstrap:
136
+ weighted_residuals = residuals * sample_weights
137
+ else:
138
+ weighted_residuals = residuals
139
+
140
+ if self.dynamic_depth:
141
+ progress = i / self.n_estimators
142
+ current_max_depth = max(1, int(self.max_depth * (1 - progress * 0.5)))
143
+ else:
144
+ current_max_depth = self.max_depth
145
+
146
+ if self.adaptive_prior and self.sigma_prior is None:
147
+ sigma_prior = 1.0 / np.sqrt(n_samples)
148
+ elif self.sigma_prior is not None:
149
+ sigma_prior = self.sigma_prior
150
+ else:
151
+ sigma_prior = 1.0
152
+
153
+ tree = BayesianDecisionTree(
154
+ max_depth=current_max_depth,
155
+ min_samples_leaf=self.min_samples_leaf,
156
+ sigma_prior=sigma_prior)
157
+ tree.fit(X, weighted_residuals)
158
+
159
+ update = tree.predict(X)
160
+ current_pred += self.learning_rate * update
161
+ self._trees.append(tree)
162
+
163
+ return self
164
+
165
+ def predict(self, X):
166
+ X = np.array(X)
167
+ current_pred = np.full(X.shape[0], self.base_score_)
168
+ for tree in self._trees:
169
+ current_pred += self.learning_rate * tree.predict(X)
170
+ return current_pred
@@ -0,0 +1,116 @@
1
+ import numpy as np
2
+
3
+ class BayesianDecisionTree:
4
+ """Optimized decision tree with Bayesian leaf weight estimation."""
5
+
6
+ def __init__(self, max_depth=2, min_samples_leaf=5, sigma_prior=1.0, n_splits=10):
7
+ self.max_depth = max_depth
8
+ self.min_samples_leaf = min_samples_leaf
9
+ self.sigma_prior = sigma_prior
10
+ self.n_splits = n_splits
11
+ self.tree_ = None
12
+
13
+ def _variance(self, y):
14
+ if len(y) <= 1:
15
+ return 0.0
16
+ return np.var(y)
17
+
18
+ def _bayesian_weight(self, residuals):
19
+ n = len(residuals)
20
+ sum_r = np.sum(residuals)
21
+ sigma_noise = self._variance(residuals)
22
+ if sigma_noise == 0:
23
+ return sum_r / n if n > 0 else 0.0
24
+ shrinkage = sigma_noise / (self.sigma_prior ** 2)
25
+ return sum_r / (n + shrinkage)
26
+
27
+ def _best_split(self, X, residuals):
28
+ best_gain = -np.inf
29
+ best_feature = None
30
+ best_threshold = None
31
+
32
+ n_features = X.shape[1]
33
+ n = len(residuals)
34
+
35
+ for feature in range(n_features):
36
+ values = X[:, feature]
37
+ sort_idx = np.argsort(values)
38
+ sorted_values = values[sort_idx]
39
+ sorted_residuals = residuals[sort_idx]
40
+
41
+ # Prefix sums for O(1) variance computation
42
+ cumsum = np.cumsum(sorted_residuals)
43
+ cumsum2 = np.cumsum(sorted_residuals ** 2)
44
+ total_sum = cumsum[-1]
45
+ total_sum2 = cumsum2[-1]
46
+ parent_var = (total_sum2 - total_sum**2 / n) / n if n > 1 else 0
47
+
48
+ # Full search over all valid split positions
49
+ for pos in range(self.min_samples_leaf, n - self.min_samples_leaf + 1):
50
+ left_n = pos
51
+ right_n = n - pos
52
+
53
+ left_sum = cumsum[pos - 1]
54
+ left_sum2 = cumsum2[pos - 1]
55
+ left_var = (left_sum2 - left_sum**2 / left_n) / left_n if left_n > 1 else 0
56
+
57
+ right_sum = total_sum - left_sum
58
+ right_sum2 = total_sum2 - left_sum2
59
+ right_var = (right_sum2 - right_sum**2 / right_n) / right_n if right_n > 1 else 0
60
+
61
+ gain = parent_var - (left_n / n * left_var + right_n / n * right_var)
62
+
63
+ if gain > best_gain:
64
+ best_gain = gain
65
+ best_feature = feature
66
+ best_threshold = (sorted_values[pos - 1] + sorted_values[pos]) / 2
67
+
68
+ return best_feature, best_threshold, best_gain
69
+
70
+ def _build_tree(self, X, residuals, depth=0):
71
+ n = len(residuals)
72
+
73
+ if (depth >= self.max_depth or
74
+ n < self.min_samples_leaf * 2 or
75
+ len(np.unique(residuals)) == 1):
76
+ return {
77
+ 'type': 'leaf',
78
+ 'weight': self._bayesian_weight(residuals),
79
+ 'n_samples': n
80
+ }
81
+
82
+ feature, threshold, gain = self._best_split(X, residuals)
83
+
84
+ if feature is None or gain <= 0:
85
+ return {
86
+ 'type': 'leaf',
87
+ 'weight': self._bayesian_weight(residuals),
88
+ 'n_samples': n
89
+ }
90
+
91
+ left_mask = X[:, feature] <= threshold
92
+ right_mask = ~left_mask
93
+
94
+ return {
95
+ 'type': 'node',
96
+ 'feature': feature,
97
+ 'threshold': threshold,
98
+ 'n_samples': n,
99
+ 'left': self._build_tree(X[left_mask], residuals[left_mask], depth + 1),
100
+ 'right': self._build_tree(X[right_mask], residuals[right_mask], depth + 1)
101
+ }
102
+
103
+ def fit(self, X, residuals):
104
+ self.tree_ = self._build_tree(np.array(X), np.array(residuals))
105
+ return self
106
+
107
+ def _predict_one(self, x, node):
108
+ if node['type'] == 'leaf':
109
+ return node['weight']
110
+ if x[node['feature']] <= node['threshold']:
111
+ return self._predict_one(x, node['left'])
112
+ else:
113
+ return self._predict_one(x, node['right'])
114
+
115
+ def predict(self, X):
116
+ return np.array([self._predict_one(x, self.tree_) for x in np.array(X)])
@@ -0,0 +1,147 @@
1
+ Metadata-Version: 2.4
2
+ Name: smallgbm
3
+ Version: 1.0.0
4
+ Summary: Gradient boosting optimized for small datasets (n < 1000)
5
+ Home-page: https://github.com/nsdmlk/SmallGBM
6
+ Author: Emelyanov Ilya
7
+ Author-email: Nsdmlk@yandex.ru
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: License :: OSI Approved :: MIT License
10
+ Classifier: Operating System :: OS Independent
11
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
12
+ Classifier: Intended Audience :: Science/Research
13
+ Requires-Python: >=3.8
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: numpy>=1.20.0
17
+ Requires-Dist: scikit-learn>=1.0.0
18
+ Dynamic: author
19
+ Dynamic: author-email
20
+ Dynamic: classifier
21
+ Dynamic: description
22
+ Dynamic: description-content-type
23
+ Dynamic: home-page
24
+ Dynamic: license-file
25
+ Dynamic: requires-dist
26
+ Dynamic: requires-python
27
+ Dynamic: summary
28
+
29
+
30
+ # SmallGBM
31
+
32
+ <p align="center">
33
+ <b>Gradient boosting that actually works on small data.</b>
34
+ </p>
35
+
36
+ <p align="center">
37
+ <img src="https://img.shields.io/badge/version-1.0.0-blue" alt="version">
38
+ <img src="https://img.shields.io/badge/python-3.8+-green" alt="python">
39
+ <img src="https://img.shields.io/badge/license-MIT-brightgreen" alt="license">
40
+ <img src="https://img.shields.io/badge/pip%20install-smallgbm-orange" alt="pip">
41
+ </p>
42
+
43
+ ---
44
+
45
+ ## Why SmallGBM?
46
+
47
+ XGBoost and LightGBM are built for scale. They shine on thousands of rows. But when you only have **50, 100, or 500 samples**, their default hyperparameters fail — overfitting, instability, unpredictable results.
48
+
49
+ **SmallGBM** is designed from the ground up for datasets with fewer than 1000 samples.
50
+
51
+ | Feature | SmallGBM | XGBoost | LightGBM |
52
+ | ---------------------------- | -------- | ------- | -------- |
53
+ | Bayesian leaf weights | ✅ | ❌ | ❌ |
54
+ | Adaptive regularization | ✅ | ❌ | ❌ |
55
+ | No bootstrap (uses all data) | ✅ | ❌ | ❌ |
56
+ | Stable under label noise | ✅ | ❌ | ❌ |
57
+ | scikit-learn compatible | ✅ | ✅ | ✅ |
58
+
59
+ ---
60
+
61
+ ## Noise Stability
62
+
63
+ SmallGBM degrades gracefully when labels are noisy — unlike XGBoost and LightGBM which drop sharply.
64
+
65
+ <p align="center">
66
+ <img src="docs/noise_comparison.png" width="600" alt="Noise stability comparison">
67
+ </p>
68
+
69
+ *At 20% label noise, SmallGBM is the best performer. Bayesian regularization keeps it stable when others collapse.*
70
+
71
+ ---
72
+
73
+ ## Learning Curve
74
+
75
+ Clear, predictable improvement as data grows. Reliable performance starts at **n ≈ 40**.
76
+
77
+ <p align="center">
78
+ <img src="docs/learning_curve.png" width="600" alt="Learning curve">
79
+ </p>
80
+
81
+ *No sudden jumps, no catastrophic failures. A safe choice when data is limited.*
82
+
83
+ ---
84
+
85
+ ## Installation
86
+
87
+ ```bash
88
+ pip install smallgbm
89
+ ```
90
+
91
+ ## Quickstart
92
+
93
+ ```python
94
+ from smallgbm import SmallGBMClassifier
95
+
96
+ model = SmallGBMClassifier()
97
+ model.fit(X_train, y_train)
98
+ proba = model.predict_proba(X_test)
99
+ ```
100
+
101
+ ## API
102
+
103
+ ### SmallGBMClassifier
104
+
105
+ | Parameter | Default | Description |
106
+ | -------------------- | ------- | ------------------------- |
107
+ | `n_estimators` | 50 | Number of boosting rounds |
108
+ | `max_depth` | 3 | Maximum tree depth |
109
+ | `min_samples_leaf` | 3 | Minimum samples per leaf |
110
+ | `learning_rate` | 0.1 | Shrinkage factor |
111
+ | `sigma_prior` | 0.5 | Bayesian prior strength |
112
+
113
+ ### SmallGBMRegressor
114
+
115
+ Same parameters, for regression tasks.
116
+
117
+ ```python
118
+ from smallgbm import SmallGBMRegressor
119
+
120
+ model = SmallGBMRegressor()
121
+ model.fit(X_train, y_train)
122
+ preds = model.predict(X_test)
123
+ ```
124
+
125
+ ---
126
+
127
+ ## Research
128
+
129
+ Full characterisation notebook with 5 experiments: `benchmark_final.ipynb`
130
+
131
+ - Noise stability analysis
132
+ - Prior sensitivity
133
+ - Sample size curve
134
+ - Regression performance
135
+ - Class imbalance tolerance
136
+
137
+ ---
138
+
139
+ ## License
140
+
141
+ MIT © [Emelyanov Ilya](https://github.com/nsdmlk) 2026
142
+
143
+ ---
144
+
145
+ <p align="center">
146
+ <sub>Built with ❤️ for the small data community</sub>
147
+ </p>
@@ -0,0 +1,12 @@
1
+ LICENSE
2
+ MANIFEST.in
3
+ README.md
4
+ setup.py
5
+ smallgbm/__init__.py
6
+ smallgbm/smallgbm.py
7
+ smallgbm/tree.py
8
+ smallgbm.egg-info/PKG-INFO
9
+ smallgbm.egg-info/SOURCES.txt
10
+ smallgbm.egg-info/dependency_links.txt
11
+ smallgbm.egg-info/requires.txt
12
+ smallgbm.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ numpy>=1.20.0
2
+ scikit-learn>=1.0.0
@@ -0,0 +1 @@
1
+ smallgbm