binboost 0.2.5__tar.gz → 0.2.6__tar.gz

binboost-0.2.6/PKG-INFO

@@ -0,0 +1,429 @@
+Metadata-Version: 2.4
+Name: binboost
+Version: 0.2.6
+Summary: Gradient Boosting Berbasis Aturan Logika Adaptif untuk Fitur Biner
+Author: Rangga Wahyu Pratama
+License: MIT
+Keywords: gradient boosting,logical rules,binary features,interpretable machine learning
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.21.0
+Requires-Dist: pandas>=1.3.0
+Dynamic: author
+Dynamic: license-file
+Dynamic: requires-python
+
+# BinBoost
+
+**Gradient boosting with adaptive logic rules for binary and multi-class classification.**
+
+BinBoost builds an additive ensemble in which every weak learner is a small logical rule over the input features, for example `(A AND NOT B) OR C`. Numeric features are binarized again at every boosting iteration with thresholds chosen from the current gradient. Each rule contributes one of two learned weights to the score of a sample, depending on whether the sample satisfies the rule. Because every weak learner is a readable rule, the fitted model can be inspected directly, without a post-hoc explanation tool.
+
+The package depends only on NumPy. Pandas DataFrames are accepted as input and their column names are used in rule descriptions.
+
+## Contents
+
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [Multi-class classification](#multi-class-classification)
+- [Reading the model](#reading-the-model)
+- [How it works](#how-it-works)
+- [Input requirements](#input-requirements)
+- [API reference](#api-reference)
+- [Package layout](#package-layout)
+- [License](#license)
+
+## Installation
+
+```bash
+pip install binboost
+```
+
+## Quick start
+
+```python
+from sklearn.datasets import load_breast_cancer
+from sklearn.model_selection import train_test_split
+from binboost import BinBoost
+
+data = load_breast_cancer(as_frame=True)
+X_train, X_test, y_train, y_test = train_test_split(
+    data.data, data.target, test_size=0.25, random_state=0, stratify=data.target
+)
+
+model = BinBoost(n_estimators=50, random_state=0)
+model.fit(X_train, y_train)
+
+print(model.predict(X_test))
+print(model.predict_proba(X_test))   # shape (n_samples, 2)
+print(model.score(X_test, y_test))   # accuracy
+print(model.rules_[:2])
+```
+
+```text
+['(worst concave points AND mean concave points)', '((worst radius AND worst concavity) AND worst concave points)']
+```
+
+`rule_summary_` lists every rule together with its two weights:
+
+```python
+import pandas as pd
+print(pd.DataFrame(model.rule_summary_).head(2))
+```
+
+```text
+   iterasi                                                         aturan  bobot_memenuhi  bobot_tidak_memenuhi  panjang_aturan  threshold_prediksi
+0        1                 (worst concave points AND mean concave points)       -2.514364              1.204648               2            0.570929
+1        2  ((worst radius AND worst concavity) AND worst concave points)       -1.920358              1.219571               3            0.570929
+```
+
+Labels for `BinBoost` must be `0` or `1`. For more than two classes use `OneVsRestBinBoost`.
+
+## Multi-class classification
+
+`OneVsRestBinBoost` trains one `BinBoost` model per class (one-vs-rest). Every keyword argument is forwarded to `BinBoost`, so the hyperparameters are identical.
+
+```python
+from sklearn.datasets import load_iris
+from sklearn.model_selection import train_test_split
+from binboost.multiclass import OneVsRestBinBoost
+
+iris = load_iris(as_frame=True)
+X_train, X_test, y_train, y_test = train_test_split(
+    iris.data, iris.target, test_size=0.3, random_state=0, stratify=iris.target
+)
+
+model = OneVsRestBinBoost(mode='multiclass', n_estimators=50, random_state=0)
+model.fit(X_train, y_train)          # labels may be integers or strings
+
+print(model.predict(X_test))
+print(model.predict_proba(X_test))   # rows are normalized to sum to 1
+print(model.n_rules_)                # rules summed over all classes
+```
+
+Two modes are available:
+
+| `mode` | `predict_proba` | `predict` |
+|---|---|---|
+| `'multiclass'` (default) | Per-class probabilities normalized so that each row sums to 1. A row whose raw scores are all zero gets a uniform distribution. | The class with the largest normalized probability. |
+| `'multilabel'` | Raw per-class probabilities, not normalized. | One binary column per class, each using its own sub-model and prediction threshold. |
+
+## Reading the model
+
+### Rule structure
+
+A rule combines features from left to right with the operators `AND`, `OR` and optionally `XOR`. There is no operator precedence: `((A AND B) OR C)` is evaluated exactly as the parentheses indicate. Each feature in a rule can be negated, and `rules_` shows negation as `NOT`.
+
+The score of a sample is
+
+```
+F(x) = F0 + learning_rate * sum over rules of ( w1 if x satisfies the rule else w0 )
+P(y = 1 | x) = sigmoid(F(x))
+```
+
+`F0_` is the initial score, `rule_weights_[i] = [w0, w1]` holds the two weights of rule `i`, and the class decision compares `P(y = 1 | x)` with the prediction threshold.
+
+### Global view
+
+```python
+model.rules_                      # text of every rule, negations written as NOT
+model.rule_summary_               # list of dicts, one per rule, with both weights
+model.feature_importances_        # normalized importance per feature
+model.feature_usage_              # how often each feature appears in a rule
+model.thresholds_                 # binarization thresholds per numeric feature and iteration
+model.apply(X)                    # (n_samples, n_rules) matrix of rule activations
+```
+
+`jelaskan()` prints every rule in an IF / THEN / ELSE layout:
+
+```python
+model.jelaskan(maks=2, urut='pengaruh')
+```
+
+```text
+Skor awal +0.518
+Skor semua aturan dijumlahkan lalu diubah jadi probabilitas lewat sigmoid
+Kelas positif adalah 1
+Kelas positif dipilih jika probabilitas minimal 0.571
+Skor positif mendorong ke kelas positif dan skor negatif mendorong ke arah sebaliknya
+
+Aturan 1 dengan pengaruh 0.744
+JIKA  worst concave points > 0.14235
+      DAN mean concave points > 0.051455
+MAKA skor -0.503
+SELAIN ITU skor +0.241
+
+Aturan 2 dengan pengaruh 0.628
+JIKA  worst radius > 15.455
+      DAN worst concavity > 0.2108
+      DAN worst concave points > 0.1151
+MAKA skor -0.384
+SELAIN ITU skor +0.244
+```
+
+How to read it:
+
+- **Numeric features** are written against the binarization threshold used in that rule's iteration, for example `worst radius > 15.455`. A negated numeric feature becomes `<=`.
+- **Binary features** are written as `feature = 1`, and a negated binary feature as `feature = 0`. No `NOT` appears in this output.
+- **Layout.** When all operators in a rule are the same, the rule is a vertical list with the operator at the start of each line. When the operators are mixed, the rule stays on one line with parentheses so that the left-to-right evaluation order is unambiguous.
+- **Scores** are the contribution of the rule to the final score, already multiplied by the learning rate. They therefore differ from the raw values in `rule_weights_`.
+- **Influence** (`pengaruh`) is the absolute difference between the two scores of the rule. `urut='pengaruh'` sorts rules by it.
+
+The text of these reports is in Indonesian. A short glossary:
+
+| Output term | Meaning |
+|---|---|
+| `JIKA` | IF |
+| `MAKA` | THEN |
+| `SELAIN ITU` | ELSE |
+| `DAN` / `ATAU` | AND / OR (`XOR` is unchanged) |
+| `Aturan` | Rule |
+| `pengaruh` | influence |
+| `Skor awal` | initial score |
+| `terpenuhi` / `tidak terpenuhi` | satisfied / not satisfied |
+| `Nilai sampel` | sample values |
+
+`rule_summary_` uses Indonesian keys as well: `iterasi` (iteration), `aturan` (rule), `bobot_memenuhi` (weight when satisfied), `bobot_tidak_memenuhi` (weight when not satisfied), `panjang_aturan` (rule length) and `threshold_prediksi` (prediction threshold).
+
+`ringkas_ambang()` summarizes the thresholds that rules actually use for each numeric feature:
+
+```python
+print(pd.DataFrame(model.ringkas_ambang()).head(3))
+```
+
+```text
+                  fitur  jumlah_dipakai  ambang_terkecil  ambang_tengah  ambang_terbesar
+0          worst radius              17          12.8350       16.80500         17.67500
+1         worst texture              17          20.2250       23.26500         25.83000
+2  worst concave points              16           0.1151        0.12865          0.14655
+```
+
+The columns give the feature name, the number of rules that use it, and the smallest, median and largest threshold among those rules.
+
+### Local view
+
+`pilih_sampel` picks representative samples automatically, and `jelaskan_sampel` explains them rule by rule.
+
+```python
+X_eval = X_test.reset_index(drop=True)
+
+model.pilih_sampel(X_eval, y_test.values)
+model.jelaskan_sampel(X_eval, y_test.values, maks=8)
+
+# Explain specific rows instead (positional row indices, labels optional)
+model.jelaskan_sampel(X_eval, indeks=[5, 12])
+```
+
+For every class, `pilih_sampel` returns up to three samples (`indeks` is `None` when a class has no sample of that kind):
+
+| `jenis` | Meaning |
+|---|---|
+| `'mudah'` | Correctly classified, farthest from the decision boundary. |
+| `'sulit'` | Correctly classified, closest to the decision boundary. |
+| `'salah'` | Misclassified, farthest from the decision boundary. |
+
+Distance to the boundary is measured as the difference between the final score and the logit of the prediction threshold, so it stays meaningful when the threshold is not 0.5. For `OneVsRestBinBoost` it is the gap between the two largest normalized probabilities.
+
+A report lists the initial score, the `maks` rules with the largest absolute contribution (including the sample's own values for the features involved), one line that aggregates the remaining rules, and the final score and probability:
+
+```text
+Jenis sampel : Salah dan yakin untuk kelas 1
+Indeks baris : 13
+Label asli   : 1
+Prediksi     : 0
+
+Skor awal +0.518
+Aturan 1 terpenuhi skor -0.503
+    worst concave points > 0.14235 DAN mean concave points > 0.051455
+    Nilai sampel worst concave points 0.1505 dan mean concave points 0.07798
+Aturan 3 terpenuhi skor -0.335
+    worst perimeter > 103.25 DAN worst concavity > 0.2162 DAN mean concave points > 0.049885
+    Nilai sampel worst perimeter 107 dan worst concavity 0.6783 dan mean concave points 0.07798
+Sisa 48 aturan lainnya skor -0.852
+Skor akhir -1.171
+Probabilitas kelas positif 0.237
+Keputusan kelas 0 karena probabilitas 0.237 kurang dari 0.571
+```
+
+Every rule adds a score, either `w1` when it is satisfied or `w0` when it is not, so a sample has one contribution per rule. The report therefore shows the largest ones and folds the rest into a single line, which keeps the sum complete. In multi-class mode a misclassified sample is explained with two sub-models: the predicted class (why it won) and the true class (why it lost).
+
+## How it works
+
+**Training loop.** Training starts from the log-odds of the (weighted) positive rate as the initial score `F0`. Each iteration then does the following:
+
+1. Compute the gradient and Hessian of the selected loss at the current scores.
+2. Draw a random subsample without replacement (`subsample`).
+3. Binarize every numeric feature on the subsample with the selected strategy. This yields one threshold per numeric feature for this iteration.
+4. Optionally keep only the candidate features whose gain is above a percentile (`feature_selection_threshold`).
+5. Find a rule with beam search (below).
+6. Apply the iteration's thresholds to the full training set and evaluate the rule there.
+7. Compute two Newton-style weights, one for the samples that satisfy the rule and one for the others, using `w = sum(g) / (sum(H) + lambda)`, with `lambda = lambda0 * L / sqrt(n + 1)`. Here `L` is the rule length and `n` is the number of samples on that side.
+8. Update the scores with `learning_rate * (w1 * h + w0 * (1 - h))`, where `h` is 1 for samples that satisfy the rule and 0 otherwise.
+
+Training stops after `n_estimators` iterations, when no candidate rule passes the acceptance tests, or earlier when `n_iter_no_change` is set and the training loss stops improving by more than `tol`.
+
+**Rule search.** Candidates are scored by how much they reduce the (weighted) squared error when fitting the current gradient with a single-weight rule, and all candidates are scored with matrix products rather than loops. The search starts from single features. Each feature is tried in both polarities, original and negated. A beam of the `beam_width` best rules is then extended one feature at a time with the allowed operators up to `max_rule_length`, and extension stops as soon as it no longer improves the best score. A rule is only accepted when:
+
+- at least `min_samples_rule` samples satisfy it, and
+- its error reduction is at least `gamma` times the current gradient energy. Because this threshold is relative, one value can be used across datasets and losses. A value of `0.0` disables the pruning.
+
+`rule_complexity_penalty` makes the score of a candidate worse in proportion to its length.
+
+**Binarization strategies** (`binarize_strategy`):
+
+| Strategy | Threshold |
+|---|---|
+| `'gradient'` | Recomputed every iteration: the candidate cut that maximizes the absolute sum of the current gradient over the samples above it. |
+| `'quantile'` | Fixed per feature: the median of the candidate cut points. |
+| `'uniform'` | Fixed per feature: the midpoint between the minimum and maximum value. |
+| `'kmeans'` | Fixed per feature: the midpoint between the two centroids of a one-dimensional 2-means clustering. |
+
+Candidate cut points are midpoints between consecutive unique values. `n_thresholds='auto'` caps them at 256 per feature, taking an evenly spaced subset when there are more. Binarized values are `1` when the feature is strictly greater than the threshold.
+
+**One-hot groups.** Binary columns that are mutually exclusive and add up to exactly one in every row are detected as a one-hot group (`ohe_groups_`). With `ohe_policy='block'` a single rule can never contain two categories of the same group, which rules out nonsensical rules such as `(Color_Red AND Color_Blue)`.
+
+**Prediction threshold.** With `threshold='auto'` the cut-off on `P(y = 1)` is chosen after training by maximizing Youden's J (sensitivity + specificity - 1) on the training data, taking the smallest threshold that reaches the maximum. This means the cut-off is not fixed at 0.5. Pass a float to use a fixed value.
+
+**Feature importance.** For each rule, the absolute difference `|w1 - w0|` is added to every feature the rule contains, and the totals are normalized to sum to 1.
+
+**Losses.** `'logistic'` is the standard log-loss. `'focal'` is the focal loss with `focal_gamma` and `focal_alpha`. `'poly'` is the PolyLoss variant controlled by `poly_epsilon`. Each loss provides its own gradient and Hessian for the Newton weights.
+
+**Sample weights.** `fit` accepts non-negative `sample_weight`. The weights are normalized to a mean of 1, so only their ratios matter. They enter the initial score, the gradients and Hessians, the rule search, the Newton weights, the training loss and the Youden threshold. `min_samples_rule` counts samples whose weight is positive. Subsampling stays uniform. Weights that are all equal give the same result as no weights.
+
+## Input requirements
+
+- `X` is converted to a float64 matrix. Numeric columns are detected automatically as every column that is not purely `0`/`1`. Missing values are not handled, so impute them before fitting.
+- `y` for `BinBoost` must contain only `0` and `1`. `OneVsRestBinBoost` accepts any labels, with at least two distinct classes.
+- Categorical features with three or more categories must be one-hot encoded. Keep every category column (do not drop one) so that each group adds up to exactly one per row and is recognized as a group:
+
+```python
+from sklearn.preprocessing import OneHotEncoder
+
+encoder = OneHotEncoder(sparse_output=False)   # no drop=...
+X_encoded = encoder.fit_transform(X[['categorical_column']])
+```
+
+If one column per group is dropped (for example `drop='first'`), the columns no longer add up to one in every row. The group is then not detected, and `ohe_policy` has no effect for it.
+
+- `pilih_sampel` and `jelaskan_sampel` expect `y` values of `0`/`1` for `BinBoost` and use positional row indices. `OneVsRestBinBoost.pilih_sampel` supports `mode='multiclass'` only.
+
+## API reference
+
+### `BinBoost`
+
+```python
+BinBoost(
+    n_estimators=100, learning_rate=0.2, loss='logistic',
+    poly_epsilon=1.0, focal_gamma=2.0, focal_alpha=0.25,
+    max_rule_length=4, operators=None, use_xor=False,
+    rule_complexity_penalty=0.0, feature_selection_threshold=0.0,
+    beam_width=5, subsample=0.8, min_samples_rule=5,
+    lambda0=3.0, gamma=0.001, binarize_strategy='gradient',
+    n_thresholds='auto', n_iter_no_change=None, tol=1e-4,
+    random_state=None, warm_start=False, threshold='auto',
+    ohe_policy='block',
+)
+```
+
+| Parameter | Default | Description |
+|---|---|---|
+| `n_estimators` | `100` | Maximum number of boosting iterations (rules). |
+| `learning_rate` | `0.2` | Shrinkage applied to the contribution of every rule. |
+| `loss` | `'logistic'` | `'logistic'`, `'focal'` or `'poly'`. |
+| `poly_epsilon` | `1.0` | Epsilon of the PolyLoss. Used only when `loss='poly'`. |
+| `focal_gamma` | `2.0` | Gamma of the focal loss. Used only when `loss='focal'`. |
+| `focal_alpha` | `0.25` | Alpha of the focal loss. Used only when `loss='focal'`. |
+| `max_rule_length` | `4` | Maximum number of features in one rule. |
+| `operators` | `None` | Logical operators allowed in a rule. `None` means `['AND', 'OR']`. |
+| `use_xor` | `False` | Adds `'XOR'` to the operators. |
+| `rule_complexity_penalty` | `0.0` | Makes longer candidate rules score worse in proportion to their length. |
+| `feature_selection_threshold` | `0.0` | Percentile of per-feature gain below which features are not considered as candidates in an iteration. `0.0` keeps all features and `0.5` keeps those above the median gain. |
+| `beam_width` | `5` | Number of best partial rules kept at each step of the beam search. |
+| `subsample` | `0.8` | Fraction of the training data drawn without replacement in each iteration. |
+| `min_samples_rule` | `5` | Minimum number of samples that must satisfy a rule. A float below `1.0` is read as a fraction of the sample count. |
+| `lambda0` | `3.0` | Base regularization of the Newton weights, `lambda = lambda0 * L / sqrt(n + 1)`. |
+| `gamma` | `0.001` | Relative minimum error reduction (as a multiple of the gradient energy) needed to accept a rule. `0.0` disables it. |
+| `binarize_strategy` | `'gradient'` | `'gradient'`, `'quantile'`, `'uniform'` or `'kmeans'`. |
+| `n_thresholds` | `'auto'` | Number of candidate thresholds per numeric feature. `'auto'` means at most 256. |
+| `n_iter_no_change` | `None` | Stop when the training loss has not improved for this many iterations. `None` disables early stopping. |
+| `tol` | `1e-4` | Minimum loss improvement that counts as progress for early stopping. |
+| `random_state` | `None` | Seed for the subsampling. |
+| `warm_start` | `False` | Continue training from the existing ensemble and add more rules. |
+| `threshold` | `'auto'` | Prediction threshold on `P(y = 1)`. `'auto'` uses Youden's J on the training data. A float fixes it. |
+| `ohe_policy` | `'block'` | `'block'` forbids two categories of one one-hot group in the same rule and `'allow'` permits it. |
+
+### Methods
+
+| Method | Description |
+|---|---|
+| `fit(X, y, sample_weight=None)` | Train the model. Returns `self`. |
+| `predict(X)` | Class labels `0` or `1` using the prediction threshold. |
+| `predict_proba(X)` | Array of shape `(n_samples, 2)`. |
+| `predict_log_proba(X)` | Logarithm of `predict_proba`. |
+| `score(X, y)` | Accuracy. |
+| `staged_predict(X)` / `staged_predict_proba(X)` | Generators that yield the prediction after each boosting iteration. |
+| `apply(X)` | Rule activations of shape `(n_samples, n_rules)`. |
+| `get_params()` / `set_params(**params)` | Read and change hyperparameters. |
+| `jelaskan(urut='iterasi', maks=None, cetak=True, nama_positif=1)` | IF / THEN / ELSE description of the rules. `urut` is `'iterasi'` or `'pengaruh'`, `maks` limits the number of rules, and the text is returned as a string. |
+| `ringkas_ambang()` | Per-feature summary of the thresholds that rules use. |
+| `pilih_sampel(X, y, jenis=('mudah', 'sulit', 'salah'))` | Select representative samples per class. |
+| `jelaskan_sampel(X, y=None, indeks=None, jenis=('mudah', 'sulit', 'salah'), maks=8, cetak=True)` | Rule-by-rule explanation of selected or given samples. `y` is required unless `indeks` is given. |
+
+### Attributes and properties
+
+| Name | Description |
+|---|---|
+| `estimators_` | List of the learned `Rule` objects. |
+| `F0_` | Initial score (log-odds of the positive rate). |
+| `rule_weights_` | Array of shape `(n_rules, 2)` with columns `[w0, w1]`: the weight when the rule is not satisfied and when it is satisfied. |
+| `rules_` | Text of every rule, with `NOT` for negated features. |
+| `rule_summary_` | List of dicts with the keys `iterasi`, `aturan`, `bobot_memenuhi`, `bobot_tidak_memenuhi`, `panjang_aturan`, `threshold_prediksi`. |
+| `feature_importances_` | Normalized importance per feature. |
+| `feature_usage_` | Dict of how often each feature appears in rules. |
+| `thresholds_` | Dict mapping each numeric feature index to its list of thresholds, one per iteration. |
+| `train_score_` | Training loss after each iteration. |
+| `n_rules_` / `n_estimators_` | Number of rules actually learned. |
+| `n_features_in_` | Number of features seen during `fit`. |
+| `feature_names_in_` | Column names when `fit` received a DataFrame, otherwise `None`. |
+| `numeric_cols_` | Indices of the columns treated as numeric. |
+| `ohe_groups_` | Detected one-hot groups as a dict, or `None`. |
+
+### `OneVsRestBinBoost`
+
+```python
+OneVsRestBinBoost(mode='multiclass', **binboost_params)
+```
+
+`fit(X, y, sample_weight=None)`, `predict`, `predict_proba` and `score` work as described above. Other members:
+
+| Name | Description |
+|---|---|
+| `classes_`, `n_classes_` | The class labels and their count. |
+| `estimators_ovr_` | The per-class `BinBoost` models, in the order of `classes_`. |
+| `rule_summary_` | Combined rule summaries with an extra `kelas` column that names the class of each sub-model. |
+| `feature_usage_` | Feature usage summed over all classes. |
+| `n_rules_` | Total number of rules across classes. |
+| `jelaskan(urut='iterasi', maks=None, cetak=True)` | The `BinBoost.jelaskan` description for every class model. The class is the positive class of its own section. |
+| `ringkas_ambang()` | Threshold summaries of all models with a `kelas` column. |
+| `pilih_sampel(X, y, jenis=...)` | Representative samples per class (`mode='multiclass'` only). |
+| `jelaskan_sampel(X, y, jenis=..., maks=8, cetak=True)` | Explanation of the selected samples, including the combined class probabilities. |
+
+## Package layout
+
+| Module | Contents |
+|---|---|
+| `binboost/binboost.py` | `BinBoost`: the boosting loop, prediction, thresholding and the explanation methods. |
+| `binboost/rule.py` | `Rule` (evaluation and text rendering) and `BeamSearchRuleFinder` (vectorized beam search). |
+| `binboost/binarizer.py` | `GradientBinarizer`: the four binarization strategies. |
+| `binboost/loss.py` | `LogisticLoss`, `FocalLoss`, `PolyLoss`, `get_loss` and a numerically stable `sigmoid`. |
+| `binboost/multiclass.py` | `OneVsRestBinBoost`: the one-vs-rest wrapper. |
+
+## License
+
+MIT