scikit-doc 0.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- scikit_doc/__init__.py +328 -0
- scikit_doc/data/NOTEBOOK_EXAMPLES.md +3355 -0
- scikit_doc/data/SKLEARN_ROOT_HELP_1_6_1.txt +412 -0
- scikit_doc/data/USER_GUIDE.md +1088 -0
- scikit_doc/data/WORKED_EXAMPLES.md +4038 -0
- scikit_doc/data/examples.json +1864 -0
- scikit_doc/examples/customer_preprocessing.py +34 -0
- scikit_doc/examples/data_inspection.py +45 -0
- scikit_doc/examples/feature_selection.py +37 -0
- scikit_doc/examples/flight_features.py +49 -0
- scikit_doc/examples/house_cleaning.py +64 -0
- scikit_doc/examples/model_search.py +60 -0
- scikit_doc/examples/pca_ridgecv.py +22 -0
- scikit_doc/examples/regression_models.py +32 -0
- scikit_doc-0.1.0.dist-info/METADATA +21 -0
- scikit_doc-0.1.0.dist-info/RECORD +19 -0
- scikit_doc-0.1.0.dist-info/WHEEL +5 -0
- scikit_doc-0.1.0.dist-info/licenses/LICENSE +21 -0
- scikit_doc-0.1.0.dist-info/top_level.txt +1 -0
|
@@ -0,0 +1,1088 @@
|
|
|
1
|
+
# SCIKIT-DOC — Practical scikit-learn reference
|
|
2
|
+
|
|
3
|
+
A step-by-step handbook for tabular data inspection, cleaning, feature engineering,
|
|
4
|
+
preprocessing, regression, model selection, dimensionality reduction and feature
|
|
5
|
+
selection. It is designed to be read directly with Python's built-in `help()`.
|
|
6
|
+
|
|
7
|
+
## Quick start
|
|
8
|
+
|
|
9
|
+
```python
|
|
10
|
+
import scikit_doc as skdoc
|
|
11
|
+
help(skdoc) # full manual and all worked examples
|
|
12
|
+
skdoc.show() # rich Markdown in a notebook
|
|
13
|
+
skdoc.topics() # section index
|
|
14
|
+
skdoc.examples() # browse all worked examples
|
|
15
|
+
skdoc.api("Ridge") # installed scikit-learn API docstring
|
|
16
|
+
skdoc.search("OneHotEncoder")
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
The PyPI distribution is `scikit-doc`; the Python import name is `scikit_doc`.
|
|
20
|
+
|
|
21
|
+
## The complete workflow
|
|
22
|
+
|
|
23
|
+
For a supervised learning task, keep the following sequence visible in your notebook.
|
|
24
|
+
|
|
25
|
+
1. **Read the instructions literally.** Identify the target column, columns to exclude,
|
|
26
|
+
required split rule, `random_state`, requested transformer order, estimator parameters,
|
|
27
|
+
scoring metric, cross-validation count and exact value to report.
|
|
28
|
+
2. **Load the file.** Use `pd.read_csv`. Do not accidentally treat a saved index as a
|
|
29
|
+
feature. Inspect `df.head()`, `df.shape`, `df.columns`, `df.dtypes` and `df.info()`.
|
|
30
|
+
3. **Understand each column.** Check `nunique`, `unique`, `value_counts`, summary
|
|
31
|
+
statistics and plausible ranges. A column stored as `object` may be categorical even
|
|
32
|
+
when its values look numeric; a numeric column may contain a sentinel such as `-1`.
|
|
33
|
+
4. **Separate features and target.** Set `y = df[target]` and `X = df.drop(columns=[target])`.
|
|
34
|
+
Exclude identifiers that have no predictive meaning when instructed.
|
|
35
|
+
5. **Create the split exactly once.** For a random split, use `train_test_split` with the
|
|
36
|
+
specified `test_size` and seed. For time-based or fixed-row splits, use boolean masks
|
|
37
|
+
or `.iloc` exactly as specified. Keep the original train and test labels aligned.
|
|
38
|
+
6. **Inspect missing and unknown values.** Count both real nulls (`isna`) and special
|
|
39
|
+
strings or impossible numeric values. Turn confirmed sentinels into `np.nan` before
|
|
40
|
+
using scikit-learn imputers.
|
|
41
|
+
7. **Learn preprocessing from training data only.** Fit imputers, encoders and scalers on
|
|
42
|
+
`X_train`; transform `X_train` and `X_test` with that same fitted object. Never fit a
|
|
43
|
+
transformer separately on test data.
|
|
44
|
+
8. **Preserve feature order.** A `ColumnTransformer` emits columns in transformer-list
|
|
45
|
+
order. This matters for feature indices, coefficient positions and PCA inputs.
|
|
46
|
+
9. **Fit the requested estimator or pipeline.** Keep unspecified parameters at their
|
|
47
|
+
defaults. Use a `Pipeline` when transformations must be fitted inside cross-validation.
|
|
48
|
+
10. **Evaluate on the held-out test set.** Predict once, then calculate the requested
|
|
49
|
+
metric. Do not choose hyperparameters using the test set.
|
|
50
|
+
11. **Inspect fitted attributes.** Common examples are `coef_`, `intercept_`, `n_iter_`,
|
|
51
|
+
`best_params_`, `best_estimator_`, `support_`, `ranking_`, `scores_`, `pvalues_`,
|
|
52
|
+
`explained_variance_ratio_` and `n_components_`.
|
|
53
|
+
12. **Report exactly what was requested.** Distinguish a fraction from a percentage,
|
|
54
|
+
a variance ratio from an eigenvalue, an index from a feature name, and rounding from
|
|
55
|
+
truncation.
|
|
56
|
+
|
|
57
|
+
## 1. Reading and inspecting tabular data
|
|
58
|
+
|
|
59
|
+
### Load a CSV
|
|
60
|
+
|
|
61
|
+
```python
|
|
62
|
+
import numpy as np
|
|
63
|
+
import pandas as pd
|
|
64
|
+
|
|
65
|
+
df = pd.read_csv("data.csv")
|
|
66
|
+
print(df.head())
|
|
67
|
+
print(df.shape) # (rows, columns)
|
|
68
|
+
print(df.shape[0]) # number of samples
|
|
69
|
+
print(df.shape[1]) # number of columns, including target if present
|
|
70
|
+
print(df.columns.tolist())
|
|
71
|
+
print(df.dtypes)
|
|
72
|
+
df.info()
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Use `index_col=0` only when the file genuinely stores an index in its first column.
|
|
76
|
+
If the file has no index column, do not discard its first feature.
|
|
77
|
+
|
|
78
|
+
### Find unique categories and numerical columns
|
|
79
|
+
|
|
80
|
+
```python
|
|
81
|
+
print(df["qualification"].unique())
|
|
82
|
+
print(df["qualification"].nunique())
|
|
83
|
+
print(df["qualification"].value_counts(dropna=False))
|
|
84
|
+
|
|
85
|
+
numeric_columns = df.select_dtypes(include="number").columns.tolist()
|
|
86
|
+
object_columns = df.select_dtypes(include=["object", "category"]).columns.tolist()
|
|
87
|
+
print(numeric_columns)
|
|
88
|
+
print(object_columns)
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
A boolean/integer flag is numerically typed, but whether it should be treated as a
|
|
92
|
+
continuous feature or a category is a modelling decision. Use the requested dtype rule
|
|
93
|
+
when a task explicitly asks for `int`/`float` columns.
|
|
94
|
+
|
|
95
|
+
### Missing-value audit
|
|
96
|
+
|
|
97
|
+
```python
|
|
98
|
+
missing_by_column = df.isna().sum().sort_values(ascending=False)
|
|
99
|
+
missing_percent = (df.isna().mean() * 100).sort_values(ascending=False)
|
|
100
|
+
print(missing_by_column)
|
|
101
|
+
print(missing_percent.round(2))
|
|
102
|
+
print(df.isna().sum().sum())
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
`isna()` detects `NaN`, `None` and pandas missing sentinels. It does not detect arbitrary
|
|
106
|
+
strings like `"?"`, `"UNKNOWN"`, `"UNK"`, `"not available"` or impossible numbers.
|
|
107
|
+
|
|
108
|
+
### Detect unknown and implausible values
|
|
109
|
+
|
|
110
|
+
Always inspect values before defining a sentinel rule.
|
|
111
|
+
|
|
112
|
+
```python
|
|
113
|
+
print(sorted(df["RM"].dropna().unique()))
|
|
114
|
+
print(sorted(df["AGE"].dropna().unique())[:10])
|
|
115
|
+
print(df["RIVERSIDE"].value_counts(dropna=False))
|
|
116
|
+
|
|
117
|
+
bad_rooms = df["RM"].lt(1) # only if values below one are invalid here
|
|
118
|
+
bad_age = df["AGE"].lt(0) # only if negative age is invalid here
|
|
119
|
+
bad_riverside = df["RIVERSIDE"].eq("UNKNOWN")
|
|
120
|
+
|
|
121
|
+
print("invalid room count:", int(bad_rooms.sum()))
|
|
122
|
+
print("invalid age:", int(bad_age.sum()))
|
|
123
|
+
print("unknown river status:", int(bad_riverside.sum()))
|
|
124
|
+
print("combined:", int(bad_rooms.sum() + bad_age.sum() + bad_riverside.sum()))
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
The threshold is domain-specific. Do not mark every zero as missing unless the feature
|
|
128
|
+
specification says zero is impossible. After counting, replace invalid entries:
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
df.loc[bad_rooms, "RM"] = np.nan
|
|
132
|
+
df.loc[bad_age, "AGE"] = np.nan
|
|
133
|
+
df.loc[bad_riverside, "RIVERSIDE"] = np.nan
|
|
134
|
+
# Or, for a textual sentinel:
|
|
135
|
+
df["Property"] = df["Property"].replace("?", np.nan)
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
To calculate a statistic while ignoring invalid values, use a filtered Series:
|
|
139
|
+
|
|
140
|
+
```python
|
|
141
|
+
valid_age_mean = df.loc[~bad_age, "AGE"].mean()
|
|
142
|
+
unknown_share_percent = 100 * bad_riverside.mean()
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
`mean()` and `median()` ignore genuine nulls by default. They do not ignore a sentinel
|
|
146
|
+
number until you mask it or replace it with `np.nan`.
|
|
147
|
+
|
|
148
|
+
### Filtering, counting and conditional logic
|
|
149
|
+
|
|
150
|
+
Boolean masks are the most reliable way to translate a sentence into code.
|
|
151
|
+
|
|
152
|
+
```python
|
|
153
|
+
mask = (
|
|
154
|
+
df["Year"].eq(2021)
|
|
155
|
+
& df["Locality"].eq("Stamford")
|
|
156
|
+
& df["num_bathrooms"].ge(2)
|
|
157
|
+
& df["Face"].isin(["South", "West"])
|
|
158
|
+
)
|
|
159
|
+
count = int(mask.sum())
|
|
160
|
+
|
|
161
|
+
# Exclude incomplete rows before counting:
|
|
162
|
+
valid = df["AGE"].notna() & df["RIVERSIDE"].notna()
|
|
163
|
+
count_valid = int((valid & df["RIVERSIDE"].eq("NO") & df["AGE"].gt(60)).sum())
|
|
164
|
+
|
|
165
|
+
# Inclusive interval: values from 3 through 5:
|
|
166
|
+
count_interval = int(df["HIGHWAYCOUNT"].between(3, 5, inclusive="both").sum())
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Use `&` for AND, `|` for OR, `~` for NOT. Put parentheses around each condition.
|
|
170
|
+
Do not use Python's `and` or `or` with pandas Series.
|
|
171
|
+
|
|
172
|
+
### Grouping, sorting and top/bottom records
|
|
173
|
+
|
|
174
|
+
```python
|
|
175
|
+
# Average outcome by group, highest first:
|
|
176
|
+
mean_by_source = df.groupby("Source")["Price"].mean().sort_values(ascending=False)
|
|
177
|
+
|
|
178
|
+
# Most common category after filtering:
|
|
179
|
+
subset = df.loc[
|
|
180
|
+
df["Age"].le(35)
|
|
181
|
+
& df["Location"].eq("Suburban")
|
|
182
|
+
& df["Credit Score"].ge(600)
|
|
183
|
+
]
|
|
184
|
+
most_common_policy = subset["Policy Type"].value_counts().idxmax()
|
|
185
|
+
|
|
186
|
+
# Average of the 10 cheapest / most expensive outcomes:
|
|
187
|
+
mean_10_lowest = df["PRICE"].nsmallest(10).mean()
|
|
188
|
+
mean_10_highest = df["PRICE"].nlargest(10).mean()
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
`nsmallest` and `nlargest` avoid manually sorting and slicing. For ties, decide whether
|
|
192
|
+
the request means exactly ten rows or all rows tied at the boundary.
|
|
193
|
+
|
|
194
|
+
### Create bins with `pd.cut`
|
|
195
|
+
|
|
196
|
+
```python
|
|
197
|
+
bins = [-np.inf, 15, 25, 35, np.inf]
|
|
198
|
+
labels = ["low", "middle", "upper-middle", "high"]
|
|
199
|
+
df["PRICE_BAND"] = pd.cut(
|
|
200
|
+
df["PRICE"], bins=bins, labels=labels, right=False
|
|
201
|
+
)
|
|
202
|
+
print(df["PRICE_BAND"].value_counts(sort=False))
|
|
203
|
+
print(df["PRICE_BAND"].value_counts().idxmin())
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
`right=False` creates left-closed, right-open intervals: `[15, 25)` includes 15 but not
|
|
207
|
+
25. Use the boundary convention stated in the task.
|
|
208
|
+
|
|
209
|
+
### Correlation matrices and heatmaps
|
|
210
|
+
|
|
211
|
+
```python
|
|
212
|
+
numeric = df.select_dtypes(include="number")
|
|
213
|
+
corr = numeric.corr(method="pearson")
|
|
214
|
+
|
|
215
|
+
# Find the strongest / weakest off-diagonal pair without counting the diagonal:
|
|
216
|
+
pairs = corr.where(~np.eye(corr.shape[0], dtype=bool)).stack()
|
|
217
|
+
print(pairs.sort_values().head()) # most negative
|
|
218
|
+
print(pairs.sort_values(ascending=False).head()) # most positive
|
|
219
|
+
|
|
220
|
+
# Optional visualisation:
|
|
221
|
+
import matplotlib.pyplot as plt
|
|
222
|
+
import seaborn as sns
|
|
223
|
+
plt.figure(figsize=(10, 8))
|
|
224
|
+
sns.heatmap(corr, annot=True, fmt=".2f", cmap="coolwarm", center=0)
|
|
225
|
+
plt.tight_layout()
|
|
226
|
+
plt.show()
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Correlation is defined only for numeric columns. It measures association, not causation.
|
|
230
|
+
A feature's correlation with another feature is different from its correlation with the
|
|
231
|
+
target; remove the target first when a task asks for feature-to-feature pairs.
|
|
232
|
+
|
|
233
|
+
## 2. Splitting data correctly
|
|
234
|
+
|
|
235
|
+
### Random train/test split
|
|
236
|
+
|
|
237
|
+
```python
|
|
238
|
+
from sklearn.model_selection import train_test_split
|
|
239
|
+
|
|
240
|
+
TARGET = "PRICE"
|
|
241
|
+
X = df.drop(columns=[TARGET])
|
|
242
|
+
y = df[TARGET]
|
|
243
|
+
|
|
244
|
+
X_train, X_test, y_train, y_test = train_test_split(
|
|
245
|
+
X, y, test_size=0.25, random_state=42
|
|
246
|
+
)
|
|
247
|
+
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
The test fraction is a fraction of rows. The exact row count depends on the input row
|
|
251
|
+
count and split rounding. `random_state` makes the split reproducible. Keep the same
|
|
252
|
+
split for every subsequent preprocessing/model step.
|
|
253
|
+
|
|
254
|
+
### Split chronologically by a year column
|
|
255
|
+
|
|
256
|
+
```python
|
|
257
|
+
train_rows = df["Year"] <= 2019
|
|
258
|
+
test_rows = df["Year"].between(2020, 2022, inclusive="both")
|
|
259
|
+
|
|
260
|
+
train_df = df.loc[train_rows].copy()
|
|
261
|
+
test_df = df.loc[test_rows].copy()
|
|
262
|
+
X_train = train_df.drop(columns=["Sale Price"])
|
|
263
|
+
y_train = train_df["Sale Price"]
|
|
264
|
+
X_test = test_df.drop(columns=["Sale Price"])
|
|
265
|
+
y_test = test_df["Sale Price"]
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
This is not a random split. Verify the year ranges and ensure no rows are lost or
|
|
269
|
+
included in both masks.
|
|
270
|
+
|
|
271
|
+
### Split by fixed row positions
|
|
272
|
+
|
|
273
|
+
```python
|
|
274
|
+
train_df = df.iloc[:8000].copy() # positions 0 through 7999
|
|
275
|
+
test_df = df.iloc[8000:].copy() # positions 8000 through end
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
`.iloc` is positional; `.loc` is label-based. If the requirement refers to row index
|
|
279
|
+
labels rather than row positions, use `.loc` with the intended labels.
|
|
280
|
+
|
|
281
|
+
### Last column is the target
|
|
282
|
+
|
|
283
|
+
```python
|
|
284
|
+
X = df.iloc[:, :-1]
|
|
285
|
+
y = df.iloc[:, -1]
|
|
286
|
+
X_train, X_test, y_train, y_test = train_test_split(
|
|
287
|
+
X, y, test_size=0.25, random_state=42
|
|
288
|
+
)
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
Do this only when the file specification confirms that the final column is the target.
|
|
292
|
+
|
|
293
|
+
## 3. Imputation without leakage
|
|
294
|
+
|
|
295
|
+
### Manual train-statistic imputation
|
|
296
|
+
|
|
297
|
+
Compute the statistic on training data only, then use the same value for both sets.
|
|
298
|
+
|
|
299
|
+
```python
|
|
300
|
+
X_train = X_train.copy()
|
|
301
|
+
X_test = X_test.copy()
|
|
302
|
+
|
|
303
|
+
income_mean = X_train["income"].mean()
|
|
304
|
+
policy_mode = X_train["policy"].mode(dropna=True).iloc[0]
|
|
305
|
+
area_mode = X_train["area"].mode(dropna=True).iloc[0]
|
|
306
|
+
claim_median = X_train["claim_amount"].median()
|
|
307
|
+
|
|
308
|
+
# Inspect counts before filling if a count is also required:
|
|
309
|
+
policy_missing_in_test = int(X_test["policy"].isna().sum())
|
|
310
|
+
|
|
311
|
+
for part in (X_train, X_test):
|
|
312
|
+
part["income"] = part["income"].fillna(income_mean)
|
|
313
|
+
part["policy"] = part["policy"].fillna(policy_mode)
|
|
314
|
+
part["area"] = part["area"].fillna(area_mode)
|
|
315
|
+
part["claim_amount"] = part["claim_amount"].fillna(claim_median)
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
For string sentinels, exclude the sentinel before computing the mode:
|
|
319
|
+
|
|
320
|
+
```python
|
|
321
|
+
valid_property = X_train.loc[X_train["Property"].ne("?"), "Property"]
|
|
322
|
+
property_mode = valid_property.mode().iloc[0]
|
|
323
|
+
for part in (X_train, X_test):
|
|
324
|
+
part["Property"] = part["Property"].replace("?", property_mode)
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
The training set is the only source of learned fill values. Do not calculate a separate
|
|
328
|
+
mean/mode/median on test data.
|
|
329
|
+
|
|
330
|
+
### `SimpleImputer` inside a pipeline
|
|
331
|
+
|
|
332
|
+
```python
|
|
333
|
+
from sklearn.impute import SimpleImputer
|
|
334
|
+
from sklearn.pipeline import Pipeline
|
|
335
|
+
from sklearn.preprocessing import StandardScaler
|
|
336
|
+
|
|
337
|
+
numeric_pipe = Pipeline([
|
|
338
|
+
("imputer", SimpleImputer(strategy="median")),
|
|
339
|
+
("scaler", StandardScaler()),
|
|
340
|
+
])
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
Common strategies are `"mean"`, `"median"`, `"most_frequent"` and `"constant"`.
|
|
344
|
+
Use `fill_value=...` with `strategy="constant"`. Fit only on training data.
|
|
345
|
+
|
|
346
|
+
### Several columns with different strategies
|
|
347
|
+
|
|
348
|
+
```python
|
|
349
|
+
fill_values = {
|
|
350
|
+
"Age": X_train["Age"].mean(),
|
|
351
|
+
"Annual Income": X_train["Annual Income"].median(),
|
|
352
|
+
"Number of Dependents": X_train["Number of Dependents"].mode().iloc[0],
|
|
353
|
+
"Health Score": X_train["Health Score"].mean(),
|
|
354
|
+
"Credit Score": X_train["Credit Score"].median(),
|
|
355
|
+
"Customer Feedback": X_train["Customer Feedback"].mode().iloc[0],
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
X_train = X_train.fillna(fill_values)
|
|
359
|
+
X_test = X_test.fillna(fill_values)
|
|
360
|
+
assert X_train.isna().sum().sum() == 0
|
|
361
|
+
assert X_test.isna().sum().sum() == 0
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
When a rule says statistics should exclude all rows with any missing/unknown value, first
|
|
365
|
+
create a complete-case subset for that statistic, rather than relying on a column-level
|
|
366
|
+
mean that only drops nulls in that one column:
|
|
367
|
+
|
|
368
|
+
```python
|
|
369
|
+
complete_train = X_train.dropna(subset=["Age", "Annual Income", "Credit Score"])
|
|
370
|
+
age_mean = complete_train["Age"].mean()
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
## 4. Encoding categorical variables
|
|
374
|
+
|
|
375
|
+
### One-hot encoding
|
|
376
|
+
|
|
377
|
+
```python
|
|
378
|
+
from sklearn.preprocessing import OneHotEncoder
|
|
379
|
+
|
|
380
|
+
encoder = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
|
|
381
|
+
X_train_encoded = encoder.fit_transform(X_train[["gender", "area", "policy"]])
|
|
382
|
+
X_test_encoded = encoder.transform(X_test[["gender", "area", "policy"]])
|
|
383
|
+
print(encoder.get_feature_names_out())
|
|
384
|
+
print(X_train_encoded.shape)
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
Each learned category becomes a binary column. The number of output columns is the sum
|
|
388
|
+
of the number of categories for each input feature, unless `drop` removes a reference
|
|
389
|
+
category or other options change the representation. Use `get_feature_names_out()` to
|
|
390
|
+
count and inspect the columns; do not guess from the number of input variables.
|
|
391
|
+
|
|
392
|
+
`handle_unknown="ignore"` avoids an error for a category first seen in test data. For an
|
|
393
|
+
unknown category, its one-hot block is all zeros. That is not the same as ordinal encoding
|
|
394
|
+
an unknown category to `-1`.
|
|
395
|
+
|
|
396
|
+
### Ordinal encoding with an explicit order
|
|
397
|
+
|
|
398
|
+
Use `OrdinalEncoder` only when categories have a meaningful order or an explicit order is
|
|
399
|
+
specified.
|
|
400
|
+
|
|
401
|
+
```python
|
|
402
|
+
from sklearn.preprocessing import OrdinalEncoder
|
|
403
|
+
|
|
404
|
+
education_order = ["High School", "Bachelor", "Masters", "Doctorate"]
|
|
405
|
+
frequency_order = ["Never", "Rarely", "Monthly", "Weekly", "Daily"]
|
|
406
|
+
encoder = OrdinalEncoder(
|
|
407
|
+
categories=[education_order, frequency_order],
|
|
408
|
+
handle_unknown="use_encoded_value",
|
|
409
|
+
unknown_value=-1,
|
|
410
|
+
)
|
|
411
|
+
X_train_ord = encoder.fit_transform(X_train[["Education Level", "Exercise Frequency"]])
|
|
412
|
+
X_test_ord = encoder.transform(X_test[["Education Level", "Exercise Frequency"]])
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
For nominal labels with no order, use one-hot encoding. If an order is specified, preserve
|
|
416
|
+
its exact sequence. `unknown_value=-1` makes unseen categories visible and countable:
|
|
417
|
+
|
|
418
|
+
```python
|
|
419
|
+
unknown_encoded_count = int((X_test_ord == -1).sum())
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
## 5. Scaling numerical variables
|
|
423
|
+
|
|
424
|
+
### Standard scaling
|
|
425
|
+
|
|
426
|
+
For each feature, `StandardScaler` subtracts the training mean and divides by the
|
|
427
|
+
training standard deviation.
|
|
428
|
+
|
|
429
|
+
```python
|
|
430
|
+
from sklearn.preprocessing import StandardScaler
|
|
431
|
+
|
|
432
|
+
scaler = StandardScaler()
|
|
433
|
+
X_train_scaled = scaler.fit_transform(X_train[["vintage"]])
|
|
434
|
+
X_test_scaled = scaler.transform(X_test[["vintage"]])
|
|
435
|
+
print(X_train_scaled.max())
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
Keep a one-column DataFrame (`[['vintage']]`) rather than a Series (`['vintage']`),
|
|
439
|
+
because scikit-learn expects a two-dimensional feature matrix.
|
|
440
|
+
|
|
441
|
+
### Min-max scaling
|
|
442
|
+
|
|
443
|
+
```python
|
|
444
|
+
from sklearn.preprocessing import MinMaxScaler
|
|
445
|
+
|
|
446
|
+
scaler = MinMaxScaler()
|
|
447
|
+
X_train_mm = scaler.fit_transform(X_train[["Duration"]])
|
|
448
|
+
X_test_mm = scaler.transform(X_test[["Duration"]])
|
|
449
|
+
print(X_test_mm.max())
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
The default output range is `[0, 1]`. A test value may be below zero or above one if it
|
|
453
|
+
falls outside the minimum/maximum observed in training data. This is expected.
|
|
454
|
+
|
|
455
|
+
### Get the transformed matrix mean or sum
|
|
456
|
+
|
|
457
|
+
```python
|
|
458
|
+
transformed_mean = X_train_transformed.mean()
|
|
459
|
+
transformed_test_mean = X_test_transformed.mean()
|
|
460
|
+
first_rows_total = X_test_transformed[:50].sum()
|
|
461
|
+
maximum_anywhere = X_test_transformed.max()
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
These are scalar summaries of the entire matrix. They are not per-feature statistics.
|
|
465
|
+
If a per-feature summary is needed, use `.mean(axis=0)`, `.max(axis=0)` or a DataFrame
|
|
466
|
+
with the transformed feature names.
|
|
467
|
+
|
|
468
|
+
## 6. ColumnTransformer and Pipeline
|
|
469
|
+
|
|
470
|
+
### Make output order explicit
|
|
471
|
+
|
|
472
|
+
```python
|
|
473
|
+
from sklearn.compose import ColumnTransformer
|
|
474
|
+
from sklearn.pipeline import Pipeline
|
|
475
|
+
from sklearn.impute import SimpleImputer
|
|
476
|
+
from sklearn.preprocessing import OneHotEncoder, StandardScaler, MinMaxScaler
|
|
477
|
+
|
|
478
|
+
categorical = ["gender", "area", "qualification", "policy"]
|
|
479
|
+
numeric = ["income", "vintage", "claim_amount"]
|
|
480
|
+
|
|
481
|
+
preprocessor = ColumnTransformer(
|
|
482
|
+
transformers=[
|
|
483
|
+
("categorical", OneHotEncoder(handle_unknown="ignore", sparse_output=False), categorical),
|
|
484
|
+
("numeric", StandardScaler(), numeric),
|
|
485
|
+
],
|
|
486
|
+
remainder="drop",
|
|
487
|
+
verbose_feature_names_out=False,
|
|
488
|
+
)
|
|
489
|
+
|
|
490
|
+
X_train_t = preprocessor.fit_transform(X_train)
|
|
491
|
+
X_test_t = preprocessor.transform(X_test)
|
|
492
|
+
print(X_train_t.shape, X_test_t.shape)
|
|
493
|
+
print(preprocessor.get_feature_names_out())
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
The transformer list determines output block order. If a task prescribes categorical
|
|
497
|
+
columns first and numerical columns second, list them in that order. If it prescribes a
|
|
498
|
+
specific per-column order, use multiple named transformer blocks as needed.
|
|
499
|
+
|
|
500
|
+
### Different scaler/imputer choices by column
|
|
501
|
+
|
|
502
|
+
```python
|
|
503
|
+
preprocessor = ColumnTransformer([
|
|
504
|
+
("crime", StandardScaler(), ["CRIM"]),
|
|
505
|
+
("zoning", MinMaxScaler(), ["ZN"]),
|
|
506
|
+
("industry", MinMaxScaler(), ["INDUS"]),
|
|
507
|
+
("pollution", StandardScaler(), ["POLINDEX"]),
|
|
508
|
+
("other_numeric", MinMaxScaler(), ["DIS", "HIGHWAYCOUNT", "TAX", "PTRATIO", "IMM", "BPL"]),
|
|
509
|
+
("rooms", Pipeline([
|
|
510
|
+
("imputer", SimpleImputer(strategy="mean")),
|
|
511
|
+
("scale", MinMaxScaler()),
|
|
512
|
+
]), ["RM"]),
|
|
513
|
+
("age", Pipeline([
|
|
514
|
+
("imputer", SimpleImputer(strategy="median")),
|
|
515
|
+
("scale", StandardScaler()),
|
|
516
|
+
]), ["AGE"]),
|
|
517
|
+
("river", Pipeline([
|
|
518
|
+
("imputer", SimpleImputer(strategy="most_frequent")),
|
|
519
|
+
("encode", OneHotEncoder(drop="first", sparse_output=False,
|
|
520
|
+
handle_unknown="ignore")),
|
|
521
|
+
]), ["RIVERSIDE"]),
|
|
522
|
+
], remainder="drop")
|
|
523
|
+
```
|
|
524
|
+
|
|
525
|
+
Each tuple is `(name, transformer, columns)`. Reusing a scaler in separate blocks is fine
|
|
526
|
+
and often required when columns need different transformations. For output order, the
|
|
527
|
+
blocks appear in the order listed.
|
|
528
|
+
|
|
529
|
+
### Full leakage-safe pipeline
|
|
530
|
+
|
|
531
|
+
```python
|
|
532
|
+
from sklearn.compose import ColumnTransformer
|
|
533
|
+
from sklearn.impute import SimpleImputer
|
|
534
|
+
from sklearn.pipeline import Pipeline
|
|
535
|
+
from sklearn.preprocessing import OneHotEncoder, StandardScaler
|
|
536
|
+
from sklearn.linear_model import Ridge
|
|
537
|
+
|
|
538
|
+
numeric_features = X.select_dtypes(include="number").columns.tolist()
|
|
539
|
+
categorical_features = X.select_dtypes(exclude="number").columns.tolist()
|
|
540
|
+
|
|
541
|
+
numeric_pipe = Pipeline([
|
|
542
|
+
("imputer", SimpleImputer(strategy="median")),
|
|
543
|
+
("scaler", StandardScaler()),
|
|
544
|
+
])
|
|
545
|
+
categorical_pipe = Pipeline([
|
|
546
|
+
("imputer", SimpleImputer(strategy="most_frequent")),
|
|
547
|
+
("encoder", OneHotEncoder(handle_unknown="ignore")),
|
|
548
|
+
])
|
|
549
|
+
preprocessor = ColumnTransformer([
|
|
550
|
+
("numeric", numeric_pipe, numeric_features),
|
|
551
|
+
("categorical", categorical_pipe, categorical_features),
|
|
552
|
+
])
|
|
553
|
+
model = Pipeline([
|
|
554
|
+
("preprocess", preprocessor),
|
|
555
|
+
("regressor", Ridge(alpha=1.0)),
|
|
556
|
+
])
|
|
557
|
+
model.fit(X_train, y_train)
|
|
558
|
+
predictions = model.predict(X_test)
|
|
559
|
+
```
|
|
560
|
+
|
|
561
|
+
A pipeline ensures that imputation, category discovery and scaling are refitted inside
|
|
562
|
+
each training fold during cross-validation. This prevents validation-fold leakage.
|
|
563
|
+
|
|
564
|
+
## 7. Regression estimators and metrics
|
|
565
|
+
|
|
566
|
+
### LinearRegression
|
|
567
|
+
|
|
568
|
+
```python
|
|
569
|
+
from sklearn.linear_model import LinearRegression
|
|
570
|
+
from sklearn.metrics import mean_absolute_error
|
|
571
|
+
|
|
572
|
+
model = LinearRegression()
|
|
573
|
+
model.fit(X_train, y_train)
|
|
574
|
+
y_pred = model.predict(X_test)
|
|
575
|
+
print(mean_absolute_error(y_test, y_pred))
|
|
576
|
+
print(model.score(X_test, y_test)) # R-squared for regressors
|
|
577
|
+
```
|
|
578
|
+
|
|
579
|
+
### Ridge and Lasso
|
|
580
|
+
|
|
581
|
+
```python
|
|
582
|
+
from sklearn.linear_model import Ridge, Lasso
|
|
583
|
+
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
|
|
584
|
+
|
|
585
|
+
ridge = Ridge(alpha=10, random_state=42).fit(X_train, y_train)
|
|
586
|
+
lasso = Lasso(alpha=100, random_state=42).fit(X_train, y_train)
|
|
587
|
+
|
|
588
|
+
ridge_pred = ridge.predict(X_test)
|
|
589
|
+
lasso_pred = lasso.predict(X_test)
|
|
590
|
+
print("Ridge MAE:", mean_absolute_error(y_test, ridge_pred))
|
|
591
|
+
print("Lasso MAE:", mean_absolute_error(y_test, lasso_pred))
|
|
592
|
+
print("Ridge R2:", r2_score(y_test, ridge_pred))
|
|
593
|
+
print("Lasso intercept:", lasso.intercept_)
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
`alpha` controls regularisation strength; larger values generally shrink coefficients
|
|
597
|
+
more strongly. Feature scaling is important for penalties that act on coefficient size.
|
|
598
|
+
|
|
599
|
+
### SGDRegressor
|
|
600
|
+
|
|
601
|
+
```python
|
|
602
|
+
from sklearn.linear_model import SGDRegressor
|
|
603
|
+
from sklearn.metrics import mean_squared_error, r2_score
|
|
604
|
+
|
|
605
|
+
sgd = SGDRegressor(random_state=42).fit(X_train, y_train)
|
|
606
|
+
y_pred = sgd.predict(X_test)
|
|
607
|
+
rmse = mean_squared_error(y_test, y_pred) ** 0.5
|
|
608
|
+
print("RMSE:", rmse)
|
|
609
|
+
print("R2:", r2_score(y_test, y_pred))
|
|
610
|
+
print("iterations used:", sgd.n_iter_)
|
|
611
|
+
|
|
612
|
+
# Specify the number of passes explicitly when requested:
|
|
613
|
+
sgd_limited = SGDRegressor(random_state=42, max_iter=25).fit(X_train, y_train)
|
|
614
|
+
print(sgd_limited.score(X_test, y_test))
|
|
615
|
+
```
|
|
616
|
+
|
|
617
|
+
`max_iter` is the maximum number of epochs. `n_iter_` reports the number actually used;
|
|
618
|
+
early stopping by the convergence tolerance may stop training before the maximum.
|
|
619
|
+
`warm_start=True` reuses the previous solution when fitting the same estimator again.
|
|
620
|
+
|
|
621
|
+
### Metric definitions
|
|
622
|
+
|
|
623
|
+
- **MAE**: average absolute prediction error; same units as the target.
|
|
624
|
+
- **MSE**: average squared prediction error; squared target units.
|
|
625
|
+
- **RMSE**: square root of MSE; same units as the target.
|
|
626
|
+
- **R²**: coefficient of determination; 1 is perfect, 0 is the constant-mean baseline,
|
|
627
|
+
and a negative value can occur when predictions are worse than that baseline.
|
|
628
|
+
|
|
629
|
+
```python
|
|
630
|
+
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
|
|
631
|
+
|
|
632
|
+
mae = mean_absolute_error(y_test, y_pred)
|
|
633
|
+
mse = mean_squared_error(y_test, y_pred)
|
|
634
|
+
rmse = np.sqrt(mse)
|
|
635
|
+
r2 = r2_score(y_test, y_pred)
|
|
636
|
+
```
|
|
637
|
+
|
|
638
|
+
### Coefficient values and feature indices
|
|
639
|
+
|
|
640
|
+
```python
|
|
641
|
+
absolute_coefficients = np.abs(model.coef_)
|
|
642
|
+
most_important_index = int(np.argmax(absolute_coefficients))
|
|
643
|
+
least_important_index = int(np.argmin(absolute_coefficients))
|
|
644
|
+
maximum_absolute_coefficient = float(absolute_coefficients.max())
|
|
645
|
+
|
|
646
|
+
# Intercept is separate and is not part of coef_:
|
|
647
|
+
print(model.intercept_)
|
|
648
|
+
|
|
649
|
+
# Map the index back to a feature name:
|
|
650
|
+
feature_names = np.asarray(X_train.columns)
|
|
651
|
+
print(feature_names[most_important_index])
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
This positional mapping is valid only if `coef_` and `feature_names` describe exactly the
|
|
655
|
+
same transformed feature order. After a `ColumnTransformer`, use
|
|
656
|
+
`preprocessor.get_feature_names_out()` rather than the original DataFrame columns.
|
|
657
|
+
|
|
658
|
+
## 8. Cross-validation
|
|
659
|
+
|
|
660
|
+
```python
|
|
661
|
+
from sklearn.model_selection import KFold, cross_val_score
|
|
662
|
+
from sklearn.linear_model import LinearRegression
|
|
663
|
+
|
|
664
|
+
cv = KFold(n_splits=5, shuffle=True, random_state=42)
|
|
665
|
+
scores = cross_val_score(
|
|
666
|
+
LinearRegression(), X_train, y_train,
|
|
667
|
+
cv=cv, scoring="r2", n_jobs=-1,
|
|
668
|
+
)
|
|
669
|
+
print(scores)
|
|
670
|
+
print("mean:", scores.mean())
|
|
671
|
+
print("minimum:", scores.min())
|
|
672
|
+
```
|
|
673
|
+
|
|
674
|
+
If no scoring argument is supplied, the estimator's default `score` is used. For
|
|
675
|
+
regressors this is usually R². Specify `scoring="r2"` when you want the intent to be
|
|
676
|
+
unambiguous. `KFold(shuffle=True, random_state=...)` provides repeatable shuffled folds.
|
|
677
|
+
|
|
678
|
+
## 9. GridSearchCV and parameter tuning
|
|
679
|
+
|
|
680
|
+
### Tune an SGDRegressor
|
|
681
|
+
|
|
682
|
+
```python
|
|
683
|
+
from sklearn.linear_model import SGDRegressor
|
|
684
|
+
from sklearn.model_selection import GridSearchCV
|
|
685
|
+
|
|
686
|
+
parameter_grid = {
|
|
687
|
+
"penalty": ["l1", "l2", "elasticnet"],
|
|
688
|
+
"alpha": [1e-5, 1e-4, 1e-3, 1e-2, 1e-1],
|
|
689
|
+
"tol": [1e-3, 1e-2],
|
|
690
|
+
}
|
|
691
|
+
search = GridSearchCV(
|
|
692
|
+
estimator=SGDRegressor(random_state=42),
|
|
693
|
+
param_grid=parameter_grid,
|
|
694
|
+
cv=5,
|
|
695
|
+
scoring="neg_mean_squared_error",
|
|
696
|
+
refit=True,
|
|
697
|
+
n_jobs=-1,
|
|
698
|
+
)
|
|
699
|
+
search.fit(X_train, y_train)
|
|
700
|
+
print(search.best_params_)
|
|
701
|
+
best_model = search.best_estimator_
|
|
702
|
+
y_pred = best_model.predict(X_test)
|
|
703
|
+
print(np.sqrt(mean_squared_error(y_test, y_pred)))
|
|
704
|
+
```
|
|
705
|
+
|
|
706
|
+
Scikit-learn maximises scores. Loss metrics are therefore exposed as negative values in
|
|
707
|
+
cross-validation scoring, for example `neg_mean_absolute_error` and
|
|
708
|
+
`neg_mean_squared_error`. The negative sign is a scoring convention; convert to a
|
|
709
|
+
positive MAE/RMSE for final reporting.
|
|
710
|
+
|
|
711
|
+
`best_params_` is a dictionary. `best_estimator_` is the refitted estimator when
|
|
712
|
+
`refit=True`. `best_score_` is the cross-validation score in the scoring convention,
|
|
713
|
+
so it can be negative for error metrics.
|
|
714
|
+
|
|
715
|
+
### Tune a pipeline
|
|
716
|
+
|
|
717
|
+
Use `step_name__parameter_name` to tune nested parameters.
|
|
718
|
+
|
|
719
|
+
```python
|
|
720
|
+
from sklearn.decomposition import PCA
|
|
721
|
+
from sklearn.linear_model import Lasso
|
|
722
|
+
from sklearn.pipeline import Pipeline
|
|
723
|
+
from sklearn.model_selection import GridSearchCV
|
|
724
|
+
|
|
725
|
+
pipe = Pipeline([
|
|
726
|
+
("pca", PCA()),
|
|
727
|
+
("lasso", Lasso()),
|
|
728
|
+
])
|
|
729
|
+
parameter_grid = {
|
|
730
|
+
"pca__n_components": [0.85, 0.95],
|
|
731
|
+
"lasso__alpha": [10, 1, 0.1, 0.01],
|
|
732
|
+
}
|
|
733
|
+
search = GridSearchCV(
|
|
734
|
+
pipe, parameter_grid,
|
|
735
|
+
scoring="neg_mean_absolute_error",
|
|
736
|
+
cv=5,
|
|
737
|
+
n_jobs=-1,
|
|
738
|
+
refit=True,
|
|
739
|
+
)
|
|
740
|
+
search.fit(X_train, y_train)
|
|
741
|
+
print(search.best_params_)
|
|
742
|
+
y_pred = search.predict(X_test)
|
|
743
|
+
print(mean_absolute_error(y_test, y_pred))
|
|
744
|
+
```
|
|
745
|
+
|
|
746
|
+
A float `PCA(n_components=0.95)` means retain enough components to explain at least 95%
|
|
747
|
+
of training variance. It does not mean retain 0.95 components. The fitted integer count is
|
|
748
|
+
available as `search.best_estimator_.named_steps["pca"].n_components_`.
|
|
749
|
+
|
|
750
|
+
## 10. PCA and RidgeCV
|
|
751
|
+
|
|
752
|
+
```python
|
|
753
|
+
from sklearn.decomposition import PCA
|
|
754
|
+
from sklearn.linear_model import RidgeCV
|
|
755
|
+
|
|
756
|
+
pca = PCA(n_components=7, svd_solver="full", whiten=True, random_state=42)
|
|
757
|
+
X_train_pca = pca.fit_transform(X_train)
|
|
758
|
+
X_test_pca = pca.transform(X_test)
|
|
759
|
+
print(pca.explained_variance_ratio_.sum())
|
|
760
|
+
|
|
761
|
+
ridge_cv = RidgeCV(alphas=[0.01, 0.1, 1, 10, 100], cv=5)
|
|
762
|
+
ridge_cv.fit(X_train_pca, y_train)
|
|
763
|
+
print(ridge_cv.alpha_)
|
|
764
|
+
print(ridge_cv.score(X_test_pca, y_test))
|
|
765
|
+
```
|
|
766
|
+
|
|
767
|
+
Fit PCA on training features only; call `transform` on test features. Do not fit a second
|
|
768
|
+
PCA model on test data.
|
|
769
|
+
|
|
770
|
+
Important fitted attributes:
|
|
771
|
+
|
|
772
|
+
- `explained_variance_ratio_`: fraction of total variance represented by each component.
|
|
773
|
+
- `explained_variance_ratio_.sum()`: cumulative fraction represented by retained components.
|
|
774
|
+
- `explained_variance_[0]`: eigenvalue/variance of the first principal component.
|
|
775
|
+
- `n_components_`: actual number of components selected by a variance-fraction setting.
|
|
776
|
+
- `components_`: component directions in the original feature space.
|
|
777
|
+
|
|
778
|
+
A fraction such as `0.83` is 83%. If a request asks for the eigenvalue, use
|
|
779
|
+
`explained_variance_[0]`; do not substitute `explained_variance_ratio_[0]`.
|
|
780
|
+
|
|
781
|
+
PCA is scale-sensitive. If features have different units or magnitudes, standardise them
|
|
782
|
+
before PCA unless the specified workflow says otherwise.
|
|
783
|
+
|
|
784
|
+
## 11. PolynomialFeatures with Lasso
|
|
785
|
+
|
|
786
|
+
```python
|
|
787
|
+
from sklearn.preprocessing import PolynomialFeatures
|
|
788
|
+
from sklearn.linear_model import Lasso
|
|
789
|
+
from sklearn.pipeline import Pipeline
|
|
790
|
+
from sklearn.metrics import mean_squared_error, r2_score
|
|
791
|
+
|
|
792
|
+
poly_model = Pipeline([
|
|
793
|
+
("poly", PolynomialFeatures(degree=2, interaction_only=False)),
|
|
794
|
+
("lasso", Lasso(alpha=2, warm_start=True, random_state=0)),
|
|
795
|
+
])
|
|
796
|
+
poly_model.fit(X_train, y_train)
|
|
797
|
+
y_pred = poly_model.predict(X_test)
|
|
798
|
+
print("RMSE:", np.sqrt(mean_squared_error(y_test, y_pred)))
|
|
799
|
+
print("R2:", r2_score(y_test, y_pred))
|
|
800
|
+
```
|
|
801
|
+
|
|
802
|
+
`degree=2` adds squared terms and pairwise interactions when `interaction_only=False`.
|
|
803
|
+
With `interaction_only=True`, only products of distinct original features are created;
|
|
804
|
+
squared terms are excluded. Polynomial expansion can greatly increase the feature count.
|
|
805
|
+
|
|
806
|
+
Grid-search degree and alpha together:
|
|
807
|
+
|
|
808
|
+
```python
|
|
809
|
+
parameter_grid = {
|
|
810
|
+
"poly__degree": [1, 2],
|
|
811
|
+
"lasso__alpha": np.logspace(-2, 1, num=4),
|
|
812
|
+
}
|
|
813
|
+
search = GridSearchCV(
|
|
814
|
+
Pipeline([
|
|
815
|
+
("poly", PolynomialFeatures()),
|
|
816
|
+
("lasso", Lasso()),
|
|
817
|
+
]),
|
|
818
|
+
parameter_grid,
|
|
819
|
+
scoring="neg_mean_squared_error",
|
|
820
|
+
cv=5,
|
|
821
|
+
n_jobs=-1,
|
|
822
|
+
)
|
|
823
|
+
search.fit(X_train, y_train)
|
|
824
|
+
print(search.best_params_)
|
|
825
|
+
print(search.score(X_test, y_test))
|
|
826
|
+
```
|
|
827
|
+
|
|
828
|
+
Use a pipeline so each fold fits the polynomial expansion and estimator together.
|
|
829
|
+
|
|
830
|
+
## 12. Feature selection
|
|
831
|
+
|
|
832
|
+
### SelectKBest with f_regression
|
|
833
|
+
|
|
834
|
+
```python
|
|
835
|
+
from sklearn.feature_selection import SelectKBest, f_regression
|
|
836
|
+
from sklearn.linear_model import SGDRegressor
|
|
837
|
+
from sklearn.pipeline import Pipeline
|
|
838
|
+
|
|
839
|
+
selection_model = Pipeline([
|
|
840
|
+
("select", SelectKBest(score_func=f_regression, k=4)),
|
|
841
|
+
("regressor", SGDRegressor(random_state=42)),
|
|
842
|
+
])
|
|
843
|
+
selection_model.fit(X_train, y_train)
|
|
844
|
+
selector = selection_model.named_steps["select"]
|
|
845
|
+
selected_mask = selector.get_support()
|
|
846
|
+
selected_names = np.asarray(X_train.columns)[selected_mask]
|
|
847
|
+
selected_indices = np.flatnonzero(selected_mask)
|
|
848
|
+
ranked_indices = np.argsort(selector.scores_)[::-1][:4]
|
|
849
|
+
ranked_names = np.asarray(X_train.columns)[ranked_indices]
|
|
850
|
+
print(ranked_names)
|
|
851
|
+
print(selector.scores_)
|
|
852
|
+
print(selector.pvalues_)
|
|
853
|
+
print((selector.pvalues_ < 0.05).sum())
|
|
854
|
+
```
|
|
855
|
+
|
|
856
|
+
`get_support()` reflects selected columns. `scores_` and `pvalues_` are aligned to the
|
|
857
|
+
original input feature order, not only the selected subset. Sort with `np.argsort` to
|
|
858
|
+
report selected columns in descending score order. Handle `NaN` scores if a feature is
|
|
859
|
+
constant or the statistic is undefined.
|
|
860
|
+
|
|
861
|
+
### SequentialFeatureSelector
|
|
862
|
+
|
|
863
|
+
```python
|
|
864
|
+
from sklearn.feature_selection import SequentialFeatureSelector
|
|
865
|
+
from sklearn.linear_model import LinearRegression
|
|
866
|
+
from sklearn.model_selection import KFold
|
|
867
|
+
|
|
868
|
+
cv = KFold(n_splits=5, shuffle=True, random_state=42)
|
|
869
|
+
sfs = SequentialFeatureSelector(
|
|
870
|
+
LinearRegression(),
|
|
871
|
+
n_features_to_select=6,
|
|
872
|
+
direction="forward",
|
|
873
|
+
cv=cv,
|
|
874
|
+
n_jobs=-1,
|
|
875
|
+
)
|
|
876
|
+
sfs.fit(X_train, y_train)
|
|
877
|
+
print(np.flatnonzero(sfs.get_support()))
|
|
878
|
+
print(np.asarray(X_train.columns)[sfs.get_support()])
|
|
879
|
+
```
|
|
880
|
+
|
|
881
|
+
Forward selection starts with no features and adds one feature at a time according to the
|
|
882
|
+
cross-validation score. Backward selection starts with all features and removes them.
|
|
883
|
+
|
|
884
|
+
### Recursive Feature Elimination
|
|
885
|
+
|
|
886
|
+
```python
|
|
887
|
+
from sklearn.feature_selection import RFE
|
|
888
|
+
from sklearn.linear_model import LinearRegression
|
|
889
|
+
|
|
890
|
+
rfe = RFE(
|
|
891
|
+
estimator=LinearRegression(),
|
|
892
|
+
n_features_to_select=X_train.shape[1] - 2,
|
|
893
|
+
)
|
|
894
|
+
rfe.fit(X_train, y_train)
|
|
895
|
+
eliminated_indices = np.flatnonzero(~rfe.support_)
|
|
896
|
+
eliminated_names = np.asarray(X_train.columns)[~rfe.support_]
|
|
897
|
+
print(eliminated_indices)
|
|
898
|
+
print(eliminated_names)
|
|
899
|
+
print(rfe.ranking_)
|
|
900
|
+
```
|
|
901
|
+
|
|
902
|
+
`support_` is `True` for retained columns. The eliminated positions are
|
|
903
|
+
`np.flatnonzero(~rfe.support_)`. `ranking_ == 1` means selected; larger values indicate
|
|
904
|
+
how early a feature was eliminated. The estimator must be compatible with the data.
|
|
905
|
+
|
|
906
|
+
## 13. Time and text feature engineering
|
|
907
|
+
|
|
908
|
+
### Normalize inconsistent strings
|
|
909
|
+
|
|
910
|
+
```python
|
|
911
|
+
df["Additional_Info"] = df["Additional_Info"].replace({"No Info": "No info"})
|
|
912
|
+
count = ((df["Airline"] == "Jet Airways") &
|
|
913
|
+
(df["Additional_Info"] == "No info")).sum()
|
|
914
|
+
```
|
|
915
|
+
|
|
916
|
+
### Parse duration text into minutes
|
|
917
|
+
|
|
918
|
+
```python
|
|
919
|
+
import re
|
|
920
|
+
|
|
921
|
+
def duration_to_minutes(value):
|
|
922
|
+
if pd.isna(value):
|
|
923
|
+
return np.nan
|
|
924
|
+
text = str(value).strip()
|
|
925
|
+
hours = re.search(r"(\d+(?:\.\d+)?)\s*h", text, flags=re.I)
|
|
926
|
+
minutes = re.search(r"(\d+(?:\.\d+)?)\s*m", text, flags=re.I)
|
|
927
|
+
h = float(hours.group(1)) if hours else 0.0
|
|
928
|
+
m = float(minutes.group(1)) if minutes else 0.0
|
|
929
|
+
return 60 * h + m
|
|
930
|
+
|
|
931
|
+
df["Duration"] = df["Duration"].map(duration_to_minutes)
|
|
932
|
+
print(df["Duration"].median())
|
|
933
|
+
```
|
|
934
|
+
|
|
935
|
+
This accepts values such as `"27h"`, `"1h 30m"` and `"50m"`. Missing values stay
|
|
936
|
+
missing. Check unusual input strings before using this in production.
|
|
937
|
+
|
|
938
|
+
### Convert clock times into day parts
|
|
939
|
+
|
|
940
|
+
```python
|
|
941
|
+
def time_of_day(value):
|
|
942
|
+
if pd.isna(value):
|
|
943
|
+
return np.nan
|
|
944
|
+
hour = int(str(value).split(":", 1)[0])
|
|
945
|
+
if 5 <= hour < 12:
|
|
946
|
+
return "Morning"
|
|
947
|
+
if 12 <= hour < 17:
|
|
948
|
+
return "Afternoon"
|
|
949
|
+
if 17 <= hour < 20:
|
|
950
|
+
return "Evening"
|
|
951
|
+
return "Night" # 20:00 through 04:59
|
|
952
|
+
|
|
953
|
+
df["Dep_Time"] = df["Dep_Time"].map(time_of_day)
|
|
954
|
+
df["Arrival_Time"] = df["Arrival_Time"].map(time_of_day)
|
|
955
|
+
print(pd.crosstab(df["Dep_Time"], df["Arrival_Time"]))
|
|
956
|
+
evening_to_night = (
|
|
957
|
+
df["Dep_Time"].eq("Evening") & df["Arrival_Time"].eq("Night")
|
|
958
|
+
).sum()
|
|
959
|
+
```
|
|
960
|
+
|
|
961
|
+
### Encode weekends
|
|
962
|
+
|
|
963
|
+
```python
|
|
964
|
+
df["WeekDay"] = df["WeekDay"].isin(["Saturday", "Sunday"]).astype("int8")
|
|
965
|
+
weekend_count = int(df["WeekDay"].sum())
|
|
966
|
+
```
|
|
967
|
+
|
|
968
|
+
### Ordinal encoding with unseen categories
|
|
969
|
+
|
|
970
|
+
```python
|
|
971
|
+
from sklearn.preprocessing import OrdinalEncoder
|
|
972
|
+
|
|
973
|
+
stop_order = ["non-stop", "1 stop", "2 stops", "3 stops", "4 stops"]
|
|
974
|
+
time_order = ["Morning", "Afternoon", "Evening", "Night"]
|
|
975
|
+
encoder = OrdinalEncoder(
|
|
976
|
+
categories=[stop_order, time_order],
|
|
977
|
+
handle_unknown="use_encoded_value",
|
|
978
|
+
unknown_value=-1,
|
|
979
|
+
)
|
|
980
|
+
```
|
|
981
|
+
|
|
982
|
+
If `handle_unknown="use_encoded_value"`, count unknown entries after transformation
|
|
983
|
+
rather than assuming all values were known.
|
|
984
|
+
|
|
985
|
+
## 14. Numerical answer formatting
|
|
986
|
+
|
|
987
|
+
Some graders require ordinary rounding; others explicitly require truncation. Use the
|
|
988
|
+
instruction exactly.
|
|
989
|
+
|
|
990
|
+
```python
|
|
991
|
+
from decimal import Decimal, ROUND_DOWN, ROUND_HALF_UP
|
|
992
|
+
|
|
993
|
+
def format_number(value, decimals=3, *, truncate=False):
|
|
994
|
+
quant = Decimal("1").scaleb(-decimals)
|
|
995
|
+
number = Decimal(str(float(value)))
|
|
996
|
+
mode = ROUND_DOWN if truncate else ROUND_HALF_UP
|
|
997
|
+
return format(number.quantize(quant, rounding=mode), f".{decimals}f")
|
|
998
|
+
|
|
999
|
+
print(format_number(3.1415926, 3)) # 3.142
|
|
1000
|
+
print(format_number(3.1415926, 3, truncate=True)) # 3.141
|
|
1001
|
+
```
|
|
1002
|
+
|
|
1003
|
+
For arrays, first extract the scalar you need. `np.round` rounds; `np.trunc` truncates
|
|
1004
|
+
toward zero. Do not silently apply a different precision rule to the value than the one
|
|
1005
|
+
requested.
|
|
1006
|
+
|
|
1007
|
+
## 15. Common errors and how to diagnose them
|
|
1008
|
+
|
|
1009
|
+
- **Wrong row count after splitting:** verify total rows, `test_size`, seed, stratification,
|
|
1010
|
+
and whether the split should be random, chronological or positional.
|
|
1011
|
+
- **Missing-value counts disagree:** inspect `unique()` and `value_counts(dropna=False)`;
|
|
1012
|
+
`isna()` does not know that `-1` or `"UNKNOWN"` is a sentinel.
|
|
1013
|
+
- **Unknown category error:** use `handle_unknown="ignore"` for one-hot encoding or
|
|
1014
|
+
`handle_unknown="use_encoded_value", unknown_value=-1` for ordinal encoding.
|
|
1015
|
+
- **Different transformed column count:** check category cardinalities, `drop`, input
|
|
1016
|
+
column selection, `remainder`, and the order of the `ColumnTransformer` blocks.
|
|
1017
|
+
- **Feature index maps to the wrong name:** obtain names from the exact transformed
|
|
1018
|
+
matrix (`get_feature_names_out`) and preserve its output order.
|
|
1019
|
+
- **PCA component count seems wrong:** if `n_components` is a float, inspect the fitted
|
|
1020
|
+
`n_components_`; the float is an explained-variance threshold.
|
|
1021
|
+
- **Grid-search score is negative:** error scorers are negated so that higher is better.
|
|
1022
|
+
The final MAE/RMSE should be recomputed as a positive metric.
|
|
1023
|
+
- **Model results differ:** confirm data version, row order, split seed, preprocessing
|
|
1024
|
+
fit scope, transformer order, model parameters and scikit-learn version.
|
|
1025
|
+
- **Convergence warning:** inspect scale, `max_iter`, `tol` and solver settings. Do not
|
|
1026
|
+
change parameters that were explicitly fixed by a reproducibility requirement.
|
|
1027
|
+
|
|
1028
|
+
## 16. Full scikit-learn documentation and API reference
|
|
1029
|
+
|
|
1030
|
+
This package does not freeze or redistribute a copy of the entire official documentation.
|
|
1031
|
+
Instead, use its local `api()` function to read the live docstrings from the scikit-learn
|
|
1032
|
+
version installed in the active notebook. This makes parameter descriptions, signatures,
|
|
1033
|
+
attributes and version-specific details match the runtime.
|
|
1034
|
+
|
|
1035
|
+
```python
|
|
1036
|
+
import scikit_doc as skdoc
|
|
1037
|
+
|
|
1038
|
+
print(skdoc.api("sklearn.preprocessing.StandardScaler"))
|
|
1039
|
+
print(skdoc.api("sklearn.compose.ColumnTransformer"))
|
|
1040
|
+
print(skdoc.api("sklearn.model_selection.GridSearchCV"))
|
|
1041
|
+
print(skdoc.api("sklearn.linear_model.Ridge"))
|
|
1042
|
+
print(skdoc.api_catalog(module="preprocessing"))
|
|
1043
|
+
# Compile the complete public API docstrings from the installed version:
|
|
1044
|
+
reference_text = skdoc.api_reference()
|
|
1045
|
+
# Or keep the output focused:
|
|
1046
|
+
preprocessing_reference = skdoc.api_reference(module="preprocessing")
|
|
1047
|
+
```
|
|
1048
|
+
|
|
1049
|
+
The official documentation is the complete authoritative reference:
|
|
1050
|
+
|
|
1051
|
+
- User Guide: https://scikit-learn.org/stable/user_guide.html
|
|
1052
|
+
- API Reference: https://scikit-learn.org/stable/api/
|
|
1053
|
+
- Examples: https://scikit-learn.org/stable/auto_examples/index.html
|
|
1054
|
+
- Model evaluation: https://scikit-learn.org/stable/modules/model_evaluation.html
|
|
1055
|
+
- Preprocessing: https://scikit-learn.org/stable/modules/preprocessing.html
|
|
1056
|
+
- Composition and pipelines: https://scikit-learn.org/stable/modules/compose.html
|
|
1057
|
+
- Feature selection: https://scikit-learn.org/stable/modules/feature_selection.html
|
|
1058
|
+
- Decomposition and PCA: https://scikit-learn.org/stable/modules/decomposition.html
|
|
1059
|
+
- Linear models: https://scikit-learn.org/stable/modules/linear_model.html
|
|
1060
|
+
|
|
1061
|
+
The installed scikit-learn API surface is larger than any focused study guide. The
|
|
1062
|
+
`api()` helper returns one installed-version docstring, `api_catalog()` indexes public
|
|
1063
|
+
objects, and `api_reference()` compiles the live public API documentation for all discovered
|
|
1064
|
+
modules (or a selected module/query). The official links provide the complete and current
|
|
1065
|
+
user guide, API reference, examples and release notes. The entire official website is not
|
|
1066
|
+
copied into this distribution; its maintained version remains available at those links.
|
|
1067
|
+
|
|
1068
|
+
## 17. Included worked-example collection
|
|
1069
|
+
|
|
1070
|
+
The package bundles a catalog generated from every supplied notebook and the companion
|
|
1071
|
+
text source. Examples are reorganized by topic, with source-specific labels removed:
|
|
1072
|
+
|
|
1073
|
+
- Dataset dimensions, data types, unique categories, missing-value profiles and numeric
|
|
1074
|
+
feature discovery.
|
|
1075
|
+
- Correlation pairs, heatmaps, conditional counts, grouped means, ranges and category bins.
|
|
1076
|
+
- Outlier and sentinel detection for numeric and categorical fields.
|
|
1077
|
+
- Random, chronological and fixed-position splitting.
|
|
1078
|
+
- Train-only mean/median/mode imputation and missing-value audits.
|
|
1079
|
+
- Standard, min-max, one-hot and ordinal transformations, including explicit category
|
|
1080
|
+
order, unseen values and transformed matrix summaries.
|
|
1081
|
+
- Flight duration parsing, day-part conversion, cross-tabulation and weekend indicators.
|
|
1082
|
+
- LinearRegression, Ridge, Lasso and SGDRegressor evaluation and fitted attributes.
|
|
1083
|
+
- Cross-validation, GridSearchCV, PCA/Lasso, polynomial/Lasso, RidgeCV and RFE/SFS/
|
|
1084
|
+
SelectKBest workflows.
|
|
1085
|
+
- Customer, housing, property-sale, flight-price and premium-estimation examples.
|
|
1086
|
+
|
|
1087
|
+
Use `scikit_doc.examples()` to list scripts and `scikit_doc.example("filename.py")` to read one.
|
|
1088
|
+
Use `scikit_doc.guide("PCA")` to extract the relevant part of this handbook.
|