scikit-tools 0.1.1__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- scikit_tools-0.2.0/PKG-INFO +61 -0
- scikit_tools-0.2.0/README.md +32 -0
- scikit_tools-0.2.0/SCIKIT_TOOLS.md +47 -0
- scikit_tools-0.2.0/pyproject.toml +22 -0
- scikit_tools-0.2.0/scikit_tools/__init__.py +1 -0
- scikit_tools-0.2.0/scikit_tools/correlation_covariance.py +19 -0
- scikit_tools-0.2.0/scikit_tools/data_transformation.py +26 -0
- scikit_tools-0.2.0/scikit_tools/descriptive_stats.py +24 -0
- scikit_tools-0.2.0/scikit_tools/eda_plots.py +34 -0
- scikit_tools-0.2.0/scikit_tools/hypothesis_testing.py +22 -0
- scikit_tools-0.2.0/scikit_tools/inspect_dataset.py +28 -0
- scikit_tools-0.2.0/scikit_tools/kmeans_clustering.py +29 -0
- scikit_tools-0.2.0/scikit_tools/linear_regression.py +32 -0
- scikit_tools-0.2.0/scikit_tools/logistic_regression.py +30 -0
- scikit_tools-0.2.0/scikit_tools/missing_values.py +33 -0
- scikit_tools-0.2.0/scikit_tools/numpy_stats.py +31 -0
- scikit_tools-0.2.0/scikit_tools/outlier_detection.py +39 -0
- scikit_tools-0.2.0/scikit_tools/pca.py +39 -0
- scikit_tools-0.2.0/scikit_tools.egg-info/PKG-INFO +61 -0
- scikit_tools-0.2.0/scikit_tools.egg-info/SOURCES.txt +24 -0
- {scikit_tools-0.1.1/src → scikit_tools-0.2.0}/scikit_tools.egg-info/requires.txt +5 -4
- {scikit_tools-0.1.1 → scikit_tools-0.2.0}/setup.cfg +4 -4
- scikit_tools-0.2.0/tests/test.py +31 -0
- scikit_tools-0.2.0/tests/test_scikit_tools.py +67 -0
- scikit_tools-0.1.1/PKG-INFO +0 -19
- scikit_tools-0.1.1/README.md +0 -2
- scikit_tools-0.1.1/pyproject.toml +0 -29
- scikit_tools-0.1.1/src/scikit_tools/__init__.py +0 -1
- scikit_tools-0.1.1/src/scikit_tools/solutions.py +0 -205
- scikit_tools-0.1.1/src/scikit_tools.egg-info/PKG-INFO +0 -19
- scikit_tools-0.1.1/src/scikit_tools.egg-info/SOURCES.txt +0 -9
- {scikit_tools-0.1.1/src → scikit_tools-0.2.0}/scikit_tools.egg-info/dependency_links.txt +0 -0
- {scikit_tools-0.1.1/src → scikit_tools-0.2.0}/scikit_tools.egg-info/top_level.txt +0 -0
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: scikit-tools
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Thirteen runnable, straight-through scikit-learn and data-analysis examples
|
|
5
|
+
Requires-Python: >=3.10
|
|
6
|
+
Description-Content-Type: text/markdown
|
|
7
|
+
Requires-Dist: ipython
|
|
8
|
+
Requires-Dist: matplotlib
|
|
9
|
+
Requires-Dist: numpy
|
|
10
|
+
Requires-Dist: pandas
|
|
11
|
+
Requires-Dist: scikit-learn
|
|
12
|
+
Requires-Dist: scipy
|
|
13
|
+
Requires-Dist: seaborn
|
|
14
|
+
|
|
15
|
+
# scikit-tools
|
|
16
|
+
|
|
17
|
+
`scikit-tools` provides 13 standalone, straight-through Python examples. Each
|
|
18
|
+
topic file contains its own imports, dataset setup, and analysis code; examples
|
|
19
|
+
are not hidden inside functions.
|
|
20
|
+
|
|
21
|
+
## Install from a local checkout
|
|
22
|
+
|
|
23
|
+
```console
|
|
24
|
+
python -m pip install -e .
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
This installs the package and its dependencies. Each topic loads the Iris
|
|
28
|
+
dataset provided by `sklearn.datasets.load_iris`, formats its columns to match
|
|
29
|
+
the examples, and does not download or write a CSV file.
|
|
30
|
+
|
|
31
|
+
## Open or run a topic
|
|
32
|
+
|
|
33
|
+
```console
|
|
34
|
+
python -m scikit_tools.numpy_stats
|
|
35
|
+
python -m scikit_tools.pca
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
You can also open any topic script under `scikit_tools` directly in your
|
|
39
|
+
editor, inspect or copy its raw code, and run it with the editor's Python run
|
|
40
|
+
command. These are scripts: importing one executes its example.
|
|
41
|
+
|
|
42
|
+
## Topics
|
|
43
|
+
|
|
44
|
+
1. NumPy statistics (`numpy_stats.py`)
|
|
45
|
+
2. Inspecting the Iris dataset (`inspect_dataset.py`)
|
|
46
|
+
3. Handling missing values (`missing_values.py`)
|
|
47
|
+
4. Data transformation (`data_transformation.py`)
|
|
48
|
+
5. Histogram and scatter-plot EDA (`eda_plots.py`)
|
|
49
|
+
6. Box plot and IQR outlier detection (`outlier_detection.py`)
|
|
50
|
+
7. Descriptive statistics (`descriptive_stats.py`)
|
|
51
|
+
8. Correlation and covariance (`correlation_covariance.py`)
|
|
52
|
+
9. Independent-samples t-test (`hypothesis_testing.py`)
|
|
53
|
+
10. Linear regression (`linear_regression.py`)
|
|
54
|
+
11. Logistic-regression classification (`logistic_regression.py`)
|
|
55
|
+
12. K-means clustering (`kmeans_clustering.py`)
|
|
56
|
+
13. PCA and classification (`pca.py`)
|
|
57
|
+
|
|
58
|
+
The examples use the Iris columns `SepalLengthCm`, `SepalWidthCm`,
|
|
59
|
+
`PetalLengthCm`, `PetalWidthCm`, and `Species`. The species names use the
|
|
60
|
+
`Iris-setosa` style. The missing values example intentionally adds missing values to a DataFrame
|
|
61
|
+
copy and fills them; it does not change the source dataset.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# IEEEXtreme Python Toolkit & OA / DSA Practice
|
|
2
|
+
|
|
3
|
+
A reusable, contest-oriented reference library and practice repository for advanced DSA, Online Assessments (OA), and competition simulations.
|
|
4
|
+
|
|
5
|
+
## DSA & OA Practice Problems
|
|
6
|
+
|
|
7
|
+
| # | Problem | Key Topics & Algorithms | Solutions |
|
|
8
|
+
|---|---|---|---|
|
|
9
|
+
| 01 | [Tree Vector Norm Maximization](problems/tree_vector_norm/EXPLANATION.md) | Tree DP, Convex Duality, QuickHull, Branch-and-Bound Pruning | [Python](problems/tree_vector_norm/solution.py), [C++](problems/tree_vector_norm/solution.cpp), [Tests](problems/tree_vector_norm/test_solution.py) |
|
|
10
|
+
|
|
11
|
+
## Toolkit Contents
|
|
12
|
+
|
|
13
|
+
- `template.py` — compact starting template suitable for copying into an editor.
|
|
14
|
+
- `io_utils.py` — fast token input, parsing, and output helpers.
|
|
15
|
+
- `sorting_searching.py`, `two_pointers.py` — common array patterns.
|
|
16
|
+
- `data_structures.py` — DSU, Fenwick and segment trees, trie, monotonic structures.
|
|
17
|
+
- `graphs.py`, `advanced_graphs.py` — traversals, shortest paths, MST, SCC, bridges, LCA, Euler tours.
|
|
18
|
+
- `dynamic_programming.py`, `number_theory.py`, `combinatorics.py`, `strings.py`, `geometry.py`, `bitwise.py`, `backtracking.py`.
|
|
19
|
+
- `docs/` — API notes, assumptions, and complexity guide.
|
|
20
|
+
- `tests/` — unit tests for representative edge cases and algorithms.
|
|
21
|
+
- `category_templates.py` — focused patterns mapped to every category visible in the supplied screenshot.
|
|
22
|
+
- `docs/category-map.md` — category-to-snippet index, archive research notes, and platform-use boundary.
|
|
23
|
+
|
|
24
|
+
## Use
|
|
25
|
+
|
|
26
|
+
Copy `template.py` as the actual contest starting file, then add only the needed algorithm snippets. Import library code locally while practicing, for example `from graphs import dijkstra`. Some contest editors accept only one submitted file, so copy required functions/classes into that file when needed. No package installation is required.
|
|
27
|
+
|
|
28
|
+
Run checks from this directory with `python -m unittest discover -s tests -v`.
|
|
29
|
+
|
|
30
|
+
## Notes
|
|
31
|
+
|
|
32
|
+
These are general algorithms, not task-specific solutions. Check each docstring for indexing, weight, and input assumptions. Python integers are arbitrary precision. Graph vertices and sequence positions are zero-indexed unless stated otherwise. Dijkstra requires nonnegative edge weights. The library is for local reference and practice; platform upload availability and competition rules are determined by the platform/organizers.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# scikit-tools
|
|
2
|
+
|
|
3
|
+
`scikit-tools` provides 13 standalone, straight-through Python examples. Each
|
|
4
|
+
topic file contains its own imports, dataset setup, and analysis code; examples
|
|
5
|
+
are not hidden inside functions.
|
|
6
|
+
|
|
7
|
+
## Install from a local checkout
|
|
8
|
+
|
|
9
|
+
```console
|
|
10
|
+
python -m pip install -e .
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
This installs the package and its dependencies. Each topic loads the Iris
|
|
14
|
+
dataset provided by `sklearn.datasets.load_iris`, formats its columns to match
|
|
15
|
+
the examples, and does not download or write a CSV file.
|
|
16
|
+
|
|
17
|
+
## Open or run a topic
|
|
18
|
+
|
|
19
|
+
```console
|
|
20
|
+
python -m scikit_tools.numpy_stats
|
|
21
|
+
python -m scikit_tools.pca
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
You can also open any topic script under `scikit_tools` directly in your
|
|
25
|
+
editor, inspect or copy its raw code, and run it with the editor's Python run
|
|
26
|
+
command. These are scripts: importing one executes its example.
|
|
27
|
+
|
|
28
|
+
## Topics
|
|
29
|
+
|
|
30
|
+
1. NumPy statistics (`numpy_stats.py`)
|
|
31
|
+
2. Inspecting the Iris dataset (`inspect_dataset.py`)
|
|
32
|
+
3. Handling missing values (`missing_values.py`)
|
|
33
|
+
4. Data transformation (`data_transformation.py`)
|
|
34
|
+
5. Histogram and scatter-plot EDA (`eda_plots.py`)
|
|
35
|
+
6. Box plot and IQR outlier detection (`outlier_detection.py`)
|
|
36
|
+
7. Descriptive statistics (`descriptive_stats.py`)
|
|
37
|
+
8. Correlation and covariance (`correlation_covariance.py`)
|
|
38
|
+
9. Independent-samples t-test (`hypothesis_testing.py`)
|
|
39
|
+
10. Linear regression (`linear_regression.py`)
|
|
40
|
+
11. Logistic-regression classification (`logistic_regression.py`)
|
|
41
|
+
12. K-means clustering (`kmeans_clustering.py`)
|
|
42
|
+
13. PCA and classification (`pca.py`)
|
|
43
|
+
|
|
44
|
+
The examples use the Iris columns `SepalLengthCm`, `SepalWidthCm`,
|
|
45
|
+
`PetalLengthCm`, `PetalWidthCm`, and `Species`. The species names use the
|
|
46
|
+
`Iris-setosa` style. The missing values example intentionally adds missing values to a DataFrame
|
|
47
|
+
copy and fills them; it does not change the source dataset.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "scikit-tools"
|
|
7
|
+
version = "0.2.0"
|
|
8
|
+
description = "Thirteen runnable, straight-through scikit-learn and data-analysis examples"
|
|
9
|
+
readme = "SCIKIT_TOOLS.md"
|
|
10
|
+
requires-python = ">=3.10"
|
|
11
|
+
dependencies = [
|
|
12
|
+
"ipython",
|
|
13
|
+
"matplotlib",
|
|
14
|
+
"numpy",
|
|
15
|
+
"pandas",
|
|
16
|
+
"scikit-learn",
|
|
17
|
+
"scipy",
|
|
18
|
+
"seaborn",
|
|
19
|
+
]
|
|
20
|
+
|
|
21
|
+
[tool.setuptools]
|
|
22
|
+
packages = ["scikit_tools"]
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
"""Runnable, straight-through examples for 13 scikit-learn/data-analysis topics."""
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from sklearn.datasets import load_iris
|
|
3
|
+
|
|
4
|
+
iris = load_iris(as_frame=True)
|
|
5
|
+
df = iris.data.rename(columns={
|
|
6
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
7
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
8
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
9
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
10
|
+
})
|
|
11
|
+
df['Species'] = iris.target.map({
|
|
12
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
13
|
+
})
|
|
14
|
+
|
|
15
|
+
covariance_val = df['PetalLengthCm'].cov(df['PetalWidthCm'])
|
|
16
|
+
correlation_val = df['PetalLengthCm'].corr(df['PetalWidthCm'])
|
|
17
|
+
|
|
18
|
+
print(f"Covariance: {covariance_val:.4f}")
|
|
19
|
+
print(f"Correlation Coefficient: {correlation_val:.4f}")
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from IPython.display import display
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
from sklearn.preprocessing import StandardScaler
|
|
5
|
+
|
|
6
|
+
iris = load_iris(as_frame=True)
|
|
7
|
+
df = iris.data.rename(columns={
|
|
8
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
9
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
10
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
11
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
12
|
+
})
|
|
13
|
+
df['Species'] = iris.target.map({
|
|
14
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
15
|
+
})
|
|
16
|
+
|
|
17
|
+
# One-hot encoding for Species
|
|
18
|
+
df_encoded = pd.get_dummies(df, columns=['Species'])
|
|
19
|
+
|
|
20
|
+
# Standardize numerical attributes
|
|
21
|
+
num_cols = ['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']
|
|
22
|
+
scaler = StandardScaler()
|
|
23
|
+
df_encoded[num_cols] = scaler.fit_transform(df_encoded[num_cols])
|
|
24
|
+
|
|
25
|
+
print("--- Transformed Data Preview ---")
|
|
26
|
+
display(df_encoded.head())
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from IPython.display import display
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
|
|
5
|
+
iris = load_iris(as_frame=True)
|
|
6
|
+
df = iris.data.rename(columns={
|
|
7
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
8
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
9
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
10
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
11
|
+
})
|
|
12
|
+
df['Species'] = iris.target.map({
|
|
13
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
14
|
+
})
|
|
15
|
+
|
|
16
|
+
attributes = ['SepalLengthCm', 'PetalLengthCm']
|
|
17
|
+
stats_summary = pd.DataFrame({
|
|
18
|
+
'Mean': df[attributes].mean(),
|
|
19
|
+
'Median': df[attributes].median(),
|
|
20
|
+
'Sample Variance': df[attributes].var(),
|
|
21
|
+
'Standard Deviation': df[attributes].std()
|
|
22
|
+
})
|
|
23
|
+
|
|
24
|
+
display(stats_summary)
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
import matplotlib.pyplot as plt
|
|
3
|
+
import seaborn as sns
|
|
4
|
+
from sklearn.datasets import load_iris
|
|
5
|
+
|
|
6
|
+
iris = load_iris(as_frame=True)
|
|
7
|
+
df = iris.data.rename(columns={
|
|
8
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
9
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
10
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
11
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
12
|
+
})
|
|
13
|
+
df['Species'] = iris.target.map({
|
|
14
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
15
|
+
})
|
|
16
|
+
|
|
17
|
+
plt.figure(figsize=(14, 5))
|
|
18
|
+
|
|
19
|
+
# Histogram
|
|
20
|
+
plt.subplot(1, 2, 1)
|
|
21
|
+
sns.histplot(data=df, x='PetalLengthCm', kde=True, color='skyblue')
|
|
22
|
+
plt.title('Histogram of PetalLengthCm')
|
|
23
|
+
plt.xlabel('Petal Length (cm)')
|
|
24
|
+
plt.ylabel('Frequency')
|
|
25
|
+
|
|
26
|
+
# Scatter Plot
|
|
27
|
+
plt.subplot(1, 2, 2)
|
|
28
|
+
sns.scatterplot(data=df, x='SepalLengthCm', y='PetalLengthCm', hue='Species', palette='Set1')
|
|
29
|
+
plt.title('SepalLengthCm vs PetalLengthCm')
|
|
30
|
+
plt.xlabel('Sepal Length (cm)')
|
|
31
|
+
plt.ylabel('Petal Length (cm)')
|
|
32
|
+
|
|
33
|
+
plt.tight_layout()
|
|
34
|
+
plt.show()
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from scipy import stats
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
|
|
5
|
+
iris = load_iris(as_frame=True)
|
|
6
|
+
df = iris.data.rename(columns={
|
|
7
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
8
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
9
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
10
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
11
|
+
})
|
|
12
|
+
df['Species'] = iris.target.map({
|
|
13
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
14
|
+
})
|
|
15
|
+
|
|
16
|
+
setosa_sepal = df[df['Species'] == 'Iris-setosa']['SepalLengthCm']
|
|
17
|
+
versicolor_sepal = df[df['Species'] == 'Iris-versicolor']['SepalLengthCm']
|
|
18
|
+
|
|
19
|
+
t_stat, p_val = stats.ttest_ind(setosa_sepal, versicolor_sepal, equal_var=False)
|
|
20
|
+
|
|
21
|
+
print(f"T-statistic: {t_stat:.4f}")
|
|
22
|
+
print(f"P-value: {p_val:.4e}")
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from IPython.display import display
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
|
|
5
|
+
iris = load_iris(as_frame=True)
|
|
6
|
+
df = iris.data.rename(columns={
|
|
7
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
8
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
9
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
10
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
11
|
+
})
|
|
12
|
+
df['Species'] = iris.target.map({
|
|
13
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
14
|
+
})
|
|
15
|
+
|
|
16
|
+
print("--- First 5 Records ---")
|
|
17
|
+
display(df.head())
|
|
18
|
+
|
|
19
|
+
print(f"\nNumber of rows: {df.shape[0]}, Number of columns: {df.shape[1]}")
|
|
20
|
+
|
|
21
|
+
print("\n--- Column Names & Data Types ---")
|
|
22
|
+
print(df.dtypes)
|
|
23
|
+
|
|
24
|
+
print("\n--- Summary Statistics ---")
|
|
25
|
+
display(df.describe())
|
|
26
|
+
|
|
27
|
+
print("\nNumerical attributes: SepalLengthCm, SepalWidthCm, PetalLengthCm, PetalWidthCm")
|
|
28
|
+
print("Target column: Species")
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
import matplotlib.pyplot as plt
|
|
3
|
+
import seaborn as sns
|
|
4
|
+
from IPython.display import display
|
|
5
|
+
from sklearn.datasets import load_iris
|
|
6
|
+
from sklearn.cluster import KMeans
|
|
7
|
+
|
|
8
|
+
iris = load_iris(as_frame=True)
|
|
9
|
+
df = iris.data.rename(columns={
|
|
10
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
11
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
12
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
13
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
14
|
+
})
|
|
15
|
+
df['Species'] = iris.target.map({
|
|
16
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
17
|
+
})
|
|
18
|
+
|
|
19
|
+
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
20
|
+
|
|
21
|
+
kmeans = KMeans(n_clusters=3, random_state=42, n_init='auto')
|
|
22
|
+
df['Cluster'] = kmeans.fit_predict(X)
|
|
23
|
+
|
|
24
|
+
plt.figure(figsize=(6, 4))
|
|
25
|
+
sns.scatterplot(data=df, x='SepalLengthCm', y='PetalLengthCm', hue='Cluster', palette='Set1')
|
|
26
|
+
plt.title('K-Means Clustering Assignments (k=3)')
|
|
27
|
+
plt.show()
|
|
28
|
+
|
|
29
|
+
display(df[['SepalLengthCm', 'PetalLengthCm', 'Species', 'Cluster']].head())
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from IPython.display import display
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
from sklearn.model_selection import train_test_split
|
|
5
|
+
from sklearn.linear_model import LinearRegression
|
|
6
|
+
|
|
7
|
+
iris = load_iris(as_frame=True)
|
|
8
|
+
df = iris.data.rename(columns={
|
|
9
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
10
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
11
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
12
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
13
|
+
})
|
|
14
|
+
df['Species'] = iris.target.map({
|
|
15
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
16
|
+
})
|
|
17
|
+
|
|
18
|
+
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalWidthCm']]
|
|
19
|
+
y = df['PetalLengthCm']
|
|
20
|
+
|
|
21
|
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
|
22
|
+
|
|
23
|
+
model = LinearRegression()
|
|
24
|
+
model.fit(X_train, y_train)
|
|
25
|
+
|
|
26
|
+
predictions = model.predict(X_test)
|
|
27
|
+
|
|
28
|
+
print("Model Coefficients:", {col: coef for col, coef in zip(X.columns, model.coef_)})
|
|
29
|
+
print("Intercept:", model.intercept_)
|
|
30
|
+
|
|
31
|
+
results = pd.DataFrame({'Actual': y_test, 'Predicted': predictions})
|
|
32
|
+
display(results.head())
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from sklearn.datasets import load_iris
|
|
3
|
+
from sklearn.model_selection import train_test_split
|
|
4
|
+
from sklearn.linear_model import LogisticRegression
|
|
5
|
+
from sklearn.metrics import accuracy_score, confusion_matrix
|
|
6
|
+
|
|
7
|
+
iris = load_iris(as_frame=True)
|
|
8
|
+
df = iris.data.rename(columns={
|
|
9
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
10
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
11
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
12
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
13
|
+
})
|
|
14
|
+
df['Species'] = iris.target.map({
|
|
15
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
16
|
+
})
|
|
17
|
+
|
|
18
|
+
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
19
|
+
y = df['Species']
|
|
20
|
+
|
|
21
|
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
|
22
|
+
|
|
23
|
+
clf = LogisticRegression(max_iter=200)
|
|
24
|
+
clf.fit(X_train, y_train)
|
|
25
|
+
|
|
26
|
+
y_pred = clf.predict(X_test)
|
|
27
|
+
|
|
28
|
+
print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
|
|
29
|
+
print("\nConfusion Matrix:")
|
|
30
|
+
print(confusion_matrix(y_test, y_pred))
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
import numpy as np
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
|
|
5
|
+
iris = load_iris(as_frame=True)
|
|
6
|
+
df = iris.data.rename(columns={
|
|
7
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
8
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
9
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
10
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
11
|
+
})
|
|
12
|
+
df['Species'] = iris.target.map({
|
|
13
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
14
|
+
})
|
|
15
|
+
df_copy = df.copy()
|
|
16
|
+
|
|
17
|
+
# Introduce missing values
|
|
18
|
+
df_copy.loc[[5, 10, 15], 'SepalLengthCm'] = np.nan
|
|
19
|
+
df_copy.loc[[20, 25], 'Species'] = np.nan
|
|
20
|
+
|
|
21
|
+
print("--- Missing Value Count Before Imputation ---")
|
|
22
|
+
print(df_copy.isnull().sum())
|
|
23
|
+
|
|
24
|
+
# Impute numerical missing values with median
|
|
25
|
+
median_sepal = df_copy['SepalLengthCm'].median()
|
|
26
|
+
df_copy['SepalLengthCm'] = df_copy['SepalLengthCm'].fillna(median_sepal)
|
|
27
|
+
|
|
28
|
+
# Impute categorical missing values with mode
|
|
29
|
+
mode_species = df_copy['Species'].mode()[0]
|
|
30
|
+
df_copy['Species'] = df_copy['Species'].fillna(mode_species)
|
|
31
|
+
|
|
32
|
+
print("\n--- Missing Value Count After Imputation ---")
|
|
33
|
+
print(df_copy.isnull().sum())
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
import numpy as np
|
|
3
|
+
from sklearn.datasets import load_iris
|
|
4
|
+
|
|
5
|
+
iris = load_iris(as_frame=True)
|
|
6
|
+
df = iris.data.rename(columns={
|
|
7
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
8
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
9
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
10
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
11
|
+
})
|
|
12
|
+
df['Species'] = iris.target.map({
|
|
13
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
14
|
+
})
|
|
15
|
+
|
|
16
|
+
# Select SepalLengthCm and convert to NumPy array
|
|
17
|
+
sepal_length = df['SepalLengthCm'].to_numpy()
|
|
18
|
+
|
|
19
|
+
# Calculate basic statistics
|
|
20
|
+
total_sum = np.sum(sepal_length)
|
|
21
|
+
mean_val = np.mean(sepal_length)
|
|
22
|
+
max_val = np.max(sepal_length)
|
|
23
|
+
min_val = np.min(sepal_length)
|
|
24
|
+
first_five = sepal_length[:5]
|
|
25
|
+
|
|
26
|
+
# Display results
|
|
27
|
+
print(f"Sum: {total_sum:.2f}")
|
|
28
|
+
print(f"Mean: {mean_val:.2f}")
|
|
29
|
+
print(f"Maximum: {max_val:.2f}")
|
|
30
|
+
print(f"Minimum: {min_val:.2f}")
|
|
31
|
+
print(f"First five values: {first_five}")
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
import matplotlib.pyplot as plt
|
|
3
|
+
import seaborn as sns
|
|
4
|
+
from IPython.display import display
|
|
5
|
+
from sklearn.datasets import load_iris
|
|
6
|
+
|
|
7
|
+
iris = load_iris(as_frame=True)
|
|
8
|
+
df = iris.data.rename(columns={
|
|
9
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
10
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
11
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
12
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
13
|
+
})
|
|
14
|
+
df['Species'] = iris.target.map({
|
|
15
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
16
|
+
})
|
|
17
|
+
|
|
18
|
+
# Box Plot
|
|
19
|
+
plt.figure(figsize=(6, 4))
|
|
20
|
+
sns.boxplot(y=df['SepalWidthCm'], color='lightgreen')
|
|
21
|
+
plt.title('Box Plot of SepalWidthCm')
|
|
22
|
+
plt.ylabel('Sepal Width (cm)')
|
|
23
|
+
plt.show()
|
|
24
|
+
|
|
25
|
+
# Calculate IQR & Outliers
|
|
26
|
+
Q1 = df['SepalWidthCm'].quantile(0.25)
|
|
27
|
+
Q3 = df['SepalWidthCm'].quantile(0.75)
|
|
28
|
+
IQR = Q3 - Q1
|
|
29
|
+
lower_bound = Q1 - 1.5 * IQR
|
|
30
|
+
upper_bound = Q3 + 1.5 * IQR
|
|
31
|
+
|
|
32
|
+
outliers = df[(df['SepalWidthCm'] < lower_bound) | (df['SepalWidthCm'] > upper_bound)]
|
|
33
|
+
|
|
34
|
+
print(f"Q1 (25th percentile): {Q1:.2f}")
|
|
35
|
+
print(f"Q3 (75th percentile): {Q3:.2f}")
|
|
36
|
+
print(f"IQR (Interquartile Range): {IQR:.2f}")
|
|
37
|
+
print(f"Outlier limits: {lower_bound:.2f} to {upper_bound:.2f}")
|
|
38
|
+
print(f"\nDetected Outliers:")
|
|
39
|
+
display(outliers[['SepalWidthCm', 'Species']])
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
import pandas as pd
|
|
2
|
+
from sklearn.datasets import load_iris
|
|
3
|
+
from sklearn.preprocessing import StandardScaler
|
|
4
|
+
from sklearn.decomposition import PCA
|
|
5
|
+
from sklearn.model_selection import train_test_split
|
|
6
|
+
from sklearn.linear_model import LogisticRegression
|
|
7
|
+
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
|
|
8
|
+
|
|
9
|
+
iris = load_iris(as_frame=True)
|
|
10
|
+
df = iris.data.rename(columns={
|
|
11
|
+
'sepal length (cm)': 'SepalLengthCm',
|
|
12
|
+
'sepal width (cm)': 'SepalWidthCm',
|
|
13
|
+
'petal length (cm)': 'PetalLengthCm',
|
|
14
|
+
'petal width (cm)': 'PetalWidthCm'
|
|
15
|
+
})
|
|
16
|
+
df['Species'] = iris.target.map({
|
|
17
|
+
index: f'Iris-{name}' for index, name in enumerate(iris.target_names)
|
|
18
|
+
})
|
|
19
|
+
|
|
20
|
+
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
21
|
+
y = df['Species']
|
|
22
|
+
|
|
23
|
+
# Standardize
|
|
24
|
+
X_scaled = StandardScaler().fit_transform(X)
|
|
25
|
+
|
|
26
|
+
# PCA to 2 components
|
|
27
|
+
pca = PCA(n_components=2)
|
|
28
|
+
X_pca = pca.fit_transform(X_scaled)
|
|
29
|
+
|
|
30
|
+
X_train, X_test, y_train, y_test = train_test_split(X_pca, y, test_size=0.2, random_state=42)
|
|
31
|
+
|
|
32
|
+
clf = LogisticRegression()
|
|
33
|
+
clf.fit(X_train, y_train)
|
|
34
|
+
y_pred = clf.predict(X_test)
|
|
35
|
+
|
|
36
|
+
print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
|
|
37
|
+
print(f"Precision (Macro): {precision_score(y_test, y_pred, average='macro'):.4f}")
|
|
38
|
+
print(f"Recall (Macro): {recall_score(y_test, y_pred, average='macro'):.4f}")
|
|
39
|
+
print(f"F1-Score (Macro): {f1_score(y_test, y_pred, average='macro'):.4f}")
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: scikit-tools
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Thirteen runnable, straight-through scikit-learn and data-analysis examples
|
|
5
|
+
Requires-Python: >=3.10
|
|
6
|
+
Description-Content-Type: text/markdown
|
|
7
|
+
Requires-Dist: ipython
|
|
8
|
+
Requires-Dist: matplotlib
|
|
9
|
+
Requires-Dist: numpy
|
|
10
|
+
Requires-Dist: pandas
|
|
11
|
+
Requires-Dist: scikit-learn
|
|
12
|
+
Requires-Dist: scipy
|
|
13
|
+
Requires-Dist: seaborn
|
|
14
|
+
|
|
15
|
+
# scikit-tools
|
|
16
|
+
|
|
17
|
+
`scikit-tools` provides 13 standalone, straight-through Python examples. Each
|
|
18
|
+
topic file contains its own imports, dataset setup, and analysis code; examples
|
|
19
|
+
are not hidden inside functions.
|
|
20
|
+
|
|
21
|
+
## Install from a local checkout
|
|
22
|
+
|
|
23
|
+
```console
|
|
24
|
+
python -m pip install -e .
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
This installs the package and its dependencies. Each topic loads the Iris
|
|
28
|
+
dataset provided by `sklearn.datasets.load_iris`, formats its columns to match
|
|
29
|
+
the examples, and does not download or write a CSV file.
|
|
30
|
+
|
|
31
|
+
## Open or run a topic
|
|
32
|
+
|
|
33
|
+
```console
|
|
34
|
+
python -m scikit_tools.numpy_stats
|
|
35
|
+
python -m scikit_tools.pca
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
You can also open any topic script under `scikit_tools` directly in your
|
|
39
|
+
editor, inspect or copy its raw code, and run it with the editor's Python run
|
|
40
|
+
command. These are scripts: importing one executes its example.
|
|
41
|
+
|
|
42
|
+
## Topics
|
|
43
|
+
|
|
44
|
+
1. NumPy statistics (`numpy_stats.py`)
|
|
45
|
+
2. Inspecting the Iris dataset (`inspect_dataset.py`)
|
|
46
|
+
3. Handling missing values (`missing_values.py`)
|
|
47
|
+
4. Data transformation (`data_transformation.py`)
|
|
48
|
+
5. Histogram and scatter-plot EDA (`eda_plots.py`)
|
|
49
|
+
6. Box plot and IQR outlier detection (`outlier_detection.py`)
|
|
50
|
+
7. Descriptive statistics (`descriptive_stats.py`)
|
|
51
|
+
8. Correlation and covariance (`correlation_covariance.py`)
|
|
52
|
+
9. Independent-samples t-test (`hypothesis_testing.py`)
|
|
53
|
+
10. Linear regression (`linear_regression.py`)
|
|
54
|
+
11. Logistic-regression classification (`logistic_regression.py`)
|
|
55
|
+
12. K-means clustering (`kmeans_clustering.py`)
|
|
56
|
+
13. PCA and classification (`pca.py`)
|
|
57
|
+
|
|
58
|
+
The examples use the Iris columns `SepalLengthCm`, `SepalWidthCm`,
|
|
59
|
+
`PetalLengthCm`, `PetalWidthCm`, and `Species`. The species names use the
|
|
60
|
+
`Iris-setosa` style. The missing values example intentionally adds missing values to a DataFrame
|
|
61
|
+
copy and fills them; it does not change the source dataset.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
README.md
|
|
2
|
+
SCIKIT_TOOLS.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
scikit_tools/__init__.py
|
|
5
|
+
scikit_tools/correlation_covariance.py
|
|
6
|
+
scikit_tools/data_transformation.py
|
|
7
|
+
scikit_tools/descriptive_stats.py
|
|
8
|
+
scikit_tools/eda_plots.py
|
|
9
|
+
scikit_tools/hypothesis_testing.py
|
|
10
|
+
scikit_tools/inspect_dataset.py
|
|
11
|
+
scikit_tools/kmeans_clustering.py
|
|
12
|
+
scikit_tools/linear_regression.py
|
|
13
|
+
scikit_tools/logistic_regression.py
|
|
14
|
+
scikit_tools/missing_values.py
|
|
15
|
+
scikit_tools/numpy_stats.py
|
|
16
|
+
scikit_tools/outlier_detection.py
|
|
17
|
+
scikit_tools/pca.py
|
|
18
|
+
scikit_tools.egg-info/PKG-INFO
|
|
19
|
+
scikit_tools.egg-info/SOURCES.txt
|
|
20
|
+
scikit_tools.egg-info/dependency_links.txt
|
|
21
|
+
scikit_tools.egg-info/requires.txt
|
|
22
|
+
scikit_tools.egg-info/top_level.txt
|
|
23
|
+
tests/test.py
|
|
24
|
+
tests/test_scikit_tools.py
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
[egg_info]
|
|
2
|
-
tag_build =
|
|
3
|
-
tag_date = 0
|
|
4
|
-
|
|
1
|
+
[egg_info]
|
|
2
|
+
tag_build =
|
|
3
|
+
tag_date = 0
|
|
4
|
+
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
from pathlib import Path
|
|
2
|
+
import scikit_tools as st
|
|
3
|
+
|
|
4
|
+
def pca_analysis(filepath='Iris.csv'):
|
|
5
|
+
"""Topic 13: Standardize, compress with 2-Component PCA, and classify."""
|
|
6
|
+
import pandas as pd
|
|
7
|
+
from sklearn.preprocessing import StandardScaler
|
|
8
|
+
from sklearn.decomposition import PCA
|
|
9
|
+
from sklearn.model_selection import train_test_split
|
|
10
|
+
from sklearn.linear_model import LogisticRegression
|
|
11
|
+
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
|
|
12
|
+
df = pd.read_csv(filepath)
|
|
13
|
+
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
14
|
+
y = df['Species']
|
|
15
|
+
X_scaled = StandardScaler().fit_transform(X)
|
|
16
|
+
pca = PCA(n_components=2)
|
|
17
|
+
X_pca = pca.fit_transform(X_scaled)
|
|
18
|
+
X_train, X_test, y_train, y_test = train_test_split(X_pca, y, test_size=0.2, random_state=42)
|
|
19
|
+
clf = LogisticRegression()
|
|
20
|
+
clf.fit(X_train, y_train)
|
|
21
|
+
y_pred = clf.predict(X_test)
|
|
22
|
+
return {
|
|
23
|
+
'accuracy': accuracy_score(y_test, y_pred),
|
|
24
|
+
'precision': precision_score(y_test, y_pred, average='macro'),
|
|
25
|
+
'recall': recall_score(y_test, y_pred, average='macro'),
|
|
26
|
+
'f1_score': f1_score(y_test, y_pred, average='macro')
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
if __name__ == '__main__':
|
|
31
|
+
print(pca_analysis(Path(__file__).with_name('Iris.csv')))
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
import os
|
|
2
|
+
from pathlib import Path
|
|
3
|
+
import subprocess
|
|
4
|
+
import sys
|
|
5
|
+
import tempfile
|
|
6
|
+
import unittest
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
|
10
|
+
TOPIC_MODULES = [
|
|
11
|
+
"numpy_stats",
|
|
12
|
+
"inspect_dataset",
|
|
13
|
+
"missing_values",
|
|
14
|
+
"data_transformation",
|
|
15
|
+
"eda_plots",
|
|
16
|
+
"outlier_detection",
|
|
17
|
+
"descriptive_stats",
|
|
18
|
+
"correlation_covariance",
|
|
19
|
+
"hypothesis_testing",
|
|
20
|
+
"linear_regression",
|
|
21
|
+
"logistic_regression",
|
|
22
|
+
"kmeans_clustering",
|
|
23
|
+
"pca",
|
|
24
|
+
]
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
class ScikitToolsScriptTests(unittest.TestCase):
|
|
28
|
+
def test_all_topics_run_from_a_clean_working_directory(self):
|
|
29
|
+
with tempfile.TemporaryDirectory() as working_dir:
|
|
30
|
+
env = os.environ.copy()
|
|
31
|
+
env["MPLBACKEND"] = "Agg"
|
|
32
|
+
env["PYTHONPATH"] = str(PROJECT_ROOT)
|
|
33
|
+
for idx, module in enumerate(TOPIC_MODULES, start=1):
|
|
34
|
+
with self.subTest(module=module):
|
|
35
|
+
result = subprocess.run(
|
|
36
|
+
[sys.executable, str(PROJECT_ROOT / "scikit_tools" / f"{module}.py")],
|
|
37
|
+
cwd=working_dir,
|
|
38
|
+
env=env,
|
|
39
|
+
capture_output=True,
|
|
40
|
+
text=True,
|
|
41
|
+
timeout=60,
|
|
42
|
+
)
|
|
43
|
+
self.assertEqual(
|
|
44
|
+
result.returncode,
|
|
45
|
+
0,
|
|
46
|
+
msg=f"module {module} failed:\n{result.stdout}\n{result.stderr}",
|
|
47
|
+
)
|
|
48
|
+
self.assertFalse((Path(working_dir) / "iris.csv").exists())
|
|
49
|
+
if module == "missing_values":
|
|
50
|
+
after_imputation = result.stdout.split(
|
|
51
|
+
"--- Missing Value Count After Imputation ---", 1
|
|
52
|
+
)[1]
|
|
53
|
+
self.assertRegex(after_imputation, r"SepalLengthCm\s+0")
|
|
54
|
+
self.assertRegex(after_imputation, r"Species\s+0")
|
|
55
|
+
|
|
56
|
+
def test_every_topic_loads_the_bundled_sklearn_dataset(self):
|
|
57
|
+
for module in TOPIC_MODULES:
|
|
58
|
+
with self.subTest(module=module):
|
|
59
|
+
source = (PROJECT_ROOT / "scikit_tools" / f"{module}.py").read_text(
|
|
60
|
+
encoding="utf-8"
|
|
61
|
+
)
|
|
62
|
+
self.assertIn("from sklearn.datasets import load_iris", source)
|
|
63
|
+
self.assertNotIn("from ._common", source)
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
if __name__ == "__main__":
|
|
67
|
+
unittest.main()
|
scikit_tools-0.1.1/PKG-INFO
DELETED
|
@@ -1,19 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: scikit-tools
|
|
3
|
-
Version: 0.1.1
|
|
4
|
-
Summary: Clean, dependency-managed toolkit with fully self-contained topic functions
|
|
5
|
-
Author-email: Your Name <your.email@example.com>
|
|
6
|
-
Classifier: Programming Language :: Python :: 3
|
|
7
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
8
|
-
Classifier: Operating System :: OS Independent
|
|
9
|
-
Requires-Python: >=3.8
|
|
10
|
-
Description-Content-Type: text/markdown
|
|
11
|
-
Requires-Dist: pandas
|
|
12
|
-
Requires-Dist: numpy
|
|
13
|
-
Requires-Dist: matplotlib
|
|
14
|
-
Requires-Dist: seaborn
|
|
15
|
-
Requires-Dist: scipy
|
|
16
|
-
Requires-Dist: scikit-learn
|
|
17
|
-
|
|
18
|
-
# scikit-tools
|
|
19
|
-
A clean, streamlined scientific data science toolkit with self-contained dataset fetching and direct topic interfaces.
|
scikit_tools-0.1.1/README.md
DELETED
|
@@ -1,29 +0,0 @@
|
|
|
1
|
-
[build-system]
|
|
2
|
-
requires = ["setuptools>=61.0"]
|
|
3
|
-
build-backend = "setuptools.build_meta"
|
|
4
|
-
|
|
5
|
-
[project]
|
|
6
|
-
name = "scikit-tools"
|
|
7
|
-
version = "0.1.1"
|
|
8
|
-
description = "Clean, dependency-managed toolkit with fully self-contained topic functions"
|
|
9
|
-
authors = [
|
|
10
|
-
{ name="Your Name", email="your.email@example.com" }
|
|
11
|
-
]
|
|
12
|
-
readme = "README.md"
|
|
13
|
-
requires-python = ">=3.8"
|
|
14
|
-
classifiers = [
|
|
15
|
-
"Programming Language :: Python :: 3",
|
|
16
|
-
"License :: OSI Approved :: MIT License",
|
|
17
|
-
"Operating System :: OS Independent",
|
|
18
|
-
]
|
|
19
|
-
dependencies = [
|
|
20
|
-
"pandas",
|
|
21
|
-
"numpy",
|
|
22
|
-
"matplotlib",
|
|
23
|
-
"seaborn",
|
|
24
|
-
"scipy",
|
|
25
|
-
"scikit-learn"
|
|
26
|
-
]
|
|
27
|
-
|
|
28
|
-
[tool.setuptools.packages.find]
|
|
29
|
-
where = ["src"]
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
from .solutions import *
|
|
@@ -1,205 +0,0 @@
|
|
|
1
|
-
def ensure_dataset(filepath='iris.csv'):
|
|
2
|
-
"""Ensures that the local dataset exists. If not, automatically downloads and formats it."""
|
|
3
|
-
import os
|
|
4
|
-
import pandas as pd
|
|
5
|
-
if not os.path.exists(filepath):
|
|
6
|
-
url = 'https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv'
|
|
7
|
-
df = pd.read_csv(url)
|
|
8
|
-
df.columns = ['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm', 'Species']
|
|
9
|
-
df['Species'] = df['Species'].map(lambda x: f"Iris-{x}")
|
|
10
|
-
df.to_csv(filepath, index=False)
|
|
11
|
-
|
|
12
|
-
def numpy_stats(filepath='iris.csv'):
|
|
13
|
-
"""Topic 1: Basic numerical operations and statistics using NumPy."""
|
|
14
|
-
import pandas as pd
|
|
15
|
-
import numpy as np
|
|
16
|
-
ensure_dataset(filepath)
|
|
17
|
-
df = pd.read_csv(filepath)
|
|
18
|
-
sepal_length = df['SepalLengthCm'].to_numpy()
|
|
19
|
-
return {
|
|
20
|
-
'sum': np.sum(sepal_length),
|
|
21
|
-
'mean': np.mean(sepal_length),
|
|
22
|
-
'max': np.max(sepal_length),
|
|
23
|
-
'min': np.min(sepal_length),
|
|
24
|
-
'first_five': sepal_length[:5]
|
|
25
|
-
}
|
|
26
|
-
|
|
27
|
-
def inspect_dataset(filepath='iris.csv'):
|
|
28
|
-
"""Topic 2: Load DataFrame and inspect its structure, columns, and shape."""
|
|
29
|
-
import pandas as pd
|
|
30
|
-
ensure_dataset(filepath)
|
|
31
|
-
df = pd.read_csv(filepath)
|
|
32
|
-
return {
|
|
33
|
-
'head': df.head(),
|
|
34
|
-
'shape': df.shape,
|
|
35
|
-
'dtypes': df.dtypes,
|
|
36
|
-
'description': df.describe()
|
|
37
|
-
}
|
|
38
|
-
|
|
39
|
-
def handle_missing_values(filepath='iris.csv'):
|
|
40
|
-
"""Topic 3: Introduce mock missing values and handle them using median and mode."""
|
|
41
|
-
import pandas as pd
|
|
42
|
-
import numpy as np
|
|
43
|
-
ensure_dataset(filepath)
|
|
44
|
-
df_copy = pd.read_csv(filepath).copy()
|
|
45
|
-
df_copy.loc[[5, 10, 15], 'SepalLengthCm'] = np.nan
|
|
46
|
-
df_copy.loc[[20, 25], 'Species'] = np.nan
|
|
47
|
-
|
|
48
|
-
median_sepal = df_copy['SepalLengthCm'].median()
|
|
49
|
-
df_copy['SepalLengthCm'] = df_copy['SepalLengthCm'].fillna(median_sepal)
|
|
50
|
-
mode_species = df_copy['Species'].mode()[0]
|
|
51
|
-
df_copy['Species'] = df_copy['Species'].fillna(mode_species)
|
|
52
|
-
return df_copy
|
|
53
|
-
|
|
54
|
-
def data_transformation(filepath='iris.csv'):
|
|
55
|
-
"""Topic 4: Perform one-hot encoding and standardization (scaling)."""
|
|
56
|
-
import pandas as pd
|
|
57
|
-
from sklearn.preprocessing import StandardScaler
|
|
58
|
-
ensure_dataset(filepath)
|
|
59
|
-
df = pd.read_csv(filepath)
|
|
60
|
-
df_encoded = pd.get_dummies(df, columns=['Species'])
|
|
61
|
-
num_cols = ['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']
|
|
62
|
-
scaler = StandardScaler()
|
|
63
|
-
df_encoded[num_cols] = scaler.fit_transform(df_encoded[num_cols])
|
|
64
|
-
return df_encoded
|
|
65
|
-
|
|
66
|
-
def eda_plots(filepath='iris.csv'):
|
|
67
|
-
"""Topic 5: Generate and render Histogram and Scatter Plots for EDA."""
|
|
68
|
-
import pandas as pd
|
|
69
|
-
import matplotlib.pyplot as plt
|
|
70
|
-
import seaborn as sns
|
|
71
|
-
ensure_dataset(filepath)
|
|
72
|
-
df = pd.read_csv(filepath)
|
|
73
|
-
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
|
|
74
|
-
sns.histplot(data=df, x='PetalLengthCm', kde=True, color='skyblue', ax=axes[0])
|
|
75
|
-
axes[0].set_title('Histogram of PetalLengthCm')
|
|
76
|
-
sns.scatterplot(data=df, x='SepalLengthCm', y='PetalLengthCm', hue='Species', palette='Set1', ax=axes[1])
|
|
77
|
-
axes[1].set_title('SepalLengthCm vs PetalLengthCm')
|
|
78
|
-
plt.tight_layout()
|
|
79
|
-
plt.show()
|
|
80
|
-
|
|
81
|
-
def outlier_detection(filepath='iris.csv'):
|
|
82
|
-
"""Topic 6: Quantify outliers using Interquartile Range (IQR) bounds."""
|
|
83
|
-
import pandas as pd
|
|
84
|
-
ensure_dataset(filepath)
|
|
85
|
-
df = pd.read_csv(filepath)
|
|
86
|
-
Q1 = df['SepalWidthCm'].quantile(0.25)
|
|
87
|
-
Q3 = df['SepalWidthCm'].quantile(0.75)
|
|
88
|
-
IQR = Q3 - Q1
|
|
89
|
-
lower_bound = Q1 - 1.5 * IQR
|
|
90
|
-
upper_bound = Q3 + 1.5 * IQR
|
|
91
|
-
outliers = df[(df['SepalWidthCm'] < lower_bound) | (df['SepalWidthCm'] > upper_bound)]
|
|
92
|
-
return {
|
|
93
|
-
'IQR': IQR,
|
|
94
|
-
'limits': (lower_bound, upper_bound),
|
|
95
|
-
'outliers': outliers
|
|
96
|
-
}
|
|
97
|
-
|
|
98
|
-
def descriptive_stats(filepath='iris.csv'):
|
|
99
|
-
"""Topic 7: Calculate detailed descriptive metrics."""
|
|
100
|
-
import pandas as pd
|
|
101
|
-
ensure_dataset(filepath)
|
|
102
|
-
df = pd.read_csv(filepath)
|
|
103
|
-
attributes = ['SepalLengthCm', 'PetalLengthCm']
|
|
104
|
-
return pd.DataFrame({
|
|
105
|
-
'Mean': df[attributes].mean(),
|
|
106
|
-
'Median': df[attributes].median(),
|
|
107
|
-
'Sample Variance': df[attributes].var(),
|
|
108
|
-
'Standard Deviation': df[attributes].std()
|
|
109
|
-
})
|
|
110
|
-
|
|
111
|
-
def correlation_covariance(filepath='iris.csv'):
|
|
112
|
-
"""Topic 8: Evaluate covariance and Pearson correlation coefficient."""
|
|
113
|
-
import pandas as pd
|
|
114
|
-
ensure_dataset(filepath)
|
|
115
|
-
df = pd.read_csv(filepath)
|
|
116
|
-
return {
|
|
117
|
-
'covariance': df['PetalLengthCm'].cov(df['PetalWidthCm']),
|
|
118
|
-
'correlation': df['PetalLengthCm'].corr(df['PetalWidthCm'])
|
|
119
|
-
}
|
|
120
|
-
|
|
121
|
-
def hypothesis_testing(filepath='iris.csv'):
|
|
122
|
-
"""Topic 9: Run an independent T-test on SepalLengthCm between Setosa and Versicolor."""
|
|
123
|
-
import pandas as pd
|
|
124
|
-
from scipy import stats
|
|
125
|
-
ensure_dataset(filepath)
|
|
126
|
-
df = pd.read_csv(filepath)
|
|
127
|
-
setosa_sepal = df[df['Species'] == 'Iris-setosa']['SepalLengthCm']
|
|
128
|
-
versicolor_sepal = df[df['Species'] == 'Iris-versicolor']['SepalLengthCm']
|
|
129
|
-
t_stat, p_val = stats.ttest_ind(setosa_sepal, versicolor_sepal, equal_var=False)
|
|
130
|
-
return {'t_statistic': t_stat, 'p_value': p_val}
|
|
131
|
-
|
|
132
|
-
def linear_regression(filepath='iris.csv'):
|
|
133
|
-
"""Topic 10: Linear Regression predicting PetalLengthCm."""
|
|
134
|
-
import pandas as pd
|
|
135
|
-
from sklearn.model_selection import train_test_split
|
|
136
|
-
from sklearn.linear_model import LinearRegression
|
|
137
|
-
ensure_dataset(filepath)
|
|
138
|
-
df = pd.read_csv(filepath)
|
|
139
|
-
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalWidthCm']]
|
|
140
|
-
y = df['PetalLengthCm']
|
|
141
|
-
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
|
142
|
-
model = LinearRegression()
|
|
143
|
-
model.fit(X_train, y_train)
|
|
144
|
-
predictions = model.predict(X_test)
|
|
145
|
-
return {
|
|
146
|
-
'coefficients': {col: coef for col, coef in zip(X.columns, model.coef_)},
|
|
147
|
-
'intercept': model.intercept_,
|
|
148
|
-
'predictions': pd.DataFrame({'Actual': y_test, 'Predicted': predictions})
|
|
149
|
-
}
|
|
150
|
-
|
|
151
|
-
def logistic_regression(filepath='iris.csv'):
|
|
152
|
-
"""Topic 11: Multi-class Logistic Regression for classification."""
|
|
153
|
-
import pandas as pd
|
|
154
|
-
from sklearn.model_selection import train_test_split
|
|
155
|
-
from sklearn.linear_model import LogisticRegression
|
|
156
|
-
from sklearn.metrics import accuracy_score, confusion_matrix
|
|
157
|
-
ensure_dataset(filepath)
|
|
158
|
-
df = pd.read_csv(filepath)
|
|
159
|
-
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
160
|
-
y = df['Species']
|
|
161
|
-
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
|
162
|
-
clf = LogisticRegression(max_iter=200)
|
|
163
|
-
clf.fit(X_train, y_train)
|
|
164
|
-
y_pred = clf.predict(X_test)
|
|
165
|
-
return {
|
|
166
|
-
'accuracy': accuracy_score(y_test, y_pred),
|
|
167
|
-
'confusion_matrix': confusion_matrix(y_test, y_pred)
|
|
168
|
-
}
|
|
169
|
-
|
|
170
|
-
def kmeans_clustering(filepath='iris.csv'):
|
|
171
|
-
"""Topic 12: Cluster attributes unsupervised using K-Means."""
|
|
172
|
-
import pandas as pd
|
|
173
|
-
from sklearn.cluster import KMeans
|
|
174
|
-
ensure_dataset(filepath)
|
|
175
|
-
df = pd.read_csv(filepath)
|
|
176
|
-
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
177
|
-
kmeans = KMeans(n_clusters=3, random_state=42, n_init='auto')
|
|
178
|
-
df['Cluster'] = kmeans.fit_predict(X)
|
|
179
|
-
return df
|
|
180
|
-
|
|
181
|
-
def pca_analysis(filepath='iris.csv'):
|
|
182
|
-
"""Topic 13: Standardize, compress with 2-Component PCA, and classify."""
|
|
183
|
-
import pandas as pd
|
|
184
|
-
from sklearn.preprocessing import StandardScaler
|
|
185
|
-
from sklearn.decomposition import PCA
|
|
186
|
-
from sklearn.model_selection import train_test_split
|
|
187
|
-
from sklearn.linear_model import LogisticRegression
|
|
188
|
-
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
|
|
189
|
-
ensure_dataset(filepath)
|
|
190
|
-
df = pd.read_csv(filepath)
|
|
191
|
-
X = df[['SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm']]
|
|
192
|
-
y = df['Species']
|
|
193
|
-
X_scaled = StandardScaler().fit_transform(X)
|
|
194
|
-
pca = PCA(n_components=2)
|
|
195
|
-
X_pca = pca.fit_transform(X_scaled)
|
|
196
|
-
X_train, X_test, y_train, y_test = train_test_split(X_pca, y, test_size=0.2, random_state=42)
|
|
197
|
-
clf = LogisticRegression()
|
|
198
|
-
clf.fit(X_train, y_train)
|
|
199
|
-
y_pred = clf.predict(X_test)
|
|
200
|
-
return {
|
|
201
|
-
'accuracy': accuracy_score(y_test, y_pred),
|
|
202
|
-
'precision': precision_score(y_test, y_pred, average='macro'),
|
|
203
|
-
'recall': recall_score(y_test, y_pred, average='macro'),
|
|
204
|
-
'f1_score': f1_score(y_test, y_pred, average='macro')
|
|
205
|
-
}
|
|
@@ -1,19 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: scikit-tools
|
|
3
|
-
Version: 0.1.1
|
|
4
|
-
Summary: Clean, dependency-managed toolkit with fully self-contained topic functions
|
|
5
|
-
Author-email: Your Name <your.email@example.com>
|
|
6
|
-
Classifier: Programming Language :: Python :: 3
|
|
7
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
8
|
-
Classifier: Operating System :: OS Independent
|
|
9
|
-
Requires-Python: >=3.8
|
|
10
|
-
Description-Content-Type: text/markdown
|
|
11
|
-
Requires-Dist: pandas
|
|
12
|
-
Requires-Dist: numpy
|
|
13
|
-
Requires-Dist: matplotlib
|
|
14
|
-
Requires-Dist: seaborn
|
|
15
|
-
Requires-Dist: scipy
|
|
16
|
-
Requires-Dist: scikit-learn
|
|
17
|
-
|
|
18
|
-
# scikit-tools
|
|
19
|
-
A clean, streamlined scientific data science toolkit with self-contained dataset fetching and direct topic interfaces.
|
|
@@ -1,9 +0,0 @@
|
|
|
1
|
-
README.md
|
|
2
|
-
pyproject.toml
|
|
3
|
-
src/scikit_tools/__init__.py
|
|
4
|
-
src/scikit_tools/solutions.py
|
|
5
|
-
src/scikit_tools.egg-info/PKG-INFO
|
|
6
|
-
src/scikit_tools.egg-info/SOURCES.txt
|
|
7
|
-
src/scikit_tools.egg-info/dependency_links.txt
|
|
8
|
-
src/scikit_tools.egg-info/requires.txt
|
|
9
|
-
src/scikit_tools.egg-info/top_level.txt
|
|
File without changes
|
|
File without changes
|