eb-evaluation 0.2.0__tar.gz → 0.2.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. eb_evaluation-0.2.2/PKG-INFO +119 -0
  2. eb_evaluation-0.2.2/README.md +86 -0
  3. eb_evaluation-0.2.2/pyproject.toml +92 -0
  4. eb_evaluation-0.2.2/src/eb_evaluation.egg-info/PKG-INFO +119 -0
  5. eb_evaluation-0.2.2/src/eb_evaluation.egg-info/requires.txt +12 -0
  6. eb_evaluation-0.2.0/PKG-INFO +0 -169
  7. eb_evaluation-0.2.0/README.md +0 -148
  8. eb_evaluation-0.2.0/pyproject.toml +0 -41
  9. eb_evaluation-0.2.0/src/eb_evaluation.egg-info/PKG-INFO +0 -169
  10. eb_evaluation-0.2.0/src/eb_evaluation.egg-info/requires.txt +0 -7
  11. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/LICENSE +0 -0
  12. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/setup.cfg +0 -0
  13. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/__init__.py +0 -0
  14. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/adjustment/__init__.py +0 -0
  15. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/adjustment/_utils.py +0 -0
  16. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/adjustment/readiness_adjustment.py +0 -0
  17. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/__init__.py +0 -0
  18. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/cost_ratio.py +0 -0
  19. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/entity.py +0 -0
  20. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/group.py +0 -0
  21. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/hierarchy.py +0 -0
  22. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/panel.py +0 -0
  23. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/sensitivity.py +0 -0
  24. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/single.py +0 -0
  25. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/dataframe/tolerance.py +0 -0
  26. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/model_selection/__init__.py +0 -0
  27. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/model_selection/auto_engine.py +0 -0
  28. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/model_selection/compare.py +0 -0
  29. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/model_selection/cwsl_regressor.py +0 -0
  30. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/model_selection/electric_barometer.py +0 -0
  31. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/utils/__init__.py +0 -0
  32. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation/utils/validation.py +0 -0
  33. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation.egg-info/SOURCES.txt +0 -0
  34. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation.egg-info/dependency_links.txt +0 -0
  35. {eb_evaluation-0.2.0 → eb_evaluation-0.2.2}/src/eb_evaluation.egg-info/top_level.txt +0 -0
@@ -0,0 +1,119 @@
1
+ Metadata-Version: 2.4
2
+ Name: eb-evaluation
3
+ Version: 0.2.2
4
+ Summary: Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics.
5
+ Author-email: "Kyle Corrie (Economistician)" <kcorrie@economistician.com>
6
+ License-Expression: BSD-3-Clause
7
+ Project-URL: Homepage, https://github.com/Economistician/eb-evaluation
8
+ Project-URL: Repository, https://github.com/Economistician/eb-evaluation
9
+ Project-URL: Issues, https://github.com/Economistician/eb-evaluation/issues
10
+ Project-URL: Documentation, https://github.com/Economistician/eb-docs
11
+ Keywords: electric-barometer,forecast-evaluation,asymmetric-loss,readiness,forecasting,pandas
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3 :: Only
14
+ Classifier: Programming Language :: Python :: 3.10
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Operating System :: OS Independent
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ License-File: LICENSE
22
+ Requires-Dist: numpy>=1.24
23
+ Requires-Dist: pandas>=2.0
24
+ Requires-Dist: eb-metrics<0.3,>=0.2
25
+ Requires-Dist: eb-adapters<0.3,>=0.2
26
+ Provides-Extra: test
27
+ Requires-Dist: pytest>=8.0; extra == "test"
28
+ Requires-Dist: scikit-learn>=1.3; extra == "test"
29
+ Provides-Extra: dev
30
+ Requires-Dist: pytest>=8.0; extra == "dev"
31
+ Requires-Dist: pytest-cov>=5.0; extra == "dev"
32
+ Dynamic: license-file
33
+
34
+ # Electric Barometer · Evaluation (`eb-evaluation`)
35
+
36
+ [![CI](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml/badge.svg)](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml)
37
+ ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
38
+ ![Python Versions](https://img.shields.io/pypi/pyversions/eb-evaluation)
39
+ ![PyPI](https://img.shields.io/pypi/v/eb-evaluation)
40
+
41
+ Evaluation and model selection utilities for applying Electric Barometer metrics across entities, groups, and operational contexts.
42
+
43
+ ---
44
+
45
+ ## Overview
46
+
47
+ `eb-evaluation` provides the evaluation and model selection layer of the Electric Barometer ecosystem. It applies metric primitives to forecasts and observations across entities, groups, and hierarchical structures, enabling consistent assessment of forecasting performance in operational settings.
48
+
49
+ The package focuses on DataFrame-first evaluation workflows, including tolerance-based scoring, cost-sensitive comparison, and readiness-oriented adjustment logic. It does not define feature construction or model interfaces; instead, it consumes standardized inputs from upstream layers and produces evaluation outputs that can be used for model selection, reporting, and decision support.
50
+
51
+ ---
52
+
53
+ ## Role in the Electric Barometer Ecosystem
54
+
55
+ `eb-evaluation` defines the evaluation and model selection layer used throughout the Electric Barometer ecosystem. It is responsible for applying metric primitives to forecasts and observations across entities, groups, and hierarchies, enabling consistent comparison of forecasting performance in operational contexts.
56
+
57
+ This package focuses exclusively on evaluation logic, aggregation semantics, and selection workflows. It does not perform feature construction, model training, or metric definition. Those responsibilities are handled by adjacent layers that generate inputs, adapt model interfaces, or define metric behavior.
58
+
59
+ By separating evaluation orchestration from metric semantics and model implementation details, `eb-evaluation` provides a stable, DataFrame-first foundation for decision-aligned model comparison and readiness assessment across heterogeneous forecasting pipelines.
60
+
61
+ ---
62
+
63
+ ## Installation
64
+
65
+ `eb-evaluation` is distributed as a standard Python package.
66
+
67
+ ```bash
68
+ pip install eb-evaluation
69
+ ```
70
+
71
+ The package supports Python 3.10 and later.
72
+
73
+ ---
74
+
75
+ ## Core Concepts
76
+
77
+ - **DataFrame-first evaluation** — Evaluation logic operates directly on tabular forecast and observation data, enabling transparent aggregation, grouping, and comparison across entities and hierarchies.
78
+ - **Cost- and tolerance-aware scoring** — Forecast performance is assessed using metrics that reflect asymmetric cost, acceptable deviation thresholds, and operational risk rather than purely symmetric statistical error.
79
+ - **Hierarchical and panel semantics** — Evaluation respects entity boundaries, grouping structure, and temporal alignment, ensuring correctness in multi-level forecasting environments.
80
+ - **Model comparability** — Forecasts produced by heterogeneous models can be evaluated and compared using a consistent set of metrics and aggregation rules.
81
+ - **Readiness-oriented selection** — Model selection emphasizes execution feasibility and operational adequacy, not just aggregate accuracy, supporting decision-aligned forecasting workflows.
82
+
83
+ ---
84
+
85
+ ## Minimal Example
86
+
87
+ The example below shows how forecasts and observations can be evaluated and compared across entities using Electric Barometer metrics in a DataFrame-first workflow.
88
+
89
+ ```python
90
+ import pandas as pd
91
+ from eb_evaluation.dataframe.compare import compare_models
92
+
93
+ # Example evaluation data
94
+ df = pd.DataFrame({
95
+ "entity_id": ["A", "A", "B", "B"],
96
+ "date": pd.to_datetime(["2024-01-01", "2024-01-02"] * 2),
97
+ "actual": [10, 12, 7, 9],
98
+ "model_a": [9, 11, 8, 10],
99
+ "model_b": [11, 13, 6, 8],
100
+ })
101
+
102
+ # Compare models using a common evaluation contract
103
+ results = compare_models(
104
+ df,
105
+ actual_col="actual",
106
+ prediction_cols=["model_a", "model_b"],
107
+ entity_col="entity_id",
108
+ time_col="date",
109
+ )
110
+
111
+ print(results)
112
+ ```
113
+
114
+ ---
115
+
116
+ ## License
117
+
118
+ BSD 3-Clause License.
119
+ © 2025 Kyle Corrie.
@@ -0,0 +1,86 @@
1
+ # Electric Barometer · Evaluation (`eb-evaluation`)
2
+
3
+ [![CI](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml/badge.svg)](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml)
4
+ ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
5
+ ![Python Versions](https://img.shields.io/pypi/pyversions/eb-evaluation)
6
+ ![PyPI](https://img.shields.io/pypi/v/eb-evaluation)
7
+
8
+ Evaluation and model selection utilities for applying Electric Barometer metrics across entities, groups, and operational contexts.
9
+
10
+ ---
11
+
12
+ ## Overview
13
+
14
+ `eb-evaluation` provides the evaluation and model selection layer of the Electric Barometer ecosystem. It applies metric primitives to forecasts and observations across entities, groups, and hierarchical structures, enabling consistent assessment of forecasting performance in operational settings.
15
+
16
+ The package focuses on DataFrame-first evaluation workflows, including tolerance-based scoring, cost-sensitive comparison, and readiness-oriented adjustment logic. It does not define feature construction or model interfaces; instead, it consumes standardized inputs from upstream layers and produces evaluation outputs that can be used for model selection, reporting, and decision support.
17
+
18
+ ---
19
+
20
+ ## Role in the Electric Barometer Ecosystem
21
+
22
+ `eb-evaluation` defines the evaluation and model selection layer used throughout the Electric Barometer ecosystem. It is responsible for applying metric primitives to forecasts and observations across entities, groups, and hierarchies, enabling consistent comparison of forecasting performance in operational contexts.
23
+
24
+ This package focuses exclusively on evaluation logic, aggregation semantics, and selection workflows. It does not perform feature construction, model training, or metric definition. Those responsibilities are handled by adjacent layers that generate inputs, adapt model interfaces, or define metric behavior.
25
+
26
+ By separating evaluation orchestration from metric semantics and model implementation details, `eb-evaluation` provides a stable, DataFrame-first foundation for decision-aligned model comparison and readiness assessment across heterogeneous forecasting pipelines.
27
+
28
+ ---
29
+
30
+ ## Installation
31
+
32
+ `eb-evaluation` is distributed as a standard Python package.
33
+
34
+ ```bash
35
+ pip install eb-evaluation
36
+ ```
37
+
38
+ The package supports Python 3.10 and later.
39
+
40
+ ---
41
+
42
+ ## Core Concepts
43
+
44
+ - **DataFrame-first evaluation** — Evaluation logic operates directly on tabular forecast and observation data, enabling transparent aggregation, grouping, and comparison across entities and hierarchies.
45
+ - **Cost- and tolerance-aware scoring** — Forecast performance is assessed using metrics that reflect asymmetric cost, acceptable deviation thresholds, and operational risk rather than purely symmetric statistical error.
46
+ - **Hierarchical and panel semantics** — Evaluation respects entity boundaries, grouping structure, and temporal alignment, ensuring correctness in multi-level forecasting environments.
47
+ - **Model comparability** — Forecasts produced by heterogeneous models can be evaluated and compared using a consistent set of metrics and aggregation rules.
48
+ - **Readiness-oriented selection** — Model selection emphasizes execution feasibility and operational adequacy, not just aggregate accuracy, supporting decision-aligned forecasting workflows.
49
+
50
+ ---
51
+
52
+ ## Minimal Example
53
+
54
+ The example below shows how forecasts and observations can be evaluated and compared across entities using Electric Barometer metrics in a DataFrame-first workflow.
55
+
56
+ ```python
57
+ import pandas as pd
58
+ from eb_evaluation.dataframe.compare import compare_models
59
+
60
+ # Example evaluation data
61
+ df = pd.DataFrame({
62
+ "entity_id": ["A", "A", "B", "B"],
63
+ "date": pd.to_datetime(["2024-01-01", "2024-01-02"] * 2),
64
+ "actual": [10, 12, 7, 9],
65
+ "model_a": [9, 11, 8, 10],
66
+ "model_b": [11, 13, 6, 8],
67
+ })
68
+
69
+ # Compare models using a common evaluation contract
70
+ results = compare_models(
71
+ df,
72
+ actual_col="actual",
73
+ prediction_cols=["model_a", "model_b"],
74
+ entity_col="entity_id",
75
+ time_col="date",
76
+ )
77
+
78
+ print(results)
79
+ ```
80
+
81
+ ---
82
+
83
+ ## License
84
+
85
+ BSD 3-Clause License.
86
+ © 2025 Kyle Corrie.
@@ -0,0 +1,92 @@
1
+ ######################################
2
+ # Project metadata
3
+ ######################################
4
+ [project]
5
+ name = "eb-evaluation"
6
+ version = "0.2.2"
7
+ description = "Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics."
8
+ readme = "README.md"
9
+ requires-python = ">=3.10"
10
+ license = "BSD-3-Clause"
11
+ license-files = ["LICENSE*"]
12
+
13
+ authors = [
14
+ { name = "Kyle Corrie (Economistician)", email = "kcorrie@economistician.com" }
15
+ ]
16
+
17
+ keywords = [
18
+ "electric-barometer",
19
+ "forecast-evaluation",
20
+ "asymmetric-loss",
21
+ "readiness",
22
+ "forecasting",
23
+ "pandas",
24
+ ]
25
+
26
+ classifiers = [
27
+ "Programming Language :: Python :: 3",
28
+ "Programming Language :: Python :: 3 :: Only",
29
+ "Programming Language :: Python :: 3.10",
30
+ "Programming Language :: Python :: 3.11",
31
+ "Programming Language :: Python :: 3.12",
32
+ "Programming Language :: Python :: 3.13",
33
+ "Operating System :: OS Independent",
34
+ ]
35
+
36
+ ######################################
37
+ # Core runtime dependencies
38
+ ######################################
39
+ dependencies = [
40
+ "numpy>=1.24",
41
+ "pandas>=2.0",
42
+
43
+ # Electric Barometer compatibility band (0.2.x)
44
+ "eb-metrics>=0.2,<0.3",
45
+ "eb-adapters>=0.2,<0.3",
46
+ ]
47
+
48
+ ######################################
49
+ # Project URLs
50
+ ######################################
51
+ [project.urls]
52
+ Homepage = "https://github.com/Economistician/eb-evaluation"
53
+ Repository = "https://github.com/Economistician/eb-evaluation"
54
+ Issues = "https://github.com/Economistician/eb-evaluation/issues"
55
+ Documentation = "https://github.com/Economistician/eb-docs"
56
+
57
+ ######################################
58
+ # Optional dependencies (extras)
59
+ ######################################
60
+ [project.optional-dependencies]
61
+
62
+ # CI / test-only dependencies
63
+ test = [
64
+ "pytest>=8.0",
65
+ "scikit-learn>=1.3",
66
+ ]
67
+
68
+ # Local developer tooling
69
+ dev = [
70
+ "pytest>=8.0",
71
+ "pytest-cov>=5.0",
72
+ ]
73
+
74
+ ######################################
75
+ # Build system
76
+ ######################################
77
+ [build-system]
78
+ requires = ["setuptools>=64", "wheel"]
79
+ build-backend = "setuptools.build_meta"
80
+
81
+ ######################################
82
+ # Package discovery
83
+ ######################################
84
+ [tool.setuptools.packages.find]
85
+ where = ["src"]
86
+
87
+ ######################################
88
+ # Pytest configuration
89
+ ######################################
90
+ [tool.pytest.ini_options]
91
+ pythonpath = ["src"]
92
+ addopts = "-ra"
@@ -0,0 +1,119 @@
1
+ Metadata-Version: 2.4
2
+ Name: eb-evaluation
3
+ Version: 0.2.2
4
+ Summary: Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics.
5
+ Author-email: "Kyle Corrie (Economistician)" <kcorrie@economistician.com>
6
+ License-Expression: BSD-3-Clause
7
+ Project-URL: Homepage, https://github.com/Economistician/eb-evaluation
8
+ Project-URL: Repository, https://github.com/Economistician/eb-evaluation
9
+ Project-URL: Issues, https://github.com/Economistician/eb-evaluation/issues
10
+ Project-URL: Documentation, https://github.com/Economistician/eb-docs
11
+ Keywords: electric-barometer,forecast-evaluation,asymmetric-loss,readiness,forecasting,pandas
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3 :: Only
14
+ Classifier: Programming Language :: Python :: 3.10
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Operating System :: OS Independent
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ License-File: LICENSE
22
+ Requires-Dist: numpy>=1.24
23
+ Requires-Dist: pandas>=2.0
24
+ Requires-Dist: eb-metrics<0.3,>=0.2
25
+ Requires-Dist: eb-adapters<0.3,>=0.2
26
+ Provides-Extra: test
27
+ Requires-Dist: pytest>=8.0; extra == "test"
28
+ Requires-Dist: scikit-learn>=1.3; extra == "test"
29
+ Provides-Extra: dev
30
+ Requires-Dist: pytest>=8.0; extra == "dev"
31
+ Requires-Dist: pytest-cov>=5.0; extra == "dev"
32
+ Dynamic: license-file
33
+
34
+ # Electric Barometer · Evaluation (`eb-evaluation`)
35
+
36
+ [![CI](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml/badge.svg)](https://github.com/Economistician/eb-evaluation/actions/workflows/ci.yml)
37
+ ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
38
+ ![Python Versions](https://img.shields.io/pypi/pyversions/eb-evaluation)
39
+ ![PyPI](https://img.shields.io/pypi/v/eb-evaluation)
40
+
41
+ Evaluation and model selection utilities for applying Electric Barometer metrics across entities, groups, and operational contexts.
42
+
43
+ ---
44
+
45
+ ## Overview
46
+
47
+ `eb-evaluation` provides the evaluation and model selection layer of the Electric Barometer ecosystem. It applies metric primitives to forecasts and observations across entities, groups, and hierarchical structures, enabling consistent assessment of forecasting performance in operational settings.
48
+
49
+ The package focuses on DataFrame-first evaluation workflows, including tolerance-based scoring, cost-sensitive comparison, and readiness-oriented adjustment logic. It does not define feature construction or model interfaces; instead, it consumes standardized inputs from upstream layers and produces evaluation outputs that can be used for model selection, reporting, and decision support.
50
+
51
+ ---
52
+
53
+ ## Role in the Electric Barometer Ecosystem
54
+
55
+ `eb-evaluation` defines the evaluation and model selection layer used throughout the Electric Barometer ecosystem. It is responsible for applying metric primitives to forecasts and observations across entities, groups, and hierarchies, enabling consistent comparison of forecasting performance in operational contexts.
56
+
57
+ This package focuses exclusively on evaluation logic, aggregation semantics, and selection workflows. It does not perform feature construction, model training, or metric definition. Those responsibilities are handled by adjacent layers that generate inputs, adapt model interfaces, or define metric behavior.
58
+
59
+ By separating evaluation orchestration from metric semantics and model implementation details, `eb-evaluation` provides a stable, DataFrame-first foundation for decision-aligned model comparison and readiness assessment across heterogeneous forecasting pipelines.
60
+
61
+ ---
62
+
63
+ ## Installation
64
+
65
+ `eb-evaluation` is distributed as a standard Python package.
66
+
67
+ ```bash
68
+ pip install eb-evaluation
69
+ ```
70
+
71
+ The package supports Python 3.10 and later.
72
+
73
+ ---
74
+
75
+ ## Core Concepts
76
+
77
+ - **DataFrame-first evaluation** — Evaluation logic operates directly on tabular forecast and observation data, enabling transparent aggregation, grouping, and comparison across entities and hierarchies.
78
+ - **Cost- and tolerance-aware scoring** — Forecast performance is assessed using metrics that reflect asymmetric cost, acceptable deviation thresholds, and operational risk rather than purely symmetric statistical error.
79
+ - **Hierarchical and panel semantics** — Evaluation respects entity boundaries, grouping structure, and temporal alignment, ensuring correctness in multi-level forecasting environments.
80
+ - **Model comparability** — Forecasts produced by heterogeneous models can be evaluated and compared using a consistent set of metrics and aggregation rules.
81
+ - **Readiness-oriented selection** — Model selection emphasizes execution feasibility and operational adequacy, not just aggregate accuracy, supporting decision-aligned forecasting workflows.
82
+
83
+ ---
84
+
85
+ ## Minimal Example
86
+
87
+ The example below shows how forecasts and observations can be evaluated and compared across entities using Electric Barometer metrics in a DataFrame-first workflow.
88
+
89
+ ```python
90
+ import pandas as pd
91
+ from eb_evaluation.dataframe.compare import compare_models
92
+
93
+ # Example evaluation data
94
+ df = pd.DataFrame({
95
+ "entity_id": ["A", "A", "B", "B"],
96
+ "date": pd.to_datetime(["2024-01-01", "2024-01-02"] * 2),
97
+ "actual": [10, 12, 7, 9],
98
+ "model_a": [9, 11, 8, 10],
99
+ "model_b": [11, 13, 6, 8],
100
+ })
101
+
102
+ # Compare models using a common evaluation contract
103
+ results = compare_models(
104
+ df,
105
+ actual_col="actual",
106
+ prediction_cols=["model_a", "model_b"],
107
+ entity_col="entity_id",
108
+ time_col="date",
109
+ )
110
+
111
+ print(results)
112
+ ```
113
+
114
+ ---
115
+
116
+ ## License
117
+
118
+ BSD 3-Clause License.
119
+ © 2025 Kyle Corrie.
@@ -0,0 +1,12 @@
1
+ numpy>=1.24
2
+ pandas>=2.0
3
+ eb-metrics<0.3,>=0.2
4
+ eb-adapters<0.3,>=0.2
5
+
6
+ [dev]
7
+ pytest>=8.0
8
+ pytest-cov>=5.0
9
+
10
+ [test]
11
+ pytest>=8.0
12
+ scikit-learn>=1.3
@@ -1,169 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: eb-evaluation
3
- Version: 0.2.0
4
- Summary: Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics.
5
- Author-email: "Kyle Corrie (Economistician)" <kcorrie@economistician.com>
6
- License-Expression: BSD-3-Clause
7
- Project-URL: Homepage, https://github.com/Economistician/eb-evaluation
8
- Project-URL: Repository, https://github.com/Economistician/eb-evaluation
9
- Project-URL: Issues, https://github.com/Economistician/eb-evaluation/issues
10
- Project-URL: Documentation, https://github.com/Economistician/eb-docs
11
- Requires-Python: >=3.10
12
- Description-Content-Type: text/markdown
13
- License-File: LICENSE
14
- Requires-Dist: numpy>=1.24
15
- Requires-Dist: pandas>=2.0
16
- Requires-Dist: eb-metrics>=0.1.1
17
- Provides-Extra: dev
18
- Requires-Dist: pytest>=8.0; extra == "dev"
19
- Requires-Dist: pytest-cov>=5.0; extra == "dev"
20
- Dynamic: license-file
21
-
22
- # Electric Barometer Evaluation (`eb-evaluation`)
23
-
24
- ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
25
- ![Python Versions](https://img.shields.io/badge/Python-3.10%2B-blue)
26
- [![Docs](https://img.shields.io/badge/docs-electric--barometer-blue)](https://economistician.github.io/eb-docs/)
27
- ![Project Status](https://img.shields.io/badge/Status-Alpha-yellow)
28
-
29
- This repository contains the **evaluation and orchestration layer** of the
30
- *Electric Barometer* ecosystem.
31
-
32
- `eb-evaluation` sits above core metric implementations (`eb-metrics`) and
33
- provides structured tools for applying Electric Barometer concepts to
34
- real-world forecasting workflows, including readiness adjustment, model
35
- comparison, sensitivity analysis, and dataframe-based evaluation.
36
-
37
- Conceptual definitions and theoretical framing for the evaluation logic are
38
- maintained in the companion research repository:
39
- **`eb-papers`**.
40
-
41
- ---
42
-
43
- ## Naming convention
44
-
45
- Electric Barometer packages follow standard Python packaging conventions:
46
-
47
- - **Distribution names** (used with `pip install`) use hyphens
48
- e.g. `pip install eb-evaluation`
49
- - **Python import paths** use underscores
50
- e.g. `import eb_evaluation`
51
-
52
- This distinction is intentional and consistent across the Electric Barometer
53
- ecosystem.
54
-
55
- ---
56
-
57
- ## Role Within Electric Barometer
58
-
59
- Within the Electric Barometer ecosystem:
60
-
61
- - **`eb-papers`** defines *concepts, frameworks, and meaning*
62
- - **`eb-metrics`** implements *individual metrics*
63
- - **`eb-evaluation`** orchestrates *how metrics are applied, combined, and interpreted*
64
-
65
- This repository focuses on *evaluation logic*, not raw metric computation.
66
-
67
- ---
68
-
69
- ## What This Library Provides
70
-
71
- - **Readiness adjustment logic** for modifying evaluation outputs based on
72
- operational readiness signals
73
- - **Model selection and comparison utilities** grounded in asymmetric loss and
74
- readiness-aware metrics
75
- - **Sensitivity and tolerance analysis** for cost ratios and service thresholds
76
- - **DataFrame-oriented evaluation tools** for entity-level and time-based analysis
77
- - **Feature engineering utilities** to support evaluation pipelines
78
-
79
- ---
80
-
81
- ## Scope
82
-
83
- This repository focuses on **evaluation workflows and orchestration**, not
84
- low-level metric definitions.
85
-
86
- **In scope:**
87
- - Applying EB metrics to datasets and model outputs
88
- - Combining metrics into readiness-aware evaluation artifacts
89
- - Model comparison and selection logic
90
- - Sensitivity analysis and tolerance handling
91
-
92
- **Out of scope:**
93
- - Metric definitions and loss formulations (see `eb-metrics`)
94
- - Conceptual frameworks and theory (see `eb-papers`)
95
- - Model training or forecasting algorithms
96
-
97
- ---
98
-
99
- ## Installation
100
-
101
- Once published, the package will be installable via PyPI:
102
-
103
- ```bash
104
- pip install eb-evaluation
105
- ```
106
-
107
- For development or local use:
108
-
109
- ```bash
110
- pip install -e .
111
- ```
112
-
113
- ---
114
-
115
- ## Package Structure
116
-
117
- The repository follows a modern Python package layout:
118
-
119
- ```text
120
- eb-evaluation/
121
- ├── src/eb_evaluation/
122
- │ ├── adjustment/ # Readiness and evaluation adjustments
123
- │ ├── dataframe/ # DataFrame-based evaluation utilities
124
- │ ├── features/ # Feature engineering helpers
125
- │ ├── model_selection/ # Model comparison and selection logic
126
- │ └── utils/ # Shared validation and helpers
127
- │
128
- ├── tests/ # Unit tests mirroring package structure
129
- ├── pyproject.toml # Build and dependency configuration
130
- ├── README.md # Project documentation
131
- └── LICENSE # BSD-3-Clause license
132
- ```
133
-
134
- ---
135
-
136
- ## Relationship to Other EB Repositories
137
-
138
- - `eb-papers`
139
- Source of truth for conceptual definitions and evaluation philosophy.
140
-
141
- - `eb-metrics`
142
- Provides the metric implementations used during evaluation.
143
-
144
- - `eb-evaluation`
145
- Orchestrates evaluation workflows using adapted models.
146
-
147
- - `eb-adapters`
148
- Ensures heterogeneous models can be evaluated consistently.
149
-
150
- When discrepancies arise, conceptual intent in `eb-papers` should be treated as authoritative.
151
-
152
- ---
153
-
154
- ## Development and Testing
155
-
156
- Tests are located under the `tests/` directory and mirror the package structure.
157
-
158
- To run the test suite:
159
-
160
- ```bash
161
- pytest
162
- ```
163
-
164
- ---
165
-
166
- ## Status
167
-
168
- This package is under active development.
169
- Public APIs may evolve prior to the first stable release.
@@ -1,148 +0,0 @@
1
- # Electric Barometer Evaluation (`eb-evaluation`)
2
-
3
- ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
4
- ![Python Versions](https://img.shields.io/badge/Python-3.10%2B-blue)
5
- [![Docs](https://img.shields.io/badge/docs-electric--barometer-blue)](https://economistician.github.io/eb-docs/)
6
- ![Project Status](https://img.shields.io/badge/Status-Alpha-yellow)
7
-
8
- This repository contains the **evaluation and orchestration layer** of the
9
- *Electric Barometer* ecosystem.
10
-
11
- `eb-evaluation` sits above core metric implementations (`eb-metrics`) and
12
- provides structured tools for applying Electric Barometer concepts to
13
- real-world forecasting workflows, including readiness adjustment, model
14
- comparison, sensitivity analysis, and dataframe-based evaluation.
15
-
16
- Conceptual definitions and theoretical framing for the evaluation logic are
17
- maintained in the companion research repository:
18
- **`eb-papers`**.
19
-
20
- ---
21
-
22
- ## Naming convention
23
-
24
- Electric Barometer packages follow standard Python packaging conventions:
25
-
26
- - **Distribution names** (used with `pip install`) use hyphens
27
- e.g. `pip install eb-evaluation`
28
- - **Python import paths** use underscores
29
- e.g. `import eb_evaluation`
30
-
31
- This distinction is intentional and consistent across the Electric Barometer
32
- ecosystem.
33
-
34
- ---
35
-
36
- ## Role Within Electric Barometer
37
-
38
- Within the Electric Barometer ecosystem:
39
-
40
- - **`eb-papers`** defines *concepts, frameworks, and meaning*
41
- - **`eb-metrics`** implements *individual metrics*
42
- - **`eb-evaluation`** orchestrates *how metrics are applied, combined, and interpreted*
43
-
44
- This repository focuses on *evaluation logic*, not raw metric computation.
45
-
46
- ---
47
-
48
- ## What This Library Provides
49
-
50
- - **Readiness adjustment logic** for modifying evaluation outputs based on
51
- operational readiness signals
52
- - **Model selection and comparison utilities** grounded in asymmetric loss and
53
- readiness-aware metrics
54
- - **Sensitivity and tolerance analysis** for cost ratios and service thresholds
55
- - **DataFrame-oriented evaluation tools** for entity-level and time-based analysis
56
- - **Feature engineering utilities** to support evaluation pipelines
57
-
58
- ---
59
-
60
- ## Scope
61
-
62
- This repository focuses on **evaluation workflows and orchestration**, not
63
- low-level metric definitions.
64
-
65
- **In scope:**
66
- - Applying EB metrics to datasets and model outputs
67
- - Combining metrics into readiness-aware evaluation artifacts
68
- - Model comparison and selection logic
69
- - Sensitivity analysis and tolerance handling
70
-
71
- **Out of scope:**
72
- - Metric definitions and loss formulations (see `eb-metrics`)
73
- - Conceptual frameworks and theory (see `eb-papers`)
74
- - Model training or forecasting algorithms
75
-
76
- ---
77
-
78
- ## Installation
79
-
80
- Once published, the package will be installable via PyPI:
81
-
82
- ```bash
83
- pip install eb-evaluation
84
- ```
85
-
86
- For development or local use:
87
-
88
- ```bash
89
- pip install -e .
90
- ```
91
-
92
- ---
93
-
94
- ## Package Structure
95
-
96
- The repository follows a modern Python package layout:
97
-
98
- ```text
99
- eb-evaluation/
100
- ├── src/eb_evaluation/
101
- │ ├── adjustment/ # Readiness and evaluation adjustments
102
- │ ├── dataframe/ # DataFrame-based evaluation utilities
103
- │ ├── features/ # Feature engineering helpers
104
- │ ├── model_selection/ # Model comparison and selection logic
105
- │ └── utils/ # Shared validation and helpers
106
- │
107
- ├── tests/ # Unit tests mirroring package structure
108
- ├── pyproject.toml # Build and dependency configuration
109
- ├── README.md # Project documentation
110
- └── LICENSE # BSD-3-Clause license
111
- ```
112
-
113
- ---
114
-
115
- ## Relationship to Other EB Repositories
116
-
117
- - `eb-papers`
118
- Source of truth for conceptual definitions and evaluation philosophy.
119
-
120
- - `eb-metrics`
121
- Provides the metric implementations used during evaluation.
122
-
123
- - `eb-evaluation`
124
- Orchestrates evaluation workflows using adapted models.
125
-
126
- - `eb-adapters`
127
- Ensures heterogeneous models can be evaluated consistently.
128
-
129
- When discrepancies arise, conceptual intent in `eb-papers` should be treated as authoritative.
130
-
131
- ---
132
-
133
- ## Development and Testing
134
-
135
- Tests are located under the `tests/` directory and mirror the package structure.
136
-
137
- To run the test suite:
138
-
139
- ```bash
140
- pytest
141
- ```
142
-
143
- ---
144
-
145
- ## Status
146
-
147
- This package is under active development.
148
- Public APIs may evolve prior to the first stable release.
@@ -1,41 +0,0 @@
1
- [project]
2
- name = "eb-evaluation"
3
- version = "0.2.0"
4
- description = "Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics."
5
- readme = "README.md"
6
- requires-python = ">=3.10"
7
- license = "BSD-3-Clause"
8
- license-files = ["LICENSE*"]
9
-
10
- authors = [
11
- { name = "Kyle Corrie (Economistician)", email = "kcorrie@economistician.com" }
12
- ]
13
-
14
- dependencies = [
15
- "numpy>=1.24",
16
- "pandas>=2.0",
17
- "eb-metrics>=0.1.1",
18
- ]
19
-
20
- [project.urls]
21
- Homepage = "https://github.com/Economistician/eb-evaluation"
22
- Repository = "https://github.com/Economistician/eb-evaluation"
23
- Issues = "https://github.com/Economistician/eb-evaluation/issues"
24
- Documentation = "https://github.com/Economistician/eb-docs"
25
-
26
- [project.optional-dependencies]
27
- dev = [
28
- "pytest>=8.0",
29
- "pytest-cov>=5.0",
30
- ]
31
-
32
- [build-system]
33
- requires = ["setuptools>=64", "wheel"]
34
- build-backend = "setuptools.build_meta"
35
-
36
- [tool.setuptools.packages.find]
37
- where = ["src"]
38
-
39
- [tool.pytest.ini_options]
40
- pythonpath = ["src"]
41
- addopts = "-ra"
@@ -1,169 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: eb-evaluation
3
- Version: 0.2.0
4
- Summary: Electric Barometer: DataFrame-based evaluation utilities for CWSL and related metrics.
5
- Author-email: "Kyle Corrie (Economistician)" <kcorrie@economistician.com>
6
- License-Expression: BSD-3-Clause
7
- Project-URL: Homepage, https://github.com/Economistician/eb-evaluation
8
- Project-URL: Repository, https://github.com/Economistician/eb-evaluation
9
- Project-URL: Issues, https://github.com/Economistician/eb-evaluation/issues
10
- Project-URL: Documentation, https://github.com/Economistician/eb-docs
11
- Requires-Python: >=3.10
12
- Description-Content-Type: text/markdown
13
- License-File: LICENSE
14
- Requires-Dist: numpy>=1.24
15
- Requires-Dist: pandas>=2.0
16
- Requires-Dist: eb-metrics>=0.1.1
17
- Provides-Extra: dev
18
- Requires-Dist: pytest>=8.0; extra == "dev"
19
- Requires-Dist: pytest-cov>=5.0; extra == "dev"
20
- Dynamic: license-file
21
-
22
- # Electric Barometer Evaluation (`eb-evaluation`)
23
-
24
- ![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)
25
- ![Python Versions](https://img.shields.io/badge/Python-3.10%2B-blue)
26
- [![Docs](https://img.shields.io/badge/docs-electric--barometer-blue)](https://economistician.github.io/eb-docs/)
27
- ![Project Status](https://img.shields.io/badge/Status-Alpha-yellow)
28
-
29
- This repository contains the **evaluation and orchestration layer** of the
30
- *Electric Barometer* ecosystem.
31
-
32
- `eb-evaluation` sits above core metric implementations (`eb-metrics`) and
33
- provides structured tools for applying Electric Barometer concepts to
34
- real-world forecasting workflows, including readiness adjustment, model
35
- comparison, sensitivity analysis, and dataframe-based evaluation.
36
-
37
- Conceptual definitions and theoretical framing for the evaluation logic are
38
- maintained in the companion research repository:
39
- **`eb-papers`**.
40
-
41
- ---
42
-
43
- ## Naming convention
44
-
45
- Electric Barometer packages follow standard Python packaging conventions:
46
-
47
- - **Distribution names** (used with `pip install`) use hyphens
48
- e.g. `pip install eb-evaluation`
49
- - **Python import paths** use underscores
50
- e.g. `import eb_evaluation`
51
-
52
- This distinction is intentional and consistent across the Electric Barometer
53
- ecosystem.
54
-
55
- ---
56
-
57
- ## Role Within Electric Barometer
58
-
59
- Within the Electric Barometer ecosystem:
60
-
61
- - **`eb-papers`** defines *concepts, frameworks, and meaning*
62
- - **`eb-metrics`** implements *individual metrics*
63
- - **`eb-evaluation`** orchestrates *how metrics are applied, combined, and interpreted*
64
-
65
- This repository focuses on *evaluation logic*, not raw metric computation.
66
-
67
- ---
68
-
69
- ## What This Library Provides
70
-
71
- - **Readiness adjustment logic** for modifying evaluation outputs based on
72
- operational readiness signals
73
- - **Model selection and comparison utilities** grounded in asymmetric loss and
74
- readiness-aware metrics
75
- - **Sensitivity and tolerance analysis** for cost ratios and service thresholds
76
- - **DataFrame-oriented evaluation tools** for entity-level and time-based analysis
77
- - **Feature engineering utilities** to support evaluation pipelines
78
-
79
- ---
80
-
81
- ## Scope
82
-
83
- This repository focuses on **evaluation workflows and orchestration**, not
84
- low-level metric definitions.
85
-
86
- **In scope:**
87
- - Applying EB metrics to datasets and model outputs
88
- - Combining metrics into readiness-aware evaluation artifacts
89
- - Model comparison and selection logic
90
- - Sensitivity analysis and tolerance handling
91
-
92
- **Out of scope:**
93
- - Metric definitions and loss formulations (see `eb-metrics`)
94
- - Conceptual frameworks and theory (see `eb-papers`)
95
- - Model training or forecasting algorithms
96
-
97
- ---
98
-
99
- ## Installation
100
-
101
- Once published, the package will be installable via PyPI:
102
-
103
- ```bash
104
- pip install eb-evaluation
105
- ```
106
-
107
- For development or local use:
108
-
109
- ```bash
110
- pip install -e .
111
- ```
112
-
113
- ---
114
-
115
- ## Package Structure
116
-
117
- The repository follows a modern Python package layout:
118
-
119
- ```text
120
- eb-evaluation/
121
- ├── src/eb_evaluation/
122
- │ ├── adjustment/ # Readiness and evaluation adjustments
123
- │ ├── dataframe/ # DataFrame-based evaluation utilities
124
- │ ├── features/ # Feature engineering helpers
125
- │ ├── model_selection/ # Model comparison and selection logic
126
- │ └── utils/ # Shared validation and helpers
127
- │
128
- ├── tests/ # Unit tests mirroring package structure
129
- ├── pyproject.toml # Build and dependency configuration
130
- ├── README.md # Project documentation
131
- └── LICENSE # BSD-3-Clause license
132
- ```
133
-
134
- ---
135
-
136
- ## Relationship to Other EB Repositories
137
-
138
- - `eb-papers`
139
- Source of truth for conceptual definitions and evaluation philosophy.
140
-
141
- - `eb-metrics`
142
- Provides the metric implementations used during evaluation.
143
-
144
- - `eb-evaluation`
145
- Orchestrates evaluation workflows using adapted models.
146
-
147
- - `eb-adapters`
148
- Ensures heterogeneous models can be evaluated consistently.
149
-
150
- When discrepancies arise, conceptual intent in `eb-papers` should be treated as authoritative.
151
-
152
- ---
153
-
154
- ## Development and Testing
155
-
156
- Tests are located under the `tests/` directory and mirror the package structure.
157
-
158
- To run the test suite:
159
-
160
- ```bash
161
- pytest
162
- ```
163
-
164
- ---
165
-
166
- ## Status
167
-
168
- This package is under active development.
169
- Public APIs may evolve prior to the first stable release.
@@ -1,7 +0,0 @@
1
- numpy>=1.24
2
- pandas>=2.0
3
- eb-metrics>=0.1.1
4
-
5
- [dev]
6
- pytest>=8.0
7
- pytest-cov>=5.0
File without changes
File without changes