automl-contracts 0.1.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- automl_contracts-0.1.4/CHANGELOG.md +105 -0
- automl_contracts-0.1.4/LICENSE.md +28 -0
- automl_contracts-0.1.4/MANIFEST.in +13 -0
- automl_contracts-0.1.4/PKG-INFO +457 -0
- automl_contracts-0.1.4/README.md +432 -0
- automl_contracts-0.1.4/pyproject.toml +89 -0
- automl_contracts-0.1.4/setup.cfg +4 -0
- automl_contracts-0.1.4/src/automl_contracts/__init__.py +190 -0
- automl_contracts-0.1.4/src/automl_contracts/config/__init__.py +81 -0
- automl_contracts-0.1.4/src/automl_contracts/config/algorithms.py +256 -0
- automl_contracts-0.1.4/src/automl_contracts/config/complexity_guard.py +199 -0
- automl_contracts-0.1.4/src/automl_contracts/config/ensemble.py +95 -0
- automl_contracts-0.1.4/src/automl_contracts/config/general.py +650 -0
- automl_contracts-0.1.4/src/automl_contracts/config/general_rules.py +415 -0
- automl_contracts-0.1.4/src/automl_contracts/config/phases.py +298 -0
- automl_contracts-0.1.4/src/automl_contracts/config/root.py +68 -0
- automl_contracts-0.1.4/src/automl_contracts/config/sanity_gate.py +305 -0
- automl_contracts-0.1.4/src/automl_contracts/custom_metrics.py +94 -0
- automl_contracts-0.1.4/src/automl_contracts/definitions.py +131 -0
- automl_contracts-0.1.4/src/automl_contracts/errors.py +38 -0
- automl_contracts-0.1.4/src/automl_contracts/feature_selection.py +93 -0
- automl_contracts-0.1.4/src/automl_contracts/metrics.py +60 -0
- automl_contracts-0.1.4/src/automl_contracts/preprocessing.py +783 -0
- automl_contracts-0.1.4/src/automl_contracts/protocols.py +26 -0
- automl_contracts-0.1.4/src/automl_contracts/py.typed +0 -0
- automl_contracts-0.1.4/src/automl_contracts/result.py +630 -0
- automl_contracts-0.1.4/src/automl_contracts/schemas/config.schema.json +1728 -0
- automl_contracts-0.1.4/src/automl_contracts/search_space.py +199 -0
- automl_contracts-0.1.4/src/automl_contracts.egg-info/SOURCES.txt +26 -0
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `automl-contracts` are documented here. The format is
|
|
4
|
+
based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the
|
|
5
|
+
project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
The contract is versioned independently of `configurable-automl-engine`.
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.1.4] - 2026-11-19
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- The contract is published to public PyPI. Installation no longer requires a
|
|
15
|
+
private registry, a personal access token or the `--no-deps` workaround, and
|
|
16
|
+
the engine resolves the declared contract range automatically.
|
|
17
|
+
|
|
18
|
+
## [0.1.3] - 2026-11-19
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
|
|
22
|
+
- Version bump for publication; the public API is unchanged.
|
|
23
|
+
|
|
24
|
+
## [0.1.2] - 2026-11-18
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- Version bump for publication; the public API is unchanged.
|
|
29
|
+
|
|
30
|
+
## [0.1.1] - 2026-11-18
|
|
31
|
+
|
|
32
|
+
### Note
|
|
33
|
+
|
|
34
|
+
- The `contracts-v0.1.1` tag points to a commit where
|
|
35
|
+
`automl-contracts/pyproject.toml` declares version `0.1.0`. The tag does not
|
|
36
|
+
introduce a `0.1.1` package version, so no `0.1.1` changes are recorded.
|
|
37
|
+
|
|
38
|
+
## [0.1.0] - 2026-11-15
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- Initial public contract with configuration models, result DTOs, metric and
|
|
43
|
+
algorithm names, enumerations, exceptions, protocols and the configuration
|
|
44
|
+
JSON Schema.
|
|
45
|
+
- `py.typed` marker and the canonical
|
|
46
|
+
`automl_contracts/schemas/config.schema.json` package data.
|
|
47
|
+
- Public structural protocols `MetricFunction` and `ScorerCallable`.
|
|
48
|
+
- Stable error codes `ERR_CUSTOM_NAME_INVALID`, `ERR_REMOVED_PLUGIN_KEY`,
|
|
49
|
+
`ERR_REMOVED_REFINE_WINNER` and `ERR_REMOVED_SANITY_GATE_MODE`.
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- **BREAKING (0.x MINOR):** `CustomMetricSpec` performs structural validation
|
|
54
|
+
only. Callable import, signature, direction and name-collision checks run at
|
|
55
|
+
engine entry points instead of in the model constructor. Constructing the
|
|
56
|
+
model with an invalid callable now succeeds; `train_best_model`,
|
|
57
|
+
`AutoMLExperiment`, `optimize` and `ModelTrainer` still reject it before
|
|
58
|
+
training.
|
|
59
|
+
- **BREAKING (0.x MINOR):** direct construction of `Config`, `GeneralCfg` and
|
|
60
|
+
the re-exported engine paths performs structural validation only. Algorithm
|
|
61
|
+
dependency checks, hyperparameter compatibility, custom-metric runtime
|
|
62
|
+
checks, `check_start_method`, `check_threads_timeouts` and the joblib part of
|
|
63
|
+
`check_n_folds` moved to the engine's `validate_runtime_config` and its
|
|
64
|
+
public entry points.
|
|
65
|
+
- **BREAKING (0.x MINOR):** runtime validation errors raised by the engine's
|
|
66
|
+
`read_config`, `load_config`, `train_best_model` and `AutoMLExperiment` are
|
|
67
|
+
`ValueError` (and `pydantic_core.PydanticCustomError` for custom metrics)
|
|
68
|
+
instead of `pydantic.ValidationError`. `ValidationError` remains for
|
|
69
|
+
structural validation of contract models.
|
|
70
|
+
- **BREAKING (0.x MINOR):** reserved custom-metric names are no longer rejected
|
|
71
|
+
by the direct `Config`/`GeneralCfg` constructor. The contract checks only the
|
|
72
|
+
name format (`ERR_CUSTOM_NAME_INVALID`); the engine checks collisions with
|
|
73
|
+
built-in and scikit-learn names (`ERR_CUSTOM_NAME_COLLISION`).
|
|
74
|
+
- **BREAKING (0.x MINOR):** `CustomMetricSpec.natural_greater_is_better` is not
|
|
75
|
+
part of the contract. The engine exposes `natural_greater_is_better(spec)` in
|
|
76
|
+
`configurable_automl_engine.common.custom_metrics`; code reading the property
|
|
77
|
+
must switch to that function.
|
|
78
|
+
- **BREAKING (0.x MINOR):** `AlgoCfg.get_required_package` and
|
|
79
|
+
`AlgoCfg.get_unknown_hyperparameters` moved out of the model; the engine
|
|
80
|
+
provides the corresponding runtime helpers.
|
|
81
|
+
- Result models accept plain Python numbers; the engine normalizes NumPy
|
|
82
|
+
scalars at its boundary.
|
|
83
|
+
- `ExperimentResult` resolves built-in metric directions from the static
|
|
84
|
+
`BUILTIN_METRIC_DIRECTIONS` table and takes custom-metric directions from the
|
|
85
|
+
payload `metric_direction`; it no longer resolves sklearn metrics through a
|
|
86
|
+
process-local registry. The engine keeps the runtime direction and
|
|
87
|
+
name-collision checks for arbitrary sklearn names.
|
|
88
|
+
- `WORST_SCORE_THRESHOLD` is defined as the float32 minimum literal and no
|
|
89
|
+
longer requires NumPy.
|
|
90
|
+
|
|
91
|
+
### Deprecated
|
|
92
|
+
|
|
93
|
+
- `ScalingType` remains available as an alias in
|
|
94
|
+
`automl_contracts.preprocessing`; prefer `StringScaling` or the concrete
|
|
95
|
+
scaling models.
|
|
96
|
+
|
|
97
|
+
### Migration
|
|
98
|
+
|
|
99
|
+
- Import contract symbols from `automl_contracts` or its stable submodules.
|
|
100
|
+
The old `configurable_automl_engine` paths remain as re-exports for the
|
|
101
|
+
`0.x` contract line. See the migration table in the package README and in
|
|
102
|
+
`API_REFERENCE.md`.
|
|
103
|
+
- Regenerate `config.schema.json` after any model change. The canonical file is
|
|
104
|
+
`automl_contracts/schemas/config.schema.json`; the root `config.schema.json`
|
|
105
|
+
is a generated compatibility mirror.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
|
|
2
|
+
Copyright (c) 2026, Leonid Lapshin
|
|
3
|
+
All rights reserved.
|
|
4
|
+
|
|
5
|
+
Redistribution and use in source and binary forms, with or without
|
|
6
|
+
modification, are permitted provided that the following conditions are met:
|
|
7
|
+
|
|
8
|
+
* Redistributions of source code must retain the above copyright notice, this
|
|
9
|
+
list of conditions and the following disclaimer.
|
|
10
|
+
|
|
11
|
+
* Redistributions in binary form must reproduce the above copyright notice,
|
|
12
|
+
this list of conditions and the following disclaimer in the documentation
|
|
13
|
+
and/or other materials provided with the distribution.
|
|
14
|
+
|
|
15
|
+
* Neither the name of the copyright holder nor the names of its
|
|
16
|
+
contributors may be used to endorse or promote products derived from
|
|
17
|
+
this software without specific prior written permission.
|
|
18
|
+
|
|
19
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
20
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
21
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
22
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
|
23
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
24
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
25
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
26
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
27
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
28
|
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
include README.md
|
|
2
|
+
include CHANGELOG.md
|
|
3
|
+
include LICENSE.md
|
|
4
|
+
include src/automl_contracts/py.typed
|
|
5
|
+
include src/automl_contracts/schemas/*.json
|
|
6
|
+
prune tests
|
|
7
|
+
prune dist
|
|
8
|
+
prune scripts
|
|
9
|
+
global-exclude __pycache__ *.py[cod]
|
|
10
|
+
prune src/automl_contracts.egg-info
|
|
11
|
+
prune .mypy_cache
|
|
12
|
+
prune .ruff_cache
|
|
13
|
+
prune .import_linter_cache
|
|
@@ -0,0 +1,457 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: automl-contracts
|
|
3
|
+
Version: 0.1.4
|
|
4
|
+
Summary: Public data contracts of the Configurable AutoML Engine.
|
|
5
|
+
Author-email: Leonid Lapshin <leonid.lapshin.a@gmail.com>
|
|
6
|
+
License-Expression: BSD-3-Clause
|
|
7
|
+
Project-URL: Homepage, https://github.com/LapshinLeonid/configurable_automl_engine
|
|
8
|
+
Project-URL: Issues, https://github.com/LapshinLeonid/configurable_automl_engine/issues
|
|
9
|
+
Keywords: automl,contracts,pydantic,typing
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Typing :: Typed
|
|
16
|
+
Classifier: Intended Audience :: Developers
|
|
17
|
+
Classifier: Development Status :: 4 - Beta
|
|
18
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE.md
|
|
22
|
+
Requires-Dist: pydantic<3,>=2.12.5
|
|
23
|
+
Requires-Dist: pydantic-core<3,>=2
|
|
24
|
+
Dynamic: license-file
|
|
25
|
+
|
|
26
|
+
# automl-contracts
|
|
27
|
+
|
|
28
|
+
`automl-contracts` is the standalone public data contract of the
|
|
29
|
+
[Configurable AutoML Engine](https://github.com/LapshinLeonid/configurable_automl_engine).
|
|
30
|
+
It describes the configuration format, the result DTOs, the enumerations and
|
|
31
|
+
value sets, the public exceptions, the constants, the structural protocols and
|
|
32
|
+
the JSON Schema that external consumers may rely on.
|
|
33
|
+
|
|
34
|
+
The distribution is intentionally small: it depends only on the Python
|
|
35
|
+
standard library, Pydantic v2 and `pydantic-core`. It does not install the
|
|
36
|
+
engine, scikit-learn, pandas, NumPy, Optuna, PyYAML or any other runtime
|
|
37
|
+
dependency of the engine.
|
|
38
|
+
|
|
39
|
+
The package is useful when you need to:
|
|
40
|
+
|
|
41
|
+
- validate or generate an engine configuration without training a model;
|
|
42
|
+
- build a UI, a configuration editor or a CI validator on top of the schema;
|
|
43
|
+
- type-check custom metrics and the objects exchanged with the engine;
|
|
44
|
+
- inspect result DTOs without importing the training stack;
|
|
45
|
+
- share one definition of the contract between several services.
|
|
46
|
+
|
|
47
|
+
## What is and is not a contract
|
|
48
|
+
|
|
49
|
+
The engine separates three levels of responsibility:
|
|
50
|
+
|
|
51
|
+
| Level | What belongs there | Where it lives |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| Public library contract | Configuration models and value sets, result DTOs, statuses, public exceptions, stable error codes, public constants, `Protocol` interfaces and the JSON Schema | `automl-contracts` |
|
|
54
|
+
| Configuration contract | The structure of a configuration, field types, defaults, required fields and structural cross-field rules | `automl-contracts` |
|
|
55
|
+
| Engine implementation | Training, preprocessing, HPO, winner selection, artifacts, reports, experiment storage, explainability, parallel execution, installed-package and environment checks | `configurable-automl-engine` |
|
|
56
|
+
|
|
57
|
+
The rule for moving a definition into this package is deliberately narrow:
|
|
58
|
+
**only definitions and pure Pydantic rules move**. Anything that needs
|
|
59
|
+
scikit-learn, pandas, NumPy, Optuna, PyYAML, joblib, the installed-package
|
|
60
|
+
registry or environment inspection stays in the engine.
|
|
61
|
+
|
|
62
|
+
In particular, the following are not part of this package: `ModelTrainer`,
|
|
63
|
+
`train_model`, `optimize`, `create_model`, `FeatureSelector`,
|
|
64
|
+
`DataOversampler`, `build_preprocessor`, `DEFAULT_SPACES`, `read_config`, HPO
|
|
65
|
+
runners, reporters, the experiment store, export helpers and explainability
|
|
66
|
+
helpers. They remain public engine APIs with their own compatibility rules.
|
|
67
|
+
|
|
68
|
+
Borderline DTOs (`RunSummary`, `LeaderboardEntry`, `FeatureImportance`,
|
|
69
|
+
`EnsembleMember`, `ReporterArtifact`) are engine-only while the contract is in
|
|
70
|
+
the `0.x` series. Adding any of them later is an additive public API change and
|
|
71
|
+
must be reflected in `CHANGELOG.md` and in the compatibility matrix.
|
|
72
|
+
|
|
73
|
+
The inclusion rules, the compatibility matrix and the maintainer process are
|
|
74
|
+
documented in this guide and in
|
|
75
|
+
[the API reference](https://github.com/LapshinLeonid/configurable_automl_engine/blob/main/API_REFERENCE.md#contracts-package-automl-contracts).
|
|
76
|
+
|
|
77
|
+
## Package contents and stable import paths
|
|
78
|
+
|
|
79
|
+
The importable module is `automl_contracts`. Its top-level namespace
|
|
80
|
+
re-exports the public symbols and fixes them in `__all__`; importing it must not
|
|
81
|
+
import the engine.
|
|
82
|
+
|
|
83
|
+
| Module | What you may import from it |
|
|
84
|
+
|---|---|
|
|
85
|
+
| `automl_contracts` | The supported top-level facade: configuration models, result DTOs, enums, constants, exceptions and protocols listed in `automl_contracts.__all__` |
|
|
86
|
+
| `automl_contracts.definitions` | `ValidationStrategy`, `EncodingStrategy`, `SerializationFormat`, `TargetTransform`, `ParallelStrategy`, `RunStatus`, `RunStepStatus`, `RunPhaseAction`, `MetricDirection`, `AVAILABLE_ALGORITHMS`, `WORST_SCORE_THRESHOLD` |
|
|
87
|
+
| `automl_contracts.errors` | `TrainingError`, `InvalidPredictionError` |
|
|
88
|
+
| `automl_contracts.metrics` | `AVAILABLE_METRICS`, `ALIAS_TO_SKLEARN`, `BUILTIN_METRIC_DIRECTIONS`; `to_sklearn_name` is an internal compatibility helper kept for the old engine path |
|
|
89
|
+
| `automl_contracts.custom_metrics` | `CustomMetricKind`, `CustomMetricSpec`, `ERR_CUSTOM_NAME_INVALID` |
|
|
90
|
+
| `automl_contracts.protocols` | `MetricFunction`, `ScorerCallable` |
|
|
91
|
+
| `automl_contracts.search_space` | `SearchSpaceEntry`, `BaseDistribution`, `CategoricalSpace`, `NumericSpace`, `FloatSpace`, `IntSpace` |
|
|
92
|
+
| `automl_contracts.feature_selection` | `FeatureSelectionMode`, `FeatureSelectionMethod`, `FeatureSelectionCfg` |
|
|
93
|
+
| `automl_contracts.preprocessing` | `ObjectScalingSpec`, `ScalingSpec`, scaling and outlier specifications, `PreprocessingPreset`, `PreprocessingOverride`, `AlgorithmClass`, `coerce_scaling`, `scaling_type`, `coerce_outliers`, `format_outliers`, `merge_preprocessing_overrides`, `NO_OUTLIERS` and the deprecated `ScalingType` alias |
|
|
94
|
+
| `automl_contracts.result` | `AlgorithmRun`, `ExperimentResult` |
|
|
95
|
+
| `automl_contracts.config` | `Config`, `GeneralCfg`, `HPOPhaseCfg`, `SanityGateCfg`, `ComplexityGuardCfg`, `EnsembleCfg`, `AlgoCfg`, `OversamplingCfg`, `AlgorithmsConfig` |
|
|
96
|
+
| `automl_contracts.config.general` | `GeneralCfg`, `ComparisonMetric`, `InfPolicy`, `DatetimePolicy`, `SplitAggregation`, `AUTO_HOLDOUT_FRACTION`, `AUTO_HOLDOUT_MIN_ROWS` |
|
|
97
|
+
| `automl_contracts.config.phases` | `HPOPhaseAction`, `SelectStrategy`, `SelectOutput`, `PruningCfg`, `PruningStrategy`, `HPOPhaseCfg`, `DEFAULT_PHASE_N_TRIALS`, `ERR_REMOVED_REFINE_WINNER` |
|
|
98
|
+
| `automl_contracts.config.sanity_gate` | `SanityGateCfg`, `CorridorMode`, `GeneralizationGapMode`, `ERR_REMOVED_SANITY_GATE_MODE` |
|
|
99
|
+
| `automl_contracts.config.complexity_guard` | `ComplexityGuardCfg`, `ComplexityGuardMode`, `ComplexityGuardUserRangePolicy`, `ComplexityGuardSizeBasis` |
|
|
100
|
+
| `automl_contracts.config.ensemble` | `EnsembleCfg`, `EnsembleWeightMethod` |
|
|
101
|
+
| `automl_contracts.config.algorithms` | `AlgoCfg`, `OversamplingCfg`, `OversamplingAlgorithm`, `AlgorithmsConfig`, `AlgorithmsConfigType`, `ERR_REMOVED_PLUGIN_KEY` |
|
|
102
|
+
|
|
103
|
+
`ALIAS_TO_SKLEARN` and `BUILTIN_METRIC_DIRECTIONS` are public module data used
|
|
104
|
+
to interpret metric names and result payloads. `to_sklearn_name` is exported
|
|
105
|
+
for compatibility but is an internal helper; new code should use the alias map
|
|
106
|
+
instead.
|
|
107
|
+
|
|
108
|
+
`automl_contracts.config.root` and `automl_contracts.config.general_rules` are
|
|
109
|
+
internal implementation modules. `automl_contracts/schemas/config.schema.json`
|
|
110
|
+
is published package data, not a Python module. `ScalingType` is kept only as a
|
|
111
|
+
deprecated alias in `automl_contracts.preprocessing`; prefer `StringScaling` or
|
|
112
|
+
the concrete scaling models.
|
|
113
|
+
|
|
114
|
+
The authoritative list of top-level names is `automl_contracts.__all__`.
|
|
115
|
+
`automl_contracts.__version__` is read from the installed distribution
|
|
116
|
+
metadata.
|
|
117
|
+
|
|
118
|
+
## Installation
|
|
119
|
+
|
|
120
|
+
`automl-contracts` is published on public PyPI, so the contract and its runtime
|
|
121
|
+
dependencies install in one step:
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
python -m pip install automl-contracts
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
The engine declares `automl-contracts>=0.1.0,<0.2.0`, so
|
|
128
|
+
`pip install configurable-automl-engine` installs a compatible contract
|
|
129
|
+
automatically. The contract release is published before the engine release that
|
|
130
|
+
depends on it. A clean environment needs no credentials, private index or
|
|
131
|
+
`--no-deps` workaround. Run `python -m pip check` after installation to confirm
|
|
132
|
+
that the requirements resolve.
|
|
133
|
+
|
|
134
|
+
### Index configuration and dependency-confusion risk
|
|
135
|
+
|
|
136
|
+
The project publishes and owns the `automl-contracts` name on PyPI, so the
|
|
137
|
+
documented installation resolves to the maintained distribution. Keep a single
|
|
138
|
+
trusted index and do not add an `--extra-index-url` that can serve another
|
|
139
|
+
distribution under the same name: pip gives an additional index no priority, so
|
|
140
|
+
a same-named package can win the resolution.
|
|
141
|
+
|
|
142
|
+
If an organization needs an internal mirror, mirror the PyPI wheel and sdist
|
|
143
|
+
with their hashes, pin the exact version in requirements or constraints, and
|
|
144
|
+
review every package that resolves from the additional index. Keep `pip.conf`,
|
|
145
|
+
`.pypirc`, Docker layers and CI logs free of credentials.
|
|
146
|
+
|
|
147
|
+
### Publication pipeline
|
|
148
|
+
|
|
149
|
+
A contract release is published to public PyPI by
|
|
150
|
+
`.github/workflows/publish-contracts.yaml`. The SourceCraft `sync-to-github`
|
|
151
|
+
workflow pushes the `contracts-v<version>` tag to GitHub, which starts the
|
|
152
|
+
release workflow. The workflow uses PyPI Trusted Publishing, so the repository
|
|
153
|
+
stores no long-lived PyPI token. It runs only for a `contracts-v<version>` tag
|
|
154
|
+
and refuses to publish when:
|
|
155
|
+
|
|
156
|
+
- the tag does not match the `contracts-v<version>` mask;
|
|
157
|
+
- the tag version differs from the static version in `pyproject.toml`;
|
|
158
|
+
- any release gate fails: contract boundaries, release configuration, Ruff,
|
|
159
|
+
Mypy, import layers, contract tests, artifact metadata, the clean
|
|
160
|
+
installation of the exact wheel or the consumer E2E on that wheel;
|
|
161
|
+
- the PyPI trusted publisher for the workflow is not configured or rejects the
|
|
162
|
+
OIDC token.
|
|
163
|
+
|
|
164
|
+
The workflow checks out the tag, builds the wheel and sdist, and uploads exactly
|
|
165
|
+
the artifacts that passed the gates with `pypa/gh-action-pypi-publish`.
|
|
166
|
+
Published versions are immutable: an attempt to upload the same version again
|
|
167
|
+
fails instead of being skipped. After the upload the workflow installs
|
|
168
|
+
`automl-contracts==<version>` back from PyPI in two clean environments, once
|
|
169
|
+
from the wheel and once by building the sdist, and validates a reference
|
|
170
|
+
configuration through the public models, protocols and packaged JSON Schema,
|
|
171
|
+
without the engine.
|
|
172
|
+
|
|
173
|
+
The SourceCraft `publish-contracts` workflow additionally mirrors the released
|
|
174
|
+
artifacts to the organization registry for private consumers. It requires the
|
|
175
|
+
full `refs/tags/contracts-v<version>` reference to point at `HEAD` before the
|
|
176
|
+
private upload. Public PyPI is the canonical index documented here, and the
|
|
177
|
+
engine resolves the contract from it.
|
|
178
|
+
|
|
179
|
+
## Using the public API
|
|
180
|
+
|
|
181
|
+
### Configuration models
|
|
182
|
+
|
|
183
|
+
```python
|
|
184
|
+
from automl_contracts import Config, GeneralCfg, TargetTransform
|
|
185
|
+
|
|
186
|
+
config = Config.model_validate(
|
|
187
|
+
{
|
|
188
|
+
"general": {
|
|
189
|
+
"comparison_metric": "r2",
|
|
190
|
+
"target_transform": "log1p",
|
|
191
|
+
"holdout_fraction": 0.2,
|
|
192
|
+
},
|
|
193
|
+
"algorithms": {
|
|
194
|
+
"ridge": {},
|
|
195
|
+
"random_forest": {"enable": True},
|
|
196
|
+
},
|
|
197
|
+
}
|
|
198
|
+
)
|
|
199
|
+
|
|
200
|
+
assert isinstance(config.general, GeneralCfg)
|
|
201
|
+
assert config.general.target_transform is TargetTransform.log1p
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The direct contract constructor performs **structural** validation only:
|
|
205
|
+
unknown keys, invalid types, invalid phases and structural cross-field rules.
|
|
206
|
+
Runtime checks that need the engine's environment (installed algorithm
|
|
207
|
+
packages, estimator signatures, callable imports, thread/process limits) run
|
|
208
|
+
at the engine entry points, not in this package. An engine consumer should
|
|
209
|
+
still call `read_config`, `train_best_model` or `AutoMLExperiment` for the full
|
|
210
|
+
check. The internal `load_config` helper is not a public entry point.
|
|
211
|
+
|
|
212
|
+
### Custom metrics and protocols
|
|
213
|
+
|
|
214
|
+
```python
|
|
215
|
+
from automl_contracts import CustomMetricKind, CustomMetricSpec, MetricFunction
|
|
216
|
+
|
|
217
|
+
|
|
218
|
+
def mape(y_true, y_pred, **kwargs) -> float:
|
|
219
|
+
"""A custom metric with the engine calling convention."""
|
|
220
|
+
...
|
|
221
|
+
|
|
222
|
+
|
|
223
|
+
metric: MetricFunction = mape
|
|
224
|
+
kind: CustomMetricKind = "metric"
|
|
225
|
+
spec = CustomMetricSpec(
|
|
226
|
+
path="my_project.metrics:mape",
|
|
227
|
+
kind=kind,
|
|
228
|
+
greater_is_better=False,
|
|
229
|
+
)
|
|
230
|
+
|
|
231
|
+
assert spec.cache_key[0] == "my_project.metrics:mape"
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
`CustomMetricSpec` is a data declaration. Creating it does not import the
|
|
235
|
+
callable; imports, signatures, direction conflicts and name collisions are
|
|
236
|
+
checked by the engine before training. `MetricFunction` and `ScorerCallable`
|
|
237
|
+
are structural `Protocol` definitions for static typing; they are not runtime
|
|
238
|
+
registries and are not `runtime_checkable`.
|
|
239
|
+
|
|
240
|
+
### Exceptions and result DTOs
|
|
241
|
+
|
|
242
|
+
```python
|
|
243
|
+
from automl_contracts import (
|
|
244
|
+
AlgorithmRun,
|
|
245
|
+
ExperimentResult,
|
|
246
|
+
InvalidPredictionError,
|
|
247
|
+
RunStatus,
|
|
248
|
+
TrainingError,
|
|
249
|
+
)
|
|
250
|
+
|
|
251
|
+
try:
|
|
252
|
+
...
|
|
253
|
+
except InvalidPredictionError as exc:
|
|
254
|
+
# Both a TrainingError and a ValueError: a model returned non-finite
|
|
255
|
+
# predictions on rows it was required to predict.
|
|
256
|
+
print(exc)
|
|
257
|
+
except TrainingError as exc:
|
|
258
|
+
# The engine's data or training-parameter error.
|
|
259
|
+
print(exc)
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
`ExperimentResult` and `AlgorithmRun` are serializable DTOs: the engine writes
|
|
263
|
+
them, the store and reporters read them. Their `model_dump(mode="json")`
|
|
264
|
+
output is JSON-safe. The contract accepts plain Python numbers; the engine
|
|
265
|
+
normalizes NumPy scalars at its boundary.
|
|
266
|
+
|
|
267
|
+
## JSON Schema without the engine
|
|
268
|
+
|
|
269
|
+
The canonical schema ships as package data and is available after installing
|
|
270
|
+
only `automl-contracts`:
|
|
271
|
+
|
|
272
|
+
```python
|
|
273
|
+
import json
|
|
274
|
+
from importlib.resources import files
|
|
275
|
+
|
|
276
|
+
schema_path = files("automl_contracts").joinpath("schemas/config.schema.json")
|
|
277
|
+
schema = json.loads(schema_path.read_text(encoding="utf-8"))
|
|
278
|
+
|
|
279
|
+
print(schema["title"])
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
You can also generate the schema object in memory:
|
|
283
|
+
|
|
284
|
+
```python
|
|
285
|
+
from automl_contracts import Config
|
|
286
|
+
|
|
287
|
+
schema = Config.model_json_schema()
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
The JSON file is a build artifact of the Pydantic models, not an independent
|
|
291
|
+
source of truth. In a repository checkout, regenerate the canonical file and
|
|
292
|
+
the root compatibility mirror after changing any model:
|
|
293
|
+
|
|
294
|
+
```bash
|
|
295
|
+
python automl-contracts/scripts/generate_schema.py
|
|
296
|
+
python schema_generator.py
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
A release candidate must pass a schema reproducibility check: regenerate the
|
|
300
|
+
schema in a temporary location and require it to match the checked-in canonical
|
|
301
|
+
file byte for byte. `tests/contracts/test_json_schema.py` performs this check
|
|
302
|
+
as part of `pytest tests/contracts` in CI. Do not edit `config.schema.json` by
|
|
303
|
+
hand.
|
|
304
|
+
|
|
305
|
+
## Versioning and compatibility
|
|
306
|
+
|
|
307
|
+
`automl-contracts` uses its own SemVer version, independent of the engine
|
|
308
|
+
version. The version is static in `automl-contracts/pyproject.toml`; contract
|
|
309
|
+
releases are tagged `contracts-v<version>`, while engine releases are tagged
|
|
310
|
+
`v<version>`. `setuptools-scm` is used by the engine only, and its tag filter
|
|
311
|
+
accepts only the `v<version>` family. A `contracts-v<version>` tag at HEAD
|
|
312
|
+
therefore neither supplies an engine version nor breaks an engine build.
|
|
313
|
+
|
|
314
|
+
The current line is `0.1.x`. Until `1.0.0`:
|
|
315
|
+
|
|
316
|
+
- potentially incompatible changes are released as a MINOR version and marked
|
|
317
|
+
`BREAKING` in `CHANGELOG.md` with a migration note;
|
|
318
|
+
- PATCH is reserved for strictly compatible fixes;
|
|
319
|
+
- an incompatible change is never hidden in a PATCH release;
|
|
320
|
+
- every public API or schema change is recorded in
|
|
321
|
+
[`automl-contracts/CHANGELOG.md`](https://github.com/LapshinLeonid/configurable_automl_engine/blob/main/automl-contracts/CHANGELOG.md).
|
|
322
|
+
|
|
323
|
+
From `1.0.0`, the usual SemVer rules apply:
|
|
324
|
+
|
|
325
|
+
| Change | Release |
|
|
326
|
+
|---|---|
|
|
327
|
+
| Compatible bug fix that does not change public behavior, types or schema semantics | PATCH |
|
|
328
|
+
| New public type, protocol, module or optional field with a safe default | MINOR |
|
|
329
|
+
| New value in an `Enum` or a closed `Literal` set | MINOR or MAJOR depending on consumer behavior; a consumer that rejects an unknown value makes it MAJOR |
|
|
330
|
+
| New required field | MAJOR |
|
|
331
|
+
| Removed or renamed public symbol, field, method or value | MAJOR |
|
|
332
|
+
| Changed type or semantics of a public field or method | MAJOR |
|
|
333
|
+
| Removed old engine import path | MAJOR |
|
|
334
|
+
| Description-only or field-order-only schema change | PATCH |
|
|
335
|
+
| Schema change that breaks producers or consumers | MAJOR |
|
|
336
|
+
|
|
337
|
+
Adding a value to an `Enum` or a closed `Literal` is a producer/consumer
|
|
338
|
+
decision, not a cosmetic one: old validators may reject a payload produced by a
|
|
339
|
+
newer library. Classify it by the actual compatibility impact and state the
|
|
340
|
+
impact in the changelog.
|
|
341
|
+
|
|
342
|
+
### Compatibility matrix
|
|
343
|
+
|
|
344
|
+
The engine declares the contract range in its own `pyproject.toml`:
|
|
345
|
+
|
|
346
|
+
```toml
|
|
347
|
+
dependencies = [
|
|
348
|
+
"automl-contracts>=0.1.0,<0.2.0",
|
|
349
|
+
]
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
| `automl-contracts` | `configurable-automl-engine` | Status |
|
|
353
|
+
|---|---|---|
|
|
354
|
+
| `0.1.x` | Releases that declare `>=0.1.0,<0.2.0` | Supported; compatibility is part of the release gate |
|
|
355
|
+
| `<0.1.0` | Any | Not supported |
|
|
356
|
+
| `>=0.2.0,<1.0.0` | The current engine line | Requires a new engine release with an updated range |
|
|
357
|
+
| `>=1.0.0,<2.0.0` | A future engine that declares `>=1.0.0,<2.0.0` | Supported after the policy and range update |
|
|
358
|
+
|
|
359
|
+
The engine's range is deliberately narrow during `0.x`: a MINOR contract
|
|
360
|
+
release may contain an incompatible change, so the engine must not accept
|
|
361
|
+
`0.2.0` automatically. A combination outside the declared range is not
|
|
362
|
+
supported even if an import happens to work. Changing the range is a separate
|
|
363
|
+
change with its own changelog entry; the contract tests verify the declared
|
|
364
|
+
range.
|
|
365
|
+
|
|
366
|
+
When the engine and the contract change together, update both distributions in
|
|
367
|
+
one PR, bump the contract version by SemVer, update the engine's range and run
|
|
368
|
+
both build/test jobs before merging.
|
|
369
|
+
|
|
370
|
+
## Migration from engine imports
|
|
371
|
+
|
|
372
|
+
The engine keeps the old public import paths as re-exports of the same
|
|
373
|
+
objects. This is a transition period, not a second source of truth: there must
|
|
374
|
+
never be a duplicate class with the same name.
|
|
375
|
+
|
|
376
|
+
The table below covers the most common imports. The complete matrix, including
|
|
377
|
+
every submodule and the engine-only symbols, is in the
|
|
378
|
+
[API reference](https://github.com/LapshinLeonid/configurable_automl_engine/blob/main/API_REFERENCE.md#migration-from-engine-imports).
|
|
379
|
+
|
|
380
|
+
| Before | After |
|
|
381
|
+
|---|---|
|
|
382
|
+
| `from configurable_automl_engine import Config` | `from automl_contracts import Config` |
|
|
383
|
+
| `from configurable_automl_engine import AlgorithmRun, ExperimentResult` | `from automl_contracts import AlgorithmRun, ExperimentResult` |
|
|
384
|
+
| `from configurable_automl_engine import CustomMetricSpec` | `from automl_contracts import CustomMetricSpec` |
|
|
385
|
+
| `from configurable_automl_engine.common.definitions import ValidationStrategy, TargetTransform, MetricDirection, WORST_SCORE_THRESHOLD` | `from automl_contracts import ...` (or `automl_contracts.definitions`) |
|
|
386
|
+
| `from configurable_automl_engine.common.errors import TrainingError, InvalidPredictionError` | `from automl_contracts import TrainingError, InvalidPredictionError` (or `automl_contracts.errors`) |
|
|
387
|
+
| `from configurable_automl_engine.training_engine.config_parser import Config, GeneralCfg, HPOPhaseCfg` | `from automl_contracts import Config, GeneralCfg, HPOPhaseCfg` (or the matching `automl_contracts.config.*` module) |
|
|
388
|
+
| `from configurable_automl_engine.training_engine.config.general import GeneralCfg` | `from automl_contracts.config.general import GeneralCfg` |
|
|
389
|
+
| `from configurable_automl_engine.training_engine.config.ensemble import EnsembleCfg, EnsembleWeightMethod` | `from automl_contracts.config.ensemble import EnsembleCfg, EnsembleWeightMethod` |
|
|
390
|
+
| `from configurable_automl_engine.feature_selection import FeatureSelectionCfg` | `from automl_contracts.feature_selection import FeatureSelectionCfg` |
|
|
391
|
+
| `from configurable_automl_engine.preprocessing_presets import PreprocessingPreset` | `from automl_contracts.preprocessing import PreprocessingPreset` |
|
|
392
|
+
| `from configurable_automl_engine.outliers_presets import NoOutliers` | `from automl_contracts.preprocessing import NoOutliers` |
|
|
393
|
+
| `from configurable_automl_engine.hpo.default_spaces import SearchSpaceEntry, FloatSpace` | `from automl_contracts.search_space import SearchSpaceEntry, FloatSpace` |
|
|
394
|
+
| `from configurable_automl_engine.experiment.result import ExperimentResult` | `from automl_contracts.result import ExperimentResult` |
|
|
395
|
+
| `from configurable_automl_engine.models import AVAILABLE_ALGORITHMS` | `from automl_contracts import AVAILABLE_ALGORITHMS` |
|
|
396
|
+
| `from configurable_automl_engine.common.metrics import AVAILABLE_METRICS, to_sklearn_name` | `from automl_contracts.metrics import AVAILABLE_METRICS, to_sklearn_name` (the second name is an internal compatibility helper) |
|
|
397
|
+
| `from configurable_automl_engine.common.custom_metrics import CustomMetricSpec, CustomMetricKind` | `from automl_contracts.custom_metrics import CustomMetricSpec, CustomMetricKind` |
|
|
398
|
+
| `from configurable_automl_engine.trainer import TrainingError` | `from automl_contracts.errors import TrainingError` |
|
|
399
|
+
| `from configurable_automl_engine.trainer_params import TrainingError` | `from automl_contracts.errors import TrainingError` |
|
|
400
|
+
| `from configurable_automl_engine.tuner import WORST_SCORE_THRESHOLD` | `from automl_contracts import WORST_SCORE_THRESHOLD` |
|
|
401
|
+
| `from configurable_automl_engine.preprocessing import EncodingStrategy` | `from automl_contracts import EncodingStrategy` |
|
|
402
|
+
|
|
403
|
+
Engine-only APIs keep their old paths and do not move into the contract:
|
|
404
|
+
`read_config`, `train_best_model`, `AutoMLExperiment`, `ModelTrainer`,
|
|
405
|
+
`train_model`, `optimize`, `create_model`, `FeatureSelector`,
|
|
406
|
+
`DataOversampler`, reporters, `ExperimentStore`, `export_experiment`,
|
|
407
|
+
`explain` and `DEFAULT_SPACES`. The internal `load_config` helper is used by
|
|
408
|
+
the engine but is not a public entry point.
|
|
409
|
+
|
|
410
|
+
Re-export policy:
|
|
411
|
+
|
|
412
|
+
- all old paths in the compatibility matrix remain importable for the whole
|
|
413
|
+
`0.x` contract line;
|
|
414
|
+
- re-exported classes, enums and exceptions are identical objects (`is`);
|
|
415
|
+
type aliases and constants have equal values and types;
|
|
416
|
+
- `ScalingType` is a deprecated alias and is not part of the top-level
|
|
417
|
+
`__all__`;
|
|
418
|
+
- removal of an old path requires a MAJOR release, a deprecation period and a
|
|
419
|
+
changelog entry that names the replacement.
|
|
420
|
+
|
|
421
|
+
The exhaustive table, including submodule imports and engine-only paths, lives
|
|
422
|
+
in [the API reference](https://github.com/LapshinLeonid/configurable_automl_engine/blob/main/API_REFERENCE.md#migration-from-engine-imports).
|
|
423
|
+
|
|
424
|
+
## Changing the contract
|
|
425
|
+
|
|
426
|
+
A contract change and the engine change that consumes it are made in one pull
|
|
427
|
+
request. The mandatory order is:
|
|
428
|
+
|
|
429
|
+
1. Decide the compatibility class of the change (PATCH, MINOR or MAJOR) and
|
|
430
|
+
write it down before editing the code.
|
|
431
|
+
2. Update the contract models, enums, protocols or exceptions.
|
|
432
|
+
3. Regenerate the canonical JSON Schema and the root compatibility mirror.
|
|
433
|
+
4. Bump the static version in `automl-contracts/pyproject.toml` when the
|
|
434
|
+
public API or schema changes.
|
|
435
|
+
5. Update `automl-contracts/CHANGELOG.md` with the change, its compatibility
|
|
436
|
+
class, the affected public API or schema and the migration instructions.
|
|
437
|
+
6. Update the inclusion rules and the compatibility matrix in
|
|
438
|
+
`API_REFERENCE.md` when a boundary, a decision or a public path changes.
|
|
439
|
+
7. Update the consumer documentation and the migration table when an import
|
|
440
|
+
path changes.
|
|
441
|
+
8. Update the engine's dependency range when the contract line changes.
|
|
442
|
+
9. Run the contract tests, the engine tests, `ruff`, `mypy`, `lint-imports`,
|
|
443
|
+
the schema reproducibility check and both isolated builds.
|
|
444
|
+
10. Publish only after merge, from the `contracts-v*` tag, after the version
|
|
445
|
+
check passes.
|
|
446
|
+
|
|
447
|
+
A documentation-only change does not bump the contract version. A public API
|
|
448
|
+
or schema change without a changelog entry, a compatibility-matrix update where
|
|
449
|
+
required, a regenerated schema and a version decision is incomplete and must
|
|
450
|
+
not be merged. The contract tests check the documented public surface, the
|
|
451
|
+
declared version range and the Pydantic declarations; they do not pin a
|
|
452
|
+
snapshot of that surface and do not read `CHANGELOG.md`. The review checklist
|
|
453
|
+
is what catches a missing changelog entry or an undocumented compatibility
|
|
454
|
+
change.
|
|
455
|
+
|
|
456
|
+
The maintainer process is described above. Installation and index guidance
|
|
457
|
+
are documented in the [installation section](#installation).
|