algorithm-nexus 0.3.0__tar.gz → 1.0.0__tar.gz

{algorithm_nexus-0.3.0 → algorithm_nexus-1.0.0}/.pre-commit-config.yaml

@@ -17,7 +17,7 @@ repos:
 
   # Ruff - Python linter and formatter
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.15.20
+    rev: v0.16.8
     hooks:
       - id: ruff
         args: [--fix]
@@ -35,7 +35,7 @@ repos:
 
   # Codespell - Check for common misspellings
   - repo: https://github.com/codespell-project/codespell
-    rev: v2.4.2
+    rev: v2.4.3
     hooks:
       - id: codespell
         additional_dependencies: ["tomli"]
@@ -43,7 +43,7 @@ repos:
 
   # UV lock check
   - repo: https://github.com/astral-sh/uv-pre-commit
-    rev: 0.11.26
+    rev: 0.12.17
     hooks:
       - id: uv-lock
       - id: uv-export
@@ -64,7 +64,7 @@ repos:
 
   # Markdownlint - Markdown linting
   - repo: https://github.com/DavidAnson/markdownlint-cli2
-    rev: v0.23.0
+    rev: v0.23.3
     hooks:
       - id: markdownlint-cli2
 
@@ -76,7 +76,7 @@ repos:
 
   # Tombi - TOML formatting
   - repo: https://github.com/tombi-toml/tombi-pre-commit
-    rev: v1.2.0
+    rev: v1.5.5
     hooks:
       - id: tombi-format
         args: ["--offline"]

{algorithm_nexus-0.3.0 → algorithm_nexus-1.0.0}/.secrets.baseline

@@ -1,9 +1,9 @@
 {
   "exclude": {
-    "files": "^.secrets$|^.secrets.baseline$",
+    "files": "^.secrets$|^.secrets.baseline$|^requirements-candidate.txt$|^requirements-ecosystem.txt$|^requirements-product.txt$",
     "lines": null
   },
-  "generated_at": "2026-08-18T13:37:11Z",
+  "generated_at": "2026-09-11T09:26:16Z",
   "plugins_used": [
     {
       "name": "AWSKeyDetector"
@@ -76,18 +76,7 @@
       "name": "TwilioKeyDetector"
     }
   ],
-  "results": {
-    "requirements-ecosystem.txt": [
-      {
-        "hashed_secret": "52b44e0998fd3bcb1929ac25818191fae050cf1d",
-        "is_secret": false,
-        "is_verified": false,
-        "line_number": 825,
-        "type": "Secret Keyword",
-        "verified_result": null
-      }
-    ]
-  },
+  "results": {},
   "version": "0.13.1+ibm.64.dss",
   "word_list": {
     "file": null,

{algorithm_nexus-0.3.0 → algorithm_nexus-1.0.0}/CHANGELOG.md

@@ -1,3 +1,11 @@
+## [0.3.0](https://github.com/ibm/algorithm-nexus/compare/f6eeb79b27485ed30089ec3b8e216d8756b1971a..0.3.0) - 2026-08-31
+#### Features
+- (**dependencies**) Updated vllm to 0.28.x and terratorch to 1.2.13 (#218) - ([7bdaea9](https://github.com/ibm/algorithm-nexus/commit/7bdaea9a97534de8376c1826c424b35e65b0bf1b)) - Christian Pinto
+#### Documentation
+- (**changelog**) add release notes for 0.2.3 (#215) - ([f6eeb79](https://github.com/ibm/algorithm-nexus/commit/f6eeb79b27485ed30089ec3b8e216d8756b1971a)) - DRL-NextGen
+
+- - -
+
 ## [0.2.3](https://github.com/ibm/algorithm-nexus/compare/36f3d99129ddf64de7f5098e4f1c290b991efa18..0.2.3) - 2026-08-25
 #### Features
 - (**dependencies**) Relaxed vllm dependency in product target (#214) - ([5926fdf](https://github.com/ibm/algorithm-nexus/commit/5926fdf6e38d8def91ab5ceed2a2b01b258d24c9)) - Christian Pinto

{algorithm_nexus-0.3.0 → algorithm_nexus-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: algorithm-nexus
-Version: 0.3.0
+Version: 1.0.0
 Summary: A comprehensive library for AI model delivery, testing, and benchmarking
 Project-URL: Homepage, https://github.com/IBM/algorithm-nexus
 Project-URL: Issues, https://github.com/IBM/algorithm-nexus/issues
@@ -14,8 +14,8 @@ Requires-Dist: biomed-multi-alignment[vllm]; extra == 'candidate'
 Requires-Dist: bmfm-targets[vllm]>=0.20.0; extra == 'candidate'
 Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'candidate'
 Requires-Dist: terrakit==0.2.0; extra == 'candidate'
-Requires-Dist: terratorch[vllm]==1.2.10; extra == 'candidate'
-Requires-Dist: vllm==0.26.0; extra == 'candidate'
+Requires-Dist: terratorch[vllm]==1.2.13; extra == 'candidate'
+Requires-Dist: vllm<0.31.0,>=0.30.0; extra == 'candidate'
 Provides-Extra: cli
 Requires-Dist: ado-core>=2.0; extra == 'cli'
 Requires-Dist: pydantic>=2.13.3; extra == 'cli'
@@ -33,7 +33,7 @@ Provides-Extra: product
 Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'product'
 Requires-Dist: terrakit==0.2.0; extra == 'product'
 Requires-Dist: terratorch[vllm]==1.2.13; extra == 'product'
-Requires-Dist: vllm<0.29.0,>=0.28.0; extra == 'product'
+Requires-Dist: vllm<0.31.0,>=0.30.0; extra == 'product'
 Description-Content-Type: text/markdown
 
 # Algorithm Nexus

algorithm_nexus-1.0.0/docs/contributing/benchmarks/add_benchmark_binding.md

@@ -0,0 +1,210 @@
+<!--
+Copyright IBM Corporation 2026
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# How to bind an experiment to a problem
+
+This guide shows you how to register a benchmark binding in Algorithm Nexus.
+
+A **benchmark binding** maps an experiment's property and metric names onto a
+problem's instance properties and metrics. A submission's `space.yaml` uses the
+experiment's names; the binding is how those names correspond to the instance.
+
+The steps below bind `rlx_coloring` to the `graph_coloring` problem. After that
+walk-through, [Alternatives](#alternatives) covers matching names, several
+experiments in one file, categorical mappings, and static filters.
+
+## Prerequisites
+
+Work from a local checkout of your Algorithm Nexus fork with the CLI extra
+installed:
+
+```bash
+cd algorithm-nexus
+uv sync --group dev --extra cli
+git checkout -b add-rlx-coloring-binding
+```
+
+The `graph_coloring` problem is already registered. To add it, see
+[How to add a benchmark problem and instance](./add_benchmark_problem.md).
+
+The `rlx_coloring` experiment is already registered. To add it, see
+[How to add a benchmark experiment](./add_benchmark_experiment.md).
+
+## 1. Create the binding
+
+Create the bindings folder and write a YAML file that names the problem, the
+experiment, and the field mappings.
+
+```bash
+mkdir -p experiments/rlx_coloring/bindings
+```
+
+Create `experiments/rlx_coloring/bindings/graph_coloring_binding.yaml`:
+
+```yaml
+bindings:
+    - benchmarkIdentifier: graph_coloring
+      experiment:
+          actuatorIdentifier: custom_experiments
+          experimentIdentifier: rlx_coloring
+          experimentVersion: 1.0.0
+      targetMapping: rlx # The target (algorithm) of this experiment is always "rlx".
+      instanceMapping:
+          - benchmark: # benchmarks num_vertices is experiments n_nodes parameter
+                identifier: num_vertices
+            experiment:
+                identifier: n_nodes
+          - benchmark:
+                identifier: edge_density
+            experiment:
+                identifier: density
+          - benchmark:
+                identifier: graph_family
+            experiment:
+                identifier: graph_type
+      metricMapping:
+          - benchmark: #the benchmarks num_colors_used output is given by the experiment colors output
+                identifier: num_colors_used
+            experiment:
+                identifier: colors
+          - benchmark:
+                identifier: elapsed_ms
+            experiment:
+                identifier: runtime_ms
+```
+
+`instanceMapping` pairs each instance property with the experiment input that
+carries it. `metricMapping` does the same for result columns. `targetMapping`
+names the experiment property that identifies the target.
+
+In the above example, the experiment only implements a single "benchmark target".
+Hence, the mapping it static.
+An experiment that allows choosing an algorithms via a parameter, would have a mapping
+here as in the `instanceMapping` and `metricMapping` sections.
+
+The full field list is in
+[Logical Benchmarks and Instances](../../design/benchmarking/logical_benchmarks_and_instances.md#3-benchmark-binding).
+
+You now have:
+
+```text
+experiments/rlx_coloring/
+├── experiment_package.yaml
+└── bindings/
+    └── graph_coloring_binding.yaml
+```
+
+## 2. Validate
+
+```bash
+uv run nexus validate experiments --experiment rlx_coloring
+```
+
+A successful run lists the experiment with status `success`. The binding is
+ready to commit.
+
+## 3. Open a pull request
+
+```bash
+git add experiments/rlx_coloring/bindings
+git commit -s -m "feat(benchmark): Bind rlx_coloring to graph_coloring"
+git push origin add-rlx-coloring-binding
+```
+
+Open a pull request from your fork to the Algorithm Nexus main branch.
+
+To apply this experiment to an instance, continue with
+[How to add a benchmark submission](./add_benchmark_submission.md).
+
+---
+
+## Alternatives
+
+### Bind when names already match
+
+When the experiment already uses the problem's property and metric names, the
+binding names the problem, the experiment, and the target:
+
+```yaml
+bindings:
+    - benchmarkIdentifier: sorting
+      experiment:
+          actuatorIdentifier: custom_experiments
+          experimentIdentifier: bubble_sort
+          experimentVersion: 1.0.0
+      targetMapping: algorithm
+```
+
+### Bind several experiments in one file
+
+To bind more than one experiment from the same package, add an entry per
+experiment under `bindings:`:
+
+```yaml
+bindings:
+    - benchmarkIdentifier: sorting
+      experiment:
+          actuatorIdentifier: custom_experiments
+          experimentIdentifier: bubble_sort
+          experimentVersion: 1.0.0
+      targetMapping: algorithm
+    - benchmarkIdentifier: sorting
+      experiment:
+          actuatorIdentifier: custom_experiments
+          experimentIdentifier: merge_sort
+          experimentVersion: 1.0.0
+      targetMapping: algorithm
+```
+
+### Map a categorical instance value to experiment predicates
+
+To collapse a range of experiment inputs onto one instance category, use a
+categorical value mapping:
+
+```yaml
+instanceMapping:
+    - categoricalValue:
+          property:
+              identifier: workload
+          value: steady_state_heavy
+      predicate:
+          - identifier: request_rate
+            propertyDomain:
+                values: [-1]
+          - identifier: max_concurrency
+            propertyDomain:
+                domainRange: [200, 500]
+                variableType: CONTINUOUS_VARIABLE_TYPE
+```
+
+### Pin a property that is implicit in the problem or the experiment
+
+To assert a constant without reading it from results or the instance, add
+`staticFilters`:
+
+```yaml
+staticFilters:
+    experimentFilters:
+        - property:
+              identifier: dataset
+          value: random
+    benchmarkFilters:
+        - property:
+              identifier: graph_family
+          value: random_regular
+```
+
+See
+[section 3 of Logical Benchmarks and Instances](../../design/benchmarking/logical_benchmarks_and_instances.md#3-benchmark-binding)
+for the mapping variants.
+
+## See also
+
+- [Logical Benchmarks and Instances](../../design/benchmarking/logical_benchmarks_and_instances.md)
+  — binding schema and aggregation
+- [How to add a benchmark problem and instance](./add_benchmark_problem.md)
+- [How to add a benchmark experiment](./add_benchmark_experiment.md)
+- [How to add a benchmark submission](./add_benchmark_submission.md)
+- [`nexus validate experiments`](../../getting-started/cli-reference.md#nexus-validate-experiments)

algorithm_nexus-1.0.0/docs/contributing/benchmarks/add_benchmark_experiment.md

@@ -0,0 +1,149 @@
+<!--
+Copyright IBM Corporation 2026
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# How to add a benchmark experiment
+
+This guide shows you how to register a benchmark experiment in Algorithm Nexus.
+
+A **benchmark experiment** is an `ado` custom experiment, distributed as a Python
+package, that evaluates a target (an algorithm or model) and collects
+measurements. Registering the python package in Algorithm Nexus
+makes all the experiments it contains available to submissions.
+
+The steps below register a graph-coloring experiment whose package is already
+on PyPI. After that walk-through, [Alternatives](#alternatives) covers GitHub
+URLs, packages that expose several experiments, and authoring a new package.
+
+To register the problem this experiment relates to, see
+[How to add a benchmark problem and instance](./add_benchmark_problem.md).
+
+## Prerequisites
+
+Work from a local checkout of your Algorithm Nexus fork with the CLI extra
+installed:
+
+```bash
+cd algorithm-nexus
+uv sync --group dev --extra cli
+git checkout -b add-rlx-coloring-experiment
+```
+
+The following assumes a python package containing the experiment
+is published on PyPI (or GitHub; see
+[Point the specifier at a GitHub URL](#point-the-specifier-at-a-github-url)).
+
+## 1. Create the experiment folder
+
+Create the experiment folder and write `experiment_package.yaml`. This file
+names the Python package and the experiment identifiers it exposes.
+
+```bash
+mkdir -p experiments/rlx_coloring
+```
+
+Create `experiments/rlx_coloring/experiment_package.yaml`:
+
+```yaml
+experiment_package:
+    requirement_specifier: "rlx-coloring"
+    experiments:
+        - rlx_coloring
+```
+
+`requirement_specifier` is a PyPI package name (optionally with a version
+constraint) or a GitHub URL. The `experiments` list is the set of experiment
+ids submissions will reference.
+
+The full field list is in
+[Benchmark Experiments and Submissions](../../design/benchmarking/benchmark_experiments_and_submissions.md#31-experiment-package-registration-in-experiment_packageyaml).
+
+You now have:
+
+```text
+experiments/rlx_coloring/
+└── experiment_package.yaml
+```
+
+## 2. Validate
+
+```bash
+uv run nexus validate experiments --experiment rlx_coloring
+```
+
+A successful run lists the experiment with status `success`. The registration is
+ready to commit.
+
+## 3. Open a pull request
+
+```bash
+git add experiments/rlx_coloring
+git commit -s -m "feat(benchmark): Add rlx_coloring experiment"
+git push origin add-rlx-coloring-experiment
+```
+
+Open a pull request from your fork to the Algorithm Nexus main branch.
+
+To map this experiment onto a problem, continue with
+[How to bind an experiment to a problem](./add_benchmark_binding.md).
+To apply this experiment to a target and instance, continue with
+[How to add a benchmark submission](./add_benchmark_submission.md).
+
+---
+
+## Alternatives
+
+### Point the specifier at a GitHub URL
+
+To register a package hosted on GitHub, set `requirement_specifier` to the
+repository URL:
+
+```yaml
+experiment_package:
+    requirement_specifier: "https://github.com/<org>/<experiment-package-repo>"
+    experiments:
+        - rlx_coloring
+```
+
+Then validate as in [step 2](#2-validate).
+
+### Declare several experiment ids from one package
+
+To expose more than one experiment from the same package, list each identifier:
+
+```yaml
+experiment_package:
+    requirement_specifier: "sorting-benchmarks>=1.0.0"
+    experiments:
+        - bubble_sort
+        - merge_sort
+        - quick_sort
+```
+
+Each id in the list is available to submissions under this experiment folder.
+
+### Author a new experiment package
+
+To create the Python package itself, follow the
+[ado custom experiment template](https://ibm.github.io/ado/actuators/creating-custom-experiments/),
+publish the package on PyPI or GitHub, then continue from
+[step 1](#1-create-the-experiment-folder).
+
+### List registered experiments
+
+To see which experiments Algorithm Nexus already registers:
+
+```bash
+uv run nexus list benchmark-experiments
+```
+
+## See also
+
+- [Benchmark Experiments and Submissions](../../design/benchmarking/benchmark_experiments_and_submissions.md)
+  — schema and folder layout
+- [How to bind an experiment to a problem](./add_benchmark_binding.md)
+- [How to add a benchmark submission](./add_benchmark_submission.md)
+- [How to add a benchmark problem and instance](./add_benchmark_problem.md)
+- [`nexus validate experiments`](../../getting-started/cli-reference.md#nexus-validate-experiments)
+- [`nexus list benchmark-experiments`](../../getting-started/cli-reference.md#nexus-list-benchmark-experiments)