algorithm-nexus 1.1.0__tar.gz → 1.1.1__tar.gz

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/.agents/skills/add-benchmark-instance/SKILL.md

@@ -91,6 +91,27 @@ problemPropertyValues:
       identifier: <property_id_2>
     value: <value_2>
 
+# (Optional) Generative properties used to create the instance (e.g. seed).
+# Not declared in benchmark.yaml; must not reuse a problem property identifier.
+instancePropertyValues:
+  - property:
+      identifier: <generator_property_name>
+    value: <value>
+
+# (Optional) variants: use when one instance covers several values of a problem property.
+# A property that varies is omitted from problemPropertyValues and given a value in EVERY variant.
+# List every combination of interest; files must exist in the artifact's artifactsLocation.
+variants:
+  - problemPropertyValues:
+      - property:
+          identifier: <varying_property_id>
+        value: <value>
+    instanceArtifactValues:
+      - property:
+          identifier: <artifact_property_id>
+        values:
+          - <file-name>
+
 # (Optional) Include instanceArtifacts if the instance ships with input files
 instanceArtifacts:
   - property:

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/.pre-commit-config.yaml

@@ -17,7 +17,7 @@ repos:
 
   # Ruff - Python linter and formatter
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.16.10
+    rev: v0.17.3
     hooks:
       - id: ruff
         args: [--fix]
@@ -25,7 +25,7 @@ repos:
 
   # Conventional commits
   - repo: https://github.com/compilerla/conventional-pre-commit
-    rev: v4.4.0
+    rev: v4.5.0
     hooks:
       - id: conventional-pre-commit
         stages: [commit-msg]
@@ -43,7 +43,7 @@ repos:
 
   # UV lock check
   - repo: https://github.com/astral-sh/uv-pre-commit
-    rev: 0.12.23
+    rev: 0.13.6
     hooks:
       - id: uv-lock
       - id: uv-export
@@ -64,7 +64,7 @@ repos:
 
   # Markdownlint - Markdown linting
   - repo: https://github.com/DavidAnson/markdownlint-cli2
-    rev: v0.23.3
+    rev: v0.24.0
     hooks:
       - id: markdownlint-cli2
 
@@ -76,7 +76,7 @@ repos:
 
   # Tombi - TOML formatting
   - repo: https://github.com/tombi-toml/tombi-pre-commit
-    rev: v1.7.2
+    rev: v1.9.8
     hooks:
       - id: tombi-format
         args: ["--offline"]

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/CHANGELOG.md

@@ -1,3 +1,27 @@
+## [1.1.0](https://github.com/ibm/algorithm-nexus/compare/995ebfa4cccd7d6237dbbeda53ec151ab7fa1743..1.1.0) - 2026-10-13
+#### Features
+- (**Schema**) Updated models and cli to the latest schema for logical benchmarks and experiments (#295) - ([cc441c1](https://github.com/ibm/algorithm-nexus/commit/cc441c18922327912a932d343c2e80b66d3a7d0e)) - Christian Pinto
+- (**benchmarks**) validate instance benchmarkIdentifier matches parent benchmarkIdentifier (#302) - ([33be13b](https://github.com/ibm/algorithm-nexus/commit/33be13b034e34bc78074a94213bcf82b8340b706)) - Vassilis Vassiliadis
+- (**benchmarks**) require exactly one value for each problem property (#305) - ([c2fbd74](https://github.com/ibm/algorithm-nexus/commit/c2fbd74ccac5a49525b38767951c582c31241524)) - Vassilis Vassiliadis
+- (**dependencies**) Removed terrakit from candidate and product dependency groups (#315) - ([3b04e46](https://github.com/ibm/algorithm-nexus/commit/3b04e46b518f56d5c65b03860f6233e20057e58d)) - Christian Pinto
+- (**dependencies**) Updated vllm to version 0.31.0 for candidate and product dependency groups (#313) - ([4111b9b](https://github.com/ibm/algorithm-nexus/commit/4111b9b0f12dbe9f0aa4176982e77db21dbfa7fc)) - Christian Pinto
+- (**design**) support optional metadata in Nexus manifests (#310) - ([f438b6b](https://github.com/ibm/algorithm-nexus/commit/f438b6b7e8e4f20b741958ecd17e2cc18f454541)) - Vassilis Vassiliadis
+- (**design**) Updated logical benchmarks, instances and bindings (#290) - ([574f009](https://github.com/ibm/algorithm-nexus/commit/574f009ecad098f6485d1002cb2a98962c5df98c)) - Christian Pinto
+- (**docs**) Aligned contributing docs to latest schema (#299) - ([7693ef3](https://github.com/ibm/algorithm-nexus/commit/7693ef33764f41054d218b5a7c9773a530dbf76b)) - Christian Pinto
+- (**packages**) Temporarily removing biomed-multi-alignment (#308) - ([8e435f3](https://github.com/ibm/algorithm-nexus/commit/8e435f3287fb3238964a1459007d47f02f1a7fcc)) - Christian Pinto
+- (**skills**) Added skills for creation of logical-benchmark and instance (#303) - ([d488d65](https://github.com/ibm/algorithm-nexus/commit/d488d65eab10ac0f4b9b86aba3c52dc3364aa3f9)) - Christian Pinto
+#### Bug Fixes
+- (**cli**) Solved bug in validate logical-benchmarks that allowed any folder or file to be in a benchmark (#289) - ([c72e1c3](https://github.com/ibm/algorithm-nexus/commit/c72e1c3a49796c5fbc474d6138e040577d810ae5)) - Christian Pinto
+- (**cli**) Updated cli commands to use the new benchmarks/experiments folders structure (#285) - ([daff761](https://github.com/ibm/algorithm-nexus/commit/daff761e16f504668ff45bf6f63d76d78a49a226)) - Christian Pinto
+#### Documentation
+- (**benchmarks**) fix typo (#281) - ([bd6f5c4](https://github.com/ibm/algorithm-nexus/commit/bd6f5c46d578a1f5c33277ccedc1450eb582ca34)) - Vassilis Vassiliadis
+- (**changelog**) add release notes for 1.0.0 (#283) - ([995ebfa](https://github.com/ibm/algorithm-nexus/commit/995ebfa4cccd7d6237dbbeda53ec151ab7fa1743)) - DRL-NextGen
+#### Build system
+- (**deps**) update dependencies (#291) - ([2e16af8](https://github.com/ibm/algorithm-nexus/commit/2e16af88b97319d67ed61be171180fc1ffce749d)) - DRL-NextGen
+- (**hooks**) update pre-commit hooks (#292) - ([5960bec](https://github.com/ibm/algorithm-nexus/commit/5960becdbceb52dcf35d305cffeda55bc34109ff)) - DRL-NextGen
+
+- - -
+
 ## [1.0.0](https://github.com/ibm/algorithm-nexus/compare/f3f8054bba7a0ac14581b1d45e9c655cba984266..1.0.0) - 2026-09-29
 #### Documentation
 - (**benchmarks**) Improved benchmarks terminology and adapted schema accordingly (#250) - ([57fcfa7](https://github.com/ibm/algorithm-nexus/commit/57fcfa75960716c1adadadb5ad8ff819f839f385)) - Christian Pinto

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: algorithm-nexus
-Version: 1.1.0
+Version: 1.1.1
 Summary: A comprehensive library for AI model delivery, testing, and benchmarking
 Project-URL: Homepage, https://github.com/IBM/algorithm-nexus
 Project-URL: Issues, https://github.com/IBM/algorithm-nexus/issues
@@ -13,7 +13,7 @@ Provides-Extra: candidate
 Requires-Dist: bmfm-targets[vllm]>=0.20.0; extra == 'candidate'
 Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'candidate'
 Requires-Dist: terratorch[vllm]==1.2.13; extra == 'candidate'
-Requires-Dist: vllm<0.32.0,>=0.31.0; extra == 'candidate'
+Requires-Dist: vllm<0.34.0,>=0.33.0; extra == 'candidate'
 Provides-Extra: cli
 Requires-Dist: ado-core>=2.0; extra == 'cli'
 Requires-Dist: pydantic>=2.13.3; extra == 'cli'
@@ -28,8 +28,8 @@ Requires-Dist: terratorch>=1.2.7; extra == 'ecosystem'
 Requires-Dist: tokamind; extra == 'ecosystem'
 Provides-Extra: product
 Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'product'
-Requires-Dist: terratorch[vllm]==1.2.13; extra == 'product'
-Requires-Dist: vllm<0.32.0,>=0.31.0; extra == 'product'
+Requires-Dist: terratorch[vllm]==1.2.14; extra == 'product'
+Requires-Dist: vllm<0.34.0,>=0.33.0; extra == 'product'
 Description-Content-Type: text/markdown
 
 # Algorithm Nexus

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/docs/contributing/benchmarks/add_benchmark_experiment.md

@@ -123,10 +123,29 @@ experiment_package:
 
 Each id in the list is available to submissions under this experiment folder.
 
+### Label an experiment's compute type
+
+To label the compute type of an experiment, add a `compute-type-<experiment>`
+key to the `metadata` block of `experiment_package.yaml`. The `<experiment>`
+part must be one of the ids in the `experiments` list:
+
+```yaml
+experiment_package:
+    requirement_specifier: "rlx-coloring"
+    experiments:
+        - rlx_coloring
+
+metadata:
+    compute-type-rlx_coloring: Classical
+```
+
+See [Compute type metadata](../../design/benchmarking/benchmark_experiments_and_submissions.md#6-compute-type-metadata)
+for suggested values.
+
 ### Author a new experiment package
 
 To create the Python package itself, follow the
-[ado custom experiment template](https://ibm.github.io/ado/actuators/creating-custom-experiments/),
+[ado custom experiment template](https://ibm.github.io/ado/latest/developer-guide/creating-custom-experiments/),
 publish the package on PyPI or GitHub, then continue from
 [step 1](#1-create-the-experiment-folder).
 

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/docs/contributing/benchmarks/add_benchmark_problem.md

@@ -196,6 +196,100 @@ benchmarks/graph_coloring/instances/erdos_renyi_50_02/
     └── graph.json
 ```
 
+### Cover several values of a property with variants
+
+When the files of one instance differ by the value of a problem property (e.g. a
+sweep over `edge_density`), define a single instance with `variants` instead of
+one instance per value. A property that varies is omitted from
+`problemPropertyValues` and given a value in **every** variant. Each variant
+lists the artifact files, in the artifact's `artifactsLocation`, that belong to
+it:
+
+```yaml
+instanceIdentifier: erdos_renyi_50
+benchmarkIdentifier: graph_coloring
+description: 50-node Erdos-Renyi graphs with different edge densities
+problemPropertyValues:
+    - property:
+          identifier: graph_family
+      value: erdos_renyi
+    - property:
+          identifier: num_vertices
+      value: 50
+instanceArtifacts:
+    - property:
+          identifier: graph
+      artifactsLocation: graph_files
+variants:
+    - problemPropertyValues:
+          - property:
+                identifier: edge_density
+            value: 0.2
+      instanceArtifactValues:
+          - property:
+                identifier: graph
+            values:
+                - graph_02.dimacs
+                - graph_02.json
+    - problemPropertyValues:
+          - property:
+                identifier: edge_density
+            value: 0.5
+      instanceArtifactValues:
+          - property:
+                identifier: graph
+            values:
+                - graph_05.dimacs
+                - graph_05.json
+```
+
+The instance directory then looks like this:
+
+```text
+benchmarks/graph_coloring/instances/erdos_renyi_50/
+├── instance.yaml
+└── graph_files/
+    ├── graph_02.dimacs
+    ├── graph_02.json
+    ├── graph_05.dimacs
+    └── graph_05.json
+```
+
+- Every value a property can take for this instance must be declared in a
+  variant.
+- For artifacts that change with the variant, list all combinations of interest
+  for each property that has variants.
+
+Validation fails if a variant is missing a value, redefines a property that has
+a top-level value, duplicates another variant, or lists a file that does not
+exist. Bindings need no changes; see
+[Instances with variants](../../design/benchmarking/logical_benchmarks_and_instances.md#instances-with-variants).
+
+### Record where a benchmark comes from
+
+To record the source of a problem that comes from a public benchmark library,
+add the source keys to the `metadata` block of `benchmark.yaml`:
+
+```yaml
+metadata:
+    benchmark-source-name: QOBLIB
+    benchmark-source-url: https://zib-aopt.github.io/QOBLIB/problem/07/
+```
+
+To also point to the matching instance in that source, set the instance keys
+in the `metadata` block of `instance.yaml`. Repeat `benchmark-source-name`,
+since an instance does not inherit the metadata of its benchmark:
+
+```yaml
+metadata:
+    benchmark-source-name: QOBLIB
+    benchmark-instance-source-name: ibm32
+    benchmark-instance-source-url: https://zib-aopt.github.io/QOBLIB/instance.html?problem=07&name=ibm32
+```
+
+See [Benchmark source metadata](../../design/benchmarking/logical_benchmarks_and_instances.md#9-benchmark-source-metadata)
+for the list of keys.
+
 ### Add an instance to an existing problem
 
 To add another instance of a problem that is already registered, create a new
@@ -225,7 +319,7 @@ propertyDomain:
 ```
 
 To leave a categorical property open, omit `propertyDomain`. See the
-[ado property domain documentation](https://ibm.github.io/ado/core-concepts/properties-and-domains/)
+[ado property domain documentation](https://ibm.github.io/ado/latest/concepts/properties-and-domains/)
 and
 [section 2 of Logical Benchmarks and Instances](../../design/benchmarking/logical_benchmarks_and_instances.md#2-logical-benchmark-definition).
 

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/docs/contributing/benchmarks/add_benchmark_submission.md

@@ -101,7 +101,7 @@ Here `n_nodes`, `density`, and `graph_type` carry `num_vertices`,
 target.
 
 See
-[Using your custom experiment in a discoveryspace](https://ibm.github.io/ado/actuators/creating-custom-experiments/#using-your-custom-experiment-in-a-discoveryspace)
+[Using your custom experiment in a discoveryspace](https://ibm.github.io/ado/latest/developer-guide/creating-custom-experiments/#using-your-custom-experiment-in-a-discoveryspace)
 for the `space.yaml` syntax.
 
 You now have:

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/docs/contributing/benchmarks/add_instance_binding.md

@@ -105,8 +105,17 @@ experiments/rlx_coloring/
 uv run nexus validate experiments --experiment rlx_coloring
 ```
 
-A successful run lists the experiment with status `success`. The binding is
-ready to commit.
+A successful run lists the binding file with status `success`. Besides the
+experiment package, the command checks that:
+
+- the binding matches the schema and its `instanceBindingIdentifier` is unique
+- `experimentIdentifier` is listed in the experiment's `experiment_package.yaml`
+- `instanceReference` points to an existing benchmark and instance
+- every mapped metric and problem property exists in the benchmark
+- every `instanceArtifactMapping` names an artifact declared by the instance,
+  and each file in `validValues` exists in that artifact's `artifactsLocation`
+
+The binding is ready to commit.
 
 ## 3. Open a pull request
 

{algorithm_nexus-1.1.0 → algorithm_nexus-1.1.1}/docs/design/benchmarking/benchmark_experiments_and_submissions.md

@@ -67,7 +67,7 @@ submissions. Logical benchmarks, instances, and results are specified in
     - in this design, experiments are declared in
       `experiments/<name>/experiment_package.yaml`
     - All benchmark experiments follow the
-      [ADO custom experiment template](https://ibm.github.io/ado/actuators/creating-custom-experiments/)
+      [ADO custom experiment template](https://ibm.github.io/ado/latest/developer-guide/creating-custom-experiments/)
 
 - **Benchmark submission**
     - one registered use of a benchmark experiment on a benchmark instance for a
@@ -175,6 +175,11 @@ experiment_package:
         - bubble_sort
         - merge_sort
         - quick_sort
+
+metadata:
+    compute-type-bubble_sort: Classical
+    compute-type-merge_sort: Classical
+    compute-type-quick_sort: Classical
 ```
 
 GitHub URL example:
@@ -192,7 +197,8 @@ experiment_package:
   GitHub URL (local paths are rejected by schema validation)
 - `experiment_package.experiments` — required; non-empty list of experiment
   identifiers exposed by the package
-- `metadata` — optional; flat string-to-string dictionary for arbitrary key-value annotations (keys and values up to 4,096 characters)
+- `metadata` — optional; flat string-to-string dictionary for arbitrary
+  key-value annotations (keys and values up to 4,096 characters).
 
 ### 3.2 Benchmark Submissions in `submissions/`
 
@@ -235,7 +241,7 @@ experiments:
   identifiers declared in the sibling `experiment_package.yaml`
 
 For reference on the expected discoveryspace structure, see
-[Using your custom experiment in a discoveryspace](https://ibm.github.io/ado/actuators/creating-custom-experiments/#using-your-custom-experiment-in-a-discoveryspace).
+[Using your custom experiment in a discoveryspace](https://ibm.github.io/ado/latest/developer-guide/creating-custom-experiments/#using-your-custom-experiment-in-a-discoveryspace).
 
 ---
 
@@ -286,3 +292,38 @@ Fetching details on experiments and submissions — such as expected input,
 metrics exported, etc. — can be obtained with a combination of the `nexus` CLI
 for listing and the `ado` CLI for full experiment details after installing the
 relevant experiment package.
+
+### 5.4 Leaderboard and Discovery Filtering by Compute Type
+
+Leaderboards and experiment discovery interfaces can use the optional `metadata` in `experiment_package.yaml` to
+filter and categorize experiments. For example, a user might want to search for, or filter a leaderboard by,
+experiments of a specific compute type (e.g. classical algorithms, AI-based approaches, or hybrid quantum workflows).
+
+Using the metadata conventions, a discovery interface or leaderboard can parse labels like the following to
+automatically tag and filter results:
+
+```yaml
+metadata:
+    compute-type-bubble_sort: Classical
+    compute-type-neural_sort: AI
+    compute-type-merge_sort: AI-Classical
+```
+
+---
+
+## 6. Compute Type Metadata
+
+The optional `metadata` block of `experiment_package.yaml` can label the compute type of each experiment the package
+lists. Nexus does not interpret these keys directly. They are conventions that tools reading Nexus manifests can rely
+on.
+
+<!-- markdownlint-disable line-length -->
+
+| Key                         | Description                                                                                                                     |
+| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
+| `compute-type-<experiment>` | Compute type of `<experiment>`, which is one of the identifiers listed under `experiment_package.experiments` in the same file. |
+
+<!-- markdownlint-enable line-length -->
+
+The value is a free-form string. Suggested values are `Quantum`, `AI`, and `Classical`. For a hybrid, join two of them
+with a hyphen, for example `Quantum-Classical`, `Quantum-AI`, or `AI-Classical`.