algorithm-nexus 0.3.0__tar.gz → 1.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- algorithm_nexus-1.1.0/.agents/skills/add-benchmark-instance/SKILL.md +153 -0
- algorithm_nexus-1.1.0/.agents/skills/add-logical-benchmark/SKILL.md +187 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.pre-commit-config.yaml +5 -5
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.secrets.baseline +3 -14
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/CHANGELOG.md +32 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/PKG-INFO +6 -26
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/README.md +2 -18
- algorithm_nexus-1.1.0/docs/contributing/benchmarks/add_benchmark_experiment.md +149 -0
- algorithm_nexus-1.1.0/docs/contributing/benchmarks/add_benchmark_problem.md +263 -0
- algorithm_nexus-1.1.0/docs/contributing/benchmarks/add_benchmark_submission.md +217 -0
- algorithm_nexus-1.1.0/docs/contributing/benchmarks/add_instance_binding.md +211 -0
- algorithm_nexus-1.1.0/docs/contributing/benchmarks/index.md +59 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/contributing/index.md +3 -3
- algorithm_nexus-0.3.0/docs/design/benchmark_system.md → algorithm_nexus-1.1.0/docs/design/benchmarking/benchmark_execution.md +61 -80
- algorithm_nexus-1.1.0/docs/design/benchmarking/benchmark_experiments_and_submissions.md +288 -0
- algorithm_nexus-1.1.0/docs/design/benchmarking/index.md +64 -0
- algorithm_nexus-1.1.0/docs/design/benchmarking/logical_benchmarks_and_instances.md +755 -0
- algorithm_nexus-1.1.0/docs/design/index.md +20 -0
- algorithm_nexus-1.1.0/docs/design/nexus_package.md +155 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/getting-started/cli-reference.md +93 -95
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/index.md +2 -1
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/benchmark.md +99 -84
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/index.md +1 -1
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/pyproject.toml +12 -7
- algorithm_nexus-1.1.0/requirements-candidate.txt +3837 -0
- algorithm_nexus-1.1.0/requirements-ecosystem.txt +3485 -0
- algorithm_nexus-1.1.0/requirements-product.txt +3281 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/cli.py +5 -24
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/ado_validator.py +16 -14
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/benchmark_manager.py +453 -311
- algorithm_nexus-1.1.0/src/algorithm_nexus/commands/list.py +377 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/utils.py +23 -182
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/validate.py +320 -144
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/venv_manager.py +43 -16
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/models.py +329 -60
- algorithm_nexus-1.1.0/templates/experiment/bindings/binding-name.yaml +33 -0
- algorithm_nexus-1.1.0/templates/experiment/experiment_package.yaml +13 -0
- algorithm_nexus-1.1.0/templates/experiment/submissions/submission-name/space.yaml +19 -0
- algorithm_nexus-1.1.0/templates/logical-benchmark/benchmark.yaml +35 -0
- algorithm_nexus-1.1.0/templates/logical-benchmark/instances/instance-name/instance.yaml +26 -0
- algorithm_nexus-1.1.0/templates/nexus-package-template/nexus.yaml +5 -0
- algorithm_nexus-1.1.0/templates/packages/package-name/nexus.yaml +5 -0
- algorithm_nexus-1.1.0/tests/fixtures/experiments/vllm-performance/experiment_package.yaml +5 -0
- algorithm_nexus-1.1.0/tests/fixtures/experiments/vllm-performance/submissions/granite-guidellm/space.yaml +45 -0
- algorithm_nexus-1.1.0/tests/fixtures/experiments/vllm-performance/submissions/granite-vllm/space.yaml +44 -0
- algorithm_nexus-1.1.0/tests/fixtures/experiments-invalid/missing-config/README.md +11 -0
- algorithm_nexus-1.1.0/tests/fixtures/experiments-invalid/missing-config/submissions/run1/space.yaml +8 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_binding_unknown_metric.yaml +10 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_binding_unknown_property.yaml +8 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_instance_unknown_property.yaml +8 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_missing_required_field.yaml +7 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_ranking_unknown_metric.yaml +15 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_static_filters_overlap_with_instance_mapping.yaml +8 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/invalid_static_filters_unknown_benchmark_property.yaml +8 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_full.yaml +20 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_instance_field.yaml +13 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_metric_mapping_no_metrics_defined.yaml +10 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_no_bindings.yaml +11 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_ranking_desc.yaml +15 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_ranking_field.yaml +16 -0
- algorithm_nexus-1.1.0/tests/fixtures/logical_benchmarks/valid_static_filters_both_directions.yaml +9 -0
- algorithm_nexus-1.1.0/tests/fixtures/packages/valid-package-with-benchmarks/nexus.yaml +4 -0
- algorithm_nexus-0.3.0/tests/test_cli_list_get.py → algorithm_nexus-1.1.0/tests/test_cli_list.py +176 -189
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/test_cli_run.py +51 -91
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/test_models.py +265 -117
- algorithm_nexus-1.1.0/tests/test_utils.py +72 -0
- algorithm_nexus-1.1.0/tests/test_validate_benchmarks.py +469 -0
- algorithm_nexus-1.1.0/tests/test_validate_logical_benchmarks.py +874 -0
- algorithm_nexus-1.1.0/uv.lock +9135 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/zensical.toml +14 -7
- algorithm_nexus-0.3.0/docs/contributing/add_benchmark.md +0 -428
- algorithm_nexus-0.3.0/docs/design/benchmark_integration_design.md +0 -363
- algorithm_nexus-0.3.0/docs/design/benchmark_metadata_convention.md +0 -477
- algorithm_nexus-0.3.0/docs/design/index.md +0 -14
- algorithm_nexus-0.3.0/docs/design/nexus_package.md +0 -197
- algorithm_nexus-0.3.0/requirements-candidate.txt +0 -4876
- algorithm_nexus-0.3.0/requirements-ecosystem.txt +0 -4032
- algorithm_nexus-0.3.0/requirements-product.txt +0 -3813
- algorithm_nexus-0.3.0/src/algorithm_nexus/commands/get.py +0 -120
- algorithm_nexus-0.3.0/src/algorithm_nexus/commands/list.py +0 -349
- algorithm_nexus-0.3.0/templates/nexus-package-template/benchmark_instances/example-benchmark/space.yaml +0 -18
- algorithm_nexus-0.3.0/templates/nexus-package-template/models/your-model-name/benchmark_instances/example-benchmark/space.yaml +0 -18
- algorithm_nexus-0.3.0/templates/nexus-package-template/nexus.yaml +0 -20
- algorithm_nexus-0.3.0/templates/packages/package-name/benchmark_instances/example-benchmark/space.yaml +0 -18
- algorithm_nexus-0.3.0/templates/packages/package-name/models/model-name/benchmark_instances/example-benchmark/space.yaml +0 -18
- algorithm_nexus-0.3.0/templates/packages/package-name/nexus.yaml +0 -20
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/invalid_binding_id_mismatch.yaml +0 -19
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/invalid_binding_unknown_metric.yaml +0 -26
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/invalid_binding_unknown_property.yaml +0 -24
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/invalid_missing_required_field.yaml +0 -8
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/valid_full.yaml +0 -50
- algorithm_nexus-0.3.0/tests/fixtures/logical_benchmarks/valid_no_bindings.yaml +0 -14
- algorithm_nexus-0.3.0/tests/fixtures/packages/invalid-package-benchmarks/models/bad-model/benchmark_instances/unregistered-experiment-test/space.yaml +0 -5
- algorithm_nexus-0.3.0/tests/fixtures/packages/invalid-package-benchmarks/models/bad-model/model.yaml +0 -4
- algorithm_nexus-0.3.0/tests/fixtures/packages/invalid-package-benchmarks/nexus.yaml +0 -13
- algorithm_nexus-0.3.0/tests/fixtures/packages/valid-package-with-benchmarks/nexus.yaml +0 -15
- algorithm_nexus-0.3.0/tests/test_utils.py +0 -192
- algorithm_nexus-0.3.0/tests/test_validate_benchmarks.py +0 -271
- algorithm_nexus-0.3.0/tests/test_validate_logical_benchmarks.py +0 -113
- algorithm_nexus-0.3.0/uv.lock +0 -10707
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.agents/skills/add-nexus-package/SKILL.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.agents/skills/update-lock-file/SKILL.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.copywrite.hcl +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.github/PULL_REQUEST_TEMPLATE/new_nexus_package.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.github/dco.yml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.gitignore +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.markdownlint-cli2.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.prettierrc.toml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.python-version +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/.yamlfmt +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/CONTRIBUTING.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/LICENSE +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/MAINTAINERS.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/benchmarks/.gitkeep +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/assets/images/favicon.ico +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/assets/images/logo-ibm-dark.png +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/assets/images/logo-ibm.png +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/contributing/add_new_nexus_package.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/contributing/guidelines.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/contributing/owner_responsibilities.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/design/dependency-resolution.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/design/project_release.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/getting-started/index.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/getting-started/installation.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/model_vllm.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/models_testing.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/nexus_package.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/requirements/packaging_and_dependency_reqs.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/stylesheets/extra.css +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/theme/favicon.ico +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/theme/logo-ibm-dark.png +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/docs/theme/logo-ibm.png +0 -0
- {algorithm_nexus-0.3.0/packages → algorithm_nexus-1.1.0/experiments}/.gitkeep +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/mendUnifiedAgent.config +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/package.json +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/invalid-package-benchmarks/models/bad-model/benchmark_instances/missing-space-yaml → algorithm_nexus-1.1.0/packages}/.gitkeep +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/biomed-multi-alignment/models/biomed.omics.bl.sm.ma-ted-458m/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/biomed-multi-alignment/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/bmfm-targets/models/biomed.rna.llama.32m.mlm.multitask.v1.vllm/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/bmfm-targets/models/biomed.rna.llama.32m.mlm.multitask.v1.vllm/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/bmfm-targets/models/biomed.rna.llama.47m.wced.multitask.v1.vllm/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/bmfm-targets/models/biomed.rna.llama.47m.wced.multitask.v1.vllm/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/bmfm-targets/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/gridfm-graphkit/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terrakit/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/prithvi/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/prithvi/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/terramind_base_fire/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/terramind_base_fire/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/terramind_base_flood/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/models/terramind_base_flood/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/terratorch/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/tokamind/models/tokamind-base-v2/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/packages/tokamind/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/renovate.json +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/ruff.toml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/__init__.py +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/__init__.py +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/src/algorithm_nexus/commands/run.py +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/templates/nexus-package-template/README.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/templates/nexus-package-template/models/your-model-name/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/templates/packages/package-name/README.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/templates/packages/package-name/models/model-name/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/__init__.py +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/.gitignore +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/invalid-package/models/broken-model/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/invalid-package/models/undeclared-model/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/invalid-package/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/terratorch/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/terratorch/benchmark_submissions}/base-test/space.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_submissions}/fire-test/space.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_submissions}/flood-test/config.json +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/terratorch/models/prithvi/benchmark_submissions}/flood-test/space.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/terratorch/models/prithvi/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/terratorch/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/tokamind/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/tokamind/benchmark_submissions}/base-test/space.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/tokamind/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package/AGENTS.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package/models/example-model/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package/models/example-model/usage.md +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package/models/minimal-model/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package/nexus.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/valid-package-with-benchmarks/models/test-model/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/valid-package-with-benchmarks/models/test-model/benchmark_submissions}/flood-segmentation-test/space.yaml +0 -0
- {algorithm_nexus-0.3.0/tests/fixtures/packages/valid-package-with-benchmarks/models/test-model/benchmark_instances → algorithm_nexus-1.1.0/tests/fixtures/packages/valid-package-with-benchmarks/models/test-model/benchmark_submissions}/performance-baseline/space.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/fixtures/packages/valid-package-with-benchmarks/models/test-model/model.yaml +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tests/test_cli_validation.py +0 -0
- {algorithm_nexus-0.3.0 → algorithm_nexus-1.1.0}/tombi.toml +0 -0
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: add-benchmark-instance
|
|
3
|
+
description: >
|
|
4
|
+
Step-by-step guidance for adding a new problem instance (benchmark instance)
|
|
5
|
+
to an existing logical benchmark in Algorithm Nexus. Creates instance.yaml
|
|
6
|
+
(and optional artifact files), validates the instance definition, and opens a
|
|
7
|
+
pull request. Use when the user wants to add a new problem instance, benchmark
|
|
8
|
+
instance, or concrete instance to Algorithm Nexus.
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Adding a Benchmark Instance to Algorithm Nexus
|
|
12
|
+
|
|
13
|
+
## Prerequisites
|
|
14
|
+
|
|
15
|
+
The following information is required before creating a benchmark instance:
|
|
16
|
+
|
|
17
|
+
- **Target logical benchmark (`benchmark_id`)**: The snake-case identifier of the existing problem class
|
|
18
|
+
(e.g. `graph_coloring`, `portfolio_optimization`). The benchmark definition must already exist at
|
|
19
|
+
`benchmarks/<benchmark_id>/benchmark.yaml`.
|
|
20
|
+
> **Note**: If the target problem does not exist yet, use the skill in
|
|
21
|
+
> `.agents/skills/add-logical-benchmark` to add the logical benchmark first before proceeding.
|
|
22
|
+
- `instance_id` (string, required): Unique identifier for this instance, e.g. `erdos_renyi_50_02` or
|
|
23
|
+
`100nodes_random3regular`. This becomes the folder name under
|
|
24
|
+
`benchmarks/<benchmark_id>/instances/<instance_id>/` and the `instanceIdentifier` field value.
|
|
25
|
+
- `description` (string, optional/recommended): Human-readable description of this concrete instance.
|
|
26
|
+
- `problem_property_values` (list of PropertyValue, required if `problemProperties` are defined in
|
|
27
|
+
`benchmark.yaml`): Concrete values corresponding to the problem properties defined in
|
|
28
|
+
`benchmarks/<benchmark_id>/benchmark.yaml`. Each entry specifies:
|
|
29
|
+
- `property.identifier`: The identifier matching a `problemProperty` from `benchmark.yaml`.
|
|
30
|
+
- `value`: The concrete value for this instance (e.g. integer, float, string).
|
|
31
|
+
- `instance_artifacts` (list of InstanceArtifact, optional): If the instance provides pre-generated
|
|
32
|
+
input files (such as graphs, datasets, or mathematical models) that algorithms consume directly:
|
|
33
|
+
- `property.identifier`: Identifier for the artifact slot (e.g. `graph`, `model`, `market_data`).
|
|
34
|
+
- `artifactsLocation`: Subfolder name relative to the instance directory containing the files.
|
|
35
|
+
|
|
36
|
+
If the user has not provided enough information, ask for the missing details before creating files.
|
|
37
|
+
|
|
38
|
+
## Steps
|
|
39
|
+
|
|
40
|
+
1. **Gather information & verify problem existence** — verify the logical benchmark exists and confirm
|
|
41
|
+
all instance property values and artifacts.
|
|
42
|
+
2. **Create the instance definition** — write
|
|
43
|
+
`benchmarks/<benchmark_id>/instances/<instance_id>/instance.yaml` and create artifact files if applicable.
|
|
44
|
+
3. **Validate** — run `uv run nexus validate logical-benchmarks` and resolve any errors.
|
|
45
|
+
4. **Commit, push, and open a PR** — commit changes to a new branch and open a pull request.
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## Step 1 — Gather Information & Verify Problem Existence
|
|
50
|
+
|
|
51
|
+
1. Verify that the parent logical benchmark exists at `benchmarks/<benchmark_id>/benchmark.yaml`.
|
|
52
|
+
- If `benchmark.yaml` does not exist or the user has not specified a problem, stop and guide the
|
|
53
|
+
user to create the benchmark problem first using the skill in `.agents/skills/add-logical-benchmark`.
|
|
54
|
+
2. Inspect `benchmarks/<benchmark_id>/benchmark.yaml` to retrieve the declared `problemProperties`.
|
|
55
|
+
3. Check whether an instance with the same `instance_id` already exists under
|
|
56
|
+
`benchmarks/<benchmark_id>/instances/<instance_id>/`. If it already exists, stop and inform the user.
|
|
57
|
+
4. Distinguish between problem properties and instance artifacts:
|
|
58
|
+
- **Problem property values** define the parameters of the problem (e.g. `num_vertices: 50`,
|
|
59
|
+
`edge_density: 0.2`) and must match properties declared in `benchmark.yaml`.
|
|
60
|
+
- **Instance artifacts** are concrete input files (e.g. `.dimacs`, `.mps`, `.json`, `.csv`) stored
|
|
61
|
+
in a subfolder and referenced in `instanceArtifacts`. Artifact identifiers do not need to be
|
|
62
|
+
listed in `problemProperties`.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Step 2 — Create the Instance Definition
|
|
67
|
+
|
|
68
|
+
Create the instance folder:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
mkdir -p benchmarks/<benchmark_id>/instances/<instance_id>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Write `benchmarks/<benchmark_id>/instances/<instance_id>/instance.yaml` following the schema below,
|
|
75
|
+
including the IBM copyright header:
|
|
76
|
+
|
|
77
|
+
```yaml
|
|
78
|
+
# Copyright IBM Corp. 2026
|
|
79
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
80
|
+
|
|
81
|
+
instanceIdentifier: <instance_id>
|
|
82
|
+
benchmarkIdentifier: <benchmark_id>
|
|
83
|
+
description: >
|
|
84
|
+
<description>
|
|
85
|
+
|
|
86
|
+
problemPropertyValues:
|
|
87
|
+
- property:
|
|
88
|
+
identifier: <property_id_1>
|
|
89
|
+
value: <value_1>
|
|
90
|
+
- property:
|
|
91
|
+
identifier: <property_id_2>
|
|
92
|
+
value: <value_2>
|
|
93
|
+
|
|
94
|
+
# (Optional) Include instanceArtifacts if the instance ships with input files
|
|
95
|
+
instanceArtifacts:
|
|
96
|
+
- property:
|
|
97
|
+
identifier: <artifact_property_name>
|
|
98
|
+
artifactsLocation: <artifacts_subfolder>
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
### Artifact Files (Optional)
|
|
102
|
+
|
|
103
|
+
If the instance includes artifact files:
|
|
104
|
+
|
|
105
|
+
1. Create the subfolder under the instance directory:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
mkdir -p benchmarks/<benchmark_id>/instances/<instance_id>/<artifacts_subfolder>
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
2. Place or generate the required artifact files in that subfolder.
|
|
112
|
+
3. Ensure `artifactsLocation` in `instance.yaml` matches the exact name of the subfolder.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Step 3 — Validate
|
|
117
|
+
|
|
118
|
+
Run the validation command from the repository root:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
uv run nexus validate logical-benchmarks
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
To validate only the specific benchmark and its instances:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
uv run nexus validate logical-benchmarks --file benchmarks/<benchmark_id>
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
A successful validation prints `success` for the benchmark. If validation fails:
|
|
131
|
+
|
|
132
|
+
1. Read the error messages carefully (e.g. unknown problem property identifier, schema validation
|
|
133
|
+
errors, missing `artifactsLocation` subfolder).
|
|
134
|
+
2. Fix `instance.yaml` or artifact files.
|
|
135
|
+
3. Re-run validation until all checks pass.
|
|
136
|
+
|
|
137
|
+
Do not proceed to the PR step until validation succeeds without errors.
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Step 4 — Commit, Push, and Open a PR
|
|
142
|
+
|
|
143
|
+
Create a feature branch, stage all instance files, commit, and push:
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
git checkout -b add-<instance_id>-instance
|
|
147
|
+
git add benchmarks/<benchmark_id>/instances/<instance_id>
|
|
148
|
+
git commit -s -m "feat(benchmark): Add <instance_id> instance to <benchmark_id>"
|
|
149
|
+
git push origin add-<instance_id>-instance
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Then open a pull request from the new branch to the Algorithm Nexus `main` branch.
|
|
153
|
+
Use the `create_pr_workflow` if available to generate the PR description automatically.
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: add-logical-benchmark
|
|
3
|
+
description: >
|
|
4
|
+
Step-by-step guidance for adding a new problem class (logical benchmark) to
|
|
5
|
+
Algorithm Nexus. Creates benchmark.yaml, validates it, and opens a pull
|
|
6
|
+
request. Use when the user wants to add a new benchmark problem, problem
|
|
7
|
+
class, or logical-benchmark to Algorithm Nexus.
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Adding a Logical Benchmark (Problem Class) to Algorithm Nexus
|
|
11
|
+
|
|
12
|
+
## Prerequisites
|
|
13
|
+
|
|
14
|
+
The following information is required before starting. If any required item is
|
|
15
|
+
missing, ask the user before proceeding.
|
|
16
|
+
|
|
17
|
+
- `benchmark_id` (string, required): Snake-case unique identifier for the
|
|
18
|
+
problem, e.g. `graph_coloring`. This becomes both the folder name and the
|
|
19
|
+
`benchmarkIdentifier` value.
|
|
20
|
+
- `title` (string, optional): Short human-readable display name.
|
|
21
|
+
- `description` (string, required): What the benchmark evaluates — algorithm
|
|
22
|
+
class, inputs, goal.
|
|
23
|
+
- `problem_properties` (list, required): The properties that characterise the
|
|
24
|
+
*abstract* problem — complexity parameters, size dimensions, structural
|
|
25
|
+
settings. Each property needs an `identifier` and an optional human-readable
|
|
26
|
+
`description`. A `propertyDomain` is optional (see Step 2).
|
|
27
|
+
- `metrics` (list, optional): Canonical output metric names that every
|
|
28
|
+
experiment binding will report under. Each metric has an `identifier` and an
|
|
29
|
+
optional `metadata.description` — use it to explain what the metric measures
|
|
30
|
+
and its unit, rather than leaving that detail to the README.
|
|
31
|
+
- `ranking_metric` and `ranking_order` (`asc` or `desc`, optional): Which
|
|
32
|
+
metric drives the leaderboard and in which direction.
|
|
33
|
+
- `owner` (string, optional): GitHub team or username, e.g. `@my-team`.
|
|
34
|
+
|
|
35
|
+
If the user has not provided enough information to fill in all required fields,
|
|
36
|
+
ask for it before creating any files.
|
|
37
|
+
|
|
38
|
+
## Steps
|
|
39
|
+
|
|
40
|
+
1. **Gather information** — confirm all required prerequisite information.
|
|
41
|
+
2. **Create the problem definition** — write `benchmarks/<benchmark_id>/benchmark.yaml`.
|
|
42
|
+
3. **Write the README** — write `benchmarks/<benchmark_id>/README.md`.
|
|
43
|
+
4. **Validate** — run `uv run nexus validate logical-benchmarks` and fix any errors.
|
|
44
|
+
5. **Commit, push, and open a PR** — commit all changes to a new branch and open a pull request.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Step 1 — Gather Information
|
|
49
|
+
|
|
50
|
+
Before writing any file, confirm all prerequisite information is available.
|
|
51
|
+
Inspect the workspace to check whether a benchmark with the same identifier
|
|
52
|
+
already exists under `benchmarks/`. If it does, stop and inform the user.
|
|
53
|
+
|
|
54
|
+
### Distinguish problem properties from instance artifacts
|
|
55
|
+
|
|
56
|
+
Before finalising `problemProperties`, review each proposed property with the
|
|
57
|
+
following question:
|
|
58
|
+
|
|
59
|
+
> **Does this property describe the abstract problem (its size, structure, or
|
|
60
|
+
> parameters), or does it represent a pre-generated input file that an
|
|
61
|
+
> algorithm will consume directly?**
|
|
62
|
+
|
|
63
|
+
- **Scalar/categorical parameters** that characterise problem complexity (e.g.
|
|
64
|
+
number of nodes, graph family, budget, risk-aversion weight) → declare as
|
|
65
|
+
`problemProperties` in `benchmark.yaml`.
|
|
66
|
+
- **Files or datasets** that are derived from those parameters and fed directly
|
|
67
|
+
to algorithms (e.g. a graph in DIMACS format, an MPS model file, a JSON
|
|
68
|
+
representation) → these are **instance artifacts**, not problem properties.
|
|
69
|
+
They belong in `instanceArtifacts` on the instance, not in `problemProperties`
|
|
70
|
+
on the benchmark. Do not add them to `problemProperties`.
|
|
71
|
+
|
|
72
|
+
If the user describes a property that is clearly a file input, flag it and
|
|
73
|
+
explain that it should be an instance artifact instead. Only proceed with
|
|
74
|
+
writing `benchmark.yaml` once this distinction has been applied.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Step 2 — Create the Problem Definition
|
|
79
|
+
|
|
80
|
+
Create the benchmark folder and `benchmark.yaml`:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
mkdir -p benchmarks/<benchmark_id>
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Write `benchmarks/<benchmark_id>/benchmark.yaml` with the following structure,
|
|
87
|
+
including the IBM copyright header:
|
|
88
|
+
|
|
89
|
+
```yaml
|
|
90
|
+
# Copyright IBM Corp. 2026
|
|
91
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
92
|
+
|
|
93
|
+
benchmarkIdentifier: <benchmark_id>
|
|
94
|
+
title: <title>
|
|
95
|
+
description: >
|
|
96
|
+
<description>
|
|
97
|
+
problemProperties:
|
|
98
|
+
- identifier: <property_id>
|
|
99
|
+
metadata:
|
|
100
|
+
description: <property description>
|
|
101
|
+
# propertyDomain is optional — omit it when the property is open-ended
|
|
102
|
+
propertyDomain:
|
|
103
|
+
variableType: <DOMAIN_TYPE>
|
|
104
|
+
values: [...] # for CATEGORICAL_VARIABLE_TYPE or DISCRETE_VARIABLE_TYPE
|
|
105
|
+
metrics:
|
|
106
|
+
- identifier: <metric_id>
|
|
107
|
+
metadata:
|
|
108
|
+
description: <what this metric measures and its unit>
|
|
109
|
+
ranking:
|
|
110
|
+
metric: <metric_id>
|
|
111
|
+
order: asc # or desc
|
|
112
|
+
owner: "<owner>"
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Only include `title`, `metrics`, `ranking`, and `owner` if the information is
|
|
116
|
+
available. Only include `propertyDomain` when it adds meaningful constraint or
|
|
117
|
+
documentation value; omit it for open-ended properties.
|
|
118
|
+
|
|
119
|
+
**Property domain types** — choose the right `variableType` when a domain is
|
|
120
|
+
declared:
|
|
121
|
+
|
|
122
|
+
| User intent | `variableType` | Extra fields |
|
|
123
|
+
| --- | --- | --- |
|
|
124
|
+
| A fixed set of named categories | `CATEGORICAL_VARIABLE_TYPE` | `values: [a, b, c]` |
|
|
125
|
+
| A fixed set of integer counts | `DISCRETE_VARIABLE_TYPE` | `values: [10, 50, 100]` |
|
|
126
|
+
| Any real number | `CONTINUOUS_VARIABLE_TYPE` | *(none, or `domainRange: [lo, hi]`)* |
|
|
127
|
+
| No constraint / open-ended | omit `propertyDomain` entirely | — |
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Step 3 — Write the README
|
|
132
|
+
|
|
133
|
+
Create `benchmarks/<benchmark_id>/README.md` alongside `benchmark.yaml`. The
|
|
134
|
+
README gives a human-readable problem statement that is not constrained by YAML
|
|
135
|
+
syntax. It should cover:
|
|
136
|
+
|
|
137
|
+
- **Problem formulation** — what is being optimised or evaluated, objective,
|
|
138
|
+
constraints.
|
|
139
|
+
- **Instance structure** — what the problem properties represent and how they
|
|
140
|
+
combine to define a concrete instance.
|
|
141
|
+
- **References** — papers, datasets, or external benchmark suites the problem
|
|
142
|
+
is drawn from (if any).
|
|
143
|
+
|
|
144
|
+
Write the README from the information gathered in Step 1. If the user has not
|
|
145
|
+
provided enough detail to write a meaningful formulation, ask before proceeding.
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Step 4 — Validate
|
|
150
|
+
|
|
151
|
+
Run the validation command from the repository root:
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
uv run nexus validate logical-benchmarks
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
To validate only the new benchmark:
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
uv run nexus validate logical-benchmarks --file benchmarks/<benchmark_id>
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
A successful run prints `success` for the benchmark folder. If validation fails:
|
|
164
|
+
|
|
165
|
+
1. Read the error messages carefully — they are usually Pydantic schema errors
|
|
166
|
+
or missing required fields.
|
|
167
|
+
2. Fix `benchmark.yaml`.
|
|
168
|
+
3. Re-run validation until all errors are resolved.
|
|
169
|
+
|
|
170
|
+
Do not proceed to the PR step until validation passes with no errors.
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## Step 5 — Commit, Push, and Open a PR
|
|
175
|
+
|
|
176
|
+
Stage and commit everything:
|
|
177
|
+
|
|
178
|
+
```bash
|
|
179
|
+
git checkout -b add-<benchmark_id>-benchmark
|
|
180
|
+
git add benchmarks/<benchmark_id>
|
|
181
|
+
git commit -s -m "feat(benchmark): Add <benchmark_id> problem class"
|
|
182
|
+
git push origin add-<benchmark_id>-benchmark
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
Then open a pull request from the new branch to the Algorithm Nexus main branch.
|
|
186
|
+
Use the `create_pr_workflow` if available to generate the PR description
|
|
187
|
+
automatically.
|
|
@@ -17,7 +17,7 @@ repos:
|
|
|
17
17
|
|
|
18
18
|
# Ruff - Python linter and formatter
|
|
19
19
|
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
20
|
-
rev: v0.
|
|
20
|
+
rev: v0.16.10
|
|
21
21
|
hooks:
|
|
22
22
|
- id: ruff
|
|
23
23
|
args: [--fix]
|
|
@@ -35,7 +35,7 @@ repos:
|
|
|
35
35
|
|
|
36
36
|
# Codespell - Check for common misspellings
|
|
37
37
|
- repo: https://github.com/codespell-project/codespell
|
|
38
|
-
rev: v2.4.
|
|
38
|
+
rev: v2.4.3
|
|
39
39
|
hooks:
|
|
40
40
|
- id: codespell
|
|
41
41
|
additional_dependencies: ["tomli"]
|
|
@@ -43,7 +43,7 @@ repos:
|
|
|
43
43
|
|
|
44
44
|
# UV lock check
|
|
45
45
|
- repo: https://github.com/astral-sh/uv-pre-commit
|
|
46
|
-
rev: 0.
|
|
46
|
+
rev: 0.12.23
|
|
47
47
|
hooks:
|
|
48
48
|
- id: uv-lock
|
|
49
49
|
- id: uv-export
|
|
@@ -64,7 +64,7 @@ repos:
|
|
|
64
64
|
|
|
65
65
|
# Markdownlint - Markdown linting
|
|
66
66
|
- repo: https://github.com/DavidAnson/markdownlint-cli2
|
|
67
|
-
rev: v0.23.
|
|
67
|
+
rev: v0.23.3
|
|
68
68
|
hooks:
|
|
69
69
|
- id: markdownlint-cli2
|
|
70
70
|
|
|
@@ -76,7 +76,7 @@ repos:
|
|
|
76
76
|
|
|
77
77
|
# Tombi - TOML formatting
|
|
78
78
|
- repo: https://github.com/tombi-toml/tombi-pre-commit
|
|
79
|
-
rev: v1.2
|
|
79
|
+
rev: v1.7.2
|
|
80
80
|
hooks:
|
|
81
81
|
- id: tombi-format
|
|
82
82
|
args: ["--offline"]
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"exclude": {
|
|
3
|
-
"files": "^.secrets$|^.secrets.baseline$",
|
|
3
|
+
"files": "^.secrets$|^.secrets.baseline$|^requirements-candidate.txt$|^requirements-ecosystem.txt$|^requirements-product.txt$",
|
|
4
4
|
"lines": null
|
|
5
5
|
},
|
|
6
|
-
"generated_at": "2026-
|
|
6
|
+
"generated_at": "2026-09-11T09:26:16Z",
|
|
7
7
|
"plugins_used": [
|
|
8
8
|
{
|
|
9
9
|
"name": "AWSKeyDetector"
|
|
@@ -76,18 +76,7 @@
|
|
|
76
76
|
"name": "TwilioKeyDetector"
|
|
77
77
|
}
|
|
78
78
|
],
|
|
79
|
-
"results": {
|
|
80
|
-
"requirements-ecosystem.txt": [
|
|
81
|
-
{
|
|
82
|
-
"hashed_secret": "52b44e0998fd3bcb1929ac25818191fae050cf1d",
|
|
83
|
-
"is_secret": false,
|
|
84
|
-
"is_verified": false,
|
|
85
|
-
"line_number": 825,
|
|
86
|
-
"type": "Secret Keyword",
|
|
87
|
-
"verified_result": null
|
|
88
|
-
}
|
|
89
|
-
]
|
|
90
|
-
},
|
|
79
|
+
"results": {},
|
|
91
80
|
"version": "0.13.1+ibm.64.dss",
|
|
92
81
|
"word_list": {
|
|
93
82
|
"file": null,
|
|
@@ -1,3 +1,35 @@
|
|
|
1
|
+
## [1.0.0](https://github.com/ibm/algorithm-nexus/compare/f3f8054bba7a0ac14581b1d45e9c655cba984266..1.0.0) - 2026-09-29
|
|
2
|
+
#### Documentation
|
|
3
|
+
- (**benchmarks**) Improved benchmarks terminology and adapted schema accordingly (#250) - ([57fcfa7](https://github.com/ibm/algorithm-nexus/commit/57fcfa75960716c1adadadb5ad8ff819f839f385)) - Christian Pinto
|
|
4
|
+
- (**changelog**) add release notes for 0.3.0 (#219) - ([f3f8054](https://github.com/ibm/algorithm-nexus/commit/f3f8054bba7a0ac14581b1d45e9c655cba984266)) - DRL-NextGen
|
|
5
|
+
- (**experiments**) Refactored experiment and benchmarks folder structure (#259) - ([5482773](https://github.com/ibm/algorithm-nexus/commit/5482773886999e63f91f2c4b81b796c3e809596c)) - Christian Pinto
|
|
6
|
+
- (**reqa**) Update benchmarking reqs (#261) - ([7c811f9](https://github.com/ibm/algorithm-nexus/commit/7c811f987aea21760cdcf8ec4fc99a6de54ae65c)) - Michael Johnston
|
|
7
|
+
- benchmark how-tos (#273) - ([2ac3469](https://github.com/ibm/algorithm-nexus/commit/2ac34696396edc7baaf455d5f6acc6f17fd03da8)) - Michael Johnston
|
|
8
|
+
- update benchmark design doc structure and harmonise terminology (#266) - ([9f251bd](https://github.com/ibm/algorithm-nexus/commit/9f251bd8c5460ffe23e75908c98bd2a77e8a48e3)) - Michael Johnston
|
|
9
|
+
#### Build system
|
|
10
|
+
- (**dependencies**) Updated vllm to v0.30.0 (#251) - ([cc4198f](https://github.com/ibm/algorithm-nexus/commit/cc4198f8eec52b0207cae128d1a1e80bbe87c949)) - Christian Pinto
|
|
11
|
+
- (**deps**) update dependencies (#270) - ([12538be](https://github.com/ibm/algorithm-nexus/commit/12538be9eb933ba1ee66381e5fb9d8ef2ce26a0d)) - DRL-NextGen
|
|
12
|
+
- (**deps**) update dependencies (#248) - ([cfef198](https://github.com/ibm/algorithm-nexus/commit/cfef1989e42b9ddb68ca6acb57991eb1368d866e)) - DRL-NextGen
|
|
13
|
+
- (**deps**) update dependencies (#245) - ([8b72898](https://github.com/ibm/algorithm-nexus/commit/8b72898d5e14732971eb068537818b1af3112b32)) - DRL-NextGen
|
|
14
|
+
- (**deps**) update dependencies (#239) - ([ff886d6](https://github.com/ibm/algorithm-nexus/commit/ff886d684711b62cc2a1beb6f90043cfb11dd966)) - DRL-NextGen
|
|
15
|
+
- (**deps**) Updated vllm (0.29.) and terratorch (1.2.13) versions in candidate target (#238) - ([0a8c880](https://github.com/ibm/algorithm-nexus/commit/0a8c880cba0b1c96d8287e4d1eaf9a830561b44a)) - Christian Pinto
|
|
16
|
+
- (**deps**) update dependencies (#234) - ([c7c30d1](https://github.com/ibm/algorithm-nexus/commit/c7c30d19d7e67b5e30bb73549cc33a7cccbca33b)) - DRL-NextGen
|
|
17
|
+
- (**hooks**) update pre-commit hooks (#249) - ([55834b0](https://github.com/ibm/algorithm-nexus/commit/55834b0e516d2302d7dce30f6b7787de09fb1c64)) - DRL-NextGen
|
|
18
|
+
- (**hooks**) update pre-commit hooks (#244) - ([fb1c11d](https://github.com/ibm/algorithm-nexus/commit/fb1c11d42940c4f99cce92fc999dd2a1df6054e2)) - DRL-NextGen
|
|
19
|
+
- (**hooks**) update pre-commit hooks (#227) - ([c2cd126](https://github.com/ibm/algorithm-nexus/commit/c2cd12647aafa4579390ffb53efc661231893f88)) - DRL-NextGen
|
|
20
|
+
#### Miscellaneous Chores
|
|
21
|
+
- exclude autogenerated requirements file from detect-secrets (#233) - ([a337773](https://github.com/ibm/algorithm-nexus/commit/a33777333e749c047511922600aa0d008411f0a0)) - Alessandro Pomponio
|
|
22
|
+
|
|
23
|
+
- - -
|
|
24
|
+
|
|
25
|
+
## [0.3.0](https://github.com/ibm/algorithm-nexus/compare/f6eeb79b27485ed30089ec3b8e216d8756b1971a..0.3.0) - 2026-08-31
|
|
26
|
+
#### Features
|
|
27
|
+
- (**dependencies**) Updated vllm to 0.28.x and terratorch to 1.2.13 (#218) - ([7bdaea9](https://github.com/ibm/algorithm-nexus/commit/7bdaea9a97534de8376c1826c424b35e65b0bf1b)) - Christian Pinto
|
|
28
|
+
#### Documentation
|
|
29
|
+
- (**changelog**) add release notes for 0.2.3 (#215) - ([f6eeb79](https://github.com/ibm/algorithm-nexus/commit/f6eeb79b27485ed30089ec3b8e216d8756b1971a)) - DRL-NextGen
|
|
30
|
+
|
|
31
|
+
- - -
|
|
32
|
+
|
|
1
33
|
## [0.2.3](https://github.com/ibm/algorithm-nexus/compare/36f3d99129ddf64de7f5098e4f1c290b991efa18..0.2.3) - 2026-08-25
|
|
2
34
|
#### Features
|
|
3
35
|
- (**dependencies**) Relaxed vllm dependency in product target (#214) - ([5926fdf](https://github.com/ibm/algorithm-nexus/commit/5926fdf6e38d8def91ab5ceed2a2b01b258d24c9)) - Christian Pinto
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: algorithm-nexus
|
|
3
|
-
Version:
|
|
3
|
+
Version: 1.1.0
|
|
4
4
|
Summary: A comprehensive library for AI model delivery, testing, and benchmarking
|
|
5
5
|
Project-URL: Homepage, https://github.com/IBM/algorithm-nexus
|
|
6
6
|
Project-URL: Issues, https://github.com/IBM/algorithm-nexus/issues
|
|
@@ -10,12 +10,10 @@ Maintainer-email: Christian Pinto <christian.pinto@ibm.com>
|
|
|
10
10
|
License-File: LICENSE
|
|
11
11
|
Requires-Python: <3.14,>=3.12
|
|
12
12
|
Provides-Extra: candidate
|
|
13
|
-
Requires-Dist: biomed-multi-alignment[vllm]; extra == 'candidate'
|
|
14
13
|
Requires-Dist: bmfm-targets[vllm]>=0.20.0; extra == 'candidate'
|
|
15
14
|
Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'candidate'
|
|
16
|
-
Requires-Dist:
|
|
17
|
-
Requires-Dist:
|
|
18
|
-
Requires-Dist: vllm==0.26.0; extra == 'candidate'
|
|
15
|
+
Requires-Dist: terratorch[vllm]==1.2.13; extra == 'candidate'
|
|
16
|
+
Requires-Dist: vllm<0.32.0,>=0.31.0; extra == 'candidate'
|
|
19
17
|
Provides-Extra: cli
|
|
20
18
|
Requires-Dist: ado-core>=2.0; extra == 'cli'
|
|
21
19
|
Requires-Dist: pydantic>=2.13.3; extra == 'cli'
|
|
@@ -23,7 +21,6 @@ Requires-Dist: pyyaml>=6.0.3; extra == 'cli'
|
|
|
23
21
|
Requires-Dist: rich>=15.0.0; extra == 'cli'
|
|
24
22
|
Requires-Dist: typer>=0.25.0; extra == 'cli'
|
|
25
23
|
Provides-Extra: ecosystem
|
|
26
|
-
Requires-Dist: biomed-multi-alignment; extra == 'ecosystem'
|
|
27
24
|
Requires-Dist: bmfm-targets; extra == 'ecosystem'
|
|
28
25
|
Requires-Dist: gridfm-graphkit==0.8.1; extra == 'ecosystem'
|
|
29
26
|
Requires-Dist: terrakit>=0.2.0; extra == 'ecosystem'
|
|
@@ -31,9 +28,8 @@ Requires-Dist: terratorch>=1.2.7; extra == 'ecosystem'
|
|
|
31
28
|
Requires-Dist: tokamind; extra == 'ecosystem'
|
|
32
29
|
Provides-Extra: product
|
|
33
30
|
Requires-Dist: opencv-python-headless!=5.0.0.93; extra == 'product'
|
|
34
|
-
Requires-Dist: terrakit==0.2.0; extra == 'product'
|
|
35
31
|
Requires-Dist: terratorch[vllm]==1.2.13; extra == 'product'
|
|
36
|
-
Requires-Dist: vllm<0.
|
|
32
|
+
Requires-Dist: vllm<0.32.0,>=0.31.0; extra == 'product'
|
|
37
33
|
Description-Content-Type: text/markdown
|
|
38
34
|
|
|
39
35
|
# Algorithm Nexus
|
|
@@ -250,7 +246,7 @@ nexus list benchmark-packages ./packages --nexus-package terratorch
|
|
|
250
246
|
#### List benchmark experiments
|
|
251
247
|
|
|
252
248
|
```bash
|
|
253
|
-
nexus list
|
|
249
|
+
nexus list experiments [PACKAGES_ROOT]
|
|
254
250
|
```
|
|
255
251
|
|
|
256
252
|
Lists all benchmark experiments with their associated benchmark packages and
|
|
@@ -259,22 +255,7 @@ Nexus packages. Filter by a specific Nexus package with `--nexus-package`.
|
|
|
259
255
|
**Example:**
|
|
260
256
|
|
|
261
257
|
```bash
|
|
262
|
-
nexus list
|
|
263
|
-
```
|
|
264
|
-
|
|
265
|
-
#### Get benchmark requirements
|
|
266
|
-
|
|
267
|
-
```bash
|
|
268
|
-
nexus get benchmark-requirements NEXUS_PACKAGE [PACKAGES_ROOT]
|
|
269
|
-
```
|
|
270
|
-
|
|
271
|
-
Retrieves the benchmark requirement specifiers for a specific Nexus package.
|
|
272
|
-
Output in requirements.txt format with `-o txt` for use with pip.
|
|
273
|
-
|
|
274
|
-
**Example:**
|
|
275
|
-
|
|
276
|
-
```bash
|
|
277
|
-
nexus get benchmark-requirements terratorch ./packages -o txt --output-file requirements.txt
|
|
258
|
+
nexus list experiments ./packages -o json
|
|
278
259
|
```
|
|
279
260
|
|
|
280
261
|
### Output formats
|
|
@@ -284,7 +265,6 @@ Most list commands support multiple output formats:
|
|
|
284
265
|
- **Table** (default): Human-readable table output
|
|
285
266
|
- **JSON** (`-o json`): Machine-readable JSON format
|
|
286
267
|
- **CSV** (`-o csv`): Comma-separated values for spreadsheets
|
|
287
|
-
- **TXT** (`-o txt`): Requirements file format (get commands only)
|
|
288
268
|
|
|
289
269
|
Use `--output-file` to write output to a file instead of stdout.
|
|
290
270
|
|
|
@@ -212,7 +212,7 @@ nexus list benchmark-packages ./packages --nexus-package terratorch
|
|
|
212
212
|
#### List benchmark experiments
|
|
213
213
|
|
|
214
214
|
```bash
|
|
215
|
-
nexus list
|
|
215
|
+
nexus list experiments [PACKAGES_ROOT]
|
|
216
216
|
```
|
|
217
217
|
|
|
218
218
|
Lists all benchmark experiments with their associated benchmark packages and
|
|
@@ -221,22 +221,7 @@ Nexus packages. Filter by a specific Nexus package with `--nexus-package`.
|
|
|
221
221
|
**Example:**
|
|
222
222
|
|
|
223
223
|
```bash
|
|
224
|
-
nexus list
|
|
225
|
-
```
|
|
226
|
-
|
|
227
|
-
#### Get benchmark requirements
|
|
228
|
-
|
|
229
|
-
```bash
|
|
230
|
-
nexus get benchmark-requirements NEXUS_PACKAGE [PACKAGES_ROOT]
|
|
231
|
-
```
|
|
232
|
-
|
|
233
|
-
Retrieves the benchmark requirement specifiers for a specific Nexus package.
|
|
234
|
-
Output in requirements.txt format with `-o txt` for use with pip.
|
|
235
|
-
|
|
236
|
-
**Example:**
|
|
237
|
-
|
|
238
|
-
```bash
|
|
239
|
-
nexus get benchmark-requirements terratorch ./packages -o txt --output-file requirements.txt
|
|
224
|
+
nexus list experiments ./packages -o json
|
|
240
225
|
```
|
|
241
226
|
|
|
242
227
|
### Output formats
|
|
@@ -246,7 +231,6 @@ Most list commands support multiple output formats:
|
|
|
246
231
|
- **Table** (default): Human-readable table output
|
|
247
232
|
- **JSON** (`-o json`): Machine-readable JSON format
|
|
248
233
|
- **CSV** (`-o csv`): Comma-separated values for spreadsheets
|
|
249
|
-
- **TXT** (`-o txt`): Requirements file format (get commands only)
|
|
250
234
|
|
|
251
235
|
Use `--output-file` to write output to a file instead of stdout.
|
|
252
236
|
|