sci-modeling-bench 0.1.0__tar.gz

sci_modeling_bench-0.1.0/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+All notable changes to SciModelingBench are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-07-16
+
+### Added
+
+- Initial Python 3.10+ package with typed public exports.
+- Core Dataset framework for resolving Hugging Face revisions to immutable
+  commits, selecting collection configs and splits, loading regular or
+  streaming datasets, and checking declared schemas and row counts.
+- Strict collection and dataset manifests with provenance metadata, semantic
+  input, target, and context fields, common constraints, and split metadata.
+- Structured validation reports and dataset-specific validator hooks that
+  report violations without mutating observations or candidates.
+- Read-only, lazily loaded Knowledge resources pinned to the Dataset revision.
+- Minimal Objective and Protocol contracts for validated target evaluation and
+  construction of agent-visible inputs.
+- One black-box optimization integration for TFBind8 `SIX6_REF_R1`, including
+  the complete canonical 65,536-sequence landscape, exact E-score lookup, and
+  a Design-Bench-compatible bottom-50% Protocol exposing 32,768 unique offline
+  candidates.
+- Reproducible TFBind8 release builder and `smb-build-tfbind8` command with
+  source checks, canonicalization, integrity validation, provenance output,
+  and optional parity checks against legacy Design-Bench arrays.
+- Public documentation for the framework concepts and TFBind8 integration.
+
+### Known Limitations
+
+- Dataset, Objective, and Protocol interfaces remain experimental, and a Task
+  or evaluation-harness interface is not included in this release.
+- Dataset artifacts are distributed separately from the Python package and
+  retain their own source, citation, provenance, and license metadata.
+
+[0.1.0]: https://github.com/xukp20/sci-modeling-bench/releases/tag/v0.1.0

sci_modeling_bench-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SciModelingBench contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sci_modeling_bench-0.1.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include CHANGELOG.md
+include LICENSE
+include README.md
+include THIRD_PARTY_NOTICES.md
+recursive-include docs *.md
+recursive-include tests *.py

sci_modeling_bench-0.1.0/PKG-INFO

@@ -0,0 +1,128 @@
+Metadata-Version: 2.4
+Name: sci-modeling-bench
+Version: 0.1.0
+Summary: An observation-grounded benchmark framework for scientific modeling and design.
+Author: SciModelingBench contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/xukp20/sci-modeling-bench
+Project-URL: Documentation, https://github.com/xukp20/sci-modeling-bench/tree/main/docs
+Project-URL: Repository, https://github.com/xukp20/sci-modeling-bench.git
+Project-URL: Issues, https://github.com/xukp20/sci-modeling-bench/issues
+Project-URL: Changelog, https://github.com/xukp20/sci-modeling-bench/blob/main/CHANGELOG.md
+Keywords: agent,benchmark,black-box-optimization,scientific-machine-learning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: datasets>=3.1
+Requires-Dist: huggingface-hub>=0.26
+Requires-Dist: numpy>=1.26
+Requires-Dist: pydantic>=2.10
+Requires-Dist: pyarrow>=15
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: twine>=5.1; extra == "dev"
+Dynamic: license-file
+
+# SciModelingBench
+
+SciModelingBench is a Python framework for reproducible scientific modeling
+and design benchmarks. Version 0.1.0 provides the first experimental package
+surface for loading versioned observations, validating scientific schemas,
+constructing agent-visible inputs, and evaluating candidates against trusted
+objectives.
+
+## Installation
+
+SciModelingBench requires Python 3.10 or later.
+
+From PyPI:
+
+```bash
+python -m pip install "sci-modeling-bench==0.1.0"
+```
+
+From the GitHub v0.1.0 source tag:
+
+```bash
+python -m pip install \
+  "git+https://github.com/xukp20/sci-modeling-bench.git@v0.1.0"
+```
+
+## Scope
+
+SciModelingBench provides:
+
+- revision-pinned loading of scientific datasets hosted on Hugging Face;
+- semantic manifests, schemas, provenance metadata, and structured validation;
+- trusted Objectives for candidate-to-target evaluation;
+- Protocols that construct the information exposed to an optimization agent;
+- optional, lazily loaded domain-knowledge resources.
+
+The package does not yet define Tasks, submission formats, metrics, query
+budgets, agent workflows, or evaluation harnesses.
+
+## Minimal TFBind8 Example
+
+The first integration exposes the canonical TFBind8 `SIX6_REF_R1` landscape,
+an exact black-box Objective, and the Design-Bench bottom-50% offline-data
+Protocol:
+
+```python
+from sci_modeling_bench.suites.design_bench import (
+    TFBind8Dataset,
+    TFBind8DesignBenchProtocol,
+    TFBind8ExactObjective,
+)
+
+dataset = TFBind8Dataset.from_hub(
+    revision="2ee2856f4255bb6a64c11b6c2660a6f41418e654"
+)
+offline_data = TFBind8DesignBenchProtocol().build_input(dataset)
+score = TFBind8ExactObjective(dataset).evaluate({"sequence": "AACCGGTT"})
+```
+
+The TFBind8 observations are downloaded from the public SciModelingBench
+Hugging Face organization and are not bundled in the Python wheel.
+
+## Core Concepts
+
+- **Dataset** binds immutable observations to metadata, semantic fields,
+  validation rules, splits, and optional knowledge.
+- **Objective** validates candidates and returns declared target values while
+  preserving batch order and repeated candidates.
+- **Protocol** derives the data or context visible to an agent without
+  modifying the underlying Dataset.
+- **Knowledge** provides read-only explanatory resources pinned to the same
+  dataset revision.
+
+## Dataset Artifacts
+
+Dataset artifacts are hosted separately from this package. Neither the PyPI
+distribution nor the GitHub source repository bundles observation tables.
+Loaders resolve dedicated Hugging Face Dataset repositories at pinned commits;
+the package contains the framework, integrations, validators, and reproducible
+builders.
+
+## Documentation
+
+- [Documentation index](https://github.com/xukp20/sci-modeling-bench/tree/main/docs)
+- [Architecture and core concepts](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/architecture/core-concepts.md)
+- [Dataset API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/dataset.md)
+- [Objective API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/objective.md)
+- [Protocol API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/protocol.md)
+- [TFBind8 integration](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/suites/design-bench/tfbind8.md)
+- [Changelog](https://github.com/xukp20/sci-modeling-bench/blob/main/CHANGELOG.md)
+
+## Release Status
+
+Version 0.1.0 is the initial experimental release. Dataset, Objective, and
+Protocol interfaces are implemented but may change as additional scientific
+benchmarks are integrated. The TFBind8 code path has been validated against
+the pinned public Hugging Face artifact and the legacy Design-Bench arrays.

sci_modeling_bench-0.1.0/README.md

@@ -0,0 +1,96 @@
+# SciModelingBench
+
+SciModelingBench is a Python framework for reproducible scientific modeling
+and design benchmarks. Version 0.1.0 provides the first experimental package
+surface for loading versioned observations, validating scientific schemas,
+constructing agent-visible inputs, and evaluating candidates against trusted
+objectives.
+
+## Installation
+
+SciModelingBench requires Python 3.10 or later.
+
+From PyPI:
+
+```bash
+python -m pip install "sci-modeling-bench==0.1.0"
+```
+
+From the GitHub v0.1.0 source tag:
+
+```bash
+python -m pip install \
+  "git+https://github.com/xukp20/sci-modeling-bench.git@v0.1.0"
+```
+
+## Scope
+
+SciModelingBench provides:
+
+- revision-pinned loading of scientific datasets hosted on Hugging Face;
+- semantic manifests, schemas, provenance metadata, and structured validation;
+- trusted Objectives for candidate-to-target evaluation;
+- Protocols that construct the information exposed to an optimization agent;
+- optional, lazily loaded domain-knowledge resources.
+
+The package does not yet define Tasks, submission formats, metrics, query
+budgets, agent workflows, or evaluation harnesses.
+
+## Minimal TFBind8 Example
+
+The first integration exposes the canonical TFBind8 `SIX6_REF_R1` landscape,
+an exact black-box Objective, and the Design-Bench bottom-50% offline-data
+Protocol:
+
+```python
+from sci_modeling_bench.suites.design_bench import (
+    TFBind8Dataset,
+    TFBind8DesignBenchProtocol,
+    TFBind8ExactObjective,
+)
+
+dataset = TFBind8Dataset.from_hub(
+    revision="2ee2856f4255bb6a64c11b6c2660a6f41418e654"
+)
+offline_data = TFBind8DesignBenchProtocol().build_input(dataset)
+score = TFBind8ExactObjective(dataset).evaluate({"sequence": "AACCGGTT"})
+```
+
+The TFBind8 observations are downloaded from the public SciModelingBench
+Hugging Face organization and are not bundled in the Python wheel.
+
+## Core Concepts
+
+- **Dataset** binds immutable observations to metadata, semantic fields,
+  validation rules, splits, and optional knowledge.
+- **Objective** validates candidates and returns declared target values while
+  preserving batch order and repeated candidates.
+- **Protocol** derives the data or context visible to an agent without
+  modifying the underlying Dataset.
+- **Knowledge** provides read-only explanatory resources pinned to the same
+  dataset revision.
+
+## Dataset Artifacts
+
+Dataset artifacts are hosted separately from this package. Neither the PyPI
+distribution nor the GitHub source repository bundles observation tables.
+Loaders resolve dedicated Hugging Face Dataset repositories at pinned commits;
+the package contains the framework, integrations, validators, and reproducible
+builders.
+
+## Documentation
+
+- [Documentation index](https://github.com/xukp20/sci-modeling-bench/tree/main/docs)
+- [Architecture and core concepts](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/architecture/core-concepts.md)
+- [Dataset API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/dataset.md)
+- [Objective API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/objective.md)
+- [Protocol API](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/api/protocol.md)
+- [TFBind8 integration](https://github.com/xukp20/sci-modeling-bench/blob/main/docs/suites/design-bench/tfbind8.md)
+- [Changelog](https://github.com/xukp20/sci-modeling-bench/blob/main/CHANGELOG.md)
+
+## Release Status
+
+Version 0.1.0 is the initial experimental release. Dataset, Objective, and
+Protocol interfaces are implemented but may change as additional scientific
+benchmarks are integrated. The TFBind8 code path has been validated against
+the pinned public Hugging Face artifact and the legacy Design-Bench arrays.

sci_modeling_bench-0.1.0/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,24 @@
+# Third-Party Notices
+
+SciModelingBench integrates benchmark definitions and derived data from
+third-party projects. The project code license does not replace the terms that
+apply to external datasets and publications.
+
+## Design-Bench
+
+The TFBind8 integration reproduces the task semantics and preprocessing of
+Design-Bench:
+
+- Project: https://github.com/brandontrabucco/design-bench
+- Referenced revision: `e52939588421b5433f6f2e9b359cf013c542bd89`
+- Copyright: Copyright (c) 2020 Brandon Trabucco
+- License: MIT License
+
+## TFBind8 Data
+
+The TFBind8 observations originate from the protein-binding microarray data
+published by Barrera et al. (Science, 2016), UniPROBE accession `BAR15A`.
+SciModelingBench publishes only the derived contiguous 8-mer landscape and
+does not assert that the project MIT license applies to those observations.
+The Hugging Face Dataset card records the source, citations, checksums,
+transformations, and applicable upstream terms.

sci_modeling_bench-0.1.0/docs/README.md

@@ -0,0 +1,33 @@
+# SciModelingBench Documentation
+
+SciModelingBench is an observation-grounded framework for scientific modeling
+and design benchmarks. The first release provides experimental `Dataset`,
+`Objective`, and `Protocol` APIs plus an end-to-end black-box optimization
+integration for TFBind8.
+
+## Start Here
+
+- [Architecture and core concepts](architecture/core-concepts.md) explains the
+  framework boundaries, trust model, and data flow.
+- [API documentation](api/README.md) covers the three implemented public
+  interfaces:
+  - [Dataset API](api/dataset.md)
+  - [Objective API](api/objective.md)
+  - [Protocol API](api/protocol.md)
+- [TFBind8](suites/design-bench/tfbind8.md) documents the first supported
+  black-box optimization setting, its canonical data, and Design-Bench parity.
+
+## First-Release Scope
+
+The implemented layers load revision-pinned scientific observations, construct
+the information visible to an optimization agent, and evaluate candidates
+against trusted targets. A benchmark `Task`, runner, query-budget policy,
+submission format, and metrics layer are not implemented. Those responsibilities
+remain with the external harness in this release.
+
+The public interfaces are experimental. Pin dataset revisions and package
+versions in reproducible work.
+
+## Maintainers
+
+- [Release process](development/releasing.md)

sci_modeling_bench-0.1.0/docs/api/README.md

@@ -0,0 +1,13 @@
+# API Documentation
+
+The first release exposes three experimental interfaces:
+
+- [Dataset API](dataset.md): revision-pinned observations, manifests, semantic
+  schemas, validation, splits, and knowledge resources.
+- [Objective API](objective.md): trusted, Dataset-bound candidate evaluation.
+- [Protocol API](protocol.md): construction of agent-visible benchmark input.
+
+For how these interfaces fit together, read
+[Architecture and core concepts](../architecture/core-concepts.md). The
+[TFBind8 suite guide](../suites/design-bench/tfbind8.md) shows all three in one
+black-box optimization workflow.

sci_modeling_bench-0.1.0/docs/api/dataset.md

@@ -0,0 +1,293 @@
+# Dataset API
+
+> **Status:** Experimental. Pin the package version and dataset revision in
+> reproducible work.
+
+The `Dataset` layer represents published scientific observations together with
+their identity, provenance, semantic field roles, validity rules, split
+metadata, and optional domain knowledge. It intentionally does not select
+task-specific agent input or evaluate candidate designs.
+
+## Loading a Dataset
+
+Load a repository config directly:
+
+```python
+from sci_modeling_bench import Dataset
+
+dataset = Dataset.from_hub(
+    "sci-modeling-bench/design-bench",
+    config_name="tfbind8",
+    revision="full-commit-sha",
+)
+```
+
+`Dataset.from_hub()` has this signature:
+
+```python
+Dataset.from_hub(
+    repo_id: str,
+    config_name: str | None = None,
+    revision: str | None = None,
+    *,
+    token: str | None = None,
+    validator: DatasetValidator | None = None,
+) -> Dataset
+```
+
+When `config_name` is omitted, the collection manifest's `default_config` is
+used. The requested revision, including the repository default when `None`, is
+resolved once to a full commit SHA. All manifests, data, and knowledge resources
+are read from that SHA with remote code disabled.
+
+For a reusable or replaceable source reference, use `HubDatasetSource`:
+
+```python
+from sci_modeling_bench import Dataset, HubDatasetSource
+
+source = HubDatasetSource(
+    repo_id="internal-mirror/design-bench",
+    config_name="tfbind8",
+    revision="full-commit-sha",
+)
+dataset = Dataset.from_source(source)
+```
+
+`Dataset.from_source(source, *, token=None, validator=None)` applies the same
+resolution and loading behavior. `HubDatasetSource` stores only repository,
+config, and revision identity. Authentication comes from the local Hugging Face
+configuration or the explicit `token` argument.
+
+Dataset-specific classes may provide a default source. For example,
+`TFBind8Dataset.from_hub()` fills in the Design-Bench repository and config
+while still accepting a mirror or pinned revision.
+
+## Dataset Properties
+
+| Property | Type | Meaning |
+|---|---|---|
+| `repo_id` | `str` | Hugging Face Dataset repository used by this instance |
+| `resolved_revision` | `str` | Full commit SHA used for every repository access |
+| `config_name` | `str` | Selected repository config |
+| `metadata` | `DatasetMetadata` | Dataset ID, version, description, license, sources, and citations |
+| `schema` | `DatasetSchema` | Input, target, and context field declarations |
+| `default_split` | `str` | Manifest-selected split name |
+| `splits` | `tuple[DatasetSplit, ...]` | Immutable split metadata |
+| `available_splits` | `tuple[str, ...]` | Declared split names in manifest order |
+| `knowledge` | `Knowledge` | Read-only mapping of lazy text resources |
+| `features` | Hugging Face `Features` | Physical features for the loaded default split |
+
+Metadata, schema, field, split, constraint, violation, and validation-report
+objects are immutable Pydantic models. `HubDatasetSource` and knowledge resource
+metadata are immutable dataclasses.
+
+## Splits and Data Loading
+
+```python
+split_metadata = dataset.split()  # Default split
+named_metadata = dataset.split("six6_ref_r1")
+
+observations = dataset.load()
+stream = dataset.load("six6_ref_r1", streaming=True)
+```
+
+`split(name=None)` returns metadata for a declared split and raises `ValueError`
+for an undeclared name. `load(split=None, *, streaming=False)` returns a standard
+Hugging Face `Dataset` or, for streaming loads, an `IterableDataset`.
+
+Non-streaming data are cached by split on the `Dataset` instance. When Hugging
+Face feature metadata is available, loading checks that every manifest-declared
+field exists. When a split declares `num_rows`, non-streaming loading also
+checks the row count. Streaming loads are not cached and cannot perform the
+row-count check.
+
+A published split is an observation group with one schema and shared metadata.
+It is not an agent-visible train/test view; constructing that view is a
+[Protocol](protocol.md) responsibility.
+
+## Semantic Schema
+
+`dataset.schema` contains ordered `inputs`, `targets`, and `context` tuples of
+`FieldSpec` objects. Field names must be unique across all roles. Each field has
+a name, description, optional unit, `required` flag, and zero or more common
+constraints.
+
+| Constraint kind | Model | Behavior |
+|---|---|---|
+| `range` | `NumericRangeConstraint` | Minimum and maximum numeric bounds; each bound can be inclusive or exclusive |
+| `allowed_values` | `AllowedValuesConstraint` | Finite allowed set for scalar values or sequence leaves |
+| `alphabet` | `AlphabetConstraint` | Allowed single-character symbols for a string |
+| `length` | `LengthConstraint` | Minimum and maximum length for strings or sized sequences |
+| `finite` | `FiniteConstraint` | Reject NaN and infinite numeric leaves |
+
+Example field declarations in a config manifest:
+
+```json
+{
+  "inputs": [
+    {
+      "name": "sequence",
+      "description": "DNA sequence of exactly eight nucleotides.",
+      "constraints": [
+        {"kind": "alphabet", "symbols": ["A", "C", "G", "T"]},
+        {"kind": "length", "minimum": 8, "maximum": 8}
+      ]
+    }
+  ],
+  "targets": [
+    {
+      "name": "normalized_e_score",
+      "description": "Condition-level normalized E-score.",
+      "constraints": [
+        {"kind": "finite"},
+        {"kind": "range", "minimum": 0.0, "maximum": 1.0}
+      ]
+    }
+  ],
+  "context": []
+}
+```
+
+Hugging Face `Features` remain the authority for physical dtype and shape.
+Field specifications add semantic roles and scientific constraints.
+
+## Validation
+
+```python
+input_report = dataset.validate_inputs(
+    {"sequence": "AACCGGTT"},
+    context=None,
+)
+target_report = dataset.validate_targets(
+    {"normalized_e_score": 0.5},
+    inputs={"sequence": "AACCGGTT"},
+    context=None,
+)
+row_report = dataset.validate_observation(observation)
+full_report = dataset.validate_dataset()
+```
+
+The validation methods return `ValidationReport`; expected data errors are not
+raised as exceptions. A report contains immutable `Violation` objects with a
+machine-readable `code`, message, optional field and row index, and `error` or
+`warning` severity. `report.valid` is false when at least one error is present.
+
+- `validate_inputs()` checks the input role, context role, physical features,
+  common constraints, and custom input hooks. Unknown role fields are rejected.
+- `validate_targets()` checks the target and context roles, then passes optional
+  inputs and context to custom target checks. It does not separately validate
+  the supplied `inputs` mapping.
+- `validate_observation()` checks all declared fields in one row and permits
+  additional published columns.
+- `validate_dataset(data=None)` validates every row and then runs the custom
+  dataset-level hook. It can be expensive for large or streaming data.
+
+Validation reports findings without clipping, repairing, or mutating values.
+
+For scientific checks that cannot be expressed in a manifest, subclass
+`DatasetValidator` and pass an instance while loading:
+
+```python
+from sci_modeling_bench import DatasetValidator, ValidationReport, Violation
+
+
+class RejectHomopolymers(DatasetValidator):
+    def validate_inputs(self, inputs, context=None):
+        sequence = inputs.get("sequence")
+        if isinstance(sequence, str) and sequence and len(set(sequence)) == 1:
+            return ValidationReport(
+                violations=(
+                    Violation(
+                        code="homopolymer",
+                        field="sequence",
+                        message="homopolymers are not accepted",
+                    ),
+                )
+            )
+        return ValidationReport()
+```
+
+Custom validators are local code. Dataset repositories cannot provide remotely
+executed validators.
+
+## Knowledge Resources
+
+`dataset.knowledge` is always a read-only mapping and is falsey when the
+manifest declares no resources. Resource contents are loaded only when read:
+
+```python
+for name, resource in dataset.knowledge.items():
+    print(name, resource.title, resource.media_type)
+
+overview = dataset.knowledge.read_text("domain_overview")
+# Equivalent: dataset.knowledge["domain_overview"].read_text()
+```
+
+Supported media types are `text/markdown` and `text/plain`. Resources are read
+from the Dataset's resolved repository revision. They should explain scientific
+meaning, priors, constraints, and limitations, not contain executable code or
+agent workflow instructions.
+
+## Repository Contract
+
+One Hugging Face Dataset repository can publish multiple related configs:
+
+```text
+README.md
+scimodelingbench.json
+manifests/
+└── tfbind8.json
+data/
+└── tfbind8/
+    └── six6_ref_r1.parquet
+knowledge/
+└── tfbind8/
+    └── domain-overview.md
+```
+
+The Dataset Card's YAML metadata maps configs and splits to data files.
+`scimodelingbench.json` maps the same config names to semantic manifests:
+
+```json
+{
+  "schema_version": 1,
+  "default_config": "tfbind8",
+  "configs": {
+    "tfbind8": {"manifest": "manifests/tfbind8.json"}
+  }
+}
+```
+
+The config manifest schema version is `1` and defines these field groups:
+
+| Field | Meaning |
+|---|---|
+| `dataset_id`, `version`, `description`, `license` | Stable identity and release metadata |
+| `default_split`, `splits` | Published split declarations; each split has a name, description, optional row count, optional sources, and scalar attributes |
+| `inputs`, `targets`, `context` | Semantic field roles and constraints |
+| `source`, `citation` | Upstream provenance and citation records |
+| `knowledge` | Stable snake-case names mapped to repository-relative text resources |
+
+`source`, `citation`, `context`, and `knowledge` may be omitted or empty. Input,
+target, and split collections may not be empty. Unknown manifest fields,
+duplicate names, unsafe repository paths, and references to undeclared defaults
+are rejected. Raw data and preprocessing code may stay upstream, but provenance
+should record the source version, revision, and checksum when available.
+
+Dataset loading never executes Python code from the remote repository.
+
+## Exceptions
+
+| Exception | Raised when |
+|---|---|
+| `ManifestError` | A collection or config manifest is missing, malformed, or internally inconsistent |
+| `DatasetLoadError` | A Hub revision or repository resource cannot be resolved or loaded |
+| `SchemaMismatchError` | Loaded data omit declared columns or violate a declared non-streaming row count |
+| `ValueError` | A caller requests an undeclared split or constructs an invalid `HubDatasetSource` |
+
+All package-specific exceptions inherit from `SciModelingBenchError`.
+
+## Concrete Dataset
+
+See [TFBind8](../suites/design-bench/tfbind8.md) for the first Dataset-specific
+loader, validator, exact Objective, and offline-data Protocol.