deep-learning-core 0.0.2__tar.gz → 0.0.2a4__tar.gz

{deep_learning_core-0.0.2 → deep_learning_core-0.0.2a4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deep-learning-core
-Version: 0.0.2
+Version: 0.0.2a4
 Summary: Reusable deep learning framework core.
 Project-URL: Homepage, https://github.com/Blazkowiz47/dl-core
 Project-URL: Documentation, https://github.com/Blazkowiz47/dl-core/tree/main/readme
@@ -53,13 +53,13 @@ Requires-Dist: torch
 Requires-Dist: torchvision
 Requires-Dist: tqdm
 Provides-Extra: azure
-Requires-Dist: deep-learning-azure; extra == 'azure'
+Requires-Dist: deep-learning-azure==0.0.2a4; extra == 'azure'
 Provides-Extra: dev
 Requires-Dist: pytest; extra == 'dev'
 Provides-Extra: mlflow
-Requires-Dist: deep-learning-mlflow; extra == 'mlflow'
+Requires-Dist: deep-learning-mlflow==0.0.2a4; extra == 'mlflow'
 Provides-Extra: wandb
-Requires-Dist: deep-learning-wandb; extra == 'wandb'
+Requires-Dist: deep-learning-wandb==0.0.2a4; extra == 'wandb'
 Description-Content-Type: text/markdown
 
 # deep-learning-core
@@ -226,6 +226,7 @@ Then:
    - `src/datasets/my_exp.py`
    - `configs/base.yaml`
    - `experiments/lr_sweep.yaml`
+   - `AGENTS.md`
 2. implement the generated dataset wrapper under `src/datasets/my_exp.py`
 3. adjust `configs/base.yaml` so it points at the dataset/model/trainer you want
 4. start with:
@@ -256,13 +257,68 @@ scaffold a `wandb` callback block, W&B tracking defaults, and `.env.example`.
 - [`dl-mlflow`](https://github.com/Blazkowiz47/dl-mlflow)
 - [`dl-wandb`](https://github.com/Blazkowiz47/dl-wandb)
 
-To add a new local component scaffold inside the experiment repo:
+## Scaffold Commands
+
+Each `dl-core add ...` command creates the new module and updates the matching
+local package `__init__.py` export list under `src/`.
+
+Common local component scaffolds:
+
+```bash
+uv run dl-core add model MyResNet
+uv run dl-core add trainer MyTrainer
+uv run dl-core add callback MyMetrics
+uv run dl-core add metric_manager MyManager
+uv run dl-core add sampler MySampler
+uv run dl-core add criterion MyLoss
+uv run dl-core add augmentation MyAugmentation
+uv run dl-core add metric MyMetric
+uv run dl-core add executor MyExecutor
+```
+
+Sweep scaffolds are supported too:
 
 ```bash
-uv run dl-core add augmentation Custom1
+uv run dl-core add sweep DebugSweep
+uv run dl-core add sweep AzureEval --tracking azure_mlflow
+uv run dl-core add sweep MlflowBaseline --tracking mlflow
+uv run dl-core add sweep WandbAblation --tracking wandb
 ```
 
-Dataset scaffolds can now target a specific wrapper base:
+Generated sweep files:
+
+- live under `experiments/`
+- extend `../configs/base_sweep.yaml`
+- include runnable defaults in `fixed`
+- start with `grid: {}`
+- let the tracker derive sweep naming from the filename unless
+  `tracking.sweep_name` overrides it
+
+You can inspect registered components and built-in base classes directly from
+the CLI:
+
+```bash
+uv run dl-core describe dataset my_dataset --root-dir .
+uv run dl-core describe model my_resnet --root-dir .
+uv run dl-core describe class dl_core.core.FrameWrapper
+uv run dl-core describe class dl_azure.datasets.AzureComputeMultiFrameWrapper
+uv run dl-core describe dataset my_dataset --root-dir . --json
+```
+
+The describe command shows:
+
+- resolved class and registered names
+- constructor signature
+- inheritance chain
+- docstring
+- declared properties
+- class-level attributes
+- public methods defined on the class
+
+It does not discover instance attributes created dynamically inside `__init__`
+without constructing the class.
+
+Dataset scaffolds can target a specific wrapper base:
 
 ```bash
 uv run dl-core add dataset MyDataset
@@ -276,6 +332,8 @@ When `dl-azure` is importable, the dataset scaffold also exposes Azure bases:
 ```bash
 uv run dl-core add dataset AzureFrames --base azure_compute_frame
 uv run dl-core add dataset AzureSeq --base azure_compute_multiframe
+uv run dl-core add dataset AzureStream --base azure_streaming
+uv run dl-core add dataset AzureStreamSeq --base azure_streaming_multiframe
 ```
 
 Plain `deep-learning-core` currently exposes dataset bases for:

{deep_learning_core-0.0.2 → deep_learning_core-0.0.2a4}/README.md

@@ -162,6 +162,7 @@ Then:
    - `src/datasets/my_exp.py`
    - `configs/base.yaml`
    - `experiments/lr_sweep.yaml`
+   - `AGENTS.md`
 2. implement the generated dataset wrapper under `src/datasets/my_exp.py`
 3. adjust `configs/base.yaml` so it points at the dataset/model/trainer you want
 4. start with:
@@ -192,13 +193,68 @@ scaffold a `wandb` callback block, W&B tracking defaults, and `.env.example`.
 - [`dl-mlflow`](https://github.com/Blazkowiz47/dl-mlflow)
 - [`dl-wandb`](https://github.com/Blazkowiz47/dl-wandb)
 
-To add a new local component scaffold inside the experiment repo:
+## Scaffold Commands
+
+Each `dl-core add ...` command creates the new module and updates the matching
+local package `__init__.py` export list under `src/`.
+
+Common local component scaffolds:
+
+```bash
+uv run dl-core add model MyResNet
+uv run dl-core add trainer MyTrainer
+uv run dl-core add callback MyMetrics
+uv run dl-core add metric_manager MyManager
+uv run dl-core add sampler MySampler
+uv run dl-core add criterion MyLoss
+uv run dl-core add augmentation MyAugmentation
+uv run dl-core add metric MyMetric
+uv run dl-core add executor MyExecutor
+```
+
+Sweep scaffolds are supported too:
 
 ```bash
-uv run dl-core add augmentation Custom1
+uv run dl-core add sweep DebugSweep
+uv run dl-core add sweep AzureEval --tracking azure_mlflow
+uv run dl-core add sweep MlflowBaseline --tracking mlflow
+uv run dl-core add sweep WandbAblation --tracking wandb
 ```
 
-Dataset scaffolds can now target a specific wrapper base:
+Generated sweep files:
+
+- live under `experiments/`
+- extend `../configs/base_sweep.yaml`
+- include runnable defaults in `fixed`
+- start with `grid: {}`
+- let the tracker derive sweep naming from the filename unless
+  `tracking.sweep_name` overrides it
+
+You can inspect registered components and built-in base classes directly from
+the CLI:
+
+```bash
+uv run dl-core describe dataset my_dataset --root-dir .
+uv run dl-core describe model my_resnet --root-dir .
+uv run dl-core describe class dl_core.core.FrameWrapper
+uv run dl-core describe class dl_azure.datasets.AzureComputeMultiFrameWrapper
+uv run dl-core describe dataset my_dataset --root-dir . --json
+```
+
+The describe command shows:
+
+- resolved class and registered names
+- constructor signature
+- inheritance chain
+- docstring
+- declared properties
+- class-level attributes
+- public methods defined on the class
+
+It does not discover instance attributes created dynamically inside `__init__`
+without constructing the class.
+
+Dataset scaffolds can target a specific wrapper base:
 
 ```bash
 uv run dl-core add dataset MyDataset
@@ -212,6 +268,8 @@ When `dl-azure` is importable, the dataset scaffold also exposes Azure bases:
 ```bash
 uv run dl-core add dataset AzureFrames --base azure_compute_frame
 uv run dl-core add dataset AzureSeq --base azure_compute_multiframe
+uv run dl-core add dataset AzureStream --base azure_streaming
+uv run dl-core add dataset AzureStreamSeq --base azure_streaming_multiframe
 ```
 
 Plain `deep-learning-core` currently exposes dataset bases for:

{deep_learning_core-0.0.2 → deep_learning_core-0.0.2a4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "deep-learning-core"
-version = "0.0.2"
+version = "0.0.2a4"
 description = "Reusable deep learning framework core."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -49,13 +49,13 @@ dependencies = [
 
 [project.optional-dependencies]
 azure = [
-    "deep-learning-azure",
+    "deep-learning-azure==0.0.2a4",
 ]
 mlflow = [
-    "deep-learning-mlflow",
+    "deep-learning-mlflow==0.0.2a4",
 ]
 wandb = [
-    "deep-learning-wandb",
+    "deep-learning-wandb==0.0.2a4",
 ]
 dev = [
     "pytest",

{deep_learning_core-0.0.2 → deep_learning_core-0.0.2a4}/readme/technical/3_sweep_system.md

@@ -25,7 +25,8 @@ default_grid:
   optimizers.lr: [1e-4, 5e-4]
 
 tracking:
-  group: my_experiment
+  # sweep_name: my_custom_sweep
+  # Optional override. Defaults to the sweep filename.
   run_name_template: "lr_{optimizers.lr}"
 
 seeds: [2025]
@@ -71,7 +72,7 @@ That local artifact contract is what powers `dl-analyze-sweep`.
 Sweep templates support a `tracking` block used for:
 
 - run name templates
-- group names
+- sweep names
 - description templates
 - auto-generated tags
 

{deep_learning_core-0.0.2 → deep_learning_core-0.0.2a4}/src/dl_core/__init__.py

@@ -29,7 +29,7 @@ _BUILTIN_COMPONENT_MODULES = (
     "trainers",
 )
 
-__version__ = "0.0.2"
+__version__ = "0.0.2a4"
 
 
 def load_builtin_components() -> None:

deep_learning_core-0.0.2a4/src/dl_core/cli.py

@@ -0,0 +1,233 @@
+"""User-facing command line entry point for ``dl-core`` utilities."""
+
+from __future__ import annotations
+
+import argparse
+
+from dl_core.component_describer import (
+    describe_component,
+    format_description,
+    format_description_json,
+    list_supported_describe_types,
+    normalize_describe_type,
+)
+from dl_core.component_scaffold import (
+    create_component_scaffold,
+    list_supported_dataset_bases,
+    list_supported_component_types,
+    normalize_component_type,
+)
+from dl_core.sweep_scaffold import (
+    create_sweep_scaffold,
+    list_supported_tracking_backends,
+    normalize_tracking_backend,
+)
+
+
+def _list_supported_add_targets() -> list[str]:
+    """Return supported targets for ``dl-core add``."""
+    return sorted([*list_supported_component_types(), "sweep"])
+
+
+def main(argv: list[str] | None = None) -> int:
+    """Run the ``dl-core`` command line interface."""
+    parser = argparse.ArgumentParser(
+        prog="dl-core",
+        description="Utilities for working with dl-core experiment repositories.",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=(
+            "Common first steps:\n"
+            "  dl-init-experiment --name my-exp --root-dir .\n"
+            "  cd my-exp\n"
+            "  uv run dl-core add dataset LocalDataset\n"
+            "  uv run dl-core add sweep DebugSweep\n"
+            "  uv run dl-core describe dataset LocalDataset --root-dir .\n"
+            "  uv run dl-core add model MyResNet\n"
+            "  uv run dl-core add callback MyMetrics\n"
+            "  uv run dl-run --config configs/base.yaml"
+        ),
+    )
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    add_parser = subparsers.add_parser(
+        "add",
+        help="Create a local component or sweep scaffold inside an experiment repository.",
+        description=(
+            "Create a local component or sweep scaffold inside an experiment "
+            "repository.\n\nUse this after dl-init-experiment when you want an "
+            "extra local dataset, model, callback, trainer, or another sweep "
+            "file.\nComponent generation writes the module and updates the "
+            "matching src/<package>/__init__.py export list for you."
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=(
+            "Dataset examples:\n"
+            "  dl-core add dataset LocalDataset\n"
+            "  dl-core add dataset FrameDataset --base frame\n"
+            "  dl-core add dataset TextDataset --base text_sequence\n"
+            "  dl-core add dataset ActDataset --base adaptive_computation\n"
+            "  dl-core add dataset AzureFrames --base azure_compute_frame\n"
+            "  dl-core add dataset AzureSeq --base azure_compute_multiframe\n\n"
+            "Sweep examples:\n"
+            "  dl-core add sweep DebugSweep\n"
+            "  dl-core add sweep AzureEval --tracking azure_mlflow\n"
+            "  dl-core add sweep WandbAblation --tracking wandb\n\n"
+            "Other component examples:\n"
+            "  dl-core add model MyResNet\n"
+            "  dl-core add trainer MyTrainer\n"
+            "  dl-core add callback MyMetrics\n"
+            "  dl-core add metric_manager MyManager\n"
+            "  dl-core add sampler MySampler\n"
+            "  dl-core add criterion MyLoss\n"
+            "  dl-core add augmentation MyAugmentation\n"
+            "  dl-core add metric MyMetric\n"
+            "  dl-core add executor MyExecutor"
+        ),
+    )
+    add_parser.add_argument(
+        "component_type",
+        help=(
+            "Component type to generate. Supported values: "
+            f"{', '.join(_list_supported_add_targets())}"
+        ),
+    )
+    add_parser.add_argument(
+        "name",
+        help="Display name for the new component. It will also be normalized.",
+    )
+    add_parser.add_argument(
+        "--base",
+        help=(
+            "Dataset scaffold base to use when component_type is 'dataset'. "
+            "Supported values in this environment: "
+            f"{', '.join(list_supported_dataset_bases())}."
+        ),
+    )
+    add_parser.add_argument(
+        "--tracking",
+        default="local",
+        help=(
+            "Tracking backend to scaffold when component_type is 'sweep'. "
+            "Supported values: "
+            f"{', '.join(list_supported_tracking_backends())}. "
+            "Defaults to local."
+        ),
+    )
+    add_parser.add_argument(
+        "--root-dir",
+        default=".",
+        help=(
+            "Path inside the target experiment repository. "
+            "Defaults to the current directory."
+        ),
+    )
+    add_parser.add_argument(
+        "--force",
+        action="store_true",
+        help="Overwrite an existing component scaffold if it already exists.",
+    )
+
+    describe_parser = subparsers.add_parser(
+        "describe",
+        help="Describe a registered component class or importable base class.",
+        description=(
+            "Inspect a registered component or importable class.\n\n"
+            "Use this when you want constructor signatures, inheritance, "
+            "properties, class attributes, and public methods without opening "
+            "the source code first."
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=(
+            "Examples:\n"
+            "  dl-core describe dataset my_dataset --root-dir .\n"
+            "  dl-core describe model tall_swin --root-dir .\n"
+            "  dl-core describe class dl_core.core.FrameWrapper\n"
+            "  dl-core describe class dl_azure.datasets.AzureComputeMultiFrameWrapper\n"
+            "  dl-core describe dataset ocim_multiframe --root-dir . --json"
+        ),
+    )
+    describe_parser.add_argument(
+        "component_type",
+        help=(
+            "Describe target type. Supported values: "
+            f"{', '.join(list_supported_describe_types())}"
+        ),
+    )
+    describe_parser.add_argument(
+        "name",
+        help=(
+            "Registered component name or fully qualified class path when "
+            "component_type is 'class'."
+        ),
+    )
+    describe_parser.add_argument(
+        "--root-dir",
+        default=".",
+        help=(
+            "Path inside the target experiment repository. "
+            "Defaults to the current directory."
+        ),
+    )
+    describe_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Print the description as JSON instead of human-readable text.",
+    )
+
+    args = parser.parse_args(argv)
+    if args.command == "add":
+        normalized_add_target = args.component_type.strip().lower().replace(
+            "-", "_"
+        ).replace(" ", "_")
+        if normalized_add_target == "sweep":
+            if args.base:
+                parser.error(
+                    "--base is only supported when component_type is 'dataset'."
+                )
+            normalize_tracking_backend(args.tracking)
+            sweep_path = create_sweep_scaffold(
+                args.name,
+                root_dir=args.root_dir,
+                tracking_backend=args.tracking,
+                force=args.force,
+            )
+            print(f"Created sweep scaffold: {sweep_path}")
+            return 0
+
+        if args.tracking != "local":
+            parser.error(
+                "--tracking is only supported when component_type is 'sweep'."
+            )
+
+        if args.base and normalize_component_type(args.component_type) != "dataset":
+            parser.error("--base is only supported when component_type is 'dataset'.")
+
+        component_path = create_component_scaffold(
+            component_type=args.component_type,
+            name=args.name,
+            root_dir=args.root_dir,
+            dataset_base=args.base,
+            force=args.force,
+        )
+        print(f"Created component scaffold: {component_path}")
+        return 0
+
+    if args.command == "describe":
+        normalize_describe_type(args.component_type)
+        description = describe_component(
+            args.component_type,
+            args.name,
+            root_dir=args.root_dir,
+        )
+        if args.json:
+            print(format_description_json(description))
+        else:
+            print(format_description(description))
+        return 0
+
+    parser.error(f"Unsupported command: {args.command}")
+    return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())