morphic 0.2.2__tar.gz → 0.2.3__tar.gz

{morphic-0.2.2 → morphic-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: morphic
-Version: 0.2.2
+Version: 0.2.3
 Summary: Dynamic Python utilities for class registration, creation, and type checking
 Author-email: Abhishek Divekar <adivekar@utexas.edu>
 License-File: LICENSE

{morphic-0.2.2 → morphic-0.2.3}/docs/user-guide/typed-cli.md

@@ -45,6 +45,8 @@ app = App(_cli_parse_args=["--name", "alice", "--seed", "7"])
 | Repeated flag | `--name a --name b` | Last one wins |
 | Registry discriminator | `--backend.__type__ http` | Picks the concrete subclass for a `Typed + Registry` field; use [`parse_cli_args`](#pattern-registry-dispatch-with-__type__-discriminator) |
 | Discriminator inside inline JSON | `--backend '{"__type__":"http",...}'` | Same dispatch via inline JSON |
+| Path to JSON / YAML file | `--llm configs/llm/qwen.json` | For fields annotated `Annotated[X, TypedPath]`; see [TypedPath fields and the CLI](#typedpath-fields-and-the-cli) |
+| Path-loaded file + deep override | `--llm path.json --llm.model X` | File sets baseline, deep flag patches one leaf |
 
 ## Quick reference: what does NOT work by default
 
@@ -1005,6 +1007,171 @@ Fix: either
 1. Split into per-field flags: `--infra '{...}' --seed 42`.
 2. Use the [JSON-File config pattern](#pattern-json-file-config-with-cli-overrides) which loads root-level JSON properly.
 
+## TypedPath fields and the CLI
+
+`TypedPath` is a feature of `Typed` itself — it makes any field accept a path to a JSON / YAML file in addition to inline values. See [Typed user guide → Loading Typed fields from JSON / YAML files](typed.md#loading-typed-fields-from-json--yaml-files-typedpath) for the full feature description (programmatic usage, dict input, error semantics, identity preservation, optional fields, edge cases).
+
+This section covers the CLI-specific behaviors: how path-loaded fields interact with `parse_cli_args`, deep overrides, and pydantic-settings.
+
+### Annotation reminder
+
+```python
+from typing import Annotated, Optional
+from morphic import Typed, TypedPath
+
+class JudgeConfig(Typed):
+    llm: Annotated[LLMConfig, TypedPath]
+    infra: Annotated[InfraConfig, TypedPath]
+    refusal_classifier: Annotated[Optional[JudgeConfig], TypedPath] = None
+```
+
+`TypedPath` is a singleton — bare, no parens. `Annotated[X, TypedPath()]` (with parens) also works since the call returns the same singleton.
+
+### CLI usage
+
+`parse_cli_args` understands TypedPath-annotated fields and accepts path strings as values:
+
+```bash
+# Path to JSON file:
+python script.py --llm configs/llm/qwen.json
+
+# Path to YAML file (pyyaml must be installed):
+python script.py --llm configs/llm/qwen.yaml
+
+# Equals form:
+python script.py --llm=configs/llm/qwen.json
+
+# Inline JSON instead of a path (no .json/.yaml suffix on the value):
+python script.py --llm '{"model":"qwen","infra":{"mode":"Ray"}}'
+```
+
+The script:
+
+```python
+class JudgeConfig(Typed):
+    llm: Annotated[LLMConfig, TypedPath]
+
+if __name__ == "__main__":
+    config = JudgeConfig.parse_cli_args()
+```
+
+### CLI deep override on a path-loaded field
+
+This is the killer feature: load a base config from disk, then patch one or more leaves on the command line:
+
+```bash
+python script.py \
+    --llm configs/llm/qwen.json \
+    --llm.model qwen-large \
+    --llm.infra.address ray://prod:10001
+```
+
+The flow:
+
+1. `parse_cli_args` scans argv, sees `--llm <path.json>`, loads the file, and replaces the value with the inline-JSON-encoded contents.
+2. Pydantic-settings parses the rewritten argv. The CLI source emits `{"llm": {<contents>}}`.
+3. The deep-override flags `--llm.model qwen-large` and `--llm.infra.address ray://...` produce `{"llm": {"model": "qwen-large"}}` and `{"llm": {"infra": {"address": "..."}}}`.
+4. Pydantic-settings' `deep_update` merges these onto the loaded baseline. The override wins on every leaf key it explicitly sets; everything else stays from the file.
+
+The merge happens at the dict level, BEFORE the inner Typed is validated, so a `--llm.<sub>` flag works regardless of whether the sub-key was present in the JSON file.
+
+### Why this works (without a custom CLI source)
+
+Pydantic-settings' CLI source treats nested-model fields as "complex" and unconditionally calls `json.loads` on their values. A path string like `configs/foo.json` isn't valid JSON, so without intervention you'd get `SettingsError: error parsing value for field "llm" from source "CliSettingsSource"`.
+
+morphic handles this by preprocessing argv inside `parse_cli_args`: it walks the model's TypedPath-annotated fields, finds matching `--field <value>` (or `--field=<value>`) tokens where the value looks like a path, loads the file, and replaces the argv token with the JSON-encoded file contents. Pydantic-settings then handles everything from there using its native code paths — no custom `CliSettingsSource` subclass, no monkeypatching.
+
+### Path resolution rules apply on the CLI too
+
+The path resolution rules from [the Typed guide](typed.md#path-resolution-rules) apply to CLI input identically:
+
+```bash
+# Absolute path: used as-is.
+python script.py --llm /abs/path/qwen.json
+
+# Relative path: resolved against the current working directory.
+python script.py --llm configs/llm/qwen.json   # cwd-relative
+
+# Inside the loaded file, relative paths are resolved against THAT file's directory:
+# configs/llm/qwen.json:   {"infra": "../infra/sync.json"}
+#                          → resolved against configs/llm/, NOT cwd
+```
+
+This makes multi-level config trees work without any "config root" environment variable or custom resolution logic.
+
+### Multiple TypedPath fields in one CLI invocation
+
+```python
+class App(Typed):
+    target: Annotated[LLMConfig, TypedPath]
+    judge: Annotated[JudgeConfig, TypedPath]
+    refusal_classifier: Annotated[Optional[JudgeConfig], TypedPath] = None
+```
+
+```bash
+python train.py \
+    --target configs/llm/qwen-2-5-7b.json \
+    --judge configs/judges/wildguard.json \
+    --judge.llm.infra.address ray://prod:10001 \
+    --refusal-classifier configs/judges/promptguard.json
+```
+
+Each TypedPath field is loaded independently. Deep overrides target each one separately.
+
+### Optional TypedPath fields on the CLI
+
+`Annotated[Optional[X], TypedPath] = None` works on the CLI:
+
+```bash
+# Field omitted → stays None.
+python script.py --target configs/llm/qwen.json
+
+# Field explicitly set to None via inline JSON literal:
+python script.py --refusal-classifier null
+
+# Field set to a path:
+python script.py --refusal-classifier configs/judges/promptguard.json
+```
+
+### CLI errors
+
+| Scenario | What happens |
+|---|---|
+| `--llm /missing.json` | `FileNotFoundError` with field name + resolved absolute path |
+| `--llm /broken.json` (invalid JSON) | `ValueError` with file path + JSON parse error |
+| `--llm /list.json` (top-level is a JSON array, not object) | `ValueError`: the file's top-level value must be a mapping |
+| `--llm "not-a-path"` (string with no `.json`/`.yaml`/`.yml` suffix) | Falls through to pydantic-settings' normal CLI parsing → `SettingsError` because the string isn't valid JSON for a complex field |
+
+### Composing TypedPath with discriminator dispatch
+
+When a TypedPath field's inner type is a `Typed + Registry` abstract base, the loaded JSON/YAML can include a `__type__` discriminator to pick the concrete subclass. See [Typed CLI guide → Pattern: Registry dispatch](#pattern-registry-dispatch-with-__type__-discriminator) for the discriminator details.
+
+```python
+class Backend(Typed, Registry, ABC):
+    name: str
+
+class HttpBackend(Backend):
+    aliases = ("http",)
+    url: str
+
+class App(Typed):
+    backend: Annotated[Backend, TypedPath]
+```
+
+`configs/backends/prod.json`:
+```json
+{
+    "__type__": "http",
+    "name": "prod",
+    "url": "https://api.example.com"
+}
+```
+
+```bash
+python script.py --backend configs/backends/prod.json
+# → app.backend is an HttpBackend instance
+```
+
 ## See also
 
 - [Typed user guide](typed.md) — the foundational guide for `Typed`, validation, lifecycle hooks, frozen models, `PrivateAttr`.

{morphic-0.2.2 → morphic-0.2.3}/docs/user-guide/typed.md

@@ -13,6 +13,7 @@ Typed is built on Pydantic's `BaseSettings` (a `BaseModel` with extra `__init__`
 - **Advanced error handling** - Enhanced error messages with detailed validation information
 - **Hierarchical type support** - Nested Typed objects, lists, and dictionaries with automatic conversion
 - **Identity preservation** - A pre-built Typed passed as a field of an outer Typed is reused, not cloned (`revalidate_instances="never"`)
+- **Load from JSON / YAML files** - Annotate any field with `Annotated[X, TypedPath]` to make it accept a path to a file in addition to instances and dicts; see [Loading Typed fields from JSON / YAML files (TypedPath)](#loading-typed-fields-from-json--yaml-files-typedpath)
 - **CLI integration** - Every Typed accepts `_cli_parse_args=...` and supports nested overrides (`--infra.ray-init.address X`); use [`Config.parse_cli_args()`](typed-cli.md#pattern-pure-native-cli) as the recommended entrypoint
 - **Registry CLI dispatch** - Typed + Registry hierarchies support `__type__` discriminator dispatch in kwargs, dict input, and CLI flags (`--backend.__type__ http`); see [Typed CLI guide](typed-cli.md#pattern-registry-dispatch-with-__type__-discriminator)
 - **Source isolation** - Environment variables and dotenv files are NOT read by default, so a field named `user` won't accidentally pick up `$USER`
@@ -350,6 +351,199 @@ except ValidationError:
     print("Assignment validation failed")
 ```
 
+## Loading Typed fields from JSON / YAML files (TypedPath)
+
+`TypedPath` is a marker that turns any field into a "path-loadable" field. The user can pass either an inline value (instance, dict) or a path to a `.json` / `.yaml` / `.yml` file, and the file's contents are loaded, parsed, and validated as the field's type.
+
+This works **everywhere** — programmatically, in dict inputs, in CLI flags. Although the CLI is the most common motivation (deeply-nested configs are awkward to express as one inline JSON string on the command line), the same field also accepts paths when constructed from Python code, which makes notebook workflows and tests straightforward.
+
+### Annotation
+
+```python
+from typing import Annotated, Optional
+from morphic import Typed, TypedPath
+
+class JudgeConfig(Typed):
+    llm: Annotated[LLMConfig, TypedPath]
+    infra: Annotated[InfraConfig, TypedPath]
+    refusal_classifier: Annotated[Optional[JudgeConfig], TypedPath] = None
+```
+
+`TypedPath` is a singleton — it does not take any arguments and you do not call it. Just write `Annotated[X, TypedPath]` (no parens). `Annotated[X, TypedPath()]` also works (the call returns the same singleton), but the bare form is preferred so users don't have to remember to instantiate it.
+
+### Three accepted input forms
+
+A field annotated `Annotated[X, TypedPath]` accepts three kinds of value, in any context (programmatic, dict, CLI):
+
+1. **A pre-built `X` instance** — passed through unchanged, identity preserved.
+2. **A dict matching `X`'s schema** — validated as `X`.
+3. **A path string or `Path` object** ending in `.json`, `.yaml`, or `.yml` — the file is read, parsed, and validated as `X`.
+
+#### Programmatic construction
+
+```python
+from pathlib import Path
+from morphic import Typed, TypedPath
+from typing import Annotated
+
+class LLMConfig(Typed):
+    model: str
+    timeout: float = 30.0
+
+class App(Typed):
+    llm: Annotated[LLMConfig, TypedPath]
+
+# Form 1: pre-built instance — identity preserved
+llm = LLMConfig(model="qwen", timeout=60.0)
+app = App(llm=llm)
+assert app.llm is llm   # same instance
+
+# Form 2: dict
+app = App(llm={"model": "qwen", "timeout": 60.0})
+assert isinstance(app.llm, LLMConfig)
+
+# Form 3a: path string (relative or absolute)
+app = App(llm="configs/llm/qwen.json")
+
+# Form 3b: pathlib.Path object
+app = App(llm=Path("configs/llm/qwen.yaml"))
+```
+
+YAML support requires `pyyaml` to be installed. If a `.yaml` / `.yml` path arrives but `pyyaml` isn't installed, you get a clear `ImportError` pointing at the missing package. JSON is always supported (Python stdlib).
+
+#### Dict input with embedded paths
+
+This pattern is common when constructing a Typed from JSON data that someone else parsed for you:
+
+```python
+data = {
+    "llm": "configs/llm/qwen.json",       # Path string inside a dict
+    "infra": {"mode": "Sync"},            # Inline dict for another field
+}
+app = App.model_validate(data)
+```
+
+The path string for `llm` triggers file loading; the inline dict for `infra` is used as-is.
+
+#### CLI input
+
+The CLI just routes to the same path-loading machinery via [`Config.parse_cli_args`](typed-cli.md#pattern-pure-native-cli):
+
+```bash
+python script.py --llm configs/llm/qwen.json --infra configs/infra/ec2-ray.json
+
+# Or override individual leaves of the loaded file:
+python script.py --llm configs/llm/qwen.json --llm.model qwen-large
+```
+
+See [the CLI guide](typed-cli.md#typedpath-load-nested-fields-from-json--yaml-files) for full CLI semantics including deep overrides on path-loaded fields.
+
+### Path resolution rules
+
+Three rules, in this order:
+
+1. **Absolute paths** are used as-is.
+2. **Relative paths inside a loaded file** are resolved against the directory of THAT file (the parent file currently being loaded).
+3. **Relative paths from outside any file** (programmatic / top-level CLI input) are resolved against the current working directory.
+
+The "parent file's directory" rule (#2) makes nested config files compose without any work from the user. Consider this layout:
+
+```
+configs/
+├── judges/
+│   └── wildguard.json   ──> {"llm": "../../llm/wildguard.json", "name": "wildguard"}
+├── llm/
+│   └── wildguard.json   ──> {"model": "...", "infra": "../infra/sync.json"}
+└── infra/
+    └── sync.json        ──> {"mode": "Sync"}
+```
+
+When you run `JudgeConfig.parse_cli_args(["--judge", "configs/judges/wildguard.json"])`:
+
+1. The outer file `configs/judges/wildguard.json` is loaded.
+2. Its `"llm"` field has the relative path `"../../llm/wildguard.json"`. This is resolved against `configs/judges/`, giving `configs/llm/wildguard.json`.
+3. That file is loaded. Its `"infra"` field has the relative path `"../infra/sync.json"`, which is resolved against `configs/llm/`, giving `configs/infra/sync.json`.
+4. That file is loaded.
+
+You don't pass any "config root" parameter; the parent-dir rule threads through all the levels automatically.
+
+### Optional fields
+
+`Annotated[Optional[X], TypedPath] = None` works as expected:
+
+```python
+class JudgeConfig(Typed):
+    llm: Annotated[LLMConfig, TypedPath]
+    refusal_classifier: Annotated[Optional[JudgeConfig], TypedPath] = None
+```
+
+The user can omit `refusal_classifier`, pass `None`, an instance, a dict, or a path. Loading only triggers when a non-`None` value is provided.
+
+### Validation always runs on loaded content
+
+Reading a file does NOT skip validation. The dict parsed from JSON/YAML goes through the same validation pipeline as any other dict input:
+
+```python
+class LLMConfig(Typed):
+    model: str
+    timeout: float = 30.0
+
+# A file with bad data:
+# config.json: {"model": "qwen", "timeout": "not-a-number"}
+
+app = App(llm="config.json")
+# ValueError: Cannot create Pydantic instance of type 'LLMConfig' ...
+#   timeout: Input should be a valid number, ...
+```
+
+The same applies to:
+- Missing required fields → caught.
+- Extra fields → rejected (Typed's `extra="forbid"`).
+- Field type mismatches → caught.
+
+### Errors
+
+| Scenario | Error type | Error message includes |
+|---|---|---|
+| Path doesn't exist | `FileNotFoundError` | field name, resolved absolute path, original input |
+| File is not a regular file (e.g., a directory named `foo.json`) | `ValueError` | field name, resolved path |
+| Corrupt JSON | `ValueError` | field name, file path, JSON parse error |
+| Corrupt YAML | `ValueError` | field name, file path, YAML parse error |
+| YAML file but `pyyaml` not installed | `ImportError` | install instruction |
+| File parses but top-level is not a JSON object / YAML mapping | `ValueError` | field name, actual type |
+| Loaded content fails inner type validation | `ValueError` (Pydantic `ValidationError` wrapped) | full field path, offending field, expected type |
+
+### What `TypedPath` is NOT
+
+These are deliberate non-features, not oversights:
+
+- **NOT a runtime type.** After validation, no field on any Typed instance has type `TypedPath`. The annotation is consumed during validation. Reading `judge.llm` always returns an `LLMConfig` instance.
+- **NOT lazy.** The file is read and parsed at construction time. If the file is missing or malformed, construction fails immediately.
+- **NOT cached.** Each construction reads the file fresh. If you load the same file twice, you get two distinct (but equal) instances.
+- **NOT for collection elements.** `List[Annotated[X, TypedPath]]` does NOT load each list element from a path. TypedPath only triggers on single-Typed fields. If you need a list of file-loadable items, load them in your code and pass the list of instances.
+- **NOT a global validator.** The path-loading machinery only runs on classes that have at least one TypedPath-annotated field. Pure Typeds with no TypedPath fields pay zero overhead per construction.
+- **NOT for remote URIs.** Only local filesystem paths are supported (no `s3://`, no `https://`). Add fetching code yourself if you need to download a config.
+
+### Mixing TypedPath with other Typed features
+
+`TypedPath` composes cleanly with everything else in Typed:
+
+- **Identity preservation**: still applies for the instance branch (Form 1). When the user passes a pre-built instance, `o.llm is the_instance`. The path-loading branch always produces a fresh instance (the file system is the source of truth).
+- **`post_initialize`**: runs once on every instance produced, including those loaded from files. The framework's [#12876 idempotency marker](#identity-preservation-and-pre-built-instances) ensures heavy `post_initialize` work runs exactly once per instance.
+- **Registry dispatch**: `Annotated[BackendBase, TypedPath]` works alongside `__type__` discriminators. The loaded JSON/YAML can include a `__type__` key to select the concrete subclass. See [Typed CLI guide → Pattern: Registry dispatch](typed-cli.md#pattern-registry-dispatch-with-__type__-discriminator).
+- **Lifecycle hooks**: `pre_initialize` and `pre_validate` run after path loading (so they see the dict, not the path string).
+
+### When to use `TypedPath`
+
+| Need | Use |
+|---|---|
+| Single root config from a JSON/YAML file | [JSON-File config pattern](typed-cli.md#pattern-json-file-config-with-cli-overrides) (small argparse wrapper) |
+| Multiple deeply-nested config files cross-referencing each other | `TypedPath` — much cleaner than chaining the wrapper for every sub-config |
+| Some fields are simple inline values, others are big nested configs | `TypedPath` on the big fields, leave the simple ones plain |
+| All config comes from CLI flags / env vars | Plain Typed; no TypedPath needed |
+
+The trojanshot use case is squarely TypedPath territory: `JudgeConfig` references an `LLMConfig` file, which references an `InfraConfig` file, and each is independently editable. The `Annotated[X, TypedPath]` annotation models this cleanly without any custom argv preprocessor or config-loader scaffolding.
+
 ## Default Value Validation and Conversion
 
 Pydantic validates and converts default values automatically, ensuring type safety and preventing common errors.

{morphic-0.2.2 → morphic-0.2.3}/src/morphic/__init__.py

@@ -91,6 +91,7 @@ from .typed import (
     TYPED_REGISTRY_DISCRIMINATOR_KEY,
     MutableTyped,
     Typed,
+    TypedPath,
     ValidationError,
     validate,
 )
@@ -104,6 +105,7 @@ __all__ = [
     "Typed",
     "MutableTyped",
     "TYPED_REGISTRY_DISCRIMINATOR_KEY",
+    "TypedPath",
     "validate",
     "ValidationError",
     "classproperty",