cimprofiles 0.1.0a2__py3-none-any.whl

cimprofiles/README.md

@@ -0,0 +1,291 @@
+# CIM Data Profiles
+
+This package contains the Python data profile modules used by cim-graph
+(and any other tool that wants the CIM classes without a database stack).
+Each profile is a package of dataclasses generated from a CIM UML export
+(via CIMTool or Enterprise Architect Schema Composer).
+
+## How profiles are selected
+
+In cim-graph, set `CIMG_CIM_PROFILE` to the dotted module path of the profile you want:
+
+```bash
+export CIMG_CIM_PROFILE="cimhub_2023"
+# or fully-qualified:
+export CIMG_CIM_PROFILE="cimprofiles.cim17.cimhub_2023"
+```
+
+Profiles live under their CIM version (`cimprofiles.cim17.cimhub_2023`). A short name,
+optionally with a sub-profile (`cim18gmdm.connectivity`), is looked up in
+`cimprofiles.PROFILE_PACKAGES` by `cimprofiles.profile_module`. The pre-0.6
+`cimgraph.data_profile.<name>` paths still work and resolve the same way.
+
+### Multi-profile merge — new in 0.5
+
+Comma-separate two or more sub-profile paths to merge them at runtime:
+
+```bash
+export CIMG_CIM_PROFILE="cimprofiles.cim18.cim18gmdm.connectivity,\
+cimprofiles.cim18.cim18gmdm.electrical,\
+cimprofiles.cim18.cim18gmdm.asset"
+```
+
+`get_cim_profile()` calls `merge_profiles()` once and memoises the result.
+Every caller in the same process — `XMLFile`, `FeederModel`, any utility
+function — receives the **same module object**, so class identity is
+maintained across all graph key lookups.
+
+The merged module satisfies the same contract as any single profile: it has
+an `__all__` list and each class is accessible via `getattr(merged, name)`.
+
+---
+
+## Available profiles
+
+### Monolithic profiles
+
+These are single-package profiles — set `CIMG_CIM_PROFILE` to the package
+name and use the classes directly.
+
+| Profile | Classes | Description |
+|---|---|---|
+| `cimhub_2023` | 397 | CIMHub 2023 distribution feeder profile (CIM100) |
+| `cimhub_2026` | 449 | CIMHub 2026 distribution feeder profile (CIM100) |
+| `cim16v33` | 533 | IEC CIM16 v33 |
+| `cim17v40` | 1930 | IEC CIM17 v40 |
+| `cim18gad` | 1348 | CIM18 GridAPPS-D profile |
+| `rc4_2021` | 571 | RC4 2021 profile |
+
+### cim18gmdm — EPRI GMDM split profile
+
+The GMDM model is split into sub-profiles by functional area. Use the
+pre-baked merged package (`cim18gmdm`) for a single-import experience, or
+compose the sub-profiles you need at runtime via the comma-spec.
+
+#### Pre-baked merged package
+
+```bash
+export CIMG_CIM_PROFILE="cimprofiles.cim18.cim18gmdm"
+```
+
+This is a committed `__init__.py` generated by `python -m cimprofiles.build_merged_profiles`.
+It re-exports all classes from all five sub-profiles (978 classes total) and
+is the equivalent of setting the comma-spec to all five sub-profiles.
+Use it when you need every profile class available statically (e.g. for
+Pylance completion on the full model).
+
+#### Sub-profiles
+
+Compose only the sub-profiles you need to keep the merged class count small
+and avoid loading profile classes irrelevant to your task.
+
+| Sub-profile | Classes | Key classes |
+|---|---|---|
+| `cim18gmdm.connectivity` | 56 | `ACLineSegment`, `ConnectivityNode`, `Terminal`, `Breaker`, `EnergyConsumer`, `PowerTransformer` |
+| `cim18gmdm.electrical` | 77 | `PerLengthPhaseImpedance`, `PhaseImpedanceData`, `TransformerTankInfo`, `ACLineSegment.length`, `ACLineSegment.PerLengthImpedance` |
+| `cim18gmdm.asset` | 24 | `Asset`, `AssetInfo`, `TransformerEndInfo`, `NoLoadTest`, `ShortCircuitTest` |
+| `cim18gmdm.location` | 39 | `Location`, `PositionPoint`, `CoordinateSystem` |
+| `cim18gmdm.marketnode` | 5 | `IndividualPnode`, `AggregateNode`, `Pnode` |
+| `cim18gmdm.diagram` | 12 | `Diagram`, `DiagramObject`, `DiagramObjectPoint` |
+| `cim18gmdm.canonical` | 765 | All classes in one file — legacy, prefer the pre-baked merged package |
+
+**Common combinations:**
+
+```bash
+# Topology + electrical parameters (most common for power flow):
+export CIMG_CIM_PROFILE="cimprofiles.cim18.cim18gmdm.connectivity,\
+cimprofiles.cim18.cim18gmdm.electrical"
+
+# Add asset catalog (transformer nameplate data, wire info):
+export CIMG_CIM_PROFILE="cimprofiles.cim18.cim18gmdm.connectivity,\
+cimprofiles.cim18.cim18gmdm.electrical,\
+cimprofiles.cim18.cim18gmdm.asset"
+
+# Full five-part merge (equivalent to using the pre-baked cim18gmdm package):
+export CIMG_CIM_PROFILE="cimprofiles.cim18.cim18gmdm.connectivity,\
+cimprofiles.cim18.cim18gmdm.electrical,\
+cimprofiles.cim18.cim18gmdm.asset,\
+cimprofiles.cim18.cim18gmdm.location,\
+cimprofiles.cim18.cim18gmdm.marketnode"
+```
+
+### cgmes_3_0_0 — IEC CGMES 3.0 split profile
+
+CGMES 3.0 is split into sub-profiles by exchange type, following the IEC
+61970-600 document structure.
+
+| Sub-profile | Classes | Description |
+|---|---|---|
+| `cgmes_3_0_0.core_equipment` | 203 | EQ — equipment definitions |
+| `cgmes_3_0_0.equipment_boundary` | 22 | EQBD — boundary equipment |
+| `cgmes_3_0_0.operation` | 39 | OP — operational limits |
+| `cgmes_3_0_0.short_circuit` | 48 | SC — short-circuit parameters |
+| `cgmes_3_0_0.steady_state_hypothesis` | 85 | SSH — schedule/setpoints |
+| `cgmes_3_0_0.state_variables` | 29 | SV — solved state |
+| `cgmes_3_0_0.topology` | 15 | TP — bus-branch topology |
+| `cgmes_3_0_0.geographical_location` | 12 | GL — geographic coordinates |
+| `cgmes_3_0_0.diagram` | 12 | DL — diagram layout |
+
+**Typical CGMES merge (EQ + SSH + SV):**
+
+```bash
+export CIMG_CIM_PROFILE="cimprofiles.cim17.cgmes_3_0_0.core_equipment,\
+cimprofiles.cim17.cgmes_3_0_0.steady_state_hypothesis,\
+cimprofiles.cim17.cgmes_3_0_0.state_variables"
+```
+
+---
+
+## Loading split exports
+
+For models spread across multiple part files, use an IEC 61970-552 metadata
+header (`.cimx` file) to load all parts in one call:
+
+```python
+import os
+from cimgraph.databases import XMLFile
+from cimgraph.models import FeederModel
+
+os.environ['CIMG_CIM_PROFILE'] = (
+    'cimprofiles.cim18.cim18gmdm.connectivity,'
+    'cimprofiles.cim18.cim18gmdm.electrical,'
+    'cimprofiles.cim18.cim18gmdm.asset'
+)
+
+# 9500node.cimx lists the individual part files as dcat:Distribution entries.
+# XMLFile resolves each accessURL relative to the .cimx file and loads all
+# parts into one graph in a single two-pass read.
+file    = XMLFile(metadata='profile_parts/9500node.cimx')
+network = FeederModel(container=None, connection=file)
+```
+
+The `.cimx` format follows IEC 61970-552 §8. Each part file is a
+`dcat:Distribution` with an `accessURL` (relative path) and a
+`dcterms:conformsTo` URI naming the sub-profile it was written against.
+
+---
+
+## Adding a custom profile
+
+1. Export your UML profile from CIMTool or Schema Composer as `.xsd`.
+2. Run `uv run xsdata generate` (or the equivalent CIMTool builder) to
+   produce the dataclass package.
+3. Place the generated package under `src/cimprofiles/` (or any importable package).
+4. Set `CIMG_CIM_PROFILE` to the dotted path of the package.
+
+For split profiles, generate one sub-package per functional area and use
+the comma-spec to merge them at runtime.
+
+---
+
+## Generating a pre-baked `__init__.py` for a split profile
+
+A pre-baked `__init__.py` lets users `import cimprofiles.<name> as cim`
+without knowing the sub-profile names, and lets Pylance/mypy resolve the full
+merged class surface statically.
+
+### 1. Register the release in `merge_releases.yaml`
+
+Add an entry to `src/cimprofiles/merge_releases.yaml`:
+
+```yaml
+releases:
+  - name: my_profile          # must match the package directory name
+    package: cimprofiles.my_profile
+    parts:
+      - cimprofiles.my_profile.connectivity
+      - cimprofiles.my_profile.electrical
+      - cimprofiles.my_profile.asset
+    description: "My profile — merged from connectivity + electrical + asset"
+```
+
+`parts` is an ordered list. First-seen order wins for classes that appear in
+multiple sub-profiles, so put the most complete sub-profile first.
+
+**Prerequisite — base class consistency:** `merge_profiles` fails fast if two
+sub-profiles declare the same class with *different* direct base classes. This
+happens when a partial sub-profile uses `Identity` as a stub base for a class
+that a fuller sub-profile roots at `IdentifiedObject` or deeper. Fix by
+regenerating the offending sub-profile from CIMTool with the full inheritance
+chain before running the build script.
+
+To diagnose conflicts before registering:
+
+```python
+import importlib
+from collections import defaultdict
+from cimprofiles.merge import merge_profiles
+
+parts = ['cimprofiles.my_profile.connectivity',
+         'cimprofiles.my_profile.electrical']
+mods = [importlib.import_module(p) for p in parts]
+
+class_defs = defaultdict(list)
+for mod in mods:
+    for name in getattr(mod, '__all__', []):
+        obj = getattr(mod, name, None)
+        if obj and isinstance(obj, type):
+            class_defs[name].append(obj)
+
+for name, defs in sorted(class_defs.items()):
+    bases = set()
+    for cls in defs:
+        for base in cls.__mro__[1:]:
+            if base is not object:
+                bases.add(base.__name__)
+                break
+    if len(bases) > 1:
+        print(f'{name}: conflicting bases {sorted(bases)}')
+```
+
+### 2. Run the build script
+
+```bash
+uv run python -m cimprofiles.build_merged_profiles
+```
+
+This writes a committed `__init__.py` into each package listed in
+`merge_releases.yaml` and smoke-tests the import. Re-run after adding or
+changing sub-profile parts.
+
+The generated file is a normal Python module — Pylance reads it directly.
+It re-exports every class from the appropriate sub-profile, so the source
+of each class is traceable and static analysis works without the runtime
+merge machinery.
+
+### Current status
+
+| Profile | `__init__.py` | Notes |
+|---|---|---|
+| `cim18gmdm` | ✅ generated | 116 classes from 5 sub-profiles |
+| `cgmes_3_0_0` | ✅ generated | 275 classes from 9 sub-profiles (EQ+EQBD+OP+SC+SSH+SV+TP+GL+DL) |
+
+---
+
+## Runtime merge internals
+
+`cimprofiles.merge.merge_profiles(*modules)` builds a synthetic
+`types.ModuleType` by:
+
+1. Collecting every name from each module's `__all__`.
+2. Topologically sorting by inheritance (parents before children).
+3. For classes that appear in more than one sub-profile: creating a new
+   dataclass with the **union of direct fields**, reparented onto the
+   already-merged base class.
+4. For classes that appear in only one sub-profile: adopting the class
+   as-is but reparenting it onto the merged base if the base was rebuilt.
+5. Enums, CIMUnit subclasses, and the shared `Identity` root are taken from
+   the first profile that defines them.
+
+The merge fails fast if two sub-profiles declare the same class name with
+**different base classes** — that would silently drop a lineage.
+
+To generate a static type-stub file for Pylance/mypy (the replacement for
+the hand-maintained `canonical.py`):
+
+```python
+from cimprofiles.cim18.cim18gmdm import connectivity, electrical
+from cimprofiles.merge import generate_type_stubs
+
+generate_type_stubs(connectivity, electrical, output_path='cim_merged.pyi')
+```

cimprofiles/__init__.py

@@ -0,0 +1,36 @@
+from enum import Enum
+
+
+class CIM_PROFILE(Enum):
+    RC4_2021 = 'rc4_2021'
+    CIMHUB_2023 = 'cimhub_2023'
+    CIMHUB_2026 = 'cimhub_2026'
+    CIM16V33 = 'cim16v33'
+    CIM17V40 = 'cim17v40'
+    CIM18GMDM = 'cim18gmdm'
+
+
+# Short profile names, as used by CIMG_CIM_PROFILE and cimgraph.data_profile -> package
+PROFILE_PACKAGES = {
+    'cim16v33': 'cimprofiles.cim16.cim16v33',
+    'cgmes_3_0_0': 'cimprofiles.cim17.cgmes_3_0_0',
+    'cim17v40': 'cimprofiles.cim17.cim17v40',
+    'cimhub_2023': 'cimprofiles.cim17.cimhub_2023',
+    'rc4_2021': 'cimprofiles.cim17.gridappsd_rc4_2021',
+    'cim18gad': 'cimprofiles.cim18.cim18gad',
+    'cim18gmdm': 'cimprofiles.cim18.cim18gmdm',
+    'cimhub_2026': 'cimprofiles.cim18.cimhub_2026',
+}
+
+
+def profile_module(name: str) -> str:
+    """
+    Returns the module path of a profile given by short name, optionally with a
+    sub-profile: 'cimhub_2023' -> 'cimprofiles.cim17.cimhub_2023',
+    'cim18gmdm.connectivity' -> 'cimprofiles.cim18.cim18gmdm.connectivity'.
+    """
+    short_name, dot, sub_profile = name.partition('.')
+    if short_name not in PROFILE_PACKAGES:
+        raise ValueError(f'Unknown CIM profile {name!r}; known profiles: '
+                         f'{", ".join(PROFILE_PACKAGES)}')
+    return PROFILE_PACKAGES[short_name] + dot + sub_profile

cimprofiles/build_merged_profiles.py

@@ -0,0 +1,261 @@
+"""Build pre-baked merged profile __init__.py files from merge_releases.yaml.
+
+Run with the development dependencies installed (needs pyyaml):
+    uv run python -m cimprofiles.build_merged_profiles
+    uv run python -m cimprofiles.build_merged_profiles --config path/to/releases.yaml
+
+``--config`` builds releases for a profile package outside this repo (e.g.
+cim-vfm-profiles). Each release's ``package`` must be importable; its
+``__init__.py`` is written wherever that package lives.
+
+Each entry in merge_releases.yaml produces a committed __init__.py that
+re-exports the merged class surface. The file is a normal Python module —
+Pylance reads it directly, and `import cimprofiles.<name> as cim`
+resolves to the merged class set with no runtime merge machinery needed.
+
+For classes that appear in only one sub-profile, the output is a simple
+re-import. For classes that overlap across sub-profiles (same name, different
+fields in each), the output is an inline @dataclass definition that carries the
+union of all direct fields — this is the case for CGMES where e.g. ACLineSegment
+appears in both EQ (r, x, bch) and SC (r0, x0, b0ch) sub-profiles.
+
+Add this to the PyPI release process for any profile listed in the config.
+"""
+from __future__ import annotations
+
+import argparse
+import dataclasses
+import importlib
+import importlib.util
+import sys
+import textwrap
+from collections import defaultdict
+from pathlib import Path
+
+import yaml
+
+_CONFIG = Path(__file__).parent / 'merge_releases.yaml'
+
+
+def _load_config(config: Path) -> list[dict]:
+    with open(config) as f:
+        data = yaml.safe_load(f)
+    return data['releases']
+
+
+def _package_dir(package: str) -> Path:
+    """Where *package* lives — resolved by import, so external packages work too."""
+    spec = importlib.util.find_spec(package)
+    if spec is None or not spec.submodule_search_locations:
+        raise FileNotFoundError(
+            f'Package not importable: {package}\n'
+            f'Create it and add sub-profile packages before running this script.'
+        )
+    return Path(next(iter(spec.submodule_search_locations)))
+
+
+def _sanitize_metadata(meta: dict) -> dict:
+    """Strip newlines from string metadata values so repr() is safe in source."""
+    return {
+        k: v.replace('\n', ' ').strip() if isinstance(v, str) else v
+        for k, v in meta.items()
+    }
+
+
+def _field_source(f: dataclasses.Field) -> str:
+    """Emit a field(...) expression that recreates the field descriptor."""
+    parts: list[str] = []
+    if f.default is not dataclasses.MISSING:
+        parts.append(f'default={f.default!r}')
+    elif f.default_factory is not dataclasses.MISSING:
+        factory = f.default_factory
+        if factory is list:
+            parts.append('default_factory=list')
+        elif factory is dict:
+            parts.append('default_factory=dict')
+        else:
+            parts.append(f'default_factory={factory.__name__}')
+    meta = _sanitize_metadata(dict(f.metadata))
+    parts.append(f'metadata={meta!r}')
+    return f'field({", ".join(parts)})'
+
+
+def _emit_inline_class(name: str, merged_cls: type, merged_base_name: str) -> list[str]:
+    """Emit an inline @dataclass definition for an overlapping class."""
+    from cimprofiles.merge import _get_direct_fields
+    direct = _get_direct_fields(merged_cls)
+    lines: list[str] = []
+    lines.append('@dataclass(repr=False, eq=False)')
+    lines.append(f'class {name}({merged_base_name}):')
+
+    # Preserve the original docstring — make_dataclass overwrites __doc__ with
+    # the signature string, so we must emit it explicitly.
+    doc = merged_cls.__doc__
+    if doc and not doc.startswith(f'{name}('):
+        # Indent each line of the docstring body by 4 spaces.
+        indented = '\n'.join('    ' + l for l in doc.split('\n'))
+        lines.append(f'    """{indented.strip()}\n    """')
+
+    if not direct:
+        lines.append('    pass')
+    else:
+        for fname, f in direct.items():
+            ftype = f.type if isinstance(f.type, str) else str(f.type)
+            field_expr = _field_source(f)
+            # Never wrap — the metadata repr contains string literals that
+            # textwrap would break mid-literal, producing a SyntaxError.
+            lines.append(f'    {fname}: {ftype} = {field_expr}')
+    return lines
+
+
+def _build_init(release: dict) -> str:
+    """Return the generated __init__.py content for one release entry."""
+    from cimprofiles.identity import Identity
+    from cimprofiles.merge import _get_direct_fields, merge_profiles
+
+    parts = release['parts']
+    description = release.get('description', '')
+    package = release['package']
+
+    modules = [importlib.import_module(p) for p in parts]
+    merged = merge_profiles(*modules)
+
+    names = sorted(merged.__all__)
+
+    # ── Determine which classes are overlapping vs single-source ──────────
+    # A class is "overlapping" if the same name with *different field sets*
+    # appears in more than one sub-profile.  For those we emit an inline
+    # @dataclass.  For single-source (or Identity/enum/cimunit), we re-import.
+    name_to_parts: dict[str, list[str]] = defaultdict(list)
+    name_to_first_part: dict[str, str] = {}
+    for part_path, mod in zip(parts, modules):
+        for n in getattr(mod, '__all__', []):
+            if n not in name_to_first_part:
+                name_to_first_part[n] = part_path
+            name_to_parts[n].append(part_path)
+
+    # Collect the merged module's class objects
+    merged_classes: dict[str, type] = {}
+    for n in names:
+        obj = getattr(merged, n, None)
+        if obj is not None and isinstance(obj, type):
+            merged_classes[n] = obj
+
+    # An overlapping dataclass is one where the merged object is NOT the same
+    # object as in the first-seen sub-profile (merge_profiles rebuilt it).
+    from cimprofiles.merge import _is_profile_dataclass
+    overlapping: set[str] = set()
+    for n, cls in merged_classes.items():
+        if not _is_profile_dataclass(cls) or n == 'Identity':
+            continue
+        first_mod = importlib.import_module(name_to_first_part[n])
+        first_cls = getattr(first_mod, n, None)
+        if first_cls is not None and cls is not first_cls:
+            overlapping.add(n)
+
+    # ── Topological order for the inline dataclass section ─────────────────
+    # Only overlapping dataclasses need topo sorting (parents before children).
+    from cimprofiles.merge import _topo_sort
+    class_defs_for_sort: dict[str, list[type]] = {
+        n: [merged_classes[n]] for n in overlapping if n in merged_classes
+    }
+    topo_overlapping = _topo_sort(list(overlapping), class_defs_for_sort)
+
+    # ── Imports for non-overlapping names ──────────────────────────────────
+    part_to_simple: dict[str, list[str]] = defaultdict(list)
+    for n in sorted(names):
+        if n not in overlapping:
+            src = name_to_first_part.get(n, parts[0])
+            part_to_simple[src].append(n)
+
+    import_lines: list[str] = []
+    for part_path in parts:
+        part_names = part_to_simple.get(part_path, [])
+        if not part_names:
+            continue
+        header = f'from {part_path} import ('
+        joined = ', '.join(part_names)
+        wrapped_names = textwrap.fill(
+            joined,
+            width=88,
+            initial_indent='    ',
+            subsequent_indent='    ',
+            break_long_words=False,
+            break_on_hyphens=False,
+        )
+        import_lines.append(f'{header}\n{wrapped_names})')
+
+    # ── Inline @dataclass section ──────────────────────────────────────────
+    # We need 'dataclass' and 'field' in scope for the generated code.
+    inline_lines: list[str] = []
+    for n in topo_overlapping:
+        cls = merged_classes[n]
+        # Base: the first real base in the merged class's MRO
+        base = cls.__bases__[0]
+        base_name = base.__name__
+        inline_lines.append('')
+        inline_lines.extend(_emit_inline_class(n, cls, base_name))
+
+    # ── __all__ ────────────────────────────────────────────────────────────
+    all_entries = '\n'.join(f"    '{n}'," for n in names)
+    parts_comment = '\n'.join(f'#   {p}' for p in parts)
+
+    has_inline = bool(inline_lines)
+    # Inline @dataclass fields use 'Optional[...]' and 'UUID' in their type
+    # annotations.  Without these imports Pylance resolves them as Any.
+    inline_imports = (
+        'from dataclasses import dataclass, field\n'
+        'from typing import Optional\n'
+        'from uuid import UUID\n'
+        if has_inline else ''
+    )
+
+    return f'''\
+"""Merged profile: {package}
+
+{description}
+
+Parts:
+{parts_comment}
+
+Generated by scripts/build_merged_profiles.py — do not edit by hand.
+Re-run the script after adding or updating sub-profile parts.
+"""
+from __future__ import annotations
+
+{inline_imports}{chr(10).join(import_lines)}
+{''.join(line + chr(10) for line in inline_lines)}
+__all__ = [
+{all_entries}
+]
+'''
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description='Build pre-baked merged profile packages.')
+    parser.add_argument('--config', type=Path, default=_CONFIG,
+                        help=f'releases yaml (default: {_CONFIG.name})')
+    args = parser.parse_args()
+
+    releases = _load_config(args.config)
+    for release in releases:
+        package = release['package']
+
+        print(f'Building {package} ...')
+        content = _build_init(release)
+
+        # Write to the package __init__.py.
+        init_path = _package_dir(package) / '__init__.py'
+        init_path.write_text(content, encoding='utf-8')
+        print(f'  wrote {init_path}')
+
+        # Smoke-test: the written file must be importable and expose __all__.
+        if package in sys.modules:
+            del sys.modules[package]
+        mod = importlib.import_module(package)
+        assert hasattr(mod, '__all__'), f'{package}.__all__ missing after write'
+        print(f'  ok — {len(mod.__all__)} classes exported')
+
+
+if __name__ == '__main__':
+    main()