dot-metrics 0.1.0__tar.gz

dot_metrics-0.1.0/.claude/skills/integrate-dot-metrics/SKILL.md

@@ -0,0 +1,322 @@
+---
+name: integrate-dot-metrics
+description: Integrates the dot-metrics package into an existing project that already has metrics implemented. Use when the user wants to migrate ad-hoc metric functions, quality scores, constraint checks, or evaluation logic to the dot-metrics framework. Covers identifying existing metrics, defining input types, registering metrics and constraints, wiring batch evaluation, and replacing call sites.
+---
+
+# Skill: Integrate dot-metrics into an existing project
+
+## Description
+
+Step-by-step guide to migrate an existing metrics implementation to the `dot-metrics` framework. Covers identifying existing metrics, defining an input type, registering metrics and constraints, wiring batch evaluation, and replacing call sites.
+
+## When to use
+
+- A project already computes metrics (quality scores, constraint checks, KPIs, evaluation functions) with ad-hoc code
+- You want structured, typed, batch-capable metric evaluation via `dot-metrics`
+
+## Prerequisites
+
+- Python 3.12+
+- The project must already have some form of metrics or evaluation logic
+
+---
+
+## Step-by-step integration
+
+### 1. Install dot-metrics
+
+```bash
+pip install dot-metrics
+# or in pyproject.toml dependencies:
+# "dot-metrics>=0.1.0"
+```
+
+### 2. Identify the existing metrics landscape
+
+Scan the codebase for:
+- Functions that compute numeric scores, rates, counts, or pass/fail checks
+- A "compute all metrics" orchestrator function (often returns a list or dict of results)
+- The data types those functions receive as input (the "context" or "input" objects)
+
+Typical patterns you'll find:
+
+```python
+# Pattern A: Functions returning custom Metric objects
+def coverage_score(context, response) -> Metric:
+    value = preserved / total * 100
+    return Metric(name="coverage_score", value=value, type=MetricType.PERCENTAGE, unit="%", ...)
+
+# Pattern B: Functions returning raw floats
+def compute_fulfillment_rate(request, schedule) -> float:
+    return fulfilled / total
+
+# Pattern C: Dict-based definitions dispatched by key
+METRICS = {"coverage": {"fn": compute_coverage, "higher_is_better": True, ...}}
+```
+
+**Inventory what you find.** For each metric, note:
+- Its **key/name** (e.g. `"coverage_score"`)
+- Whether it's a **metric** (continuous value) or a **constraint** (pass/fail against a threshold)
+- Its **unit**, **direction** (`higher_is_better`), and any **threshold** for constraints
+- Whether the function returns **debug info** alongside the value
+
+### 3. Define the metric input type
+
+dot-metrics metric functions take **exactly one argument**. If your existing functions take multiple arguments (e.g. `context` and `response`), bundle them into a single object:
+
+```python
+from dataclasses import dataclass
+
+@dataclass
+class EvalInput:
+    context: Schedule
+    response: RecommendationResponse
+```
+
+Or use a Pydantic model, a TypedDict, a plain dict, a NamedTuple -- anything works. The key is: **one argument in, one number out**.
+
+> **Favour a type that can represent different parametric scenarios.** If you foresee running the same metric set on multiple batches (e.g. different schedules, different parameter configs), a single bundled input type makes `compute_batch()` natural.
+
+### 4. Create the MetricSet
+
+Create a dedicated module (e.g. `metrics/metric_set.py` or `evaluation/metrics.py`):
+
+```python
+from dot_metrics import MetricSet, ComputedValue
+
+metric_set: MetricSet[EvalInput] = MetricSet()
+```
+
+The generic annotation `MetricSet[EvalInput]` is optional but gives you static type checking on all registered functions.
+
+### 5. Register metrics
+
+Translate each existing metric function. Two styles are available:
+
+#### Decorator style (preferred for standalone metric modules)
+
+```python
+@metric_set.metric("coverage_score", unit="%", description="Percentage of initial items preserved")
+def coverage_score(data: EvalInput):
+    """Percentage of initial items preserved in the final output.
+
+    Range: 0-100
+    Interpretation:
+        - 90-100: Excellent retention
+        - <70: Significant loss
+    """
+    initial = len(data.context.items)
+    preserved = sum(1 for u in data.response.updates if u.action != Action.DELETE)
+    return (preserved / initial) * 100 if initial else 100.0
+```
+
+#### Imperative style (preferred when migrating many metrics in bulk)
+
+```python
+metric_set.add(
+    "coverage_score",
+    lambda data: (
+        sum(1 for u in data.response.updates if u.action != Action.DELETE)
+        / len(data.context.items) * 100
+        if data.context.items else 100.0
+    ),
+    unit="%",
+    description="Percentage of initial items preserved",
+)
+```
+
+#### Attaching debug info
+
+If a metric currently returns debug data (e.g. list of violations, IDs of affected items), return a `ComputedValue`:
+
+```python
+@metric_set.metric("scheduling_rate")
+def scheduling_rate(data: EvalInput):
+    scheduled = [e for e in data.response.entries if e.scheduled]
+    unscheduled_ids = [e.id for e in data.response.entries if not e.scheduled]
+    return ComputedValue(
+        value=len(scheduled) / len(data.context.appointments),
+        debug={"unscheduled": unscheduled_ids},
+    )
+```
+
+### 6. Register constraints
+
+Constraints are metrics with a **threshold**. A constraint passes when `value <= threshold`.
+
+```python
+@metric_set.constraint("constraint_violations", threshold=0, unit="violations")
+def constraint_violations(data: EvalInput):
+    """Total count of all constraint violations.
+
+    Range: 0+
+    Interpretation:
+        - 0: All constraints satisfied
+        - >0: Schedule has violations that need attention
+    """
+    return (
+        count_double_bookings(data)
+        + count_availability_violations(data)
+        + count_schedule_violations(data)
+    )
+```
+
+The sub-counts (`count_double_bookings`, etc.) can be helper functions, or they can themselves be registered as separate constraints if you want individual pass/fail tracking:
+
+```python
+@metric_set.constraint("double_bookings", threshold=0, unit="violations")
+def double_bookings(data: EvalInput):
+    overlaps = find_overlapping_slots(data)
+    return ComputedValue(value=len(overlaps), debug={"overlaps": overlaps})
+```
+
+### 7. Replace the orchestrator
+
+**Before** (typical pattern):
+
+```python
+def compute_metrics(context, response) -> list[Metric]:
+    metrics = []
+    metrics.append(coverage_score(context, response))
+    metrics.append(constraint_violations(context, response))
+    metrics.extend(recommendation_counts(context, response))
+    return metrics
+```
+
+**After** (single compute call):
+
+```python
+from myproject.metrics.metric_set import metric_set, EvalInput
+
+def compute_metrics(context, response):
+    return metric_set.compute(EvalInput(context=context, response=response))
+```
+
+The returned `EvalResult` gives you:
+- `result.score("coverage_score")` -- float value
+- `result.metrics["coverage_score"].debug` -- debug dict
+- `result.constraints_ok` -- all constraints passed?
+- `result.violations` -- list of failed constraints
+- `result.assert_constraints()` -- raise if any failed
+
+### 8. Wire up batch evaluation
+
+If your code runs metrics on multiple scenarios (benchmark runs, parameter sweeps, A/B comparisons), use `compute_batch()`:
+
+```python
+# Dict-keyed batch (recommended -- keys become result labels)
+inputs = {
+    "baseline": EvalInput(context=ctx, response=baseline_response),
+    "optimized_v1": EvalInput(context=ctx, response=v1_response),
+    "optimized_v2": EvalInput(context=ctx, response=v2_response),
+}
+batch = metric_set.compute_batch(inputs)
+
+batch["baseline"].score("coverage_score")       # single value
+batch.scores("coverage_score")                  # {"baseline": 95.0, "optimized_v1": 97.2, ...}
+
+# List batch (indexed by position)
+batch = metric_set.compute_batch([input1, input2, input3])
+batch[0].score("coverage_score")
+```
+
+**Even if you only have single-input evaluation today, prefer structuring your code to support batch.** Future visualization features (3D parametric graphs, comparative charts) will consume `BatchResult` natively.
+
+### 9. Update call sites
+
+Find all places that currently call the old orchestrator or access the old metric objects. Common patterns to search for:
+
+```
+# Search for old call sites
+grep -r "compute_metrics" --include="*.py"
+grep -r "\.value" --include="*.py" | grep -i metric
+```
+
+**Before:**
+```python
+metrics = compute_metrics(context, response)
+coverage = next(m for m in metrics if m.name == "coverage_score")
+print(f"Coverage: {coverage.value}%")
+violations = [m for m in metrics if m.name == "constraint_violations"]
+```
+
+**After:**
+```python
+result = compute_metrics(context, response)
+print(f"Coverage: {result.score('coverage_score')}%")
+print(f"Constraints OK: {result.constraints_ok}")
+for v in result.violations:
+    print(f"  FAILED: {v.key} = {v.value} (threshold: {v.threshold})")
+```
+
+### 10. Terminal visualization
+
+```python
+from dot_metrics import draw_terminal_chart
+
+result = metric_set.compute(data)
+print(draw_terminal_chart(result))
+```
+
+For metrics where lower is better (e.g. latency, violation counts), set `higher_is_better=False` during registration. For absolute metrics (unbounded values like time-in-ms), add `metadata={"absolute": True}` -- the chart will normalize them differently.
+
+### 11. Clean up
+
+Once all call sites are migrated:
+
+- Delete the old `Metric` / `MetricType` model classes (if they were project-specific)
+- Delete the old orchestrator function
+- Delete any manual normalization / chart code that dot-metrics now handles
+- Remove metric-related utility functions that are now inlined in metric registrations
+
+---
+
+## File organization recommendations
+
+```
+your_project/
+  metrics/
+    __init__.py          # exports metric_set and EvalInput
+    metric_set.py        # MetricSet definition + EvalInput dataclass
+    quality.py           # @metric_set.metric / .constraint registrations
+    performance.py       # more registrations
+```
+
+Keep all registrations as module-level decorators. They execute at import time, so importing the module registers the metrics. Your `__init__.py` just needs:
+
+```python
+from your_project.metrics.metric_set import metric_set, EvalInput
+
+# Import modules to trigger registration
+import your_project.metrics.quality
+import your_project.metrics.performance
+```
+
+---
+
+## Migration checklist
+
+- [ ] `dot-metrics` added to dependencies
+- [ ] Input type defined (single object bundling all metric inputs)
+- [ ] `MetricSet` created with generic annotation
+- [ ] All metrics registered (decorator or imperative)
+- [ ] All constraints registered with thresholds
+- [ ] Debug info attached via `ComputedValue` where needed
+- [ ] Docstrings added for metric documentation (Range, Interpretation, Notes)
+- [ ] Orchestrator replaced with `metric_set.compute()`
+- [ ] Batch evaluation wired where applicable
+- [ ] Call sites updated to use `EvalResult` API
+- [ ] Old metric models and utilities removed
+- [ ] `higher_is_better` and `metadata={"absolute": True}` set correctly for visualization
+
+---
+
+## Common gotchas
+
+**Metric functions must take exactly one argument.** If your existing functions take `(context, response)`, you must bundle them. Don't try to use `functools.partial` or closures to work around this -- define a proper input type.
+
+**Constraints pass when `value <= threshold`.** This is a "lower is better" model for violations. If you have a constraint like "availability must be >= 95%", invert it: register the shortfall (`100 - availability`) with `threshold=5`.
+
+**Don't duplicate computation.** If multiple metrics share expensive computation (e.g. building a conflict graph), extract it into a helper that takes the input type and cache or precompute as needed. The metric functions can then call the shared helper.
+
+**Batch inputs must all be the same type.** `compute_batch()` runs the same metric set on each input. If you need different metric sets for different scenarios, use separate `MetricSet` instances.

dot_metrics-0.1.0/.gitignore

@@ -0,0 +1,23 @@
+# Python > Build
+*.py[cod]
+*.egg-info
+**/__pycache__/
+
+# Python > Cache
+.pytest_cache/
+
+# Python > Coverage
+.coverage
+coverage_html_report/
+
+# Environment
+.venv/
+
+# Ide
+.vscode/
+
+# Local files
+local/
+
+# Build artifacts
+dist/

dot_metrics-0.1.0/LICENSE

@@ -0,0 +1 @@
+TODO: TO BE COMPLETED

dot_metrics-0.1.0/PKG-INFO

@@ -0,0 +1,312 @@
+Metadata-Version: 2.4
+Name: dot-metrics
+Version: 0.1.0
+Summary: A lightweight metrics and constraint evaluation framework
+Project-URL: Homepage, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-metrics
+Project-URL: Repository, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-metrics
+Project-URL: Issues, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-metrics/-/issues
+Author-email: deepika Team <contact@deepika.ai>
+License: TODO: TO BE COMPLETED
+License-File: LICENSE
+Keywords: constraints,deepika,evaluation,metrics,open-toolbox
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.12
+Requires-Dist: pydantic>=2.0
+Description-Content-Type: text/markdown
+
+# dot-metrics
+
+![Python Version](https://img.shields.io/badge/python-3.12%2B-blue)
+![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen)
+
+**dot-metrics** is a lightweight metrics and constraint evaluation framework. Define metrics and constraints, run them against your data, and get structured results with debug info.
+
+## Install
+
+```bash
+pip install dot-metrics
+```
+
+## Concept
+
+A `MetricSet` holds metric and constraint definitions. Call `compute(data)` to evaluate them.
+
+```
+MetricSet
+ ├── metrics:     {"coverage": MetricDefinition}
+ └── constraints: {"errors":   ConstraintDefinition}
+          │
+          ▼
+     set.compute(data)
+          │
+          ▼
+      EvalResult
+       ├── metrics:     {"coverage": Metric}
+       └── constraints: {"errors":   Constraint}
+```
+
+**Metrics** are continuous measurements (e.g. coverage rate, score).
+**Constraints** are pass/fail checks against a threshold (e.g. error count ≤ 0).
+
+A constraint passes when `value <= threshold`.
+
+## Quick start
+
+```python
+from dot_metrics import MetricSet
+
+metric_set = MetricSet()
+
+@metric_set.metric("coverage")
+def coverage(data):
+    return data["covered"] / data["total"]
+
+@metric_set.constraint("errors", threshold=0)
+def errors(data):
+    return data["error_count"]
+
+result = metric_set.compute({"covered": 90, "total": 100, "error_count": 0})
+
+result.score("coverage")     # 0.9
+result.constraints_ok        # True
+```
+
+## Defining metrics and constraints
+
+### Decorator style
+
+```python
+metric_set = MetricSet()
+
+@metric_set.metric("latency_ms", unit="ms", higher_is_better=False)
+def latency(data):
+    return data["total_ms"] / data["requests"]
+
+@metric_set.constraint("error_rate", threshold=0.01, unit="%")
+def error_rate(data):
+    return data["errors"] / data["requests"]
+```
+
+### Imperative style
+
+```python
+metric_set = MetricSet()
+metric_set.add("coverage", lambda data: data["covered"] / data["total"])
+metric_set.add_constraint("errors", lambda data: data["error_count"], threshold=0)
+```
+
+Both styles are equivalent. `add()` and `add_constraint()` accept the same keyword arguments as the decorators.
+
+### Parameters
+
+**`metric_set.metric(key, *, unit="", description="", higher_is_better=True, metadata={})`**
+**`metric_set.add(key, fn, *, unit="", description="", higher_is_better=True, metadata={})`**
+
+**`metric_set.constraint(key, *, threshold, unit="", description="", metadata={})`**
+**`metric_set.add_constraint(key, fn, *, threshold, unit="", description="", metadata={})`**
+
+- `key` — unique name for the metric/constraint
+- `threshold` — constraint passes when `value <= threshold`
+- `higher_is_better` — affects terminal chart rendering
+- `metadata` — arbitrary dict, passed through to results
+
+## Computing results
+
+```python
+result = metric_set.compute(data)
+```
+
+Every metric and constraint function must accept **exactly one argument** — the data object. `data` can be anything: a dict, dataclass, Pydantic model, etc.
+
+### Accessing results
+
+```python
+result.score("coverage")                  # float — metric value
+result.metrics["coverage"].value          # same
+result.metrics["coverage"].unit           # ""
+result.metrics["coverage"].debug          # {} by default
+
+result.constraints["errors"].value        # float
+result.constraints["errors"].passed       # True/False
+result.constraints["errors"].threshold    # 0
+
+result.constraints_ok                     # True if all constraints passed
+result.violations                         # list of failed Constraint objects
+result.assert_constraints()               # raises ValueError if any failed
+```
+
+## Attaching debug info
+
+Return a `ComputedValue` instead of a plain float to attach structured debug data:
+
+```python
+from dot_metrics import MetricSet, ComputedValue
+
+metric_set = MetricSet()
+
+@metric_set.metric("coverage")
+def coverage(data):
+    missed = [x for x in data if not x["covered"]]
+    return ComputedValue(value=1 - len(missed) / len(data), debug={"missed": missed})
+
+result = metric_set.compute(data)
+result.metrics["coverage"].debug    # {"missed": [...]}
+```
+
+`ComputedValue` works the same way for constraints.
+
+## Batch evaluation
+
+Evaluate a set of inputs in one call:
+
+```python
+# dict of inputs
+batch = metric_set.compute_batch({"run_1": data1, "run_2": data2})
+batch["run_1"].score("coverage")            # 0.9
+batch.scores("coverage")                   # {"run_1": 0.9, "run_2": 0.85}
+
+# list of inputs
+batch = metric_set.compute_batch([data1, data2, data3])
+batch[0].score("coverage")                 # indexed by position
+```
+
+`BatchResult` supports iteration, `len()`, and `.items()`.
+
+## Metric documentation
+
+Add a Google-style docstring to a metric or constraint function and it gets parsed into a `help` dict on both the definition and the result:
+
+```python
+@metric_set.metric("coverage", unit="%")
+def coverage(data):
+    """Percentage of code paths covered by tests.
+
+    Range: 0-100
+    Interpretation:
+        - 90-100: Excellent
+        - 70-90:  Good
+        - <70:    Needs improvement
+    Notes:
+        - Returns 0 for empty input.
+    """
+    return sum(1 for x in data if x["covered"]) / len(data)
+
+metric_set.metrics["coverage"].help
+# {
+#   "summary": "Percentage of code paths covered by tests.",
+#   "range": "0-100",
+#   "interpretation": "- 90-100: Excellent\n- 70-90:  Good\n- <70:    Needs improvement",
+#   "notes": "- Returns 0 for empty input."
+# }
+
+result = metric_set.compute(data)
+result.metrics["coverage"].help     # same dict
+```
+
+Supported sections: `Range:`, `Interpretation:`, `Notes:`. No docstring → `help` is `{}`.
+
+## Typing
+
+`MetricSet` is generic over the input type `T`. Annotating it lets static type checkers (mypy, pyright) verify that every registered function accepts the right type:
+
+```python
+from dataclasses import dataclass
+from dot_metrics import MetricSet
+
+@dataclass
+class SchedulingData:
+    appointments: list
+    solution: list
+
+metric_set: MetricSet[SchedulingData] = MetricSet()
+metric_set.add("rate", lambda d: len(d.solution) / len(d.appointments))
+
+result = metric_set.compute(SchedulingData(appointments=[...], solution=[...]))
+```
+
+The annotation is optional — omitting it is fine and everything still works at runtime.
+
+## Terminal chart
+
+```python
+from dot_metrics import draw_terminal_chart
+
+print(draw_terminal_chart(result))
+# coverage  ████████████████████  0.90
+```
+
+`draw_terminal_chart(result, width=40, char="█")` returns a string — use `print()` to display it.
+
+## Full example
+
+```python
+from dot_metrics import MetricSet, ComputedValue
+
+appointments = [
+    {"id": "A1", "patient": "Alice",   "duration": 30},
+    {"id": "A2", "patient": "Bob",     "duration": 60},
+    {"id": "A3", "patient": "Charlie", "duration": 30},
+]
+
+solution = [
+    {"appointment_id": "A1", "practitioner": "Dr. Martin", "slot": "09:00", "scheduled": True},
+    {"appointment_id": "A2", "practitioner": "Dr. Martin", "slot": "09:00", "scheduled": True},  # conflict!
+    {"appointment_id": "A3", "practitioner": "Dr. Martin", "slot": "10:00", "scheduled": True},
+]
+
+metric_set = MetricSet()
+
+@metric_set.metric("scheduling_rate")
+def scheduling_rate(data):
+    scheduled = [e for e in data["solution"] if e["scheduled"]]
+    unscheduled = [e["appointment_id"] for e in data["solution"] if not e["scheduled"]]
+    return ComputedValue(value=len(scheduled) / len(data["appointments"]), debug={"unscheduled": unscheduled})
+
+@metric_set.constraint("conflicts", threshold=0)
+def count_conflicts(data):
+    seen = {}
+    conflicts = []
+    for entry in data["solution"]:
+        key = (entry["practitioner"], entry["slot"])
+        if key in seen:
+            conflicts.append((seen[key], entry["appointment_id"]))
+        seen[key] = entry["appointment_id"]
+    return ComputedValue(value=len(conflicts), debug={"conflicts": conflicts})
+
+result = metric_set.compute({"appointments": appointments, "solution": solution})
+
+result.score("scheduling_rate")                     # 1.0
+result.constraints_ok                               # False
+result.constraints["conflicts"].debug               # {"conflicts": [("A1", "A2")]}
+```
+
+## Reference
+
+| Import | Description |
+|--------|-------------|
+| `MetricSet` | Main class — holds definitions, runs computation |
+| `EvalResult` | Output of `compute()` — holds `Metric` and `Constraint` dicts |
+| `BatchResult` | Output of `compute_batch()` — maps keys to `EvalResult` |
+| `ComputedValue` | Wraps a float return value with optional debug data |
+| `Metric` | Computed metric result |
+| `Constraint` | Computed constraint result with `passed` flag |
+| `MetricDefinition` | Stored metric definition (in `metric_set.metrics`) |
+| `ConstraintDefinition` | Stored constraint definition (in `metric_set.constraints`) |
+| `draw_terminal_chart` | Renders a Unicode bar chart from an `EvalResult` |
+
+## Contributing & Development
+
+See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) and [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
+
+## License
+
+See [LICENSE](LICENSE) for details.
+
+## Contact
+
+deepika Team — contact@deepika.ai
+Project: [gitlab.com/deepika6190303/deepika-open-toolbox/dot-metrics](https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-metrics)