cmpnd 0.1.0__tar.gz → 0.3.0__tar.gz

cmpnd-0.3.0/.gitignore

@@ -0,0 +1,43 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.env
+.venv
+env/
+venv/
+ENV/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+.nox/
+
+# Misc
+*.log
+.DS_Store

cmpnd-0.3.0/CLAUDE.md

@@ -0,0 +1,116 @@
+# Claude Code Instructions
+
+Part of the CMPND monorepo. See `../CLAUDE.md` for cross-cutting rules (hash architecture, null handling, testing philosophy).
+
+## Documentation discipline (docs are as-built, updated in the same commit)
+
+This mirrors the practice that has kept `serve/docs/` in sync with the Go
+code while the top-level `docs/` rotted: **the docs are part of the diff.**
+
+- This component's docs live in `python-sdk/docs/`. They describe what the
+  SDK does **today**, not what it should do.
+- A change that adds or removes a subsystem, swaps a seam (the callback, the
+  exporter transport, the IR/hash, the optimizer tracker), or changes the wire
+  shape the SDK emits **updates the matching doc in the same commit.** If a
+  change leaves a doc describing the old behavior, the change isn't done.
+- The `cmpnd-*` skills under `.claude/skills/` point at these docs rather than
+  duplicating them — keep the doc authoritative and let the skill be the thin
+  trigger. (Anchor paths are checked by `bin/check-skill-anchors`.)
+- `docs/plans/*` is append-only design history and is exempt — don't rewrite it.
+
+## Running Tests & Checks
+
+```bash
+# uv provides the interpreter (managed Python); no venv activate needed.
+uv run python -m pytest tests/           # all tests (excluding e2e)
+uv run python -m pytest tests/ -k "not e2e"  # skip e2e explicitly
+uv run ruff check .                      # lint
+uv run ruff format --check .             # format check
+```
+
+## Key Files
+
+- `cmpnd/identity.py` - Unified hash computations (xxhash, SHA256)
+- `cmpnd/ir/` - Structural program-identity IR (algebra, encoder, hash, reflector)
+- `cmpnd/callback.py` - DSPy instrumentation, signature extraction
+- `cmpnd/optimizers/tracker.py` - Optimization tracking
+- `cmpnd/packaging.py` - Source ZIP generation for deploy (ported from deploy/local_cmpnd.py)
+- `cmpnd/deployment.py` - `deploy()` function: SDK -> Rails -> AWS deploy stack
+
+## Reflector ↔ DSPy compatibility
+
+`cmpnd/ir/reflection.py` reads internal attributes off live `dspy.Module`
+instances (`m.signature`, `m.predict.signature`, `m.tools`, `m.module`,
+`m.trainset`, …). A DSPy release that renames or removes any of those
+breaks every customer's traces and optimization runs with an unhandled
+`AttributeError`.
+
+`tests/test_reflector_smoke.py` is the contract. Its `KIND_FIXTURES`
+table declares one fixture per IR kind (the DSPy class to construct,
+the attributes the reflector reads, the expected IR shape). Two
+meta-tests fail loudly when this table drifts out of sync:
+
+- new `Module` kind in `cmpnd/ir/program.py` without a fixture
+- new `isinstance(m, dspy.X)` or `_opt("X")` branch in
+  `cmpnd/ir/reflection.py` without a fixture
+
+**When you change the reflector or add an IR kind, update
+`KIND_FIXTURES` in the same change.** When you bump DSPy, run
+`pytest tests/test_reflector_smoke.py` first — failures point at the
+exact attribute or class that moved.
+## CLI has two front-ends — kept in sync by `cli/verbs.py`
+
+`cmpnd <subcommand>` (argparse, one-shot) and bare `cmpnd` (interactive
+Python shell, `cmpnd/cli/shell.py`) drive the **same** handler functions
+in `cmpnd/cli/commands/`. The dual binding for every cross-cutting verb
+(ps, deploy, status, logs, inspect, run) is declared **once** in
+`cmpnd/cli/verbs.py`. Both front-ends consume that list:
+
+- `top.register()` resolves `set_defaults(func=...)` through `verbs.by_name`.
+- `Deployment` in `shell.py` is decorated with `@verbs.assert_entity_methods`,
+  which fails at import time if a verb's method is missing.
+- `tests/test_cli/test_parity.py` enumerates the actual argparse choices,
+  REPL namespace, and entity methods and cross-checks them against VERBS.
+  Adding a CLI verb without a shell binding (or vice versa) fails CI.
+
+**To add a new dual-front-end verb**: add a `Verb(...)` row to
+`cli/verbs.py`, add the `add_parser` call in `top.register()`, and add
+the method on the entity (or function in `_make_namespace`). All three
+must agree; the parity test asserts so.
+
+When you touch a handler:
+
+- **User-facing hints must use `_hint_*` helpers** in `cli/commands/top.py`
+  (`_hint_ps`, `_hint_run`, `_hint_status`, `_hint_inspect`,
+  `_hint_traces_search`). They branch on `_SHELL_MODE` to emit
+  `deps.d_xxx.run({...})` inside the shell vs `cmpnd run d_xxx '{...}'`
+  outside. Never hardcode `cmpnd <verb>` strings in handler output.
+
+- **New shell-addressable entity**: subclass `Entity` in `cli/shell.py`,
+  add a `_<entity>_key` function returning a Python-safe attribute name
+  (UUID-leading entities need a non-digit prefix like `d_` / `t_`), and
+  expose it via `Registry(...)` in `_make_namespace` (top-level) or as a
+  `@property` on a parent entity (scoped, see `Deployment.traces`).
+
+- **Table-as-repr**: if you want typing the registry at the REPL to
+  render the same table as the corresponding CLI `ps`-style command,
+  give the Entity subclass a `_render_table(cls, entities) -> str`
+  classmethod. Reuse the CLI's renderer by extracting it from the
+  handler (see `top.render_deployments_table`) so the two front-ends
+  produce byte-identical tables.
+
+- **Attribute-name scheme has one home**: `top._attr_name` (deployments,
+  `d_<8hex>`) and `top._trace_attr_name` (traces, `t_<8hex>`) are the
+  canonical sources. `shell._deployment_key` / `shell._trace_key` are
+  thin record-adapters over them, and the CLI's table renderers swap
+  the id column to the attribute form when `_SHELL_MODE` is on, so what
+  the user reads is exactly what they tab-complete.
+
+- **Test isolation**: `shell.launch()` flips the module-level
+  `top._SHELL_MODE`. The autouse `_reset_shell_mode` fixture in
+  `tests/test_cli/test_shell.py` restores it; any new test that mutates
+  `_SHELL_MODE` outside that file must restore it itself.
+
+## Release
+
+Tag-free. Bump `[project] version` in `pyproject.toml` (`uv version --bump {minor,patch}` or by hand), open a PR, merge to `main`. A change to that version on `main` triggers `.github/workflows/release-sdk.yml`, which publishes to PyPI only when the version is new (build → TestPyPI → PyPI); a merge that doesn't bump the version no-ops. Runtime `cmpnd.__version__` reads installed metadata. Full flow: [`docs/releasing.md`](../docs/releasing.md) Part 3 and `docs/plans/2026-07-02-sdk-pypi-tag-free-publish.md`.

cmpnd-0.3.0/PKG-INFO

@@ -0,0 +1,372 @@
+Metadata-Version: 2.4
+Name: cmpnd
+Version: 0.3.0
+Summary: DSPy observability and deployment SDK for cmpnd
+Project-URL: Homepage, https://cmpnd.ai
+Author-email: cmpnd <hello@cmpnd.ai>
+License: MIT
+Keywords: dspy,llm,observability,tracing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Requires-Dist: dspy>=3.1.3
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: ipython>=8.20
+Requires-Dist: pydantic>=2.0
+Requires-Dist: xxhash>=3.0.0
+Provides-Extra: dev
+Requires-Dist: basedpyright>=1.39; extra == 'dev'
+Requires-Dist: hypothesis>=6.100; extra == 'dev'
+Requires-Dist: jsonschema>=4.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest-rerunfailures>=14.0; extra == 'dev'
+Requires-Dist: pytest-xdist>=3.0; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.4.0; extra == 'dev'
+Provides-Extra: stub
+Requires-Dist: cmpnd-deploy; (python_version >= '3.11') and extra == 'stub'
+Requires-Dist: fastapi>=0.110; extra == 'stub'
+Requires-Dist: python-multipart>=0.0.9; extra == 'stub'
+Requires-Dist: uvicorn[standard]>=0.27; extra == 'stub'
+Description-Content-Type: text/markdown
+
+# cmpnd
+
+DSPy observability and deployment SDK for cmpnd.
+
+Automatically trace your DSPy programs with one line of code, and deploy them with one more.
+
+## Installation
+
+```bash
+uv add cmpnd      # or: pip install cmpnd
+```
+
+## Quick Start
+
+```python
+import cmpnd
+
+# Configure with your API key
+cmpnd.configure(api_key="ck_xxx", project="my-project")
+
+# Enable automatic DSPy instrumentation
+cmpnd.auto_instrument()
+
+# Your DSPy code is now automatically traced!
+import dspy
+
+lm = dspy.LM("openai/gpt-4o-mini")
+dspy.configure(lm=lm)
+
+cot = dspy.ChainOfThought("question -> answer")
+result = cot(question="What is DSPy?")
+# Trace is automatically sent to cmpnd
+
+# Correlate the result with its platform trace (e.g. to attach feedback later):
+cmpnd.trace_id(result)  # "a1b2c3d4-..."  (None if not produced under instrumentation)
+```
+
+## Configuration
+
+### Using environment variables
+
+```bash
+export CMPND_API_KEY="ck_your_api_key"
+export CMPND_ENDPOINT="https://platform.cmpnd.ai"  # optional
+```
+
+```python
+import cmpnd
+
+cmpnd.configure()  # Reads from environment
+cmpnd.auto_instrument()
+```
+
+### Configuration options
+
+```python
+cmpnd.configure(
+    api_key="ck_xxx",                # Required: API key
+    endpoint="https://platform.cmpnd.ai", # Optional: Backend URL
+    project="my-project",            # Optional: Project name
+    batch_size=100,                  # Optional: Batch size for export
+    flush_interval_seconds=5.0,      # Optional: Flush interval
+    capture_inputs=True,             # Optional: Capture function inputs
+    capture_outputs=True,            # Optional: Capture function outputs
+    default_tags={"env": "prod"},    # Optional: Tags for all traces
+)
+```
+
+## Deployment
+
+Deploy a DSPy module to the cmpnd platform for server-side execution:
+
+```python
+import cmpnd
+import dspy
+
+cmpnd.configure(api_key="ck_xxx")
+
+lm = dspy.LM("groq/llama-3.1-8b-instant", temperature=0.0, max_tokens=16384)
+dspy.configure(lm=lm)
+
+module = dspy.Predict("question -> answer")
+deployed = cmpnd.deploy(module)
+
+# The returned program is callable — run it server-side like the local module:
+result = deployed(question="What is DSPy?")
+print(result.answer)
+```
+
+The SDK packages the module's source code into a ZIP, uploads it to the platform, and polls until the server-side parser finishes. Returns a `DeployedProgram` you can **call directly** to execute server-side; `str(deployed)` is its `deployment_id` and `deployed.deployment_id` exposes it explicitly.
+
+### With a metric (for optimization)
+
+```python
+def accuracy(example, pred, trace=None):
+    return example.answer == pred.answer
+
+deployment_id = cmpnd.deploy(module, metric=accuracy)
+```
+
+### Options
+
+```python
+cmpnd.deploy(
+    module,              # Required: a DSPy module
+    metric=None,         # Optional: metric callable for optimization support
+    timeout=300,         # Optional: seconds to wait for parsing (default 300)
+)
+```
+
+### How it works
+
+1. The SDK generates source code from the live module (LM config, signature, types)
+2. `POST /api/v1/deployments` creates a deployment and returns a presigned S3 upload URL
+3. The source ZIP is uploaded directly to S3
+4. The server-side parser extracts the program and LM configuration
+5. The SDK polls until status is `ready`, then returns the `DeployedProgram`
+
+The deployed program's LM model string (e.g. `groq/llama-3.1-8b-instant`) and configuration (temperature, max_tokens) are extracted automatically from the source.
+
+## Optimization
+
+Optimize a DSPy program through the platform. `cmpnd.optimize()` mirrors `cmpnd.deploy()`: **it returns a `DeployedProgram` you call directly** — it just also tells you the program's `score`.
+
+```python
+import cmpnd
+import dspy
+
+cmpnd.configure(api_key="ck_xxx")
+dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
+
+program = dspy.ChainOfThought("topic -> haiku")
+
+# gepa needs a 5-arg feedback metric returning a dspy.Prediction(score=, feedback=)
+def metric(example, pred, trace=None, pred_name=None, pred_trace=None):
+    ok = example.season.lower() not in pred.haiku.lower()
+    return dspy.Prediction(score=float(ok),
+                           feedback=None if ok else "Don't name the season.")
+
+best = cmpnd.optimize(
+    program,
+    optimizer="gepa",                # or "bootstrap_fewshot", "random_search"
+    trainset=train,
+    valset=val,
+    metric=metric,
+    reflection_model="groq/llama-3.3-70b-versatile",   # gepa only
+)
+
+print(best.score)                       # best score found
+print(best(topic="late autumn").haiku)  # call it directly — same as deploy()
+```
+
+The metric's arity must match the optimizer (checked before any network call): **gepa** needs a 5-arg feedback metric, while **bootstrap_fewshot** / **random_search** need a `<=3`-arg scalar metric.
+
+### Reusing an existing deployment
+
+Skip the initial deploy by passing the `deployment_id` of a ready deployment as the starting program:
+
+```python
+best = cmpnd.optimize(program, optimizer="bootstrap_fewshot",
+                      trainset=train, valset=val, metric=metric,
+                      deployment_id=deployed.deployment_id)
+```
+
+### Options
+
+```python
+cmpnd.optimize(
+    program,                 # Required: the DSPy module to optimize
+    optimizer="gepa",        # Required: "gepa" | "bootstrap_fewshot" | "random_search"
+    trainset=train,          # Required: training examples
+    valset=val,              # Required: validation examples
+    metric=metric,           # Required: the evaluation metric (arity-checked)
+    config=None,             # Optional: optimizer-specific config (allow-listed server-side)
+    deployment_id=None,      # Optional: reuse a ready deployment as the starting program
+    reflection_model=None,   # Optional (gepa): litellm model for the reflection LM
+    max_steps=10,            # Optional: maximum optimize steps
+    patience=2,              # Optional: stop after N consecutive non-improving steps
+    seed=0,                  # Optional: starting seed; advanced each step
+)
+```
+
+On failure, `optimize()` raises `cmpnd.OptimizeError` (a 4xx initiating a step — e.g. 409 no-data / mismatch) or `cmpnd.OptimizeFailed` (a step that transitioned to `failed`).
+
+### From the CLI
+
+```bash
+cmpnd optimize gepa task.py --reflection-model groq/llama-3.3-70b-versatile
+```
+
+The snippet (a `.py` file or `-` for stdin) binds `program`, `trainset`, `valset`, and `metric`; the CLI prints the best deployment id.
+
+## Custom Spans
+
+Add custom spans to trace non-DSPy code:
+
+### Using the decorator
+
+```python
+from cmpnd import trace, SpanType
+
+@trace(name="fetch_documents", span_type=SpanType.RETRIEVE)
+def fetch_documents(query: str) -> list[str]:
+    # Your retrieval logic
+    return documents
+```
+
+### Using the context manager
+
+```python
+from cmpnd import start_span, SpanType
+
+def run_pipeline(query: str):
+    with start_span("vector_search", span_type=SpanType.RETRIEVE) as span:
+        span.set_attribute("index", "my-faiss-index")
+        docs = search(query)
+        span.set_outputs({"doc_count": len(docs)})
+
+    return generate(query, docs)
+```
+
+## What Gets Traced
+
+The SDK automatically captures:
+
+### Module Execution
+- Module type (Predict, ChainOfThought, ReAct, etc.)
+- Signature name and instructions
+- Input/output field names
+- Demo count
+
+### LM Calls
+- Model name and provider
+- Token usage (prompt, completion, total)
+- Request/response content
+
+### Adapters
+- Format and parse operations
+- Input/output transformations
+
+### Tools
+- Tool name and description
+- Invocation inputs/outputs
+
+### Evaluations
+- Evaluation scores
+- Program being evaluated
+
+## Span Types
+
+Available span types for categorization:
+
+- `SpanType.MODULE` - Generic DSPy module
+- `SpanType.PREDICT` - Predict module
+- `SpanType.CHAIN_OF_THOUGHT` - ChainOfThought module
+- `SpanType.REACT` - ReAct agent
+- `SpanType.RETRIEVE` - Retrieval operations
+- `SpanType.LM_CALL` - Language model calls
+- `SpanType.ADAPTER_FORMAT` - Adapter formatting
+- `SpanType.ADAPTER_PARSE` - Adapter parsing
+- `SpanType.TOOL` - Tool invocations
+- `SpanType.EVALUATION` - Evaluation runs
+
+## API Reference
+
+### `cmpnd.configure()`
+
+Initialize the SDK with your API key and options.
+
+### `cmpnd.auto_instrument()`
+
+Automatically register the callback with DSPy.
+
+### `cmpnd.CmpndCallback`
+
+The callback class for manual registration:
+
+```python
+import dspy
+import cmpnd
+
+cmpnd.configure(api_key="ck_xxx")
+dspy.configure(callbacks=[cmpnd.CmpndCallback()])
+```
+
+### `cmpnd.deploy()`
+
+Deploy a DSPy module to the cmpnd platform. Returns a callable `DeployedProgram`.
+
+### `cmpnd.optimize()`
+
+Optimize a DSPy program through the platform. Returns the best `DeployedProgram` (call it directly; `.score` is the best score found). Raises `cmpnd.OptimizeError` / `cmpnd.OptimizeFailed` on failure.
+
+### `cmpnd.execute()`
+
+Execute a deployed program by `deployment_id`, returning a `dspy.Prediction`. Usually you just call a `DeployedProgram` instead, which dispatches here.
+
+### `cmpnd.DeployedProgram`
+
+A deployed, callable program returned by `cmpnd.deploy()` and `cmpnd.optimize()`. Call it with the original module's input fields; `str()` and `.deployment_id` give its id, and `.score` is its score when it came from `optimize()` (else `None`).
+
+### `cmpnd.trace()`
+
+Decorator for custom traced functions.
+
+### `cmpnd.start_span()`
+
+Context manager for custom spans.
+
+### `cmpnd.get_current_trace()`
+
+Get the current trace (if any).
+
+### `cmpnd.get_current_span()`
+
+Get the current span (if any).
+
+### `cmpnd.trace_id()`
+
+Return the platform trace id stamped on a prediction returned by an instrumented program, or `None` if it wasn't produced under instrumentation. Use it to correlate a prediction with its trace, e.g. to attach feedback via `POST /api/v1/traces/{cmpnd.trace_id(pred)}/metadata`.
+
+### `cmpnd.flush_exporter()`
+
+Force-publish everything the background exporter has queued, keeping it running. Use this when a mid-run reader needs a just-produced trace to exist server-side before fetching it — unlike `shutdown_exporter()`, the exporter stays alive so later spans still publish. Returns `True` if the drain completed within the timeout, `False` if it timed out or no exporter is running — check it before fetching so you don't read a trace that hasn't published yet.
+
+### `cmpnd.shutdown_exporter()`
+
+Gracefully shutdown the background exporter.
+
+## License
+
+MIT