traigent 0.21.3.dev0__tar.gz → 0.22.0__tar.gz

{traigent-0.21.3.dev0 → traigent-0.22.0}/CHANGELOG.md

@@ -6,6 +6,55 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+
+- `ObservabilityClient.flush(timeout=0)` and `close(timeout=0)` now perform an
+  immediate poll rather than treating zero as the configured 30-second flush
+  timeout. No deadline-exceeded warning is emitted for this intentional
+  poll-style use. No-argument flush and close retain synchronous delivery.
+- `items_pending` and transport `pending_items` now count both buffered and
+  in-flight payloads, so a timed-out sender remains visible until it completes.
+  `get_stats()` also exposes `oldest_inflight_age_seconds` to diagnose custom
+  senders that do not return.
+
+### Fixed
+
+- Observability delivery keeps exactly one sender active, including after a
+  bounded flush returns before its sender completes. A later flush waits for
+  that sender within its own deadline before beginning another batch.
+- Health callbacks run after transport locks are released, and batch-delivery
+  failure events explicitly identify their batch (`item_count` and capped
+  `trace_ids`) instead of implying one callback per payload.
+- The atexit observability close is explicitly bounded by the default
+  `flush_timeout` (30 seconds). This intentionally favors a bounded interpreter
+  shutdown over guaranteed delivery of a very slow final backlog.
+
+## [0.21.3] - 2026-07-11
+
+### Added
+
+- Additive SmartOps Planner V2 client and `traigent guidance` CLI with strict
+  treatment/profile provenance, opaque decision execution, server-integrity-tagged structured
+  resolution, explicit idempotent receipt submission, and lifecycle reopen
+  commands.
+  The existing `traigent next-steps` v1 surface remains unchanged.
+
+### Fixed
+
+- Correct the registered observability issue-list MCP wrapper so it matches the
+  six-argument helper contract instead of raising a runtime ``TypeError``.
+- Enforce Planner V2 treatment isolation even outside strict experiment mode;
+  strict mode additionally rejects fallback and incomplete context.
+- Validate exact action economics, implementation hash, attempt-bound argv,
+  receipt scope, and live equal execution/lease expiries before presenting a
+  resolved operation. The server integrity tag remains opaque to public clients.
+- Describe rule parity and certified session-utility advantage without implying
+  policy agreement or a product KPI guarantee, and validate reopen/receipt
+  lifecycle transitions fail-closed.
+
+- Declare `tenacity` as a dependency so litellm's retry helpers
+  (`completion_with_retries`) work on a clean install (#1824).
+
 ## [0.21.0] - 2026-07-07
 
 Observability hardening release. Enterprise-grade privacy posture for the

{traigent-0.21.3.dev0/traigent.egg-info → traigent-0.22.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: traigent
-Version: 0.21.3.dev0
+Version: 0.22.0
 Summary: Enterprise-grade LLM optimization platform with advanced analytics and AI-powered insights
 Author-email: Traigent Team <opensource@traigent.ai>
 License-Expression: AGPL-3.0-only OR LicenseRef-Traigent-Commercial
@@ -37,6 +37,7 @@ Requires-Dist: cryptography>=46.0.7
 Requires-Dist: PyJWT[crypto]<3.0.0,>=2.13.0
 Requires-Dist: backoff>=2.2.0
 Requires-Dist: litellm<2,>=1.87.1
+Requires-Dist: tenacity>=8.1.0
 Requires-Dist: rank-bm25
 Requires-Dist: psutil>=5.8.0
 Requires-Dist: pydantic>=2.0.0

{traigent-0.21.3.dev0 → traigent-0.22.0}/docs/api-reference/complete-function-specification.md

@@ -529,6 +529,38 @@ class OptimizationResult:
 `metadata["source"]` records result provenance as one of `cloud_brain`,
 `local_fallback`, `explicit_local`, or `offline`.
 
+#### `best_config` in multi-objective runs (authoritative accessor)
+
+When you declare more than one objective — e.g. `objectives=["accuracy", "cost"]`,
+**including the default equal weights** — `results.best_config` is the winner of
+the declared **weighted aggregate**, not the highest-accuracy config. This is the
+same winner the run's own post-hoc analysis crowns:
+
+- `results.best_config` **==** `results.calculate_weighted_scores()["best_weighted_config"]`
+  **==** the `best_weighted_config` in the persisted `weighted_results_v2.json`.
+
+The two artifacts of one run agree — **including on ties**: both selectors gather
+every config within the conservative weighted-score tolerance of the maximum and
+break the tie identically (secondary metrics in the declared objective order, then
+`trial_id`), so a two-endpoint Pareto tie (both configs sharing the top weighted
+aggregate) crowns the same winner in both places rather than opposite first-by-order
+picks. `results.best_score` remains the winner's primary-objective value (e.g. its
+accuracy); the weighted basis of the selection is surfaced per-trial in
+`metrics["score"]` and in `metadata["session_summary"]["weighted_selection"]`.
+`auto_load_best` and `get_best_config()` load this same weighted winner.
+
+The agreement holds for the common run. Two narrow cases can still diverge, by
+design: (1) a custom `promotion_policy.tie_breaker` (anything other than the default
+`min_abs_deviation`) is honoured by the terminal selector but not by the post-hoc
+accessor, which always uses the default; and (2) the terminal selector ranks only
+**ranking-eligible** trials (those passing the comparability/coverage gate) while
+`calculate_weighted_scores()` ranks all **successful** trials, so a successful but
+ranking-ineligible config can appear only in the post-hoc set.
+
+Single-objective runs are unchanged (highest primary-objective config); banded
+objectives select by band proximity. Declaring non-uniform weights simply shifts
+the aggregate toward the heavier objective. (Issues #1682, #1846.)
+
 ### TrialResult
 
 ```python

{traigent-0.21.3.dev0 → traigent-0.22.0}/docs/api-reference/telemetry.md

@@ -122,6 +122,23 @@ token usage from prompts. Pass measured `input_tokens`, `output_tokens`,
 `total_tokens`, or `cost_usd` from the provider response when recording
 generation spans; otherwise usage is reported as unknown, not zero.
 
+### Content-Free Execution Lineage
+
+`ExecutionContextDTO` carries only versioned lineage identifiers. Client and
+environment defaults are merged into each trace; fields omitted from a
+per-trace DTO inherit those defaults. An explicit `None` clears a default and
+is emitted as JSON `null`:
+
+```python
+from traigent.observability import ExecutionContextDTO
+
+context = ExecutionContextDTO(toolset_id=None)
+client.start_trace("agent-run", execution_context=context)
+```
+
+Dictionary inputs have the same explicit-null behavior. Free-form metadata and
+content fields are not accepted in execution context.
+
 ### OpenTelemetry Tracing
 
 OpenTelemetry tracing is opt-in and uses a separate flag from telemetry
@@ -361,6 +378,59 @@ For HIPAA compliance or handling sensitive data:
 
 4. **Review Evaluation Dataset**: Ensure your evaluation datasets don't contain PII
 
+## Observability transport health
+
+`ObservabilityClient.get_stats()` returns a process-local snapshot of buffered
+trace delivery. Use it to alert on local loss before a trace reaches the
+backend. The snapshot includes `dropped_items`, `dropped_by_reason`,
+`queue_depth`, `inflight_items`, `oldest_inflight_age_seconds`, and
+`retry_attempts`.
+
+```python
+from traigent.observability import ObservabilityClient, ObservabilityConfig
+
+
+def report_transport_health(event_type: str, payload: dict) -> None:
+    # Every local drop is reported. Delivery failures are one event per batch:
+    # payloads include item_count and up to 20 trace_ids alongside drop_reason,
+    # cumulative dropped_items, queue_depth, and event-specific details.
+    metrics_sink.emit(event_type, payload)
+
+
+client = ObservabilityClient(
+    ObservabilityConfig(health_callback=report_transport_health)
+)
+stats = client.get_stats()
+if stats["dropped_items"]:
+    metrics_sink.emit("observability_transport_snapshot", stats)
+```
+
+`queue_depth` counts payloads still buffered; `inflight_items` counts payloads
+already handed to a sender but not yet completed. A non-null
+`oldest_inflight_age_seconds` makes a custom sender that does not return
+diagnosable; custom senders should enforce their own request timeout.
+`dropped_by_reason` is bounded to the transport's in-process lifetime and
+distinguishes reasons such as
+`queue_full`, `payload_too_large`, `payload_not_json_serializable`,
+`transport_closed`, and `batch_delivery_failed`. These are local SDK metrics;
+they are deliberately not added to the ingest wire payload until the strict
+Schema and Backend contract is updated together.
+
+`health_callback` is invoked only after SDK transport locks are released. It
+may run on a background delivery thread, and callback exceptions are swallowed.
+Keep it fast and safe for concurrent invocation. Batch-delivery failures emit a
+single batch-shaped event with `item_count`, up to 20 `trace_ids`, and
+`trace_ids_truncated`; queue, oversized-payload, serialization, and closed-
+transport drops each emit their own event.
+
+`flush(timeout=0)` and `close(timeout=0)` are immediate poll-style calls: they
+return without starting a sender and do not emit a deadline-exceeded warning.
+Previously, zero was treated like the configured flush timeout. An explicit
+positive timeout bounds the wait; no-argument flush/close preserves synchronous
+delivery. The atexit close path explicitly uses the default 30-second
+`flush_timeout`, so shutdown can trade undelivered tail items for a bounded
+interpreter exit.
+
 ## FAQ
 
 ### Q: Is telemetry enabled by default?

{traigent-0.21.3.dev0 → traigent-0.22.0}/docs/guides/evaluation.md

@@ -198,6 +198,31 @@ enable_mock_mode_for_quickstart()
 
 For shell-only fixtures, `TRAIGENT_MOCK_LLM=true` remains available outside production for backwards compatibility, but direct user-set activation emits `DeprecationWarning`.
 
+## Reading trial results — the trial-metrics contract
+
+Each `TrialResult` (the items in `result.trials`) exposes several score-like
+values. They do **not** all mean the same thing:
+
+| Field | Meaning |
+|---|---|
+| `trial.score` | The trial's **optimization signal** — the exact value `best_config` argmaxes. For a single-objective run it is the objective metric (e.g. accuracy); for a weighted multi-objective run it is the weighted selection basis. **Read this when you want "the trial's score".** |
+| `trial.metrics["score"]` | Same optimization signal, kept in the metrics map for serialization parity. |
+| `trial.metrics["accuracy"]` (and other objective keys) | The individual metric values. When a custom `scoring_function` defines the objective, `metrics["accuracy"]` is the custom scorer's mean. |
+| `trial.metrics["exact_match_default"]` | Present **only** when a custom `scoring_function` owns the `accuracy` objective. It holds the SDK's built-in case-insensitive exact-match scorer as a **diagnostic** — it is never the optimization signal and is not what `best_config` uses. |
+
+```python
+result = my_agent.optimize_sync(max_trials=10)
+for trial in result.trials:
+    # Correct: trial.score is the signal the optimizer used.
+    print(trial.config, trial.score)
+```
+
+Before this contract was fixed (issue #1845), `trial.score` was `None` and
+`metrics["score"]` held the built-in exact-match scorer even when a custom
+`scoring_function` was supplied — so a consumer doing
+`trial.score or trial.metrics["score"]` would log a number the optimizer never
+used, disagreeing with `best_config` whenever the custom scorer was non-binary.
+
 ## Troubleshooting
 
 - **Paraphrased answers score 0.0**: The default `LocalEvaluator` accuracy is exact / case-insensitive match; it is not semantic. Provide your own `scoring_function` (or `metric_functions={"accuracy": ...}`) that performs embedding similarity or LLM-judge scoring. `LocalExecutionAdapter` also raises a visible error when dataset examples are tagged `evaluation_type: "semantic"` without a configured scorer.

{traigent-0.21.3.dev0 → traigent-0.22.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "traigent"
-version = "0.21.3.dev0"
+version = "0.22.0"
 authors = [
     {name = "Traigent Team", email = "opensource@traigent.ai"},
 ]
@@ -44,6 +44,7 @@ dependencies = [
     "PyJWT[crypto]>=2.13.0,<3.0.0",  # >=2.13.0: Dependabot alerts — DoS, SSRF via JWKClient, HS256 forgery, alg bypass, kid DoS
     "backoff>=2.2.0",
     "litellm>=1.87.1,<2",  # Floor: 1.84+ escaped the 1.83.x python-dotenv==1.0.1 and openai==2.24.0 exact sub-pins; 1.87.1 is the first stable release verified against this SDK. Upper bound <2 guards against a future breaking major. The demo (TraigentDemo) requires 1.88.0; this range admits it. Only the public API surface (completion, acompletion, cost_per_token, token_counter, model_cost, ModelResponse) is used — stable across the 1.x line.
+    "tenacity>=8.1.0",  # tenacity completes litellm's retry surface: litellm imports it lazily in its public *_with_retries helpers (and the retry path exercised on litellm 1.91.0, #1824) but doesn't declare it — declare it so litellm's retry path works on a clean install
     "rank-bm25",
     "psutil>=5.8.0",
     # Required by API decorators and config validation

{traigent-0.21.3.dev0 → traigent-0.22.0}/requirements/requirements.txt

@@ -11,5 +11,6 @@ claude-code-sdk>=0.0.14
 mcp>=1.27.0
 backoff>=2.2.0
 litellm>=1.87.1,<2
+tenacity>=8.1.0
 rank-bm25
 psutil>=5.8.0

{traigent-0.21.3.dev0 → traigent-0.22.0}/traigent/analytics/__init__.py

@@ -16,6 +16,9 @@ from __future__ import annotations
 
 import warnings as _warnings
 
+# Planner V2 is a core protocol, independent of the optional analytics plugin.
+from .planner import PlannerV2Client
+
 _PLUGIN_AVAILABLE = False
 
 # Try to import from the plugin first
@@ -131,6 +134,7 @@ __all__ = [
     # Example Insights
     "ExampleInsightsClient",
     "NextStepsClient",
+    "PlannerV2Client",
     "OptimizationPlanClient",
     # Historical analytics
     "HistoricalAnalyticsEngine",