traigent 0.20.0__tar.gz → 0.21.0.dev0__tar.gz

{traigent-0.20.0 → traigent-0.21.0.dev0}/CHANGELOG.md

@@ -6,6 +6,81 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [0.21.0] - 2026-07-07
+
+Observability hardening release. Enterprise-grade privacy posture for the
+`@observe` instrumentation plus latency/usage accuracy fixes across the
+SDK, backend, and portal.
+
+### Changed
+
+- **BREAKING (behavioral): `@observe` is now metadata-only by default.** The
+  decorator and `ObserveContext` no longer send function arguments or return
+  values to the observability backend by default — only span metadata (name,
+  type, timing, tags, session/environment, and token/cost fields you pass).
+  To restore raw input/output capture, opt in per span with
+  `@observe(..., content_mode="record")` or globally with
+  `TRAIGENT_OBSERVABILITY_CONTENT=record`. Use `content_mode="redacted"` to
+  emit redaction placeholders. This closes an egress path where prompt/response
+  content left the process by default. **Migration:** if you relied on seeing
+  prompts/outputs in the dashboard, set `content_mode="record"` on those spans
+  (only for spans whose I/O is safe to store).
+- Recorded span latency now measures only the wrapped function body. Previously
+  the span start was stamped before SDK trace-setup work, so transport/setup
+  overhead (tens of ms) was misattributed to application latency.
+- Token and cost usage fields are nullable end-to-end (SDK DTO → backend schema
+  and model → portal). Unknown usage now renders as `—` instead of a misleading
+  `0` / `$0.0000`; a measured zero still renders as `0`. Completed generation
+  spans with no usage log a one-time warning rather than implying auto-capture.
+
+### Fixed
+
+- Observability transport no longer blocks the calling application thread on
+  batch-size flushes: sends run on a daemon flush thread, and payload
+  redaction/serialization happen before acquiring the transport lock. An age
+  timer stays armed as a backstop so a tail item is never stranded if the flush
+  thread is in its death window when the next item arrives.
+- Telemetry drops (queue full, oversized/non-serializable payloads, closed
+  transport, failed batches) are no longer silent: each updates stats/errors and
+  emits a `health_callback` event.
+- Unknown `ENVIRONMENT`/`TRAIGENT_ENV` labels now log a warning (without leaking
+  the raw value) instead of being silently ignored; a missing `session_id`
+  renders as `—`.
+
+## [0.20.1] - 2026-07-06
+
+Patch release for the managed optimization dispatch fix wave and adjacent
+SDK honesty/hardening fixes shipped on develop after 0.20.0.
+
+### Changed
+- Named smart optimization for managed-backend users now binds supported
+  algorithm names (`bayesian`, `tpe`, `optuna`, `optuna_tpe`, and
+  `optuna_random`) to the typed backend Optuna strategy and serializes
+  `optimization_strategy` on session creation. Unsupported smart names such
+  as `nsga2` and `cmaes` now fail before backend session creation with a
+  capability message instead of creating an orphaned session or returning a
+  misleading zero-trial result. (#1758, #1752)
+
+### Fixed
+- Dict-input evaluation now warns when metadata-only parameters are defaulted
+  instead of silently accepting the defaulted shape. (#1756, #1746)
+- Local run-log persistence preserves trial/configuration fidelity and keeps
+  orchestrator builder compatibility. (#1757, #1747, #1748)
+- CLI algorithm/model surfaces now match public behavior and remove unsourced
+  Claude model guesses. (#1759, #1751)
+- Callback failures derived from `BaseException` are contained without
+  breaking optimization control flow. (#1760, #1749)
+- Objective reporting now honestly records unmatched objective metrics instead
+  of implying a measurement exists. (#1761, #1691)
+- API-key validation failures are classified as auth/permission failures rather
+  than generic backend/network errors. (#1762, #1754)
+- Prior unreleased develop fixes since 0.20.0: public onboarding skills repo
+  selection (#1737, #1623), early-stop decorator propagation (#1739, #1692),
+  unpriceable OpenRouter cost handling (#1740, #1597), redaction fail-open
+  tightening (#1742, #1649, #1650), exact ruff pin extraction in preflight
+  (#1743, #1550), analytics insight value validation (#1744, #1662), and
+  companion error message threading (#1745, #1885).
+
 ## [0.20.0] - 2026-07-05
 
 Fail-closed hardening release. A systematic silent-failure audit of the

{traigent-0.20.0/traigent.egg-info → traigent-0.21.0.dev0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: traigent
-Version: 0.20.0
+Version: 0.21.0.dev0
 Summary: Enterprise-grade LLM optimization platform with advanced analytics and AI-powered insights
 Author-email: Traigent Team <opensource@traigent.ai>
 License-Expression: AGPL-3.0-only OR LicenseRef-Traigent-Commercial
@@ -121,7 +121,7 @@ Requires-Dist: types-PyYAML>=6.0.0; extra == "dev"
 Requires-Dist: types-requests>=2.31.0; extra == "dev"
 Requires-Dist: types-bleach>=6.0.0; extra == "dev"
 Requires-Dist: pre-commit>=3.0.0; extra == "dev"
-Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: ruff==0.15.12; extra == "dev"
 Requires-Dist: bandit>=1.7.0; extra == "dev"
 Requires-Dist: hypothesis>=6.100.0; extra == "dev"
 Provides-Extra: docs

{traigent-0.20.0 → traigent-0.21.0.dev0}/docs/api-reference/telemetry.md

@@ -41,7 +41,8 @@ During optimization runs, Traigent collects:
 ### What is NOT Collected
 
 Outside tuned configuration values explicitly sent on the default portal-backed
-path, Traigent does **not** collect:
+path, and outside observability content you explicitly opt into recording,
+Traigent does **not** collect:
 
 - **User prompts or inputs**
 - **LLM responses or outputs**
@@ -88,6 +89,39 @@ Use `TRAIGENT_DISABLE_TELEMETRY=true` for SDK telemetry opt-out, effective
 values on privacy-mode submissions, and `offline=True` for zero Traigent backend
 egress.
 
+### `@observe` Content Egress
+
+The `@observe` decorator and `ObserveContext` are metadata-only by default. They
+can send trace names, observation names, status, timing, tags, environment,
+release, IDs, explicit metadata, and token/cost fields you pass, but they omit
+function arguments, explicit context `input_data`, return values, and output data
+unless you opt in.
+
+To send prompt/output content for observability, opt in explicitly:
+
+```python
+@observe("llm-call", content_mode="record")
+def call_model(prompt: str) -> str:
+    ...
+```
+
+Alternatively set `TRAIGENT_OBSERVABILITY_CONTENT=record`. The supported content
+modes are:
+
+- `metadata` (default): omit input and output content
+- `redacted`: send `{"redacted": true}` placeholders for input and output
+- `record`: send captured input and output content
+
+`redact_input=True` and `redact_output=True` force redaction placeholders for the
+corresponding side even when `content_mode="record"`. The transport still applies
+pattern-based secret scrubbing before send, but that scrubber is a final safety
+net, not the content-egress policy.
+
+`@observe(observation_type=GENERATION)` does not estimate LangChain or provider
+token usage from prompts. Pass measured `input_tokens`, `output_tokens`,
+`total_tokens`, or `cost_usd` from the provider response when recording
+generation spans; otherwise usage is reported as unknown, not zero.
+
 ### OpenTelemetry Tracing
 
 OpenTelemetry tracing is opt-in and uses a separate flag from telemetry
@@ -179,6 +213,7 @@ The following values are recognized as "opt-out" (case-insensitive):
 - `"true"`
 - `"1"`
 - `"yes"`
+- `"on"`
 
 Any other value (including unset) means telemetry is enabled.
 

{traigent-0.20.0 → traigent-0.21.0.dev0}/docs/features/onboarding.md

@@ -20,7 +20,7 @@ In a terminal, onboarding can offer to:
 
 - add `traigent[integrations]` to the project
 - start device login unless `--no-login` is set
-- install Traigent agent skills through `npx skills add Traigent/agents-skills`
+- install Traigent agent skills through `npx skills add Traigent/traigent-skills`
 - register MCP for detected agents when `traigent mcp --help` succeeds
 - run the packaged mock quickstart
 - print the first prompt for the detected coding agent

{traigent-0.20.0 → traigent-0.21.0.dev0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "traigent"
-version = "0.20.0"
+version = "0.21.0.dev0"
 authors = [
     {name = "Traigent Team", email = "opensource@traigent.ai"},
 ]
@@ -177,7 +177,9 @@ dev = [
     "types-requests>=2.31.0",
     "types-bleach>=6.0.0",
     "pre-commit>=3.0.0",
-    "ruff>=0.1.0",
+    "ruff==0.15.12",  # Exact pin (issue #1550): CI preflight installs this exact
+    # version so an unrelated ruff release can't retroactively fail PRs on
+    # pre-existing lint debt. Bump deliberately, in the same PR as any fix-up.
     "bandit>=1.7.0",
     "hypothesis>=6.100.0",  # Property-based testing for validation logic
 ]

{traigent-0.20.0 → traigent-0.21.0.dev0}/requirements/requirements-dev.txt

@@ -18,7 +18,7 @@ isort>=5.10.0
 flake8>=5.0.0
 mypy>=1.0.0
 pre-commit>=3.0.0
-ruff>=0.1.0
+ruff==0.15.12
 
 # Visualization
 matplotlib>=3.5.0

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/api/decorators.py

@@ -701,10 +701,16 @@ _OPTIMIZE_DEFAULTS: dict[str, Any] = {
     "execution": None,
     "mock": None,
     "max_trials": DEFAULT_MAX_TRIALS,
-    # Early stopping parameters
-    "plateau_window": None,  # Stop if no improvement for N trials
-    "plateau_epsilon": None,  # Improvement threshold for plateau detection
-    "semantic_saturation": None,  # Opt-in semantic saturation stop condition config
+    # NOTE: plateau_window / plateau_epsilon / semantic_saturation are
+    # intentionally NOT listed here (issue #1692). They are early-stop
+    # runtime-override keys (see _ALLOWED_RUNTIME_OVERRIDE_KEYS) that must
+    # flow through `combined_runtime_overrides` -> `_decorator_runtime_overrides`
+    # -> the orchestrator's stop-condition config, both at decoration time and
+    # at call time. Adding them to _OPTIMIZE_DEFAULTS pulls them into
+    # _DIRECT_OPTION_KEYS, which reroutes decorator-time values into the dead
+    # `combined_settings` bucket (nothing ever reads combined_settings[...]
+    # for these keys) instead of `combined_runtime_overrides`, silently
+    # no-op'ing early stopping when configured on the decorator.
     # Multi-agent configuration
     "agents": None,  # Explicit agent definitions
     "agent_prefixes": None,  # Prefix-based agent inference

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/api/types.py

@@ -727,6 +727,11 @@ class OptimizationResult:
     # Stop reason - why the optimization stopped (see class docstring for values)
     stop_reason: StopReason | None = None
 
+    # Selection reason for no-winner or fail-closed terminal results. This is
+    # distinct from stop_reason: stop_reason explains why execution stopped,
+    # while reason_code explains why result selection did not produce a winner.
+    reason_code: str | None = None
+
     # Run identification and cloud link
     experiment_id: str | None = None
     cloud_url: str | None = None
@@ -781,6 +786,8 @@ class OptimizationResult:
     @property
     def success_rate(self) -> float:
         """Calculate trial success rate."""
+        if "OBJECTIVE_UNMATCHED" in self.warning_codes:
+            return 0.0
         if not self.trials:
             return 0.0
         return len(self.successful_trials) / len(self.trials)

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/cli/auth_commands.py

@@ -503,7 +503,8 @@ class TraigentAuthCLI:
 
         try:
             async with aiohttp.ClientSession(
-                timeout=aiohttp.ClientTimeout(total=15)
+                timeout=aiohttp.ClientTimeout(total=15),
+                trust_env=True,
             ) as session:
                 async with session.post(url, headers=headers) as response:
                     response_text = await response.text()
@@ -896,7 +897,7 @@ class TraigentAuthCLI:
             return False
 
         try:
-            async with aiohttp.ClientSession() as session:
+            async with aiohttp.ClientSession(trust_env=True) as session:
                 authorization = await self._start_device_authorization(session)
                 self._display_device_authorization(authorization)
                 credentials = await self._poll_device_token(
@@ -1026,7 +1027,7 @@ class TraigentAuthCLI:
 
         console.print(f"[dim]POST {login_url}[/dim]")
 
-        async with aiohttp.ClientSession() as session:
+        async with aiohttp.ClientSession(trust_env=True) as session:
             async with session.post(
                 login_url,
                 json={"email": email, "password": password},
@@ -1113,7 +1114,7 @@ class TraigentAuthCLI:
         console.print(f"\n[dim]POST {api_key_url}[/dim]")
         console.print(f"[dim]Creating API key: {key_name}[/dim]")
 
-        async with aiohttp.ClientSession() as session:
+        async with aiohttp.ClientSession(trust_env=True) as session:
             async with session.post(
                 api_key_url,
                 json=key_payload,
@@ -1446,7 +1447,7 @@ class TraigentAuthCLI:
             try:
                 import aiohttp
 
-                async with aiohttp.ClientSession():
+                async with aiohttp.ClientSession(trust_env=True):
                     # Backend may not expose a revoke endpoint yet; placeholder for future call.
                     pass  # Silently continue if revocation fails
             except Exception as e:
@@ -1985,7 +1986,8 @@ async def _check_api_key(backend_api_url: str, key: str) -> bool:
 
     try:
         async with aiohttp.ClientSession(
-            timeout=aiohttp.ClientTimeout(total=15)
+            timeout=aiohttp.ClientTimeout(total=15),
+            trust_env=True,
         ) as session:
             async with session.post(
                 url,

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/cli/main.py

@@ -15,7 +15,6 @@ from rich.table import Table
 
 from traigent import get_version_info
 from traigent.api.functions import (
-    get_available_strategies,
     list_recommendation_agent_types,
     recommend_configuration_space,
 )
@@ -32,6 +31,7 @@ from traigent.cli.auth_commands import auth
 from traigent.cli.hooks_commands import hooks
 from traigent.cli.local_commands import register_local_commands
 from traigent.cli.sync_commands import register_sync_command
+from traigent.config.types import accepted_algorithm_values
 from traigent.evaluators import (
     list_eval_recommendation_task_types,
     recommend_evaluator,
@@ -508,34 +508,33 @@ def info() -> None:
 
 @cli.command()
 def algorithms() -> None:
-    """List available optimization algorithms."""
-    strategies = get_available_strategies()
+    """List public optimization algorithms accepted by SDK validation."""
 
     console.print("\n[bold blue]Available Optimization Algorithms[/bold blue]\n")
 
-    for name, info in strategies.items():
-        console.print(f"[bold green]{info['name']}[/bold green] ([cyan]{name}[/cyan])")
-        console.print(f"  {info['description']}")
-        console.print(f"  Best for: {info['best_for']}")
-
-        # Capabilities
-        capabilities = []
-        if info["supports_continuous"]:
-            capabilities.append("continuous")
-        if info["supports_categorical"]:
-            capabilities.append("categorical")
-        if info["deterministic"]:
-            capabilities.append("deterministic")
-
-        console.print(f"  Supports: {', '.join(capabilities)}")
+    table = Table(show_header=True, header_style=_TABLE_HEADER_STYLE)
+    table.add_column("Algorithm", style="cyan")
+    table.add_column("Availability")
+    table.add_column("Notes")
+
+    for name in accepted_algorithm_values():
+        if name == "auto":
+            availability = "[green]connected[/green]"
+            notes = "Default cloud smart optimizer/TPE path when authenticated."
+        elif name in {"grid", "random"}:
+            availability = "[green]local[/green]"
+            notes = "Runs in-process; can sync results when authenticated."
+        else:
+            availability = "[yellow]connected[/yellow]"
+            notes = "Accepted public smart name; routed through Traigent cloud."
 
-        # Parameters
-        if info["parameters"]:
-            console.print("  Parameters:")
-            for param, desc in info["parameters"].items():
-                console.print(f"    • {param}: {desc}")
+        table.add_row(name, availability, notes)
 
-        console.print()
+    console.print(table)
+    console.print(
+        "\n[dim]Runtime-only registry names are intentionally omitted here because "
+        "they are not accepted by public SDK validation.[/dim]"
+    )
 
 
 @cli.command("recommend")

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/cli/onboard_commands.py

@@ -33,7 +33,7 @@ FIRST_PROMPT_TOOL_LINE: dict[AgentName, str] = {
 }
 PLAN_JSON_BEGIN = "BEGIN_TRAIGENT_ONBOARD_PLAN_JSON"
 PLAN_JSON_END = "END_TRAIGENT_ONBOARD_PLAN_JSON"
-SKILLS_COMMAND = ["npx", "skills", "add", "Traigent/agents-skills"]
+SKILLS_COMMAND = ["npx", "skills", "add", "Traigent/traigent-skills"]
 AUTH_STATUS_COMMAND = ["traigent", "auth", "status"]
 
 

{traigent-0.20.0 → traigent-0.21.0.dev0}/traigent/cloud/analytics_client.py

@@ -127,6 +127,8 @@ _RUN_EXAMPLE_INSIGHTS_REQUIRED_KEYS = frozenset(
     }
 )
 _EXAMPLE_INSIGHT_REF_PATTERN = re.compile(r"^exref_[0-9a-f]{16}$")
+_EXAMPLE_INSIGHT_MAX_SAFE_EXAMPLE_REFS = 50
+_EXAMPLE_INSIGHTS_DATASET_QUALITIES = frozenset({"low", "medium", "high"})
 _EXAMPLE_INSIGHT_REVIEW_PRIORITIES = frozenset({"critical", "high", "medium", "low"})
 _EXAMPLE_INSIGHT_DIFFICULTY_BUCKETS = frozenset({"low", "medium", "high", "unknown"})
 _EXAMPLE_INSIGHT_SUSPICIOUS_FLAGS = frozenset(
@@ -277,7 +279,13 @@ def _project_aggregate(data: Any, keys: frozenset[str], *, what: str) -> dict[st
     Unknown nested keys can carry proprietary signals or raw backend state.
     This function is the nested-aggregate counterpart of the per-row construction
     in :func:`_project_example_insight_row`: both rebuild output by allowlist so
-    nothing unknown can pass through regardless of what the backend sends.
+    nothing unknown can pass through regardless of what the backend sends. Note
+    that this allowlists by *key* only — callers whose values carry a specific
+    contract (e.g. ``summary.dataset_quality``, ``cohorts[].safe_example_refs``)
+    must additionally validate those values themselves (see
+    :func:`_project_example_insights_summary` and
+    :func:`_project_example_insights_cohort`); an unvalidated value allowlisted
+    by key alone could still smuggle raw/proprietary content through.
     """
     if not isinstance(data, dict):
         raise AnalyticsClientError(
@@ -286,6 +294,81 @@ def _project_aggregate(data: Any, keys: frozenset[str], *, what: str) -> dict[st
     return {k: v for k, v in data.items() if k in keys}
 
 
+def _assert_allowed_string(value: Any, allowed: frozenset[str], *, what: str) -> str:
+    """Validate that ``value`` is one of a fixed set of enum strings.
+
+    Mirrors the JS SDK's ``assertAllowedString`` (traigent-js analytics-client.ts)
+    used to close the same value-validation gap for cross-SDK parity.
+    """
+    if not isinstance(value, str) or value not in allowed:
+        raise AnalyticsClientError(
+            f"Malformed example insights response: {what} must be one of: "
+            f"{', '.join(sorted(allowed))}."
+        )
+    return value
+
+
+def _assert_safe_example_refs(value: Any, *, what: str) -> list[str]:
+    """Validate a cohort's ``safe_example_refs`` list.
+
+    Each element must be an opaque ``exref_`` reference matching the backend
+    contract (``run_analytics_service._safe_example_refs``), and the list must
+    not exceed the backend's cap. Mirrors the JS SDK's
+    ``assertSafeExampleRefs`` for cross-SDK parity (Traigent#1662).
+    """
+    if not isinstance(value, list):
+        raise AnalyticsClientError(
+            f"Malformed example insights response: {what} must be a list."
+        )
+    if len(value) > _EXAMPLE_INSIGHT_MAX_SAFE_EXAMPLE_REFS:
+        raise AnalyticsClientError(
+            f"Malformed example insights response: {what} must contain at most "
+            f"{_EXAMPLE_INSIGHT_MAX_SAFE_EXAMPLE_REFS} refs."
+        )
+    for ref in value:
+        if not isinstance(ref, str) or not _EXAMPLE_INSIGHT_REF_PATTERN.match(ref):
+            raise AnalyticsClientError(
+                f"Malformed example insights response: {what} must only contain "
+                "refs matching ^exref_[0-9a-f]{16}$."
+            )
+    return list(value)
+
+
+def _project_example_insights_summary(data: Any) -> dict[str, Any]:
+    """Project + value-validate the ``summary`` aggregate.
+
+    ``dataset_quality`` is required and must be one of the backend's frozen
+    ``Literal["low", "medium", "high"]`` enum values; a backend regression
+    emitting a raw/unexpected value must fail closed rather than pass through.
+    """
+    summary = _project_aggregate(data, _EXAMPLE_INSIGHTS_SUMMARY_KEYS, what="summary")
+    summary["dataset_quality"] = _assert_allowed_string(
+        summary.get("dataset_quality"),
+        _EXAMPLE_INSIGHTS_DATASET_QUALITIES,
+        what="summary.dataset_quality",
+    )
+    return summary
+
+
+def _project_example_insights_cohort(cohort: Any, *, index: int) -> dict[str, Any]:
+    """Project + value-validate one ``cohorts[]`` aggregate.
+
+    ``safe_example_refs`` is optional per cohort, but when present each element
+    must be a well-formed opaque ref within the backend's cap — a backend
+    regression placing raw prompt text/emails in this list must not pass
+    through the SDK.
+    """
+    projected = _project_aggregate(
+        cohort, _EXAMPLE_INSIGHTS_COHORT_KEYS, what=f"cohorts[{index}]"
+    )
+    if "safe_example_refs" in projected:
+        projected["safe_example_refs"] = _assert_safe_example_refs(
+            projected["safe_example_refs"],
+            what=f"cohorts[{index}].safe_example_refs",
+        )
+    return projected
+
+
 def _quote_segment(value: str, *, field: str) -> str:
     """URL-encode a path segment, rejecting empty values up front.
 
@@ -687,9 +770,13 @@ class BackendAnalyticsClient:
         from projected safe fields so raw signals cannot pass through even on a
         backend regression. ``summary``, ``cohorts``, ``recommendations``, and
         ``redactions`` are backend-redacted, templated, canary-enforced safe
-        aggregate guidance and are forwarded as-is. This reader deliberately
-        allowlists by construction, unlike the other thin pass-through readers in
-        this module, because it is the IP-sensitive per-example endpoint.
+        aggregate guidance, projected by key allowlist. ``summary.dataset_quality``
+        and ``cohorts[].safe_example_refs`` additionally get per-value shape
+        validation (enum / opaque-ref-pattern + length cap): a backend regression
+        placing raw content in those two value families must fail closed rather
+        than pass through (Traigent#1662). This reader deliberately allowlists by
+        construction, unlike the other thin pass-through readers in this module,
+        because it is the IP-sensitive per-example endpoint.
         """
         path = (
             "/api/v1/analytics/runs/"
@@ -729,21 +816,13 @@ class BackendAnalyticsClient:
         return {
             "run_id": payload["run_id"],
             "privacy_mode": payload["privacy_mode"],
-            "summary": _project_aggregate(
-                payload["summary"],
-                _EXAMPLE_INSIGHTS_SUMMARY_KEYS,
-                what="summary",
-            ),
+            "summary": _project_example_insights_summary(payload["summary"]),
             "example_rows": [
                 _project_example_insight_row(row, index=index)
                 for index, row in enumerate(rows)
             ],
             "cohorts": [
-                _project_aggregate(
-                    cohort,
-                    _EXAMPLE_INSIGHTS_COHORT_KEYS,
-                    what=f"cohorts[{i}]",
-                )
+                _project_example_insights_cohort(cohort, index=i)
                 for i, cohort in enumerate(cohorts_raw)
             ],
             "recommendations": [