@elevasis/sdk 0.5.12 → 0.5.14

package/reference/{roadmap/index.mdx → roadmap.mdx}

@@ -1,147 +1,163 @@
----
-title: Roadmap
-description: Planned SDK features -- error taxonomy, retry semantics, circuit breaker, metrics, alerting, and resource lifecycle extensions
-loadWhen: "Asking about future features or planned capabilities"
----
-
-**Status:** Planned -- none of these features are implemented yet.
-
-For currently implemented behavior, see [Runtime](../runtime/index.mdx).
-
----
-
-## Structured Error Taxonomy
-
-The current runtime reports errors as plain strings. A future SDK version will introduce a structured error taxonomy. All SDK errors will extend `ResourceError`, the base class for errors surfaced through the execution protocol.
-
-Every error carries: `message` (string), `code` (string enum), `details` (optional structured data), and `retryable` (boolean).
-
-**Error types:**
-
-- **`ResourceError`** -- Base class for all SDK errors
-- **`ValidationError`** -- Input or output schema validation failed. Thrown automatically when Zod `.parse()` fails. Code: `VALIDATION_ERROR`. Not retryable.
-- **`StepError`** -- A workflow step handler threw. Includes `stepId` and `stepName`. Code: `STEP_ERROR`. Retryable (transient failures may succeed on retry).
-- **`ToolError`** -- A tool execution failed. Includes `toolName`. Code: `TOOL_ERROR`. Retryability depends on the underlying error.
-- **`TimeoutError`** -- Execution exceeded the deadline. Code: `TIMEOUT`. Retryable.
-- **`CancellationError`** -- Execution was cancelled by the platform or user. Code: `CANCELLED`. Not retryable.
-
----
-
-## Retry Semantics
-
-Retries are platform-side only -- workers are ephemeral and never retry internally.
-
-- **Configuration:** Per-resource via `maxRetries` (default: 0) and `backoffStrategy` (exponential with jitter)
-- **Retryable conditions:** Worker crash or timeout (worker terminated by `AbortSignal`)
-- **Non-retryable conditions:** Worker reports `status: 'failed'` (handler ran and returned an error -- application logic, not infrastructure failure), user cancellation
-- **Idempotency:** On retry, the same `executionId` is reused. Design handlers to be idempotent where possible.
-
----
-
-## Workflow Step Failure
-
-Default behavior is fail-fast: when a step throws, the workflow fails immediately.
-
-- **Error handler:** Optional `onError` callback per step. The callback receives the error and can return a recovery value or rethrow to propagate the failure.
-- **Partial output:** Steps completed before the failure are included in the error response. The platform receives: `failedStepId`, `completedSteps[]` (IDs of successfully completed steps), and `partialOutput` (the last successful step's output).
-
-**Step error response format:**
-
-```json
-{
-  "status": "failed",
-  "error": {
-    "code": "STEP_ERROR",
-    "message": "Email delivery failed: invalid address",
-    "stepId": "send-welcome",
-    "stepName": "Send Welcome Email"
-  },
-  "completedSteps": ["validate"],
-  "partialOutput": { "clientName": "Jane", "isValid": true }
-}
-```
-
----
-
-## Agent Failure Modes
-
-Agents have multiple failure paths due to the LLM loop:
-
-- **Max iterations reached:** The agent returns the best output produced so far, plus a warning flag (`maxIterationsReached: true`). This is a graceful termination, not an error.
-- **Tool crash:** Tool errors are caught by the SDK runtime, formatted as a tool result, and sent back to the LLM. The LLM decides whether to retry, try a different approach, or give up.
-- **Model refusal:** If the model refuses the prompt, the SDK retries once with an adjusted system prompt. If the retry also refuses, the agent fails with `code: 'MODEL_REFUSAL'`.
-- **Model API error:** Network errors, rate limits, or server errors from the model provider. The SDK retries with exponential backoff (3 attempts, 1s/2s/4s), then fails with `code: 'MODEL_ERROR'`.
-- **Agent error response includes:** `iterationCount` (LLM iterations completed), `toolCallHistory` (array of tool calls made), `lastModelResponse` (final response from the LLM before failure).
-
----
-
-## Circuit Breaker
-
-The platform will implement a circuit breaker to prevent runaway failures:
-
-- **Trip condition:** 5 consecutive failures on the same resource within a 10-minute window
-- **Action:** Pause executions for 60 seconds. New execution requests for that resource return `503` with: "Resource temporarily unavailable (circuit breaker tripped)"
-- **Auto-recovery:** After the 60-second pause, the next execution attempt is allowed through. If it succeeds, the circuit breaker resets. If it fails, the pause extends (120s, then 240s, capped at 5 minutes).
-- **Alerting:** You are notified via webhook callback or email when the circuit breaker trips. Configurable per organization.
-
----
-
-## Metrics
-
-### Auto-Collected
-
-The SDK runtime and platform will automatically collect these metrics for every execution:
-
-- `execution_duration_ms` -- Total wall-clock time from request received to result sent
-- `step_duration_ms` -- Per-step timing for workflows (array of `{ stepId, durationMs }`)
-- `iteration_count` -- Number of LLM loop iterations for agents
-- `ai_token_usage` -- Token counts per model call: `{ prompt_tokens, completion_tokens, total_tokens }`
-- `ai_cost_usd` -- Calculated from model pricing multiplied by token usage
-- `tool_call_count` -- Total number of tool invocations during the execution
-- `tool_call_duration_ms` -- Per-tool timing (array of `{ toolName, durationMs }`)
-- `error_count` -- Number of errors encountered (including recovered errors)
-
-### Cost Attribution
-
-Metrics are aggregated at multiple levels:
-
-- **Per-execution:** Total AI spend and total duration
-- **Per-resource:** Aggregated over configurable time periods (daily, weekly, monthly)
-- **Per-organization:** Total platform cost (execution time + AI spend + managed hosting compute)
-- **Visibility:** Platform dashboard and the CLI via `elevasis-sdk executions <resourceId>`
-
-### Developer-Defined Metrics
-
-A future SDK version will support custom metrics emitted from your handlers:
-
-- `sdk.metrics.counter('custom_name', value)` -- Increment a counter
-- `sdk.metrics.gauge('queue_depth', value)` -- Set a point-in-time gauge value
-- Custom metrics are stored alongside auto-collected metrics and queryable through the same APIs
-
----
-
-## Alerting
-
-Developer-configurable alerts for production monitoring:
-
-- **Error rate threshold:** Notify when more than X% of executions fail within Y minutes
-- **Latency percentile:** Notify when p95 execution duration exceeds a threshold
-- **Cost budget:** Notify when daily or weekly AI spend exceeds a configured limit
-- **Channel:** Webhook callback to a developer-provided URL (integrates with Slack, PagerDuty, and similar services via webhook)
-
----
-
-## Resource Lifecycle Extensions
-
-### Deprecation Status
-
-Beyond the current `dev` and `production` statuses, two additional statuses are planned:
-
-- **`deprecated`** -- Marked via the platform UI. Existing executions continue working. New executions show a warning: "This resource is deprecated." The resource still appears in the platform and can still be triggered.
-- **`offline`** -- Set automatically when a deployment is unregistered (for example, after a failed deploy or explicit deletion). Clears automatically on the next successful deploy. No executions are accepted while a resource is offline.
-
-Deprecation requires no automatic removal -- the developer must explicitly delete the resource to remove it.
-
----
-
-**Last Updated:** 2026-02-25
+---
+title: Roadmap
+description: Planned SDK features -- error taxonomy, retry semantics, circuit breaker, metrics, alerting, and resource lifecycle extensions
+loadWhen: "Asking about future features or planned capabilities"
+---
+
+**Status:** Mixed -- some features below are implemented, others remain planned. Each section notes its current status.
+
+For currently implemented behavior, see [Runtime](runtime.mdx).
+
+---
+
+## Structured Error Taxonomy
+
+**Status: Partially implemented.** The runtime has a structured error hierarchy (`ExecutionError`, `PlatformToolError`, `ToolingError`) with error codes and context fields. The taxonomy below describes a planned _redesign_ that is not yet implemented.
+
+The current runtime reports errors as plain strings. A future SDK version will introduce a structured error taxonomy. All SDK errors will extend `ResourceError`, the base class for errors surfaced through the execution protocol.
+
+Every error carries: `message` (string), `code` (string enum), `details` (optional structured data), and `retryable` (boolean).
+
+**Error types:**
+
+- **`ResourceError`** -- Base class for all SDK errors
+- **`ValidationError`** -- Input or output schema validation failed. Thrown automatically when Zod `.parse()` fails. Code: `VALIDATION_ERROR`. Not retryable.
+- **`StepError`** -- A workflow step handler threw. Includes `stepId` and `stepName`. Code: `STEP_ERROR`. Retryable (transient failures may succeed on retry).
+- **`ToolError`** -- A tool execution failed. Includes `toolName`. Code: `TOOL_ERROR`. Retryability depends on the underlying error.
+- **`TimeoutError`** -- Execution exceeded the deadline. Code: `TIMEOUT`. Retryable.
+- **`CancellationError`** -- Execution was cancelled by the platform or user. Code: `CANCELLED`. Not retryable.
+
+---
+
+## Retry Semantics
+
+**Status: Planned.**
+
+Retries are platform-side only -- workers are ephemeral and never retry internally.
+
+- **Configuration:** Per-resource via `maxRetries` (default: 0) and `backoffStrategy` (exponential with jitter)
+- **Retryable conditions:** Worker crash or timeout (worker terminated by `AbortSignal`)
+- **Non-retryable conditions:** Worker reports `status: 'failed'` (handler ran and returned an error -- application logic, not infrastructure failure), user cancellation
+- **Idempotency:** On retry, the same `executionId` is reused. Design handlers to be idempotent where possible.
+
+---
+
+## Workflow Step Failure
+
+**Status: Planned.** The current runtime uses fail-fast behavior with `WorkflowStepError`. The `onError` callback, `completedSteps`, and `partialOutput` features described below are not yet implemented.
+
+Default behavior is fail-fast: when a step throws, the workflow fails immediately.
+
+- **Error handler:** Optional `onError` callback per step. The callback receives the error and can return a recovery value or rethrow to propagate the failure.
+- **Partial output:** Steps completed before the failure are included in the error response. The platform receives: `failedStepId`, `completedSteps[]` (IDs of successfully completed steps), and `partialOutput` (the last successful step's output).
+
+**Proposed step error response format** (not yet implemented -- subject to change):
+
+```json
+{
+  "status": "failed",
+  "error": {
+    "code": "STEP_ERROR",
+    "message": "Email delivery failed: invalid address",
+    "stepId": "send-welcome",
+    "stepName": "Send Welcome Email"
+  },
+  "completedSteps": ["validate"],
+  "partialOutput": { "clientName": "Jane", "isValid": true }
+}
+```
+
+---
+
+## Agent Failure Modes
+
+**Status: Implemented** (SDK 0.4.2). Agent execution runs in ephemeral worker threads with full tool calling support via `PostMessageLLMAdapter`.
+
+Agents have multiple failure paths due to the LLM loop:
+
+- **Max iterations reached:** The agent returns the best output produced so far, plus a warning flag (`maxIterationsReached: true`). This is a graceful termination, not an error.
+- **Tool crash:** Tool errors are caught by the SDK runtime, formatted as a tool result, and sent back to the LLM. The LLM decides whether to retry, try a different approach, or give up.
+- **Model refusal:** If the model refuses the prompt, the SDK retries once with an adjusted system prompt. If the retry also refuses, the agent fails with `code: 'MODEL_REFUSAL'`.
+- **Model API error:** Network errors, rate limits, or server errors from the model provider. The SDK retries with exponential backoff (3 attempts, 1s/2s/4s), then fails with `code: 'MODEL_ERROR'`.
+- **Agent error response includes:** `iterationCount` (LLM iterations completed), `toolCallHistory` (array of tool calls made), `lastModelResponse` (final response from the LLM before failure).
+
+---
+
+## Circuit Breaker
+
+**Status: Planned.**
+
+The platform will implement a circuit breaker to prevent runaway failures:
+
+- **Trip condition:** 5 consecutive failures on the same resource within a 10-minute window
+- **Action:** Pause executions for 60 seconds. New execution requests for that resource return `503` with: "Resource temporarily unavailable (circuit breaker tripped)"
+- **Auto-recovery:** After the 60-second pause, the next execution attempt is allowed through. If it succeeds, the circuit breaker resets. If it fails, the pause extends (120s, then 240s, capped at 5 minutes).
+- **Alerting:** You are notified via webhook callback or email when the circuit breaker trips. Configurable per organization.
+
+---
+
+## Metrics
+
+**Status: Planned.**
+
+### Auto-Collected
+
+The SDK runtime and platform will automatically collect these metrics for every execution:
+
+- `execution_duration_ms` -- Total wall-clock time from request received to result sent
+- `step_duration_ms` -- Per-step timing for workflows (array of `{ stepId, durationMs }`)
+- `iteration_count` -- Number of LLM loop iterations for agents
+- `ai_token_usage` -- Token counts per model call: `{ prompt_tokens, completion_tokens, total_tokens }`
+- `ai_cost_usd` -- Calculated from model pricing multiplied by token usage
+- `tool_call_count` -- Total number of tool invocations during the execution
+- `tool_call_duration_ms` -- Per-tool timing (array of `{ toolName, durationMs }`)
+- `error_count` -- Number of errors encountered (including recovered errors)
+
+### Cost Attribution
+
+Metrics are aggregated at multiple levels:
+
+- **Per-execution:** Total AI spend and total duration
+- **Per-resource:** Aggregated over configurable time periods (daily, weekly, monthly)
+- **Per-organization:** Total platform cost (execution time + AI spend + managed hosting compute)
+- **Visibility:** Platform dashboard and the CLI via `elevasis-sdk executions <resourceId>`
+
+### Developer-Defined Metrics
+
+A future SDK version will support custom metrics emitted from your handlers:
+
+- `sdk.metrics.counter('custom_name', value)` -- Increment a counter
+- `sdk.metrics.gauge('queue_depth', value)` -- Set a point-in-time gauge value
+- Custom metrics are stored alongside auto-collected metrics and queryable through the same APIs
+
+---
+
+## Alerting
+
+**Status: Planned.**
+
+Developer-configurable alerts for production monitoring:
+
+- **Error rate threshold:** Notify when more than X% of executions fail within Y minutes
+- **Latency percentile:** Notify when p95 execution duration exceeds a threshold
+- **Cost budget:** Notify when daily or weekly AI spend exceeds a configured limit
+- **Channel:** Webhook callback to a developer-provided URL (integrates with Slack, PagerDuty, and similar services via webhook)
+
+---
+
+## Resource Lifecycle Extensions
+
+**Status: Planned.**
+
+### Deprecation Status
+
+Beyond the current `dev` and `prod` statuses, two additional statuses are planned:
+
+- **`deprecated`** -- Marked via the platform UI. Existing executions continue working. New executions show a warning: "This resource is deprecated." The resource still appears in the platform and can still be triggered.
+- **`offline`** -- Set automatically when a deployment is unregistered (for example, after a failed deploy or explicit deletion). Clears automatically on the next successful deploy. No executions are accepted while a resource is offline.
+
+Deprecation requires no automatic removal -- the developer must explicitly delete the resource to remove it.
+
+---
+
+**Last Updated:** 2026-03-08