rootmark 0.1.0__tar.gz → 0.1.2__tar.gz

rootmark-0.1.2/CHANGELOG.md

@@ -0,0 +1,144 @@
+# Changelog
+
+Notable changes to the `rootmark` SDK. Versions follow the collector's own
+releases loosely; where the two must match, it says so.
+
+## 0.1.2
+
+**Fixes a defect in 0.1.1 that stopped runs ever completing.** Upgrade from
+0.1.1; there is no reason to stay on it.
+
+### Fixed
+
+- **A discarded span held its run open for ever.** 0.1.1 made `start_span`
+  write a provisional row, but `discard_span` still emitted nothing -- correct
+  when a discarded span had never been mentioned, wrong once it had. The
+  provisional row said `running` and nothing superseded it, and a run is
+  finished only when all of its spans are. Every run containing a discarded
+  span therefore stayed in progress permanently.
+
+  It was not a corner case. The boto3 interceptor discards its span on *every*
+  Bedrock call made inside a Strands agent, because the Strands patch is
+  already tracing the same call. On the first deployment running both, 18 of
+  18 runs recorded after the upgrade were stuck "running" and none could ever
+  complete.
+
+  Two changes, because one of them is a guarantee and the other is the right
+  shape:
+
+  * `discard_span` now **retracts** a span it announced, writing a terminal
+    `cancelled` row at the end sentinel. Any caller that announced a span can
+    discard it without stranding it. `cancelled` rather than `success`,
+    because nothing was measured -- a 0 ms success would be averaged in as a
+    real and very fast call -- and it is not an error, so error rates are
+    untouched.
+  * `start_span` takes `announce=False` for a span opened speculatively, and
+    the boto3 interceptor uses it. A span that may turn out to belong to
+    another layer is no longer announced at open, so discarding it costs
+    nothing and leaves no `cancelled` row on every trace.
+
+  The cost of the second change: a bare Bedrock call made through boto3
+  *outside* a Strands agent is no longer visible while it runs, only once it
+  completes. Inside Strands -- the common case, and the one that was broken --
+  the agent span above it keeps in-progress visibility.
+
+### Repairing existing data
+
+Runs already stuck cannot fix themselves: their provisional rows will never be
+superseded. Write the retraction the fixed code would have written -- a
+`cancelled` row at `row_version` 2147483647 with the same `(project_id,
+trace_id, start_time, span_id)` -- for every span matching
+
+```sql
+SELECT * FROM trace_events FINAL
+WHERE status NOT IN ('success','error','timeout','cancelled')
+```
+
+then re-project the affected runs. On the deployment above this moved 18 runs
+from `running` to `success` and left no open spans.
+
+## 0.1.1
+
+**Upgrading from 0.1.0 requires a collector upgrade first.** See *Compatibility*
+below — this is not optional, and getting the order wrong loses data.
+
+### Added
+
+- **Spans are sent when they start, not only when they finish.** Before this a
+  span reached Rootmark only from `end_span`, so anything in flight was simply
+  absent — and a step paused for human review was indistinguishable from one
+  wedged against a model that had stopped answering. A run is now visible from
+  the moment its first span opens.
+- `rootmark.mark_waiting(span, reason)` — say a span is deliberately not
+  working, and what it is waiting on. `reason` is free text in the caller's own
+  vocabulary (`"human_review"`, `"rate_limit"`, `"upstream"`); the SDK does not
+  fix the set, because encoding one application's review workflow into the
+  schema would make it everyone's.
+- `heartbeat=` on `rootmark.span()` — re-announce a long span on an interval so
+  a killed process can be told apart from a slow one. Opt-in per span, because
+  a heartbeat on a 200 ms call is volume spent to say nothing. Intervals below
+  five seconds are raised to five rather than refused.
+- `step_key` / `step_kind` on spans, and `SpanEmitter.set_step()` for callers
+  that learn what a step is only after starting it. `step_key` is a stable
+  identity for which logical step a span is; `step_kind` is what it was for.
+  A kind the SDK does not recognise is kept, not dropped.
+- `SpanStatus` gains `running`, `waiting` and `cancelled`, plus an `is_terminal`
+  property. The first three values are unchanged.
+
+### Changed
+
+- **`TraceEvent` carries `row_version` and `waiting_reason`.** `row_version`
+  orders the several writes a span now produces so the collector can keep the
+  latest: start is 1, heartbeats climb, the end row takes a sentinel above every
+  possible heartbeat. A rank rather than a timestamp, because rows arrive from
+  different processes over an at-least-once channel and a clock cannot order
+  them.
+- **Bedrock guardrail traces no longer include `modelOutput`**, and the size cap
+  rose from 4 KB to 16 KB. Bedrock echoes the model's whole answer back inside
+  the guardrail trace; on a long answer that alone overran the old cap, so the
+  payload was stored as a truncated string and the `inputAssessment` naming the
+  policy that fired was never reached. A real deployment lost a `PROMPT_ATTACK`
+  block that way. The removal is recorded under `rootmarkOmitted`, so a reader
+  can tell a pruned trace from one Bedrock never sent the key for.
+  **If you read `extra.bedrock_guardrail_trace` expecting `modelOutput`, read
+  the model's response from the span's outputs instead.**
+- A non-terminal span reports `duration_ms` of 0 and an `end_time` equal to its
+  `start_time`. **Read the status, not the duration**, to decide whether a span
+  has finished — a 0 ms span is otherwise a plausible-looking lie.
+
+### Compatibility
+
+| | |
+|---|---|
+| Collector | Needs ClickHouse migrations `007`–`009`. Upgrade it **before** the SDK. |
+| Python | 3.12+, unchanged |
+| Wire format | Additive; `row_version` and `waiting_reason` are new fields |
+
+A collector older than this release deduplicates buffered spans on
+`(project_id, span_id)` and keeps the **first** row it sees. With a 0.1.1
+client that first row is the *start* row, so the end row — the one carrying the
+result, the duration and the token counts — is discarded. Runs appear and never
+finish. Where the two rows land in different batches instead, the old storage
+engine has no version column to order them by and the survivor is undefined.
+
+Neither failure reports an error. Upgrade the collector first, or stay on 0.1.0.
+
+### Cost
+
+Each span is now sent at least twice, so **a run produces roughly twice the
+rows** it did on 0.1.0, plus one per heartbeat where heartbeats are enabled.
+The collector collapses the pair when it merges, so stored size settles back;
+what doubles is what crosses the network and passes through ingest. There is no
+flag to turn it off — the alternative is being blind to work in progress, which
+is the whole point of the release.
+
+## 0.1.0
+
+First public release. Span tree recording, framework instrumentation for
+OpenAI, LangChain, CrewAI, Strands and Bedrock via botocore, batched background
+delivery with a spill file and replay-safe batch identity, and
+`suppress_payloads` for collectors outside the trust boundary.
+
+No licence is declared, which means all rights reserved. That is unchanged in
+0.1.1 and remains the thing to fix before this package is genuinely installable
+by anyone outside the organisation.

{rootmark-0.1.0 → rootmark-0.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: rootmark
-Version: 0.1.0
+Version: 0.1.2
 Summary: Rootmark SDK -- agent instrumentation
 Project-URL: Homepage, https://rootmark.dev
 Project-URL: Documentation, https://rootmark.dev/docs/
@@ -51,6 +51,34 @@ pip install rootmark
 
 Requires Python 3.12 or newer.
 
+> **Upgrading from 0.1.0 needs a collector upgrade too.** Since 0.1.1 a span is
+> sent when it *starts* as well as when it ends, so that work in progress is
+> visible at all. A collector older than this release stores the first row it
+> sees for a span and discards the rest — which is the end row, the one
+> carrying the result and the duration. Runs would appear and never finish.
+> Upgrade the collector first (it needs ClickHouse migrations `007`–`009`), or
+> stay on 0.1.0.
+
+> **0.1.1 is broken; use 0.1.2 or later.** In 0.1.1 a span that the SDK opened
+> and then discarded was left looking like it was still running, which held its
+> whole run open permanently. Any Bedrock call made through boto3 inside a
+> Strands agent hits it, on every call.
+
+## What changed in 0.1.1
+
+- **Runs are visible while they are still going.** A span is recorded when it
+  starts, not only when it finishes.
+- **`rootmark.mark_waiting(span, reason)`** distinguishes a step paused on a
+  human from one wedged against a model that stopped answering.
+- **`heartbeat=`** on `rootmark.span()`, so a long step keeps saying it is alive.
+- **Steps**: `step_key` and `step_kind` name which logical step a span is, so
+  the collector can aggregate the same step across runs.
+- Bedrock guardrail traces no longer carry `modelOutput`, and the size cap rose
+  from 4 KB to 16 KB. A long answer used to overrun the cap and take the policy
+  assessment with it. Read the model's response from the span's outputs instead.
+
+See `CHANGELOG.md` for the full list.
+
 ## Quickstart
 
 ```python
@@ -85,6 +113,49 @@ agent loop spawns, because the binding follows the async context.
 `workflow_name` should not be `"default"`: that value lets framework
 instrumentation override the name you bind per run with its own agent id.
 
+## Naming the steps of a run
+
+```python
+with rootmark.span("fetch_clauses", run_type=RunType.TOOL,
+                   step_key="fetch_clauses", step_kind="retrieval"):
+    ...
+```
+
+`step_key` is a stable identity for *which* logical step this is, constant
+across runs; `step_kind` is what it was for (`retrieval`, `generation`,
+`routing`, `review`, `tool`, `memory`). Together they let the collector answer
+"which step is slow" without anyone reading individual traces.
+
+Both are optional, and a kind the SDK does not recognise is kept rather than
+dropped — your vocabulary is allowed to be ahead of ours. Derive `step_key`
+from something stable: a key built from a timestamp or a request id fragments
+its own aggregate, and nothing can detect that for you.
+
+## Long-running steps
+
+```python
+with rootmark.span("await_approval", run_type=RunType.TOOL, heartbeat=30) as span:
+    rootmark.mark_waiting(span, "human_review")
+    decision = wait_for_approval()      # minutes, or hours
+```
+
+`mark_waiting` says the step is deliberately not working, and what it is
+waiting on. The reason is free text in your own vocabulary — `human_review`,
+`rate_limit`, `upstream` — and is shown back verbatim. It is the difference
+between a run somebody has to investigate and one they can leave alone.
+
+`heartbeat` re-announces the span every N seconds while it runs. Leave it unset
+for anything that finishes quickly: a span is already visible from the moment
+it starts, and a heartbeat for a 200 ms call is bandwidth spent to say nothing.
+Set it on the long ones, where the useful question is not "is it running" but
+"is anyone still on the other end" — without it, a step whose process was
+killed looks exactly like one that is simply taking its time. Intervals below
+five seconds are raised to five rather than refused.
+
+**This roughly doubles the rows a run produces**, since each span is now sent
+at least twice. The collector collapses the pair when it merges, so stored size
+settles back; what doubles is what crosses the network.
+
 ## Framework support
 
 The base install instruments nothing on its own beyond direct use. Install the

rootmark-0.1.2/README.md

@@ -0,0 +1,167 @@
+# Rootmark SDK
+
+Agent instrumentation for [Rootmark](https://rootmark.dev) — observe, verify and
+govern agent runs.
+
+The SDK records what your agents actually did: the span tree of a run, the model
+calls inside it, token counts, and the payloads you choose to keep. It sends
+them to a Rootmark collector, which you can self-host.
+
+```bash
+pip install rootmark
+```
+
+Requires Python 3.12 or newer.
+
+> **Upgrading from 0.1.0 needs a collector upgrade too.** Since 0.1.1 a span is
+> sent when it *starts* as well as when it ends, so that work in progress is
+> visible at all. A collector older than this release stores the first row it
+> sees for a span and discards the rest — which is the end row, the one
+> carrying the result and the duration. Runs would appear and never finish.
+> Upgrade the collector first (it needs ClickHouse migrations `007`–`009`), or
+> stay on 0.1.0.
+
+> **0.1.1 is broken; use 0.1.2 or later.** In 0.1.1 a span that the SDK opened
+> and then discarded was left looking like it was still running, which held its
+> whole run open permanently. Any Bedrock call made through boto3 inside a
+> Strands agent hits it, on every call.
+
+## What changed in 0.1.1
+
+- **Runs are visible while they are still going.** A span is recorded when it
+  starts, not only when it finishes.
+- **`rootmark.mark_waiting(span, reason)`** distinguishes a step paused on a
+  human from one wedged against a model that stopped answering.
+- **`heartbeat=`** on `rootmark.span()`, so a long step keeps saying it is alive.
+- **Steps**: `step_key` and `step_kind` name which logical step a span is, so
+  the collector can aggregate the same step across runs.
+- Bedrock guardrail traces no longer carry `modelOutput`, and the size cap rose
+  from 4 KB to 16 KB. A long answer used to overrun the cap and take the policy
+  assessment with it. Read the model's response from the span's outputs instead.
+
+See `CHANGELOG.md` for the full list.
+
+## Quickstart
+
+```python
+import rootmark
+
+rootmark.init(
+    api_key="pk_...",                 # project API key
+    project_id="...",                 # UUID
+    api_key_id="...",                 # UUID
+    api_url="https://rootmark.example.com",
+    workflow_name="my-service",       # not "default" -- see below
+)
+
+rootmark.patch()                      # instrument supported frameworks in place
+```
+
+That is the whole integration for the common case. `patch()` installs
+instrumentation process-wide, so call it once at startup and before the first
+agent invocation.
+
+## Grouping and labelling runs
+
+```python
+with rootmark.workflow_override("contract-review"):
+    with rootmark.bind_context(tenant_id="acme", session_id=session, tags=["surface:chat"]):
+        result = agent(prompt)
+```
+
+`bind_context` applies to everything that runs inside it, including tasks the
+agent loop spawns, because the binding follows the async context.
+
+`workflow_name` should not be `"default"`: that value lets framework
+instrumentation override the name you bind per run with its own agent id.
+
+## Naming the steps of a run
+
+```python
+with rootmark.span("fetch_clauses", run_type=RunType.TOOL,
+                   step_key="fetch_clauses", step_kind="retrieval"):
+    ...
+```
+
+`step_key` is a stable identity for *which* logical step this is, constant
+across runs; `step_kind` is what it was for (`retrieval`, `generation`,
+`routing`, `review`, `tool`, `memory`). Together they let the collector answer
+"which step is slow" without anyone reading individual traces.
+
+Both are optional, and a kind the SDK does not recognise is kept rather than
+dropped — your vocabulary is allowed to be ahead of ours. Derive `step_key`
+from something stable: a key built from a timestamp or a request id fragments
+its own aggregate, and nothing can detect that for you.
+
+## Long-running steps
+
+```python
+with rootmark.span("await_approval", run_type=RunType.TOOL, heartbeat=30) as span:
+    rootmark.mark_waiting(span, "human_review")
+    decision = wait_for_approval()      # minutes, or hours
+```
+
+`mark_waiting` says the step is deliberately not working, and what it is
+waiting on. The reason is free text in your own vocabulary — `human_review`,
+`rate_limit`, `upstream` — and is shown back verbatim. It is the difference
+between a run somebody has to investigate and one they can leave alone.
+
+`heartbeat` re-announces the span every N seconds while it runs. Leave it unset
+for anything that finishes quickly: a span is already visible from the moment
+it starts, and a heartbeat for a 200 ms call is bandwidth spent to say nothing.
+Set it on the long ones, where the useful question is not "is it running" but
+"is anyone still on the other end" — without it, a step whose process was
+killed looks exactly like one that is simply taking its time. Intervals below
+five seconds are raised to five rather than refused.
+
+**This roughly doubles the rows a run produces**, since each span is now sent
+at least twice. The collector collapses the pair when it merges, so stored size
+settles back; what doubles is what crosses the network.
+
+## Framework support
+
+The base install instruments nothing on its own beyond direct use. Install the
+extra for what you actually run:
+
+```bash
+pip install "rootmark[openai]"    # OpenAI 1.x and 2.x
+pip install "rootmark[langchain]" # LangChain callback handler
+pip install "rootmark[crewai]"    # CrewAI event listeners
+pip install "rootmark[strands]"   # Strands agents / AWS Bedrock
+pip install "rootmark[boto3]"     # Bedrock via botocore, no code change
+```
+
+For boto3, a session built by hand rather than the default one needs
+`rootmark.patch_session(session)` — the botocore hooks only reach the default
+session otherwise.
+
+## Delivery
+
+Spans are batched on a background thread and flushed on a timer, so nothing
+blocks a request. A failed export is written to a spill file and retried,
+keeping its batch identity so the collector can recognise a replay rather than
+double-count it. Ordering is preserved: a batch that cannot be delivered holds
+back the ones behind it.
+
+Point `spill_file` somewhere writable. The default is relative to the working
+directory, which in a container is frequently owned by root while the process
+runs as someone else — and a spill file that cannot be written is buffering you
+silently do not have.
+
+Call `rootmark.flush()` before a process exits if you want outstanding spans
+delivered, and `rootmark.shutdown()` to stop the processor.
+
+## Keeping payloads out
+
+```python
+rootmark.init(..., suppress_payloads=True)
+```
+
+Spans are still recorded — timings, token counts, the shape of the run — but
+prompt and completion text is not sent. Use this when the collector sits outside
+the trust boundary that the data does not leave.
+
+## Links
+
+- Documentation: <https://rootmark.dev/docs/>
+- Source: <https://github.com/IvanAdactic/rootmark>

{rootmark-0.1.0 → rootmark-0.1.2}/pyproject.toml

@@ -1,10 +1,12 @@
 [project]
 name = "rootmark"
-version = "0.1.0"
+version = "0.1.2"
 description = "Rootmark SDK -- agent instrumentation"
 readme = "README.md"
 requires-python = ">=3.12"
-# No licence declared for 0.1.0; to be added in a later release.
+# No licence declared as of 0.1.2; to be added in a later release. Without
+# one the package defaults to all rights reserved -- see the 0.1.0 release
+# commit (329e16c) for the reasoning and why it still needs revisiting.
 keywords = ["agents", "observability", "tracing", "llm", "instrumentation"]
 classifiers = [
     "Development Status :: 4 - Beta",