nodloop 0.9.0__tar.gz → 0.10.0__tar.gz

{nodloop-0.9.0 → nodloop-0.10.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: nodloop
-Version: 0.9.0
+Version: 0.10.0
 Summary: Record AI runs and the verdicts people give on them, and hand the knowledge they approve to the next run in the same place
 Project-URL: Homepage, https://github.com/jeon-jihyeon/nodloop
 Project-URL: Repository, https://github.com/jeon-jihyeon/nodloop
@@ -41,23 +41,22 @@ async with Client() as c:
     run = await c.record("support-bot", answer, labels=acme, applied=knowledge.items)
 
     # later, when a person reads the answer
-    await c.judge(run, "reject", reason="the refund window was missing", reason_code="scope")
+    await c.judge(run, "reject", "ann", reason="the refund window was missing", reason_code="scope")
     candidate = await c.propose("judgment", "Quote the refund window before the steps", from_run=run)
-    approval = await c.approve(candidate.id, candidate.version, approver="ann")
 ```
 
-From then on every `knowledge("support-bot", acme)` carries the item, and a run of another tenant never does. `judge` takes `approve`, `edit` with `edited=` holding the corrected output, `reject` with what was wrong, or `withdraw` to take back the verdict before it.
+The person approves the candidate in a terminal of their own with `nodloop knowledge approve <id> --version <n>`, or the client approves it through a nodloop server under an approver key the person holds, added with `--person`. Over stdio `approve` records a proposal and changes nothing, since a program proves no person. From then on every `knowledge("support-bot", acme)` carries the item, and a run of another tenant never does. `judge` takes `approve`, `edit` with `edited=` holding the corrected output, `reject` with what was wrong, or `withdraw` to take back the verdict before it, and always the name of the reviewer.
 
 | Method | Returns |
 |---|---|
 | `knowledge(producer, labels, holdout_key)` | `Knowledge` with `items`, the `text` to put in the prompt and `withheld`. With a `holdout_key` such as `<service>:<hour>`, the `holdout` share of config.json gets no items and `withheld` instead |
-| `record(producer, output, *, labels, applied, subject, withheld, holdout_key)` | the run id a verdict cites. Pass `knowledge.withheld` as `withheld` and the `holdout_key` the run passed to `knowledge`. `report effect` counts only runs recorded with the key and counts a run without it as `undrawn`. `withheld` takes `Ref` or `Item` values |
-| `judge(run, verdict, reason, reason_code, edited, reviewer)` | nothing |
+| `record(producer, output, *, labels, applied, subject, withheld, holdout_key, model)` | the run id a verdict cites. Pass `knowledge.withheld` as `withheld` and the `holdout_key` the run passed to `knowledge`. `report effect` counts only runs recorded with the key and counts a run without it as `undrawn`. `withheld` takes `Ref` or `Item` values. `model` names the one model the producer runs, which the OTLP export sends as `gen_ai.request.model`. Leave it out when the producer may switch models |
+| `judge(run, verdict, reviewer, reason, reason_code, edited)` | nothing. `reviewer` is required, so a verdict without a name never reads as a person's |
 | `propose(kind, content, from_run, producer, labels, new_labels, item_id, veto)` | `Candidate` with `id`, `version`, `status` and `overlaps`, the current items of the same kind one run may carry with it |
-| `approve(item_id, version, approver, labels=)` | Approves as proposed, or for the runs carrying `labels` when given, and `{}` for every run of the producer. `Approval` with `id`, `version`, `status`, `approver`, `veto`, `compaction_due` when one run would carry more than five items, and `export_error` and `folder_error` when a step after the approval failed |
+| `approve(item_id, version, approver, labels=)` | Under a key of a nodloop server that a person holds, approves as proposed, or for the runs carrying `labels` when given, and `{}` for every run of the producer. Anywhere else it records a proposal the person is asked about and the status stays. `Approval` with `id`, `version`, `status`, `approver`, `veto`, `proposal` when a proposal was recorded, and `export_error` and `folder_error` when a step after the approval failed |
 | `check_call(tool, arguments, producer)` | `Decision` with `action` of allow, block or ask, `veto` and `reason` |
-| `outcome(run, result, confirmed_cause, note, reviewer)` | nothing. `result` is a `Result`, confirmed, refuted or inconclusive: what a real check found, apart from the verdict on the output |
-| `report(name)` | `Report` with `report`, the report as JSON, and `since`, the RFC3339 start of the runs it read or empty for every run. `name` is a `ReportName`: `loop`, `extract`, `critic`, `effect`, `health` or `replay` |
+| `outcome(run, result, confirmed_cause, note, reviewer)` | nothing. `result` is a `Result`, confirmed, refuted or inconclusive: what a real check found, apart from the verdict on the output. Over stdio `reviewer` is required |
+| `report(name)` | `Report` with `report`, the report as JSON, and `since`, the RFC3339 start of the runs it read or empty for every run. `name` is a `ReportName`: `loop`, `extract`, `critic`, `effect`, `health` or `replay`. `critic` is `{critics, reactions}`: the rows of the draft critics and of the reaction classifiers |
 
 A tool that refuses, such as a proposal from a run nobody corrected, raises `ToolError` with the reason.
 
@@ -79,7 +78,7 @@ run = await c.record(
 )
 
 # later, when the cause was checked
-await c.outcome(run, "confirmed", confirmed_cause="partition 7 was hot")
+await c.outcome(run, "confirmed", confirmed_cause="partition 7 was hot", reviewer="ann")
 effect = (await c.report("effect")).report
 ```
 
@@ -95,7 +94,7 @@ from nodloop.langchain import NodloopMiddleware
 mw = NodloopMiddleware(c, "support-bot", acme)
 agent = create_agent(model, tools, middleware=[mw])
 await agent.ainvoke({"messages": [...]})
-await c.judge(mw.run, "approve")
+await c.judge(mw.run, "approve", "ann")
 ```
 
 OpenAI Agents SDK hooks and a tool input guardrail:

{nodloop-0.9.0 → nodloop-0.10.0}/README.md

@@ -23,23 +23,22 @@ async with Client() as c:
     run = await c.record("support-bot", answer, labels=acme, applied=knowledge.items)
 
     # later, when a person reads the answer
-    await c.judge(run, "reject", reason="the refund window was missing", reason_code="scope")
+    await c.judge(run, "reject", "ann", reason="the refund window was missing", reason_code="scope")
     candidate = await c.propose("judgment", "Quote the refund window before the steps", from_run=run)
-    approval = await c.approve(candidate.id, candidate.version, approver="ann")
 ```
 
-From then on every `knowledge("support-bot", acme)` carries the item, and a run of another tenant never does. `judge` takes `approve`, `edit` with `edited=` holding the corrected output, `reject` with what was wrong, or `withdraw` to take back the verdict before it.
+The person approves the candidate in a terminal of their own with `nodloop knowledge approve <id> --version <n>`, or the client approves it through a nodloop server under an approver key the person holds, added with `--person`. Over stdio `approve` records a proposal and changes nothing, since a program proves no person. From then on every `knowledge("support-bot", acme)` carries the item, and a run of another tenant never does. `judge` takes `approve`, `edit` with `edited=` holding the corrected output, `reject` with what was wrong, or `withdraw` to take back the verdict before it, and always the name of the reviewer.
 
 | Method | Returns |
 |---|---|
 | `knowledge(producer, labels, holdout_key)` | `Knowledge` with `items`, the `text` to put in the prompt and `withheld`. With a `holdout_key` such as `<service>:<hour>`, the `holdout` share of config.json gets no items and `withheld` instead |
-| `record(producer, output, *, labels, applied, subject, withheld, holdout_key)` | the run id a verdict cites. Pass `knowledge.withheld` as `withheld` and the `holdout_key` the run passed to `knowledge`. `report effect` counts only runs recorded with the key and counts a run without it as `undrawn`. `withheld` takes `Ref` or `Item` values |
-| `judge(run, verdict, reason, reason_code, edited, reviewer)` | nothing |
+| `record(producer, output, *, labels, applied, subject, withheld, holdout_key, model)` | the run id a verdict cites. Pass `knowledge.withheld` as `withheld` and the `holdout_key` the run passed to `knowledge`. `report effect` counts only runs recorded with the key and counts a run without it as `undrawn`. `withheld` takes `Ref` or `Item` values. `model` names the one model the producer runs, which the OTLP export sends as `gen_ai.request.model`. Leave it out when the producer may switch models |
+| `judge(run, verdict, reviewer, reason, reason_code, edited)` | nothing. `reviewer` is required, so a verdict without a name never reads as a person's |
 | `propose(kind, content, from_run, producer, labels, new_labels, item_id, veto)` | `Candidate` with `id`, `version`, `status` and `overlaps`, the current items of the same kind one run may carry with it |
-| `approve(item_id, version, approver, labels=)` | Approves as proposed, or for the runs carrying `labels` when given, and `{}` for every run of the producer. `Approval` with `id`, `version`, `status`, `approver`, `veto`, `compaction_due` when one run would carry more than five items, and `export_error` and `folder_error` when a step after the approval failed |
+| `approve(item_id, version, approver, labels=)` | Under a key of a nodloop server that a person holds, approves as proposed, or for the runs carrying `labels` when given, and `{}` for every run of the producer. Anywhere else it records a proposal the person is asked about and the status stays. `Approval` with `id`, `version`, `status`, `approver`, `veto`, `proposal` when a proposal was recorded, and `export_error` and `folder_error` when a step after the approval failed |
 | `check_call(tool, arguments, producer)` | `Decision` with `action` of allow, block or ask, `veto` and `reason` |
-| `outcome(run, result, confirmed_cause, note, reviewer)` | nothing. `result` is a `Result`, confirmed, refuted or inconclusive: what a real check found, apart from the verdict on the output |
-| `report(name)` | `Report` with `report`, the report as JSON, and `since`, the RFC3339 start of the runs it read or empty for every run. `name` is a `ReportName`: `loop`, `extract`, `critic`, `effect`, `health` or `replay` |
+| `outcome(run, result, confirmed_cause, note, reviewer)` | nothing. `result` is a `Result`, confirmed, refuted or inconclusive: what a real check found, apart from the verdict on the output. Over stdio `reviewer` is required |
+| `report(name)` | `Report` with `report`, the report as JSON, and `since`, the RFC3339 start of the runs it read or empty for every run. `name` is a `ReportName`: `loop`, `extract`, `critic`, `effect`, `health` or `replay`. `critic` is `{critics, reactions}`: the rows of the draft critics and of the reaction classifiers |
 
 A tool that refuses, such as a proposal from a run nobody corrected, raises `ToolError` with the reason.
 
@@ -61,7 +60,7 @@ run = await c.record(
 )
 
 # later, when the cause was checked
-await c.outcome(run, "confirmed", confirmed_cause="partition 7 was hot")
+await c.outcome(run, "confirmed", confirmed_cause="partition 7 was hot", reviewer="ann")
 effect = (await c.report("effect")).report
 ```
 
@@ -77,7 +76,7 @@ from nodloop.langchain import NodloopMiddleware
 mw = NodloopMiddleware(c, "support-bot", acme)
 agent = create_agent(model, tools, middleware=[mw])
 await agent.ainvoke({"messages": [...]})
-await c.judge(mw.run, "approve")
+await c.judge(mw.run, "approve", "ann")
 ```
 
 OpenAI Agents SDK hooks and a tool input guardrail:

{nodloop-0.9.0 → nodloop-0.10.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "nodloop"
-version = "0.9.0"
+version = "0.10.0"
 description = "Record AI runs and the verdicts people give on them, and hand the knowledge they approve to the next run in the same place"
 readme = "README.md"
 license = "MIT"

nodloop-0.10.0/src/nodloop/__init__.py

@@ -0,0 +1,38 @@
+"""Record AI runs and the verdicts people give on them, and hand the knowledge they approve to the next run in the same place"""
+
+from .client import (
+    Approval,
+    Candidate,
+    Client,
+    Decision,
+    Intervention,
+    InterventionAction,
+    Item,
+    Knowledge,
+    Page,
+    Ref,
+    Report,
+    ReportName,
+    Result,
+    ToolError,
+)
+from .records import AnyRecord, RecordKind
+
+__all__ = [
+    "AnyRecord",
+    "Approval",
+    "Candidate",
+    "Client",
+    "Decision",
+    "Intervention",
+    "InterventionAction",
+    "Item",
+    "Knowledge",
+    "Page",
+    "RecordKind",
+    "Ref",
+    "Report",
+    "ReportName",
+    "Result",
+    "ToolError",
+]

{nodloop-0.9.0 → nodloop-0.10.0}/src/nodloop/claude_agent.py

@@ -7,6 +7,7 @@
 
 from __future__ import annotations
 
+import time
 from typing import Any
 
 from claude_agent_sdk import HookMatcher
@@ -34,6 +35,8 @@ class NodloopHooks:
         self.withheld: list[Ref] = []
         # The run the last answer was recorded as so a verdict can cite it
         self.run: str = ""
+        # When the prompt of the turn came so the run records how long the turn took
+        self._started = time.monotonic()
 
     def hooks(self) -> dict[str, list[HookMatcher]]:
         return {
@@ -43,6 +46,7 @@ class NodloopHooks:
         }
 
     async def prompt(self, input_data: dict[str, Any], tool_use_id: str | None, context: Any) -> dict[str, Any]:
+        self._started = time.monotonic()
         knowledge = await self.client.knowledge(self.producer, self.labels, self.holdout_key)
         self.applied, self.withheld = knowledge.items, knowledge.withheld
         if not knowledge.text:
@@ -55,7 +59,13 @@ class NodloopHooks:
         answer = input_data.get("last_assistant_message") or ""
         if answer.strip():
             self.run = await self.client.record(
-                self.producer, answer, labels=self.labels, applied=self.applied, withheld=self.withheld, holdout_key=self.holdout_key
+                self.producer,
+                answer,
+                labels=self.labels,
+                applied=self.applied,
+                withheld=self.withheld,
+                holdout_key=self.holdout_key,
+                duration_ms=int((time.monotonic() - self._started) * 1000),
             )
         return {}
 

{nodloop-0.9.0 → nodloop-0.10.0}/src/nodloop/client.py

@@ -13,16 +13,37 @@ from mcp.client.stdio import stdio_client
 from mcp.client.streamable_http import create_mcp_http_client, streamable_http_client
 
 from . import binary
-
-__all__ = ["Approval", "Candidate", "Client", "Decision", "Item", "Knowledge", "Ref", "Report", "ReportName", "Result", "ToolError"]
+from .records import AnyRecord, InterventionRecord, RecordKind
+
+__all__ = [
+    "Approval",
+    "Candidate",
+    "Client",
+    "Decision",
+    "Intervention",
+    "InterventionAction",
+    "Item",
+    "Knowledge",
+    "Page",
+    "Ref",
+    "Report",
+    "ReportName",
+    "Result",
+    "ToolError",
+]
 
 # The binary release the client fetches when no nodloop is installed
-VERSION = "0.9.0"
+VERSION = "0.10.0"
 
 
 # The reports report reads
+# Deprecated: page the raw records with records instead since they carry the published contract
 ReportName = Literal["loop", "extract", "critic", "effect", "health", "replay"]
 
+# The actions of the write side
+# A test checks them against the generated record type of an intervention
+InterventionAction = Literal["judge", "answer", "dismiss", "pause", "resume", "knob", "reanchor", "digest", "reverse"]
+
 # What a real check found for a run
 Result = Literal["confirmed", "refuted", "inconclusive"]
 
@@ -80,21 +101,22 @@ class Candidate:
 
 @dataclass(frozen=True)
 class Approval:
-    """An approved knowledge version"""
+    """An approved knowledge version or the agent proposal of a call that proves no person while the version stays as it was"""
 
     id: str
     version: int
     status: str
     # The approver the records keep, which a server sets to the key's name
     approver: str
-    # Whether the approved version carries a veto the guard now applies
+    # Whether the version carries a veto the guard applies once approved
     veto: bool = False
-    # Whether one run would carry more than five items so a compaction is due
-    compaction_due: bool = False
     # Why approved.md or the veto file could not be written, while the approval itself stands
     export_error: str = ""
     # Why the items one run carries with it could not be read, while the approval itself stands
     folder_error: str = ""
+    # The intervention recorded as an agent proposal when the call proved no person
+    # Empty when it approved
+    proposal: str = ""
 
 
 @dataclass(frozen=True)
@@ -112,7 +134,10 @@ class Knowledge:
 
 @dataclass(frozen=True)
 class Report:
-    """One report of the records with the start of the runs it read"""
+    """One report of the records with the start of the runs it read
+
+    Deprecated: page the raw records with records instead since they carry the published contract
+    """
 
     # The report as JSON
     report: Any
@@ -120,6 +145,39 @@ class Report:
     since: str = ""
 
 
+@dataclass(frozen=True)
+class Page:
+    """One page of raw records in the published schema of their kind, in the order they were appended"""
+
+    # Each record as the TypedDict of its kind in nodloop.records
+    records: list[AnyRecord]
+    # The cursor to pass as since for the next page
+    # A page with nothing new still answers one to pass again
+    next: str = ""
+
+
+@dataclass(frozen=True)
+class Intervention:
+    """What one intervention of the write side recorded"""
+
+    # The intervention record in its published schema and None when the person declined the form
+    intervention: InterventionRecord | None
+    # person when a key a person holds decided and agent for a proposal the person is asked about
+    authority: str = ""
+    proof: str = ""
+    # The knowledge records it wrote, each with id, version and status
+    records: list[dict[str, Any]] = field(default_factory=list)
+    # The answer event an answer wrote
+    answer: str = ""
+    # The cursor of a digest to pass as since to the records of answer, intervention, decide, expose and knowledge
+    since: str = ""
+    # The decide that asks the person about a proposal
+    question: str = ""
+    # The person declined the form so nothing was recorded
+    declined: bool = False
+    export_error: str = ""
+
+
 def _ref(v: Item | Ref | dict[str, Any]) -> dict[str, Any]:
     """The id and version the run tool takes from an item or a ref"""
     if isinstance(v, dict):
@@ -198,6 +256,7 @@ class Client:
         holdout_key is a string kept for one run such as a service and an hour block and the holdout share of config.json draws on it
         A drawn run gets no item and withheld instead
         Without a key nothing is withheld
+        the correlation label may name an external conversation without becoming inferred knowledge scope
         """
         args: dict[str, Any] = {"producer": producer, "labels": labels or {}}
         if holdout_key:
@@ -217,6 +276,8 @@ class Client:
         subject: str = "",
         withheld: list[Item | Ref] | None = None,
         holdout_key: str = "",
+        model: str = "",
+        duration_ms: int = 0,
     ) -> str:
         """Records one output and returns the run id a verdict cites
 
@@ -224,6 +285,10 @@ class Client:
         Never together with applied
         holdout_key is the key the run passed to knowledge
         report effect counts a run without it only as undrawn
+        the correlation label preserves an external id independently of the returned run id and holdout_key
+        model is the one model the producer runs and the OTLP export names it
+        Leave it out when the producer may switch models
+        duration_ms is how long the run took up to this call so its OTLP span has a length
         """
         args: dict[str, Any] = {"producer": producer, "output": output, "labels": labels or {}}
         if applied:
@@ -234,18 +299,29 @@ class Client:
             args["subject"] = subject
         if holdout_key:
             args["holdout_key"] = holdout_key
+        if model:
+            args["model"] = model
+        if duration_ms:
+            args["duration_ms"] = duration_ms
         return (await self.call("run", args))["trace_id"]
 
     async def judge(
         self,
         run: str,
         verdict: str,
+        *,
+        reviewer: str,
         reason: str = "",
         reason_code: str = "",
         edited: Any = None,
-        reviewer: str = "",
     ) -> None:
-        """Records a verdict on a run: approve, edit with the corrected output, reject with what was wrong, or withdraw an earlier one"""
+        """Records a verdict on a run: approve, edit with the corrected output, reject with what was wrong, or withdraw an earlier one
+
+        reviewer names who judged so a verdict without a name never reads as a person's
+        nodloop is refused since only the capture of a session records verdicts under it
+        Every argument after verdict is keyword only
+        So a call of an earlier release that passed the reason third fails instead of naming it the reviewer
+        """
         args: dict[str, Any] = {"trace_id": run, "verdict": verdict}
         for key, value in (("reason", reason), ("reason_code", reason_code), ("reviewer", reviewer)):
             if value:
@@ -259,6 +335,7 @@ class Client:
 
         A different fact from the verdict on the output
         confirmed_cause only goes with confirmed
+        reviewer names who checked and is required over stdio while a server key names itself
         """
         args: dict[str, Any] = {"trace_id": run, "result": result}
         for key, value in (("confirmed_cause", confirmed_cause), ("note", note), ("reviewer", reviewer)):
@@ -267,10 +344,31 @@ class Client:
         await self.call("outcome", args)
 
     async def report(self, name: ReportName) -> Report:
-        """One report of the records as JSON with the start of the window it read"""
+        """One report of the records as JSON with the start of the window it read
+
+        Deprecated: page the raw records with records instead since they carry the published contract
+        report is removed in the next release
+        """
         answer = await self.call("report", {"name": name})
         return Report(answer["report"], answer.get("since", ""))
 
+    async def records(self, kind: RecordKind, since: str = "", limit: int = 0, content: bool = False) -> Page:
+        """One page of the raw records of a kind in its published schema, in the order they were appended after the cursor since
+
+        Pass next back as since for the following page, so a record appended late with an earlier time still comes once
+        limit is 100 by default and at most 1000
+        The free text of the kind such as outputs, reasons and lesson content is left out unless content is true
+        """
+        args: dict[str, Any] = {"kind": kind}
+        if since:
+            args["since"] = since
+        if limit:
+            args["limit"] = limit
+        if content:
+            args["content"] = True
+        answer = await self.call("records", args)
+        return Page(answer.get("records") or [], answer.get("next", ""))
+
     async def propose(
         self,
         kind: str,
@@ -300,8 +398,9 @@ class Client:
         return Candidate(answer["id"], answer["version"], answer["status"], answer.get("overlaps") or [])
 
     async def approve(self, item_id: str, version: int, approver: str, *, labels: dict[str, list[str]] | None = None) -> Approval:
-        """Approves a candidate on behalf of the named person
+        """Approves a candidate under a key of nodloop-server that a person holds
 
+        Any other call such as one over stdio records an agent proposal the person is asked about: the status stays and proposal names it
         labels approves it for the runs carrying them instead of the proposed scope and an empty mapping for every run of its producer
         """
         args: dict[str, Any] = {"id": item_id, "version": version, "approver": approver}
@@ -314,9 +413,48 @@ class Client:
             answer["status"],
             answer["approver"],
             veto=bool(answer.get("veto")),
-            compaction_due=bool((answer.get("folder") or {}).get("compaction_due")),
             export_error=answer.get("export_error", ""),
             folder_error=answer.get("folder_error", ""),
+            proposal=answer.get("proposal", ""),
+        )
+
+    async def intervene(
+        self,
+        action: InterventionAction,
+        target: str = "",
+        *,
+        choice: str = "",
+        labels: dict[str, list[str]] | None = None,
+        value: float | None = None,
+        session: str = "",
+    ) -> Intervention:
+        """Submits one intervention of the write side: judge, answer, dismiss, pause, resume, knob, reanchor, digest or reverse
+
+        target is an item version as id:version, a compaction, the decide of a question, the change kind of a pause,
+        the pause a resume ends, a knob such as audit_tier1, the producer of a place or the intervention a reverse takes back
+        Under a key of nodloop-server that a person holds it is the person's decision at once
+        Any other call such as one over stdio records an agent proposal the person is asked about and changes nothing
+        A digest needs no proof and answers since, the cursor after the person's last decision
+        """
+        args: dict[str, Any] = {"action": action}
+        for key, given in (("target", target), ("choice", choice), ("session", session)):
+            if given:
+                args[key] = given
+        if labels is not None:
+            args["labels"] = labels
+        if value is not None:
+            args["value"] = value
+        answer = await self.call("intervene", args)
+        return Intervention(
+            answer.get("intervention"),
+            authority=answer.get("authority", ""),
+            proof=answer.get("proof", ""),
+            records=answer.get("records") or [],
+            answer=answer.get("answer", ""),
+            since=answer.get("since", ""),
+            question=answer.get("question", ""),
+            declined=bool(answer.get("declined")),
+            export_error=answer.get("export_error", ""),
         )
 
     async def check_call(self, tool: str, arguments: dict[str, Any], producer: str = "") -> Decision:

{nodloop-0.9.0 → nodloop-0.10.0}/src/nodloop/langchain.py

@@ -7,6 +7,7 @@
 
 from __future__ import annotations
 
+import time
 from collections.abc import Callable
 from typing import Any
 
@@ -43,12 +44,15 @@ class NodloopMiddleware(AgentMiddleware):
         self._knowledge: Knowledge | None = None
         # The run the final answer was recorded as so a verdict can cite it
         self.run: str = ""
+        # When the first model call of the run came so the run records how long it took
+        self._started = time.monotonic()
 
     def _labels(self, state: Any) -> Labels:
         return self.labels(state) if callable(self.labels) else self.labels
 
     async def awrap_model_call(self, request: Any, handler: Callable[[Any], Any]) -> Any:
         if self._knowledge is None:
+            self._started = time.monotonic()
             self._knowledge = await self.client.knowledge(self.producer, self._labels(request.state), self.holdout_key)
             self.applied, self.withheld = self._knowledge.items, self._knowledge.withheld
         text = self._knowledge.text
@@ -66,6 +70,7 @@ class NodloopMiddleware(AgentMiddleware):
                 applied=self.applied,
                 withheld=self.withheld,
                 holdout_key=self.holdout_key,
+                duration_ms=int((time.monotonic() - self._started) * 1000),
             )
         self._knowledge = None
 

{nodloop-0.9.0 → nodloop-0.10.0}/src/nodloop/openai_agents.py

@@ -8,6 +8,7 @@
 from __future__ import annotations
 
 import json
+import time
 from collections.abc import Callable
 from typing import Any
 
@@ -34,11 +35,14 @@ class NodloopHooks(RunHooks):
         self.withheld: list[Ref] = []
         # The run the final output was recorded as so a verdict can cite it
         self.run: str = ""
+        # When the instructions were built so the run records how long it took
+        self._started = time.monotonic()
 
     def instructions(self, base: str) -> Callable[[Any, Any], Any]:
         """Dynamic instructions: the base and the approved items under it"""
 
         async def build(context: Any, agent: Any) -> str:
+            self._started = time.monotonic()
             knowledge = await self.client.knowledge(self.producer, self.labels, self.holdout_key)
             self.applied, self.withheld = knowledge.items, knowledge.withheld
             return f"{base}\n\n{knowledge.text}" if knowledge.text else base
@@ -48,7 +52,13 @@ class NodloopHooks(RunHooks):
     async def on_agent_end(self, context: Any, agent: Any, output: Any) -> None:
         text = output if isinstance(output, str) else json.dumps(output, default=str)
         self.run = await self.client.record(
-            self.producer, text, labels=self.labels, applied=self.applied, withheld=self.withheld, holdout_key=self.holdout_key
+            self.producer,
+            text,
+            labels=self.labels,
+            applied=self.applied,
+            withheld=self.withheld,
+            holdout_key=self.holdout_key,
+            duration_ms=int((time.monotonic() - self._started) * 1000),
         )