chap-analytics 0.1.0__tar.gz

chap_analytics-0.1.0/CHANGELOG.md

@@ -0,0 +1,32 @@
+# Changelog
+
+`chap-analytics` versions on its own track. It is a reader of the protocol,
+so a protocol release moves the coordinator packages together and leaves this
+one where it is until the tables change.
+
+## 0.1.0
+
+First release: stage one of the analytics roadmap.
+
+- Eleven documented tables, projected from a CHAP audit chain by replaying
+  the envelope stream: `events`, `tasks`, `decisions`, `overrides`,
+  `patch_ops`, `participants`, `deliberations`, `votes`, `whispers`,
+  `handoffs`, `routing`. Every column declared with its dtype and provenance.
+- Four loaders: a SQLite store, a JSON export, an `audit.read` endpoint, and
+  a `Coordinator` in the same process.
+- Review passes kept apart, so a task sent back for revision is measured from
+  each pass's own opening and `outcome` reads from the decision that settled
+  the last one.
+- The corrected artefact reconstructed from the patch with the coordinator's
+  own RFC 6902 semantics, and checked against what the coordinator stored by
+  the differential suite.
+- Successors minted by escalation and supersession counted as tasks, with
+  what they inherit from the original.
+- `id_certain` and `assignee_certain`, so a row says which of its values are
+  inferred.
+- A redactor hook that covers every artefact-bearing field, in both the
+  envelope stream and the snapshot.
+- `chap_analytics.sample.support_desk()`, a generated week of review work; a
+  walkthrough notebook and a printed report over it.
+- A differential suite: random workspaces against a live coordinator, read
+  both ways and checked against what the coordinator holds.

chap_analytics-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for describing the origin of the Work and
+      reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 The CHAP Authors
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

chap_analytics-0.1.0/MANIFEST.in

@@ -0,0 +1,2 @@
+include CHANGELOG.md
+include tests/conftest.py

chap_analytics-0.1.0/PKG-INFO

@@ -0,0 +1,307 @@
+Metadata-Version: 2.4
+Name: chap-analytics
+Version: 0.1.0
+Summary: A CHAP audit chain as documented pandas tables: every task, decision, override, whisper, handoff and vote, ready to analyse
+Author-email: Brightbeam AI <oss@brightbeam.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/BrightbeamAI/chap/tree/main/packages/chap-analytics
+Project-URL: Repository, https://github.com/BrightbeamAI/chap
+Project-URL: Bugs, https://github.com/BrightbeamAI/chap/issues
+Project-URL: Changelog, https://github.com/BrightbeamAI/chap/blob/main/packages/chap-analytics/CHANGELOG.md
+Project-URL: Walkthrough, https://github.com/BrightbeamAI/chap/blob/main/packages/chap-analytics/examples/chap_analytics_walkthrough.ipynb
+Project-URL: Specification, https://github.com/BrightbeamAI/chap/blob/main/SPECIFICATION.md
+Project-URL: Roadmap, https://github.com/BrightbeamAI/chap/blob/main/ANALYTICS_ROADMAP.md
+Keywords: chap,audit,analytics,pandas,human-in-the-loop,agent
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=2.0
+Provides-Extra: coordinator
+Requires-Dist: chap-coordinator>=0.2.13; extra == "coordinator"
+Provides-Extra: notebook
+Requires-Dist: nbformat>=5; extra == "notebook"
+Requires-Dist: nbclient>=0.9; extra == "notebook"
+Requires-Dist: ipykernel>=6; extra == "notebook"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: chap-coordinator>=0.2.13; extra == "dev"
+Requires-Dist: nbformat>=5; extra == "dev"
+Dynamic: license-file
+
+# chap-analytics
+
+[![PyPI](https://img.shields.io/pypi/v/chap-analytics?style=flat-square&logo=pypi&logoColor=white&label=PyPI)](https://pypi.org/project/chap-analytics/)
+[![Python](https://img.shields.io/pypi/pyversions/chap-analytics?style=flat-square)](https://pypi.org/project/chap-analytics/)
+[![Licence](https://img.shields.io/badge/licence-Apache_2.0-7c3aed?style=flat-square)](https://github.com/BrightbeamAI/chap/blob/main/LICENSE)
+
+A CHAP audit chain, as pandas tables.
+
+A CHAP chain records what people decided about agent work: what an agent
+produced, what a person changed, why, under which rule, and when. That is a
+human-labelled evaluation set with provenance, generated as a side effect of
+ordinary review. This package projects the chain into eleven documented tables
+so it can be analysed as one.
+
+It is stage one of the
+[analytics roadmap](https://github.com/BrightbeamAI/chap/blob/main/ANALYTICS_ROADMAP.md)
+and it stops at the tables. Statistics belong in a layer above, where their
+assumptions can be stated.
+
+## Install
+
+```bash
+pip install chap-analytics
+```
+
+Python 3.10 or later. `pandas` is the only dependency. Reading a chain from a
+live `Coordinator` in the same process, or generating the sample week, needs
+`chap-coordinator` as well:
+
+```bash
+pip install 'chap-analytics[coordinator]'
+```
+
+Reading a SQLite file, a JSON export or an HTTP endpoint needs the base
+install alone.
+
+## Quick start
+
+```python
+from chap_analytics import from_sqlite, frames
+
+chain = from_sqlite("./chap.db", workspace="wsp_support")
+f = frames(chain)
+
+print(f.summary())
+
+# Which part of the output do reviewers keep correcting?
+f.patch_ops.groupby("top_path").size().sort_values(ascending=False)
+
+# Does the agent's confidence track its outcomes?
+f.tasks.groupby("outcome")["confidence"].describe()
+
+# Are reviewers refining the agent's decision or reversing it?
+f.overrides.intent_preserved.value_counts(dropna=False)
+
+# Everything, as files.
+f.to_csv("./week")
+```
+
+## A worked week
+
+`chap_analytics.sample.support_desk()` generates a week at a support desk
+against a real coordinator. An agent drafts replies to customer tickets and
+three people review them. They approve most, correct some and send a few back.
+One declares a conflict of interest, work changes hands at a shift change, the
+agent asks two questions and gets one answer, a policy exception goes to a
+vote, and one ticket is escalated to legal. Every action is a CHAP envelope,
+and the chain comes back ready to project.
+
+```python
+from chap_analytics import frames
+from chap_analytics.sample import support_desk
+
+f = frames(support_desk())
+```
+
+Two guided versions of the same week ship with the repository:
+
+- [`examples/chap_analytics_walkthrough.ipynb`](https://github.com/BrightbeamAI/chap/blob/main/packages/chap-analytics/examples/chap_analytics_walkthrough.ipynb)
+  follows the envelopes from `audit.read` to the tables, with each table's
+  grain, columns, dtypes and provenance explained beside the frames.
+- [`examples/support_desk.py`](https://github.com/BrightbeamAI/chap/blob/main/packages/chap-analytics/examples/support_desk.py)
+  prints ten short analyses of the week, each a count or a median with a line
+  on what it is for.
+
+The week is generated on each run, so it reflects the coordinator that is
+installed. The test suite runs both.
+
+## The tables
+
+| Table | Grain |
+|---|---|
+| `events` | one row per audit log entry |
+| `tasks` | one row per task |
+| `decisions` | one row per approve, reject, override or abstain |
+| `overrides` | one row per correction, with its diff summarised |
+| `patch_ops` | one row per RFC 6902 operation within an override |
+| `participants` | one row per participant per workspace |
+| `deliberations` | one row per group decision |
+| `votes` | one row per vote |
+| `whispers` | one row per deadline-bound question |
+| `handoffs` | one row per proposed handoff |
+| `routing` | one row per routing decision |
+
+Every column is declared in
+[`schema.py`](https://github.com/BrightbeamAI/chap/blob/main/packages/chap-analytics/chap_analytics/schema.py)
+with its dtype and its provenance. `describe()` prints the whole contract:
+
+```python
+from chap_analytics import describe
+print(describe())
+```
+
+A column the source lacked a value for is present and null. Code downstream
+can reference any column and see missingness rather than a `KeyError`.
+
+## Four sources
+
+```python
+from chap_analytics import from_sqlite, from_url, from_json, from_coordinator
+
+from_sqlite("./chap.db", workspace="wsp_support")     # the full snapshot
+from_url("http://localhost:8080/chap", "wsp_support") # envelopes, via audit.read
+from_json("export.json")                              # an audit.read result, a bare entry list, or a snapshot
+from_coordinator(coord, workspace="wsp_support")      # a live coordinator, in-process
+```
+
+A SQLite file or a live coordinator carries the workspace snapshot: every
+task, override, deliberation and handoff as the coordinator holds it, and the
+audit log beside them. `audit.read`, which is what an MCP client can obtain,
+returns the audit log alone: every request parameter, in order, and
+hash-linked where the coordinator chains its log.
+
+The tables are **replayed** from the envelope stream, so most of what is worth
+analysing is available from either source. The artefact under review arrives
+on `review.request` and the patch on `decide.override`, so the before and the
+after of every correction come from the envelopes themselves. Three things
+come from the snapshot alone: deliberation and routing outcomes, which the
+server computes; the assignee a `task.route` chose, which `assignee_certain`
+flags; and the certainty of which server-minted id belongs to which creation,
+which the section on identity below explains.
+
+How far the two reads agree is measured. Random workspaces are driven against
+a real coordinator and read both ways. For every task, decision, override,
+whisper, deliberation and handoff, the row either matches what the coordinator
+holds or is marked `id_certain` false.
+
+## What the projection handles for you
+
+**Parses `confidence`.** Fractional values travel the CHAP wire as decimal
+strings, because canonicalisation admits integers alone
+([SPECIFICATION §7](https://github.com/BrightbeamAI/chap/blob/main/SPECIFICATION.md)).
+The tables carry a float.
+
+**Censors open work.** `lifetime_s` is null while a task is unsettled, and
+`settled` says which rows are censored. A mean over the finished work alone
+flatters every latency figure; the flag lets you say so.
+
+**Knows when a decision settled a review.** Under `all_approve` or `quorum:N`
+an approval may leave the review open. `is_final` is computed with the rule the
+coordinator applies. Under `all_approve` the coordinator waits on the reviewers
+it can name, and a review addressed to a group alone has none to wait on, so
+it treats that review as first-approve.
+
+**Separates the review passes.** A task sent back for revision is reviewed
+again, and the coordinator starts that review afresh. `review_index` says which
+pass a decision belongs to, `n_reviews` counts the passes, `latency_s` is
+measured from each pass's own opening, and `outcome` is read from the decision
+that settled the last pass.
+
+**Reconstructs the correction.** `based_on` is the artefact the reviewer saw
+and `result` is the patch applied to it, using the coordinator's own RFC 6902
+semantics. Both are available from `audit.read` alone. Where the snapshot
+carries the artefact the coordinator stored, that one is used, and the
+differential suite requires the replayed one to equal it.
+
+**Counts the tasks the server minted.** `escalate.raise` and
+`control.supersede` create a successor without a `task.create` envelope, and
+give it what the original had. An escalation successor takes the original's
+mode, and its kind where the spec omits one. A supersession successor takes
+the original's assignee and mode where the spec omits them, and gets the same
+`review_required` default a `task.create` would. The successor is a row like
+any other, with `supersedes` linking it back.
+
+**Keeps the orphans.** A task created and left alone still gets a row.
+
+**Says how sure it is.** `id_certain` and `assignee_certain` mark the rows where
+the chain admits more than one reading, so the population to draw conclusions
+from is a filter away.
+
+## Redaction
+
+Artefacts hold whatever the agent was working on: customer messages,
+contracts, source code. A redactor sees every one of them before anything
+reaches a table.
+
+```python
+from chap_analytics import from_sqlite, redact_artefacts, frames
+
+f = frames(from_sqlite("./chap.db", redact=redact_artefacts))
+```
+
+What goes: task inputs and outputs, the artefact under review and the
+corrected one, the values a patch writes, free-text whisper answers under
+either of their two names, lapse defaults, the inputs of a successor an
+escalation or supersession mints, and the copies a snapshot holds. All of it
+goes in both the envelope stream and the workspace snapshot that also contains
+it. A test plants a marker in each of those places and searches every cell of
+every table for all of them.
+
+What stays, because it is the analysis: counts, rates, latencies, tags, policy
+references, patch paths and operations, which option a whisper answer chose,
+and the words participants wrote about the work. A reviewer's rationale,
+comment and decline reason, a handoff summary and a deliberation question are
+the people's account of what they did, and the point of the chain. A whisper
+question is the agent's own words to a person, and it stays too; an agent that
+quotes customer content into a question puts that content in the log, and a
+deployment that needs it removed strips the `question` field before loading.
+
+## Identity
+
+Server-minted identifiers are returned in the *result* of a call, and the
+audit log records the envelopes. A task id therefore becomes visible when a
+later envelope acts on it, and which creation produced which id is worked out
+afterwards.
+
+With the snapshot, the pairing is settled by what the creation envelope and
+the stored task agree they are, with the order of appearance breaking ties.
+From envelopes alone, the order of appearance is the evidence: it settles the
+pairing when work proceeds one task at a time, and leaves it open when two
+tasks are created before either is touched. The row says which case it is in.
+`id_certain` is true where the evidence leaves one reading, so
+
+```python
+f.tasks[f.tasks.id_certain]
+```
+
+is the population to draw conclusions about individual tasks from. Counts hold
+either way: a creation that matches no observed id is given one marked
+`unidentified`, and the total stays right.
+
+The protocol settles more than the ordering can. A lapse notice concerns a
+whisper whose deadline had passed. An answer comes from someone the whisper was
+addressed to. A vote comes from someone the deliberation invited. An acceptance
+comes from the named recipient. An id the caller supplied is in the opening
+envelope itself. Each of those is used before the ordering is consulted, and
+`frames.summary()` reports how many rows remain inferred.
+
+## Tests
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+The fixtures drive a real coordinator. A generated fixture captures what the
+coordinator does, and fails when the protocol moves.
+
+Alongside the written cases there is a differential suite: random sequences
+of accepted calls against a real coordinator, read both ways. The
+tasks, decisions, overrides, whispers, deliberations and handoffs are checked
+against what that coordinator holds, the corrected artefacts against the ones
+it stored, and a floor is asserted on how many envelope-only rows are certain.
+
+## Licence
+
+Apache 2.0.