qlm-measure 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,169 @@
1
+ Metadata-Version: 2.4
2
+ Name: qlm-measure
3
+ Version: 0.1.0
4
+ Summary: Open measurement SDK for education AI — evidence events, record verification, and dataset clients
5
+ Author: Quantum Learning Machines
6
+ License: Apache-2.0
7
+ Project-URL: Homepage, https://play.quantumlearningmachines.com/developer
8
+ Project-URL: Repository, https://github.com/quantumkumar/qlm-measure
9
+ Keywords: education,measurement,evidence,learning-analytics,misconceptions
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: License :: OSI Approved :: Apache Software License
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Topic :: Education
14
+ Classifier: Topic :: Scientific/Engineering
15
+ Requires-Python: >=3.9
16
+ Description-Content-Type: text/markdown
17
+
18
+ # @qlm/measure
19
+
20
+ Open measurement SDK for education AI — evidence events, record verification, and dataset clients.
21
+
22
+ **v0.1.0 · Apache-2.0 · [Documentation](https://play.quantumlearningmachines.com/developer) · [Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)**
23
+
24
+ ## Limitations (read first)
25
+
26
+ - This SDK is the **public interface layer**, not the measurement engine. The estimation service (Bayesian posteriors, IRT ability estimation, calibration) is QLM's hosted engine.
27
+ - The verifier checks **bookkeeping integrity**, not estimation correctness. It can tell you if a record was tampered with. It cannot tell you if the posteriors are accurate. That distinction is the design.
28
+ - Evidence event schemas are **v0.1** and may change in minor versions. Pin your dependency.
29
+ - The ontology client requires the QLM API to be reachable. No offline mode.
30
+ - Classifier runners require model weights downloaded separately from Hugging Face.
31
+
32
+ ## What it is
33
+
34
+ 1. **Typed evidence-event vocabulary** (`ObservationEvent`) — a schema any tool can emit to describe a student interaction, with scaffold level, depth, epistemic mode.
35
+ 2. **Buffered telemetry emitter** (`MeasurementEmitter`) — sends events to the measurement service with sendBeacon fallback.
36
+ 3. **Evidence record format + replay verifier** (`verifyRecord`, `replayToVersion`) — audits a record's hash-chain and posterior-chain integrity without containing any estimation mathematics.
37
+ 4. **Dataset clients** (`OntologyClient`) — typed wrappers over the public misconception ontology, learning graph, and standards alignment APIs.
38
+ 5. **Engine client** (`EngineClient`) — commercial boundary for the hosted measurement service.
39
+
40
+ ## What it is NOT
41
+
42
+ - Not the estimation engine (that runs server-side at QLM)
43
+ - Not a standalone learning analytics system
44
+ - Not a replacement for the measurement service
45
+ - Not a classifier or scorer (those are separate open-weights releases)
46
+
47
+ ## Install
48
+
49
+ ```bash
50
+ npm install @qlm/measure
51
+ ```
52
+
53
+ ## Quickstart
54
+
55
+ ### 1. Query the misconception ontology (no auth)
56
+
57
+ ```typescript
58
+ import { OntologyClient } from "@qlm/measure/clients";
59
+
60
+ const client = new OntologyClient();
61
+ const misconceptions = await client.getMisconceptions("math");
62
+ console.log(`${misconceptions.length} math misconceptions`);
63
+ ```
64
+
65
+ ### 2. Emit measurement events
66
+
67
+ ```typescript
68
+ import { MeasurementEmitter } from "@qlm/measure/emitter";
69
+
70
+ const emitter = new MeasurementEmitter({
71
+ baseUrl: "https://play.quantumlearningmachines.com",
72
+ product: "my-tool",
73
+ });
74
+
75
+ emitter.emit({
76
+ eventType: "answer_submitted",
77
+ eventCategory: "learning",
78
+ domain: "math",
79
+ topic: "fractions",
80
+ payload: { questionId: "q1", correct: true },
81
+ });
82
+ ```
83
+
84
+ ### 3. Verify an evidence record (tamper detection)
85
+
86
+ ```typescript
87
+ import { verifyRecord } from "@qlm/measure/verifier";
88
+
89
+ const result = verifyRecord(record);
90
+ if (!result.valid) {
91
+ for (const v of result.violations) {
92
+ console.log(`[${v.category}] v${v.version}: ${v.message}`);
93
+ }
94
+ }
95
+ ```
96
+
97
+ ## Schema Reference
98
+
99
+ ### ObservationEvent (evidence input)
100
+
101
+ | Field | Type | Required | Description |
102
+ |-------|------|----------|-------------|
103
+ | studentId | string | yes | Pseudonymous ID (no PII) |
104
+ | sessionId | string | yes | Session identifier |
105
+ | source | string | yes | Source system |
106
+ | timestamp | string | yes | ISO 8601 |
107
+ | correct | boolean | yes | Was the response correct? |
108
+ | responseTimeMs | number | yes | Response time in ms |
109
+ | domain | string | yes | Subject domain |
110
+ | scaffoldType | ScaffoldType | no | What preceded this response |
111
+ | misconceptionId | string | no | Detected misconception ID |
112
+ | classifierConfidence | number | no | Classifier confidence (0-1) |
113
+ | skillId | string | no | Skill being assessed |
114
+ | score | number | no | Response score (0-1) |
115
+ | difficulty | number | no | Item difficulty (0-1) |
116
+ | epistemicMode | EpistemicMode | no | How student arrived at answer |
117
+ | ext | Record | no | Extension namespace |
118
+
119
+ ### Enums
120
+
121
+ **ScaffoldType:** `none`, `socratic`, `probing`, `metacognitive`, `scaffolding`, `explain`, `hint`, `demonstrate`
122
+
123
+ **EpistemicMode:** `experience`, `inference`, `analogy`, `testimony`
124
+
125
+ **DepthLevel:** `surface`, `conceptual`, `transfer`, `integration`
126
+
127
+ **TriageResult:** `correct`, `slip`, `misconception`, `disengagement`, `ambiguous`
128
+
129
+ ## Verifier
130
+
131
+ The verifier checks:
132
+ - Schema validity (required fields, correct types)
133
+ - Strictly monotonic version numbers
134
+ - Timestamp monotonicity (configurable tolerance)
135
+ - Hash-chain integrity (prevHash linkage, entryHash recompute)
136
+ - Posterior chain consistency (each prior = previous updated)
137
+ - Compaction-boundary integrity
138
+ - Enum validity
139
+
140
+ What it deliberately **cannot** check:
141
+ - Whether posteriors are correctly computed (that requires the engine)
142
+ - Whether evidential weights are appropriate (those are calibrated server-side)
143
+ - Whether triage classifications are accurate (those use rules not in this package)
144
+
145
+ This is the design: **verify instead of trust**.
146
+
147
+ ## Privacy
148
+
149
+ - `studentId` MUST be a pseudonymous identifier. Never include names, emails, or other PII.
150
+ - Events are buffered locally and sent via sendBeacon/fetch. No cookies are set.
151
+ - The SDK does not store any data persistently.
152
+
153
+ ## Versioning
154
+
155
+ - Semver from 0.1.0.
156
+ - Schema changes in minor versions (0.2, 0.3).
157
+ - Breaking changes in major versions (1.0).
158
+ - Pin your dependency version.
159
+
160
+ ## Links
161
+
162
+ - [Developer portal](https://play.quantumlearningmachines.com/developer)
163
+ - [Open Measurement Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)
164
+ - [Misconception ontology](https://play.quantumlearningmachines.com/developer)
165
+ - [Model cards on Hugging Face](https://huggingface.co/QuantumLearningMachines)
166
+
167
+ ## License
168
+
169
+ Apache-2.0. See [LICENSE](./LICENSE).
@@ -0,0 +1,152 @@
1
+ # @qlm/measure
2
+
3
+ Open measurement SDK for education AI — evidence events, record verification, and dataset clients.
4
+
5
+ **v0.1.0 · Apache-2.0 · [Documentation](https://play.quantumlearningmachines.com/developer) · [Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)**
6
+
7
+ ## Limitations (read first)
8
+
9
+ - This SDK is the **public interface layer**, not the measurement engine. The estimation service (Bayesian posteriors, IRT ability estimation, calibration) is QLM's hosted engine.
10
+ - The verifier checks **bookkeeping integrity**, not estimation correctness. It can tell you if a record was tampered with. It cannot tell you if the posteriors are accurate. That distinction is the design.
11
+ - Evidence event schemas are **v0.1** and may change in minor versions. Pin your dependency.
12
+ - The ontology client requires the QLM API to be reachable. No offline mode.
13
+ - Classifier runners require model weights downloaded separately from Hugging Face.
14
+
15
+ ## What it is
16
+
17
+ 1. **Typed evidence-event vocabulary** (`ObservationEvent`) — a schema any tool can emit to describe a student interaction, with scaffold level, depth, epistemic mode.
18
+ 2. **Buffered telemetry emitter** (`MeasurementEmitter`) — sends events to the measurement service with sendBeacon fallback.
19
+ 3. **Evidence record format + replay verifier** (`verifyRecord`, `replayToVersion`) — audits a record's hash-chain and posterior-chain integrity without containing any estimation mathematics.
20
+ 4. **Dataset clients** (`OntologyClient`) — typed wrappers over the public misconception ontology, learning graph, and standards alignment APIs.
21
+ 5. **Engine client** (`EngineClient`) — commercial boundary for the hosted measurement service.
22
+
23
+ ## What it is NOT
24
+
25
+ - Not the estimation engine (that runs server-side at QLM)
26
+ - Not a standalone learning analytics system
27
+ - Not a replacement for the measurement service
28
+ - Not a classifier or scorer (those are separate open-weights releases)
29
+
30
+ ## Install
31
+
32
+ ```bash
33
+ npm install @qlm/measure
34
+ ```
35
+
36
+ ## Quickstart
37
+
38
+ ### 1. Query the misconception ontology (no auth)
39
+
40
+ ```typescript
41
+ import { OntologyClient } from "@qlm/measure/clients";
42
+
43
+ const client = new OntologyClient();
44
+ const misconceptions = await client.getMisconceptions("math");
45
+ console.log(`${misconceptions.length} math misconceptions`);
46
+ ```
47
+
48
+ ### 2. Emit measurement events
49
+
50
+ ```typescript
51
+ import { MeasurementEmitter } from "@qlm/measure/emitter";
52
+
53
+ const emitter = new MeasurementEmitter({
54
+ baseUrl: "https://play.quantumlearningmachines.com",
55
+ product: "my-tool",
56
+ });
57
+
58
+ emitter.emit({
59
+ eventType: "answer_submitted",
60
+ eventCategory: "learning",
61
+ domain: "math",
62
+ topic: "fractions",
63
+ payload: { questionId: "q1", correct: true },
64
+ });
65
+ ```
66
+
67
+ ### 3. Verify an evidence record (tamper detection)
68
+
69
+ ```typescript
70
+ import { verifyRecord } from "@qlm/measure/verifier";
71
+
72
+ const result = verifyRecord(record);
73
+ if (!result.valid) {
74
+ for (const v of result.violations) {
75
+ console.log(`[${v.category}] v${v.version}: ${v.message}`);
76
+ }
77
+ }
78
+ ```
79
+
80
+ ## Schema Reference
81
+
82
+ ### ObservationEvent (evidence input)
83
+
84
+ | Field | Type | Required | Description |
85
+ |-------|------|----------|-------------|
86
+ | studentId | string | yes | Pseudonymous ID (no PII) |
87
+ | sessionId | string | yes | Session identifier |
88
+ | source | string | yes | Source system |
89
+ | timestamp | string | yes | ISO 8601 |
90
+ | correct | boolean | yes | Was the response correct? |
91
+ | responseTimeMs | number | yes | Response time in ms |
92
+ | domain | string | yes | Subject domain |
93
+ | scaffoldType | ScaffoldType | no | What preceded this response |
94
+ | misconceptionId | string | no | Detected misconception ID |
95
+ | classifierConfidence | number | no | Classifier confidence (0-1) |
96
+ | skillId | string | no | Skill being assessed |
97
+ | score | number | no | Response score (0-1) |
98
+ | difficulty | number | no | Item difficulty (0-1) |
99
+ | epistemicMode | EpistemicMode | no | How student arrived at answer |
100
+ | ext | Record | no | Extension namespace |
101
+
102
+ ### Enums
103
+
104
+ **ScaffoldType:** `none`, `socratic`, `probing`, `metacognitive`, `scaffolding`, `explain`, `hint`, `demonstrate`
105
+
106
+ **EpistemicMode:** `experience`, `inference`, `analogy`, `testimony`
107
+
108
+ **DepthLevel:** `surface`, `conceptual`, `transfer`, `integration`
109
+
110
+ **TriageResult:** `correct`, `slip`, `misconception`, `disengagement`, `ambiguous`
111
+
112
+ ## Verifier
113
+
114
+ The verifier checks:
115
+ - Schema validity (required fields, correct types)
116
+ - Strictly monotonic version numbers
117
+ - Timestamp monotonicity (configurable tolerance)
118
+ - Hash-chain integrity (prevHash linkage, entryHash recompute)
119
+ - Posterior chain consistency (each prior = previous updated)
120
+ - Compaction-boundary integrity
121
+ - Enum validity
122
+
123
+ What it deliberately **cannot** check:
124
+ - Whether posteriors are correctly computed (that requires the engine)
125
+ - Whether evidential weights are appropriate (those are calibrated server-side)
126
+ - Whether triage classifications are accurate (those use rules not in this package)
127
+
128
+ This is the design: **verify instead of trust**.
129
+
130
+ ## Privacy
131
+
132
+ - `studentId` MUST be a pseudonymous identifier. Never include names, emails, or other PII.
133
+ - Events are buffered locally and sent via sendBeacon/fetch. No cookies are set.
134
+ - The SDK does not store any data persistently.
135
+
136
+ ## Versioning
137
+
138
+ - Semver from 0.1.0.
139
+ - Schema changes in minor versions (0.2, 0.3).
140
+ - Breaking changes in major versions (1.0).
141
+ - Pin your dependency version.
142
+
143
+ ## Links
144
+
145
+ - [Developer portal](https://play.quantumlearningmachines.com/developer)
146
+ - [Open Measurement Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)
147
+ - [Misconception ontology](https://play.quantumlearningmachines.com/developer)
148
+ - [Model cards on Hugging Face](https://huggingface.co/QuantumLearningMachines)
149
+
150
+ ## License
151
+
152
+ Apache-2.0. See [LICENSE](./LICENSE).
@@ -0,0 +1,28 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68.0", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "qlm-measure"
7
+ version = "0.1.0"
8
+ description = "Open measurement SDK for education AI — evidence events, record verification, and dataset clients"
9
+ readme = "README.md"
10
+ license = {text = "Apache-2.0"}
11
+ requires-python = ">=3.9"
12
+ authors = [{name = "Quantum Learning Machines"}]
13
+ keywords = ["education", "measurement", "evidence", "learning-analytics", "misconceptions"]
14
+ classifiers = [
15
+ "Development Status :: 3 - Alpha",
16
+ "License :: OSI Approved :: Apache Software License",
17
+ "Programming Language :: Python :: 3",
18
+ "Topic :: Education",
19
+ "Topic :: Scientific/Engineering",
20
+ ]
21
+
22
+ [project.urls]
23
+ Homepage = "https://play.quantumlearningmachines.com/developer"
24
+ Repository = "https://github.com/quantumkumar/qlm-measure"
25
+
26
+ [tool.setuptools.packages.find]
27
+ where = ["."]
28
+ include = ["qlm_measure*"]
@@ -0,0 +1,27 @@
1
+ """
2
+ qlm-measure — Open measurement SDK for education AI.
3
+
4
+ Python mirror of @qlm/measure (npm). Provides:
5
+ - Evidence event schema (ObservationEvent, MeasurementEvent)
6
+ - Evidence record format + verifier
7
+ - Dataset clients (OntologyClient)
8
+ - Engine client (EngineClient, commercial boundary)
9
+
10
+ Apache-2.0 · v0.1.0
11
+ """
12
+
13
+ __version__ = "0.1.0"
14
+
15
+ from .schema import (
16
+ ObservationEvent,
17
+ MeasurementEvent,
18
+ EvidenceEntry,
19
+ EvidenceRecord,
20
+ ScaffoldType,
21
+ EpistemicMode,
22
+ DepthLevel,
23
+ TriageResult,
24
+ COMPACTION_RETENTION,
25
+ )
26
+ from .verifier import verify_record, replay_to_version, posterior_at_version
27
+ from .clients import OntologyClient, EngineClient
@@ -0,0 +1,79 @@
1
+ """
2
+ Dataset and engine clients — Python mirror.
3
+ """
4
+
5
+ from __future__ import annotations
6
+ import json
7
+ from typing import Any, Optional
8
+ from urllib.request import urlopen, Request
9
+ from urllib.parse import urlencode
10
+
11
+
12
+ class OntologyClient:
13
+ """Typed wrapper over the QLM dataset export API. Zero-auth."""
14
+
15
+ def __init__(self, base_url: str = "https://play.quantumlearningmachines.com"):
16
+ self.base_url = base_url
17
+
18
+ def fetch_dataset(
19
+ self,
20
+ dataset: str,
21
+ domain: Optional[str] = None,
22
+ fmt: Optional[str] = None,
23
+ ) -> Any:
24
+ params = {"dataset": dataset}
25
+ if domain:
26
+ params["domain"] = domain
27
+ if fmt:
28
+ params["format"] = fmt
29
+
30
+ url = f"{self.base_url}/api/labpath/dataset-export?{urlencode(params)}"
31
+ req = Request(url, headers={"Accept": "application/json"})
32
+ with urlopen(req) as resp:
33
+ return json.loads(resp.read().decode())
34
+
35
+ def get_misconceptions(self, domain: Optional[str] = None) -> list[dict]:
36
+ data = self.fetch_dataset("misconceptions", domain=domain)
37
+ return data.get("misconceptions", data) if isinstance(data, dict) else data
38
+
39
+ def get_learning_graph(self, domain: Optional[str] = None) -> list[dict]:
40
+ data = self.fetch_dataset("learning-graph", domain=domain)
41
+ return data.get("learningGraph", data) if isinstance(data, dict) else data
42
+
43
+ def get_standards(self, domain: Optional[str] = None) -> list[dict]:
44
+ data = self.fetch_dataset("standards", domain=domain)
45
+ return data.get("standardsAlignment", data) if isinstance(data, dict) else data
46
+
47
+
48
+ class EngineClient:
49
+ """
50
+ Commercial boundary for the QLM measurement engine.
51
+
52
+ The estimation service is QLM's hosted engine. This client wraps
53
+ the API for submitting observations and retrieving state.
54
+ """
55
+
56
+ def __init__(self, base_url: str, api_key: str):
57
+ self.base_url = base_url
58
+ self.api_key = api_key
59
+
60
+ def _request(self, method: str, path: str, body: Optional[dict] = None) -> dict:
61
+ url = f"{self.base_url}{path}"
62
+ data = json.dumps(body).encode() if body else None
63
+ req = Request(
64
+ url,
65
+ data=data,
66
+ method=method,
67
+ headers={
68
+ "Content-Type": "application/json",
69
+ "Authorization": f"Bearer {self.api_key}",
70
+ },
71
+ )
72
+ with urlopen(req) as resp:
73
+ return json.loads(resp.read().decode())
74
+
75
+ def update_student_model(self, observation: dict) -> dict:
76
+ return self._request("POST", "/api/student-model/update", observation)
77
+
78
+ def get_state(self, student_id: str) -> dict:
79
+ return self._request("GET", f"/api/student-model/state?studentId={student_id}")
@@ -0,0 +1,126 @@
1
+ """
2
+ Public event schema — Python mirror of TypeScript types.
3
+
4
+ All types are TypedDict for structural typing without runtime overhead.
5
+ """
6
+
7
+ from __future__ import annotations
8
+ from typing import TypedDict, Literal, Optional, Any
9
+
10
+ # ── Enums (as Literal unions) ─────────────────────────────────────────────────
11
+
12
+ ScaffoldType = Literal[
13
+ "none", "socratic", "probing", "metacognitive",
14
+ "scaffolding", "explain", "hint", "demonstrate",
15
+ ]
16
+
17
+ EpistemicMode = Literal["experience", "inference", "analogy", "testimony"]
18
+
19
+ DepthLevel = Literal["surface", "conceptual", "transfer", "integration"]
20
+
21
+ TriageResult = Literal["correct", "slip", "misconception", "disengagement", "ambiguous"]
22
+
23
+ EventCategory = Literal["learning", "teaching", "assessment", "interaction", "system"]
24
+
25
+ COMPACTION_RETENTION = 500
26
+
27
+ # ── Internal -> Public mapping ────────────────────────────────────────────────
28
+
29
+ _EPISTEMIC_MAP = {
30
+ "direct": "experience",
31
+ "reason": "inference",
32
+ "compare": "analogy",
33
+ "told": "testimony",
34
+ "experience": "experience",
35
+ "inference": "inference",
36
+ "analogy": "analogy",
37
+ "testimony": "testimony",
38
+ }
39
+
40
+
41
+ def to_public_epistemic_mode(value: str) -> EpistemicMode:
42
+ return _EPISTEMIC_MAP.get(value, "testimony") # type: ignore[return-value]
43
+
44
+
45
+ # ── ObservationEvent ──────────────────────────────────────────────────────────
46
+
47
+ class ObservationEvent(TypedDict, total=False):
48
+ studentId: str
49
+ sessionId: str
50
+ source: str
51
+ timestamp: str
52
+ correct: bool
53
+ responseTimeMs: int
54
+ domain: str
55
+ scaffoldType: Optional[ScaffoldType]
56
+ misconceptionId: Optional[str]
57
+ classifierConfidence: Optional[float]
58
+ classifierLabel: Optional[str]
59
+ skillId: Optional[str]
60
+ score: Optional[float]
61
+ difficulty: Optional[float]
62
+ consecutiveErrors: Optional[int]
63
+ sessionErrorRate: Optional[float]
64
+ explanationQuality: Optional[float]
65
+ isTransferProblem: Optional[bool]
66
+ connectionsMade: Optional[int]
67
+ epistemicMode: Optional[EpistemicMode]
68
+ selfCorrected: Optional[bool]
69
+ ext: Optional[dict[str, Any]]
70
+
71
+
72
+ # ── MeasurementEvent ──────────────────────────────────────────────────────────
73
+
74
+ class MeasurementEvent(TypedDict, total=False):
75
+ eventType: str
76
+ eventCategory: EventCategory
77
+ sessionId: Optional[str]
78
+ classroomId: Optional[str]
79
+ domain: Optional[str]
80
+ topic: Optional[str]
81
+ mode: Optional[str]
82
+ difficulty: Optional[float]
83
+ payload: Optional[dict[str, Any]]
84
+ qualitySignal: Optional[float]
85
+ confidence: Optional[float]
86
+ durationMs: Optional[int]
87
+ parentEventId: Optional[str]
88
+
89
+
90
+ # ── Evidence Record ───────────────────────────────────────────────────────────
91
+
92
+ class NonInterventionDecision(TypedDict):
93
+ intervened: bool
94
+ confidence: float
95
+
96
+
97
+ class RedactedEvent(TypedDict):
98
+ redacted: bool
99
+ eventHash: str
100
+
101
+
102
+ class EvidenceEntry(TypedDict, total=False):
103
+ version: int
104
+ timestamp: str
105
+ event: ObservationEvent | RedactedEvent
106
+ scaffoldType: ScaffoldType
107
+ depthLevel: DepthLevel
108
+ epistemicMode: Optional[EpistemicMode]
109
+ triageResult: TriageResult
110
+ evidentialWeight: float
111
+ priorPosterior: float
112
+ updatedPosterior: float
113
+ nonInterventionDecision: Optional[NonInterventionDecision]
114
+ entryHash: str
115
+ prevHash: str
116
+
117
+
118
+ class CompactedBefore(TypedDict):
119
+ version: int
120
+ summaryHash: str
121
+
122
+
123
+ class EvidenceRecord(TypedDict, total=False):
124
+ studentScopeId: str
125
+ entries: list[EvidenceEntry]
126
+ compactedBefore: Optional[CompactedBefore]
@@ -0,0 +1,166 @@
1
+ """
2
+ Evidence Record Verifier — Python mirror.
3
+
4
+ Checks bookkeeping integrity WITHOUT any estimation mathematics.
5
+ """
6
+
7
+ from __future__ import annotations
8
+ import hashlib
9
+ import json
10
+ from dataclasses import dataclass, field
11
+ from typing import Optional
12
+
13
+ from .schema import EvidenceRecord, EvidenceEntry
14
+
15
+ VALID_SCAFFOLD_TYPES = {"none", "socratic", "probing", "metacognitive", "scaffolding", "explain", "hint", "demonstrate"}
16
+ VALID_DEPTH_LEVELS = {"surface", "conceptual", "transfer", "integration"}
17
+ VALID_EPISTEMIC_MODES = {"experience", "inference", "analogy", "testimony"}
18
+ VALID_TRIAGE_RESULTS = {"correct", "slip", "misconception", "disengagement", "ambiguous"}
19
+
20
+
21
+ @dataclass
22
+ class Violation:
23
+ version: int
24
+ category: str # schema | version | timestamp | hash | posterior | compaction | enum
25
+ message: str
26
+
27
+
28
+ @dataclass
29
+ class VerificationResult:
30
+ valid: bool
31
+ violations: list[Violation] = field(default_factory=list)
32
+ entries_checked: int = 0
33
+
34
+
35
+ def _canonicalize(obj: object) -> str:
36
+ """Canonical JSON: sorted keys, no whitespace."""
37
+ return json.dumps(obj, sort_keys=True, separators=(",", ":"), default=str)
38
+
39
+
40
+ def _sha256(s: str) -> str:
41
+ return hashlib.sha256(s.encode()).hexdigest()
42
+
43
+
44
+ def _hash_entry(entry: EvidenceEntry) -> str:
45
+ hashable = {
46
+ "version": entry.get("version"),
47
+ "timestamp": entry.get("timestamp"),
48
+ "event": entry.get("event"),
49
+ "scaffoldType": entry.get("scaffoldType"),
50
+ "depthLevel": entry.get("depthLevel"),
51
+ "epistemicMode": entry.get("epistemicMode"),
52
+ "triageResult": entry.get("triageResult"),
53
+ "evidentialWeight": entry.get("evidentialWeight"),
54
+ "priorPosterior": entry.get("priorPosterior"),
55
+ "updatedPosterior": entry.get("updatedPosterior"),
56
+ "nonInterventionDecision": entry.get("nonInterventionDecision"),
57
+ }
58
+ return _sha256(_canonicalize(hashable))
59
+
60
+
61
+ def verify_record(
62
+ record: EvidenceRecord,
63
+ timestamp_tolerance_ms: int = 1000,
64
+ ) -> VerificationResult:
65
+ """Verify bookkeeping integrity of an evidence record."""
66
+ violations: list[Violation] = []
67
+
68
+ scope_id = record.get("studentScopeId", "")
69
+ if not scope_id:
70
+ violations.append(Violation(0, "schema", "Missing or empty studentScopeId"))
71
+
72
+ entries = record.get("entries", [])
73
+ if not isinstance(entries, list):
74
+ violations.append(Violation(0, "schema", "entries must be a list"))
75
+ return VerificationResult(valid=False, violations=violations, entries_checked=0)
76
+
77
+ compacted = record.get("compactedBefore")
78
+ prev_version = compacted["version"] if compacted else 0
79
+ prev_hash = compacted.get("summaryHash", "") if compacted else ""
80
+ last_updated: Optional[float] = None
81
+
82
+ from datetime import datetime
83
+
84
+ prev_ts_str = ""
85
+ for entry in entries:
86
+ v = entry.get("version", -1)
87
+
88
+ # Schema
89
+ if not isinstance(v, int) or v < 1:
90
+ violations.append(Violation(v, "schema", "version must be a positive integer"))
91
+ if not entry.get("timestamp"):
92
+ violations.append(Violation(v, "schema", "timestamp is required"))
93
+ if not isinstance(entry.get("evidentialWeight"), (int, float)):
94
+ violations.append(Violation(v, "schema", "evidentialWeight must be a number"))
95
+ if not isinstance(entry.get("priorPosterior"), (int, float)):
96
+ violations.append(Violation(v, "schema", "priorPosterior must be a number"))
97
+ if not isinstance(entry.get("updatedPosterior"), (int, float)):
98
+ violations.append(Violation(v, "schema", "updatedPosterior must be a number"))
99
+
100
+ # Monotonic version
101
+ if v <= prev_version:
102
+ violations.append(Violation(v, "version", f"Version {v} <= previous {prev_version}"))
103
+
104
+ # Timestamp monotonicity
105
+ ts_str = entry.get("timestamp", "")
106
+ if prev_ts_str and ts_str:
107
+ try:
108
+ prev_ms = datetime.fromisoformat(prev_ts_str.replace("Z", "+00:00")).timestamp() * 1000
109
+ curr_ms = datetime.fromisoformat(ts_str.replace("Z", "+00:00")).timestamp() * 1000
110
+ if curr_ms < prev_ms - timestamp_tolerance_ms:
111
+ violations.append(Violation(v, "timestamp", f"Timestamp goes backward"))
112
+ except (ValueError, TypeError):
113
+ pass
114
+
115
+ # Enum validation
116
+ st = entry.get("scaffoldType")
117
+ if st and st not in VALID_SCAFFOLD_TYPES:
118
+ violations.append(Violation(v, "enum", f"Invalid scaffoldType: {st}"))
119
+ dl = entry.get("depthLevel")
120
+ if dl and dl not in VALID_DEPTH_LEVELS:
121
+ violations.append(Violation(v, "enum", f"Invalid depthLevel: {dl}"))
122
+ em = entry.get("epistemicMode")
123
+ if em and em not in VALID_EPISTEMIC_MODES:
124
+ violations.append(Violation(v, "enum", f"Invalid epistemicMode: {em}"))
125
+ tr = entry.get("triageResult")
126
+ if tr and tr not in VALID_TRIAGE_RESULTS:
127
+ violations.append(Violation(v, "enum", f"Invalid triageResult: {tr}"))
128
+
129
+ # Hash chain
130
+ if entry.get("prevHash") != prev_hash:
131
+ violations.append(Violation(v, "hash", f"prevHash mismatch"))
132
+ computed = _hash_entry(entry)
133
+ if entry.get("entryHash") != computed:
134
+ violations.append(Violation(v, "hash", f"entryHash mismatch"))
135
+
136
+ # Posterior chain
137
+ if last_updated is not None:
138
+ prior = entry.get("priorPosterior")
139
+ if isinstance(prior, (int, float)) and abs(prior - last_updated) > 1e-9:
140
+ violations.append(Violation(v, "posterior", f"priorPosterior ({prior}) != previous updatedPosterior ({last_updated})"))
141
+
142
+ prev_version = v
143
+ prev_ts_str = ts_str
144
+ prev_hash = entry.get("entryHash", "")
145
+ up = entry.get("updatedPosterior")
146
+ if isinstance(up, (int, float)):
147
+ last_updated = up
148
+
149
+ return VerificationResult(
150
+ valid=len(violations) == 0,
151
+ violations=violations,
152
+ entries_checked=len(entries),
153
+ )
154
+
155
+
156
+ def replay_to_version(record: EvidenceRecord, target_version: int) -> list[EvidenceEntry]:
157
+ """Return entries up to and including target_version."""
158
+ return [e for e in record.get("entries", []) if e.get("version", 0) <= target_version]
159
+
160
+
161
+ def posterior_at_version(record: EvidenceRecord, target_version: int) -> Optional[float]:
162
+ """Get the posterior at a specific version."""
163
+ entries = replay_to_version(record, target_version)
164
+ if not entries:
165
+ return None
166
+ return entries[-1].get("updatedPosterior")
@@ -0,0 +1,169 @@
1
+ Metadata-Version: 2.4
2
+ Name: qlm-measure
3
+ Version: 0.1.0
4
+ Summary: Open measurement SDK for education AI — evidence events, record verification, and dataset clients
5
+ Author: Quantum Learning Machines
6
+ License: Apache-2.0
7
+ Project-URL: Homepage, https://play.quantumlearningmachines.com/developer
8
+ Project-URL: Repository, https://github.com/quantumkumar/qlm-measure
9
+ Keywords: education,measurement,evidence,learning-analytics,misconceptions
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: License :: OSI Approved :: Apache Software License
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Topic :: Education
14
+ Classifier: Topic :: Scientific/Engineering
15
+ Requires-Python: >=3.9
16
+ Description-Content-Type: text/markdown
17
+
18
+ # @qlm/measure
19
+
20
+ Open measurement SDK for education AI — evidence events, record verification, and dataset clients.
21
+
22
+ **v0.1.0 · Apache-2.0 · [Documentation](https://play.quantumlearningmachines.com/developer) · [Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)**
23
+
24
+ ## Limitations (read first)
25
+
26
+ - This SDK is the **public interface layer**, not the measurement engine. The estimation service (Bayesian posteriors, IRT ability estimation, calibration) is QLM's hosted engine.
27
+ - The verifier checks **bookkeeping integrity**, not estimation correctness. It can tell you if a record was tampered with. It cannot tell you if the posteriors are accurate. That distinction is the design.
28
+ - Evidence event schemas are **v0.1** and may change in minor versions. Pin your dependency.
29
+ - The ontology client requires the QLM API to be reachable. No offline mode.
30
+ - Classifier runners require model weights downloaded separately from Hugging Face.
31
+
32
+ ## What it is
33
+
34
+ 1. **Typed evidence-event vocabulary** (`ObservationEvent`) — a schema any tool can emit to describe a student interaction, with scaffold level, depth, epistemic mode.
35
+ 2. **Buffered telemetry emitter** (`MeasurementEmitter`) — sends events to the measurement service with sendBeacon fallback.
36
+ 3. **Evidence record format + replay verifier** (`verifyRecord`, `replayToVersion`) — audits a record's hash-chain and posterior-chain integrity without containing any estimation mathematics.
37
+ 4. **Dataset clients** (`OntologyClient`) — typed wrappers over the public misconception ontology, learning graph, and standards alignment APIs.
38
+ 5. **Engine client** (`EngineClient`) — commercial boundary for the hosted measurement service.
39
+
40
+ ## What it is NOT
41
+
42
+ - Not the estimation engine (that runs server-side at QLM)
43
+ - Not a standalone learning analytics system
44
+ - Not a replacement for the measurement service
45
+ - Not a classifier or scorer (those are separate open-weights releases)
46
+
47
+ ## Install
48
+
49
+ ```bash
50
+ npm install @qlm/measure
51
+ ```
52
+
53
+ ## Quickstart
54
+
55
+ ### 1. Query the misconception ontology (no auth)
56
+
57
+ ```typescript
58
+ import { OntologyClient } from "@qlm/measure/clients";
59
+
60
+ const client = new OntologyClient();
61
+ const misconceptions = await client.getMisconceptions("math");
62
+ console.log(`${misconceptions.length} math misconceptions`);
63
+ ```
64
+
65
+ ### 2. Emit measurement events
66
+
67
+ ```typescript
68
+ import { MeasurementEmitter } from "@qlm/measure/emitter";
69
+
70
+ const emitter = new MeasurementEmitter({
71
+ baseUrl: "https://play.quantumlearningmachines.com",
72
+ product: "my-tool",
73
+ });
74
+
75
+ emitter.emit({
76
+ eventType: "answer_submitted",
77
+ eventCategory: "learning",
78
+ domain: "math",
79
+ topic: "fractions",
80
+ payload: { questionId: "q1", correct: true },
81
+ });
82
+ ```
83
+
84
+ ### 3. Verify an evidence record (tamper detection)
85
+
86
+ ```typescript
87
+ import { verifyRecord } from "@qlm/measure/verifier";
88
+
89
+ const result = verifyRecord(record);
90
+ if (!result.valid) {
91
+ for (const v of result.violations) {
92
+ console.log(`[${v.category}] v${v.version}: ${v.message}`);
93
+ }
94
+ }
95
+ ```
96
+
97
+ ## Schema Reference
98
+
99
+ ### ObservationEvent (evidence input)
100
+
101
+ | Field | Type | Required | Description |
102
+ |-------|------|----------|-------------|
103
+ | studentId | string | yes | Pseudonymous ID (no PII) |
104
+ | sessionId | string | yes | Session identifier |
105
+ | source | string | yes | Source system |
106
+ | timestamp | string | yes | ISO 8601 |
107
+ | correct | boolean | yes | Was the response correct? |
108
+ | responseTimeMs | number | yes | Response time in ms |
109
+ | domain | string | yes | Subject domain |
110
+ | scaffoldType | ScaffoldType | no | What preceded this response |
111
+ | misconceptionId | string | no | Detected misconception ID |
112
+ | classifierConfidence | number | no | Classifier confidence (0-1) |
113
+ | skillId | string | no | Skill being assessed |
114
+ | score | number | no | Response score (0-1) |
115
+ | difficulty | number | no | Item difficulty (0-1) |
116
+ | epistemicMode | EpistemicMode | no | How student arrived at answer |
117
+ | ext | Record | no | Extension namespace |
118
+
119
+ ### Enums
120
+
121
+ **ScaffoldType:** `none`, `socratic`, `probing`, `metacognitive`, `scaffolding`, `explain`, `hint`, `demonstrate`
122
+
123
+ **EpistemicMode:** `experience`, `inference`, `analogy`, `testimony`
124
+
125
+ **DepthLevel:** `surface`, `conceptual`, `transfer`, `integration`
126
+
127
+ **TriageResult:** `correct`, `slip`, `misconception`, `disengagement`, `ambiguous`
128
+
129
+ ## Verifier
130
+
131
+ The verifier checks:
132
+ - Schema validity (required fields, correct types)
133
+ - Strictly monotonic version numbers
134
+ - Timestamp monotonicity (configurable tolerance)
135
+ - Hash-chain integrity (prevHash linkage, entryHash recompute)
136
+ - Posterior chain consistency (each prior = previous updated)
137
+ - Compaction-boundary integrity
138
+ - Enum validity
139
+
140
+ What it deliberately **cannot** check:
141
+ - Whether posteriors are correctly computed (that requires the engine)
142
+ - Whether evidential weights are appropriate (those are calibrated server-side)
143
+ - Whether triage classifications are accurate (those use rules not in this package)
144
+
145
+ This is the design: **verify instead of trust**.
146
+
147
+ ## Privacy
148
+
149
+ - `studentId` MUST be a pseudonymous identifier. Never include names, emails, or other PII.
150
+ - Events are buffered locally and sent via sendBeacon/fetch. No cookies are set.
151
+ - The SDK does not store any data persistently.
152
+
153
+ ## Versioning
154
+
155
+ - Semver from 0.1.0.
156
+ - Schema changes in minor versions (0.2, 0.3).
157
+ - Breaking changes in major versions (1.0).
158
+ - Pin your dependency version.
159
+
160
+ ## Links
161
+
162
+ - [Developer portal](https://play.quantumlearningmachines.com/developer)
163
+ - [Open Measurement Manifesto](https://play.quantumlearningmachines.com/resources/open-measurement-layer)
164
+ - [Misconception ontology](https://play.quantumlearningmachines.com/developer)
165
+ - [Model cards on Hugging Face](https://huggingface.co/QuantumLearningMachines)
166
+
167
+ ## License
168
+
169
+ Apache-2.0. See [LICENSE](./LICENSE).
@@ -0,0 +1,10 @@
1
+ README.md
2
+ pyproject.toml
3
+ qlm_measure/__init__.py
4
+ qlm_measure/clients.py
5
+ qlm_measure/schema.py
6
+ qlm_measure/verifier.py
7
+ qlm_measure.egg-info/PKG-INFO
8
+ qlm_measure.egg-info/SOURCES.txt
9
+ qlm_measure.egg-info/dependency_links.txt
10
+ qlm_measure.egg-info/top_level.txt
@@ -0,0 +1 @@
1
+ qlm_measure
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+