masterytrace-cli 0.1.0__tar.gz

masterytrace_cli-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.pyc
+*.egg-info/
+.venv/
+venv/
+dist/
+build/
+.pytest_cache/
+.mypy_cache/

masterytrace_cli-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Rudrendu Paul
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

masterytrace_cli-0.1.0/PKG-INFO

@@ -0,0 +1,362 @@
+Metadata-Version: 2.4
+Name: masterytrace-cli
+Version: 0.1.0
+Summary: Mastery Measurement API and CLI: fits Bayesian Knowledge Tracing (BKT) and Item Response Theory (2PL IRT) models to learner response logs and reports per-learner, per-skill mastery estimates.
+Project-URL: Homepage, https://github.com/RudrenduPaul/MasteryTrace
+Project-URL: Repository, https://github.com/RudrenduPaul/MasteryTrace
+Project-URL: Bug Tracker, https://github.com/RudrenduPaul/MasteryTrace/issues
+Project-URL: Changelog, https://github.com/RudrenduPaul/MasteryTrace/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/RudrenduPaul/MasteryTrace/blob/main/docs/getting-started.md
+Project-URL: Author - Rudrendu Paul, https://github.com/RudrenduPaul
+Project-URL: Author - Sourav Nandy, https://github.com/Sourav-nandy-ai
+Author: Rudrendu Paul, Sourav Nandy
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent-native,bayesian-knowledge-tracing,bkt,cli,edtech,education,irt,item-response-theory,learning-analytics,psychometrics
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Education
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Provides-Extra: dev
+Requires-Dist: build<2,>=1.0; extra == 'dev'
+Requires-Dist: pytest<9,>=7.0; extra == 'dev'
+Requires-Dist: twine<7,>=5.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# masterytrace-cli (Python)
+
+Mastery Measurement API and CLI: fits Bayesian Knowledge Tracing (BKT) and
+Item Response Theory (2-parameter logistic IRT) models to learner response
+logs and reports per-learner, per-skill mastery estimates.
+
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/RudrenduPaul/MasteryTrace/blob/main/LICENSE)
+[![CI](https://github.com/RudrenduPaul/MasteryTrace/actions/workflows/ci.yml/badge.svg)](https://github.com/RudrenduPaul/MasteryTrace/actions/workflows/ci.yml)
+
+**PyPI status**: publishing this package as `masterytrace-cli` is in
+progress. The first publish attempt hit PyPI's account-level
+`429 Too many new projects created` anti-abuse rate limit -- a
+platform-side throttle on new package names, unrelated to this code or
+to any account security issue -- so `pip install masterytrace-cli` is not
+live yet. The PyPI version/Python-versions badges above will be restored
+once the package is live; install from source below in the meantime.
+
+## Why this exists
+
+Most tutoring products and AI tutoring agents track raw percent-correct,
+which conflates a lucky guess with real mastery and never says how
+confident the estimate actually is. BKT and IRT are the two psychometric
+models built to fix that, but they live almost entirely as Python-only
+libraries with no CLI a non-Python tool (or an agent shelling out to a
+subprocess) can call directly. MasteryTrace is a CLI and library that
+turns a log of `{learnerId, skillId, correct, timestamp}` response events
+into a posterior mastery probability (BKT) and an ability estimate (IRT)
+per learner per skill. This package is the Python distribution -- a
+genuine, independent port of the npm package's TypeScript source, not a
+wrapper around a Node binary.
+
+## Install
+
+Once published:
+
+```bash
+pip install masterytrace-cli
+```
+
+or with [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv add masterytrace-cli
+```
+
+Until then, install from source (this checkout builds and installs
+identically to how the PyPI package will):
+
+```bash
+git clone https://github.com/RudrenduPaul/MasteryTrace.git
+cd MasteryTrace/python
+pip install -e .
+```
+
+This installs a `masterytrace` console script and the `masterytrace`
+importable package. No runtime dependencies beyond the Python standard
+library -- the forward-recursion and gradient-ascent math in this port
+are both plain scalar/list arithmetic, exactly what the TypeScript
+original does with `Float64Array` and plain numbers, so a numpy/scipy
+dependency would not simplify anything here.
+
+**Honest note on the npm side:** the complementary JS/TS distribution
+(`masterytrace-cli` on npm) is not yet published -- that is a deliberate,
+unrelated decision by the maintainer, not a reflection of the npm
+package's readiness (it passes CI and has a working `dist/` build). This
+PyPI package is independent of that decision and installs and runs on its
+own.
+
+## Quickstart
+
+```bash
+masterytrace init
+masterytrace record events.json
+masterytrace score
+masterytrace report
+```
+
+`init` scaffolds a sample `events.json` (3 learners, 3 skills, several
+responses each) and a default `masterytrace.config.json` in the current
+directory. Real output from that flow:
+
+```
+$ masterytrace init
+Created: events.json, masterytrace.config.json
+Next: run 'masterytrace record events.json' to load it, then 'masterytrace score'.
+
+$ masterytrace record events.json
+Stored 58 event(s) to /path/to/.masterytrace/events.json
+(record replaces any previously stored event log; see --help for details.)
+
+$ masterytrace score
+Scored 58 event(s) with model(s): both
+Wrote /path/to/.masterytrace/scores.json
+
+$ masterytrace report
+learner        skill                  model  metric                         value    responses
+-------------  ---------------------  -----  -----------------------------  -------  ---------
+learner-ada    fractions              bkt    posterior_mastery_probability  0.9994   6
+learner-ada    fractions              irt    ability_theta                  0.7349   6
+learner-ada    linear-equations       bkt    posterior_mastery_probability  0.9746   7
+learner-brook  fractions              bkt    posterior_mastery_probability  0.0612   6
+learner-cyrus  reading-comprehension  bkt    posterior_mastery_probability  0.9947   7
+...
+```
+
+`report` also takes `--format markdown` or `--format json`, and every
+command accepts a global `--json` flag for machine-readable output on
+stdout, with a real exit code contract (`0` success, `1` general/usage
+error, `2` bad event data) so a script or agent invoking this CLI can
+branch on the result without parsing text.
+
+Your own event log is a JSON array of `{learnerId, skillId, correct,
+timestamp}` objects, or a CSV with header
+`learner_id,skill_id,correct,timestamp`. `timestamp` must be ISO 8601;
+`correct` is a boolean (JSON) or `true`/`false`/`1`/`0` (CSV), and any
+other value in a CSV `correct` cell is rejected as a validation error
+rather than silently treated as false. Event log files over 100 MB are
+rejected up front with a clear error.
+
+## CLI command reference
+
+| Command | Arguments | Options | Does |
+| --- | --- | --- | --- |
+| `masterytrace init` | | `--force` | Scaffolds a sample `events.json` and `masterytrace.config.json` in the current directory. Skips files that already exist unless `--force` is passed. |
+| `masterytrace record <path>` | `<path>`: JSON or CSV event log | | Validates an event log and stores it to `.masterytrace/events.json`. Always replaces any previously stored log. |
+| `masterytrace score` | | `--model <bkt\|irt\|both>` (default `both`) | Fits and scores the stored event log, writing the result to `.masterytrace/scores.json`. |
+| `masterytrace report` | | `--format <table\|json\|markdown>` (default `table`) | Reads `.masterytrace/scores.json` and prints a per-learner, per-skill mastery table. |
+
+Global option: `--json` forces machine-readable JSON on stdout for any
+command, overriding `--format` on `report`.
+
+Exit codes: `0` success, `1` general or usage error (bad flag, missing
+file), `2` validation error (the event log itself is malformed).
+
+## Library API
+
+```python
+from masterytrace import (
+    # Event schema and validation
+    ResponseEvent, parse_response_events, EventValidationError,
+
+    # Engine: runs one or both models
+    run_scoring, EngineConfig, EngineResult,
+
+    # BKT
+    BktModel, BKT_DEFAULT_PARAMS, run_forward_recursion, fit_skill_params_by_grid_search,
+    BktParams, BktConfig, BktFittedModel,
+
+    # IRT
+    IrtModel, probability_correct,
+    IrtItemParams, IrtLearnerResult, IrtConfig, IrtFittedModel,
+
+    # Generic JSON/CSV event log adapter
+    generic_adapter, parse_csv, GenericAdapter,
+)
+
+events = parse_response_events([
+    {"learnerId": "l1", "skillId": "fractions", "correct": True, "timestamp": "2026-01-01T00:00:00Z"},
+    {"learnerId": "l1", "skillId": "fractions", "correct": False, "timestamp": "2026-01-02T00:00:00Z"},
+])
+
+result = run_scoring(events, "both")
+# result.reports[0].model == "bkt", result.reports[1].model == "irt"
+# each learner's report.learners[i].skills[j].value is the mastery estimate
+```
+
+`BktModel` and `IrtModel` both implement the same `fit(events)` then
+`score(fitted_model)` two-step interface, so the engine, and your own
+code, can treat them interchangeably.
+
+**A deliberate naming divergence from the npm package**: this Python
+port's JSON output and dataclass field names are `snake_case`
+(`learner_id`, `response_count`, `item_discrimination`) rather than the
+npm CLI's `camelCase` (`learnerId`, `responseCount`,
+`itemDiscrimination`), matching each language's own convention. The data
+shape -- which fields exist and what they mean -- is identical.
+
+## How BKT and IRT work
+
+MasteryTrace implements two independent psychometric models. They answer
+different questions and produce different kinds of numbers, so
+`masterytrace score --model both` runs them side by side rather than
+picking one.
+
+### Bayesian Knowledge Tracing (BKT)
+
+BKT models one learner's mastery of one skill as a hidden binary state
+(knows it / does not know it yet) and updates a probability of "knows it"
+after every response, using four parameters: `p_init` (prior probability
+the learner already knows the skill), `p_transit` (probability of
+learning the skill between one attempt and the next), `p_slip`
+(probability of an incorrect answer despite knowing the skill), and
+`p_guess` (probability of a correct answer despite not knowing the
+skill). For each response, the forward recursion first updates the belief
+given the observed outcome (Bayes' rule), then advances it for possible
+learning before the next attempt:
+
+```
+after correct:   P(know | obs) = P(know) * (1 - p_slip) / [P(know) * (1 - p_slip) + (1 - P(know)) * p_guess]
+after incorrect: P(know | obs) = P(know) * p_slip       / [P(know) * p_slip       + (1 - P(know)) * (1 - p_guess)]
+P(know)_next = P(know | obs) + (1 - P(know | obs)) * p_transit
+```
+
+MasteryTrace runs this recursion per learner per skill, in chronological
+order, and reports the final posterior as that learner's mastery
+probability for that skill. If you set `"bkt": {"fit": true}` in
+`masterytrace.config.json`, each skill's four parameters are fit from
+your own data by a coarse grid search (7 x 7 x 5 x 5 candidate
+combinations) that minimizes squared error between predicted and observed
+correctness, instead of using the textbook defaults (`p_init=0.4,
+p_transit=0.3, p_slip=0.1, p_guess=0.2`). This is the same coarse,
+dependency-free grid search the TypeScript original implements -- not a
+full EM/Baum-Welch fit -- ported line-for-line, including its parameter
+grid values.
+
+### Item Response Theory (2PL IRT)
+
+IRT models one continuous learner ability (`theta`) per learner and two
+parameters per skill treated as an "item": discrimination (`a`, how
+sharply the item separates high- and low-ability learners) and difficulty
+(`b`). The probability of a correct response under the 2-parameter
+logistic model is:
+
+```
+P(correct) = sigmoid(a * (theta - b))
+```
+
+MasteryTrace fits all of these jointly by gradient ascent on the
+log-likelihood (joint MLE), with a small L2 penalty pulling `theta`/`b`
+toward 0 and `a` toward 1. That penalty is what keeps the fit finite for
+a learner or skill with an all-correct or all-incorrect record, where the
+unregularized likelihood would otherwise be maximized at infinity.
+Because the 2PL model is only identified up to a shift and scale of
+`theta` (shifting `theta` and `b` by the same constant, or scaling
+`theta`/`b` while dividing `a` accordingly, leaves every predicted
+probability unchanged), the fit re-centers `theta` to mean 0 and standard
+deviation 1 after every iteration -- the standard way to pin down a
+single solution. This port mirrors the TypeScript original's hand-rolled
+batch gradient ascent line for line (same update rule, same per-iteration
+gauge fix, same default 500 iterations / 0.5 learning rate / 0.01
+regularization), rather than calling into an external optimizer -- there
+is no scipy.optimize equivalent in play to port to, since the original
+does not use one either.
+
+### A real recovery check
+
+`tests/test_irt.py` fits the model against a synthetic dataset built from
+known ground-truth `theta`/`a`/`b` values (4,000 responses across 5
+learners and 4 skills, same synthetic setup as the TypeScript suite's own
+recovery test) and checks that the recovered parameters land close to the
+true ones once put through the same gauge normalization, within the same
+0.3 absolute-error tolerance the TypeScript test uses.
+
+## Benchmark
+
+Run locally against synthetic event logs (single core, in-process library
+calls, no subprocess/CLI startup overhead):
+
+| Dataset | Events | BKT fit+score | IRT fit+score (500 iterations) |
+| --- | --- | --- | --- |
+| Small | 10,000 (50 learners x 20 skills x 10 responses) | 0.05s | 2.86s |
+| Large | 100,000 (100 learners x 50 skills x 20 responses) | 0.09s | 14.42s |
+
+BKT with grid-search fitting (`"bkt": {"fit": true}`, a 1,225-combination
+grid search per skill) on the 10,000-event dataset took 5.20s.
+
+**Honest comparison with the TypeScript package**: the npm package's own
+documented benchmark reports the 100,000-event IRT fit at 0.54s (as part
+of a `--model both` subprocess run) -- roughly 27x faster than this
+Python port's 14.42s for the same fit alone. This gap is expected and
+disclosed, not a porting bug: IRT's gradient-ascent fit is a tight
+numeric loop, and V8's JIT compiles that loop far more aggressively than
+CPython's interpreter executes the equivalent Python loop. The fitted
+*values* are what this port keeps faithful (see the recovery check
+above, and the identical-to-4-decimal-places output in the Quickstart
+section); wall-clock fit time on large datasets is not. See
+[SECURITY.md](https://github.com/RudrenduPaul/MasteryTrace/blob/main/SECURITY.md)'s
+"known limitation" section for what this means for very large,
+untrusted event logs.
+
+## CI integration
+
+```yaml
+- uses: actions/checkout@v4
+- uses: actions/setup-python@v5
+  with:
+    python-version: '3.12'
+- run: pip install masterytrace-cli
+- run: masterytrace init
+- run: masterytrace record events.json
+- run: masterytrace score --model both
+- run: masterytrace report --format json > mastery-report.json
+```
+
+Full walkthrough in
+[docs/integrations/ci.md](https://github.com/RudrenduPaul/MasteryTrace/blob/main/docs/integrations/ci.md).
+
+## Security
+
+MasteryTrace never `eval()`s or `exec()`s anything read from an event
+log; event data is only ever parsed as JSON/CSV and pattern-matched
+against the validation schema. The generic adapter refuses symlinked
+paths and files over 100 MB before reading them, closing the two most
+likely ways a maliciously crafted event-log path could cause unexpected
+behavior. **Honest note**: this project does not currently publish SLSA
+provenance, Sigstore signatures, or an SBOM, and has no OpenSSF Scorecard
+badge set up -- none of that infrastructure exists yet for either
+distribution, so it isn't claimed here. See
+[SECURITY.md](https://github.com/RudrenduPaul/MasteryTrace/blob/main/SECURITY.md)
+for the vulnerability disclosure process.
+
+## Contributing
+
+See [CONTRIBUTING.md](https://github.com/RudrenduPaul/MasteryTrace/blob/main/CONTRIBUTING.md)
+for the full guide, covering both the TypeScript and Python codebases.
+
+```bash
+cd python
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT, see [LICENSE](https://github.com/RudrenduPaul/MasteryTrace/blob/main/LICENSE).