mta-audit 0.1.0__tar.gz

mta_audit-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,28 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - run: python -m pip install --upgrade pip
+      - run: python -m pip install -e ".[dev]"
+      - run: pytest
+      - run: ruff check src tests
+      - if: matrix.python-version == '3.11'
+        run: mypy

mta_audit-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,23 @@
+name: Publish
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: read
+  id-token: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: python -m pip install --upgrade pip build
+      - run: python -m build
+      - uses: pypa/gh-action-pypi-publish@release/v1

mta_audit-0.1.0/.gitignore

@@ -0,0 +1,14 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.coverage
+.pytest_cache/
+.ruff_cache/
+.venv/
+build/
+dist/
+data/
+*.tsv.gz
+.mypy_cache/
+benchmarks/zeta_*.py
+benchmarks/results/zeta_*/

mta_audit-0.1.0/CHANGELOG.md

@@ -0,0 +1,30 @@
+# Changelog
+
+All notable changes follow [Keep a Changelog](https://keepachangelog.com/) conventions.
+
+## 0.1.0 - 2026-09-15
+
+### Added
+
+- Add a standalone Python library for auditing observational multi-touch
+  attribution reliability.
+- Add typed event validation and converter/non-converter journey construction.
+- Add first-touch, last-touch, linear, and absorbing Markov attribution.
+- Add data-quality, duplicate, path, window, identity-loss, touchpoint-loss,
+  temporal-stability, and model-disagreement checks.
+- Add deterministic journey-level bootstrap attribution intervals and rank frequencies.
+- Add decision robustness and rank/share assumption-sensitivity matrices.
+- Add typed insufficient-evidence statuses that abstain from aggregate scoring.
+- Add six deterministic known-condition stress scenarios.
+- Add standalone Markdown and HTML reports.
+- Add complete-user sampling with report-visible metadata.
+- Add public Criteo and Hugging Face synthetic dataset adapters.
+- Add independent JD-style and Criteo Research label-mismatch synthetic generators.
+- Add a Poisson multi-touch synthesizer compatible with the PubliusV CSV schema.
+- Add optional adapters for IgnazioDS journeys and triangulation user-level MTA tables.
+- Identify Markov only when NULL/non-converter paths are observed.
+- Score identity robustness in the default audit run.
+- Treat date-grain repeated touches as informational, not duplicate failures.
+- Fail data quality when converting events have zero conversion value.
+- Add transparent configurable scoring weights, interpretation bands, and a
+  reproducible score-sensitivity benchmark.

mta_audit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 mta-audit contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mta_audit-0.1.0/PKG-INFO

@@ -0,0 +1,389 @@
+Metadata-Version: 2.5
+Name: mta-audit
+Version: 0.1.0
+Summary: Reliability audits for multi-touch attribution data and models.
+Project-URL: Homepage, https://github.com/morantejr/mta-audit
+Project-URL: Documentation, https://github.com/morantejr/mta-audit/tree/main/docs
+Project-URL: Issues, https://github.com/morantejr/mta-audit/issues
+Author: mta-audit contributors
+License: MIT License
+        
+        Copyright (c) 2026 mta-audit contributors
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: attribution,audit,data-quality,marketing-analytics
+Classifier: Development Status :: 3 - Alpha
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: numpy>=1.26
+Requires-Dist: pandas>=2.1
+Provides-Extra: dev
+Requires-Dist: matplotlib>=3.8; extra == 'dev'
+Requires-Dist: mypy>=1.11; extra == 'dev'
+Requires-Dist: pandas-stubs>=2.1; extra == 'dev'
+Requires-Dist: pyarrow>=15; extra == 'dev'
+Requires-Dist: pytest-cov>=5; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Provides-Extra: parquet
+Requires-Dist: pyarrow>=15; extra == 'parquet'
+Provides-Extra: viz
+Requires-Dist: matplotlib>=3.8; extra == 'viz'
+Description-Content-Type: text/markdown
+
+# mta-audit
+
+**Stress-testing and reliability testing for multi-touch attribution.**
+
+`mta-audit` evaluates whether campaign conclusions remain stable when
+attribution windows, identity quality, touchpoint availability, attribution
+models, sampling, and other measurement assumptions change.
+
+> It does not estimate causal lift or incrementality.
+
+This project is a standalone reliability library. Attribution systems still
+allocate credit; this package scores whether journeys and model outputs are
+stable enough to use.
+
+Optional public-dataset adapters only map foreign tables onto the package event
+schema.
+
+## What it catches
+
+The public Criteo benchmark uses an explicit 8,000-user sample: 21,307 mapped
+events, 639 anonymized campaigns, 8,144 journeys, and 586 conversions.
+
+- Injecting 1–10% duplicate events was detected every time and reduced the
+  scoped data-quality score from **72.35 to 50.20–50.70**.
+- Conversion-window campaign ranks reached a minimum Spearman correlation of
+  **0.588** across 1/3/7/14/30-day windows.
+- Dropping 30% of eligible touchpoints produced **2.2% attribution-share
+  drift** and reduced campaign rank correlation to **0.972**.
+- Fragmenting 20% of identities increased journeys from **8,144 to 8,908**.
+
+See the [rendered sample audit](benchmarks/results/sample_report.html), its
+[Markdown source](benchmarks/results/sample_report.md), and the full
+[Criteo reliability scorecard](benchmarks/results/criteo_benchmark.md).
+The sample is synthetic and deterministic; the Criteo figures are measured,
+not illustrative.
+
+## Installation
+
+Python 3.11–3.13 is supported.
+
+```bash
+pip install "mta-audit @ git+https://github.com/morantejr/mta-audit.git@v0.1.0"
+pip install "mta-audit[viz] @ git+https://github.com/morantejr/mta-audit.git@v0.1.0"
+```
+
+PyPI publication is pending its account-side Trusted Publisher registration.
+After that is connected, the shorter command will be `pip install mta-audit`.
+For an unreleased main-branch snapshot:
+
+```bash
+pip install "mta-audit @ git+https://github.com/morantejr/mta-audit.git"
+```
+
+For contributors: `pip install -e ".[dev]"`.
+
+## Quickstart
+
+```python
+import pandas as pd
+from mta_audit import MTAAudit
+
+data = pd.DataFrame({
+    "user_id": ["a", "a", "b", "b"],
+    "timestamp": pd.to_datetime([
+        "2026-01-01", "2026-01-03", "2026-01-02", "2026-01-04"
+    ]),
+    "channel": ["search", "email", "social", "direct"],
+    "conversion": [False, True, False, True],
+})
+
+report = MTAAudit(data=data).run(
+    bootstrap=True,
+    n_bootstrap=200,
+    random_state=42,
+)
+print(report.summary())
+print(report.decision_robustness())
+report.sensitivity_matrix(metric="rank")
+report.to_dataframe()
+report.to_json()
+```
+
+Example output:
+
+```text
+MTA Audit Reliability Report
+Overall audit score: 67.5/100 — Fragile
+Component scores:
+- Data Quality: …
+- Window Stability: …
+- Identity Robustness: …
+- Path Quality: …
+- Model Agreement: …
+- Temporal Stability: insufficient evidence
+Checks run: 12 | Visible findings: 7
+Markov removal effects: not evaluated
+Channel grain: channel
+
+Decision robustness:
+Search remains top-two in 94% of tested scenarios.
+Display and Social frequently change position; avoid a strong reallocation
+between them based on this observational attribution analysis alone.
+```
+
+Custom schemas and the advanced API remain concise:
+
+```python
+audit = MTAAudit(
+    data=data,
+    user_col="visitor_id",
+    timestamp_col="event_time",
+    channel_col="source",
+    conversion_col="converted",
+    scoring_weights={"data_quality": 2, "model_agreement": 1},
+)
+report = audit.run(
+    attribution_models=["first_touch", "last_touch", "linear", "markov"],
+    conversion_windows=[7, 14, 30, 60],
+    checks=["data_quality", "conversion_window", "model_disagreement"],
+    simulations={"identity_loss": {"rates": [0.05, 0.10, 0.20]}},
+    bootstrap=True,
+    n_bootstrap=500,
+    random_state=42,
+)
+```
+
+For reusable bootstrap settings:
+
+```python
+from mta_audit import BootstrapConfig
+
+report = MTAAudit(data).run(
+    bootstrap=BootstrapConfig(
+        n_iterations=500,
+        confidence_level=0.90,
+        random_state=42,
+    )
+)
+report.bootstrap.to_dataframe()
+```
+
+Bootstrap resamples complete journeys, never touchpoint rows. Its percentile
+intervals and rank frequencies measure observational attribution stability;
+they are not causal confidence intervals.
+
+## Checks
+
+- Required fields, nulls, timestamps, channels, and duplicate events
+- Conversion-window sensitivity and delayed-converter contamination
+- First-touch, last-touch, linear, and absorbing Markov model disagreement
+- Path sparsity and channel concentration
+- Weekly or monthly temporal stability
+- Converter/non-converter exposure comparison
+- Deterministic identity, touchpoint, impression, and click-loss simulations
+
+## Methodology
+
+Every audit emits a normalized 0–100 **audit robustness score**, severity,
+observed metric, and recommendation. The score does not mean percent correct.
+Related checks feed six reliability components:
+`data_quality`, `window_stability`, `identity_robustness`, `path_quality`,
+`model_agreement`, and `temporal_stability`. The overall formula is:
+
+```text
+reliability = Σ(component score × normalized available-component weight)
+```
+
+Only completed components are included; skipped checks are never treated as
+perfect. Attribution models allocate observed conversion value. Markov credit
+uses channel-removal effects from an absorbing transition chain.
+
+Qualitative bands are Robust (90–100), Stable (80–89), Caution (70–79),
+Fragile (60–69), and Highly Fragile (<60). The component scorecard is more
+important than the aggregate.
+
+## Shareable reports and local sampling
+
+```python
+markdown = report.to_markdown()
+html = report.to_html()
+
+from mta_audit import sample_for_audit
+sample = sample_for_audit(data, max_journeys=100_000, random_state=42)
+sampled_report = MTAAudit(sample).run()
+print(sampled_report.sample_info)
+```
+
+Sampling preserves complete user histories and marks the resulting report.
+This remains a local pandas package; no warehouse or distributed execution is
+included.
+
+## Thresholds and configuration
+
+Default scoring weights are 20%, 15%, 15%, 15%, 20%, and 15% in the component
+order above. Any non-negative relative values can be passed through
+`scoring_weights`; they are normalized automatically over available components.
+These are explicit policy defaults, not empirically estimated constants:
+data quality and model agreement receive 20% because invalid inputs and
+conclusion-changing model choice are direct decision risks; the other four
+dimensions receive equal 15% weights. The round-number bands are likewise
+communication thresholds, not confidence intervals.
+
+The reproducible [score-policy sensitivity report](benchmarks/results/scoring_sensitivity.md)
+reweights the committed Criteo components and shifts every rating cutoff by
+five points. It shows how much the aggregate and label depend on those policy
+choices. Component scores must remain visible; users should override weights
+when their decision context differs.
+Audit-specific threshold dataclasses such as `ConversionWindowThresholds`,
+`ContaminationThresholds`, and `TemporalStabilityThresholds` expose warning
+cutoffs. See [the methodology](docs/methodology.md) for formulas.
+
+## Visualization
+
+With the `viz` extra installed, reports provide `plot_scorecard()`,
+`plot_model_comparison()`, `plot_window_sensitivity()`, and
+`plot_channel_volatility()`. Each returns a matplotlib `Axes`.
+
+## Limitations
+
+Results depend on identity resolution, channel taxonomy, event collection, and
+the selected lookback/conversion windows. Sparse paths make transition estimates
+unstable. Converter/non-converter comparisons are confounded and are not lift
+estimates. Tracking-loss simulations test sensitivity to specified corruption,
+not the true unknown missingness process.
+
+Converter-only sequencing tables do **not** identify Markov removal effects.
+The report sets `markov_identified=false` in that case. Date-grain
+timestamps are not treated as duplicates. Campaign-name "channels" are flagged as
+fine grain.
+
+Kaggle does not currently publish a licensed event-level multi-touch path dataset
+with converters and non-converters comparable to Criteo. Public fixtures:
+
+- Criteo Attribution Modeling (real impressions; see walkthrough below)
+- Hugging Face [synthetic attribution benchmark](https://huggingface.co/datasets/lucianfialho/synthetic-attribution-benchmark) (CC BY 4.0)
+- Independent generators inspired by JD MTA (Du et al., 2019) and Criteo Research
+  robust-label experiments (Bompaire et al., 2020): `simulate_jd_mta_events`,
+  `simulate_criteo_label_events`, `simulate_synthesizer_events`
+- Optional CSV adapters for IgnazioDS journeys and the triangulation project's
+  user-level `df_mta.csv` (`load_ignazio_attribution`, `load_triangulation_mta`).
+  Those files are not stored in git. Weekly MMM / geo tables are out of scope.
+
+## Roadmap
+
+Current stage: **0.1.0 alpha**. Next:
+
+1. Keep 0.1.x installable and documented (stability contract in
+   [docs/stability.md](docs/stability.md)).
+2. **Beta** when `run()` / report fields stay boring across a few tagged
+   releases and the Criteo reference scorecard is re-run before each tag.
+3. **1.0** when scoring defaults are frozen and third parties can depend on
+   the public API. Warehouse extracts stay out of this repository.
+
+Later, optional work: more public-dataset adapters and richer performance
+benchmarks.
+
+See [concepts](docs/concepts.md), [methodology](docs/methodology.md),
+[API reference](docs/api.md), [stability](docs/stability.md),
+[releasing](docs/releasing.md), the
+[external practitioner review protocol](docs/practitioner-review.md), and
+[examples](examples/quickstart.py).
+
+## Criteo walkthrough
+
+The public Criteo Attribution Modeling dataset (CC-BY-NC-SA-4.0) is the
+real-world integration set for this package. **It is not stored in git.**
+
+1. Download `criteo_attribution_dataset.tsv.gz` from
+   [Hugging Face](https://huggingface.co/datasets/criteo/criteo-attribution-dataset)
+   (Criteo's published copy of Diemert et al., 2017).
+2. Save it as `./data/criteo_attribution_dataset.tsv.gz`, or let the loader
+   cache it under `~/.cache/mta-audit/` with `download=True`.
+3. Map impressions through the adapter — do **not** pass Criteo's impression-level
+   `conversion` flag straight into `conversion_col`.
+
+```python
+from mta_audit import MTAAudit
+from mta_audit.datasets import load_criteo_attribution, CriteoAttributionAdapter
+
+# Explicit representative sample; pass sample_users=None for the full file.
+df = load_criteo_attribution(path="./data/criteo_attribution_dataset.tsv.gz", sample_users=20000)
+# equivalent: CriteoAttributionAdapter("./data/criteo_attribution_dataset.tsv.gz").load()
+
+report = MTAAudit(data=df).run(
+    attribution_models=["first_touch", "last_touch", "linear", "markov"]
+)
+print(report.summary())
+```
+
+Public synthetic journeys (named channels + NULL paths, CC BY 4.0):
+
+```python
+from mta_audit.datasets import load_synthetic_attribution
+
+events = load_synthetic_attribution(download=True)
+report = MTAAudit(data=events).run(attribution_models=["linear", "markov"])
+print(report.markov_identified)
+```
+
+Independent research-style synthetics (no third-party generators vendored):
+
+```python
+from mta_audit.datasets import (
+    simulate_criteo_label_events,
+    simulate_jd_mta_events,
+    simulate_synthesizer_events,
+)
+
+jd = simulate_jd_mta_events(n_users=200, random_state=42)
+labels = simulate_criteo_label_events(n_users=400, random_state=42)
+synth = simulate_synthesizer_events(n_users=300, random_state=42)
+```
+
+Unit tests never require the file. Integration tests and the benchmark do:
+
+```bash
+pytest                    # default, no Criteo download
+pytest -m criteo          # loads the local/public dataset
+python benchmarks/criteo_benchmark.py --download --sample-users 20000
+```
+
+Results from a completed run (8,000-user explicit sample) live in
+[`benchmarks/results/criteo_benchmark.md`](benchmarks/results/criteo_benchmark.md).
+On that run, injected duplicates dropped the data-quality score from ~72 to ~50,
+20% identity loss raised journey count from 8,144 to 8,908, and conversion-window
+ranks correlated at only 0.59. The notebook `examples/criteo_example.ipynb` is
+the measurement-team walkthrough.
+
+Criteo campaigns are anonymized. Treat outputs as campaign-level attribution
+reliability diagnostics, not named-channel budget recommendations.
+
+Removal-effect Markov on ~700 campaigns is implemented with a numpy transition
+matrix. `MarkovAttribution(max_channels=N)` can explicitly collapse a long tail
+into `__OTHER__`; the benchmark does **not** do that by default.