goad-toolkit 0.2.7__tar.gz → 0.2.8__tar.gz

goad_toolkit-0.2.8/.claude/worktrees/quizzical-solomon-b62511/.claude/settings.local.json

@@ -0,0 +1,7 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(cd /Users/rgrouls/code/courses/goad_toolkit && cat pyproject.toml)"
+    ]
+  }
+}

goad_toolkit-0.2.8/.claude/worktrees/quizzical-solomon-b62511/CHANGELOG.md

@@ -0,0 +1,152 @@
+# Changelog
+
+## 0.2.7
+
+### Added
+
+- `ScatterPlot`, `RegPlot` and `CorrelationHeatmap`: the relation plots for two-variable
+  questions, in their order of use — a scatter first, a fit once the scatter has shown which
+  kind, a correlation matrix when there are too many pairs to plot by hand.
+- `ScatterPlot` defaults `alpha` below 1, since overplotting is the standard way a scatter
+  misreports its own density.
+- `RegPlot` wraps `sns.regplot`, which is axis-level and so can draw onto a layer's axis.
+  `scatter=False` lays a fit over an existing `ScatterPlot`; `lowess` together with
+  `order != 1` raises, since both answer "what shape" and only one can apply.
+- `CorrelationHeatmap` pins the colour scale to `[-1, 1]` centred on 0 and defaults to a
+  diverging colormap, so the sign of a correlation stays readable. Non-numeric columns are
+  dropped, and a frame with nothing numeric left raises rather than drawing an empty grid.
+- docs/04 §4.4 covers the three and their layering; docs/08 §8.10 documents the seaborn
+  axis-label override — a seaborn call with `x=`/`y=` writes the column names over
+  `PlotSettings`.
+
+## 0.2.6
+
+### Added
+
+- `NullDistribution` and `NullResult`: shuffle one column, re-measure, and compare the
+  observed value against the cloud that produces. The statistic stays the caller's —
+  `NullDistribution` takes a callable and ships no ready-made statistics.
+- `NullResult.p_value` counts rather than tests. Both one-sided counts include the observed
+  arrangement, so the smallest value it can return is `1 / (n_iter + 1)`.
+- `NullPlot`: the null as a grey `HistogramPlot` with the observed value marked, its value
+  and p-value in the legend, stated as a bound (`p < …`) once the count floor is reached,
+  since an exact number there would overstate what a thousand shuffles support.
+- `VerticalLine`, the numeric counterpart to `VerticalDate` and the layer `NullPlot`
+  composes.
+- docs/06 §6.7 carries the prose, including the two things the shuffle test does not defend
+  against: the garden of forking paths, and a confounder that shuffling breaks along with
+  the association.
+
+## 0.2.5
+
+### Added
+
+- `RegexFeature` transform: one new column derived from a regular expression over a text
+  column, in `count` (occurrences per row), `has` (bool) or `extract` (first capture group,
+  NaN where nothing matched) mode. The output column is named by `feature`, because
+  `Pipeline.add(name=...)` already claims `name` for the step.
+- `mode="extract"` reports its match rate through `loguru`: it is the one mode that fails
+  silently, filling with NaN where a wrong pattern in the other two gives a visibly zero or
+  all-`False` column.
+- docs/02 §2.5 documents the shipped transform with its mode table and coverage log; §2.4
+  keeps `IsWeekend` as the write-your-own example.
+
+## 0.2.4
+
+### Added
+
+- `bernoulli`, `binomial`, `nbinom` and `beta` are default registry families. `pareto` is
+  deliberately not among them: lesson 4 has the student register it as the extensibility
+  exercise, and the class docstring shows the call.
+- `QQPlot`: sample quantiles against a frozen distribution's theoretical quantiles with a
+  `y = x` reference line, for the tail mismatches a `PlotFits` histogram is too
+  bulk-dominated to show.
+- `ECDFPlot`: one or two empirical CDFs, bin-free.
+- `fit_table`: turns a `fit()` result list into a dataframe ranked by log-likelihood — one
+  row per family with params, log-likelihood, KS statistic and p-value, and which criteria
+  it won. `FailedFit` entries carry their message and sort last, since a missing likelihood
+  is not a small one.
+
+### Fixed
+
+- Log-likelihood uses `logpmf` for discrete families and `logpdf` for continuous ones, so
+  every discrete family reports a real likelihood and the comparison between them means
+  something.
+- The lower bound for a single-parameter discrete family is `1e-3`, low enough for a
+  probability (bernoulli's `p`) as well as a count-scale rate (poisson's lambda).
+- `best_likelihood` and `best_ks` are marked independently: a KS p-value that never clears
+  the `p > 0` threshold — routine for a discrete family on a large, tied sample — leaves
+  the likelihood winner marked. An unknown `criterion` raises `ValueError` whether or not
+  a winner was found.
+
+## 0.2.3
+
+### Added
+
+- `TimeFeatures` transform: derives `date`, `hour`, `day_name`, `isoweek` and `year_week`
+  from a timestamp column, with a `features=[...]` selector. The default is the three
+  lesson 1 relies on, and an unknown feature name raises `ValueError`.
+- `DecomposePlot`: `seasonal_decompose` as the four-panel observed/trend/seasonal/residual
+  view, composing a `LinePlot` per panel.
+- `ACFPlot`: autocorrelations as bars with a Bartlett confidence band. NaN input raises,
+  since `statsmodels.acf` returns an all-NaN result instead of raising.
+
+### Changed
+
+- `statsmodels` is a dependency.
+
+## 0.2.2
+
+### Added
+
+- Analysis-method rows on exploratory versus confirmatory questions and on mechanism: which
+  of the two a question is decides what would count as evidence, and a significant result
+  with no nameable mechanism points at a confounder rather than a discovery.
+- Analysis-method rows on the unit of analysis — the unit a claim is about against the unit
+  of a row, and the number of independent units behind each group being compared — with
+  §9.3 naming pseudoreplication and showing the n=4000/n=5 bar chart.
+- An analysis-method row on how many comparisons were considered, with §9.6 on the garden
+  of forking paths: the arithmetic of fifteen tries at a one-in-twenty threshold, and the
+  three defences (write the question down, split the data, report the count).
+- A visual-critique row separating "A differs from B" from "A is the highest", since every
+  set of numbers has a maximum and a ranking claim needs the ranking to be stable.
+- A counting step at the head of the distribution method: how many observations, and how
+  many independent ones.
+
+These land in docs/09-analysis-method.md and in the matching `goad_mcp` checklists.
+
+## 0.2.1
+
+### Breaking
+
+- **`FileConfig` gained `date_column` and `date_index`.** `FileHandler.load` reads csv and
+  parquet (reader picked from the suffix), parses `date_column` as dates and, when
+  `date_index` is true, moves it into the index. The defaults (`"date"`, `True`) keep the
+  covid pipeline working unchanged, but a `FileConfig` subclass that overrode `load` to
+  support another shape can now express it as configuration instead. A file whose suffix has
+  no reader, or a `date_column` that is not in the frame, raises `ValueError`. Parquet needs
+  `pyarrow`, available as the `parquet` extra.
+- **`DistributionRegistry` is a normal class.** Constructing one no longer returns a shared
+  process-wide instance, so a registration affects only the registry it was made on. Code
+  that registered a family on one registry and relied on a separately constructed
+  `DistributionFitter()` picking it up must now pass it: `DistributionFitter(registry)`.
+
+### Added
+
+- `goad` console script: fits every family in the registry to one column of a csv or parquet
+  file and prints the fits ranked by log-likelihood (`goad data/residuals.csv residual`).
+- `DistributionFitter` accepts an optional `DistributionRegistry`.
+- `filehandler.reader_for`, `filehandler.write_frame` and `filehandler.parse_dates` as
+  reusable pieces; `FileHandler.save` writes parquet as well as csv.
+- A pytest suite under `tests/`.
+
+### Fixed
+
+- `BarWithDates.build` and `VerticalDate.build` return `(fig, ax)` like every other `build`.
+- Plots draw on `self.ax` / `self.fig` instead of module-level `plt` calls, so a layer placed
+  with `plot_on_axes` lands on the axis it was handed rather than on matplotlib's current one.
+- The `goad` console script points at a module that exists.
+
+### Changed
+
+- `__version__` is read from the installed package metadata.

goad_toolkit-0.2.8/.remember/archive.md

@@ -0,0 +1,3 @@
+# Archive
+
+(empty)

goad_toolkit-0.2.8/.remember/logs/memory-2026-08-12.log

@@ -0,0 +1,28 @@
+10:46:23 [hook] session-start: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:46:23 [hook] run-consolidation: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:46:24 [consolidation] start
+10:46:34 [consolidation] tokens: 9+6254cache→792out ($0.013050)
+10:46:34 [consolidation] done: 1 files consolidated
+10:46:36 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:46:49 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:46:54 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:00 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:04 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:10 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:15 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:22 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:27 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:29 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:39 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:46 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:50 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:59 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:59 [hook] save-session: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:47:59 [extract] session 16b4788a-5507-4216-98ec-06bac13a9aba
+10:47:59 [extract] 17 exchanges (1 human)
+10:47:59 [extract] 1 human msgs < 3, skip
+10:48:45 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:48:50 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:48:54 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:49:08 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember
+10:49:13 [hook] post-tool: PROJECT_DIR=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical-solomon-b62511 PIPELINE_DIR=/Users/rgrouls/.claude/plugins/cache/claude-plugins-official/remember/0.19.0 PYTHON=python3 REMEMBER_DIR=/Users/rgrouls/code/courses/goad_toolkit/.remember

goad_toolkit-0.2.8/.remember/recent.md

@@ -0,0 +1,7 @@
+# Recent
+
+## 2026-08-10
+Fixed 5 defects in goad_toolkit (feat/docs-and-mcp), shipped v0.2.0. Removed DistributionRegistry singleton, parameterized FileHandler.load for csv/parquet with date col support, fixed BasePlot.build return contracts. Refactored visualizer.py from module-level plt.* to instance methods (self.ax/.fig), added CLI.
+
+## Identity Candidates
+- IDENTITY CANDIDATE: Architectural preference for instance methods over module-level state and singleton patterns

goad_toolkit-0.2.8/.remember/tmp/capture-alive

@@ -0,0 +1 @@
+16b4788a-5507-4216-98ec-06bac13a9aba

goad_toolkit-0.2.8/.remember/tmp/last-save-ts

@@ -0,0 +1 @@
+1786524479

goad_toolkit-0.2.8/.remember/tmp/save-session.pid

@@ -0,0 +1 @@
+70230

{goad_toolkit-0.2.7 → goad_toolkit-0.2.8}/.remember/tmp/session-slug

@@ -4,4 +4,4 @@ project_dir=/Users/rgrouls/code/courses/goad_toolkit/.claude/worktrees/quizzical
 slug=-Users-rgrouls-code-courses-goad-toolkit--claude-worktrees-quizzical-solomon-b62511
 sessions_dir=/Users/rgrouls/.claude/projects/-Users-rgrouls-code-courses-goad-toolkit--claude-worktrees-quizzical-solomon-b62511
 memory_dir=/Users/rgrouls/code/courses/goad_toolkit/.remember
-session_id=90574e63-97cd-4c11-87e5-23b656c56fd9
+session_id=16b4788a-5507-4216-98ec-06bac13a9aba

{goad_toolkit-0.2.7/.claude/worktrees/quizzical-solomon-b62511 → goad_toolkit-0.2.8}/MCP_SERVER.md

@@ -103,13 +103,13 @@ You pin a version with `GOAD_REF` — a git ref (branch, tag, or commit). That o
 selects both the script (in the URL) and the docs it fetches (inside the script), so a whole
 cohort runs exactly the same thing and the two cannot drift apart.
 
-> **Note:** the docs and this server land in **v0.2.0**. Until that tag exists, point
-> `GOAD_REF` at the branch (`feat/docs-and-mcp`) or work from a local clone (see below).
+> **Note:** the docs and this server land in **v0.2.2**. 
+> You can also point `GOAD_REF` to a branch or work from a local clone (see below).
 
 ### Claude Code
 
 ```bash
-claude mcp add goad -e GOAD_REF=v0.2.0 -- \
+claude mcp add goad -e GOAD_REF=v0.2.2 -- \
   sh -c 'uv run --no-project https://raw.githubusercontent.com/raoulg/goad_toolkit/$GOAD_REF/goad_mcp.py'
 ```
 
@@ -123,7 +123,7 @@ Add to `.cursor/mcp.json` (per-project) or `~/.cursor/mcp.json` (global):
     "goad": {
       "command": "sh",
       "args": ["-c", "uv run --no-project https://raw.githubusercontent.com/raoulg/goad_toolkit/$GOAD_REF/goad_mcp.py"],
-      "env": { "GOAD_REF": "v0.2.0" }
+      "env": { "GOAD_REF": "v0.2.2" }
     }
   }
 }
@@ -137,7 +137,7 @@ Add to `~/.codex/config.toml`:
 [mcp_servers.goad]
 command = "sh"
 args = ["-c", "uv run --no-project https://raw.githubusercontent.com/raoulg/goad_toolkit/$GOAD_REF/goad_mcp.py"]
-env = { GOAD_REF = "v0.2.0" }
+env = { GOAD_REF = "v0.2.2" }
 ```
 
 ### Claude Desktop
@@ -151,7 +151,7 @@ app:
     "goad": {
       "command": "sh",
       "args": ["-c", "uv run --no-project https://raw.githubusercontent.com/raoulg/goad_toolkit/$GOAD_REF/goad_mcp.py"],
-      "env": { "GOAD_REF": "v0.2.0" }
+      "env": { "GOAD_REF": "v0.2.2" }
     }
   }
 }
@@ -187,7 +187,7 @@ add with the new tag:
 
 ```bash
 claude mcp remove goad -s user
-claude mcp add goad -s user -e GOAD_REF=v0.2.0 -- \
+claude mcp add goad -s user -e GOAD_REF=v0.2.2 -- \
   sh -c 'uv run --no-project https://raw.githubusercontent.com/raoulg/goad_toolkit/$GOAD_REF/goad_mcp.py'
 ```
 

{goad_toolkit-0.2.7 → goad_toolkit-0.2.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: goad-toolkit
-Version: 0.2.7
+Version: 0.2.8
 Summary: An extensible toolkit for Goal Oriented Analysis of Data
 Project-URL: Github, https://github.com/raoulg/goad_toolkit
 Author-email: raoul grouls <Raoul.Grouls@han.nl>

{goad_toolkit-0.2.7/.claude/worktrees/quizzical-solomon-b62511 → goad_toolkit-0.2.8}/README.md

@@ -183,6 +183,36 @@ For the [kstest](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stat
 The plots are sorted by log-likelihood, which means there is no good fit with a distribution in this case.
 ![residuals](img/distribution_fit.png)
 
+### 🎲 Is it real? The shuffle test
+
+Shuffle the labels, measure again, and see where the real number lands. No test, no
+assumptions, no table of critical values — just a picture of what your measurement does when
+the label means nothing:
+
+```python
+from goad_toolkit.analytics import NullDistribution
+from goad_toolkit.visualizer import NullPlot, PlotSettings
+
+def gap(frame):
+    means = frame.groupby("is_bot")["length"].mean()
+    return means[True] - means[False]
+
+result = NullDistribution(gap, n_iter=2000, seed=1).run(data, label="is_bot")
+settings = PlotSettings(
+    title="Mean message length: bots minus humans",
+    xlabel="difference in mean length (characters)",
+    ylabel="density under shuffled labels",
+)
+NullPlot(settings).plot(result=result)
+```
+
+![null distribution](img/null-distribution.png)
+
+The grey cloud is the statistic under shuffled labels; the line is what the real data did.
+Writing the statistic is your job — that is the claim. See
+[Models and residuals](docs/06-models-and-residuals.md) §6.7 for what the p-value can and
+cannot carry.
+
 ### 🧩 Extending with Custom Distributions
 
 You can easily register new distributions:

{goad_toolkit-0.2.7/.claude/worktrees/quizzical-solomon-b62511 → goad_toolkit-0.2.8}/docs/02-pipelines.md

@@ -58,6 +58,8 @@ input frame is safe, and the steps stay cheap.
 | `SelectDataRange` | keep rows in a date range | `start_date`, `end_date` |
 | `RollingAvg` | rolling mean, drops the leading NaNs | `column`, `window`, `rename` |
 | `ZScaler` | standardise to mean 0, std 1 | `column`, `rename` |
+| `TimeFeatures` | derive calendar columns from a timestamp | `column`, `features` |
+| `RegexFeature` | count / flag / extract a pattern in a text column | `column`, `pattern`, `feature`, `mode` |
 
 `rename=True` writes to a new column (`deaths_shifted`, `deaths_zscore`, …) instead of
 overwriting. Prefer it. An overwritten column is a step you cannot debug, and the whole
@@ -75,20 +77,17 @@ Subclass `TransformBase` and implement `transform`. That is the entire contract.
 from goad_toolkit.datatransforms import TransformBase
 import pandas as pd
 
-class TimeFeatures(TransformBase):
-    """Derive calendar features from a timestamp column."""
+class IsWeekend(TransformBase):
+    """Flag Saturday/Sunday from a timestamp column."""
 
     def transform(self, data: pd.DataFrame, column: str) -> pd.DataFrame:
         ts = pd.to_datetime(data[column])
-        data["date"] = ts.dt.date
-        data["hour"] = ts.dt.hour
-        data["day_name"] = ts.dt.day_name()
-        data["isoweek"] = ts.dt.isocalendar().week
+        data["is_weekend"] = ts.dt.dayofweek >= 5
         return data
 ```
 
 ```python
-pipeline.add(TimeFeatures, column="timestamp")
+pipeline.add(IsWeekend, column="timestamp")
 ```
 
 What the base class does for you:
@@ -106,39 +105,39 @@ Two rules for your `transform`:
 - **Name your parameters explicitly** in the signature. `def transform(self, data, column,
   window)` documents itself; `**kwargs` does not, and the validation cannot help you.
 
-## 2.5 A transform for feature enrichment
+## 2.5 `RegexFeature`: feature enrichment from text
 
-The single most useful custom transform for text data, and a good template:
+The single most useful transform for text data, and the reason `TransformBase` is worth
+subclassing at all. It ships:
 
 ```python
-class RegexFeature(TransformBase):
-    """Add a feature extracted from a text column with a regular expression."""
-
-    def transform(
-        self,
-        data: pd.DataFrame,
-        column: str,
-        pattern: str,
-        feature: str,
-        mode: str = "count",
-    ) -> pd.DataFrame:
-        text = data[column].fillna("")
-        if mode == "count":
-            data[feature] = text.str.count(pattern)
-        elif mode == "has":
-            data[feature] = text.str.contains(pattern, regex=True)
-        elif mode == "extract":
-            data[feature] = text.str.extract(pattern, expand=False)
-        else:
-            raise ValueError(f"mode must be count/has/extract, got {mode!r}")
-        return data
-```
+from goad_toolkit.datatransforms import RegexFeature
 
-```python
 pipeline.add(RegexFeature, name="url_flag",
              column="message", pattern=r"https?://\S+", feature="has_url", mode="has")
 ```
 
+Three modes, each writing one new column named by `feature`:
+
+| `mode` | writes | use for |
+|---|---|---|
+| `"count"` | how many times the pattern occurs (int) | how many URLs, how many question marks |
+| `"has"` | whether it occurs at all (bool) | flags you will group or filter on |
+| `"extract"` | the first capture group, NaN where nothing matched | pulling a value *out* of the text |
+
+`"extract"` needs exactly one capture group in `pattern`, and it is the mode worth being
+careful with. `count` and `has` fail visibly when a pattern is wrong — a column of all zeros
+or all `False` is hard to miss. Extraction fails *silently*, filling with NaN, so it reports
+its own coverage through `loguru`:
+
+```
+mentions: extracted 'addressed_to' from 92,415/627,172 rows (14.7%); 534,757 rows had no match
+```
+
+That number is the point. A pattern that matches 15% of rows may be exactly right — on IRC,
+most messages do not address anyone — or it may be silently broken. The log line makes you
+decide which, instead of finding out four notebooks later.
+
 Note the two different `name`s: `Pipeline.add(name=...)` names the *step*, and
 `TransformBase.__init__` consumes it. So the new-column parameter has to be called something
 else — `feature` here. Any transform that wants to name an output column hits this.

{goad_toolkit-0.2.7/.claude/worktrees/quizzical-solomon-b62511 → goad_toolkit-0.2.8}/docs/03-plot-composition.md

@@ -148,12 +148,26 @@ step 3.
 | `LinePlot` | `sns.lineplot` | the primitive most compositions build on |
 | `ComparePlot` | two lines with labels | `LinePlot` twice via `plot_on` |
 | `VerticalDate` | a labelled vertical line at a date | a layer, not a plot |
+| `VerticalLine` | the same at a number | a layer, not a plot |
 | `ComparePlotDate` | `ComparePlot` + `VerticalDate` | |
+| `BarPlot` | `sns.barplot`, one bar per category | see [Five families §4.1](04-five-families.md) |
+| `GroupedBarPlot` | bars split by `hue` | two categorical variables at once |
+| `HeatmapPlot` | `sns.heatmap`, pivoting the frame if asked | when both variables have many levels |
+| `BarbellPlot` | before and after per category, joined | the change is the message |
+| `HighlightCategory` | recolours bars: grey, plus the one that matters | a layer, and it needs no help from what drew them |
+| `ScatterPlot` | `sns.scatterplot` | the first plot of any relation question |
+| `RegPlot` | `sns.regplot` — `fit_reg`, `lowess`, `order` | `scatter=False` layers it over a `ScatterPlot` |
+| `CorrelationHeatmap` | `.corr()` as a heatmap, scale pinned to [-1, 1] | where to look next, not a ranking |
 | `BarWithDates` | bars with a month locator on the x axis | |
 | `ResidualPlot` | `BarWithDates` + `VerticalDate` | the residual-over-time view |
+| `DecomposePlot` | `seasonal_decompose`, one panel each | observed/trend/seasonal/residual, overrides `plot` |
+| `ACFPlot` | `acf` as bars with a confidence band | see [Five families §4.2](04-five-families.md) |
 | `HistogramPlot` | `sns.histplot`, `stat="density"` | density so a pdf can be overlaid |
 | `DistPlot` | a scipy distribution's pdf or pmf | estimates its own x-range from `ppf` |
 | `PlotFits` | histogram + fitted pdf, one panel per fit | see [Distributions](05-distributions.md) |
+| `QQPlot` | sample quantiles vs. a distribution's theoretical quantiles | the tails, not the bulk — see [Distributions §5.5](05-distributions.md) |
+| `ECDFPlot` | one or two empirical CDFs, bin-free | the right tool for comparing two samples |
+| `NullPlot` | a shuffled statistic, with the observed value marked | `HistogramPlot` + `VerticalLine`, see [Models and residuals §6.7](06-models-and-residuals.md) |
 
 `HistogramPlot` defaults to `sqrt(n)` bins capped at 50. That default is a starting point,
 not an answer — bin count changes what a histogram appears to say, and choosing it is part of
@@ -164,6 +178,30 @@ falling back to `pmf` for discrete families. Left to itself it picks an x-range
 0.1st to the 99.9th percentile, which is right for most things and wrong for anything with a
 very heavy tail — pass `x_range` explicitly there.
 
+`QQPlot` takes a sample and a *frozen* distribution and plots sorted data against the
+distribution's quantiles at matching plotting positions, with a y=x reference line —
+where `PlotFits`' histogram is dominated by the bulk of the data, this is where a tail
+mismatch actually shows up.
+
+`ECDFPlot` plots one sample, or two with `compare=`, as step functions of the empirical CDF —
+no bin width to choose, so nothing about the shape is a plotting decision.
+
+`HighlightCategory` is the clearest case for why layers beat settings. It recolours the bars
+already on the axis — grey for everything, one colour for the categories you name — so it
+works over `BarPlot`, over a bare `sns.barplot`, over pandas' `.plot.bar`, and over the
+`BasePlot` subclass you wrote yourself. None of them need to know it exists:
+
+```python
+settings = PlotSettings(title="Survival rate by deck", highlight=["D"])
+bars = BarPlot(settings)
+bars.plot(data=decks, x="deck", y="survival")
+bars.plot_on(HighlightCategory(settings))
+```
+
+`highlight`, `base_color` and `highlight_color` live on `PlotSettings` so that the call stays
+about the data, but they are only defaults — pass `categories=` at the call to override.
+Naming a category that is not on the axis raises, and lists the ones that are.
+
 ## 3.7 Writing your own plot
 
 Two questions decide the shape:
@@ -216,9 +254,11 @@ lines of code.
   `self.ax.axvline(...)`, `self.fig.suptitle(...)`. Keeping to them is what makes
   `plot_on_axes` reliable.
 
-`PlotFits` overrides `plot` rather than `build`, because it needs to know how many panels to
-create before the figure exists. Its `build` is a no-op. It is the exception, not a pattern to
-copy.
+`PlotFits` and `DecomposePlot` override `plot` rather than `build`, because both need to know
+how many panels to create before the figure exists — variable for `PlotFits` (one per fit),
+fixed at four for `DecomposePlot`. Either way, `build` alone cannot decide the panel count
+before `create_figure` runs, so both leave it as a no-op. This is the exception, not a pattern
+to copy for a plot that only ever needs one axis.
 
 ---
 

{goad_toolkit-0.2.7/.claude/worktrees/quizzical-solomon-b62511 → goad_toolkit-0.2.8}/docs/04-five-families.md

@@ -42,9 +42,22 @@ barbell when each group has a before and an after and the *change* is the point.
   labelled adds nothing and costs attention. Grey, plus colour on the one bar you are talking
   about.
 
-**In GOAD:** the category counts come from a `Pipeline` (`groupby` in a transform), and the
-bars from a `BasePlot` subclass over `sns.barplot`. Ordering and the top-k cut belong in the
-pipeline, so they are recorded, not in the plot call, where they are invisible.
+**In GOAD:** `BarPlot`, `GroupedBarPlot`, `HeatmapPlot` and `BarbellPlot`, with
+`HighlightCategory` layered on top when one category is the point.
+
+The category counts come from a `Pipeline` (`groupby` in a transform). Ordering and the top-k
+cut belong in the pipeline too, so they are recorded, not in the plot call, where they are
+invisible — which is why `BarPlot` draws the frame in the order it is handed and offers no
+`sort` of its own.
+
+The four are the same question at different sizes: one categorical variable is `BarPlot`, two
+is `GroupedBarPlot` until the legend gets long, and past that it is `HeatmapPlot`.
+`BarbellPlot` is the case where each group has a before and an after — two bar charts make the
+reader do the subtraction, a barbell draws it.
+
+Lesson 2 has you write a `BarPlot` of your own before any of this. That exercise is about the
+`BasePlot` extension point, and it is not made pointless by the library shipping one, any more
+than writing a sort is made pointless by `sorted` existing.
 
 ---
 
@@ -76,10 +89,12 @@ the period is long relative to your intuition.
   the same data support different stories. Choose the window from the mechanism you are
   studying, before you look.
 
-**In GOAD:** `ShiftValues`, `DiffValues`, `RollingAvg`, `SelectDataRange` are all pipeline
-steps, so the smoothing window is a named parameter you can change in one line and rerun.
-`ComparePlot` puts raw and smoothed on the same axes; `VerticalDate` marks the moment you
-claim something changed, with a label, on the plot.
+**In GOAD:** `ShiftValues`, `DiffValues`, `RollingAvg`, `SelectDataRange`, `TimeFeatures` are
+all pipeline steps, so the smoothing window is a named parameter you can change in one line
+and rerun. `ComparePlot` puts raw and smoothed on the same axes; `VerticalDate` marks the
+moment you claim something changed, with a label, on the plot. `DecomposePlot` wraps
+`seasonal_decompose` into the four-panel observed/trend/seasonal/residual view; `ACFPlot`
+wraps `acf` with its confidence band, for finding a period you did not already suspect.
 
 ---
 
@@ -114,7 +129,8 @@ fitted pdf drawn over the histogram to see where it agrees and where it does not
   sample sizes are near-unreadable. Normalise, or use ECDFs, or do a two-sample test.
 
 **In GOAD:** `HistogramPlot` (density-scaled, so a pdf overlays correctly), `DistPlot` for a
-parametric curve, and `DistributionFitter` + `PlotFits` for the full ranked comparison. See
+parametric curve, `DistributionFitter` + `PlotFits` for the full ranked comparison, and
+`QQPlot` / `ECDFPlot` for the tail diagnostics a histogram cannot show. See
 [Distributions](05-distributions.md).
 
 ---
@@ -146,9 +162,22 @@ to look.
   are not independent evidence. This is exactly why regularisation is the better tool for
   "which variables matter" — see [Models and residuals](06-models-and-residuals.md).
 
-**In GOAD:** the scatter and the fit are `BasePlot` subclasses; the interesting work is in
-[Models and residuals](06-models-and-residuals.md), because a relation you can state as a
-model is a relation whose failures you can inspect.
+**In GOAD:** `ScatterPlot`, `RegPlot` and `CorrelationHeatmap`, in that order of use.
+
+```python
+scatter = ScatterPlot(settings)
+scatter.plot(data=data, x="messages", y="length", hue="is_bot")
+scatter.plot_on(RegPlot(settings), data=data, x="messages", y="length", scatter=False)
+```
+
+The layering is the point: the scatter is drawn first and the fit goes on top of it, so the
+choice of `fit_reg`, `order` or `lowess` is made after you have seen the shape rather than
+before. `CorrelationHeatmap` is the entry point when there are too many pairs to plot — it
+tells you which scatter to make next, and nothing more than that.
+
+The interesting work is still in [Models and residuals](06-models-and-residuals.md), because
+a relation you can state as a model is a relation whose failures you can inspect — and
+§6.7 is how you check that the relation is not something a shuffle would have produced.
 
 ---