calkit-python 0.47.5__py3-none-any.whl → 0.47.6__py3-none-any.whl

calkit/agent_skills/build-paper-pipeline/SKILL.md

@@ -84,8 +84,11 @@ building a pipeline from scratch.
    `calkit.yaml`: the `question` as the paper poses it, the `answer` as the
    paper states it with `{name}` placeholders where numbers go, and
    `evidence` naming the results file and key, the figure, and the
-   publication section that carries the argument. Run
-   `calkit check questions` and fix what it reports.
+   publication section that carries the argument. The results file must be
+   one a stage from step 3 writes, never one written by hand. Where the
+   claim hinges on a threshold, write a conditional answer; the
+   `check-questions` skill covers both. Run `calkit check questions` and
+   fix what it reports.
 
 6. **Run the pipeline, then check your own work.** Run `calkit run`, then
    `calkit check repro`, which reads the manuscript back and reports any

calkit/agent_skills/check-questions/SKILL.md

@@ -21,12 +21,17 @@ Deterministic, done by `calkit check questions` — never re-derive by hand:
 - every evidence path exists;
 - every `value` key resolves in its file, and every `{name}` placeholder in
   the prose resolves and formats;
+- every clause of a conditional answer parses, names only evidence, and
+  renders, including the clauses the current values don't select;
 - every publication `label` still exists in the LaTeX source;
 - no evidence has changed (Git history for Git-tracked outputs, `dvc.lock`
   for DVC-tracked ones) since the commit that last edited the question;
-- each evidence path is produced by a pipeline stage, or declared with
-  `imported_from` or `created_by`. This one is advisory: it is reported as
-  `unattributed` and does not fail the check.
+- each `value` entry reads a file a pipeline stage produces, or one
+  declared with `imported_from`; anything else is an error, since a number
+  nothing computes is a magic number;
+- each other evidence path is produced by a pipeline stage, or declared
+  with `imported_from` or `created_by`. This one is advisory: it is
+  reported as `unattributed` and does not fail the check.
 
 Judgment, done here — the check reads paths and hashes, and cannot read a
 sentence:
@@ -35,6 +40,8 @@ sentence:
   this skill exists;
 - does the answer still follow from the evidence, given what changed;
 - are numbers retyped into the prose that should be `{name}` placeholders;
+- does a claim resting on a threshold use a conditional answer, with a
+  threshold chosen before the value was known;
 - is the answer concise, and does it point at the publication section that
   carries the argument rather than repeating it.
 
@@ -104,7 +111,8 @@ evidence without showing them both.
    from. If a stage should produce it, that is a pipeline gap worth
    reporting. If it was imported or made by hand, declare it under
    `figures`, `datasets`, or `publications` with `imported_from` or
-   `created_by` so the project says so.
+   `created_by` so the project says so. A `value` entry with no stage is
+   reported as an error rather than `unattributed`: give it a stage.
 4. For each **error**, fix the reference: a missing path means the pipeline
    has not been run or pulled; a bad key or placeholder means a results
    file was restructured; a missing label means the publication was
@@ -128,6 +136,38 @@ project, not a tidy-up.
   what it means; leave the reasoning to the publication.
 - Numbers come from `value` evidence via placeholders, formatted to the
   precision the claim needs: `{ratio:.1f}x`, `{error:.0%}`.
+- A `value` entry must read a file a pipeline stage writes. A placeholder
+  over a results file written by hand, including one you write, is a
+  retyped number with extra steps, and the check fails it. If no stage
+  produces the number, add one (`/calkit:add-pipeline-stage`) instead of
+  writing the file. Never declare a file `imported_from` or `created_by`
+  to clear that error unless it really came from there, and never edit a
+  results file to change what an answer says.
+- When the claim itself depends on a value, not just the number in it,
+  e.g., significant or not, which method wins, write a conditional answer
+  so the wording follows the evidence on a rerun:
+
+  ```yaml
+  answer:
+    if p < 0.05: "The closure cuts error by {improvement:.1f}x."
+    else: "The closure does not measurably reduce error."
+  evidence:
+    - kind: value
+      path: results/closure.json
+      key: p-value
+      name: p
+    - kind: value
+      path: results/closure.json
+      key: improvement
+  ```
+
+  Conditions name `value` evidence, so those names must be valid Python
+  identifiers; give the entry a `name` otherwise. Every branch must be a
+  claim the evidence would support if it held. The threshold is part of
+  the claim: take it from the field's convention or the user, never pick
+  it to make the current branch hold, and don't use branches to hedge a
+  claim that should simply be weakened.
+
 - Point at the publication with a `publication` evidence entry carrying
   `section` (for the reader) and `label` (for the check), instead of an
   `explanation` that restates the argument.

calkit/cli/list.py

@@ -243,7 +243,7 @@ def list_questions(
         render_question,
     )
 
-    def _texts(question: dict) -> list[str]:
+    def _texts(question: dict) -> list[str | dict]:
         evidence = question.get("evidence") or []
         return [question.get(f) or "" for f in TEMPLATED_FIELDS] + [
             ev.get("explanation") or ""
@@ -260,16 +260,18 @@ def list_questions(
         # let a fresh clone read as a project that types its braces. Any
         # placeholder left standing counts, whether it names evidence that
         # could not be read or names nothing at all: a brace meant to stay
-        # in the text is written '{{' and never reaches here.
+        # in the text is written '{{' and never reaches here. A conditional
+        # answer still in clauses is one whose conditions could not be read.
         unfilled = any(
-            placeholders(text)
+            isinstance(text, dict) or placeholders(text)
             for q in rendered
             if isinstance(q, dict)
             for text in _texts(q)
         )
         if unfilled:
             warn(
-                "Some placeholders could not be filled from the evidence. "
+                "Some placeholders or conditions could not be filled from "
+                "the evidence. "
                 "Run 'calkit check questions' to see why; 'calkit pull' if "
                 "the results files are not here yet.",
                 err=json_output,

calkit/models/core.py

@@ -1841,7 +1841,15 @@ class Question(BaseModel):
 
     question: str
     hypothesis: str | None = None
-    answer: str | None = None
+    answer: str | dict[str, str] | None = Field(
+        default=None,
+        description=(
+            "The claim the evidence supports. A mapping keyed by 'if "
+            "<condition>', 'elif <condition>' and 'else' picks its wording "
+            "from the evidence, so an answer resting on a threshold states "
+            "the other outcome instead of going stale when a value moves."
+        ),
+    )
     notes: str | None = Field(
         default=None,
         description=(

calkit/questions.py

@@ -32,13 +32,15 @@ updates itself.
 
 from __future__ import annotations
 
+import ast
 import glob
 import io
 import json
+import operator
 import os
 import re
 import string
-from typing import Any, Literal
+from typing import Any, Literal, TypeGuard
 
 from pydantic import BaseModel, Field
 
@@ -245,14 +247,208 @@ def placeholders(text: str) -> list[str]:
     return [m.group(1) for m in _PLACEHOLDER.finditer(text or "")]
 
 
-def render(text: str | None, values: dict[str, Any]) -> str | None:
+def _placeholder_problems(text: str, values: dict[str, Any]) -> list[str]:
+    if "{" not in text:
+        return []
+    try:
+        _FORMATTER.vformat(text, (), values)
+    except KeyError as e:
+        return [
+            f"placeholder {{{e.args[0]}}} names no evidence; write "
+            "'{{' and '}}' for braces meant to stay in the text"
+        ]
+    except (ValueError, IndexError) as e:
+        return [f"cannot render {text[:40]!r}...: {e}"]
+    return []
+
+
+_IF_KEY = re.compile(r"^\s*(if|elif)\s+(.+?)\s*$")
+_ELSE_KEY = re.compile(r"^\s*else\s*$")
+_COMPARISONS = {
+    ast.Lt: operator.lt,
+    ast.LtE: operator.le,
+    ast.Gt: operator.gt,
+    ast.GtE: operator.ge,
+    ast.Eq: operator.eq,
+    ast.NotEq: operator.ne,
+}
+
+
+def is_conditional(value: Any) -> TypeGuard[dict]:
+    """Whether a value picks its wording with ``if``/``elif``/``else``."""
+    return isinstance(value, dict)
+
+
+def parse_conditional(clauses: dict) -> list[tuple[str | None, str]]:
+    """Read a conditional's keys into ordered ``(condition, wording)``.
+
+    The condition is ``None`` for ``else``. Keys are read in the order
+    they appear in the file, so the clauses are tried in the order they
+    were written.
+    """
+    parsed: list[tuple[str | None, str]] = []
+    for position, (key, wording) in enumerate(clauses.items()):
+        opened = _IF_KEY.match(str(key))
+        if opened:
+            keyword, condition = opened.groups()
+            if keyword == "if" and position:
+                raise ValueError("only the first clause may be 'if'")
+            if keyword == "elif" and not parsed:
+                raise ValueError("'elif' with no 'if' before it")
+            parsed.append((condition, str(wording)))
+            continue
+        if _ELSE_KEY.match(str(key)):
+            if not parsed:
+                raise ValueError("'else' with no 'if' before it")
+            parsed.append((None, str(wording)))
+            continue
+        raise ValueError(f"expected 'if', 'elif' or 'else', got {key!r}")
+    if not parsed:
+        raise ValueError("a conditional needs at least an 'if' clause")
+    for condition, _ in parsed[:-1]:
+        if condition is None:
+            raise ValueError("'else' must be the last clause")
+    return parsed
+
+
+def _operand(node: ast.AST, values: dict[str, Any]) -> Any:
+    """One side of a comparison, resolved against the evidence values.
+
+    Names and literals are read directly; anything else is arithmetic and
+    goes to the same evaluator the calculations use, so there is one
+    audited path for arithmetic rather than two.
+    """
+    if isinstance(node, ast.Constant):
+        return node.value
+    if isinstance(node, ast.Name):
+        if node.id not in values:
+            raise KeyError(node.id)
+        return values[node.id]
+    for inner in ast.walk(node):
+        if isinstance(inner, ast.Name) and inner.id not in values:
+            raise KeyError(inner.id)
+    import arithmetic_eval  # type: ignore[import-untyped]
+
+    return arithmetic_eval.evaluate(ast.unparse(node), values)
+
+
+def _truth(node: ast.AST, values: dict[str, Any]) -> bool:
+    if isinstance(node, ast.BoolOp):
+        # Not short-circuited, so a misspelled name is an error whatever
+        # the current values are
+        outcomes = [_truth(v, values) for v in node.values]
+        if isinstance(node.op, ast.And):
+            return all(outcomes)
+        return any(outcomes)
+    if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
+        return not _truth(node.operand, values)
+    if isinstance(node, ast.Compare):
+        left = _operand(node.left, values)
+        for op, comparator in zip(node.ops, node.comparators):
+            right = _operand(comparator, values)
+            compare = _COMPARISONS.get(type(op))
+            if compare is None:
+                raise ValueError(
+                    f"{type(op).__name__} is not a supported comparison"
+                )
+            if not compare(left, right):
+                return False
+            left = right
+        return True
+    raise ValueError("a condition must compare values, e.g., 'p < 0.05'")
+
+
+def evaluate_condition(expression: str, values: dict[str, Any]) -> bool:
+    """Evaluate one ``if``/``elif`` condition against the evidence values.
+
+    Only comparisons, ``and``/``or``/``not`` and arithmetic are allowed, so
+    a condition read from ``calkit.yaml`` cannot call anything.
+    """
+    try:
+        tree = ast.parse(expression, mode="eval")
+    except SyntaxError as e:
+        raise ValueError(f"cannot parse condition {expression!r}: {e}") from e
+    try:
+        return bool(_truth(tree.body, values))
+    except KeyError:
+        # A name like 'paired-gain.vawt-8' reads as arithmetic and
+        # attribute access, so the failure would otherwise name a
+        # fragment of it and look like missing evidence.
+        unusable = [
+            name
+            for name in values
+            if not name.isidentifier() and name in expression
+        ]
+        if unusable:
+            raise ValueError(
+                f"condition {expression!r} refers to {unusable[0]!r}, which "
+                "cannot be read as a variable; give that evidence a 'name' "
+                "that is a valid Python identifier"
+            ) from None
+        raise
+    except ValueError:
+        raise
+    except Exception as e:
+        # E.g., comparing a string to a number, or syntax arithmetic_eval
+        # refuses, so callers only have to handle one kind of bad condition
+        raise ValueError(
+            f"cannot evaluate condition {expression!r}: {e}"
+        ) from e
+
+
+def check_conditional(clauses: dict, values: dict[str, Any]) -> list[str]:
+    """Problems with every clause of a conditional, not just the one chosen.
+
+    The clauses the current values don't select are the ones a rerun will
+    reach, so a typo in them is an error now rather than after the rerun.
+    """
+    try:
+        parsed = parse_conditional(clauses)
+    except ValueError as e:
+        return [f"conditional answer: {e}"]
+    messages: list[str] = []
+    held = False
+    for condition, wording in parsed:
+        if condition is None:
+            held = True
+        else:
+            try:
+                held = evaluate_condition(condition, values) or held
+            except KeyError as e:
+                messages.append(
+                    f"condition {condition!r} names no evidence {e.args[0]!r}"
+                )
+            except ValueError as e:
+                messages.append(str(e))
+        messages += _placeholder_problems(wording, values)
+    if not messages and not held:
+        messages.append(
+            "no condition of the conditional answer holds and there is no "
+            "'else' clause"
+        )
+    return messages
+
+
+def select_branch(clauses: dict, values: dict[str, Any]) -> str:
+    """The wording whose condition holds, for a conditional answer."""
+    for condition, wording in parse_conditional(clauses):
+        if condition is None or evaluate_condition(condition, values):
+            return wording
+    raise ValueError("no condition held and there is no 'else' clause")
+
+
+def render(text: str | dict | None, values: dict[str, Any]) -> str | None:
     """Fill a question text's placeholders from its evidence values.
 
     Raises ``KeyError`` for a name with no evidence and ``ValueError`` for
     a format spec the value cannot satisfy, so a template that cannot be
     rendered is an error rather than a silently unfilled sentence.
     """
-    if text is None or "{" not in text:
+    if text is None:
+        return text
+    if isinstance(text, dict):
+        text = select_branch(text, values)
+    if "{" not in text:
         return text
     return _FORMATTER.vformat(text, (), values)
 
@@ -596,7 +792,11 @@ def _check_publication_label(
 
 
 def _is_attributed(
-    path: str, stage: str | None, ck_info: dict, wdir: str
+    path: str,
+    stage: str | None,
+    ck_info: dict,
+    wdir: str,
+    computed: bool = False,
 ) -> bool:
     """Whether the project says where an evidence path came from.
 
@@ -605,7 +805,9 @@ def _is_attributed(
     may be declared as an artifact that records an import or a person,
     which is what :func:`calkit.provenance.has_provenance` reads---an
     imported dataset or a hand-drawn schematic is accounted for even
-    though there is nothing upstream to point at.
+    though there is nothing upstream to point at. With ``computed``, a
+    person doesn't count, since a number someone typed into a file is
+    still a magic number.
     """
     from calkit.provenance import has_provenance
 
@@ -624,7 +826,12 @@ def _is_attributed(
             if (
                 isinstance(artifact, dict)
                 and artifact.get("path") == path
-                and has_provenance(artifact)
+                and (
+                    artifact.get("stage") is not None
+                    or artifact.get("imported_from") is not None
+                    if computed
+                    else has_provenance(artifact)
+                )
             ):
                 return True
     return False
@@ -808,7 +1015,19 @@ def check_evidence(
                     ],
                 )
             )
-    if out.status == "ok" and not _is_attributed(
+    if (
+        out.status in ("ok", "changed")
+        and is_value_evidence(ev)
+        and not _is_attributed(path, out.stage, ck_info, wdir, computed=True)
+    ):
+        out.status = "error"
+        out.message = (
+            "no pipeline stage computes this value, so nothing can show "
+            "where the number came from or keep it current; produce the file "
+            "with a stage, or declare it with 'imported_from' if another "
+            "project computed it"
+        )
+    elif out.status == "ok" and not _is_attributed(
         path, out.stage, ck_info, wdir
     ):
         out.status = "unattributed"
@@ -874,17 +1093,10 @@ def check_question(
         ev.get("explanation") for ev in evidence
     ]
     for t in texts:
-        if not t or "{" not in t:
-            continue
-        try:
-            render(t, values)
-        except KeyError as e:
-            messages.append(
-                f"placeholder {{{e.args[0]}}} names no evidence; write "
-                "'{{' and '}}' for braces meant to stay in the text"
-            )
-        except (ValueError, IndexError) as e:
-            messages.append(f"cannot render {t[:40]!r}...: {e}")
+        if is_conditional(t):
+            messages += check_conditional(t, values)
+        elif t:
+            messages += _placeholder_problems(t, values)
     # Worst first, matching what the hub shows against each question: an
     # answer resting on nothing anyone can find is worse off than one
     # resting on something merely out of date.

calkit/tests/cli/test_check.py

@@ -1182,11 +1182,24 @@ def test_check_questions(tmp_dir):
         calkit.ryaml.dump(ck_info, f)
     subprocess.check_call(["git", "add", "-A"])
     subprocess.check_call(["git", "commit", "-q", "-m", "Answer"])
+    # No stage computes the value, so it is a magic number and fails
+    proc = subprocess.run(
+        ["calkit", "check", "questions"], capture_output=True, text=True
+    )
+    assert proc.returncode != 0
+    assert "no pipeline stage computes this value" in proc.stdout
+    # Another project computed it, which is traceable
+    ck_info["datasets"] = [
+        {
+            "path": "results/findings.json",
+            "imported_from": {"project": "someone/else"},
+        }
+    ]
+    with open("calkit.yaml", "w") as f:
+        calkit.ryaml.dump(ck_info, f)
+    subprocess.check_call(["git", "commit", "-q", "-am", "Import it"])
     out = subprocess.check_output(["calkit", "check", "questions"], text=True)
     assert "Answers backed by current evidence: 1/1" in out
-    # Nothing in the project says where the results file came from, which
-    # is advice rather than a failure: the answer may be perfectly good
-    assert "Evidence with nothing recorded behind it: 1" in out
     # Listing renders the placeholder from the results file
     out = subprocess.check_output(["calkit", "list", "questions"], text=True)
     assert "answer: 8 of eight do." in out
@@ -1252,19 +1265,6 @@ def test_check_questions(tmp_dir):
     subprocess.check_call(["git", "commit", "-q", "-am", "Review"])
     out = subprocess.check_output(["calkit", "check", "questions"], text=True)
     assert "changed after the answer was written" not in out
-    # Declaring where the results file came from clears the advice
-    ck_info = calkit.load_calkit_info()
-    ck_info["datasets"] = [
-        {
-            "path": "results/findings.json",
-            "created_by": {"name": "A person"},
-        }
-    ]
-    with open("calkit.yaml", "w") as f:
-        calkit.ryaml.dump(ck_info, f)
-    subprocess.check_call(["git", "commit", "-q", "-am", "Declare"])
-    out = subprocess.check_output(["calkit", "check", "questions"], text=True)
-    assert "nothing recorded behind it" not in out
     # A console that cannot encode a check mark gets a '?' rather than a
     # UnicodeEncodeError, which on Windows would kill the command with no
     # output at all
@@ -1289,8 +1289,25 @@ def test_check_questions(tmp_dir):
     with open("calkit.yaml", "w") as f:
         calkit.ryaml.dump(ck_info, f)
     out = subprocess.check_output(["calkit", "list", "questions"], text=True)
-    assert "placeholders could not be filled" in out
+    assert "could not be filled" in out
     assert "It is {missing} of them." in out
+    # A conditional answer is listed as the branch that holds, and one with
+    # no branch holding is listed as written, with the same warning
+    ck_info["questions"][0]["answer"] = {
+        "if n_top > 4": "Most do.",
+        "else": "{n_top} do.",
+    }
+    with open("calkit.yaml", "w") as f:
+        calkit.ryaml.dump(ck_info, f)
+    out = subprocess.check_output(["calkit", "list", "questions"], text=True)
+    assert "answer: 0 do." in out
+    assert "could not be filled" not in out
+    del ck_info["questions"][0]["answer"]["else"]
+    with open("calkit.yaml", "w") as f:
+        calkit.ryaml.dump(ck_info, f)
+    out = subprocess.check_output(["calkit", "list", "questions"], text=True)
+    assert "could not be filled" in out
+    assert "if n_top > 4: Most do." in out
     # A project with no questions says so rather than printing nothing
     os.remove("calkit.yaml")
     with open("calkit.yaml", "w") as f:

calkit/tests/test_questions.py

@@ -14,11 +14,14 @@ from calkit.questions import (
     QuestionsStatus,
     check_question,
     check_questions,
+    evaluate_condition,
     format_status,
+    parse_conditional,
     placeholders,
     render,
     render_question,
     resolve_key,
+    select_branch,
 )
 
 
@@ -92,6 +95,16 @@ def test_history_is_read_once_for_all_questions(tmp_dir):
     with open("results/findings.json", "w") as f:
         json.dump({f"k{i}": i for i in range(4)}, f)
     ck_info = {
+        "pipeline": {
+            "stages": {
+                "summarize": {
+                    "kind": "python-script",
+                    "environment": "py",
+                    "script_path": "s.py",
+                    "outputs": ["results/findings.json"],
+                }
+            }
+        },
         "questions": [
             {
                 "question": f"Q{i}?",
@@ -106,7 +119,7 @@ def test_history_is_read_once_for_all_questions(tmp_dir):
                 ],
             }
             for i in range(4)
-        ]
+        ],
     }
     with open("calkit.yaml", "w") as f:
         calkit.ryaml.dump(ck_info, f)
@@ -235,6 +248,32 @@ def test_check_questions(tmp_dir):
     # makes it and it is not declared with an import or a person
     assert q4.evidence[4].status == "unattributed"
     assert [ev.path for ev in status.unattributed] == ["figures/plot.png"]
+    # A value no stage computes is a magic number, so it fails rather than
+    # being advice; an import is traceable, a person typing it in is not
+    with open("results/typed.json", "w") as f:
+        json.dump({"n": 3}, f)
+    typed = {
+        "question": "Typed?",
+        "answer": "{n}",
+        "evidence": [
+            {"kind": "value", "path": "results/typed.json", "key": "n"}
+        ],
+    }
+    for declared, expected in [
+        (None, "error"),
+        ({"created_by": "someone"}, "error"),
+        ({"imported_from": {"project": "a/b"}}, "ok"),
+    ]:
+        info = dict(ck_info)
+        if declared:
+            info["datasets"] = [{"path": "results/typed.json"} | declared]
+        checked = check_question(5, typed, info, ".")
+        assert checked.status == expected, declared
+        if expected == "error":
+            assert "no pipeline stage computes" in (
+                checked.evidence[0].message or ""
+            )
+    os.remove("results/typed.json")
     rendered = render_question(ck_info["questions"][3], ck_info, ".")
     assert rendered["answer"] == "8 of eight do, a 5.1x gain."
     assert rendered["evidence"][2]["explanation"] == "The best is a."
@@ -530,3 +569,118 @@ def test_check_questions_pipeline_and_pins(tmp_dir):
     assert checked.status == "ok"
     assert checked.evidence[0].current == 2
     assert checked.evidence[0].git_ref == sha
+
+
+def test_conditional_answers(tmp_dir):
+    values = {"p": 0.007, "rho": 0.8373, "n": 17, "leader": "turns-max"}
+    # Comparisons, chaining, boolean operators, strings, and arithmetic
+    assert evaluate_condition("p < 0.05", values)
+    assert not evaluate_condition("p >= 0.05", values)
+    assert evaluate_condition("0.0 <= p < 0.05", values)
+    assert evaluate_condition("p < 0.05 and rho > 0.8", values)
+    assert evaluate_condition("p > 0.5 or rho > 0.8", values)
+    assert evaluate_condition("not p > 0.5", values)
+    assert evaluate_condition("leader == 'turns-max'", values)
+    assert evaluate_condition("n / 2 > 8", values)
+    # A name with no evidence is an error, not a false condition, and
+    # nothing may be called or used as a bare value
+    with pytest.raises(KeyError):
+        evaluate_condition("missing < 1", values)
+    with pytest.raises(Exception):
+        evaluate_condition("len(leader) > 1", values)
+    with pytest.raises(ValueError):
+        evaluate_condition("p", values)
+    # A comparison the values can't make is a ValueError like the rest
+    with pytest.raises(ValueError, match="cannot evaluate"):
+        evaluate_condition("leader < 0.5", values)
+    # A name that is not a valid identifier cannot be read as a variable,
+    # and says so rather than reporting a fragment of itself as missing
+    with pytest.raises(ValueError, match="valid Python identifier"):
+        evaluate_condition("paired-gain > 0.1", {"paired-gain": 0.25})
+    # Clauses are tried in the order written, with None marking the else
+    clauses = {
+        "if p < 0.05": "strong, rho {rho:.2f}",
+        "elif p < 0.1": "weak",
+        "else": "none",
+    }
+    assert [c for c, _ in parse_conditional(clauses)] == [
+        "p < 0.05",
+        "p < 0.1",
+        None,
+    ]
+    assert select_branch(clauses, {"p": 0.007}) == "strong, rho {rho:.2f}"
+    assert select_branch(clauses, {"p": 0.08}) == "weak"
+    assert select_branch(clauses, {"p": 0.9}) == "none"
+    # Malformed clause sets are errors rather than silent misreadings
+    with pytest.raises(ValueError):
+        parse_conditional({"elif p < 1": "x"})
+    with pytest.raises(ValueError):
+        parse_conditional({"else": "x"})
+    with pytest.raises(ValueError):
+        parse_conditional({"when p < 1": "x"})
+    with pytest.raises(ValueError):
+        parse_conditional({"if p < 1": "x", "else": "y", "elif p < 2": "z"})
+    with pytest.raises(ValueError):
+        parse_conditional({"if p < 1": "x", "if p < 2": "y"})
+    # Nothing holding with no else is an error rather than a blank answer
+    with pytest.raises(ValueError):
+        select_branch({"if p < 0.05": "strong"}, {"p": 0.9})
+    # Rendering picks the branch, then fills its placeholders
+    answer = {
+        "if p < 0.05": "{leader} predicts it (rho {rho:+.2f})",
+        "else": "no feature predicts it",
+    }
+    assert render(answer, values) == "turns-max predicts it (rho +0.84)"
+    assert render(answer, values | {"p": 0.2}) == "no feature predicts it"
+    # Plain strings are untouched by the conditional path
+    assert render("if and only if", values) == "if and only if"
+    # Checking reports every clause's problems, including the clauses the
+    # current values don't select, rather than crashing or passing
+    with open("r.json", "w") as f:
+        json.dump({"p": 0.2, "leader": "x"}, f)
+    evidence = [
+        {"kind": "value", "path": "r.json", "key": "p"},
+        {"kind": "value", "path": "r.json", "key": "leader"},
+    ]
+    cases = [
+        ({"if p < 0.5": "{leader}", "else": "no"}, []),
+        (
+            {"if p < 0.05": "x"},
+            ["no condition of the conditional answer holds"],
+        ),
+        ({"when p < 1": "x"}, ["conditional answer: expected 'if'"]),
+        (
+            {"if leader < 0.05": "x", "else": "y"},
+            ["cannot evaluate condition 'leader < 0.05'"],
+        ),
+        (
+            {"if p > 0.1": "yes", "elif typo < 1": "x", "else": "{rhoo:.2f}"},
+            [
+                "condition 'typo < 1' names no evidence 'typo'",
+                "placeholder {rhoo} names no evidence",
+            ],
+        ),
+    ]
+    ck_info = {
+        "pipeline": {
+            "stages": {
+                "scan": {
+                    "kind": "python-script",
+                    "environment": "py",
+                    "script_path": "s.py",
+                    "outputs": ["r.json"],
+                }
+            }
+        }
+    }
+    for answer, expected in cases:
+        checked = check_question(
+            1,
+            {"question": "q", "answer": answer, "evidence": evidence},
+            ck_info,
+            ".",
+        )
+        messages = checked.message or ""
+        assert (checked.status == "error") == bool(expected), answer
+        for fragment in expected:
+            assert fragment in messages, (answer, messages)

{calkit_python-0.47.5.dist-info → calkit_python-0.47.6.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: calkit-python
-Version: 0.47.5
+Version: 0.47.6
 Summary: Reproducibility simplified.
 Project-URL: Homepage, https://calkit.org
 Project-URL: Issues, https://github.com/calkit/calkit/issues

{calkit_python-0.47.5.dist-info → calkit_python-0.47.6.dist-info}/RECORD

@@ -31,7 +31,7 @@ calkit/overleaf.py,sha256=ycCX6G28CTd8IprCAh4o9ihtgECi4dR1hwwgfUnBZmw,53210
 calkit/pipeline.py,sha256=2y6NsgncyGXBFVcyCQxXjVajA28fAdGUOMlfFTjzBfw,105006
 calkit/procedures.py,sha256=ZZtoWWmq_H0dyuUb6M-g_y77GRsJzxLMovT_X0RtYrM,2860
 calkit/provenance.py,sha256=tnj_2yWkVk929GEKr76lacO2jCzGCzpz8_Ygr6Mds_Q,33863
-calkit/questions.py,sha256=ZpxPjqFP5Aik9cbPyV4MB5HnbpGDzuqBk62QVXV2dOo,37276
+calkit/questions.py,sha256=7VZ2KTe9vvpKRQAJADYbXxkhjl1y46C5aPJzYTX3CMQ,45142
 calkit/releases.py,sha256=RDURVbJJ1M0UNmOvlEs1-eNof6H6fPplxeKT9BcfcLw,32277
 calkit/reproducibility.py,sha256=sNsstw1tWyLPQoZMKldRD8NCOpvU988zSMhhQBXIvGE,34354
 calkit/schema.py,sha256=grCxoUc7JEPhntow__ibS45hdSCqpdAHBwd2dmtiwEM,2704
@@ -47,7 +47,7 @@ calkit/cli/dev.py,sha256=cu4QsfDiA_KurGx4WJcgA2qgfghMoZutE8kl4IZbUcM,1070
 calkit/cli/hub.py,sha256=ZBOlL7-mV57q_ynr4hPuhV7NlkKEkINSRksXD-jjiOg,2972
 calkit/cli/import_.py,sha256=QOZEfKUmqMntle47PRZhN8cs7gfWCBte_smPZqksl54,25563
 calkit/cli/latex.py,sha256=XtahLrNaYdqtqt_mMoP_wfHXIEpdecGlSW6jPR_yulA,51907
-calkit/cli/list.py,sha256=KoTZNR8r_4ewvWjh4ATGbmcrt8HrflDJXFoOf2VhtY8,20463
+calkit/cli/list.py,sha256=DCWIX9LpTU6KpE5KKcT_DDjoRz5xPxQgzeixMsp36tM,20620
 calkit/cli/new.py,sha256=VUgPE5iOpflnuJxUNcy-GzAR-IIs9KRDwwkQtblWfoQ,158690
 calkit/cli/notebooks.py,sha256=aR4I0qatOtkrAxEG5PVLej20SBLKo8qr-lxDSzgljUQ,38836
 calkit/cli/office.py,sha256=jVSTvbl91TCzlglST5O2b8hdApZA_QHw_UiEQFJqxdc,1754
@@ -64,7 +64,7 @@ calkit/dvc/zip.py,sha256=nf4EzOyc5nI_OA3M6S7wwZpL7N4EQz9yCf8FbnPjlxM,21857
 calkit/jupyterlab/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 calkit/jupyterlab/routes.py,sha256=sARrFjWnHVV-LunrgqO4DCt_x7XLhVPAqA1eaUFf7HQ,51443
 calkit/models/__init__.py,sha256=vGYF4MZBGzDk1BPBzpp1bRb59RCJ2Bj2kzHuVVRacug,120
-calkit/models/core.py,sha256=U3C3UzCj19je5K0G_Kt2NnwsopZ1zDplE_nNqLAvGvw,77280
+calkit/models/core.py,sha256=3Z0o6_vLOQ-CKix6iFFmz2FsAr6w1nSv93Xmi1INcjw,77660
 calkit/models/docx.py,sha256=rgR1Y5s_yEomOeC6YlhHvsMDC9TmZp3rPN1wEt_93ds,2539
 calkit/models/io.py,sha256=0nup-gDz3FbiD8AT5pTpxji6XN_rW5AsijErsIFdpbg,2330
 calkit/models/iteration.py,sha256=_QajYhWVXV9ge9naX8ZkT1PBNzfc75cpl2BR2f8VPOo,2945
@@ -124,7 +124,7 @@ calkit/tests/test_overleaf.py,sha256=Itc4K3mYInHi36ZWhqH7drCHLCDbH1SXk3z_fB0-W8o
 calkit/tests/test_pipeline.py,sha256=27BfaE9oCwj9BfDdLQWr-Yco_mpJ1mvULTgajbfLOEI,135627
 calkit/tests/test_procedures.py,sha256=NJUIDy-gkou1BBABeWjazm__-8ZxHwvzzjkLm6kbviI,11359
 calkit/tests/test_provenance.py,sha256=bkL6a7E2Q-JINS8oLS5hs5x-2IzRQkzKJ8HWxSeiemo,23818
-calkit/tests/test_questions.py,sha256=CNdl6T0OjQRZFfOnVzjnsI1O5h7OPVwlaxI1bUZ2QWY,20563
+calkit/tests/test_questions.py,sha256=XSKwPfJpOY8HmhpY0uwf33rZU2gbJnCZz57f35m6GL8,26790
 calkit/tests/test_releases.py,sha256=tiQvrAYMxhg0mHzhcRGi0avAu_rJGhPdhK_lCg2Sb9k,19224
 calkit/tests/test_reproducibility.py,sha256=QIkUugP_1zqUDlgYZsEnkO144FqIQpamt6v7aDPabnE,10273
 calkit/tests/test_resources.py,sha256=Wm2hr_Vy-Dk8rSElYbIzlZnCbomJInpTvUMOxT-mg04,6746
@@ -132,7 +132,7 @@ calkit/tests/test_schema.py,sha256=3h7R23bjqQntfuygrKzGwQfWPBZxX9fdBhnu6AmBqp8,8
 calkit/tests/test_templates.py,sha256=yWln6iro64_vqCgJElhSFOthA7xrHXE1GXDJJ9JEBMQ,2613
 calkit/tests/test_workspace.py,sha256=l1txfs2Oa71vbC9dEHrlcSCXinUeV19SwMZlWqdSpro,60163
 calkit/tests/cli/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-calkit/tests/cli/test_check.py,sha256=Pv7s-ccmjz6tFf8wwwlZiutyJbfsDc8KbPtwUYQNsgk,51642
+calkit/tests/cli/test_check.py,sha256=gcix6EcPCQdPNx_VcdnYNU3So5PE4oIqGDguz80fJnk,52303
 calkit/tests/cli/test_config.py,sha256=AAz-cwCgYM_XTLZdv58XAax1GmvAWgDfksEpn7iiezs,5393
 calkit/tests/cli/test_core.py,sha256=eMEzdL7YKp6Ss5ey4HVtzvjWzqk9KtVJegFjCfhGDu8,1640
 calkit/tests/cli/test_delete.py,sha256=qIbgwKc8nBjmkynzyVuVAjrQB96XSmGRet_jfxs19_0,750
@@ -163,26 +163,26 @@ calkit/tests/models/test_core.py,sha256=iDi0mvgZDBj2n7bo-cyKd8dvRZHl5ywWbcab4BPs
 calkit/tests/models/test_iteration.py,sha256=vjuDKwyYQAV4w1qF9VJWQEHksq3KRd6gtT39_a-RhnI,644
 calkit/tests/models/test_pipeline.py,sha256=6oNoJV7ookVKpeQcgs0sOSVaO99PvX_xbb-cToTc3ZQ,33446
 calkit/agent_skills/add-pipeline-stage/SKILL.md,sha256=kpYYb-rJsS4B9p1Ub3gL21irCG5ZZ4DvWZpdJeEwPZk,3772
-calkit/agent_skills/build-paper-pipeline/SKILL.md,sha256=gAvaTeFj58ZaMj1Qt7dd17KOJfg9bUfpkjxHoeFZjxs,6628
-calkit/agent_skills/check-questions/SKILL.md,sha256=WlQWWuS-TgsoDU504WdLR6UW3GrcjYNdvqELfX2rdf4,7633
+calkit/agent_skills/build-paper-pipeline/SKILL.md,sha256=aV9t86FxQNkeXyswM03fb5LW3ns4pcsWQyYNZ33niLI,6829
+calkit/agent_skills/check-questions/SKILL.md,sha256=6-NVWX2O_soAL0vg38NsBgA_h3roUT3pHwKQ5cwDxQw,9576
 calkit/agent_skills/check-reproducibility/SKILL.md,sha256=5SGmtnbq8bSJW0Zwav_aVzfuhilHwBkkdxGuZvPHxGU,4699
 calkit/agent_skills/conventions/SKILL.md,sha256=kK0Sa54uIORAslE3U53sXJUG-TMdM-Ie72hCMIzzbxQ,8909
 calkit/agent_skills/create-pipeline/SKILL.md,sha256=2FGw1iDQVRk5FUq79GP3lPdgvgof2Wi1O9O-abGGtgs,5422
-calkit_python-0.47.5.data/data/etc/jupyter/jupyter_server_config.d/calkit.json,sha256=CWrZP--JDGz8fsvbAlKr_duTL1vDriGT48SL1YCtkY0,81
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/package.json,sha256=0hnHqhCly59Wg7urFGahc2mSupCWwiPfwKWCYfnbB2E,6202
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/schemas/calkit/package.json.orig,sha256=egkbkHmJYzDARH3_4qpoF75_Svpj_dfPaIwRU-PNjW4,6064
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/schemas/calkit/plugin.json,sha256=YSpIrwpwB-lQBk9Mwv-npedWTOOZreO6nlWZhqlWyGI,840
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/506.c34b070098184e33.js,sha256=0PxdmgdZvLeh6RBcfgN1pcriNO8oCeTK5WJvUIEA3MY,112384
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/57d13a7399cec3c5.png,sha256=svHD7-cMtTnRITFwugwsVaB9nZ-h8A61ose8z32HiRE,24850
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/616.528427eb54a4a0c0.js,sha256=d0rHq7cqAjj1GNAMLOpl8dyY8yyKUQcLPCOUr-wJGdQ,8129
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/740.6bf87276e9e5788a.js,sha256=4E0sK8x-ldVkL-1sM07wgJGS1vtVD9Uly7VtoFTi23c,221
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/899.f9b9f9bf705a6493.js,sha256=RNTIItG9k-A_Ip54fRR3QMCjiffkY0bo8zQ2Xh7TyOY,57096
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/935.46ecc6bf99aa593a.js,sha256=Cmp0KWM5YDZ-zcsJWWgqCMIF6adeHQH80rxz1sas45E,51183
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/remoteEntry.109a3a8379365e59.js,sha256=4JavPLcn703qVSC2E1_MD1thrJNL4N-eITjnfNhSDSM,15066
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/style.js,sha256=LTlA3M58hwjqmhomdcrNRO9GM58zZcf0V76WX9HI6EQ,146
-calkit_python-0.47.5.data/data/share/jupyter/labextensions/calkit/static/third-party-licenses.json,sha256=2BGdItLJwO3fAopLl4eJvp26TEwuscYC0-yFaF-LaLU,13683
-calkit_python-0.47.5.dist-info/METADATA,sha256=lscQ40SyBEjeYn_HTruIytM3ImK_zYKsdyF_rQWL5eQ,16997
-calkit_python-0.47.5.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
-calkit_python-0.47.5.dist-info/entry_points.txt,sha256=TtpHpcv2vFVAVYxFVp0XrwCV_z7FWIOguEiypXRDoOw,131
-calkit_python-0.47.5.dist-info/licenses/LICENSE,sha256=9ZamCaSUTZk9rcrnf-sWFKLOHr3ws-S_dgKMegW4nw8,1056
-calkit_python-0.47.5.dist-info/RECORD,,
+calkit_python-0.47.6.data/data/etc/jupyter/jupyter_server_config.d/calkit.json,sha256=CWrZP--JDGz8fsvbAlKr_duTL1vDriGT48SL1YCtkY0,81
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/package.json,sha256=0hnHqhCly59Wg7urFGahc2mSupCWwiPfwKWCYfnbB2E,6202
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/schemas/calkit/package.json.orig,sha256=egkbkHmJYzDARH3_4qpoF75_Svpj_dfPaIwRU-PNjW4,6064
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/schemas/calkit/plugin.json,sha256=YSpIrwpwB-lQBk9Mwv-npedWTOOZreO6nlWZhqlWyGI,840
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/506.c34b070098184e33.js,sha256=0PxdmgdZvLeh6RBcfgN1pcriNO8oCeTK5WJvUIEA3MY,112384
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/57d13a7399cec3c5.png,sha256=svHD7-cMtTnRITFwugwsVaB9nZ-h8A61ose8z32HiRE,24850
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/616.528427eb54a4a0c0.js,sha256=d0rHq7cqAjj1GNAMLOpl8dyY8yyKUQcLPCOUr-wJGdQ,8129
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/740.6bf87276e9e5788a.js,sha256=4E0sK8x-ldVkL-1sM07wgJGS1vtVD9Uly7VtoFTi23c,221
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/899.f9b9f9bf705a6493.js,sha256=RNTIItG9k-A_Ip54fRR3QMCjiffkY0bo8zQ2Xh7TyOY,57096
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/935.46ecc6bf99aa593a.js,sha256=Cmp0KWM5YDZ-zcsJWWgqCMIF6adeHQH80rxz1sas45E,51183
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/remoteEntry.109a3a8379365e59.js,sha256=4JavPLcn703qVSC2E1_MD1thrJNL4N-eITjnfNhSDSM,15066
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/style.js,sha256=LTlA3M58hwjqmhomdcrNRO9GM58zZcf0V76WX9HI6EQ,146
+calkit_python-0.47.6.data/data/share/jupyter/labextensions/calkit/static/third-party-licenses.json,sha256=2BGdItLJwO3fAopLl4eJvp26TEwuscYC0-yFaF-LaLU,13683
+calkit_python-0.47.6.dist-info/METADATA,sha256=OeP79x--jBv_sZbbCf6uUoALyK6Dr6821tDz2vgR9sA,16997
+calkit_python-0.47.6.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
+calkit_python-0.47.6.dist-info/entry_points.txt,sha256=TtpHpcv2vFVAVYxFVp0XrwCV_z7FWIOguEiypXRDoOw,131
+calkit_python-0.47.6.dist-info/licenses/LICENSE,sha256=9ZamCaSUTZk9rcrnf-sWFKLOHr3ws-S_dgKMegW4nw8,1056
+calkit_python-0.47.6.dist-info/RECORD,,