plainbook 0.2.2__tar.gz → 0.2.4__tar.gz

{plainbook-0.2.2 → plainbook-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: plainbook
-Version: 0.2.2
+Version: 0.2.4
 Summary: Plain Language Notebooks
 Author-email: Luca de Alfaro <dealfaro@acm.org>
 License-Expression: BSD-3-Clause
@@ -14,6 +14,7 @@ Requires-Dist: anthropic
 Requires-Dist: bottle
 Requires-Dist: cherrypy
 Requires-Dist: google-genai
+Requires-Dist: py-machineid
 Requires-Dist: snapshot-kernel>=0.1.4
 Requires-Dist: nbformat
 Requires-Dist: pyyaml
@@ -31,20 +32,22 @@ The use of plain language allows you to share your data analysis and science wit
 Plainbook can use multiple AIs to check that the code faithfully implements the natural language description, and can include tests to further validate the notebook. 
 When you share a notebook, the recipients can also check that the code under the hood implements the natural-language tasks. Recipients can also edit the plainbook, regenerate the code, and rerun it, just as in Jupyter notebooks. 
 
-Thus, the goal of the Plainbook project is to replicate in natural language what made Juptyter notebooks so successful: the ability to share together code and results, so that any recipient can validate and modify the notebook.
+Thus, the goal of the Plainbook project is to replicate in natural language what made Jupyter notebooks so successful: the ability to share together code and results, so that any recipient can validate and modify the notebook.
 
-### Trying Plainbook 
+### Try Plainbook Now
 
-* [30s short video](https://youtu.be/0t4ND8wPoYA)
-* [5 min Introductory video](https://youtu.be/Mkv5cl5rA7s). 
-* **Run it on Codespaces:** You can run plainbook right here on GitHub, with a trial Claude key, as follows:
-    * Create a codespace (Click on Code > Codespaces in the GitHub interface).
-    * Wait about 3 minutes, until after it finishes the postStartCommand.
-    * Once you see the button "Open in Browser" for port 8080 on the lower right, press it. 
-    * The Claude key provided in the trial has a low spending limit, and might not work.  You can add your own keys. 
+**Quick Start Videos:**
+* [30-second demo](https://youtu.be/0t4ND8wPoYA)
+* [5-minute introduction](https://youtu.be/Mkv5cl5rA7s)
 
-Example plainbooks:
-* Plainbook on soccer matches: [Download Soccer_w_Tests.plnb](https://github.com/lucadealfaro/plainbook/raw/main/examples/Soccer_w_Tests.plnb).  
+**Run on GitHub Codespaces (no installation needed):**
+1. Click **Code** → **Codespaces** in the GitHub interface
+2. Wait ~3 minutes for the environment to set up
+3. Click **Open in Browser** for port 8080
+4. A trial Claude API key is provided; you can add your own in Settings
+
+**Example Notebooks:**
+* [Soccer World Cup Analysis](https://github.com/lucadealfaro/plainbook/raw/main/examples/Soccer_w_Tests.plnb) — demonstrates action cells, tests, and AI validation
 
 
 ## Installation and use
@@ -58,12 +61,19 @@ pip install plainbook
 To open a plainbook (which will be created if it does not exist): 
 
 ```bash
-plainbook notebook.nlb
+plainbook notebook.plnb
 ```
 
 You can use any file name you like, with any extension you like. 
 
-**AI API Keys.** You need a Gemini or Claude API key to use Plainbook.  Click on the Settings button (the gear on the top right) to see instructions on how to set them.  The lead developer is spending only about $2/month on Gemini and Claude together, so the expense should be very low.
+**AI API Keys.** You need a Gemini or Claude API key to use Plainbook. Click on the Settings button (the gear on the top right) to see instructions on how to set them. Both providers offer free trial credits, and usage costs are typically low for regular notebook work.
+
+### Key Features
+
+- **Natural language notebooks:** Describe what you want in plain English; AI generates and validates the code. 
+- **Multiple AI providers:** Use Gemini, Claude, or both—cross-check implementations for robustness.
+- **Built-in testing:** Write test cells to verify notebook behavior automatically.
+- **Shareable & reproducible:** Share notebooks with others who can modify, regenerate, and rerun your work.
 
 ### Resources
 
@@ -71,7 +81,7 @@ You can use any file name you like, with any extension you like.
 * [Pypi package](https://pypi.org/project/plainbook/).
 
 
-## Plainbook Structure ##
+## Plainbook Structure
 
 Plainbooks consist of three types of cells: 
 

{plainbook-0.2.2 → plainbook-0.2.4}/README.md

@@ -9,20 +9,22 @@ The use of plain language allows you to share your data analysis and science wit
 Plainbook can use multiple AIs to check that the code faithfully implements the natural language description, and can include tests to further validate the notebook. 
 When you share a notebook, the recipients can also check that the code under the hood implements the natural-language tasks. Recipients can also edit the plainbook, regenerate the code, and rerun it, just as in Jupyter notebooks. 
 
-Thus, the goal of the Plainbook project is to replicate in natural language what made Juptyter notebooks so successful: the ability to share together code and results, so that any recipient can validate and modify the notebook.
+Thus, the goal of the Plainbook project is to replicate in natural language what made Jupyter notebooks so successful: the ability to share together code and results, so that any recipient can validate and modify the notebook.
 
-### Trying Plainbook 
+### Try Plainbook Now
 
-* [30s short video](https://youtu.be/0t4ND8wPoYA)
-* [5 min Introductory video](https://youtu.be/Mkv5cl5rA7s). 
-* **Run it on Codespaces:** You can run plainbook right here on GitHub, with a trial Claude key, as follows:
-    * Create a codespace (Click on Code > Codespaces in the GitHub interface).
-    * Wait about 3 minutes, until after it finishes the postStartCommand.
-    * Once you see the button "Open in Browser" for port 8080 on the lower right, press it. 
-    * The Claude key provided in the trial has a low spending limit, and might not work.  You can add your own keys. 
+**Quick Start Videos:**
+* [30-second demo](https://youtu.be/0t4ND8wPoYA)
+* [5-minute introduction](https://youtu.be/Mkv5cl5rA7s)
 
-Example plainbooks:
-* Plainbook on soccer matches: [Download Soccer_w_Tests.plnb](https://github.com/lucadealfaro/plainbook/raw/main/examples/Soccer_w_Tests.plnb).  
+**Run on GitHub Codespaces (no installation needed):**
+1. Click **Code** → **Codespaces** in the GitHub interface
+2. Wait ~3 minutes for the environment to set up
+3. Click **Open in Browser** for port 8080
+4. A trial Claude API key is provided; you can add your own in Settings
+
+**Example Notebooks:**
+* [Soccer World Cup Analysis](https://github.com/lucadealfaro/plainbook/raw/main/examples/Soccer_w_Tests.plnb) — demonstrates action cells, tests, and AI validation
 
 
 ## Installation and use
@@ -36,12 +38,19 @@ pip install plainbook
 To open a plainbook (which will be created if it does not exist): 
 
 ```bash
-plainbook notebook.nlb
+plainbook notebook.plnb
 ```
 
 You can use any file name you like, with any extension you like. 
 
-**AI API Keys.** You need a Gemini or Claude API key to use Plainbook.  Click on the Settings button (the gear on the top right) to see instructions on how to set them.  The lead developer is spending only about $2/month on Gemini and Claude together, so the expense should be very low.
+**AI API Keys.** You need a Gemini or Claude API key to use Plainbook. Click on the Settings button (the gear on the top right) to see instructions on how to set them. Both providers offer free trial credits, and usage costs are typically low for regular notebook work.
+
+### Key Features
+
+- **Natural language notebooks:** Describe what you want in plain English; AI generates and validates the code. 
+- **Multiple AI providers:** Use Gemini, Claude, or both—cross-check implementations for robustness.
+- **Built-in testing:** Write test cells to verify notebook behavior automatically.
+- **Shareable & reproducible:** Share notebooks with others who can modify, regenerate, and rerun your work.
 
 ### Resources
 
@@ -49,7 +58,7 @@ You can use any file name you like, with any extension you like.
 * [Pypi package](https://pypi.org/project/plainbook/).
 
 
-## Plainbook Structure ##
+## Plainbook Structure
 
 Plainbooks consist of three types of cells: 
 

plainbook-0.2.4/plainbook/action_log.py

@@ -0,0 +1,322 @@
+"""Action log for user studies.
+
+When Plainbook is run with ``--log``, every mutating API call and every
+``active_cell_change`` client event is appended as a structured entry to
+``notebook.metadata['log']``. See ``Log.md`` at the repo root for the schema
+and analysis recipes.
+"""
+
+import datetime
+import hashlib
+import json
+import sys
+import time
+import traceback
+from functools import wraps
+
+from bottle import request, HTTPError
+
+
+LOG_ENABLED = False
+LOGVIEW_ENABLED = False
+_plainbook = None
+
+_LOG_SIZE_WARN_THRESHOLD = 10_000
+_warned_size = False
+
+
+def bind(plainbook, enabled):
+    """Wire the module to the Plainbook instance created by main.py.
+    On first enable, capture log_initial_state so the /log_view replay can
+    start from the notebook's state at that moment."""
+    global _plainbook, LOG_ENABLED
+    _plainbook = plainbook
+    LOG_ENABLED = bool(enabled)
+    if LOG_ENABLED and plainbook is not None:
+        if 'log_initial_state' not in plainbook.nb.metadata:
+            with plainbook._lock:
+                plainbook.nb.metadata['log_initial_state'] = _make_initial_snapshot(plainbook.nb)
+                plainbook._write()
+
+
+def _make_initial_snapshot(nb):
+    """Compact snapshot of current cells + notebook metadata, used as the
+    starting point for replay. Outputs and execution_count are omitted
+    (not replayable)."""
+    cells = []
+    for c in nb.cells:
+        md = dict(c.get('metadata') or {})
+        md.pop('_last_logged_hash', None)
+        cells.append({
+            'id': c.get('id'),
+            'cell_type': c.get('cell_type'),
+            'source': c.get('source', ''),
+            'metadata': md,
+        })
+    meta = {k: v for k, v in nb.metadata.items()
+            if k not in ('log', 'log_initial_state')}
+    return {
+        'ts_captured': _now_iso(),
+        'cells': cells,
+        'metadata': meta,
+    }
+
+
+OP_LOG_CONFIG = {
+    "set_key": {"redact_params": ["gemini_api_key", "claude_api_key"], "snapshot": False},
+    "set_active_ai": {"snapshot": False},
+    "edit_explanation": {"snapshot": True},
+    "edit_code": {"snapshot": True},
+    "edit_markdown": {"snapshot": True},
+    "clear_code": {"snapshot": True},
+    "insert_cell": {"snapshot": True},
+    "delete_cell": {"snapshot": False},
+    "move_cell": {"snapshot": False},
+    "execute_cell": {"snapshot": True, "drop_result_fields": ["outputs", "details"]},
+    "execute_test_cell": {"snapshot": True, "drop_result_fields": ["outputs"]},
+    "run_unit_test_cell": {"snapshot": True, "drop_result_fields": ["outputs"]},
+    "reset_kernel": {"snapshot": False},
+    "interrupt_kernel": {"snapshot": False},
+    "clear_outputs": {"snapshot": False},
+    "cancel_ai_request": {"snapshot": False},
+    "generate_code": {"snapshot": True, "truncate_result_fields": {"code": 8192}},
+    "generate_test_code": {"snapshot": True, "truncate_result_fields": {"code": 8192}},
+    "generate_unit_test_cell_code": {"snapshot": True, "truncate_result_fields": {"code": 8192}},
+    "validate_code": {"snapshot": True},
+    "validate_unit_test_code": {"snapshot": True},
+    "set_validation_visibility": {"snapshot": True},
+    "set_unit_test_validation_visibility": {"snapshot": True},
+    "save_unit_tests": {"snapshot": True},
+    "save_unit_test_explanation": {"snapshot": True},
+    "save_unit_test_code": {"snapshot": True},
+    "clear_unit_test_code": {"snapshot": True},
+    "clear_unit_test_outputs": {"snapshot": True},
+    "lock_notebook": {"snapshot": False},
+    "set_share_output": {"snapshot": False},
+    "set_files": {"snapshot": False, "truncate_param_fields": {"files": 4096, "missing_files": 4096}},
+    "set_ai_instructions": {"snapshot": False, "truncate_param_fields": {"ai_instructions": 4096}},
+    "reset_tokens": {"snapshot": False},
+}
+
+
+def _now_iso():
+    now = datetime.datetime.now(datetime.timezone.utc)
+    return now.strftime("%Y-%m-%dT%H:%M:%S.") + f"{now.microsecond // 1000:03d}Z"
+
+
+def _redact(value):
+    s = str(value) if value is not None else ""
+    return f"<redacted len={len(s)}>"
+
+
+def _truncate(value, max_bytes):
+    try:
+        s = value if isinstance(value, str) else json.dumps(value, default=str)
+    except Exception:
+        s = str(value)
+    if len(s) <= max_bytes:
+        return value
+    return s[:max_bytes] + f"...<truncated len={len(s)}>"
+
+
+def _filter_params(op_name, params):
+    if not params:
+        return params
+    cfg = OP_LOG_CONFIG.get(op_name, {})
+    out = dict(params)
+    for k in cfg.get("redact_params", ()):
+        if k in out and out[k]:
+            out[k] = _redact(out[k])
+    for k, max_bytes in cfg.get("truncate_param_fields", {}).items():
+        if k in out:
+            out[k] = _truncate(out[k], max_bytes)
+    return out
+
+
+def _filter_result(op_name, result):
+    if not isinstance(result, dict):
+        return result
+    cfg = OP_LOG_CONFIG.get(op_name, {})
+    out = {k: v for k, v in result.items() if k not in ("state", "unit_test_state")}
+    for k in cfg.get("drop_result_fields", ()):
+        out.pop(k, None)
+    for k, max_bytes in cfg.get("truncate_result_fields", {}).items():
+        if k in out and out[k] is not None:
+            out[k] = _truncate(out[k], max_bytes)
+    return out
+
+
+def _hash_cell_content(source, description):
+    h = hashlib.sha1()
+    h.update((source or "").encode("utf-8"))
+    h.update(b"\x00")
+    h.update((description or "").encode("utf-8"))
+    return h.hexdigest()
+
+
+def _extract_error_outputs(outputs):
+    error = None
+    stderr_parts = []
+    for o in outputs or []:
+        otype = o.get("output_type")
+        if otype == "error":
+            error = {
+                "ename": o.get("ename", ""),
+                "evalue": o.get("evalue", ""),
+                "traceback": list(o.get("traceback", [])),
+            }
+        elif otype == "stream" and o.get("name") == "stderr":
+            text = o.get("text", "")
+            if isinstance(text, list):
+                text = "".join(text)
+            stderr_parts.append(text)
+    stderr = "".join(stderr_parts) if stderr_parts else None
+    return error, stderr
+
+
+def _cell_snapshot(plainbook, cell_index):
+    if cell_index is None:
+        return None
+    try:
+        cell = plainbook.nb.cells[cell_index]
+    except (IndexError, TypeError, AttributeError):
+        return None
+    source = cell.get("source", "")
+    description = cell.metadata.get("explanation", "") if hasattr(cell, "metadata") else ""
+    new_hash = _hash_cell_content(source, description)
+    prev_hash = cell.metadata.get("_last_logged_hash") if hasattr(cell, "metadata") else None
+    changed = new_hash != prev_hash
+    snap = {
+        "cell_id": cell.get("id"),
+        "cell_type": cell.get("cell_type"),
+        "changed": changed,
+    }
+    if changed:
+        snap["source"] = source
+        snap["description"] = description
+        cell.metadata["_last_logged_hash"] = new_hash
+    outputs = cell.get("outputs", []) if cell.get("cell_type") in ("code", "test") else []
+    error, stderr = _extract_error_outputs(outputs)
+    if error is not None:
+        snap["error"] = error
+    if stderr is not None:
+        snap["stderr"] = stderr
+    return snap
+
+
+def _resolve_cell_index(op_name, params, result):
+    if not isinstance(params, dict):
+        return None
+    idx = params.get("cell_index")
+    if idx is not None:
+        return idx
+    if op_name == "insert_cell" and isinstance(result, dict):
+        return result.get("index")
+    return None
+
+
+def _resolve_cell_id(plainbook, cell_index):
+    if cell_index is None:
+        return None
+    try:
+        return plainbook.nb.cells[cell_index].get("id")
+    except (IndexError, TypeError, AttributeError):
+        return None
+
+
+def _build_entry(op_name, params, result, duration_ms, error_repr):
+    cell_index = _resolve_cell_index(op_name, params, result)
+    cfg = OP_LOG_CONFIG.get(op_name, {"snapshot": cell_index is not None})
+    entry = {
+        "ts_server": _now_iso(),
+        "source": "server",
+        "op": op_name,
+        "params": _filter_params(op_name, params),
+        "result": _filter_result(op_name, result) if result is not None else None,
+        "cell_id": _resolve_cell_id(_plainbook, cell_index),
+        "cell_index": cell_index,
+        "duration_ms": duration_ms,
+        "error": error_repr,
+    }
+    if cfg.get("snapshot", False):
+        snap = _cell_snapshot(_plainbook, cell_index)
+        if snap is not None:
+            entry["cell_snapshot"] = snap
+            if entry["cell_id"] is None:
+                entry["cell_id"] = snap.get("cell_id")
+    return entry
+
+
+def _append(entry):
+    global _warned_size
+    if _plainbook is None:
+        return
+    _plainbook.append_log_entry(entry)
+    try:
+        n = len(_plainbook.nb.metadata.get("log", []))
+    except Exception:
+        return
+    if n > _LOG_SIZE_WARN_THRESHOLD and not _warned_size:
+        print(f"Warning: action log has {n} entries (>{_LOG_SIZE_WARN_THRESHOLD}); "
+              f"notebook file will continue to grow.", file=sys.stderr)
+        _warned_size = True
+
+
+def logged(op_name):
+    """Decorator for Bottle POST routes. Captures request.json, invokes the
+    handler, then appends a filtered log entry. Exceptions are logged and
+    re-raised."""
+    def outer(fn):
+        @wraps(fn)
+        def wrapper(*a, **kw):
+            if LOGVIEW_ENABLED:
+                raise HTTPError(403, 'Notebook is in read-only --logview mode; mutations are not allowed.')
+            if not LOG_ENABLED:
+                return fn(*a, **kw)
+            try:
+                params = request.json if request.content_type and "json" in request.content_type else None
+            except Exception:
+                params = None
+            start = time.perf_counter()
+            error_repr = None
+            result = None
+            try:
+                result = fn(*a, **kw)
+                return result
+            except Exception as exc:
+                error_repr = repr(exc)
+                raise
+            finally:
+                duration_ms = int((time.perf_counter() - start) * 1000)
+                try:
+                    entry = _build_entry(op_name, params, result, duration_ms, error_repr)
+                    _append(entry)
+                except Exception as log_exc:
+                    print(f"Warning: action_log append failed for op={op_name}: {log_exc}",
+                          file=sys.stderr)
+                    if _plainbook is not None and getattr(_plainbook, "debug", False):
+                        traceback.print_exc()
+        return wrapper
+    return outer
+
+
+def append_client_event(evt):
+    """Record a client-originated event (active_cell_change). evt is the raw
+    JSON body posted to /log_client_event."""
+    if not LOG_ENABLED or _plainbook is None:
+        return
+    if not isinstance(evt, dict):
+        return
+    op = evt.get("op") or "client_event"
+    entry = {
+        "ts_server": _now_iso(),
+        "ts_client": evt.get("ts_client"),
+        "source": "client",
+        "op": op,
+        "from_id": evt.get("from_id"),
+        "from_index": evt.get("from_index"),
+        "to_id": evt.get("to_id"),
+        "to_index": evt.get("to_index"),
+        "duration_on_prev_ms": evt.get("duration_on_prev_ms"),
+    }
+    _append(entry)

{plainbook-0.2.2 → plainbook-0.2.4}/plainbook/ai_common.py

@@ -78,9 +78,59 @@ You should return the words YES (if the code meets the instructions) or NO (if i
 followed by a brief explanation.
 """
 
-VALIDATION_FEEDBACK_PREAMBLE = """The previous code for this cell does not seem to be correct. 
+VALIDATION_FEEDBACK_PREAMBLE = """The previous code for this cell does not seem to be correct.
 Here are comments on it given by an AI model:"""
 
+NOTEBOOK_VERIFY_INSTRUCTIONS = """
+You are auditing a Jupyter notebook on behalf of a user who needs assurance that the
+notebook is both correct and safe to run.
+You will receive every non-test code cell in order. For each cell you are given:
+  - the human-written description (the intended behavior),
+  - the Python code,
+  - the variables present after executing the cell,
+  - and possibly the cell's output.
+
+For every cell, verify BOTH of the following:
+  (a) The code implements what the description says (no missing steps, no extra steps
+      beyond what the description implies, no logic that contradicts it).
+  (b) The code performs no dangerous operations. Treat as dangerous, in particular:
+      deleting or overwriting files (os.remove, shutil.rmtree, Path.unlink, open(..., 'w')
+      on paths the description does not call out), modifying environment variables,
+      mutating system configuration or registry, executing shell commands
+      (os.system, subprocess.*, !shell, %%bash), installing packages,
+      and network calls to hosts the description does not mention.
+      Reading input files the description references is NOT dangerous.
+
+Reply on the very first line with EXACTLY one of these tokens:
+  OK            -- if every cell passes both (a) and (b).
+  VIOLATIONS    -- if any cell fails either check.
+
+If VIOLATIONS, follow on the next line with a markdown bullet list. Each bullet must
+name the offending cell (use its name if given, else "cell N") and describe the issue
+in one sentence. Do not include any other prose.
+"""
+
+TEST_VERIFY_INSTRUCTIONS = """
+You are auditing the test cells of a Jupyter notebook for safety only.
+You will receive every test cell in order. For each cell you are given the Python code
+and the variables present at the time of execution. You are NOT given a description and
+you should NOT verify whether the test logic is "correct" -- only whether it is safe.
+
+Verify that no test cell performs dangerous operations. Treat as dangerous, in particular:
+deleting or overwriting files (os.remove, shutil.rmtree, Path.unlink, open(..., 'w') on
+arbitrary paths), modifying environment variables, mutating system configuration,
+executing shell commands (os.system, subprocess.*, !shell, %%bash), installing packages,
+and network calls. Assertions, comparisons, and reads of in-memory variables are fine.
+
+Reply on the very first line with EXACTLY one of these tokens:
+  OK            -- if every test cell is safe.
+  VIOLATIONS    -- if any test cell performs a dangerous operation.
+
+If VIOLATIONS, follow on the next line with a markdown bullet list. Each bullet must
+name the offending test cell (use its name if given, else "test cell N") and describe
+the dangerous operation in one sentence. Do not include any other prose.
+"""
+
 VARIABLES_FOR_TARGET_DESCRIPTION = """In generating the test setup, the goal is to set the value of these variables, 
 possibly simplifying them if they are long or complex, so that a human can make sense of their values 
 and can check the results of the execution of the test cell.  The variables are these:\n"""
@@ -381,3 +431,21 @@ def parse_validation_response(text):
         return dict(is_valid=False, message=clean_start(r[2:]))
     else:
         return dict(is_valid=False, message=r)
+
+
+def parse_verify_response(text):
+    """Parse an OK / VIOLATIONS verification response into a result dict.
+    The first line carries the verdict; the rest is the (markdown) message body."""
+    r = text.strip()
+    first, _, rest = r.partition("\n")
+    head = first.strip().upper()
+    body = rest.strip()
+    if head.startswith("OK"):
+        return dict(is_valid=True, message=body)
+    if head.startswith("VIOLATIONS"):
+        # If the model did not include any body, surface a generic message so the
+        # red bar is never empty.
+        return dict(is_valid=False, message=body or "Violations were reported but no details were given.")
+    # Fallback: model did not follow the protocol -- treat as a violation and
+    # show the entire response so the user can see what happened.
+    return dict(is_valid=False, message=r)

{plainbook-0.2.2 → plainbook-0.2.4}/plainbook/claude.py

@@ -8,6 +8,8 @@ from .ai_common import (
     UNIT_TEST_SYSTEM_INSTRUCTIONS,
     CHECKING_INSTRUCTIONS,
     NAME_GENERATION_INSTRUCTIONS,
+    NOTEBOOK_VERIFY_INSTRUCTIONS,
+    TEST_VERIFY_INSTRUCTIONS,
     add_tokens,
     build_context_prompt,
     build_unit_test_prompt,
@@ -15,6 +17,7 @@ from .ai_common import (
     dump_ai_request,
     log_ai_request_size,
     parse_validation_response,
+    parse_verify_response,
     strip_markdown_code_fences,
 )
 
@@ -274,6 +277,43 @@ Validation Result:
     return parse_validation_response(response_text)
 
 
+def _claude_verify(api_key, system_instructions, payload, label, model=None,
+                   debug=False, dump_ai_requests=False):
+    client = _get_client(api_key)
+    model = model or CLAUDE_MODEL
+    if debug:
+        log_ai_request_size(f"claude {label}", system_instructions, payload)
+    if dump_ai_requests:
+        dump_ai_request(dump_ai_requests, f"claude {label}", {
+            "model": model, "max_tokens": 2048,
+            "system": system_instructions,
+            "messages": [{"role": "user", "content": payload}],
+        })
+    message = client.messages.create(
+        model=model,
+        max_tokens=2048,
+        system=system_instructions,
+        messages=[{"role": "user", "content": payload}],
+    )
+    add_tokens(message.usage.input_tokens, message.usage.output_tokens)
+    response_text = message.content[0].text
+    if debug:
+        print("Response:", response_text)
+    return parse_verify_response(response_text)
+
+
+def claude_verify_notebook(api_key, payload, model=None, debug=False, dump_ai_requests=False):
+    return _claude_verify(api_key, NOTEBOOK_VERIFY_INSTRUCTIONS, payload,
+                          "verify_notebook", model=model, debug=debug,
+                          dump_ai_requests=dump_ai_requests)
+
+
+def claude_verify_tests(api_key, payload, model=None, debug=False, dump_ai_requests=False):
+    return _claude_verify(api_key, TEST_VERIFY_INSTRUCTIONS, payload,
+                          "verify_tests", model=model, debug=debug,
+                          dump_ai_requests=dump_ai_requests)
+
+
 def claude_generate_cell_name(api_key, explanation, model=None, debug=False, dump_ai_requests=False):
     client = _get_client(api_key)
     model = model or CLAUDE_MODEL