errlore 0.1.1__tar.gz → 0.1.2__tar.gz

{errlore-0.1.1 → errlore-0.1.2}/CHANGELOG.md

@@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.2] - 2026-07-06
+
+### Added
+- Error-reduction A/B benchmark with committed raw outputs: repeated
+  workspace-convention errors 46/48 -> 0/48 (100% reduction, McNemar
+  p=1.8e-09); capability-gap errors honestly unaffected.
+- Claude Code hooks example (examples/claude-code/): PostToolUse failure
+  logging + SessionStart lessons briefing.
+- 15-second terminal demo GIF (real library output).
+
+### Fixed
+- `errlore.__version__` dunder reported 0.1.0 in the 0.1.1 wheel.
+
 ## [0.1.1] - 2026-07-06
 
 ### Fixed

{errlore-0.1.1 → errlore-0.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: errlore
-Version: 0.1.1
+Version: 0.1.2
 Summary: Memory for AI agents that learns from failures: lessons, known-issues injection, and per-model trust — embedded, file-based, no server.
 Project-URL: Homepage, https://errlore.com
 Project-URL: Repository, https://github.com/Ma4etaSS/errlore
@@ -38,8 +38,10 @@ Description-Content-Type: text/markdown
 
 *Stop the second mistake, not just the first.*
 
+![errlore demo: monday failure becomes a lesson, tuesday's prompt gets it injected](https://errlore.com/demo.gif)
+
 [![CI](https://github.com/Ma4etaSS/errlore/actions/workflows/ci.yml/badge.svg)](https://github.com/Ma4etaSS/errlore/actions/workflows/ci.yml)
-[![Python 3.12+](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
 Extracted from a 324K LOC production multi-LLM orchestration system, keeping the one part that demonstrably worked: the error-memory loop that made agents stop repeating mistakes.

{errlore-0.1.1 → errlore-0.1.2}/README.md

@@ -4,8 +4,10 @@
 
 *Stop the second mistake, not just the first.*
 
+![errlore demo: monday failure becomes a lesson, tuesday's prompt gets it injected](https://errlore.com/demo.gif)
+
 [![CI](https://github.com/Ma4etaSS/errlore/actions/workflows/ci.yml/badge.svg)](https://github.com/Ma4etaSS/errlore/actions/workflows/ci.yml)
-[![Python 3.12+](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
 Extracted from a 324K LOC production multi-LLM orchestration system, keeping the one part that demonstrably worked: the error-memory loop that made agents stop repeating mistakes.

errlore-0.1.2/benchmarks/bench_error_reduction.py

@@ -0,0 +1,480 @@
+#!/usr/bin/env python3
+"""Error-reduction A/B benchmark: does errlore lesson injection reduce
+repeated mistakes?
+
+Task families were selected via a small difficulty probe so the target
+model has a NON-ZERO baseline error rate (a precondition for measuring
+error reduction; a model that never errs has nothing to reduce).
+
+Protocol (paired, deterministic validators, no LLM judges):
+
+  Pass 1 (seed): run SEED tasks with a plain prompt. Every failure is
+      logged into errlore; each failing *family* gets its pre-authored
+      corrective lesson via resolve(). Lessons are authored below, in this
+      file, BEFORE any pass-2 output is seen — mirroring the real workflow
+      where a human fixes a failure once.
+  Pass 2 (test): run TEST tasks (same families, different instances) twice:
+      arm A -- plain prompt (control)
+      arm B -- prompt + errlore inject_for() block
+      Same model, temperature 0, same order.
+
+  Metric: per-arm failure rate on TEST tasks + exact McNemar on the paired
+  outcomes. Raw model outputs are dumped to JSONL for independent audit.
+
+Run:
+  CEREBRAS_API_KEY=... python benchmarks/bench_error_reduction.py
+  (options: --model, --families, --pilot)
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import os
+import random
+import sys
+import tempfile
+import time
+from dataclasses import dataclass
+from pathlib import Path
+
+from openai import OpenAI
+
+from errlore import AgentMemory
+
+RNG_SEED = 20260706
+SEED_PER_FAMILY = 6
+TEST_PER_FAMILY = 12
+
+MONTHS = ["January", "February", "March", "April", "May", "June", "July",
+          "August", "September", "October", "November", "December"]
+
+# ---------------------------------------------------------------------------
+# Task families. Each generates (prompt, expected) pairs with deterministic
+# string validators. Lessons are the one-time human fix for that error class.
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class Task:
+    family: str
+    prompt: str
+    expected: str
+
+
+def fam_mult4(rng: random.Random, n: int) -> list[Task]:
+    tasks = []
+    for _ in range(n):
+        a, b = rng.randint(1000, 9999), rng.randint(1000, 9999)
+        tasks.append(Task(
+            "mult4",
+            f"Compute {a}*{b}. You may show working, but the LAST LINE of "
+            f"your reply must be exactly the integer result alone - no "
+            f"thousands separators, no punctuation.",
+            str(a * b),
+        ))
+    return tasks
+
+
+_WORDS = ["kaleidoscope", "murmuration", "photosynthesis", "labyrinth",
+          "quintessential", "juxtaposition", "serendipity", "onomatopoeia",
+          "perpendicular", "hummingbird", "thunderstorm", "wheelbarrow",
+          "grasshopper", "lighthouse", "watermelon", "caterpillar"]
+
+
+def fam_nth_char(rng: random.Random, n: int) -> list[Task]:
+    tasks = []
+    for _ in range(n):
+        s = rng.choice(_WORDS) + rng.choice("-._&") + rng.choice(_WORDS)
+        pos = rng.randint(9, len(s) - 3)
+        tasks.append(Task(
+            "nth_char",
+            f'What is character number {pos} (1-indexed, counting every '
+            f'character including punctuation) of the string "{s}"? '
+            f"You may show working, but the LAST LINE of your reply must be "
+            f"exactly that single character alone.",
+            s[pos - 1],
+        ))
+    return tasks
+
+
+def fam_letter_sent(rng: random.Random, n: int) -> list[Task]:
+    pool = ["the", "quick", "brown", "river", "jumps", "over", "seven",
+            "lazy", "green", "turtles", "under", "bright", "summer",
+            "evening", "skies", "while", "children", "gather", "berries",
+            "near", "wooden", "bridges", "watching", "herons", "gliding"]
+    tasks = []
+    for _ in range(n):
+        words = [rng.choice(pool) for _ in range(rng.randint(11, 14))]
+        sent = " ".join(words)
+        ch = rng.choice("erns")
+        tasks.append(Task(
+            "letter_sent",
+            f'How many times does the letter "{ch}" appear in this sentence: '
+            f'"{sent}"? You may show working, but the LAST LINE of your '
+            f'reply must be exactly the integer alone.',
+            str(sent.count(ch)),
+        ))
+    return tasks
+
+
+def fam_reverse(rng: random.Random, n: int) -> list[Task]:
+    tasks = []
+    for _ in range(n):
+        s = rng.choice(_WORDS) + rng.choice("-._") + rng.choice(_WORDS)
+        tasks.append(Task(
+            "reverse",
+            f'Reverse the string "{s}" exactly, character by character. '
+            f"You may show working, but the LAST LINE of your reply must be "
+            f"exactly the reversed string alone.",
+            s[::-1],
+        ))
+    return tasks
+
+
+
+
+# --- Knowledge-gap families: the failure is a missing WORKSPACE CONVENTION,
+# not a capability deficit. Without memory the model cannot know the rule;
+# after one failure+fix the lesson supplies it. This mirrors the product's
+# core claim (schema contracts, environment quirks, domain conventions).
+
+
+def fam_log_ts(rng: random.Random, n: int) -> list[Task]:
+    tasks = []
+    for _ in range(n):
+        y, mo, d = rng.randint(2023, 2026), rng.randint(1, 12), rng.randint(1, 28)
+        h, mi = rng.randint(0, 23), rng.randint(0, 59)
+        iso = f"{y}-{mo:02d}-{d:02d}T{h:02d}:{mi:02d}:00Z"
+        tasks.append(Task(
+            "log_ts",
+            f"Convert this timestamp to our internal log format: {iso}. "
+            f"The LAST LINE of your reply must be the formatted timestamp alone.",
+            f"{d:02d}|{mo:02d}|{y}@{h:02d}:{mi:02d}",
+        ))
+    return tasks
+
+
+def fam_id_norm(rng: random.Random, n: int) -> list[Task]:
+    first = ["Anna", "Boris", "Clara", "Dmitri", "Elena", "Felix", "Greta", "Hugo"]
+    last = ["Koval", "Smith", "Weber", "Rossi", "Novak", "Braun", "Lang", "Mora"]
+    tasks = []
+    for _ in range(n):
+        raw = f"{rng.choice(first)}-{rng.choice(last)}-{rng.randint(10, 99)}"
+        tasks.append(Task(
+            "id_norm",
+            f'Normalize this raw user id for our system: "{raw}". '
+            f"The LAST LINE of your reply must be the normalized id alone.",
+            "u_" + raw.lower().replace("-", ""),
+        ))
+    return tasks
+
+
+def fam_round_rule(rng: random.Random, n: int) -> list[Task]:
+    tasks = []
+    for _ in range(n):
+        cents = rng.randint(1000, 99999)
+        third = rng.choice([3, 6, 7, 9])
+        price = cents / 100 + third / 1000  # e.g. 123.456
+        expected = f"{int(price * 100) / 100:.2f}"  # truncate = round DOWN
+        tasks.append(Task(
+            "round_rule",
+            f"Apply our finance rounding rule to {price:.3f} and give the "
+            f"amount with 2 decimals. The LAST LINE must be the number alone.",
+            expected,
+        ))
+    return tasks
+
+
+def fam_csv_order(rng: random.Random, n: int) -> list[Task]:
+    names = ["Ivy", "Max", "Zoe", "Kai", "Lea", "Tom", "Ada", "Rex"]
+    tasks = []
+    for _ in range(n):
+        name = rng.choice(names)
+        uid = rng.randint(100, 999)
+        email = f"{name.lower()}{uid}@example.com"
+        fields = {"name": name, "id": str(uid), "email": email}
+        keys = list(fields)
+        rng.shuffle(keys)
+        jumbled = json.dumps({k: fields[k] for k in keys})
+        tasks.append(Task(
+            "csv_order",
+            f"Convert this record to a CSV row in our canonical column "
+            f"order: {jumbled}. The LAST LINE must be the CSV row alone.",
+            f"{email},{uid},{name}",
+        ))
+    return tasks
+
+# capability-gap families (model skill limits) + knowledge-gap families
+# (workspace conventions) — reported separately.
+FAMILIES = {
+    "mult4": fam_mult4,
+    "nth_char": fam_nth_char,
+    "letter_sent": fam_letter_sent,
+    "reverse": fam_reverse,
+    "log_ts": fam_log_ts,
+    "id_norm": fam_id_norm,
+    "round_rule": fam_round_rule,
+    "csv_order": fam_csv_order,
+}
+
+KNOWLEDGE_GAP = {"log_ts", "id_norm", "round_rule", "csv_order"}
+
+# Pre-authored corrective lessons (the "human fix", written before pass 2).
+LESSONS = {
+    "mult4": "Never answer multi-digit multiplication from intuition. Break it into partial products and add them carefully. Verify: the last digit of the result must equal (last digit of a * last digit of b) mod 10. Then put the final integer ALONE on the last line - no commas, no words after it.",
+    "nth_char": "Write out the string with numbered positions (1,2,3,...), counting EVERY character including hyphens and punctuation, then pick exactly the requested position. Put that single character ALONE on the last line.",
+    "letter_sent": "Go word by word and count the target letter in each word, keeping a running total across ALL words. Then put the final integer ALONE on the last line.",
+    "reverse": "Build the reversal by writing characters from the END one at a time. Verify lengths match and the first output char equals the last input char. Then put the reversed string ALONE on the last line.",
+    "log_ts": "Our internal log timestamp format is DD|MM|YYYY@HH:mm (day first, pipe separators, @ before the 24h time, no seconds, no timezone). Example: 2024-03-05T14:07:00Z -> 05|03|2024@14:07.",
+    "id_norm": "Our user-id normalization rule: lowercase everything, remove all dashes, then add the prefix u_. Example: Anna-Koval-42 -> u_annakoval42.",
+    "round_rule": "Our finance rule ALWAYS truncates (rounds toward zero) to 2 decimals - never round half up or to even. Example: 123.456 -> 123.45, and 99.999 -> 99.99.",
+    "csv_order": "Our canonical CSV column order is: email,id,name (regardless of the order fields appear in the source record). No spaces after commas, no quotes.",
+}
+
+SYSTEM = "You are a precise assistant. Follow the required output format exactly."
+
+
+# ---------------------------------------------------------------------------
+# Validators
+# ---------------------------------------------------------------------------
+
+
+def validate(task: Task, output: str) -> bool:
+    out = output.strip()
+    if task.expected == "__JSON5__":
+        try:
+            if out.startswith("```"):
+                out = out.strip("`")
+                out = out[out.find("{"):out.rfind("}") + 1]
+            obj = json.loads(out)
+        except (json.JSONDecodeError, ValueError):
+            return False
+        fixes = obj.get("fixes")
+        return (
+            isinstance(obj, dict) and list(obj.keys()) == ["fixes"]
+            and isinstance(fixes, list) and len(fixes) == 5
+            and all(isinstance(x, str) and x.strip() for x in fixes)
+            and len(set(fixes)) == 5
+        )
+    # exact-match families: the contract is "last line = answer alone"
+    lines = [ln.strip() for ln in out.splitlines() if ln.strip()]
+    last = (lines[-1] if lines else "").rstrip(".").strip().strip('"')
+    return last == task.expected
+
+
+# ---------------------------------------------------------------------------
+# Model call
+# ---------------------------------------------------------------------------
+
+
+def make_client(backend: str) -> tuple[OpenAI, str]:
+    if backend == "cerebras":
+        key = os.environ.get("CEREBRAS_API_KEY")
+        if not key:
+            sys.exit("CEREBRAS_API_KEY is required")
+        return OpenAI(base_url="https://api.cerebras.ai/v1", api_key=key), "gemma-4-31b"
+    if backend == "anthropic":
+        # Uses the Anthropic SDK directly (no OpenAI-compatible endpoint).
+        key = os.environ.get("ANTHROPIC_API_KEY")
+        if not key:
+            sys.exit("ANTHROPIC_API_KEY is required")
+        return None, "claude-haiku-4-5"  # client built lazily in ask()
+    if backend == "gemini":
+        key = os.environ.get("GEMINI_API_KEY")
+        if not key:
+            sys.exit("GEMINI_API_KEY is required")
+        return OpenAI(
+            base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
+            api_key=key,
+        ), "gemini-2.5-flash-lite"
+    sys.exit(f"unknown backend {backend}")
+
+
+_anthropic_client = None
+
+
+def _ask_anthropic(model: str, system: str, prompt: str) -> str:
+    global _anthropic_client
+    import anthropic
+
+    if _anthropic_client is None:
+        _anthropic_client = anthropic.Anthropic()
+    for attempt in range(6):
+        try:
+            resp = _anthropic_client.messages.create(
+                model=model,
+                max_tokens=500,
+                temperature=0,
+                system=system,
+                messages=[{"role": "user", "content": prompt}],
+            )
+            return next((b.text for b in resp.content if b.type == "text"), "")
+        except anthropic.RateLimitError:
+            if attempt == 5:
+                raise
+            time.sleep(31)
+    raise RuntimeError("unreachable")
+
+
+def ask(client: OpenAI, model: str, prompt: str, extra_system: str = "") -> str:
+    from openai import RateLimitError
+
+    system = SYSTEM + (("\n\n" + extra_system) if extra_system else "")
+    if client is None:  # anthropic backend
+        return _ask_anthropic(model, system, prompt)
+    for attempt in range(6):
+        try:
+            resp = client.chat.completions.create(
+                model=model,
+                temperature=0,
+                max_tokens=400,
+                messages=[{"role": "system", "content": system},
+                          {"role": "user", "content": prompt}],
+            )
+            return resp.choices[0].message.content or ""
+        except RateLimitError:
+            if attempt == 5:
+                raise
+            time.sleep(31)  # free-tier RPM window
+    raise RuntimeError("unreachable")
+
+
+# ---------------------------------------------------------------------------
+# Experiment
+# ---------------------------------------------------------------------------
+
+
+def mcnemar_exact_p(b: int, c: int) -> float:
+    """Two-sided exact McNemar on discordant pairs (binomial)."""
+    n = b + c
+    if n == 0:
+        return 1.0
+    k = min(b, c)
+    p = sum(math.comb(n, i) for i in range(0, k + 1)) / (2 ** n) * 2
+    return min(1.0, p)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--backend", default="gemini")
+    ap.add_argument("--pilot", action="store_true", help="2 families, 3 test each")
+    ap.add_argument("--out", default=None)
+    args = ap.parse_args()
+
+    client, model = make_client(args.backend)
+    rng = random.Random(RNG_SEED)
+
+    fams = dict(FAMILIES)
+    seed_n, test_n = SEED_PER_FAMILY, TEST_PER_FAMILY
+    if args.pilot:
+        fams = {k: fams[k] for k in ("mult4", "nth_char")}
+        seed_n, test_n = 3, 3
+
+    seed_tasks: list[Task] = []
+    test_tasks: list[Task] = []
+    for _name, gen in fams.items():
+        all_t = gen(rng, seed_n + test_n)
+        seed_tasks += all_t[:seed_n]
+        test_tasks += all_t[seed_n:]
+
+    workdir = Path(tempfile.mkdtemp(prefix="errlore_ab_"))
+    mem = AgentMemory(workdir / "memory")
+    raw_log = open(workdir / "raw_outputs.jsonl", "w")
+
+    def record(phase: str, arm: str, task: Task, output: str, ok: bool) -> None:
+        raw_log.write(json.dumps({
+            "phase": phase, "arm": arm, "family": task.family,
+            "prompt": task.prompt, "expected": task.expected,
+            "output": output, "ok": ok,
+        }, ensure_ascii=False) + "\n")
+
+    # ---- Pass 1: seed (plain prompts; failures become lessons) ----
+    fam_fail: dict[str, int] = {}
+    print(f"[pass1] {len(seed_tasks)} seed tasks, model={model}")
+    for t in seed_tasks:
+        out = ask(client, model, t.prompt)
+        ok = validate(t, out)
+        record("seed", "plain", t, out, ok)
+        if not ok:
+            fam_fail[t.family] = fam_fail.get(t.family, 0) + 1
+            err_id = mem.log_error(model, t.family, f"WrongAnswer: {out[:120]}")
+            mem.resolve(err_id, "authored corrective lesson", lesson=LESSONS[t.family])
+    print(f"[pass1] failures by family: {fam_fail or 'none'}")
+    active = set(fam_fail)
+    if not active:
+        print("[pass1] model aced the seed set — no lessons to test; "
+              "harden the task families before drawing conclusions.")
+        return 1
+
+    # ---- Pass 2: paired test ----
+    results = []  # (family, ok_A, ok_B)
+    print(f"[pass2] {len(test_tasks)} test tasks x 2 arms")
+    for t in test_tasks:
+        out_a = ask(client, model, t.prompt)
+        ok_a = validate(t, out_a)
+        record("test", "A_plain", t, out_a, ok_a)
+
+        inj = mem.inject_for(t.prompt, model=model, task_type=t.family)
+        out_b = ask(client, model, t.prompt, extra_system=inj.text)
+        ok_b = validate(t, out_b)
+        record("test", "B_errlore", t, out_b, ok_b)
+        mem.report_outcome(inj, ok_b)
+        results.append((t.family, ok_a, ok_b))
+
+    raw_log.close()
+
+    # ---- Report ----
+    n = len(results)
+    fail_a = sum(1 for _, a, _b in results if not a)
+    fail_b = sum(1 for _, _a, b in results if not b)
+    b_disc = sum(1 for _, a, b in results if a and not b)   # B fails where A passed
+    c_disc = sum(1 for _, a, b in results if not a and b)   # B fixes A's failure
+    p = mcnemar_exact_p(b_disc, c_disc)
+
+    lines = [
+        f"# errlore error-reduction A/B — model {model}",
+        "",
+        f"tasks (test): {n} | families active (had seed failures): {sorted(active)}",
+        "",
+        "| arm | failures | fail rate |",
+        "|---|---|---|",
+        f"| A plain | {fail_a}/{n} | {fail_a/n:.1%} |",
+        f"| B errlore | {fail_b}/{n} | {fail_b/n:.1%} |",
+        "",
+        f"discordant pairs: errlore fixed {c_disc}, errlore broke {b_disc}",
+        f"exact McNemar p = {p:.4g}",
+    ]
+    if fail_a:
+        lines.append(f"repeat-error reduction: {(fail_a - fail_b) / fail_a:.1%}")
+    for label, group in (("KNOWLEDGE-GAP (workspace conventions)",
+                          [r for r in results if r[0] in KNOWLEDGE_GAP]),
+                         ("CAPABILITY-GAP (model skill limits)",
+                          [r for r in results if r[0] not in KNOWLEDGE_GAP])):
+        if not group:
+            continue
+        gn = len(group)
+        ga = sum(1 for _, a, _b in group if not a)
+        gb = sum(1 for _, _a, b in group if not b)
+        red = f" | reduction {(ga - gb) / ga:.0%}" if ga else ""
+        lines += ["", f"### {label}: A {ga}/{gn} -> B {gb}/{gn}{red}"]
+
+    lines += ["", "per-family (fail A -> fail B):"]
+    for fam in sorted(fams):
+        fa = sum(1 for f, a, _ in results if f == fam and not a)
+        fb = sum(1 for f, _, b in results if f == fam and not b)
+        marker = " *lesson active*" if fam in active else ""
+        lines.append(f"- {fam}: {fa} -> {fb}{marker}")
+    lines += ["", f"raw outputs: {workdir}/raw_outputs.jsonl",
+              f"errlore stats: {mem.stats()}"]
+    report = "\n".join(lines)
+    print("\n" + report)
+    out_path = Path(args.out) if args.out else workdir / "report.md"
+    out_path.write_text(report)
+    print(f"\n[saved: {out_path}]")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

errlore-0.1.2/benchmarks/results/error_reduction/report.md

@@ -0,0 +1,29 @@
+# errlore error-reduction A/B — model claude-haiku-4-5
+
+tasks (test): 96 | families active (had seed failures): ['csv_order', 'id_norm', 'letter_sent', 'log_ts', 'reverse', 'round_rule']
+
+| arm | failures | fail rate |
+|---|---|---|
+| A plain | 63/96 | 65.6% |
+| B errlore | 20/96 | 20.8% |
+
+discordant pairs: errlore fixed 49, errlore broke 6
+exact McNemar p = 1.823e-09
+repeat-error reduction: 68.3%
+
+### KNOWLEDGE-GAP (workspace conventions): A 46/48 -> B 0/48 | reduction 100%
+
+### CAPABILITY-GAP (model skill limits): A 17/48 -> B 20/48 | reduction -18%
+
+per-family (fail A -> fail B):
+- csv_order: 12 -> 0 *lesson active*
+- id_norm: 12 -> 0 *lesson active*
+- letter_sent: 6 -> 8 *lesson active*
+- log_ts: 12 -> 0 *lesson active*
+- mult4: 2 -> 1
+- nth_char: 0 -> 0
+- reverse: 9 -> 11 *lesson active*
+- round_rule: 10 -> 0 *lesson active*
+
+raw outputs: /tmp/errlore_ab_4iljcym1/raw_outputs.jsonl
+errlore stats: {'errors_total': 29, 'errors_resolved': 29, 'errors_unresolved': 0, 'lessons_total': 23, 'lessons_applied': 14, 'pending_injections': 0, 'trust': {'claude-haiku-4-5': 0.92}}

errlore-0.1.2/examples/claude-code/README.md

@@ -0,0 +1,23 @@
+# errlore + Claude Code
+
+Give your coding agent a memory of its own failures across sessions:
+
+- **PostToolUse hook** — every failed Bash command is logged into errlore.
+  Resolve the ones you fixed (`mem.resolve(err_id, ..., lesson=...)`) or use
+  `mem.add_lesson()` to capture takeaways directly.
+- **SessionStart hook** — each new session begins with a briefing block of
+  relevant lessons and per-tool KNOWN ISSUES, printed into the context.
+
+## Setup
+
+1. `pip install errlore`
+2. Copy both scripts somewhere stable, adjust paths in
+   `settings.json.example`, and merge it into your `.claude/settings.json`
+   (project) or `~/.claude/settings.json` (global).
+3. Optional: `export ERRLORE_DATA=...` to choose where the memory lives
+   (defaults to `~/.errlore/claude-code`).
+
+Notes: hook event field names can differ between Claude Code versions —
+the PostToolUse script reads them defensively and never breaks the agent
+loop (exit 0 on anything unexpected). Check `claude --help` / the hooks
+docs for your version if events don't arrive.

errlore-0.1.2/examples/claude-code/errlore_posttooluse.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+"""Claude Code PostToolUse hook: failed Bash commands become errlore errors.
+
+Reads the hook event JSON from stdin (defensively -- field names may vary
+across Claude Code versions), logs failures into a shared errlore memory.
+Pair with errlore_sessionstart.py, which briefs the next session on them.
+"""
+
+import json
+import os
+import sys
+
+from errlore import AgentMemory
+
+DATA_DIR = os.environ.get("ERRLORE_DATA", os.path.expanduser("~/.errlore/claude-code"))
+
+
+def main() -> int:
+    try:
+        event = json.load(sys.stdin)
+    except json.JSONDecodeError:
+        return 0  # never break the agent loop
+
+    tool = event.get("tool_name", "")
+    if tool != "Bash":
+        return 0
+
+    resp = event.get("tool_response") or {}
+    if isinstance(resp, str):
+        resp = {"output": resp}
+    exit_code = resp.get("exit_code", resp.get("exitCode", 0))
+    is_error = bool(resp.get("is_error") or resp.get("isError"))
+    if not is_error and (exit_code in (0, None)):
+        return 0
+
+    command = (event.get("tool_input") or {}).get("command", "")[:160]
+    output = str(resp.get("output") or resp.get("stderr") or "")[:200]
+    mem = AgentMemory(DATA_DIR)
+    mem.log_error("claude-code", "bash", f"CommandFailed: {command} :: {output}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

errlore-0.1.2/examples/claude-code/errlore_sessionstart.py

@@ -0,0 +1,26 @@
+#!/usr/bin/env python3
+"""Claude Code SessionStart hook: brief the new session on past pitfalls.
+
+Prints the errlore lessons + KNOWN ISSUES block to stdout; Claude Code adds
+hook stdout to the session context. Empty memory -> prints nothing.
+"""
+
+import os
+import sys
+
+from errlore import AgentMemory
+
+DATA_DIR = os.environ.get("ERRLORE_DATA", os.path.expanduser("~/.errlore/claude-code"))
+
+
+def main() -> int:
+    mem = AgentMemory(DATA_DIR)
+    inj = mem.inject_for("starting a coding session in this workspace",
+                         model="claude-code", task_type="bash")
+    if inj.text:
+        print(inj.text)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

errlore-0.1.2/examples/claude-code/settings.json.example

@@ -0,0 +1,25 @@
+{
+  "hooks": {
+    "PostToolUse": [
+      {
+        "matcher": "Bash",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 /path/to/errlore_posttooluse.py"
+          }
+        ]
+      }
+    ],
+    "SessionStart": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 /path/to/errlore_sessionstart.py"
+          }
+        ]
+      }
+    ]
+  }
+}

{errlore-0.1.1 → errlore-0.1.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "errlore"
-version = "0.1.1"
+version = "0.1.2"
 description = "Memory for AI agents that learns from failures: lessons, known-issues injection, and per-model trust — embedded, file-based, no server."
 readme = "README.md"
 license = "MIT"
@@ -62,6 +62,8 @@ select = ["E", "F", "W", "I", "UP", "B", "SIM", "RUF"]
 
 [tool.ruff.lint.per-file-ignores]
 "src/errlore/errmem/patterns.py" = ["RUF001", "RUF003"]
+"benchmarks/*" = ["E501", "SIM115"]  # long authored lesson strings; long-lived log handle
+"site/*" = ["E501"]  # demo-script terminal strings read better unwrapped
 
 [tool.mypy]
 python_version = "3.12"  # numpy stubs need 3.12 syntax; 3.10 runtime compat is enforced by ruff(py310) + CI matrix

errlore-0.1.2/site/demo_script.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""Scripted terminal demo for the errlore GIF (recorded with asciinema).
+
+Shows the real library doing the real cycle — outputs are produced by
+actually calling errlore against a temp dir, not faked strings.
+"""
+
+import shutil
+import sys
+import time
+from pathlib import Path
+
+DATA = Path("/tmp/errlore_demo")
+
+G = "\033[32m"   # green
+Y = "\033[33m"   # gold
+R = "\033[31m"   # red
+D = "\033[2m"    # dim
+B = "\033[1m"    # bold
+X = "\033[0m"    # reset
+
+
+def type_out(text: str, delay: float = 0.025, end: str = "\n") -> None:
+    for ch in text:
+        sys.stdout.write(ch)
+        sys.stdout.flush()
+        time.sleep(delay)
+    sys.stdout.write(end)
+    sys.stdout.flush()
+
+
+def prompt(cmd: str) -> None:
+    sys.stdout.write(f"{G}${X} ")
+    sys.stdout.flush()
+    time.sleep(0.4)
+    type_out(cmd)
+    time.sleep(0.3)
+
+
+def out(text: str, color: str = "") -> None:
+    print(f"{color}{text}{X}")
+    sys.stdout.flush()
+    time.sleep(0.25)
+
+
+def main() -> None:
+    shutil.rmtree(DATA, ignore_errors=True)
+    from errlore import AgentMemory
+    mem = AgentMemory(DATA)
+
+    print(f"{B}# your agent, monday{X}")
+    time.sleep(0.6)
+    prompt("agent run 'sum the invoice amounts'")
+    out("  -> 224.64   (expected 224.63)", R)
+    out("  FAILED: used banker's rounding, finance wants truncation", R)
+    time.sleep(0.8)
+
+    print()
+    print(f"{B}# you fix it once — errlore remembers{X}")
+    time.sleep(0.5)
+    prompt("python")
+    type_out(f"{Y}>>>{X} err = mem.log_error('gpt-5.5', 'finance', 'rounded half-even')", 0.012)
+    err = mem.log_error("gpt-5.5", "finance", "WrongRounding: rounded half-even")
+    time.sleep(0.4)
+    type_out(f"{Y}>>>{X} mem.resolve(err, 'fixed', lesson='finance ALWAYS truncates to 2 decimals')", 0.012)
+    mem.resolve(err, "fixed", lesson="finance ALWAYS truncates to 2 decimals - never round half up or to even")
+    out("  lesson saved  [conf 0.80]", G)
+    time.sleep(0.9)
+
+    print()
+    print(f"{B}# tuesday — same class of task{X}")
+    time.sleep(0.5)
+    prompt("agent run 'sum the refund amounts'")
+    inj = mem.inject_for("sum the refund amounts", model="gpt-5.5", task_type="finance")
+    out("  errlore injected into the prompt:", Y)
+    for line in inj.text.strip().splitlines():
+        print(f"{D}  | {line}{X}")
+        time.sleep(0.18)
+    time.sleep(0.5)
+    out("  -> 148.90   correct", G)
+    mem.report_outcome(inj, success=True)
+    out("  outcome reported: lesson reinforced [conf 0.80 -> 0.90]", G)
+    time.sleep(1.0)
+
+    print()
+    print(f"{B}pip install errlore{X}   {D}errlore.com — MIT, offline, no telemetry{X}")
+    time.sleep(2.2)
+
+
+if __name__ == "__main__":
+    main()

{errlore-0.1.1 → errlore-0.1.2}/site/index.html

@@ -5,6 +5,11 @@
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <title>errlore — memory for AI agents that learns from failures</title>
 <meta name="description" content="Your agent keeps making the same mistakes. errlore turns failures into lessons, injects them into prompts, and tracks which model to trust. Embedded, file-based, offline.">
+<meta property="og:title" content="errlore — memory for AI agents that learns from failures">
+<meta property="og:description" content="Every resolved error becomes a lesson, injected into future prompts. 100% elimination of repeated convention errors in a paired A/B. Offline, file-based, MIT.">
+<meta property="og:image" content="https://errlore.com/og.png">
+<meta property="og:url" content="https://errlore.com">
+<meta name="twitter:card" content="summary_large_image">
 <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%2312100d'/%3E%3Ctext x='16' y='23' font-family='monospace' font-size='20' fill='%23d9a441' text-anchor='middle'%3Ee%3C/text%3E%3C/svg%3E">
 <style>
   :root{
@@ -89,17 +94,18 @@
   <div class="links">
     <a href="https://github.com/Ma4etaSS/errlore">GitHub</a>
     <a href="https://pypi.org/project/errlore/">PyPI</a>
-    <a href="#quickstart">Docs</a>
+    <a href="#quickstart">Quickstart</a>
   </div>
 </nav>
 
 <header>
   <h1>Your agent keeps making <em>the same mistakes.</em><br>errlore makes that stop.</h1>
-  <p class="sub">A memory layer for AI agents that learns from failures: every resolved
+  <p class="sub" style="margin-top:14px;font-style:italic;color:var(--gold)">Stop the second mistake, not just the first.</p>
+  <p class="sub" style="margin-top:14px">A memory layer for AI agents that learns from failures: every resolved
   error becomes a lesson, lessons are injected into future prompts, and Bayesian trust
   weights track which model to believe for which job.</p>
   <div style="display:flex;gap:14px;flex-wrap:wrap;align-items:center;margin-top:36px">
-    <div class="pip" style="margin-top:0"><span class="dollar">$</span> pip install errlore</div>
+    <div class="pip" style="margin-top:0;cursor:pointer" onclick="navigator.clipboard.writeText('pip install errlore');this.querySelector('.cp').textContent='copied!'" title="Click to copy"><span class="dollar">$</span> pip install errlore <span class="cp" style="color:var(--gold-dim);font-size:12px;margin-left:8px">copy</span></div>
     <a href="https://github.com/Ma4etaSS/errlore" style="display:inline-block;border:1px solid var(--gold-dim);border-radius:8px;padding:14px 22px;font-family:var(--mono);font-size:15px;color:var(--gold)">View on GitHub &rarr;</a>
   </div>
   <div class="cta-row">MIT &middot; Python 3.10+ &middot; 186 tests &middot; CI passing &middot; offline &middot; no telemetry &middot; JSONL, file-locked</div>
@@ -141,6 +147,11 @@
   </div>
 </section>
 
+<section>
+  <h2>Fifteen seconds of the whole idea</h2>
+  <img src="/demo.gif" alt="errlore demo: a failure becomes a lesson; the next similar task gets it injected and succeeds" style="width:100%;border:1px solid var(--line);border-radius:10px" loading="lazy">
+</section>
+
 <section id="quickstart">
   <h2>Sixty seconds to a learning agent</h2>
   <div class="term">
@@ -181,16 +192,22 @@ mem.report_outcome(inj, success=<span class="c-kw">True</span>)</pre>
     <tr><td class="mono">recall@5</td><td class="zero">0.000</td><td><b>0.675</b></td></tr>
     <tr><td class="mono">MRR</td><td class="zero">0.000</td><td><b>0.488</b></td></tr>
   </table>
-  <p class="note">Reproduce it yourself: <span class="mono">python benchmarks/bench_retrieval.py</span> — the gold set ships in the repo.</p>
+  <p class="note">The gold set is deliberately adversarial — every query is paraphrased to
+  share zero words with its lesson. On natural queries with shared vocabulary, plain
+  word-overlap does fine; embeddings earn their keep on paraphrased ones.
+  Reproduce it yourself: <span class="mono">python benchmarks/bench_retrieval.py</span> — the gold set ships in the repo.</p>
 </section>
 
 <section>
   <h2>Drops into what you already run</h2>
-  <div class="grid2">
-    <div class="card"><h3>Open WebUI</h3><p>Two functions — a Filter that injects lessons
+  <div class="loops">
+    <div class="loop"><h3>Claude Code</h3><p>Hooks wire errlore into your coding agent:
+    failed commands become lessons, and each session starts with a briefing of past
+    pitfalls. See <span class="mono">examples/claude-code/</span>.</p></div>
+    <div class="loop"><h3>Open WebUI</h3><p>Two functions — a Filter that injects lessons
     into every chat and a feedback Action that closes the loop from a button under each
     message. Paste, enable, done.</p></div>
-    <div class="card"><h3>Python SDKs</h3><p>Working examples for the OpenAI SDK,
+    <div class="loop"><h3>Python SDKs</h3><p>Working examples for the OpenAI SDK,
     Anthropic SDK, and LangChain in <span class="mono">examples/</span> — each runnable
     offline, each showing the full learn-inject-reinforce cycle.</p></div>
   </div>
@@ -207,6 +224,8 @@ mem.report_outcome(inj, success=<span class="c-kw">True</span>)</pre>
     <tr><td class="mono">tracks per-model weaknesses</td><td class="zero">no</td><td class="zero">no</td><td><b>KNOWN ISSUES + trust</b></td></tr>
     <tr><td class="mono">works offline, no server</td><td>yes</td><td>depends</td><td><b>yes</b></td></tr>
   </table>
+  <p class="note">Not a vector DB, not chat-history RAG, not observability. One job:
+  failures become lessons that reach the next prompt.</p>
 </section>
 
 <section>
@@ -228,6 +247,11 @@ mem.report_outcome(inj, success=<span class="c-kw">True</span>)</pre>
   </div>
 </section>
 
+<section style="padding-top:0">
+  <p class="note" style="text-align:center">Async API, log compaction and multi-agent memory are next —
+  <a href="https://github.com/Ma4etaSS/errlore">watch releases on GitHub</a> to follow.</p>
+</section>
+
 <footer>
   <div>errlore · MIT · <a href="https://github.com/Ma4etaSS/errlore">github.com/Ma4etaSS/errlore</a></div>
   <div><a href="https://pypi.org/project/errlore/">pypi.org/project/errlore</a></div>

errlore-0.1.2/site/llms.txt

@@ -0,0 +1,31 @@
+# errlore
+
+> Memory for AI agents that learns from failures. Every resolved error
+> becomes a lesson, injected into future prompts; per-model KNOWN ISSUES
+> warnings; Bayesian per-model trust. Embedded, file-based, offline,
+> no telemetry. MIT. Python 3.10+.
+
+Positioning: a failure-memory / post-error learning layer — not a generic
+"memory for AI" store. Unlike mem0/Zep/Letta (which store user facts and
+session context), errlore stores the agent's own resolved failures and
+closes the loop: it tracks whether an injected lesson actually helped
+(reinforce/decay) and which model to trust per task domain.
+
+Proof: paired A/B (deterministic validators, raw outputs committed):
+repeated workspace-convention errors 46/48 -> 0/48 (100% reduction,
+exact McNemar p=1.8e-09). Capability-gap errors (e.g. letter counting)
+are honestly NOT improved. See the benchmark in the repository.
+
+## Links
+- [Website](https://errlore.com)
+- [Quickstart](https://errlore.com/#quickstart)
+- [GitHub](https://github.com/Ma4etaSS/errlore)
+- [PyPI](https://pypi.org/project/errlore/)
+- [Changelog](https://github.com/Ma4etaSS/errlore/blob/main/CHANGELOG.md)
+- [Error-reduction benchmark](https://github.com/Ma4etaSS/errlore/tree/main/benchmarks)
+- [Open WebUI integration](https://github.com/Ma4etaSS/errlore/tree/main/integrations/openwebui)
+- [Examples: OpenAI / Anthropic / LangChain / Claude Code](https://github.com/Ma4etaSS/errlore/tree/main/examples)
+
+## Install
+pip install errlore            # core, zero heavy deps
+pip install errlore[embeddings] # + local semantic lesson retrieval

{errlore-0.1.1 → errlore-0.1.2}/src/errlore/__init__.py

@@ -1,6 +1,6 @@
 """errlore -- memory for AI agents that learns from failures."""
 
-__version__ = "0.1.0"
+__version__ = "0.1.2"
 
 from errlore.facade import AgentMemory, Injection
 from errlore.lessons.store import LessonStore