agent-self-edit 0.1.0__tar.gz

agent_self_edit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Debashish Ghosal
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agent_self_edit-0.1.0/PKG-INFO

@@ -0,0 +1,166 @@
+Metadata-Version: 2.4
+Name: agent-self-edit
+Version: 0.1.0
+Summary: An agent that rewrites its own system prompt from execution feedback
+Author: Debashish Ghosal
+License: MIT
+Project-URL: Homepage, https://github.com/deghosal-2026/agent-self-edit
+Project-URL: Repository, https://github.com/deghosal-2026/agent-self-edit.git
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.1
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: scipy>=1.10
+Requires-Dist: tomli>=2.0; python_version < "3.11"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-cov>=4.1; extra == "dev"
+Requires-Dist: ruff>=0.1; extra == "dev"
+Requires-Dist: mypy>=1.7; extra == "dev"
+Dynamic: license-file
+
+# AgentSelfEdit
+
+**An agent that rewrites its own system prompt from execution feedback — proposing edits, A/B testing them against a held-out task set, and promoting only statistically-proven winners under deterministic guardrails.**
+
+The agentic ecosystem raced ahead on orchestration and retrieval, but skipped the capability that would make agents feel alive: the ability to get measurably better at their own job over time. AgentSelfEdit turns prompt optimization into a self-sustaining, evidence-driven loop with provenance, rollback, and guardrails.
+
+## Why
+
+Most production agents are prompt-tuned once by hand — usually by the human who wrote them, usually once, then never again. The prompt freezes the moment it ships. Every recurring failure is silently absorbed until a human manually tunes again. And the agent that sees the most failure data is the least able to use it.
+
+Two common answers are not enough:
+- **"Reflection" is not learning.** Appending a paragraph of prose to context makes prompts longer, not better. The prompt itself never changes, so the same failure repeats tomorrow.
+- **Sharing the raw prompt optimizer** with an LLM breaks everything. Unmanaged, LLM-judged edits poison the baseline within a few iterations.
+
+AgentSelfEdit turns prompt optimization into a **self-sustaining, evidence-driven loop** with provenance, rollback, and guardrails — and it is designed as a **sidecar**. It does not modify the agent's runtime. It observes execution traces and proposes prompt edits.
+
+## Quick Start
+
+```bash
+# Install
+pip install agent-self-edit
+
+# Scaffold a project
+agent-self-edit init
+
+# Run the self-improvement loop
+agent-self-edit run --once
+
+# Or in Docker (requires local OMLX or OpenRouter key)
+docker build -t agent-self-edit .
+docker run --rm --network=host agent-self-edit run --once
+```
+
+## How It Works
+
+```
+Agent executes task ──▶ Execution trace stored (SQLite)
+                                │
+                                ▼
+                      Feedback Analyzer (LLM)
+                      reviews traces, proposes concrete edits,
+                      each with a written hypothesis
+                                │
+                                ▼
+─────────────────────  A/B Test Engine  ─────────────────────
+  candidate edit vs current prompt on a held-out task set:
+  win rate, bootstrap confidence interval, effect size,
+  permutation p-value, per-task breakdown
+────────────────────────────────────────────────────────────
+                                │
+                                ▼
+                Promotion Gate (deterministic checks)
+                1. Sample floor     4. Frozen sections
+                2. Effect size      5. Edit-distance limit
+                3. Confidence p-val 6. Drift detection
+                                │
+              ┌─────────────────┼─────────────────┐
+              ▼                 ▼                 ▼
+          Promoted         Near-miss        Rejected
+     prompt updated in     logged for       archived with
+     versioned Registry    human review     full reasoning
+```
+
+1. **Analyze** — After each task, an LLM reviews the execution trace and identifies what went wrong: the specific instruction that was missing, ambiguous, or wrong.
+2. **Propose** — It proposes one or more concrete, minimal prompt edits, each with a stated hypothesis about why it should improve outcomes.
+3. **Test** — Each candidate edit is A/B tested against the current prompt on a held-out task set, with confidence intervals, effect-size thresholds, and a minimum sample floor before any promotion decision.
+4. **Promote or Archive** — The statistically-proven winner becomes the new baseline. The loser is archived with its full reasoning.
+5. **Guard** — Frozen core sections, edit-distance limits, and drift detection keep the agent from rewriting itself into a worse version over hundreds of iterations.
+
+## Core Components
+
+| Component | What it does |
+|---|---|
+| **Feedback Analyzer** | An LLM that reviews execution traces and produces structured failure diagnoses plus concrete prompt-edit proposals, each with a written hypothesis. **It has no authority** — it only proposes; every proposal goes through A/B test + gate. |
+| **A/B Test Engine** | Compares a candidate prompt against the current one on a held-out task set. Reports win rate, bootstrap confidence intervals, effect size, permutation p-value, and per-task breakdown. |
+| **Promotion Gate** | The safety-critical component. Six deterministic checks in fail-fast order: sample floor, effect size, confidence interval, frozen sections, edit-distance, drift. Outcomes: **promote**, **reject**, or **near-miss** (logged for human review). The gate is **code, not prompts** — verifiable, testable, non-negotiable. |
+| **Prompt Registry** | File-based versioned store of every prompt with full lineage: diff from previous version, the analyzer's hypothesis, A/B test results, guardrail results, timestamp, model version, trigger trace IDs. Supports `diff`, `rollback`, `lineage`. SHA-256 integrity per version. |
+| **Guardrail Module** | Frozen section annotations, edit-distance calculation, and TF-IDF drift (embedding drift in v0.2.0). |
+| **Diff Visualization** | Side-by-side or inline diff showing exactly what changed between prompt versions, what stayed the same (frozen core), and why (guardrail evidence). |
+| **CLI** | `agent-self-edit` with `init`, `run`, `status`, `diff`, `rollback`, `guardrails`, `lineage`, `propose`, `ingest`, `validate`. Shortest path from `pip install` to a working loop is under 5 minutes. |
+
+## Guardrails
+
+The promotion gate is deterministic — never LLM-judged — and runs six checks before any edit is promoted:
+
+1. **Sample floor** — minimum number of A/B trials completed
+2. **Effect size** — improvement exceeds a configurable minimum threshold
+3. **Confidence interval** — p-value below the confidence threshold
+4. **Frozen core sections** — user-annotated sections the analyzer cannot modify
+5. **Edit-distance limit** — maximum lines changed per cycle (configurable)
+6. **Drift detection** — semantic similarity to the original prompt, alerts on divergence
+
+Design target: < 1% of good edits rejected (false positives), < 0.1% of bad edits promoted (false negatives), 100% rollback success.
+
+## Trigger Modes
+
+- **Batch** — analyze after N tasks (default: 50)
+- **Time-based** — analyze every N hours
+- **Manual** — analyze on demand
+
+## Where It Helps
+
+Any agent that repeats a similar task type and sees execution feedback beats its prompt frozen on day one:
+
+- **Customer support ticket classification** — a change that helps one category can't degrade another.
+- **Code review / PR analysis** — false positives on docs-only PRs are learned and fixed.
+- **Data extraction / entity parsing** — PDF-with-tables extraction doesn't regress plain-text extraction.
+- **Content moderation** — false-positive / false-negative tradeoffs are measured, not guessed.
+- **Sales outreach personalization** — winning variants are proven, not copy-pasted.
+- **Documentation generation** — output length and tone adapt from feedback.
+
+## Roadmap
+
+| Version | Focus |
+|---|---|
+| **v0.1.0** | Prove the loop — core loop, statistical gate, CLI, guardrails, Docker, field test (in progress — M10) |
+| **v0.2.0** | Trust + visibility — web dashboard, drift detection, near-miss feedback, REST API, shadow mode |
+| **v0.3.0** | Scale + adapters — framework adapters, multi-failure clustering, adaptive sample floors, evals integration |
+| **v0.4.0** | Fleet — fleet-wide shared-rules learning, cost-aware improvement, promotion analytics |
+| **v1.0.0** | General availability — stable API, production deployment guide |
+
+## Status
+
+🚧 **Pre-release v0.1.0.** Core loop complete: trace ingestion, feedback analyzer, A/B test engine, promotion gate, prompt registry, CLI (10 commands), Docker support, diff visualization, rollback, guardrails. **434 hermetic tests + 9 Docker tests pass.** Docker full-loop integration test runs against OMLX real LLM (9/9 tests pass in 112s). Field test in progress (M10): 3-model comparison (4B/9B/cloud) completed, 10-iteration improvement run pending.
+
+## Field Test Status
+
+| Component | Status |
+|-----------|--------|
+| F-01 Trace ingestion | ✅ works |
+| F-02 Feedback analyzer | ✅ works (proposes edits against OMLX) |
+| F-03 A/B test engine | ✅ works (2 distinct prompts, real statistics) |
+| F-04 Promotion gate | ✅ works (6 checks, deterministic) |
+| F-05 Prompt registry | ✅ works (versioned, lineage, rollback) |
+| F-09 CLI | ✅ works (10 commands) |
+| F-14 Docker support | ✅ works (9/9 tests, full loop against OMLX) |
+| M10 Field test | ⬜ 10-iteration improvement run pending (#100) |
+
+See [mini field test report](docs/field-test/v0.1.0/mini-field-test-report.md) for 3-model (4B, 9B, cloud) comparison results.
+
+## License
+
+MIT — see [LICENSE](LICENSE)

agent_self_edit-0.1.0/README.md

@@ -0,0 +1,143 @@
+# AgentSelfEdit
+
+**An agent that rewrites its own system prompt from execution feedback — proposing edits, A/B testing them against a held-out task set, and promoting only statistically-proven winners under deterministic guardrails.**
+
+The agentic ecosystem raced ahead on orchestration and retrieval, but skipped the capability that would make agents feel alive: the ability to get measurably better at their own job over time. AgentSelfEdit turns prompt optimization into a self-sustaining, evidence-driven loop with provenance, rollback, and guardrails.
+
+## Why
+
+Most production agents are prompt-tuned once by hand — usually by the human who wrote them, usually once, then never again. The prompt freezes the moment it ships. Every recurring failure is silently absorbed until a human manually tunes again. And the agent that sees the most failure data is the least able to use it.
+
+Two common answers are not enough:
+- **"Reflection" is not learning.** Appending a paragraph of prose to context makes prompts longer, not better. The prompt itself never changes, so the same failure repeats tomorrow.
+- **Sharing the raw prompt optimizer** with an LLM breaks everything. Unmanaged, LLM-judged edits poison the baseline within a few iterations.
+
+AgentSelfEdit turns prompt optimization into a **self-sustaining, evidence-driven loop** with provenance, rollback, and guardrails — and it is designed as a **sidecar**. It does not modify the agent's runtime. It observes execution traces and proposes prompt edits.
+
+## Quick Start
+
+```bash
+# Install
+pip install agent-self-edit
+
+# Scaffold a project
+agent-self-edit init
+
+# Run the self-improvement loop
+agent-self-edit run --once
+
+# Or in Docker (requires local OMLX or OpenRouter key)
+docker build -t agent-self-edit .
+docker run --rm --network=host agent-self-edit run --once
+```
+
+## How It Works
+
+```
+Agent executes task ──▶ Execution trace stored (SQLite)
+                                │
+                                ▼
+                      Feedback Analyzer (LLM)
+                      reviews traces, proposes concrete edits,
+                      each with a written hypothesis
+                                │
+                                ▼
+─────────────────────  A/B Test Engine  ─────────────────────
+  candidate edit vs current prompt on a held-out task set:
+  win rate, bootstrap confidence interval, effect size,
+  permutation p-value, per-task breakdown
+────────────────────────────────────────────────────────────
+                                │
+                                ▼
+                Promotion Gate (deterministic checks)
+                1. Sample floor     4. Frozen sections
+                2. Effect size      5. Edit-distance limit
+                3. Confidence p-val 6. Drift detection
+                                │
+              ┌─────────────────┼─────────────────┐
+              ▼                 ▼                 ▼
+          Promoted         Near-miss        Rejected
+     prompt updated in     logged for       archived with
+     versioned Registry    human review     full reasoning
+```
+
+1. **Analyze** — After each task, an LLM reviews the execution trace and identifies what went wrong: the specific instruction that was missing, ambiguous, or wrong.
+2. **Propose** — It proposes one or more concrete, minimal prompt edits, each with a stated hypothesis about why it should improve outcomes.
+3. **Test** — Each candidate edit is A/B tested against the current prompt on a held-out task set, with confidence intervals, effect-size thresholds, and a minimum sample floor before any promotion decision.
+4. **Promote or Archive** — The statistically-proven winner becomes the new baseline. The loser is archived with its full reasoning.
+5. **Guard** — Frozen core sections, edit-distance limits, and drift detection keep the agent from rewriting itself into a worse version over hundreds of iterations.
+
+## Core Components
+
+| Component | What it does |
+|---|---|
+| **Feedback Analyzer** | An LLM that reviews execution traces and produces structured failure diagnoses plus concrete prompt-edit proposals, each with a written hypothesis. **It has no authority** — it only proposes; every proposal goes through A/B test + gate. |
+| **A/B Test Engine** | Compares a candidate prompt against the current one on a held-out task set. Reports win rate, bootstrap confidence intervals, effect size, permutation p-value, and per-task breakdown. |
+| **Promotion Gate** | The safety-critical component. Six deterministic checks in fail-fast order: sample floor, effect size, confidence interval, frozen sections, edit-distance, drift. Outcomes: **promote**, **reject**, or **near-miss** (logged for human review). The gate is **code, not prompts** — verifiable, testable, non-negotiable. |
+| **Prompt Registry** | File-based versioned store of every prompt with full lineage: diff from previous version, the analyzer's hypothesis, A/B test results, guardrail results, timestamp, model version, trigger trace IDs. Supports `diff`, `rollback`, `lineage`. SHA-256 integrity per version. |
+| **Guardrail Module** | Frozen section annotations, edit-distance calculation, and TF-IDF drift (embedding drift in v0.2.0). |
+| **Diff Visualization** | Side-by-side or inline diff showing exactly what changed between prompt versions, what stayed the same (frozen core), and why (guardrail evidence). |
+| **CLI** | `agent-self-edit` with `init`, `run`, `status`, `diff`, `rollback`, `guardrails`, `lineage`, `propose`, `ingest`, `validate`. Shortest path from `pip install` to a working loop is under 5 minutes. |
+
+## Guardrails
+
+The promotion gate is deterministic — never LLM-judged — and runs six checks before any edit is promoted:
+
+1. **Sample floor** — minimum number of A/B trials completed
+2. **Effect size** — improvement exceeds a configurable minimum threshold
+3. **Confidence interval** — p-value below the confidence threshold
+4. **Frozen core sections** — user-annotated sections the analyzer cannot modify
+5. **Edit-distance limit** — maximum lines changed per cycle (configurable)
+6. **Drift detection** — semantic similarity to the original prompt, alerts on divergence
+
+Design target: < 1% of good edits rejected (false positives), < 0.1% of bad edits promoted (false negatives), 100% rollback success.
+
+## Trigger Modes
+
+- **Batch** — analyze after N tasks (default: 50)
+- **Time-based** — analyze every N hours
+- **Manual** — analyze on demand
+
+## Where It Helps
+
+Any agent that repeats a similar task type and sees execution feedback beats its prompt frozen on day one:
+
+- **Customer support ticket classification** — a change that helps one category can't degrade another.
+- **Code review / PR analysis** — false positives on docs-only PRs are learned and fixed.
+- **Data extraction / entity parsing** — PDF-with-tables extraction doesn't regress plain-text extraction.
+- **Content moderation** — false-positive / false-negative tradeoffs are measured, not guessed.
+- **Sales outreach personalization** — winning variants are proven, not copy-pasted.
+- **Documentation generation** — output length and tone adapt from feedback.
+
+## Roadmap
+
+| Version | Focus |
+|---|---|
+| **v0.1.0** | Prove the loop — core loop, statistical gate, CLI, guardrails, Docker, field test (in progress — M10) |
+| **v0.2.0** | Trust + visibility — web dashboard, drift detection, near-miss feedback, REST API, shadow mode |
+| **v0.3.0** | Scale + adapters — framework adapters, multi-failure clustering, adaptive sample floors, evals integration |
+| **v0.4.0** | Fleet — fleet-wide shared-rules learning, cost-aware improvement, promotion analytics |
+| **v1.0.0** | General availability — stable API, production deployment guide |
+
+## Status
+
+🚧 **Pre-release v0.1.0.** Core loop complete: trace ingestion, feedback analyzer, A/B test engine, promotion gate, prompt registry, CLI (10 commands), Docker support, diff visualization, rollback, guardrails. **434 hermetic tests + 9 Docker tests pass.** Docker full-loop integration test runs against OMLX real LLM (9/9 tests pass in 112s). Field test in progress (M10): 3-model comparison (4B/9B/cloud) completed, 10-iteration improvement run pending.
+
+## Field Test Status
+
+| Component | Status |
+|-----------|--------|
+| F-01 Trace ingestion | ✅ works |
+| F-02 Feedback analyzer | ✅ works (proposes edits against OMLX) |
+| F-03 A/B test engine | ✅ works (2 distinct prompts, real statistics) |
+| F-04 Promotion gate | ✅ works (6 checks, deterministic) |
+| F-05 Prompt registry | ✅ works (versioned, lineage, rollback) |
+| F-09 CLI | ✅ works (10 commands) |
+| F-14 Docker support | ✅ works (9/9 tests, full loop against OMLX) |
+| M10 Field test | ⬜ 10-iteration improvement run pending (#100) |
+
+See [mini field test report](docs/field-test/v0.1.0/mini-field-test-report.md) for 3-model (4B, 9B, cloud) comparison results.
+
+## License
+
+MIT — see [LICENSE](LICENSE)

agent_self_edit-0.1.0/pyproject.toml

@@ -0,0 +1,62 @@
+[build-system]
+requires = ["setuptools>=64"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "agent-self-edit"
+version = "0.1.0"
+description = "An agent that rewrites its own system prompt from execution feedback"
+readme = "README.md"
+license = {text = "MIT"}
+authors = [{name = "Debashish Ghosal"}]
+requires-python = ">=3.10"
+dependencies = [
+    "click>=8.1",
+    "pyyaml>=6.0",
+    "numpy>=1.24",
+    "scipy>=1.10",
+    "tomli>=2.0; python_version < '3.11'",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.4",
+    "pytest-cov>=4.1",
+    "ruff>=0.1",
+    "mypy>=1.7",
+]
+
+[project.urls]
+Homepage = "https://github.com/deghosal-2026/agent-self-edit"
+Repository = "https://github.com/deghosal-2026/agent-self-edit.git"
+
+[project.scripts]
+agent-self-edit = "agent_self_edit.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint.per-file-ignores]
+"field-test/**" = ["E501"]
+"tests/**" = ["E501"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "N", "W"]
+
+[tool.mypy]
+strict = true
+python_version = "3.10"
+ignore_missing_imports = true
+disallow_untyped_defs = true
+disallow_any_unimported = false
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "--cov=agent_self_edit --cov-report=term-missing"
+markers = [
+    "docker: marks tests that require a Docker daemon and optionally OMLX",
+]

agent_self_edit-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

agent_self_edit-0.1.0/src/agent_self_edit/__init__.py

@@ -0,0 +1 @@
+"""AgentSelfEdit — An agent that rewrites its own system prompt from execution feedback."""