driftfix 0.2.0__tar.gz → 0.2.1__tar.gz

{driftfix-0.2.0 → driftfix-0.2.1}/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 0.2.1
+- Test commands no longer see `ANTHROPIC_API_KEY` or GitHub tokens
+- Report the real cost when the agent stops on its budget (was $0.00)
+- The pinning-back guard now allows widening the requirement to admit the new version (e.g. dropping `<2.0.0`)
+
 ## 0.2.0
 - 5 more benchmark cases: pandas 2, Django 5, Flask 3, Pillow 10, PyYAML 6
 - `driftfix --version`

{driftfix-0.2.0 → driftfix-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: driftfix
-Version: 0.2.0
+Version: 0.2.1
 Summary: Dependabot bumps it. CI breaks. driftfix fixes the code.
 Project-URL: Homepage, https://github.com/Gthejesraj/driftfix
 Project-URL: Issues, https://github.com/Gthejesraj/driftfix/issues
@@ -22,6 +22,7 @@ Classifier: Topic :: Software Development :: Build Tools
 Classifier: Topic :: Software Development :: Quality Assurance
 Requires-Python: >=3.10
 Requires-Dist: claude-agent-sdk>=0.2
+Requires-Dist: packaging>=23
 Provides-Extra: dev
 Requires-Dist: pytest>=8; extra == 'dev'
 Requires-Dist: ruff>=0.6; extra == 'dev'
@@ -84,6 +85,23 @@ for free with `python bench/run.py --check`.
 **Found an upgrade it can't fix? That's the most useful contribution:** see
 [CONTRIBUTING.md](CONTRIBUTING.md).
 
+### Real projects
+
+The synthetic cases above are small. On real open-source projects broken by
+pydantic 1 → 2 ([study](https://github.com/Gthejesraj/driftfix-study), 39
+candidates → 4 real breaks):
+
+| Project | Tests broken | Result | Cost |
+|---|---|---|---|
+| internetarchive/fatcat-scholar | 15 of 115 | ✅ fixed, 11 files | $2.38 |
+| antonagestam/phantom-types | 2 of 677 | ✅ fixed, one test assertion edit to review | $2.93 |
+| epi2me-labs/ezcharts | 5 of 37 | ✅ fixed, including the model generator | ~$1 |
+| bdd100k/bdd100k | 2 of 46 | ⚠️ tests pass, but via a workaround for a third-party library | $0.95 |
+
+Real fixes cost $1–3, not $0.20, so set `budget` to at least 3 for real
+codebases. Always read the diff: "tests pass" and "correct" aren't the same
+thing. Details in the study's [RESULTS.md](https://github.com/Gthejesraj/driftfix-study/blob/main/RESULTS.md).
+
 ## Quick start: GitHub Action
 
 1. Add `.github/workflows/driftfix.yml`:
@@ -140,9 +158,10 @@ Exit codes: `0` fixed or nothing to fix, `1` not fixed, `2` dirty working tree.
 
 - **Your tests decide.** driftfix reruns them after the agent finishes. The
   agent saying it's done doesn't count.
-- **No pinning back.** Changing the upgraded package's requirement or editing
-  lock files is rejected. Adding a package the upgrade split out (like
-  `pydantic-settings`) is allowed.
+- **No pinning back.** A requirement that excludes the new version, or any
+  lock file edit, is rejected. Widening a requirement so the new version is
+  allowed (dropping an old `<2` cap) is fine, and so is adding a package the
+  upgrade split out (like `pydantic-settings`).
 - **No deleting or skipping tests.** The agent is told not to, and every test
   change shows up in the diff you review.
 - **Reviewable.** It needs a clean git tree, so every change is a diff, and it
@@ -172,6 +191,13 @@ Native Renovate support is on the roadmap.
 workflows from commits pushed with the default `GITHUB_TOKEN`. Pass a personal
 access token or GitHub App token to `actions/checkout` if you want that.
 
+**Can it fix deprecations before they break?** Yes. Make the test command
+treat them as errors, e.g. `pytest -q -W error::DeprecationWarning` (some
+libraries use their own warning class, like
+`-W error::starlette.exceptions.StarletteDeprecationWarning`). On a real
+31-file FastAPI app, this caught Starlette's `httpx` → `httpx2` deprecation and
+fixed it with a one-line dependency swap ($0.40).
+
 **Mocked tests?** driftfix can only catch what your tests catch. A test that
 mocks the library can pass while the real call is broken.
 

{driftfix-0.2.0 → driftfix-0.2.1}/README.md

@@ -55,6 +55,23 @@ for free with `python bench/run.py --check`.
 **Found an upgrade it can't fix? That's the most useful contribution:** see
 [CONTRIBUTING.md](CONTRIBUTING.md).
 
+### Real projects
+
+The synthetic cases above are small. On real open-source projects broken by
+pydantic 1 → 2 ([study](https://github.com/Gthejesraj/driftfix-study), 39
+candidates → 4 real breaks):
+
+| Project | Tests broken | Result | Cost |
+|---|---|---|---|
+| internetarchive/fatcat-scholar | 15 of 115 | ✅ fixed, 11 files | $2.38 |
+| antonagestam/phantom-types | 2 of 677 | ✅ fixed, one test assertion edit to review | $2.93 |
+| epi2me-labs/ezcharts | 5 of 37 | ✅ fixed, including the model generator | ~$1 |
+| bdd100k/bdd100k | 2 of 46 | ⚠️ tests pass, but via a workaround for a third-party library | $0.95 |
+
+Real fixes cost $1–3, not $0.20, so set `budget` to at least 3 for real
+codebases. Always read the diff: "tests pass" and "correct" aren't the same
+thing. Details in the study's [RESULTS.md](https://github.com/Gthejesraj/driftfix-study/blob/main/RESULTS.md).
+
 ## Quick start: GitHub Action
 
 1. Add `.github/workflows/driftfix.yml`:
@@ -111,9 +128,10 @@ Exit codes: `0` fixed or nothing to fix, `1` not fixed, `2` dirty working tree.
 
 - **Your tests decide.** driftfix reruns them after the agent finishes. The
   agent saying it's done doesn't count.
-- **No pinning back.** Changing the upgraded package's requirement or editing
-  lock files is rejected. Adding a package the upgrade split out (like
-  `pydantic-settings`) is allowed.
+- **No pinning back.** A requirement that excludes the new version, or any
+  lock file edit, is rejected. Widening a requirement so the new version is
+  allowed (dropping an old `<2` cap) is fine, and so is adding a package the
+  upgrade split out (like `pydantic-settings`).
 - **No deleting or skipping tests.** The agent is told not to, and every test
   change shows up in the diff you review.
 - **Reviewable.** It needs a clean git tree, so every change is a diff, and it
@@ -143,6 +161,13 @@ Native Renovate support is on the roadmap.
 workflows from commits pushed with the default `GITHUB_TOKEN`. Pass a personal
 access token or GitHub App token to `actions/checkout` if you want that.
 
+**Can it fix deprecations before they break?** Yes. Make the test command
+treat them as errors, e.g. `pytest -q -W error::DeprecationWarning` (some
+libraries use their own warning class, like
+`-W error::starlette.exceptions.StarletteDeprecationWarning`). On a real
+31-file FastAPI app, this caught Starlette's `httpx` → `httpx2` deprecation and
+fixed it with a one-line dependency swap ($0.40).
+
 **Mocked tests?** driftfix can only catch what your tests catch. A test that
 mocks the library can pass while the real call is broken.
 

{driftfix-0.2.0 → driftfix-0.2.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "driftfix"
-version = "0.2.0"
+version = "0.2.1"
 description = "Dependabot bumps it. CI breaks. driftfix fixes the code."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -24,7 +24,7 @@ classifiers = [
   "Topic :: Software Development :: Build Tools",
   "Topic :: Software Development :: Quality Assurance",
 ]
-dependencies = ["claude-agent-sdk>=0.2"]
+dependencies = ["claude-agent-sdk>=0.2", "packaging>=23"]
 
 [project.optional-dependencies]
 dev = ["pytest>=8", "ruff>=0.6"]

{driftfix-0.2.0 → driftfix-0.2.1}/src/driftfix/cli.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import argparse
 import asyncio
+import os
 import re
 import subprocess
 import sys
@@ -11,8 +12,13 @@ from dataclasses import dataclass
 from importlib.metadata import version
 from pathlib import Path
 
+from packaging.requirements import InvalidRequirement, Requirement
+from packaging.version import InvalidVersion, Version
+
 # Pinning back to the old version is not a fix. Adding a split-out package is fine.
 MANIFESTS = {"requirements.txt", "pyproject.toml", "setup.py", "setup.cfg", "Pipfile", "package.json"}
+# The test suite runs the upgraded package's code; it shouldn't see our credentials.
+SECRETS = {"ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "CLAUDE_CODE_OAUTH_TOKEN", "GITHUB_TOKEN", "GH_TOKEN"}
 LOCKFILES = {"Pipfile.lock", "poetry.lock", "uv.lock", "package-lock.json", "yarn.lock", "pnpm-lock.yaml"}
 
 PROMPT = """\
@@ -24,7 +30,8 @@ $ {test_command}
 Fix this repository's code so it works with the new version of `{package}`.
 - Read the package's changelog / migration guide for the relevant versions if you need to.
 - Change application code. Change tests only where they use `{package}`'s API directly.
-- Never pin, downgrade, or change the requirement for `{package}`, and never edit lock files.
+- Never pin or downgrade `{package}`, and never edit lock files. You may only widen its
+  requirement so the new version is allowed (e.g. drop an old upper bound).
 - If the new version moved code into a separate package, you may add that package to the
   dependency manifest and install it into the current environment.
 - Never delete or skip tests to make them pass.
@@ -41,7 +48,8 @@ class TestRun:
 def run_tests(command: str, repo: Path, timeout: int) -> TestRun:
     try:
         proc = subprocess.run(
-            command, shell=True, cwd=repo, capture_output=True, text=True, timeout=timeout
+            command, shell=True, cwd=repo, capture_output=True, text=True, timeout=timeout,
+            env={k: v for k, v in os.environ.items() if k not in SECRETS},
         )
     except subprocess.TimeoutExpired:
         return TestRun(False, f"timed out after {timeout}s")
@@ -56,8 +64,17 @@ def changed_files(repo: Path) -> list[str]:
     return [line[3:].split(" -> ")[-1] for line in out.splitlines()]
 
 
-def pins_package(repo: Path, path: str, package: str) -> bool:
-    """Did this edit touch a lock file, or any requirement line naming the upgraded package?"""
+def admits(line: str, new_version: str | None) -> bool:
+    """Is this requirement line one that allows the upgraded version (e.g. a loosened cap)?"""
+    try:
+        req = Requirement(line.strip().strip("\"',"))
+        return new_version is not None and req.specifier.contains(Version(new_version), prereleases=True)
+    except (InvalidRequirement, InvalidVersion):
+        return False
+
+
+def pins_package(repo: Path, path: str, package: str, new_version: str | None = None) -> bool:
+    """Did this edit touch a lock file, or add a requirement for the package that excludes the new version?"""
     if Path(path).name in LOCKFILES:
         return True
     if Path(path).name not in MANIFESTS:
@@ -65,13 +82,13 @@ def pins_package(repo: Path, path: str, package: str) -> bool:
     diff = subprocess.run(
         ["git", "diff", "-U0", "--", path], cwd=repo, capture_output=True, text=True
     ).stdout
-    lines = (
-        [line[1:] for line in diff.splitlines() if line[:1] in "+-" and line[:3] not in ("+++", "---")]
+    added = (
+        [line[1:] for line in diff.splitlines() if line[:1] == "+" and line[:3] != "+++"]
         if diff else (repo / path).read_text().splitlines()  # new, untracked file
     )
     names = "|".join(re.sub(r"[-_.]", "[-_.]", n) for n in re.split(r"[,\s]+", package) if n)
     pattern = re.compile(rf"(?<![\w.-])({names})(?![\w.-])", re.IGNORECASE)
-    return any(pattern.search(line) for line in lines)
+    return any(pattern.search(line) and not admits(line, new_version) for line in added)
 
 
 def build_prompt(package: str, old: str | None, new: str | None, command: str, out: str) -> str:
@@ -95,9 +112,12 @@ async def run_agent(prompt: str, repo: Path, model: str, max_turns: int, budget:
         skills=[],
     )
     summary, cost = "", 0.0
-    async for message in query(prompt=prompt, options=options):
-        if isinstance(message, ResultMessage):
-            summary, cost = message.result or "", message.total_cost_usd or 0.0
+    try:
+        async for message in query(prompt=prompt, options=options):
+            if isinstance(message, ResultMessage):
+                summary, cost = message.result or "", message.total_cost_usd or 0.0
+    except Exception as exc:  # e.g. budget reached; the cost was already reported above
+        return f"Agent stopped: `{exc}`", cost
     return summary, cost
 
 
@@ -120,7 +140,7 @@ def fix(args: argparse.Namespace) -> int:
         summary, cost = f"Agent error: `{exc}`", 0.0
 
     touched = [f for f in changed_files(repo) if f not in baseline]
-    pinned = [f for f in touched if pins_package(repo, f, args.package)]
+    pinned = [f for f in touched if pins_package(repo, f, args.package, args.to_version)]
     after = run_tests(args.test, repo, args.timeout)  # don't trust the agent's word
     ok = after.passed and bool(touched) and not pinned
 

{driftfix-0.2.0 → driftfix-0.2.1}/tests/test_fix.py

@@ -83,3 +83,41 @@ def test_pins_package():
     assert pins_package(repo, "requirements.txt", "pydantic-core")  # name normalization
     assert not pins_package(repo, "requirements.txt", "pydantic")   # pydantic-core is a different package
     assert not pins_package(repo, "app.py", "pydantic")
+
+
+def test_tests_cannot_see_api_key(tmp_path, monkeypatch):
+    monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-secret")
+    leak = f"{sys.executable} -c 'import os, sys; sys.exit(\"ANTHROPIC_API_KEY\" in os.environ)'"
+    assert cli.run_tests(leak, tmp_path, 30).passed
+
+
+def test_cost_survives_agent_error(monkeypatch):
+    import asyncio
+    import sys as _sys
+    import types
+
+    class ResultMessage:
+        result, total_cost_usd = "", 1.02
+
+    async def query(**kw):
+        yield ResultMessage()
+        raise RuntimeError("Reached maximum budget ($1)")
+
+    fake = types.SimpleNamespace(
+        ClaudeAgentOptions=lambda **kw: None, ResultMessage=ResultMessage, query=query
+    )
+    monkeypatch.setitem(_sys.modules, "claude_agent_sdk", fake)
+    summary, cost = asyncio.run(cli.run_agent("p", Path("."), "m", 1, 1.0))
+    assert cost == 1.02 and "maximum budget" in summary
+
+
+def test_loosening_cap_is_allowed_pinning_back_is_not(tmp_path):
+    from driftfix.cli import pins_package
+    repo = make_repo(tmp_path, True)
+    (repo / "requirements.txt").write_text("lib>=2.0.0\n")         # was lib==2, loosened
+    assert not pins_package(repo, "requirements.txt", "lib", "2.13.5")
+    (repo / "requirements.txt").write_text("lib<2\n")              # pinned back
+    assert pins_package(repo, "requirements.txt", "lib", "2.13.5")
+    (repo / "requirements.txt").write_text("lib>=2.0.0\n")         # unknown target version: be strict
+    assert pins_package(repo, "requirements.txt", "lib", None)
+    assert pins_package(repo, "requirements.txt", "lib", "1.x")    # unparsable version: be strict