agent-lineup 0.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- agent_lineup-0.1.0.dist-info/METADATA +107 -0
- agent_lineup-0.1.0.dist-info/RECORD +25 -0
- agent_lineup-0.1.0.dist-info/WHEEL +5 -0
- agent_lineup-0.1.0.dist-info/entry_points.txt +2 -0
- agent_lineup-0.1.0.dist-info/licenses/LICENSE +21 -0
- agent_lineup-0.1.0.dist-info/top_level.txt +1 -0
- lineup/__init__.py +3 -0
- lineup/__main__.py +5 -0
- lineup/banner.py +70 -0
- lineup/changes.py +403 -0
- lineup/cli.py +440 -0
- lineup/functions.py +152 -0
- lineup/git.py +161 -0
- lineup/guard.py +187 -0
- lineup/hookjson.py +58 -0
- lineup/hooks.py +326 -0
- lineup/lineup-banner.txt +8 -0
- lineup/llm.py +158 -0
- lineup/paths.py +57 -0
- lineup/rollback.py +252 -0
- lineup/rules.py +229 -0
- lineup/snapshot.py +170 -0
- lineup/store.py +232 -0
- lineup/summary.py +54 -0
- lineup/table.py +54 -0
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: agent-lineup
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: See what your coding agent just changed, which edits are risky, and undo a file.
|
|
5
|
+
License: MIT
|
|
6
|
+
Requires-Python: >=3.10
|
|
7
|
+
Description-Content-Type: text/markdown
|
|
8
|
+
License-File: LICENSE
|
|
9
|
+
Provides-Extra: dev
|
|
10
|
+
Requires-Dist: pytest; extra == "dev"
|
|
11
|
+
Dynamic: license-file
|
|
12
|
+
|
|
13
|
+
<p align="center"><img src="assets/lineup-logo.svg" alt="lineup logo" width="324"></p>
|
|
14
|
+
|
|
15
|
+
# lineup
|
|
16
|
+
|
|
17
|
+
`lineup` shows what your coding agent (Claude Code or Codex) just changed, flags the risky edits, and lets you undo a file.
|
|
18
|
+
|
|
19
|
+
## Demo
|
|
20
|
+
|
|
21
|
+
After the agent's reply, the hook prints a short line:
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
⚠ ≈ lineup: 2 HIGH
|
|
25
|
+
2 auth.py auth or payment file changed
|
|
26
|
+
3 test_utils.py assertion removed
|
|
27
|
+
Undo: lineup rollback 2
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`lineup risky` shows every changed file and the rule behind its score:
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
| ID | File | Risk | Why (rule) | Other |
|
|
34
|
+
| 1 | .env | HIGH | .env or key file changed, contents not read | Can't roll back |
|
|
35
|
+
| 2 | auth.py | HIGH | auth or payment file changed | |
|
|
36
|
+
| 3 | test_utils.py | HIGH | assertion removed | |
|
|
37
|
+
| 4 | utils.py | LOW | code edited, + function added: mul | |
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
`lineup rollback 2` (or `lineup rollback auth.py`) puts the file back:
|
|
41
|
+
|
|
42
|
+
```
|
|
43
|
+
2 auth.py restored
|
|
44
|
+
Your versions from before this rollback are saved in .git/lineup/backups/2026-11-06_004042/.
|
|
45
|
+
Also changed: .env (ID 1), test_utils.py (ID 3), utils.py (ID 4). Check whether they depend on this file.
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Install
|
|
49
|
+
|
|
50
|
+
Needs Python 3.10 or newer, git, and [pipx](https://pipx.pypa.io).
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
pipx install git+https://github.com/HARSHAVARDHAN-RAJU5/agent-lineup
|
|
54
|
+
cd your-repo
|
|
55
|
+
lineup init --agent claude
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Use `--agent codex` if you use Codex, or leave `--agent` out for both.
|
|
59
|
+
|
|
60
|
+
`lineup init` shows the hooks it will add and asks before writing. Then start a new Claude Code session. In Codex, run `/hooks` and trust the lineup hooks.
|
|
61
|
+
|
|
62
|
+
- No pipx: `python -m pip install --user pipx`, then `python -m pipx ensurepath`, then open a new terminal.
|
|
63
|
+
- No git: the install fails with "Cannot find command 'git'". Install git first.
|
|
64
|
+
- Upgrade: `pipx upgrade agent-lineup`. If it says it's already up to date, run `pipx install --force git+https://github.com/HARSHAVARDHAN-RAJU5/agent-lineup`.
|
|
65
|
+
- Uninstall: run `lineup init --remove` in each repo first, then `pipx uninstall agent-lineup`. To delete lineup's saved data too, delete the `.git/lineup` folder in each repo.
|
|
66
|
+
|
|
67
|
+
## Commands
|
|
68
|
+
|
|
69
|
+
| Command | What it does |
|
|
70
|
+
|---|---|
|
|
71
|
+
| `lineup init` | Adds the hooks for Claude Code (`.claude/settings.local.json`) and Codex (`.codex/hooks.json`). `--agent claude` or `--agent codex` picks one. `--remove` takes them out again. |
|
|
72
|
+
| `lineup diff` | What changed since the session started, with a short AI summary. |
|
|
73
|
+
| `lineup risky` | Scores each changed file HIGH, MED or LOW using fixed rules, and shows the rule. |
|
|
74
|
+
| `lineup rollback 3 4` | Undoes files by ID or name. Your current version is backed up first. |
|
|
75
|
+
| `lineup report` | Usage counts to send as tester feedback. No code or file names. |
|
|
76
|
+
|
|
77
|
+
## What gets sent to the AI
|
|
78
|
+
|
|
79
|
+
Only `lineup diff` uses AI. It sends the changed lines to the agent you already have (Claude Code with Haiku, or Codex), with no tools. Nothing goes anywhere else, and lineup needs no API key.
|
|
80
|
+
|
|
81
|
+
- `.env`, key and credential files are never sent. They're listed by name only.
|
|
82
|
+
- Lockfile contents are left out. Secret-looking text is hidden before sending.
|
|
83
|
+
- The hooks, `risky`, `rollback` and `report` use no AI. They run only on your machine.
|
|
84
|
+
- lineup keeps its photo, file copies and backups in `.git/lineup/` in your repo. Private files are kept as fingerprints only, never copied.
|
|
85
|
+
|
|
86
|
+
## Run it in a second terminal
|
|
87
|
+
|
|
88
|
+
You can run lineup with `!` in the same session (for example `! lineup risky`), but the agent may comment on the output. Its commentary isn't part of lineup's check: the scores come from fixed rules, not from the agent.
|
|
89
|
+
|
|
90
|
+
To avoid this, open a second terminal in the same folder and run lineup there.
|
|
91
|
+
|
|
92
|
+
## Known limits
|
|
93
|
+
|
|
94
|
+
- **Claude Code is tested. Codex hooks aren't fully tested yet**, and you must trust them with `/hooks` before they run.
|
|
95
|
+
- The secret scan can miss things. Don't rely on it alone.
|
|
96
|
+
- Tested on Windows. Mac and Linux aren't tested yet.
|
|
97
|
+
- Rollback can't undo private files, files over 10 MB, links, or changes that are already committed.
|
|
98
|
+
- Scores come from simple rules, so expect some false flags.
|
|
99
|
+
|
|
100
|
+
## Develop
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
pip install -e ".[dev]"
|
|
104
|
+
pytest
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Testers: [docs/testers.md](https://github.com/HARSHAVARDHAN-RAJU5/agent-lineup/blob/main/docs/testers.md).
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
agent_lineup-0.1.0.dist-info/licenses/LICENSE,sha256=9wRuZjooHxTu9-cZixscqDVmvWSSRrlURu2SROXQt64,1091
|
|
2
|
+
lineup/__init__.py,sha256=u1fxKaH0tBg_6A3Y91fw4OzCPcnrKdyWCJTdjsiSuQE,109
|
|
3
|
+
lineup/__main__.py,sha256=ZkCO9HQ-l25sXDLfGaKhrtjGaUbg_lxuPlPruGrqhTw,58
|
|
4
|
+
lineup/banner.py,sha256=QarbGZLEqbrKyCg0GzOF9zi1rgE3vpPyoCCANbV_md4,2241
|
|
5
|
+
lineup/changes.py,sha256=pOG6DAI0cA5ukThdLHLb3jQ3O_R8DEvu6_bD5127dnQ,16653
|
|
6
|
+
lineup/cli.py,sha256=x_wqFl-81uzlK5faoDW3kuHzYjQ1yTO-ng10JQBNHSU,18639
|
|
7
|
+
lineup/functions.py,sha256=JdU5UsVvo4p_2GEtngUoak4q0zPAmt_yS_-OP2Xu3Sc,4884
|
|
8
|
+
lineup/git.py,sha256=MRhvjXGzwXK-NpLE2INu8HlzV1e9nzvFpjVAtnezVF0,5880
|
|
9
|
+
lineup/guard.py,sha256=W_nC6SaFQBd4s0U99gVw0yVA6jOOy1Cx9qM6bWQ_giA,7418
|
|
10
|
+
lineup/hookjson.py,sha256=AW1M31HQHWeVpH1fLMjNSwufMfyT9iZTsXysfAuzGts,2062
|
|
11
|
+
lineup/hooks.py,sha256=JALQyKPrxDtrrPh9Q-gk8EaC5hF8K6daaKiOldw0LsQ,13411
|
|
12
|
+
lineup/lineup-banner.txt,sha256=qays0dU83qnGGm5BdhTpfDZweELAbG3wW6RDh4ysbgM,708
|
|
13
|
+
lineup/llm.py,sha256=lrcS3O8tFasFq2bWfHwLuaS3huQj7WeQHTGh85yAhpQ,5852
|
|
14
|
+
lineup/paths.py,sha256=yHCD6NJfk7UlMKO3PX7S2VTFHOBXFqnM8T8_IE-WuwQ,2044
|
|
15
|
+
lineup/rollback.py,sha256=V4Rhgmxl8f7u1uuu36LP2HwFTbSinM0LRbAHBS6_KVk,10238
|
|
16
|
+
lineup/rules.py,sha256=oElNVDgYEKHK5qgyyEYt6B8GZI1KSYe2ycCeXq9UuSY,9420
|
|
17
|
+
lineup/snapshot.py,sha256=KtVOj6VepfhhTP9iUmnu7r6hQyjrXtzJ2nT7OFqmm_8,7257
|
|
18
|
+
lineup/store.py,sha256=Ita6eCk6y6O-5z2SuzFcVttusIBQdvoYcpzVpYySiaY,7385
|
|
19
|
+
lineup/summary.py,sha256=CgYbyelm9qbLW2_mxWJ4MuP1joqD-weEmKr8NBDrhoo,2651
|
|
20
|
+
lineup/table.py,sha256=XHtCG4HQpyNj5HLPhP87US2D9VqcdlXP3Nk_m3Repyw,2236
|
|
21
|
+
agent_lineup-0.1.0.dist-info/METADATA,sha256=r0nd429mG28JfPeypVraeRwsNeC-XSoCZAAazmJ6Wlo,4805
|
|
22
|
+
agent_lineup-0.1.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
|
|
23
|
+
agent_lineup-0.1.0.dist-info/entry_points.txt,sha256=4kUTJxSAL5faNMua1h-N4gO5XlLg5fnw7fYAXqCg0jA,43
|
|
24
|
+
agent_lineup-0.1.0.dist-info/top_level.txt,sha256=6vFg2PzbsMrFEZHnUq1mv4jRZGnOFHc6Vf1pBfzxWm0,7
|
|
25
|
+
agent_lineup-0.1.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Harshavardhan
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
lineup
|
lineup/__init__.py
ADDED
lineup/__main__.py
ADDED
lineup/banner.py
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
"""The lineup banner, for the user's own terminal only (init, --version, no command).
|
|
2
|
+
|
|
3
|
+
Never printed by hooks, diff, risky or rollback, so nothing the agent reads ever contains it.
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
import os
|
|
7
|
+
import sys
|
|
8
|
+
from importlib import resources
|
|
9
|
+
|
|
10
|
+
BLUE = "\x1b[38;2;111;140;216m" # #6F8CD8, readable on dark terminals
|
|
11
|
+
BLUE_256 = "\x1b[38;5;68m" # nearest 256-colour shade, for terminals without 24-bit colour
|
|
12
|
+
RESET = "\x1b[0m"
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
def text():
|
|
16
|
+
return resources.files("lineup").joinpath("lineup-banner.txt").read_text(encoding="utf-8")
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def _windows_colour_on(stream):
|
|
20
|
+
"""Turn on colour codes in a Windows console. False if it can't be done."""
|
|
21
|
+
try:
|
|
22
|
+
import ctypes
|
|
23
|
+
import msvcrt
|
|
24
|
+
handle = msvcrt.get_osfhandle(stream.fileno())
|
|
25
|
+
kernel32 = ctypes.windll.kernel32
|
|
26
|
+
mode = ctypes.c_uint32()
|
|
27
|
+
if not kernel32.GetConsoleMode(handle, ctypes.byref(mode)):
|
|
28
|
+
return False
|
|
29
|
+
return bool(mode.value & 0x0004 or kernel32.SetConsoleMode(handle, mode.value | 0x0004))
|
|
30
|
+
except (AttributeError, OSError, ValueError, ImportError):
|
|
31
|
+
return False
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def colour(stream):
|
|
35
|
+
"""The colour code to use, or "" for plain text."""
|
|
36
|
+
if "NO_COLOR" in os.environ or os.environ.get("TERM") == "dumb":
|
|
37
|
+
return ""
|
|
38
|
+
if not is_terminal(stream):
|
|
39
|
+
return ""
|
|
40
|
+
if sys.platform == "win32":
|
|
41
|
+
return BLUE if _windows_colour_on(stream) else ""
|
|
42
|
+
if os.environ.get("COLORTERM", "").lower() in ("truecolor", "24bit"):
|
|
43
|
+
return BLUE
|
|
44
|
+
return BLUE_256
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def is_terminal(stream):
|
|
48
|
+
try:
|
|
49
|
+
return stream.isatty()
|
|
50
|
+
except (AttributeError, ValueError):
|
|
51
|
+
return False
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
def show(stream=None):
|
|
55
|
+
"""Print the banner, only to a real terminal (piped or redirected output gets none)."""
|
|
56
|
+
stream = stream or sys.stdout
|
|
57
|
+
if not is_terminal(stream):
|
|
58
|
+
return
|
|
59
|
+
banner = text()
|
|
60
|
+
code = colour(stream)
|
|
61
|
+
if code:
|
|
62
|
+
banner = code + banner.rstrip("\n") + RESET + "\n"
|
|
63
|
+
try:
|
|
64
|
+
stream.write(banner)
|
|
65
|
+
except UnicodeEncodeError: # e.g. piped output on Windows: send the exact UTF-8 bytes
|
|
66
|
+
stream.flush()
|
|
67
|
+
stream.buffer.write(banner.encode("utf-8"))
|
|
68
|
+
stream.buffer.flush()
|
|
69
|
+
stream.write("\n")
|
|
70
|
+
stream.flush()
|
lineup/changes.py
ADDED
|
@@ -0,0 +1,403 @@
|
|
|
1
|
+
"""Finds what changed in the repo and sorts it into Additions, Removals, Edits and Schema."""
|
|
2
|
+
|
|
3
|
+
import difflib
|
|
4
|
+
import hashlib
|
|
5
|
+
import os
|
|
6
|
+
import posixpath
|
|
7
|
+
import re
|
|
8
|
+
from dataclasses import dataclass, field
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
|
|
11
|
+
from lineup import functions, git, hookjson, paths, store, table
|
|
12
|
+
from lineup.guard import is_private, private_pathspecs
|
|
13
|
+
|
|
14
|
+
BIG_FILE_BYTES = 10 * 1024 * 1024
|
|
15
|
+
|
|
16
|
+
LOCKFILES = {
|
|
17
|
+
"package-lock.json", "yarn.lock", "pnpm-lock.yaml", "poetry.lock", "pipfile.lock",
|
|
18
|
+
"cargo.lock", "go.sum", "composer.lock", "gemfile.lock", "uv.lock", "bun.lockb",
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
@dataclass
|
|
23
|
+
class Change:
|
|
24
|
+
path: str
|
|
25
|
+
kind: str # "added", "deleted" or "modified"
|
|
26
|
+
file_kind: str = "code"
|
|
27
|
+
private: bool = False
|
|
28
|
+
big: bool = False
|
|
29
|
+
binary: bool = False
|
|
30
|
+
symlink: bool = False # never followed: only where it points is compared
|
|
31
|
+
copy_of_private: bool = False # same bytes as a private file (e.g. .env renamed to settings.txt)
|
|
32
|
+
lineup_hooks_only: bool = False # an agent settings file where only lineup's own hooks changed
|
|
33
|
+
lines_added: int = 0
|
|
34
|
+
lines_removed: int = 0
|
|
35
|
+
diff: str = "" # never filled for private, big, binary or symlink files
|
|
36
|
+
funcs_added: list = field(default_factory=list)
|
|
37
|
+
funcs_removed: list = field(default_factory=list)
|
|
38
|
+
deps_added: list = field(default_factory=list)
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
# ---- what kind of file is this (by path only) ----
|
|
42
|
+
|
|
43
|
+
_TEST_DIRS = {"test", "tests", "__tests__", "spec", "specs"}
|
|
44
|
+
_TEST_NAME = re.compile(r"(^test_.*\.py$|_test\.(py|go)$|\.(test|spec)\.[jt]sx?$|^conftest\.py$)")
|
|
45
|
+
_SCHEMA = re.compile(r"(^|/)(migrations?|alembic)/|\.sql$|(^|/)schema\.(prisma|rb|sql)$|(^|/)db/schema")
|
|
46
|
+
_DOCS = re.compile(r"\.(md|rst|adoc)$|^docs?/|(^|/)(readme|changelog|license)(\.\w+)?$", re.I)
|
|
47
|
+
_CONFIG = re.compile(r"\.(ya?ml|toml|ini|cfg|conf)$|(^|/)dockerfile|(^|/)docker-compose|(^|/)\.github/|(^|/)\.claude/", re.I)
|
|
48
|
+
_AUTH_WORDS = {
|
|
49
|
+
"auth", "authn", "authz", "authenticate", "authentication", "authorize", "authorization", "oauth",
|
|
50
|
+
"login", "logins", "session", "sessions", "password", "passwords",
|
|
51
|
+
"payment", "payments", "billing", "checkout", "checkouts",
|
|
52
|
+
}
|
|
53
|
+
# Words in a path: split at / _ - . digits and capitals. "src/AuthService.ts" -> src, Auth, Service, ts
|
|
54
|
+
_WORD = re.compile(r"[A-Z]+(?![a-z])|[A-Z]?[a-z]+")
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def path_words(path):
|
|
58
|
+
return {word.lower() for word in _WORD.findall(path)}
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def is_auth(path):
|
|
62
|
+
"""Whole words only: auth_service.py and AuthService.ts match, author.py and authority.py don't."""
|
|
63
|
+
return bool(path_words(path) & _AUTH_WORDS)
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
def file_kind(path):
|
|
67
|
+
"""One word describing the file: private, test, schema, lockfile, dependencies, config, docs, auth or code."""
|
|
68
|
+
name = posixpath.basename(path).lower()
|
|
69
|
+
parts = set(path.lower().split("/")[:-1])
|
|
70
|
+
if is_private(path):
|
|
71
|
+
return "private"
|
|
72
|
+
if parts & _TEST_DIRS or _TEST_NAME.search(name):
|
|
73
|
+
return "test"
|
|
74
|
+
if _SCHEMA.search(path.lower()):
|
|
75
|
+
return "schema"
|
|
76
|
+
if name in LOCKFILES:
|
|
77
|
+
return "lockfile"
|
|
78
|
+
if functions.is_dependency_file(path):
|
|
79
|
+
return "dependencies"
|
|
80
|
+
if _CONFIG.search(path):
|
|
81
|
+
return "config"
|
|
82
|
+
if _DOCS.search(path):
|
|
83
|
+
return "docs"
|
|
84
|
+
if is_auth(path):
|
|
85
|
+
return "auth"
|
|
86
|
+
return "code"
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
# ---- reading both versions ----
|
|
90
|
+
|
|
91
|
+
def _is_binary(data):
|
|
92
|
+
return b"\0" in data[:8192]
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
def _decode(data):
|
|
96
|
+
return data.decode("utf-8", "replace") if data is not None else ""
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
def fingerprint(file):
|
|
100
|
+
"""SHA-256 of a file: changes if the file changes, but can't be turned back into it."""
|
|
101
|
+
digest = hashlib.sha256()
|
|
102
|
+
with open(file, "rb") as f:
|
|
103
|
+
for chunk in iter(lambda: f.read(1 << 20), b""):
|
|
104
|
+
digest.update(chunk)
|
|
105
|
+
return digest.hexdigest()
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
def is_link(file):
|
|
109
|
+
"""True for a symlink or a Windows junction (a shortcut). Checked without opening it."""
|
|
110
|
+
return paths.is_link(file)
|
|
111
|
+
|
|
112
|
+
|
|
113
|
+
def read_link(file):
|
|
114
|
+
"""Where a link points, as bytes. Reads the link itself, never the file it points to."""
|
|
115
|
+
return os.fsencode(os.readlink(file))
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
def link_hash(file):
|
|
119
|
+
"""Fingerprint of where a link points (never of what it points to)."""
|
|
120
|
+
return hashlib.sha256(read_link(file)).hexdigest()
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
def _link_change(root, base, row, path, file):
|
|
124
|
+
"""A path that is a symlink now: compare only where it points. Never opens the target."""
|
|
125
|
+
target = read_link(file)
|
|
126
|
+
at_start = (row or {}).get("at_start")
|
|
127
|
+
if at_start == "symlink":
|
|
128
|
+
if hashlib.sha256(target).hexdigest() == row.get("link"):
|
|
129
|
+
return None
|
|
130
|
+
kind = "modified"
|
|
131
|
+
elif at_start == "missing":
|
|
132
|
+
kind = "added"
|
|
133
|
+
elif at_start:
|
|
134
|
+
kind = "modified"
|
|
135
|
+
else:
|
|
136
|
+
old = git.show_file(root, (row or {}).get("base", base), path) # git stores a link as its target
|
|
137
|
+
if old == target:
|
|
138
|
+
return None
|
|
139
|
+
kind = "added" if old is None else "modified"
|
|
140
|
+
private = is_private(path)
|
|
141
|
+
return Change(path=path, kind=kind, file_kind="private" if private else file_kind(path),
|
|
142
|
+
private=private, symlink=True)
|
|
143
|
+
|
|
144
|
+
|
|
145
|
+
def _build(root, path, kind, old_size, load_old, new_size=None, load_new=None):
|
|
146
|
+
"""Compare the before and after of one file. Returns None if nothing really changed.
|
|
147
|
+
|
|
148
|
+
The "after" is the file on disk, unless `new_size` and `load_new` give another version.
|
|
149
|
+
"""
|
|
150
|
+
change = Change(path=path, kind=kind, file_kind=file_kind(path))
|
|
151
|
+
if change.file_kind == "private":
|
|
152
|
+
change.private = True
|
|
153
|
+
return change # never read a private file
|
|
154
|
+
file = Path(root, path)
|
|
155
|
+
if new_size is None:
|
|
156
|
+
if kind != "deleted" and is_link(file):
|
|
157
|
+
change.symlink = True
|
|
158
|
+
return change # never follow a link
|
|
159
|
+
new_size = file.stat().st_size if kind != "deleted" else 0
|
|
160
|
+
load_new = file.read_bytes
|
|
161
|
+
if max(old_size, new_size) > BIG_FILE_BYTES:
|
|
162
|
+
change.big = True
|
|
163
|
+
return change
|
|
164
|
+
|
|
165
|
+
old = (load_old() or b"") if kind != "added" else b""
|
|
166
|
+
new = (load_new() or b"") if kind != "deleted" else b""
|
|
167
|
+
if kind == "modified" and old == new:
|
|
168
|
+
return None
|
|
169
|
+
if _is_binary(old) or _is_binary(new):
|
|
170
|
+
change.binary = True
|
|
171
|
+
return change
|
|
172
|
+
|
|
173
|
+
old_text, new_text = _decode(old), _decode(new)
|
|
174
|
+
old_lines, new_lines = old_text.splitlines(), new_text.splitlines()
|
|
175
|
+
diff = list(difflib.unified_diff(old_lines, new_lines, f"a/{path}", f"b/{path}", lineterm=""))
|
|
176
|
+
if kind == "modified" and not diff:
|
|
177
|
+
return None # e.g. only line endings differ
|
|
178
|
+
body = diff[2:] # skip the ---/+++ header
|
|
179
|
+
change.lines_added = sum(1 for line in body if line.startswith("+"))
|
|
180
|
+
change.lines_removed = sum(1 for line in body if line.startswith("-"))
|
|
181
|
+
change.diff = "\n".join(diff)
|
|
182
|
+
if path in hookjson.SETTINGS_FILES:
|
|
183
|
+
change.lineup_hooks_only = hookjson.only_lineup_changed(old_text, new_text)
|
|
184
|
+
change.funcs_added, change.funcs_removed = functions.function_changes(path, old_text, new_text)
|
|
185
|
+
change.deps_added = functions.new_dependencies(path, old_text, new_text)
|
|
186
|
+
return change
|
|
187
|
+
|
|
188
|
+
|
|
189
|
+
SECRET_COPY_MAX_BYTES = 1024 * 1024 # private files are small; don't fingerprint big files for this
|
|
190
|
+
|
|
191
|
+
|
|
192
|
+
def private_fingerprints(root, rows=None, changed=()):
|
|
193
|
+
"""Fingerprints of every private file we know: from the photo, and private files on disk now."""
|
|
194
|
+
hashes = {r["hash"] for r in (rows or {}).values() if r.get("at_start") == "private" and r.get("hash")}
|
|
195
|
+
names = {p for p in changed if is_private(p)}
|
|
196
|
+
names.update(p for p in git.ignored_files(root, private_pathspecs()) if is_private(p))
|
|
197
|
+
names.update(p for p in git.tracked_files(root, private_pathspecs()) if is_private(p))
|
|
198
|
+
for name in names:
|
|
199
|
+
file = Path(root, name)
|
|
200
|
+
if paths.unsafe(root, name):
|
|
201
|
+
continue # never follow a link, even inside the repo
|
|
202
|
+
if not is_link(file) and file.is_file() and file.stat().st_size > 0:
|
|
203
|
+
hashes.add(fingerprint(file))
|
|
204
|
+
return hashes
|
|
205
|
+
|
|
206
|
+
|
|
207
|
+
def is_copy_of_private(file, hashes):
|
|
208
|
+
"""True if `file` has exactly the same bytes as a private file. Hashes it; never reads it as text."""
|
|
209
|
+
if not hashes or is_private(Path(file).name) or is_link(file) or not file.is_file():
|
|
210
|
+
return False # a private name is just private, not a "copy"
|
|
211
|
+
size = file.stat().st_size
|
|
212
|
+
return 0 < size <= SECRET_COPY_MAX_BYTES and fingerprint(file) in hashes
|
|
213
|
+
|
|
214
|
+
|
|
215
|
+
def seen_fingerprint(root, path):
|
|
216
|
+
"""What lineup saw of a changed file at a check: its fingerprint, "deleted", or None if not fingerprinted
|
|
217
|
+
(private, behind a link, outside the repo, or big)."""
|
|
218
|
+
file = Path(root, path)
|
|
219
|
+
if paths.unsafe(root, path) or is_private(path) or is_link(file):
|
|
220
|
+
return None
|
|
221
|
+
if not os.path.lexists(file):
|
|
222
|
+
return "deleted"
|
|
223
|
+
if not file.is_file() or file.stat().st_size > BIG_FILE_BYTES:
|
|
224
|
+
return None
|
|
225
|
+
return fingerprint(file)
|
|
226
|
+
|
|
227
|
+
|
|
228
|
+
def _private_copy(path, kind):
|
|
229
|
+
return Change(path=path, kind=kind, file_kind="private", private=True, copy_of_private=True)
|
|
230
|
+
|
|
231
|
+
|
|
232
|
+
def _kind(existed, exists_now):
|
|
233
|
+
if not existed:
|
|
234
|
+
return "added" if exists_now else None
|
|
235
|
+
return "modified" if exists_now else "deleted"
|
|
236
|
+
|
|
237
|
+
|
|
238
|
+
def _compare(root, base, row, path, secret_hashes=frozenset()):
|
|
239
|
+
"""Find the "before" of one file (photo first, then git) and compare it with now.
|
|
240
|
+
|
|
241
|
+
A file with the same bytes as a private file (`secret_hashes`) is treated as private and never read.
|
|
242
|
+
"""
|
|
243
|
+
file = Path(root, path)
|
|
244
|
+
reason = paths.unsafe(root, path)
|
|
245
|
+
if reason == paths.OUTSIDE:
|
|
246
|
+
return None # never read anything outside the repo
|
|
247
|
+
if reason and paths.link_in_folders(root, path): # e.g. a folder replaced by a junction
|
|
248
|
+
at_start = (row or {}).get("at_start")
|
|
249
|
+
existed = at_start not in (None, "missing") or git.blob_size(root, (row or {}).get("base", base), path) is not None
|
|
250
|
+
kind = _kind(existed, os.path.lexists(file)) or "modified"
|
|
251
|
+
return Change(path=path, kind=kind, file_kind="private" if is_private(path) else file_kind(path),
|
|
252
|
+
private=is_private(path), symlink=True)
|
|
253
|
+
if is_link(file):
|
|
254
|
+
return _link_change(root, base, row, path, file)
|
|
255
|
+
exists_now = file.is_file()
|
|
256
|
+
at_start = (row or {}).get("at_start")
|
|
257
|
+
|
|
258
|
+
if at_start == "symlink": # was a link in the photo: nothing of it was read, so name only
|
|
259
|
+
return Change(path=path, kind=_kind(True, exists_now), file_kind=file_kind(path),
|
|
260
|
+
private=is_private(path), symlink=True)
|
|
261
|
+
|
|
262
|
+
if at_start == "private": # only a fingerprint in the photo
|
|
263
|
+
if exists_now and fingerprint(file) == row["hash"]:
|
|
264
|
+
return None
|
|
265
|
+
return Change(path=path, kind=_kind(True, exists_now), file_kind="private", private=True)
|
|
266
|
+
|
|
267
|
+
if at_start == "big": # only size and time in the photo
|
|
268
|
+
if exists_now:
|
|
269
|
+
stat = file.stat()
|
|
270
|
+
if stat.st_size == row.get("size") and stat.st_mtime_ns == row.get("mtime_ns"):
|
|
271
|
+
return None
|
|
272
|
+
return Change(path=path, kind=_kind(True, exists_now), file_kind=file_kind(path), big=True)
|
|
273
|
+
|
|
274
|
+
if at_start == "uncommitted": # a saved copy in the photo
|
|
275
|
+
data = store.read_copy(root, path) or b""
|
|
276
|
+
kind = _kind(True, exists_now)
|
|
277
|
+
if kind != "deleted" and is_copy_of_private(file, secret_hashes):
|
|
278
|
+
return _private_copy(path, kind)
|
|
279
|
+
return _build(root, path, kind, len(data), lambda: data)
|
|
280
|
+
|
|
281
|
+
if at_start == "missing": # already gone when the photo was taken
|
|
282
|
+
kind = _kind(False, exists_now)
|
|
283
|
+
if kind and is_copy_of_private(file, secret_hashes):
|
|
284
|
+
return _private_copy(path, kind)
|
|
285
|
+
return _build(root, path, kind, 0, lambda: b"") if kind else None
|
|
286
|
+
|
|
287
|
+
rev = (row or {}).get("base", base) # not in the photo: git has the "before"
|
|
288
|
+
old_size = git.blob_size(root, rev, path)
|
|
289
|
+
kind = _kind(old_size is not None, exists_now)
|
|
290
|
+
if not kind:
|
|
291
|
+
return None
|
|
292
|
+
if kind != "deleted" and is_copy_of_private(file, secret_hashes):
|
|
293
|
+
before = git.show_file(root, rev, path) if kind == "modified" else None
|
|
294
|
+
if before is not None and hashlib.sha256(before).hexdigest() == fingerprint(file):
|
|
295
|
+
return None # unchanged since the photo (only hashed, never read as text)
|
|
296
|
+
return _private_copy(path, kind)
|
|
297
|
+
return _build(root, path, kind, old_size or 0, lambda: git.show_file(root, rev, path))
|
|
298
|
+
|
|
299
|
+
|
|
300
|
+
def committed_changes(root, old, new):
|
|
301
|
+
"""Files changed by the commits between `old` and `new`, each compared with its version at `old`."""
|
|
302
|
+
found = []
|
|
303
|
+
for path in git.files_between(root, old, new):
|
|
304
|
+
old_size, new_size = git.blob_size(root, old, path), git.blob_size(root, new, path)
|
|
305
|
+
kind = _kind(old_size is not None, new_size is not None)
|
|
306
|
+
if not kind:
|
|
307
|
+
continue
|
|
308
|
+
change = _build(root, path, kind, old_size or 0, lambda p=path: git.show_file(root, old, p),
|
|
309
|
+
new_size or 0, lambda p=path: git.show_file(root, new, p))
|
|
310
|
+
if change:
|
|
311
|
+
found.append(change)
|
|
312
|
+
return found
|
|
313
|
+
|
|
314
|
+
|
|
315
|
+
def find_changes(root, table=None):
|
|
316
|
+
"""Every file that changed since the photo (`table`), or since the last commit if there's no photo."""
|
|
317
|
+
if table:
|
|
318
|
+
base, rows = table["start_commit"], table["files"]
|
|
319
|
+
else:
|
|
320
|
+
base, rows = git.head_commit(root) or git.EMPTY_TREE, {}
|
|
321
|
+
|
|
322
|
+
paths = {path for _, path in git.changed_files(root, base)}
|
|
323
|
+
paths.update(git.untracked_files(root))
|
|
324
|
+
paths.update(rows)
|
|
325
|
+
if table: # a hidden private file created after the photo
|
|
326
|
+
paths.update(p for p in git.ignored_files(root, private_pathspecs()) if is_private(p))
|
|
327
|
+
|
|
328
|
+
secret_hashes = private_fingerprints(root, rows, paths)
|
|
329
|
+
found = []
|
|
330
|
+
for path in sorted(paths):
|
|
331
|
+
change = _compare(root, base, rows.get(path), path, secret_hashes)
|
|
332
|
+
if change:
|
|
333
|
+
found.append(change)
|
|
334
|
+
return found
|
|
335
|
+
|
|
336
|
+
|
|
337
|
+
# ---- sorting into groups ----
|
|
338
|
+
|
|
339
|
+
def _what(change):
|
|
340
|
+
if change.symlink:
|
|
341
|
+
return "link, not followed"
|
|
342
|
+
if change.private:
|
|
343
|
+
return "private, contents not read"
|
|
344
|
+
if change.big:
|
|
345
|
+
return "big file, not read"
|
|
346
|
+
if change.binary:
|
|
347
|
+
return "binary file"
|
|
348
|
+
return f"+{change.lines_added} -{change.lines_removed}"
|
|
349
|
+
|
|
350
|
+
|
|
351
|
+
def sort_into_groups(changes):
|
|
352
|
+
"""Group changes for display: {group: [(name, description, file path)]}."""
|
|
353
|
+
from lineup.rules import assertions_removed # here: rules is only needed for display
|
|
354
|
+
|
|
355
|
+
groups = {"Additions": [], "Removals": [], "Edits": [], "Schema": []}
|
|
356
|
+
for c in changes:
|
|
357
|
+
def add(title, name, what, path=c.path):
|
|
358
|
+
groups[title].append((name, what, path))
|
|
359
|
+
|
|
360
|
+
test = c.file_kind == "test"
|
|
361
|
+
not_read = c.private or c.big or c.binary or c.symlink
|
|
362
|
+
if c.kind == "added":
|
|
363
|
+
what = "new test file" if test else "new file"
|
|
364
|
+
add("Additions", c.path, f"{what}, {_what(c)}" if not_read else what)
|
|
365
|
+
elif c.kind == "deleted":
|
|
366
|
+
what = "test file deleted" if test else "file deleted"
|
|
367
|
+
add("Removals", c.path, f"{what}, {_what(c)}" if not_read else what)
|
|
368
|
+
else:
|
|
369
|
+
add("Edits", c.path, _what(c) if c.file_kind in ("code", "private") else f"{c.file_kind}, {_what(c)}")
|
|
370
|
+
|
|
371
|
+
if c.kind != "deleted":
|
|
372
|
+
for name in c.funcs_added:
|
|
373
|
+
label = "test added" if test else "function added"
|
|
374
|
+
add("Additions", name, f"{label} ({c.path})")
|
|
375
|
+
for dep in c.deps_added:
|
|
376
|
+
add("Additions", dep, f"new package ({c.path})")
|
|
377
|
+
if c.kind != "added":
|
|
378
|
+
for name in c.funcs_removed:
|
|
379
|
+
label = "test removed" if test else "function removed"
|
|
380
|
+
add("Removals", name, f"{label} ({c.path})")
|
|
381
|
+
lost = assertions_removed(c)
|
|
382
|
+
if lost:
|
|
383
|
+
add("Removals", "assertion" if lost == 1 else "assertions", f"{lost} removed ({c.path})")
|
|
384
|
+
|
|
385
|
+
if c.file_kind == "schema":
|
|
386
|
+
add("Schema", c.path, c.kind)
|
|
387
|
+
return groups
|
|
388
|
+
|
|
389
|
+
|
|
390
|
+
def format_groups(groups, base_label, box=None, ids=None):
|
|
391
|
+
"""The groups as one table: Group | ID | Name | What, with a line between groups.
|
|
392
|
+
|
|
393
|
+
`ids` is {path: ID} (the same IDs as lineup risky); rows about a function, package or assertion
|
|
394
|
+
show the ID of the file they're in.
|
|
395
|
+
"""
|
|
396
|
+
ids = ids or {}
|
|
397
|
+
blocks = []
|
|
398
|
+
for title, items in groups.items():
|
|
399
|
+
rows = items or [("nothing", "", None)]
|
|
400
|
+
blocks.append([(title if i == 0 else "", "" if path is None else ids.get(path, "-"), name, what)
|
|
401
|
+
for i, (name, what, path) in enumerate(rows)])
|
|
402
|
+
body = table.draw(("Group", "ID", "Name", "What"), blocks, (9, 4, 40, 50), box)
|
|
403
|
+
return f"lineup diff (compared with {base_label})\n\n{body}"
|