gipper 0.3.0__tar.gz → 0.4.0__tar.gz

{gipper-0.3.0 → gipper-0.4.0}/AGENTS.md

@@ -54,6 +54,7 @@ working *on* gipper.
 | `costs.py` | the whole-run estimate and score for one offer |
 | `history.py` | memory, read from the ledger: failure rates, ready times, link speeds, compute times, billing factors |
 | `run.py` | the run: plan, rent, preflight, upload, detached job, log, checkpoints, collect, teardown |
+| `custody.py` | what reached each machine: manifests, scanning, the incident report |
 | `lifecycle.py` | verified teardown |
 | `reap.py` | the 15-minute sweep for forgotten machines |
 | `billing.py` | providers' bills → `spend.billed`; spend for the daily cap |

{gipper-0.3.0 → gipper-0.4.0}/CHANGELOG.md

@@ -1,5 +1,33 @@
 # Changelog
 
+## 0.4.0 (2026-11-05)
+
+A forensic trail of what reached each rented machine. When a provider reports an
+incident, `gipper custody report` answers what to rotate and what can't be vouched
+for. See [docs/custody.md](docs/custody.md).
+
+### Added
+
+- **Every `Remote.put`, from jobs and client scripts alike, takes a manifest first**
+  of exactly what tar will send (path, size, sha256, mode), and writes a
+  `custody.upload` receipt before any byte moves. If the manifest can't be taken,
+  the upload doesn't happen.
+- **Uploads are scanned** for sensitive file names and credential patterns. Findings
+  are recorded and printed, never the matched text. Provider keys and values from
+  `custody_secret_files` are always refused. `custody_on_sensitive` or a job's
+  `[custody]` table can refuse other findings, with an `allow` list for reviewed
+  false positives. Jobs are checked before a machine is rented.
+- **`Remote.put_secrets`** places credentials deliberately, as a mode-600 env file
+  sent over stdin, recording names and keyed fingerprints only.
+- **`custody.machine` receipts:** provider, machine, address, image, env names, and
+  hashes of the job file and command.
+- **`gipper custody`:** `report` (rotate list, sensitive uploads, machines, gaps),
+  `show`, `find` (which machines received this file), `check` (was this exact value
+  ever placed on a machine; use it to confirm a rotation), and `declare` (record a
+  scp, a tunnel, or a machine gipper didn't rent).
+- `report --check-providers` compares providers' bills with the ledger, to reveal
+  machines gipper never knew about.
+
 ## 0.3.0 (2026-11-04)
 
 ### Added

{gipper-0.3.0 → gipper-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gipper
-Version: 0.3.0
+Version: 0.4.0
 Summary: Run GPU jobs across cloud markets with cost estimates, verified teardown, and an append-only ledger
 Author: Marc Lavallee
 License-Expression: MIT
@@ -174,6 +174,18 @@ compute_s = 900
 setup_download_gb = 5                 # weights and wheels the job fetches itself
 ```
 
+## After a provider incident
+
+Every upload is recorded before it leaves: a manifest of what was sent, sensitive-looking
+files flagged, and provider keys refused outright. When a provider reports a problem, one
+command gives you what to rotate, what to review, and what the trail can't vouch for:
+
+```
+gipper custody report --provider runpod --since 2026-10-01 --check-providers
+```
+
+See [docs/custody.md](docs/custody.md), including how to record what went around gipper.
+
 ## How gipper chooses
 
 Every offer is judged on the whole run, not its hourly price

{gipper-0.3.0 → gipper-0.4.0}/README.md

@@ -147,6 +147,18 @@ compute_s = 900
 setup_download_gb = 5                 # weights and wheels the job fetches itself
 ```
 
+## After a provider incident
+
+Every upload is recorded before it leaves: a manifest of what was sent, sensitive-looking
+files flagged, and provider keys refused outright. When a provider reports a problem, one
+command gives you what to rotate, what to review, and what the trail can't vouch for:
+
+```
+gipper custody report --provider runpod --since 2026-10-01 --check-providers
+```
+
+See [docs/custody.md](docs/custody.md), including how to record what went around gipper.
+
 ## How gipper chooses
 
 Every offer is judged on the whole run, not its hourly price

{gipper-0.3.0 → gipper-0.4.0}/ROADMAP.md

@@ -9,4 +9,7 @@ Potential next steps, without delivery commitments:
 - Coordinate owned GPUs and rented capacity through requests and leases.
 - Add priorities and pre-emption with explicit resource ownership.
 
+- Extend the custody trail to outputs brought back from a machine, and look inside
+  uploaded archives.
+
 These features are not implemented by the current CLI.

{gipper-0.3.0 → gipper-0.4.0}/SECURITY.md

@@ -31,6 +31,10 @@ no promised response time.
   or a mode-600 dotenv file. Known provider key values are redacted from API
   errors and receipts, but arbitrary secrets printed by jobs are not detected.
   The `[env]` name check is a guard, not a general secret detector.
+- `gipper custody report` lists what reached rented machines, for use after a
+  provider incident. It covers what passed through gipper or was declared; see
+  [docs/custody.md](docs/custody.md) for its limits. Custody manifests hold file
+  paths and hashes, and fingerprints are keyed hashes, never secret values.
 - Receipts include paths, provider resource identifiers, host addresses, and
   commands; logs and collected files can contain sensitive job data. Restrict
   filesystem permissions and review everything before sharing it.

{gipper-0.3.0 → gipper-0.4.0}/artifact.toml

@@ -6,20 +6,19 @@ kind = "reference-guide"
 form = "collection"
 status = "live"
 created = "2026-11-04"
-updated = "2026-11-04"
+updated = "2026-11-05"
 
-# Raw HTML is owned by site/; scripts/build_site.py renders public Markdown guides.
 [build]
-site = "docs"
 commands = ["uv run --group site python scripts/build_site.py"]
+site = "docs"
 
 [deploy]
 target = "github-pages"
 
 [deploy.github-pages]
-mode = "docs"
+commit = true
 docs_dir = "docs"
-subpath = ""
+mode = "docs"
 own_root = true
-commit = true
 push = false
+subpath = ""

{gipper-0.3.0 → gipper-0.4.0}/docs/clients.md

@@ -40,7 +40,7 @@ These names are stable within 0.x:
 | `gipper.providers.runpod.Runpod().api()` | the raw Runpod client |
 | `gipper.ledger.read(since=, kinds=, job=)` / `.write(kind, payload, job=)` | receipts |
 | `gipper.history.History.load()` | memory: `.failure(p, machine)`, `.ready_s`, `.mbps`, `.compute_s(workload, gpu)`, `.billing_factor(p)` |
-| `gipper.ssh.Remote(host, port, user)` | `.run`, `.put`, `.get` over gipper's key |
+| `gipper.ssh.Remote(host, port, user)` | `.run`, `.put`, `.put_secrets`, `.get` over gipper's key. `.put` records a custody manifest first and can raise `gipper.custody.Refused` |
 
 Kept for scripts written before 0.2: `gipper.runpod.api.Api` and
 `gipper.runpod.run._wait_ready`. Any other `gipper.runpod.run` name raises an
@@ -57,6 +57,8 @@ instead of silently patching nothing.
 - `spend` (computed at teardown) and `spend.billed` (the provider's bill, from
   `reconcile`; the latest one per machine counts). `gipper.billing.spend_since(t)`
   combines them by machine.
+- `custody.machine`, `custody.upload`, `custody.uploaded`, `custody.secret`,
+  `custody.note`: what reached a machine ([custody.md](custody.md)).
 - Receipts without `provider` predate 2026-10-26 and are Runpod's.
 
 An editable install imports code directly from its checkout. Use an isolated

gipper-0.4.0/docs/custody.md

@@ -0,0 +1,101 @@
+# Custody: what reached each machine
+
+When a provider tells you a machine hosting your work may have been compromised, you
+need three answers quickly: what was on it, which credentials to rotate, and what you
+can't be sure about. gipper keeps that trail as it works.
+
+```
+gipper custody report --provider runpod --since 2026-10-01 --check-providers
+gipper custody show POD_ID                # every file sent to one machine
+gipper custody find ./data/private.csv    # which machines received this file
+gipper custody check ./secrets.env        # was this exact value ever placed on a machine
+```
+
+`report` has four sections:
+
+- **Rotate:** credentials placed on machines, with how many machines and distinct values.
+- **Review:** uploaded files that look sensitive.
+- **Machines:** each rental with its machine ID, job, uploads and env names.
+- **Gaps:** what the report can't vouch for. Read this section before relying on the others.
+
+Add `--json` for the whole report as data. `--pod`, `--machine` and `--until` narrow it.
+
+## What is recorded
+
+**Each upload.** `Remote.put` reads the tar stream it is about to send, so exclude rules
+apply exactly as they will on the wire. It stores a manifest (path, size, sha256, mode)
+under `~/.local/share/gipper/custody` with owner-only permissions, then writes a
+`custody.upload` receipt. Only then does anything leave. If the manifest can't be taken,
+the upload doesn't happen. A `custody.uploaded` receipt records the outcome. An upload
+with no outcome, or a failed one, may still have delivered bytes, so treat it as arrived.
+
+**Each machine.** `custody.machine` records provider, machine ID, address, image, env
+names, and hashes of the job file and command.
+
+**Sensitive material.** Uploads are checked for file names like `.env`, `id_ed25519`
+and `*.pem`, and content patterns like private keys and common API tokens. Findings keep
+the path and rule, never the matched text. Files over 2 MB are hashed but not scanned,
+and archives are not opened. Both appear under Gaps.
+
+**Credentials.** Values are never recorded. A fingerprint is a keyed hash, with the key
+kept beside the ledger, so `check` can recognise a value without the ledger being
+usable to guess it. After you rotate, `check` on the new values should report that none
+has been seen on a machine.
+
+## Placing a credential on a machine
+
+Don't upload credential files. Do this instead, which records the names:
+
+```python
+remote.put_secrets({"SERVICE_TOKEN": token}, "/root/job/service.env")
+```
+
+The file is mode 600, and the values travel over stdin rather than the command line. It
+is receipted before sending, so a failed write still counts as exposed.
+
+## Recording what went around gipper
+
+gipper can't see a `scp` you ran by hand, a tunnel, or a machine created in a provider's
+console. Declare them, and they join the report:
+
+```
+gipper custody declare --provider runpod --pod POD_ID \
+  --secret SERVICE_TOKEN --from-env ./secrets.env --path /root/job/service.env --how scp
+gipper custody declare --provider runpod --pod POD_ID --file ./bundle --path /root/job/bundle --how scp
+gipper custody declare --provider runpod --pod POD_ID --tunnel "reverse tunnel to local port 8917"
+```
+
+Use `--at` for something that happened earlier, and `--confidence inferred` when you
+worked it out rather than saw it. `report --check-providers` compares providers' bills
+with the ledger and lists machines gipper has never heard of. This call is read-only.
+
+## Refusing uploads
+
+```toml
+# ~/.config/gipper/config.toml
+custody_on_sensitive = "refuse"                # default "warn": record it and say so
+custody_secret_files = ["~/.config/app/.env"]  # these values must never be uploaded
+```
+
+```toml
+# in a job
+[custody]
+on_sensitive = "refuse"
+allow = ["tests/fixtures/*.pem"]               # reviewed false positives
+```
+
+Provider keys and values from `custody_secret_files` are always refused, even when a
+path is allowed. Jobs are checked before renting, so a refusal doesn't cost a machine.
+
+## Limits
+
+- It records what passed through gipper or was declared. Commands run on a machine, and
+  anything a job downloaded for itself, are not recorded. The command is kept as a hash.
+- Outputs brought back from a machine have no manifest yet.
+- Scanning catches common shapes and known values, not every secret. A clean scan is not
+  proof that a file holds none.
+- Providers may add credentials of their own to a machine, such as a machine-scoped API
+  key. Check your provider's console for keys that outlived their machines.
+- The ledger is append-only by convention and isn't tamper-proof. Treat it as your own
+  record, not as evidence against an attacker who had access to this machine.
+- A trail only exists from the release that adds it. Earlier rentals show under Gaps.

{gipper-0.3.0 → gipper-0.4.0}/gipper/__init__.py

@@ -1,2 +1,2 @@
 """gipper: GPU jobs across cloud markets."""
-__version__ = "0.3.0"
+__version__ = "0.4.0"

{gipper-0.3.0 → gipper-0.4.0}/gipper/cli.py

@@ -10,6 +10,10 @@
     gipper pod cost [--since YYYY-MM-DD]          spend per job: billed where known, else computed
     gipper pod account                            balance per provider
     gipper ledger [--since YYYY-MM-DD] [--job ID] raw receipts
+    gipper custody report [--provider P] [--since D] [--check-providers]     what reached rented machines
+    gipper custody find FILE|SHA256|PATH-PART     which machines received this
+    gipper custody check ENV_FILE                 was this exact value ever placed on a machine
+    gipper custody declare --provider P --pod ID --secret NAME [...]         record what went around gipper
 
 Providers: runpod, vast, shadeform (keys RUNPOD_API_KEY, VAST_API_KEY,
 SHADEFORM_API_KEY in the environment or ~/.config/gipper/.env). Vast and Shadeform list offers without a key.
@@ -166,6 +170,51 @@ def _pod(args):
         print(f"total computed ${computed:.3f}; billed where known, else computed: ${billed_total:.3f}")
 
 
+def _custody(a):
+    from gipper import custody
+    if a.cmd == "report":
+        rep = custody.report(provider=a.provider, pod=a.pod, machine=a.machine, since=a.since, until=a.until,
+                             check_providers=a.check_providers)
+        if a.json:
+            print(json.dumps(rep, indent=1, default=str))
+        else:
+            custody.show(rep)
+    elif a.cmd == "show":
+        found = [m for m in custody.collect().values() if m["pod"] == a.pod]
+        if not found:
+            raise SystemExit(f"{a.pod}: not in the ledger")
+        for m in found:
+            print(f"{m['provider']}:{m['pod']}  machine {m['machine'] or '?'}  {m['name'] or '?'}")
+            for placed in m["secrets"]:
+                names = ", ".join(s["name"] for s in placed["secrets"])
+                print(f"  secrets  {placed['at']}  {names}  -> {placed.get('path') or '?'}  {placed.get('how') or ''}".rstrip())
+            for u in m["uploads"]:
+                state = "refused" if u.get("refused") else "declared" if u.get("declared") else {True: "sent", False: "failed", None: "outcome unknown"}[u.get("ok")]
+                print(f"  upload   {u['at']}  {u['src']} -> {u['dst']}  [{state}]  manifest {u['manifest']}")
+                flagged = {f["path"]: f["rule"] for f in u.get("findings") or []}
+                for e in custody.load_manifest(u["manifest"])["entries"]:
+                    if e["type"] != "dir":
+                        print(f"    {(e.get('sha256') or '')[:16]:16s} {e['size']:>12d}  {e['path']}" + (f"  !{flagged[e['path']]}" if e["path"] in flagged else ""))
+            for n in m["notes"]:
+                print(f"  note     {n['at']}  {n.get('tunnel') or ''} {n.get('note') or ''}".rstrip())
+    elif a.cmd == "find":
+        hits = custody.find(a.target)
+        for h in hits:
+            print(f"{h['at']}  {h['machine']:32s} {h['path']}  (from {h['src']}; {'sent' if h['arrived'] else 'outcome unknown' if h['arrived'] is None else 'failed'})")
+        print(f"{len(hits)} match(es)")
+    elif a.cmd == "check":
+        rows = custody.check(a.env_file)
+        for r in rows:
+            print(f"{r['name']:28s} " + (f"EXPOSED on {', '.join(r['exposed_on'])}" if r["exposed_on"] else "not seen on any machine"))
+    elif a.cmd == "declare":
+        written = custody.declare(provider=a.provider, pod=a.pod, secrets=a.secret, from_env=a.from_env, path=a.path, files=a.file,
+                                  tunnel=a.tunnel, note=a.note, how=a.how, confidence=a.confidence, machine=a.machine,
+                                  host=a.host, port=a.port, at=a.at)
+        if not written:
+            raise SystemExit("nothing to declare: give --secret, --file, --tunnel or --note")
+        print(f"{len(written)} receipt(s) written for {a.provider}:{a.pod}")
+
+
 def main(argv=None):
     sys.stdout.reconfigure(line_buffering=True)  # live output when redirected to a file
     signal.signal(signal.SIGPIPE, signal.SIG_DFL)  # `gipper pod hosts | head` ends quietly
@@ -201,9 +250,42 @@ def main(argv=None):
     lg = sub.add_parser("ledger", help="raw receipts")
     lg.add_argument("--since")
     lg.add_argument("--job")
+    cu = sub.add_parser("custody", help="what reached rented machines: the incident trail")
+    cs = cu.add_subparsers(dest="cmd", required=True)
+    rp = cs.add_parser("report", help="rotate list, sensitive uploads, machines and gaps")
+    rp.add_argument("--provider", choices=PROVIDERS)
+    rp.add_argument("--pod")
+    rp.add_argument("--machine")
+    rp.add_argument("--since", help="machines in use on or after this date")
+    rp.add_argument("--until", help="machines in use on or before this date")
+    rp.add_argument("--check-providers", action="store_true", help="also ask providers' bills for machines the ledger doesn't know (read-only)")
+    rp.add_argument("--json", action="store_true")
+    sh = cs.add_parser("show", help="every file sent to one machine")
+    sh.add_argument("pod")
+    fd = cs.add_parser("find", help="which machines received a file")
+    fd.add_argument("target", help="local file, sha256, or part of a path")
+    ck = cs.add_parser("check", help="has each value in this file been placed on a machine (values are not shown)")
+    ck.add_argument("env_file")
+    dc = cs.add_parser("declare", help="record what reached a machine outside gipper: scp, a tunnel, a machine it didn't rent")
+    dc.add_argument("--provider", required=True)
+    dc.add_argument("--pod", required=True)
+    dc.add_argument("--machine")
+    dc.add_argument("--host")
+    dc.add_argument("--port", type=int)
+    dc.add_argument("--secret", action="append", default=[], help="credential name, repeatable")
+    dc.add_argument("--from-env", help="read those names' values from this file, to fingerprint them")
+    dc.add_argument("--path", help="where it landed on the machine")
+    dc.add_argument("--file", action="append", default=[], help="local file or directory sent, repeatable: takes a manifest")
+    dc.add_argument("--tunnel", help="e.g. \"reverse tunnel to local port 8917\"")
+    dc.add_argument("--how", default="", help="e.g. scp")
+    dc.add_argument("--note", default="")
+    dc.add_argument("--confidence", choices=["direct", "inferred"], default="direct")
+    dc.add_argument("--at", help="when it happened, if not now (ISO time)")
     a = ap.parse_args(argv)
     if a.group == "pod":
         return _pod(a)
+    if a.group == "custody":
+        return _custody(a)
     if a.group == "ledger":
         from gipper import ledger
         for rec in ledger.read(since=a.since, job=a.job):

{gipper-0.3.0 → gipper-0.4.0}/gipper/config.py

@@ -35,6 +35,8 @@ DEFAULTS = {
     "hour_value_usd": 3.0,
     "failure_cost_usd": 1.0,
     "history_days": 90,             # how far back the ledger counts as memory
+    "custody_on_sensitive": "warn", # warn: record and say so; refuse: stop the upload (see docs/custody.md)
+    "custody_secret_files": [],     # env or JSON files whose values must never be uploaded
 }