git-remote-ops 0.1.0 → 0.2.0

package/AGENTS.md

@@ -6,48 +6,50 @@ contributor-internal lore (parser invariants, harness boot), see `docs/`.
 ## What this library is
 
 Read-only Git client over smart HTTP. No `.git` dir, no subprocess, no working tree. Fetches one
-commit / one tree / one blob from a remote and hands back decoded bytes.
+commit / one tree / one blob from a remote and stores decoded objects in caller-supplied `storeDir`.
 
 ## What it is NOT
 
 - Not a `git` replacement. No push, no working tree, no ref writes, no merges.
 - Not a clone manager. No `have` negotiation, no incremental sync — every fetch is from-scratch with
   `done`.
-- Not streaming. Packs are held in memory in full.
+- Not a fully streaming parser. Extracted packs are loaded in memory during parse.
 
 If consumer needs writes or full clones, shell out to `git` instead.
 
 ## Install
 
 ```ts
-import { RemoteGit } from "jsr:@local/git-remote-ops";
+import { RemoteGit } from "git-remote-ops";
 ```
 
-Deno only. Requires `--allow-net`.
+Node.js 24 LTS package. Use pnpm for local development and npm-compatible package installs for consumers.
 
 CLI:
 
 ```sh
-deno install --global -A -n git-remote-ops src/cli.ts
+pnpm install
+pnpm build
+pnpm exec git-remote-ops --help
 ```
 
 ## Core API
 
 One class. All methods async. All return `Result<T, GitRemoteOpsError>` from
-[`better-result`](https://jsr.io/@local/better-result) — never throw.
+`better-result` — never throw.
 
 ```ts
-const client = new RemoteGit(url, { logger? , diagnostic? });
+const client = new RemoteGit(url, { storeDir, logger?, diagnostic? });
 
 await client.discover();      // -> ServerProfile
 await client.probe(verbose?); // -> ServerProfile (filter probe)
 await client.lsRefs();        // -> Map<refName, sha>
 await client.resolveRef(ref); // -> sha
-await client.fetchCommit(ref, { depth?, filter?, parseFull? });
+await client.fetchCommit(ref, { depth?, filter?, parseScope? });
 await client.fetchBlob(sha);
 await client.fetchTree(sha);
 await client.fetchTreeForCommit(ref, opts);
-client.getObject(sha);        // cache lookup, no network
+await client.getObject(sha);  // disk store lookup, no network
 ```
 
 ## Result handling
@@ -71,26 +73,26 @@ if you want exceptions.
 
 ## Caching behaviour consumers should know
 
-One `RemoteGit` instance keeps:
+`RemoteGit` keeps one `ServerProfile` in memory. Objects live on disk under `storeDir`:
 
-- One `ServerProfile` — populated on first call, reused after.
-- All materialized objects in a `Map<sha, GitObject>`, **process-lifetime**.
+- `objects/<aa>/<rest>` — Git-compatible loose objects.
+- `incoming/` — transient raw responses and extracted packs.
+- `snapshots/` — depth-1 snapshot markers.
 
 Implications:
 
-- Reuse instances across calls — commit → tree → blob dedupes `want`s.
-- Don't hold an instance forever in long-running processes if memory matters. Throw it away when
-  done with that remote.
-- Concurrent calls on one instance share the cache; no internal locking.
+- Reuse `storeDir` across calls/processes — commit → tree → blob dedupes `want`s from disk.
+- `getObject` is async because it reads the loose-object store.
+- Concurrent writers rely on atomic temp-file rename only; no lockfile yet.
 
 ## Choosing options
 
-| Goal                        | Recipe                                                                        |
-| --------------------------- | ----------------------------------------------------------------------------- |
-| One commit's metadata       | `fetchCommit(ref, { depth: 1, filter: "blob:none" })`                         |
-| List files at snapshot      | `fetchTreeForCommit(ref, { depth: 1, filter: "blob:none", parseFull: true })` |
-| One known blob              | `fetchBlob(sha)` (no profile probe needed)                                    |
-| Path → blob without network | `resolvePathToBlob` from `objects/tree.ts` against `client` object cache      |
+| Goal                        | Recipe                                                                               |
+| --------------------------- | ------------------------------------------------------------------------------------ |
+| One commit's metadata       | `fetchCommit(ref, { depth: 1, filter: "blob:none" })`                                |
+| List files at snapshot      | `fetchTreeForCommit(ref, { depth: 1, filter: "blob:none" })`                         |
+| One known blob              | `fetchBlob(sha)` (no profile probe needed)                                           |
+| Path → blob without network | read trees from `getObject` / `LooseObjectStore`, then use `objects/tree.ts` helpers |
 
 `depth: 1` + `filter: "blob:none"` is the cheap default. Both gracefully degrade if server doesn't
 advertise the capability — depth drops, filter logs at info and proceeds.
@@ -100,10 +102,10 @@ advertise the capability — depth drops, filter logs at info and proceeds.
 Library silent unless told otherwise.
 
 ```ts
-import { Logger } from "jsr:@local/git-remote-ops";
+import { Logger } from "git-remote-ops";
 
 const logger = new Logger({ level: "debug" });
-const client = new RemoteGit(url, { logger });
+const client = new RemoteGit(url, { logger, storeDir });
 // ...
 console.error(logger.summary()); // metrics table
 ```
@@ -122,7 +124,7 @@ git-remote-ops list-files  <url> [--ref HEAD] [--depth N] [--filter SPEC | --no-
                                  [--details]
 git-remote-ops cat-blob    <url> <blob-sha>
 
-Global: -q | -v | --debug | --stats
+Global: --store-dir <path> | -q | -v | --debug | --stats
 ```
 
 Stdout = data. Stderr = logs + errors. Exit 1 on any `Result.err`.
@@ -132,8 +134,8 @@ Full reference: `docs/cli.md`.
 ## Common pitfalls
 
 **Calling `fetchTree(commit.tree)` after `fetchCommit(ref)` with default options.** Default
-`parseFull: false` only materializes the commit itself. Use `fetchTreeForCommit` or pass
-`{ parseFull: true }` to `fetchCommit`.
+`parseScope: "target"` only materializes the commit itself. Use `fetchTreeForCommit` or pass
+`{ parseScope: "all" }` to `fetchCommit`.
 
 **Expecting full history.** `depth: 1` (the CLI default) gives one commit. Drop depth for full
 history; expect a much larger pack.
@@ -141,11 +143,10 @@ history; expect a much larger pack.
 **Treating `RefNotFoundError` as fatal.** A 40-char hex sha is accepted even when not in the ad —
 try direct sha if your ref is unusual.
 
-**Confusing `getObject` with a fetch.** `getObject` only checks the cache. Returns `undefined` if
-not yet materialized. No network call.
+**Confusing `getObject` with a fetch.** `getObject` only checks the disk store. Returns `undefined`
+if not yet materialized. No network call.
 
-**Throwing away the client between calls.** Re-discovers, re-probes, re-fetches the same objects.
-Reuse it.
+**Throwing away `storeDir` between calls.** Re-fetches the same objects. Reuse the directory.
 
 ## Where to look in source
 
@@ -170,7 +171,7 @@ Every file carries a module-level JSDoc banner with intent. Every constant docum
 
 ## Versioning
 
-Pre-1.0. API may shift. Pin exact version in `deno.json` imports.
+Pre-1.0. API may shift. Pin exact package versions in `package.json`.
 
 ## License
 

package/README.md

@@ -1,247 +1,121 @@
 # git-remote-ops
 
-**Read-only Git over smart HTTP, without `git`.** A single Deno/TypeScript process speaks the same
-wire protocol that `git fetch` does, then materializes the parts you actually want — one commit, one
-tree, one file — straight into memory.
+Read-only Git remote operations over smart HTTP.
 
-No `.git` directory. No subprocess. No filesystem state to clean up.
+Fetch commit metadata, trees, file lists, or blobs from a remote Git repository without cloning and without shelling out to `git`.
+
+## Install
 
 ```bash
-deno run --allow-net jsr:@local/git-remote-ops/cli \
-  cat-blob https://github.com/torvalds/linux.git \
-  e8c39d0f… > Makefile
+npm install git-remote-ops
 ```
 
-The blob lands in `Makefile`. The local working set never grows past a few KB of cached pack
-metadata.
-
-## Why bother
-
-A surprising amount of production code shells out to `git` just to read a file. That works, but it
-forces a clone — every byte of every revision of every path — to retrieve one revision of one path.
-On a large monorepo the clone is the entire latency budget.
-
-The Git smart-HTTP protocol has supported targeted fetches for years. With shallow depth
-(`--depth=1`) and partial-clone filters (`--filter=blob:none`, `--filter=tree:0`) you can ask the
-server for _just the snapshot you care about_, often a few hundred kilobytes total. Mainline `git`
-exposes this, but to use it you still need an on-disk clone and a forked process.
-
-`git-remote-ops` is the thin client you'd write if you only ever did this one thing: a few hundred
-lines of TypeScript that build the same `want`/`done` request `git` would, parse the packfile that
-comes back, and hand you the decoded objects.
-
-## What's in the box
-
-Six subcommands, all read-only:
+Run CLI without installing into project:
 
-| Command      | Does                                                          |
-| ------------ | ------------------------------------------------------------- |
-| `probe`      | Capability sniff: protocol version, shallow, filter dialects. |
-| `ls-refs`    | List advertised refs as `<sha> <name>`.                       |
-| `cat-commit` | Fetch + decode a commit object.                               |
-| `cat-tree`   | Fetch + decode a root tree (or a specific tree by sha).       |
-| `list-files` | Walk a snapshot and emit every path.                          |
-| `cat-blob`   | Stream one blob's raw bytes to stdout.                        |
-
-And the same operations are available as a library:
-
-```typescript
-import { RemoteGit } from "jsr:@local/git-remote-ops";
-
-const client = new RemoteGit("https://github.com/owner/repo.git");
-const { sha, commit } = (await client.fetchCommit("HEAD")).unwrap();
-const tree = (await client.fetchTree(commit.tree)).unwrap();
-const blob = (await client.fetchBlob(tree[0].sha)).unwrap();
+```bash
+npx git-remote-ops --help
 ```
 
-Every public call returns a `Result<T, GitRemoteOpsError>` from
-[`better-result`](https://jsr.io/@local/better-result) — never throws.
-
-## How it works
+## CLI
 
-The work splits into four layers, and the codebase mirrors that split.
+Every command needs `--store-dir`. Reuse same directory to avoid re-downloading objects.
 
+```bash
+git-remote-ops --store-dir /tmp/git-remote-ops-cache ls-refs https://github.com/owner/repo.git
 ```
-                 ┌──────────────────────────┐
-src/client.ts ─→ │  RemoteGit (public API)  │
-                 └────────────┬─────────────┘
-                              │
-     ┌────────────────────────┼───────────────────────┐
-     ▼                        ▼                       ▼
-src/protocol/            src/pack/                src/objects/
-pkt-line, refs,         pack parser, delta,       commit + tree
-upload-pack             zlib bridge               decoders
-     │                        ▲
-     ▼                        │
-src/transport.ts ─────────────┘
-(fetch wrappers)
-```
-
-### 1. Transport — `src/transport.ts`
-
-Two `fetch` calls do all the network work:
-
-1. `GET /info/refs?service=git-upload-pack` — the ref advertisement.
-2. `POST /git-upload-pack` — the actual fetch, body framed below.
-
-Both helpers thread an optional `Logger` so byte counts and durations roll into the same metrics
-struct the CLI's `--stats` flag prints.
-
-### 2. Protocol — `src/protocol/`
-
-**pkt-line framing.** Everything Git sends over HTTP is wrapped in a 4-byte ASCII-hex length prefix.
-Length `0000` is a flush; `0001` and `0002` are v2 control packets. `parsePktLines` walks a buffer
-end-to-end, returning subarray views — no payload bytes are copied. Trailing `\n`s are preserved
-because some callers care.
-
-**Ref advertisement.** v0/v1 puts capabilities NUL-separated on the first ref line; v2 puts them on
-standalone pkt-lines after `version 2`. We parse both, merge the capability sets, and prefer v2 when
-both are advertised.
-
-**`git-upload-pack` request.** A `want <sha>` line per object, an optional `deepen <n>` for shallow,
-an optional `filter <spec>` for partial-clone, a flush, then `done`. The v2 encoder adds a
-`command=fetch` header section and turns capabilities into separate argument lines instead of riding
-the first `want`.
-
-**Sideband demux.** The response is itself a stream of pkt-lines; each data line starts with a
-channel byte. Channel 1 is the packfile, channel 2 is human-readable progress, channel 3 is fatal
-stderr. `extractPack` looks at the first real data line, picks the right strategy (raw `PACK`
-signature, sideband, or PACK-search fallback), and returns the concatenated channel-1 bytes.
-
-### 3. Packfile — `src/pack/`
 
-A packfile is `PACK` + version + count + N variable-length object entries + a trailing SHA-1. Each
-entry has a type/size header (a varint with a 3-bit type field jammed into the first byte) followed
-by a zlib-compressed body.
-
-The parser walks entries in order. For non-delta types it inflates and hashes the body in one shot.
-For deltas — two flavours, `OBJ_OFS_DELTA` (base referenced by negative byte offset) and
-`OBJ_REF_DELTA` (base referenced by 20-byte sha) — it applies the delta against the resolved base
-and re-hashes the reconstructed object.
-
-Two implementation notes worth flagging:
-
-**zlib boundaries.** Multiple deflate streams sit back-to-back inside the pack with no length
-prefix. The streaming API doesn't tell you how many input bytes a single decode consumed, so we
-reach into `node:zlib`'s `_processChunk` and read `bytesWritten` to advance the cursor. It's an
-implementation detail of Node's zlib binding, but it's a stable one, and the alternative is parsing
-deflate headers ourselves.
+```bash
+git-remote-ops --store-dir /tmp/git-remote-ops-cache \
+  cat-commit https://github.com/owner/repo.git --ref main
+```
 
-**Targeted bailout.** If you only need a single commit, the parser accepts a `targets` set and
-returns as soon as those shas are materialized — even mid-pack. On a large snapshot pack that's the
-difference between parsing five megabytes of objects and parsing fifty kilobytes.
+```bash
+git-remote-ops --store-dir /tmp/git-remote-ops-cache \
+  list-files https://github.com/owner/repo.git --ref main
+```
 
-**Delta application.** The delta stream is a tiny opcode language: "copy `n` bytes from base at
-offset `o`" or "insert these `n` literal bytes". A size of zero on a copy opcode means 64 KiB.
-Out-of-bounds copies become typed errors, never silent corruption.
+```bash
+git-remote-ops --store-dir /tmp/git-remote-ops-cache \
+  cat-blob https://github.com/owner/repo.git <blob-sha> > file.bin
+```
 
-### 4. Objects — `src/objects/`
+Commands:
 
-Commit and tree decoders. Both inputs are the _uncompressed_ bodies produced by layer 3; neither
-parser cares how the bytes got there.
+```text
+probe       <url>
+ls-refs     <url>
+cat-commit  <url> [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter]
+cat-tree    <url> [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter] [--tree-sha <sha>]
+list-files  <url> [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter] [--details]
+cat-blob    <url> <blob-sha>
+```
 
-A tree is `<mode> <name>\0<sha20>` repeated, with no length prefix. A commit is RFC822-ish headers +
-blank line + free-form message; we keep `tree`, first `parent`, `author`, `committer`. Multiple
-`parent` lines (merge commits) collapse to the first because this client only walks toward snapshots
-— never along history.
+See [`docs/cli.md`](docs/cli.md) for full CLI reference.
 
-`walkTree` and `resolvePathToBlob` round out the layer for callers that want to flatten a tree or
-resolve `path/to/file` to a blob sha without touching the network.
+## API
 
-### 5. The client — `src/client.ts`
+```ts
+import { RemoteGit } from "git-remote-ops";
 
-`RemoteGit` ties the layers together and caches two things across calls:
+const git = new RemoteGit("https://github.com/owner/repo.git", {
+  storeDir: "/tmp/git-remote-ops-cache",
+});
 
-- A `ServerProfile` (refs + capabilities + filter probe results), populated lazily on the first call
-  and shared by everything after.
-- Every materialized object, in a process-lifetime `Map`. Subsequent fetches dedupe `want` lists
-  against this map, so a typical commit → tree → blob sequence only pays the network cost for new
-  objects.
+const commit = await git.fetchCommit("HEAD", {
+  depth: 1,
+  filter: "blob:none",
+});
+if (commit.isErr()) throw commit.error;
 
-There's a small bit of negotiation logic: shallow depth is dropped if the server didn't advertise
-`shallow`; filters are dropped (with an info log) if the server didn't advertise `filter`. `probe()`
-exists because some servers _advertise_ filter support but don't honour it — we send minimal-cost
-test fetches with `blob:none` and `tree:0` and look at the returned pack to find out.
+const tree = await git.fetchTree(commit.value.commit.tree);
+if (tree.isErr()) throw tree.error;
 
-## Errors
+console.log(tree.value);
+```
 
-Every public function returns `Result<T, E>`. Errors are tagged classes from `better-result`, so you
-can narrow on `_tag` and pull structured fields off:
+```ts
+const files = await git.fetchTreeForCommit("main", {
+  depth: 1,
+  filter: "blob:none",
+});
 
-```typescript
-const result = await client.fetchBlob(sha);
-if (result.isErr()) {
-  switch (result.error._tag) {
-    case "TransportError": // .status, .url, .method available
-    case "PackParseError": // .offset, .reason
-    case "ObjectNotFoundError": // .sha
-      // …
+if (files.isOk()) {
+  for (const entry of files.value.entries) {
+    console.log(entry.mode, entry.sha, entry.name);
   }
 }
 ```
 
-The full union is exported as `GitRemoteOpsError`. See `src/errors.ts`.
-
-## Observability
-
-The bundled `Logger` does three jobs at once:
-
-- Levelled messages (`info`, `debug`, `trace`) with namespaced children.
-- Cumulative metrics: HTTP request count / bytes / time, pack objects by type / bytes / parse time.
-- An aligned summary table for `--stats`.
-
-The library never logs above `info` unless you ask. Pass a configured `Logger` via
-`new RemoteGit(url, { logger })`, or hand it a `diagnostic` callback and the constructor will route
-debug-level lines to it.
-
-## CLI
-
-```text
-git-remote-ops <command> [options] <url>
-
-Global flags:
-  -q, --quiet       silent
-  -v, --verbose     debug
-      --debug       trace
-      --stats       print metrics summary after completion
-
-probe <url>
-ls-refs <url>
-cat-commit <url> [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter]
-cat-tree <url>   [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter]
-                 [--tree-sha <sha>]
-list-files <url> [--ref HEAD] [--depth 1] [--filter blob:none | --no-filter]
-                 [--details]
-cat-blob <url> <blob-sha>
+```ts
+const blob = await git.fetchBlob("<blob-sha>");
+if (blob.isOk()) {
+  await writeFile("file.bin", blob.value);
+}
 ```
 
-See `docs/cli.md` for the long-form reference.
-
-## Development
-
-```bash
-deno task check        # type-check
-deno task lint
-deno task fmt:check
-deno task test         # unit tests (no network)
-deno task test:integration   # spins up a local git server
-deno task verify       # fmt + lint + check, what CI runs
+Core methods:
+
+```ts
+await git.probe();
+await git.lsRefs();
+await git.resolveRef(ref);
+await git.fetchCommit(ref, options?);
+await git.fetchTree(treeSha);
+await git.fetchTreeForCommit(ref, options?);
+await git.fetchBlob(blobSha);
+await git.getObject(sha);
 ```
 
-Tests for each layer live next to their code as `*.test.ts`. The integration tests under
-`src/testing/integration/` exercise a real `git-upload-pack` talking to the client — see
-`docs/git-server-harness-AGENTS.md` for how the harness boots.
+All public methods return `Result<T, GitRemoteOpsError>` from `better-result`.
+
+## Cache
 
-## Limitations
+`storeDir` persists fetched Git objects. Reuse it across calls or CLI runs.
 
-By design this client only does fetches. It can't push, can't write to a working tree, can't manage
-refs. It also makes no attempt to negotiate `have` lines — every request is a from-scratch fetch
-with `done`, on the assumption that you're after small slices of remote history rather than
-incrementally syncing a clone.
+## Limits
 
-The packfile is held in memory in full. For large monorepos that fits in practice (kilobytes with
-`blob:none` + `--depth=1`), but a streaming parser would be a reasonable evolution.
+Fetch only. No push, checkout, ref writes, or full clone management.
 
 ## License
 
-See `LICENSE`.
+MIT. See [`LICENSE`](LICENSE).