gridlinegpu 0.1.9__tar.gz → 0.2.0__tar.gz

{gridlinegpu-0.1.9 → gridlinegpu-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gridlinegpu
-Version: 0.1.9
+Version: 0.2.0
 Summary: Python client for Gridline YAML workload routing
 Author: Gridline GPU
 License: Proprietary
@@ -28,13 +28,13 @@ Linux.
 1. **Sign up** at https://gridlinegpu.com/signup and verify your email.
 2. **Request access** at https://gridlinegpu.com/get-started: accept the partner beta terms and
    describe your use case in a few lines.
-3. **Wait for approval.** Gridline approves each request by hand and grants your account beta
-   credit; the get-started page shows the status. There is no payment step: jobs draw on that
-   credit.
+3. **Wait for approval.** Gridline checks each request; approval grants your account beta
+   credit, and the get-started page shows the status. There is no payment step: jobs draw on
+   that credit.
 4. **Install** the SDK with the source extra (Python 3.10 or newer):
 
    ```sh
-   python -m pip install 'gridlinegpu[source-contract]>=0.1.9,<0.2'
+   python -m pip install 'gridlinegpu[source-contract]>=0.2.0,<0.3'
    ```
 
 5. **Create a token** once. Browser approval saves a private, scoped token file; it is never
@@ -151,6 +151,230 @@ compiler. Without it, the YAML commands and `client.access()` still work and sou
 `source_dependencies_missing` with the install line. The "Source runs (beta)" section below has the
 limits, costs and failure codes.
 
+## Functions: `App`, `@app.function`, `.remote()` and `.spawn()` (preview)
+
+Decorate a Python function instead of writing YAML. Each call becomes one ordinary source job,
+with the same access, credit holds, limits, isolation and billing as `run()`:
+
+```python
+import gridlinegpu as gl
+
+app = gl.App("transcribe")
+image = gl.Image.curated("gridline/hf-torch-2.5@r1")
+
+@app.function(image=image, gpu="any:16GB", outputs={"transcript": "transcript.txt"})
+def transcribe(audio: gl.File, language: str = "en") -> dict:
+    text = run_model(audio.path, language)
+    (gl.output_dir() / "transcript.txt").write_text(text)
+    return {"chars": len(text)}
+
+print(transcribe.remote(gl.File("audio/a.flac")))   # waits; files land in gridline-outputs/
+call = transcribe.spawn(gl.File("audio/b.flac"))     # returns once the job is admitted
+print(call.job_id, call.get())
+```
+
+How the parts work:
+
+- **Code.** The function's module and the local modules and packages it imports (read from the
+  source, never run) are sent with the job. Add data files with `include=["data/*.json"]`. Inside
+  the job, `gridlinegpu` is a small stand-in, so the module imports in any image; only the
+  function runs, never `local_entrypoint` or `if __name__ == "__main__"` code.
+- **Arguments and return values** are JSON data plus `gl.File` arguments. There is no pickle.
+  Each `gl.File` is uploaded as an input file (at most 512 MiB). The return value is at most
+  8 MiB. Larger results are files written under `gl.output_dir()` and declared in `outputs`
+  (at most 128 MiB each).
+- **Options:**
+  - `gpu="any:<N>GB"` sets the minimum GPU memory (default 8 GB).
+  - `models={"base": gl.Model.hf(repo, revision)}` is fetched by the platform and found at
+    `gl.model_path("base")`.
+  - `kind="training"` marks a training job.
+  - `timeout=` is a local limit, enforced best-effort while the SDK waits; past it the job is
+    cancelled.
+- **Not yet available:** named GPU types, `max_price`, `capacity` and built images. Each
+  raises when used.
+
+**Many calls: `.map()` and `.starmap()`.** These return a `MapRun`; iterating it submits the calls
+and yields their values. Nothing is submitted before that.
+
+```python
+for value in transcribe.map([gl.File(path) for path in sorted(Path("audio").glob("*.flac"))],
+                           kwargs={"language": "en"}):
+    print(value)
+results = list(transcribe.starmap([(gl.File("a.flac"), "en"), (gl.File("b.flac"), "de")],
+                                  max_spend_usd=5.0, return_exceptions=True))
+```
+
+A map runs one of two ways:
+
+- **Warm pool.** With `max_spend_usd` (and `warm=None`, the default) or `warm=True`, a worker
+  pool of 1 to `max_workers` (at most 4) GPU workers runs the calls. The module loads once per
+  worker.
+  - Each call is one item, delivered at least once, with exactly one result committed.
+  - The function's `timeout` is the item timeout (60 to 3,600 seconds).
+  - A pooled function declares at most 7 outputs, each a file name directly in
+    `gl.output_dir()`.
+  - An account without pools falls back to chunks, unless `warm=True`.
+- **Chunks of ordinary jobs.** `per_job` calls per job (default 16; 1 when the function
+  declares outputs), at most `max_running` jobs at once (default 2, the account's
+  running-jobs limit). A call that raises is recorded and the job runs its next call.
+
+Common to both:
+
+- Values come back in input order (`order="completed"` yields each as it finishes).
+- `@app.function(retries=N)` (0 to 4) gives each failed call N more attempts.
+- A call that still fails raises `gl.FunctionError` (`function_raised`, the job's failure code,
+  or `item_dead`). With `return_exceptions=True` it is yielded instead.
+- Each job's or item's files land under `gridline-outputs/<function>/<job or pool id>/`.
+
+Inside a job, a decorated function is the plain function: call other functions with
+`.local()`. `.remote()` and `.map()` are refused there, because the job has no network.
+
+**Fine-tuning in one call.** `gl.finetune.lora()` and `gl.finetune.qlora()` submit one training job on
+`gridline/hf-finetune-2.5@r1`. The job runs Gridline's recipe, trl `SFTTrainer` with a peft LoRA:
+bf16 for LoRA, 4-bit NF4 for QLoRA. The platform fetches the base model at its pinned revision;
+the dataset is a JSON-lines file whose rows have a `text` field:
+
+```python
+run = gl.finetune.qlora(gl.Model.hf("HuggingFaceTB/SmolLM2-135M", revision="93efa2f097d58c2a74874c7e644dbc9b0cee75a2"),
+                        "data/train.jsonl", eval_dataset="data/eval.jsonl", epochs=2, rank=16)
+metrics = run.get(outputs_dir="adapters/smol")   # adapter_model.safetensors, adapter_config.json, metrics.json
+print(metrics["eval_loss_before"], metrics["eval_loss_after"])
+```
+
+Options: `epochs`, `learning_rate`, `rank`, `alpha`, `dropout`, `max_seq_length`,
+`batch_size`, `max_steps`, `target_modules` and `gpu` (default `"any:16GB"`). The adapter is at
+most 128 MiB. Full fine-tunes (all weights) are not offered yet.
+
+**From the command line.** Running submits paid jobs, so it needs `--apply`:
+
+```sh
+gridline app show transcribe_app.py                 # each function's generated YAML; nothing is sent
+gridline run --apply transcribe_app.py --folder audio   # the local_entrypoint, arguments typed from it
+gridline run --apply transcribe_app.py::transcribe --audio a.flac --language en
+gridline run --apply --detach transcribe_app.py::transcribe --audio a.flac   # print the job id and exit
+gridline pools status <pool id>                     # also: items, close, cancel --apply
+```
+
+`gridline run` imports the file the way `python app.py` would. Each `--name value` is converted
+by the parameter's annotation or default: `int`, `float`, `bool`, `str`, or `gl.File` for a path.
+- **Failures:** a failed job raises `gl.FunctionError` with the job's failure code. An uncertain
+  submission raises `SubmissionOutcomeUnknown`, as `run()` does; continue it with
+  `transcribe.resume(error.submission_id)`.
+
+## Many files: batches of ordinary jobs
+
+`client.run_batch()` (0.1.10 or newer) runs one workload over many files, at most `per_job` files
+(1 to 64) per job.
+Each chunk is an ordinary `run()`, with the same admission, funding, routing, isolation and
+billing as any job, and nothing new on the server. The workload declares one input per slot,
+`item_00` to `item_<per_job - 1>` (`gridlinegpu.batch_slots(per_job)` lists the names). Give each
+slot a default `source`, such as a small placeholder file, and its own `inputs/` path:
+
+```yaml
+task:
+  # ...
+  command: [python, transcribe.py, --count, {parameter: count}]
+inputs:
+  item_00: {source: audio/empty.flac, path: inputs/item_00.flac}
+  item_01: {source: audio/empty.flac, path: inputs/item_01.flac}
+  # ... through item_31 for per_job=32
+parameters:
+  count: {type: integer, default: 0}
+```
+
+```python
+from pathlib import Path
+
+run = client.run_batch("transcribe.yaml", sorted(Path("audio").glob("*.flac")), per_job=32,
+                       count_parameter="count")
+for done in run.results():          # one BatchResult per job, as each one finishes
+    if done.succeeded:
+        done.job.outputs(f"results/{done.index:04d}")
+    else:
+        print(done.index, done.files, done.result.failure_code)
+```
+
+- **Nothing is submitted until `results()` is iterated.** At most `max_running` jobs (default 2,
+  the per-account running-jobs default) run at once.
+- **When the account is at its running-jobs limit,** the chunk waits and the same submission is
+  resumed once a job finishes.
+- **A short last chunk** leaves its unused slots at their default. Use `count_parameter` to tell
+  the job how many real files it got (an `integer` parameter bound in `command`).
+- **Limits:** files are project-relative paths, at most 512 MiB each and 1 GiB per job, as for any
+  input.
+- **Cost:** every job pays its own start-up (pod start, image, model load). Choose `per_job` so
+  each job's work is well above that.
+- **Errors:** an error other than the running-jobs limit is raised as `run()` raises it.
+  - `run.jobs` lists what was submitted, and `run.remaining` lists what was not.
+  - An error with a `submission_id` is resumed with `client.resume()`, never run again.
+- **Without a client:** `gridlinegpu.batch(files, per_job)` returns the input maps alone, one per
+  job, to pass as `run(..., inputs=...)`.
+
+## Worker pools (beta)
+
+A worker pool runs one workload over a stream of files on 1 to 4 warm GPU workers. The model
+loads once per worker instead of once per file. Pools are in beta: the account must be enabled
+for them (`pool_owner_not_allowed` otherwise). `client.pool()` and `gridlinegpu.worker` need
+0.1.10 or newer.
+
+```python
+from pathlib import Path
+
+pool = client.pool("transcribe.yaml", max_workers=2, max_spend_usd=5.0, unit_name="audio_second")
+pool.put_many(sorted(Path("audio").glob("*.flac")))   # enqueue any time while the pool is open
+pool.close()                                          # no more items: workers drain, then stop
+for item in pool.results(directory="results"):       # each item once it is finished; done
+    print(item.key, item.state, item.last_error_detail)   # items' outputs land in results/<key>/
+```
+
+`pool.outputs(key, directory)` downloads one done item's outputs. Each file is checked against its
+listed size and SHA-256 before it appears, and an existing identical file is kept. Outputs are
+kept for 7 days after the item finished (`pool_item_output_expired` after that).
+
+The workload's command runs once per worker and takes items with `gridlinegpu.worker`:
+
+```python
+import gridlinegpu.worker
+
+model = load_model()
+for item in gridlinegpu.worker.items():
+    text = model.transcribe(item.input_path)
+    (item.output_dir / "transcript.json").write_text(text)
+    item.done()                     # or item.fail("why"); an exception hands the item back failed
+```
+
+What a pool is:
+
+- **Bounded.** A pool has hard limits: `max_duration_seconds` (default 3,600, at most 24 hours),
+  `max_items` (at most 5,000), `max_spend_usd` and `max_workers` (at most 4). Each worker is an
+  ordinary 1-GPU job with the same routing, isolation, holds and settlement as `run()`.
+- **Scaling.** Gridline starts workers while queued work justifies them and stops a worker after
+  `idle_seconds` (default 120) without items. An idle pool costs nothing; enqueue again to
+  restart it.
+- **Billing.** Billing is pass-through: you pay the workers' GPU time, with no pool fee.
+  `status()["money"]` shows what is held and settled. The per-item cost is informational: each
+  worker's settled cost is shared by busy time, and cold start and idle time are shown
+  separately.
+- **Items.** Each item has one input file (at most 512 MiB). Uploads go in batches of at most 64
+  files and 1 GiB. An item has at most 8 output files, 128 MiB each and 256 MiB in all.
+- **Retries.** A failed item is retried after a backoff, up to `max_attempts` (default 3, at
+  most 5), then kept as `dead` with its last error. `pool.retry(key)` requeues it once. A lost
+  machine requeues an item without using an attempt.
+- **Keys.** A key defaults to the file's SHA-256, so the same file is enqueued once. Pass `key=`
+  to name items yourself.
+- **At least once.** An item can run twice, for example when a worker is lost after finishing
+  it or a lease expires. Only one result is ever committed, so make your work idempotent.
+  Transcription and most batch inference already are.
+- **Shared process.** Items of one pool share a warm process inside a worker. They are not
+  isolated from each other, only from other accounts.
+- **No network.** As with `run()`, your code has no network. Items arrive as local files over a
+  Unix socket that Gridline's supervisor serves.
+
+If `pool()` ends with `submission_pending` (uncertain), call `client.resume_pool(submission_id)`.
+It resends the same request and returns the same pool. `client.pool_handle(pool_id)` reattaches
+to a pool from another process.
+
+
 ## YAML routing workloads
 
 Submit one YAML workload through Gridline's authenticated API. The Python client stages
@@ -158,7 +382,7 @@ files referenced by YAML, returns a route for review, then launches only that ex
 fresh plan when you supply a cost ceiling and idempotency key.
 
 ```sh
-python -m pip install 'gridlinegpu>=0.1.9,<0.2'
+python -m pip install 'gridlinegpu>=0.2.0,<0.3'
 ```
 
 Run browser setup once. It creates a 30-day scoped token in a private file, attempts to
@@ -272,7 +496,7 @@ step in the SDK. Request access first (see the quickstart above); `client.access
 `gridline access` show where your request stands.
 
 ```sh
-python -m pip install 'gridlinegpu[source-contract]>=0.1.9,<0.2'
+python -m pip install 'gridlinegpu[source-contract]>=0.2.0,<0.3'
 gridline setup --apply   # once: saves a private, scoped token file
 gridline access          # "approved" before your first run
 ```
@@ -395,7 +619,7 @@ Under `hardened_user_separation` each process has an 8 GiB data-memory limit, so
 base model straight to the GPU (`device_map` or 4-bit loading) rather than building it in
 CPU memory. While a job runs, about $1.78 of credits is held per window of about an hour;
 it settles at the actual cost after teardown (the acceptance job cost about $0.06). Full
-fine-tunes that return merged weights need larger outputs and are coming soon.
+fine-tunes that return merged weights need larger outputs and are not yet offered.
 
 The rest of this section describes how source runs behave.
 

{gridlinegpu-0.1.9 → gridlinegpu-0.2.0}/README.md

@@ -10,13 +10,13 @@ Linux.
 1. **Sign up** at https://gridlinegpu.com/signup and verify your email.
 2. **Request access** at https://gridlinegpu.com/get-started: accept the partner beta terms and
    describe your use case in a few lines.
-3. **Wait for approval.** Gridline approves each request by hand and grants your account beta
-   credit; the get-started page shows the status. There is no payment step: jobs draw on that
-   credit.
+3. **Wait for approval.** Gridline checks each request; approval grants your account beta
+   credit, and the get-started page shows the status. There is no payment step: jobs draw on
+   that credit.
 4. **Install** the SDK with the source extra (Python 3.10 or newer):
 
    ```sh
-   python -m pip install 'gridlinegpu[source-contract]>=0.1.9,<0.2'
+   python -m pip install 'gridlinegpu[source-contract]>=0.2.0,<0.3'
    ```
 
 5. **Create a token** once. Browser approval saves a private, scoped token file; it is never
@@ -133,6 +133,230 @@ compiler. Without it, the YAML commands and `client.access()` still work and sou
 `source_dependencies_missing` with the install line. The "Source runs (beta)" section below has the
 limits, costs and failure codes.
 
+## Functions: `App`, `@app.function`, `.remote()` and `.spawn()` (preview)
+
+Decorate a Python function instead of writing YAML. Each call becomes one ordinary source job,
+with the same access, credit holds, limits, isolation and billing as `run()`:
+
+```python
+import gridlinegpu as gl
+
+app = gl.App("transcribe")
+image = gl.Image.curated("gridline/hf-torch-2.5@r1")
+
+@app.function(image=image, gpu="any:16GB", outputs={"transcript": "transcript.txt"})
+def transcribe(audio: gl.File, language: str = "en") -> dict:
+    text = run_model(audio.path, language)
+    (gl.output_dir() / "transcript.txt").write_text(text)
+    return {"chars": len(text)}
+
+print(transcribe.remote(gl.File("audio/a.flac")))   # waits; files land in gridline-outputs/
+call = transcribe.spawn(gl.File("audio/b.flac"))     # returns once the job is admitted
+print(call.job_id, call.get())
+```
+
+How the parts work:
+
+- **Code.** The function's module and the local modules and packages it imports (read from the
+  source, never run) are sent with the job. Add data files with `include=["data/*.json"]`. Inside
+  the job, `gridlinegpu` is a small stand-in, so the module imports in any image; only the
+  function runs, never `local_entrypoint` or `if __name__ == "__main__"` code.
+- **Arguments and return values** are JSON data plus `gl.File` arguments. There is no pickle.
+  Each `gl.File` is uploaded as an input file (at most 512 MiB). The return value is at most
+  8 MiB. Larger results are files written under `gl.output_dir()` and declared in `outputs`
+  (at most 128 MiB each).
+- **Options:**
+  - `gpu="any:<N>GB"` sets the minimum GPU memory (default 8 GB).
+  - `models={"base": gl.Model.hf(repo, revision)}` is fetched by the platform and found at
+    `gl.model_path("base")`.
+  - `kind="training"` marks a training job.
+  - `timeout=` is a local limit, enforced best-effort while the SDK waits; past it the job is
+    cancelled.
+- **Not yet available:** named GPU types, `max_price`, `capacity` and built images. Each
+  raises when used.
+
+**Many calls: `.map()` and `.starmap()`.** These return a `MapRun`; iterating it submits the calls
+and yields their values. Nothing is submitted before that.
+
+```python
+for value in transcribe.map([gl.File(path) for path in sorted(Path("audio").glob("*.flac"))],
+                           kwargs={"language": "en"}):
+    print(value)
+results = list(transcribe.starmap([(gl.File("a.flac"), "en"), (gl.File("b.flac"), "de")],
+                                  max_spend_usd=5.0, return_exceptions=True))
+```
+
+A map runs one of two ways:
+
+- **Warm pool.** With `max_spend_usd` (and `warm=None`, the default) or `warm=True`, a worker
+  pool of 1 to `max_workers` (at most 4) GPU workers runs the calls. The module loads once per
+  worker.
+  - Each call is one item, delivered at least once, with exactly one result committed.
+  - The function's `timeout` is the item timeout (60 to 3,600 seconds).
+  - A pooled function declares at most 7 outputs, each a file name directly in
+    `gl.output_dir()`.
+  - An account without pools falls back to chunks, unless `warm=True`.
+- **Chunks of ordinary jobs.** `per_job` calls per job (default 16; 1 when the function
+  declares outputs), at most `max_running` jobs at once (default 2, the account's
+  running-jobs limit). A call that raises is recorded and the job runs its next call.
+
+Common to both:
+
+- Values come back in input order (`order="completed"` yields each as it finishes).
+- `@app.function(retries=N)` (0 to 4) gives each failed call N more attempts.
+- A call that still fails raises `gl.FunctionError` (`function_raised`, the job's failure code,
+  or `item_dead`). With `return_exceptions=True` it is yielded instead.
+- Each job's or item's files land under `gridline-outputs/<function>/<job or pool id>/`.
+
+Inside a job, a decorated function is the plain function: call other functions with
+`.local()`. `.remote()` and `.map()` are refused there, because the job has no network.
+
+**Fine-tuning in one call.** `gl.finetune.lora()` and `gl.finetune.qlora()` submit one training job on
+`gridline/hf-finetune-2.5@r1`. The job runs Gridline's recipe, trl `SFTTrainer` with a peft LoRA:
+bf16 for LoRA, 4-bit NF4 for QLoRA. The platform fetches the base model at its pinned revision;
+the dataset is a JSON-lines file whose rows have a `text` field:
+
+```python
+run = gl.finetune.qlora(gl.Model.hf("HuggingFaceTB/SmolLM2-135M", revision="93efa2f097d58c2a74874c7e644dbc9b0cee75a2"),
+                        "data/train.jsonl", eval_dataset="data/eval.jsonl", epochs=2, rank=16)
+metrics = run.get(outputs_dir="adapters/smol")   # adapter_model.safetensors, adapter_config.json, metrics.json
+print(metrics["eval_loss_before"], metrics["eval_loss_after"])
+```
+
+Options: `epochs`, `learning_rate`, `rank`, `alpha`, `dropout`, `max_seq_length`,
+`batch_size`, `max_steps`, `target_modules` and `gpu` (default `"any:16GB"`). The adapter is at
+most 128 MiB. Full fine-tunes (all weights) are not offered yet.
+
+**From the command line.** Running submits paid jobs, so it needs `--apply`:
+
+```sh
+gridline app show transcribe_app.py                 # each function's generated YAML; nothing is sent
+gridline run --apply transcribe_app.py --folder audio   # the local_entrypoint, arguments typed from it
+gridline run --apply transcribe_app.py::transcribe --audio a.flac --language en
+gridline run --apply --detach transcribe_app.py::transcribe --audio a.flac   # print the job id and exit
+gridline pools status <pool id>                     # also: items, close, cancel --apply
+```
+
+`gridline run` imports the file the way `python app.py` would. Each `--name value` is converted
+by the parameter's annotation or default: `int`, `float`, `bool`, `str`, or `gl.File` for a path.
+- **Failures:** a failed job raises `gl.FunctionError` with the job's failure code. An uncertain
+  submission raises `SubmissionOutcomeUnknown`, as `run()` does; continue it with
+  `transcribe.resume(error.submission_id)`.
+
+## Many files: batches of ordinary jobs
+
+`client.run_batch()` (0.1.10 or newer) runs one workload over many files, at most `per_job` files
+(1 to 64) per job.
+Each chunk is an ordinary `run()`, with the same admission, funding, routing, isolation and
+billing as any job, and nothing new on the server. The workload declares one input per slot,
+`item_00` to `item_<per_job - 1>` (`gridlinegpu.batch_slots(per_job)` lists the names). Give each
+slot a default `source`, such as a small placeholder file, and its own `inputs/` path:
+
+```yaml
+task:
+  # ...
+  command: [python, transcribe.py, --count, {parameter: count}]
+inputs:
+  item_00: {source: audio/empty.flac, path: inputs/item_00.flac}
+  item_01: {source: audio/empty.flac, path: inputs/item_01.flac}
+  # ... through item_31 for per_job=32
+parameters:
+  count: {type: integer, default: 0}
+```
+
+```python
+from pathlib import Path
+
+run = client.run_batch("transcribe.yaml", sorted(Path("audio").glob("*.flac")), per_job=32,
+                       count_parameter="count")
+for done in run.results():          # one BatchResult per job, as each one finishes
+    if done.succeeded:
+        done.job.outputs(f"results/{done.index:04d}")
+    else:
+        print(done.index, done.files, done.result.failure_code)
+```
+
+- **Nothing is submitted until `results()` is iterated.** At most `max_running` jobs (default 2,
+  the per-account running-jobs default) run at once.
+- **When the account is at its running-jobs limit,** the chunk waits and the same submission is
+  resumed once a job finishes.
+- **A short last chunk** leaves its unused slots at their default. Use `count_parameter` to tell
+  the job how many real files it got (an `integer` parameter bound in `command`).
+- **Limits:** files are project-relative paths, at most 512 MiB each and 1 GiB per job, as for any
+  input.
+- **Cost:** every job pays its own start-up (pod start, image, model load). Choose `per_job` so
+  each job's work is well above that.
+- **Errors:** an error other than the running-jobs limit is raised as `run()` raises it.
+  - `run.jobs` lists what was submitted, and `run.remaining` lists what was not.
+  - An error with a `submission_id` is resumed with `client.resume()`, never run again.
+- **Without a client:** `gridlinegpu.batch(files, per_job)` returns the input maps alone, one per
+  job, to pass as `run(..., inputs=...)`.
+
+## Worker pools (beta)
+
+A worker pool runs one workload over a stream of files on 1 to 4 warm GPU workers. The model
+loads once per worker instead of once per file. Pools are in beta: the account must be enabled
+for them (`pool_owner_not_allowed` otherwise). `client.pool()` and `gridlinegpu.worker` need
+0.1.10 or newer.
+
+```python
+from pathlib import Path
+
+pool = client.pool("transcribe.yaml", max_workers=2, max_spend_usd=5.0, unit_name="audio_second")
+pool.put_many(sorted(Path("audio").glob("*.flac")))   # enqueue any time while the pool is open
+pool.close()                                          # no more items: workers drain, then stop
+for item in pool.results(directory="results"):       # each item once it is finished; done
+    print(item.key, item.state, item.last_error_detail)   # items' outputs land in results/<key>/
+```
+
+`pool.outputs(key, directory)` downloads one done item's outputs. Each file is checked against its
+listed size and SHA-256 before it appears, and an existing identical file is kept. Outputs are
+kept for 7 days after the item finished (`pool_item_output_expired` after that).
+
+The workload's command runs once per worker and takes items with `gridlinegpu.worker`:
+
+```python
+import gridlinegpu.worker
+
+model = load_model()
+for item in gridlinegpu.worker.items():
+    text = model.transcribe(item.input_path)
+    (item.output_dir / "transcript.json").write_text(text)
+    item.done()                     # or item.fail("why"); an exception hands the item back failed
+```
+
+What a pool is:
+
+- **Bounded.** A pool has hard limits: `max_duration_seconds` (default 3,600, at most 24 hours),
+  `max_items` (at most 5,000), `max_spend_usd` and `max_workers` (at most 4). Each worker is an
+  ordinary 1-GPU job with the same routing, isolation, holds and settlement as `run()`.
+- **Scaling.** Gridline starts workers while queued work justifies them and stops a worker after
+  `idle_seconds` (default 120) without items. An idle pool costs nothing; enqueue again to
+  restart it.
+- **Billing.** Billing is pass-through: you pay the workers' GPU time, with no pool fee.
+  `status()["money"]` shows what is held and settled. The per-item cost is informational: each
+  worker's settled cost is shared by busy time, and cold start and idle time are shown
+  separately.
+- **Items.** Each item has one input file (at most 512 MiB). Uploads go in batches of at most 64
+  files and 1 GiB. An item has at most 8 output files, 128 MiB each and 256 MiB in all.
+- **Retries.** A failed item is retried after a backoff, up to `max_attempts` (default 3, at
+  most 5), then kept as `dead` with its last error. `pool.retry(key)` requeues it once. A lost
+  machine requeues an item without using an attempt.
+- **Keys.** A key defaults to the file's SHA-256, so the same file is enqueued once. Pass `key=`
+  to name items yourself.
+- **At least once.** An item can run twice, for example when a worker is lost after finishing
+  it or a lease expires. Only one result is ever committed, so make your work idempotent.
+  Transcription and most batch inference already are.
+- **Shared process.** Items of one pool share a warm process inside a worker. They are not
+  isolated from each other, only from other accounts.
+- **No network.** As with `run()`, your code has no network. Items arrive as local files over a
+  Unix socket that Gridline's supervisor serves.
+
+If `pool()` ends with `submission_pending` (uncertain), call `client.resume_pool(submission_id)`.
+It resends the same request and returns the same pool. `client.pool_handle(pool_id)` reattaches
+to a pool from another process.
+
+
 ## YAML routing workloads
 
 Submit one YAML workload through Gridline's authenticated API. The Python client stages
@@ -140,7 +364,7 @@ files referenced by YAML, returns a route for review, then launches only that ex
 fresh plan when you supply a cost ceiling and idempotency key.
 
 ```sh
-python -m pip install 'gridlinegpu>=0.1.9,<0.2'
+python -m pip install 'gridlinegpu>=0.2.0,<0.3'
 ```
 
 Run browser setup once. It creates a 30-day scoped token in a private file, attempts to
@@ -254,7 +478,7 @@ step in the SDK. Request access first (see the quickstart above); `client.access
 `gridline access` show where your request stands.
 
 ```sh
-python -m pip install 'gridlinegpu[source-contract]>=0.1.9,<0.2'
+python -m pip install 'gridlinegpu[source-contract]>=0.2.0,<0.3'
 gridline setup --apply   # once: saves a private, scoped token file
 gridline access          # "approved" before your first run
 ```
@@ -377,7 +601,7 @@ Under `hardened_user_separation` each process has an 8 GiB data-memory limit, so
 base model straight to the GPU (`device_map` or 4-bit loading) rather than building it in
 CPU memory. While a job runs, about $1.78 of credits is held per window of about an hour;
 it settles at the actual cost after teardown (the acceptance job cost about $0.06). Full
-fine-tunes that return merged weights need larger outputs and are coming soon.
+fine-tunes that return merged weights need larger outputs and are not yet offered.
 
 The rest of this section describes how source runs behave.
 

{gridlinegpu-0.1.9 → gridlinegpu-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gridlinegpu"
-version = "0.1.9"
+version = "0.2.0"
 description = "Python client for Gridline YAML workload routing"
 readme = "README.md"
 requires-python = ">=3.10"

{gridlinegpu-0.1.9 → gridlinegpu-0.2.0}/src/gridlinegpu/__init__.py

@@ -1,14 +1,21 @@
 """Gridline YAML workload client."""
 
+from ._batch import BatchResult, BatchRun, batch
+from ._batch import slot_names as batch_slots
+from .app import (App, File, Function, FunctionCall, FunctionError, Image, MapRun, Model, Output, model_path,
+                  output_dir)
 from .client import GridlineClient, PlannedWorkload
 from .errors import (
     GridlineError, LaunchOutcomeUnknown, OutputsIncomplete, ReceiptPersistenceError, SubmissionOutcomeUnknown,
     WaitTimeout, WorkloadValidationError,
 )
+from . import finetune  # noqa: E402 - after app's names, which its job module imports
 
 __all__ = [
-    "GridlineClient", "GridlineError", "LaunchOutcomeUnknown", "OutputsIncomplete", "PlannedWorkload",
-    "ReceiptPersistenceError", "SubmissionOutcomeUnknown", "WaitTimeout", "WorkloadValidationError",
+    "App", "BatchResult", "BatchRun", "File", "Function", "FunctionCall", "FunctionError", "GridlineClient",
+    "GridlineError", "Image", "LaunchOutcomeUnknown", "MapRun", "Model", "Output", "OutputsIncomplete", "PlannedWorkload",
+    "ReceiptPersistenceError", "SubmissionOutcomeUnknown", "WaitTimeout", "WorkloadValidationError", "batch",
+    "batch_slots", "finetune", "model_path", "output_dir",
 ]