cloudgpu-mcp 0.16.1 → 0.16.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -2
- package/examples/verified-tasks/README.md +48 -0
- package/examples/verified-tasks/cases.json +90 -0
- package/examples/verified-tasks/causal-lora-part-1.py +7 -0
- package/examples/verified-tasks/causal-lora-part-2.py +76 -0
- package/examples/verified-tasks/causal-lora.sh +4 -0
- package/examples/verified-tasks/comfy-wan-video.py +204 -0
- package/examples/verified-tasks/evidence.json +113 -0
- package/examples/verified-tasks/gpu-environment.py +6 -0
- package/examples/verified-tasks/run.mjs +52 -0
- package/package.json +1 -1
- package/src/agent-readiness.js +1 -1
- package/src/api.js +13 -0
- package/src/index.js +5 -3
- package/src/job-batches.js +20 -0
- package/src/job-input.js +2 -0
package/README.md
CHANGED
|
@@ -396,7 +396,7 @@ With `persist_summary=true`, the Job draft explicitly collects only `pipeline-su
|
|
|
396
396
|
|
|
397
397
|
### Funded connection, storage growth and remote MCP
|
|
398
398
|
|
|
399
|
-
Version 0.16.0 was verified on npm and the official Registry on 2026-11-13. The current stdio inventory is
|
|
399
|
+
Version 0.16.0 was verified on npm and the official Registry on 2026-11-13. The current stdio inventory is 57 tools. The remote endpoint at `https://cloudgpu.app/mcp` uses Streamable HTTP JSON responses and OAuth authorization code with S256 PKCE; it exposes 23 tools safe without local filesystem/SSH access. Public discovery needs no login; account and paid operations require scoped owner authorization. Always check live backend capabilities and actual source-specific billing.
|
|
400
400
|
|
|
401
401
|
`connect_account` returns a direct Google link and email alternative. Pending connection state is private, origin-bound and survives restart; the server allows 30 minutes for approval. Browser payment does not approve compute. `request_top_up` creates only a short-lived owner link; `wait_for_funds` requires a real credit signal and sufficient live balance. Neither creates a Job.
|
|
402
402
|
|
|
@@ -404,4 +404,18 @@ Dedicated storage budget is a separate, explicitly confirmed wallet reservation,
|
|
|
404
404
|
|
|
405
405
|
Multipart resume keeps the same reservation and original file hash. GPU restore renewal is file-scoped, temporary and feature-gated; no general account key is installed on a GPU. Local fixtures do not certify TB-sized transfers. Backend automatic public-model preparation currently has a 600-second deadline: a 9.8GB Wan video case exceeded it on the measured link. Prepared images or explicitly reviewed bounded download commands need their own acceptance; do not promise that every size below the model byte limit loads in time.
|
|
406
406
|
|
|
407
|
-
The Python SDK
|
|
407
|
+
The Python SDK and optional framework adapters are under [`python/`](python/README.md). PyPI publication requires an owner-configured Trusted Publisher. Real LangChain Core 1.8.0, OpenAI Agents SDK 0.24.0 and CrewAI 1.6.1 tool/invocation and anonymous HTTP acceptance passed; no paid model or GPU operation was used. See the Python README for exact scope and reproduction.
|
|
408
|
+
|
|
409
|
+
The reviewed [Python SDK0.1.0 release](https://github.com/cloudgpu-app/cloudgpu-mcp/releases/tag/python-v0.1.0) has a public wheel, source distribution and SHA256SUMS. The public wheel was downloaded and its checksum and ten SDK tests passed. PyPI is not yet configured; use the exact release URL rather than an unrelated package. Hermes root installation and existing-owner private connection/catalog acceptance passed after packaging/profile fixes in cloudgpu-app/hermes-cloudgpu#5.
|
|
410
|
+
|
|
411
|
+
### Reproducible paid task cases
|
|
412
|
+
|
|
413
|
+
See [environment check, synthetic causal-LM LoRA and tiny ComfyUI video cases](examples/verified-tasks/README.md) for pinned scripts, observed fees, download checksums and release evidence. They preview locally by default; paid submission needs owner approval. Resume an accepted Job by ID to download results without another GPU. For persistent tasks, `assess_task_fit` accepts `expected_output_bytes`; current direct collection and bounded network fallback limits are returned by the live contract.
|
|
414
|
+
|
|
415
|
+
### Durable server batches and whole-hour Jobs (0.16.3)
|
|
416
|
+
|
|
417
|
+
Use `prepare_job_batch` to review every command, fresh compute quote, deadline and aggregate GPU cap. Pass the exact manifest/hash and explicit user approval to `run_server_job_batch`. The server atomically commits all child Job rows or none; it dispatches from the database after client/server restarts, waiting for account/Key execution slots. `get_server_job_batch(batch_id)` returns bounded states/cost summaries; use individual `get_job` for detailed outputs. `cancel_server_job_batch` requires approval and preserves charges for already-started hours. Failed commands are not automatically rerun, and an uncertain provider result never authorizes another GPU.
|
|
418
|
+
|
|
419
|
+
REST equivalents: `POST /v1/job-batches` with `Idempotency-Key`, `total_budget_usd` and 1–32 `jobs`; `GET /v1/job-batches/{batch_id}`; `POST /v1/job-batches/{batch_id}/cancel`. The MCP reviewed manifest remains capped at eight children. Replaying the identical key/body returns the existing children. Existing `run_job_batch` retains its explicitly non-atomic client submission behavior for compatibility; the server tool never falls back to it.
|
|
420
|
+
|
|
421
|
+
For whole-hour Jobs, call `estimate_cost` for the exact configuration and accept the minimum/billing period. Supply its `billing_quote_id` plus `billing_accepted=true` to `run_job` or each batch child, in addition to the compute `quote_id` and execution approval. Each allocation needs a separate current billing quote. The Job cap must cover the minimum; elapsed portions of a started hour are not voluntarily refunded. Billing quotes last five minutes: an expired queued Job fails without allocation and requires a new reviewed request. It is never silently renewed or routed under different billing terms. Recovery uses durable provider receipts and independent upstream presence checks; ambiguous intents remain `needs_attention`.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Reproducible bounded GPU cases
|
|
2
|
+
|
|
3
|
+
Real internal acceptance, including model loading, execution, output download/SHA-256 and upstream release checks. Fees are observed final wallet charges, not future price guarantees.
|
|
4
|
+
|
|
5
|
+
| Case | Final run elapsed | Final GPU fee | Full development deliveries / submissions |
|
|
6
|
+
|---|---:|---:|---:|
|
|
7
|
+
| NVIDIA GPU environment check, RTX 3090 | 191 s | $0.0048 | 1 / 1 |
|
|
8
|
+
| Synthetic causal-LM LoRA, SmolLM2-135M, 8 steps, RTX 3090 | 190 s | $0.0097 | 1 / 4 |
|
|
9
|
+
| Wan2.1 1.3B ComfyUI video, 256×256, 17 frames, 8 steps, RTX 4090 | 1,064 s | $0.1105 | 1 / 12 |
|
|
10
|
+
|
|
11
|
+
These development submissions used evolving preparation and harness code. They include cancellations, unavailable stock, networking, dtype, encoder and output-transport failures. They are a small internal sample, not a commercial SLA or model-quality benchmark. The final video spent 874.102 s downloading/checking models and 18.776703 s starting ComfyUI, generating and encoding; remaining time includes provisioning and collection/release. Its MP4 was independently decoded as 17 H.264 frames. LoRA is a real nonzero trained adapter on four synthetic text records; it is not image/diffusion LoRA.
|
|
12
|
+
|
|
13
|
+
All 20 submitted acceptance Jobs are terminal; 19 allocated instances were independently confirmed released. One out-of-stock submission had no instance ID. GPU wallet fees including every failed/cancelled attempt and three separate output transport probes total **$0.9552 / $10 approved**. Storage reserve/settlement and provider invoices are separate. See [evidence.json](evidence.json) and live https://cloudgpu.app/agent-evidence.json.
|
|
14
|
+
|
|
15
|
+
## Run
|
|
16
|
+
|
|
17
|
+
From the repository root after `npm ci`, preview a bundled case locally:
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
node examples/verified-tasks/run.mjs gpu-environment
|
|
21
|
+
node examples/verified-tasks/run.mjs causal-lora
|
|
22
|
+
node examples/verified-tasks/run.mjs comfy-wan-video
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Review the scripts, live readiness/quote and spending/time ceilings. With your owner-approved connected credentials (or CLOUDGPU_API_KEY in your private environment), submit one case:
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
CGPU_SUBMIT=YES CGPU_JOB_KEY=my-new-reviewed-case-1 node examples/verified-tasks/run.mjs gpu-environment
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The runner prints the accepted Job ID immediately, waits for terminal/final cost, downloads only the requested ready outputs and verifies SHA-256. Files stay in a private `cloudgpu-results/<job_id>` directory; `CGPU_RESULTS_DIR` selects another private directory. It does not delete your persistent outputs.
|
|
32
|
+
|
|
33
|
+
Resume a known accepted Job without another quote or GPU:
|
|
34
|
+
|
|
35
|
+
```sh
|
|
36
|
+
CGPU_SUBMIT=YES CGPU_JOB_ID=job_<your-existing-id> node examples/verified-tasks/run.mjs gpu-environment
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
An uncertain initial submission must be reconciled with the original request/idempotency key; do not invent a replacement key or rerun blindly. Changing a case or beginning another task requires a new quote and approval. The final video runner's real resume/download path was also exercised against the accepted video Job.
|
|
40
|
+
|
|
41
|
+
## Limits and prerequisites
|
|
42
|
+
|
|
43
|
+
- Caps in `cases.json` are spending ceilings, not predicted charges. Quote current stock, image, disk and all preparation/download/output time. Hourly suppliers can require 3,600-second minimums; use only candidates with actual Job support and appropriate billing approval.
|
|
44
|
+
- Video requires a compatible existing ComfyUI image. The reviewed `flux-1-dev` image supplied ComfyUI for this specific test; this does not make arbitrary images compatible. No custom nodes are installed. ModelScope revisions and all model SHA-256 values are pinned. A pinned imageio-ffmpeg wheel supplies an encoder when absent; its mirror bytes must match the official wheel and binary hashes.
|
|
45
|
+
- Automatic model preparation remains bounded to 600 seconds. This example explicitly downloads large models in a separately reviewed bounded command (45-minute download / 60-minute Job ceiling); it does not silently extend the platform loader.
|
|
46
|
+
- Explicit automatic Job outputs are at most 100 MiB. On GPUs without direct object-storage connectivity, the verified control-plane fallback handles at most 8 MiB total, one relay at a time, using existing reservations and checksum checks. It does not relay authorization/validation denials. Larger outputs require verified direct connectivity or an approved export plan.
|
|
47
|
+
- These examples accept no new paid-storage charges; ensure existing quota before starting. Workspace/file capacity and paid storage budgets are separately reported by `/v1/storage`. Model weights go directly to the rented GPU, not into a CloudGPU model cache.
|
|
48
|
+
- GPU disk is erased on release. Inspect `output_collection`, final cost and release state. Commercial success rates and provider health are separately reported by the live readiness/evidence endpoints; unknown values remain unknown.
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
{
|
|
2
|
+
"gpu-environment": {
|
|
3
|
+
"gpu_model": "RTX 3090",
|
|
4
|
+
"gpu_count": 1,
|
|
5
|
+
"max_minutes": 8,
|
|
6
|
+
"max_spend_usd": 0.25,
|
|
7
|
+
"outputs": {
|
|
8
|
+
"files": [
|
|
9
|
+
"environment-check.json"
|
|
10
|
+
],
|
|
11
|
+
"max_bytes": 100000,
|
|
12
|
+
"accept_storage_charges": false
|
|
13
|
+
},
|
|
14
|
+
"script": "gpu-environment.py",
|
|
15
|
+
"interpreter": "python3"
|
|
16
|
+
},
|
|
17
|
+
"causal-lora": {
|
|
18
|
+
"gpu_model": "RTX 3090",
|
|
19
|
+
"gpu_count": 1,
|
|
20
|
+
"model_references": [
|
|
21
|
+
{
|
|
22
|
+
"name": "smollm",
|
|
23
|
+
"repo_id": "HuggingFaceTB/SmolLM2-135M",
|
|
24
|
+
"revision": "93efa2f097d58c2a74874c7e644dbc9b0cee75a2",
|
|
25
|
+
"files": [
|
|
26
|
+
{
|
|
27
|
+
"path": "config.json",
|
|
28
|
+
"size_bytes": 704,
|
|
29
|
+
"sha256": "1d556eab73b69c7f11f64c557a2f9c6f440bd4c6b89bb2584a6b498c92603843"
|
|
30
|
+
},
|
|
31
|
+
{
|
|
32
|
+
"path": "generation_config.json",
|
|
33
|
+
"size_bytes": 111,
|
|
34
|
+
"sha256": "2056c988e990b0d13670f63f2f3b87b3b6d07edaf7a3416998ba27dab2d8a059"
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"path": "model.safetensors",
|
|
38
|
+
"size_bytes": 269060552,
|
|
39
|
+
"sha256": "80521b40281d6ce74e35c9282c22539e75aa0ac8578892b2a59955ef78d55da1"
|
|
40
|
+
},
|
|
41
|
+
{
|
|
42
|
+
"path": "special_tokens_map.json",
|
|
43
|
+
"size_bytes": 831,
|
|
44
|
+
"sha256": "e786b595b9a23148bf1630df78d9037a048ea671e48bfd3549a1e3c233742bb3"
|
|
45
|
+
},
|
|
46
|
+
{
|
|
47
|
+
"path": "tokenizer.json",
|
|
48
|
+
"size_bytes": 2104556,
|
|
49
|
+
"sha256": "9ca9acddb6525a194ec8ac7a87f24fbba7232a9a15ffa1af0c1224fcd888e47c"
|
|
50
|
+
},
|
|
51
|
+
{
|
|
52
|
+
"path": "tokenizer_config.json",
|
|
53
|
+
"size_bytes": 3658,
|
|
54
|
+
"sha256": "4bb9af56a342753d39374f4016a16574cab299fe088e896f425ce3c433f61424"
|
|
55
|
+
}
|
|
56
|
+
]
|
|
57
|
+
}
|
|
58
|
+
],
|
|
59
|
+
"max_minutes": 30,
|
|
60
|
+
"max_spend_usd": 3,
|
|
61
|
+
"outputs": {
|
|
62
|
+
"files": [
|
|
63
|
+
"lora-acceptance.json",
|
|
64
|
+
"adapter_config.json",
|
|
65
|
+
"adapter_model.safetensors"
|
|
66
|
+
],
|
|
67
|
+
"max_bytes": 10000000,
|
|
68
|
+
"accept_storage_charges": false
|
|
69
|
+
},
|
|
70
|
+
"script": "causal-lora.sh",
|
|
71
|
+
"interpreter": "bash"
|
|
72
|
+
},
|
|
73
|
+
"comfy-wan-video": {
|
|
74
|
+
"gpu_model": "RTX 4090",
|
|
75
|
+
"gpu_count": 1,
|
|
76
|
+
"template": "flux-1-dev",
|
|
77
|
+
"max_minutes": 60,
|
|
78
|
+
"max_spend_usd": 2,
|
|
79
|
+
"outputs": {
|
|
80
|
+
"files": [
|
|
81
|
+
"video.mp4",
|
|
82
|
+
"video-acceptance.json"
|
|
83
|
+
],
|
|
84
|
+
"max_bytes": 100000000,
|
|
85
|
+
"accept_storage_charges": false
|
|
86
|
+
},
|
|
87
|
+
"script": "comfy-wan-video.py",
|
|
88
|
+
"interpreter": "python3"
|
|
89
|
+
}
|
|
90
|
+
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
import importlib.metadata as m,pathlib,torch
|
|
2
|
+
assert torch.cuda.is_available(),'CUDA image required; not installing or replacing torch'
|
|
3
|
+
p=pathlib.Path.home()/'cloudgpu-test-torch-constraints.txt';items=[]
|
|
4
|
+
for name in ['torch','torchvision','torchaudio']:
|
|
5
|
+
try:items.append(name+'=='+m.version(name))
|
|
6
|
+
except m.PackageNotFoundError:pass
|
|
7
|
+
p.write_text(chr(10).join(items)+chr(10))
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
import os,time,pathlib,json,subprocess,hashlib
|
|
2
|
+
started=time.monotonic()
|
|
3
|
+
import torch
|
|
4
|
+
if not torch.cuda.is_available():raise RuntimeError('CUDA required')
|
|
5
|
+
from safetensors.torch import load_file
|
|
6
|
+
OUT=pathlib.Path(os.environ['CLOUDGPU_OUTPUT_DIR']);OUT.mkdir(mode=0o700,parents=True,exist_ok=True)
|
|
7
|
+
data=pathlib.Path.cwd()/'acceptance-text.jsonl'
|
|
8
|
+
texts=['A small cloud GPU can run a bounded training task and save its results.','The synthetic dataset contains no customer information or private text.','A low rank adapter changes a small number of trainable parameters.','Always release the GPU after the task finishes and verify its cost.']
|
|
9
|
+
data.write_text(''.join(json.dumps({'text':text})+chr(10) for text in texts))
|
|
10
|
+
import importlib.util,json,math,os,pathlib,stat
|
|
11
|
+
|
|
12
|
+
def training_dtype(torch):
|
|
13
|
+
# Device capability alone does not guarantee the installed torch CUDA kernels.
|
|
14
|
+
if not torch.cuda.is_bf16_supported(): return torch.float32
|
|
15
|
+
try:
|
|
16
|
+
torch.triu(torch.ones((2,2),device='cuda',dtype=torch.bfloat16),diagonal=1)
|
|
17
|
+
except RuntimeError as error:
|
|
18
|
+
if 'not implemented' not in str(error): raise
|
|
19
|
+
return torch.float32
|
|
20
|
+
return torch.bfloat16
|
|
21
|
+
|
|
22
|
+
def run(options, project_file, output_dir):
|
|
23
|
+
# Reuse a reviewed image; do not reinstall CUDA/torch or run remote model code.
|
|
24
|
+
if any(importlib.util.find_spec(m) is None for m in ['torch','transformers','peft','safetensors']):
|
|
25
|
+
raise ValueError('Select an image with torch, transformers, peft and safetensors installed')
|
|
26
|
+
import torch
|
|
27
|
+
from transformers import AutoTokenizer,AutoModelForCausalLM
|
|
28
|
+
from peft import LoraConfig,get_peft_model
|
|
29
|
+
if not torch.cuda.is_available(): raise ValueError('A CUDA training environment is required')
|
|
30
|
+
source=project_file(options['dataset_path'],33554432)
|
|
31
|
+
texts=[]
|
|
32
|
+
with source.open(encoding='utf-8') as f:
|
|
33
|
+
for line in f:
|
|
34
|
+
if len(texts)>=1000: raise ValueError('Dataset exceeds 1000 records')
|
|
35
|
+
row=json.loads(line);text=row.get('text') if isinstance(row,dict) else None
|
|
36
|
+
if not isinstance(text,str) or not text.strip() or len(text)>8192: raise ValueError('Use JSONL records with a bounded text field')
|
|
37
|
+
texts.append(text)
|
|
38
|
+
if not texts: raise ValueError('Dataset is empty')
|
|
39
|
+
ref=options['model_reference']
|
|
40
|
+
model_root=pathlib.Path(os.environ['CLOUDGPU_MODEL_DIR']).resolve()/ref['name']
|
|
41
|
+
if not model_root.is_dir() or model_root.is_symlink(): raise ValueError('Pinned model directory is unavailable')
|
|
42
|
+
os.environ['HF_HUB_OFFLINE']='1';os.environ['TRANSFORMERS_OFFLINE']='1';os.environ['HF_HUB_DISABLE_TELEMETRY']='1'
|
|
43
|
+
torch.manual_seed(0)
|
|
44
|
+
tokenizer=AutoTokenizer.from_pretrained(str(model_root),local_files_only=True,trust_remote_code=False)
|
|
45
|
+
dtype=training_dtype(torch)
|
|
46
|
+
model=AutoModelForCausalLM.from_pretrained(str(model_root),local_files_only=True,trust_remote_code=False,torch_dtype=dtype).to('cuda')
|
|
47
|
+
model=get_peft_model(model,LoraConfig(r=options['rank'],lora_alpha=options['rank']*2,lora_dropout=0.05,bias='none',task_type='CAUSAL_LM',target_modules=options['target_modules']))
|
|
48
|
+
model.train();optimizer=torch.optim.AdamW((p for p in model.parameters() if p.requires_grad),lr=options['learning_rate'])
|
|
49
|
+
loss_value=None
|
|
50
|
+
for step in range(options['steps']):
|
|
51
|
+
batch=tokenizer(texts[step%len(texts)],return_tensors='pt',truncation=True,max_length=options['sequence_length'])
|
|
52
|
+
if batch['input_ids'].shape[-1]<2: raise ValueError('Training record has too few tokens')
|
|
53
|
+
batch={k:v.to('cuda') for k,v in batch.items()};batch['labels']=batch['input_ids'].clone()
|
|
54
|
+
optimizer.zero_grad(set_to_none=True);loss=model(**batch).loss
|
|
55
|
+
loss_value=float(loss.detach().float().cpu())
|
|
56
|
+
if not math.isfinite(loss_value): raise ValueError('Training loss is not finite')
|
|
57
|
+
loss.backward();torch.nn.utils.clip_grad_norm_(model.parameters(),1.0);optimizer.step()
|
|
58
|
+
model.peft_config['default'].base_model_name_or_path=ref['repo_id']
|
|
59
|
+
model.peft_config['default'].revision=ref['revision']
|
|
60
|
+
model.save_pretrained(str(output_dir),safe_serialization=True)
|
|
61
|
+
for name in ['adapter_config.json','adapter_model.safetensors']:
|
|
62
|
+
p=output_dir/name
|
|
63
|
+
if p.is_symlink() or not stat.S_ISREG(p.stat().st_mode) or p.stat().st_size<=0:
|
|
64
|
+
raise ValueError('Expected adapter output missing')
|
|
65
|
+
return {'task':'causal-lm-lora','steps_completed':options['steps'],'records':len(texts),'last_loss':loss_value,'base_model':ref['repo_id'],'revision':ref['revision'],'rank':options['rank'],'training_dtype':str(dtype)}
|
|
66
|
+
|
|
67
|
+
options={'dataset_path':str(data),'model_reference':{'name': 'smollm', 'repo_id': 'HuggingFaceTB/SmolLM2-135M', 'revision': '93efa2f097d58c2a74874c7e644dbc9b0cee75a2', 'files': [{'path': 'config.json', 'size_bytes': 704, 'sha256': '1d556eab73b69c7f11f64c557a2f9c6f440bd4c6b89bb2584a6b498c92603843'}, {'path': 'generation_config.json', 'size_bytes': 111, 'sha256': '2056c988e990b0d13670f63f2f3b87b3b6d07edaf7a3416998ba27dab2d8a059'}, {'path': 'model.safetensors', 'size_bytes': 269060552, 'sha256': '80521b40281d6ce74e35c9282c22539e75aa0ac8578892b2a59955ef78d55da1'}, {'path': 'special_tokens_map.json', 'size_bytes': 831, 'sha256': 'e786b595b9a23148bf1630df78d9037a048ea671e48bfd3549a1e3c233742bb3'}, {'path': 'tokenizer.json', 'size_bytes': 2104556, 'sha256': '9ca9acddb6525a194ec8ac7a87f24fbba7232a9a15ffa1af0c1224fcd888e47c'}, {'path': 'tokenizer_config.json', 'size_bytes': 3658, 'sha256': '4bb9af56a342753d39374f4016a16574cab299fe088e896f425ce3c433f61424'}]},'rank':4,'target_modules':['q_proj','v_proj'],'steps':8,'learning_rate':0.0002,'sequence_length':64}
|
|
68
|
+
def local_file(path,limit):
|
|
69
|
+
p=pathlib.Path(path).resolve();assert p==data.resolve() and p.is_file() and not p.is_symlink() and p.stat().st_size<=limit;return p
|
|
70
|
+
result=run(options,local_file,OUT)
|
|
71
|
+
weights=load_file(str(OUT/'adapter_model.safetensors'))
|
|
72
|
+
result['nonzero_trained_lora_B']=any('lora_B' in name and float(tensor.abs().sum())>0 for name,tensor in weights.items())
|
|
73
|
+
assert result['nonzero_trained_lora_B'],'adapter did not update'
|
|
74
|
+
result.update({'status':'passed','cuda_available':True,'gpu':torch.cuda.get_device_name(0),'training_and_save_seconds':round(time.monotonic()-started,6),'adapter_bytes':(OUT/'adapter_model.safetensors').stat().st_size,'scope':'8-step synthetic-text causal-LM LoRA smoke test; not image LoRA or a quality benchmark'})
|
|
75
|
+
(OUT/'lora-acceptance.json').write_text(json.dumps(result))
|
|
76
|
+
print('LORA_CUDA_ACCEPTANCE_OK '+json.dumps(result))
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
set -eu
|
|
2
|
+
python3 -c "$(printf '%s' 'aW1wb3J0IGltcG9ydGxpYi5tZXRhZGF0YSBhcyBtLHBhdGhsaWIsdG9yY2gKYXNzZXJ0IHRvcmNoLmN1ZGEuaXNfYXZhaWxhYmxlKCksJ0NVREEgaW1hZ2UgcmVxdWlyZWQ7IG5vdCBpbnN0YWxsaW5nIG9yIHJlcGxhY2luZyB0b3JjaCcKcD1wYXRobGliLlBhdGguaG9tZSgpLydjbG91ZGdwdS10ZXN0LXRvcmNoLWNvbnN0cmFpbnRzLnR4dCc7aXRlbXM9W10KZm9yIG5hbWUgaW4gWyd0b3JjaCcsJ3RvcmNodmlzaW9uJywndG9yY2hhdWRpbyddOgogdHJ5Oml0ZW1zLmFwcGVuZChuYW1lKyc9PScrbS52ZXJzaW9uKG5hbWUpKQogZXhjZXB0IG0uUGFja2FnZU5vdEZvdW5kRXJyb3I6cGFzcwpwLndyaXRlX3RleHQoY2hyKDEwKS5qb2luKGl0ZW1zKStjaHIoMTApKQo=' | base64 -d)"
|
|
3
|
+
if ! python3 -m pip install --no-input --disable-pip-version-check --no-cache-dir -c "$HOME/cloudgpu-test-torch-constraints.txt" transformers==4.46.3 peft==0.13.2 accelerate==1.0.1 huggingface-hub==0.26.5 safetensors==0.5.3 >"$HOME/cloudgpu-test-dependencies.log" 2>&1; then echo ACCEPTANCE_DEPENDENCY_SETUP_FAILED; exit 92; fi
|
|
4
|
+
python3 -c "$(printf '%s' 'aW1wb3J0IG9zLHRpbWUscGF0aGxpYixqc29uLHN1YnByb2Nlc3MsaGFzaGxpYgpzdGFydGVkPXRpbWUubW9ub3RvbmljKCkKaW1wb3J0IHRvcmNoCmlmIG5vdCB0b3JjaC5jdWRhLmlzX2F2YWlsYWJsZSgpOnJhaXNlIFJ1bnRpbWVFcnJvcignQ1VEQSByZXF1aXJlZCcpCmZyb20gc2FmZXRlbnNvcnMudG9yY2ggaW1wb3J0IGxvYWRfZmlsZQpPVVQ9cGF0aGxpYi5QYXRoKG9zLmVudmlyb25bJ0NMT1VER1BVX09VVFBVVF9ESVInXSk7T1VULm1rZGlyKG1vZGU9MG83MDAscGFyZW50cz1UcnVlLGV4aXN0X29rPVRydWUpCmRhdGE9cGF0aGxpYi5QYXRoLmN3ZCgpLydhY2NlcHRhbmNlLXRleHQuanNvbmwnCnRleHRzPVsnQSBzbWFsbCBjbG91ZCBHUFUgY2FuIHJ1biBhIGJvdW5kZWQgdHJhaW5pbmcgdGFzayBhbmQgc2F2ZSBpdHMgcmVzdWx0cy4nLCdUaGUgc3ludGhldGljIGRhdGFzZXQgY29udGFpbnMgbm8gY3VzdG9tZXIgaW5mb3JtYXRpb24gb3IgcHJpdmF0ZSB0ZXh0LicsJ0EgbG93IHJhbmsgYWRhcHRlciBjaGFuZ2VzIGEgc21hbGwgbnVtYmVyIG9mIHRyYWluYWJsZSBwYXJhbWV0ZXJzLicsJ0Fsd2F5cyByZWxlYXNlIHRoZSBHUFUgYWZ0ZXIgdGhlIHRhc2sgZmluaXNoZXMgYW5kIHZlcmlmeSBpdHMgY29zdC4nXQpkYXRhLndyaXRlX3RleHQoJycuam9pbihqc29uLmR1bXBzKHsndGV4dCc6dGV4dH0pK2NocigxMCkgZm9yIHRleHQgaW4gdGV4dHMpKQppbXBvcnQgaW1wb3J0bGliLnV0aWwsanNvbixtYXRoLG9zLHBhdGhsaWIsc3RhdAoKZGVmIHRyYWluaW5nX2R0eXBlKHRvcmNoKToKICAgICMgRGV2aWNlIGNhcGFiaWxpdHkgYWxvbmUgZG9lcyBub3QgZ3VhcmFudGVlIHRoZSBpbnN0YWxsZWQgdG9yY2ggQ1VEQSBrZXJuZWxzLgogICAgaWYgbm90IHRvcmNoLmN1ZGEuaXNfYmYxNl9zdXBwb3J0ZWQoKTogcmV0dXJuIHRvcmNoLmZsb2F0MzIKICAgIHRyeToKICAgICAgICB0b3JjaC50cml1KHRvcmNoLm9uZXMoKDIsMiksZGV2aWNlPSdjdWRhJyxkdHlwZT10b3JjaC5iZmxvYXQxNiksZGlhZ29uYWw9MSkKICAgIGV4Y2VwdCBSdW50aW1lRXJyb3IgYXMgZXJyb3I6CiAgICAgICAgaWYgJ25vdCBpbXBsZW1lbnRlZCcgbm90IGluIHN0cihlcnJvcik6IHJhaXNlCiAgICAgICAgcmV0dXJuIHRvcmNoLmZsb2F0MzIKICAgIHJldHVybiB0b3JjaC5iZmxvYXQxNgoKZGVmIHJ1bihvcHRpb25zLCBwcm9qZWN0X2ZpbGUsIG91dHB1dF9kaXIpOgogICAgIyBSZXVzZSBhIHJldmlld2VkIGltYWdlOyBkbyBub3QgcmVpbnN0YWxsIENVREEvdG9yY2ggb3IgcnVuIHJlbW90ZSBtb2RlbCBjb2RlLgogICAgaWYgYW55KGltcG9ydGxpYi51dGlsLmZpbmRfc3BlYyhtKSBpcyBOb25lIGZvciBtIGluIFsndG9yY2gnLCd0cmFuc2Zvcm1lcnMnLCdwZWZ0Jywnc2FmZXRlbnNvcnMnXSk6CiAgICAgICAgcmFpc2UgVmFsdWVFcnJvcignU2VsZWN0IGFuIGltYWdlIHdpdGggdG9yY2gsIHRyYW5zZm9ybWVycywgcGVmdCBhbmQgc2FmZXRlbnNvcnMgaW5zdGFsbGVkJykKICAgIGltcG9ydCB0b3JjaAogICAgZnJvbSB0cmFuc2Zvcm1lcnMgaW1wb3J0IEF1dG9Ub2tlbml6ZXIsQXV0b01vZGVsRm9yQ2F1c2FsTE0KICAgIGZyb20gcGVmdCBpbXBvcnQgTG9yYUNvbmZpZyxnZXRfcGVmdF9tb2RlbAogICAgaWYgbm90IHRvcmNoLmN1ZGEuaXNfYXZhaWxhYmxlKCk6IHJhaXNlIFZhbHVlRXJyb3IoJ0EgQ1VEQSB0cmFpbmluZyBlbnZpcm9ubWVudCBpcyByZXF1aXJlZCcpCiAgICBzb3VyY2U9cHJvamVjdF9maWxlKG9wdGlvbnNbJ2RhdGFzZXRfcGF0aCddLDMzNTU0NDMyKQogICAgdGV4dHM9W10KICAgIHdpdGggc291cmNlLm9wZW4oZW5jb2Rpbmc9J3V0Zi04JykgYXMgZjoKICAgICAgICBmb3IgbGluZSBpbiBmOgogICAgICAgICAgICBpZiBsZW4odGV4dHMpPj0xMDAwOiByYWlzZSBWYWx1ZUVycm9yKCdEYXRhc2V0IGV4Y2VlZHMgMTAwMCByZWNvcmRzJykKICAgICAgICAgICAgcm93PWpzb24ubG9hZHMobGluZSk7dGV4dD1yb3cuZ2V0KCd0ZXh0JykgaWYgaXNpbnN0YW5jZShyb3csZGljdCkgZWxzZSBOb25lCiAgICAgICAgICAgIGlmIG5vdCBpc2luc3RhbmNlKHRleHQsc3RyKSBvciBub3QgdGV4dC5zdHJpcCgpIG9yIGxlbih0ZXh0KT44MTkyOiByYWlzZSBWYWx1ZUVycm9yKCdVc2UgSlNPTkwgcmVjb3JkcyB3aXRoIGEgYm91bmRlZCB0ZXh0IGZpZWxkJykKICAgICAgICAgICAgdGV4dHMuYXBwZW5kKHRleHQpCiAgICBpZiBub3QgdGV4dHM6IHJhaXNlIFZhbHVlRXJyb3IoJ0RhdGFzZXQgaXMgZW1wdHknKQogICAgcmVmPW9wdGlvbnNbJ21vZGVsX3JlZmVyZW5jZSddCiAgICBtb2RlbF9yb290PXBhdGhsaWIuUGF0aChvcy5lbnZpcm9uWydDTE9VREdQVV9NT0RFTF9ESVInXSkucmVzb2x2ZSgpL3JlZlsnbmFtZSddCiAgICBpZiBub3QgbW9kZWxfcm9vdC5pc19kaXIoKSBvciBtb2RlbF9yb290LmlzX3N5bWxpbmsoKTogcmFpc2UgVmFsdWVFcnJvcignUGlubmVkIG1vZGVsIGRpcmVjdG9yeSBpcyB1bmF2YWlsYWJsZScpCiAgICBvcy5lbnZpcm9uWydIRl9IVUJfT0ZGTElORSddPScxJztvcy5lbnZpcm9uWydUUkFOU0ZPUk1FUlNfT0ZGTElORSddPScxJztvcy5lbnZpcm9uWydIRl9IVUJfRElTQUJMRV9URUxFTUVUUlknXT0nMScKICAgIHRvcmNoLm1hbnVhbF9zZWVkKDApCiAgICB0b2tlbml6ZXI9QXV0b1Rva2VuaXplci5mcm9tX3ByZXRyYWluZWQoc3RyKG1vZGVsX3Jvb3QpLGxvY2FsX2ZpbGVzX29ubHk9VHJ1ZSx0cnVzdF9yZW1vdGVfY29kZT1GYWxzZSkKICAgIGR0eXBlPXRyYWluaW5nX2R0eXBlKHRvcmNoKQogICAgbW9kZWw9QXV0b01vZGVsRm9yQ2F1c2FsTE0uZnJvbV9wcmV0cmFpbmVkKHN0cihtb2RlbF9yb290KSxsb2NhbF9maWxlc19vbmx5PVRydWUsdHJ1c3RfcmVtb3RlX2NvZGU9RmFsc2UsdG9yY2hfZHR5cGU9ZHR5cGUpLnRvKCdjdWRhJykKICAgIG1vZGVsPWdldF9wZWZ0X21vZGVsKG1vZGVsLExvcmFDb25maWcocj1vcHRpb25zWydyYW5rJ10sbG9yYV9hbHBoYT1vcHRpb25zWydyYW5rJ10qMixsb3JhX2Ryb3BvdXQ9MC4wNSxiaWFzPSdub25lJyx0YXNrX3R5cGU9J0NBVVNBTF9MTScsdGFyZ2V0X21vZHVsZXM9b3B0aW9uc1sndGFyZ2V0X21vZHVsZXMnXSkpCiAgICBtb2RlbC50cmFpbigpO29wdGltaXplcj10b3JjaC5vcHRpbS5BZGFtVygocCBmb3IgcCBpbiBtb2RlbC5wYXJhbWV0ZXJzKCkgaWYgcC5yZXF1aXJlc19ncmFkKSxscj1vcHRpb25zWydsZWFybmluZ19yYXRlJ10pCiAgICBsb3NzX3ZhbHVlPU5vbmUKICAgIGZvciBzdGVwIGluIHJhbmdlKG9wdGlvbnNbJ3N0ZXBzJ10pOgogICAgICAgIGJhdGNoPXRva2VuaXplcih0ZXh0c1tzdGVwJWxlbih0ZXh0cyldLHJldHVybl90ZW5zb3JzPSdwdCcsdHJ1bmNhdGlvbj1UcnVlLG1heF9sZW5ndGg9b3B0aW9uc1snc2VxdWVuY2VfbGVuZ3RoJ10pCiAgICAgICAgaWYgYmF0Y2hbJ2lucHV0X2lkcyddLnNoYXBlWy0xXTwyOiByYWlzZSBWYWx1ZUVycm9yKCdUcmFpbmluZyByZWNvcmQgaGFzIHRvbyBmZXcgdG9rZW5zJykKICAgICAgICBiYXRjaD17azp2LnRvKCdjdWRhJykgZm9yIGssdiBpbiBiYXRjaC5pdGVtcygpfTtiYXRjaFsnbGFiZWxzJ109YmF0Y2hbJ2lucHV0X2lkcyddLmNsb25lKCkKICAgICAgICBvcHRpbWl6ZXIuemVyb19ncmFkKHNldF90b19ub25lPVRydWUpO2xvc3M9bW9kZWwoKipiYXRjaCkubG9zcwogICAgICAgIGxvc3NfdmFsdWU9ZmxvYXQobG9zcy5kZXRhY2goKS5mbG9hdCgpLmNwdSgpKQogICAgICAgIGlmIG5vdCBtYXRoLmlzZmluaXRlKGxvc3NfdmFsdWUpOiByYWlzZSBWYWx1ZUVycm9yKCdUcmFpbmluZyBsb3NzIGlzIG5vdCBmaW5pdGUnKQogICAgICAgIGxvc3MuYmFja3dhcmQoKTt0b3JjaC5ubi51dGlscy5jbGlwX2dyYWRfbm9ybV8obW9kZWwucGFyYW1ldGVycygpLDEuMCk7b3B0aW1pemVyLnN0ZXAoKQogICAgbW9kZWwucGVmdF9jb25maWdbJ2RlZmF1bHQnXS5iYXNlX21vZGVsX25hbWVfb3JfcGF0aD1yZWZbJ3JlcG9faWQnXQogICAgbW9kZWwucGVmdF9jb25maWdbJ2RlZmF1bHQnXS5yZXZpc2lvbj1yZWZbJ3JldmlzaW9uJ10KICAgIG1vZGVsLnNhdmVfcHJldHJhaW5lZChzdHIob3V0cHV0X2Rpciksc2FmZV9zZXJpYWxpemF0aW9uPVRydWUpCiAgICBmb3IgbmFtZSBpbiBbJ2FkYXB0ZXJfY29uZmlnLmpzb24nLCdhZGFwdGVyX21vZGVsLnNhZmV0ZW5zb3JzJ106CiAgICAgICAgcD1vdXRwdXRfZGlyL25hbWUKICAgICAgICBpZiBwLmlzX3N5bWxpbmsoKSBvciBub3Qgc3RhdC5TX0lTUkVHKHAuc3RhdCgpLnN0X21vZGUpIG9yIHAuc3RhdCgpLnN0X3NpemU8PTA6CiAgICAgICAgICAgIHJhaXNlIFZhbHVlRXJyb3IoJ0V4cGVjdGVkIGFkYXB0ZXIgb3V0cHV0IG1pc3NpbmcnKQogICAgcmV0dXJuIHsndGFzayc6J2NhdXNhbC1sbS1sb3JhJywnc3RlcHNfY29tcGxldGVkJzpvcHRpb25zWydzdGVwcyddLCdyZWNvcmRzJzpsZW4odGV4dHMpLCdsYXN0X2xvc3MnOmxvc3NfdmFsdWUsJ2Jhc2VfbW9kZWwnOnJlZlsncmVwb19pZCddLCdyZXZpc2lvbic6cmVmWydyZXZpc2lvbiddLCdyYW5rJzpvcHRpb25zWydyYW5rJ10sJ3RyYWluaW5nX2R0eXBlJzpzdHIoZHR5cGUpfQoKb3B0aW9ucz17J2RhdGFzZXRfcGF0aCc6c3RyKGRhdGEpLCdtb2RlbF9yZWZlcmVuY2UnOnsnbmFtZSc6ICdzbW9sbG0nLCAncmVwb19pZCc6ICdIdWdnaW5nRmFjZVRCL1Ntb2xMTTItMTM1TScsICdyZXZpc2lvbic6ICc5M2VmYTJmMDk3ZDU4YzJhNzQ4NzRjN2U2NDRkYmM5YjBjZWU3NWEyJywgJ2ZpbGVzJzogW3sncGF0aCc6ICdjb25maWcuanNvbicsICdzaXplX2J5dGVzJzogNzA0LCAnc2hhMjU2JzogJzFkNTU2ZWFiNzNiNjljN2YxMWY2NGM1NTdhMmY5YzZmNDQwYmQ0YzZiODliYjI1ODRhNmI0OThjOTI2MDM4NDMnfSwgeydwYXRoJzogJ2dlbmVyYXRpb25fY29uZmlnLmpzb24nLCAnc2l6ZV9ieXRlcyc6IDExMSwgJ3NoYTI1Nic6ICcyMDU2Yzk4OGU5OTBiMGQxMzY3MGY2M2YyZjNiODdiM2I2ZDA3ZWRhZjdhMzQxNjk5OGJhMjdkYWIyZDhhMDU5J30sIHsncGF0aCc6ICdtb2RlbC5zYWZldGVuc29ycycsICdzaXplX2J5dGVzJzogMjY5MDYwNTUyLCAnc2hhMjU2JzogJzgwNTIxYjQwMjgxZDZjZTc0ZTM1YzkyODJjMjI1MzllNzVhYTBhYzg1Nzg4OTJiMmE1OTk1NWVmNzhkNTVkYTEnfSwgeydwYXRoJzogJ3NwZWNpYWxfdG9rZW5zX21hcC5qc29uJywgJ3NpemVfYnl0ZXMnOiA4MzEsICdzaGEyNTYnOiAnZTc4NmI1OTViOWEyMzE0OGJmMTYzMGRmNzhkOTAzN2EwNDhlYTY3MWU0OGJmZDM1NDlhMWUzYzIzMzc0MmJiMyd9LCB7J3BhdGgnOiAndG9rZW5pemVyLmpzb24nLCAnc2l6ZV9ieXRlcyc6IDIxMDQ1NTYsICdzaGEyNTYnOiAnOWNhOWFjZGRiNjUyNWExOTRlYzhhYzdhODdmMjRmYmJhNzIzMmE5YTE1ZmZhMWFmMGMxMjI0ZmNkODg4ZTQ3Yyd9LCB7J3BhdGgnOiAndG9rZW5pemVyX2NvbmZpZy5qc29uJywgJ3NpemVfYnl0ZXMnOiAzNjU4LCAnc2hhMjU2JzogJzRiYjlhZjU2YTM0Mjc1M2QzOTM3NGY0MDE2YTE2NTc0Y2FiMjk5ZmUwODhlODk2ZjQyNWNlM2M0MzNmNjE0MjQnfV19LCdyYW5rJzo0LCd0YXJnZXRfbW9kdWxlcyc6WydxX3Byb2onLCd2X3Byb2onXSwnc3RlcHMnOjgsJ2xlYXJuaW5nX3JhdGUnOjAuMDAwMiwnc2VxdWVuY2VfbGVuZ3RoJzo2NH0KZGVmIGxvY2FsX2ZpbGUocGF0aCxsaW1pdCk6CiBwPXBhdGhsaWIuUGF0aChwYXRoKS5yZXNvbHZlKCk7YXNzZXJ0IHA9PWRhdGEucmVzb2x2ZSgpIGFuZCBwLmlzX2ZpbGUoKSBhbmQgbm90IHAuaXNfc3ltbGluaygpIGFuZCBwLnN0YXQoKS5zdF9zaXplPD1saW1pdDtyZXR1cm4gcApyZXN1bHQ9cnVuKG9wdGlvbnMsbG9jYWxfZmlsZSxPVVQpCndlaWdodHM9bG9hZF9maWxlKHN0cihPVVQvJ2FkYXB0ZXJfbW9kZWwuc2FmZXRlbnNvcnMnKSkKcmVzdWx0Wydub256ZXJvX3RyYWluZWRfbG9yYV9CJ109YW55KCdsb3JhX0InIGluIG5hbWUgYW5kIGZsb2F0KHRlbnNvci5hYnMoKS5zdW0oKSk+MCBmb3IgbmFtZSx0ZW5zb3IgaW4gd2VpZ2h0cy5pdGVtcygpKQphc3NlcnQgcmVzdWx0Wydub256ZXJvX3RyYWluZWRfbG9yYV9CJ10sJ2FkYXB0ZXIgZGlkIG5vdCB1cGRhdGUnCnJlc3VsdC51cGRhdGUoeydzdGF0dXMnOidwYXNzZWQnLCdjdWRhX2F2YWlsYWJsZSc6VHJ1ZSwnZ3B1Jzp0b3JjaC5jdWRhLmdldF9kZXZpY2VfbmFtZSgwKSwndHJhaW5pbmdfYW5kX3NhdmVfc2Vjb25kcyc6cm91bmQodGltZS5tb25vdG9uaWMoKS1zdGFydGVkLDYpLCdhZGFwdGVyX2J5dGVzJzooT1VULydhZGFwdGVyX21vZGVsLnNhZmV0ZW5zb3JzJykuc3RhdCgpLnN0X3NpemUsJ3Njb3BlJzonOC1zdGVwIHN5bnRoZXRpYy10ZXh0IGNhdXNhbC1MTSBMb1JBIHNtb2tlIHRlc3Q7IG5vdCBpbWFnZSBMb1JBIG9yIGEgcXVhbGl0eSBiZW5jaG1hcmsnfSkKKE9VVC8nbG9yYS1hY2NlcHRhbmNlLmpzb24nKS53cml0ZV90ZXh0KGpzb24uZHVtcHMocmVzdWx0KSkKcHJpbnQoJ0xPUkFfQ1VEQV9BQ0NFUFRBTkNFX09LICcranNvbi5kdW1wcyhyZXN1bHQpKQo=' | base64 -d)"
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
import os,pathlib,time,json,hashlib,urllib.request,urllib.error,urllib.parse,concurrent.futures,shutil,threading
|
|
2
|
+
_source={'modelscope_repo': 'Comfy-Org/Wan_2.1_ComfyUI_repackaged', 'files': [{'path': 'split_files/diffusion_models/wan2.1_t2v_1.3B_fp16.safetensors', 'size_bytes': 2838303560, 'sha256': 'be531024cd9018cb5b48c40cfbb6a6191645b1c792eb8bf4f8c1c6e10f924dc5', 'modelscope_revision': '462e77efa4287219475d8910e0ebb7bfb65313da'}, {'path': 'split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors', 'size_bytes': 6735906897, 'sha256': 'c3355d30191f1f066b26d93fba017ae9809dce6c627dda5f6a66eaa651204f68', 'modelscope_revision': '28eb857b47db293db0455bfba5bdefdcafa4036c'}, {'path': 'split_files/vae/wan_2.1_vae.safetensors', 'size_bytes': 253815318, 'sha256': '2fc39d31359a4b0a64f55876d8ff7fa8d780956ae2cb13463b0223e15148976b', 'modelscope_revision': '28eb857b47db293db0455bfba5bdefdcafa4036c'}], 'trusted_redirect_hosts': ['modelscope.cn'], 'all_three_independent_hashes_match': True, 'probe_http': 206, 'probe_bytes': 1048576, 'probe_seconds': 1.156, 'content_length': '1048576'}
|
|
3
|
+
# Owner QA only: validate the existing image before the large model transfer.
|
|
4
|
+
import subprocess,zipfile,io,sys
|
|
5
|
+
_qa_candidates=[]
|
|
6
|
+
for _proc in pathlib.Path('/proc').iterdir():
|
|
7
|
+
if not _proc.name.isdigit():continue
|
|
8
|
+
try:
|
|
9
|
+
_args=(_proc/'cmdline').read_bytes().split(b'\0');_cwd=(_proc/'cwd').resolve()
|
|
10
|
+
if b'main.py'in b' '.join(_args)and(_cwd/'comfy').is_dir()and(_cwd/'main.py').is_file():
|
|
11
|
+
_py=pathlib.Path(os.fsdecode(_args[0]))if _args and os.path.isabs(os.fsdecode(_args[0]))and pathlib.Path(os.fsdecode(_args[0])).is_file()else(_proc/'exe').resolve();_qa_candidates.append((_cwd,_py))
|
|
12
|
+
except(OSError,RuntimeError):pass
|
|
13
|
+
if not _qa_candidates:
|
|
14
|
+
for _p in ['/root/ComfyUI','/root/autodl-tmp/ComfyUI','/workspace/ComfyUI']:
|
|
15
|
+
if(pathlib.Path(_p)/'main.py').is_file():_qa_candidates.append((pathlib.Path(_p),pathlib.Path(sys.executable)));break
|
|
16
|
+
if not _qa_candidates:raise RuntimeError('ComfyUI image preflight failed')
|
|
17
|
+
_qa_cwd,_qa_python=_qa_candidates[0]
|
|
18
|
+
_check=subprocess.run([str(_qa_python),'-c','import torch,PIL,comfy;assert torch.cuda.is_available();print("QA_COMFY_TORCH_READY")'],cwd=str(_qa_cwd),capture_output=True,text=True,timeout=60)
|
|
19
|
+
if _check.returncode:raise RuntimeError('Existing ComfyUI interpreter preflight failed')
|
|
20
|
+
_qa_bin=pathlib.Path.cwd()/'qa-reviewed-bin';_qa_bin.mkdir(mode=0o700,exist_ok=True)
|
|
21
|
+
_qa_ffmpeg=_qa_bin/'ffmpeg'
|
|
22
|
+
if not shutil.which('ffmpeg'):
|
|
23
|
+
_wheel_url='https://files.pythonhosted.org/packages/a0/2d/43c8522a2038e9d0e7dbdf3a61195ecc31ca576fb1527a528c877e87d973/imageio_ffmpeg-0.6.0-py3-none-manylinux2014_x86_64.whl'
|
|
24
|
+
_wheel_url=_wheel_url.replace('https://files.pythonhosted.org/packages/','https://pypi.tuna.tsinghua.edu.cn/packages/')
|
|
25
|
+
_encoder_deadline=time.monotonic()+120
|
|
26
|
+
def _encoder_range(start):
|
|
27
|
+
end=min(29498236,start+4*1024*1024-1);request=urllib.request.Request(_wheel_url,headers={'Range':'bytes='+str(start)+'-'+str(end)})
|
|
28
|
+
with urllib.request.urlopen(request,timeout=15)as r:
|
|
29
|
+
host=urllib.parse.urlsplit(r.geturl()).hostname
|
|
30
|
+
if not host or not(host=='tuna.tsinghua.edu.cn'or host.endswith('.tuna.tsinghua.edu.cn'))or r.status!=206 or r.headers.get('Content-Range')!='bytes '+str(start)+'-'+str(end)+'/29498237':raise RuntimeError('Encoder range source changed')
|
|
31
|
+
data=bytearray()
|
|
32
|
+
while len(data)<end-start+1:
|
|
33
|
+
if time.monotonic()>_encoder_deadline:raise TimeoutError('Encoder download deadline')
|
|
34
|
+
chunk=r.read1(min(1024**2,end-start+1-len(data)))
|
|
35
|
+
if not chunk:raise RuntimeError('Incomplete encoder range')
|
|
36
|
+
data.extend(chunk)
|
|
37
|
+
return bytes(data)
|
|
38
|
+
with concurrent.futures.ThreadPoolExecutor(max_workers=4)as _pool:_wheel=b''.join(_pool.map(_encoder_range,range(0,29498237,4*1024*1024)))
|
|
39
|
+
if len(_wheel)!=29498237 or hashlib.sha256(_wheel).hexdigest()!='c7e46fcec401dd990405049d2e2f475e2b397779df2519b544b8aab515195282':raise RuntimeError('Encoder wheel integrity failed')
|
|
40
|
+
with zipfile.ZipFile(io.BytesIO(_wheel))as _z:_binary=_z.read('imageio_ffmpeg/binaries/ffmpeg-linux-x86_64-v7.0.2')
|
|
41
|
+
if hashlib.sha256(_binary).hexdigest()!='e7e7fb30477f717e6f55f9180a70386c62677ef8a4d4d1a5d948f4098aa3eb99':raise RuntimeError('Encoder binary integrity failed')
|
|
42
|
+
_qa_ffmpeg.write_bytes(_binary);_qa_ffmpeg.chmod(0o700);os.environ['PATH']=str(_qa_bin)+os.pathsep+os.environ.get('PATH','')
|
|
43
|
+
subprocess.run([shutil.which('ffmpeg'),'-version'],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,check=True,timeout=10)
|
|
44
|
+
print('VIDEO_IMAGE_PREFLIGHT_OK '+json.dumps({'comfy_interpreter_checked':True,'cuda_available':True,'encoder_verified':True}),flush=True)
|
|
45
|
+
|
|
46
|
+
_started=time.monotonic();_deadline=_started+2700;_root=pathlib.Path.cwd()/'video-modelscope-parallel';_root.mkdir(mode=0o700,exist_ok=False);_stop=threading.Event()
|
|
47
|
+
_required=sum(f['size_bytes']for f in _source['files']);_fds={};_counts={};_lock=threading.Lock();_last_print={}
|
|
48
|
+
if shutil.disk_usage(_root).free<_required+1024**3:raise RuntimeError('insufficient_model_disk')
|
|
49
|
+
def _trusted(url):
|
|
50
|
+
p=urllib.parse.urlsplit(url)
|
|
51
|
+
return p.scheme=='https'and(p.hostname=='modelscope.cn'or(p.hostname or'').endswith('.modelscope.cn'))and p.port in(None,443)and not p.username and not p.password
|
|
52
|
+
class _Redirect(urllib.request.HTTPRedirectHandler):
|
|
53
|
+
def redirect_request(self,req,fp,code,msg,headers,url):
|
|
54
|
+
if not _trusted(url):
|
|
55
|
+
print('VIDEO_REJECTED_REDIRECT '+json.dumps({'host':urllib.parse.urlsplit(url).hostname,'scheme':urllib.parse.urlsplit(url).scheme}),flush=True);raise ValueError('untrusted_redirect')
|
|
56
|
+
return super().redirect_request(req,fp,code,msg,headers,url)
|
|
57
|
+
def _range(index,entry,start,end):
|
|
58
|
+
expected=end-start+1;received=0;attempt=0
|
|
59
|
+
while received<expected:
|
|
60
|
+
try:
|
|
61
|
+
if _stop.is_set()or time.monotonic()>_deadline:raise TimeoutError()
|
|
62
|
+
offset=start+received;url='https://www.modelscope.cn/api/v1/models/'+_source['modelscope_repo']+'/repo?'+urllib.parse.urlencode({'Revision':entry['modelscope_revision'],'FilePath':entry['path']})
|
|
63
|
+
request=urllib.request.Request(url,headers={'User-Agent':'CloudGPU-Verified-Public-Model/1','Range':'bytes='+str(offset)+'-'+str(end)})
|
|
64
|
+
with urllib.request.build_opener(_Redirect()).open(request,timeout=15)as response:
|
|
65
|
+
if not _trusted(response.geturl())or response.status!=206:raise ValueError('range_response_required')
|
|
66
|
+
if response.headers.get('Content-Range')!='bytes '+str(offset)+'-'+str(end)+'/'+str(entry['size_bytes']):raise ValueError('range_identity_mismatch')
|
|
67
|
+
while received<expected:
|
|
68
|
+
if _stop.is_set()or time.monotonic()>_deadline:raise TimeoutError()
|
|
69
|
+
chunk=response.read1(min(1024**2,expected-received))
|
|
70
|
+
if not chunk:raise OSError('incomplete_range')
|
|
71
|
+
n=os.pwrite(_fds[index],chunk,start+received)
|
|
72
|
+
if n!=len(chunk):raise OSError('partial_disk_write')
|
|
73
|
+
received+=n
|
|
74
|
+
with _lock:
|
|
75
|
+
_counts[index]+=n;now=time.monotonic()
|
|
76
|
+
if now-_last_print[index]>=20:
|
|
77
|
+
print('VIDEO_MODEL_PROGRESS '+json.dumps({'source':'modelscope_parallel_range','file_index':index,'bytes':_counts[index],'elapsed_seconds':round(now-_started,3)}),flush=True);_last_print[index]=now
|
|
78
|
+
if received!=expected:raise OSError('incomplete_range')
|
|
79
|
+
except Exception as error:
|
|
80
|
+
attempt+=1
|
|
81
|
+
if attempt>=3 or isinstance(error,(ValueError,TimeoutError)):
|
|
82
|
+
_stop.set();code='http_'+str(error.code)if isinstance(error,urllib.error.HTTPError)else(str(error)if isinstance(error,ValueError)else type(error).__name__)
|
|
83
|
+
print('VIDEO_MODEL_FAILURE '+json.dumps({'file_index':index,'failure_code':code}),flush=True);raise RuntimeError('verified_range_download_failed')from None
|
|
84
|
+
jobs=[];paths=[]
|
|
85
|
+
try:
|
|
86
|
+
for index,entry in enumerate(_source['files']):
|
|
87
|
+
path=_root/'wan'/entry['path'];path.parent.mkdir(mode=0o700,parents=True,exist_ok=True);_counts[index]=0;_last_print[index]=time.monotonic();paths.append(path)
|
|
88
|
+
cached=False
|
|
89
|
+
for base in ['/root/ComfyUI/models','/root/autodl-tmp/ComfyUI/models','/workspace/ComfyUI/models']:
|
|
90
|
+
folder=pathlib.Path(base)
|
|
91
|
+
if not folder.is_dir():continue
|
|
92
|
+
for candidate in list(folder.glob('**/'+path.name))[:10]:
|
|
93
|
+
if not candidate.is_file()or candidate.is_symlink()or candidate.stat().st_size!=entry['size_bytes']:continue
|
|
94
|
+
digest=hashlib.sha256()
|
|
95
|
+
with candidate.open('rb')as stream:
|
|
96
|
+
for chunk in iter(lambda:stream.read(4*1024**2),b''):digest.update(chunk)
|
|
97
|
+
if digest.hexdigest()==entry['sha256']:
|
|
98
|
+
shutil.copyfile(candidate,path);path.chmod(0o600);_counts[index]=entry['size_bytes'];cached=True
|
|
99
|
+
print('VIDEO_PROVIDER_CACHE_MATCH '+json.dumps({'file_index':index,'bytes':entry['size_bytes'],'sha256':entry['sha256']}),flush=True);break
|
|
100
|
+
if cached:break
|
|
101
|
+
if cached:continue
|
|
102
|
+
fd=os.open(path,os.O_RDWR|os.O_CREAT|os.O_EXCL,0o600);os.ftruncate(fd,entry['size_bytes']);_fds[index]=fd
|
|
103
|
+
for start in range(0,entry['size_bytes'],128*1024**2):jobs.append((index,entry,start,min(entry['size_bytes']-1,start+128*1024**2-1)))
|
|
104
|
+
with concurrent.futures.ThreadPoolExecutor(max_workers=8)as pool:list(pool.map(lambda args:_range(*args),jobs))
|
|
105
|
+
for index,(entry,path)in enumerate(zip(_source['files'],paths)):
|
|
106
|
+
digest=hashlib.sha256()
|
|
107
|
+
with path.open('rb')as stream:
|
|
108
|
+
for chunk in iter(lambda:stream.read(4*1024**2),b''):digest.update(chunk)
|
|
109
|
+
if digest.hexdigest()!=entry['sha256']or _counts[index]!=entry['size_bytes']:raise RuntimeError('full_model_integrity_mismatch')
|
|
110
|
+
print('VIDEO_MODEL_READY '+json.dumps({'file_index':index,'bytes':entry['size_bytes'],'sha256':entry['sha256']}),flush=True)
|
|
111
|
+
finally:
|
|
112
|
+
for fd in _fds.values():os.close(fd)
|
|
113
|
+
os.environ['CLOUDGPU_MODEL_DIR']=str(_root)
|
|
114
|
+
print('VIDEO_ALL_MODELS_READY '+json.dumps({'bytes':_required,'seconds':round(time.monotonic()-_started,3),'independent_hashes_verified':True,'source':'modelscope_parallel_range'}),flush=True)
|
|
115
|
+
|
|
116
|
+
import os,pathlib,subprocess,time,json,urllib.request,uuid,hashlib,shutil,sys
|
|
117
|
+
started=time.monotonic();out=pathlib.Path(os.environ['CLOUDGPU_OUTPUT_DIR']);out.mkdir(parents=True,exist_ok=True)
|
|
118
|
+
modelroot=pathlib.Path(os.environ['CLOUDGPU_MODEL_DIR'])/'wan'/'split_files'
|
|
119
|
+
candidates=[]
|
|
120
|
+
for p in pathlib.Path('/proc').iterdir():
|
|
121
|
+
if p.name.isdigit():
|
|
122
|
+
try:
|
|
123
|
+
args=(p/'cmdline').read_bytes().split(b'\0');cwd=(p/'cwd').resolve()
|
|
124
|
+
if b'main.py' in b' '.join(args) and (cwd/'comfy').is_dir() and (cwd/'main.py').is_file():candidates.append((cwd,pathlib.Path(os.fsdecode(args[0])) if args and os.path.isabs(os.fsdecode(args[0])) and pathlib.Path(os.fsdecode(args[0])).is_file() else (p/'exe').resolve()))
|
|
125
|
+
except (OSError,RuntimeError):pass
|
|
126
|
+
for p in ['/root/ComfyUI','/root/autodl-tmp/ComfyUI','/workspace/ComfyUI']:
|
|
127
|
+
root=pathlib.Path(p)
|
|
128
|
+
if (root/'main.py').is_file() and not candidates:candidates.append((root,pathlib.Path(sys.executable)))
|
|
129
|
+
if not candidates:raise RuntimeError('Reviewed ComfyUI installation missing; no remote installation performed')
|
|
130
|
+
root,python=candidates[0]
|
|
131
|
+
ffmpeg=shutil.which('ffmpeg')
|
|
132
|
+
if not ffmpeg:
|
|
133
|
+
result=subprocess.run([str(python),'-c','import imageio_ffmpeg;print(imageio_ffmpeg.get_ffmpeg_exe())'],capture_output=True,text=True,timeout=10)
|
|
134
|
+
if result.returncode==0 and pathlib.Path(result.stdout.strip()).is_file():ffmpeg=result.stdout.strip()
|
|
135
|
+
if not ffmpeg:raise RuntimeError('MP4 encoder missing; no unreviewed software installed')
|
|
136
|
+
work=pathlib.Path.cwd()/'comfy-video-acceptance';work.mkdir(mode=0o700,exist_ok=True)
|
|
137
|
+
config=work/'models.yaml';config.write_text('acceptance:\n base_path: '+str(modelroot)+'\n diffusion_models: diffusion_models\n text_encoders: text_encoders\n vae: vae\n')
|
|
138
|
+
client=urllib.request.build_opener(urllib.request.ProxyHandler({}));base='http://127.0.0.1:8189'
|
|
139
|
+
def request(path,body=None,limit=16*1024*1024):
|
|
140
|
+
data=None if body is None else json.dumps(body).encode()
|
|
141
|
+
with client.open(urllib.request.Request(base+path,data=data,headers={'Content-Type':'application/json'}),timeout=20) as r:
|
|
142
|
+
value=r.read(limit+1)
|
|
143
|
+
if len(value)>limit:raise RuntimeError('Application response exceeds bound')
|
|
144
|
+
return json.loads(value)
|
|
145
|
+
log=(work/'server.log').open('wb')
|
|
146
|
+
proc=subprocess.Popen([str(python),str(root/'main.py'),'--listen','127.0.0.1','--port','8189','--extra-model-paths-config',str(config),'--output-directory',str(work/'frames'),'--disable-auto-launch'],cwd=str(root),stdout=log,stderr=log)
|
|
147
|
+
try:
|
|
148
|
+
deadline=time.monotonic()+150
|
|
149
|
+
while True:
|
|
150
|
+
if proc.poll() is not None:raise RuntimeError('Isolated ComfyUI process exited during startup')
|
|
151
|
+
try:info=request('/object_info');break
|
|
152
|
+
except (OSError,ValueError):
|
|
153
|
+
if time.monotonic()>deadline:raise RuntimeError('Isolated ComfyUI startup deadline')
|
|
154
|
+
time.sleep(2)
|
|
155
|
+
graph={
|
|
156
|
+
'1':{'class_type':'UNETLoader','inputs':{'unet_name':'wan2.1_t2v_1.3B_fp16.safetensors','weight_dtype':'default'}},
|
|
157
|
+
'2':{'class_type':'CLIPLoader','inputs':{'clip_name':'umt5_xxl_fp8_e4m3fn_scaled.safetensors','type':'wan','device':'default'}},
|
|
158
|
+
'3':{'class_type':'VAELoader','inputs':{'vae_name':'wan_2.1_vae.safetensors'}},
|
|
159
|
+
'4':{'class_type':'CLIPTextEncode','inputs':{'text':'A small paper boat floating gently on a calm lake, daylight, peaceful landscape','clip':['2',0]}},
|
|
160
|
+
'5':{'class_type':'CLIPTextEncode','inputs':{'text':'blurry, low quality, text, watermark','clip':['2',0]}},
|
|
161
|
+
'6':{'class_type':'EmptyHunyuanLatentVideo','inputs':{'width':256,'height':256,'length':17,'batch_size':1}},
|
|
162
|
+
'7':{'class_type':'ModelSamplingSD3','inputs':{'model':['1',0],'shift':8.0}},
|
|
163
|
+
'8':{'class_type':'KSampler','inputs':{'model':['7',0],'positive':['4',0],'negative':['5',0],'latent_image':['6',0],'seed':20261113,'steps':8,'cfg':5.0,'sampler_name':'euler','scheduler':'simple','denoise':1.0}},
|
|
164
|
+
'9':{'class_type':'VAEDecode','inputs':{'samples':['8',0],'vae':['3',0]}},
|
|
165
|
+
'10':{'class_type':'SaveImage','inputs':{'images':['9',0],'filename_prefix':'acceptance_video'}}}
|
|
166
|
+
for node in graph.values():
|
|
167
|
+
if node['class_type'] not in info:raise RuntimeError('Required reviewed core node unavailable: '+node['class_type'])
|
|
168
|
+
schema=info[node['class_type']]['input']
|
|
169
|
+
if any(k not in node['inputs'] for k in schema.get('required',{})):raise RuntimeError('Required node input changed')
|
|
170
|
+
for key,value in node['inputs'].items():
|
|
171
|
+
rule=schema.get('required',{}).get(key) or schema.get('optional',{}).get(key)
|
|
172
|
+
if not rule:raise RuntimeError('Unknown core input')
|
|
173
|
+
if isinstance(rule[0],list) and value not in rule[0]:raise RuntimeError('Model or enum missing from isolated model paths')
|
|
174
|
+
queued=request('/prompt',{'prompt':graph,'client_id':uuid.uuid4().hex});pid=queued.get('prompt_id')
|
|
175
|
+
if not pid or queued.get('node_errors'):raise RuntimeError('Reviewed video workflow was not accepted')
|
|
176
|
+
deadline=time.monotonic()+300
|
|
177
|
+
while True:
|
|
178
|
+
history=request('/history/'+pid).get(pid)
|
|
179
|
+
if history:
|
|
180
|
+
status=history.get('status',{})
|
|
181
|
+
if status.get('status_str')=='error':raise RuntimeError('ComfyUI video workflow failed')
|
|
182
|
+
if status.get('completed') is True:break
|
|
183
|
+
if time.monotonic()>deadline:raise RuntimeError('Video generation deadline')
|
|
184
|
+
time.sleep(2)
|
|
185
|
+
images=history.get('outputs',{}).get('10',{}).get('images',[])
|
|
186
|
+
if len(images)!=17:raise RuntimeError('Expected exactly 17 video frames')
|
|
187
|
+
# Encode only files listed by this unique prompt; never collect arbitrary GPU files.
|
|
188
|
+
frame_dir=work/'encode';frame_dir.mkdir(mode=0o700)
|
|
189
|
+
for i,image in enumerate(images):
|
|
190
|
+
import urllib.parse
|
|
191
|
+
query=urllib.parse.urlencode({'filename':image['filename'],'subfolder':image.get('subfolder',''),'type':'output'})
|
|
192
|
+
with client.open(base+'/view?'+query,timeout=20) as r:data=r.read(4*1024*1024+1)
|
|
193
|
+
if len(data)>4*1024*1024 or not data.startswith(b'\x89PNG\r\n\x1a\n'):raise RuntimeError('Invalid generated frame')
|
|
194
|
+
(frame_dir/('%04d.png'%i)).write_bytes(data)
|
|
195
|
+
subprocess.run([ffmpeg,'-v','error','-y','-framerate','8','-i',str(frame_dir/'%04d.png'),'-c:v','libx264','-pix_fmt','yuv420p',str(out/'video.mp4')],check=True,timeout=60)
|
|
196
|
+
video=out/'video.mp4'
|
|
197
|
+
if not 0<video.stat().st_size<100*1024*1024:raise RuntimeError('Video output bound failed')
|
|
198
|
+
summary={'status':'passed','task':'comfyui_wan_video','width':256,'height':256,'frames':17,'fps':8,'steps':8,'seed':20261113,'generation_startup_encode_seconds':round(time.monotonic()-started,6),'video_bytes':video.stat().st_size,'sha256':hashlib.sha256(video.read_bytes()).hexdigest(),'scope':'Tiny synthetic video smoke test; not a quality benchmark or customer browser recovery validation'}
|
|
199
|
+
(out/'video-acceptance.json').write_text(json.dumps(summary));print('COMFY_VIDEO_ACCEPTANCE_OK '+json.dumps(summary))
|
|
200
|
+
finally:
|
|
201
|
+
proc.terminate()
|
|
202
|
+
try:proc.wait(timeout=15)
|
|
203
|
+
except subprocess.TimeoutExpired:proc.kill();proc.wait(timeout=5)
|
|
204
|
+
log.close()
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
{
|
|
2
|
+
"observed_at": "2026-11-13T18:00:44Z",
|
|
3
|
+
"scope": "Internal acceptance evidence; not commercial traffic, not an SLA or a provider benchmark",
|
|
4
|
+
"cases": [
|
|
5
|
+
{
|
|
6
|
+
"task_type": "gpu_environment_check",
|
|
7
|
+
"verification_status": "verified_internal_smoke",
|
|
8
|
+
"observed_at": "2026-11-13T10:43:39Z",
|
|
9
|
+
"submitted_attempts": 1,
|
|
10
|
+
"executed_attempts": 1,
|
|
11
|
+
"successful_attempts": 1,
|
|
12
|
+
"success_rate": 1.0,
|
|
13
|
+
"total_elapsed_seconds": 191,
|
|
14
|
+
"final_gpu_wallet_cost_usd": 0.0048,
|
|
15
|
+
"wallet_delta_verified": true,
|
|
16
|
+
"artifact_sha256_verified": true,
|
|
17
|
+
"platform_release_recorded": true,
|
|
18
|
+
"upstream_release_verified": true,
|
|
19
|
+
"environment": {
|
|
20
|
+
"case": "gpu_environment_check",
|
|
21
|
+
"status": "passed",
|
|
22
|
+
"utc": "2026-11-13T10:43:00.758443+00:00",
|
|
23
|
+
"gpu": [
|
|
24
|
+
"NVIDIA GeForce RTX 3090, 24576 MiB, 595.71.05"
|
|
25
|
+
],
|
|
26
|
+
"python": "3.8.10",
|
|
27
|
+
"elapsed_seconds": 0.071337,
|
|
28
|
+
"scope": "NVIDIA driver and device visibility; not a model inference benchmark"
|
|
29
|
+
},
|
|
30
|
+
"sample_warning": "One internal smoke test; not a commercial success-rate estimate, model benchmark or uptime guarantee.",
|
|
31
|
+
"upstream_release_evidence": "Authenticated complete deployment list absence checked at 2026-11-13T12:09:54.643997+00:00"
|
|
32
|
+
},
|
|
33
|
+
{
|
|
34
|
+
"task_type": "comfyui_video",
|
|
35
|
+
"verification_status": "verified_internal_smoke",
|
|
36
|
+
"success_rate": 0.08333333333333333,
|
|
37
|
+
"attempts": 12,
|
|
38
|
+
"total_elapsed_seconds": 1064,
|
|
39
|
+
"final_cost_usd": 0.1105,
|
|
40
|
+
"successful_attempts": 1,
|
|
41
|
+
"submitted_attempts": 12,
|
|
42
|
+
"failed_attempts": 5,
|
|
43
|
+
"cancelled_attempts": 6,
|
|
44
|
+
"all_attempts_final_gpu_cost_usd": 0.8775,
|
|
45
|
+
"platform_release_recorded": true,
|
|
46
|
+
"upstream_release_verified": true,
|
|
47
|
+
"allocated_gpu_releases_verified": true,
|
|
48
|
+
"unverified_provisioning_attempts": 1,
|
|
49
|
+
"generation_observed": true,
|
|
50
|
+
"generation_successful_attempts": 2,
|
|
51
|
+
"artifact_sha256_verified": true,
|
|
52
|
+
"observed_at": "2026-11-13T18:00:44Z",
|
|
53
|
+
"transfer_method": "control_plane_relay",
|
|
54
|
+
"relayed_bytes": 21426,
|
|
55
|
+
"model_download_and_sha256_seconds": 874.102,
|
|
56
|
+
"generation_startup_encode_seconds": 18.776703,
|
|
57
|
+
"video_sha256": "4f39c7eedeb79159293e396d6a3e549419bb88aca7a136be42ba0a4ddad4f24c",
|
|
58
|
+
"decoded_video": {
|
|
59
|
+
"codec": "h264",
|
|
60
|
+
"width": 256,
|
|
61
|
+
"height": 256,
|
|
62
|
+
"frames": 17,
|
|
63
|
+
"fps": 8,
|
|
64
|
+
"steps": 8
|
|
65
|
+
},
|
|
66
|
+
"upstream_release_scope": "Successful case and all allocated development attempts verified absent from the complete provider list. One historical out-of-stock response has no instance ID and remains excluded.",
|
|
67
|
+
"scope_note": "Tiny synthetic Wan2.1 1.3B 17-frame smoke. Twelve development submissions used evolving preparation/harness code; 1 full delivery success, not a commercial SLA or video quality benchmark. Models download directly onto the rented GPU. Customer extension/app-restart browser issue remains separately unresolved."
|
|
68
|
+
},
|
|
69
|
+
{
|
|
70
|
+
"task_type": "causal_lm_lora",
|
|
71
|
+
"verification_status": "verified_internal_smoke",
|
|
72
|
+
"success_rate": 0.25,
|
|
73
|
+
"attempts": 4,
|
|
74
|
+
"total_elapsed_seconds": 190,
|
|
75
|
+
"final_cost_usd": 0.0097,
|
|
76
|
+
"scope_note": "8-step synthetic-text causal-LM LoRA smoke on RTX 3090, including nonzero trained adapter. Not diffusion/image LoRA, model-quality benchmark or commercial success-rate estimate. Four submitted attempts: cancellation, network preparation failure, BF16 kernel failure, then successful FP32-compatible run. Tiny internal sample.",
|
|
77
|
+
"submitted_attempts": 4,
|
|
78
|
+
"successful_attempts": 1,
|
|
79
|
+
"cancelled_before_training_attempts": 1,
|
|
80
|
+
"model_preparation_failed_attempts": 1,
|
|
81
|
+
"training_failed_attempts": 1,
|
|
82
|
+
"all_attempts_final_gpu_cost_usd": 0.0339,
|
|
83
|
+
"wallet_delta_verified": true,
|
|
84
|
+
"artifact_sha256_verified": true,
|
|
85
|
+
"platform_release_recorded": true,
|
|
86
|
+
"upstream_release_verified": true,
|
|
87
|
+
"environment": {
|
|
88
|
+
"task": "causal-lm-lora",
|
|
89
|
+
"steps_completed": 8,
|
|
90
|
+
"records": 4,
|
|
91
|
+
"last_loss": 4.992096424102783,
|
|
92
|
+
"base_model": "HuggingFaceTB/SmolLM2-135M",
|
|
93
|
+
"revision": "93efa2f097d58c2a74874c7e644dbc9b0cee75a2",
|
|
94
|
+
"rank": 4,
|
|
95
|
+
"training_dtype": "torch.float32",
|
|
96
|
+
"nonzero_trained_lora_B": true,
|
|
97
|
+
"status": "passed",
|
|
98
|
+
"cuda_available": true,
|
|
99
|
+
"gpu": "NVIDIA GeForce RTX 3090",
|
|
100
|
+
"training_and_save_seconds": 3.99557,
|
|
101
|
+
"adapter_bytes": 937056,
|
|
102
|
+
"scope": "8-step synthetic-text causal-LM LoRA smoke test; not image LoRA or a quality benchmark"
|
|
103
|
+
},
|
|
104
|
+
"upstream_release_evidence": "Authenticated complete deployment list absence checked at 2026-11-13T12:09:54.643997+00:00"
|
|
105
|
+
}
|
|
106
|
+
],
|
|
107
|
+
"aggregate_final_gpu_test_wallet_cost_usd": 0.9552,
|
|
108
|
+
"separate_transport_probe_cost_usd": 0.039,
|
|
109
|
+
"approved_budget_usd": 10,
|
|
110
|
+
"allocated_instances_released": 19,
|
|
111
|
+
"unallocated_out_of_stock_submission": 1,
|
|
112
|
+
"reliability_sample": "Internal development acceptance only; obtain a fresh quote. Not commercial success rates or an SLA."
|
|
113
|
+
}
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
import datetime,json,os,pathlib,platform,subprocess,time
|
|
2
|
+
started=time.monotonic()
|
|
3
|
+
r=subprocess.run(['nvidia-smi','--query-gpu=name,memory.total,driver_version','--format=csv,noheader'],text=True,capture_output=True,timeout=20)
|
|
4
|
+
if r.returncode or not r.stdout.strip():raise RuntimeError('GPU driver check failed')
|
|
5
|
+
out={'case':'gpu_environment_check','status':'passed','utc':datetime.datetime.now(datetime.timezone.utc).isoformat(),'gpu':r.stdout.strip().splitlines(),'python':platform.python_version(),'elapsed_seconds':round(time.monotonic()-started,6),'scope':'NVIDIA driver and device visibility; not a model inference benchmark'}
|
|
6
|
+
p=pathlib.Path(os.environ['CLOUDGPU_OUTPUT_DIR']);p.mkdir(mode=0o700,parents=True,exist_ok=True);(p/'environment-check.json').write_text(json.dumps(out));print('GPU_ENVIRONMENT_ACCEPTANCE_OK '+json.dumps(out))
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
// Local dry-run by default. Submission requires current owner approval.
|
|
2
|
+
import fs from 'node:fs/promises';
|
|
3
|
+
import {fileURLToPath} from 'node:url';
|
|
4
|
+
import path from 'node:path';
|
|
5
|
+
import {createHash} from 'node:crypto';
|
|
6
|
+
import {CloudGpuApi} from '../../src/api.js';
|
|
7
|
+
import {PersistentStorage} from '../../src/storage.js';
|
|
8
|
+
import {loadStoredKey} from '../../src/credentials.js';
|
|
9
|
+
const dir=path.dirname(fileURLToPath(import.meta.url));
|
|
10
|
+
const cases=JSON.parse(await fs.readFile(path.join(dir,'cases.json'),'utf8'));
|
|
11
|
+
const name=process.argv[2];if(!Object.hasOwn(cases,name))throw Error('Choose gpu-environment, causal-lora or comfy-wan-video.');
|
|
12
|
+
const {script,interpreter,...request}=cases[name];if(!['python3','bash'].includes(interpreter)||path.basename(script)!==script)throw Error('Invalid bundled script');
|
|
13
|
+
const bytes=await fs.readFile(path.join(dir,script));
|
|
14
|
+
request.command=`${interpreter} -c "$(printf '%s' '${bytes.toString('base64')}' | base64 -d)"`;
|
|
15
|
+
if(Buffer.byteLength(request.command)>32768)throw Error('Command exceeds Job limit');
|
|
16
|
+
const summary={case:name,gpu_model:request.gpu_model,template:request.template||null,max_minutes:request.max_minutes,max_spend_usd:request.max_spend_usd,script_sha256:createHash('sha256').update(bytes).digest('hex')};
|
|
17
|
+
if(process.env.CGPU_SUBMIT!=='YES'){
|
|
18
|
+
console.log(JSON.stringify({...summary,submitted:false,next:'Review script, current quote and storage terms. Submit with CGPU_SUBMIT=YES and a stable new CGPU_JOB_KEY. Resume a known Job with CGPU_JOB_ID.'},null,2));
|
|
19
|
+
}else{
|
|
20
|
+
const api=new CloudGpuApi({apiKey:process.env.CLOUDGPU_API_KEY||loadStoredKey()?.api_key});
|
|
21
|
+
let job;
|
|
22
|
+
if(process.env.CGPU_JOB_ID){
|
|
23
|
+
if(!/^job_[a-f0-9]{32}$/.test(process.env.CGPU_JOB_ID))throw Error('Invalid existing Job ID');
|
|
24
|
+
job=await api.job(process.env.CGPU_JOB_ID);
|
|
25
|
+
}else{
|
|
26
|
+
const key=process.env.CGPU_JOB_KEY;if(!/^[A-Za-z0-9._:-]{1,128}$/.test(key||''))throw Error('A stable new submission key is required.');
|
|
27
|
+
const quote=await api.quote({requirements:{gpu_model:request.gpu_model,gpu_count:request.gpu_count,...(request.template?{template:request.template}:{})},estimated_duration_seconds:request.max_minutes*60,max_budget_usd:request.max_spend_usd});
|
|
28
|
+
if(!quote.id||!quote.candidates?.some(c=>c.supports_jobs&&c.within_budget&&c.can_start_with_balance))throw Error('No eligible bounded Job candidate; nothing submitted.');
|
|
29
|
+
request.quote_id=quote.id;job=await api.createJob(request,key);
|
|
30
|
+
}
|
|
31
|
+
console.log(JSON.stringify({...summary,job_id:job.id,status:job.status,next:'Retain this Job ID. Resume this same Job after interruption. An uncertain initial submission requires reconciliation; do not generate a replacement key or GPU.'}));
|
|
32
|
+
const deadline=Date.now()+(request.max_minutes*60+900)*1000;
|
|
33
|
+
while(!['succeeded','failed','cancelled','needs_attention'].includes(job.status)||job.cost_status!=='final'){
|
|
34
|
+
if(job.status==='needs_attention'||Date.now()>=deadline)throw Error(`Reconcile existing Job ${job.id}; no replacement submitted.`);
|
|
35
|
+
await new Promise(resolve=>setTimeout(resolve,5000));job=await api.job(job.id);
|
|
36
|
+
}
|
|
37
|
+
if(job.status!=='succeeded'||job.output_collection?.status!=='succeeded')throw Error(`Existing Job ${job.id} ended without complete output delivery; inspect it before any new submission.`);
|
|
38
|
+
const files=job.output_collection.files,allowed=new Set(request.outputs.files);
|
|
39
|
+
if(files.length!==allowed.size||new Set(files.map(f=>f.path)).size!==allowed.size||files.some(f=>!allowed.has(f.path)||path.basename(f.path)!==f.path||f.status!=='ready'))throw Error('Unexpected output manifest');
|
|
40
|
+
const root=path.resolve(process.env.CGPU_RESULTS_DIR||path.join('cloudgpu-results',job.id));await fs.mkdir(root,{recursive:true,mode:0o700});
|
|
41
|
+
const info=await fs.lstat(root);if(info.isSymbolicLink()||!info.isDirectory()||(info.mode&0o077))throw Error('Results directory must be private');
|
|
42
|
+
const storage=new PersistentStorage(api),verified=[];
|
|
43
|
+
for(const file of files){
|
|
44
|
+
const destination=path.join(root,file.path);
|
|
45
|
+
try{const previous=await fs.lstat(destination);if(previous.isSymbolicLink()||!previous.isFile())throw Error('Unsafe existing result target');}catch(e){if(e.code!=='ENOENT')throw e;}
|
|
46
|
+
await storage.download(file.file_id,destination);await fs.chmod(destination,0o600);
|
|
47
|
+
const sha=await storage.digest(destination),stat=await fs.stat(destination);
|
|
48
|
+
if(stat.size!==file.size_bytes||sha!==file.sha256)throw Error('Downloaded output integrity mismatch');
|
|
49
|
+
verified.push({path:file.path,size_bytes:stat.size,sha256:sha,local_file:destination});
|
|
50
|
+
}
|
|
51
|
+
console.log(JSON.stringify({case:name,job_id:job.id,status:'succeeded',cost_usd:job.cost_usd,cost_status:job.cost_status,total_elapsed_seconds:job.finished_at-job.created_at,transfer_method:job.output_collection.transfer_method||'direct_object_storage',outputs:verified,platform_release_recorded:true,independent_upstream_release:'See the published internal acceptance evidence; platform status alone is not an upstream health probe.'}));
|
|
52
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cloudgpu-mcp",
|
|
3
|
-
"version": "0.16.
|
|
3
|
+
"version": "0.16.3",
|
|
4
4
|
"mcpName": "io.github.cloudgpu-app/cloudgpu-mcp",
|
|
5
5
|
"description": "MCP server and command-line tool for cloudgpu.app: let Claude Code, Cursor, Cline, opencode, goose and other MCP clients (or you, from a terminal) rent a GPU with explicit billing quotes, run a job, fetch the results and shut it down, with spending limits the platform enforces.",
|
|
6
6
|
"type": "module",
|
package/src/agent-readiness.js
CHANGED
|
@@ -2,5 +2,5 @@ import { z } from "zod";
|
|
|
2
2
|
/** Anonymous tools intentionally omit Authorization even when this client has a key. */
|
|
3
3
|
export function registerAgentReadinessTools(server,{api,wrap,ok}){
|
|
4
4
|
server.registerTool("get_agent_readiness",{title:"Read-only Agent self-check",description:"No login or spending: capabilities, source-specific billing, task prerequisites, limitations, recent aggregate Job evidence and known issues. Unknown data is not a healthy or verified result.",inputSchema:{},annotations:{readOnlyHint:true,openWorldHint:true}},wrap(async()=>ok(await api.request("GET","/v1/public/agent-self-check",undefined,{noAuth:true}))));
|
|
5
|
-
server.registerTool("assess_task_fit",{title:"Assess whether a task fits CloudGPU",description:"Read-only task suitability before quoting a GPU. Returns blockers, unverified requirements and evidence status; does not benchmark, approve spending or guarantee availability.",inputSchema:{task_type:z.enum(["gpu_environment_check","python_batch","comfyui_video","causal_lm_lora"]),needs_persistent_outputs:z.boolean().default(true),requires_kernel_control:z.boolean().default(false),requires_nested_virtualization:z.boolean().default(false),requires_resale:z.boolean().default(false)},annotations:{readOnlyHint:true,openWorldHint:true}},wrap(async input=>ok(await api.request("POST","/v1/public/task-fit",input,{noAuth:true}))));
|
|
5
|
+
server.registerTool("assess_task_fit",{title:"Assess whether a task fits CloudGPU",description:"Read-only task suitability before quoting a GPU. Returns blockers, unverified requirements and evidence status; does not benchmark, approve spending or guarantee availability.",inputSchema:{task_type:z.enum(["gpu_environment_check","python_batch","comfyui_video","causal_lm_lora"]),needs_persistent_outputs:z.boolean().default(true),expected_output_bytes:z.number().int().min(0).max(Number.MAX_SAFE_INTEGER).optional(),requires_kernel_control:z.boolean().default(false),requires_nested_virtualization:z.boolean().default(false),requires_resale:z.boolean().default(false)},annotations:{readOnlyHint:true,openWorldHint:true}},wrap(async input=>ok(await api.request("POST","/v1/public/task-fit",input,{noAuth:true}))));
|
|
6
6
|
}
|
package/src/api.js
CHANGED
|
@@ -111,6 +111,19 @@ export class CloudGpuApi {
|
|
|
111
111
|
if (body.model_references?.length) return this.createModelJob(body,idempotencyKey);
|
|
112
112
|
return this.request("POST", "/v1/jobs", body, { idempotencyKey });
|
|
113
113
|
}
|
|
114
|
+
async createServerJobBatch(body,idempotencyKey) {
|
|
115
|
+
if(!idempotencyKey)throw new Error("Server batch requires a stable idempotency key");
|
|
116
|
+
const readiness=await this.request("GET","/v1/public/agent-self-check",undefined,{noAuth:true});
|
|
117
|
+
if(readiness.durable_batch?.enabled!==true)throw new ApiError(503,"durable_batch_unavailable","Durable batch is unavailable; no client-side submission fallback was attempted.");
|
|
118
|
+
if(body.jobs.some(j=>j.outputs||j.model_references?.length)){
|
|
119
|
+
const caps=await this.request("GET","/v1/public/compute-capabilities",undefined,{noAuth:true});
|
|
120
|
+
if(body.jobs.some(j=>j.outputs)&&(caps.automatic_output_collection!==true||caps.storage!==true))throw new ApiError(503,"outputs_unavailable","Output persistence unavailable; no batch submitted.");
|
|
121
|
+
if(body.jobs.some(j=>j.model_references?.length)&&caps.model_references!==true)throw new ApiError(503,"model_references_unavailable","Pinned model loading unavailable; no batch submitted.");
|
|
122
|
+
}
|
|
123
|
+
return this.request("POST","/v1/job-batches",body,{idempotencyKey});
|
|
124
|
+
}
|
|
125
|
+
serverJobBatch(id) {if(!/^[a-f0-9]{32}$/.test(id))throw Error("Invalid server batch id");return this.request("GET",`/v1/job-batches/${id}`);}
|
|
126
|
+
cancelServerJobBatch(id) {if(!/^[a-f0-9]{32}$/.test(id))throw Error("Invalid server batch id");return this.request("POST",`/v1/job-batches/${id}/cancel`,{});}
|
|
114
127
|
async createOutputJob(body,idempotencyKey) {
|
|
115
128
|
const caps=await this.request("GET","/v1/public/compute-capabilities",undefined,{noAuth:true});
|
|
116
129
|
if(caps.automatic_output_collection!==true || caps.storage!==true)
|
package/src/index.js
CHANGED
|
@@ -40,7 +40,7 @@ import {
|
|
|
40
40
|
shq,
|
|
41
41
|
} from "./guards.js";
|
|
42
42
|
|
|
43
|
-
const VERSION = "0.16.
|
|
43
|
+
const VERSION = "0.16.3";
|
|
44
44
|
const HERE = path.dirname(fileURLToPath(import.meta.url));
|
|
45
45
|
const RULES = fs.readFileSync(path.join(HERE, "..", "rules", "cloudgpu.md"), "utf8");
|
|
46
46
|
|
|
@@ -67,7 +67,8 @@ const session = new Map();
|
|
|
67
67
|
|
|
68
68
|
const INSTRUCTIONS = `CloudGPU (cloudgpu.app) provides GPU compute scheduling for AI agents from a prepaid balance. Billing is source-specific: per-minute or minimum one hour with each started hour charged in full. Never infer the period from an hourly price label.
|
|
69
69
|
Planning: compare_compute_costs compares billed GPU cost for explicit candidates and duration; it is not a benchmark or creation approval. Check existing list_instances before renting: multiple approved commands can use the same running rental within its existing budget, time and idle limits. Separate run_job calls each provision and release independently; prepare_job_rerun does not reuse a live GPU. Never extend limits or keep idle GPUs merely to consume prepaid time.
|
|
70
|
-
|
|
70
|
+
Server batches: prepare_job_batch also prepares the exact manifest for run_server_job_batch. The new tool atomically enqueues on the server, persists across client/server restarts, and requires an aggregate cap and individually approved billing. Use get_server_job_batch/cancel_server_job_batch; never substitute client partial submission when unavailable. No automatic failed-command retries.
|
|
71
|
+
Legacy client batches: prepare_job_batch reviews up to eight individually quoted Jobs and their combined GPU budget. run_job_batch requires an exact approved plan hash; partial submission is possible. Retain the manifest and stable batch key to reconcile lost responses. No automatic execution retries; get_job_batch never proves independent supplier release. Storage charges are additional to GPU caps.
|
|
71
72
|
Shared allocation: prepare_task_pipeline prepares 1–8 reviewed sequential commands as a single Job draft, or for an approved existing rental. It creates nothing. Independent run_job calls do not share a GPU. Pipeline summary persistence is explicit and does not save arbitrary results or authorize storage charges.
|
|
72
73
|
Task recipes: list_task_templates and prepare_task_template return local read-only drafts, never a paid submission. Check prerequisites, quote and obtain explicit approval before run_job. Prefer local CPU processing when a GPU is unnecessary.
|
|
73
74
|
Workflow: get_account and list_gpus -> estimate_cost -> show the quote to the user and get an explicit yes -> create_instance -> get_status until running -> upload -> run_command (background for long jobs) -> tail_logs -> download results -> stop_instance -> report the actual cost.
|
|
@@ -539,7 +540,7 @@ if (!cfg.readOnly) {
|
|
|
539
540
|
inputSchema: jobInputShape(cfg),
|
|
540
541
|
annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true, openWorldHint: true },
|
|
541
542
|
},
|
|
542
|
-
wrap(async ({ command, gpu_model, template, gpu_count = 1, disk_gb, workspace_version_id, model_references, outputs, quote_id, max_spend_usd, max_hours, user_confirmed, idempotency_key }) => {
|
|
543
|
+
wrap(async ({ command, gpu_model, template, gpu_count = 1, disk_gb, workspace_version_id, model_references, outputs, quote_id, billing_quote_id, billing_accepted, max_spend_usd, max_hours, user_confirmed, idempotency_key }) => {
|
|
543
544
|
if (user_confirmed !== true) {
|
|
544
545
|
return fail("Not started: tell the user the GPU, budget and time limit, wait for an explicit yes, then call again with user_confirmed=true.");
|
|
545
546
|
}
|
|
@@ -561,6 +562,7 @@ if (!cfg.readOnly) {
|
|
|
561
562
|
if (outputs) body.outputs = outputs;
|
|
562
563
|
if (model_references?.length) body.model_references = model_references;
|
|
563
564
|
if (quote_id) body.quote_id = quote_id;
|
|
565
|
+
if (billing_quote_id) {if(billing_accepted!==true)return fail("Not started: accept the source-specific minimum charge and billing period first.");body.billing_quote_id=billing_quote_id;body.billing_accepted=true;}
|
|
564
566
|
const job = await api.createJob(body, key);
|
|
565
567
|
return ok({
|
|
566
568
|
...job,
|
package/src/job-batches.js
CHANGED
|
@@ -68,12 +68,32 @@ export async function getJobBatch(api,ids){
|
|
|
68
68
|
for(const id of ids){try{const j=await api.job(id);if(j.id!==id||!['queued','provisioning','running','stopping','needs_attention','succeeded','failed','cancelled'].includes(j.status))throw Error('Unrecognized Job response');items.push({job_id:id,status:j.status,cost_usd:j.cost_usd??null,cost_status:j.cost_status??null,automatic_retry_allowed:false,release_risk:j.status==='needs_attention'||j.status==='stopping'});}catch{items.push({job_id:id,status:'unavailable',automatic_retry_allowed:false,release_risk:true});}}
|
|
69
69
|
return {items,all_terminal:items.every(j=>['succeeded','failed','cancelled'].includes(j.status)),requires_reconciliation:items.some(j=>j.release_risk),automatic_retry_allowed:false,note:'Status and costs come from the existing Job API. Terminal state is not an independent supplier audit; stopping, needs_attention or unavailable must never trigger a new GPU.'};
|
|
70
70
|
}
|
|
71
|
+
export async function submitServerJobBatch(api,raw,cfg){
|
|
72
|
+
if(api instanceof CloudGpuApi)api=new CloudGpuApi({apiKey:api.apiKey,baseUrl:api.baseUrl,fetchImpl:api.fetch,userAgent:api.userAgent});
|
|
73
|
+
const {approved_plan_sha256,user_confirmed,...input}=raw;
|
|
74
|
+
if(user_confirmed!==true)throw Error('Approve every command, billing period and the aggregate budget first');
|
|
75
|
+
const plan=prepareJobBatch(input,cfg);
|
|
76
|
+
if(approved_plan_sha256!==plan.plan_sha256)throw Error('Approved batch manifest changed');
|
|
77
|
+
const quoteIds=new Set();
|
|
78
|
+
for(const item of plan.items)if(item.body.billing_quote_id){
|
|
79
|
+
if(item.body.billing_accepted!==true)throw Error('Accept the exact source billing period and minimum first');
|
|
80
|
+
if(quoteIds.has(item.body.billing_quote_id))throw Error('Each hourly allocation requires its own accepted billing quote');quoteIds.add(item.body.billing_quote_id);
|
|
81
|
+
}
|
|
82
|
+
const result=await api.createServerJobBatch({total_budget_usd:plan.total_budget_usd,jobs:plan.items.map(i=>i.body)},'server-batch:'+plan.batch_key);
|
|
83
|
+
if(!/^[a-f0-9]{32}$/.test(result?.batch_id)||result.durable!==true||!Array.isArray(result.jobs)||result.jobs.length!==plan.items.length||result.jobs.some(j=>!/^job_[a-f0-9]{32}$/.test(j.id)))
|
|
84
|
+
throw Error('Unrecognized batch receipt; reconcile with the identical manifest/key before retrying');
|
|
85
|
+
return {...result,batch_key:plan.batch_key,plan_sha256:plan.plan_sha256,atomic_enqueue:true,automatic_command_retries:false,next:'Use get_server_job_batch. Never replace an uncertain or failed task automatically; the same approved key replays this batch.'};
|
|
86
|
+
}
|
|
87
|
+
|
|
71
88
|
export function registerBatchTools(server,{api,cfg,wrap,ok}){
|
|
72
89
|
const input=schema(cfg);
|
|
73
90
|
const receipts=()=>new BatchReceipts({apiKey:'apiKey' in api?api.apiKey:cfg.apiKey,baseUrl:api.baseUrl||cfg.baseUrl});
|
|
74
91
|
server.registerTool('get_job_batch_receipt',{title:'Read a private local batch receipt',description:'Read only saved accepted Job IDs and uncertain submission intents for the original batch key, API key and endpoint. No API request or GPU creation. Local state is historical, not live release evidence; a missing entry does not prove non-creation. Commands, tokens and signed URLs are never stored.',inputSchema:{batch_key:z.string().min(1).max(80).regex(/^[A-Za-z0-9_-]+$/)},annotations:{readOnlyHint:true,destructiveHint:false,idempotentHint:true,openWorldHint:false}},wrap(async({batch_key})=>ok(await receipts().snapshot(batch_key))));
|
|
75
92
|
server.registerTool('prepare_job_batch',{title:'Review a bounded Job batch',description:'Local, read-only preparation for up to eight Jobs with fresh per-Job quotes and one aggregate cap. No API requests, reservations, installations or automatic execution retries. Retain the immutable plan hash and batch key.',inputSchema:input.shape,annotations:{readOnlyHint:true,destructiveHint:false,idempotentHint:true,openWorldHint:false}},wrap(async p=>ok(prepareJobBatch(p,cfg))));
|
|
76
93
|
server.registerTool('get_job_batch',{title:'Inspect a Job batch',description:'Read-only status and cost summary for explicit Job IDs. Does not retry, cancel, create or claim independent supplier release verification.',inputSchema:{job_ids:z.array(z.string().regex(/^job_[a-f0-9]{32}$/)).min(1).max(8)},annotations:{readOnlyHint:true,destructiveHint:false,idempotentHint:true,openWorldHint:true}},wrap(async({job_ids})=>ok(await getJobBatch(api,job_ids))));
|
|
94
|
+
server.registerTool('get_server_job_batch',{title:'Read a durable server batch',description:'Read persisted child Job states and costs after either client or server restarts. Does not create or retry resources.',inputSchema:{batch_id:z.string().regex(/^[a-f0-9]{32}$/)},annotations:{readOnlyHint:true,destructiveHint:false,idempotentHint:true,openWorldHint:true}},wrap(async({batch_id})=>ok(await api.serverJobBatch(batch_id))));
|
|
77
95
|
if(cfg.readOnly)return;
|
|
96
|
+
server.registerTool('run_server_job_batch',{title:'Atomically enqueue an approved server batch',description:'Paid. Use the exact reviewed prepare_job_batch manifest/hash. Enqueue is atomic and persists on the server; each Job reserves its own budget and waits for account/key concurrency. No automatic command retries and no partial client-side fallback. Whole-hour Jobs require a separate accepted current billing quote per allocation.',inputSchema:{...input.shape,approved_plan_sha256:z.string().regex(/^[a-f0-9]{64}$/),user_confirmed:z.boolean()},annotations:{readOnlyHint:false,destructiveHint:true,idempotentHint:true,openWorldHint:true}},wrap(async p=>ok(await submitServerJobBatch(api,p,cfg))));
|
|
97
|
+
server.registerTool('cancel_server_job_batch',{title:'Cancel a server batch',description:'Cancel queued children without starting a GPU; request safe shutdown of active children. Preserve outputs first. Previously started whole-hour charges remain payable.',inputSchema:{batch_id:z.string().regex(/^[a-f0-9]{32}$/),user_confirmed:z.boolean()},annotations:{readOnlyHint:false,destructiveHint:true,idempotentHint:true,openWorldHint:true}},wrap(async({batch_id,user_confirmed})=>{if(user_confirmed!==true)throw Error('Confirm output preservation and cancellation first');return ok(await api.cancelServerJobBatch(batch_id));}));
|
|
78
98
|
server.registerTool('run_job_batch',{title:'Submit an approved bounded Job batch',description:'Paid: submit an explicitly approved immutable manifest using existing durable Jobs, each with a quote, budget and deadline. Aggregate worst-case budget is capped. Partial success is possible. On lost responses replay the IDENTICAL manifest/key; never create replacement GPUs or retry failed execution automatically. Persist the returned receipt. No transaction rollback or independent batch scheduler.',inputSchema:{...input.shape,approved_plan_sha256:z.string().regex(/^[a-f0-9]{64}$/),user_confirmed:z.boolean()},annotations:{readOnlyHint:false,destructiveHint:true,idempotentHint:true,openWorldHint:true}},wrap(async p=>ok(await submitJobBatch(api,p,cfg,receipts()))));
|
|
79
99
|
}
|
package/src/job-input.js
CHANGED
|
@@ -14,6 +14,8 @@ export function jobInputShape(cfg){return {
|
|
|
14
14
|
accept_storage_charges: z.boolean().optional(),
|
|
15
15
|
}).strict().optional().describe("Explicit relative output files under $CLOUDGPU_OUTPUT_DIR; no globs, symlinks, hidden or credential files. Default total limit 10 MiB, maximum 100 MiB. Only set accept_storage_charges=true after the user agrees to existing paid storage terms. Inspect output_collection after completion; the GPU is released even if saving fails."),
|
|
16
16
|
quote_id: z.string().optional().describe("Server quote from quote_compute, starting quote_; binds configuration and checks price at submission"),
|
|
17
|
+
billing_quote_id: z.string().regex(/^[a-f0-9-]{36}$/).optional().describe("Accepted source-specific billing quote from estimate_cost; expires after five minutes. Required for whole-hour sources."),
|
|
18
|
+
billing_accepted: z.boolean().optional().describe("true only after the user accepts the quoted minimum and billing period; budget must cover that minimum"),
|
|
17
19
|
max_spend_usd: z.number().positive().describe(`Budget for this job in USD (at most ${cfg.maxSpend})`),
|
|
18
20
|
max_hours: z.number().positive().describe(`Hard time limit in hours (at most ${cfg.maxHours})`),
|
|
19
21
|
user_confirmed: z.boolean().describe("true only if the user explicitly approved the GPU, budget and time limit"),
|