seamless-jobserver 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,99 @@
1
+ Metadata-Version: 2.4
2
+ Name: seamless-jobserver
3
+ Version: 0.1.0
4
+ Summary: Jobserver service for asynchronous Seamless transformation execution
5
+ Author: Sjoerd de Vries
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/sjdv1982/seamless
8
+ Project-URL: Repository, https://github.com/sjdv1982/seamless
9
+ Project-URL: Issues, https://github.com/sjdv1982/seamless/issues
10
+ Keywords: seamless,jobserver,transformations,service
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Science/Research
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3 :: Only
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Topic :: Scientific/Engineering
18
+ Classifier: Topic :: Software Development :: Libraries
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ Requires-Dist: aiohttp
22
+ Requires-Dist: seamless-core
23
+ Requires-Dist: seamless-config
24
+ Requires-Dist: seamless-remote
25
+ Requires-Dist: seamless-transformer
26
+
27
+ # seamless-jobserver
28
+
29
+ `seamless-jobserver` is a lightweight async HTTP service in the [Seamless](https://github.com/sjdv1982/seamless) ecosystem. It receives transformation jobs over HTTP, dispatches them to spawned Seamless worker processes, and returns the results. It is one of the four backend services that `seamless-remote` connects to.
30
+
31
+ **This is a standalone service, not a library.** User workflow code never imports `seamless-jobserver` — it runs as an independent process, normally launched by `seamless-config` via `remote-http-launcher`. The only user-facing entry point is the `seamless-jobserver` CLI command.
32
+
33
+ ## How it works
34
+
35
+ The jobserver is a single-module (`jobserver.py`) aiohttp server. On startup it spawns a pool of Seamless worker processes (via `seamless-transformer`). Incoming transformation requests arrive over HTTP, are dispatched to the worker pool, and results are returned to the caller — which is always `seamless-remote`'s `jobserver_remote` module, never user code directly.
36
+
37
+ ### HTTP endpoints
38
+
39
+ | Endpoint | Method | Description |
40
+ | --- | --- | --- |
41
+ | `/` | GET | Welcome / version check |
42
+ | `/healthcheck` | GET | Liveness probe |
43
+ | `/run-transformation` | GET | Execute a transformation and return the result checksum |
44
+
45
+ ### Lifecycle
46
+
47
+ 1. `seamless-config` writes a status file and launches `seamless-jobserver` (via `remote-http-launcher` or directly).
48
+ 2. The jobserver picks a port (fixed or random from a range), spawns worker processes, and starts listening.
49
+ 3. It writes its port and status back to the status file so `seamless-remote` can discover it.
50
+ 4. If no requests arrive within the inactivity timeout, the jobserver shuts down automatically.
51
+
52
+ ---
53
+
54
+ ## Relation to the Seamless ecosystem
55
+
56
+ ```text
57
+ seamless-transformer (user-facing API: direct, delayed)
58
+ │
59
+ │ delegate job
60
+ ▼
61
+ seamless-remote
62
+ │ jobserver_remote
63
+ │ async HTTP
64
+ ▼
65
+ seamless-jobserver ◄── this package
66
+ │
67
+ │ dispatches to worker pool
68
+ ▼
69
+ seamless-transformer worker (spawned child processes)
70
+ ```
71
+
72
+ The jobserver sits between `seamless-remote` (which sends it work) and `seamless-transformer` (whose worker processes do the actual computation). It is a simpler, single-machine alternative to the Dask-based execution path provided by `seamless-dask`.
73
+
74
+ ---
75
+
76
+ ## CLI
77
+
78
+ ```bash
79
+ seamless-jobserver [options]
80
+ ```
81
+
82
+ | Flag | Default | Description |
83
+ | --- | --- | --- |
84
+ | `--port PORT` | — | Listen on a specific port (mutually exclusive with `--port-range`) |
85
+ | `--port-range START END` | — | Pick a random free port from a range |
86
+ | `--host HOST` | `0.0.0.0` | Listening address |
87
+ | `--status-file PATH` | — | JSON file for reporting port and status |
88
+ | `--timeout SECONDS` | — | Auto-shutdown after this many seconds of inactivity |
89
+ | `--workers N` | `1` | Number of worker processes to spawn |
90
+
91
+ ---
92
+
93
+ ## Installation
94
+
95
+ ```bash
96
+ pip install seamless-jobserver
97
+ ```
98
+
99
+ Requires Python >= 3.10. Dependencies: `aiohttp`, `seamless-core`, `seamless-transformer`. Also imports `seamless-remote` and `seamless-config` at runtime for client cleanup and remote-client configuration respectively.
@@ -0,0 +1,73 @@
1
+ # seamless-jobserver
2
+
3
+ `seamless-jobserver` is a lightweight async HTTP service in the [Seamless](https://github.com/sjdv1982/seamless) ecosystem. It receives transformation jobs over HTTP, dispatches them to spawned Seamless worker processes, and returns the results. It is one of the four backend services that `seamless-remote` connects to.
4
+
5
+ **This is a standalone service, not a library.** User workflow code never imports `seamless-jobserver` — it runs as an independent process, normally launched by `seamless-config` via `remote-http-launcher`. The only user-facing entry point is the `seamless-jobserver` CLI command.
6
+
7
+ ## How it works
8
+
9
+ The jobserver is a single-module (`jobserver.py`) aiohttp server. On startup it spawns a pool of Seamless worker processes (via `seamless-transformer`). Incoming transformation requests arrive over HTTP, are dispatched to the worker pool, and results are returned to the caller — which is always `seamless-remote`'s `jobserver_remote` module, never user code directly.
10
+
11
+ ### HTTP endpoints
12
+
13
+ | Endpoint | Method | Description |
14
+ | --- | --- | --- |
15
+ | `/` | GET | Welcome / version check |
16
+ | `/healthcheck` | GET | Liveness probe |
17
+ | `/run-transformation` | GET | Execute a transformation and return the result checksum |
18
+
19
+ ### Lifecycle
20
+
21
+ 1. `seamless-config` writes a status file and launches `seamless-jobserver` (via `remote-http-launcher` or directly).
22
+ 2. The jobserver picks a port (fixed or random from a range), spawns worker processes, and starts listening.
23
+ 3. It writes its port and status back to the status file so `seamless-remote` can discover it.
24
+ 4. If no requests arrive within the inactivity timeout, the jobserver shuts down automatically.
25
+
26
+ ---
27
+
28
+ ## Relation to the Seamless ecosystem
29
+
30
+ ```text
31
+ seamless-transformer (user-facing API: direct, delayed)
32
+ │
33
+ │ delegate job
34
+ ▼
35
+ seamless-remote
36
+ │ jobserver_remote
37
+ │ async HTTP
38
+ ▼
39
+ seamless-jobserver ◄── this package
40
+ │
41
+ │ dispatches to worker pool
42
+ ▼
43
+ seamless-transformer worker (spawned child processes)
44
+ ```
45
+
46
+ The jobserver sits between `seamless-remote` (which sends it work) and `seamless-transformer` (whose worker processes do the actual computation). It is a simpler, single-machine alternative to the Dask-based execution path provided by `seamless-dask`.
47
+
48
+ ---
49
+
50
+ ## CLI
51
+
52
+ ```bash
53
+ seamless-jobserver [options]
54
+ ```
55
+
56
+ | Flag | Default | Description |
57
+ | --- | --- | --- |
58
+ | `--port PORT` | — | Listen on a specific port (mutually exclusive with `--port-range`) |
59
+ | `--port-range START END` | — | Pick a random free port from a range |
60
+ | `--host HOST` | `0.0.0.0` | Listening address |
61
+ | `--status-file PATH` | — | JSON file for reporting port and status |
62
+ | `--timeout SECONDS` | — | Auto-shutdown after this many seconds of inactivity |
63
+ | `--workers N` | `1` | Number of worker processes to spawn |
64
+
65
+ ---
66
+
67
+ ## Installation
68
+
69
+ ```bash
70
+ pip install seamless-jobserver
71
+ ```
72
+
73
+ Requires Python >= 3.10. Dependencies: `aiohttp`, `seamless-core`, `seamless-transformer`. Also imports `seamless-remote` and `seamless-config` at runtime for client cleanup and remote-client configuration respectively.
@@ -0,0 +1,376 @@
1
+ from aiohttp import web
2
+ import asyncio
3
+ import contextlib
4
+ import json
5
+ import os
6
+ import random
7
+ import signal
8
+ import socket
9
+ import sys
10
+ import time
11
+
12
+ from seamless import Checksum, Buffer
13
+ from seamless.transformer import spawn
14
+ from seamless_transformer import worker
15
+ import seamless
16
+ from seamless.util.get_event_loop import get_event_loop
17
+
18
+ try:
19
+ from seamless_remote.client import close_all_clients as _close_all_clients
20
+ except ImportError:
21
+ _close_all_clients = None
22
+
23
+
24
+ STATUS_FILE_WAIT_TIMEOUT = 20.0
25
+ INACTIVITY_CHECK_INTERVAL = 1.0
26
+
27
+ status_tracker = None
28
+
29
+
30
+ def is_port_in_use(address, port):
31
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
32
+ return s.connect_ex((address, port)) == 0
33
+
34
+
35
+ def wait_for_status_file(path: str, timeout: float = STATUS_FILE_WAIT_TIMEOUT):
36
+ deadline = time.monotonic() + timeout
37
+ while True:
38
+ try:
39
+ with open(path, "r", encoding="utf-8") as status_stream:
40
+ contents = json.load(status_stream)
41
+ break
42
+ except FileNotFoundError:
43
+ if time.monotonic() >= deadline:
44
+ print(
45
+ f"Status file '{path}' not found after {int(timeout)} seconds",
46
+ file=sys.stderr,
47
+ )
48
+ sys.exit(1)
49
+ time.sleep(0.1)
50
+ continue
51
+ except json.JSONDecodeError as exc:
52
+ print(
53
+ f"Status file '{path}' is not valid JSON: {exc}",
54
+ file=sys.stderr,
55
+ )
56
+ sys.exit(1)
57
+
58
+ if not isinstance(contents, dict):
59
+ print(
60
+ f"Status file '{path}' must contain a JSON object",
61
+ file=sys.stderr,
62
+ )
63
+ sys.exit(1)
64
+
65
+ return contents
66
+
67
+
68
+ class StatusFileTracker:
69
+ def __init__(self, path: str, base_contents: dict, port: int):
70
+ self.path = path
71
+ self._base_contents = dict(base_contents)
72
+ self.port = port
73
+ self.running_written = False
74
+
75
+ def _write(self, payload: dict):
76
+ tmp_path = f"{self.path}.tmp"
77
+ with open(tmp_path, "w", encoding="utf-8") as status_stream:
78
+ json.dump(payload, status_stream)
79
+ status_stream.write("\n")
80
+ os.replace(tmp_path, self.path)
81
+
82
+ def write_running(self):
83
+ payload = dict(self._base_contents)
84
+ payload["port"] = self.port
85
+ payload["status"] = "running"
86
+ self._write(payload)
87
+ self._base_contents = payload
88
+ self.running_written = True
89
+
90
+ def write_failed(self):
91
+ payload = dict(self._base_contents)
92
+ payload["status"] = "failed"
93
+ self._write(payload)
94
+
95
+
96
+ def raise_startup_error(exc: BaseException):
97
+ if status_tracker and not status_tracker.running_written:
98
+ status_tracker.write_failed()
99
+ raise exc
100
+
101
+
102
+ def pick_random_free_port(host: str, start: int, end: int) -> int:
103
+ if start < 0 or end > 65535:
104
+ raise RuntimeError("--port-range values must be between 0 and 65535")
105
+ if start > end:
106
+ raise RuntimeError("--port-range START must be less than or equal to END")
107
+
108
+ span = end - start + 1
109
+ attempted = set()
110
+ while len(attempted) < span:
111
+ port = random.randint(start, end)
112
+ if port in attempted:
113
+ continue
114
+ attempted.add(port)
115
+ try:
116
+ with socket.create_server((host, port), reuse_port=False):
117
+ pass
118
+ except OSError:
119
+ continue
120
+ return port
121
+
122
+ raise RuntimeError(f"No free port available in range {start}-{end}")
123
+
124
+
125
+ class JobServer:
126
+ future = None
127
+
128
+ def __init__(
129
+ self,
130
+ host,
131
+ port,
132
+ *,
133
+ timeout_seconds=None,
134
+ status_tracker=None,
135
+ ):
136
+ self.host = host
137
+ self.port = port
138
+ self._timeout_seconds = timeout_seconds
139
+ self._status_tracker = status_tracker
140
+ self._timeout_task = None
141
+ self._last_request = None
142
+ self._runner = None
143
+ self._site = None
144
+
145
+ async def _start(self):
146
+ if is_port_in_use(self.host, self.port):
147
+ print("ERROR: %s port %d already in use" % (self.host, self.port))
148
+ raise Exception
149
+
150
+ app = web.Application(client_max_size=10e9)
151
+ app.add_routes(
152
+ [
153
+ web.get("/", self._welcome),
154
+ web.get("/healthcheck", self._healthcheck),
155
+ web.get("/run-transformation", self._run_transformation),
156
+ ]
157
+ )
158
+ runner = web.AppRunner(app)
159
+ await runner.setup()
160
+ site = web.TCPSite(runner, self.host, self.port)
161
+ await site.start()
162
+ self._runner = runner
163
+ self._site = site
164
+ if self._status_tracker and not self._status_tracker.running_written:
165
+ self._status_tracker.write_running()
166
+ print(f"Welcome to the Seamless jobserver on {self.host}:{self.port}")
167
+ sys.stdout.flush()
168
+ if self._timeout_seconds is not None:
169
+ self._last_request = time.monotonic()
170
+ loop = asyncio.get_running_loop()
171
+ self._timeout_task = loop.create_task(self._monitor_inactivity())
172
+
173
+ def start(self):
174
+ if self.future is not None:
175
+ return
176
+ loop = get_event_loop()
177
+ if loop.is_closed():
178
+ loop = asyncio.new_event_loop()
179
+ asyncio.set_event_loop(loop)
180
+ coro = self._start()
181
+ self.future = loop.create_task(coro)
182
+
183
+ async def stop(self):
184
+ if self._timeout_task:
185
+ self._timeout_task.cancel()
186
+ with contextlib.suppress(asyncio.CancelledError):
187
+ await self._timeout_task
188
+ self._timeout_task = None
189
+ if self._site is not None:
190
+ await self._site.stop()
191
+ self._site = None
192
+ if self._runner is not None:
193
+ await self._runner.cleanup()
194
+ self._runner = None
195
+
196
+ async def _monitor_inactivity(self):
197
+ try:
198
+ while True:
199
+ await asyncio.sleep(INACTIVITY_CHECK_INTERVAL)
200
+ if self._last_request is None:
201
+ continue
202
+ if time.monotonic() - self._last_request >= self._timeout_seconds:
203
+ loop = asyncio.get_running_loop()
204
+ loop.call_soon(loop.stop)
205
+ break
206
+ except asyncio.CancelledError:
207
+ raise
208
+
209
+ def _register_activity(self):
210
+ if self._timeout_seconds is not None:
211
+ self._last_request = time.monotonic()
212
+
213
+ async def _healthcheck(self, _):
214
+ self._register_activity()
215
+ return web.Response(status=200, body="OK")
216
+
217
+ async def _welcome(self, _):
218
+ self._register_activity()
219
+ return web.Response(status=200, body="Seamless jobserver is running")
220
+
221
+ async def _run_transformation(self, request):
222
+ self._register_activity()
223
+ try:
224
+ payload = await request.json()
225
+ except Exception as exc:
226
+ return web.Response(status=400, text=f"Invalid JSON: {exc}")
227
+
228
+ try:
229
+ transformation_dict = payload["transformation_dict"]
230
+ tf_checksum = Checksum(payload["tf_checksum"])
231
+ tf_dunder = payload.get("tf_dunder", {})
232
+ scratch = bool(payload.get("scratch", False))
233
+ except Exception as exc:
234
+ return web.Response(status=400, text=f"Invalid payload: {exc}")
235
+
236
+ tf_checksum_hex = tf_checksum.hex()
237
+ print(f"[jobserver] Received transformation {tf_checksum_hex}", flush=True)
238
+ try:
239
+ result_checksum = await worker.dispatch_to_workers(
240
+ transformation_dict,
241
+ tf_checksum=tf_checksum,
242
+ tf_dunder=tf_dunder,
243
+ scratch=scratch,
244
+ )
245
+ except Exception as exc:
246
+ return web.Response(status=500, text=str(exc))
247
+
248
+ if isinstance(result_checksum, str):
249
+ return web.Response(status=500, text=result_checksum)
250
+
251
+ result_checksum = Checksum(result_checksum)
252
+ if not scratch:
253
+ result_buf = await result_checksum.resolution()
254
+ assert isinstance(result_buf, Buffer)
255
+ print(
256
+ f"[jobserver] Completed transformation {tf_checksum_hex} -> {result_checksum.hex()}",
257
+ flush=True,
258
+ )
259
+ return web.Response(status=200, text=result_checksum.hex())
260
+
261
+
262
+ def main():
263
+ import argparse
264
+
265
+ p = argparse.ArgumentParser()
266
+ port_group = p.add_mutually_exclusive_group()
267
+ port_group.add_argument("--port", type=int, help="Network port")
268
+ port_group.add_argument(
269
+ "--port-range",
270
+ type=int,
271
+ nargs=2,
272
+ metavar=("START", "END"),
273
+ help="Inclusive port range to select a random free port from",
274
+ )
275
+ p.add_argument("--host", default="0.0.0.0")
276
+ p.add_argument(
277
+ "--status-file",
278
+ type=str,
279
+ help="JSON file used to report server status",
280
+ )
281
+ p.add_argument(
282
+ "--timeout",
283
+ type=float,
284
+ help="Stop the server after this many seconds of inactivity",
285
+ )
286
+ p.add_argument(
287
+ "--workers",
288
+ type=int,
289
+ default=1,
290
+ help="Number of worker processes to spawn",
291
+ )
292
+ args = p.parse_args()
293
+
294
+ global status_tracker
295
+
296
+ selected_port = args.port if args.port is not None else 5533
297
+ status_file_path = args.status_file
298
+ status_tracker = None
299
+ parameters = {}
300
+ if status_file_path:
301
+ status_file_contents = wait_for_status_file(status_file_path)
302
+ parameters = status_file_contents.get("parameters", {}) or {}
303
+ status_tracker = StatusFileTracker(
304
+ status_file_path, status_file_contents, args.port
305
+ )
306
+
307
+ if args.port_range:
308
+ start, end = args.port_range
309
+ try:
310
+ selected_port = pick_random_free_port(args.host, start, end)
311
+ except BaseException as exc:
312
+ raise_startup_error(exc)
313
+ if status_tracker:
314
+ status_tracker.port = selected_port
315
+
316
+ timeout_seconds = args.timeout
317
+ if timeout_seconds is not None and timeout_seconds <= 0:
318
+ raise_startup_error(RuntimeError("--timeout must be a positive number"))
319
+ if args.workers is not None and args.workers <= 0:
320
+ raise_startup_error(RuntimeError("--workers must be a positive integer"))
321
+
322
+ def raise_system_exit(*args, **kwargs):
323
+ raise SystemExit
324
+
325
+ signal.signal(signal.SIGTERM, raise_system_exit)
326
+ signal.signal(signal.SIGHUP, raise_system_exit)
327
+ signal.signal(signal.SIGINT, raise_system_exit)
328
+
329
+ try:
330
+ spawn(args.workers)
331
+ if parameters:
332
+ from seamless.config import set_remote_clients
333
+
334
+ set_remote_clients(parameters)
335
+ except BaseException as exc:
336
+ raise_startup_error(exc)
337
+
338
+ job_server = JobServer(
339
+ args.host,
340
+ selected_port,
341
+ timeout_seconds=timeout_seconds,
342
+ status_tracker=status_tracker,
343
+ )
344
+ job_server.start()
345
+
346
+ loop = get_event_loop()
347
+ try:
348
+ loop.run_forever()
349
+ except KeyboardInterrupt:
350
+ pass
351
+ except BaseException:
352
+ if status_tracker and not status_tracker.running_written:
353
+ status_tracker.write_failed()
354
+ raise
355
+ finally:
356
+ loop.run_until_complete(job_server.stop())
357
+ closed_ok = False
358
+ try:
359
+ seamless.close()
360
+ closed_ok = True
361
+ except Exception:
362
+ pass
363
+ if not closed_ok:
364
+ try:
365
+ worker.shutdown_workers()
366
+ except Exception:
367
+ pass
368
+ if _close_all_clients is not None:
369
+ try:
370
+ _close_all_clients()
371
+ except Exception:
372
+ pass
373
+
374
+
375
+ if __name__ == "__main__":
376
+ main()
@@ -0,0 +1,41 @@
1
+ [build-system]
2
+ requires = ["setuptools>=69", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "seamless-jobserver"
7
+ version = "0.1.0"
8
+ description = "Jobserver service for asynchronous Seamless transformation execution"
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ license = "MIT"
12
+ authors = [{name = "Sjoerd de Vries"}]
13
+ dependencies = [
14
+ "aiohttp",
15
+ "seamless-core",
16
+ "seamless-config",
17
+ "seamless-remote",
18
+ "seamless-transformer",
19
+ ]
20
+ keywords = ["seamless", "jobserver", "transformations", "service"]
21
+ classifiers = [
22
+ "Development Status :: 3 - Alpha",
23
+ "Intended Audience :: Science/Research",
24
+ "Intended Audience :: Developers",
25
+ "Programming Language :: Python :: 3",
26
+ "Programming Language :: Python :: 3 :: Only",
27
+ "Operating System :: OS Independent",
28
+ "Topic :: Scientific/Engineering",
29
+ "Topic :: Software Development :: Libraries",
30
+ ]
31
+
32
+ [project.urls]
33
+ Homepage = "https://github.com/sjdv1982/seamless"
34
+ Repository = "https://github.com/sjdv1982/seamless"
35
+ Issues = "https://github.com/sjdv1982/seamless/issues"
36
+
37
+ [project.scripts]
38
+ seamless-jobserver = "jobserver:main"
39
+
40
+ [tool.setuptools]
41
+ py-modules = ["jobserver"]
@@ -0,0 +1,99 @@
1
+ Metadata-Version: 2.4
2
+ Name: seamless-jobserver
3
+ Version: 0.1.0
4
+ Summary: Jobserver service for asynchronous Seamless transformation execution
5
+ Author: Sjoerd de Vries
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/sjdv1982/seamless
8
+ Project-URL: Repository, https://github.com/sjdv1982/seamless
9
+ Project-URL: Issues, https://github.com/sjdv1982/seamless/issues
10
+ Keywords: seamless,jobserver,transformations,service
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Science/Research
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3 :: Only
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Topic :: Scientific/Engineering
18
+ Classifier: Topic :: Software Development :: Libraries
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ Requires-Dist: aiohttp
22
+ Requires-Dist: seamless-core
23
+ Requires-Dist: seamless-config
24
+ Requires-Dist: seamless-remote
25
+ Requires-Dist: seamless-transformer
26
+
27
+ # seamless-jobserver
28
+
29
+ `seamless-jobserver` is a lightweight async HTTP service in the [Seamless](https://github.com/sjdv1982/seamless) ecosystem. It receives transformation jobs over HTTP, dispatches them to spawned Seamless worker processes, and returns the results. It is one of the four backend services that `seamless-remote` connects to.
30
+
31
+ **This is a standalone service, not a library.** User workflow code never imports `seamless-jobserver` — it runs as an independent process, normally launched by `seamless-config` via `remote-http-launcher`. The only user-facing entry point is the `seamless-jobserver` CLI command.
32
+
33
+ ## How it works
34
+
35
+ The jobserver is a single-module (`jobserver.py`) aiohttp server. On startup it spawns a pool of Seamless worker processes (via `seamless-transformer`). Incoming transformation requests arrive over HTTP, are dispatched to the worker pool, and results are returned to the caller — which is always `seamless-remote`'s `jobserver_remote` module, never user code directly.
36
+
37
+ ### HTTP endpoints
38
+
39
+ | Endpoint | Method | Description |
40
+ | --- | --- | --- |
41
+ | `/` | GET | Welcome / version check |
42
+ | `/healthcheck` | GET | Liveness probe |
43
+ | `/run-transformation` | GET | Execute a transformation and return the result checksum |
44
+
45
+ ### Lifecycle
46
+
47
+ 1. `seamless-config` writes a status file and launches `seamless-jobserver` (via `remote-http-launcher` or directly).
48
+ 2. The jobserver picks a port (fixed or random from a range), spawns worker processes, and starts listening.
49
+ 3. It writes its port and status back to the status file so `seamless-remote` can discover it.
50
+ 4. If no requests arrive within the inactivity timeout, the jobserver shuts down automatically.
51
+
52
+ ---
53
+
54
+ ## Relation to the Seamless ecosystem
55
+
56
+ ```text
57
+ seamless-transformer (user-facing API: direct, delayed)
58
+ │
59
+ │ delegate job
60
+ ▼
61
+ seamless-remote
62
+ │ jobserver_remote
63
+ │ async HTTP
64
+ ▼
65
+ seamless-jobserver ◄── this package
66
+ │
67
+ │ dispatches to worker pool
68
+ ▼
69
+ seamless-transformer worker (spawned child processes)
70
+ ```
71
+
72
+ The jobserver sits between `seamless-remote` (which sends it work) and `seamless-transformer` (whose worker processes do the actual computation). It is a simpler, single-machine alternative to the Dask-based execution path provided by `seamless-dask`.
73
+
74
+ ---
75
+
76
+ ## CLI
77
+
78
+ ```bash
79
+ seamless-jobserver [options]
80
+ ```
81
+
82
+ | Flag | Default | Description |
83
+ | --- | --- | --- |
84
+ | `--port PORT` | — | Listen on a specific port (mutually exclusive with `--port-range`) |
85
+ | `--port-range START END` | — | Pick a random free port from a range |
86
+ | `--host HOST` | `0.0.0.0` | Listening address |
87
+ | `--status-file PATH` | — | JSON file for reporting port and status |
88
+ | `--timeout SECONDS` | — | Auto-shutdown after this many seconds of inactivity |
89
+ | `--workers N` | `1` | Number of worker processes to spawn |
90
+
91
+ ---
92
+
93
+ ## Installation
94
+
95
+ ```bash
96
+ pip install seamless-jobserver
97
+ ```
98
+
99
+ Requires Python >= 3.10. Dependencies: `aiohttp`, `seamless-core`, `seamless-transformer`. Also imports `seamless-remote` and `seamless-config` at runtime for client cleanup and remote-client configuration respectively.
@@ -0,0 +1,9 @@
1
+ README.md
2
+ jobserver.py
3
+ pyproject.toml
4
+ seamless_jobserver.egg-info/PKG-INFO
5
+ seamless_jobserver.egg-info/SOURCES.txt
6
+ seamless_jobserver.egg-info/dependency_links.txt
7
+ seamless_jobserver.egg-info/entry_points.txt
8
+ seamless_jobserver.egg-info/requires.txt
9
+ seamless_jobserver.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ seamless-jobserver = jobserver:main
@@ -0,0 +1,5 @@
1
+ aiohttp
2
+ seamless-core
3
+ seamless-config
4
+ seamless-remote
5
+ seamless-transformer
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+