superkernels 0.9.0__tar.gz → 0.9.6__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of superkernels might be problematic. Click here for more details.

@@ -1,12 +1,12 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: superkernels
3
- Version: 0.9.0
3
+ Version: 0.9.6
4
4
  Summary: A local ipykernel adapter for remote Super Kernel runtimes.
5
5
  Author-email: Leo Li <leoustc@icloud.com>
6
6
  License-Expression: Apache-2.0
7
7
  Project-URL: Homepage, https://superkernel.leoustc.com
8
- Project-URL: Source, https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernel
9
- Classifier: Development Status :: 3 - Alpha
8
+ Project-URL: Source, https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernels
9
+ Classifier: Development Status :: 4 - Beta
10
10
  Classifier: Programming Language :: Python :: 3
11
11
  Classifier: Programming Language :: Python :: 3 :: Only
12
12
  Classifier: Operating System :: OS Independent
@@ -38,6 +38,16 @@ PyTorch environment.
38
38
  - Keeps authentication tokens out of Jupyter kernelspecs and command-line
39
39
  arguments.
40
40
 
41
+ This is the shared host adapter for both the VS Code and Jupyter plugins.
42
+ Local runtime records describe notebook sessions, not remote runner health.
43
+ Publication and shutdown cleanup use the shared client-state lock, and each
44
+ record retains its last cell-start timestamp across heartbeat refreshes.
45
+
46
+ Silent requests still execute but suppress notebook output. If a connection
47
+ failure leaves an execution unresolved, rerun its original code to resume it;
48
+ different code is rejected until the previous result is recovered or the kernel
49
+ is restarted. Code is never automatically replayed after a lost reply handle.
50
+
41
51
  ## Installation
42
52
 
43
53
  SuperKernel requires Python 3.10 or newer.
@@ -54,8 +64,7 @@ This package contains the notebook runtime only. Gateway login, remote runtime
54
64
  creation, and local kernel registration are handled by a compatible
55
65
  SuperKernel client integration.
56
66
 
57
- The PyPI distribution is named `superkernels`. The Python import remains
58
- `superkernel`.
67
+ The PyPI distribution and Python import are both named `superkernels`.
59
68
 
60
69
  ## Python API
61
70
 
@@ -63,13 +72,13 @@ The kernel class is available for integrations that need to inspect or extend
63
72
  the local runtime:
64
73
 
65
74
  ```python
66
- from superkernel import SuperKernel
75
+ from superkernels import SuperKernel
67
76
  ```
68
77
 
69
78
  The supported process entry point for a configured Jupyter kernelspec is:
70
79
 
71
80
  ```text
72
- python -m superkernel.kernel
81
+ python -m superkernels.kernel
73
82
  ```
74
83
 
75
84
  Its connection and authentication arguments are generated by the client
@@ -81,10 +90,19 @@ SuperKernel reads authentication from a client-managed local file immediately
81
90
  before each remote request. Credentials are not embedded in the kernelspec,
82
91
  stored by this package, or passed directly on the process command line. The
83
92
  runtime fails closed when its authentication file is missing or invalid.
93
+ It validates the version-1 session metadata, expiry, and Gateway binding before
94
+ every request; renewing the private file takes effect without restarting it.
95
+
96
+ ## Tests
97
+
98
+ Run `python -m pytest -q superkernels/tests` from the repository root with
99
+ the runtime and pytest installed. Tests use temporary private state and do not
100
+ read or modify saved user credentials.
84
101
 
85
102
  ## Project status
86
103
 
87
- Version 0.9.0 is the first public alpha release. The package is suitable for
88
- evaluation, but its integration contract may still evolve before 1.0.
104
+ Version 0.9.6 is the current public beta release. Version 0.9.5 remains the
105
+ first beta, version 0.9.0 remains the compatibility baseline, and the
106
+ integration contract may still evolve before 1.0.
89
107
 
90
108
  Learn more at [superkernel.leoustc.com](https://superkernel.leoustc.com).
@@ -17,6 +17,16 @@ PyTorch environment.
17
17
  - Keeps authentication tokens out of Jupyter kernelspecs and command-line
18
18
  arguments.
19
19
 
20
+ This is the shared host adapter for both the VS Code and Jupyter plugins.
21
+ Local runtime records describe notebook sessions, not remote runner health.
22
+ Publication and shutdown cleanup use the shared client-state lock, and each
23
+ record retains its last cell-start timestamp across heartbeat refreshes.
24
+
25
+ Silent requests still execute but suppress notebook output. If a connection
26
+ failure leaves an execution unresolved, rerun its original code to resume it;
27
+ different code is rejected until the previous result is recovered or the kernel
28
+ is restarted. Code is never automatically replayed after a lost reply handle.
29
+
20
30
  ## Installation
21
31
 
22
32
  SuperKernel requires Python 3.10 or newer.
@@ -33,8 +43,7 @@ This package contains the notebook runtime only. Gateway login, remote runtime
33
43
  creation, and local kernel registration are handled by a compatible
34
44
  SuperKernel client integration.
35
45
 
36
- The PyPI distribution is named `superkernels`. The Python import remains
37
- `superkernel`.
46
+ The PyPI distribution and Python import are both named `superkernels`.
38
47
 
39
48
  ## Python API
40
49
 
@@ -42,13 +51,13 @@ The kernel class is available for integrations that need to inspect or extend
42
51
  the local runtime:
43
52
 
44
53
  ```python
45
- from superkernel import SuperKernel
54
+ from superkernels import SuperKernel
46
55
  ```
47
56
 
48
57
  The supported process entry point for a configured Jupyter kernelspec is:
49
58
 
50
59
  ```text
51
- python -m superkernel.kernel
60
+ python -m superkernels.kernel
52
61
  ```
53
62
 
54
63
  Its connection and authentication arguments are generated by the client
@@ -60,10 +69,19 @@ SuperKernel reads authentication from a client-managed local file immediately
60
69
  before each remote request. Credentials are not embedded in the kernelspec,
61
70
  stored by this package, or passed directly on the process command line. The
62
71
  runtime fails closed when its authentication file is missing or invalid.
72
+ It validates the version-1 session metadata, expiry, and Gateway binding before
73
+ every request; renewing the private file takes effect without restarting it.
74
+
75
+ ## Tests
76
+
77
+ Run `python -m pytest -q superkernels/tests` from the repository root with
78
+ the runtime and pytest installed. Tests use temporary private state and do not
79
+ read or modify saved user credentials.
63
80
 
64
81
  ## Project status
65
82
 
66
- Version 0.9.0 is the first public alpha release. The package is suitable for
67
- evaluation, but its integration contract may still evolve before 1.0.
83
+ Version 0.9.6 is the current public beta release. Version 0.9.5 remains the
84
+ first beta, version 0.9.0 remains the compatibility baseline, and the
85
+ integration contract may still evolve before 1.0.
68
86
 
69
87
  Learn more at [superkernel.leoustc.com](https://superkernel.leoustc.com).
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "superkernels"
7
- version = "0.9.0"
7
+ version = "0.9.6"
8
8
  description = "A local ipykernel adapter for remote Super Kernel runtimes."
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.10"
@@ -14,7 +14,7 @@ authors = [
14
14
  { name = "Leo Li", email = "leoustc@icloud.com" }
15
15
  ]
16
16
  classifiers = [
17
- "Development Status :: 3 - Alpha",
17
+ "Development Status :: 4 - Beta",
18
18
  "Programming Language :: Python :: 3",
19
19
  "Programming Language :: Python :: 3 :: Only",
20
20
  "Operating System :: OS Independent",
@@ -28,7 +28,7 @@ dependencies = [
28
28
 
29
29
  [project.urls]
30
30
  Homepage = "https://superkernel.leoustc.com"
31
- Source = "https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernel"
31
+ Source = "https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernels"
32
32
 
33
33
  [tool.setuptools]
34
34
  package-dir = { "" = "src" }
@@ -7,13 +7,13 @@ from importlib.metadata import PackageNotFoundError, version
7
7
  try:
8
8
  __version__ = version("superkernels")
9
9
  except PackageNotFoundError:
10
- __version__ = "0.9.0"
10
+ __version__ = "0.9.6"
11
11
 
12
12
 
13
13
  def __getattr__(name: str):
14
14
  """Load the local ipykernel only when a client explicitly requests it."""
15
15
  if name == "SuperKernel":
16
- from superkernel.kernel import SuperKernel
16
+ from superkernels.kernel import SuperKernel
17
17
 
18
18
  return SuperKernel
19
19
  raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -31,15 +31,15 @@ from requests.exceptions import (
31
31
  ConnectionError as RequestsConnectionError,
32
32
  )
33
33
 
34
- from superkernel import __version__
35
- from superkernel.logfile import DEFAULT_LOG_FILE, append_log
36
- from superkernel.schema import (
34
+ from superkernels import __version__
35
+ from superkernels.logfile import DEFAULT_LOG_FILE, append_log
36
+ from superkernels.schema import (
37
37
  CodeExecutionRequest,
38
38
  CodeExecutionResumeRequest,
39
39
  ControlRequest,
40
40
  ControlResponse,
41
41
  )
42
- from superkernel.state import (
42
+ from superkernels.state import (
43
43
  _delete_runtime_status,
44
44
  _normalize_gateway_url,
45
45
  _read_auth_token,
@@ -51,13 +51,17 @@ class StaleSuperKernelError(RuntimeError):
51
51
  """Signal that this saved local registration cannot be used anymore."""
52
52
 
53
53
 
54
+ class UnresolvedExecutionError(RuntimeError):
55
+ """Refuse different code while a previous execution needs reconciliation."""
56
+
57
+
54
58
  AdmissionState = Literal["idle", "waiting", "admitting", "admitted"]
55
59
 
56
60
 
57
61
  class SuperKernel(Kernel):
58
62
  """Forward notebook execution to one gateway-provisioned Super Kernel."""
59
63
 
60
- implementation = "superkernel"
64
+ implementation = "superkernels"
61
65
  implementation_version = __version__
62
66
  language = "python"
63
67
  language_version = f"{sys.version_info.major}.{sys.version_info.minor}"
@@ -110,12 +114,13 @@ class SuperKernel(Kernel):
110
114
  self._log_path = Path(log_file).expanduser() if log_file else DEFAULT_LOG_FILE
111
115
  self._runtime_status = "idle"
112
116
  self._last_execution_at: float | None = None
113
- self._runtime_status_lock = threading.Lock()
117
+ self._silent_execution = False
118
+ self._runtime_status_lock = threading.RLock()
114
119
  self._runtime_status_stop = threading.Event()
115
120
  self._publish_runtime_status()
116
121
  self._runtime_status_thread = threading.Thread(
117
122
  target=self._runtime_status_heartbeat,
118
- name=f"superkernel-status-{self.session_id}",
123
+ name=f"superkernels-status-{self.session_id}",
119
124
  daemon=True,
120
125
  )
121
126
  self._runtime_status_thread.start()
@@ -128,6 +133,8 @@ class SuperKernel(Kernel):
128
133
  return None
129
134
 
130
135
  def _write_stream(self, name: str, text: str) -> None:
136
+ if getattr(self, "_silent_execution", False):
137
+ return
131
138
  self.send_response(self.iopub_socket, "stream", {"name": name, "text": text})
132
139
 
133
140
  def _write_runner_stream(self, runner_id: str, name: str, text: str) -> None:
@@ -161,6 +168,8 @@ class SuperKernel(Kernel):
161
168
  self._runner_virtual_newlines.clear()
162
169
 
163
170
  def _send_display_data(self, data: dict[str, Any]) -> None:
171
+ if getattr(self, "_silent_execution", False):
172
+ return
164
173
  self.send_response(
165
174
  self.iopub_socket,
166
175
  "display_data",
@@ -176,15 +185,15 @@ class SuperKernel(Kernel):
176
185
  def _publish_runtime_status(self) -> None:
177
186
  try:
178
187
  with self._runtime_status_lock:
179
- status = self._runtime_status
180
- last_execution_at = self._last_execution_at
181
- _write_runtime_status(
182
- self.gateway_url,
183
- self.remote_kernel_id,
184
- self.session_id,
185
- status,
186
- last_execution_at,
187
- )
188
+ if self._runtime_status_stop.is_set():
189
+ return
190
+ _write_runtime_status(
191
+ self.gateway_url,
192
+ self.remote_kernel_id,
193
+ self.session_id,
194
+ self._runtime_status,
195
+ self._last_execution_at,
196
+ )
188
197
  except Exception as exc: # noqa: BLE001 - status must not break the kernel
189
198
  self._log(
190
199
  f"Local runtime status update failed: {type(exc).__name__}: {exc}"
@@ -195,7 +204,7 @@ class SuperKernel(Kernel):
195
204
  self._runtime_status = status
196
205
  if status == "busy":
197
206
  self._last_execution_at = time.time()
198
- self._publish_runtime_status()
207
+ self._publish_runtime_status()
199
208
 
200
209
  def _runtime_status_heartbeat(self) -> None:
201
210
  while not self._runtime_status_stop.wait(5.0):
@@ -205,6 +214,11 @@ class SuperKernel(Kernel):
205
214
  digest = hashlib.sha256(code.encode("utf-8")).hexdigest()
206
215
  if self._pending_code_digest == digest and self._pending_execution_id:
207
216
  return self._pending_execution_id
217
+ if self._pending_execution_id:
218
+ raise UnresolvedExecutionError(
219
+ "A previous execution is unresolved. Re-run its original code to "
220
+ "reconnect, or restart the kernel before submitting different code."
221
+ )
208
222
  self._pending_code_digest = digest
209
223
  self._pending_execution_id = f"exec_{uuid.uuid4().hex}"
210
224
  self._pending_reply_id = None
@@ -255,9 +269,16 @@ class SuperKernel(Kernel):
255
269
  except json.JSONDecodeError as exc:
256
270
  raise RuntimeError(f"Gateway sent invalid SSE JSON: {exc}") from exc
257
271
 
258
- if event_type in {"runner_execution_started", "runner_execution_finished"}:
272
+ if event_type == "runner_execution_started":
259
273
  return None, False
260
274
 
275
+ if event_type == "runner_execution_finished":
276
+ succeeded = str(payload.get("status") or "") == "success"
277
+ # The runner atomically publishes a cluster-wide interrupt before
278
+ # this failure event. Return to Jupyter immediately instead of
279
+ # waiting for every interrupted rank to report its final state.
280
+ return (None, False) if succeeded else ("failed", True)
281
+
261
282
  if event_type in {"result", "stream"}:
262
283
  data = str(payload.get("data") or "")
263
284
  if data:
@@ -270,6 +291,12 @@ class SuperKernel(Kernel):
270
291
  return None, str(payload.get("status") or "success") != "success"
271
292
 
272
293
  if event_type == "execution_finished":
294
+ if payload.get("status") == "kernel_restarted":
295
+ self._write_stream(
296
+ "stderr",
297
+ "[Super Kernel] Cluster restarted or recovered. "
298
+ "In-memory notebook state was reset; workspace files were retained.\n",
299
+ )
273
300
  succeeded = str(payload.get("status") or "") == "success"
274
301
  return "completed" if succeeded else "failed", not succeeded
275
302
 
@@ -297,7 +324,7 @@ class SuperKernel(Kernel):
297
324
  "Accept": "text/event-stream",
298
325
  "Content-Type": "application/json",
299
326
  }
300
- access_token = _read_auth_token(self.auth_file)
327
+ access_token = _read_auth_token(self.auth_file, self.gateway_url)
301
328
  headers["Authorization"] = f"Bearer {access_token}"
302
329
  if last_event_id is not None:
303
330
  headers["Last-Event-ID"] = str(last_event_id)
@@ -406,7 +433,7 @@ class SuperKernel(Kernel):
406
433
  f"action={action} execution_id={execution_id}"
407
434
  )
408
435
  headers = {"Content-Type": "application/json"}
409
- access_token = _read_auth_token(self.auth_file)
436
+ access_token = _read_auth_token(self.auth_file, self.gateway_url)
410
437
  headers["Authorization"] = f"Bearer {access_token}"
411
438
  response = requests.post(
412
439
  self.control_endpoint,
@@ -438,13 +465,17 @@ class SuperKernel(Kernel):
438
465
  # Keep the remote stream strictly aligned with Jupyter's local kernel.
439
466
  # This also protects direct/concurrent callers outside the shell loop.
440
467
  with self._execution_lock:
441
- return self._do_execute(
442
- code,
443
- silent,
444
- store_history,
445
- user_expressions,
446
- allow_stdin,
447
- )
468
+ self._silent_execution = silent
469
+ try:
470
+ return self._do_execute(
471
+ code,
472
+ silent,
473
+ store_history,
474
+ user_expressions,
475
+ allow_stdin,
476
+ )
477
+ finally:
478
+ self._silent_execution = False
448
479
 
449
480
  def _do_execute(
450
481
  self,
@@ -454,28 +485,41 @@ class SuperKernel(Kernel):
454
485
  user_expressions: dict[str, Any] | None = None,
455
486
  allow_stdin: bool = False,
456
487
  ) -> dict[str, Any]:
457
- if silent or not code.strip():
488
+ if not code.strip():
458
489
  return self._reply("ok")
459
- self._set_runtime_status("busy")
460
- self._reset_runner_stream()
461
- execution_id = self._execution_id(code)
462
- request = CodeExecutionRequest(
463
- kernel_id=self.remote_kernel_id,
464
- execution_id=execution_id,
465
- session_id=self.session_id,
466
- execution_count=self.execution_count,
467
- code=code,
468
- )
469
490
  try:
491
+ request = CodeExecutionRequest(
492
+ kernel_id=self.remote_kernel_id,
493
+ execution_id="validation",
494
+ session_id=self.session_id,
495
+ execution_count=self.execution_count,
496
+ code=code,
497
+ )
498
+ execution_id = self._execution_id(code)
499
+ request.execution_id = execution_id
500
+ self._set_runtime_status("busy")
501
+ self._reset_runner_stream()
470
502
  succeeded = self._stream_execute(request)
471
503
  except KeyboardInterrupt:
472
- self._interrupt_pending_execution(execution_id)
504
+ if self._pending_execution_id:
505
+ self._interrupt_pending_execution(self._pending_execution_id)
473
506
  self._write_stream("stderr", "[Super Kernel] Execution interrupted.\n")
474
507
  self._clear_pending_execution()
475
508
  self._set_runtime_status("idle")
476
509
  return self._reply("abort")
477
510
  except RequestException as exc:
478
511
  self._log(f"Gateway request failed: {type(exc).__name__}: {exc}")
512
+ if (
513
+ isinstance(exc, HTTPError)
514
+ and exc.response is not None
515
+ and exc.response.status_code == 409
516
+ ):
517
+ self._clear_pending_execution()
518
+ self._write_stream(
519
+ "stderr",
520
+ "[Super Kernel] Previous execution is unavailable; its outcome is unknown. "
521
+ "No code was replayed. Re-run explicitly when the cluster is ready.\n",
522
+ )
479
523
  self._write_stream("stderr", "[Super Kernel] Error.\n")
480
524
  self._set_runtime_status("error")
481
525
  if (
@@ -488,6 +532,10 @@ class SuperKernel(Kernel):
488
532
  StaleSuperKernelError("Saved Super Kernel is unavailable"),
489
533
  )
490
534
  return self._reply("error", RuntimeError("Error"))
535
+ except UnresolvedExecutionError as exc:
536
+ self._write_stream("stderr", f"[Super Kernel] {exc}\n")
537
+ self._set_runtime_status("error")
538
+ return self._reply("error", exc)
491
539
  except Exception as exc: # noqa: BLE001 - kernel protocol boundary
492
540
  self._log(
493
541
  f"Execution failed: {type(exc).__name__}: {exc}\n"
@@ -531,13 +579,14 @@ class SuperKernel(Kernel):
531
579
  except Exception as exc: # noqa: BLE001 - shutdown must complete
532
580
  self._log(f"Super Kernel restart failed: {type(exc).__name__}: {exc}")
533
581
  self._clear_pending_execution()
534
- self._runtime_status_stop.set()
535
582
  try:
536
- _delete_runtime_status(
537
- self.gateway_url,
538
- self.remote_kernel_id,
539
- self.session_id,
540
- )
583
+ with self._runtime_status_lock:
584
+ self._runtime_status_stop.set()
585
+ _delete_runtime_status(
586
+ self.gateway_url,
587
+ self.remote_kernel_id,
588
+ self.session_id,
589
+ )
541
590
  except Exception as exc: # noqa: BLE001 - shutdown must complete
542
591
  self._log(
543
592
  f"Local runtime status cleanup failed: {type(exc).__name__}: {exc}"
@@ -547,7 +596,7 @@ class SuperKernel(Kernel):
547
596
 
548
597
  def _build_arg_parser() -> argparse.ArgumentParser:
549
598
  parser = argparse.ArgumentParser(
550
- prog="python -m superkernel.kernel",
599
+ prog="python -m superkernels.kernel",
551
600
  description="Launch a local Jupyter kernel bound to one Super Kernel ID.",
552
601
  )
553
602
  parser.add_argument("--gateway-url", required=True)
@@ -578,7 +627,7 @@ def main(argv: list[str] | None = None) -> None:
578
627
  argv if argv is not None else sys.argv[1:]
579
628
  )
580
629
  try:
581
- _read_auth_token(args.auth_file)
630
+ _read_auth_token(args.auth_file, args.gateway_url)
582
631
  except RuntimeError as exc:
583
632
  raise SystemExit(str(exc)) from exc
584
633
  SuperKernel.boot = {
@@ -1,3 +1,5 @@
1
+ """Best-effort logging for the local SuperKernels runtime."""
2
+
1
3
  from __future__ import annotations
2
4
 
3
5
  import datetime as dt
@@ -158,7 +158,7 @@ class CodeExecutionResumeRequest(BaseModel):
158
158
 
159
159
 
160
160
  class ControlRequest(BaseModel):
161
- """Body sent by ``superkernel.kernel`` to ``POST /kernel/control``."""
161
+ """Body sent by ``superkernels.kernel`` to ``POST /kernel/control``."""
162
162
 
163
163
  model_config = ConfigDict(extra="forbid")
164
164
 
@@ -1,13 +1,16 @@
1
- """Private local state access required by the SuperKernel runtime."""
1
+ """Private local state access required by the SuperKernels runtime."""
2
2
 
3
3
  from __future__ import annotations
4
4
 
5
+ import datetime as dt
5
6
  import hashlib
6
7
  import json
7
8
  import os
8
9
  import re
9
10
  import stat
10
11
  import time
12
+ from collections.abc import Iterator
13
+ from contextlib import contextmanager
11
14
  from pathlib import Path
12
15
  from urllib.parse import urlsplit, urlunsplit
13
16
 
@@ -45,7 +48,7 @@ def _ensure_private_directory(path: Path) -> Path:
45
48
  try:
46
49
  details = path.lstat()
47
50
  except FileNotFoundError:
48
- path.mkdir(parents=True, exist_ok=False, mode=0o700)
51
+ path.mkdir(parents=True, exist_ok=True, mode=0o700)
49
52
  details = path.lstat()
50
53
  if (
51
54
  not stat.S_ISDIR(details.st_mode)
@@ -57,13 +60,42 @@ def _ensure_private_directory(path: Path) -> Path:
57
60
  return path
58
61
 
59
62
 
60
- def _read_auth_token(auth_file: str) -> str:
63
+ @contextmanager
64
+ def _shared_state_lock() -> Iterator[None]:
65
+ """Use the same directory lock as both management clients."""
66
+ lock_dir = _ensure_private_directory(_shared_state_dir()) / ".write-lock"
67
+ for _ in range(80):
68
+ try:
69
+ lock_dir.mkdir(mode=0o700)
70
+ break
71
+ except FileExistsError:
72
+ try:
73
+ details = lock_dir.lstat()
74
+ if (
75
+ stat.S_ISDIR(details.st_mode)
76
+ and details.st_uid == os.geteuid()
77
+ and time.time() - details.st_mtime > 30
78
+ ):
79
+ lock_dir.rmdir()
80
+ continue
81
+ except OSError:
82
+ continue
83
+ time.sleep(0.025)
84
+ else:
85
+ raise RuntimeError("Super Kernel state is busy")
86
+ try:
87
+ yield
88
+ finally:
89
+ lock_dir.rmdir()
90
+
91
+
92
+ def _read_auth_token(auth_file: str, gateway_url: str | None = None) -> str:
61
93
  """Read the current UI-provisioned Gateway session without caching it."""
62
94
  path = Path(auth_file).expanduser()
63
95
  flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
64
96
  try:
65
97
  descriptor = os.open(path, flags)
66
- except FileNotFoundError as exc:
98
+ except OSError as exc:
67
99
  raise RuntimeError("Super Kernel authentication is unavailable") from exc
68
100
  try:
69
101
  details = os.fstat(descriptor)
@@ -80,8 +112,28 @@ def _read_auth_token(auth_file: str) -> str:
80
112
  if descriptor >= 0:
81
113
  os.close(descriptor)
82
114
  token = payload.get("access_token") if isinstance(payload, dict) else None
83
- if not isinstance(token, str) or not token:
115
+ if not isinstance(token, str) or not token.startswith("sks_"):
84
116
  raise RuntimeError("Super Kernel authentication is unavailable")
117
+ try:
118
+ expires = dt.datetime.fromisoformat(
119
+ payload["expires_at"].replace("Z", "+00:00")
120
+ )
121
+ valid = (
122
+ payload.get("version") == 1
123
+ and payload.get("auth_method") == "api_key"
124
+ and payload.get("token_type") == "Bearer"
125
+ and isinstance(payload.get("user_id"), str)
126
+ and bool(payload["user_id"].strip())
127
+ and expires.tzinfo is not None
128
+ and expires > dt.datetime.now(dt.timezone.utc)
129
+ )
130
+ saved_gateway = _normalize_gateway_url(payload["gateway_url"])
131
+ if gateway_url is not None:
132
+ valid = valid and saved_gateway == _normalize_gateway_url(gateway_url)
133
+ except (KeyError, TypeError, ValueError, AttributeError) as exc:
134
+ raise RuntimeError("Invalid Super Kernel authentication") from exc
135
+ if not valid:
136
+ raise RuntimeError("Super Kernel authentication is invalid or expired")
85
137
  return token
86
138
 
87
139
 
@@ -109,6 +161,19 @@ def _write_runtime_status(
109
161
  last_execution_at: float | None = None,
110
162
  ) -> None:
111
163
  """Publish a non-secret runtime observation for both plugin UIs."""
164
+ with _shared_state_lock():
165
+ _write_runtime_status_locked(
166
+ gateway_url, remote_kernel_id, session_id, status, last_execution_at
167
+ )
168
+
169
+
170
+ def _write_runtime_status_locked(
171
+ gateway_url: str,
172
+ remote_kernel_id: str,
173
+ session_id: str,
174
+ status: str,
175
+ last_execution_at: float | None,
176
+ ) -> None:
112
177
  if status not in {"idle", "busy", "error"}:
113
178
  raise RuntimeError("Invalid local Super Kernel status")
114
179
  root = _ensure_private_directory(_shared_state_dir())
@@ -119,24 +184,26 @@ def _write_runtime_status(
119
184
  )
120
185
  status_file = _runtime_status_file(gateway_url, remote_kernel_id, session_id)
121
186
  temporary = kernel_dir / (f".{status_file.name}.{os.getpid()}.{time.time_ns()}.tmp")
122
- temporary.write_text(
123
- json.dumps(
124
- {
125
- "version": 1,
126
- "gateway_url": _normalize_gateway_url(gateway_url),
127
- "remote_kernel_id": remote_kernel_id,
128
- "session_id": session_id,
129
- "status": status,
130
- "updated_at": time.time(),
131
- "last_execution_at": last_execution_at,
132
- "pid": os.getpid(),
133
- }
134
- ),
135
- encoding="utf-8",
136
- )
137
- temporary.chmod(0o600)
138
- os.replace(temporary, status_file)
139
- status_file.chmod(0o600)
187
+ descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
188
+ try:
189
+ with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
190
+ stream.write(
191
+ json.dumps(
192
+ {
193
+ "version": 1,
194
+ "gateway_url": _normalize_gateway_url(gateway_url),
195
+ "remote_kernel_id": remote_kernel_id,
196
+ "session_id": session_id,
197
+ "status": status,
198
+ "updated_at": time.time(),
199
+ "last_execution_at": last_execution_at,
200
+ "pid": os.getpid(),
201
+ }
202
+ ),
203
+ )
204
+ os.replace(temporary, status_file)
205
+ finally:
206
+ temporary.unlink(missing_ok=True)
140
207
 
141
208
 
142
209
  def _delete_runtime_status(
@@ -144,7 +211,7 @@ def _delete_runtime_status(
144
211
  remote_kernel_id: str,
145
212
  session_id: str,
146
213
  ) -> None:
147
- try:
148
- _runtime_status_file(gateway_url, remote_kernel_id, session_id).unlink()
149
- except FileNotFoundError:
150
- pass
214
+ with _shared_state_lock():
215
+ _runtime_status_file(gateway_url, remote_kernel_id, session_id).unlink(
216
+ missing_ok=True
217
+ )
@@ -1,12 +1,12 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: superkernels
3
- Version: 0.9.0
3
+ Version: 0.9.6
4
4
  Summary: A local ipykernel adapter for remote Super Kernel runtimes.
5
5
  Author-email: Leo Li <leoustc@icloud.com>
6
6
  License-Expression: Apache-2.0
7
7
  Project-URL: Homepage, https://superkernel.leoustc.com
8
- Project-URL: Source, https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernel
9
- Classifier: Development Status :: 3 - Alpha
8
+ Project-URL: Source, https://github.com/leoustc/collection-pip/tree/main/jupyter-superkernel/superkernels
9
+ Classifier: Development Status :: 4 - Beta
10
10
  Classifier: Programming Language :: Python :: 3
11
11
  Classifier: Programming Language :: Python :: 3 :: Only
12
12
  Classifier: Operating System :: OS Independent
@@ -38,6 +38,16 @@ PyTorch environment.
38
38
  - Keeps authentication tokens out of Jupyter kernelspecs and command-line
39
39
  arguments.
40
40
 
41
+ This is the shared host adapter for both the VS Code and Jupyter plugins.
42
+ Local runtime records describe notebook sessions, not remote runner health.
43
+ Publication and shutdown cleanup use the shared client-state lock, and each
44
+ record retains its last cell-start timestamp across heartbeat refreshes.
45
+
46
+ Silent requests still execute but suppress notebook output. If a connection
47
+ failure leaves an execution unresolved, rerun its original code to resume it;
48
+ different code is rejected until the previous result is recovered or the kernel
49
+ is restarted. Code is never automatically replayed after a lost reply handle.
50
+
41
51
  ## Installation
42
52
 
43
53
  SuperKernel requires Python 3.10 or newer.
@@ -54,8 +64,7 @@ This package contains the notebook runtime only. Gateway login, remote runtime
54
64
  creation, and local kernel registration are handled by a compatible
55
65
  SuperKernel client integration.
56
66
 
57
- The PyPI distribution is named `superkernels`. The Python import remains
58
- `superkernel`.
67
+ The PyPI distribution and Python import are both named `superkernels`.
59
68
 
60
69
  ## Python API
61
70
 
@@ -63,13 +72,13 @@ The kernel class is available for integrations that need to inspect or extend
63
72
  the local runtime:
64
73
 
65
74
  ```python
66
- from superkernel import SuperKernel
75
+ from superkernels import SuperKernel
67
76
  ```
68
77
 
69
78
  The supported process entry point for a configured Jupyter kernelspec is:
70
79
 
71
80
  ```text
72
- python -m superkernel.kernel
81
+ python -m superkernels.kernel
73
82
  ```
74
83
 
75
84
  Its connection and authentication arguments are generated by the client
@@ -81,10 +90,19 @@ SuperKernel reads authentication from a client-managed local file immediately
81
90
  before each remote request. Credentials are not embedded in the kernelspec,
82
91
  stored by this package, or passed directly on the process command line. The
83
92
  runtime fails closed when its authentication file is missing or invalid.
93
+ It validates the version-1 session metadata, expiry, and Gateway binding before
94
+ every request; renewing the private file takes effect without restarting it.
95
+
96
+ ## Tests
97
+
98
+ Run `python -m pytest -q superkernels/tests` from the repository root with
99
+ the runtime and pytest installed. Tests use temporary private state and do not
100
+ read or modify saved user credentials.
84
101
 
85
102
  ## Project status
86
103
 
87
- Version 0.9.0 is the first public alpha release. The package is suitable for
88
- evaluation, but its integration contract may still evolve before 1.0.
104
+ Version 0.9.6 is the current public beta release. Version 0.9.5 remains the
105
+ first beta, version 0.9.0 remains the compatibility baseline, and the
106
+ integration contract may still evolve before 1.0.
89
107
 
90
108
  Learn more at [superkernel.leoustc.com](https://superkernel.leoustc.com).
@@ -0,0 +1,14 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/superkernels/__init__.py
5
+ src/superkernels/kernel.py
6
+ src/superkernels/logfile.py
7
+ src/superkernels/schema.py
8
+ src/superkernels/state.py
9
+ src/superkernels.egg-info/PKG-INFO
10
+ src/superkernels.egg-info/SOURCES.txt
11
+ src/superkernels.egg-info/dependency_links.txt
12
+ src/superkernels.egg-info/requires.txt
13
+ src/superkernels.egg-info/top_level.txt
14
+ tests/test_adapter.py
@@ -0,0 +1 @@
1
+ superkernels
@@ -0,0 +1,193 @@
1
+ """Host-neutral runtime regressions; never touch real client credentials."""
2
+
3
+ import datetime as dt
4
+ import json
5
+ import threading
6
+ from concurrent.futures import ThreadPoolExecutor
7
+
8
+ import pytest
9
+
10
+ from superkernels import kernel as runtime
11
+ from superkernels import state
12
+
13
+ GATEWAY = "https://example.test"
14
+
15
+
16
+ @pytest.fixture
17
+ def adapter(monkeypatch, tmp_path):
18
+ monkeypatch.setenv("SUPERKERNEL_HOME", str(tmp_path / "state"))
19
+ host = runtime.SuperKernel.__new__(runtime.SuperKernel)
20
+ host.gateway_url = GATEWAY
21
+ host.remote_kernel_id = "rk_test"
22
+ host.session_id = "notebook-a"
23
+ host._runtime_status = "idle"
24
+ host._last_execution_at = None
25
+ host._runtime_status_lock = threading.RLock()
26
+ host._runtime_status_stop = threading.Event()
27
+ host._execution_lock = threading.Lock()
28
+ host._silent_execution = False
29
+ host._pending_execution_id = None
30
+ host._pending_code_digest = None
31
+ host._pending_reply_id = None
32
+ host._pending_cursor_id = None
33
+ host._runner_virtual_newlines = set()
34
+ host._log = lambda _: None
35
+ host.execution_count = 1
36
+ host._reply = lambda status, error=None: {"status": status}
37
+ host._write_stream = lambda *args: None
38
+ return host
39
+
40
+
41
+ def test_status_write_and_delete_take_shared_lock(monkeypatch, tmp_path):
42
+ monkeypatch.setenv("SUPERKERNEL_HOME", str(tmp_path))
43
+ original = state._write_runtime_status_locked
44
+
45
+ def check(*args):
46
+ assert (tmp_path / ".write-lock").is_dir()
47
+ original(*args)
48
+
49
+ monkeypatch.setattr(state, "_write_runtime_status_locked", check)
50
+ state._write_runtime_status(GATEWAY, "rk_test", "a", "busy", 12)
51
+ record = state._runtime_status_file(GATEWAY, "rk_test", "a")
52
+ assert record.stat().st_mode & 0o777 == 0o600
53
+ assert json.loads(record.read_text())["last_execution_at"] == 12
54
+ state._delete_runtime_status(GATEWAY, "rk_test", "a")
55
+ assert not record.exists()
56
+ assert not (tmp_path / ".write-lock").exists()
57
+
58
+
59
+ def test_heartbeat_cannot_overwrite_transition_or_recreate_shutdown(
60
+ adapter, monkeypatch
61
+ ):
62
+ entered = threading.Event()
63
+ release = threading.Event()
64
+ writes = []
65
+
66
+ def write(*args):
67
+ if not writes:
68
+ entered.set()
69
+ assert release.wait(2)
70
+ writes.append(args[3:])
71
+
72
+ monkeypatch.setattr(runtime, "_write_runtime_status", write)
73
+ with ThreadPoolExecutor(2) as pool:
74
+ old = pool.submit(adapter._publish_runtime_status)
75
+ assert entered.wait(2)
76
+ new = pool.submit(adapter._set_runtime_status, "busy")
77
+ release.set()
78
+ old.result(2)
79
+ new.result(2)
80
+ assert [entry[0] for entry in writes] == ["idle", "busy"]
81
+ assert writes[-1][1] is not None
82
+ adapter.do_shutdown()
83
+ adapter._publish_runtime_status()
84
+ assert len(writes) == 2
85
+
86
+
87
+ def test_notebooks_keep_separate_records(monkeypatch, tmp_path):
88
+ monkeypatch.setenv("SUPERKERNEL_HOME", str(tmp_path))
89
+ with ThreadPoolExecutor(2) as pool:
90
+ jobs = [
91
+ pool.submit(
92
+ state._write_runtime_status, GATEWAY, "rk_test", session, status
93
+ )
94
+ for session, status in [("a", "busy"), ("b", "idle")]
95
+ ]
96
+ for job in jobs:
97
+ job.result(3)
98
+ for session, expected in [("a", "busy"), ("b", "idle")]:
99
+ record = state._runtime_status_file(GATEWAY, "rk_test", session)
100
+ assert json.loads(record.read_text())["status"] == expected
101
+
102
+
103
+ @pytest.fixture
104
+ def auth_file(tmp_path):
105
+ path = tmp_path / "auth.json"
106
+ payload = dict(
107
+ version=1,
108
+ gateway_url=GATEWAY,
109
+ auth_method="api_key",
110
+ token_type="Bearer",
111
+ access_token="sks_test",
112
+ user_id="user",
113
+ expires_at=(
114
+ dt.datetime.now(dt.timezone.utc) + dt.timedelta(hours=1)
115
+ ).isoformat(),
116
+ )
117
+ path.write_text(json.dumps(payload))
118
+ path.chmod(0o600)
119
+ return path, payload
120
+
121
+
122
+ @pytest.mark.parametrize(
123
+ "field,value",
124
+ [
125
+ ("version", 2),
126
+ ("token_type", "Other"),
127
+ ("auth_method", "other"),
128
+ ("user_id", ""),
129
+ ("access_token", "sk_wrong"),
130
+ ("gateway_url", "https://other.test"),
131
+ ("expires_at", "2000-01-01T00:00:00+00:00"),
132
+ ("expires_at", "2999-01-01T00:00:00"),
133
+ ("expires_at", "bad"),
134
+ ],
135
+ )
136
+ def test_auth_rejects_invalid_state(auth_file, field, value):
137
+ path, payload = auth_file
138
+ payload[field] = value
139
+ path.write_text(json.dumps(payload))
140
+ with pytest.raises(RuntimeError):
141
+ state._read_auth_token(str(path), GATEWAY)
142
+
143
+
144
+ def test_auth_renewal_and_file_safety(auth_file, tmp_path):
145
+ path, payload = auth_file
146
+ assert state._read_auth_token(str(path), GATEWAY) == "sks_test"
147
+ payload["access_token"] = "sks_renewed"
148
+ path.write_text(json.dumps(payload))
149
+ assert state._read_auth_token(str(path), GATEWAY) == "sks_renewed"
150
+ link = tmp_path / "link.json"
151
+ link.symlink_to(path)
152
+ with pytest.raises(RuntimeError):
153
+ state._read_auth_token(str(link), GATEWAY)
154
+ path.chmod(0o644)
155
+ with pytest.raises(RuntimeError):
156
+ state._read_auth_token(str(path), GATEWAY)
157
+
158
+
159
+ def test_silent_request_executes_and_suppresses_output(adapter):
160
+ sent = []
161
+ executed = []
162
+ adapter._write_stream = runtime.SuperKernel._write_stream.__get__(adapter)
163
+ adapter.send_response = lambda *args: sent.append(args)
164
+
165
+ def execute(request):
166
+ executed.append(request.code)
167
+ adapter._write_stream("stdout", "hidden")
168
+ return True
169
+
170
+ adapter._stream_execute = execute
171
+ assert adapter.do_execute("x = 42", True)["status"] == "ok"
172
+ assert executed == ["x = 42"]
173
+ assert sent == []
174
+ assert not adapter._silent_execution
175
+
176
+
177
+ def test_invalid_request_does_not_leave_busy_or_pending(adapter):
178
+ adapter._stream_execute = lambda _: pytest.fail("must not submit invalid code")
179
+ assert adapter.do_execute("x" * 1_000_001, False)["status"] == "error"
180
+ assert adapter._runtime_status == "error"
181
+ assert adapter._pending_execution_id is None
182
+
183
+
184
+ def test_different_code_preserves_unresolved_handle(adapter):
185
+ execution_id = adapter._execution_id("original")
186
+ adapter._pending_reply_id = "skr_previous"
187
+ adapter._pending_cursor_id = "12-0"
188
+ adapter._stream_execute = lambda _: pytest.fail("must not submit different code")
189
+ assert adapter.do_execute("different", False)["status"] == "error"
190
+ assert adapter._pending_execution_id == execution_id
191
+ assert adapter._pending_reply_id == "skr_previous"
192
+ assert adapter._pending_cursor_id == "12-0"
193
+ assert adapter._execution_id("original") == execution_id
@@ -1,13 +0,0 @@
1
- LICENSE
2
- README.md
3
- pyproject.toml
4
- src/superkernel/__init__.py
5
- src/superkernel/kernel.py
6
- src/superkernel/logfile.py
7
- src/superkernel/schema.py
8
- src/superkernel/state.py
9
- src/superkernels.egg-info/PKG-INFO
10
- src/superkernels.egg-info/SOURCES.txt
11
- src/superkernels.egg-info/dependency_links.txt
12
- src/superkernels.egg-info/requires.txt
13
- src/superkernels.egg-info/top_level.txt
@@ -1 +0,0 @@
1
- superkernel
File without changes
File without changes