gpubk 0.1.0__tar.gz

gpubk-0.1.0/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+All notable changes are documented here. The project follows Semantic Versioning once a public release is published.
+
+## 0.1.0 - 2026-07-12
+
+- Use a concise English README by default and ship a matching Simplified Chinese guide in source distributions.
+- Adopt GPUbk as the public project brand and `gpubk` as the PyPI distribution while retaining the `bk` command and protocol namespace.
+- Add 5-minute shared/exclusive scheduling with atomic queueing and VRAM admission.
+- Add compact curses TUI, date/weekday timeline, shared lanes, and interactive Add/Edit.
+- Add NVML monitoring, privacy-safe usage audits, historical load forecasts, and live-aware placement.
+- Add per-user scheduled job workers with private specs and automatic `CUDA_VISIBLE_DEVICES`.
+- Add WAL recovery, configurable private/shared file modes, backups, and concurrent-process tests.
+- Add stable JSON agent context/recommendation, advisory external allocator protocol, MCP server, and bundled Codex Skill.
+- Bind audit display names to the process UID and render user-level systemd units with the active Python installation.
+- Reuse a parsed per-GPU reservation index, tail-read audit logs, and bound the hot-ledger retention window.
+- Expose GPU model and temperature in privacy-safe Agent and MCP context.
+- Auto-detect visible GPUs when no administrator count is configured, while preserving explicit limits.
+- Keep TUI headers and keyboard hints complete on terminals as narrow as 72 columns.
+- Pin all third-party GitHub Actions to immutable commits and test that release invariant.
+- Fail closed on an unreadable ledger without a valid backup, while still allowing durable journal recovery.
+- Add machine-readable MCP risk annotations for read-only, idempotent, destructive, and closed-world tools.
+- Bind shared ledgers to one scheduling and storage policy so per-user environment overrides cannot silently change capacity rules.
+- Keep ordinary read-only CLI, Agent, and MCP calls free of empty-directory initialization side effects while preserving durable WAL recovery.
+- Reject untrusted shared configuration, symbolic-link and special-file redirection across ledger, audit, and private job paths.
+- Bound external allocator output and terminate its whole process group on timeout or protocol overflow.
+- Adopt Apache-2.0 and publish maintainer and GitHub project metadata for the `gpubk` distribution.
+- Add a paged colorized TUI help center, explicit `f`/`g`/`r` guidance, and an embedded Quick Tour.
+- Center the live timeline around a visible NOW marker, permit read-only reservation history browsing, and keep GPU capacity/utilization/memory columns aligned.
+- Reject edits after a reservation starts or when an exact replacement start is in the past across both TUI and scheduler APIs.
+- Add auto-detected dark/light TUI themes, a live theme toggle, and terminal-default neutral text for readable black and white backgrounds.
+- Normalize editable and installed coverage paths in CI so subprocess data cannot count the same package twice and falsely fail the coverage gate.
+- Make secure-config tests independent of the host umask used on shared lab servers.
+- Show allocatable free VRAM in compact GPU rows and never render partially truncated trailing metrics.
+- Add token-free Trusted Publishing through TestPyPI verification and protected PyPI promotion of one immutable artifact.
+- Add package-structure, metadata, and medium/high-severity static security gates to CI.
+- Replace optimization-sensitive production assertions with explicit fail-closed runtime checks.
+- Keep Agent recommendations and telemetry-history reads side-effect free when the data directory does not exist.
+- Add retry-safe structured Agent and MCP edits, structured Agent cancellation, capability discovery, and operation-intent mismatch rejection.

gpubk-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

gpubk-0.1.0/MANIFEST.in

@@ -0,0 +1,8 @@
+include CHANGELOG.md
+include LICENSE
+include README.zh-CN.md
+include RELEASING.md
+include SECURITY.md
+recursive-include src/bk/data *
+recursive-include tests test_*.py
+recursive-include benchmarks *.py

gpubk-0.1.0/PKG-INFO

@@ -0,0 +1,298 @@
+Metadata-Version: 2.4
+Name: gpubk
+Version: 0.1.0
+Summary: Offline, load-aware GPU booking CLI, curses TUI, worker, and agent interface.
+Author-email: lzzmm <cortexcyh@gmail.com>
+Maintainer-email: lzzmm <cortexcyh@gmail.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/lzzmm/gpubk
+Project-URL: Repository, https://github.com/lzzmm/gpubk
+Project-URL: Issues, https://github.com/lzzmm/gpubk/issues
+Project-URL: Changelog, https://github.com/lzzmm/gpubk/blob/main/CHANGELOG.md
+Keywords: gpu,scheduler,booking,tui,nvml,mcp,agent
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Environment :: Console :: Curses
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: gpu
+Requires-Dist: nvidia-ml-py>=12; extra == "gpu"
+Provides-Extra: mcp
+Requires-Dist: mcp<2,>=1.28.1; extra == "mcp"
+Provides-Extra: all
+Requires-Dist: nvidia-ml-py>=12; extra == "all"
+Requires-Dist: mcp<2,>=1.28.1; extra == "all"
+Dynamic: license-file
+
+# GPUbk
+
+**English** | [简体中文](https://github.com/lzzmm/gpubk/blob/main/README.zh-CN.md)
+
+GPUbk is a GPU booking tool for shared Linux servers. The package is named
+`gpubk`; the command is the shorter `bk`.
+
+It works offline, stores data in local files, and has no required runtime
+dependencies. Users can book GPUs from a plain terminal prompt, a curses TUI,
+JSON commands, or an optional local MCP server.
+
+## What It Covers
+
+- Shared and exclusive reservations in 5-minute intervals.
+- Automatic queueing, live GPU awareness, and per-GPU VRAM budgets.
+- A compact timeline that works on dark and light terminals.
+- Scheduled commands with automatic `CUDA_VISIBLE_DEVICES`.
+- NVML process monitoring and recent-load history.
+- Stable JSON, MCP tools, a bundled Codex Skill, and an optional external allocator.
+- Atomic file transactions, UID ownership checks, backups, and an append-only audit log.
+
+GPUbk is a cooperative scheduler. It does not replace Linux device permissions
+or stop a user with direct access to `/dev/nvidia*` from bypassing the tool.
+
+## Install
+
+GPUbk requires Python 3.10 or newer.
+
+```bash
+python3 -m pip install gpubk          # core CLI and TUI; no dependencies
+python3 -m pip install 'gpubk[gpu]'  # add low-overhead NVML telemetry
+python3 -m pip install 'gpubk[mcp]'  # add the local MCP server
+python3 -m pip install 'gpubk[all]'  # both optional extras
+```
+
+Verify the installation:
+
+```bash
+bk --version
+bk --help
+```
+
+## Book GPUs
+
+Shared mode is the default:
+
+```bash
+bk 1 30m                         # one GPU for 30 minutes
+bk 2 1h30m --mem 12g            # 12 GiB expected VRAM per GPU
+bk s 1 2h --gpu 3               # explicit shared mode on GPU 3
+bk x 2 4h                        # exclusive mode
+bk 1 1h --start 2026-07-12T20:00:00+08:00
+```
+
+Manage your reservations with a list number or short ID:
+
+```bash
+bk l
+bk e 1 --duration 2h
+bk d 1
+bk doctor                         # read-only ledger checks
+```
+
+Scheduling rules are intentionally small:
+
+- Start times and durations use 5-minute boundaries.
+- Without `--start`, GPUbk picks the earliest valid slot and prints `queued:`
+  when the reservation starts later.
+- With `--start`, the time is exact. A conflict returns an error instead of
+  silently moving the reservation.
+- Shared capacity is counted per overlapping reservation. Exclusive
+  reservations cannot overlap anything.
+- `--mem` is expected VRAM **per GPU**. Administrators can require it for all
+  shared reservations.
+- Times shown to users are local. The ledger stores UTC.
+
+Automatic placement considers reservations, physical free VRAM, current GPU
+processes, recent load, and near-future booking pressure. A process without a
+reservation is reported and avoided when another suitable GPU is free.
+
+## Terminal Interfaces
+
+`bk` opens a normal line-oriented prompt and keeps the terminal background.
+`bk t` opens the full-screen TUI.
+
+```bash
+bk
+bk t
+```
+
+Useful TUI keys:
+
+| Key | Action |
+| --- | --- |
+| `a` / `e` / `d` | Add, edit, or cancel a reservation |
+| `Tab`, `↑`, `↓` | Move between reservations and GPU details |
+| `←`, `→` | Browse the timeline; move time in Add/Edit |
+| `Space` | Toggle the current GPU in Add/Edit |
+| `+`, `-` | Change timeline zoom or reservation duration |
+| `1`-`9` | Pick a GPU count and jump to the nearest valid slot |
+| `s`, `x` | Switch between shared and exclusive in Add/Edit |
+| `f`, `g` | Find any suitable GPUs, or keep the selected GPUs fixed |
+| `n` | Return to the live `NOW` window |
+| `c` | Toggle the dark/light theme |
+| `?` | Open the paged help and quick tour |
+| `Enter`, `Esc`, `q` | Submit, cancel the current action, or quit |
+
+The timeline can show past reservations, but history is read-only. Add and Edit
+always validate the selected interval again inside the locked scheduler
+transaction.
+
+## Run a Command at Reservation Time
+
+Put the command after `--`:
+
+```bash
+bk 2 1h30m --mem 12g -- python train.py --config exp.yaml
+bk j                    # list scheduled jobs
+bk w                    # run this user's due jobs
+```
+
+The worker sets `CUDA_VISIBLE_DEVICES`, `CUDA_DEVICE_ORDER`,
+`BK_RESERVATION_ID`, and `BK_RESERVED_GPUS`. Commands and working directories
+stay in UID-owned `0600` job specs; they are not written to the shared ledger.
+The worker uses `shell=False`. Use an explicit shell only when shell syntax is
+required:
+
+```bash
+bk 1 30m -- sh -lc 'python train.py > train.log 2>&1'
+```
+
+For unattended jobs, each user can install the bundled systemd user unit:
+
+```bash
+bk service install worker
+systemctl --user daemon-reload
+systemctl --user enable --now bk-worker.service
+```
+
+## Monitoring and Placement
+
+Install the `gpu` extra, then run a single sample or a low-overhead monitor:
+
+```bash
+bk m --once
+bk m
+bk u --rollups
+```
+
+NVML is initialized once and device handles are reused. The monitor records
+bounded load summaries plus process start, stop, and authorization changes; it
+does not append a full snapshot every second. Without NVML, GPUbk falls back to
+`nvidia-smi` with less process detail.
+
+Process status is based on the process UID and active reservation:
+`ok`, `wrong-gpu`, `unreserved`, `unknown`, or `system`. Command lines are
+reduced to safe labels before shared logging.
+
+The monitor also has a user service:
+
+```bash
+bk service install monitor
+systemctl --user daemon-reload
+systemctl --user enable --now bk-monitor.service
+```
+
+## Agents and MCP
+
+Agents should use the versioned JSON interface instead of parsing terminal text:
+
+```bash
+bk agent context --compact
+bk agent recommend 2 1h30m --mem 12g --compact
+bk 2 1h30m --mem 12g --op-id run-20260712-001 --json
+bk agent edit 6e957ef1 --duration 2h --op-id edit-20260712-001 --compact
+bk agent cancel 6e957ef1 --compact
+```
+
+Create and edit operations require a stable operation ID. An identical retry
+returns `status=exists`; reusing the ID for a different write is rejected.
+Recommendations are read-only. Identity always comes from the local process UID.
+
+Run the optional stdio MCP server with:
+
+```bash
+bk-mcp                       # same as: bk mcp
+bk skill install            # installs the bundled Codex Skill
+```
+
+The MCP server provides context, recommendation, create, list, edit, cancel,
+and private job-log tools. It listens on stdio only; each user runs their own
+process. Tool schemas include read-only, idempotent, destructive, and
+closed-world annotations.
+
+An administrator may also set `BK_ALLOCATOR_COMMAND` to a trusted local program
+that reads `bk.allocator.v1` JSON and returns a GPU ordering. Its output is
+advisory: every result still passes the built-in conflict, VRAM, time, UID, and
+transaction checks. See the [Agent protocol](https://github.com/lzzmm/gpubk/blob/main/src/bk/data/codex-skill/gpubk/references/protocol.md).
+
+## Shared Server Setup
+
+Create one setgid directory for the lab group:
+
+```bash
+sudo install -d -m 2770 -o root -g gpuusers /data2/shared/bk
+export BK_DATA_DIR=/data2/shared/bk
+```
+
+Put a root-owned `config.json` in that directory:
+
+```json
+{
+  "gpu_count": 8,
+  "max_shared_users": 2,
+  "queue_search_hours": 168,
+  "ledger_retention_days": 90,
+  "require_shared_memory": true,
+  "shared_memory_reserve_mb": 512,
+  "file_mode": "0660",
+  "dir_mode": "2770"
+}
+```
+
+```bash
+sudo chown root:gpuusers /data2/shared/bk/config.json
+sudo chmod 0644 /data2/shared/bk/config.json
+```
+
+All users and user services must use the same `BK_DATA_DIR`. The first write
+binds scheduling and storage policy into the ledger; clients with conflicting
+settings fail closed. Verify `flock` and atomic rename on the actual NFS or FUSE
+mount before deployment. Every writer must use GPUbk.
+
+See [SECURITY.md](https://github.com/lzzmm/gpubk/blob/main/SECURITY.md) for the supported boundary, file safety, WAL
+recovery, private job specs, MCP isolation, and administrator responsibilities.
+
+## Try It Without a GPU
+
+Booking and the TUI can run with simulated GPU count:
+
+```bash
+BK_DATA_DIR=/tmp/gpubk-demo BK_GPU_COUNT=4 bk t
+BK_DATA_DIR=/tmp/gpubk-demo BK_GPU_COUNT=4 bk 1 30m
+```
+
+The cards show unknown hardware metrics, but scheduling, shared capacity, the
+timeline, Add/Edit, logs, and Agent JSON remain usable.
+
+## Development
+
+```bash
+python3 -m pip install -e '.[mcp,gpu]'
+PYTHONPATH=src python3 -m unittest discover -s tests -p 'test_*.py'
+PYTHONPATH=src python3 benchmarks/scheduler_queue.py
+```
+
+Project documents: [Security](https://github.com/lzzmm/gpubk/blob/main/SECURITY.md) ·
+[Release process](https://github.com/lzzmm/gpubk/blob/main/RELEASING.md) ·
+[Changelog](https://github.com/lzzmm/gpubk/blob/main/CHANGELOG.md) ·
+[Apache-2.0 license](https://github.com/lzzmm/gpubk/blob/main/LICENSE)