hypershell 2.6.6__tar.gz → 2.7.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (185) hide show
  1. hypershell-2.7.1/.github/FUNDING.yml +12 -0
  2. hypershell-2.7.1/.github/workflows/publish.yml +19 -0
  3. hypershell-2.7.1/.github/workflows/tests.yml +24 -0
  4. hypershell-2.7.1/Apptainer +38 -0
  5. hypershell-2.7.1/Dockerfile +34 -0
  6. {hypershell-2.6.6 → hypershell-2.7.1}/PKG-INFO +8 -1
  7. {hypershell-2.6.6 → hypershell-2.7.1}/README.rst +4 -0
  8. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/cluster_desc.rst +2 -0
  9. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_param_ref.rst +4 -0
  10. hypershell-2.7.1/docs/_include/initdb_desc.rst +8 -0
  11. hypershell-2.7.1/docs/_include/initdb_help.rst +20 -0
  12. hypershell-2.7.1/docs/_include/initdb_usage.rst +1 -0
  13. hypershell-2.7.1/docs/_include/submit_desc.rst +16 -0
  14. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/submit_help.rst +8 -5
  15. hypershell-2.7.1/docs/_include/submit_usage.rst +2 -0
  16. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_info_usage.rst +1 -1
  17. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_run_desc.rst +1 -1
  18. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_run_help.rst +1 -1
  19. hypershell-2.7.1/docs/_include/task_run_usage.rst +1 -0
  20. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_search_desc.rst +1 -1
  21. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_search_help.rst +7 -0
  22. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_search_usage.rst +7 -1
  23. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_update_desc.rst +1 -1
  24. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_update_help.rst +2 -2
  25. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_update_usage.rst +2 -2
  26. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_wait_usage.rst +1 -1
  27. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/data.rst +10 -0
  28. hypershell-2.7.1/docs/blog/20250504_2_7_0_release.rst +228 -0
  29. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/index.rst +18 -0
  30. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/index.rst +7 -6
  31. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_info.rst +2 -2
  32. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_run.rst +2 -2
  33. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_search.rst +2 -2
  34. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_submit.rst +2 -2
  35. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_update.rst +2 -2
  36. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/task_wait.rst +2 -2
  37. {hypershell-2.6.6 → hypershell-2.7.1}/docs/getting_started.rst +4 -4
  38. {hypershell-2.6.6 → hypershell-2.7.1}/docs/index.rst +1 -1
  39. {hypershell-2.6.6 → hypershell-2.7.1}/docs/manual.rst +67 -65
  40. {hypershell-2.6.6 → hypershell-2.7.1}/pyproject.toml +19 -5
  41. {hypershell-2.6.6 → hypershell-2.7.1}/share/bash_completion.d/hs +44 -35
  42. {hypershell-2.6.6 → hypershell-2.7.1}/share/man/man1/hs.1 +250 -209
  43. hypershell-2.7.1/share/man/man1/hsx.1 +2080 -0
  44. {hypershell-2.6.6 → hypershell-2.7.1}/share/man/man1/hyper-shell.1 +250 -209
  45. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/__init__.py +39 -16
  46. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/cluster/__init__.py +2 -2
  47. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/config.py +5 -5
  48. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/logging.py +2 -2
  49. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/platform.py +16 -10
  50. hypershell-2.7.1/src/hypershell/core/pretty_print.py +47 -0
  51. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/queue.py +1 -1
  52. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/sys.py +2 -2
  53. hypershell-2.7.1/src/hypershell/core/types.py +57 -0
  54. hypershell-2.7.1/src/hypershell/core/uuid.py +19 -0
  55. hypershell-2.7.1/src/hypershell/data/__init__.py +276 -0
  56. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/data/core.py +17 -2
  57. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/data/model.py +27 -41
  58. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/submit.py +102 -34
  59. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/task.py +69 -36
  60. hypershell-2.7.1/tests/__init__.py +75 -0
  61. hypershell-2.7.1/tests/conftest.py +27 -0
  62. hypershell-2.7.1/tests/test_cluster.py +184 -0
  63. {hypershell-2.6.6/tests/unit/test_database → hypershell-2.7.1/tests/test_data}/test_core.py +3 -1
  64. hypershell-2.7.1/tests/test_data/test_model.py +46 -0
  65. hypershell-2.7.1/tests/test_initdb.py +82 -0
  66. hypershell-2.7.1/tests/test_main.py +179 -0
  67. hypershell-2.7.1/tests/test_server.py +102 -0
  68. hypershell-2.7.1/tests/test_submit.py +214 -0
  69. {hypershell-2.6.6 → hypershell-2.7.1}/uv.lock +981 -1099
  70. hypershell-2.6.6/Dockerfile +0 -27
  71. hypershell-2.6.6/docs/_include/initdb_desc.rst +0 -5
  72. hypershell-2.6.6/docs/_include/initdb_help.rst +0 -9
  73. hypershell-2.6.6/docs/_include/initdb_usage.rst +0 -1
  74. hypershell-2.6.6/docs/_include/submit_desc.rst +0 -9
  75. hypershell-2.6.6/docs/_include/submit_usage.rst +0 -2
  76. hypershell-2.6.6/docs/_include/task_run_usage.rst +0 -1
  77. hypershell-2.6.6/docs/_static/logo.svg +0 -102
  78. hypershell-2.6.6/src/hypershell/core/types.py +0 -32
  79. hypershell-2.6.6/src/hypershell/data/__init__.py +0 -137
  80. hypershell-2.6.6/tests/__init__.py +0 -4
  81. hypershell-2.6.6/tests/unit/__init__.py +0 -4
  82. hypershell-2.6.6/tests/unit/core/__init__.py +0 -2
  83. {hypershell-2.6.6 → hypershell-2.7.1}/.dockerignore +0 -0
  84. {hypershell-2.6.6 → hypershell-2.7.1}/.gitignore +0 -0
  85. {hypershell-2.6.6 → hypershell-2.7.1}/.readthedocs.yaml +0 -0
  86. {hypershell-2.6.6 → hypershell-2.7.1}/CODE_OF_CONDUCT.md +0 -0
  87. {hypershell-2.6.6 → hypershell-2.7.1}/LICENSE +0 -0
  88. {hypershell-2.6.6 → hypershell-2.7.1}/docs/Makefile +0 -0
  89. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/citation.rst +0 -0
  90. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/client_desc.rst +0 -0
  91. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/client_help.rst +0 -0
  92. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/client_usage.rst +0 -0
  93. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/cluster_help.rst +0 -0
  94. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/cluster_usage.rst +0 -0
  95. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_edit_desc.rst +0 -0
  96. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_edit_help.rst +0 -0
  97. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_edit_usage.rst +0 -0
  98. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_get_desc.rst +0 -0
  99. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_get_help.rst +0 -0
  100. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_get_usage.rst +0 -0
  101. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_intro.rst +0 -0
  102. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_intro_alt.rst +0 -0
  103. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_pythonpath.rst +0 -0
  104. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_set_desc.rst +0 -0
  105. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_set_help.rst +0 -0
  106. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_set_usage.rst +0 -0
  107. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_site_vars.rst +0 -0
  108. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_task_env.rst +0 -0
  109. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_task_env_alt.rst +0 -0
  110. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_which_desc.rst +0 -0
  111. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_which_help.rst +0 -0
  112. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/config_which_usage.rst +0 -0
  113. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/database.rst +0 -0
  114. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/database_alt.rst +0 -0
  115. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/desc.rst +0 -0
  116. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/exit_status.rst +0 -0
  117. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/license.rst +0 -0
  118. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/server_desc.rst +0 -0
  119. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/server_help.rst +0 -0
  120. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/server_usage.rst +0 -0
  121. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/signals.rst +0 -0
  122. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_info_desc.rst +0 -0
  123. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_info_help.rst +0 -0
  124. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_submit_desc.rst +0 -0
  125. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_submit_help.rst +0 -0
  126. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_submit_usage.rst +0 -0
  127. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_wait_desc.rst +0 -0
  128. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/task_wait_help.rst +0 -0
  129. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/templates.rst +0 -0
  130. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_include/templates_alt.rst +0 -0
  131. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_static/custom.css +0 -0
  132. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_static/logo-dark-mode.png +0 -0
  133. {hypershell-2.6.6 → hypershell-2.7.1}/docs/_static/logo-light-mode.png +0 -0
  134. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/client.rst +0 -0
  135. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/cluster.rst +0 -0
  136. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/index.rst +0 -0
  137. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/server.rst +0 -0
  138. {hypershell-2.6.6 → hypershell-2.7.1}/docs/api/submit.rst +0 -0
  139. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20230329_2_2_0_release.rst +0 -0
  140. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20230413_2_3_0_release.rst +0 -0
  141. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20230602_2_4_0_release.rst +0 -0
  142. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20240518_2_5_0_release.rst +0 -0
  143. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20240706_2_5_2_release.rst +0 -0
  144. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20241115_2_6_0_release.rst +0 -0
  145. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20241115_announce_logo.rst +0 -0
  146. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20241231_2_6_1_release.rst +0 -0
  147. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20250215_2_6_5_release.rst +0 -0
  148. {hypershell-2.6.6 → hypershell-2.7.1}/docs/blog/20250405_2_6_6_release.rst +0 -0
  149. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/client.rst +0 -0
  150. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/cluster.rst +0 -0
  151. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/config_edit.rst +0 -0
  152. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/config_get.rst +0 -0
  153. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/config_set.rst +0 -0
  154. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/config_which.rst +0 -0
  155. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/initdb.rst +0 -0
  156. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/server.rst +0 -0
  157. {hypershell-2.6.6 → hypershell-2.7.1}/docs/cli/submit.rst +0 -0
  158. {hypershell-2.6.6 → hypershell-2.7.1}/docs/conf.py +0 -0
  159. {hypershell-2.6.6 → hypershell-2.7.1}/docs/config.rst +0 -0
  160. {hypershell-2.6.6 → hypershell-2.7.1}/docs/database.rst +0 -0
  161. {hypershell-2.6.6 → hypershell-2.7.1}/docs/install.rst +0 -0
  162. {hypershell-2.6.6 → hypershell-2.7.1}/docs/logging.rst +0 -0
  163. {hypershell-2.6.6 → hypershell-2.7.1}/docs/roadmap.rst +0 -0
  164. {hypershell-2.6.6 → hypershell-2.7.1}/docs/templates.rst +0 -0
  165. {hypershell-2.6.6 → hypershell-2.7.1}/docs/tutorial/advanced.rst +0 -0
  166. {hypershell-2.6.6 → hypershell-2.7.1}/docs/tutorial/basic.rst +0 -0
  167. {hypershell-2.6.6 → hypershell-2.7.1}/docs/tutorial/distributed.rst +0 -0
  168. {hypershell-2.6.6 → hypershell-2.7.1}/docs/tutorial/hybrid.rst +0 -0
  169. {hypershell-2.6.6 → hypershell-2.7.1}/share/zsh/site-functions/_hs +0 -0
  170. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/client.py +0 -0
  171. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/cluster/local.py +0 -0
  172. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/cluster/remote.py +0 -0
  173. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/cluster/ssh.py +0 -0
  174. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/__init__.py +0 -0
  175. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/config.py +0 -0
  176. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/exceptions.py +0 -0
  177. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/fsm.py +0 -0
  178. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/heartbeat.py +0 -0
  179. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/remote.py +0 -0
  180. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/signal.py +0 -0
  181. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/tag.py +0 -0
  182. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/template.py +0 -0
  183. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/core/thread.py +0 -0
  184. {hypershell-2.6.6 → hypershell-2.7.1}/src/hypershell/server.py +0 -0
  185. {hypershell-2.6.6/tests/unit/test_database → hypershell-2.7.1/tests/test_data}/__init__.py +0 -0
@@ -0,0 +1,12 @@
1
+ # These are supported funding model platforms
2
+
3
+ github: glentner
4
+ patreon: # Replace with a single Patreon username
5
+ open_collective: # Replace with a single Open Collective username
6
+ ko_fi: # Replace with a single Ko-fi username
7
+ tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8
+ community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9
+ liberapay: # Replace with a single Liberapay username
10
+ issuehunt: # Replace with a single IssueHunt username
11
+ otechie: # Replace with a single Otechie username
12
+ custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
@@ -0,0 +1,19 @@
1
+ name: Publish
2
+ on:
3
+ release:
4
+ types: [created]
5
+
6
+ jobs:
7
+ publish:
8
+ runs-on: ubuntu-latest
9
+ steps:
10
+ - uses: actions/checkout@v4
11
+
12
+ - name: Install uv
13
+ uses: astral-sh/setup-uv@v5
14
+ with:
15
+ enable-cache: true
16
+ cache-dependency-glob: "uv.lock"
17
+
18
+ - name: Publish to PyPI
19
+ run: uv build && uv publish --token ${{ secrets.PYPI_TOKEN }}
@@ -0,0 +1,24 @@
1
+ name: Tests
2
+ on:
3
+ push:
4
+ branches: [ master, develop ]
5
+ pull_request:
6
+ branches: [ develop ]
7
+
8
+ jobs:
9
+ tests:
10
+ runs-on: ubuntu-latest
11
+ strategy:
12
+ matrix:
13
+ python-version: [ '3.9', '3.10', '3.11', '3.12', '3.13' ]
14
+ steps:
15
+ - uses: actions/checkout@v4
16
+
17
+ - name: Install uv
18
+ uses: astral-sh/setup-uv@v5
19
+ with:
20
+ enable-cache: true
21
+ cache-dependency-glob: "uv.lock"
22
+
23
+ - name: Run tests
24
+ run: uv run --python ${{ matrix.python-version }} pytest -v
@@ -0,0 +1,38 @@
1
+ Bootstrap: docker
2
+ From: python:3.13-slim
3
+ Stage: final
4
+
5
+ %labels
6
+ Version="2.7.0"
7
+ Author="glentner@purdue.edu"
8
+
9
+ %files
10
+ ./src /app/src
11
+ ./uv.lock /app/uv.lock
12
+ ./LICENSE /app/LICENSE
13
+ ./README.rst /app/README.rst
14
+ ./pyproject.toml /app/pyproject.toml
15
+
16
+ %post
17
+ export DEBIAN_FRONTEND=noninteractive
18
+ apt-get -yqq update && apt-get -yqq upgrade
19
+ apt-get -yqq install curl build-essential postgresql libpq-dev
20
+ rm -rf /var/lib/apt/lists/*
21
+
22
+ curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="/bin" sh
23
+
24
+ export UV_LINK_MODE=hardlink
25
+ export UV_CACHE_DIR=/opt/uv-cache
26
+ export UV_COMPILE_BYTECODE=1
27
+ export UV_PROJECT_ENVIRONMENT=/opt/hypershell
28
+
29
+ cd /app
30
+ /bin/uv sync --frozen --all-packages --python 3.13
31
+ ln -sf /opt/hypershell/bin/hs /bin/hs
32
+
33
+ %environment
34
+ export HYPERSHELL_LOGGING_LEVEL=TRACE
35
+ export HYPERSHELL_LOGGING_STYLE=SYSTEM
36
+
37
+ %runscript
38
+ exec /bin/hs "$@"
@@ -0,0 +1,34 @@
1
+ FROM python:3.13-slim
2
+ COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
3
+
4
+ LABEL version="2.7.0"
5
+ LABEL authors="glentner@purdue.edu"
6
+ LABEL org.opencontainers.image.source="https://github.com/hypershell/hypershell"
7
+ LABEL org.opencontainers.image.description="HyperShell Base Image"
8
+ LABEL org.opencontainers.image.licenses="Apache-2.0"
9
+
10
+ RUN DEBIAN_FRONTEND=noninteractive \
11
+ apt-get -yqq update && apt-get -yqq upgrade && \
12
+ apt-get -yqq install build-essential postgresql libpq-dev && \
13
+ rm -rf /var/lib/apt/lists/*
14
+
15
+ RUN addgroup --gid 1001 --system hypershell && \
16
+ adduser --no-create-home --shell /bin/false --disabled-password --uid 1001 --system --group hypershell
17
+
18
+ RUN mkdir -p /var/lib/hypershell /var/log/hypershell
19
+
20
+ ENV UV_CACHE_DIR=/opt/uv-cache \
21
+ UV_LINK_MODE=hardlink \
22
+ UV_COMPILE_BYTECODE=1 \
23
+ UV_PROJECT_ENVIRONMENT=/opt/hypershell \
24
+ HYPERSHELL_LOGGING_LEVEL=TRACE \
25
+ HYPERSHELL_LOGGING_STYLE=SYSTEM
26
+
27
+ WORKDIR /app
28
+ COPY . .
29
+ RUN /bin/rm -rf .venv .git
30
+ RUN uv sync --frozen --all-packages --python 3.13
31
+
32
+ USER hypershell
33
+ ENTRYPOINT ["/opt/hypershell/bin/hs", "server"]
34
+ CMD ["--forever"]
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: hypershell
3
- Version: 2.6.6
3
+ Version: 2.7.1
4
4
  Summary: A cross-platform, high-throughput computing utility for processing shell commands over a distributed, asynchronous queue.
5
5
  Author-email: Geoffrey Lentner <glentner@purdue.edu>
6
6
  License-Expression: Apache-2.0
@@ -16,6 +16,7 @@ Classifier: Programming Language :: Python :: 3.10
16
16
  Classifier: Programming Language :: Python :: 3.11
17
17
  Classifier: Programming Language :: Python :: 3.12
18
18
  Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Programming Language :: Python :: 3.14
19
20
  Classifier: Topic :: Utilities
20
21
  Requires-Python: >=3.9
21
22
  Requires-Dist: cmdkit[toml]>=2.7.7
@@ -26,6 +27,8 @@ Requires-Dist: sqlalchemy>=2.0.29
26
27
  Requires-Dist: tomlkit>=0.13.2
27
28
  Provides-Extra: postgres
28
29
  Requires-Dist: psycopg2>=2.9.9; extra == 'postgres'
30
+ Provides-Extra: uuid7
31
+ Requires-Dist: uuid-utils>=0.10.0; extra == 'uuid7'
29
32
  Description-Content-Type: text/x-rst
30
33
 
31
34
  HyperShell v2: Distributed Task Execution for HPC
@@ -39,6 +42,10 @@ HyperShell v2: Distributed Task Execution for HPC
39
42
  :target: https://github.com/hypershell/hypershell/releases
40
43
  :alt: Github Release
41
44
 
45
+ .. image:: https://static.pepy.tech/badge/hypershell/month
46
+ :target: https://pypi.org/project/hypershell/
47
+ :alt: PyPI Monthly Downloads
48
+
42
49
  .. image:: https://img.shields.io/badge/Python-3.9+-blue.svg
43
50
  :target: https://www.python.org/downloads
44
51
  :alt: Python Versions
@@ -9,6 +9,10 @@ HyperShell v2: Distributed Task Execution for HPC
9
9
  :target: https://github.com/hypershell/hypershell/releases
10
10
  :alt: Github Release
11
11
 
12
+ .. image:: https://static.pepy.tech/badge/hypershell/month
13
+ :target: https://pypi.org/project/hypershell/
14
+ :alt: PyPI Monthly Downloads
15
+
12
16
  .. image:: https://img.shields.io/badge/Python-3.9+-blue.svg
13
17
  :target: https://www.python.org/downloads
14
18
  :alt: Python Versions
@@ -13,3 +13,5 @@ left off.
13
13
 
14
14
  Use ``--autoscaling`` with either *fixed* or *dynamic* to run a persistent, elastically scalable
15
15
  cluster using an external ``--launcher`` to bring up clients as needed.
16
+
17
+ Use ``hsx`` in place of ``hs cluster``.
@@ -87,6 +87,10 @@
87
87
  Specify additional connection details for the underlying SQL dialect provider,
88
88
  e.g., ``sqlite3`` or ``psycopg2``.
89
89
 
90
+ ``[pragmas]``
91
+ Specify one or more pragmas to apply to database connections (SQLite only).
92
+ For example ``pragmas = { journal_mode = "wal" }`` to enable WAL-mode.
93
+
90
94
  ``*``
91
95
  Any additional arguments are forwarded to the provider, e.g., ``encoding = 'utf-8'``.
92
96
 
@@ -0,0 +1,8 @@
1
+ Initialize database.
2
+
3
+ For SQLite this happens automatically.
4
+ See also ``--initdb`` for the ``hs cluster`` command.
5
+
6
+ The available special actions are mutually exclusive.
7
+ The ``--rotate`` operation migrates completed tasks to the next database partition,
8
+ and applies a special purpose ``part:N`` tag to the new partition and remaining tasks.
@@ -0,0 +1,20 @@
1
+ Actions
2
+ ^^^^^^^
3
+
4
+ ``--vacuum``
5
+ Vacuum an existing database.
6
+
7
+ ``--backup`` *PATH*
8
+ Vacuum into backup file (SQLite only).
9
+
10
+ ``--rotate``
11
+ Rotate completed tasks to new database (SQLite only).
12
+
13
+ ``-t``, ``--truncate``
14
+ Truncate database (task metadata will be lost).
15
+
16
+ Options
17
+ ^^^^^^^
18
+
19
+ ``-y``, ``--yes``
20
+ Auto-confirm action (default will prompt).
@@ -0,0 +1 @@
1
+ ``hs`` ``initdb`` ``[-h]`` ``[--truncate | --vacuum | --rotate | --backup PATH]`` ``[--yes]``
@@ -0,0 +1,16 @@
1
+ Submit one or more tasks.
2
+
3
+ Submit one task as positional arguments.
4
+ If a single positional argument refers to a non-executable file path,
5
+ tasks will be read from the file, one per line (use ``-f``/``--task-file`` to be explicit).
6
+
7
+ Tasks are accumulated and published in bundles to the database.
8
+ The ``-b``/``--bundlesize`` and ``-w``/``--bundlewait`` options control the
9
+ size of these bundles and how long to wait before flushing tasks regardless of
10
+ how many have accumulated.
11
+
12
+ Pre-format tasks at `submit`-time with template expansion using ``--template``.
13
+ Any tags specified with ``-t``/``--tag`` are applied to all tasks submitted.
14
+
15
+ Use the special comment syntax, ``# HYPERSHELL: ...``, to include tags inline.
16
+ If the comment is alone on the line it will be applied to all tasks that follow.
@@ -1,20 +1,23 @@
1
1
  Arguments
2
2
  ^^^^^^^^^
3
3
 
4
- FILE
5
- Path to input task file (default: <stdin>).
4
+ *ARGS*...
5
+ Command-line task arguments (for single task submission).
6
6
 
7
7
  Options
8
8
  ^^^^^^^
9
9
 
10
- ``-t``, ``--template`` *CMD*
10
+ ``-f``, ``--task-file`` *FILE*
11
+ Input file containing one task per line.
12
+
13
+ ``--template`` *CMD*
11
14
  Command-line template pattern (default: "{}").
12
15
 
13
16
  This is expanded at submit-time before sending to the database.
14
17
  With the default "{}" the input command-line will be run verbatim.
15
18
  Specifying a template pattern allows for simple input arguments (e.g., file paths)
16
19
  to be transformed into some common form; such as
17
- ``-t './some_command.py {} >outputs/{/-}.out'``.
20
+ ``--template './some_command.py {} >outputs/{/-}.out'``.
18
21
 
19
22
  See section on `templates`.
20
23
 
@@ -44,7 +47,7 @@ Options
44
47
 
45
48
  See ``hs initdb`` command.
46
49
 
47
- ``--tag`` *TAG*...
50
+ ``-t``, ``--tag`` *TAG*...
48
51
  Assign one or more tags.
49
52
 
50
53
  Tags allow for user-defined tracking of information related to individual tasks or large
@@ -0,0 +1,2 @@
1
+ ``hs`` ``submit`` ``[-h]`` ``[ARGS... | -f FILE]``
2
+ ``[-b NUM]`` ``[-w SEC]`` ``[--template CMD]`` ``[-t TAG...]`` ``[--initdb]``
@@ -1,2 +1,2 @@
1
- ``hs`` ``task`` ``info`` ``[-h]``
1
+ ``hs`` ``info`` ``[-h]``
2
2
  ``ID`` ``[--stdout | --stderr | -x FIELD]`` ``[-f FORMAT]``
@@ -4,4 +4,4 @@ A database must be configured. The task will not run until scheduled
4
4
  by the server process. The <stdout> and <stderr> are written locally
5
5
  as if run locally, requires the client to have ``--capture`` enabled.
6
6
 
7
- See ``task info`` and ``task wait`` commands.
7
+ See ``info`` and ``wait`` commands.
@@ -1,7 +1,7 @@
1
1
  Arguments
2
2
  ^^^^^^^^^
3
3
 
4
- ARGS...
4
+ *ARGS*...
5
5
  Command-line arguments.
6
6
 
7
7
  The full command-line for some shell task.
@@ -0,0 +1 @@
1
+ ``hs`` ``run`` ``[-h]`` ``[-n SEC]`` ``[-t TAG [TAG...]]`` ``--`` ``ARGS...``
@@ -1,4 +1,4 @@
1
- Search for tasks in database.
1
+ Search tasks in the database.
2
2
 
3
3
  A database must be configured.
4
4
  Specifying *FIELD* names defines what is included in the output
@@ -60,3 +60,10 @@ Options
60
60
 
61
61
  ``-c``, ``--count``
62
62
  Show count of results.
63
+
64
+ ``-i``, ``--ignore-partitions``
65
+ Suppress auto-union feature (SQLite only).
66
+
67
+ When using the `sqlite` provider, all databases matching the numbering pattern
68
+ applied by ``hs initdb --rotate`` will automatically be attached as partitions
69
+ and used within a temporary view, making full task history searchable.
@@ -1,5 +1,11 @@
1
- ``hs`` ``task`` ``search`` ``[-h]``
1
+ ``hs`` ``list`` ``[-h]``
2
2
  ``[FIELD [FIELD ...]]`` ``[-w COND [COND ...]]`` ``[-t TAG [TAG...]]``
3
3
  ``[--order-by FIELD [--desc]]`` ``[--count | --limit NUM]``
4
4
  ``[-f FORMAT | --json | --csv]`` ``[-d CHAR]``
5
5
  ``[--failed | --succeeded | --completed | --remaining]``
6
+
7
+ ``hs`` ``list`` ``--fields``
8
+
9
+ ``hs`` ``list`` ``--tag-keys``
10
+
11
+ ``hs`` ``list`` ``--tag-values`` ``KEY``
@@ -1,4 +1,4 @@
1
- Update task metadata.
1
+ Update tasks in the database.
2
2
 
3
3
  Include any number of `FIELD=VALUE` or tag `KEY:VALUE` positional arguments.
4
4
 
@@ -1,7 +1,7 @@
1
1
  Arguments
2
2
  ^^^^^^^^^
3
3
 
4
- ARGS...
4
+ *ARGS*...
5
5
  Assignment pairs for update.
6
6
 
7
7
  Options
@@ -86,5 +86,5 @@ FIELD
86
86
  VALUE
87
87
  New value.
88
88
 
89
- Use ``key:value`` notation for tasks.
89
+ Use ``key:value`` notation for tags.
90
90
  Updating ``tag`` will add or update any pre-existing tag with that ``key``.
@@ -1,6 +1,6 @@
1
- ``hs`` ``task`` ``update`` ``[-h]`` ``ID`` ``FIELD`` ``VALUE``
1
+ ``hs`` ``update`` ``[-h]`` ``ID`` ``FIELD`` ``VALUE``
2
2
 
3
- ``hs`` ``task`` ``update``
3
+ ``hs`` ``update``
4
4
  ``[ARG [ARG ...]]``
5
5
  ``[--cancel | --revert | --delete | --remove-tag KEY [KEY...]]``
6
6
  ``[-w COND [COND ...]]`` ``[-t TAG [TAG...]]``
@@ -1,2 +1,2 @@
1
- ``hs`` ``task`` ``wait`` ``[-h]``
1
+ ``hs`` ``wait`` ``[-h]``
2
2
  ``ID`` ``[-n SEC]`` ``[--info [-f FORMAT] | --status | --return]``
@@ -35,6 +35,14 @@ Functions
35
35
 
36
36
  This action is destructive, cannot be reversed, and proceeds without confirmation.
37
37
 
38
+ |
39
+
40
+ .. autofunction:: vacuumdb
41
+
42
+ |
43
+
44
+ .. autofunction:: rotatedb
45
+
38
46
  -------------------
39
47
 
40
48
  Constants
@@ -44,4 +52,6 @@ Constants
44
52
 
45
53
  .. autodata:: DATABASE_ENABLED
46
54
 
55
+ .. autodata:: DATABASE_PROVIDER
56
+
47
57
  |
@@ -0,0 +1,228 @@
1
+ .. _20250504_2_7_0_release:
2
+
3
+ Release Notes (v2.7.0)
4
+ ======================
5
+
6
+ `May 4, 2025`
7
+
8
+ The v2.7.0 release includes major database features and CLI improvements.
9
+
10
+ - Automatic database rotation for SQLite
11
+ - Optional UUIDv7 mode
12
+ - SQLite pragmas
13
+ - SQLite optimization
14
+ - Simplified command-line interface
15
+ - Version info
16
+ - Improved task submission (global tags)
17
+
18
+ -----
19
+
20
+ Features
21
+ --------
22
+
23
+ |
24
+
25
+ Automatic database rotation for SQLite
26
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
27
+
28
+ When using `SQLite` as a backend for long persistent deployments the overall throughput
29
+ can suffer due to the increased size of the database. What we would like is to keep our long task
30
+ history while alleviating the burden of such a large database on task throughput. To accomplish
31
+ this we've added two new admin routines, ``vacuumdb()`` and ``rotatedb()``, and exposed them via
32
+ the command-line interface via ``hs initdb`` as ``--vacuum``, ``--backup``, and ``--rotate``.
33
+
34
+ For those unfamiliar with the concept, `VACUUM` is a database operation (PostgreSQL and SQLite)
35
+ that optimizes space and can improve performance in some cases. For SQLite, updated rows are
36
+ written to new pages, even deleting data is only flagged as such. Performing a vacuum copies
37
+ all active pages to a new database file and then replaces the original database file. This
38
+ operation is safe to perform while under load. This on its own is a useful feature to have for
39
+ long running deployments.
40
+
41
+ For SQLite, the vacuum operation allows for a file path and instead of copying the new cleaned
42
+ pages back to the original file the data is left at the given file path. This capability is what
43
+ drives the ``--backup`` feature, and in turn enables the ``--rotate`` feature.
44
+
45
+ With this release we now automatically add a ``part:0`` tag to all new tasks. This ``part``, short
46
+ for *partition*, is used to label all completed tasks in the activate database with ``part:N`` in
47
+ a single transaction, where *N* is the N-th partition chosen based on the path to the file. With
48
+ this tag as a means to safely distinguish between the completed tasks in that moment, we can
49
+ partition the activate database using a vacuum operation to a new location, essentially cloning
50
+ the database. Afterwards, we drop any non-``part:N`` tasks from the new clone and any ``part:N``
51
+ from the activate database. This can happen in real time without stopping the server.
52
+
53
+ Immediately following one of these partition events, there will be few tasks in the activate database.
54
+ While searching task history we want to allow for the full history without losing access because we
55
+ split the database. Now, when invoking task search, if SQLite is in use, we automatically check for
56
+ any/all existing partitions of the same database path and `attach` each to the session. We build a
57
+ temporary `view` that replaces the ``task`` table with a series of ``union all`` on these partitions.
58
+ Any and all queries are applied to this temporary view as if the entire history existed within the
59
+ one table.
60
+
61
+ .. note::
62
+
63
+ With most distributions of Python the `sqlite3` library is built with a compile-time option
64
+ ``SQLITE_MAX_ATTACHED=10`` by default. While ``--rotate`` can be called any number of times,
65
+ the automatic attachment of these partitions will be limited by this setting. While it's
66
+ possible to re-compile SQLite with a higher value, it cannot be increased above 125.
67
+
68
+ See `SQLite limits <https://www.sqlite.org/limits.html>`_.
69
+
70
+ |
71
+
72
+ Optional UUIDv7 Mode
73
+ ^^^^^^^^^^^^^^^^^^^^
74
+
75
+ A new package extra, ``hypershell[uuid7]``, installs a third-party dependency,
76
+ `uuid-utils <https://pypi.org/project/uuid-utils/>`_, which provides for
77
+ lexicographically-sortable version 7 UUIDs. The format specification for UUID
78
+ version 7 has essentially solidified and library support for use of this format is well supported
79
+ and widely used.
80
+
81
+ See `RFC 9562 (draft standard) <https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-7>`_
82
+ for reference.
83
+
84
+ In large-scale performance testing with HyperShell it is difficult to say precisely what the
85
+ net benefit of UUIDv7 has been on overall task throughput. The increase in performance comes from
86
+ less cache-misses in page loads due to the time ordering of task rows. It is likely the case that
87
+ task bundles will consist of tasks submitted close in time to one another. Updating these tasks is
88
+ more efficient when they reside within the same collection of pages, instead of being randomly
89
+ distributed within the b-tree.
90
+
91
+ Generation of UUIDs within the project has been refactored to an internal routine.
92
+ This routine calls ``uuid.uuid4`` from the standard library and returns a string.
93
+ If ``uuid-utils`` is available in the installation we prefer ``uuid_utils.uuid7``.
94
+ This functionality install-time behavior and is not configurable at runtime.
95
+
96
+ .. note::
97
+
98
+ On SQLite there is no real benefit to UUID v7.
99
+ SQLite employs a hidden ``rowid`` column when using a non-integer primary key.
100
+ The pages in the database are actually stored according to this hidden ``rowid``.
101
+ The overhead of the second lookup is minimal compared to the huge performance improvement
102
+ due to the page ordering.
103
+
104
+ In testing, adding the ``WITHOUT ROWID`` table modifier results in a significant reduction
105
+ in performance, and adding UUIDv7 on top of that entirely recovers this performance.
106
+ Incorporating both the table modifier and UUIDv7 is relatively insignificant.
107
+
108
+ |
109
+
110
+ SQLite Pragmas
111
+ ^^^^^^^^^^^^^^
112
+
113
+ It has always been technically possible to enable WAL-mode for SQLite by connecting outside
114
+ of HyperShell and applying the setting. But this is inelegant and doesn't help with settings
115
+ that do not persist between connections. Now you can include a ``pragmas`` section in your
116
+ configuration to automatically apply these to all database connections.
117
+
118
+ The most useful change from the default behavior is to enable the Write-Ahead Log (WAL)
119
+ ``journal_mode`` for SQLite which can improve performance in high-concurrency applications.
120
+
121
+ .. admonition:: Set user-level configuration option
122
+ :class: note
123
+
124
+ .. code-block:: shell
125
+
126
+ hs config set database.pragmas.journal_mode wal
127
+
128
+ There are many other pragmas we might set.
129
+ Not all configurations of SQLite have been tested for performance.
130
+
131
+ .. admonition:: Configuration with SQLite pragmas
132
+ :class: note
133
+
134
+ .. code-block:: toml
135
+
136
+ [database]
137
+ file = "/var/lib/hypershell/main.db"
138
+ pragmas = {journal_mode = "wal", cache_size = 100000}
139
+
140
+ |
141
+
142
+ SQLite Optimization
143
+ ^^^^^^^^^^^^^^^^^^^
144
+
145
+ On a related note, all ``vacuumdb()`` operations (used by ``--vacuum`` and ``--rotate``) add
146
+ an automatic ``pragma optimize`` operation. This is mostly harmless and can be invoked safely
147
+ on a regular basis.
148
+
149
+ See `optimize <https://sqlite.org/pragma.html#pragma_optimize>`_ for details.
150
+
151
+ |
152
+
153
+ Simplified Command-line Interface
154
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
155
+
156
+ As with previous changes, backwards compatibility has been maintained for existing workflows.
157
+ Hierarchy can be a good thing in command-line tools and is often necessary to manage complexity.
158
+ For HyperShell though there is no naming-conflict of actions to justify the added ``task`` layer.
159
+ The following subcommands have been remapped for simplicity and brevity:
160
+
161
+ - ``hs task submit`` → ``hs submit`` *++*
162
+ - ``hs task info`` → ``hs info``
163
+ - ``hs task wait`` → ``hs wait``
164
+ - ``hs task run`` → ``hs run``
165
+ - ``hs task search`` → ``hs list``
166
+ - ``hs task update`` → ``hs update``
167
+
168
+ The exception here is ``hs submit`` and ``hs task submit`` which were distinct operations.
169
+ The new ``hs submit`` provides both interfaces in a single command. Positional arguments are treated
170
+ as a single command-line task. If a single positional argument is provided and it is either ``-``
171
+ (stdin) or is a valid non-executable file path it will be read as before. This can be made explicit
172
+ using the new ``-f``/``--task-file`` option.
173
+
174
+ The new ``hs submit`` also includes better quoting behavior, properly forwarding arguments with
175
+ quoted white space.
176
+
177
+ A new entry-point, ``hsx``, is also included as shorthand for ``hs cluster``.
178
+
179
+ |
180
+
181
+ Version Info
182
+ ^^^^^^^^^^^^
183
+
184
+ The output of ``hs --version`` now includes more detailed information.
185
+
186
+ .. admonition:: Show version information
187
+ :class: note
188
+
189
+ .. code-block:: shell
190
+
191
+ hs --version
192
+
193
+ .. details:: Output
194
+
195
+ .. code-block:: none
196
+
197
+ HyperShell v2.7.0 (CPython 3.13.2)
198
+
199
+ |
200
+
201
+ Improved Task Submission
202
+ ^^^^^^^^^^^^^^^^^^^^^^^^
203
+
204
+ In the :ref:`v2.6.0 release <20241115_2_6_0_release>` we added inline tag assignments.
205
+ Any input line in a submission file which included a comment with the special ``# HYPERSHELL: ...``
206
+ syntax allows for tags to be mapped to input arguments on an individual basis.
207
+
208
+ In this release we extend this behavior to allow processing of non-task lines.
209
+ If an input task line is empty or comment-only we skip that line and do not emit a task.
210
+ If that comment contains the ``# HYPERSHELL: ...`` notation the tags will take effect on all
211
+ future lines.
212
+
213
+ The following example input file would be processed as 4 tasks.
214
+
215
+ .. admonition:: Input task file with global tags
216
+ :class: note
217
+
218
+ .. code-block:: shell
219
+
220
+ # HYPERSHELL: site:b group:1
221
+
222
+ # HYPERSHELL: case:1
223
+ echo 1
224
+ echo 2
225
+
226
+ # HYPERSHELL: case:2
227
+ echo 3
228
+ echo 4