qdir 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
qdir-1.0.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Carl Osterwisch
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
qdir-1.0.0/PKG-INFO ADDED
@@ -0,0 +1,127 @@
1
+ Metadata-Version: 2.4
2
+ Name: qdir
3
+ Version: 1.0.0
4
+ Summary: A very low overhead first-in, first-out job queue for the local machine
5
+ Author-email: Carl Osterwisch <costerwi@gmail.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/costerwi/qdir
8
+ Project-URL: Issues, https://github.com/costerwi/qdir/issues
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Operating System :: OS Independent
11
+ Requires-Python: >=3.6
12
+ Description-Content-Type: text/markdown
13
+ License-File: LICENSE
14
+ Requires-Dist: filelock
15
+ Requires-Dist: pywin32; sys_platform == "win32"
16
+ Dynamic: license-file
17
+
18
+ # qdir
19
+ A very low overhead first-in, first-out job queue for the local machine
20
+
21
+ The queue is maintained as a directory of tiny job status files.
22
+ When a new job is submitted, it will identify any existing status files and then add its own to the directory.
23
+ The job will not launch its command until all the preceeding status files are gone.
24
+ Each job will delete its own status file upon completion so the next job can begin.
25
+ There is a polling interval (default 5 seconds)
26
+ when the job checks to see if its predecessors have finished before launching its command.
27
+ A shared queue directory can be specified on a multi-user system to coordinate
28
+ job execution.
29
+ On Linux this is `/tmp` by default.
30
+
31
+ Any command may be submitted to the job queue using qsub
32
+ (see [usage](#usage) below).
33
+ There are also tools to check the status of the queue (qstat) and remove jobs (qdel).
34
+
35
+ # Installation
36
+ Requires python3 on Windows, Linux, or MacOS.
37
+ One way to get python3 on Windows is through the [Microsoft Store](https://apps.microsoft.com/detail/9PNRBTZXMB4Z).
38
+ Here is the command to install qdir once python3 is available:
39
+ ```bash
40
+ python3 -m pip install qdir
41
+ ```
42
+
43
+ See [here](https://pip.pypa.io/en/stable/installation/) if there is "no module named pip."
44
+
45
+ If there is a warning that the script directory is not in the PATH then
46
+ it must be added in order to use the command line tools.
47
+ Otherwise, [see below](#usage-without-script-directory-in-path).
48
+
49
+ ## Abaqus Finite Element Analysis (FEA) extras
50
+ The package includes additional files to help Abaqus users submit and monitor the qdir job queue.
51
+ Use the following command to configure Abaqus after qdir itself has been installed:
52
+ ```bash
53
+ python3 -m qdir.abaqus install
54
+ ```
55
+
56
+ By default, the necessary settings are appended to the end of the user `abaqus_v6.env` file.
57
+ The following command may be used to remove qdir settings for Abaqus:
58
+ ```bash
59
+ python3 -m qdir.abaqus uninstall
60
+ ```
61
+
62
+ # Usage
63
+ Below is a brief overview of usage.
64
+ The command line tools accept a `-h` or `--help` argument to list their full options.
65
+
66
+ ## Command line job submission
67
+ Submit commands or job scripts to the queue using the `qsub` command.
68
+ For example, on Linux:
69
+ ```bash
70
+ qsub ./myJobFile.sh
71
+ ```
72
+ or on Windows:
73
+ ```bash
74
+ qsub myJobFile.cmd
75
+ ```
76
+
77
+ Any printed output from the command will be saved to a file based on the job name with the job Id appended to the end.
78
+
79
+ Display command options with `qsub -h`
80
+
81
+ ## Command line queue status
82
+ Check status with the `qstat` command, for example:
83
+ ```bash
84
+ qstat
85
+ ```
86
+
87
+ ## Command line job removal
88
+ First use the `qstat` command above to find the job Ids and job names in the queue.
89
+ Delete or kill jobs with the `qdel` command followed by its jobId or job name.
90
+ For example, to delete job 12343:
91
+ ```bash
92
+ qdel 12343
93
+ ```
94
+
95
+ ## Usage without script directory in PATH
96
+ The `qsub`, `qstat`, and `qdel` scripts are shortcuts to call python3 methods.
97
+ These scripts will not be available at the command line if their directory is not in the PATH.
98
+ It's most convenient to add the directory to the PATH but it is also
99
+ possible to run the methods directly with some additional typing,
100
+ such as `python3 -m qdir.qstat` to execute `qstat`
101
+
102
+ ## Abaqus extras
103
+ Submit Abaqus jobs as usual from the command line:
104
+ ```bash
105
+ abaqus job=whatever
106
+ ```
107
+ Internally this will invoke the `qdir_submit.py` module to create a job script
108
+ and submit it to qdir.
109
+
110
+ You may submit jobs within Abaqus CAE by selecting `qdir` as the Job's queue before submitting.
111
+ You may check the queue status within CAE in the menu `Plug-ins > QDir Job Queue Monitor...`
112
+ Click on a Job to see more detailed information.
113
+ Right-click on a Job to expose the menu allowing you to delete the Job.
114
+
115
+ ## Customization
116
+ Most parameters used by qdir may be customized with command line options,
117
+ system environment variables, or directives within the job script itself.
118
+
119
+ | Option | Description | Default |
120
+ |--------|-------------|---------|
121
+ | -N *NAME*, --name *NAME* | Name for the job. | The last argument of the command |
122
+ | -q *QUEUE*, --queue *QUEUE* | Directory to store the queue files. | environment variable `QDIR_QUEUE` or `/tmp` on Linux |
123
+ | -c *INTERVAL*, --interval *INTERVAL* | Polling interval in seconds. | environment variable `QDIR_POLL_INTERVAL` or 5 |
124
+ | -i *INFO*, --info *INFO* | Specify the format of information about jobs in queue | environment variable `QDIR_INFO` or `{jobId:8} {owner:12} {status:35} {name}` |
125
+ | -o *OUTPUT*, --output *OUTPUT* | File name for combined stdout and stderr. | environment variable `QDIR_LOGNAME` or `{name}._{jobId}`
126
+ | -s *STATUS*, --status *STATUS* | Specify the format of job status messages output to terminal. | environment variable `QDIR_STATUS` or `{jobId} {name} {status} after {duration}`
127
+ | -v *VERBOSE*, --verbose *VERBOSE* | Set the verbosity level. | environment variable `QDIR_VERBOSE` or 2 |
qdir-1.0.0/README.md ADDED
@@ -0,0 +1,110 @@
1
+ # qdir
2
+ A very low overhead first-in, first-out job queue for the local machine
3
+
4
+ The queue is maintained as a directory of tiny job status files.
5
+ When a new job is submitted, it will identify any existing status files and then add its own to the directory.
6
+ The job will not launch its command until all the preceeding status files are gone.
7
+ Each job will delete its own status file upon completion so the next job can begin.
8
+ There is a polling interval (default 5 seconds)
9
+ when the job checks to see if its predecessors have finished before launching its command.
10
+ A shared queue directory can be specified on a multi-user system to coordinate
11
+ job execution.
12
+ On Linux this is `/tmp` by default.
13
+
14
+ Any command may be submitted to the job queue using qsub
15
+ (see [usage](#usage) below).
16
+ There are also tools to check the status of the queue (qstat) and remove jobs (qdel).
17
+
18
+ # Installation
19
+ Requires python3 on Windows, Linux, or MacOS.
20
+ One way to get python3 on Windows is through the [Microsoft Store](https://apps.microsoft.com/detail/9PNRBTZXMB4Z).
21
+ Here is the command to install qdir once python3 is available:
22
+ ```bash
23
+ python3 -m pip install qdir
24
+ ```
25
+
26
+ See [here](https://pip.pypa.io/en/stable/installation/) if there is "no module named pip."
27
+
28
+ If there is a warning that the script directory is not in the PATH then
29
+ it must be added in order to use the command line tools.
30
+ Otherwise, [see below](#usage-without-script-directory-in-path).
31
+
32
+ ## Abaqus Finite Element Analysis (FEA) extras
33
+ The package includes additional files to help Abaqus users submit and monitor the qdir job queue.
34
+ Use the following command to configure Abaqus after qdir itself has been installed:
35
+ ```bash
36
+ python3 -m qdir.abaqus install
37
+ ```
38
+
39
+ By default, the necessary settings are appended to the end of the user `abaqus_v6.env` file.
40
+ The following command may be used to remove qdir settings for Abaqus:
41
+ ```bash
42
+ python3 -m qdir.abaqus uninstall
43
+ ```
44
+
45
+ # Usage
46
+ Below is a brief overview of usage.
47
+ The command line tools accept a `-h` or `--help` argument to list their full options.
48
+
49
+ ## Command line job submission
50
+ Submit commands or job scripts to the queue using the `qsub` command.
51
+ For example, on Linux:
52
+ ```bash
53
+ qsub ./myJobFile.sh
54
+ ```
55
+ or on Windows:
56
+ ```bash
57
+ qsub myJobFile.cmd
58
+ ```
59
+
60
+ Any printed output from the command will be saved to a file based on the job name with the job Id appended to the end.
61
+
62
+ Display command options with `qsub -h`
63
+
64
+ ## Command line queue status
65
+ Check status with the `qstat` command, for example:
66
+ ```bash
67
+ qstat
68
+ ```
69
+
70
+ ## Command line job removal
71
+ First use the `qstat` command above to find the job Ids and job names in the queue.
72
+ Delete or kill jobs with the `qdel` command followed by its jobId or job name.
73
+ For example, to delete job 12343:
74
+ ```bash
75
+ qdel 12343
76
+ ```
77
+
78
+ ## Usage without script directory in PATH
79
+ The `qsub`, `qstat`, and `qdel` scripts are shortcuts to call python3 methods.
80
+ These scripts will not be available at the command line if their directory is not in the PATH.
81
+ It's most convenient to add the directory to the PATH but it is also
82
+ possible to run the methods directly with some additional typing,
83
+ such as `python3 -m qdir.qstat` to execute `qstat`
84
+
85
+ ## Abaqus extras
86
+ Submit Abaqus jobs as usual from the command line:
87
+ ```bash
88
+ abaqus job=whatever
89
+ ```
90
+ Internally this will invoke the `qdir_submit.py` module to create a job script
91
+ and submit it to qdir.
92
+
93
+ You may submit jobs within Abaqus CAE by selecting `qdir` as the Job's queue before submitting.
94
+ You may check the queue status within CAE in the menu `Plug-ins > QDir Job Queue Monitor...`
95
+ Click on a Job to see more detailed information.
96
+ Right-click on a Job to expose the menu allowing you to delete the Job.
97
+
98
+ ## Customization
99
+ Most parameters used by qdir may be customized with command line options,
100
+ system environment variables, or directives within the job script itself.
101
+
102
+ | Option | Description | Default |
103
+ |--------|-------------|---------|
104
+ | -N *NAME*, --name *NAME* | Name for the job. | The last argument of the command |
105
+ | -q *QUEUE*, --queue *QUEUE* | Directory to store the queue files. | environment variable `QDIR_QUEUE` or `/tmp` on Linux |
106
+ | -c *INTERVAL*, --interval *INTERVAL* | Polling interval in seconds. | environment variable `QDIR_POLL_INTERVAL` or 5 |
107
+ | -i *INFO*, --info *INFO* | Specify the format of information about jobs in queue | environment variable `QDIR_INFO` or `{jobId:8} {owner:12} {status:35} {name}` |
108
+ | -o *OUTPUT*, --output *OUTPUT* | File name for combined stdout and stderr. | environment variable `QDIR_LOGNAME` or `{name}._{jobId}`
109
+ | -s *STATUS*, --status *STATUS* | Specify the format of job status messages output to terminal. | environment variable `QDIR_STATUS` or `{jobId} {name} {status} after {duration}`
110
+ | -v *VERBOSE*, --verbose *VERBOSE* | Set the verbosity level. | environment variable `QDIR_VERBOSE` or 2 |
@@ -0,0 +1,35 @@
1
+ [project]
2
+ name = "qdir"
3
+ authors = [
4
+ { name="Carl Osterwisch", email="costerwi@gmail.com" },
5
+ ]
6
+ description = "A very low overhead first-in, first-out job queue for the local machine"
7
+ readme = "README.md"
8
+ requires-python = ">=3.6"
9
+ classifiers = [
10
+ "Programming Language :: Python :: 3",
11
+ "Operating System :: OS Independent",
12
+ ]
13
+ license = "MIT"
14
+ license-files = ["LICENSE"]
15
+ dependencies = [
16
+ "filelock",
17
+ "pywin32; sys_platform == 'win32'",
18
+ ]
19
+ dynamic = ["version"]
20
+
21
+ [project.scripts]
22
+ qdel = "qdir.qdel:main"
23
+ qstat = "qdir.qstat:main"
24
+ qsub = "qdir.qsub:main"
25
+
26
+ [project.urls]
27
+ Homepage = "https://github.com/costerwi/qdir"
28
+ Issues = "https://github.com/costerwi/qdir/issues"
29
+
30
+ [build-system]
31
+ requires = ["setuptools"]
32
+ build-backend = "setuptools.build_meta"
33
+
34
+ [tool.setuptools.dynamic]
35
+ version = {attr = "qdir.__version__"}
qdir-1.0.0/setup.cfg ADDED
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,288 @@
1
+ #!/bin/env python3
2
+ """A very low overhead first-in, first-out job queue for the local machine
3
+
4
+ The queue is maintained as a directory of tiny job status files.
5
+ When a new job is submitted, it will identify any existing status files and
6
+ then add its own to the directory.
7
+ The job will not launch its command until all the preceeding status files are gone.
8
+ Each job will delete its own status file upon completion so the next job can begin.
9
+ There is a polling interval (default 5 seconds)
10
+ when the job checks to see if its predecessors have finished before launching its command.
11
+ A shared queue directory can be specified on a multi-user system to coordinate
12
+ job execution.
13
+ On Linux this is `/tmp` by default.
14
+ """
15
+
16
+ __author__ = "Carl Osterwisch"
17
+ __version__ = "1.0.0"
18
+ __url__ = "https://github.com/costerwi/qdir/"
19
+
20
+ import os
21
+ import signal
22
+ import subprocess
23
+ import sys
24
+ from argparse import ArgumentParser, Namespace
25
+ from pathlib import Path
26
+ from tempfile import gettempdir
27
+ from time import asctime, perf_counter, sleep
28
+
29
+ from filelock import FileLock
30
+
31
+ default = Namespace
32
+
33
+ parser = ArgumentParser(epilog=f"qdir {__version__} {__url__}")
34
+ default.queue = Path(os.environ.get("QDIR_QUEUE", gettempdir()))
35
+ parser.add_argument(
36
+ "-q",
37
+ "--queue",
38
+ type=Path,
39
+ help="Directory to store the queue files. "
40
+ f"Default is environment variable QDIR_QUEUE or {default.queue}",
41
+ default=default.queue,
42
+ )
43
+ default.verbose = int(os.environ.get("QDIR_VERBOSE", "2"))
44
+ parser.add_argument(
45
+ "-v",
46
+ "--verbose",
47
+ type=int,
48
+ help="Verbosity level. "
49
+ f"Default is environment variable QDIR_VERBOSE or {default.verbose}",
50
+ default=default.verbose,
51
+ )
52
+
53
+ qFileSuffix = os.environ.get("QDIR_SUFFIX", ".qdir") # suffix for queue files
54
+
55
+
56
+ class Queue:
57
+ def __init__(self, **kwargs):
58
+ """Create the queue, allowing parameters to override defaults"""
59
+ for parameter, value in vars(default).items():
60
+ setattr(self, parameter, kwargs.pop(parameter, value))
61
+ if not self.queue.is_dir():
62
+ raise FileNotFoundError(f"The QDIR directory {self.queue} does not exist")
63
+ if not os.access(self.queue, os.W_OK):
64
+ raise PermissionError(f"The QDIR directory {self.queue} is not writable")
65
+
66
+ def getQueuedJobs(self) -> list[Path]:
67
+ """Return a list of qFilePath for currently queued jobs, sorted oldest to newest."""
68
+
69
+ queuedJobs = [filePath for filePath in self.queue.glob("*" + qFileSuffix)]
70
+ if len(queuedJobs) == 0:
71
+ return queuedJobs # easy answer
72
+
73
+ # Collecct a set of active process ids
74
+ if os.name == "nt":
75
+ import win32process
76
+
77
+ processIds = {str(pid) for pid in win32process.EnumProcesses()}
78
+ else:
79
+ procPath = Path("/proc")
80
+ if procPath.is_dir():
81
+ processIds = {p.stem for p in procPath.iterdir() if p.stem.isnumeric()}
82
+ else:
83
+ c = subprocess.run(
84
+ ["pgrep", "y"], stdout=subprocess.PIPE, text=True, check=True
85
+ )
86
+ processIds = {pid for pid in c.stdout.strip().split()}
87
+
88
+ def is_qFile(path):
89
+ """Verify the path is an active job file with a matching process id"""
90
+ if not path.stem.isnumeric():
91
+ return False # file name must be an integer
92
+ if not (path.is_file()):
93
+ return False # path must represent a file
94
+ if path.stem in processIds:
95
+ return True # file name must match a running process number
96
+
97
+ # Otherwise, the process may have died or the computer was rebooted with
98
+ # this job in the queue. This orphaned qFile may be removed.
99
+ print(
100
+ "Removing orphaned queue file",
101
+ self.info.format(**self.getJobInfo(path)),
102
+ file=sys.stderr,
103
+ )
104
+ try:
105
+ path.unlink(missing_ok=True)
106
+ except OSError as e:
107
+ print(e, file=sys.stderr)
108
+
109
+ # Filter valid qFiles
110
+ queuedJobs = [filePath for filePath in queuedJobs if is_qFile(filePath)]
111
+ return sorted(queuedJobs, key=lambda qFilePath: qFilePath.stat().st_mtime)
112
+
113
+ def getJobInfo(self, qFilePath: Path) -> dict:
114
+ """Return information about the job specified in qFilePath"""
115
+ jobInfo = {
116
+ "jobId": qFilePath.stem,
117
+ "owner": qFilePath.owner(),
118
+ "name": "",
119
+ "command": "",
120
+ "directory": "",
121
+ "behind": "",
122
+ "status": "",
123
+ "history": [], # list[str]
124
+ }
125
+ try:
126
+ with qFilePath.open() as qFile:
127
+ for line in qFile:
128
+ keyValue = line.strip().split("\t", 2)
129
+ if len(keyValue) == 2:
130
+ jobInfo.update([keyValue])
131
+ else:
132
+ status = keyValue[0]
133
+ jobInfo["history"].append(status)
134
+ jobInfo["status"] = status # latest update
135
+ except OSError as e:
136
+ status = repr(e)
137
+ jobInfo["history"].append(status)
138
+ jobInfo["status"] = status
139
+ return jobInfo
140
+
141
+ def dequeue(self, qFilePath: Path):
142
+ """Remove a job from the queue"""
143
+ assert qFilePath.parent == self.queue
144
+ assert qFilePath.suffix == qFileSuffix
145
+ assert qFilePath.stem.isnumeric()
146
+ # Removing qFile will dequeue the job but the effect is delayed by polling interval
147
+ qFilePath.unlink(missing_ok=True)
148
+ if os.name == "posix":
149
+ # On Linux the SIGTERM signal has immediate effect
150
+ try:
151
+ os.kill(int(qFilePath.stem), signal.SIGTERM)
152
+ except OSError: # no such process
153
+ pass
154
+
155
+ def enqueue(self, cmd: list[str]):
156
+ """Wait in queue until prior jobs finish, then run cmd"""
157
+ import getpass
158
+ import shutil
159
+
160
+ if len(cmd) < 1:
161
+ raise SyntaxError("Missing cmd")
162
+ if self.verbose >= 1 and not shutil.which(cmd[0]):
163
+ print(f"WARNING: {cmd[0]} is not in PATH", file=sys.stderr)
164
+ if not self.interval > 0:
165
+ self.interval = 1
166
+
167
+ startTime = perf_counter() # time job was enqueued
168
+ job = None # subprocess running the cmd, if any
169
+
170
+ jobInfo = { # information about this job
171
+ "jobId": str(os.getpid()), # process id of this script
172
+ "owner": getpass.getuser(),
173
+ "name": self.name or cmd[-1],
174
+ "command": " ".join(cmd),
175
+ "directory": Path.cwd(),
176
+ }
177
+
178
+ def updateStatus(priority, status, file=None):
179
+ """Notify user of status change and optionally record in file"""
180
+ jobInfo["status"] = status
181
+ jobInfo["time"] = asctime()
182
+ jobInfo["duration"] = humanDuration(perf_counter() - startTime)
183
+ if self.verbose >= priority:
184
+ # Echo status to the terminal
185
+ print(self.status.format(**jobInfo))
186
+ if file is not None:
187
+ # Record status history in specified qFile
188
+ print("{time} {status}".format(**jobInfo), file=file)
189
+
190
+ qFilePath = (self.queue / jobInfo["jobId"]).with_suffix(qFileSuffix)
191
+ lock = FileLock(self.queue / "qdir.lock") # prevent previousJobs race
192
+ with lock.acquire(timeout=5):
193
+ previousJobs = self.getQueuedJobs()
194
+ jobInfo["behind"] = " ".join([job.stem for job in previousJobs])
195
+ with qFilePath.open("w") as qFile:
196
+ os.environ["QDIR_FILE"] = qFile.name # will be exported to cmd
197
+ # Initialize qFile contents
198
+ for key in "name", "command", "directory", "behind":
199
+ value = jobInfo.get(key)
200
+ if value:
201
+ print(key, value, sep="\t", file=qFile)
202
+ updateStatus(priority=2, status="Queued", file=qFile)
203
+
204
+ def terminateJob(sig=None, frame=None):
205
+ """Terminate this job if running or dequeue if not"""
206
+ nonlocal job
207
+ result = ""
208
+ qFilePath.unlink(missing_ok=True) # remove its qFile
209
+ if job is None:
210
+ # removed from the queue before starting
211
+ result = ".. Dequeued"
212
+ elif hasattr(job, "pid"): # job is or was running
213
+ if job.poll() is None:
214
+ # job is still running
215
+ if self.verbose >= 2:
216
+ result = ".. Killed"
217
+ if os.name == "nt":
218
+ job.send_signal(signal.CTRL_BREAK_EVENT)
219
+ else:
220
+ # pass SIGTERM signal to all child processes in this process group
221
+ os.killpg(os.getpgid(job.pid), signal.SIGTERM)
222
+ elif self.verbose >= 1:
223
+ # job is no longer running; check returncode
224
+ if job.returncode <= 0:
225
+ result = "** Done"
226
+ else:
227
+ result = "-- Error"
228
+ if result:
229
+ updateStatus(priority=2, status=result)
230
+ job = result # prevent this from duplicate reporting
231
+
232
+ signal.signal(
233
+ signal.SIGTERM, terminateJob
234
+ ) # catch termination signal (ignored on Windows)
235
+
236
+ # Wait for all previousJobs to complete
237
+ previousJobs.reverse() # check newest jobs first
238
+ while qFilePath.exists() and any(
239
+ otherFile.exists() for otherFile in previousJobs
240
+ ):
241
+ sleep(self.interval)
242
+
243
+ # Launch the cmd only if this job's qFile still exists
244
+ if qFilePath.exists():
245
+ try:
246
+ # Append the job starting status to qFile
247
+ stat = qFilePath.stat()
248
+ with qFilePath.open("a") as qFile:
249
+ print(
250
+ "Waited in queue",
251
+ humanDuration(perf_counter() - startTime),
252
+ file=qFile,
253
+ )
254
+ updateStatus(priority=3, status="Started", file=qFile)
255
+ # Keep mtime unchanged so the job files sort correctly
256
+ os.utime(qFilePath, (stat.st_atime, stat.st_mtime))
257
+ except OSError:
258
+ pass # ignore status update exceptions
259
+ logFilePath = Path(self.output.format(**jobInfo))
260
+ with logFilePath.open("w") as logFile:
261
+ opts = {}
262
+ if os.name == "nt":
263
+ opts["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP
264
+ try:
265
+ job = subprocess.Popen(
266
+ cmd,
267
+ shell=False,
268
+ start_new_session=True,
269
+ stdin=subprocess.DEVNULL,
270
+ stdout=logFile,
271
+ stderr=subprocess.STDOUT,
272
+ **opts,
273
+ )
274
+ startTime = perf_counter() # reset to time job started running
275
+ while qFilePath.exists() and job.poll() is None:
276
+ # job is still running
277
+ sleep(self.interval)
278
+ except OSError as e:
279
+ print(e, file=logFile)
280
+
281
+ terminateJob() # Clean up at the end
282
+
283
+
284
+ def humanDuration(seconds: float) -> str:
285
+ for unit, per in ("days", 86400), ("hours", 3600), ("minutes", 60):
286
+ if seconds >= 1.1 * per:
287
+ return f"{seconds / per:.2g} {unit}"
288
+ return f"{seconds:.2g} seconds"
@@ -0,0 +1,5 @@
1
+ """Modules to help Abaqus interact with the qdir job queue
2
+
3
+ Install with:
4
+ python3 -m qdir.abaqus install
5
+ """
@@ -0,0 +1,66 @@
1
+ """Install or uninstall qdir for Abaqus.
2
+
3
+ This script will modify the user's abaqus_v6.env file with
4
+ settings for the qdir job queue.
5
+ """
6
+
7
+ from __future__ import print_function, with_statement # use Python3 syntax
8
+
9
+ import os
10
+ import sys
11
+ from argparse import ArgumentParser
12
+
13
+ settings = """# Settings for qdir job queue
14
+ sys.path.append(r"{}") # directory of qdir_submit.py
15
+ import qdir_submit
16
+ qdir_submit.init(queues)
17
+ plugin_central_dir += os.pathsep + qdir_submit.dirname
18
+ run_mode = BATCH # command line will submit to queue by default"""
19
+
20
+ parser = ArgumentParser(description=__doc__)
21
+ parser.add_argument(
22
+ "action",
23
+ choices=["install", "uninstall"],
24
+ help="Specify what to do.",
25
+ )
26
+ default = os.path.expanduser("~/abaqus_v6.env")
27
+ parser.add_argument(
28
+ "env",
29
+ nargs="?",
30
+ default=default,
31
+ help="Optionally specify the .env file to modify. Default is {}".format(default),
32
+ )
33
+
34
+
35
+ def uninstall(user_env):
36
+ """Remove qdir settings from user's abaqus_v6.env"""
37
+ new_env = user_env + ".new"
38
+ with open(new_env, "w") as new_env_file:
39
+ try:
40
+ # copy current settings
41
+ with open(user_env) as env_file:
42
+ for line in env_file:
43
+ if not "qdir" in line: # skip any qdir settings
44
+ print(line.rstrip(), file=new_env_file)
45
+ os.remove(user_env)
46
+ except FileNotFoundError:
47
+ pass
48
+ os.rename(new_env, user_env)
49
+
50
+
51
+ def install(user_env):
52
+ """Append qdir settings to user's abaqus_v6.env"""
53
+ uninstall(user_env) # remove any previous settings
54
+ dirname = os.path.dirname(os.path.normpath(__file__))
55
+ with open(user_env, "a") as user_env_file:
56
+ print(settings.format(dirname), file=user_env_file)
57
+
58
+
59
+ if __name__ == "__main__":
60
+ options = parser.parse_args(sys.argv[1:])
61
+ if "uninstall" == options.action:
62
+ print("Removing qdir settings from", options.env)
63
+ uninstall(options.env)
64
+ else:
65
+ print("Adding qdir settings to", options.env)
66
+ install(options.env)