rda-python-dscheck 2.0.6__py3-none-any.whl → 2.0.7__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. rda_python_dscheck/dscheck.usg +507 -411
  2. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/METADATA +1 -1
  3. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/RECORD +7 -7
  4. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/WHEEL +0 -0
  5. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/entry_points.txt +0 -0
  6. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/licenses/LICENSE +0 -0
  7. {rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/top_level.txt +0 -0
rda_python_dscheck/dscheck.usg CHANGED
@@ -1,103 +1,157 @@
1
1
 
2
2
  1 INTRODUCTION
3
3
 
4
- 'dscheck' is a utility for adding, removing, viewing, and processing recorded
5
- commands of other utility programs in the Research Data Archive Management System
6
- (GDEXMS). Utility programs 'dsarch', 'dsupdt', and 'dsrqst' support delayed
7
- execution (also called batch processing): when a command is deferred, its
8
- information and the directory from which it was invoked are saved to GDEXDB as
9
- a check record. Specialist-defined commands can also be deferred by adding them
10
- to 'dscheck' control via Action -AC (-AddCheck).
11
-
12
- Check records are processed automatically by a centralized 'dscheck' daemon,
13
- though they can also be processed manually from the command line. While a
14
- recorded command is running, its check record is locked in GDEXDB to prevent
15
- duplicate executions. Once a command finishes, its check record is automatically
16
- moved to check history.
17
-
18
- When a recorded command fails due to a system, storage, or database error, the
19
- check record is normally purged with status 'E' (error). However, if the utility
20
- program supports check-reprocessing — as 'dsarch' does — the record is retained
21
- in GDEXDB until the command succeeds or the maximum retry limit is reached.
22
- 'dsrqst' and 'dsupdt' handle their own failure recovery and do not require
23
- check-reprocessing.
24
-
25
- Completed check records are retained in GDEXDB as history and can be viewed via
26
- the utility program 'viewcheckusage'.
27
-
28
- 'dscheck' supports the following major functions:
29
- - Set daemon control records for individual specialists to configure how many
30
- concurrent processes of a specified command can run on a given host, and
31
- the host priorities that determine which host is selected for processing
32
- a check. Without daemon control information, recorded commands will not
33
- start automatically.
34
- - Add a check record for delayed execution of any specialist-defined command
35
- - View utility command information currently saved in GDEXDB
36
- - Email the current status of a specified command or list of commands,
37
- including any error messages, to a specialist
38
- - Delete recorded command information when a command is no longer needed
39
- - Unlock a check record when lock information was not cleaned up properly
40
- after a command failure
41
- - Interrupt a running command and recursively kill all associated child
42
- processes
43
- - Add due 'dsrqst' and 'dsupdt' actions into dscheck records
44
- - Process commands recorded in GDEXDB, including ones that previously failed
45
-
46
- The specialist who submits a command under 'dscheck' control remains its
47
- exclusive owner in GDEXDB, preventing the command from being processed or
48
- deleted accidentally by other specialists.
49
-
50
- The following sections describe general 'dscheck' usage first, then Action
51
- options in detail, followed by Mode and Info options.
4
+ 'dscheck' is the command-line interface to the centralized batch-processing
5
+ system for the GDEX Research Data Archive Management System (GDEXMS). It lets
6
+ DECS specialists schedule, monitor, control, and clean up deferred (delayed)
7
+ executions of utility programs such as 'dsarch', 'dsupdt', and 'dsrqst', as
8
+ well as any specialist-defined command.
9
+
10
+ The mental model has three layers:
11
+
12
+ 1. A check record - a row in GDEXDB that captures one deferred command:
13
+ its name, arguments, owner, working directory, target
14
+ host, retry policy, and current status.
15
+ 2. A daemon control - a row in GDEXDB that tells the centralized 'dscheck'
16
+ daemon how many concurrent processes of a given
17
+ command a given specialist may run on a given host,
18
+ and at what host priority.
19
+ 3. The dscheck daemon - a long-running process that wakes on a fixed
20
+ interval, adds due 'dsupdt'/'dsrqst' check records,
21
+ and starts (or restarts) commands from check
22
+ records on hosts chosen according to daemon
23
+ control configuration.
24
+
25
+ Typical lifecycle of a deferred command:
26
+
27
+ added ---> pending ---> running ---> finished (purged to history)
28
+ \ \
29
+ \ +---> failed (E) [purged]
30
+ \ \
31
+ +--> retried (if reprocessing supported, e.g. dsarch)
32
+
33
+ A check record is locked while its command runs so the same command is never
34
+ started twice. When the command finishes successfully, or fails terminally,
35
+ the check record is moved to check history. History records can be reviewed
36
+ with the utility 'viewcheckusage'.
37
+
38
+ Failure handling depends on the calling utility. 'dsarch' supports
39
+ check-reprocessing: failed checks remain in GDEXDB until the command
40
+ succeeds or its maximum retry count is reached. 'dsrqst' and 'dsupdt' have
41
+ their own recovery paths and do not rely on dscheck retries; for them, an
42
+ errored check is purged with status 'E'.
43
+
44
+ A check is owned exclusively by the specialist who submitted it. Other
45
+ specialists cannot accidentally process or delete it.
46
+
47
+ What 'dscheck' can do:
48
+
49
+ - Configure daemon controls per (specialist, command, host) so that the
50
+ daemon knows where, how many at a time, and in what host order to run
51
+ commands. Without daemon controls, recorded commands are not started
52
+ automatically.
53
+ - Add a check record to schedule a deferred run of any specialist-defined
54
+ command.
55
+ - Inspect daemon controls and check records currently in GDEXDB.
56
+ - Email the current status of one or more checks (including any error
57
+ messages) to a specialist.
58
+ - Remove check records that are no longer needed.
59
+ - Unlock a check whose lock was left behind after an abnormal termination.
60
+ - Interrupt a running check and recursively kill all of its child
61
+ processes.
62
+ - Add due 'dsupdt' and 'dsrqst' work to dscheck records.
63
+ - Process checks (start them on the configured hosts) in daemon mode or
64
+ on demand.
65
+ - Verify daemon-host connectivity for a specialist.
66
+
67
+ The remainder of this document is organized as follows. Section 2 covers
68
+ general usage and conventions. Section 3 describes Action options grouped
69
+ by what they manipulate (daemon controls, check records, check processing,
70
+ host connectivity, batch options). Section 4 lists Mode options that modify
71
+ how an action behaves. Section 5 lists Information options that pass values
72
+ into an action.
52
73
 
53
74
  2 GENERAL DSCHECK USAGE
54
75
 
55
76
  dscheck [Action Option] [Mode Options] [Info Options]
56
- or
77
+ or
57
78
  dscheck [-(IF|InputFile)] InputFileNames
58
79
 
59
- Brackets [] indicate optional elements. A pipe '|' within parentheses, as in
60
- (A|B), means either A or B may be used. Options fall into three categories:
61
- Action options - specify the task to execute
62
- Mode options - modify how an action behaves
63
- Info options - pass one or more values to the action
64
-
65
- An option can be given in short or long form — for example, -DS or -Dataset.
66
- Some options have alias names for convenience; for example, -UnLock is an alias
67
- for Mode option -UL (-UnLockCheck). Option names are case-insensitive, but
68
- values following Info options are case-sensitive.
69
-
70
- Specify exactly one Action option per 'dscheck' invocation. Based on the chosen
71
- action, certain Info options are required, others are optional, and specific
72
- Mode options may be used to alter the action's behavior.
73
-
74
- All options except -IF (-InputFile) may be given either on the command line or
75
- in input files. Input file names are provided via -IF and can only be specified
76
- on the command line. See the -IF option description for details on how to format
77
- options in input files. One or more input files may be combined with command-line
78
- options. The option name -IF itself may be omitted when a single input file is
79
- given on the command line and all other options are contained within that file.
80
-
81
- When retrieving daemon or check information, Info options serve as query filters.
82
- Four special characters can refine queries further — quote or escape them on the
83
- command line to prevent shell interpretation:
84
- '!' - exclude matches; must appear immediately after the option name
85
- '<' - less-than comparison on the following value
86
- '>' - greater-than comparison on the following value
87
- '<>' - range between the following two values
88
-
89
- Combining '!' and '<' as "'!' '<' OptionValue" expresses a 'greater than or
90
- equal to OptionValue' condition.
91
-
92
- The description of an individual option is shown when 'dscheck' is run as
93
-
94
- dscheck [Option] -(h|help) [Option]
95
-
96
- A description is shown for the option placed either before or after -(h|help).
97
- If no option is specified, or 'dscheck' is run without arguments, the full
98
- document is displayed via the UNIX utility 'more'. A hard copy of this document
99
- can be printed from the saved file, dscheck.usg, under python package
100
- rda_python_dscheck.
80
+ Notation:
81
+ [] - the enclosed element is optional.
82
+ (A|B) - either A or B may be used (a short form and a long form).
83
+ -OPT - an option name. Names are case-insensitive; values are not.
84
+
85
+ Every dscheck invocation has at most one Action option, which selects the
86
+ task to perform. Mode options change how the chosen action behaves. Info
87
+ options carry the data the action needs (indices, names, hostnames, dates,
88
+ etc.). Each action documents which Info options are required and which are
89
+ optional.
90
+
91
+ Many options have an alias for convenience; for example, -UnLock is an
92
+ alias for the Mode option -UL (-UnLockCheck). Both short and long forms
93
+ are accepted, and aliases are noted with each option.
94
+
95
+ 2.1 Quick Start
96
+
97
+ Show the full document, paged through 'more':
98
+ dscheck
99
+
100
+ Show the description of a single option:
101
+ dscheck -h -AC
102
+ dscheck -AC -h
103
+
104
+ List your own daemon control records:
105
+ dscheck -GD
106
+
107
+ List your own active check records:
108
+ dscheck -GC
109
+
110
+ Add a deferred run of a script on PBS:
111
+ dscheck -AC -CM myscript.sh -HN PBS
112
+
113
+ Email yourself the status of all your active checks:
114
+ dscheck -EC
115
+
116
+ Start the dscheck daemon (typical for the operator):
117
+ dscheck -PC -DM start
118
+
119
+ 2.2 Filters and Special Characters in Queries
120
+
121
+ For Get-style actions (-GD, -GC), Info options are interpreted as query
122
+ filters. Four characters refine those filters; quote or escape them on the
123
+ command line so the shell does not consume them:
124
+
125
+ '!' - negate the match. Must appear immediately after the option name.
126
+ '<' - less-than comparison against the next value.
127
+ '>' - greater-than comparison against the next value.
128
+ '<>' - range between the next two values (inclusive).
129
+
130
+ The combination "'!' '<' value" expresses 'greater than or equal to value'.
131
+
132
+ 2.3 Input Files
133
+
134
+ Every option except -IF (-InputFile) may be supplied either on the command
135
+ line or in an input file. -IF is itself only valid on the command line.
136
+ A single dscheck invocation may combine command-line options with one or
137
+ more input files. If the only options are inside a single input file, the
138
+ -IF prefix may be omitted:
139
+
140
+ dscheck mycheck.in
141
+
142
+ See the description of -IF in Section 5.2 for the input-file format
143
+ (comments, action/mode markers, single-value assignments, and tabular
144
+ multi-value assignments, including delimiter overrides).
145
+
146
+ 2.4 Getting Help
147
+
148
+ dscheck # full document, displayed via 'more'
149
+ dscheck -(h|help) # full document
150
+ dscheck [Option] -(h|help) # description of [Option]
151
+ dscheck -(h|help) [Option] # same as above
152
+
153
+ A printable copy of this document is the file dscheck.usg installed under
154
+ the python package rda_python_dscheck.
101
155
 
102
156
  #The online HTML version of this document is available at
103
157
  #https://gdex-docs-dscheck.readthedocs.io
@@ -105,46 +159,51 @@ rda_python_dscheck.
105
159
 
106
160
  3 ACTION OPTIONS
107
161
 
108
- Action options specify what task 'dscheck' executes. They take no values.
109
- Specifying more than one Action option at a time is not allowed.
110
-
111
- Based on the information being manipulated, actions are divided into four
112
- categories:
113
- Daemon Control Actions - create, delete, modify and view daemon control
114
- information in GDEXDB, for specified specialists,
115
- commands, and hostnames
116
- Check Actions - add, delete, unlock and view check information
117
- for active individual checks
118
- Check Process Actions - process checks by starting commands on remote hosts
119
- as configured in daemon controls; purge checks by
120
- recording commands and execution results into check
121
- history; interrupt running commands by killing the
122
- current process and all its children; and email
123
- check status to a specialist
124
- Daemon Host Connectivity - check connectivity of daemon hosts for specialists
162
+ An Action option selects what 'dscheck' does. Action options take no
163
+ values, and exactly one Action option may be supplied per invocation.
164
+
165
+ Action options are grouped into four categories:
166
+
167
+ Daemon Control Actions - create, modify, view, and delete daemon
168
+ control records that govern how, where, and
169
+ how concurrently commands run.
170
+ Check Actions - add, view, delete, and unlock individual
171
+ check records.
172
+ Check Process Actions - start commands from check records, interrupt
173
+ running commands, or email check status.
174
+ Daemon Host Connectivity - verify that daemon hosts are reachable for
175
+ a specialist.
176
+
177
+ A fifth, narrower action -SO (-SetOptions) builds batch options dynamically
178
+ and is documented in Section 3.5.
125
179
 
126
180
  3.1 Daemon Control Actions
127
181
 
128
- A daemon control record configures how many concurrent processes of a given
129
- command a specialist may run on a specified host, and the priority order in
130
- which hosts are selected. The centralized daemon reads these records
131
- periodically, so specialists can update daemon control values at any time
132
- without restarting the daemon.
182
+ A daemon control record answers three questions for the dscheck daemon:
133
183
 
134
- Daemon control records are managed with the following actions:
135
- Set Daemon Control - create or modify daemon control records for specified
136
- specialists, commands, and hostnames
137
- Get Daemon Control - retrieve existing daemon control information
138
- Delete Daemon Control - remove one or more daemon control records
184
+ - Whose checks? (specialist login)
185
+ - For which command? (utility program name, or 'ALL')
186
+ - On which host, at what concurrency, at what priority?
187
+ (hostname, ProcessLimit, Priority)
188
+
189
+ Daemon controls are read by the daemon on every cycle, so changes take
190
+ effect without restarting the daemon. A specialist with no daemon control
191
+ records will have no checks started automatically.
192
+
193
+ Available actions:
194
+
195
+ Set Daemon Control - create or modify daemon control records.
196
+ Get Daemon Control - retrieve existing daemon control records.
197
+ Delete Daemon Control - remove daemon control records by index.
139
198
 
140
199
  3.1.1 Set Daemon Control
141
- -SD or -SetDaemon, creates or modifies daemon control records in GDEXDB for
142
- given specialist login names, commands, and hostnames. One or more records
143
- can be set per execution.
200
+ -SD or -SetDaemon, creates or modifies daemon control records in GDEXDB
201
+ for given specialist login names, commands, and hostnames. One or many
202
+ records can be set per invocation.
144
203
 
145
204
  dscheck -(SD|SetDaemon) [Mode Option]
146
205
  [-(DI|DaemonIndex) controlIndices]
147
- [-(CM|Command) UtilityProgramNames] # ALL for any command names
206
+ [-(CM|Command) UtilityProgramNames] # ALL for any command name
148
207
  [-(SN|Specialist) DECSSpecialists]
149
208
  [-(HN|HostName) HostMachineNames]
150
209
  [-(MH|MatchHost) FlagToMatchHostname]
@@ -152,33 +211,33 @@ categories:
152
211
  [-(PO|Priority) HostListOrder]
153
212
 
154
213
  Available mode option:
155
- -(ND|NewDaemon) - adds a new daemon control record to GDEXDB
214
+ -(ND|NewDaemon) - add a new daemon control record (required when the
215
+ daemon index is 0).
156
216
 
157
- If a daemon control record already exists in GDEXDB for the given specialist,
158
- command, and hostname, the record is updated. A new record is created when the
159
- daemon index is 0 and Mode option -ND (-NewDaemon) is present. The combination
160
- of specialist login name, command name, and hostname must be unique for each
161
- daemon control record.
217
+ An existing record matching the (specialist, command, hostname) triple
218
+ is updated in place. To create a new record, supply daemon index 0 and
219
+ the -ND mode option; this guards against creating records by accident.
220
+ The (specialist, command, hostname) triple must be unique.
162
221
 
163
- Specify hostname 'PBS' to submit the command to the PBS batch control system.
164
- If the specified command name is not found in the daemon control, the general
165
- 'ALL' configuration is used.
222
+ Special hostnames and commands:
223
+ HN = PBS - submit the command to the PBS batch system.
224
+ CM = ALL - apply to any command not otherwise configured for this
225
+ specialist.
166
226
 
167
- For example, to set daemon control for schuster on PBS hosts for all commands,
168
- allowing up to 4 concurrent checks at priority 1 (lower number = higher
169
- priority), using input file daemon.ctl:
227
+ Example - configure schuster on PBS for any command, with up to 4
228
+ concurrent checks at priority 1 (lower number = higher priority), driven
229
+ from input file daemon.ctl:
170
230
 
171
- dscheck -SD -ND -IF daemon.ctl
231
+ dscheck -SD -ND -IF daemon.ctl
172
232
 
173
- <<Content of input file daemon.ctl>>
174
- DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
175
- 0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
233
+ <<Content of input file daemon.ctl>>
234
+ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
235
+ 0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
176
236
 
177
237
 
178
238
  3.1.2 Get Daemon Control
179
- -GD or -GetDaemon, retrieves daemon control records for specified commands,
180
- specialists, or hostnames. If no conditions are provided, only records owned
181
- by the specialist running 'dscheck' are returned.
239
+ -GD or -GetDaemon, retrieves daemon control records. With no Info-option
240
+ filters, only records owned by the current specialist are returned.
182
241
 
183
242
  dscheck -(GD|GetDaemon) [Mode Option]
184
243
  [-(FN|FieldNames) FieldNameString]
@@ -192,57 +251,61 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
192
251
  [-(DB|Debug) DebugModeInfo]
193
252
 
194
253
  Available mode option:
195
- -(FO|FormatOutput) - formats each column to a uniform fixed width
254
+ -(FO|FormatOutput) - pad each column to a uniform width for readability.
196
255
 
197
- Use -FN (-FieldNames) to specify which daemon control fields to retrieve.
198
- Defaults to all available fields when -FN is not provided.
256
+ -FN (-FieldNames) selects which daemon-control fields to retrieve as a
257
+ string of single-letter codes. When -FN is omitted, all fields are
258
+ returned.
199
259
 
200
- Valid daemon control field names and their corresponding Info options:
260
+ Field codes for daemon control records:
201
261
 
202
- Names Info Options Descriptions
262
+ Code Info Option Description
203
263
  I -(DI|DaemonIndex) daemon control index
204
- C -(CM|Command) command names of utility programs
264
+ C -(CM|Command) command name of a utility program
205
265
  H -(HN|Hostname) computer hostname
206
- M -(MH|MatchHost) Flags to control hostname match
207
- S -(SN|Specialist) DECS specialist the daemon set for
208
- P -(PL|ProcessLimit) Max check count to be processed at the same times
209
- O -(PO|Priority) host priority a specified command to start on
266
+ M -(MH|MatchHost) flag controlling hostname match
267
+ S -(SN|Specialist) DECS specialist this control applies to
268
+ P -(PL|ProcessLimit) max concurrent processes for this control
269
+ O -(PO|Priority) host priority within this control
210
270
 
211
- Results can be filtered by specialist (-SN), and/or other conditions.
212
- Options -SN, -CM, and -HN accept the '%' wildcard.
271
+ -SN, -CM, and -HN accept the SQL-style '%' wildcard. Pass -SN ALL to
272
+ retrieve every specialist's records (or a specific other specialist by
273
+ login name).
213
274
 
214
- To view records owned by another specialist, provide their login name via
215
- -SN (-Specialist). To view all control records, use -SN ALL.
275
+ Example - retrieve all daemon control records owned by you:
216
276
 
217
- For example, to retrieve all daemon control records set for you:
218
-
219
- dscheck GD
277
+ dscheck -GD
220
278
 
221
279
 
222
280
  3.1.3 Delete Daemon Control
223
281
  -DL or -Delete (Alias: -RM, -Remove), removes one or more daemon control
224
- records from GDEXDB for the specified daemon control indices.
282
+ records from GDEXDB by daemon control index.
225
283
 
226
284
  dscheck -(DL|Delete)
227
285
  -(DI|DaemonIndex) DaemonControlIndices
228
286
  [-(DB|Debug) DebugModeInfo]
229
287
 
230
- Deleting a daemon control record removes the specific configuration for a
231
- command, specialist, and hostname combination.
288
+ Deleting a daemon control record removes the configuration for one
289
+ (specialist, command, hostname) combination. Existing check records are
290
+ not removed.
291
+
232
292
 
233
293
  3.2 Check Actions
234
294
 
235
- Delayed command executions for due 'dsupdt' and 'dsrqst' actions are recorded
236
- automatically. Other commands — including 'dsarch' and specialist-defined ones
237
- — can be added to 'dscheck' manually. The following actions manage check records:
238
- Add Check - add a new check record for a deferred command
239
- Get Check - retrieve command information from check records
240
- Delete Check - delete check records for commands that no longer need processing
241
- Unlock Check - unlock a check record whose command aborted without releasing
242
- the lock
295
+ Check records are created automatically by the dscheck daemon for due
296
+ 'dsupdt' and 'dsrqst' actions. Other commands - including 'dsarch' calls
297
+ and any specialist-defined script - are added manually with -AC.
298
+
299
+ Available actions:
300
+
301
+ Add Check - add a check record for a deferred command.
302
+ Get Check - retrieve check records and field values.
303
+ Delete Check - delete check records that should no longer run.
304
+ Unlock Check - clear leftover lock information after an abnormal exit.
243
305
 
244
306
  3.2.1 Add Check
245
- -AC or -AddCheck, adds a check record for delayed command execution.
307
+ -AC or -AddCheck, adds a check record so that a command runs later,
308
+ driven by the dscheck daemon (or by an explicit -PC invocation).
246
309
 
247
310
  dscheck -(AC|AddCheck) [Mode Option]
248
311
  -(CM|Command) CommandNames
@@ -262,48 +325,54 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
262
325
  [-(DB|Debug) DebugModeInfo]
263
326
 
264
327
  Available mode option:
265
- -(AW|AnyWhere) - leaves the working directory empty in the check record so
266
- the command can be processed from any location
328
+ -(AW|AnyWhere) - leave the working directory empty in the check record
329
+ so the command may run from any location.
330
+
331
+ Defaults:
332
+ Owner - the specialist running 'dscheck'.
333
+ Working directory - the current directory at the time of -AC.
334
+ Host - none specified; daemon control governs placement.
267
335
 
268
- The command name is required. If not specified, the current specialist is
269
- set as the owner of the check record and the current directory is used as
270
- the working directory when the command runs later.
336
+ Use -PI (-ParentIndex) to make the new check wait for an existing parent
337
+ check to finish before it can run. Use -QS to pass options through to
338
+ qsub when the command will run as a PBS batch job.
271
339
 
272
- Pass additional PBS batch options via -QS (-QsubOptions). Use -PI
273
- (-ParentIndex) to put the current command on hold until a parent check
274
- finishes.
340
+ Limitations:
341
+ - Shell redirections and pipes (>, >>, |, <) are not supported in the
342
+ deferred command line. Wrap such commands in a shell script and add
343
+ the script.
275
344
 
276
- Commands with redirections or pipes are not supported for delayed mode.
277
- Wrap complex commands in a simple shell script instead.
345
+ Examples:
278
346
 
279
- For example, to list files containing 'test' in the current directory,
280
- capturing stdout to a log and displaying it on screen:
347
+ List files containing 'test' in the current directory, capture stdout to
348
+ a log, and also display it on screen:
281
349
 
282
- dscheck -AC -CM test1.sh -HN PBS
350
+ dscheck -AC -CM test1.sh -HN PBS
283
351
 
284
- <<Content of shell script test1.sh>>
285
- #!/bin/sh
286
- (ls -l | grep test) | tee test1.out
352
+ <<Content of shell script test1.sh>>
353
+ #!/bin/sh
354
+ (ls -l | grep test) | tee test1.out
287
355
 
288
- For example, to list files containing 'test', capturing stdout and stderr
289
- to separate log files:
356
+ List files containing 'test', capturing stdout and stderr to separate
357
+ log files:
290
358
 
291
- dscheck -AC -CM test2.sh -HN PBS
359
+ dscheck -AC -CM test2.sh -HN PBS
292
360
 
293
- <<Content of shell script test2.sh>>
294
- #!/bin/sh
295
- (ls -l | grep test) 1> test2.out 2>test2.err
361
+ <<Content of shell script test2.sh>>
362
+ #!/bin/sh
363
+ (ls -l | grep test) 1> test2.out 2>test2.err
296
364
 
297
- For example, to add command 'test3' to 'dscheck' for deferred execution on PBS:
365
+ Add the executable 'test3' for deferred execution on PBS:
298
366
 
299
- dscheck -AC -CM test3 -HN PBS
367
+ dscheck -AC -CM test3 -HN PBS
300
368
 
301
- The command 'test3' must be executable in the current working directory on
302
- PBS machines.
369
+ The command 'test3' must be executable from the working directory on
370
+ the PBS machine.
303
371
 
304
372
 
305
373
  3.2.2 Get Check
306
- -GC or -GetCheck, retrieves check information recorded in GDEXDB.
374
+ -GC or -GetCheck, retrieves check records from GDEXDB. With no filter,
375
+ returns active check records owned by the current specialist.
307
376
 
308
377
  dscheck -(GC|GetCheck) [Mode Options]
309
378
  [-(FN|FieldNames) FieldNameString]
@@ -322,91 +391,93 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
322
391
  [-(DB|Debug) DebugModeInfo]
323
392
 
324
393
  Available mode options:
325
- -(CS|CheckStatus) - displays detailed check status, including progress
326
- percentage for running commands and error messages
327
- for failed ones
328
- -(FO|FormatOutput) - formats each column to a uniform fixed width
394
+ -(CS|CheckStatus) - include detailed status: progress percentage for
395
+ running commands and error messages for failed ones.
396
+ -(FO|FormatOutput) - pad each column to a uniform width for readability.
329
397
 
330
- Use -FN (-FieldNames) to specify which check fields to retrieve.
331
- Defaults to "COVTUPFJDNW" when -FN is not given.
398
+ -FN (-FieldNames) selects which fields to retrieve as a string of
399
+ single-letter codes. When -FN is omitted, the default field set is
400
+ "COVTUPFJDNW".
332
401
 
333
- Valid check field names and their corresponding Info options:
402
+ Field codes for check records:
334
403
 
335
- Names Info Options Descriptions
404
+ Code Info Option Description
336
405
  C -(CI|CheckIndex) check index
337
406
  O -(CM|Command) original command name
338
- V -(AV|ArgumentVector) argument line following command, up to 100 chars
339
- T -(DS|Dataset) dataset ID, the original command run against
340
- A -(AN|ActionName) action name for a given command
341
- U -(ST|Status) check status for a recorded command
342
- B -(DF|DownFlags) Storage system down flags: D-DRDATA,G-GLADE,O-ObjectStore
343
- P -(PQ|PBSQueue) PBS batch queue name: gdex or htc
344
- R -(PI|ParentIndex) parent check index the current one to wait on
345
- F -(FC|FileCount) number of files need to be processed
346
- J -(DC|DoneCount) number of files are processed already
347
- K -(TC|TryCount) number of tries the command is executed
348
- L -(MC|MaxCount) upper limit for number of command tries
349
- Z -(SZ|DataSize) total bytes of data processed for the command
350
- D -(CD|CheckDate) date the command is initially recorded
351
- Y -(CT|CheckTime) time the command is initially recorded
352
- H -(HN|HostName) host names the command can or cannot run on
353
- N -(SN|Specialist) specialist login name who starts the command
354
- W -(WD|WorkDir) working directory where the command is started
355
- M -(MO|Modules) include modules to load to batch job script
356
- I -(EV|Environments) include environment variables to load to batch job script
357
- Q -(QS|QsubOptions) additional PBS batch options for qsub
358
- X -(AX|ArgumentExtra) additional argument line beyond 100 characters
359
- E -(ER|ErrorMessage) error message from failed command
360
-
361
- Filter results by check index via -CI (-CheckIndex). Without any condition,
362
- only check records owned by the current specialist are returned.
407
+ V -(AV|ArgumentVector) argument string after the command (<= 100 chars)
408
+ T -(DS|Dataset) dataset ID the command runs against
409
+ A -(AN|ActionName) action name for the command
410
+ U -(ST|Status) check status for the recorded command
411
+ B -(DF|DownFlags) storage down flags: D-DRDATA, G-GLADE,
412
+ O-ObjectStore
413
+ P -(PQ|PBSQueue) PBS batch queue name (e.g. gdex, htc)
414
+ R -(PI|ParentIndex) parent check index this check waits on
415
+ F -(FC|FileCount) number of files to be processed
416
+ J -(DC|DoneCount) number of files already processed
417
+ K -(TC|TryCount) number of times the command has run
418
+ L -(MC|MaxCount) maximum number of tries allowed
419
+ Z -(SZ|DataSize) total bytes processed for the command
420
+ D -(CD|CheckDate) date the command was first recorded
421
+ Y -(CT|CheckTime) time the command was first recorded
422
+ H -(HN|HostName) host(s) the command can or cannot run on
423
+ N -(SN|Specialist) specialist who owns the check
424
+ W -(WD|WorkDir) working directory for the command
425
+ M -(MO|Modules) modules to load in the batch job script
426
+ I -(EV|Environments) environment variables for the batch job
427
+ Q -(QS|QsubOptions) additional qsub options
428
+ X -(AX|ArgumentExtra) additional argument string beyond 100 chars
429
+ E -(ER|ErrorMessage) error message from a failed command
430
+
431
+ Use -CI (-CheckIndex) to fetch specific records. To inspect another
432
+ specialist's records, supply -SN with the appropriate login name.
363
433
 
364
434
 
365
435
  3.2.3 Delete Check
366
- -DL or -Delete, removes one or more 'dscheck' records from GDEXDB for
367
- the specified check indices.
436
+ -DL or -Delete, removes check records from GDEXDB by check index.
368
437
 
369
438
  dscheck -(DL|Delete)
370
439
  -(CI|CheckIndex) CheckIndices
371
440
  [-(DB|Debug) DebugModeInfo]
372
441
 
373
- A deleted check record is saved to check history and can be viewed later
374
- via the utility program 'viewcheckusage'.
442
+ A deleted check is moved to check history and remains visible via the
443
+ utility program 'viewcheckusage'.
375
444
 
376
445
 
377
446
  3.2.4 Unlock Check
378
- -UL or -UnLockCheck (Alias: -UnLock), unlocks check records whose commands
379
- aborted abnormally during processing.
447
+ -UL or -UnLockCheck (Alias: -UnLock), clears stale lock information from
448
+ a check record.
380
449
 
381
- When a command is running, its process ID and hostname are saved in the
382
- check record. If the process aborts abnormally, this lock information may
383
- not be cleared properly. Use this action to remove the lock so the command
384
- can be reprocessed or purged.
450
+ When a check is being processed, the running process ID and hostname are
451
+ written into the check record. If the process aborts abnormally, that
452
+ lock may not be cleared. Unlocking a check allows it to be reprocessed
453
+ or purged.
385
454
 
386
455
  dscheck -(UL|UnLockCheck)
387
456
  -(CI|CheckIndex) CheckIndices
388
457
  [-(DB|Debug) DebugModeInfo]
389
458
 
390
- A check index is required to unlock a recorded command.
459
+ -CI is required.
391
460
 
392
461
 
393
462
  3.3 Check Process Actions
394
463
 
395
- Deferred commands recorded in GDEXDB are started (or restarted) automatically
396
- by the centralized 'dscheck' daemon. Running commands can be interrupted at any
397
- time, and all child processes of the interrupted command are also cleaned up.
398
- The current check status can be gathered and emailed to a specialist.
464
+ The dscheck daemon starts and restarts deferred commands automatically.
465
+ Operators (and, in non-daemon mode, specialists) can also drive command
466
+ execution from the command line, interrupt running commands, and email
467
+ status summaries.
399
468
 
400
- Available actions:
401
- Process Check - start a command from its check record, in daemon or
402
- non-daemon mode
403
- Interrupt Check - interrupt a running command for a given check record and
404
- kill all its child processes
405
- Email Check - email a specialist the current status of their check records
469
+ Available actions:
470
+
471
+ Process Check - start commands from check records (daemon or
472
+ non-daemon mode).
473
+ Interrupt Check - stop a running command and recursively kill its
474
+ process tree.
475
+ Email Check - email a specialist the current status of one or more
476
+ check records.
406
477
 
407
478
  3.3.1 Process Check
408
- -PC or -ProcessCheck, starts commands from check records in daemon or
409
- non-daemon mode. Check indices may be specified in non-daemon mode.
479
+ -PC or -ProcessCheck, drives command execution from check records, in
480
+ either daemon mode or non-daemon mode.
410
481
 
411
482
  dscheck -(PC|ProcessCheck) [Mode Options]
412
483
  [-(CI|CheckIndex) CheckIndices]
@@ -417,31 +488,32 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
417
488
  [-(DB|Debug) DebugModeInfo]
418
489
 
419
490
  Available mode options:
420
- -(BG|BackGround) - runs as a background process, suppressing screen output
421
- and errors
422
- -(CP|CheckPending) - checks and kills long-pending checks
423
- -(LO|LogOn) - enables detailed logging in daemon mode
424
- -(NC|NoCommand) - suppresses remote command execution
425
- -(WU|WithdsUpdt) - in non-daemon mode, adds check records for due 'dsupdt'
426
- actions configured in update control records
427
- -(WR|WithdsRqst) - in non-daemon mode, adds check records for 'dsrqst'
428
- records due to be built or purged
429
-
430
- In daemon mode, 'dscheck' sleeps for two minutes (120 seconds by default,
431
- or as configured via -WI) between processing cycles. On each wake-up, it
432
- first adds due 'dsupdt' and 'dsrqst' check records, then starts or restarts
433
- commands from check records according to the host priorities set in daemon
434
- controls.
435
-
436
- In non-daemon mode, Mode options -WU and -WR must be present for due
437
- 'dsupdt' and 'dsrqst' actions to be added to check records. All current
438
- check records (unless indices are specified) are then processed on machines
439
- according to the daemon control configuration.
491
+ -(BG|BackGround) - run as a background process; suppress screen output.
492
+ -(CP|CheckPending) - identify and kill long-pending checks.
493
+ -(LO|LogOn) - enable detailed logging in daemon mode.
494
+ -(NC|NoCommand) - do not actually execute remote commands (dry run).
495
+ -(WU|WithdsUpdt) - in non-daemon mode, add check records for due
496
+ 'dsupdt' actions configured in update controls.
497
+ -(WR|WithdsRqst) - in non-daemon mode, add check records for 'dsrqst'
498
+ records due to be built or purged.
499
+
500
+ Daemon mode (-DM start):
501
+ The daemon sleeps for -WI seconds (default 120) between cycles. Each
502
+ cycle, it adds due 'dsupdt' and 'dsrqst' check records, then starts
503
+ or restarts checks on hosts chosen by daemon control priorities.
504
+ -DM stop, -DM logon, and -DM logoff stop the daemon, enable detailed
505
+ logging, and disable detailed logging respectively.
506
+
507
+ Non-daemon mode:
508
+ Without -DM, dscheck processes the current set of check records once.
509
+ -WU and -WR must be set explicitly if you want due 'dsupdt'/'dsrqst'
510
+ work to be added before processing. -CI restricts processing to
511
+ specific check indices.
440
512
 
441
513
  3.3.2 Interrupt Check
442
- -IC or -InterruptCheck, interrupts checks whose commands are currently
443
- running and recursively kills all child processes. Locks are released when
444
- the check is interrupted.
514
+ -IC or -InterruptCheck, stops a running command and recursively kills
515
+ its child processes. The check's lock is released as part of the
516
+ interrupt.
445
517
 
446
518
  dscheck -(IC|InterruptCheck) [Mode Option]
447
519
  -(CI|CheckIndex) CheckIndices
@@ -449,13 +521,14 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
449
521
 
450
522
  Available mode option:
451
523
  -(FI|ForceInterrupt) - required to interrupt a check that is currently
452
- being processed
524
+ being processed; without it dscheck only warns.
453
525
 
454
- A check index is required to interrupt its command.
526
+ -CI is required.
455
527
 
456
528
  3.3.3 Email Check
457
- -EC or -EmailCheck, sends an email to a specialist with the current status
458
- of all their check records, unless more specific conditions are provided.
529
+ -EC or -EmailCheck, emails a specialist a status report for their check
530
+ records. With no filters, the report covers all of the recipient's
531
+ active checks; otherwise it is restricted to the records that match.
459
532
 
460
533
  dscheck -(EC|EmailCheck)
461
534
  [-(CI|CheckIndex) CheckIndices]
@@ -469,10 +542,12 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
469
542
  [-(CC|CarbonCopy) Cc'dEmailAddresses]
470
543
  [-(DB|Debug) DebugModeInfo]
471
544
 
472
- Any error messages recorded in the check record are included in the email.
545
+ Any error messages stored on a failed check are included in the email.
546
+
473
547
 
474
548
  3.4 Daemon Host Connectivity
475
- -CH or -CheckHost, checks connectivity of daemon hosts for specialists.
549
+ -CH or -CheckHost, verifies that the hosts referenced in daemon control
550
+ records are reachable from the current host for the given specialist.
476
551
 
477
552
  dscheck -(CH|CheckHost)
478
553
  [-(DI|DaemonIndex) controlIndices]
@@ -483,9 +558,12 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
483
558
  [-(PO|Priority) ProcessPriority]
484
559
  [-(DB|Debug) DebugModeInfo]
485
560
 
561
+
486
562
  3.5 Set Batch Options Dynamically
487
- -SO or -SetOptions (Alias: -SetBatchOptions), processes batch options with
488
- a leading '!' to build options dynamically.
563
+ -SO or -SetOptions (Alias: -SetBatchOptions), evaluates batch options
564
+ whose values begin with '!' so that PBS options can be assembled
565
+ dynamically (for example, by resolving environment- or workload-derived
566
+ values at submission time).
489
567
 
490
568
  dscheck -(SO|SetOptions)
491
569
  [-(HN|HostName) HostNames]
@@ -498,204 +576,222 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
498
576
 
499
577
  4 MODE OPTIONS
500
578
 
501
- Mode options modify the behavior of Action options. They are all optional and
502
- take no values.
579
+ Mode options modify the behavior of an Action option. They take no values.
580
+ Each Mode option is meaningful only with specific actions; the relevant
581
+ action is noted with each one below.
503
582
 
504
- -AW or -AnyWhere, used with Action -AC (-AddCheck), leaves the working
505
- directory empty in the check record so the command can run from any location.
583
+ -AW or -AnyWhere, used with -AC (-AddCheck): leave the check's working
584
+ directory empty so the command may run from any location.
506
585
 
507
- -BG or -BackGround (Alias: -b), runs as a background process, suppressing
586
+ -BG or -BackGround (Alias: -b), run as a background process and suppress
508
587
  all screen output and error messages.
509
588
 
510
- -CP or -CheckPending, when used with Action -PC (-ProcessCheck), checks and
511
- kills long-pending checks.
589
+ -CP or -CheckPending, used with -PC (-ProcessCheck): identify and kill
590
+ checks that have been pending too long.
591
+
592
+ -CS or -CheckStatus, used with -GC (-GetCheck): include detailed status
593
+ for each check, including progress percentage for running commands and
594
+ error messages for failed ones.
512
595
 
513
- -CS or -CheckStatus, displays detailed status information for recorded
514
- commands, including progress percentage for running commands and error
515
- messages for failed ones.
596
+ -FI or -ForceInterrupt, required with -IC (-InterruptCheck) to actually
597
+ interrupt a check that is currently being processed; without it, dscheck
598
+ prints a warning instead of interrupting.
516
599
 
517
- -FI or -ForceInterrupt, required to interrupt a check that is currently being
518
- processed; without it, a warning message is displayed instead.
600
+ -FO or -FormatOutput, used with Get-style actions (-GD, -GC): pad each
601
+ column to a uniform width computed from the data, for readability.
519
602
 
520
- -FO or -FormatOutput, formats column output for GET actions. A uniform width,
521
- evaluated dynamically, is applied to all values of a given field.
603
+ -LO or -LogOn (Alias: -LoggingOn), used with -PC -DM start: enable
604
+ detailed daemon logging from startup. Detailed logging is off by default.
522
605
 
523
- -LO or -LogOn (Alias: -LoggingOn), enables detailed logging when 'dscheck'
524
- starts in daemon mode. Detailed logging is off by default.
606
+ -MD or -MyDataset, allows a specialist to access check information for
607
+ a dataset owned by a different specialist.
525
608
 
526
- -MD or -MyDataset, allows a specialist to access check information for a
527
- dataset owned by another specialist.
609
+ -NC or -NoCommand (Alias: -NoRemoteCommand), used with -PC: skip the
610
+ actual remote command execution (dry run).
528
611
 
529
- -NC or -NoCommand (Alias: -NoRemoteCommand), suppresses remote command
530
- execution when used with Action -PC (-ProcessCheck).
612
+ -ND or -NewDaemon, required with -SD (-SetDaemon) when the daemon
613
+ control index is 0, so that a new daemon control record is created.
614
+ This guard prevents creating records by accident.
531
615
 
532
- -ND or -NewDaemon, required to add a new daemon control record when Action
533
- -SD (-SetDaemon) is run with a daemon control index of 0. This flag prevents
534
- daemon control records from being added unintentionally.
616
+ -NT or -NoTrim, skip stripping spaces and comments from input values
617
+ when reading input files; this speeds up parsing of large input files.
535
618
 
536
- -NT or -NoTrim, skips stripping spaces and comments from input values to
537
- speed up reading of input files.
619
+ -WR or -WithdsRqst, used with -PC (-ProcessCheck) in non-daemon mode:
620
+ add 'dsrqst' records due to be built or purged to check records before
621
+ processing.
538
622
 
539
- -WR or -WithdsRqst, adds 'dsrqst' records due to be built or purged to check
540
- records, for use with Action -PC (-ProcessCheck) in non-daemon mode.
623
+ -WU or -WithdsUpdt, used with -PC (-ProcessCheck) in non-daemon mode:
624
+ add due 'dsupdt' update controls to check records before processing.
541
625
 
542
- -WU or -WithdsUpdt, adds due 'dsupdt' update controls to check records, for
543
- use with Action -PC (-ProcessCheck) in non-daemon mode.
544
626
 
545
627
  5 INFORMATION OPTIONS
546
628
 
547
629
  Info options pass data values into 'dscheck'. They come in two forms:
548
- Single-Value Info Options - accept exactly one value per invocation
549
- Multi-Value Info Options - accept one or more values per invocation
630
+
631
+ Single-Value Info Options - accept exactly one value.
632
+ Multi-Value Info Options - accept one or more values, comma-separated
633
+ on the command line or row-separated in an
634
+ input file.
550
635
 
551
636
  5.1 Single-Value Info Options
552
637
 
553
- A single-value Info option accepts exactly one value. Providing no value or
554
- more than one causes an error.
638
+ A single-value Info option accepts exactly one value. Supplying zero or
639
+ multiple values is an error.
555
640
 
556
- -DM or -DaemonMode, passes a daemon mode value — start, stop, logon, or
557
- logoff — to Action -PC (-ProcessCheck) to start, stop, enable, or disable
558
- detailed logging, respectively.
641
+ -DM or -DaemonMode, the daemon mode for -PC (-ProcessCheck). Valid
642
+ values are start, stop, logon, and logoff: start the daemon, stop the
643
+ daemon, enable detailed logging, or disable detailed logging.
559
644
 
560
- -DV or -Divider (Alias: -Delimiter, -Separator), the column delimiter for
561
- multi-value Info options in input files. Defaults to '<:>'.
645
+ -DV or -Divider (Alias: -Delimiter, -Separator), the column delimiter
646
+ used by multi-value tabular assignments in input files. Defaults to '<:>'.
562
647
 
563
- -ES or -EqualSign, the token used to assign a value to an option in input
648
+ -ES or -EqualSign, the token that assigns a value to an option in input
564
649
  files. Defaults to '<=>'.
565
650
 
566
- -FN or -FieldNames, a string of single-letter field codes selecting which
567
- fields to retrieve for check information via Action -GC (-GetCheck).
651
+ -FN or -FieldNames, a string of single-letter field codes selecting
652
+ which fields to retrieve. Used with -GC (-GetCheck) and -GD (-GetDaemon);
653
+ see Sections 3.1.2 and 3.2.2 for the code tables.
568
654
 
569
- -LH or -LocalHost, the local hostname on which to process checks for Action
570
- -PC (-ProcessCheck). Defaults to the current hostname. Specify PBS to process
571
- batch jobs.
655
+ -LH or -LocalHost, the local hostname on which to process checks for
656
+ -PC (-ProcessCheck). Defaults to the current hostname. Pass PBS to
657
+ process batch jobs.
572
658
 
573
- -MT or -MaxrunTime, the maximum run time for daemon mode. Defaults to 0
574
- (unlimited). For example, 5000 means 5000 seconds, and 1D means 1 day
659
+ -MT or -MaxrunTime, the maximum daemon run time. Defaults to 0
660
+ (unlimited). Examples: 5000 means 5000 seconds; 1D means one day
575
661
  (86400 seconds).
576
662
 
577
- -OF or -OutputFile, redirects output to a file instead of the screen.
663
+ -OF or -OutputFile, redirect output to a file instead of the screen.
578
664
 
579
- -ON or -OrderNames, a string of single-letter field codes controlling the
580
- sort order of results from Action -GC (-GetCheck).
665
+ -ON or -OrderNames, a string of single-letter field codes that
666
+ controls the sort order of -GC (-GetCheck) output.
581
667
 
582
- -AO or -ActOption, the token that marks Action and Mode options in input
583
- files. Defaults to '<!>'.
668
+ -AO or -ActOption, the token that marks Action and Mode options in
669
+ input files. Defaults to '<!>'.
584
670
 
585
- -WI or -WaitInterval, the time the 'dscheck' daemon sleeps between processing
586
- cycles. Defaults to 2 minutes (120 seconds).
671
+ -WI or -WaitInterval, the time the dscheck daemon sleeps between
672
+ cycles. Defaults to 120 seconds (2 minutes).
587
673
 
588
674
  5.2 Multi-Value Info Options
589
675
 
590
- A multi-value Info option accepts a list of one or more values. Supplying
591
- zero values causes an error.
676
+ A multi-value Info option accepts a list of one or more values. Supplying
677
+ zero values is an error.
592
678
 
593
- -AN or -ActionName (Alias: -Action), the action name associated with a given
594
- command recorded in a check record.
679
+ -AN or -ActionName (Alias: -Action), the action name associated with
680
+ a command in a check record.
595
681
 
596
- -AV or -ArgumentVector, the space-delimited argument string following the
597
- command, up to 100 characters. Quote individual arguments containing spaces
598
- with single quotes.
682
+ -AV or -ArgumentVector, the space-delimited argument string that
683
+ follows the command, up to 100 characters. Quote individual arguments
684
+ containing spaces with single quotes.
599
685
 
600
686
  -AX or -ArgumentExtra, the space-delimited argument string beyond 100
601
- characters. Non-empty only when the argument vector exceeds 100 characters.
687
+ characters. Non-empty only when the argument vector exceeds 100
688
+ characters.
602
689
 
603
- -CC or -CarbonCopy, one or more additional email addresses for Cc'd
604
- notification of check status. For DECS specialists, login names are
605
- acceptable; for other domains, full email addresses are required.
690
+ -CC or -CarbonCopy, additional email addresses for Cc'd notification
691
+ of check status. DECS specialists may be specified by login name; for
692
+ other domains, use full email addresses.
606
693
 
607
- -CD or -CheckDate, the date the recorded command was first processed.
694
+ -CD or -CheckDate, the date a recorded command was first processed.
608
695
 
609
- -CI or -CheckIndex, the check record indices for commands recorded in GDEXDB.
610
- A check record is automatically purged when the command finishes.
696
+ -CI or -CheckIndex, the indices of check records in GDEXDB. A check
697
+ record is moved to history once its command finishes.
611
698
 
612
- -CM or -Command (Alias: -CommandName), the command name recorded in a check
699
+ -CM or -Command (Alias: -CommandName), the command name in a check
613
700
  record.
614
701
 
615
- -CT or -CheckTime, the time the recorded command was first processed.
702
+ -CT or -CheckTime, the time a recorded command was first processed.
616
703
 
617
- -DB or -Debug, enables debug logging. Up to 3 values may be given: debug
618
- level (required), log file path, and log file name. The level can be a
619
- single integer (e.g., 1000 means levels 1 to 1000) or a range (e.g.,
620
- 200-1000). Defaults: log path '${DSSHOME}/dssdb/log', file name 'mydss.dbg'.
704
+ -DB or -Debug, enable debug logging. Up to three values may be given:
705
+ the debug level (required), the log file path, and the log file name.
706
+ The level is either a single integer (e.g. 1000 means levels 1 to 1000)
707
+ or a range (e.g. 200-1000). Defaults: log path '${DSSHOME}/dssdb/log',
708
+ file name 'mydss.dbg'.
621
709
 
622
710
  -DC or -DoneCount, the number of files successfully processed so far.
623
711
 
624
- -DF or -DownFlags, storage system down flags. Supported values: D-DRDATA,
625
- G-GLADE, O-ObjectStore. Up to 5 flags may be combined.
712
+ -DF or -DownFlags, storage system down flags. Valid flags: D-DRDATA,
713
+ G-GLADE, O-ObjectStore. Up to five flags may be combined.
626
714
 
627
- -DI or -DaemonIndex, daemon control indices. A daemon control record holds
628
- the configuration for how many processes of a given command a specialist may
629
- run on a given host, and at what priority.
715
+ -DI or -DaemonIndex, daemon control indices. A daemon control record
716
+ defines how many concurrent processes of a given command a specialist
717
+ may run on a given host, and at what priority.
630
718
 
631
- -DS or -Dataset, dataset numbers (also called dataset IDs), in the format
719
+ -DS or -Dataset, dataset numbers (also called dataset IDs). Format:
632
720
  [a-z]NNNNNN.
633
721
 
634
722
  -ER or -ErrorMessage, the error message from a failed check command.
635
723
 
636
- -EV or -Environments (Alias: -Envs), environment variables required to
637
- execute a command as a batch job, in the form VarName=VarValue and separated
638
- by commas. These variables are set in the batch startup script.
724
+ -EV or -Environments (Alias: -Envs), environment variables required
725
+ to run a command as a batch job, in the form VarName=VarValue, separated
726
+ by commas. The variables are set in the batch startup script.
639
727
 
640
- -FC or -FileCount, the number of files to be processed by a check command.
728
+ -FC or -FileCount, the number of files to be processed by a check
729
+ command.
641
730
 
642
- -HN or -HostName, the host names on which the check can or cannot be
731
+ -HN or -HostName, host names on which the check can or cannot be
643
732
  processed.
644
733
 
645
- -IF or -InputFile, one or more input file names provided on the command line.
646
- Input files hold options and their values for a 'dscheck' execution.
647
-
648
- In an input file:
649
- - Lines beginning with '#' are comments and are ignored.
650
- - Option names may be given in short, long, or alias form.
651
- - Action/Mode options: OptionName<!> (marker changeable via -AO)
652
- - Single-value assignments: OptionName<=>OptionValue, one per line
653
- (delimiter changeable via -ES; default '<=>')
654
- - Multi-value (tabular) assignments: a title row followed by data rows,
655
- with columns separated by '<:>' (changeable via -DV).
656
- - Tabular data ends at EOF or when a new title line or single-value
657
- assignment is encountered.
658
- - If the last column contains multi-line text, append the separator to
659
- every row (including the title line).
660
-
661
- -MC or -MaxCount, the maximum number of times a failed check command may be
662
- retried.
734
+ -IF or -InputFile, one or more input file names supplied on the command
735
+ line. An input file holds options and their values for a dscheck
736
+ invocation.
737
+
738
+ Input file format:
739
+ - Lines starting with '#' are comments and are ignored.
740
+ - Option names may appear in short, long, or alias form.
741
+ - Action/Mode markers: OptionName<!>
742
+ (marker overridable via -AO)
743
+ - Single-value assignments: OptionName<=>OptionValue
744
+ (one per line; '<=>' overridable via -ES)
745
+ - Multi-value tabular assignments:
746
+ a title row followed by data rows, columns separated by '<:>'
747
+ ('<:>' overridable via -DV).
748
+ - A tabular block ends at EOF, at the next title line, or at the next
749
+ single-value assignment.
750
+ - If the last column of a tabular block contains multi-line text,
751
+ append the column separator to every row, including the title row.
752
+
753
+ -MC or -MaxCount, the maximum number of times a failed check command
754
+ may be retried.
663
755
 
664
756
  -MH or -MatchHost (Alias: -MatchHostname), controls how hostnames are
665
- matched: 'G' (general match — includes empty hostname or exclusive hostname
666
- lists where the current host is not listed) or 'M' (exact match — only
667
- the specified hostname).
757
+ matched. 'G' is a general match - it includes empty hostnames and
758
+ exclusive lists where the current host is not listed. 'M' is an exact
759
+ match against the specified hostname.
668
760
 
669
- -MO or -Modules (Alias: -Mods), module names, separated by commas, needed
670
- to execute a command as a batch job. These modules are loaded in the batch
761
+ -MO or -Modules (Alias: -Mods), comma-separated module names required
762
+ to run a command as a batch job. The modules are loaded in the batch
671
763
  startup script.
672
764
 
673
- -PI or -ParentIndex, the parent check index that the current check must wait
674
- for before it can run.
765
+ -PI or -ParentIndex, the parent check index that the current check
766
+ must wait for before it can run.
675
767
 
676
- -PO or -Priority, the priority of a given host, determining the order in
677
- which hosts are selected to start a 'dscheck' process.
768
+ -PO or -Priority, the priority of a host within a daemon control,
769
+ determining the order in which hosts are tried when starting a check.
678
770
 
679
- -PL or -ProcessLimit, the maximum number of concurrent processes that can be
680
- started for a given command and specialist on a specified host. Also used with
681
- Action -SO (-SetOptions) to limit concurrent cron processes.
771
+ -PL or -ProcessLimit, the maximum number of concurrent processes
772
+ allowed for a given (command, specialist) on a given host. Also used
773
+ with -SO (-SetOptions) to bound concurrent cron processes.
682
774
 
683
- -PQ or -PBSQueue, the PBS batch queue to which the current check is submitted.
775
+ -PQ or -PBSQueue, the PBS batch queue to which the current check is
776
+ submitted.
684
777
 
685
- -QS or -QsubOptions (Alias: -PBSOptions), options passed to qsub when running
686
- a command as a PBS batch job. Quote the value on the command line, e.g.,
687
- -QS '-l walltime=12:00:00'.
778
+ -QS or -QsubOptions (Alias: -PBSOptions), options passed to qsub when
779
+ the command runs as a PBS batch job. Quote the value on the command
780
+ line, e.g. -QS '-l walltime=12:00:00'.
688
781
 
689
- -SN or -Specialist, the specialist who ran the original command. Used with
690
- Action -GC (-GetCheck) to view check records owned by a specific specialist.
782
+ -SN or -Specialist, the specialist who originally ran the command.
783
+ Used with -GC (-GetCheck) and -GD (-GetDaemon) to view records owned
784
+ by a particular specialist.
691
785
 
692
- -ST or -Status, the check status for a recorded command, including progress
693
- percentage for running commands and error messages for failed ones.
786
+ -ST or -Status, the check status for a recorded command, including
787
+ progress percentage for running commands and error messages for failed
788
+ ones.
694
789
 
695
- -SZ or -DataSize, the total bytes of data processed for the check record.
790
+ -SZ or -DataSize, the total number of bytes processed for the check
791
+ record.
696
792
 
697
- -TC or -TryCount, the number of times the check command has been attempted,
698
- up to the limit set via -MC.
793
+ -TC or -TryCount, the number of times the check command has been
794
+ attempted, up to the limit set by -MC.
699
795
 
700
- -WD or -WorkDir (Alias: -WorkDirectory), the working directory from which
701
- the recorded command is started.
796
+ -WD or -WorkDir (Alias: -WorkDirectory), the working directory from
797
+ which the recorded command is started.
{rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/METADATA RENAMED
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rda_python_dscheck
3
- Version: 2.0.6
3
+ Version: 2.0.7
4
4
  Summary: RDA python package to add and process batch jobs
5
5
  Author-email: Zaihua Ji <zji@ucar.edu>
6
6
  Project-URL: Homepage, https://github.com/NCAR/rda-python-dscheck
{rda_python_dscheck-2.0.6.dist-info → rda_python_dscheck-2.0.7.dist-info}/RECORD RENAMED
@@ -2,11 +2,11 @@ rda_python_dscheck/PgCheck.py,sha256=lvh0XSWJID9J9fjqmDtpq6xW2ULC-bChAxAkm8MFzbk
2
2
  rda_python_dscheck/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
3
3
  rda_python_dscheck/ds_check.py,sha256=-js5Ex1k7V8ziE_k-ld5Z0l44QGYhvQtdYri9vsqNJA,24303
4
4
  rda_python_dscheck/dscheck.py,sha256=-vv7gw4xYIa1xofWo-o3972dDnR_FQZxZz6_S_kCCuk,24805
5
- rda_python_dscheck/dscheck.usg,sha256=L0HZRoFGIqBPVQGjJQmb6mxMBsuqpN8QSp0RQ4YGjmc,30895
5
+ rda_python_dscheck/dscheck.usg,sha256=upCHGwRcjrLoue4Z9nE3pMDZ3s09Qq1ujEH2eJhb2cg,32036
6
6
  rda_python_dscheck/pg_check.py,sha256=Riv1nJYocSKLDsOIIYmOmkKlWZLKDrA1uPpbUEYo1uo,67388
7
- rda_python_dscheck-2.0.6.dist-info/licenses/LICENSE,sha256=1dck4EAQwv8QweDWCXDx-4Or0S8YwiCstaso_H57Pno,1097
8
- rda_python_dscheck-2.0.6.dist-info/METADATA,sha256=ZA1dxKZhTevLCRPZNlS5jr2YcS_ILiKtLSFGq-ZfSWc,812
9
- rda_python_dscheck-2.0.6.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
10
- rda_python_dscheck-2.0.6.dist-info/entry_points.txt,sha256=frVByX3Vp_Pd_zq93SXfItr5OMTd5tKeZzu5pbVXiGA,60
11
- rda_python_dscheck-2.0.6.dist-info/top_level.txt,sha256=NrvWqcHMRlQZJb0w85JrDlSXTp9DF-pDRQaAEABzFts,19
12
- rda_python_dscheck-2.0.6.dist-info/RECORD,,
7
+ rda_python_dscheck-2.0.7.dist-info/licenses/LICENSE,sha256=1dck4EAQwv8QweDWCXDx-4Or0S8YwiCstaso_H57Pno,1097
8
+ rda_python_dscheck-2.0.7.dist-info/METADATA,sha256=ezkK1sDQXrTcaTtHRlemEpeBOBQDmCOcHFLwuJainQk,812
9
+ rda_python_dscheck-2.0.7.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
10
+ rda_python_dscheck-2.0.7.dist-info/entry_points.txt,sha256=frVByX3Vp_Pd_zq93SXfItr5OMTd5tKeZzu5pbVXiGA,60
11
+ rda_python_dscheck-2.0.7.dist-info/top_level.txt,sha256=NrvWqcHMRlQZJb0w85JrDlSXTp9DF-pDRQaAEABzFts,19
12
+ rda_python_dscheck-2.0.7.dist-info/RECORD,,