rda-python-dscheck 2.0.5__py3-none-any.whl → 2.0.7__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. rda_python_dscheck/dscheck.usg +581 -525
  2. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/METADATA +1 -1
  3. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/RECORD +7 -7
  4. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/WHEEL +0 -0
  5. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/entry_points.txt +0 -0
  6. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/licenses/LICENSE +0 -0
  7. {rda_python_dscheck-2.0.5.dist-info → rda_python_dscheck-2.0.7.dist-info}/top_level.txt +0 -0
rda_python_dscheck/dscheck.usg CHANGED
@@ -1,109 +1,157 @@
1
1
 
2
2
  1 INTRODUCTION
3
3
 
4
- Program 'dscheck' is a utility to add, remove, view and process recorded commands
5
- of other utility programs in Research Data Archive Management System (GDEXMS). For
6
- delayed execution, or called batch process, of utility programs, 'dsarch', 'dsupdt'
7
- and 'dsrqst', the commands information and the directories where the commands are
8
- initiated are saved into GDEXDB as check records. For any other specialist-defined
9
- commands, they can be also put in delayed mode if the commands are added to
10
- 'dscheck' control via Action -AC (-AddCheck).
11
-
12
- The check records are processed automatically via a centralized 'dscheck' daemon,
13
- although they can be processed manually on command line. During the execution of a
14
- recorded command, the check record is locked in GDEXDB to prevent multiple executions
15
- of the same command. The check records that retain command information are
16
- automatically purged into check history when the commands are finished.
17
-
18
- When a recorded command fails due to failures of computer system, storage disk/tape,
19
- or the Database Management System, the check record is normally purged with status 'E'
20
- for error, unless check-reprocessing ability is built into the utility program, such
21
- as 'dsarch'. For a check-reprocessing command, the check record is retained in GDEXDB
22
- until the command is processed successfully or the number of executions reaches the
23
- try limits allowed. Utility programs 'dsrqst' and 'dsupdt' carry their own failure-
24
- recovering ability and they do not need check-reprocessing.
25
-
26
- The purged check records are retained in GDEXDB. The check history can be viewed via
27
- utility program 'viewcheckusage'.
28
-
29
- Program 'dscheck' supports the following major functions:
30
- - Set daemon control records for individual specialists to configure how many
31
- concurrent processes of a specified command can be executed on a given host, and
32
- the host priorities to define the order of which host is picked for processing
33
- a check. Without the daemon control information, a recorded command will not be
34
- started automatically.
35
- - ADD a check record for delayed command execution of any specialist-defined commands
36
- - View utility command information currently saved in GDEXDB
37
- - Email the current status of a specified utility command or a list of commands,
38
- and include error messages if any, to a specialist
39
- - Delete recorded command information, because the command is not needed anymore
40
- - Unlock a given recorded command in case lock information was not cleaned
41
- properly when the command has failed
42
- - Interrupt a utility command that is under execution and kill recursively all
43
- the associated child processes
44
- - Add the due 'dsrqst' and 'dsupdt' actions into dscheck records
45
- - Process commands that have been recorded into GDEXDB or the ones that have
46
- failed previously.
47
-
48
- The specialist who executes a utility command under 'dscheck' control remains
49
- the exclusive owner of the check record in GDEXDB. This prevents the command from being
50
- processed or deleted accidentally by other specialists.
51
-
52
- In the following sections, general usages of 'dscheck' are described first; and
53
- detail descriptions of Action options are given; and finally Mode and Info
54
- options are explained.
4
+ 'dscheck' is the command-line interface to the centralized batch-processing
5
+ system for the GDEX Research Data Archive Management System (GDEXMS). It lets
6
+ DECS specialists schedule, monitor, control, and clean up deferred (delayed)
7
+ executions of utility programs such as 'dsarch', 'dsupdt', and 'dsrqst', as
8
+ well as any specialist-defined command.
9
+
10
+ The mental model has three layers:
11
+
12
+ 1. A check record - a row in GDEXDB that captures one deferred command:
13
+ its name, arguments, owner, working directory, target
14
+ host, retry policy, and current status.
15
+ 2. A daemon control - a row in GDEXDB that tells the centralized 'dscheck'
16
+ daemon how many concurrent processes of a given
17
+ command a given specialist may run on a given host,
18
+ and at what host priority.
19
+ 3. The dscheck daemon - a long-running process that wakes on a fixed
20
+ interval, adds due 'dsupdt'/'dsrqst' check records,
21
+ and starts (or restarts) commands from check
22
+ records on hosts chosen according to daemon
23
+ control configuration.
24
+
25
+ Typical lifecycle of a deferred command:
26
+
27
+ added ---> pending ---> running ---> finished (purged to history)
28
+ \ \
29
+ \ +---> failed (E) [purged]
30
+ \ \
31
+ +--> retried (if reprocessing supported, e.g. dsarch)
32
+
33
+ A check record is locked while its command runs so the same command is never
34
+ started twice. When the command finishes successfully, or fails terminally,
35
+ the check record is moved to check history. History records can be reviewed
36
+ with the utility 'viewcheckusage'.
37
+
38
+ Failure handling depends on the calling utility. 'dsarch' supports
39
+ check-reprocessing: failed checks remain in GDEXDB until the command
40
+ succeeds or its maximum retry count is reached. 'dsrqst' and 'dsupdt' have
41
+ their own recovery paths and do not rely on dscheck retries; for them, an
42
+ errored check is purged with status 'E'.
43
+
44
+ A check is owned exclusively by the specialist who submitted it. Other
45
+ specialists cannot accidentally process or delete it.
46
+
47
+ What 'dscheck' can do:
48
+
49
+ - Configure daemon controls per (specialist, command, host) so that the
50
+ daemon knows where, how many at a time, and in what host order to run
51
+ commands. Without daemon controls, recorded commands are not started
52
+ automatically.
53
+ - Add a check record to schedule a deferred run of any specialist-defined
54
+ command.
55
+ - Inspect daemon controls and check records currently in GDEXDB.
56
+ - Email the current status of one or more checks (including any error
57
+ messages) to a specialist.
58
+ - Remove check records that are no longer needed.
59
+ - Unlock a check whose lock was left behind after an abnormal termination.
60
+ - Interrupt a running check and recursively kill all of its child
61
+ processes.
62
+ - Add due 'dsupdt' and 'dsrqst' work to dscheck records.
63
+ - Process checks (start them on the configured hosts) in daemon mode or
64
+ on demand.
65
+ - Verify daemon-host connectivity for a specialist.
66
+
67
+ The remainder of this document is organized as follows. Section 2 covers
68
+ general usage and conventions. Section 3 describes Action options grouped
69
+ by what they manipulate (daemon controls, check records, check processing,
70
+ host connectivity, batch options). Section 4 lists Mode options that modify
71
+ how an action behaves. Section 5 lists Information options that pass values
72
+ into an action.
55
73
 
56
74
  2 GENERAL DSCHECK USAGE
57
75
 
58
76
  dscheck [Action Option] [Mode Options] [Info Options]
59
- or
77
+ or
60
78
  dscheck [-(IF|InputFile)] InputFileNames
61
79
 
62
- Quotes [] indicate optional. A pipeline '|' in parentheses as in format (A|B)
63
- means either A or B can be used. The options applied to 'dscheck' are divided
64
- into three categories, Action, Mode, and Information (Info for short) options.
65
- Action options are used to specify what task this utility program is to execute,
66
- Mode options are used to modify behaviors of the actions, and the Info options
67
- are used to pass information, one or multiple values, to run 'dscheck'. An option
68
- can be given in form of either short name or long name, -DS or -Dataset for
69
- example. Some options have alias names for convenience; -UnLock, for example,
70
- is an alias option name for Mode option -UL (-UnLockCheck). Option names can be
71
- given in either upper or lower cases, while the values following Info options
72
- are case sensitive.
73
-
74
- Specify one of the Action options each time to execute 'dscheck'. Based on what
75
- Action is chosen, some of the Info options are mandatory and others are optional
76
- and certain Mode options can be applied to alter the behaviors of the Action.
77
-
78
- All options, except Info option -IF (-InputFile), can be given either on command
79
- line or in input files. Input file names are presented per Info option -IF and
80
- can only be provided on command line. Refer to the description of Info option -IF
81
- (-InputFile) for details on how to present options in input files. One or multiple
82
- input files, combined with options on command line, are allowed to run 'dscheck'.
83
- The option name, -IF (-InputFile), itself can be omitted if a single input file is
84
- given on command line and all other option information are provided inside the
85
- input file.
86
-
87
- When information of daemons and checks are retrieved, Info options are used to
88
- specify conditions for querying information from GDEXDB. Some special signs can be
89
- used to further confine the information with special and complicated conditions;
90
- they are '!', '<', '>' and '<>'. These special signs, if provided on command
91
- line, must be quoted or escaped to avoid being interpreted by the Unix shell.
92
- The '!', or \!, means exclusion to the following value(s) and it must be the
93
- first item following an Info option name, while '<' or '>' mean greater or less
94
- than the following value and '<>' means between the following two values.
95
- Combine '!' and '<', as syntax "'!' '<' OptionValue", to make a condition of
96
- 'larger than or equal to OptionValue'.
97
-
98
- Description of an individual option is displayed if 'dscheck' is issued on
99
- command line as
100
-
101
- dscheck [Option] -(h|help) [Option]
102
-
103
- A description is displayed for an option given either before or after -(h|help).
104
- If no option is specified or 'dscheck' is issued by itself, this whole document
105
- is displayed via UNIX utility 'more'. A hard copy of this help document can be
106
- printed from the saved file, dscheck.usg, under python package rda_python_dscheck.
80
+ Notation:
81
+ [] - the enclosed element is optional.
82
+ (A|B) - either A or B may be used (a short form and a long form).
83
+ -OPT - an option name. Names are case-insensitive; values are not.
84
+
85
+ Every dscheck invocation has at most one Action option, which selects the
86
+ task to perform. Mode options change how the chosen action behaves. Info
87
+ options carry the data the action needs (indices, names, hostnames, dates,
88
+ etc.). Each action documents which Info options are required and which are
89
+ optional.
90
+
91
+ Many options have an alias for convenience; for example, -UnLock is an
92
+ alias for the Mode option -UL (-UnLockCheck). Both short and long forms
93
+ are accepted, and aliases are noted with each option.
94
+
95
+ 2.1 Quick Start
96
+
97
+ Show the full document, paged through 'more':
98
+ dscheck
99
+
100
+ Show the description of a single option:
101
+ dscheck -h -AC
102
+ dscheck -AC -h
103
+
104
+ List your own daemon control records:
105
+ dscheck -GD
106
+
107
+ List your own active check records:
108
+ dscheck -GC
109
+
110
+ Add a deferred run of a script on PBS:
111
+ dscheck -AC -CM myscript.sh -HN PBS
112
+
113
+ Email yourself the status of all your active checks:
114
+ dscheck -EC
115
+
116
+ Start the dscheck daemon (typical for the operator):
117
+ dscheck -PC -DM start
118
+
119
+ 2.2 Filters and Special Characters in Queries
120
+
121
+ For Get-style actions (-GD, -GC), Info options are interpreted as query
122
+ filters. Four characters refine those filters; quote or escape them on the
123
+ command line so the shell does not consume them:
124
+
125
+ '!' - negate the match. Must appear immediately after the option name.
126
+ '<' - less-than comparison against the next value.
127
+ '>' - greater-than comparison against the next value.
128
+ '<>' - range between the next two values (inclusive).
129
+
130
+ The combination "'!' '<' value" expresses 'greater than or equal to value'.
131
+
132
+ 2.3 Input Files
133
+
134
+ Every option except -IF (-InputFile) may be supplied either on the command
135
+ line or in an input file. -IF is itself only valid on the command line.
136
+ A single dscheck invocation may combine command-line options with one or
137
+ more input files. If the only options are inside a single input file, the
138
+ -IF prefix may be omitted:
139
+
140
+ dscheck mycheck.in
141
+
142
+ See the description of -IF in Section 5.2 for the input-file format
143
+ (comments, action/mode markers, single-value assignments, and tabular
144
+ multi-value assignments, including delimiter overrides).
145
+
146
+ 2.4 Getting Help
147
+
148
+ dscheck # full document, displayed via 'more'
149
+ dscheck -(h|help) # full document
150
+ dscheck [Option] -(h|help) # description of [Option]
151
+ dscheck -(h|help) [Option] # same as above
152
+
153
+ A printable copy of this document is the file dscheck.usg installed under
154
+ the python package rda_python_dscheck.
107
155
 
108
156
  #The online HTML version of this document is available at
109
157
  #https://gdex-docs-dscheck.readthedocs.io
@@ -111,86 +159,85 @@ printed from the saved file, dscheck.usg, under python package rda_python_dschec
111
159
 
112
160
  3 ACTION OPTIONS
113
161
 
114
- Action options are used to specify what task 'dscheck' executes. No values
115
- are allowed to follow Action options. Multiple Action options provided
116
- simultaneously are blocked.
117
-
118
- Based on the information being manipulated, the actions are divided into three
119
- categories:
120
- Daemon Control Actions - create, delete, modify and view daemon control information
121
- in GDEXDB, of specified specialist, command and hostname
122
- Check Actions - add, delete, unlock and view check information of the
123
- active individual checks
124
- Check Process Actions - process checks by starting commands on remote hosts as
125
- configured in daemon controls and purge checks by recording
126
- the commands and their execution information into check
127
- history; interrupt the current executing commands by killing
128
- the current process and its all children; and email status
129
- of current checks
130
- Daemon Host Connectivity - check connectivity of daemon hosts for specialists
162
+ An Action option selects what 'dscheck' does. Action options take no
163
+ values, and exactly one Action option may be supplied per invocation.
164
+
165
+ Action options are grouped into four categories:
166
+
167
+ Daemon Control Actions - create, modify, view, and delete daemon
168
+ control records that govern how, where, and
169
+ how concurrently commands run.
170
+ Check Actions - add, view, delete, and unlock individual
171
+ check records.
172
+ Check Process Actions - start commands from check records, interrupt
173
+ running commands, or email check status.
174
+ Daemon Host Connectivity - verify that daemon hosts are reachable for
175
+ a specialist.
176
+
177
+ A fifth, narrower action -SO (-SetOptions) builds batch options dynamically
178
+ and is documented in Section 3.5.
131
179
 
132
180
  3.1 Daemon Control Actions
133
181
 
134
- A daemon control record for a command, a specialist and a hostname is used to
135
- configure how many concurrent processes the specified command can be executed
136
- for the specialist on specified hostname, and the priority the hostname is
137
- picked to start the command. A running centralized daemon reads this record
138
- periodically in case the configuration is changed while the daemon is still
139
- running, so that specialists can reset the values in daemon control records to
140
- change the behavior of dscheck daemon dynamically without shutting the daemon down.
141
-
142
- Daemon control information can be created, modified and viewed via Actions
143
- included in this section:
144
- Set Daemon Control - create and modify daemon control information for specified
145
- specialists, commands and hostnames
146
- Get Daemon Control - retrieve information of existing daemon controls
147
- Delete Daemon Control - delete one or multiple daemon control records
182
+ A daemon control record answers three questions for the dscheck daemon:
183
+
184
+ - Whose checks? (specialist login)
185
+ - For which command? (utility program name, or 'ALL')
186
+ - On which host, at what concurrency, at what priority?
187
+ (hostname, ProcessLimit, Priority)
188
+
189
+ Daemon controls are read by the daemon on every cycle, so changes take
190
+ effect without restarting the daemon. A specialist with no daemon control
191
+ records will have no checks started automatically.
192
+
193
+ Available actions:
194
+
195
+ Set Daemon Control - create or modify daemon control records.
196
+ Get Daemon Control - retrieve existing daemon control records.
197
+ Delete Daemon Control - remove daemon control records by index.
148
198
 
149
199
  3.1.1 Set Daemon Control
150
- -SD or -SetDaemon, creates and modifies daemon control information into
151
- GDEXDB for given specialist login names, commands, and hostnames of computers on
152
- which the check commands are processed. One or multiple records can be set each
153
- time.
200
+ -SD or -SetDaemon, creates or modifies daemon control records in GDEXDB
201
+ for given specialist login names, commands, and hostnames. One or many
202
+ records can be set per invocation.
154
203
 
155
204
  dscheck -(SD|SetDaemon) [Mode Option]
156
205
  [-(DI|DaemonIndex) controlIndices]
157
- [-(CM|Command) UtilityProgramNames] # ALL for any command names
206
+ [-(CM|Command) UtilityProgramNames] # ALL for any command name
158
207
  [-(SN|Specialist) DECSSpecialists]
159
208
  [-(HN|HostName) HostMachineNames]
160
209
  [-(MH|MatchHost) FlagToMatchHostname]
161
210
  [-(PL|ProcessLimit) MaxNumberOfProcesses],
162
211
  [-(PO|Priority) HostListOrder]
163
212
 
164
- Mode option that can be specified for this action include:
165
- -(ND|NewDaemon) - sets a new daemon control record into GDEXDB
213
+ Available mode option:
214
+ -(ND|NewDaemon) - add a new daemon control record (required when the
215
+ daemon index is 0).
216
+
217
+ An existing record matching the (specialist, command, hostname) triple
218
+ is updated in place. To create a new record, supply daemon index 0 and
219
+ the -ND mode option; this guards against creating records by accident.
220
+ The (specialist, command, hostname) triple must be unique.
166
221
 
167
- If information of a daemon control exists already in GDEXDB for a given specialist,
168
- a command and a hostname, the daemon control record is modified; otherwise, a new
169
- daemon control record is added if daemon index is 0 and Mode option -ND (-NewDaemon)
170
- is present. Combination of specialist login name, command name and hostname of
171
- computer must be unique for each daemon control record.
222
+ Special hostnames and commands:
223
+ HN = PBS - submit the command to the PBS batch system.
224
+ CM = ALL - apply to any command not otherwise configured for this
225
+ specialist.
172
226
 
173
- Specify host name 'PBS' for putting the command in the PBS batch control system. If
174
- a specified command name is not found in the daemon control, the general 'dscheck'
175
- configuration for command name 'ALL' is used.
176
-
177
- For example, set daemon control information for schuster, all commands on PBS hosts,
178
- for maximum 4 checks can be processed at the same time with priority 1, the smaller
179
- the number the higher the priority is, via input file daemon.ctl:
227
+ Example - configure schuster on PBS for any command, with up to 4
228
+ concurrent checks at priority 1 (lower number = higher priority), driven
229
+ from input file daemon.ctl:
180
230
 
181
- dscheck -SD -ND -IF daemon.ctl
231
+ dscheck -SD -ND -IF daemon.ctl
182
232
 
183
- <<Content of input file daemon.ctl>>
184
- DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
185
- 0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
233
+ <<Content of input file daemon.ctl>>
234
+ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
235
+ 0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
186
236
 
187
237
 
188
238
  3.1.2 Get Daemon Control
189
- -GD or -GetDaemon, retrieves daemon control information for given commands,
190
- specialists or hostnames. Daemon control information of specified specialists
191
- are retrieved if the specialist login names are provided. Without specified
192
- condition, only the daemon control records set for the specialist who runs
193
- 'dscheck' are retrieved.
239
+ -GD or -GetDaemon, retrieves daemon control records. With no Info-option
240
+ filters, only records owned by the current specialist are returned.
194
241
 
195
242
  dscheck -(GD|GetDaemon) [Mode Option]
196
243
  [-(FN|FieldNames) FieldNameString]
@@ -200,68 +247,65 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
200
247
  [-(HN|HostName) HostMachineName]
201
248
  [-(PL|ProcessLimit) MaxNumberOfProcesses]
202
249
  [-(PO|Priority) ProcessPriority]
203
- [-(OF|OutputFile) OutputFileName]
250
+ [-(OF|OutputFile) OutputFileName]
204
251
  [-(DB|Debug) DebugModeInfo]
205
252
 
206
- Mode option that can be specified for getting check control Action:
207
- -(FO|FormatOutput) - format the column output with a fix width for all values
208
- of a given field
253
+ Available mode option:
254
+ -(FO|FormatOutput) - pad each column to a uniform width for readability.
209
255
 
210
- Use Info option -FN (-FieldNames) to specify what daemon control fields to be
211
- retrieved. It defaults to all available fields if option -FN is not provided.
256
+ -FN (-FieldNames) selects which daemon-control fields to retrieve as a
257
+ string of single-letter codes. When -FN is omitted, all fields are
258
+ returned.
212
259
 
213
- Valid field names of daemon controls and their corresponding Info option
214
- names:
260
+ Field codes for daemon control records:
215
261
 
216
- Names Info Options Descriptions
262
+ Code Info Option Description
217
263
  I -(DI|DaemonIndex) daemon control index
218
- C -(CM|Command) command names of utility programs
264
+ C -(CM|Command) command name of a utility program
219
265
  H -(HN|Hostname) computer hostname
220
- M -(MH|MatchHost) Flags to control hostname match
221
- S -(SN|Specialist) DECS specialist the daemon set for
222
- P -(PL|ProcessLimit) Max check count to be processed at the same times
223
- O -(PO|Priority) host priority a specified command to start on
224
-
225
- Daemon control information can be retrieved for specified specialist per option
226
- -SN (-Specialist), and/or other conditions. Info option -SN, -CM and -HN accept
227
- wildcard input of '%' for matching any number of characters.
228
-
229
- If daemon control index is not given, 'dscheck' gathers only the daemon control
230
- records owned by the specialist who executes this getting daemon control Action.
231
- To view daemon control records owned by another specialist, you need specify Info
232
- option -SN (-Specialist). To view all control records, you simply provide option
233
- -SN with value of 'ALL'.
234
-
235
- For example, to get all daemon control information currently set for you:
236
-
237
- dscheck GD
266
+ M -(MH|MatchHost) flag controlling hostname match
267
+ S -(SN|Specialist) DECS specialist this control applies to
268
+ P -(PL|ProcessLimit) max concurrent processes for this control
269
+ O -(PO|Priority) host priority within this control
270
+
271
+ -SN, -CM, and -HN accept the SQL-style '%' wildcard. Pass -SN ALL to
272
+ retrieve every specialist's records (or a specific other specialist by
273
+ login name).
274
+
275
+ Example - retrieve all daemon control records owned by you:
276
+
277
+ dscheck -GD
238
278
 
239
279
 
240
280
  3.1.3 Delete Daemon Control
241
- -DL or -Delete (Alias: -RM, -Remove), deletes one or multiple daemon control records
242
- from GDEXDB for given daemon control indices.
243
-
281
+ -DL or -Delete (Alias: -RM, -Remove), removes one or more daemon control
282
+ records from GDEXDB by daemon control index.
283
+
244
284
  dscheck -(DL|Delete)
245
285
  -(DI|DaemonIndex) DaemonControlIndices
246
286
  [-(DB|Debug) DebugModeInfo]
247
287
 
248
- Use this action to delete daemon control information. Delete a daemon
249
- control record to remove the daemon specific configuration for a command,
250
- a specialist and a hostname.
288
+ Deleting a daemon control record removes the configuration for one
289
+ (specialist, command, hostname) combination. Existing check records are
290
+ not removed.
291
+
251
292
 
252
293
  3.2 Check Actions
253
- Delayed mode command executions for due actions of 'dsupdt' and 'dsrqst' are
254
- recorded automatically, while other commands, including 'dsarch' and specialist-
255
- defined ones, can be manually added into 'dscheck'. Command information can be
256
- added, viewed and manipulated via 'dscheck' actions:
257
- Add Check - Add a new check record for a specified command
258
- Get Check - get the command information recorded in check records
259
- Delete Check - delete check records for no need of processing the commands
260
- Unlock Check - unlock check records in case that its recorded command is aborted
261
- without cleaning the lock
294
+
295
+ Check records are created automatically by the dscheck daemon for due
296
+ 'dsupdt' and 'dsrqst' actions. Other commands - including 'dsarch' calls
297
+ and any specialist-defined script - are added manually with -AC.
298
+
299
+ Available actions:
300
+
301
+ Add Check - add a check record for a deferred command.
302
+ Get Check - retrieve check records and field values.
303
+ Delete Check - delete check records that should no longer run.
304
+ Unlock Check - clear leftover lock information after an abnormal exit.
262
305
 
263
306
  3.2.1 Add Check
264
- -AC or -AddCheck, adds check information for a delayed mode command execution.
307
+ -AC or -AddCheck, adds a check record so that a command runs later,
308
+ driven by the dscheck daemon (or by an explicit -PC invocation).
265
309
 
266
310
  dscheck -(AC|AddCheck) [Mode Option]
267
311
  -(CM|Command) CommandNames
@@ -277,53 +321,58 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
277
321
  [-(MO|Modules) ModuleList]
278
322
  [-(EV|Environments) EnvironmentPairList]
279
323
  [-(QS|QsubOptions) PBSBatchOptions]
280
- [-(OF|OutputFile) OutputFileName]
324
+ [-(OF|OutputFile) OutputFileName]
281
325
  [-(DB|Debug) DebugModeInfo]
282
326
 
283
- Mode option that can be specified for adding check control Action:
284
- -(AW|AnyWhere) - sets Working directory empty in check record to start processing
285
- the check anywhere.
327
+ Available mode option:
328
+ -(AW|AnyWhere) - leave the working directory empty in the check record
329
+ so the command may run from any location.
330
+
331
+ Defaults:
332
+ Owner - the specialist running 'dscheck'.
333
+ Working directory - the current directory at the time of -AC.
334
+ Host - none specified; daemon control governs placement.
335
+
336
+ Use -PI (-ParentIndex) to make the new check wait for an existing parent
337
+ check to finish before it can run. Use -QS to pass options through to
338
+ qsub when the command will run as a PBS batch job.
339
+
340
+ Limitations:
341
+ - Shell redirections and pipes (>, >>, |, <) are not supported in the
342
+ deferred command line. Wrap such commands in a shell script and add
343
+ the script.
286
344
 
287
- Command name is mandatory for adding a new check for delayed mode execution.
288
- Unless they are specified, the current specialist who adds the command is defaulted
289
- as the owner of the added check record, and the current path is defaulted as the
290
- working directory when the command is executed later.
345
+ Examples:
291
346
 
292
- Specify additional PBS batch options via Info option -QS (-QsubOptions) to add a check;
293
- and specify a parent check index to put the current command on hold until the parent
294
- check is finished.
347
+ List files containing 'test' in the current directory, capture stdout to
348
+ a log, and also display it on screen:
295
349
 
296
- Commands containing redirections and pipes are not supported for delayed mode.
297
- A simple shell script can be used to wrap a complicated command.
298
-
299
- For example, to list file names in the current directory with name containing 'test'
300
- by catching the standard output into a log and display it on screen:
301
-
302
- dscheck -AC -CM test1.sh -HN PBS
350
+ dscheck -AC -CM test1.sh -HN PBS
303
351
 
304
- <<Content of shell script test1.sh>>
305
- #!/bin/sh
306
- (ls -l | grep test) | tee test1.out
352
+ <<Content of shell script test1.sh>>
353
+ #!/bin/sh
354
+ (ls -l | grep test) | tee test1.out
307
355
 
308
- For example, to list file names in the current directory with name containing 'test'
309
- by catching the standard output and error into separate log files:
356
+ List files containing 'test', capturing stdout and stderr to separate
357
+ log files:
310
358
 
311
- dscheck -AC -CM test2.sh -HN PBS
359
+ dscheck -AC -CM test2.sh -HN PBS
312
360
 
313
- <<Content of shell script test2.sh>>
314
- #!/bin/sh
315
- (ls -l | grep test) 1> test2.out 2>test2.err
361
+ <<Content of shell script test2.sh>>
362
+ #!/bin/sh
363
+ (ls -l | grep test) 1> test2.out 2>test2.err
316
364
 
317
- For example, to add testing command 'test2' into 'dscheck' for delayed mode execution on
318
- PBS:
365
+ Add the executable 'test3' for deferred execution on PBS:
319
366
 
320
- dscheck -AC -CM test3 -HN PBS
367
+ dscheck -AC -CM test3 -HN PBS
321
368
 
322
- The command 'test3' must be executable at the current working directory on PBS machines.
369
+ The command 'test3' must be executable from the working directory on
370
+ the PBS machine.
323
371
 
324
372
 
325
373
  3.2.2 Get Check
326
- -GC or -GetCheck, gets check information recorded in GDEXDB.
374
+ -GC or -GetCheck, retrieves check records from GDEXDB. With no filter,
375
+ returns active check records owned by the current specialist.
327
376
 
328
377
  dscheck -(GC|GetCheck) [Mode Options]
329
378
  [-(FN|FieldNames) FieldNameString]
@@ -338,93 +387,97 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
338
387
  [-(CT|CheckTime) CommandTime]
339
388
  [-(WD|WorkDir) WorkingDirectory]
340
389
  [-(PQ|PBSQueue) PBSBatchQueue]
341
- [-(OF|OutputFile) OutputFileName]
390
+ [-(OF|OutputFile) OutputFileName]
342
391
  [-(DB|Debug) DebugModeInfo]
343
392
 
344
- Mode options that can be specified for getting check Action:
345
- -(CS|CheckStatus) - check and show detail information on check status
346
- -(FO|FormatOutput) - format the column output with a fix width for all values
347
- of a given field
393
+ Available mode options:
394
+ -(CS|CheckStatus) - include detailed status: progress percentage for
395
+ running commands and error messages for failed ones.
396
+ -(FO|FormatOutput) - pad each column to a uniform width for readability.
348
397
 
349
- Use Info option -FN (-FieldNames) to specify what check fields to retrieve.
350
- It defaults to "COVTUPFJDNW" if -FN is not given.
398
+ -FN (-FieldNames) selects which fields to retrieve as a string of
399
+ single-letter codes. When -FN is omitted, the default field set is
400
+ "COVTUPFJDNW".
351
401
 
352
- Valid check field names and their corresponding Info options:
402
+ Field codes for check records:
353
403
 
354
- Names Info Options Descriptions
404
+ Code Info Option Description
355
405
  C -(CI|CheckIndex) check index
356
406
  O -(CM|Command) original command name
357
- V -(AV|ArgumentVector) argument line following command, up to 100 chars
358
- T -(DS|Dataset) dataset ID, the original command run against
359
- A -(AN|ActionName) action name for a given command
360
- U -(ST|Status) check status for a recorded command
361
- B -(DF|DownFlags) Storage system down flags: D-DRDATA,G-GLADE,O-ObjectStore
362
- P -(PQ|PBSQueue) PBS batch queue name: gdex or htc
363
- R -(PI|ParentIndex) parent check index the current one to wait on
364
- F -(FC|FileCount) number of files need to be processed
365
- J -(DC|DoneCount) number of files are processed already
366
- K -(TC|TryCount) number of tries the command is executed
367
- L -(MC|MaxCount) upper limit for number of command tries
368
- Z -(SZ|DataSize) total bytes of data processed for the command
369
- D -(CD|CheckDate) date the command is initially recorded
370
- Y -(CT|CheckTime) time the command is initially recorded
371
- H -(HN|HostName) host names the command can or cannot run on
372
- N -(SN|Specialist) specialist login name who starts the command
373
- W -(WD|WorkDir) working directory where the command is started
374
- M -(MO|Modules) include modules to load to batch job script
375
- I -(EV|Environments) include environment variables to load to batch job script
376
- Q -(QS|QsubOptions) additional PBS batch options for qsub
377
- X -(AX|ArgumentExtra) additional argument line beyond 100 characters
378
- E -(ER|ErrorMessage) error message from failed command
379
-
380
- Check information can be retrieved for specified check index per option
381
- -CI (-CheckIndex). Without any condition, the check records owned by the
382
- current specialist are retrieved.
407
+ V -(AV|ArgumentVector) argument string after the command (<= 100 chars)
408
+ T -(DS|Dataset) dataset ID the command runs against
409
+ A -(AN|ActionName) action name for the command
410
+ U -(ST|Status) check status for the recorded command
411
+ B -(DF|DownFlags) storage down flags: D-DRDATA, G-GLADE,
412
+ O-ObjectStore
413
+ P -(PQ|PBSQueue) PBS batch queue name (e.g. gdex, htc)
414
+ R -(PI|ParentIndex) parent check index this check waits on
415
+ F -(FC|FileCount) number of files to be processed
416
+ J -(DC|DoneCount) number of files already processed
417
+ K -(TC|TryCount) number of times the command has run
418
+ L -(MC|MaxCount) maximum number of tries allowed
419
+ Z -(SZ|DataSize) total bytes processed for the command
420
+ D -(CD|CheckDate) date the command was first recorded
421
+ Y -(CT|CheckTime) time the command was first recorded
422
+ H -(HN|HostName) host(s) the command can or cannot run on
423
+ N -(SN|Specialist) specialist who owns the check
424
+ W -(WD|WorkDir) working directory for the command
425
+ M -(MO|Modules) modules to load in the batch job script
426
+ I -(EV|Environments) environment variables for the batch job
427
+ Q -(QS|QsubOptions) additional qsub options
428
+ X -(AX|ArgumentExtra) additional argument string beyond 100 chars
429
+ E -(ER|ErrorMessage) error message from a failed command
430
+
431
+ Use -CI (-CheckIndex) to fetch specific records. To inspect another
432
+ specialist's records, supply -SN with the appropriate login name.
383
433
 
384
434
 
385
435
  3.2.3 Delete Check
386
- -DL or -Delete, deletes one or multiple 'dscheck' records from GDEXDB
387
- for given check indices.
388
-
436
+ -DL or -Delete, removes check records from GDEXDB by check index.
437
+
389
438
  dscheck -(DL|Delete)
390
439
  -(CI|CheckIndex) CheckIndices
391
440
  [-(DB|Debug) DebugModeInfo]
392
441
 
393
- Use this action to delete check information. A deleted check record is saved
394
- into check history for viewing later via utility program 'viewcheckusage'.
442
+ A deleted check is moved to check history and remains visible via the
443
+ utility program 'viewcheckusage'.
395
444
 
396
445
 
397
446
  3.2.4 Unlock Check
398
- -UL or -UnLockCheck, (Alias: -UnLock), unlocks check records whose
399
- commands aborted abnormally during processing.
400
-
401
- Process ID and computer hostname are saved in a check record when the recorded
402
- command is running. If the process aborts abnormally, the PID and hostname may
403
- sometimes be not cleaned properly. Use this action to clean up the locking
404
- information to allow the command to be reprocessed or purged.
447
+ -UL or -UnLockCheck (Alias: -UnLock), clears stale lock information from
448
+ a check record.
449
+
450
+ When a check is being processed, the running process ID and hostname are
451
+ written into the check record. If the process aborts abnormally, that
452
+ lock may not be cleared. Unlocking a check allows it to be reprocessed
453
+ or purged.
405
454
 
406
455
  dscheck -(UL|UnLockCheck)
407
456
  -(CI|CheckIndex) CheckIndices
408
457
  [-(DB|Debug) DebugModeInfo]
409
458
 
410
- It is mandatory to provide a check index to remove lock on a recorded command.
459
+ -CI is required.
411
460
 
412
461
 
413
462
  3.3 Check Process Actions
414
- Delayed mode command executions recorded in GDEXDB are automatically started,
415
- or restarted, by the common 'dscheck' daemon. Running commands can be interrupted
416
- at any time and the child processed of the interrupted commands are also cleaned up.
417
- The status of the current check records can be gathered and emailed to a specialist.
418
-
419
- Here are the actions for process checks:
420
- Process Check - start a command from its information recorded in 'dscheck'
421
- Interrupt Check - interrupt a running command for given check record and clean the
422
- child processes
423
- Email Check - email a specialist for the status of the current check records
463
+
464
+ The dscheck daemon starts and restarts deferred commands automatically.
465
+ Operators (and, in non-daemon mode, specialists) can also drive command
466
+ execution from the command line, interrupt running commands, and email
467
+ status summaries.
468
+
469
+ Available actions:
470
+
471
+ Process Check - start commands from check records (daemon or
472
+ non-daemon mode).
473
+ Interrupt Check - stop a running command and recursively kill its
474
+ process tree.
475
+ Email Check - email a specialist the current status of one or more
476
+ check records.
424
477
 
425
478
  3.3.1 Process Check
426
- -PC or -ProcessCheck, starts process commands in check records in daemon or
427
- non-daemon modes. Check indices can be specified if in non-daemon mode.
479
+ -PC or -ProcessCheck, drives command execution from check records, in
480
+ either daemon mode or non-daemon mode.
428
481
 
429
482
  dscheck -(PC|ProcessCheck) [Mode Options]
430
483
  [-(CI|CheckIndex) CheckIndices]
@@ -434,47 +487,48 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
434
487
  [-(MT|MaxrunTime) MaximumRunTimeInSeconds]
435
488
  [-(DB|Debug) DebugModeInfo]
436
489
 
437
- Mode options that can be specified for processing check Action:
438
- -(BG|BackGround) - background process to turn off screen display for both
439
- standard outputs and errors
440
- -(CP|CheckPending) - Check and kill long pending checks
441
- -(LO|LogOn) - log detail for daemon mode
442
- -(NC|NoCommand) - does not issue remote commands if this Mode option is present
443
- -(WU|WithdsUpdt) - in non-daemon mode, add check records for due 'dsupdt' actions
444
- configured in update control records
445
- -(WR|WithdsRqst) - in non-daemon mode, add check records for 'dsrqst' records
446
- due to be built or purged
447
-
448
- In daemon mode, 'dscheck' sleeps two minutes (120 seconds as default), unless
449
- provided differently per option -WI (-WaitInterval), between processing check
450
- records. Every time it wakes up, 'dscheck' first tries to add the due update
451
- controls for command 'dsupdt' and add requests due to be built or purged for
452
- command 'dsrqst', and then starts, or restarts, commands recorded in check
453
- records on a computer according the priorities configured in the daemon controls.
454
-
455
- In non-daemon mode, Mode options -WU and -WR must be present for due actions
456
- of 'dsupdt' and 'dsrqst' to be added to check records. All current standing
457
- check records, unless check indices are specified, are processed on machines
458
- according to the configured information in daemon controls.
459
-
490
+ Available mode options:
491
+ -(BG|BackGround) - run as a background process; suppress screen output.
492
+ -(CP|CheckPending) - identify and kill long-pending checks.
493
+ -(LO|LogOn) - enable detailed logging in daemon mode.
494
+ -(NC|NoCommand) - do not actually execute remote commands (dry run).
495
+ -(WU|WithdsUpdt) - in non-daemon mode, add check records for due
496
+ 'dsupdt' actions configured in update controls.
497
+ -(WR|WithdsRqst) - in non-daemon mode, add check records for 'dsrqst'
498
+ records due to be built or purged.
499
+
500
+ Daemon mode (-DM start):
501
+ The daemon sleeps for -WI seconds (default 120) between cycles. Each
502
+ cycle, it adds due 'dsupdt' and 'dsrqst' check records, then starts
503
+ or restarts checks on hosts chosen by daemon control priorities.
504
+ -DM stop, -DM logon, and -DM logoff stop the daemon, enable detailed
505
+ logging, and disable detailed logging respectively.
506
+
507
+ Non-daemon mode:
508
+ Without -DM, dscheck processes the current set of check records once.
509
+ -WU and -WR must be set explicitly if you want due 'dsupdt'/'dsrqst'
510
+ work to be added before processing. -CI restricts processing to
511
+ specific check indices.
512
+
460
513
  3.3.2 Interrupt Check
461
- -IC or -InterruptCheck, interrupts checks whose commands are currently running
462
- and also kills recursively all the child processes running under those
463
- commands. Locks are cleaned too when the checks are interrupted.
514
+ -IC or -InterruptCheck, stops a running command and recursively kills
515
+ its child processes. The check's lock is released as part of the
516
+ interrupt.
464
517
 
465
518
  dscheck -(IC|InterruptCheck) [Mode Option]
466
519
  -(CI|CheckIndex) CheckIndices
467
520
  [-(DB|Debug) DebugModeInfo]
468
521
 
469
- Mode option that can be specified for this action:
470
- -(FI|ForceInterrupt) - it must be present for this action to interrupt a check
471
- that is under processing.
522
+ Available mode option:
523
+ -(FI|ForceInterrupt) - required to interrupt a check that is currently
524
+ being processed; without it dscheck only warns.
472
525
 
473
- It is mandatory to provide a check index to interrupt its command.
526
+ -CI is required.
474
527
 
475
528
  3.3.3 Email Check
476
- -EC or -EmailCheck, sends an email to a specialist for the current status of all check
477
- records owned by the specialist, unless more specific conditions are provided.
529
+ -EC or -EmailCheck, emails a specialist a status report for their check
530
+ records. With no filters, the report covers all of the recipient's
531
+ active checks; otherwise it is restricted to the records that match.
478
532
 
479
533
  dscheck -(EC|EmailCheck)
480
534
  [-(CI|CheckIndex) CheckIndices]
@@ -488,12 +542,13 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
488
542
  [-(CC|CarbonCopy) Cc'dEmailAddresses]
489
543
  [-(DB|Debug) DebugModeInfo]
490
544
 
491
- Command error messages are included too if any error messages are recorded in
492
- the check record.
545
+ Any error messages stored on a failed check are included in the email.
546
+
493
547
 
494
548
  3.4 Daemon Host Connectivity
495
- -CH or -CheckHost, checks connectivity of daemon hosts for specialists.
496
-
549
+ -CH or -CheckHost, verifies that the hosts referenced in daemon control
550
+ records are reachable from the current host for the given specialist.
551
+
497
552
  dscheck -(CH|CheckHost)
498
553
  [-(DI|DaemonIndex) controlIndices]
499
554
  [-(CM|Command) UtilityProgramNames]
@@ -503,10 +558,13 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
503
558
  [-(PO|Priority) ProcessPriority]
504
559
  [-(DB|Debug) DebugModeInfo]
505
560
 
561
+
506
562
  3.5 Set Batch Options Dynamically
507
- -SO or -SetOptions (Alias: -SetBatchOptions), processes batch options with
508
- a leading '!' to build options dynamically.
509
-
563
+ -SO or -SetOptions (Alias: -SetBatchOptions), evaluates batch options
564
+ whose values begin with '!' so that PBS options can be assembled
565
+ dynamically (for example, by resolving environment- or workload-derived
566
+ values at submission time).
567
+
510
568
  dscheck -(SO|SetOptions)
511
569
  [-(HN|HostName) HostNames]
512
570
  [-(PL|ProcessLimit) MaxNumberOfProcesses]
@@ -515,227 +573,225 @@ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
515
573
  [-(QS|QsubOptions) PBSBatchOptions]
516
574
  [-(DB|Debug) DebugModeInfo]
517
575
 
518
-
576
+
519
577
  4 MODE OPTIONS
520
578
 
521
- Use proper Mode options to modify behaviors of Action options. Mode options
522
- are all optional. No value is allowed to be passed in following any Mode option.
579
+ Mode options modify the behavior of an Action option. They take no values.
580
+ Each Mode option is meaningful only with specific actions; the relevant
581
+ action is noted with each one below.
523
582
 
524
- -AW or -AnyWhere, for Action -AC (-AddCheck), sets Working directory empty in
525
- check record to start processing the check anywhere.
583
+ -AW or -AnyWhere, used with -AC (-AddCheck): leave the check's working
584
+ directory empty so the command may run from any location.
526
585
 
527
- -BG or -BackGround (Alias: -b), background process. When it is present,
528
- screen display is turned off for both standard output and errors.
586
+ -BG or -BackGround (Alias: -b), run as a background process and suppress
587
+ all screen output and error messages.
529
588
 
530
- -CP or -CheckPending, if present for Action -PC (-ProcessCheck), checks and kills
531
- long pending checks.
589
+ -CP or -CheckPending, used with -PC (-ProcessCheck): identify and kill
590
+ checks that have been pending too long.
532
591
 
533
- -CS or -CheckStatus, if present, displays detailed status information of the
534
- recorded commands, including progress percentage for a running command and
535
- error message for a failed command.
536
-
537
- -FI or -ForceInterrupt, if present, force interrupts a check that is still under
538
- processing; otherwise a warning message will display.
592
+ -CS or -CheckStatus, used with -GC (-GetCheck): include detailed status
593
+ for each check, including progress percentage for running commands and
594
+ error messages for failed ones.
539
595
 
540
- -FO or -FormatOutput, if present, formats column output results for get
541
- actions. A same width, evaluated dynamically, is applied for all values of a
542
- given field.
596
+ -FI or -ForceInterrupt, required with -IC (-InterruptCheck) to actually
597
+ interrupt a check that is currently being processed; without it, dscheck
598
+ prints a warning instead of interrupting.
543
599
 
544
- -LO or -LogOn (Alias: -LoggingOn), turns detail logging on when 'dscheck' starts
545
- in Daemon mode. The detail logging is off as default if this Mode option is
546
- not present.
600
+ -FO or -FormatOutput, used with Get-style actions (-GD, -GC): pad each
601
+ column to a uniform width computed from the data, for readability.
547
602
 
548
- -MD or -MyDataset, allows a specialist to manipulate check information of
549
- a given dataset listed for another specialist.
603
+ -LO or -LogOn (Alias: -LoggingOn), used with -PC -DM start: enable
604
+ detailed daemon logging from startup. Detailed logging is off by default.
550
605
 
551
- -NC or -NoCommand (Alias: -NoRemoteCommand), does not issue remote commands if
552
- this Mode option is present for action -PC (-ProcessCheck).
606
+ -MD or -MyDataset, allows a specialist to access check information for
607
+ a dataset owned by a different specialist.
553
608
 
554
- -ND or -NewDaemon, a new daemon control can only be added when this Mode option is
555
- present and daemon control index is given as 0 when Action -SD (-SetDaemon) of
556
- 'dscheck' is executed. This Mode option blocks mistakes of adding daemon control
557
- records unintentionally.
609
+ -NC or -NoCommand (Alias: -NoRemoteCommand), used with -PC: skip the
610
+ actual remote command execution (dry run).
558
611
 
559
- -NT or -NoTrim, skips trimming of spaces and comments from input values to
560
- speed up reading input file(s).
612
+ -ND or -NewDaemon, required with -SD (-SetDaemon) when the daemon
613
+ control index is 0, so that a new daemon control record is created.
614
+ This guard prevents creating records by accident.
615
+
616
+ -NT or -NoTrim, skip stripping spaces and comments from input values
617
+ when reading input files; this speeds up parsing of large input files.
618
+
619
+ -WR or -WithdsRqst, used with -PC (-ProcessCheck) in non-daemon mode:
620
+ add 'dsrqst' records due to be built or purged to check records before
621
+ processing.
622
+
623
+ -WU or -WithdsUpdt, used with -PC (-ProcessCheck) in non-daemon mode:
624
+ add due 'dsupdt' update controls to check records before processing.
561
625
 
562
- -WR or -WithdsRqst, adds requests due to be built or purged of command 'dsrqst'
563
- for 'dscheck' Action -PC (-ProcessCheck) in non-daemon mode.
564
-
565
- -WU or -WithdsUpdt, adds due update controls of command 'dsupdt' for 'dscheck'
566
- Action -PC (-ProcessCheck) in non-daemon mode.
567
626
 
568
627
  5 INFORMATION OPTIONS
569
628
 
570
- Information options are used to pass information, one or multiple values, into
571
- 'dscheck'. Two types of Info options are used:
572
- Single-Value Info Options - pass a single value to 'dscheck'
573
- Multi-Value Info Options - pass one or multiple values to 'dscheck'
629
+ Info options pass data values into 'dscheck'. They come in two forms:
630
+
631
+ Single-Value Info Options - accept exactly one value.
632
+ Multi-Value Info Options - accept one or more values, comma-separated
633
+ on the command line or row-separated in an
634
+ input file.
574
635
 
575
636
  5.1 Single-Value Info Options
576
637
 
577
- A single-value Info option is used to pass one value into this application.
578
- One value, and one only, must follow a single-value option; otherwise an
579
- error message is displayed if no value or more than one value passed in.
638
+ A single-value Info option accepts exactly one value. Supplying zero or
639
+ multiple values is an error.
640
+
641
+ -DM or -DaemonMode, the daemon mode for -PC (-ProcessCheck). Valid
642
+ values are start, stop, logon, and logoff: start the daemon, stop the
643
+ daemon, enable detailed logging, or disable detailed logging.
580
644
 
581
- -DM or -DaemonMode, passes daemon mode values, start, stop, logon or logoff
582
- to 'dscheck' Action -PC (-ProcessCheck) to start, to stop, to log detail or
583
- not to log detail, correspondingly.
645
+ -DV or -Divider (Alias: -Delimiter, -Separator), the column delimiter
646
+ used by multi-value tabular assignments in input files. Defaults to '<:>'.
584
647
 
585
- -DV or -Divider (Alias: -Delimiter, -Separator), delimiter for separating
586
- columns of multi-value Info options in input files. It is default to '<:>'.
648
+ -ES or -EqualSign, the token that assigns a value to an option in input
649
+ files. Defaults to '<=>'.
587
650
 
588
- -ES or -EqualSign, for an equal sign of assigning one value to either a
589
- single-value option or multi-value option in input files. It is defaulted
590
- to '<=>'.
651
+ -FN or -FieldNames, a string of single-letter field codes selecting
652
+ which fields to retrieve. Used with -GC (-GetCheck) and -GD (-GetDaemon);
653
+ see Sections 3.1.2 and 3.2.2 for the code tables.
591
654
 
592
- -FN or -FieldNames, for a string of single letter field names. Values of
593
- the selected fields are retrieved for check information per actions -GC
594
- (-GetCheck).
655
+ -LH or -LocalHost, the local hostname on which to process checks for
656
+ -PC (-ProcessCheck). Defaults to the current hostname. Pass PBS to
657
+ process batch jobs.
595
658
 
596
- -LH or -LocalHost, specify a local hostname to processes checks on the host for
597
- action -PC(-ProcessCheck). It defaults to '' to use the local host name. Specify
598
- PBS to process batch jobs.
659
+ -MT or -MaxrunTime, the maximum daemon run time. Defaults to 0
660
+ (unlimited). Examples: 5000 means 5000 seconds; 1D means one day
661
+ (86400 seconds).
599
662
 
600
- -MT or -MaxrunTime, specify the maximum run time for daemon mode. It defaults to 0
601
- for unlimit time. For examples, 5000 means seconds, and 1D means 1 day for 86400
602
- seconds.
663
+ -OF or -OutputFile, redirect output to a file instead of the screen.
603
664
 
604
- -OF or -OutputFile, leads an output file name into which the output result
605
- of this application is dumped. If this option is not given, the result is
606
- displayed as standard output.
607
-
608
- -ON or -OrderNames, for a string of single letter field names use to order
609
- the results of getting check information via action -GC (-GetCheck).
610
-
611
- -AO or -ActOption, for setting Action and Mode options in input files. It is
612
- default to '<!>'.
665
+ -ON or -OrderNames, a string of single-letter field codes that
666
+ controls the sort order of -GC (-GetCheck) output.
613
667
 
614
- -WI or -WaitInterval, defaults to 2 minutes (120 seconds). It is used as a time
615
- intervals for the 'dscheck' daemon to sleep between processing check records.
668
+ -AO or -ActOption, the token that marks Action and Mode options in
669
+ input files. Defaults to '<!>'.
670
+
671
+ -WI or -WaitInterval, the time the dscheck daemon sleeps between
672
+ cycles. Defaults to 120 seconds (2 minutes).
616
673
 
617
674
  5.2 Multi-Value Info Options
618
675
 
619
- A multi-value Info option is used to pass multiple values for one Info option
620
- into 'dscheck'. At lease one value must follow each multi-value option.
621
-
622
- -AN or -ActionName (Alias: -Action), specifies an action name for given command
623
- name recorded in check record.
624
-
625
- -AV or -ArgumentVector, the space delimited argument vector string up to 100
626
- characters. It is quoted with single quotes '' for a individual argument
627
- containing spaces.
628
-
629
- -AX or -ArgumentExtra, the space delimited argument vector string beyond 100
630
- characters. This field is not empty only if the argument vector string is
631
- longer than 100 characters.
632
-
633
- -CC or -CarbonCopy, provides additional one or multiple email addresses on
634
- command line to send Cc'd email notification of the check status. For
635
- DECS specialist, login user names themselves are acceptable; otherwise full
636
- email addresses are required for email domains other than 'ucar.edu'.
637
-
638
- -CD or -CheckDate, the check date of the recorded command is first processed.
639
-
640
- -CI or -CheckIndex, check record indices for commands recorded in GDEXDB. A check
641
- record is automatically purged if the command is finished.
642
-
643
- -CM or -Command (Alias: -CommandName), the command name recorded in check record.
644
-
645
- -CT or -CheckTime, the check time of the recorded command is first processed.
646
-
647
- -DB or -Debug, turns on debug mode with specified information. This option
648
- provides up to 3 values, they are Debug Level, debug log file path and debug
649
- log file name. The debug level is mandatory for this option. It can be a
650
- single integer value, for example, 1000 means to log debug messages for debug
651
- levels 1 to 1000; or a range of values, for example, 200-1000 means to log
652
- debug messages from debug levels 200 to 1000. The default debug file path is
653
- '${DSSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'. Provide
654
- the second and third values for this option to override the defaults
655
- respectively.
656
-
657
- -DC or -DoneCount, the number of files that are processed successfully already.
658
-
659
- -DF or -DownFlags, storage system down flags. The current supported flags are:
660
- D-DRDATA, G-GLADE, O-ObjectStore. It can hold multiple flags, up to 5,
661
- for all down storage systems.
662
-
663
- -DI or -DaemonIndex, daemon control indices. A daemon control record retains
664
- configuration information for how many processes on what host and its priority
665
- can a command be started for specified specialist.
666
-
667
- -DS or -Dataset, for dataset numbers, or called dataset IDs in form as [a-z]NNNNNN.
668
-
669
- -ER or -ErrorMessage, error message of a failed check command.
670
-
671
- -EV or -Environments, (Alias: -Envs), specifies environment variables, in form of
672
- VarName=VarValue and separated by ',', needed to be set to execute a command as
673
- a batch job. The environment variables will be set in the batch starting script.
674
-
675
- -FC or -FileCount, the number of files to be processed by a check command.
676
-
677
- -HN or -HostName, specify the host names the check can or cannot be processed on.
678
-
679
- -IF or -InputFile, for input file names; one or multiple file names may be
680
- given on command line. Input files are used to hold all valid options and
681
- the associated values of Info options that need to be passed in for
682
- execution of 'dscheck'.
683
-
684
- In a input file, lines start with sign '#' are considered as comments;
685
- Option Names can be given either short, long or alias names. Action and Mode
686
- options are given in format of OptionName<!>. Single value Assignment is
687
- given in format of OptionName<=>OptionValue. One option is given on each line.
688
- Different setting sign of Action and Mode options can be provided by Info
689
- option -AO (-ActOption, default to <!>); and different equal sign of single
690
- value assignment can be provided by Info option -ES, (-EqualSign, default to
691
- '<=>'). Multi-value assignments can be given in columns delimited with
692
- separator specified per option -DV (-Divider, default to '<:>'). It starts
693
- with a column title line for multi-value option names and the rest holds
694
- values corresponding to each column titles. The value information stops at
695
- the end of the file or when a new column name line or another single value
696
- assignment appears. If the last column is a multi-line value field, an
697
- additional separator must be appended for each line, including the column
698
- title line to end lines properly.
699
-
700
- -MC or -MaxCount, the maximum number of tries a check command can be processed
701
- if the command has failed.
702
-
703
- -MH or -MatchHost (Alias: -MatchHostname), flag to control hostname match.
704
- 'G' - general match, including empty hostname specified, or exclusive hostnames
705
- given but the hostname is not in the list, and 'M' - match only the hostname
706
- specified is identical to the current host.
707
-
708
- -MO or -Modules, (Alias: -Mods), specifies module names, separated by ',',
709
- needed to be loaded to execute a command as a batch job. The modules will
710
- be set in the batch starting script.
711
-
712
- -PI or -ParentIndex, specifies a parent check index for the current check to
713
- wait on.
714
-
715
- -PO or -Priority, specifies the priority of a given host so that the host is
716
- picked in such an order to start a 'dscheck' process.
717
-
718
- -PL or -ProcessLimit, specifies how many processes can be started for specified
719
- command and specialist on a given host; Work for Action -SO (-SetOptions) to limit
720
- how many concurrent cron processes.
721
-
722
- -PQ or -PBSQueue, specifies PBS batch queue index the current check to submit to.
723
-
724
- -QS or -QsubOptions, (Alias: -PBSOptions), specifies options to execute a command
725
- as a batch job via qsub on PBS nodes. The qsub options must be quoted when presented
726
- on command line, such as, -QS '-l walltime=12:00:00'.
727
-
728
- -SN or -Specialist, the specialist who runs the original command. This Info
729
- option is working with Action -GC (-GetCheck) only to view check information
730
- of a specified specialist.
731
-
732
- -ST or -Status, check command status for a check record. Include command
733
- progress percentage if under process and error message for a failed command.
734
-
735
- -SZ or -DataSize, the total bytes of data processed for the check record.
736
-
737
- -TC or -TryCount, the number of tries for a check command being processed. It
738
- is up to the number of tries specified via info option -MC.
739
-
740
- -WD or -WorkDir (Alias: -WorkDirectory), the working directory where the
741
- the recorded command is started.
676
+ A multi-value Info option accepts a list of one or more values. Supplying
677
+ zero values is an error.
678
+
679
+ -AN or -ActionName (Alias: -Action), the action name associated with
680
+ a command in a check record.
681
+
682
+ -AV or -ArgumentVector, the space-delimited argument string that
683
+ follows the command, up to 100 characters. Quote individual arguments
684
+ containing spaces with single quotes.
685
+
686
+ -AX or -ArgumentExtra, the space-delimited argument string beyond 100
687
+ characters. Non-empty only when the argument vector exceeds 100
688
+ characters.
689
+
690
+ -CC or -CarbonCopy, additional email addresses for Cc'd notification
691
+ of check status. DECS specialists may be specified by login name; for
692
+ other domains, use full email addresses.
693
+
694
+ -CD or -CheckDate, the date a recorded command was first processed.
695
+
696
+ -CI or -CheckIndex, the indices of check records in GDEXDB. A check
697
+ record is moved to history once its command finishes.
698
+
699
+ -CM or -Command (Alias: -CommandName), the command name in a check
700
+ record.
701
+
702
+ -CT or -CheckTime, the time a recorded command was first processed.
703
+
704
+ -DB or -Debug, enable debug logging. Up to three values may be given:
705
+ the debug level (required), the log file path, and the log file name.
706
+ The level is either a single integer (e.g. 1000 means levels 1 to 1000)
707
+ or a range (e.g. 200-1000). Defaults: log path '${DSSHOME}/dssdb/log',
708
+ file name 'mydss.dbg'.
709
+
710
+ -DC or -DoneCount, the number of files successfully processed so far.
711
+
712
+ -DF or -DownFlags, storage system down flags. Valid flags: D-DRDATA,
713
+ G-GLADE, O-ObjectStore. Up to five flags may be combined.
714
+
715
+ -DI or -DaemonIndex, daemon control indices. A daemon control record
716
+ defines how many concurrent processes of a given command a specialist
717
+ may run on a given host, and at what priority.
718
+
719
+ -DS or -Dataset, dataset numbers (also called dataset IDs). Format:
720
+ [a-z]NNNNNN.
721
+
722
+ -ER or -ErrorMessage, the error message from a failed check command.
723
+
724
+ -EV or -Environments (Alias: -Envs), environment variables required
725
+ to run a command as a batch job, in the form VarName=VarValue, separated
726
+ by commas. The variables are set in the batch startup script.
727
+
728
+ -FC or -FileCount, the number of files to be processed by a check
729
+ command.
730
+
731
+ -HN or -HostName, host names on which the check can or cannot be
732
+ processed.
733
+
734
+ -IF or -InputFile, one or more input file names supplied on the command
735
+ line. An input file holds options and their values for a dscheck
736
+ invocation.
737
+
738
+ Input file format:
739
+ - Lines starting with '#' are comments and are ignored.
740
+ - Option names may appear in short, long, or alias form.
741
+ - Action/Mode markers: OptionName<!>
742
+ (marker overridable via -AO)
743
+ - Single-value assignments: OptionName<=>OptionValue
744
+ (one per line; '<=>' overridable via -ES)
745
+ - Multi-value tabular assignments:
746
+ a title row followed by data rows, columns separated by '<:>'
747
+ ('<:>' overridable via -DV).
748
+ - A tabular block ends at EOF, at the next title line, or at the next
749
+ single-value assignment.
750
+ - If the last column of a tabular block contains multi-line text,
751
+ append the column separator to every row, including the title row.
752
+
753
+ -MC or -MaxCount, the maximum number of times a failed check command
754
+ may be retried.
755
+
756
+ -MH or -MatchHost (Alias: -MatchHostname), controls how hostnames are
757
+ matched. 'G' is a general match - it includes empty hostnames and
758
+ exclusive lists where the current host is not listed. 'M' is an exact
759
+ match against the specified hostname.
760
+
761
+ -MO or -Modules (Alias: -Mods), comma-separated module names required
762
+ to run a command as a batch job. The modules are loaded in the batch
763
+ startup script.
764
+
765
+ -PI or -ParentIndex, the parent check index that the current check
766
+ must wait for before it can run.
767
+
768
+ -PO or -Priority, the priority of a host within a daemon control,
769
+ determining the order in which hosts are tried when starting a check.
770
+
771
+ -PL or -ProcessLimit, the maximum number of concurrent processes
772
+ allowed for a given (command, specialist) on a given host. Also used
773
+ with -SO (-SetOptions) to bound concurrent cron processes.
774
+
775
+ -PQ or -PBSQueue, the PBS batch queue to which the current check is
776
+ submitted.
777
+
778
+ -QS or -QsubOptions (Alias: -PBSOptions), options passed to qsub when
779
+ the command runs as a PBS batch job. Quote the value on the command
780
+ line, e.g. -QS '-l walltime=12:00:00'.
781
+
782
+ -SN or -Specialist, the specialist who originally ran the command.
783
+ Used with -GC (-GetCheck) and -GD (-GetDaemon) to view records owned
784
+ by a particular specialist.
785
+
786
+ -ST or -Status, the check status for a recorded command, including
787
+ progress percentage for running commands and error messages for failed
788
+ ones.
789
+
790
+ -SZ or -DataSize, the total number of bytes processed for the check
791
+ record.
792
+
793
+ -TC or -TryCount, the number of times the check command has been
794
+ attempted, up to the limit set by -MC.
795
+
796
+ -WD or -WorkDir (Alias: -WorkDirectory), the working directory from
797
+ which the recorded command is started.