rda-python-dscheck 2.0.3__tar.gz → 2.0.5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (20) hide show
  1. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/PKG-INFO +6 -2
  2. rda_python_dscheck-2.0.5/README.md +6 -0
  3. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/pyproject.toml +1 -1
  4. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/dscheck.usg +92 -88
  5. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/pg_check.py +1 -1
  6. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/PKG-INFO +6 -2
  7. rda_python_dscheck-2.0.3/README.md +0 -2
  8. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/LICENSE +0 -0
  9. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/MANIFEST.in +0 -0
  10. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/setup.cfg +0 -0
  11. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/PgCheck.py +0 -0
  12. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/__init__.py +0 -0
  13. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/ds_check.py +0 -0
  14. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/dscheck.py +0 -0
  15. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/SOURCES.txt +0 -0
  16. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/dependency_links.txt +0 -0
  17. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/entry_points.txt +0 -0
  18. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/requires.txt +0 -0
  19. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/top_level.txt +0 -0
  20. {rda_python_dscheck-2.0.3 → rda_python_dscheck-2.0.5}/tests/test_dscheck.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rda_python_dscheck
3
- Version: 2.0.3
3
+ Version: 2.0.5
4
4
  Summary: RDA python package to add and process batch jobs
5
5
  Author-email: Zaihua Ji <zji@ucar.edu>
6
6
  Project-URL: Homepage, https://github.com/NCAR/rda-python-dscheck
@@ -14,5 +14,9 @@ License-File: LICENSE
14
14
  Requires-Dist: rda_python_common
15
15
  Dynamic: license-file
16
16
 
17
- RDA python package to add and process batch jobs.
17
+ # dscheck
18
+
19
+ Python project to add and process batch jobs for the [NSF NCAR Geoscience Data Exchange (GDEX)](https://gdex.ucar.edu).
20
+
21
+ The user guide for this utility tool can be viewed at: [User guide](https://gdex-docs-dscheck.readthedocs.io).
18
22
 
@@ -0,0 +1,6 @@
1
+ # dscheck
2
+
3
+ Python project to add and process batch jobs for the [NSF NCAR Geoscience Data Exchange (GDEX)](https://gdex.ucar.edu).
4
+
5
+ The user guide for this utility tool can be viewed at: [User guide](https://gdex-docs-dscheck.readthedocs.io).
6
+
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "rda_python_dscheck"
7
- version = "2.0.3"
7
+ version = "2.0.5"
8
8
  authors = [
9
9
  { name="Zaihua Ji", email="zji@ucar.edu" },
10
10
  ]
@@ -2,28 +2,28 @@
2
2
  1 INTRODUCTION
3
3
 
4
4
  Program 'dscheck' is a utility to add, remove, view and process recorded commands
5
- of other utility programs in Research Data Archive Management System (RDAMS). For
5
+ of other utility programs in Research Data Archive Management System (GDEXMS). For
6
6
  delayed execution, or called batch process, of utility programs, 'dsarch', 'dsupdt'
7
7
  and 'dsrqst', the commands information and the directories where the commands are
8
- initiated are saved into RDADB as check records. For any other specialist-defined
8
+ initiated are saved into GDEXDB as check records. For any other specialist-defined
9
9
  commands, they can be also put in delayed mode if the commands are added to
10
10
  'dscheck' control via Action -AC (-AddCheck).
11
11
 
12
12
  The check records are processed automatically via a centralized 'dscheck' daemon,
13
13
  although they can be processed manually on command line. During the execution of a
14
- recorded command, the check record is locked in RDADB to prevent multiple executions
14
+ recorded command, the check record is locked in GDEXDB to prevent multiple executions
15
15
  of the same command. The check records that retain command information are
16
16
  automatically purged into check history when the commands are finished.
17
17
 
18
18
  When a recorded command fails due to failures of computer system, storage disk/tape,
19
19
  or the Database Management System, the check record is normally purged with status 'E'
20
20
  for error, unless check-reprocessing ability is built into the utility program, such
21
- as 'dsarch'. For a check-reprocessing command, the check record is retained in RDADB
22
- Until the command is processed successfully or the number of executions reaches the
21
+ as 'dsarch'. For a check-reprocessing command, the check record is retained in GDEXDB
22
+ until the command is processed successfully or the number of executions reaches the
23
23
  try limits allowed. Utility programs 'dsrqst' and 'dsupdt' carry their own failure-
24
24
  recovering ability and they do not need check-reprocessing.
25
25
 
26
- The purged check records are retained in RDADB. The check history can be viewed via
26
+ The purged check records are retained in GDEXDB. The check history can be viewed via
27
27
  utility program 'viewcheckusage'.
28
28
 
29
29
  Program 'dscheck' supports the following major functions:
@@ -33,21 +33,21 @@ Program 'dscheck' supports the following major functions:
33
33
  a check. Without the daemon control information, a recorded command will not be
34
34
  started automatically.
35
35
  - ADD a check record for delayed command execution of any specialist-defined commands
36
- - View utility command information currently saved in RDADB
36
+ - View utility command information currently saved in GDEXDB
37
37
  - Email the current status of a specified utility command or a list of commands,
38
38
  and include error messages if any, to a specialist
39
39
  - Delete recorded command information, because the command is not needed anymore
40
40
  - Unlock a given recorded command in case lock information was not cleaned
41
- properly when the command was failed
41
+ properly when the command has failed
42
42
  - Interrupt a utility command that is under execution and kill recursively all
43
43
  the associated child processes
44
44
  - Add the due 'dsrqst' and 'dsupdt' actions into dscheck records
45
- - Process commands that have been recorded into RDADB or the ones have failed
46
- previously.
45
+ - Process commands that have been recorded into GDEXDB or the ones that have
46
+ failed previously.
47
47
 
48
48
  The specialist who executes a utility command under 'dscheck' control remains
49
- the exclusive owner of the check record in RDADB. This prevents the command to be
50
- processed or deleted accidentally by other specialists.
49
+ the exclusive owner of the check record in GDEXDB. This prevents the command from being
50
+ processed or deleted accidentally by other specialists.
51
51
 
52
52
  In the following sections, general usages of 'dscheck' are described first; and
53
53
  detail descriptions of Action options are given; and finally Mode and Info
@@ -62,7 +62,7 @@ options are explained.
62
62
  Quotes [] indicate optional. A pipeline '|' in parentheses as in format (A|B)
63
63
  means either A or B can be used. The options applied to 'dscheck' are divided
64
64
  into three categories, Action, Mode, and Information (Info for short) options.
65
- Action options are used to specify what tasks this utility program to execute,
65
+ Action options are used to specify what task this utility program is to execute,
66
66
  Mode options are used to modify behaviors of the actions, and the Info options
67
67
  are used to pass information, one or multiple values, to run 'dscheck'. An option
68
68
  can be given in form of either short name or long name, -DS or -Dataset for
@@ -77,18 +77,18 @@ and certain Mode options can be applied to alter the behaviors of the Action.
77
77
 
78
78
  All options, except Info option -IF (-InputFile), can be given either on command
79
79
  line or in input files. Input file names are presented per Info option -IF and
80
- can only be provided on command line. Referring to description of Info option -IF
81
- (-InputFile) for detail on how to present options in input files. One or multiple
80
+ can only be provided on command line. Refer to the description of Info option -IF
81
+ (-InputFile) for details on how to present options in input files. One or multiple
82
82
  input files, combined with options on command line, are allowed to run 'dscheck'.
83
83
  The option name, -IF (-InputFile), itself can be omitted if a single input file is
84
84
  given on command line and all other option information are provided inside the
85
85
  input file.
86
86
 
87
87
  When information of daemons and checks are retrieved, Info options are used to
88
- specify conditions for querying information from RDADB. Some special signs can be
88
+ specify conditions for querying information from GDEXDB. Some special signs can be
89
89
  used to further confine the information with special and complicated conditions;
90
90
  they are '!', '<', '>' and '<>'. These special signs, if provided on command
91
- line, must be quoted or escaped to avoid of being interpreted by Unix OS system.
91
+ line, must be quoted or escaped to avoid being interpreted by the Unix shell.
92
92
  The '!', or \!, means exclusion to the following value(s) and it must be the
93
93
  first item following an Info option name, while '<' or '>' mean greater or less
94
94
  than the following value and '<>' means between the following two values.
@@ -102,11 +102,11 @@ command line as
102
102
 
103
103
  A description is displayed for an option given either before or after -(h|help).
104
104
  If no option is specified or 'dscheck' is issued by itself, this whole document
105
- is displayed per UNIX utility 'more'. A hard copy of this help document can be
106
- printed from the saved file: ${DSSHOME}/dssdb/prog_usage/dscheck.usg.
105
+ is displayed via UNIX utility 'more'. A hard copy of this help document can be
106
+ printed from the saved file, dscheck.usg, under python package rda_python_dscheck.
107
107
 
108
108
  #The online HTML version of this document is available at
109
- #http://dss.ucar.edu/internal/docs/dscheck/
109
+ #https://gdex-docs-dscheck.readthedocs.io
110
110
 
111
111
 
112
112
  3 ACTION OPTIONS
@@ -117,27 +117,27 @@ simultaneously are blocked.
117
117
 
118
118
  Based on the information being manipulated, the actions are divided into three
119
119
  categories:
120
- Daemon Control Actions - create, delete, modify and view daemon control information
121
- in RDADB, of specified specialist, command and hostname
122
- Check Actions - add, delete, unlock and view check information of the
123
- active individual checks
124
- Check Process Actions - process checks by starting commands on remote hosts as
125
- configured in daemon controls and purge checks by recording
126
- the commands and their execution information into check
127
- history; interrupt the current executing commands by killing
128
- the current process and its all children; and email status
129
- of current checks
120
+ Daemon Control Actions - create, delete, modify and view daemon control information
121
+ in GDEXDB, of specified specialist, command and hostname
122
+ Check Actions - add, delete, unlock and view check information of the
123
+ active individual checks
124
+ Check Process Actions - process checks by starting commands on remote hosts as
125
+ configured in daemon controls and purge checks by recording
126
+ the commands and their execution information into check
127
+ history; interrupt the current executing commands by killing
128
+ the current process and its all children; and email status
129
+ of current checks
130
130
  Daemon Host Connectivity - check connectivity of daemon hosts for specialists
131
131
 
132
132
  3.1 Daemon Control Actions
133
133
 
134
134
  A daemon control record for a command, a specialist and a hostname is used to
135
135
  configure how many concurrent processes the specified command can be executed
136
- for the specialist on specified hostname, and the priority the the hostname is
136
+ for the specialist on specified hostname, and the priority the hostname is
137
137
  picked to start the command. A running centralized daemon reads this record
138
138
  periodically in case the configuration is changed while the daemon is still
139
139
  running, so that specialists can reset the values in daemon control records to
140
- change the behave of dscheck daemon dynamically without shutting the daemon down.
140
+ change the behavior of dscheck daemon dynamically without shutting the daemon down.
141
141
 
142
142
  Daemon control information can be created, modified and viewed via Actions
143
143
  included in this section:
@@ -148,7 +148,7 @@ categories:
148
148
 
149
149
  3.1.1 Set Daemon Control
150
150
  -SD or -SetDaemon, creates and modifies daemon control information into
151
- RDADB for given specialist login names, commands, and hostnames of computers on
151
+ GDEXDB for given specialist login names, commands, and hostnames of computers on
152
152
  which the check commands are processed. One or multiple records can be set each
153
153
  time.
154
154
 
@@ -162,13 +162,13 @@ categories:
162
162
  [-(PO|Priority) HostListOrder]
163
163
 
164
164
  Mode option that can be specified for this action include:
165
- -(ND|NewDaemon) - sets a new daemon control record into RDADB
165
+ -(ND|NewDaemon) - sets a new daemon control record into GDEXDB
166
166
 
167
- If information of a daemon control exists already in RDADB for a given specialist,
167
+ If information of a daemon control exists already in GDEXDB for a given specialist,
168
168
  a command and a hostname, the daemon control record is modified; otherwise, a new
169
169
  daemon control record is added if daemon index is 0 and Mode option -ND (-NewDaemon)
170
170
  is present. Combination of specialist login name, command name and hostname of
171
- computer must be unique for for each daemon control record.
171
+ computer must be unique for each daemon control record.
172
172
 
173
173
  Specify host name 'PBS' for putting the command in the PBS batch control system. If
174
174
  a specified command name is not found in the daemon control, the general 'dscheck'
@@ -176,12 +176,12 @@ categories:
176
176
 
177
177
  For example, set daemon control information for schuster, all commands on PBS hosts,
178
178
  for maximum 4 checks can be processed at the same time with priority 1, the smaller
179
- the number the higher the priority is, via input file daemon.ctl
179
+ the number the higher the priority is, via input file daemon.ctl:
180
180
 
181
- dsrqst SD -ND -IF daemon.ctl
181
+ dscheck -SD -ND -IF daemon.ctl
182
182
 
183
183
  <<Content of input file daemon.ctl>>
184
- DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
184
+ DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
185
185
  0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
186
186
 
187
187
 
@@ -217,11 +217,11 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
217
217
  I -(DI|DaemonIndex) daemon control index
218
218
  C -(CM|Command) command names of utility programs
219
219
  H -(HN|Hostname) computer hostname
220
- M -(MH|MatchHost) Flage to control hostname match
220
+ M -(MH|MatchHost) Flags to control hostname match
221
221
  S -(SN|Specialist) DECS specialist the daemon set for
222
222
  P -(PL|ProcessLimit) Max check count to be processed at the same times
223
223
  O -(PO|Priority) host priority a specified command to start on
224
-
224
+
225
225
  Daemon control information can be retrieved for specified specialist per option
226
226
  -SN (-Specialist), and/or other conditions. Info option -SN, -CM and -HN accept
227
227
  wildcard input of '%' for matching any number of characters.
@@ -232,14 +232,14 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
232
232
  option -SN (-Specialist). To view all control records, you simply provide option
233
233
  -SN with value of 'ALL'.
234
234
 
235
- For example, to get all daemon control information currently set for you
235
+ For example, to get all daemon control information currently set for you:
236
236
 
237
237
  dscheck GD
238
238
 
239
239
 
240
240
  3.1.3 Delete Daemon Control
241
241
  -DL or -Delete (Alias: -RM, -Remove), deletes one or multiple daemon control records
242
- from RDADB for given daemon control indices.
242
+ from GDEXDB for given daemon control indices.
243
243
 
244
244
  dscheck -(DL|Delete)
245
245
  -(DI|DaemonIndex) DaemonControlIndices
@@ -289,7 +289,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
289
289
  as the owner of the added check record, and the current path is defaulted as the
290
290
  working directory when the command is executed later.
291
291
 
292
- Specified addtional PBS batch options via Info option -QS (-QSubOptions) to add a check;
292
+ Specify additional PBS batch options via Info option -QS (-QsubOptions) to add a check;
293
293
  and specify a parent check index to put the current command on hold until the parent
294
294
  check is finished.
295
295
 
@@ -297,29 +297,33 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
297
297
  A simple shell script can be used to wrap a complicated command.
298
298
 
299
299
  For example, to list file names in the current directory with name containing 'test'
300
- by catching the standard output into a log and display it on screen
300
+ by catching the standard output into a log and display it on screen:
301
+
302
+ dscheck -AC -CM test1.sh -HN PBS
301
303
 
302
304
  <<Content of shell script test1.sh>>
303
305
  #!/bin/sh
304
306
  (ls -l | grep test) | tee test1.out
305
307
 
306
308
  For example, to list file names in the current directory with name containing 'test'
307
- by catching the standard output and error into separate log files
308
-
309
+ by catching the standard output and error into separate log files:
310
+
311
+ dscheck -AC -CM test2.sh -HN PBS
312
+
309
313
  <<Content of shell script test2.sh>>
310
314
  #!/bin/sh
311
315
  (ls -l | grep test) 1> test2.out 2>test2.err
312
316
 
313
317
  For example, to add testing command 'test2' into 'dscheck' for delayed mode execution on
314
- PBS
318
+ PBS:
315
319
 
316
- dsheck AC -CM test2 -HN PBS
320
+ dscheck -AC -CM test3 -HN PBS
317
321
 
318
- The command 'test2' must be executable at the current working directory on PBS machines.
322
+ The command 'test3' must be executable at the current working directory on PBS machines.
319
323
 
320
324
 
321
325
  3.2.2 Get Check
322
- -GC or -GetCheck, gets check information recorded in RDADB.
326
+ -GC or -GetCheck, gets check information recorded in GDEXDB.
323
327
 
324
328
  dscheck -(GC|GetCheck) [Mode Options]
325
329
  [-(FN|FieldNames) FieldNameString]
@@ -343,7 +347,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
343
347
  of a given field
344
348
 
345
349
  Use Info option -FN (-FieldNames) to specify what check fields to retrieve.
346
- It defaults to "COVTUFJDNW" if -FN is not given.
350
+ It defaults to "COVTUPFJDNW" if -FN is not given.
347
351
 
348
352
  Valid check field names and their corresponding Info options:
349
353
 
@@ -354,21 +358,21 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
354
358
  T -(DS|Dataset) dataset ID, the original command run against
355
359
  A -(AN|ActionName) action name for a given command
356
360
  U -(ST|Status) check status for a recorded command
357
- B -(DF|DownFlags) Storage system down flags: H-HPSS,D-DRDATA,G-GLADE,O-ObjectStore
358
- P -(PQ|PBSQueue) PBS batch queue name: rda or htc
361
+ B -(DF|DownFlags) Storage system down flags: D-DRDATA,G-GLADE,O-ObjectStore
362
+ P -(PQ|PBSQueue) PBS batch queue name: gdex or htc
359
363
  R -(PI|ParentIndex) parent check index the current one to wait on
360
364
  F -(FC|FileCount) number of files need to be processed
361
365
  J -(DC|DoneCount) number of files are processed already
362
366
  K -(TC|TryCount) number of tries the command is executed
363
367
  L -(MC|MaxCount) upper limit for number of command tries
364
- Z -(SZ|DataSize) total tytes of data processed for the command
368
+ Z -(SZ|DataSize) total bytes of data processed for the command
365
369
  D -(CD|CheckDate) date the command is initially recorded
366
370
  Y -(CT|CheckTime) time the command is initially recorded
367
371
  H -(HN|HostName) host names the command can or cannot run on
368
372
  N -(SN|Specialist) specialist login name who starts the command
369
373
  W -(WD|WorkDir) working directory where the command is started
370
374
  M -(MO|Modules) include modules to load to batch job script
371
- I -(EV|Environments) include environment vairables to load to batch job script
375
+ I -(EV|Environments) include environment variables to load to batch job script
372
376
  Q -(QS|QsubOptions) additional PBS batch options for qsub
373
377
  X -(AX|ArgumentExtra) additional argument line beyond 100 characters
374
378
  E -(ER|ErrorMessage) error message from failed command
@@ -379,7 +383,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
379
383
 
380
384
 
381
385
  3.2.3 Delete Check
382
- -DL or -Delete, deletes one or multiple 'dscheck' records from RDADB
386
+ -DL or -Delete, deletes one or multiple 'dscheck' records from GDEXDB
383
387
  for given check indices.
384
388
 
385
389
  dscheck -(DL|Delete)
@@ -391,23 +395,23 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
391
395
 
392
396
 
393
397
  3.2.4 Unlock Check
394
- -UL or -UnLockCheck, (Alias: -UnLock), unlocks check records that their
395
- commands aborted abnormally during processes.
398
+ -UL or -UnLockCheck, (Alias: -UnLock), unlocks check records whose
399
+ commands aborted abnormally during processing.
396
400
 
397
401
  Process ID and computer hostname are saved in a check record when the recorded
398
402
  command is running. If the process aborts abnormally, the PID and hostname may
399
403
  sometimes be not cleaned properly. Use this action to clean up the locking
400
404
  information to allow the command to be reprocessed or purged.
401
405
 
402
- dscheck -(-UL|UnLockCheck)
403
- -(CI|CheckIndex) CheckIndices
404
- [-(DB|Debug) DebugModeInfo]
406
+ dscheck -(UL|UnLockCheck)
407
+ -(CI|CheckIndex) CheckIndices
408
+ [-(DB|Debug) DebugModeInfo]
405
409
 
406
410
  It is mandatory to provide a check index to remove lock on a recorded command.
407
411
 
408
412
 
409
413
  3.3 Check Process Actions
410
- Delayed mode command executions recorded in RDADB are automatically started,
414
+ Delayed mode command executions recorded in GDEXDB are automatically started,
411
415
  or restarted, by the common 'dscheck' daemon. Running commands can be interrupted
412
416
  at any time and the child processed of the interrupted commands are also cleaned up.
413
417
  The status of the current check records can be gathered and emailed to a specialist.
@@ -454,13 +458,13 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
454
458
  according to the configured information in daemon controls.
455
459
 
456
460
  3.3.2 Interrupt Check
457
- -IC or -InterruptCheck, interrupts checks that their commands are currently running
458
- and also kills recursively all the child processes that are running under the
461
+ -IC or -InterruptCheck, interrupts checks whose commands are currently running
462
+ and also kills recursively all the child processes running under those
459
463
  commands. Locks are cleaned too when the checks are interrupted.
460
464
 
461
- dscheck -(-IC|InterruptCheck) [Mode Option]
462
- -(CI|CheckIndex) CheckIndices
463
- [-(DB|Debug) DebugModeInfo]
465
+ dscheck -(IC|InterruptCheck) [Mode Option]
466
+ -(CI|CheckIndex) CheckIndices
467
+ [-(DB|Debug) DebugModeInfo]
464
468
 
465
469
  Mode option that can be specified for this action:
466
470
  -(FI|ForceInterrupt) - it must be present for this action to interrupt a check
@@ -518,13 +522,13 @@ Use proper Mode options to modify behaviors of Action options. Mode options
518
522
  are all optional. No value is allowed to be passed in following any Mode option.
519
523
 
520
524
  -AW or -AnyWhere, for Action -AC (-AddCheck), sets Working directory empty in
521
- cehck record to start processing the check anywhere.
525
+ check record to start processing the check anywhere.
522
526
 
523
- -BG or -BackGround (Alias: -b), background process. When it presents
524
- screen display is turned off for both standard outputs and errors. -(CP|CheckPending) - Check and kill long pending checks
525
-
526
- -CP or -CheckPending, if present for Action -PC (-ProcessCheck), check and kill
527
- long pending checks
527
+ -BG or -BackGround (Alias: -b), background process. When it is present,
528
+ screen display is turned off for both standard output and errors.
529
+
530
+ -CP or -CheckPending, if present for Action -PC (-ProcessCheck), checks and kills
531
+ long pending checks.
528
532
 
529
533
  -CS or -CheckStatus, if present, displays detailed status information of the
530
534
  recorded commands, including progress percentage for a running command and
@@ -545,7 +549,7 @@ are all optional. No value is allowed to be passed in following any Mode option.
545
549
  a given dataset listed for another specialist.
546
550
 
547
551
  -NC or -NoCommand (Alias: -NoRemoteCommand), does not issue remote commands if
548
- this Mode option is present for action -PC (-Processcheck).
552
+ this Mode option is present for action -PC (-ProcessCheck).
549
553
 
550
554
  -ND or -NewDaemon, a new daemon control can only be added when this Mode option is
551
555
  present and daemon control index is given as 0 when Action -SD (-SetDaemon) of
@@ -558,7 +562,7 @@ are all optional. No value is allowed to be passed in following any Mode option.
558
562
  -WR or -WithdsRqst, adds requests due to be built or purged of command 'dsrqst'
559
563
  for 'dscheck' Action -PC (-ProcessCheck) in non-daemon mode.
560
564
 
561
- -WU or -WithdsUpdt, adds due update controls of command 'dsrqst' for 'dscheck'
565
+ -WU or -WithdsUpdt, adds due update controls of command 'dsupdt' for 'dscheck'
562
566
  Action -PC (-ProcessCheck) in non-daemon mode.
563
567
 
564
568
  5 INFORMATION OPTIONS
@@ -593,7 +597,7 @@ Information options are used to pass information, one or multiple values, into
593
597
  action -PC(-ProcessCheck). It defaults to '' to use the local host name. Specify
594
598
  PBS to process batch jobs.
595
599
 
596
- -MT or -MaxrunTime, specify the maxmum run time for deamon mode. It defaults to 0
600
+ -MT or -MaxrunTime, specify the maximum run time for daemon mode. It defaults to 0
597
601
  for unlimit time. For examples, 5000 means seconds, and 1D means 1 day for 86400
598
602
  seconds.
599
603
 
@@ -633,7 +637,7 @@ Information options are used to pass information, one or multiple values, into
633
637
 
634
638
  -CD or -CheckDate, the check date of the recorded command is first processed.
635
639
 
636
- -CI or -CechkIndex, check record indices for commands recorded in RDADB. A check
640
+ -CI or -CheckIndex, check record indices for commands recorded in GDEXDB. A check
637
641
  record is automatically purged if the command is finished.
638
642
 
639
643
  -CM or -Command (Alias: -CommandName), the command name recorded in check record.
@@ -646,14 +650,14 @@ Information options are used to pass information, one or multiple values, into
646
650
  single integer value, for example, 1000 means to log debug messages for debug
647
651
  levels 1 to 1000; or a range of values, for example, 200-1000 means to log
648
652
  debug messages from debug levels 200 to 1000. The default debug file path is
649
- '$DSSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'. Provides
650
- the second and third values for this option to override the default ones
653
+ '${DSSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'. Provide
654
+ the second and third values for this option to override the defaults
651
655
  respectively.
652
656
 
653
657
  -DC or -DoneCount, the number of files that are processed successfully already.
654
658
 
655
659
  -DF or -DownFlags, storage system down flags. The current supported flags are:
656
- H-HPSS, D-DRDATA, G-GLADE, O-ObjectStore. It can hold multiple flags, up to 5,
660
+ D-DRDATA, G-GLADE, O-ObjectStore. It can hold multiple flags, up to 5,
657
661
  for all down storage systems.
658
662
 
659
663
  -DI or -DaemonIndex, daemon control indices. A daemon control record retains
@@ -666,7 +670,7 @@ Information options are used to pass information, one or multiple values, into
666
670
 
667
671
  -EV or -Environments, (Alias: -Envs), specifies environment variables, in form of
668
672
  VarName=VarValue and separated by ',', needed to be set to execute a command as
669
- a batch job. The environment varaibles will be set in the batch starting script.
673
+ a batch job. The environment variables will be set in the batch starting script.
670
674
 
671
675
  -FC or -FileCount, the number of files to be processed by a check command.
672
676
 
@@ -685,7 +689,7 @@ Information options are used to pass information, one or multiple values, into
685
689
  option -AO (-ActOption, default to <!>); and different equal sign of single
686
690
  value assignment can be provided by Info option -ES, (-EqualSign, default to
687
691
  '<=>'). Multi-value assignments can be given in columns delimited with
688
- separator specified per option -SP (-Separator, default to '<:>'). It starts
692
+ separator specified per option -DV (-Divider, default to '<:>'). It starts
689
693
  with a column title line for multi-value option names and the rest holds
690
694
  values corresponding to each column titles. The value information stops at
691
695
  the end of the file or when a new column name line or another single value
@@ -693,11 +697,11 @@ Information options are used to pass information, one or multiple values, into
693
697
  additional separator must be appended for each line, including the column
694
698
  title line to end lines properly.
695
699
 
696
- -MC or -MaxCount, the maximum number of tries for a check command can be processed
697
- if the command is failed.
700
+ -MC or -MaxCount, the maximum number of tries a check command can be processed
701
+ if the command has failed.
698
702
 
699
703
  -MH or -MatchHost (Alias: -MatchHostname), flag to control hostname match.
700
- 'G' - general match, including emty hostname specified, or exclusive hostnames
704
+ 'G' - general match, including empty hostname specified, or exclusive hostnames
701
705
  given but the hostname is not in the list, and 'M' - match only the hostname
702
706
  specified is identical to the current host.
703
707
 
@@ -718,7 +722,7 @@ Information options are used to pass information, one or multiple values, into
718
722
  -PQ or -PBSQueue, specifies PBS batch queue index the current check to submit to.
719
723
 
720
724
  -QS or -QsubOptions, (Alias: -PBSOptions), specifies options to execute a command
721
- as a batch job via qsub on PBS nodes. The qsub options must be quoted when prsented
725
+ as a batch job via qsub on PBS nodes. The qsub options must be quoted when presented
722
726
  on command line, such as, -QS '-l walltime=12:00:00'.
723
727
 
724
728
  -SN or -Specialist, the specialist who runs the original command. This Info
@@ -728,7 +732,7 @@ Information options are used to pass information, one or multiple values, into
728
732
  -ST or -Status, check command status for a check record. Include command
729
733
  progress percentage if under process and error message for a failed command.
730
734
 
731
- -SZ or -DataSize, the total bytes of data is processed for the check record.
735
+ -SZ or -DataSize, the total bytes of data processed for the check record.
732
736
 
733
737
  -TC or -TryCount, the number of tries for a check command being processed. It
734
738
  is up to the number of tries specified via info option -MC.
@@ -48,7 +48,7 @@ class PgCheck(PgOPT, PgCMD):
48
48
  'FI': [0, 'ForceInterrrupt', 0],
49
49
  'FO': [0, 'FormatOutput', 0],
50
50
  'LO': [0, 'LogOn', 0],
51
- 'MD': [0, 'PgDataset', 3],
51
+ 'MD': [0, 'MyDataset', 3],
52
52
  'NC': [0, 'NoCommand', 0],
53
53
  'ND': [0, 'NewDaemon', 0],
54
54
  'NT': [0, 'NoTrim', 0],
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rda_python_dscheck
3
- Version: 2.0.3
3
+ Version: 2.0.5
4
4
  Summary: RDA python package to add and process batch jobs
5
5
  Author-email: Zaihua Ji <zji@ucar.edu>
6
6
  Project-URL: Homepage, https://github.com/NCAR/rda-python-dscheck
@@ -14,5 +14,9 @@ License-File: LICENSE
14
14
  Requires-Dist: rda_python_common
15
15
  Dynamic: license-file
16
16
 
17
- RDA python package to add and process batch jobs.
17
+ # dscheck
18
+
19
+ Python project to add and process batch jobs for the [NSF NCAR Geoscience Data Exchange (GDEX)](https://gdex.ucar.edu).
20
+
21
+ The user guide for this utility tool can be viewed at: [User guide](https://gdex-docs-dscheck.readthedocs.io).
18
22
 
@@ -1,2 +0,0 @@
1
- RDA python package to add and process batch jobs.
2
-