rda-python-dscheck 2.0.4__tar.gz → 2.0.5__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/PKG-INFO +1 -1
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/pyproject.toml +1 -1
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/dscheck.usg +44 -44
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/PKG-INFO +1 -1
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/LICENSE +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/MANIFEST.in +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/README.md +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/setup.cfg +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/PgCheck.py +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/__init__.py +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/ds_check.py +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/dscheck.py +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck/pg_check.py +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/SOURCES.txt +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/dependency_links.txt +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/entry_points.txt +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/requires.txt +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/top_level.txt +0 -0
- {rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/tests/test_dscheck.py +0 -0
|
@@ -19,7 +19,7 @@ When a recorded command fails due to failures of computer system, storage disk/t
|
|
|
19
19
|
or the Database Management System, the check record is normally purged with status 'E'
|
|
20
20
|
for error, unless check-reprocessing ability is built into the utility program, such
|
|
21
21
|
as 'dsarch'. For a check-reprocessing command, the check record is retained in GDEXDB
|
|
22
|
-
|
|
22
|
+
until the command is processed successfully or the number of executions reaches the
|
|
23
23
|
try limits allowed. Utility programs 'dsrqst' and 'dsupdt' carry their own failure-
|
|
24
24
|
recovering ability and they do not need check-reprocessing.
|
|
25
25
|
|
|
@@ -38,16 +38,16 @@ Program 'dscheck' supports the following major functions:
|
|
|
38
38
|
and include error messages if any, to a specialist
|
|
39
39
|
- Delete recorded command information, because the command is not needed anymore
|
|
40
40
|
- Unlock a given recorded command in case lock information was not cleaned
|
|
41
|
-
properly when the command
|
|
41
|
+
properly when the command has failed
|
|
42
42
|
- Interrupt a utility command that is under execution and kill recursively all
|
|
43
43
|
the associated child processes
|
|
44
44
|
- Add the due 'dsrqst' and 'dsupdt' actions into dscheck records
|
|
45
|
-
- Process commands that have been recorded into GDEXDB or the ones have
|
|
46
|
-
previously.
|
|
45
|
+
- Process commands that have been recorded into GDEXDB or the ones that have
|
|
46
|
+
failed previously.
|
|
47
47
|
|
|
48
48
|
The specialist who executes a utility command under 'dscheck' control remains
|
|
49
|
-
the exclusive owner of the check record in GDEXDB. This prevents the command
|
|
50
|
-
processed or deleted accidentally by other specialists.
|
|
49
|
+
the exclusive owner of the check record in GDEXDB. This prevents the command from being
|
|
50
|
+
processed or deleted accidentally by other specialists.
|
|
51
51
|
|
|
52
52
|
In the following sections, general usages of 'dscheck' are described first; and
|
|
53
53
|
detail descriptions of Action options are given; and finally Mode and Info
|
|
@@ -62,7 +62,7 @@ options are explained.
|
|
|
62
62
|
Quotes [] indicate optional. A pipeline '|' in parentheses as in format (A|B)
|
|
63
63
|
means either A or B can be used. The options applied to 'dscheck' are divided
|
|
64
64
|
into three categories, Action, Mode, and Information (Info for short) options.
|
|
65
|
-
Action options are used to specify what
|
|
65
|
+
Action options are used to specify what task this utility program is to execute,
|
|
66
66
|
Mode options are used to modify behaviors of the actions, and the Info options
|
|
67
67
|
are used to pass information, one or multiple values, to run 'dscheck'. An option
|
|
68
68
|
can be given in form of either short name or long name, -DS or -Dataset for
|
|
@@ -77,8 +77,8 @@ and certain Mode options can be applied to alter the behaviors of the Action.
|
|
|
77
77
|
|
|
78
78
|
All options, except Info option -IF (-InputFile), can be given either on command
|
|
79
79
|
line or in input files. Input file names are presented per Info option -IF and
|
|
80
|
-
can only be provided on command line.
|
|
81
|
-
(-InputFile) for
|
|
80
|
+
can only be provided on command line. Refer to the description of Info option -IF
|
|
81
|
+
(-InputFile) for details on how to present options in input files. One or multiple
|
|
82
82
|
input files, combined with options on command line, are allowed to run 'dscheck'.
|
|
83
83
|
The option name, -IF (-InputFile), itself can be omitted if a single input file is
|
|
84
84
|
given on command line and all other option information are provided inside the
|
|
@@ -88,7 +88,7 @@ When information of daemons and checks are retrieved, Info options are used to
|
|
|
88
88
|
specify conditions for querying information from GDEXDB. Some special signs can be
|
|
89
89
|
used to further confine the information with special and complicated conditions;
|
|
90
90
|
they are '!', '<', '>' and '<>'. These special signs, if provided on command
|
|
91
|
-
line, must be quoted or escaped to avoid
|
|
91
|
+
line, must be quoted or escaped to avoid being interpreted by the Unix shell.
|
|
92
92
|
The '!', or \!, means exclusion to the following value(s) and it must be the
|
|
93
93
|
first item following an Info option name, while '<' or '>' mean greater or less
|
|
94
94
|
than the following value and '<>' means between the following two values.
|
|
@@ -102,7 +102,7 @@ command line as
|
|
|
102
102
|
|
|
103
103
|
A description is displayed for an option given either before or after -(h|help).
|
|
104
104
|
If no option is specified or 'dscheck' is issued by itself, this whole document
|
|
105
|
-
is displayed
|
|
105
|
+
is displayed via UNIX utility 'more'. A hard copy of this help document can be
|
|
106
106
|
printed from the saved file, dscheck.usg, under python package rda_python_dscheck.
|
|
107
107
|
|
|
108
108
|
#The online HTML version of this document is available at
|
|
@@ -133,11 +133,11 @@ categories:
|
|
|
133
133
|
|
|
134
134
|
A daemon control record for a command, a specialist and a hostname is used to
|
|
135
135
|
configure how many concurrent processes the specified command can be executed
|
|
136
|
-
for the specialist on specified hostname, and the priority the
|
|
136
|
+
for the specialist on specified hostname, and the priority the hostname is
|
|
137
137
|
picked to start the command. A running centralized daemon reads this record
|
|
138
138
|
periodically in case the configuration is changed while the daemon is still
|
|
139
139
|
running, so that specialists can reset the values in daemon control records to
|
|
140
|
-
change the
|
|
140
|
+
change the behavior of dscheck daemon dynamically without shutting the daemon down.
|
|
141
141
|
|
|
142
142
|
Daemon control information can be created, modified and viewed via Actions
|
|
143
143
|
included in this section:
|
|
@@ -168,7 +168,7 @@ categories:
|
|
|
168
168
|
a command and a hostname, the daemon control record is modified; otherwise, a new
|
|
169
169
|
daemon control record is added if daemon index is 0 and Mode option -ND (-NewDaemon)
|
|
170
170
|
is present. Combination of specialist login name, command name and hostname of
|
|
171
|
-
computer must be unique for
|
|
171
|
+
computer must be unique for each daemon control record.
|
|
172
172
|
|
|
173
173
|
Specify host name 'PBS' for putting the command in the PBS batch control system. If
|
|
174
174
|
a specified command name is not found in the daemon control, the general 'dscheck'
|
|
@@ -178,10 +178,10 @@ categories:
|
|
|
178
178
|
for maximum 4 checks can be processed at the same time with priority 1, the smaller
|
|
179
179
|
the number the higher the priority is, via input file daemon.ctl:
|
|
180
180
|
|
|
181
|
-
|
|
181
|
+
dscheck -SD -ND -IF daemon.ctl
|
|
182
182
|
|
|
183
183
|
<<Content of input file daemon.ctl>>
|
|
184
|
-
DaemonIndex<:>Command<:>
|
|
184
|
+
DaemonIndex<:>Specialist<:>Command<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
185
185
|
0<:>schuster<:>ALL<:>PBS<:>4<:>1<:>
|
|
186
186
|
|
|
187
187
|
|
|
@@ -289,7 +289,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
289
289
|
as the owner of the added check record, and the current path is defaulted as the
|
|
290
290
|
working directory when the command is executed later.
|
|
291
291
|
|
|
292
|
-
|
|
292
|
+
Specify additional PBS batch options via Info option -QS (-QsubOptions) to add a check;
|
|
293
293
|
and specify a parent check index to put the current command on hold until the parent
|
|
294
294
|
check is finished.
|
|
295
295
|
|
|
@@ -299,7 +299,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
299
299
|
For example, to list file names in the current directory with name containing 'test'
|
|
300
300
|
by catching the standard output into a log and display it on screen:
|
|
301
301
|
|
|
302
|
-
|
|
302
|
+
dscheck -AC -CM test1.sh -HN PBS
|
|
303
303
|
|
|
304
304
|
<<Content of shell script test1.sh>>
|
|
305
305
|
#!/bin/sh
|
|
@@ -308,7 +308,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
308
308
|
For example, to list file names in the current directory with name containing 'test'
|
|
309
309
|
by catching the standard output and error into separate log files:
|
|
310
310
|
|
|
311
|
-
|
|
311
|
+
dscheck -AC -CM test2.sh -HN PBS
|
|
312
312
|
|
|
313
313
|
<<Content of shell script test2.sh>>
|
|
314
314
|
#!/bin/sh
|
|
@@ -317,7 +317,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
317
317
|
For example, to add testing command 'test2' into 'dscheck' for delayed mode execution on
|
|
318
318
|
PBS:
|
|
319
319
|
|
|
320
|
-
|
|
320
|
+
dscheck -AC -CM test3 -HN PBS
|
|
321
321
|
|
|
322
322
|
The command 'test3' must be executable at the current working directory on PBS machines.
|
|
323
323
|
|
|
@@ -347,7 +347,7 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
347
347
|
of a given field
|
|
348
348
|
|
|
349
349
|
Use Info option -FN (-FieldNames) to specify what check fields to retrieve.
|
|
350
|
-
It defaults to "
|
|
350
|
+
It defaults to "COVTUPFJDNW" if -FN is not given.
|
|
351
351
|
|
|
352
352
|
Valid check field names and their corresponding Info options:
|
|
353
353
|
|
|
@@ -395,17 +395,17 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
395
395
|
|
|
396
396
|
|
|
397
397
|
3.2.4 Unlock Check
|
|
398
|
-
-UL or -UnLockCheck, (Alias: -UnLock), unlocks check records
|
|
399
|
-
commands aborted abnormally during
|
|
398
|
+
-UL or -UnLockCheck, (Alias: -UnLock), unlocks check records whose
|
|
399
|
+
commands aborted abnormally during processing.
|
|
400
400
|
|
|
401
401
|
Process ID and computer hostname are saved in a check record when the recorded
|
|
402
402
|
command is running. If the process aborts abnormally, the PID and hostname may
|
|
403
403
|
sometimes be not cleaned properly. Use this action to clean up the locking
|
|
404
404
|
information to allow the command to be reprocessed or purged.
|
|
405
405
|
|
|
406
|
-
dscheck -(
|
|
407
|
-
|
|
408
|
-
|
|
406
|
+
dscheck -(UL|UnLockCheck)
|
|
407
|
+
-(CI|CheckIndex) CheckIndices
|
|
408
|
+
[-(DB|Debug) DebugModeInfo]
|
|
409
409
|
|
|
410
410
|
It is mandatory to provide a check index to remove lock on a recorded command.
|
|
411
411
|
|
|
@@ -458,13 +458,13 @@ DaemonIndex<:>Command<:>Specialist<:>Hostname<:>ProcessLimit<:>Priority<:>
|
|
|
458
458
|
according to the configured information in daemon controls.
|
|
459
459
|
|
|
460
460
|
3.3.2 Interrupt Check
|
|
461
|
-
-IC or -InterruptCheck, interrupts checks
|
|
462
|
-
and also kills recursively all the child processes
|
|
461
|
+
-IC or -InterruptCheck, interrupts checks whose commands are currently running
|
|
462
|
+
and also kills recursively all the child processes running under those
|
|
463
463
|
commands. Locks are cleaned too when the checks are interrupted.
|
|
464
464
|
|
|
465
|
-
dscheck -(
|
|
466
|
-
|
|
467
|
-
|
|
465
|
+
dscheck -(IC|InterruptCheck) [Mode Option]
|
|
466
|
+
-(CI|CheckIndex) CheckIndices
|
|
467
|
+
[-(DB|Debug) DebugModeInfo]
|
|
468
468
|
|
|
469
469
|
Mode option that can be specified for this action:
|
|
470
470
|
-(FI|ForceInterrupt) - it must be present for this action to interrupt a check
|
|
@@ -524,11 +524,11 @@ are all optional. No value is allowed to be passed in following any Mode option.
|
|
|
524
524
|
-AW or -AnyWhere, for Action -AC (-AddCheck), sets Working directory empty in
|
|
525
525
|
check record to start processing the check anywhere.
|
|
526
526
|
|
|
527
|
-
-BG or -BackGround (Alias: -b), background process. When it
|
|
528
|
-
screen display is turned off for both standard
|
|
529
|
-
|
|
530
|
-
-CP or -CheckPending, if present for Action -PC (-ProcessCheck),
|
|
531
|
-
long pending checks
|
|
527
|
+
-BG or -BackGround (Alias: -b), background process. When it is present,
|
|
528
|
+
screen display is turned off for both standard output and errors.
|
|
529
|
+
|
|
530
|
+
-CP or -CheckPending, if present for Action -PC (-ProcessCheck), checks and kills
|
|
531
|
+
long pending checks.
|
|
532
532
|
|
|
533
533
|
-CS or -CheckStatus, if present, displays detailed status information of the
|
|
534
534
|
recorded commands, including progress percentage for a running command and
|
|
@@ -562,7 +562,7 @@ are all optional. No value is allowed to be passed in following any Mode option.
|
|
|
562
562
|
-WR or -WithdsRqst, adds requests due to be built or purged of command 'dsrqst'
|
|
563
563
|
for 'dscheck' Action -PC (-ProcessCheck) in non-daemon mode.
|
|
564
564
|
|
|
565
|
-
-WU or -WithdsUpdt, adds due update controls of command '
|
|
565
|
+
-WU or -WithdsUpdt, adds due update controls of command 'dsupdt' for 'dscheck'
|
|
566
566
|
Action -PC (-ProcessCheck) in non-daemon mode.
|
|
567
567
|
|
|
568
568
|
5 INFORMATION OPTIONS
|
|
@@ -637,7 +637,7 @@ Information options are used to pass information, one or multiple values, into
|
|
|
637
637
|
|
|
638
638
|
-CD or -CheckDate, the check date of the recorded command is first processed.
|
|
639
639
|
|
|
640
|
-
-CI or -
|
|
640
|
+
-CI or -CheckIndex, check record indices for commands recorded in GDEXDB. A check
|
|
641
641
|
record is automatically purged if the command is finished.
|
|
642
642
|
|
|
643
643
|
-CM or -Command (Alias: -CommandName), the command name recorded in check record.
|
|
@@ -650,8 +650,8 @@ Information options are used to pass information, one or multiple values, into
|
|
|
650
650
|
single integer value, for example, 1000 means to log debug messages for debug
|
|
651
651
|
levels 1 to 1000; or a range of values, for example, 200-1000 means to log
|
|
652
652
|
debug messages from debug levels 200 to 1000. The default debug file path is
|
|
653
|
-
'$DSSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'.
|
|
654
|
-
the second and third values for this option to override the
|
|
653
|
+
'${DSSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'. Provide
|
|
654
|
+
the second and third values for this option to override the defaults
|
|
655
655
|
respectively.
|
|
656
656
|
|
|
657
657
|
-DC or -DoneCount, the number of files that are processed successfully already.
|
|
@@ -689,7 +689,7 @@ Information options are used to pass information, one or multiple values, into
|
|
|
689
689
|
option -AO (-ActOption, default to <!>); and different equal sign of single
|
|
690
690
|
value assignment can be provided by Info option -ES, (-EqualSign, default to
|
|
691
691
|
'<=>'). Multi-value assignments can be given in columns delimited with
|
|
692
|
-
separator specified per option -
|
|
692
|
+
separator specified per option -DV (-Divider, default to '<:>'). It starts
|
|
693
693
|
with a column title line for multi-value option names and the rest holds
|
|
694
694
|
values corresponding to each column titles. The value information stops at
|
|
695
695
|
the end of the file or when a new column name line or another single value
|
|
@@ -697,8 +697,8 @@ Information options are used to pass information, one or multiple values, into
|
|
|
697
697
|
additional separator must be appended for each line, including the column
|
|
698
698
|
title line to end lines properly.
|
|
699
699
|
|
|
700
|
-
-MC or -MaxCount, the maximum number of tries
|
|
701
|
-
if the command
|
|
700
|
+
-MC or -MaxCount, the maximum number of tries a check command can be processed
|
|
701
|
+
if the command has failed.
|
|
702
702
|
|
|
703
703
|
-MH or -MatchHost (Alias: -MatchHostname), flag to control hostname match.
|
|
704
704
|
'G' - general match, including empty hostname specified, or exclusive hostnames
|
|
@@ -732,7 +732,7 @@ Information options are used to pass information, one or multiple values, into
|
|
|
732
732
|
-ST or -Status, check command status for a check record. Include command
|
|
733
733
|
progress percentage if under process and error message for a failed command.
|
|
734
734
|
|
|
735
|
-
-SZ or -DataSize, the total bytes of data
|
|
735
|
+
-SZ or -DataSize, the total bytes of data processed for the check record.
|
|
736
736
|
|
|
737
737
|
-TC or -TryCount, the number of tries for a check command being processed. It
|
|
738
738
|
is up to the number of tries specified via info option -MC.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/SOURCES.txt
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/requires.txt
RENAMED
|
File without changes
|
{rda_python_dscheck-2.0.4 → rda_python_dscheck-2.0.5}/src/rda_python_dscheck.egg-info/top_level.txt
RENAMED
|
File without changes
|
|
File without changes
|