rda-python-dsarch 1.0.1__py3-none-any.whl → 1.0.2__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- rda_python_dsarch/dsarch.usg +2462 -0
- {rda_python_dsarch-1.0.1.dist-info → rda_python_dsarch-1.0.2.dist-info}/METADATA +1 -1
- rda_python_dsarch-1.0.2.dist-info/RECORD +11 -0
- rda_python_dsarch-1.0.1.dist-info/RECORD +0 -10
- {rda_python_dsarch-1.0.1.dist-info → rda_python_dsarch-1.0.2.dist-info}/LICENSE +0 -0
- {rda_python_dsarch-1.0.1.dist-info → rda_python_dsarch-1.0.2.dist-info}/WHEEL +0 -0
- {rda_python_dsarch-1.0.1.dist-info → rda_python_dsarch-1.0.2.dist-info}/entry_points.txt +0 -0
- {rda_python_dsarch-1.0.1.dist-info → rda_python_dsarch-1.0.2.dist-info}/top_level.txt +0 -0
|
@@ -0,0 +1,2462 @@
|
|
|
1
|
+
|
|
2
|
+
1 INTRODUCTION
|
|
3
|
+
|
|
4
|
+
Program 'dsarch' is a comprehensive utility for archiving data onto CISL
|
|
5
|
+
Research Data Archive (RDA) Servers; and recording detailed information about
|
|
6
|
+
data files, data period, file counts and sizes of a given dataset into Research
|
|
7
|
+
Data Archive Database (RDADB). A Digital Object Identifier (DOI) can be
|
|
8
|
+
assigned to a dataset to put the dataset under DOI/Version control, and all
|
|
9
|
+
the web data files of the dataset are linked to the new DOI/Version Control.
|
|
10
|
+
Archived data files for one dataset can be organized into sub-products, called
|
|
11
|
+
groups. Data files can be further organized into subgroups. Theoretically there
|
|
12
|
+
is no limit to how many levels of subgroups that can be setup. The information
|
|
13
|
+
of data files recorded in RDADB can be also retrieved and modified using this
|
|
14
|
+
utility program.
|
|
15
|
+
|
|
16
|
+
A dataset, if not in RDADB yet, can be initiated through Metadata Manager,
|
|
17
|
+
https://dss.ucar.edu/cgi-bin/internal/dssmm. To process data archives, 'dsarch'
|
|
18
|
+
can perform the following major functions:
|
|
19
|
+
- Setup a dataset, and its groups/subgroups if specified, in the RDADB to
|
|
20
|
+
make it ready for data archiving
|
|
21
|
+
- Add, update and terminate DOI/Version Controls for a given dataset
|
|
22
|
+
- Archive data files by copying them from working area to the RDA Servers
|
|
23
|
+
- Call 'gatherxml' to evaluate file content metadata for the archived files
|
|
24
|
+
- Cross-archive data files by duplicating them between Object Stores and RDA
|
|
25
|
+
Data Disk Servers
|
|
26
|
+
- Save file information of the archived data into RDADB
|
|
27
|
+
- Retrieve dataset/group/file information saved in RDADB
|
|
28
|
+
- Cross-archive data files by duplicating them between Object Stores and RDA
|
|
29
|
+
Data Disk Servers
|
|
30
|
+
- Move data files from one dataset, or even one group, to another; or move
|
|
31
|
+
data files from one location to another
|
|
32
|
+
- Tar a provided file list from one or multiple datasets, and Backup it up
|
|
33
|
+
onto Globus Quasar Server for a backup copy, and optionally another disaster
|
|
34
|
+
backup copy
|
|
35
|
+
- Retore damaged data files from the Quasar backup server
|
|
36
|
+
- Remove files from RDA Servers
|
|
37
|
+
- Auto-restart interrupted dsarch commands recorded in RDADB
|
|
38
|
+
- Regenerate filelist and main webpages on demand after changes made to
|
|
39
|
+
a dataset
|
|
40
|
+
|
|
41
|
+
A supporting utility program, 'myconvert', was developed to convert and transfer
|
|
42
|
+
SCCS metadata (text-based flat files) into RDADB, and another program,
|
|
43
|
+
'publish_filelist', was developed to publish the RDA Server filelists
|
|
44
|
+
of datasets to user interfaces on the RDA Web Server. These are complementary
|
|
45
|
+
utilities of 'dsarch'. Check usages of them for detail help information.
|
|
46
|
+
|
|
47
|
+
The RDADB dataset flag, -UD (-UseDSARCH), must be, at least, set to 'Y' for a
|
|
48
|
+
dataset before data files can be archived and the associated file information
|
|
49
|
+
can be inserted or modified into RDADB by 'dsarch'. This flag also controls
|
|
50
|
+
publication for file lists: 'P' or 'W' to publish file list for data files on
|
|
51
|
+
RDA Server. The published filelists are part of the user interface for
|
|
52
|
+
each dataset viewed on the RDA Web Server. Check description of Action option
|
|
53
|
+
-SD (-SetDataset) and Info option -UD (-UseDSARCH) for details on how to set
|
|
54
|
+
this flag properly.
|
|
55
|
+
|
|
56
|
+
If, after moved, a data file name, including its path, is different from its
|
|
57
|
+
original file name, the new file name overrides the original file name,
|
|
58
|
+
but the original name is also recorded in RDADB with link to the new data file
|
|
59
|
+
name. Both file names are considered to be the same data file when the data
|
|
60
|
+
accesses of this file are gathered for usage metrics. Therefore, 'dsarch' is
|
|
61
|
+
recommended for moving data files, instead of direct moving commands.
|
|
62
|
+
|
|
63
|
+
Combine another untiliy program, dsquasar, to backup archived data files onto
|
|
64
|
+
Globus Quasar Server. The dsquasar helps to identify and create filelists of
|
|
65
|
+
un-backed up data files, and save them as input files for dsarch to tar and
|
|
66
|
+
back them up.
|
|
67
|
+
|
|
68
|
+
Some validation procedures are added to prevent DECS specialists from executing
|
|
69
|
+
'dsarch' accidentally on a wrong dataset. If an input file is used to hold
|
|
70
|
+
option information for processing 'dsarch' of a given dataset, the input file
|
|
71
|
+
name must start with the dataset number, as in format of 'dsnnn.n.*'; the
|
|
72
|
+
wildcard '*' matches one or multiple characters valid for file names. Another
|
|
73
|
+
validation is supported to check if a specialist is allowed to execute 'dsarch'
|
|
74
|
+
on the chosen dataset. Action stops if a specialist is not in the authorized
|
|
75
|
+
list, unless a Mode option -MD (-MyDataset) is present. The option -MD tells
|
|
76
|
+
'dsarch' to go ahead finish an Action forcefully.
|
|
77
|
+
|
|
78
|
+
For failed data archive actions, (-AW|-AS|-AH|-AQ), due to storage system downs,
|
|
79
|
+
the identical dsarch actions will be retried after the systems are backed up
|
|
80
|
+
again. This ability will only be turned on if option -BP (-d) is present and
|
|
81
|
+
the dsarch actions sent into background batch processes, and the related
|
|
82
|
+
information is saved in RDADB and
|
|
83
|
+
|
|
84
|
+
When a DOI/Version control is added to dataset, it can not be removed, although
|
|
85
|
+
it can be terminated when the dataset is not active anymore for its data are out
|
|
86
|
+
of date. The data filelist of the inactive dataset can still be reconstructed
|
|
87
|
+
per user request via the terminated DOI/Version control information. Restrictions
|
|
88
|
+
are applied to data files of a dataset that is under DOI/Version control. These
|
|
89
|
+
data files can not be moved or deleted, but new data files can be added for
|
|
90
|
+
operational datasets. When a data file is archived, normally it can not changed
|
|
91
|
+
unless new data are appended to the same data files.
|
|
92
|
+
|
|
93
|
+
When a dataset is under DOI/Version control, only the Web data files are put
|
|
94
|
+
under the version control directly. If there are saved data staged on disks
|
|
95
|
+
for internal uses or easy access to the NCAR users, it is responsibility of
|
|
96
|
+
the specialist who owns the dataset to maintain them.
|
|
97
|
+
|
|
98
|
+
In the following sections, general usage of 'dsarch' is described first; and
|
|
99
|
+
detail descriptions of Action options are given next; and then Mode and Info
|
|
100
|
+
options are explained in more detail on how to modify and pass information
|
|
101
|
+
for Action options, respectively. Examples of actions and other options are
|
|
102
|
+
interspersed through out the document.
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
2 GENERAL DSARCH USAGE
|
|
106
|
+
|
|
107
|
+
dsarch [[-(DS|Dataset)] dsnnn.n] [Action Option] [Mode Options] [Info Options]
|
|
108
|
+
or
|
|
109
|
+
dsarch [-(IF|InputFile)] InputFileNames
|
|
110
|
+
|
|
111
|
+
Quotes [] indicate optional. A pipeline '|' in parentheses as in format (A|B)
|
|
112
|
+
means either A or B can be used. The options applied to 'dsarch' are divided
|
|
113
|
+
into three categories, Action, Mode, and Information (Info for short) options.
|
|
114
|
+
Action options are used to specify what tasks this utility to execute, Mode
|
|
115
|
+
options are used to modify behaviors of the actions, and the Info options
|
|
116
|
+
are used to pass information, one or multiple values, to run 'dsarch'. An option
|
|
117
|
+
can be given in form of either short name or long name, -DS or -Dataset for
|
|
118
|
+
example. Some options have alias names for convenience; -UseRDADB, for example,
|
|
119
|
+
is an alias option name for Mode option -UD (-UseDSARCH). Option names can be
|
|
120
|
+
given in either upper or lower cases, while the values following Info options
|
|
121
|
+
are case sensitive.
|
|
122
|
+
|
|
123
|
+
Option -DS is actually an Info option which is used to specify a dataset number.
|
|
124
|
+
It is listed separately because a dataset number is a common value required for
|
|
125
|
+
most actions. If a dataset number is given as the first option following
|
|
126
|
+
'dsarch' the option name -DS (-Dataset) can be omitted.
|
|
127
|
+
|
|
128
|
+
Specify one of the Action options each time to execute 'dsarch'. Based on what
|
|
129
|
+
Action is chosen, some of the Info options are mandatory and others are optional
|
|
130
|
+
and certain Mode options can be applied to alter the behaviors of the Action.
|
|
131
|
+
|
|
132
|
+
All options, except Info option -IF (-InputFile), can be given either on command
|
|
133
|
+
line or in input files. Input file names are presented per Info option -IF and
|
|
134
|
+
can only be provided on command line. Check description of Info option -IF
|
|
135
|
+
(-InputFile) for detail on how to present options in input files. One or
|
|
136
|
+
multiple input files, combined with options on command line, are allowed to run
|
|
137
|
+
'dsarch'. The option name, -IF (-InputFile), itself can be omitted if a single
|
|
138
|
+
input file is given on command line, while the action and other option
|
|
139
|
+
information are all provided inside the input file.
|
|
140
|
+
|
|
141
|
+
When information of dataset, DOI/Version Controls, groups, and the
|
|
142
|
+
Web/Save/Help/Quasar files are retrieved from RDADB, Info options are used to
|
|
143
|
+
specify conditions for querying information from RDADB. Some special signs can
|
|
144
|
+
be used to further confine the information with special and complicated
|
|
145
|
+
conditions; they are '!', '<', '>' and '<>'. These special signs, if provided
|
|
146
|
+
on command line, must be quoted or escaped to avoid of being interpreted by
|
|
147
|
+
Unix OS system. The '!', or \!, means exclusion to the following value(s) and
|
|
148
|
+
it must be the first item following an Info option name, while '<' or '>' mean
|
|
149
|
+
greater or less than the following value, and '<>' means between the following
|
|
150
|
+
two values. Combine '!' and '<', as syntax "'!' '<' OptionValue", to make a
|
|
151
|
+
condition of 'larger than or equal to OptionValue'.
|
|
152
|
+
|
|
153
|
+
Description of an individual option is displayed if 'dsarch' is issued on
|
|
154
|
+
command line as
|
|
155
|
+
|
|
156
|
+
dsarch [Option] -(h|help) [Option]
|
|
157
|
+
|
|
158
|
+
A description is displayed for an option given either before or after -(h|help).
|
|
159
|
+
If no option is specified or 'dsarch' is issued by itself, this whole document
|
|
160
|
+
is displayed per UNIX utility 'more'. A hard copy of this help document can be
|
|
161
|
+
printed from the saved file: ~rdadata/dssdb/prog_usage/dsarch.usg.
|
|
162
|
+
|
|
163
|
+
#The online HTML version of this document is available at
|
|
164
|
+
#http://dss.ucar.edu/internal/docs/dsarch/
|
|
165
|
+
|
|
166
|
+
|
|
167
|
+
3 ACTION OPTIONS
|
|
168
|
+
|
|
169
|
+
Action options are used to specify what task 'dsarch' executes. No values
|
|
170
|
+
are allowed to follow Action options. Multiple tasks may be processed with a
|
|
171
|
+
single execution of 'dsarch' depending on what Action option is chosen. Some of
|
|
172
|
+
the comprehensive actions include automatically other simpler actions as
|
|
173
|
+
default; and others include additional actions when certain Mode options are
|
|
174
|
+
present. Multiple Action options provided simultaneously are blocked.
|
|
175
|
+
|
|
176
|
+
Some actions are setting information into and some getting information from
|
|
177
|
+
RDADB for a given dataset. A using DSARCH flag must be set to 'Y', 'I', 'P', or
|
|
178
|
+
'W' per Info option -UD (-UseDSARCH) before setting actions can be executed
|
|
179
|
+
for a dataset.
|
|
180
|
+
|
|
181
|
+
Based on the information being manipulated, the actions are divided into seven
|
|
182
|
+
categories:
|
|
183
|
+
Dataset Actions - create, modify and retrieve dataset information in RDADB
|
|
184
|
+
DOI/Version Actions - add, modify, view and terminate DOI/Version control for
|
|
185
|
+
a given dataset
|
|
186
|
+
Group Actions - create, delete, modify and retrieve group information in
|
|
187
|
+
RDADB, of a given dataset
|
|
188
|
+
File Actions - archive local data files as Saved/Web/Help files; move
|
|
189
|
+
and delete archived files on RDA Server; create, delete,
|
|
190
|
+
modify and retrieve information of archived files in
|
|
191
|
+
RDADB, of a given dataset, and groups if specified
|
|
192
|
+
All Info Actions - create, modify and retrieve all information, of dataset,
|
|
193
|
+
groups and files of a given dataset in RDADB
|
|
194
|
+
Webpage Actions - regenerate dataset main and filelist webpages
|
|
195
|
+
|
|
196
|
+
|
|
197
|
+
3.1 Dataset Actions
|
|
198
|
+
|
|
199
|
+
Dataset information can be modified and viewed via Actions included in this
|
|
200
|
+
section:
|
|
201
|
+
Set Dataset Information - modify information of an existing dataset
|
|
202
|
+
Get Dataset Information - retrieve information of an existing dataset
|
|
203
|
+
|
|
204
|
+
3.1.1 Set Dataset Information
|
|
205
|
+
-SD or -SetDataset, modifies dataset information into RDADB for a given
|
|
206
|
+
dataset number.
|
|
207
|
+
|
|
208
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](SD|SetDataset) [Mode Options]
|
|
209
|
+
[-(UD|UseDSARCH) UseDSARCHFlag] # Y, P, W, I, or N
|
|
210
|
+
[-(LC|Location) AccessLocationFlag] # C - CGD, G - Glade, O - Object Store
|
|
211
|
+
[-(BF|BackupFlag) QuasarBackupFlag] # B - Backup, D - Backup & Drdata
|
|
212
|
+
[-(DF|DataFormat) DataFormat] # BINARY, ASCII, NetCDF and etc.
|
|
213
|
+
[-(WH|WebHome) WebDataHomeDirectory]
|
|
214
|
+
[-(WP|WebPath) WebFilePath]
|
|
215
|
+
[-(SP|SavedPath) SavedFilePath]
|
|
216
|
+
[-(FL|FileLimit) FileCountLimit]
|
|
217
|
+
[-(BL|ButtonLimit) FileLimitShowButton]
|
|
218
|
+
[-(GL|GroupLevel) NumberOfGroupDisplay]
|
|
219
|
+
[-(ML|MetaLink) MetadataLink]
|
|
220
|
+
[-(NW|NoteWeb) WebNote]
|
|
221
|
+
[-(ND|NoteDocument) DocumentNote]
|
|
222
|
+
[-(NS|NoteSoftware) SoftwareNote]
|
|
223
|
+
[-(NI|NoteInternal) InternalNote]
|
|
224
|
+
[-(DB|Debug) DebugModeInfo]
|
|
225
|
+
# options for setting multiple specialist defined key/value pairs
|
|
226
|
+
[-(KV|KeyValue) KeyValuePairs] # in form of 'key=>value'
|
|
227
|
+
# options for setting multiple data periods
|
|
228
|
+
[-(GI|GroupIndex) GroupIndices] # used to identify groups for periods
|
|
229
|
+
[-(BD|BeginDate) BeginOnDates]
|
|
230
|
+
[-(BT|BeginTime) BeginAtTimes]
|
|
231
|
+
[-(ED|EndDate) EndOnDates]
|
|
232
|
+
[-(ET|EndTime) EndOnTimes]
|
|
233
|
+
|
|
234
|
+
Mode options that can be specified for setting dataset action:
|
|
235
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
236
|
+
information to be set by any one
|
|
237
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
238
|
+
values to speed up reading input file(s)
|
|
239
|
+
-(RT|ResetTGroup) - reset top group index for all the Saved/Web file records
|
|
240
|
+
-(WM|WithMetadata) - re-gathers Web file content metadata at dataset level
|
|
241
|
+
-(WN|WithFileNumber) - reevaluates and resets file counts
|
|
242
|
+
|
|
243
|
+
A given dataset number is normally added via dataset Metadata Manager (MM).
|
|
244
|
+
The Action -SD modifies dataset record, and the associated
|
|
245
|
+
data periods for groups if any. Data periods for groups can be modified by
|
|
246
|
+
'dsarch', while they can be created, removed and modified per MM.
|
|
247
|
+
|
|
248
|
+
The Web data accessing location flag of a dataset is provided via option
|
|
249
|
+
-LC (-Location). The defualt value is 'G' for accessing web files from Glade
|
|
250
|
+
Disk if there exists any web data files. A value of 'O' is set as default
|
|
251
|
+
if all web data files of the dataset are on Object Store, and a value of 'C'
|
|
252
|
+
is set as default if all web data files of the dataset are at CGD data path.
|
|
253
|
+
|
|
254
|
+
For example, set dataset information of use DSARCH flag, reset file counts and
|
|
255
|
+
data size, and beginning data date into RDADB for dataset number 'd744004'
|
|
256
|
+
|
|
257
|
+
dsarch d744004 SD -WN -UD Y -BD 1999-07-01
|
|
258
|
+
|
|
259
|
+
3.1.2 Get Dataset Information
|
|
260
|
+
-GD or -GetDataset, retrieves dataset information from RDADB for a given
|
|
261
|
+
dataset number.
|
|
262
|
+
|
|
263
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](GD|GetDataset) [Mode Options]
|
|
264
|
+
[-(FN|FieldNames) FieldNameString]
|
|
265
|
+
[-(KV|KeyValue) KeyNames] # specify key names to view their values
|
|
266
|
+
[-(OF|OutputFile) OutputFileName]
|
|
267
|
+
[-(DB|Debug) DebugModeInfo]
|
|
268
|
+
|
|
269
|
+
Mode options that can be specified for getting dataset Action:
|
|
270
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
271
|
+
values of a given field
|
|
272
|
+
-(PE|ShowPeriod) - show associated period information for dataset, and
|
|
273
|
+
groups if any
|
|
274
|
+
-(WN|WithFileNumber) - gathers and displays Saved/Web file counts
|
|
275
|
+
|
|
276
|
+
Use Info option -FN (-FieldNames) to specify what dataset fields to retrieve.
|
|
277
|
+
It defaults to 'SWFULQ' if option -FN is omitted. Information of all available
|
|
278
|
+
fields is retrieved if -FN ALL is given.
|
|
279
|
+
|
|
280
|
+
Valid dataset field names and their corresponding Info options:
|
|
281
|
+
|
|
282
|
+
Names Info Options Descriptions
|
|
283
|
+
T -(TI|Title) dataset title
|
|
284
|
+
S -(SP|SavedPath) path relative to Saved home directory
|
|
285
|
+
H -(WH|WebHome) path for the web data home directory
|
|
286
|
+
W -(WP|WebPath) path relative to web home directory
|
|
287
|
+
F -(DF|DataFormat) default data content format
|
|
288
|
+
U -(UD|UseDSARCH) use DSARCH flag
|
|
289
|
+
L -(LC|Location) Online web file access location flags
|
|
290
|
+
Q -(BF|BackupFlag) Quasar Backup flags
|
|
291
|
+
V -(GL|GroupLevel) number of group levels to display
|
|
292
|
+
C -(FL|FileLimit) file count limit for separate group filelists
|
|
293
|
+
B -(BL|ButtonLimit) file count limit for showing script buttons
|
|
294
|
+
A -(ML|MetaLink) a link to content meta at the dataset level
|
|
295
|
+
W -(NW|NoteWeb) web data description
|
|
296
|
+
I -(NI|NoteInternal) internal data description
|
|
297
|
+
D -(ND|NoteDocument) document description
|
|
298
|
+
N -(NS|NoteSoftware) software description
|
|
299
|
+
G -(GI|GroupIndex) Group Indices to identify groups for periods
|
|
300
|
+
J -(BD|BeginDate) date data begin for dataset/group
|
|
301
|
+
K -(BT|BeginTime) time data begin for dataset/group
|
|
302
|
+
X -(ED|EndDate) date data end for dataset/group
|
|
303
|
+
Y -(ET|EndTime) time data end for dataset/group
|
|
304
|
+
|
|
305
|
+
Info option -OF (-OutputFile) is used to specify a file name to save the
|
|
306
|
+
retrieved information for later usage. Result of this action is displayed
|
|
307
|
+
on screen if no output file is provided.
|
|
308
|
+
|
|
309
|
+
For example, get dataset information of d260001 for all fields and dump the
|
|
310
|
+
output into file 'd260001.ds'
|
|
311
|
+
|
|
312
|
+
dsarch d260001 GD -FN ALL -PE -OF d260001.ds
|
|
313
|
+
|
|
314
|
+
<<Content of output file d260001.ds>>
|
|
315
|
+
Dataset<=>d260001
|
|
316
|
+
Title<=>Objectively Analyzed Air-Sea Fluxes (OAFlux) For Global Oceans, By Lisan Yu et al.
|
|
317
|
+
SavedPath<=>
|
|
318
|
+
WebPath<=>
|
|
319
|
+
Location<=>G
|
|
320
|
+
DataFormat<=>
|
|
321
|
+
UseDSARCH<=>P
|
|
322
|
+
GroupLevel<=>3
|
|
323
|
+
FileLimit<=>500
|
|
324
|
+
ButtonLimit<=>2
|
|
325
|
+
DescWeb<=>
|
|
326
|
+
DescInternal<=>
|
|
327
|
+
GroupIndex<:>BeginDate<:>EndDate<:>BeginTime<:>EndTime<:>
|
|
328
|
+
1<:>1985-01-01<:>2006-12-31<:>00:00:00<:>00:00:59<:>
|
|
329
|
+
2<:>1958-01-01<:>2006-12-31<:>00:00:00<:>00:00:59<:>
|
|
330
|
+
|
|
331
|
+
|
|
332
|
+
3.2 DOI/Version Actions
|
|
333
|
+
|
|
334
|
+
DOI/Version controls can be added, modified, viewed and terminated via Actions
|
|
335
|
+
included in this section:
|
|
336
|
+
Set Version Control - add and modify DOI/Version control information
|
|
337
|
+
Get Version Control - retrieve DOI/Version control information
|
|
338
|
+
Terminate Version Control - terminate an active DVI/Version control
|
|
339
|
+
|
|
340
|
+
3.2.1 Set Version Control
|
|
341
|
+
-SV or -SetVersion, adds and modifies DOI/Version controls for a given dataset number.
|
|
342
|
+
|
|
343
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](SV|SetVersion) [Mode Options]
|
|
344
|
+
[-(VI|VersionIndex) VersionIndex]
|
|
345
|
+
[-(IV|InternalVersion) InternalVersionIndex]
|
|
346
|
+
[-(EV|ExternalVersion) AssignedVersionNumber]
|
|
347
|
+
[-(DN|DOINumber) DOInumber]
|
|
348
|
+
[-(VT|VersionStatus) VersionRecordStatus] # P-Pending, A-Active, H-History
|
|
349
|
+
[-(BD|BeginDate) VersionStartdate]
|
|
350
|
+
[-(BT|BeginTime) VersionStartTime]
|
|
351
|
+
[-(ED|EndDate) VersionEnddate]
|
|
352
|
+
[-(ET|EndTime) VersionEndTime]
|
|
353
|
+
[-(DE|Description) VersionNote]
|
|
354
|
+
|
|
355
|
+
Mode options that can be specified for setting DOI/Version control action:
|
|
356
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
357
|
+
information to be set by any one
|
|
358
|
+
-(NT|NoTrim) - skips trimming of spaces and comments from input
|
|
359
|
+
values to speed up reading input file(s)
|
|
360
|
+
-(NV|NewVersion) - adds a new version control record into RDADB
|
|
361
|
+
|
|
362
|
+
If a version index is given and it is in RDADB already, its record is modified.
|
|
363
|
+
A new version control record is added if version control index is not given or it is
|
|
364
|
+
0 and Mode option -NV (-NewVersion) is present.
|
|
365
|
+
|
|
366
|
+
When a version control record is first added into RDADB without DOI number specified,
|
|
367
|
+
it is in status of Pending. It can only be changed to Active if a DOI number
|
|
368
|
+
is provided. An active version control record is added with a new DOI number and a
|
|
369
|
+
version control index is auto-generated.
|
|
370
|
+
|
|
371
|
+
For example, put dataset d999009 under DOI/Version control
|
|
372
|
+
|
|
373
|
+
dsarch d999009 SV -NV -DN DOINumber
|
|
374
|
+
|
|
375
|
+
If the data of d999009 are updated mostly or completely to a newer version (set of
|
|
376
|
+
files) a new DOI number is needed. You may execute the same 'dsarch' command to
|
|
377
|
+
add another new active version control record with a new DOI number. The previous
|
|
378
|
+
version control record is changed automatically to status of History and its DOI number
|
|
379
|
+
is superseded now by the current DOI number. Only one active version control record can
|
|
380
|
+
exist for a give dataset.
|
|
381
|
+
|
|
382
|
+
If the data of d999009 are modified minimally a new internal version index is needed
|
|
383
|
+
with the same DOI number, you may execute the same 'dsarch' command to add another
|
|
384
|
+
new version control record with a new internal version index but the same existing DOI
|
|
385
|
+
number. The previous version control record is also changed to status of History.
|
|
386
|
+
|
|
387
|
+
One case of the minimal data change is to replace one or multiple erroneous Web
|
|
388
|
+
files under version control. The files are first moved to different filenames
|
|
389
|
+
via dsarch Action -MV. The file type is autmatically set to 'V' as Version
|
|
390
|
+
controlled type. A new internal version can be optionally added at this time. Then
|
|
391
|
+
new Web files are added as replacements.
|
|
392
|
+
|
|
393
|
+
Usage Note: Various RDA use cases are described here. It is recommended that you review
|
|
394
|
+
and know these use cases, so that the DOIs in the RDA are handled consistently.
|
|
395
|
+
|
|
396
|
+
3.2.2 Get Version Control
|
|
397
|
+
-GV or -GetVersion, retrieves DOI/Version control information from RDADB for a given
|
|
398
|
+
dataset number.
|
|
399
|
+
|
|
400
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](GV|GetVersion) [Mode Option]
|
|
401
|
+
[-(FN|FieldNames) FieldNameString]
|
|
402
|
+
[-(KV|KeyValue) KeyNames] # specify key names to view their values
|
|
403
|
+
[-(OF|OutputFile) OutputFileName]
|
|
404
|
+
[-(DB|Debug) DebugModeInfo]
|
|
405
|
+
|
|
406
|
+
Mode option that can be specified for getting DOI/Version control Action:
|
|
407
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
408
|
+
values of a given field
|
|
409
|
+
|
|
410
|
+
Use Info option -FN (-FieldNames) to specify what version control fields to retrieve.
|
|
411
|
+
It defaults to 'VIEDSJX' if option -FN is omitted. Information of all available
|
|
412
|
+
fields is retrieved if -FN ALL is given.
|
|
413
|
+
|
|
414
|
+
Valid version control field names and their corresponding Info options:
|
|
415
|
+
|
|
416
|
+
Names Info Options Descriptions
|
|
417
|
+
V -(VI|VersionIndex) version control index
|
|
418
|
+
I -(IV|InternalVersion) Internal version number
|
|
419
|
+
E -(EV|ExternalVersion) External version number to publish
|
|
420
|
+
D -(DN|DOINumber) DOI number created for the version control
|
|
421
|
+
S -(VT|VersionStatus) version control status, A-Active, P-Pending and H-History
|
|
422
|
+
J -(BD|BeginDate) date the version control starts at
|
|
423
|
+
K -(BT|BeginTime) time the version control starts at
|
|
424
|
+
X -(ED|EndDate) date the terminated version control ends by
|
|
425
|
+
Y -(ET|EndTime) time the terminated version control ends by
|
|
426
|
+
N -(DE|Description) note for the version control record
|
|
427
|
+
|
|
428
|
+
Info option -OF (-OutputFile) is used to specify a file name to save the
|
|
429
|
+
retrieved information for later usage. Result of this action is displayed
|
|
430
|
+
on screen if no output file is provided.
|
|
431
|
+
|
|
432
|
+
For example, get version control information of d132000 for all fields and dump the
|
|
433
|
+
output into file 'd132000.vrsn'
|
|
434
|
+
|
|
435
|
+
dsarch d132000 GV -FN ALL -OF d132000.vrsn
|
|
436
|
+
|
|
437
|
+
<<Content of output file d132000.vrn>>
|
|
438
|
+
Dataset<=>d132000
|
|
439
|
+
VersionIndex<:>InternalVersion<:>ExternalVersion<:>DOINumber<:>VersionStatus<:>BeginDate<:>BeginTime<:>EndDate<:>EndTime<:>Description<:>
|
|
440
|
+
9<:>1<:><:>10.5065/D6SQ8XDW<:>A<:>2013-03-04<:>14:38:44<:><:><:><:>
|
|
441
|
+
|
|
442
|
+
|
|
443
|
+
3.2.3 Terminate Version Control
|
|
444
|
+
-TV or -Terminate, terminates a version control from RDADB for a specified dataset.
|
|
445
|
+
|
|
446
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](TV|TerminateVersion) [Mode Options]
|
|
447
|
+
-(VI|VersionIndex) VersionIndix
|
|
448
|
+
[-(LN|LoginName) LoginAccountName]
|
|
449
|
+
[-(DB|Debug) DebugModeInfo]
|
|
450
|
+
|
|
451
|
+
Mode options that can be specified for terminating version control action:
|
|
452
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
453
|
+
information to be set by any one
|
|
454
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
455
|
+
values to speed up reading input file(s)
|
|
456
|
+
|
|
457
|
+
The version control record to be terminated is identified by the version index given
|
|
458
|
+
per Info option -VI (-VersionIndex) and dataset number per option -DS. This action
|
|
459
|
+
is blocked if there exist any Web data files lined to the version index. To terminate
|
|
460
|
+
the version control record with Web files under it, you gather the Web file names per
|
|
461
|
+
action option -GW (-GetWebFile), and either set the version index value to 0 or a
|
|
462
|
+
different index, or move the Web files to Saved files; then terminate the version
|
|
463
|
+
control.
|
|
464
|
+
|
|
465
|
+
For example, terminate version index 5 of d999009
|
|
466
|
+
|
|
467
|
+
dsarch d999009 TV -VI 5
|
|
468
|
+
|
|
469
|
+
After termination, an active version control record is changed to status 'H' and the
|
|
470
|
+
record is retained for future references, but for a Pending version control record, it
|
|
471
|
+
is completely removed from RDADB if it is terminated.
|
|
472
|
+
|
|
473
|
+
|
|
474
|
+
3.3 Group Actions
|
|
475
|
+
|
|
476
|
+
Groups are for further dividing a dataset into multiple products. Dataset
|
|
477
|
+
group information can be created, modified, deleted and viewed via Actions
|
|
478
|
+
included in this section:
|
|
479
|
+
Set Group Information - create or modify group information into RDADB for
|
|
480
|
+
a specified dataset
|
|
481
|
+
Get Group Information - retrieve information of existing dataset groups
|
|
482
|
+
Delete Group Information - delete one or multiple dataset groups
|
|
483
|
+
Change Group Information - change one or multiple group indices to different
|
|
484
|
+
values
|
|
485
|
+
|
|
486
|
+
3.3.1 Set Group Information
|
|
487
|
+
-SG or -SetGroup, creates and modifies group information into RDADB for a
|
|
488
|
+
given dataset number. One or multiple groups can be processed each time.
|
|
489
|
+
|
|
490
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](SG|SetGroup) [Mode Options]
|
|
491
|
+
[-(GI|GroupIndex) GroupIndices] # mandatory for new groups
|
|
492
|
+
[-(GN|GroupName) GroupNames]
|
|
493
|
+
[-(PI|ParentIndex) ParentGroupIndices] # set if subgroup
|
|
494
|
+
[-(TI|Title) GroupTitles]
|
|
495
|
+
[-(GT|GroupType) GroupTypes]
|
|
496
|
+
[-(GP|GroupPattern) FileNamePatterns]
|
|
497
|
+
[-(BF|BackupFlag) QuasarBackupFlags]
|
|
498
|
+
[-(SP|SavedPath) SavedFilePaths] # relative to dataset saved home dir
|
|
499
|
+
[-(WP|WebPath) WebFilePaths] # relative to dataset data home dir
|
|
500
|
+
[-(ML|MetaLink) MetadataLinks]
|
|
501
|
+
[-(NW|NoteWeb) WebNotes]
|
|
502
|
+
[-(NI|NoteInternal) InternalNotes]
|
|
503
|
+
[-(LN|LoginName) LoginAccountName]
|
|
504
|
+
[-(DB|Debug) DebugModeInfo]
|
|
505
|
+
|
|
506
|
+
Mode options that can be specified for setting group action:
|
|
507
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
508
|
+
information to be set by any one
|
|
509
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
510
|
+
values to speed up reading input file(s)
|
|
511
|
+
-(RT|ResetTGroup) - reset top group index for all the Saved/Web file records
|
|
512
|
+
in a group
|
|
513
|
+
-(WM|WithMetadata) - re-gathers Web file content metadata at the group level
|
|
514
|
+
-(WN|WithFileNumber) - re-evaluates and resets Saved/Web file counts for the
|
|
515
|
+
affected groups
|
|
516
|
+
|
|
517
|
+
If a group index per option -GI (-GroupIndex) exists in RDADB already for the
|
|
518
|
+
specified dataset, group record is modified; otherwise, a new group record is
|
|
519
|
+
added. For a new group record, a unique group index number must be provided.
|
|
520
|
+
|
|
521
|
+
A tree structure of groups can set by using subgroups. A subgroups is created
|
|
522
|
+
if its parent group index is identified per options -PI (-ParentIndex).
|
|
523
|
+
Theoretically there is no limits how many levels of subgroups can be set.
|
|
524
|
+
As default, three subgroup levels are displayed on group list webpages; but it
|
|
525
|
+
can be reset to any number by option -GL (-GroupLevel) per action -SD
|
|
526
|
+
(-SetDataset).
|
|
527
|
+
|
|
528
|
+
If a group type is changed from P to I, all its subgroups are set to I
|
|
529
|
+
recursively, and all Saved/Web Files under the group or its subgroups are
|
|
530
|
+
also set to I.
|
|
531
|
+
|
|
532
|
+
For example, set multiple group information of d744004 via input file
|
|
533
|
+
'd744004.grp'
|
|
534
|
+
|
|
535
|
+
dsarch SG -IF d744004.grp
|
|
536
|
+
|
|
537
|
+
<<Content of input file d744004.grp>>
|
|
538
|
+
Dataset<=>d744004
|
|
539
|
+
EndDate<=>2005-09-30 # set end data date into table 'dataset'
|
|
540
|
+
GroupIndex<:>Title<:>GroupPattern<:>DescWeb<:>
|
|
541
|
+
1<:>Blended Wind<:>.bln<:>
|
|
542
|
+
The 'bln' product is the blended wind field output.<:>
|
|
543
|
+
2<:>NCEP Re-analyses<:>.low<:>
|
|
544
|
+
The 'low' fields consist of the NCEP re-analyses splined
|
|
545
|
+
to the 0.5 x 0.5 degree grid (i.e. no QSCAT data blended).<:>
|
|
546
|
+
3<:>Superposition of QSCAT Observations<:>.sub<:>
|
|
547
|
+
The 'sub' product is output from an intermediate step in the blending process
|
|
548
|
+
that consists of the superposition of QSCAT observations on NCEP before
|
|
549
|
+
blending. This product is of no particular scientific value, but very useful
|
|
550
|
+
in identifying the satellite data swaths in each composite of 12-hours of
|
|
551
|
+
QSCAT observations.<:>
|
|
552
|
+
|
|
553
|
+
3.3.2 Get Group Information
|
|
554
|
+
-GG or -GetGroup, retrieves group information for a given dataset. Information
|
|
555
|
+
of specified groups are retrieved if the group indices or names are provided.
|
|
556
|
+
|
|
557
|
+
dsarch [-(DS|Dataset] dsnnn.n [-](GG|GetGroup)
|
|
558
|
+
[-(FN|FieldNames) FieldNameString]
|
|
559
|
+
[-(ON|OrderNames) OrderNameString]
|
|
560
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
561
|
+
[-(GN|GroupName) GroupNames]
|
|
562
|
+
[-(PI|ParentGroupIndex) ParentGroupIndices]
|
|
563
|
+
[(-OF|-OutputFile) OutputFileName]
|
|
564
|
+
[-(DB|Debug) DebugModeInfo]
|
|
565
|
+
|
|
566
|
+
Mode options that can be specified for getting group action:
|
|
567
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
568
|
+
values of a given field
|
|
569
|
+
-(RG|REcursiveGroup) - gather subgroups recursively for given group index
|
|
570
|
+
-(WN|WithFileNumber) - gathers and displays HPSS/Web file counts for groups
|
|
571
|
+
|
|
572
|
+
Use Info option -FN (-FieldNames) to specify what group fields to retrieve.
|
|
573
|
+
It defaults to 'IGXTQSW' if option -FN is not provided. Information of all
|
|
574
|
+
available fields is retrieved if -FN ALL is given.
|
|
575
|
+
|
|
576
|
+
Valid group field names and their corresponding Info options:
|
|
577
|
+
|
|
578
|
+
Names Info Options Descriptions
|
|
579
|
+
I -(GI|GroupIndex) group indices
|
|
580
|
+
G -(GN|GroupName) group names, IDs
|
|
581
|
+
X -(PI|ParentIndex) parent group indices
|
|
582
|
+
T -(TI|Title) group titles
|
|
583
|
+
R -(GT|GroupType) group types
|
|
584
|
+
P -(GP|GroupPattern) common file name patterns for each group
|
|
585
|
+
Q -(BF|BackupFlag) Quasar Backup flags
|
|
586
|
+
S -(SP|SavedPath) path relative to saved home directory for groups
|
|
587
|
+
W -(WP|WebPath) path relative to web home directory for groups
|
|
588
|
+
A -(ML|MetaLink) a link to content meta at the group level
|
|
589
|
+
D -(NW|NoteWeb) group description of data on RDA Server
|
|
590
|
+
N -(NI|NoteInternal) group description of internal HPSS data
|
|
591
|
+
|
|
592
|
+
Group information can be retrieved for specified groups per option -GI
|
|
593
|
+
(-GroupIndex) or -GN (-GroupName), or per option -PI (-ParentIndex). Info
|
|
594
|
+
option -GN accepts wildcard input of '%' for matching any number of
|
|
595
|
+
characters.
|
|
596
|
+
|
|
597
|
+
For example, get the group information of d260001 for default fields and
|
|
598
|
+
the file counts for groups 1 and 2
|
|
599
|
+
|
|
600
|
+
dsarch d260001 GG -WN -GI 1 2
|
|
601
|
+
|
|
602
|
+
<<Content of the output>>
|
|
603
|
+
Dataset<=>d260001
|
|
604
|
+
GroupIndex<:>GroupName<:>ParentIndex<:>Title<:>GroupPattern<:>BackupFlag<:>SavedPath<:>WebPath<:>
|
|
605
|
+
1<:>DAILY<:>0<:>DAILY OAFlux Products<:><:>P<:><:>daily<:>
|
|
606
|
+
2<:>MONTHLY<:>0<:>MONTHLY OAFlux Products<:><:>P<:><:>monthly<:>
|
|
607
|
+
2 group records retrieved
|
|
608
|
+
d260001-G1: DS-11094192261 DC-327 WC-327 SS-8909173908 SC-51
|
|
609
|
+
d260001-G2: DS-552780725 DC-522 WC-522 SS-465789806 SC-51
|
|
610
|
+
|
|
611
|
+
|
|
612
|
+
3.3.3 Delete Group Information
|
|
613
|
+
-DG or -DeleteGroup, deletes one or multiple groups from RDADB for given
|
|
614
|
+
group indices of a specified dataset.
|
|
615
|
+
|
|
616
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](DG|DeleteGroup) [Mode Options]
|
|
617
|
+
-(GI|GroupIndex) GroupIndices
|
|
618
|
+
[-(LN|LoginName) LoginAccountName]
|
|
619
|
+
[-(DB|Debug) DebugModeInfo]
|
|
620
|
+
|
|
621
|
+
Mode options that can be specified for deleting group action:
|
|
622
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
623
|
+
information to be set by any one
|
|
624
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
625
|
+
values to speed up reading input file(s)
|
|
626
|
+
-(RT|ResetTGroup) - reset top group index for the HPSS/Web file records
|
|
627
|
+
-(WN|WithFileNumber) - reevaluates and resets HPSS/Web file counts for the
|
|
628
|
+
affected groups
|
|
629
|
+
|
|
630
|
+
The groups to be deleted are identified by group indices given per Info
|
|
631
|
+
option -GI (-GroupIndex) and dataset number per option -DS. This action
|
|
632
|
+
is blocked if there exist any subgroups or data files under the groups to
|
|
633
|
+
be deleted. To delete the groups with files under them, you gather the files
|
|
634
|
+
on Saved or RDA Server per action options, -GS (-GetSavedFile) or -GW
|
|
635
|
+
(-GetWebFile), and set the group indices of the files to 0 or change them to
|
|
636
|
+
other group indices via action -CG (-ChangeGroup), or delete the files
|
|
637
|
+
gathered per action -DL (-Delete); then delete the groups.
|
|
638
|
+
|
|
639
|
+
For example, delete group indices 2 and 3 of d744004
|
|
640
|
+
|
|
641
|
+
dsarch d744004 DG -GI 2 3
|
|
642
|
+
|
|
643
|
+
|
|
644
|
+
3.3.4 Change Group Information
|
|
645
|
+
-CG or -ChangeGroup, changes group indices from original groups given by
|
|
646
|
+
option -OG (-OriginGroup) to new groups specified by option -GI (-GroupIndex)
|
|
647
|
+
for a given dataset.
|
|
648
|
+
|
|
649
|
+
dsarch [-(DS|dataset)] dsnnn.n [-](CG|ChangeGroup) [Mode Options]
|
|
650
|
+
-(OG|OriginGroup) OriginalGroupIndices
|
|
651
|
+
-(GI|GroupIndex) NewGroupIndices
|
|
652
|
+
[-(LN|LoginName) LoginAccountName]
|
|
653
|
+
[-(DB|Debug) DebugModeInfo]
|
|
654
|
+
|
|
655
|
+
Mode options that can be specified for changing group action:
|
|
656
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
657
|
+
information to be set by any one
|
|
658
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
659
|
+
values to speed up reading input file(s)
|
|
660
|
+
-(RT|ResetTGroup) - reset top group index for the HPSS/Web file records
|
|
661
|
+
-(WN|WithFileNumber) - reevaluates and resets HPSS/Web file counts for the
|
|
662
|
+
affected groups
|
|
663
|
+
|
|
664
|
+
A dataset group is identified by its group index. It can only be changed to
|
|
665
|
+
different value via this action -CG and both original group indices per Info
|
|
666
|
+
option -OG (-OriginGroup) and new group indices per Info option -GI
|
|
667
|
+
(-GroupIndex) are present. The Saved and Web File namesassociated to the
|
|
668
|
+
original group indices are changed to new group indices correspondingly. The
|
|
669
|
+
original group indices must exist in RDADB for the given dataset.
|
|
670
|
+
|
|
671
|
+
For example, change group indices 1 and 2, to 2 and 3 of d744004
|
|
672
|
+
|
|
673
|
+
dsarch d744004 CG -OG 1 2 -GI 2 3
|
|
674
|
+
|
|
675
|
+
|
|
676
|
+
3.4 File Actions
|
|
677
|
+
|
|
678
|
+
The saved, Web and Help files are archived and their information are recorded
|
|
679
|
+
into RDADB via actions described in this section. The files themselves, and
|
|
680
|
+
their recorded information can be manipulated via different file actions. The
|
|
681
|
+
file information recorded in RDADB can be viewed for given dataset and further
|
|
682
|
+
specified by other conditions. The file actions included in this section are:
|
|
683
|
+
Set Saved File Information - create or modify information of Saved data files
|
|
684
|
+
into RDADB
|
|
685
|
+
Get Saved File Information - retrieve information of existing saved data files
|
|
686
|
+
recorded in RDADB
|
|
687
|
+
Set Web File Information - create or modify information of data files on RDA
|
|
688
|
+
Server into RDADB
|
|
689
|
+
Get Web File Information - retrieve information of existing data files on RDA
|
|
690
|
+
Server recorded RDADB
|
|
691
|
+
Set Help File Information - create or modify information of document and
|
|
692
|
+
software Help files recorded in RDADB
|
|
693
|
+
Get Help File Information - retrieve Quasar file information recorded in RDADB
|
|
694
|
+
Set Quasar File Information - create or modify information of backup Quasar
|
|
695
|
+
files recorded in RDADB
|
|
696
|
+
Get Quasar File Information - retrieve Help file information recorded in RDADB
|
|
697
|
+
Archive Saved Files - archive saved data files, and record new records
|
|
698
|
+
into RDADB or modify records that are in RDADB
|
|
699
|
+
Archive Web Files - archive web data files, and record new records
|
|
700
|
+
into RDADB or modify records that are in RDADB
|
|
701
|
+
Archive Help Files - archive Help files onto RDA Seb Servers, and
|
|
702
|
+
record new records into RDADB or modify records
|
|
703
|
+
that are in RDADB
|
|
704
|
+
Backup Quasar Files - tar mulitple files from one or more datasets into
|
|
705
|
+
a single file, and backup it onto Globus Quasar
|
|
706
|
+
Server
|
|
707
|
+
Restore Quasar Files - retrieve quasar files identified from given Saved
|
|
708
|
+
and Web files, restore the untarrd Saved and Web
|
|
709
|
+
files for missing ones
|
|
710
|
+
Move Saved/Web/Help/Quasar Files - move physically the Saved, Web, Help and
|
|
711
|
+
Quasar files from one dataset to another,
|
|
712
|
+
or just inside of a single dataset, and
|
|
713
|
+
cross move between Save and Web files
|
|
714
|
+
Delete Saved/Web/Help/Quasar Files - delete one or multiple Saved, Web, Help,
|
|
715
|
+
and Quasar files, and their information
|
|
716
|
+
recorded in RDADB
|
|
717
|
+
|
|
718
|
+
3.4.1 Set Saved File Information
|
|
719
|
+
-SS or -SetSavedFile (Alias: -SetSaved), creates and modifies information of
|
|
720
|
+
the Saved files in RDADB for a given dataset. One or multiple files can be
|
|
721
|
+
processed each time.
|
|
722
|
+
|
|
723
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](SS|SetSavedFile) [Mode Options]
|
|
724
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
725
|
+
[-(ON|OrderNames) OrderNameString]
|
|
726
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
727
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
728
|
+
[-(GN|GroupName) GroupNames]
|
|
729
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
730
|
+
[-(PO|PatternOffset) PatternStringOffset]
|
|
731
|
+
[-(ST|SavedFileType) SavedFileTypes] # Primary, VersionControl, Work, oRiginal
|
|
732
|
+
[-(FS|FileStatus) SavedFileStatus] # Public or Interal file
|
|
733
|
+
[-(LC|Location) StorageLocationFlags] # CGD data; Glade, Object or Both
|
|
734
|
+
[-(DF|DataFormat) DataFormats] # BINARY, ASCII, NetCDF and etc.
|
|
735
|
+
[-(AF|ArchiveFormat) FileArchiveFormats] # tarred, Compressed and etc.
|
|
736
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
737
|
+
[-(SZ|Size) FileSizes]
|
|
738
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
739
|
+
[-(FD|FileDate) DateModified]
|
|
740
|
+
[-(FT|FileTime) TimeModified]
|
|
741
|
+
[-(DE|Description] SavedFileDescriptions]
|
|
742
|
+
[-(LN|LoginName) LoginAccountName]
|
|
743
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
744
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
745
|
+
[-(DB|Debug) DebugModeInfo]
|
|
746
|
+
|
|
747
|
+
Mode options that can be specified for setting Web file action:
|
|
748
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
749
|
+
standard outputs and errors
|
|
750
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
751
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
752
|
+
information to be set by any one
|
|
753
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
754
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
755
|
+
values to speed up reading input file(s)
|
|
756
|
+
-(RD|RemoveDir) - removes empty web directories
|
|
757
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
758
|
+
on Saved Server according to the order they are given per
|
|
759
|
+
option -SF. Reordering filelist can be also accomplished
|
|
760
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
761
|
+
-SW to reorder all the files of given dataset and the
|
|
762
|
+
involved groups
|
|
763
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for data files on
|
|
764
|
+
RDA Server and saves them into RDADB
|
|
765
|
+
|
|
766
|
+
Saved file names must be provided per option -SF (-SavedFile) for this action to
|
|
767
|
+
work, unless Info Option -ON (-OrderNames) is provided to reorder Saved files
|
|
768
|
+
of specified dataset and groups. The number of values passed in following
|
|
769
|
+
Info options should match the number of file names given per option -SF,
|
|
770
|
+
except for options related to dataset information and the Info options that
|
|
771
|
+
are allowed to pass in single values, such as -GI (-GroupIndex), -AF
|
|
772
|
+
(-ArchiveFormat), -DF (-DataFormat), and -ST (-SavedFileType). If a single group
|
|
773
|
+
index is provided for multiple Saved file names, the index value is shared by
|
|
774
|
+
all the given files on RDA Server.
|
|
775
|
+
|
|
776
|
+
Pattern matching is used for identify a group index for a given saved file
|
|
777
|
+
if group index is not given explicitly and Info option -PO (-PatternOffset)
|
|
778
|
+
if present. The Web file name is matched against group pattern values that
|
|
779
|
+
are previously saved in group records via Action -SG (-SetGroup).
|
|
780
|
+
|
|
781
|
+
|
|
782
|
+
3.4.2 Get Saved File Information
|
|
783
|
+
-GS or -GetSavedFile (Alias: -GetSaved), gets information of data files on Saved
|
|
784
|
+
Server for a given dataset and/or specified Saved file names. The list of data
|
|
785
|
+
files can be further specified for given group indices, file archive types,
|
|
786
|
+
and other conditions.
|
|
787
|
+
|
|
788
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(GS|GetSavedFile) [Mode Options]
|
|
789
|
+
[-(FN|FieldNames) FieldNameString]
|
|
790
|
+
[-(ON|OrderNames) OrderNameString]
|
|
791
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
792
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
793
|
+
[-(ST|SavedFileType) SavedFileTypes]
|
|
794
|
+
[-(FS|FileStatus) SavedFileStatus]
|
|
795
|
+
[-(LC|Location) StorageLocationFlags]
|
|
796
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
797
|
+
[-(GN|GroupName) GroupNames]
|
|
798
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
799
|
+
[-(DE|Description) SavedFileDescriptions]
|
|
800
|
+
[(-OF|-OutputFile) OutputFileName]
|
|
801
|
+
[-(DB|Debug) DebugModeInfo]
|
|
802
|
+
|
|
803
|
+
Mode options that can be specified for getting Saved file action:
|
|
804
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
805
|
+
values of a given field
|
|
806
|
+
-(RG|REcursiveGroup) - gather files in subgroups recursively for given group index
|
|
807
|
+
-(RN|RelativeName) - retrieves Saved file names with paths relative to
|
|
808
|
+
saved paths for groups or dataset
|
|
809
|
+
|
|
810
|
+
Use Info option -FN (-FieldNames) to specify what fields of file on Saved
|
|
811
|
+
Server to retrieve. It defaults to 'FINMTHS' if option -FN is not provided;
|
|
812
|
+
information of all available fields is retrieved if -FN ALL is given.
|
|
813
|
+
|
|
814
|
+
Valid Saved file field names and their corresponding Info options:
|
|
815
|
+
|
|
816
|
+
Names Info Options Descriptions
|
|
817
|
+
F -(SF|SavedFile) Saved file names with relative paths
|
|
818
|
+
T -(ST|SavedFileType) Saved file archiving types
|
|
819
|
+
C -(MC|MD5Checksum) Saved file MD5 Checksum
|
|
820
|
+
I -(GI|GroupIndex) group indices for Saved files
|
|
821
|
+
X -(TG|TopGroupIndex) top group indices for Saved files
|
|
822
|
+
V -(VI|VersionIndex) Version control index
|
|
823
|
+
N -(DF|DataFormat) data content formats
|
|
824
|
+
M -(AF|ArchiveFormat) file archive formats
|
|
825
|
+
O -(DO|DisplayOrder) display order indices
|
|
826
|
+
B -(QF|QuasarFile) Quasar backup file names
|
|
827
|
+
Q -(QT|QuasarType) Quasar backyp file types
|
|
828
|
+
L -(LC|Location) Saved file storage location flag
|
|
829
|
+
H -(FS|FileStatus) Saved file archiving status
|
|
830
|
+
S -(SZ|Size) Saved file sizes
|
|
831
|
+
J -(FD|FileDate) date data last modified on
|
|
832
|
+
K -(FT|FileTime) time data last modified at
|
|
833
|
+
D -(DE|Description) Saved file descriptions
|
|
834
|
+
|
|
835
|
+
The dataset number per option -DS (-Dataset) is mandatory unless Saved file
|
|
836
|
+
names are provided. An error message is prompted for specialist to pick one
|
|
837
|
+
dataset number and rerun 'dsarch' if multiple datasets are identified for
|
|
838
|
+
given Saved file information without specified dataset number. Saved file names
|
|
839
|
+
are expanded automatically to include full paths relative to the dataset Saved
|
|
840
|
+
Home Directories if the file names are given without or with incomplete paths.
|
|
841
|
+
|
|
842
|
+
Saved file information can be retrieved for further specified file archive
|
|
843
|
+
types per option -ST (-SavedFileType), groups per option -GI (-GroupIndex) or
|
|
844
|
+
-GN (-GroupName), and Saved file descriptions per -DE (-Description). Info
|
|
845
|
+
options -SF, -GN and -DE accept wildcard inputs of '%' for matching any number
|
|
846
|
+
of characters.
|
|
847
|
+
|
|
848
|
+
|
|
849
|
+
3.4.3 Set Web File Information
|
|
850
|
+
-SW or -SetWebFile (Alias: -SetWeb), creates and modifies information of files
|
|
851
|
+
on RDA Server in RDADB for a given dataset. One or multiple files can be
|
|
852
|
+
processed each time.
|
|
853
|
+
|
|
854
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(SW|SetWebFile) [Mode Options]
|
|
855
|
+
[-(WF|WebFile) webFileNames]
|
|
856
|
+
[-(ON|OrderNames) OrderNameString]
|
|
857
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
858
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
859
|
+
[-(GN|GroupName) GroupNames]
|
|
860
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
861
|
+
[-(PO|PatternOffset) PatternStringOffset]
|
|
862
|
+
[-(WT|WebFileType) WebFileTypes] # Web Data or NCAR Data
|
|
863
|
+
[-(LC|Location) StorageLocationFlags]
|
|
864
|
+
[-(FS|FileStatus) WebFileStatus] # Public or interal file
|
|
865
|
+
[-(DF|DataFormat) DataFormats] # BINARY, ASCII, NetCDF and etc.
|
|
866
|
+
[-(AF|ArchiveFormat) FileArchiveFormats] # tarred, Compressed and etc.
|
|
867
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
868
|
+
[-(SZ|Size) FileSizes]
|
|
869
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
870
|
+
[-(FD|FileDate) DateModified]
|
|
871
|
+
[-(FT|FileTime) TimeModified]
|
|
872
|
+
[-(ML|MetaLink) MetadataLinks]
|
|
873
|
+
[-(TL|ThreddLink) ThreddCatalogLinks]
|
|
874
|
+
[-(DE|Description] WebFileDescriptions]
|
|
875
|
+
[-(LN|LoginName) LoginAccountName]
|
|
876
|
+
[-(AL|AsyncLimit) AsyncProcessLimit]
|
|
877
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
878
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
879
|
+
[-(DB|Debug) DebugModeInfo]
|
|
880
|
+
|
|
881
|
+
Mode options that can be specified for setting Web file action:
|
|
882
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
883
|
+
standard outputs and errors
|
|
884
|
+
-(DX|DeleteXML) - if present, calling 'dcm' to delete file content metadata
|
|
885
|
+
of specified Web files
|
|
886
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
887
|
+
-(GX|GatherXML) - evaluates file content metadata by calling 'gatherxml'
|
|
888
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
889
|
+
information to be set by any one
|
|
890
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
891
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
892
|
+
values to speed up reading input file(s)
|
|
893
|
+
-(RD|RemoveDir) - removes empty web directories
|
|
894
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
895
|
+
on RDA Server according to the order they are given per
|
|
896
|
+
option -WF. Reordering filelist can be also accomplished
|
|
897
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
898
|
+
-SW to reorder all the files of given dataset and the
|
|
899
|
+
involved groups
|
|
900
|
+
-(RS|GXRSOptions) - works with option -GX (-GatherXML) to speed up gatherxml
|
|
901
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for data files on
|
|
902
|
+
RDA Server and saves them into RDADB
|
|
903
|
+
|
|
904
|
+
Web file names must be provided per option -WF (-WebFile) for this action to
|
|
905
|
+
work, unless Info Option -ON (-OrderNames) is provided to reorder Web files
|
|
906
|
+
of specified dataset and groups. The number of values passed in following
|
|
907
|
+
Info options should match the number of file names given per option -WF,
|
|
908
|
+
except for options related to dataset information and the Info options that
|
|
909
|
+
are allowed to pass in single values, such as -GI (-GroupIndex), -AF
|
|
910
|
+
(-ArchiveFormat), -DF (-DataFormat), and -WT (-WebFileType). If a single group
|
|
911
|
+
index is provided for multiple Web file names, the index value is shared by
|
|
912
|
+
all the given files on RDA Server.
|
|
913
|
+
|
|
914
|
+
Pattern matching is used for identify a group index for a given Web file
|
|
915
|
+
if group index is not given explicitly and Info option -PO (-PatternOffset)
|
|
916
|
+
if present. The Web file name is matched against group pattern values that
|
|
917
|
+
are previously saved in group records via Action -SG (-SetGroup).
|
|
918
|
+
|
|
919
|
+
The web file status can be set to either P (public) or I (internal) for data
|
|
920
|
+
type D (data), N (ncar data), O (document) and S (software). For a Web file is
|
|
921
|
+
set to be under an Internal group, its status is automatically forced to I.
|
|
922
|
+
|
|
923
|
+
If a Web file is set to be under an Internal group, its status is forced to I.
|
|
924
|
+
|
|
925
|
+
|
|
926
|
+
3.4.4 Get Web File Information
|
|
927
|
+
-GW or -GetWebFile (Alias: -GetWeb), gets information of data files on RDA
|
|
928
|
+
Server for a given dataset and/or specified Web file names. The list of data
|
|
929
|
+
files can be further specified for given group indices, file archive types,
|
|
930
|
+
and file descriptions.
|
|
931
|
+
|
|
932
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(GW|GetWebFile) [Mode Options]
|
|
933
|
+
[-(FN|FieldNames) FieldNameString]
|
|
934
|
+
[-(ON|OrderNames) OrderNameString]
|
|
935
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
936
|
+
[-(WF|WebFile) WebFileNames]
|
|
937
|
+
[-(WT|WebFileType) WebFileTypes]
|
|
938
|
+
[-(FS|FileStatus) WebFileStatus]
|
|
939
|
+
[-(LC|Location) StorageLocationFlags]
|
|
940
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
941
|
+
[-(GN|GroupName) GroupNames]
|
|
942
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
943
|
+
[-(DE|Description) WebFileDescriptions]
|
|
944
|
+
[(-OF|-OutputFile) OutputFileName]
|
|
945
|
+
[-(DB|Debug) DebugModeInfo]
|
|
946
|
+
|
|
947
|
+
Mode options that can be specified for getting Web file action:
|
|
948
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
949
|
+
values of a given field
|
|
950
|
+
-(RG|REcursiveGroup) - gather files in subgroups recursively for given group index
|
|
951
|
+
-(RN|RelativeName) - retrieves Web file names with paths relative to
|
|
952
|
+
saved Web paths for groups or dataset
|
|
953
|
+
|
|
954
|
+
Use Info option -FN (-FieldNames) to specify what fields of file on RDA
|
|
955
|
+
Server to retrieve. It defaults to 'FINMTHS' if option -FN is not provided;
|
|
956
|
+
information of all available fields is retrieved if -FN ALL is given.
|
|
957
|
+
|
|
958
|
+
Valid Web file field names and their corresponding Info options:
|
|
959
|
+
|
|
960
|
+
Names Info Options Descriptions
|
|
961
|
+
F -(WF|WebFile) Web file names with relative paths
|
|
962
|
+
T -(WT|WebFileType) Web file archiving types
|
|
963
|
+
C -(MC|MD5Checksum) Web file MD5 Checksum
|
|
964
|
+
I -(GI|GroupIndex) group indices for Web files
|
|
965
|
+
X -(TG|TopGroupIndex) top group indices for Web files
|
|
966
|
+
V -(VI|VersionIndex) Version control index
|
|
967
|
+
M -(AF|ArchiveFormat) file archive formats
|
|
968
|
+
N -(DF|DataFormat) data content formats
|
|
969
|
+
O -(DO|DisplayOrder) display order indices
|
|
970
|
+
B -(QF|QuasarFile) Quasar backup file names
|
|
971
|
+
Q -(QT|QuasarType) Quasar backyp fie types
|
|
972
|
+
L -(LC|Location) Web file storage location flag
|
|
973
|
+
H -(FS|FileStatus) Web file archiving status
|
|
974
|
+
S -(SZ|Size) Web file sizes
|
|
975
|
+
J -(FD|FileDate) date data last modified on
|
|
976
|
+
K -(FT|FileTime) time data last modified at
|
|
977
|
+
A -(ML|MetaLink) a link to content meta at the file level
|
|
978
|
+
E -(TL|ThreddLink) a link to Thredd Catalog page at the file level
|
|
979
|
+
D -(DE|Description) Web file descriptions
|
|
980
|
+
|
|
981
|
+
The dataset number per option -DS (-Dataset) is mandatory unless Web file
|
|
982
|
+
names are provided. An error message is prompted for specialist to pick one
|
|
983
|
+
dataset number and rerun 'dsarch' if multiple datasets are identified for
|
|
984
|
+
given Web file information without specified dataset number. Web file names
|
|
985
|
+
are expanded automatically to include full paths relative to the dataset Web
|
|
986
|
+
Home Directories if the file names are given without or with incomplete paths.
|
|
987
|
+
|
|
988
|
+
Web file information can be retrieved for further specified file archive
|
|
989
|
+
types per option -WT (-WebFileType), groups per option -GI (-GroupIndex) or
|
|
990
|
+
-GN (-GroupName), and Web file descriptions per -DE (-Description). Info
|
|
991
|
+
options -WF, -GN and -DE accept wildcard inputs of '%' for matching any number
|
|
992
|
+
of characters.
|
|
993
|
+
|
|
994
|
+
For example, get the Web file information of d540000 for default fields;
|
|
995
|
+
for Data type and wildcard matching '200' in Web file names
|
|
996
|
+
|
|
997
|
+
dsarch d540000 GW -WT D -WF %200%
|
|
998
|
+
|
|
999
|
+
<<Content of the output>>
|
|
1000
|
+
Dataset<=>d540000
|
|
1001
|
+
WebFile<:>GroupIndex<:>DataFormat<:>ArchiveFormat<:>WebFileType<:>Size<:>
|
|
1002
|
+
IMMA_2.4_1998_2000.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>1283094016<:>
|
|
1003
|
+
IMMA_2.4_2001_2002.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>1006424064<:>
|
|
1004
|
+
IMMA_2.4_2003_2004.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>1149672960<:>
|
|
1005
|
+
IMMA_2.4_2005.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>951742976<:>
|
|
1006
|
+
IMMA_2.4_2006.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>1119821824<:>
|
|
1007
|
+
IMMA_2.4_2007.tar<:>37<:>IMMA<:>Z.TAR<:>D<:>656820224<:>
|
|
1008
|
+
|
|
1009
|
+
3.4.5 Set Help File Information
|
|
1010
|
+
-SH or -SetHelpFile (Alias: -SetHelp), creates and modifies information of Help
|
|
1011
|
+
files, Documents and Software, in RDADB for a given dataset. One or multiple
|
|
1012
|
+
files can be processed each time.
|
|
1013
|
+
|
|
1014
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(SH|SetHelpFile) [Mode Options]
|
|
1015
|
+
[-(HF|HelpFile) HelpFileNames]
|
|
1016
|
+
[-(WU|WebURL) RemoteFileWebURL]
|
|
1017
|
+
[-(SR|Source) ContributionSource]
|
|
1018
|
+
[-(ON|OrderNames) OrderNameString]
|
|
1019
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
1020
|
+
[-(HT|HelpFileType) HelpFileTypes] # Document or Software
|
|
1021
|
+
[-(FS|FileStatus) HelpFileStatus] # Public or Interal file
|
|
1022
|
+
[-(LC|Location) StorageLocationFlags] # Glade, Object, Both or Remote
|
|
1023
|
+
[-(DF|DataFormat) DataFormats] # Python, PDF, and etc.
|
|
1024
|
+
[-(AF|ArchiveFormat) FileArchiveFormats] # tarred, Compressed and etc.
|
|
1025
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
1026
|
+
[-(SZ|Size) FileSizes]
|
|
1027
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
1028
|
+
[-(ID|InitialDate) HelpFileInitialDate] # Initial Publish/Release Date
|
|
1029
|
+
[-(FD|FileDate) DateModified]
|
|
1030
|
+
[-(FT|FileTime) TimeModified]
|
|
1031
|
+
[-(DE|Description] SavedFileDescriptions]
|
|
1032
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1033
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1034
|
+
|
|
1035
|
+
Mode options that can be specified for setting Web file action:
|
|
1036
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1037
|
+
standard outputs and errors
|
|
1038
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1039
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1040
|
+
information to be set by any one
|
|
1041
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1042
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1043
|
+
values to speed up reading input file(s)
|
|
1044
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
1045
|
+
according to the order they are provided
|
|
1046
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for Helpfiles and
|
|
1047
|
+
saves them into RDADB
|
|
1048
|
+
|
|
1049
|
+
Help file names must be provided per option -HF (-HelpFile) for this action to
|
|
1050
|
+
work, unless Info Option -ON (-OrderNames) is provided to reorder Help files
|
|
1051
|
+
of specified dataset and groups. The number of values passed in following
|
|
1052
|
+
Info options should match the number of file names given per option -HF,
|
|
1053
|
+
except for options related to dataset information and the Info options that
|
|
1054
|
+
are allowed to pass in single values, such as -AF (-ArchiveFormat), -DF
|
|
1055
|
+
(-DataFormat), and -HT (-HelpFileType).
|
|
1056
|
+
|
|
1057
|
+
For a Help file located remotely, such as on github, a Help file record can be
|
|
1058
|
+
added into RDADB via action -SH and a full URL link is provided via option -WU.
|
|
1059
|
+
|
|
1060
|
+
|
|
1061
|
+
3.4.6 Get Help File Information
|
|
1062
|
+
-GH or -GetHelpFile (Alias: -GetHelp), gets information of Help files, Documents
|
|
1063
|
+
and Softwares, for a given dataset and/or specified Help file names. The file
|
|
1064
|
+
list can be further specified for given file types, and other
|
|
1065
|
+
conditions.
|
|
1066
|
+
|
|
1067
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(GH|GetHelpFile) [Mode Options]
|
|
1068
|
+
[-(FN|FieldNames) FieldNameString]
|
|
1069
|
+
[-(ON|OrderNames) OrderNameString]
|
|
1070
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
1071
|
+
[-(HF|HelpFile) HelpFileNames]
|
|
1072
|
+
[-(HT|HelpFileType) HelpFileTypes] # Document or Software
|
|
1073
|
+
[-(FS|FileStatus) SavedFileStatus]
|
|
1074
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1075
|
+
[-(SR|Source) ContributionSource]
|
|
1076
|
+
[-(DE|Description) SavedFileDescriptions]
|
|
1077
|
+
[(-OF|-OutputFile) OutputFileName]
|
|
1078
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1079
|
+
|
|
1080
|
+
Mode option that can be specified for getting Help file action:
|
|
1081
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
1082
|
+
values of a given field
|
|
1083
|
+
|
|
1084
|
+
Use Info option -FN (-FieldNames) to specify what fields of Help files to
|
|
1085
|
+
retrieve. It defaults to 'FINMTHSU' if option -FN is not provided;
|
|
1086
|
+
information of all available fields is retrieved if -FN ALL is given.
|
|
1087
|
+
|
|
1088
|
+
Valid Saved file field names and their corresponding Info options:
|
|
1089
|
+
|
|
1090
|
+
Names Info Options Descriptions
|
|
1091
|
+
F -(HF|HelpFile) Help file names
|
|
1092
|
+
N -(DF|DataFormat) file content formats
|
|
1093
|
+
M -(AF|ArchiveFormat) file archive formats
|
|
1094
|
+
O -(DO|DisplayOrder) display order indices
|
|
1095
|
+
T -(HT|HelpFileType) Help file archiving types
|
|
1096
|
+
H -(FS|FileStatus) Help file archiving status
|
|
1097
|
+
L -(LC|Location) Help file storage location flag
|
|
1098
|
+
S -(SZ|Size) Help file sizes
|
|
1099
|
+
R -(SR|Source) Help file contribution source
|
|
1100
|
+
I -(ID|InitialDate) Date initially Published/Released
|
|
1101
|
+
J -(FD|FileDate) date last modified on
|
|
1102
|
+
K -(FT|FileTime) time last modified at
|
|
1103
|
+
U -(WU|WebURL' a full URL for a remote help file
|
|
1104
|
+
D -(DE|Description) file descriptions
|
|
1105
|
+
|
|
1106
|
+
The dataset number per option -DS (-Dataset) is mandatory unless Help file
|
|
1107
|
+
names are provided. An error message is prompted for specialist to pick one
|
|
1108
|
+
dataset number and rerun 'dsarch' if multiple datasets are identified for
|
|
1109
|
+
given Help file information without specified dataset number.
|
|
1110
|
+
|
|
1111
|
+
Help file information can be retrieved for further specified file archive
|
|
1112
|
+
types per option -HT (-HelpFileType), and file descriptions per -DE
|
|
1113
|
+
(-Description). Info options -HF and -DE accept wildcard inputs of '%'
|
|
1114
|
+
for matching any number of characters.
|
|
1115
|
+
|
|
1116
|
+
3.4.7 Set Quasar File Information
|
|
1117
|
+
-SQ or -SetQuasarFile (Alias: -SetQuasar), modifies information of the Quasar files
|
|
1118
|
+
in RDADB for a given dataset. One or multiple files can be processed each time.
|
|
1119
|
+
|
|
1120
|
+
dsarch [-(DS|Dataset)] dsnnn.n [-](SS|SetSavedFile) [Mode Options]
|
|
1121
|
+
[-(QF|QuasarFile) QuasarFileNames]
|
|
1122
|
+
[-(ON|OrderNames) OrderNameString]
|
|
1123
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
1124
|
+
[-(QT|QuasarFileType) QuasarFileTypes] # Backup or backup&Drdata
|
|
1125
|
+
[-(BS|BackupStatus) QuasarBackupFileStatus] # New-record, Tarred, or Achived
|
|
1126
|
+
[-(DF|DataFormat) FileContentFormat]
|
|
1127
|
+
[-(AF|ArchiveFormat) FileArchiveFormat]
|
|
1128
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
1129
|
+
[-(SZ|Size) FileSizes]
|
|
1130
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
1131
|
+
[-(FD|FileDate) DateModified]
|
|
1132
|
+
[-(FT|FileTime) TimeModified]
|
|
1133
|
+
[-(DE|Description] SavedFileDescriptions]
|
|
1134
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1135
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1136
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1137
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1138
|
+
|
|
1139
|
+
Mode options that can be specified for setting Web file action:
|
|
1140
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1141
|
+
standard outputs and errors
|
|
1142
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1143
|
+
information to be set by any one
|
|
1144
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1145
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1146
|
+
values to speed up reading input file(s)
|
|
1147
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
1148
|
+
on Quasar Server according to the order they are given per
|
|
1149
|
+
option -QF. Reordering filelist can be also accomplished
|
|
1150
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
1151
|
+
-SQ to reorder all the files of given dataset
|
|
1152
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for data files on
|
|
1153
|
+
RDA Server and saves them into RDADB
|
|
1154
|
+
|
|
1155
|
+
Quasar file names must be provided per option -QF (-QuasarFile) for this action
|
|
1156
|
+
to work, unless Info Option -ON (-OrderNames) is provided to reorder the files
|
|
1157
|
+
of specified dataset. The number of values passed in following Info options
|
|
1158
|
+
should match the number of file names given per option -QF, except for options
|
|
1159
|
+
related to dataset information and the Info options that are allowed to pass in
|
|
1160
|
+
as single values, such as -AF (-ArchiveFormat), -DF (-DataFormat), and -QT
|
|
1161
|
+
(-QuasarFileType).
|
|
1162
|
+
|
|
1163
|
+
|
|
1164
|
+
3.4.8 Get Quasar File Information
|
|
1165
|
+
-GQ or -GetQuasarFile (Alias: -GetQuasar), gets information of backup files on
|
|
1166
|
+
Quasar Server for a given dataset and/or specified Quasar file names. The file
|
|
1167
|
+
list can be further specified for file archive types, and other conditions.
|
|
1168
|
+
|
|
1169
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(GQ|GetQuasarFile) [Mode Options]
|
|
1170
|
+
[-(FN|FieldNames) FieldNameString]
|
|
1171
|
+
[-(ON|OrderNames) OrderNameString]
|
|
1172
|
+
[-(OB|OrderBy) OrderByPattern]
|
|
1173
|
+
[-(QF|QuasarFile) QuasarFileNames]
|
|
1174
|
+
[-(QT|QuasarFileType) QuasarFileTypes]
|
|
1175
|
+
[-(FS|FileStatus) QuasarFileStatus]
|
|
1176
|
+
[-(DE|Description) QuasarFileDescriptions]
|
|
1177
|
+
[(-OF|-OutputFile) OutputFileName]
|
|
1178
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1179
|
+
|
|
1180
|
+
Mode options that can be specified for getting Saved file action:
|
|
1181
|
+
-(FO|FormatOutput) - format the column output with a same width for all
|
|
1182
|
+
values of a given field
|
|
1183
|
+
|
|
1184
|
+
Use Info option -FN (-FieldNames) to specify what fields of file on Saved
|
|
1185
|
+
Server to retrieve. It defaults to 'FNMTHS' if option -FN is not provided;
|
|
1186
|
+
information of all available fields is retrieved if -FN ALL is given.
|
|
1187
|
+
|
|
1188
|
+
Valid Saved file field names and their corresponding Info options:
|
|
1189
|
+
|
|
1190
|
+
Names Info Options Descriptions
|
|
1191
|
+
F -(QF|QuasarFile) Quasar file names with relative paths
|
|
1192
|
+
T -(QT|QuasarFileType) Quasar file archiving types
|
|
1193
|
+
C -(MC|MD5Checksum) Saved file MD5 Checksum
|
|
1194
|
+
N -(DF|DataFormat) data content formats
|
|
1195
|
+
M -(AF|ArchiveFormat) file archive formats
|
|
1196
|
+
O -(DO|DisplayOrder) display order indices
|
|
1197
|
+
B -(QF|QuasarFile) Quasar backup file names
|
|
1198
|
+
Q -(QT|QuasarType) Quasar backyp fie types
|
|
1199
|
+
H -(FS|FileStatus) Quasar file archiving status
|
|
1200
|
+
S -(SZ|Size) Quasar file sizes
|
|
1201
|
+
J -(FD|FileDate) date data last modified on
|
|
1202
|
+
K -(FT|FileTime) time data last modified at
|
|
1203
|
+
D -(DE|Description) Quasar file descriptions
|
|
1204
|
+
|
|
1205
|
+
The dataset number per option -DS (-Dataset) is mandatory unless Quasar file
|
|
1206
|
+
names are provided. An error message is prompted for specialist to pick one
|
|
1207
|
+
dataset number and rerun 'dsarch' if multiple datasets are identified for
|
|
1208
|
+
given Quasar file information without specified dataset number. Quasar file
|
|
1209
|
+
names are expanded automatically to include full paths relative to the dataset
|
|
1210
|
+
Home Directories if the file names are given without or with incomplete paths.
|
|
1211
|
+
|
|
1212
|
+
Quasar file information can be retrieved for further specified file archive
|
|
1213
|
+
types per option -QT (-QuasarFileType) and Quasar file descriptions per -DE
|
|
1214
|
+
(-Description). Info options -QF and -DE accept wildcard inputs of '%' for
|
|
1215
|
+
matching any number of characters.
|
|
1216
|
+
|
|
1217
|
+
3.4.9 Archive Saved Files
|
|
1218
|
+
-AS or -ArchiveSavedFile (Alias: -ArchiveSaved|-ArchSaved|-ArchSavedFile), archives
|
|
1219
|
+
local data files as saved files and saves information of the archived files
|
|
1220
|
+
into RDADB for a given dataset. One or multiple local files can be archived
|
|
1221
|
+
each time.
|
|
1222
|
+
|
|
1223
|
+
dsarch [-(DS|dataset)] dsnnn.n -(AS|ArchiveSavedFile) [Mode Options]
|
|
1224
|
+
[-(LF|LocalFile) LocalFileNames] # mandatory unless -LD or -LL present
|
|
1225
|
+
[-(LD|LocalDirectory) LocalDirectoryName] # directory for local file names
|
|
1226
|
+
[-(LL|LocalFileList) LocalFileListName] # contains a list of local file names
|
|
1227
|
+
[-(SF|SavedFile) SavedFileNames] # default to -LF
|
|
1228
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
1229
|
+
[-(GN|GroupName) GroupNames]
|
|
1230
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
1231
|
+
[-(PO|PatternOffset) PatternStringOffset]
|
|
1232
|
+
[-(ST|SavedFileType) SavedFileTypes]
|
|
1233
|
+
[-(FS|FileStatus) SavedFileStatus] # P-Public or I-Internal
|
|
1234
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1235
|
+
[-(DF|DataFormat) DataFormat] # BINARY, ASCII, NetCDF and etc.
|
|
1236
|
+
[-(AF|ArchiveFormat) FileArchiveFormat] # tarred, compressed and etc.
|
|
1237
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
1238
|
+
[-(SZ|Size) FileSizes]
|
|
1239
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
1240
|
+
[-(DE|Description] SavedFileDescriptions]
|
|
1241
|
+
[-(VS|ValidSize) MinSizeForValidFile]
|
|
1242
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1243
|
+
[-(AL|AsyncLimit) AsyncProcessLimit]
|
|
1244
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1245
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1246
|
+
[-(DD|DeleteDir) DeleteLocalDirLevel]
|
|
1247
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1248
|
+
|
|
1249
|
+
Mode options that can be specified for archive Saved file action:
|
|
1250
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1251
|
+
standard outputs and errors
|
|
1252
|
+
-(CL|CleanLocal) - cleans local files after they are all archived onto
|
|
1253
|
+
RDA Server successfully
|
|
1254
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1255
|
+
-(GZ|GMTZone) - uses GMT dates/times as controlling times
|
|
1256
|
+
-(KP|KeepLocalPath) - retains the path names of local files as relative paths
|
|
1257
|
+
to data home directory on RDA Server
|
|
1258
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1259
|
+
information to be set by any one
|
|
1260
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1261
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1262
|
+
values to speed up reading input file(s)
|
|
1263
|
+
-(OE|OverrideExist) - overrides existing Saved files
|
|
1264
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
1265
|
+
on RDA Server according to the order they are given per
|
|
1266
|
+
option -SF. Reordering filelist can be also accomplished
|
|
1267
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
1268
|
+
-SS to reorder all the files of given dataset and the
|
|
1269
|
+
involved groups
|
|
1270
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for data files on
|
|
1271
|
+
RDA Server and saves them into RDADB
|
|
1272
|
+
-(UZ|UpzipData) - uncompresses online data with one of the uncompressing
|
|
1273
|
+
utilities, 'gunzip', 'uncompress', 'unzip' or 'bunzip',
|
|
1274
|
+
according to the file name extensions of ".gz', '.Z',
|
|
1275
|
+
'.zip' or '.bz2', respectively
|
|
1276
|
+
-(XC|CrossCopy) - copies files from existing Saved/Object data files
|
|
1277
|
+
to missing Object/Saved files; information of the Saved
|
|
1278
|
+
files in RDADB is used for missing data copies
|
|
1279
|
+
-(XM|CrossMove) - moves files from existing Saved/Object data files
|
|
1280
|
+
to missing Object/Saved files; information of the Saved
|
|
1281
|
+
files in RDADB is used for missing data copies
|
|
1282
|
+
-(ZD|ZipData) - compresses online data with one of the compressing
|
|
1283
|
+
utilities, 'gzip', 'compress', 'zip' or 'bzip',
|
|
1284
|
+
according to the archiving format 'GZ', 'Z', 'ZIP' or
|
|
1285
|
+
'BZ2' provided per option -AF (-ARchiveFormat),
|
|
1286
|
+
respectively
|
|
1287
|
+
|
|
1288
|
+
One or multiple local file names can be provided per option -LF (-LocalFile)
|
|
1289
|
+
for this action to work. If option -LF is given on command line, '*' and '?'
|
|
1290
|
+
can be used in local file names for wildcard match. File names must include
|
|
1291
|
+
relative or absolute paths if local data files are not located in the
|
|
1292
|
+
directory where 'dsarch' is started. The local file names are gathered
|
|
1293
|
+
recursively following the sub-directory tree if a local directory per Info
|
|
1294
|
+
option -LD (-LocalDirectory) is present.
|
|
1295
|
+
|
|
1296
|
+
Local file names are used as default for Saved file names unless they are
|
|
1297
|
+
explicitly given per option -SF (-SavedFile). Local files are archived onto
|
|
1298
|
+
Saved Server under the dataset home directory as default, unless relative or
|
|
1299
|
+
absolute paths are given per option -SP (-SavedPath) or Web path information for
|
|
1300
|
+
the given dataset, and groups if specified, previously saved in RDADB. The
|
|
1301
|
+
original local file names, including paths, are used if Mode option -KP
|
|
1302
|
+
(-KeepLocalPath) is specified. Option -SP is not allowed if Mode option -KP
|
|
1303
|
+
is given.
|
|
1304
|
+
|
|
1305
|
+
Group information for data files to be archived are given per option -GI
|
|
1306
|
+
(-GroupIndex) or -GN (-GroupName). Pattern matching is used for identify a
|
|
1307
|
+
group index for a given Web file if group index is not given explicitly, and
|
|
1308
|
+
the file record not exists in RDADB yet or Info option -PO (-PatternOffset)
|
|
1309
|
+
if present. The Web or local file name is matched against group pattern values
|
|
1310
|
+
that are previously saved in group records via Action -SG (-SetGroup). If no
|
|
1311
|
+
group index is provided and no group pattern matching, an index value of 0
|
|
1312
|
+
is defaulted for a data file to indicate no group.
|
|
1313
|
+
|
|
1314
|
+
The number of values passed in per Info options must be the same as the number
|
|
1315
|
+
of local file names provided, except for the Info options that are allowed to
|
|
1316
|
+
pass in single values, such as options -GI (-GroupIndex), -DF (-DataFormat),
|
|
1317
|
+
-AF (-ArchiveFormat), -ST (-SavedFileType), and -SP (-SavedPath). If a signle
|
|
1318
|
+
group index is provided for multiple given local file names, the group index
|
|
1319
|
+
is shared by all the given data files.
|
|
1320
|
+
|
|
1321
|
+
The location flag is provided via option -LC (-Location). The defualt value
|
|
1322
|
+
is 'G' for archiving a saved file onto Glade Disk only.
|
|
1323
|
+
|
|
1324
|
+
3.4.10 Archive Web Files
|
|
1325
|
+
-AW or -ArchiveWebFile (Alias: -ArchiveWeb|-ArchWeb|-ArchWebFile), archives
|
|
1326
|
+
local data files onto RDA Server and saves information of the archived files
|
|
1327
|
+
into RDADB for a given dataset. One or multiple local files can be archived
|
|
1328
|
+
each time.
|
|
1329
|
+
|
|
1330
|
+
dsarch [-(DS|dataset)] dsnnn.n -(AW|ArchiveWebFile) [Mode Options]
|
|
1331
|
+
[-(LF|LocalFile) LocalFileNames] # mandatory unless -LD or -LL present
|
|
1332
|
+
[-(LD|LocalDirectory) LocalDirectoryName] # directory for local file names
|
|
1333
|
+
[-(LL|LocalFileList) LocalFileListName] # contains a list of local file names
|
|
1334
|
+
[-(WF|WebFile) webFileNames] # default to -LF
|
|
1335
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
1336
|
+
[-(TG|TopGroupIndex) TopGroupIndices]
|
|
1337
|
+
[-(GN|GroupName) GroupNames]
|
|
1338
|
+
[-(PO|PatternOffset) PatternStringOffset]
|
|
1339
|
+
[-(WT|WebFileType) WebFileTypes] # Data, Ncar Data
|
|
1340
|
+
[-(FS|FileStatus) WebFileStatus] # P-Public or I-Internal
|
|
1341
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1342
|
+
[-(DF|DataFormat) DataFormat] # BINARY, ASCII, NetCDF and etc.
|
|
1343
|
+
[-(AF|ArchiveFormat) FileArchiveFormat] # tarred, compressed and etc.
|
|
1344
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
1345
|
+
[-(SZ|Size) FileSizes]
|
|
1346
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
1347
|
+
[-(DE|Description] WebFileDescriptions]
|
|
1348
|
+
[-(VS|ValidSize) MinSizeForValidFile]
|
|
1349
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1350
|
+
[-(AL|AsyncLimit) AsyncProcessLimit]
|
|
1351
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1352
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1353
|
+
[-(DD|DeleteDir) DeleteLocalDirLevel]
|
|
1354
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1355
|
+
|
|
1356
|
+
Mode options that can be specified for archive web file action:
|
|
1357
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1358
|
+
standard outputs and errors
|
|
1359
|
+
-(CL|CleanLocal) - cleans local files after they are all archived onto
|
|
1360
|
+
RDA Server successfully
|
|
1361
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1362
|
+
-(GX|GatherXML) - evaluates file content metadata from local file
|
|
1363
|
+
directly by calling 'gatherxml'
|
|
1364
|
+
-(GZ|GMTZone) - uses GMT dates/times as controlling times
|
|
1365
|
+
-(KP|KeepLocalPath) - retains the path names of local files as relative paths
|
|
1366
|
+
to data home directory on RDA Server
|
|
1367
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1368
|
+
information to be set by any one
|
|
1369
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1370
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1371
|
+
values to speed up reading input file(s)
|
|
1372
|
+
-(OE|OverrideExist) - overrides existing web files
|
|
1373
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
1374
|
+
on RDA Server according to the order they are given per
|
|
1375
|
+
option -SF. Reordering filelist can be also accomplished
|
|
1376
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
1377
|
+
-SS to reorder all the files of given dataset and the
|
|
1378
|
+
involved groups
|
|
1379
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for data files on
|
|
1380
|
+
Saved Server and saves them into RDADB
|
|
1381
|
+
-(UZ|UpzipData) - uncompresses online data with one of the uncompressing
|
|
1382
|
+
utilities, 'gunzip', 'uncompress', 'unzip' or 'bunzip',
|
|
1383
|
+
according to the file name extensions of ".gz', '.Z',
|
|
1384
|
+
'.zip' or '.bz2', respectively
|
|
1385
|
+
-(XC|CrossCopy) - copies files from existing web/Object data files
|
|
1386
|
+
to missing Object/Web files; information of the Web files
|
|
1387
|
+
saved in RDADB is used for missing data copies
|
|
1388
|
+
-(XM|CrossMove) - moves files from existing web/Object data files
|
|
1389
|
+
to missing Object/Web files; information of the Web files
|
|
1390
|
+
saved in RDADB is used for missing data copies
|
|
1391
|
+
-(ZD|ZipData) - compresses online data with one of the compressing
|
|
1392
|
+
utilities, 'gzip', 'compress', 'zip' or 'bzip',
|
|
1393
|
+
according to the archiving format 'GZ', 'Z', 'ZIP' or
|
|
1394
|
+
'BZ2' provided per option -AF (-ARchiveFormat),
|
|
1395
|
+
respectively
|
|
1396
|
+
|
|
1397
|
+
One or multiple local file names can be provided per option -LF (-LocalFile)
|
|
1398
|
+
for this action to work. If option -LF is given on command line, '*' and '?'
|
|
1399
|
+
can be used in local file names for wildcard match. File names must include
|
|
1400
|
+
relative or absolute paths if local data files are not located in the
|
|
1401
|
+
directory where 'dsarch' is started. The local file names are gathered
|
|
1402
|
+
recursively following the sub-directory tree if a local directory per Info
|
|
1403
|
+
option -LD (-LocalDirectory) is present.
|
|
1404
|
+
|
|
1405
|
+
Local file names are used as default for Web file names unless they are
|
|
1406
|
+
explicitly given per option -WF (-WebFile). Local files are archived onto
|
|
1407
|
+
RDA Server under the dataset home directory as default, unless relative or
|
|
1408
|
+
absolute paths are given per option -WP (-WebPath) or Web path information for
|
|
1409
|
+
the given dataset, and groups if specified, previously saved in RDADB. The
|
|
1410
|
+
original local file names, including paths, are used if Mode option -KP
|
|
1411
|
+
(-KeepLocalPath) is specified. Option -WP is not allowed if Mode option -KP
|
|
1412
|
+
is given.
|
|
1413
|
+
|
|
1414
|
+
Group information for data files to be archived are given per option -GI
|
|
1415
|
+
(-GroupIndex) or -GN (-GroupName). Pattern matching is used for identify a
|
|
1416
|
+
group index for a given Web file if group index is not given explicitly, and
|
|
1417
|
+
the file record not exists in RDADB yet or Info option -PO (-PatternOffset)
|
|
1418
|
+
if present. The Web or local file name is matched against group pattern values
|
|
1419
|
+
that are previously saved in group records via Action -SG (-SetGroup). If no
|
|
1420
|
+
group index is provided and no group pattern matching, an index value of 0
|
|
1421
|
+
is defaulted for a data file to indicate no group.
|
|
1422
|
+
|
|
1423
|
+
A web file can be archived as either P (public) or I (internal) for any data type
|
|
1424
|
+
as D (data) or N (ncar data). For a Web file is archived to be under an Internal
|
|
1425
|
+
group, its status is automatically forced to I.
|
|
1426
|
+
|
|
1427
|
+
The number of values passed in per Info options must be the same as the number
|
|
1428
|
+
of local file names provided, except for the Info options that are allowed to
|
|
1429
|
+
pass in single values, such as options -GI (-GroupIndex), -DF (-DataFormat),
|
|
1430
|
+
-AF (-ArchiveFormat), -WT (-WebFileType), and -WP (-WebPath). If a single
|
|
1431
|
+
group index is provided for multiple given local file names, the group index
|
|
1432
|
+
is shared by all the given data files.
|
|
1433
|
+
|
|
1434
|
+
The location flag is provided via option -LC (-Location). The defualt value
|
|
1435
|
+
is 'G' for archiving a web file onto Glade Disk only.
|
|
1436
|
+
|
|
1437
|
+
3.4.11 Archive Help Files
|
|
1438
|
+
-AH or -ArchiveHelpFile (Alias: -ArchiveHelp|-ArchHelp|-ArchHelpFile),
|
|
1439
|
+
archives local files as files and saved information of the archived files
|
|
1440
|
+
into RDADB for a given dataset. One or multiple local files can be archived
|
|
1441
|
+
each time.
|
|
1442
|
+
|
|
1443
|
+
dsarch [-(DS|dataset)] dsnnn.n -(AH|ArchiveHelpFile) [Mode Options]
|
|
1444
|
+
-(LF|LocalFile) LocalFileNames
|
|
1445
|
+
[-(HF|HelpFile) HelpFileNames] # default to -LF
|
|
1446
|
+
[-(HT|HelpFileType) HelpFileTypes]
|
|
1447
|
+
[-(FS|FileStatus) HelpFileStatus] # P-Public or I-Internal
|
|
1448
|
+
[-(DF|DataFormat) FileContentFormat] # PDF, Python and etc.
|
|
1449
|
+
[-(AF|ArchiveFormat) FileArchiveFormat] # tarred, compressed and etc.
|
|
1450
|
+
[-(DO|DisplayOrder] DisplayOrderIndices]
|
|
1451
|
+
[-(SZ|Size) FileSizes]
|
|
1452
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1453
|
+
[-(MC|MD5Checksum) MD5ChecksumValues]
|
|
1454
|
+
[-(DE|Description] SavedFileDescriptions]
|
|
1455
|
+
[-(VS|ValidSize) MinSizeForValidFile]
|
|
1456
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1457
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1458
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1459
|
+
[-(DD|DeleteDir) DeleteLocalDirLevel]
|
|
1460
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1461
|
+
|
|
1462
|
+
Mode options that can be specified for archive help file action:
|
|
1463
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1464
|
+
standard outputs and errors
|
|
1465
|
+
-(CL|CleanLocal) - cleans local files after they are all archived onto
|
|
1466
|
+
RDA Web Server successfully
|
|
1467
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1468
|
+
-(GZ|GMTZone) - uses GMT dates/times as controlling times
|
|
1469
|
+
-(KP|KeepLocalPath) - retains the path names of local files as relative paths
|
|
1470
|
+
to data home directory on RDA Server
|
|
1471
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1472
|
+
information to be set by any one
|
|
1473
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1474
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1475
|
+
values to speed up reading input file(s)
|
|
1476
|
+
-(OE|OverrideExist) - overrides existing web files
|
|
1477
|
+
-(RO|ResetOrder) - resets the ordering indices of the list of file names
|
|
1478
|
+
on RDA Server according to the order they are given per
|
|
1479
|
+
option -HF. Reordering filelist can be also accomplished
|
|
1480
|
+
by giving option -ON (-OrderNames), which tells Action
|
|
1481
|
+
-SH to reorder all the files of given dataset and the
|
|
1482
|
+
involved groups
|
|
1483
|
+
-(SC|SetChecksum) - evaluates the md5 checksum values for the help files
|
|
1484
|
+
and saves them into RDADB
|
|
1485
|
+
|
|
1486
|
+
One or multiple local file names can be provided per option -LF (-LocalFile)
|
|
1487
|
+
for this action to work. If option -LF is given on command line, '*' and '?'
|
|
1488
|
+
can be used in local file names for wildcard match. File names must include
|
|
1489
|
+
relative or absolute paths if local data files are not located in the
|
|
1490
|
+
directory where 'dsarch' is started. The local file names are gathered
|
|
1491
|
+
recursively following the sub-directory tree if a local directory per Info
|
|
1492
|
+
option -LD (-LocalDirectory) is present.
|
|
1493
|
+
|
|
1494
|
+
Local file names are used as default for Saved file names unless they are
|
|
1495
|
+
explicitly given per option -SF (-SavedFile). The original local file names,
|
|
1496
|
+
including paths, are used if Mode option -KP (-KeepLocalPath) is specified.
|
|
1497
|
+
|
|
1498
|
+
The number of values passed in per Info options must be the same as the number
|
|
1499
|
+
of local file names provided, except for the Info options that are allowed to
|
|
1500
|
+
pass in single values, such as options -DF (-DataFormat), -AF (-ArchiveFormat),
|
|
1501
|
+
and -HT (-HelpFileType).
|
|
1502
|
+
|
|
1503
|
+
The location flag is provided via option -LC (-Location). The defualt value
|
|
1504
|
+
is 'B' for archiving a help file onto both Web Disk and Object Store.
|
|
1505
|
+
|
|
1506
|
+
3.4.12 Backup Quasar Files
|
|
1507
|
+
-AQ or -ArchiveQuasarFile (Alias: -ArchiveQuasar|-ArchiveBackupFile|-ArchiveBackup),
|
|
1508
|
+
backs the archived saved and/or Web files up onto Globus Quasar Servers and saves
|
|
1509
|
+
their information into RDADB. All the provided saved and/or web files are backed
|
|
1510
|
+
up into a single Quasar backup file at a time. Another utility program, dsquasar,
|
|
1511
|
+
is normally used to check and collect the newly archived saved and web files.
|
|
1512
|
+
The dsquasar builds input files between 1 and 3 GB, and then call the dsarch to
|
|
1513
|
+
build a tarfile and back the tar file onto Quasar server. Reference to dsquasar
|
|
1514
|
+
help document for detail information.
|
|
1515
|
+
|
|
1516
|
+
dsarch [-(DS|dataset)] dsnnn.n -(AQ|ArchiveQuasarFile) [Mode Options]
|
|
1517
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
1518
|
+
[-(ST|SavedFileType) SavedFileTypes]
|
|
1519
|
+
[-(WF|WebFile) webFileNames]
|
|
1520
|
+
[-(WT|WebFileType) WebFileTypes]
|
|
1521
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1522
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1523
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1524
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1525
|
+
|
|
1526
|
+
Mode options that can be specified for archive web file action:
|
|
1527
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1528
|
+
standard outputs and errors
|
|
1529
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1530
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1531
|
+
information to be set by any one
|
|
1532
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1533
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1534
|
+
values to speed up reading input file(s)
|
|
1535
|
+
-(OE|OverrideExist) - overrides existing web files
|
|
1536
|
+
-(TO|TarOnly) - to tar the given Saved and/or Web files only, without physically
|
|
1537
|
+
backing up the Quasar file onto the Globus Quasar Server.
|
|
1538
|
+
-(XC|CrossCopy) - copies files from existing Quasar Backup and Disaster Recovery
|
|
1539
|
+
files to missing Backup/Drdata files; information of the quasar
|
|
1540
|
+
backup files saved in RDADB is used for missing data copies
|
|
1541
|
+
|
|
1542
|
+
One or multiple Saved/Web file names can be provided per option -SF/-WF,
|
|
1543
|
+
respectively, and for this action to work.
|
|
1544
|
+
|
|
1545
|
+
|
|
1546
|
+
3.4.13 Restore Quasar Files
|
|
1547
|
+
-RQ or -RestoreQuasarFile (Alias: -RestoreQuasar|RestoreBackupfile|-RestoreBackup),
|
|
1548
|
+
identifies Quasar files from given saved and/or Web files, retrieves the tarfiles from
|
|
1549
|
+
Globus Quasar Server, and restores the Saved and/or Web files if any missed.
|
|
1550
|
+
|
|
1551
|
+
dsarch [-(DS|dataset)] dsnnn.n -(RQ|RestoreQuasarFile) [Mode Options]
|
|
1552
|
+
[-(QF|QuasarFile) QuasarFileNames]
|
|
1553
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
1554
|
+
[-(ST|SavedFileType) SavedFileTypes]
|
|
1555
|
+
[-(WF|WebFile) webFileNames]
|
|
1556
|
+
[-(WT|WebFileType) WebFileTypes]
|
|
1557
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1558
|
+
[-(QS|QsubOptions) PBSBatchOptions]
|
|
1559
|
+
[-(BP|BatchProcess) [BatchControlInfo]]
|
|
1560
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1561
|
+
|
|
1562
|
+
Mode options that can be specified for archive web file action:
|
|
1563
|
+
-(BG|BackGround) - background process to turn off screen display for both
|
|
1564
|
+
standard outputs and errors
|
|
1565
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1566
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1567
|
+
information to be set by any one
|
|
1568
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1569
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1570
|
+
values to speed up reading input file(s)
|
|
1571
|
+
|
|
1572
|
+
One or multiple Quasar/Saved/Web file names can be provided per option -QF-SF/-WF,
|
|
1573
|
+
for this action to work. If Only Quasar file names are provided, the quasar tarfiles
|
|
1574
|
+
are retrieved from quasar Server, but no Saved or Web files are retored.
|
|
1575
|
+
|
|
1576
|
+
|
|
1577
|
+
3.4.14 Move Saved/Web/Help/Quasar Files
|
|
1578
|
+
-MV or -MoveFile, moves files archived on RDA Server from an original dataset
|
|
1579
|
+
per option -OD (-OriginDataset) to a different dataset per option -DS (-Dataset),
|
|
1580
|
+
or from one group to another in the same dataset, or just simply to different
|
|
1581
|
+
file names. One or multiple files can be moved each time.
|
|
1582
|
+
|
|
1583
|
+
If, after moved, a file name, including path, is different from its original name,
|
|
1584
|
+
the original name is recorded in RDADB with link to the new data file name.
|
|
1585
|
+
|
|
1586
|
+
Files can be moved cross servers between Web and Saved files if Mode options -TS
|
|
1587
|
+
(-ToSaved) or -TW (-ToWeb).
|
|
1588
|
+
|
|
1589
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(MV|MoveFile) [Mode Options]
|
|
1590
|
+
[-(OD|OriginDataset) OriginalDataset]
|
|
1591
|
+
[-(RF|OriginFile) OriginalSavedFileNames]
|
|
1592
|
+
-(SF|SavedFile) SavedFileNames
|
|
1593
|
+
[-(OG|OriginGroup) OriginalGroupIndices]
|
|
1594
|
+
[-(GI|GroupIndex) NewGroupIndices]
|
|
1595
|
+
[-(OT|OriginType) OriginalFileTypes]
|
|
1596
|
+
[-(ST|SavedFileType) NewSavedFileTypes]
|
|
1597
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1598
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1599
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1600
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1601
|
+
|
|
1602
|
+
or
|
|
1603
|
+
|
|
1604
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(MV|MoveFile) [Mode Options]
|
|
1605
|
+
[-(OD|OriginDataset) OriginalDataset]
|
|
1606
|
+
[-(RF|OriginFile) OriginalWebFileNames]
|
|
1607
|
+
-(WF|WebFile) WebFileNames
|
|
1608
|
+
[-(OG|OriginGroup) OriginalGroupIndices]
|
|
1609
|
+
[-(GI|GroupIndex) NewGroupIndices]
|
|
1610
|
+
[-(OT|OriginType) OriginalFileTypes]
|
|
1611
|
+
[-(WT|WebFileType) NewWebFileTypes]
|
|
1612
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1613
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1614
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1615
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1616
|
+
|
|
1617
|
+
or
|
|
1618
|
+
|
|
1619
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(MV|MoveFile) [Mode Options]
|
|
1620
|
+
[-(OD|OriginDataset) OriginalDataset]
|
|
1621
|
+
[-(RF|OriginFile) OriginalHelpFileNames]
|
|
1622
|
+
-(HF|HelpFile) HelpFileNames
|
|
1623
|
+
[-(OT|OriginType) OriginalFileTypes]
|
|
1624
|
+
[-(HT|HelpFileType) NewHelpFileTypes]
|
|
1625
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1626
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1627
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1628
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1629
|
+
|
|
1630
|
+
or
|
|
1631
|
+
|
|
1632
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(MV|MoveFile) [Mode Options]
|
|
1633
|
+
[-(OD|OriginDataset) OriginalDataset]
|
|
1634
|
+
[-(RF|OriginFile) OriginalQuasarFileNames]
|
|
1635
|
+
-(QF|QuasarFile) QuasarFileNames
|
|
1636
|
+
[-(OT|OriginType) OriginalFileTypes]
|
|
1637
|
+
[-(QT|QuasarFileType) NewQuasarFileTypes]
|
|
1638
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1639
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1640
|
+
|
|
1641
|
+
or
|
|
1642
|
+
|
|
1643
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(MV|MoveFile) [-TS|-TW] [Mode Options]
|
|
1644
|
+
[-(OD|OriginDataset) OriginalDataset]
|
|
1645
|
+
[-(OG|OriginGroup) OriginalGroupIndices]
|
|
1646
|
+
[-(WF|WebFile) WebFileNames]
|
|
1647
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
1648
|
+
[-(GI|GroupIndex) NewGroupIndices]
|
|
1649
|
+
[-(WT|WebFileType) WebFileTypes]
|
|
1650
|
+
[-(ST|SavedFileType) NewSavedFileTypes]
|
|
1651
|
+
[-(LC|Location) StorageLocationFlags]
|
|
1652
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1653
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1654
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1655
|
+
|
|
1656
|
+
|
|
1657
|
+
Mode options that can be specified for this action:
|
|
1658
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1659
|
+
information to be set by any one
|
|
1660
|
+
-(NT|NoTrim) - skips trimming of spaces and comments from input
|
|
1661
|
+
values to speed up reading input file(s)
|
|
1662
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1663
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input
|
|
1664
|
+
values to speed up reading input file(s)
|
|
1665
|
+
-(TS|ToSaved) - move Web Files to Saved files
|
|
1666
|
+
-(TW|ToWeb) - move Saved Files to Web files
|
|
1667
|
+
-(KM|KeepMetadata) - works with option -TW to keep the metadata record for later use
|
|
1668
|
+
after the web file record is moved to savefile
|
|
1669
|
+
|
|
1670
|
+
Use this action to move Saved/Web/Help/Quasar files, Saved File names, Web File Names,
|
|
1671
|
+
Help file names, or Quasar backup file names are provided per option, -SF (-SavedFile),
|
|
1672
|
+
-WF (-WebFile), -HF (-HelpFile), or -QF (-QuasarFile), respectively. It is allowed to
|
|
1673
|
+
move Web to Saved files, or Saved to Web files. A data file is only physically moved
|
|
1674
|
+
if its new and original paths are different. Dataset links for the data files are
|
|
1675
|
+
pointed to the new dataset number. For Saved and Web files, group indices are changed
|
|
1676
|
+
to new ones if specified per option -GI (-GroupIndex).
|
|
1677
|
+
|
|
1678
|
+
If content metadata is gathered for the original Web file, the metadata information
|
|
1679
|
+
is automatically transfered to the new file by cammnd 'rcm'.
|
|
1680
|
+
|
|
1681
|
+
If a Web file of a dataset under DOI/Version control is moved to a different name,
|
|
1682
|
+
or to a Saved file, its data type is automatically changed to 'V 'as Version
|
|
1683
|
+
controlled type.
|
|
1684
|
+
|
|
1685
|
+
|
|
1686
|
+
3.4.15 Delete Saved/Web/Help/Quasar Files
|
|
1687
|
+
-DL or -Delete (Alias: -RM|-Remove|-DeleteFile|-RemoveFile), deletes data
|
|
1688
|
+
files on RDA Servers and their RDADB records.
|
|
1689
|
+
|
|
1690
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(DL|Delete) [Mode Options]
|
|
1691
|
+
[-(SF|SavedFile) SavedFileNames]
|
|
1692
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
1693
|
+
-(ST|SavedFileType) SavedFileTypes
|
|
1694
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1695
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1696
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1697
|
+
|
|
1698
|
+
or
|
|
1699
|
+
|
|
1700
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(DL|Delete) [Mode Options]
|
|
1701
|
+
[-(WF|WebFile) WebFileNames]
|
|
1702
|
+
[-(GI|GroupIndex) GroupIndices]
|
|
1703
|
+
-(WT|WebFileType) WebFileTypes
|
|
1704
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1705
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1706
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1707
|
+
|
|
1708
|
+
or
|
|
1709
|
+
|
|
1710
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(DL|Delete) [Mode Options]
|
|
1711
|
+
[-(HF|HelpFile) HelpFileNames]
|
|
1712
|
+
-(HT|HelpFileType) HelpFileTypes
|
|
1713
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1714
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1715
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1716
|
+
|
|
1717
|
+
or
|
|
1718
|
+
|
|
1719
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(DL|Delete) [Mode Options]
|
|
1720
|
+
[-(QF|QuasarFile) QuasarFileNames]
|
|
1721
|
+
-(HT|HelpFileType) HelpFileTypes
|
|
1722
|
+
[-(DD|DeleteDir) DeleteDirLevel]
|
|
1723
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1724
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1725
|
+
|
|
1726
|
+
Mode options that can be specified for deleting file action:
|
|
1727
|
+
-(DX|DeleteXML) - if present, calling 'dcm' to delete file content metadata
|
|
1728
|
+
of the Web files to be deleted
|
|
1729
|
+
-(EM|EMailNotice) - sends email for summary and errors for this action
|
|
1730
|
+
-(MD|MyDataset) - overrides the default specialist and enables RDADB
|
|
1731
|
+
information to be set by any one
|
|
1732
|
+
-(NE|NoEmail) - does not send email to the specialist for failed action
|
|
1733
|
+
-(NT|NoTrim) - skip trimming of spaces and comments from input values to
|
|
1734
|
+
speed up reading input file(s)
|
|
1735
|
+
-(RD|RemoveDir) - removes empty saved or web directories
|
|
1736
|
+
|
|
1737
|
+
Use this action to delete data files on RDA Server. The Saved, Web, Help or Quasar
|
|
1738
|
+
File names are provided per option -SF (-SavedFile), -WF (-WebFile), -HF (-HelpFile),
|
|
1739
|
+
or -QF (-Quasarfile), respectively.
|
|
1740
|
+
|
|
1741
|
+
Specify the number of levels of directories via Info option -DD (-DeleteDir)
|
|
1742
|
+
for the directories to be deleted if Saved or Web files inside them are all
|
|
1743
|
+
deleted.
|
|
1744
|
+
|
|
1745
|
+
|
|
1746
|
+
3.5 All Info Actions
|
|
1747
|
+
|
|
1748
|
+
Information of dataset, groups and data files on RDA Servers can be
|
|
1749
|
+
created, modified and viewed together via actions included in this section:
|
|
1750
|
+
Get All Information - retrieve information of dataset, groups, and files on
|
|
1751
|
+
RDA Server together for a specified dataset
|
|
1752
|
+
Set All Information - modify information of existing dataset, groups and data
|
|
1753
|
+
files on RDA Server of a given dataset, and create
|
|
1754
|
+
new records for groups and data files if they are
|
|
1755
|
+
not recorded in RDADB yet
|
|
1756
|
+
|
|
1757
|
+
3.5.1 Get All Information
|
|
1758
|
+
-GA or -GetALL, retrieves dataset, group and Saved/Web/Help/Quasar file
|
|
1759
|
+
information for a given dataset. This is a comprehensive action, combining all Get
|
|
1760
|
+
Actions, -GD (-GetDataset), -GG (-GetGroup), -GS (-GetSavedFile), -GW (-GetWebFile),
|
|
1761
|
+
-GH (-GetHelpFile), and -GQ (-GetQuasarFile.
|
|
1762
|
+
|
|
1763
|
+
dsarch [-(DS|dataset)] dsnnn.n -(GA|GetAll)
|
|
1764
|
+
[-(OF|OutputFile|>) OutputFileName]
|
|
1765
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1766
|
+
|
|
1767
|
+
No Mode option is applied to this action. Information of all available fields
|
|
1768
|
+
is retrieved for dataset, groups and files.
|
|
1769
|
+
|
|
1770
|
+
Section headers are generated as [DATASET] for dataset, [DSGROUP] for
|
|
1771
|
+
groups, [SAVEDFILE] for Saved file information, [WEBFILE] for Web file information,
|
|
1772
|
+
[HELPFILE] for Help file information, and [QUASARFILE] for Quasar file information.
|
|
1773
|
+
|
|
1774
|
+
Info option -OF (-OutputFile|>) is normally used to specify a file name to
|
|
1775
|
+
save the retrieved information for later usage. Result of this action is
|
|
1776
|
+
displayed on screen if no output file is provided. The output file can be
|
|
1777
|
+
edited and treated as a input file to save the changes back to RDADB per
|
|
1778
|
+
action -SA (-SetAll).
|
|
1779
|
+
|
|
1780
|
+
3.5.2 Set All Information
|
|
1781
|
+
-SA or -SetAll, creates and modifies dataset, group and Saved/Web/Help/Quasar
|
|
1782
|
+
file information into RDADB for a given dataset number. This is a comprehensive
|
|
1783
|
+
action, combining all four Set Actions, -SD (-SetDataset), -SG (-SetGroup), -SS
|
|
1784
|
+
(-SetSavedFile), -SW (-SetWebFile), -SH (-SetHelpFile), and -SQ (-SetQuasarFile).
|
|
1785
|
+
|
|
1786
|
+
dsarch -(SA|SetAll) [Mode Options]
|
|
1787
|
+
[-(LN|LoginName) LoginAccountName]
|
|
1788
|
+
-(IF|InputFile) Input Files
|
|
1789
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1790
|
+
|
|
1791
|
+
ALL Mode options that can be applied to the Set Actions, -SD, -SG, -SW, -SH
|
|
1792
|
+
and -SQ can be specified for this action. At least one input file is need to
|
|
1793
|
+
execute this action since section headers need to be specified in an input file.
|
|
1794
|
+
One way of creating an input file is to get one per action -GA (-GetALL) for a
|
|
1795
|
+
dataset. The output file can then be edited and used as an input file to set
|
|
1796
|
+
all information of the dataset back to RDADB.
|
|
1797
|
+
|
|
1798
|
+
|
|
1799
|
+
3.6 Webpage Actions
|
|
1800
|
+
|
|
1801
|
+
Information changed for dataset/groups/files of a given dataset can be
|
|
1802
|
+
refreshed onto the main and filelist web pages of a given dataset via actions
|
|
1803
|
+
included in this section:
|
|
1804
|
+
Regenerate Dataset Webpages - regenerate main webpage and try to republish
|
|
1805
|
+
all the filelists of data on HPSS and RDA Server
|
|
1806
|
+
Increment Of Cache Number - increase the cache control number by 1 for
|
|
1807
|
+
a specified dataset; therefore, the cached
|
|
1808
|
+
filelists are refreshed accordingly
|
|
1809
|
+
|
|
1810
|
+
3.6.1 Regenerate Dataset Webpages
|
|
1811
|
+
-UW or -UpdateWeb, regenerates main and filelist webpages to catch the
|
|
1812
|
+
changes of dataset/group/file information in RDADB of a given dataset.
|
|
1813
|
+
|
|
1814
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(UW|UpdateWeb)
|
|
1815
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1816
|
+
|
|
1817
|
+
Given a dataset number to refresh its the main and filelist web pages.
|
|
1818
|
+
|
|
1819
|
+
A cache control number of each dataset is increased by 1 each time the
|
|
1820
|
+
changes of dataset/group/file information of the dataset are set into RDADB
|
|
1821
|
+
by 'dsarch'. The new cache number tells webpage utility to refresh the
|
|
1822
|
+
cached group and file information of the dataset. Changes made through other
|
|
1823
|
+
utilities may not modify this cache control number. Specialists can always
|
|
1824
|
+
manually increase this cache number by action -UC (-UpdateCache).
|
|
1825
|
+
|
|
1826
|
+
3.6.2 Increment Of Cache Number
|
|
1827
|
+
-UC or -UpdateCache, (alias: -UpdateCacheNumber), increases the cache control number
|
|
1828
|
+
by 1 for a given dataset.
|
|
1829
|
+
|
|
1830
|
+
dsarch [-(DS|Dataset)] dsnnn.n -(UC|UpdateCache)
|
|
1831
|
+
[-(DB|Debug) DebugModeInfo]
|
|
1832
|
+
|
|
1833
|
+
|
|
1834
|
+
4 MODE OPTIONS
|
|
1835
|
+
|
|
1836
|
+
Use proper Mode options to modify behaviors of Action options. Mode options
|
|
1837
|
+
are all optional. No value is allowed to be passed in following any Mode option.
|
|
1838
|
+
|
|
1839
|
+
-BG or -BackGround (Alias: -b), background process. When it presents
|
|
1840
|
+
screen display is turned off for both standard outputs and errors.
|
|
1841
|
+
|
|
1842
|
+
-CL or -CleanLocal (Alias: -CleanLocfile|-CleanLocalFile), when present for
|
|
1843
|
+
archiving actions -AS, or -AW, cleans the local files after they are all archived
|
|
1844
|
+
successfully. If local files are staged temporarily in sub-directories, the
|
|
1845
|
+
sub-directories can be removed too after all the local files are cleaned inside
|
|
1846
|
+
them. Add Info option -DD DirectoryLevel to tell how many level of the local
|
|
1847
|
+
empty directories to be removed.
|
|
1848
|
+
|
|
1849
|
+
-DX or -DeleteXML, if present, calls 'dcm' to delete file content metadata
|
|
1850
|
+
of given Web files for actions -SW (-SetWebFile). This is the default Mode for
|
|
1851
|
+
action -DL (-Delete) of Web files provided via -WF (-WebFile).
|
|
1852
|
+
|
|
1853
|
+
-EM or -EMailNotice, sends an email notice to the specialist who executes the
|
|
1854
|
+
dsarch for certain actions after a action is finished successfully or aborts
|
|
1855
|
+
abnormally. The Actions working with this Mode option include -AS, -AW,, -AH,
|
|
1856
|
+
-AQ, -SS, -SW, -SH, -SQ, -RQ, -MV and -DL. An email notice includes errors
|
|
1857
|
+
happening during the action and a brief summary to show the action progress.
|
|
1858
|
+
This option is ignored if option -d is present.
|
|
1859
|
+
|
|
1860
|
+
Specialists need to be cautious of adding this Mode option to run dsarch.
|
|
1861
|
+
If you run dsarch to archive many files with one at a time, you will receive
|
|
1862
|
+
one email for each file. To make it work better, you include all the files
|
|
1863
|
+
you want to process into a single input file and run dsarch with option -EM
|
|
1864
|
+
against the input file. This way you receive a single email notice instead.
|
|
1865
|
+
|
|
1866
|
+
-FO or -FormatOutput, if present, formats column output results for get
|
|
1867
|
+
actions. A same width, evaluated dynamically, is applied for all values of a
|
|
1868
|
+
given field.
|
|
1869
|
+
|
|
1870
|
+
-GF or -GrowingFile, if present, Action -AW (-ArchiveWebFile) will allow
|
|
1871
|
+
existing Web files being overridden although the data files are
|
|
1872
|
+
under DOI/Version control already. Normally an archived data file can not
|
|
1873
|
+
changed if it is linked to a version control record. This Mode option
|
|
1874
|
+
indicates that the data file is a growing one and needs to be updated.
|
|
1875
|
+
If Action -AW of 'dsarch' is controlled by configuration of 'dsupdt', a
|
|
1876
|
+
growing file is automatically detected and this Mode option is added to the
|
|
1877
|
+
'dsarch' command as needed.
|
|
1878
|
+
|
|
1879
|
+
-GX or -GatherXML (Alias: -Grid2XML), if present, calls 'gatherxml' to
|
|
1880
|
+
evaluate file content metadata of archived Web files. This is the default
|
|
1881
|
+
Mode for action -MV (-MoveFile) for Web files, to call 'rcm' to move the
|
|
1882
|
+
file content metadata too.
|
|
1883
|
+
|
|
1884
|
+
-GZ or -GMTZone (Alias: -GMT|GreenwichZone|UTC), uses GMT dates/times as
|
|
1885
|
+
controlling times for archiving actions, instead of mountain time.
|
|
1886
|
+
|
|
1887
|
+
-KM or -KeepMetadata (Alias: -KeepMeta), for actions -MV (-MoveFile) with
|
|
1888
|
+
option -TS (-ToSaved) to keep the metadata record for later use after
|
|
1889
|
+
the web file is moved to savefile.
|
|
1890
|
+
|
|
1891
|
+
-KP or -KeepPath (Alias: -KeepLocalPath), for actions -AS (-ArchiveSavedFile), -AW
|
|
1892
|
+
(-ArchiveWebFile) and -AH (-ArchiveHelpFile) to archive Saved/Web/Help files with
|
|
1893
|
+
the same file names as the local ones specified by -LF (-LocalFile). A relative
|
|
1894
|
+
path is reta8ined if it is part of a local file name. Options -SP (-SavedPath)
|
|
1895
|
+
and -WP (-WebPath) are ignored if -KP is given. This Mode option is ignored if
|
|
1896
|
+
Save/Web/Help file names are specified per options -SF (-SavedFile), -WF (-WebFile),
|
|
1897
|
+
and -HF (-HelpFile), respectively.
|
|
1898
|
+
|
|
1899
|
+
-MD or -MyDataset, allows a specialist to update dataset/group/file
|
|
1900
|
+
|
|
1901
|
+
-NE or -NoEmail, does not send email notice to the specialist for failed action.
|
|
1902
|
+
|
|
1903
|
+
-NT or -NoTrim, skips trimming of spaces and comments from input values to
|
|
1904
|
+
speed up reading input file(s).
|
|
1905
|
+
|
|
1906
|
+
-NV or -NewVersion, adds a new version control record for a specified dataset
|
|
1907
|
+
if this Mode option is present for Action -SV (-SetVersion).
|
|
1908
|
+
|
|
1909
|
+
-OE or -OverrideExist, if present, overrides an existing file.
|
|
1910
|
+
|
|
1911
|
+
-PE or -ShowPeriod, shows period information for dataset, and groups if provided,
|
|
1912
|
+
for action -SD (-SetDataset).
|
|
1913
|
+
|
|
1914
|
+
-RA or -RetryArchive, deprecated, does not do anything.
|
|
1915
|
+
|
|
1916
|
+
-RD or -RemoveDir (Alias: -RemoveDirectory|-RemoveEmptyDir), it works with
|
|
1917
|
+
Actions -DL, -SS and -SW to remove empty directories for a specified dataset,
|
|
1918
|
+
and groups if specified.
|
|
1919
|
+
|
|
1920
|
+
If empty directories are not cleaned after deleting files. Use Actions -SS or
|
|
1921
|
+
-SW with this Mode option, without Saved or Web file names specified, to clean
|
|
1922
|
+
the empty Web or HPSS directories, respectively for a given dataset, and
|
|
1923
|
+
groups if specified.
|
|
1924
|
+
|
|
1925
|
+
-RG or -RecursiveGroup (Alias: -RepeatGroup), when present, gathers subgroups
|
|
1926
|
+
and file information in subgroups recursively for actions -GG (-GetGroup) and
|
|
1927
|
+
-GS/GW (-GetSavedFile/GetWebfile), respectively. Group index must be provided
|
|
1928
|
+
explicitly via Info option -GI (-GroupIndex) for this Mode option to work.
|
|
1929
|
+
|
|
1930
|
+
-RN or -RelativeName, for action -GS (-GetSavedFile) and -GW (-GetWebFile) to
|
|
1931
|
+
show a file name with path relative to the Saved and Web paths set for
|
|
1932
|
+
the dataset or group, respectively.
|
|
1933
|
+
|
|
1934
|
+
-RO or -ResetOrder (Alias: -Reorder), when present, resets the display orders
|
|
1935
|
+
of the given Saved/Web/Help/Quasar file lists as the orders the data files are
|
|
1936
|
+
given for actions -SS (-SetSavedFile), -SW (-SetWebFile), -SH (-SetHelpFile),
|
|
1937
|
+
and -SQ (-SetQuasarFile), respectively. Another way of reordering the data files
|
|
1938
|
+
is to provide explicitly order index values per Info option -DO (-DisplayOrder).
|
|
1939
|
+
|
|
1940
|
+
Specify what fields to order on per option -ON (-OrderNames) to order the
|
|
1941
|
+
files accord for the whole datasets or individual groups. Check option -ON
|
|
1942
|
+
for examples of ordering Files.
|
|
1943
|
+
|
|
1944
|
+
-RS or -GXRSOptions, works with Mode option -GX (-GatherXML) to pass option
|
|
1945
|
+
R and S to command gatherxml for speeding up.
|
|
1946
|
+
|
|
1947
|
+
-RT or -ResetTIndex (Alias: -ResetTopGroup), for setting actions to reset
|
|
1948
|
+
the top group index for Saved or Web files for dataset, and groups if
|
|
1949
|
+
specified.
|
|
1950
|
+
|
|
1951
|
+
-SC or -SetChecksum, if presents, evaluates the md5 checksum values for data
|
|
1952
|
+
files on RDA Servers and saves them into RDADB
|
|
1953
|
+
|
|
1954
|
+
-TO or -TarOnly, works with Action -AQ (-ArchiveQuasarFile) to tar the given
|
|
1955
|
+
Saved and/or Web files only, without physically backing up the Quasar file
|
|
1956
|
+
onto the Globus Quasar Server.
|
|
1957
|
+
|
|
1958
|
+
-TS or -ToSaved (Alias: -ToSavedFile|-MoveToSaved), when present for Action -MV to
|
|
1959
|
+
move Web files to Saved files.
|
|
1960
|
+
|
|
1961
|
+
-TT or -TotalSummary, when present, gets the statistic summary information
|
|
1962
|
+
for Action options -GS (-GetSavedFile) and -GW (-GetWebFile).
|
|
1963
|
+
|
|
1964
|
+
-TW or -ToWeb (Alias: -ToWebFile|-MoveToWeb), when present for Action -MV to
|
|
1965
|
+
move Saved files to Web files.
|
|
1966
|
+
|
|
1967
|
+
-UZ or -UpzipData (Alias: -Unzip|-UncompressData), uncompresses online data
|
|
1968
|
+
with one of the uncompressing utilities, 'gunzip', 'uncompress', 'unzip'
|
|
1969
|
+
or 'bunzip', according to the file name extensions of ".gz', '.Z', '.zip'
|
|
1970
|
+
or '.bz2', respectively
|
|
1971
|
+
|
|
1972
|
+
-WC or -WithChecksum, when present, checks the MD5 checksums in addition to
|
|
1973
|
+
the data file sizes for online data files on RDA Server.
|
|
1974
|
+
|
|
1975
|
+
-WM or -WithMetadata, for setting actions to reset the dataset/group level
|
|
1976
|
+
content metadata for action -SD/SG (-SetDataset/SetGroup).
|
|
1977
|
+
|
|
1978
|
+
-WN or -WithFileNumber, for setting actions to reevaluate and reset the
|
|
1979
|
+
Saved or Web file counts for dataset, and groups if specified. These file
|
|
1980
|
+
counts can also be viewed for action -GD (-GetDataset) if this Mode option is
|
|
1981
|
+
preset.
|
|
1982
|
+
|
|
1983
|
+
-XC or -CrossCopy, works with action -AW/-AS to cross copy data files between
|
|
1984
|
+
Web/Saved files on Glade disk and Object files on Object store.
|
|
1985
|
+
|
|
1986
|
+
-XM or -CrossMove, works with action -AW/-AS to cross move data files between
|
|
1987
|
+
Web/Saved files on Glade disk and Object files on Object store. This Mode option
|
|
1988
|
+
is different from -XC that the original files are removed.
|
|
1989
|
+
|
|
1990
|
+
-ZD or -ZipData (Alias: -Zip|-CompressData), when present, Actions -AS
|
|
1991
|
+
(-ArchiveSavedFile) and -AW (-ArchiveWebFile) compress online data with one
|
|
1992
|
+
of the compressing utilities, 'gzip', 'compress', 'zip' and 'bzip', according
|
|
1993
|
+
to the archiving format 'GZ', 'Z', 'ZIP' or 'BZ2' provided per option -AF
|
|
1994
|
+
(-ARchiveFormat), respectively. If the original data files are compressed already,
|
|
1995
|
+
this Mode option is ignored if the same compression, and the files are uncompressed,
|
|
1996
|
+
before compressing, with one of the uncompressing utilities, 'gunzip', 'uncompress',
|
|
1997
|
+
'unzip' or 'bunzip', according to the file name extensions of ".gz', '.Z', '.zip'
|
|
1998
|
+
or '.bz2', respectively.
|
|
1999
|
+
|
|
2000
|
+
|
|
2001
|
+
5 INFORMATION OPTIONS
|
|
2002
|
+
|
|
2003
|
+
Information options are used to pass information, one or multiple values, into
|
|
2004
|
+
'dsarch'. Two types of Info options are used:
|
|
2005
|
+
Single-Value Info Options - pass a single value to 'dsarch'
|
|
2006
|
+
Multi-Value Info Options - pass one or multiple values to 'dsarch'
|
|
2007
|
+
|
|
2008
|
+
5.1 Single-Value Info Options
|
|
2009
|
+
|
|
2010
|
+
A single-value Info option is used to pass one value into this application.
|
|
2011
|
+
One value, and one only, must follow a single-value option; otherwise an
|
|
2012
|
+
error message is displayed if no value or more than one value passed in.
|
|
2013
|
+
|
|
2014
|
+
-AL or -AsyncLimit (Alias: -AsynchronousLimit), provides a value of the
|
|
2015
|
+
maximum number of background processes for 'cp' and 'gatherxml' calls. This
|
|
2016
|
+
option is ignored for PBS batch jobs.
|
|
2017
|
+
|
|
2018
|
+
-AO or -ActionOption (Alias: -ActOption), provides an ending sign to idicate
|
|
2019
|
+
the leading option is a Action or Mode option. It is defaults to '<!>'.
|
|
2020
|
+
|
|
2021
|
+
-BL or -ButtonLimit, controlling limit of available file count in filelist
|
|
2022
|
+
to show click buttons for download scripts. The default number is 2.
|
|
2023
|
+
|
|
2024
|
+
-DD or -DeleteDir (Alias: -DeleteDirLevel|-DeleteEmptyDir), it works with
|
|
2025
|
+
Actions -DL (-Delete) or -MV (-MoveFile) for removing how many level of
|
|
2026
|
+
empty directories after all files inside are deleted or moved. It works
|
|
2027
|
+
with Mode option -CL too for Actions -AS and -AW to remove empty local
|
|
2028
|
+
directories after the local files inside of them are all cleaned.
|
|
2029
|
+
|
|
2030
|
+
-DS or -Dataset, for a dataset number, or called dataset ID in form as
|
|
2031
|
+
[a-z]NNNNNN. It is mandatory for most actions.
|
|
2032
|
+
|
|
2033
|
+
-DV or -Divider (Alias: -Delimiter, -Separator), delimiter for separating
|
|
2034
|
+
columns of multi-value Info options in input files. It is default to '<:>'.
|
|
2035
|
+
|
|
2036
|
+
-ES or -EqualSign, for an equal sign of assigning one value to either a
|
|
2037
|
+
single-value option or multi-value option in input files. It is defaulted
|
|
2038
|
+
to '<=>'.
|
|
2039
|
+
|
|
2040
|
+
-FL or -FileLimit (Alias: -FilecountLimit), sets limit for how many files
|
|
2041
|
+
can be displayed on a single web page of static filelists. Files from
|
|
2042
|
+
multiple groups may be displayed on a single page if the total file count
|
|
2043
|
+
is lower than this limit, while files from a single group may be split into
|
|
2044
|
+
sub-filelists if the file count is beyond this limit. It defaults to 2000
|
|
2045
|
+
and specialists can reset it to a different value for a specified dataset
|
|
2046
|
+
via Action -SD (-SetDataset).
|
|
2047
|
+
|
|
2048
|
+
-FN or -FieldNames, for a string of single letter field names. Values of
|
|
2049
|
+
the selected fields are retrieved per actions -GD (-GetDataset),
|
|
2050
|
+
-GG (-GetGroup), -GS (-GetSavedFile), -GW (-GetWebFile), -GH (-GetHelpFile),
|
|
2051
|
+
and -GQ (-GetQuasarFile). Values of default fields are retrieved according
|
|
2052
|
+
to what GET action is specified. Valid field names are listed in corresponding
|
|
2053
|
+
Get action sections.
|
|
2054
|
+
|
|
2055
|
+
-LD or -LocalDirectory, provides a single local directory, file names under
|
|
2056
|
+
the directory, and the sub-directories recursively if any, are all collected
|
|
2057
|
+
as local files
|
|
2058
|
+
|
|
2059
|
+
-LL or -LocalFileList, provides a single file name and the file holds a list of
|
|
2060
|
+
file names, one file name on each line. These file names are used as the local
|
|
2061
|
+
file names
|
|
2062
|
+
|
|
2063
|
+
-LN or -LoginName, login name of the current user; it defaults to the current
|
|
2064
|
+
login DECS specialist who starts execution of 'dsarch'. Set this option if you
|
|
2065
|
+
try to run this utility for a specialist other than yourself. Normally this
|
|
2066
|
+
is set when this application is started by a daemon or as a cron job. Be
|
|
2067
|
+
careful of using this option since it might override the currently information
|
|
2068
|
+
of dataset/groups/files saved by other specialists in RDADB.
|
|
2069
|
+
|
|
2070
|
+
-OF or -OutputFile, leading an output file name into which the output result
|
|
2071
|
+
of this application is dumped. Output file format is similar to the format of
|
|
2072
|
+
the input files. If this option is not given, the result is displayed on
|
|
2073
|
+
screen.
|
|
2074
|
+
|
|
2075
|
+
-ON or -OrderNames, for a string of single letter field names use to order
|
|
2076
|
+
the results of GET actions, -GG (-GetGroup), -GS (-GetSavedFile), -GW
|
|
2077
|
+
(-GetWebFile), -GH (-GetHelpFile), and -GQ (-GetQuasarFile). Upper case is
|
|
2078
|
+
for Ascending order while lower case is for Descending order. This option also
|
|
2079
|
+
force reordering of files in individual groups if any file information is modified
|
|
2080
|
+
per Action options -AS (-ArchiveSavedFile), -AW (-ARchiveWebFile), -SS
|
|
2081
|
+
(-SetSavedFile), -SW (-SetWebFile), -DL (-DeleteFile) and -MV (MoveFile).
|
|
2082
|
+
|
|
2083
|
+
Web filelists of specified dataset and groups can also be accomplished per
|
|
2084
|
+
Action -SW (-SetWebFile) with presenting option -ON, as
|
|
2085
|
+
|
|
2086
|
+
dsarch dsnnn.n -SW -ON OrderNameString [-GI GroupIndices]
|
|
2087
|
+
|
|
2088
|
+
Without specified group indices, all the files within or without groups
|
|
2089
|
+
will all be reordered.
|
|
2090
|
+
|
|
2091
|
+
-PO or -PatternOffset, given for string offset of pattern matches while
|
|
2092
|
+
assigning group indices to given Saved or Web files by matches the file
|
|
2093
|
+
names to given group wildcard patterns for file set/archive actions.
|
|
2094
|
+
This is a mandatory Info option for matching group indices of files recorded
|
|
2095
|
+
in RDADB already. A binary search is used for pattern matching if an value of
|
|
2096
|
+
>= 0 is provided; while a simple linear search is used if a value of -1 is
|
|
2097
|
+
provided. Value of -1 is defaulted if this option is not provided for matching
|
|
2098
|
+
group index for a file that is not recorded in RDADB yet.
|
|
2099
|
+
|
|
2100
|
+
-QS or -QsubOptions, (Alias: -PBSOptions), specifies options to execute dsarch
|
|
2101
|
+
as a batch job via qsub on PBS nodes. The qsub options must be quoted when prsented
|
|
2102
|
+
on command line, such as, -QS '-l walltime=12:00:00'.
|
|
2103
|
+
|
|
2104
|
+
-AO or -ActOption, for setting Action and Mode options in input files. It is
|
|
2105
|
+
default to '<!>'.
|
|
2106
|
+
|
|
2107
|
+
-UD or -UseDSARCH (Alias: -UseRDADB), for flag of using DSARCH to control the
|
|
2108
|
+
given dataset. A value out of 'N', 'Y', 'I', 'P', or 'W' is saved in RDADB
|
|
2109
|
+
for a given dataset per action -SD (-SetDataset). A value other than 'N', at
|
|
2110
|
+
least 'Y', must be set for a dataset before it can manipulated by 'dsarch'.
|
|
2111
|
+
'Y' means the the dataset is ready to be modified by 'dsarch'. 'I', 'P',
|
|
2112
|
+
and 'W' mean that the dataset/group/file information of the specified dataset
|
|
2113
|
+
is ready to be published to web server for internal only, publicly, and Web
|
|
2114
|
+
file list only, correspondingly.
|
|
2115
|
+
|
|
2116
|
+
-VS or -ValidSize, a minimal size for a file to be valid for archiving process.
|
|
2117
|
+
It defaults to 100 bytes and can be reset on command line with the update actions.
|
|
2118
|
+
|
|
2119
|
+
-WI or -WaitInterval, it works for the background processes. When the number of
|
|
2120
|
+
background processes reaches the process limit, this option value tells 'dsarch'
|
|
2121
|
+
to wait this period of time before checking if any background process is finish,
|
|
2122
|
+
so that a new background process can be started for next waiting one.
|
|
2123
|
+
|
|
2124
|
+
It can be provided to set a different wait interval, such as Second, Minute,
|
|
2125
|
+
Hour or Day. For example, '-WI 5M' is for 5 minutes. A digital value only
|
|
2126
|
+
assumes a unit of Second, and '-WI 300' means 300S, for example.
|
|
2127
|
+
|
|
2128
|
+
|
|
2129
|
+
5.2 Multi-Value Info Options
|
|
2130
|
+
|
|
2131
|
+
A multi-value Info option is used to pass multiple values for one Info option
|
|
2132
|
+
into 'dsarch'. At lease one value must follow each multi-value option.
|
|
2133
|
+
|
|
2134
|
+
-AF or -ArchiveFormat (Alias: -FF|-FileFormat|-ExternalFormat), for file
|
|
2135
|
+
archiving formats of data files being archived onto RDA Servers. The file
|
|
2136
|
+
archive format indicates post process of a data file. Option values, 'BI' -
|
|
2137
|
+
binary blocked, 'C1' or 'CH' - ASCII/character blocked, can be passed
|
|
2138
|
+
in for COS block information of data files; 'TAR' means a tarred file, 'Z'
|
|
2139
|
+
means a compressed file and 'GZ' means a gzipped file, and 'BZ2' means a file
|
|
2140
|
+
is compressed via bzip2. 'C1.TAR.GZ', for example, means a file is COS ASCII
|
|
2141
|
+
blocked and then tarred and then gzipped; the order is important. The maximum
|
|
2142
|
+
length of format string is 10. Additional characters are truncated for length
|
|
2143
|
+
longer than 10.
|
|
2144
|
+
|
|
2145
|
+
-BD or -BeginDate, for begin date of data files of a given group or the whole
|
|
2146
|
+
dataset, or begin date of a version control record.
|
|
2147
|
+
|
|
2148
|
+
-BF or -BackupFlag, flags set in Datasets and/or groups to control Quasar
|
|
2149
|
+
auto-backup activities; N-No auto-backup (default value for Datasets),
|
|
2150
|
+
B-auto-Backup onto Quasar backup endpoint, D-auto-Backup onto Quasar backup&drdata
|
|
2151
|
+
endpoints, P-the same Backup flags set in parent groups (default values in groups,
|
|
2152
|
+
and parent group index 0 means to follow flags in datasets).
|
|
2153
|
+
|
|
2154
|
+
-BP or -BatchProcess (Alias: -d, -DelayedMode), delayed mode execution. When
|
|
2155
|
+
it presents, the 'dsarch' command is not executed right way, but the command
|
|
2156
|
+
information is recorded into RDADB instead and the command is executed later
|
|
2157
|
+
by the centralized daemon 'dscheck'. One or multiple host names can be specified
|
|
2158
|
+
after option -BP to force the 'dsarch' command be executed on or not on the hosts.
|
|
2159
|
+
For examples, '-d rda-data' to run on host 'rda-data'; '-d rda-data:rda-web-prod'
|
|
2160
|
+
to run on hosts 'rda-data' and 'rda-web-prod'; and '-d \!rda-web-prod:rda-data'
|
|
2161
|
+
to run on all configured hosts other than 'rda-web-prod' and 'rda-data'. Character
|
|
2162
|
+
'!' needs to be escaped for passing in on command line. A upper limit for number
|
|
2163
|
+
of tries can also be passed in with this options, 1 to 99, as '-d 5' for example.
|
|
2164
|
+
It default to 2 for command of 'dsarch' if not specified. And also dsarch actions,
|
|
2165
|
+
-(AS|AW|AH|AQ), will be restarted automatically when this option is present, if
|
|
2166
|
+
the previous processes are failed due to storage system downs.
|
|
2167
|
+
|
|
2168
|
+
-BS or -BackupStatus, Quasar Backup File status, a single letter status flag,
|
|
2169
|
+
A - files are backed up onto Globus Quasar servers; T - the backup tar files
|
|
2170
|
+
are generated, waiting to be archived; and N - backup file records and input
|
|
2171
|
+
files for Web/Saved file lists are created, waiting to be tarred.
|
|
2172
|
+
|
|
2173
|
+
-BT or -BeginTime, for begin time of data files of a given group or the whole
|
|
2174
|
+
dataset, or begin time of a version control record.
|
|
2175
|
+
|
|
2176
|
+
-DB or -Debug, to turn on debug mode with specified information. This option
|
|
2177
|
+
provides up to 3 values, they are Debug Level, debug log file path and debug
|
|
2178
|
+
log file name. The debug level is mandatory for this option. It can be a
|
|
2179
|
+
single integer value, for example, 1000 means to log debug messages for debug
|
|
2180
|
+
levels 1 to 1000; or a range of values, for example, 200-1000 means to log
|
|
2181
|
+
debug messages from debug levels 200 to 1000. The default debug file path is
|
|
2182
|
+
'$DECSHOME}/dssdb/log' and the default debug file name is 'mydss.dbg'. Provides
|
|
2183
|
+
the second and third values for this option to override the default ones
|
|
2184
|
+
respectively.
|
|
2185
|
+
|
|
2186
|
+
-DE or -Description (Alias: -Desc|-Note|-FileDesc|-FileDescription), for file
|
|
2187
|
+
descriptions of data files. Multiple lines are allowed for a description if it is
|
|
2188
|
+
passed in from an input file specified by Info option -IF (-InputFile). On
|
|
2189
|
+
commandline, a single line description with spaces between words must be quoted
|
|
2190
|
+
in 'single quotes'. It presents a version control note if it is used with
|
|
2191
|
+
version control Actions.
|
|
2192
|
+
|
|
2193
|
+
-DF or -DataFormat (Alias: -TF|-ContentFormat|-InternalFormat), data content
|
|
2194
|
+
format of files being archived onto RDA Servers. This is content format of data
|
|
2195
|
+
files. For examples, NetCDF, IMMA and etc.
|
|
2196
|
+
|
|
2197
|
+
A default data format can be set for a given dataset. It is used for archive
|
|
2198
|
+
actions when the data format is not provided explicitly.
|
|
2199
|
+
|
|
2200
|
+
-DN or -DOINumber (Alias: -DOI|-DOIName), for DOI, a Digital Object
|
|
2201
|
+
Identification, of a given dataset for public users to reference against.
|
|
2202
|
+
|
|
2203
|
+
-DO or -DisplayOrder, for display order indices of files of a given group,
|
|
2204
|
+
or of the whole dataset if no groups are set for the dataset. This Info option
|
|
2205
|
+
is ignored if Mode option -RO (-Reorder) is present.
|
|
2206
|
+
|
|
2207
|
+
-ED or -EndDate, for end date of data files of a given group or the whole
|
|
2208
|
+
dataset, or end date of a version control record when it is terminated.
|
|
2209
|
+
|
|
2210
|
+
-ET or -EndTime, for end time of data files of a given group or the whole
|
|
2211
|
+
dataset, or end time of a version control record when it is terminated.
|
|
2212
|
+
|
|
2213
|
+
-EV or -ExternalVerion, external version number. It is only used if a public
|
|
2214
|
+
version number is used for a dataset; otherwise it is defaulted to empty.
|
|
2215
|
+
|
|
2216
|
+
-FD or -FileDate, for date of a data file last modified on.
|
|
2217
|
+
|
|
2218
|
+
-FS or -FileStatus, File status, a single letter status flag, P - show file
|
|
2219
|
+
to public users; and I - to show file to specialist only.
|
|
2220
|
+
|
|
2221
|
+
-FT or -FileTime, for time of a data file last modified at.
|
|
2222
|
+
|
|
2223
|
+
-GI or -GroupIndex, for group indices. This is considered only if a dataset
|
|
2224
|
+
is divided further into groups. To make the grouping works, run this
|
|
2225
|
+
utility program with action -SG (-SetGroup) for a given dataset before
|
|
2226
|
+
archiving files to the dataset. Valid group index values are 1, 2, 3, ....,
|
|
2227
|
+
while 0 means no group which is the default index value.
|
|
2228
|
+
|
|
2229
|
+
If this option is omitted for file archiving options -AS or -AW of a dataset
|
|
2230
|
+
with groups, pattern matching logic is automatically processed; a group index
|
|
2231
|
+
is determined by matching the local file name to group patterns previously
|
|
2232
|
+
saved in RDADB.
|
|
2233
|
+
|
|
2234
|
+
-GL or -GroupLevel, number of group/subgroup levels to be displayed in the
|
|
2235
|
+
group lists. It defaults to 2. Set it to -2 to display one level initially
|
|
2236
|
+
but the group list can be expanded up to 2 levels if the subgroups exist.
|
|
2237
|
+
|
|
2238
|
+
-GN or -GroupName (Alias: -GroupID), a short string, up to 20 characters, for
|
|
2239
|
+
group ID name. Group names can be set into RDADB for a given dataset per
|
|
2240
|
+
action -SG (-SetGroup). If set, they can be provided per this option to
|
|
2241
|
+
identify group indices, and in addition, they are displayed on webpages of
|
|
2242
|
+
Web filelists instead of group indices.
|
|
2243
|
+
|
|
2244
|
+
-GP or -GroupPattern, group patterns to match local file names with group
|
|
2245
|
+
indices while archiving Saved or Web files. The group patterns from this
|
|
2246
|
+
option are saved into RDADB for specified groups of a given dataset per
|
|
2247
|
+
action -SG (-SetGroup).
|
|
2248
|
+
|
|
2249
|
+
-GT or -GroupType (Alias: -GroupDataType), set P for Public or I for
|
|
2250
|
+
Internal group types. When a group type is set from P to I, all the public
|
|
2251
|
+
subgroups under it are set to I recursively, and all the public Web files
|
|
2252
|
+
under the groups or its subgroups are set to Internal too.
|
|
2253
|
+
|
|
2254
|
+
-HF or -HelpFile, for Help file names. During Document or software archiving
|
|
2255
|
+
per action -AH (-ArchiveHelpFile), the local file names given by option -LF
|
|
2256
|
+
(-LocalFile) are used as default for the Help file names unless different
|
|
2257
|
+
file names are specified by this option.
|
|
2258
|
+
|
|
2259
|
+
-HT or -HelpFileType (Alias: -HelpType), Help file types,
|
|
2260
|
+
a single letter property flag, D-Document and S-Software. If only one type
|
|
2261
|
+
is passed in, all Help files are assumed to be the same type.
|
|
2262
|
+
|
|
2263
|
+
-ID or -InitialDate, for date of a Help, Document/Software, file last
|
|
2264
|
+
Published/Released on.
|
|
2265
|
+
|
|
2266
|
+
-IF or -InputFile, for input file names; one or multiple file names may be
|
|
2267
|
+
given on command line. A input file name must start with dataset number as
|
|
2268
|
+
in format of 'dsnnn.n.*' and the dataset number must match the dataset number
|
|
2269
|
+
given per option -DS (-Dataset). This restriction is for prevention that
|
|
2270
|
+
specialists archive data files accidentally into a wrong dataset. Input files
|
|
2271
|
+
are used to hold all valid options and the associated values of Info options
|
|
2272
|
+
that need to be passed in for execution of 'dsarch'.
|
|
2273
|
+
|
|
2274
|
+
In a input file, lines start with sign '#' are considered as comments;
|
|
2275
|
+
Option Names can be given either short, long or alias names. Action and Mode
|
|
2276
|
+
options are given in format of OptionName<!>. Single value Assignment is
|
|
2277
|
+
given in format of OptionName<=>OptionValue. One option is given on each line.
|
|
2278
|
+
Different setting sign of Action and Mode options can be provided by Info
|
|
2279
|
+
option -AO (-ActOption, default to <!>); and different equal sign of single
|
|
2280
|
+
value assignment can be provided by Info option -ES, (-EqualSign, default to
|
|
2281
|
+
'<=>'). Multi-value assignments can be given in columns delimited with
|
|
2282
|
+
separator specified per option -SP (-Separator, default to '<:>'). It starts
|
|
2283
|
+
with a column title line for multi-value option names and the rest holds
|
|
2284
|
+
values corresponding to each column titles. The value information stops at
|
|
2285
|
+
the end of the file or when a new column name line or another single value
|
|
2286
|
+
assignment appears. If the last column is a multi-line value field, an
|
|
2287
|
+
additional separator must be appended for each line, including the column
|
|
2288
|
+
title line to end lines properly.
|
|
2289
|
+
|
|
2290
|
+
-IV or -InternVerion, internal version index. It is used to indicate a different
|
|
2291
|
+
version of data with minimal changes under the same DOI number. It is defaulted
|
|
2292
|
+
to 1 for a version control record with new created DOI number.
|
|
2293
|
+
|
|
2294
|
+
-KV or -KeyVakue, to set specialist defined key/value pairs via action -SD
|
|
2295
|
+
(-SetDataset). These special key/value pairs can be viewed per option -GD
|
|
2296
|
+
(-GetDataset). Specialists can record key/value pair information for
|
|
2297
|
+
specified datasets for personal or shared information. A key can be up to
|
|
2298
|
+
40 characters and a value can be up to 128 characters.
|
|
2299
|
+
|
|
2300
|
+
For example, syntax to set a key/value pair of 'test/test it' for d540000 on
|
|
2301
|
+
command line
|
|
2302
|
+
|
|
2303
|
+
dsarch d540000 SD -KV 'test=>test it'
|
|
2304
|
+
|
|
2305
|
+
and to view this pair
|
|
2306
|
+
|
|
2307
|
+
dsarch d540000 GD -KV test
|
|
2308
|
+
|
|
2309
|
+
and to view all available key/value pairs
|
|
2310
|
+
|
|
2311
|
+
dsarch d540000 GD -KV all
|
|
2312
|
+
|
|
2313
|
+
-LC or -Location (Alias: -LocationFlag), Storage localtion flags for CGD data,
|
|
2314
|
+
Web and Saved data files, and Help files, are C-CGD data, G-Glade Disk, O-Object
|
|
2315
|
+
Store, and B-Both Storage Systems. For Help files, an additional flag R is for
|
|
2316
|
+
Remote URL. This flag is also used in dataset table as web online file access
|
|
2317
|
+
location. The valid values are G or O, which mean to access online file on Glade
|
|
2318
|
+
or Object Store, respectively.
|
|
2319
|
+
|
|
2320
|
+
-LF or -LocalFile (Alias: -LocFile|-SourceFile), for local file names,
|
|
2321
|
+
also called original or source file names. A local file is a data file that
|
|
2322
|
+
is ready to be archived as Saved, Web or Help file. It is later archived
|
|
2323
|
+
onto RDA servers according to actions -AS (-ArchiveSavedFile), -AW
|
|
2324
|
+
(-ArchiveWebFile) or -AH (-ArchiveHelpFile). UNIX wildcard is acceptable if
|
|
2325
|
+
given on command line to match multiple local file names. Absolute or
|
|
2326
|
+
relative paths need to be added to the local file names if the data files are
|
|
2327
|
+
not located in the directory where this application is running.
|
|
2328
|
+
|
|
2329
|
+
-MC or -MD5Checksum, providing md5 checksum IDs of of the given data files
|
|
2330
|
+
on RDA Servers.
|
|
2331
|
+
|
|
2332
|
+
-ML or -MetaLink (Alias: -MetadataLink), providing a link to dataset, group or
|
|
2333
|
+
file level content metadata page. This link is normally auto-generated from
|
|
2334
|
+
inside of utility program 'gatherxml', which is triggered when Mode option
|
|
2335
|
+
-GX is present for actions of archiving or setting Web data files.
|
|
2336
|
+
|
|
2337
|
+
Although it is not recommended, specialists are allowed, under certain
|
|
2338
|
+
circumstances, to generate their own detail content meta data and put a link
|
|
2339
|
+
via this option, and it overrides the auto-generated link.
|
|
2340
|
+
|
|
2341
|
+
-ND or -NoteDocument (Alias: -DocumentNote|-DescDocument), for Document
|
|
2342
|
+
descriptions of a given dataset.
|
|
2343
|
+
|
|
2344
|
+
-NI or -NoteInternal (Alias: -InternalNote|-DescInternal|-DI), for internal
|
|
2345
|
+
data descriptions of a given group or the dataset itself.
|
|
2346
|
+
|
|
2347
|
+
-NS or -NoteSoftware (Alias: -SoftwareNote|-DescSoftware), for Software
|
|
2348
|
+
descriptions of a given dataset.
|
|
2349
|
+
|
|
2350
|
+
-NW or -NoteWeb (Alias: -WebNote|-DescWeb|-DW), for descriptions
|
|
2351
|
+
of Web data on RDA Server of a given group or the dataset itself.
|
|
2352
|
+
|
|
2353
|
+
-OB or -OrderBy (Alias: -OrderByPattern), temporal patterns used to order
|
|
2354
|
+
filelist according to the string values of field names specified per
|
|
2355
|
+
option -ON (-OrderNames). 'YYYYmon', for example, means the order should
|
|
2356
|
+
be based on on 4 digit year and three letter lowercase month names as 'jan',
|
|
2357
|
+
'feb', and etc. Check option -ON (-OrderNames) for example of ordering
|
|
2358
|
+
files by temporal patterns.
|
|
2359
|
+
|
|
2360
|
+
-OD or -OriginDataset, for an original dataset number; used for action
|
|
2361
|
+
-MV (-MoveFile).
|
|
2362
|
+
|
|
2363
|
+
-OG or -OriginGroup, for original group indices, used for action -CG
|
|
2364
|
+
(-ChangeGroup) to change group indices of specified data files of a
|
|
2365
|
+
given dataset. Check option -GI (-GroupIndex) for more information.
|
|
2366
|
+
|
|
2367
|
+
-OT or -OriginType, original web file types. This option is used only
|
|
2368
|
+
for action -MV (-MoveFile) to move files by changing file types.
|
|
2369
|
+
|
|
2370
|
+
-PI or -ParentIndex (Alias: -ParentGroupIndex), for parent group indices.
|
|
2371
|
+
This is considered only if a group is divided further into subgroups.
|
|
2372
|
+
Subgrouping can continue going down to as many levels as needed.
|
|
2373
|
+
|
|
2374
|
+
-QF or -QuasarFile, for Quasar backup file names. During Quasar backing up
|
|
2375
|
+
per action -AQ (-ArchiveQuasarFile), a quasar file name must be specified
|
|
2376
|
+
by this option.
|
|
2377
|
+
|
|
2378
|
+
-QT or -QuasarFileType (Alias: -QuasarType|-BackupType), Quasar file
|
|
2379
|
+
types, a single letter property flag, B-Quasar Backup only and D-Quasar
|
|
2380
|
+
Backup and Drdata.
|
|
2381
|
+
|
|
2382
|
+
-RF or -OriginFile, for original Web file names used for action -MV
|
|
2383
|
+
(-MoveFile) to move data files from one dataset specified by option -OD
|
|
2384
|
+
(-OriginDataset) to a new dataset specified by option -DS (-Dataset).
|
|
2385
|
+
Set this option only when original file names are different from the new
|
|
2386
|
+
ones.
|
|
2387
|
+
|
|
2388
|
+
-SF or -SavedFile, for Saved file names. During Saved data archiving
|
|
2389
|
+
per action -AS (-ArchiveSavedFile), the local file names given by option -LF
|
|
2390
|
+
(-LocalFile) are used as default for the saved file names unless different
|
|
2391
|
+
file names are specified by this option.
|
|
2392
|
+
|
|
2393
|
+
-SP or -SavedPath, for saved file paths relative to the saved data home directory.
|
|
2394
|
+
For action -AS (-ArchiveSavedFile), if this option is not specified, the path
|
|
2395
|
+
values previously saved in RDADB for given dataset, and groups if specified,
|
|
2396
|
+
are used. For action -SG (-SetGroup) and -SD -(-SetDataset), path values given by
|
|
2397
|
+
this option are saved into RDADB for the specified groups and dataset.
|
|
2398
|
+
|
|
2399
|
+
The group saved paths matching well with groups are important for NCAR internal
|
|
2400
|
+
users to list group/file information on the the data storage disks directly via
|
|
2401
|
+
utility program 'rdals'. One and only one single directory must be assign to a
|
|
2402
|
+
top level group, one additional directory should be appended for each sub-group
|
|
2403
|
+
down. A web path relative to the web data home directory is saved in each group
|
|
2404
|
+
record. In this way, the web path values match exactly the physical directory
|
|
2405
|
+
tree of a specified dataset.
|
|
2406
|
+
|
|
2407
|
+
-SR or -Source, for source of a Help file contributed from.
|
|
2408
|
+
|
|
2409
|
+
-ST or -SavedFileType (Alias: -SavedType|-SavedArchiveType), Saved file types,
|
|
2410
|
+
a single letter property flag, P-Primary, R-Original, W-Work, S-Scratch,
|
|
2411
|
+
and V-Version control. If only one type is passed in, all Saved files are
|
|
2412
|
+
assumed to be the same type.
|
|
2413
|
+
|
|
2414
|
+
-SZ or -Size, for file sizes in bytes. During data archiving the sizes of data
|
|
2415
|
+
files are gathered dynamically from the local files given per option -LF
|
|
2416
|
+
(-LocalFile) unless the sizes are specified by this option otherwise.
|
|
2417
|
+
|
|
2418
|
+
-TG or -TopGroupIndex (Alias: -TopGroup), top group index values for Saved/Web
|
|
2419
|
+
files belong to the top group or its subgroups.
|
|
2420
|
+
|
|
2421
|
+
-TI or -Title, for dataset or group titles. Dataset title is readonly for
|
|
2422
|
+
'dsarch'; use Metadata Editor to modify it.
|
|
2423
|
+
|
|
2424
|
+
-TL or -ThreddLink, providing a link to file level Thredd Catalog page.
|
|
2425
|
+
Set this value to Y if Thredd Catalog information is gathered for a file.
|
|
2426
|
+
|
|
2427
|
+
-VI or -VersionIndex, version control index to have DOI/Version control on
|
|
2428
|
+
a given dataset.
|
|
2429
|
+
|
|
2430
|
+
-VT or -VersionStatus, version control status, A-Active, P-Pending and H-History.
|
|
2431
|
+
|
|
2432
|
+
-WF or -WebFile, for file names on RDA Servers. During Web data archiving
|
|
2433
|
+
per action -AW (-ArchiveWebFile), the local file names given by option -LF
|
|
2434
|
+
(-LocalFile) are used as default for the Web file names unless different
|
|
2435
|
+
file names are specified by this option.
|
|
2436
|
+
|
|
2437
|
+
-WH or -WebHome, (Alias: -WebDataHome), for web Web data home directory
|
|
2438
|
+
other than the default location. Use -SD -(-SetDataset), to set web data
|
|
2439
|
+
home directory into RDADB for the specified dataset.
|
|
2440
|
+
|
|
2441
|
+
-WP or -WebPath, for web file paths relative to the Web data home directory. For
|
|
2442
|
+
action -AW (-ArchiveWebFile), if this option is not specified, the path
|
|
2443
|
+
values previously saved in RDADB for given dataset, and groups if specified,
|
|
2444
|
+
are used. For action -SG (-SetGroup) and -SD -(-SetDataset), path values given by
|
|
2445
|
+
this option are saved into RDADB for the specified groups and dataset.
|
|
2446
|
+
|
|
2447
|
+
The group web paths matching well with groups are important for NCAR internal
|
|
2448
|
+
users to list group/file information on the the data storage disks directly via
|
|
2449
|
+
utility program 'rdals'. One and only one single directory must be assign to a
|
|
2450
|
+
top level group, one additional directory should be appended for each sub-group
|
|
2451
|
+
down. A web path relative to the web data home directory is saved in each group
|
|
2452
|
+
record. In this way, the web path values match exactly the physical directory
|
|
2453
|
+
tree of a specified dataset.
|
|
2454
|
+
|
|
2455
|
+
-WT or -WebFileType (Alias: -WebType|-WebArchiveType), web file types,
|
|
2456
|
+
D-Data and N-NCAR Data. If only one type is passed in, all Saved files are
|
|
2457
|
+
assumed to be the same type.
|
|
2458
|
+
|
|
2459
|
+
-WU or -WebURL (Alias: -URL|-WebAddress), Web URL for a Help file to link it to
|
|
2460
|
+
a remote web server, such as github.com. This option works with Action -SH
|
|
2461
|
+
(-SetHelpFile) to create a remote Document or Software file record in RDADB
|
|
2462
|
+
without local copy on RDA Servers.
|