rda-python-dsarch 2.0.20__tar.gz → 3.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rda_python_dsarch-2.0.20/src/rda_python_dsarch.egg-info → rda_python_dsarch-3.0.0}/PKG-INFO +73 -23
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/README.md +72 -22
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/pyproject.toml +1 -2
- rda_python_dsarch-3.0.0/src/rda_python_dsarch/__init__.py +29 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/dsarch.py +3 -11
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0/src/rda_python_dsarch.egg-info}/PKG-INFO +73 -23
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/SOURCES.txt +0 -1
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/entry_points.txt +0 -1
- rda_python_dsarch-2.0.20/src/rda_python_dsarch/__init__.py +0 -1
- rda_python_dsarch-2.0.20/src/rda_python_dsarch/dsarch_setup.py +0 -69
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/LICENSE +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/MANIFEST.in +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/setup.cfg +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/PgArch.py +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/PgMeta.py +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/ds_arch.py +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/dsarch.usg +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/pg_arch.py +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch/pg_meta.py +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/dependency_links.txt +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/requires.txt +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/top_level.txt +0 -0
- {rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/tests/test_dsarch.py +0 -0
{rda_python_dsarch-2.0.20/src/rda_python_dsarch.egg-info → rda_python_dsarch-3.0.0}/PKG-INFO
RENAMED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rda_python_dsarch
|
|
3
|
-
Version:
|
|
3
|
+
Version: 3.0.0
|
|
4
4
|
Summary: RDA Python package for utility program dsarch archive RDA data
|
|
5
5
|
Author-email: Zaihua Ji <zji@ucar.edu>
|
|
6
6
|
Project-URL: Homepage, https://github.com/NCAR/rda-python-dsarch
|
|
@@ -56,39 +56,85 @@ The inheritance chain is `DsArch -> PgArch -> PgMeta -> PgCMD/PgSplit`, so
|
|
|
56
56
|
`DsArch` instances expose every option, path, count, and database helper
|
|
57
57
|
through a single object.
|
|
58
58
|
|
|
59
|
-
##
|
|
60
|
-
|
|
61
|
-
`dsarch` is executed as the common user `gdexdata` via the `rda_python_setuid`
|
|
62
|
-
setuid mechanism. `rda_python_setuid` is declared as a dependency and is
|
|
63
|
-
installed automatically with this package.
|
|
59
|
+
## Environment setup
|
|
64
60
|
|
|
65
|
-
|
|
61
|
+
Create a Python environment first; package installs in the next section run
|
|
62
|
+
inside whichever environment you activate here.
|
|
66
63
|
|
|
67
|
-
|
|
64
|
+
### Option A — Python venv (DECS machines)
|
|
68
65
|
|
|
69
66
|
```bash
|
|
70
67
|
python3 -m venv $ENVHOME # e.g. /glade/u/home/gdexdata/gdexmsenv
|
|
71
68
|
source $ENVHOME/bin/activate
|
|
72
|
-
pip install rda_python_dsarch
|
|
73
69
|
```
|
|
74
70
|
|
|
75
|
-
|
|
71
|
+
### Option B — Conda (DAV/Casper)
|
|
76
72
|
|
|
77
73
|
```bash
|
|
78
|
-
conda
|
|
79
|
-
|
|
74
|
+
conda create --prefix $ENVHOME python=3.12 # e.g. /glade/work/gdexdata/conda-envs/pg-gdex
|
|
75
|
+
conda activate $ENVHOME
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Installing rda-python-dsarch
|
|
79
|
+
|
|
80
|
+
Pick whichever install mode fits your workflow. All four pull in the
|
|
81
|
+
transitive dependencies (`rda_python_common`, `rda_python_setuid`,
|
|
82
|
+
`rda_python_miscs`) automatically.
|
|
83
|
+
|
|
84
|
+
For local development, clone this repo alongside your project and install it
|
|
85
|
+
in editable mode so that changes are picked up without re-installing:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
git clone https://github.com/NCAR/rda-python-dsarch.git
|
|
89
|
+
cd rda-python-dsarch
|
|
90
|
+
pip install -e .
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
To test a specific branch (e.g. an in-progress feature or fix branch), pass
|
|
94
|
+
`-b/--branch` to `git clone`:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
git clone -b <branch-name> https://github.com/NCAR/rda-python-dsarch.git
|
|
98
|
+
cd rda-python-dsarch
|
|
99
|
+
pip install -e .
|
|
80
100
|
```
|
|
81
101
|
|
|
82
|
-
|
|
102
|
+
For a regular (non-editable) install from a checkout:
|
|
83
103
|
|
|
84
|
-
|
|
104
|
+
```bash
|
|
105
|
+
pip install /path/to/rda-python-dsarch
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For a production install on a system that uses the published distribution:
|
|
85
109
|
|
|
86
110
|
```bash
|
|
87
|
-
|
|
88
|
-
|
|
111
|
+
pip install rda_python_dsarch
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Setuid Setup
|
|
89
115
|
|
|
90
|
-
|
|
91
|
-
|
|
116
|
+
`dsarch` is executed as the common user `PGLOG['COMMONUSER']` (default
|
|
117
|
+
`gdexdata`) via the `rda_python_setuid` setuid mechanism, which is pulled
|
|
118
|
+
in automatically as a dependency. After `pip install` above, choose one
|
|
119
|
+
of the wiring options below.
|
|
120
|
+
|
|
121
|
+
### Full setuid install (requires sudo access to COMMONUSER)
|
|
122
|
+
|
|
123
|
+
Run these steps once per environment:
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
# 1. Compile the pywrapper C binary (once per environment):
|
|
127
|
+
pywrapper-install -c|--compile -n|--username gdexdata
|
|
128
|
+
|
|
129
|
+
# 2. Wire up dsarch as a setuid entry (or use 'all' to link every setuid_* at once):
|
|
130
|
+
pywrapper-install -l|--link dsarch
|
|
131
|
+
pywrapper-install -l|--link all
|
|
132
|
+
|
|
133
|
+
# 3. Optionally, install a pgstart_<loginname> binary so <loginname> (any
|
|
134
|
+
# user in the same group as PGLOG['COMMONUSER']) can run commands as
|
|
135
|
+
# themselves. Run either by PGLOG['ADMINUSER'] (default zji, if it has
|
|
136
|
+
# 'sudo -u <loginname>'), or by <loginname> directly:
|
|
137
|
+
pywrapper-install -p|--pgstart -n|--username <loginname>
|
|
92
138
|
```
|
|
93
139
|
|
|
94
140
|
`pywrapper-install` with no arguments displays the full user guide.
|
|
@@ -99,21 +145,25 @@ Users who do not need the setuid mechanism can create a direct symlink instead:
|
|
|
99
145
|
|
|
100
146
|
```bash
|
|
101
147
|
pywrapper-install -l|--link dsarch -s|--simple
|
|
148
|
+
pywrapper-install -l|--link all -s|--simple # or link every setuid_* at once
|
|
102
149
|
```
|
|
103
150
|
|
|
104
151
|
This creates `bin/dsarch -> bin/setuid_dsarch` and the program runs as the
|
|
105
152
|
current user with no privilege change.
|
|
106
153
|
|
|
107
|
-
###
|
|
154
|
+
### Update an existing installation (no sudo required)
|
|
108
155
|
|
|
109
|
-
|
|
156
|
+
When the package is upgraded and a new `pywrapper.c` is bundled, recompile and
|
|
157
|
+
reinstall all setuid binaries using the existing `pgstart_*` binaries:
|
|
110
158
|
|
|
111
159
|
```bash
|
|
112
|
-
|
|
160
|
+
pywrapper-install -u|--update
|
|
113
161
|
```
|
|
114
162
|
|
|
115
|
-
|
|
116
|
-
|
|
163
|
+
### Setup guide
|
|
164
|
+
|
|
165
|
+
The shared setuid setup guide is shown automatically if `setuid_dsarch` is
|
|
166
|
+
invoked directly before the setuid wrapper has been configured.
|
|
117
167
|
|
|
118
168
|
## Documentation sync
|
|
119
169
|
|
|
@@ -38,39 +38,85 @@ The inheritance chain is `DsArch -> PgArch -> PgMeta -> PgCMD/PgSplit`, so
|
|
|
38
38
|
`DsArch` instances expose every option, path, count, and database helper
|
|
39
39
|
through a single object.
|
|
40
40
|
|
|
41
|
-
##
|
|
42
|
-
|
|
43
|
-
`dsarch` is executed as the common user `gdexdata` via the `rda_python_setuid`
|
|
44
|
-
setuid mechanism. `rda_python_setuid` is declared as a dependency and is
|
|
45
|
-
installed automatically with this package.
|
|
41
|
+
## Environment setup
|
|
46
42
|
|
|
47
|
-
|
|
43
|
+
Create a Python environment first; package installs in the next section run
|
|
44
|
+
inside whichever environment you activate here.
|
|
48
45
|
|
|
49
|
-
|
|
46
|
+
### Option A — Python venv (DECS machines)
|
|
50
47
|
|
|
51
48
|
```bash
|
|
52
49
|
python3 -m venv $ENVHOME # e.g. /glade/u/home/gdexdata/gdexmsenv
|
|
53
50
|
source $ENVHOME/bin/activate
|
|
54
|
-
pip install rda_python_dsarch
|
|
55
51
|
```
|
|
56
52
|
|
|
57
|
-
|
|
53
|
+
### Option B — Conda (DAV/Casper)
|
|
58
54
|
|
|
59
55
|
```bash
|
|
60
|
-
conda
|
|
61
|
-
|
|
56
|
+
conda create --prefix $ENVHOME python=3.12 # e.g. /glade/work/gdexdata/conda-envs/pg-gdex
|
|
57
|
+
conda activate $ENVHOME
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## Installing rda-python-dsarch
|
|
61
|
+
|
|
62
|
+
Pick whichever install mode fits your workflow. All four pull in the
|
|
63
|
+
transitive dependencies (`rda_python_common`, `rda_python_setuid`,
|
|
64
|
+
`rda_python_miscs`) automatically.
|
|
65
|
+
|
|
66
|
+
For local development, clone this repo alongside your project and install it
|
|
67
|
+
in editable mode so that changes are picked up without re-installing:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
git clone https://github.com/NCAR/rda-python-dsarch.git
|
|
71
|
+
cd rda-python-dsarch
|
|
72
|
+
pip install -e .
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
To test a specific branch (e.g. an in-progress feature or fix branch), pass
|
|
76
|
+
`-b/--branch` to `git clone`:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
git clone -b <branch-name> https://github.com/NCAR/rda-python-dsarch.git
|
|
80
|
+
cd rda-python-dsarch
|
|
81
|
+
pip install -e .
|
|
62
82
|
```
|
|
63
83
|
|
|
64
|
-
|
|
84
|
+
For a regular (non-editable) install from a checkout:
|
|
65
85
|
|
|
66
|
-
|
|
86
|
+
```bash
|
|
87
|
+
pip install /path/to/rda-python-dsarch
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
For a production install on a system that uses the published distribution:
|
|
67
91
|
|
|
68
92
|
```bash
|
|
69
|
-
|
|
70
|
-
|
|
93
|
+
pip install rda_python_dsarch
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Setuid Setup
|
|
71
97
|
|
|
72
|
-
|
|
73
|
-
|
|
98
|
+
`dsarch` is executed as the common user `PGLOG['COMMONUSER']` (default
|
|
99
|
+
`gdexdata`) via the `rda_python_setuid` setuid mechanism, which is pulled
|
|
100
|
+
in automatically as a dependency. After `pip install` above, choose one
|
|
101
|
+
of the wiring options below.
|
|
102
|
+
|
|
103
|
+
### Full setuid install (requires sudo access to COMMONUSER)
|
|
104
|
+
|
|
105
|
+
Run these steps once per environment:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
# 1. Compile the pywrapper C binary (once per environment):
|
|
109
|
+
pywrapper-install -c|--compile -n|--username gdexdata
|
|
110
|
+
|
|
111
|
+
# 2. Wire up dsarch as a setuid entry (or use 'all' to link every setuid_* at once):
|
|
112
|
+
pywrapper-install -l|--link dsarch
|
|
113
|
+
pywrapper-install -l|--link all
|
|
114
|
+
|
|
115
|
+
# 3. Optionally, install a pgstart_<loginname> binary so <loginname> (any
|
|
116
|
+
# user in the same group as PGLOG['COMMONUSER']) can run commands as
|
|
117
|
+
# themselves. Run either by PGLOG['ADMINUSER'] (default zji, if it has
|
|
118
|
+
# 'sudo -u <loginname>'), or by <loginname> directly:
|
|
119
|
+
pywrapper-install -p|--pgstart -n|--username <loginname>
|
|
74
120
|
```
|
|
75
121
|
|
|
76
122
|
`pywrapper-install` with no arguments displays the full user guide.
|
|
@@ -81,21 +127,25 @@ Users who do not need the setuid mechanism can create a direct symlink instead:
|
|
|
81
127
|
|
|
82
128
|
```bash
|
|
83
129
|
pywrapper-install -l|--link dsarch -s|--simple
|
|
130
|
+
pywrapper-install -l|--link all -s|--simple # or link every setuid_* at once
|
|
84
131
|
```
|
|
85
132
|
|
|
86
133
|
This creates `bin/dsarch -> bin/setuid_dsarch` and the program runs as the
|
|
87
134
|
current user with no privilege change.
|
|
88
135
|
|
|
89
|
-
###
|
|
136
|
+
### Update an existing installation (no sudo required)
|
|
90
137
|
|
|
91
|
-
|
|
138
|
+
When the package is upgraded and a new `pywrapper.c` is bundled, recompile and
|
|
139
|
+
reinstall all setuid binaries using the existing `pgstart_*` binaries:
|
|
92
140
|
|
|
93
141
|
```bash
|
|
94
|
-
|
|
142
|
+
pywrapper-install -u|--update
|
|
95
143
|
```
|
|
96
144
|
|
|
97
|
-
|
|
98
|
-
|
|
145
|
+
### Setup guide
|
|
146
|
+
|
|
147
|
+
The shared setuid setup guide is shown automatically if `setuid_dsarch` is
|
|
148
|
+
invoked directly before the setuid wrapper has been configured.
|
|
99
149
|
|
|
100
150
|
## Documentation sync
|
|
101
151
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "rda_python_dsarch"
|
|
7
|
-
version = "
|
|
7
|
+
version = "3.0.0"
|
|
8
8
|
authors = [
|
|
9
9
|
{ name="Zaihua Ji", email="zji@ucar.edu" },
|
|
10
10
|
]
|
|
@@ -33,4 +33,3 @@ pythonpath = [
|
|
|
33
33
|
|
|
34
34
|
[project.scripts]
|
|
35
35
|
"setuid_dsarch" = "rda_python_dsarch.dsarch:main"
|
|
36
|
-
"dsarch-setup" = "rda_python_dsarch.dsarch_setup:main"
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
"""rda_python_dsarch: dataset archive (dsarch) utility package.
|
|
2
|
+
|
|
3
|
+
This package exposes two parallel APIs:
|
|
4
|
+
|
|
5
|
+
1. Legacy module-based API (back-compat). Import the capitalized
|
|
6
|
+
submodules and call their module-level functions, e.g.::
|
|
7
|
+
|
|
8
|
+
from rda_python_dsarch import PgArch, PgMeta
|
|
9
|
+
|
|
10
|
+
2. Class-based API (preferred for new code). Import the class from the
|
|
11
|
+
lower-case module and either instantiate or subclass it, e.g.::
|
|
12
|
+
|
|
13
|
+
from rda_python_dsarch.pg_arch import PgArch
|
|
14
|
+
from rda_python_dsarch.pg_meta import PgMeta
|
|
15
|
+
|
|
16
|
+
The legacy submodules are eagerly imported below so that
|
|
17
|
+
``from rda_python_dsarch import PgArch`` continues to return the module
|
|
18
|
+
object that existing callers expect.
|
|
19
|
+
"""
|
|
20
|
+
|
|
21
|
+
from . import PgArch, PgMeta
|
|
22
|
+
|
|
23
|
+
__version__ = "2.0.21"
|
|
24
|
+
|
|
25
|
+
__all__ = [
|
|
26
|
+
"PgArch",
|
|
27
|
+
"PgMeta",
|
|
28
|
+
"__version__",
|
|
29
|
+
]
|
|
@@ -3204,7 +3204,6 @@ class DsArch(PgArch, PgMeta):
|
|
|
3204
3204
|
tmpds = tmpgs = None
|
|
3205
3205
|
bidx = chksize = 0
|
|
3206
3206
|
bucket = self.PGLOG['OBJCTBKT']
|
|
3207
|
-
rcnt = len(self.PGLOG['WEBHOSTS'])
|
|
3208
3207
|
s = 's' if self.ALLCNT > 1 else ''
|
|
3209
3208
|
if 'OD' not in self.params:
|
|
3210
3209
|
self.params['OD'] = dsid
|
|
@@ -3353,7 +3352,6 @@ class DsArch(PgArch, PgMeta):
|
|
|
3353
3352
|
tmpds = None
|
|
3354
3353
|
bidx = chksize = 0
|
|
3355
3354
|
bucket = self.PGLOG['OBJCTBKT']
|
|
3356
|
-
rcnt = len(self.PGLOG['WEBHOSTS'])
|
|
3357
3355
|
s = 's' if self.ALLCNT > 1 else ''
|
|
3358
3356
|
if 'OD' not in self.params:
|
|
3359
3357
|
self.params['OD'] = dsid
|
|
@@ -3632,7 +3630,6 @@ class DsArch(PgArch, PgMeta):
|
|
|
3632
3630
|
bucket = self.PGLOG['OBJCTBKT']
|
|
3633
3631
|
s = 's' if self.ALLCNT > 1 else ''
|
|
3634
3632
|
bidx = chksize = 0
|
|
3635
|
-
rcnt = len(self.PGLOG['WEBHOSTS'])
|
|
3636
3633
|
self.pglog("Delete {} Web file{} from {} ...".format(self.ALLCNT, s, dsid), self.WARNLG)
|
|
3637
3634
|
reorder = metatotal = metacnt = dcnt = mcnt = ocnt = wcnt = 0
|
|
3638
3635
|
self.cache_group_info(self.ALLCNT, 0)
|
|
@@ -3741,7 +3738,6 @@ class DsArch(PgArch, PgMeta):
|
|
|
3741
3738
|
dcnd = "dsid = '{}'".format(dsid)
|
|
3742
3739
|
s = 's' if self.ALLCNT > 1 else ''
|
|
3743
3740
|
bidx = chksize = 0
|
|
3744
|
-
rcnt = len(self.PGLOG['WEBHOSTS'])
|
|
3745
3741
|
self.pglog("Delete {} Help file{} from {} ...".format(self.ALLCNT, s, dsid), self.WARNLG)
|
|
3746
3742
|
reorder = dcnt = hcnt = ocnt = mcnt = 0
|
|
3747
3743
|
self.cache_group_info(self.ALLCNT, 0)
|
|
@@ -3787,9 +3783,7 @@ class DsArch(PgArch, PgMeta):
|
|
|
3787
3783
|
if pgrec['url']:
|
|
3788
3784
|
fcnt = 1
|
|
3789
3785
|
else:
|
|
3790
|
-
fcnt =
|
|
3791
|
-
for j in range(rcnt):
|
|
3792
|
-
fcnt += self.delete_local_file(afile, self.PGOPT['emerol'])
|
|
3786
|
+
fcnt = self.delete_local_file(afile, self.PGOPT['emerol'])
|
|
3793
3787
|
if fcnt:
|
|
3794
3788
|
hcnt += 1
|
|
3795
3789
|
hflag = 0
|
|
@@ -4957,11 +4951,9 @@ def main():
|
|
|
4957
4951
|
Instantiates DsArch, parses arguments via read_parameters(), executes the
|
|
4958
4952
|
requested action via start_actions(), and exits cleanly.
|
|
4959
4953
|
"""
|
|
4960
|
-
import
|
|
4961
|
-
if os.path.basename(sys.argv[0]) == 'setuid_dsarch':
|
|
4962
|
-
from rda_python_dsarch.dsarch_setup import main as setup_main
|
|
4963
|
-
setup_main()
|
|
4954
|
+
from rda_python_setuid.setup_guide import show_setup_guide
|
|
4964
4955
|
object = DsArch()
|
|
4956
|
+
show_setup_guide(object, 'rda_python_dsarch', ['dsarch'])
|
|
4965
4957
|
object.read_parameters()
|
|
4966
4958
|
object.start_actions()
|
|
4967
4959
|
object.pgexit(0)
|
{rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0/src/rda_python_dsarch.egg-info}/PKG-INFO
RENAMED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rda_python_dsarch
|
|
3
|
-
Version:
|
|
3
|
+
Version: 3.0.0
|
|
4
4
|
Summary: RDA Python package for utility program dsarch archive RDA data
|
|
5
5
|
Author-email: Zaihua Ji <zji@ucar.edu>
|
|
6
6
|
Project-URL: Homepage, https://github.com/NCAR/rda-python-dsarch
|
|
@@ -56,39 +56,85 @@ The inheritance chain is `DsArch -> PgArch -> PgMeta -> PgCMD/PgSplit`, so
|
|
|
56
56
|
`DsArch` instances expose every option, path, count, and database helper
|
|
57
57
|
through a single object.
|
|
58
58
|
|
|
59
|
-
##
|
|
60
|
-
|
|
61
|
-
`dsarch` is executed as the common user `gdexdata` via the `rda_python_setuid`
|
|
62
|
-
setuid mechanism. `rda_python_setuid` is declared as a dependency and is
|
|
63
|
-
installed automatically with this package.
|
|
59
|
+
## Environment setup
|
|
64
60
|
|
|
65
|
-
|
|
61
|
+
Create a Python environment first; package installs in the next section run
|
|
62
|
+
inside whichever environment you activate here.
|
|
66
63
|
|
|
67
|
-
|
|
64
|
+
### Option A — Python venv (DECS machines)
|
|
68
65
|
|
|
69
66
|
```bash
|
|
70
67
|
python3 -m venv $ENVHOME # e.g. /glade/u/home/gdexdata/gdexmsenv
|
|
71
68
|
source $ENVHOME/bin/activate
|
|
72
|
-
pip install rda_python_dsarch
|
|
73
69
|
```
|
|
74
70
|
|
|
75
|
-
|
|
71
|
+
### Option B — Conda (DAV/Casper)
|
|
76
72
|
|
|
77
73
|
```bash
|
|
78
|
-
conda
|
|
79
|
-
|
|
74
|
+
conda create --prefix $ENVHOME python=3.12 # e.g. /glade/work/gdexdata/conda-envs/pg-gdex
|
|
75
|
+
conda activate $ENVHOME
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Installing rda-python-dsarch
|
|
79
|
+
|
|
80
|
+
Pick whichever install mode fits your workflow. All four pull in the
|
|
81
|
+
transitive dependencies (`rda_python_common`, `rda_python_setuid`,
|
|
82
|
+
`rda_python_miscs`) automatically.
|
|
83
|
+
|
|
84
|
+
For local development, clone this repo alongside your project and install it
|
|
85
|
+
in editable mode so that changes are picked up without re-installing:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
git clone https://github.com/NCAR/rda-python-dsarch.git
|
|
89
|
+
cd rda-python-dsarch
|
|
90
|
+
pip install -e .
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
To test a specific branch (e.g. an in-progress feature or fix branch), pass
|
|
94
|
+
`-b/--branch` to `git clone`:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
git clone -b <branch-name> https://github.com/NCAR/rda-python-dsarch.git
|
|
98
|
+
cd rda-python-dsarch
|
|
99
|
+
pip install -e .
|
|
80
100
|
```
|
|
81
101
|
|
|
82
|
-
|
|
102
|
+
For a regular (non-editable) install from a checkout:
|
|
83
103
|
|
|
84
|
-
|
|
104
|
+
```bash
|
|
105
|
+
pip install /path/to/rda-python-dsarch
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For a production install on a system that uses the published distribution:
|
|
85
109
|
|
|
86
110
|
```bash
|
|
87
|
-
|
|
88
|
-
|
|
111
|
+
pip install rda_python_dsarch
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Setuid Setup
|
|
89
115
|
|
|
90
|
-
|
|
91
|
-
|
|
116
|
+
`dsarch` is executed as the common user `PGLOG['COMMONUSER']` (default
|
|
117
|
+
`gdexdata`) via the `rda_python_setuid` setuid mechanism, which is pulled
|
|
118
|
+
in automatically as a dependency. After `pip install` above, choose one
|
|
119
|
+
of the wiring options below.
|
|
120
|
+
|
|
121
|
+
### Full setuid install (requires sudo access to COMMONUSER)
|
|
122
|
+
|
|
123
|
+
Run these steps once per environment:
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
# 1. Compile the pywrapper C binary (once per environment):
|
|
127
|
+
pywrapper-install -c|--compile -n|--username gdexdata
|
|
128
|
+
|
|
129
|
+
# 2. Wire up dsarch as a setuid entry (or use 'all' to link every setuid_* at once):
|
|
130
|
+
pywrapper-install -l|--link dsarch
|
|
131
|
+
pywrapper-install -l|--link all
|
|
132
|
+
|
|
133
|
+
# 3. Optionally, install a pgstart_<loginname> binary so <loginname> (any
|
|
134
|
+
# user in the same group as PGLOG['COMMONUSER']) can run commands as
|
|
135
|
+
# themselves. Run either by PGLOG['ADMINUSER'] (default zji, if it has
|
|
136
|
+
# 'sudo -u <loginname>'), or by <loginname> directly:
|
|
137
|
+
pywrapper-install -p|--pgstart -n|--username <loginname>
|
|
92
138
|
```
|
|
93
139
|
|
|
94
140
|
`pywrapper-install` with no arguments displays the full user guide.
|
|
@@ -99,21 +145,25 @@ Users who do not need the setuid mechanism can create a direct symlink instead:
|
|
|
99
145
|
|
|
100
146
|
```bash
|
|
101
147
|
pywrapper-install -l|--link dsarch -s|--simple
|
|
148
|
+
pywrapper-install -l|--link all -s|--simple # or link every setuid_* at once
|
|
102
149
|
```
|
|
103
150
|
|
|
104
151
|
This creates `bin/dsarch -> bin/setuid_dsarch` and the program runs as the
|
|
105
152
|
current user with no privilege change.
|
|
106
153
|
|
|
107
|
-
###
|
|
154
|
+
### Update an existing installation (no sudo required)
|
|
108
155
|
|
|
109
|
-
|
|
156
|
+
When the package is upgraded and a new `pywrapper.c` is bundled, recompile and
|
|
157
|
+
reinstall all setuid binaries using the existing `pgstart_*` binaries:
|
|
110
158
|
|
|
111
159
|
```bash
|
|
112
|
-
|
|
160
|
+
pywrapper-install -u|--update
|
|
113
161
|
```
|
|
114
162
|
|
|
115
|
-
|
|
116
|
-
|
|
163
|
+
### Setup guide
|
|
164
|
+
|
|
165
|
+
The shared setuid setup guide is shown automatically if `setuid_dsarch` is
|
|
166
|
+
invoked directly before the setuid wrapper has been configured.
|
|
117
167
|
|
|
118
168
|
## Documentation sync
|
|
119
169
|
|
{rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/SOURCES.txt
RENAMED
|
@@ -8,7 +8,6 @@ src/rda_python_dsarch/__init__.py
|
|
|
8
8
|
src/rda_python_dsarch/ds_arch.py
|
|
9
9
|
src/rda_python_dsarch/dsarch.py
|
|
10
10
|
src/rda_python_dsarch/dsarch.usg
|
|
11
|
-
src/rda_python_dsarch/dsarch_setup.py
|
|
12
11
|
src/rda_python_dsarch/pg_arch.py
|
|
13
12
|
src/rda_python_dsarch/pg_meta.py
|
|
14
13
|
src/rda_python_dsarch.egg-info/PKG-INFO
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
|
|
@@ -1,69 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env python3
|
|
2
|
-
#
|
|
3
|
-
##################################################################################
|
|
4
|
-
#
|
|
5
|
-
# Title: dsarch-setup
|
|
6
|
-
# Author: Zaihua Ji, zji@ucar.edu
|
|
7
|
-
# Date: 2026-05-12
|
|
8
|
-
# Purpose: Display the setuid setup guide for dsarch after pip install.
|
|
9
|
-
# Run automatically when setuid_dsarch is invoked directly instead
|
|
10
|
-
# of through the dsarch -> pywrapper setuid symlink.
|
|
11
|
-
#
|
|
12
|
-
# Github: https://github.com/NCAR/rda-python-dsarch.git
|
|
13
|
-
#
|
|
14
|
-
##################################################################################
|
|
15
|
-
|
|
16
|
-
import os
|
|
17
|
-
import sys
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
SETUP_GUIDE = """
|
|
21
|
-
dsarch - Setuid Setup Guide
|
|
22
|
-
===========================
|
|
23
|
-
|
|
24
|
-
dsarch must be run as the common user 'gdexdata' via the rda_python_setuid
|
|
25
|
-
mechanism. rda_python_setuid is installed automatically as a dependency.
|
|
26
|
-
|
|
27
|
-
If you are seeing this message after running 'setuid_dsarch' directly, it
|
|
28
|
-
means the setuid wrapper has not been set up yet. Follow the steps below.
|
|
29
|
-
|
|
30
|
-
Run 'pywrapper-install' with no arguments for the full pywrapper user guide.
|
|
31
|
-
|
|
32
|
-
Environment Setup
|
|
33
|
-
-----------------
|
|
34
|
-
|
|
35
|
-
Option A - Python venv (DECS machines):
|
|
36
|
-
python3 -m venv $ENVHOME # e.g. /glade/u/home/gdexdata/gdexmsenv
|
|
37
|
-
source $ENVHOME/bin/activate
|
|
38
|
-
pip install rda_python_dsarch
|
|
39
|
-
|
|
40
|
-
Option B - Conda (DAV/Casper):
|
|
41
|
-
conda activate pg-gdex # e.g. /glade/work/gdexdata/conda-envs/pg-gdex
|
|
42
|
-
pip install rda_python_dsarch
|
|
43
|
-
|
|
44
|
-
Full Setuid Install (requires sudo access to gdexdata)
|
|
45
|
-
------------------------------------------------------
|
|
46
|
-
|
|
47
|
-
# Compile the pywrapper C binary (once per environment):
|
|
48
|
-
pywrapper-install -c|--compile -u|--user gdexdata
|
|
49
|
-
|
|
50
|
-
# Wire up dsarch as a setuid entry:
|
|
51
|
-
pywrapper-install -l|--link dsarch -u|--user gdexdata
|
|
52
|
-
|
|
53
|
-
Simple Install (no sudo required, runs as current user)
|
|
54
|
-
-------------------------------------------------------
|
|
55
|
-
|
|
56
|
-
pywrapper-install -l|--link dsarch -s|--simple
|
|
57
|
-
|
|
58
|
-
This creates bin/dsarch -> bin/setuid_dsarch.
|
|
59
|
-
The program runs as the current user with no privilege change.
|
|
60
|
-
|
|
61
|
-
"""
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
def main():
|
|
65
|
-
print(SETUP_GUIDE)
|
|
66
|
-
sys.exit(0)
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
if __name__ == '__main__': main()
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/requires.txt
RENAMED
|
File without changes
|
{rda_python_dsarch-2.0.20 → rda_python_dsarch-3.0.0}/src/rda_python_dsarch.egg-info/top_level.txt
RENAMED
|
File without changes
|
|
File without changes
|