devpi-admin 1.2.0__tar.gz → 1.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/.github/workflows/publish.yml +1 -1
  2. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/.github/workflows/tests.yml +2 -2
  3. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/.gitignore +2 -0
  4. devpi_admin-1.3.0/PKG-INFO +648 -0
  5. devpi_admin-1.3.0/README.md +624 -0
  6. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/_version.py +3 -3
  7. devpi_admin-1.3.0/devpi_admin/customizer.py +270 -0
  8. devpi_admin-1.3.0/devpi_admin/main.py +1381 -0
  9. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/css/style.css +268 -63
  10. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/index.html +0 -1
  11. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/js/api.js +5 -0
  12. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/js/app.js +1462 -258
  13. devpi_admin-1.3.0/devpi_admin/tokens.py +501 -0
  14. devpi_admin-1.3.0/devpi_admin.egg-info/PKG-INFO +648 -0
  15. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin.egg-info/SOURCES.txt +7 -3
  16. devpi_admin-1.3.0/devpi_admin.egg-info/requires.txt +4 -0
  17. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/pyproject.toml +10 -1
  18. devpi_admin-1.3.0/tests/test_acl_read.py +974 -0
  19. devpi_admin-1.3.0/tests/test_filter.py +390 -0
  20. devpi_admin-1.3.0/tests/test_pipconf.py +78 -0
  21. devpi_admin-1.3.0/tests/test_tokens.py +418 -0
  22. devpi_admin-1.3.0/tests/test_view_helpers.py +79 -0
  23. devpi_admin-1.2.0/INSTALL.textile +0 -190
  24. devpi_admin-1.2.0/PKG-INFO +0 -231
  25. devpi_admin-1.2.0/README.md +0 -209
  26. devpi_admin-1.2.0/devpi_admin/main.py +0 -342
  27. devpi_admin-1.2.0/devpi_admin.egg-info/PKG-INFO +0 -231
  28. devpi_admin-1.2.0/devpi_admin.egg-info/requires.txt +0 -1
  29. devpi_admin-1.2.0/tests/test_cached_versions.py +0 -91
  30. devpi_admin-1.2.0/tests/test_helpers.py +0 -101
  31. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/LICENSE +0 -0
  32. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/__init__.py +0 -0
  33. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/favicon.svg +0 -0
  34. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/js/marked.min.js +0 -0
  35. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin/static/js/theme.js +0 -0
  36. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin.egg-info/dependency_links.txt +0 -0
  37. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin.egg-info/entry_points.txt +0 -0
  38. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/devpi_admin.egg-info/top_level.txt +0 -0
  39. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/setup.cfg +0 -0
  40. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/__init__.py +0 -0
  41. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/test_hooks.py +0 -0
  42. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/test_json_safe.py +0 -0
  43. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/test_package.py +0 -0
  44. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/test_tween.py +0 -0
  45. {devpi_admin-1.2.0 → devpi_admin-1.3.0}/tests/test_wants_html.py +0 -0
@@ -14,7 +14,7 @@ jobs:
14
14
  steps:
15
15
  - uses: actions/checkout@v4
16
16
  with:
17
- fetch-depth: 0 # needed for hatch-vcs to derive version from tag
17
+ fetch-depth: 0 # needed for setuptools-scm to derive version from tag
18
18
 
19
19
  - name: Set up Python
20
20
  uses: actions/setup-python@v5
@@ -11,12 +11,12 @@ jobs:
11
11
  runs-on: ubuntu-latest
12
12
  strategy:
13
13
  matrix:
14
- python-version: ["3.10", "3.14"]
14
+ python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
15
15
 
16
16
  steps:
17
17
  - uses: actions/checkout@v4
18
18
  with:
19
- fetch-depth: 0 # needed for hatch-vcs to read git tags
19
+ fetch-depth: 0 # needed for setuptools-scm to read git tags
20
20
 
21
21
  - name: Set up Python ${{ matrix.python-version }}
22
22
  uses: actions/setup-python@v5
@@ -13,3 +13,5 @@ build
13
13
  devpi_admin/_version.py
14
14
  *.md
15
15
  !README*.md
16
+ *.textile
17
+ dev/
@@ -0,0 +1,648 @@
1
+ Metadata-Version: 2.4
2
+ Name: devpi-admin
3
+ Version: 1.3.0
4
+ Summary: Modern web UI plugin for devpi-server — drop-in replacement for devpi-web
5
+ Author-email: Pavel Revak <pavelrevak@gmail.com>
6
+ License: MIT
7
+ Keywords: devpi,pypi,admin,web,ui
8
+ Classifier: Development Status :: 4 - Beta
9
+ Classifier: Framework :: Pyramid
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Intended Audience :: System Administrators
12
+ Classifier: License :: OSI Approved :: MIT License
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Topic :: Software Development :: Libraries
16
+ Classifier: Topic :: System :: Software Distribution
17
+ Requires-Python: >=3.9
18
+ Description-Content-Type: text/markdown
19
+ License-File: LICENSE
20
+ Requires-Dist: devpi-server<7,>=6.19
21
+ Provides-Extra: dev
22
+ Requires-Dist: pytest; extra == "dev"
23
+ Dynamic: license-file
24
+
25
+ # devpi-admin
26
+
27
+ A modern web UI plugin for [devpi-server](https://devpi.net/) - a drop-in replacement for
28
+ `devpi-web`. Ships as a Python package that registers itself as a devpi-server plugin via the
29
+ standard entry point mechanism, so a single `pip install devpi-admin` is enough.
30
+
31
+ The UI itself is a bundled single-page application (pure HTML + CSS + vanilla JavaScript, no
32
+ build step) served under `/+admin/`. All devpi REST API endpoints remain untouched - the SPA
33
+ talks to the standard devpi JSON API directly.
34
+
35
+ ## Features
36
+
37
+ ### Dashboard
38
+ - Server info with version of devpi-server and all installed plugins (auto-detected)
39
+ - Cache metrics with hit-rate bars (storage, changelog, relpath caches)
40
+ - Whoosh search index queue status
41
+ - **Replica status** (master only, authenticated users only) - per-replica cards with
42
+ authoritative `applied_serial` vs. master serial. Three states:
43
+ - **in sync** - replica matches master serial
44
+ - **lagging** - replica is behind but advancing
45
+ - **stuck** - replica has been polling the same serial for >=30 s; usually means a
46
+ server-side plugin (`devpi-admin`, `devpi-web`, ...) is missing or out of date on the replica
47
+ - **Topbar health indicator** - the `devpi admin` logo is coloured green / orange / red
48
+ on every page, refreshed every 30 s in the background:
49
+ - server reachable, all replicas in sync
50
+ - at least one replica lagging (visible to authenticated master operators)
51
+ - server not responding
52
+
53
+ ### Indexes
54
+ - Visual cards color-coded by type: green (stage), amber (volatile stage), blue (mirror)
55
+ - Warning tags for ACL edge cases:
56
+ - **`world-writable`** - `acl_upload` contains `:ANONYMOUS:`; supply-chain risk
57
+ - **`no upload`** - `acl_upload` is empty; nobody (not even owner / root) can publish
58
+ - Adaptive kebab menu - items hint whether a token will be issued:
59
+ - **`pip.conf`** (public read index, no auth needed) vs. **`pip.conf + token`** (private read)
60
+ - **`.pypirc`** (public upload, no auth needed) vs. **`.pypirc + token`** (private upload).
61
+ Hidden when nobody can upload (`acl_upload` empty).
62
+ - **`pip.conf` modal** - issues a short-lived `read`-scope token bound to the index. Returns
63
+ the full `pip.conf` (Copy / Download), a one-off `pip install --index-url ...` command, and
64
+ the raw `user:token` pair for `curl`, `devpi login`, etc. Anonymous-readable indexes show
65
+ a static pip.conf without credentials.
66
+ - **`.pypirc` modal** - issues an `upload`-scope token bound to the index. Returns the full
67
+ `.pypirc` (Copy / Download), `TWINE_*` environment variable block, a one-shot
68
+ `twine upload --repository-url ... -u ... -p ... dist/*` command, and the raw `user:token`
69
+ pair. Anonymous-upload indexes (rare; world-writable) show a static `.pypirc` without
70
+ credentials and a security warning.
71
+ - Create / edit / delete indexes via modal dialogs
72
+ - `bases` editor with drag & drop priority ordering and transitive inheritance display
73
+ - `acl_upload` and `acl_read` tag pickers with user selection dropdown
74
+ - `volatile`, `mirror_url`, `title` configuration
75
+ - **Mirror package allow/deny lists** (`package_allowlist`, `package_denylist`) — see
76
+ *Mirror access control* below
77
+
78
+ ### Read access control (`acl_read`)
79
+ - Per-index list of principals allowed to read the index (download packages, browse simple)
80
+ - Default `[:ANONYMOUS:]` - public, behaves like devpi-web
81
+ - Set to specific users (`alice`, `bob`) to make the index private
82
+ - Special principals: `:ANONYMOUS:` (everyone, including unauthenticated) and `:AUTHENTICATED:`
83
+ (any logged-in user)
84
+ - Enforced natively by devpi via the `pkg_read` permission on every download path,
85
+ plus a tween that filters out invisible indexes from the root listing (`GET /`)
86
+ and rejects direct access to private indexes with 404
87
+
88
+ ### Mirror access control (allow/deny lists)
89
+ - Per-mirror `package_allowlist` and `package_denylist` filter the projects, versions
90
+ and simple-index links served from upstream. Only `type=mirror` indexes carry
91
+ these fields; stage indexes are unaffected
92
+ - **Empty allowlist** = pass-through (everything allowed except denylist).
93
+ **Non-empty allowlist** = whitelist mode (only listed entries reach pip)
94
+ - **Denylist always wins** — overrides any allowlist match
95
+ - Entry formats (one per line in the modal):
96
+ - PEP 508 requirement — `numpy`, `numpy>=2.0`, `urllib3<1.26.5`
97
+ - Glob in name part — `mycompany-*`, `*-internal`, `mycompany-*<2.0`
98
+ - **Multi-layer enforcement** so a denylist hit cannot be bypassed:
99
+ - `+simple/<project>/` — denied versions never appear in pip's discovery (devpi-server's
100
+ customizer hooks: `get_projects_filter_iter`, `get_versions_filter_iter`,
101
+ `get_simple_links_filter_iter`)
102
+ - `/<user>/<index>` listing — denied projects vanish from the project list
103
+ - `+f/<hash>/<filename>` direct download — tween returns 404 even for previously
104
+ cached files (defense in depth against shared/bookmarked URLs). The cached file
105
+ stays on disk; removing the deny rule restores access without re-fetching upstream
106
+ - Use cases:
107
+ - **CVE blocklist** — `urllib3<1.26.5`, `cryptography<41.0.0`
108
+ - **Internal namespace ban** — `mycompany-*` keeps PyPI typosquats from shadowing
109
+ private packages on a public mirror
110
+ - **Whitelist-only mirrors** — paste curated `requirements.txt` style entries
111
+ into `package_allowlist`; everything else is blocked
112
+
113
+ ### Admin tokens (scoped, revocable)
114
+ - Opaque `adm_<id>.<secret>` tokens bound to a `(user, index, scope)` triple. Scope is
115
+ `read` (pip install) or `upload` (twine / `devpi upload`). A leaked token is contained
116
+ to **one index** and **one operation class** - no cross-index or upgrade path.
117
+ - Tokens are persisted in keyfs as **SHA-256 hashes only** - the plaintext secret is shown
118
+ exactly once at issuance. A keyfs dump (replica disk, backup) does not yield usable
119
+ tokens. Lookup compares hashes via `hmac.compare_digest` (constant-time).
120
+ - TTL configurable per-token (60 s up to 1 year), uniquely revocable
121
+ - **Tween enforcement matrix**:
122
+
123
+ | scope | allowed methods | allowed paths |
124
+ |---|---|---|
125
+ | `read` | GET, HEAD | `/+api`, `/<token.user>/<token.index>/...` |
126
+ | `upload` | GET, HEAD, POST, PUT | `/+api`, `/<token.user>/<token.index>/...` |
127
+
128
+ `DELETE` is **never** granted, even with `upload` scope - package removal must use
129
+ password auth. Anything outside the bound index path returns 403, including the SPA,
130
+ `/+admin-api/*` (so a token cannot mint further tokens), `/+login`, `/`, and `/<user>`.
131
+ - **Issuance rules**: regular users may issue for themselves; root may issue for *other*
132
+ users (admin delegation) but not for itself. Admin-token-authenticated requests cannot
133
+ issue further tokens. Issuance verifies the target user is in `acl_read` /
134
+ `acl_upload` of the target index.
135
+ - **Management rules**: list / revoke is allowed for the token owner or root. Per-index
136
+ token list endpoint shows only the caller's own tokens (root sees all).
137
+ - **Auto-cleanup**:
138
+ - User delete -> all tokens for that user removed from keyfs
139
+ - Index delete -> all tokens bound to that index removed (USER subscriber diffs the
140
+ `indexes` dict)
141
+ - Legacy tokens (pre-hash storage, or pre-`index/scope`) wiped at startup
142
+ - Audit log: failed lookups (unknown id, secret mismatch, expired, deleted user, legacy
143
+ token) are logged at WARNING/INFO so an operator can spot bruteforce attempts.
144
+ - CI/Ansible-friendly: `GET /+admin-api/pip-conf?index=user/index&ttl=3600` returns a
145
+ ready-to-use `pip.conf` (text/plain) in one HTTP call. For upload, use
146
+ `POST /+admin-api/token` with `{"scope": "upload"}`.
147
+
148
+ ### Users
149
+ - Create, edit (email, password), delete users (admin only)
150
+ - **Tokens manager** (kebab -> Tokens) - per-user list with label, **index, scope**,
151
+ expiry, issuer, IP; revoke individual or "Reset all". Wide modal layout so the table
152
+ doesn't overflow on stage indexes with long names.
153
+
154
+ ### Packages
155
+ - Client-side search with PEP 503 name normalization and relevance ranking
156
+ (exact match > prefix match > substring match, then shortest name first) so
157
+ searching `requests` in a 780k-project upstream surfaces `requests` itself, not
158
+ `django-requests-cache` first
159
+ - Stage indexes load packages automatically. Mirror indexes (e.g. `root/pypi` ≈ 780k
160
+ upstream projects, ~17 MB) require an explicit "Browse full index" click — no
161
+ auto-fetch
162
+ - Package cards with latest version and `pip install` command
163
+
164
+ ### Package detail (PyPI-like layout)
165
+ - **Sidebar**: metadata (author, license, Python version, keywords, platform, maintainer,
166
+ extras, project URLs, dependencies), `pip install` command, file downloads with upload dates
167
+ - **Version list**: every known version of the package, newest first, each linking to
168
+ its own detail view
169
+ - **README**: rendered markdown (via `marked.js`); fetched from PyPI.org for mirror packages
170
+ where devpi doesn't cache the description
171
+
172
+ ### General
173
+ - **Anonymous browsing** - visitors can explore public indexes without logging in; admin
174
+ actions (create/edit/delete) appear only after authentication. Private indexes
175
+ (`acl_read` without `:ANONYMOUS:`) are hidden from anonymous root listing.
176
+ - **Hardened SPA delivery** - strict `Content-Security-Policy` (no inline scripts,
177
+ `connect-src` limited to same-origin + `pypi.org`, `frame-ancestors 'none'`),
178
+ `X-Content-Type-Options: nosniff`, `Referrer-Policy: no-referrer`. Markdown READMEs
179
+ are sanitised before rendering (script/iframe/event handlers stripped, dangerous
180
+ URL schemes blocked).
181
+ - **Dark / light / auto theme** with half-circle icon for auto mode
182
+ - **Responsive mobile menu** with hamburger toggle
183
+ - **ESC + outside-click** dismissal for modals, dropdown menus, mobile menu
184
+ - **Login via modal** - no separate login page
185
+
186
+ ## Installation
187
+
188
+ ```bash
189
+ pip install devpi-admin
190
+ ```
191
+
192
+ This pulls in `devpi-server` as a dependency. If you are using devpi in a dedicated venv
193
+ (recommended), install the plugin into the same venv:
194
+
195
+ ```bash
196
+ /var/lib/pypi/venv/bin/pip install devpi-admin
197
+ systemctl --user restart devpi # or however you run devpi-server
198
+ ```
199
+
200
+ You should uninstall `devpi-web` - `devpi-admin` replaces it entirely:
201
+
202
+ ```bash
203
+ pip uninstall devpi-web
204
+ ```
205
+
206
+ Both plugins can technically coexist but it is not recommended. `devpi-admin` intercepts `/`
207
+ for HTML requests while `devpi-web` would still serve its own HTML on other routes like
208
+ `/<user>/<index>/<package>`, leading to a confusing mixed experience.
209
+
210
+ ### Replicas: install on every node
211
+
212
+ `devpi-admin` registers custom keyfs keys (`+admin/tokens/...`,
213
+ `+admin/user-tokens/...`, `+admin/index-tokens/...`). Master writes to these on every
214
+ token issue / revoke. **Replicas without `devpi-admin` installed cannot apply those
215
+ changelog entries** - `import_changes` fails with `AssertionError` on the missing
216
+ keyfs key, the replica rolls back to the prior serial, and replication stalls.
217
+
218
+ The dashboard's stuck-replica detection is designed exactly for this: a `stuck`
219
+ state on a replica card almost always means a plugin (typically `devpi-admin` itself,
220
+ also `devpi-web`, `devpi-postgresql`) is missing or out of date on the replica. Recovery
221
+ is straightforward:
222
+
223
+ ```bash
224
+ # on the replica
225
+ ~/.venv/bin/pip install --upgrade devpi-admin # match master version
226
+ systemctl restart devpi
227
+ ```
228
+
229
+ Replication resumes from the failed serial automatically - no manual keyfs surgery.
230
+
231
+ **Upgrade order:** replicas first, then master. If you upgrade master first and that
232
+ release introduces a new keyfs key, replicas would crash on the very next poll.
233
+
234
+ See `INSTALL.md` section 11 for full step-by-step replica setup and dashboard interpretation.
235
+
236
+ ### Recommended for production: `--restrict-modify root`
237
+
238
+ devpi-server starts in an **open** mode by default - anyone (including unauthenticated
239
+ clients) can `PUT /<newuser>` to create an account, and any logged-in user can
240
+ `PUT /<user>/<index>` to spin up indexes under their own account. The devpi-admin UI
241
+ hides those buttons from non-root users, but a direct API call (`curl`, `devpi user -c`)
242
+ will still succeed.
243
+
244
+ Pass `--restrict-modify root` to `devpi-server` to lock structural operations
245
+ (create/modify/delete of users and indexes) down to `root` only. Per-index
246
+ `acl_upload`/`acl_read` are unaffected, so day-to-day uploads and downloads keep working
247
+ under the existing per-index permissions.
248
+
249
+ ```ini
250
+ ExecStart=/opt/pypi/venv/bin/devpi-server \
251
+ --serverdir /var/lib/pypi/data \
252
+ --restrict-modify root \
253
+ ...
254
+ ```
255
+
256
+ See `INSTALL.md` for a full systemd unit example.
257
+
258
+ ## Usage
259
+
260
+ After restart, open:
261
+
262
+ ```
263
+ http://<your-devpi-host>:3141/
264
+ ```
265
+
266
+ Browser visits to `/` are redirected to `/+admin/`, which serves the SPA. Direct links like
267
+ `http://<host>:3141/+admin/#packages/ci/testing` work and can be bookmarked.
268
+
269
+ devpi CLI tools and other JSON clients are unaffected - they send `Accept: application/json`
270
+ and bypass the redirect.
271
+
272
+ ## CI/Ansible: short-lived pip.conf via the API
273
+
274
+ For automation that needs to install from a private index, store the service user's password
275
+ as a secret and let the pipeline mint a fresh short-lived `pip.conf` per run:
276
+
277
+ ```yaml
278
+ # Gitea Actions example
279
+ - name: Install dependencies
280
+ env:
281
+ DEVPI_USER: ${{ secrets.DEVPI_USER }} # e.g. "gitea-ci"
282
+ DEVPI_PASSWORD: ${{ secrets.DEVPI_PASSWORD }}
283
+ run: |
284
+ mkdir -p ~/.pip
285
+ AUTH=$(printf '%s:%s' "$DEVPI_USER" "$DEVPI_PASSWORD" | base64)
286
+ curl -sf -H "X-Devpi-Auth: $AUTH" \
287
+ "https://devpi.example.com/+admin-api/pip-conf?index=company/private&ttl=3600&wait_replicas=10" \
288
+ > ~/.pip/pip.conf
289
+ pip install -r requirements.txt
290
+ ```
291
+
292
+ ### Replication race: `wait_replicas`
293
+
294
+ When devpi runs as primary + replicas behind a load balancer, a freshly issued token
295
+ exists on the primary instantly but takes one polling cycle (~37 s by default) to reach
296
+ replicas. An Ansible-style playbook that issues a token and immediately uses it through
297
+ the LB may hit a replica that doesn't know the token yet - and get `401`.
298
+
299
+ Both `POST /+admin-api/token` and `GET /+admin-api/pip-conf` accept a `wait_replicas`
300
+ parameter. The primary blocks until every currently-polling replica has caught up to
301
+ the commit serial, bounded by 30 s. Stale replicas (silent for >2 min) are skipped so an
302
+ offline replica never blocks the caller.
303
+
304
+ ```bash
305
+ # Wait up to 10 s for replicas; default cap is 30 s if you pass `true`/`1`.
306
+ curl -sf -H "X-Devpi-Auth: $AUTH" \
307
+ "https://devpi.example.com/+admin-api/pip-conf?index=company/private&ttl=3600&wait_replicas=10" \
308
+ > ~/.pip/pip.conf
309
+ ```
310
+
311
+ For `POST /+admin-api/token`, send `{"wait_replicas": 10}` in the JSON body. The response
312
+ includes a `replication` block (`synced`, `waited`, `timed_out`, `replicas`, ...) so the
313
+ client can decide whether to retry.
314
+
315
+ The token issued is `read`-scoped - usable only for `GET`/`HEAD` on `/+api` and the
316
+ bound `/<user>/<index>/...`. It cannot upload, modify indexes, change passwords,
317
+ exchange itself for a session token, or issue another token. It expires after `ttl`
318
+ seconds. The service user must have `pkg_read` on the target index. Root may issue
319
+ for *other* users (admin delegation) but never for itself.
320
+
321
+ For uploads, `POST /+admin-api/token` with `{"scope": "upload"}` returns a token that
322
+ adds POST/PUT to the bound index - usable from `twine` or `devpi upload`:
323
+
324
+ ```yaml
325
+ # CI publish step
326
+ - name: Publish wheel
327
+ run: |
328
+ AUTH=$(printf '%s:%s' "$DEVPI_USER" "$DEVPI_PASSWORD" | base64)
329
+ TOKEN=$(curl -sf -H "X-Devpi-Auth: $AUTH" -H 'Content-Type: application/json' \
330
+ -d '{"index":"company/release","scope":"upload","ttl_seconds":900}' \
331
+ "https://devpi.example.com/+admin-api/token" | jq -r .token)
332
+ twine upload --repository-url https://devpi.example.com/company/release/ \
333
+ -u "$DEVPI_USER" -p "$TOKEN" dist/*
334
+ ```
335
+
336
+ Upload tokens still cannot DELETE - package removal must use password auth.
337
+
338
+ ### Trusted proxy for client IP logging
339
+
340
+ The `client_ip` field on issued tokens (visible in the token list) is taken from
341
+ `request.client_addr` by default. When devpi-server runs behind a reverse proxy, set
342
+ `DEVPI_ADMIN_TRUSTED_PROXIES` to a comma-separated list of CIDRs whose `X-Forwarded-For`
343
+ header should be honoured:
344
+
345
+ ```
346
+ DEVPI_ADMIN_TRUSTED_PROXIES=10.0.0.0/8,127.0.0.1
347
+ ```
348
+
349
+ Without this variable, `X-Forwarded-For` is ignored - preventing clients from forging
350
+ their logged IP.
351
+
352
+ ## How it works
353
+
354
+ `devpi-admin` registers a `devpi_server` entry point with several `@hookimpl`s:
355
+
356
+ - **`devpiserver_get_features`** - advertises the plugin in `/+api`.
357
+ - **`devpiserver_indexconfig_defaults`** - registers `acl_read` as an indexconfig field
358
+ with an `ACLList` marker so devpi normalizes its values on every `PUT`/`PATCH`.
359
+ - **`devpiserver_stage_get_principals_for_pkg_read`** - feeds `acl_read` into devpi's
360
+ pyramid ACL, which applies the `pkg_read` permission natively on every download path
361
+ (`+f/`, `+e/`, simple page).
362
+ - **`devpiserver_get_identity`** - recognizes `adm_<id>.<secret>` admin tokens, validates
363
+ them against keyfs (constant-time hash compare), sets `adm.is_admin_token` in the
364
+ request environ for downstream tween checks.
365
+ - **`devpiserver_pyramid_configure`** - registers the SPA, custom API views, the tween,
366
+ the token keyfs keys, and a USER-key subscriber that cleans up tokens on user delete
367
+ AND on per-user-index removal (diffs old vs. new `indexes` dict via `tx.get_value_at`).
368
+ Primary only - replicas are read-only.
369
+
370
+ The tween does several things on every request:
371
+
372
+ 1. **Captures replica poll info.** Matches `GET /+changelog/{N}-?` and records
373
+ `start_serial` + `last_seen` keyed by the `X-DEVPI-REPLICA-UUID` header. This is the
374
+ data source for `/+admin-api/replicas` and the dashboard's stuck-replica detection.
375
+ 2. **Validates admin tokens** by direct `tokens.lookup()` (not via pyramid identity, which
376
+ would pin a stale identity through `/+login`'s mid-request header swap). On valid
377
+ token, sets `adm.token_meta` in the request environ for the identity hook to reuse.
378
+ 3. **Enforces token scope and index binding**:
379
+ - `read` scope -> only GET/HEAD allowed
380
+ - `upload` scope -> adds POST/PUT (DELETE is *never* granted)
381
+ - URL must be `/+api` or under `/<token.user>/<token.index>/...`. Anything else
382
+ (other indexes, SPA, `/+admin-api/*`, `/+login`, root listing, `/<user>`) returns 403.
383
+ 4. **Redirects** HTML browser requests on `/` to `/+admin/` while leaving JSON requests intact.
384
+ 5. **Returns 404** for `GET /<user>/<index>/...` (index, simple, project, version, file)
385
+ when the requestor lacks `pkg_read` - devpi's own listing endpoints have no
386
+ permission check, so we add one.
387
+ 6. **Returns 403/404** for `GET /<user>` when the requestor is neither the user
388
+ themselves nor `root` - devpi otherwise leaks the full list of that user's
389
+ private indexes.
390
+ 7. **Filters the `GET /` JSON response** to remove indexes the requestor can't read,
391
+ and adds `Cache-Control: private, no-store` so a shared cache cannot serve one
392
+ user's filtered view to another.
393
+
394
+ The SPA HTML (`/+admin/`) is served with security headers - strict
395
+ `Content-Security-Policy` (no inline scripts, restricted `connect-src` to
396
+ `'self'` + `https://pypi.org` for the README fallback, `frame-ancestors 'none'`),
397
+ plus `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer`.
398
+
399
+ The plugin uses devpi-server internals: `xom.model.getstage`, `stage.list_versions`,
400
+ `stage.get_versiondata`, `stage.get_releaselinks`, `xom.keyfs`.
401
+
402
+ The mirror access control (`package_allowlist` / `package_denylist`) is implemented
403
+ on top of devpi-server's stage customizer hooks (`get_projects_filter_iter`,
404
+ `get_versions_filter_iter`, `get_simple_links_filter_iter`). devpi-server rejects
405
+ duplicate customizer registrations for a given `index_type`, so instead of providing
406
+ our own class we monkey-patch our methods onto the upstream `MirrorCustomizer`
407
+ (an empty pass-through class designed exactly for this kind of extension). The
408
+ patch runs once at module import. The tween additionally enforces denylist on
409
+ direct `+f/` downloads to neutralise previously-cached or shared file URLs.
410
+
411
+ ## Requirements
412
+
413
+ - Python 3.9+
414
+ - **devpi-server 6.19 <= version < 7.0** - we rely on `tx.get_value_at`, the
415
+ `X-DEVPI-REPLICA-UUID` header and the `polling_replicas` dict shape introduced in
416
+ 6.19; the upper bound is held until 7.x compatibility is verified.
417
+ - A browser with ES6 support (`Promise`, `fetch`, `sessionStorage`)
418
+
419
+ ## Routes (UI)
420
+
421
+ Routing is hash-based, so any of these URLs can be bookmarked or shared:
422
+
423
+ | Hash | View |
424
+ |------|------|
425
+ | `#` | Status dashboard (default) |
426
+ | `#indexes` | All indexes |
427
+ | `#indexes/<user>` | Indexes filtered by user |
428
+ | `#packages/<user>/<index>` | Packages in an index |
429
+ | `#package/<user>/<index>/<name>` | Package detail (latest version) |
430
+ | `#package/<user>/<index>/<name>?version=<ver>` | Specific version |
431
+ | `#users` | User management (requires login) |
432
+
433
+ ## API
434
+
435
+ In addition to serving the SPA, `devpi-admin` exposes its own JSON API under
436
+ `/+admin-api/`. Authentication uses the standard devpi-server header
437
+ `X-Devpi-Auth: base64(user:token)`. Responses are `application/json` unless noted
438
+ (`/+admin-api/pip-conf` returns `text/plain`).
439
+
440
+ ### Session and discovery
441
+
442
+ #### `GET /+admin-api/session`
443
+ Cheap auth check; the frontend pings this on tab focus to detect expired sessions.
444
+ - **Auth:** required
445
+ - **200:** `{"valid": true, "user": "alice"}`
446
+ - **403:** not authenticated
447
+
448
+ #### `GET /+admin-api/public-url`
449
+ Canonical "outside" URL of this deployment, derived from
450
+ `request.application_url` (respects `--outside-url` and `X-Forwarded-*` headers).
451
+ The SPA uses this for static `pip.conf` / `.pypirc` previews so they match what the
452
+ backend would emit when behind a reverse proxy.
453
+ - **Auth:** none (URL is not a secret; even anonymous viewers of public indexes need it)
454
+ - **200:** `{"url": "https://devpi.example.com"}`
455
+
456
+ ### Project metadata
457
+
458
+ #### `GET /+admin-api/versions/{user}/{index}/{project}`
459
+ All known versions of a project, newest first. Backed by `stage.list_versions()` so
460
+ the result is consistent across primary and replicas (PROJSIMPLELINKS in keyfs is
461
+ replicated via the changelog).
462
+ - **Auth:** `pkg_read` on the index
463
+ - **200:** `{"versions": ["1.0", "0.9", "0.8"]}`
464
+
465
+ #### `GET /+admin-api/versiondata/{user}/{index}/{project}/{version}`
466
+ Metadata + file links for a single version (PEP 426 / PEP 621 fields plus `+links`
467
+ with `href`, `basename`, `hash_spec`, upload `log`).
468
+ - **Auth:** `pkg_read` on the index
469
+ - **200:** `{"result": {...}}`
470
+ - **404:** version doesn't exist
471
+
472
+ ### Tokens
473
+
474
+ Tokens are opaque `adm_<id>.<secret>` strings bound to a `(user, index, scope)` triple.
475
+ Only the SHA-256 of the secret is persisted in keyfs.
476
+
477
+ #### `POST /+admin-api/token`
478
+ Issue a new token.
479
+ - **Auth:** required (regular user for self; root may issue for *other* users; admin-token
480
+ requests cannot issue further tokens)
481
+ - **Body (JSON):**
482
+ ```json
483
+ {
484
+ "user": "alice", // optional, default = authenticated; root may set freely (not "root")
485
+ "index": "alice/dev", // required
486
+ "scope": "read" | "upload", // required
487
+ "ttl_seconds": 3600, // optional; 60 <= ttl <= 1 year, default 1h
488
+ "label": "ci-build", // optional, <= 200 chars
489
+ "wait_replicas": 10 // optional; block up to N seconds for replicas to catch up
490
+ }
491
+ ```
492
+ - **200:** `{token, user, index, scope, issued_at, expires_at, label, replication?}` -
493
+ `token` is the plaintext, returned **once**.
494
+ - **403:** target user lacks scope perm on index, root issuing for itself, admin-token call, etc.
495
+ - **404:** index doesn't exist
496
+
497
+ #### `GET /+admin-api/pip-conf?index=u/i&user=&ttl=&label=&wait_replicas=`
498
+ Issue a `read` token + return a ready-to-use pip.conf in one call (CI/Ansible-friendly).
499
+ - **Auth:** required (same rules as `POST /token`)
500
+ - **200:** `text/plain`
501
+ ```ini
502
+ [global]
503
+ index-url = https://alice:adm_xxx.yyy@devpi.example.com/alice/dev/+simple/
504
+ trusted-host = devpi.example.com
505
+ ```
506
+
507
+ #### `GET /+admin-api/users/{user}/tokens`
508
+ List active tokens for a user.
509
+ - **Auth:** the user themselves, or root
510
+ - **200:** `{"result": [{id, id_short, user, index, scope, issuer, issued_at, expires_at, expires_in, label, client_ip}, ...], "count": N}`
511
+
512
+ #### `DELETE /+admin-api/users/{user}/tokens`
513
+ Revoke ALL tokens for a user.
514
+ - **Auth:** the user themselves, or root
515
+ - **200:** `{"revoked": N, "user": "alice"}`
516
+
517
+ #### `GET /+admin-api/indexes/{user}/{index}/tokens`
518
+ List tokens bound to an index. Non-root callers see only tokens they own; root sees
519
+ every token for the index. Returns 404 (not 403) when the caller has no `pkg_read` so
520
+ private index existence is not leaked.
521
+ - **Auth:** `pkg_read` on the index (404 otherwise)
522
+ - **200:** `{"result": [...], "count": N}` - same record shape as `/users/{user}/tokens`
523
+
524
+ #### `DELETE /+admin-api/tokens/{token_id}`
525
+ Revoke a single token.
526
+ - **Auth:** owner of the token, or root
527
+ - **200:** `{"revoked": true, "id": "abc..."}`
528
+ - **404:** token id not found
529
+
530
+ ### Replication observability (master only)
531
+
532
+ #### `GET /+admin-api/replicas`
533
+ Last-known poll info per replica, captured from each `GET /+changelog/{N}-` request via
534
+ a tween. The `applied_serial` field is the highest serial the replica has actually
535
+ applied (`start_serial - 1` from its most recent poll). Compare against `/+status`
536
+ `serial` for true lag.
537
+
538
+ Why this isn't `polling_replicas` from `/+status`: devpi-server overwrites
539
+ `xom.polling_replicas[uuid].serial` during streaming and gives a misleading "caught up"
540
+ reading once the response generator drains. Capturing `start_serial` at the request
541
+ boundary is the only stable signal master alone can produce.
542
+
543
+ - **Auth:** required
544
+ - **200:**
545
+ ```json
546
+ {
547
+ "result": {
548
+ "<replica-uuid>": {
549
+ "start_serial": 103,
550
+ "applied_serial": 102,
551
+ "last_seen": 1712345678.9,
552
+ "age_seconds": 3,
553
+ "stuck_seconds": 47,
554
+ "remote_ip": "10.0.0.5",
555
+ "outside_url": "https://replica.example.com"
556
+ }
557
+ }
558
+ }
559
+ ```
560
+ - Entries auto-expire after 10 min of silence. Dict size capped at 256 entries
561
+ (least-recently-seen evicted first) so an attacker spamming UUIDs cannot exhaust master memory.
562
+
563
+ ## Project layout
564
+
565
+ ```
566
+ devpi-admin/
567
+ ├── pyproject.toml
568
+ ├── README.md
569
+ ├── LICENSE
570
+ ├── .github/workflows/
571
+ │ ├── tests.yml - CI on push/PR (Python 3.10 - 3.14)
572
+ │ └── publish.yml - publish to PyPI on release
573
+ ├── dev/ - untracked dev-only prototypes (e.g. demo-graph.html)
574
+ ├── devpi_admin/
575
+ │ ├── __init__.py - version (from git tag via setuptools-scm)
576
+ │ ├── main.py - Pyramid hooks, tween, API views
577
+ │ ├── tokens.py - admin token gen / lookup / revoke / list (keyfs storage)
578
+ │ ├── customizer.py - mirror package allow/deny filter (patches MirrorCustomizer)
579
+ │ └── static/
580
+ │ ├── index.html - SPA entry point
581
+ │ ├── css/style.css
582
+ │ └── js/
583
+ │ ├── api.js - devpi REST wrapper + auth
584
+ │ ├── theme.js - theme toggle (light/dark/auto)
585
+ │ ├── marked.min.js - vendored markdown renderer
586
+ │ └── app.js - routing, views, rendering
587
+ └── tests/
588
+ ├── test_acl_read.py - acl_read hooks, tween guards (scope/index), token issuance
589
+ │ rules, _check_index_perm, USER-changed handler, replica poll
590
+ │ tween + endpoint, public-url
591
+ ├── test_filter.py - package allow/deny customizer + tween +f/ block
592
+ ├── test_hooks.py - pluggy hook registration
593
+ ├── test_json_safe.py - readonly view conversion
594
+ ├── test_package.py - entry point, static files
595
+ ├── test_pipconf.py - pip.conf credential helpers
596
+ ├── test_tokens.py - token format, issue/lookup/revoke, reset_for_index,
597
+ │ list_for_index, end-to-end cleanup chain
598
+ ├── test_tween.py - redirect behavior
599
+ ├── test_view_helpers.py - _get_stage_or_404, _check_read_access, CSP headers
600
+ └── test_wants_html.py - Accept header heuristic
601
+ ```
602
+
603
+ ## Development
604
+
605
+ ```bash
606
+ git clone <repo>
607
+ cd devpi-admin
608
+ python -m venv .venv
609
+ .venv/bin/pip install -e ".[dev]"
610
+ ```
611
+
612
+ The `dev` extra pulls in `pytest`. A bare `pip install -e .` works too - the test suite
613
+ is also runnable with the stdlib `unittest` runner.
614
+
615
+ The static files live at `devpi_admin/static/` and can be edited in place - changes show
616
+ up on the next browser reload, no restart of devpi-server required (static views read
617
+ from disk on each request). Python changes (`main.py`, `tokens.py`) require a
618
+ devpi-server restart.
619
+
620
+ Run the unit tests:
621
+
622
+ ```bash
623
+ # pytest (recommended for local development)
624
+ pytest tests/ -q
625
+
626
+ # unittest (matches the CI invocation)
627
+ PYTHONWARNINGS="ignore::UserWarning" python -m unittest discover -v tests/
628
+ ```
629
+
630
+ (The `PYTHONWARNINGS` shim hides an unrelated deprecation warning emitted by Pyramid 2.1
631
+ when it imports `pkg_resources`.)
632
+
633
+ ## Releasing
634
+
635
+ Version is derived from the git tag via `setuptools-scm`. To release:
636
+
637
+ 1. `git tag v0.1.0 && git push --tags`
638
+ 2. On GitHub: Releases -> Draft new release -> select tag -> Publish
639
+ 3. The `publish.yml` workflow runs tests, builds wheel+sdist, and uploads to PyPI via trusted
640
+ publishing (no API tokens needed - configure the GitHub environment `pypi` in PyPI settings).
641
+
642
+ ## Author
643
+
644
+ Pavel Revak <pavelrevak@gmail.com>
645
+
646
+ ## License
647
+
648
+ MIT - see [LICENSE](LICENSE).