pydepinject 0.0.4.dev0__tar.gz → 0.0.5.dev0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pydepinject-0.0.4.dev0/src/pydepinject.egg-info → pydepinject-0.0.5.dev0}/PKG-INFO +79 -1
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/README.md +78 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject/__init__.py +27 -16
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject/backends.py +50 -10
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0/src/pydepinject.egg-info}/PKG-INFO +79 -1
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject.egg-info/SOURCES.txt +0 -5
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/tests/test_pydepinject.py +117 -0
- pydepinject-0.0.4.dev0/src/requirementmanager.egg-info/PKG-INFO +0 -32
- pydepinject-0.0.4.dev0/src/requirementmanager.egg-info/SOURCES.txt +0 -7
- pydepinject-0.0.4.dev0/src/requirementmanager.egg-info/dependency_links.txt +0 -1
- pydepinject-0.0.4.dev0/src/requirementmanager.egg-info/requires.txt +0 -11
- pydepinject-0.0.4.dev0/src/requirementmanager.egg-info/top_level.txt +0 -1
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/LICENSE +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/MANIFEST.in +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/pyproject.toml +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/setup.cfg +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject.egg-info/dependency_links.txt +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject.egg-info/requires.txt +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject.egg-info/top_level.txt +0 -0
- {pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/tests/conftest.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pydepinject
|
|
3
|
-
Version: 0.0.
|
|
3
|
+
Version: 0.0.5.dev0
|
|
4
4
|
Summary: A package to dynamically inject requirements into a virtual environment.
|
|
5
5
|
Author: pydepinject
|
|
6
6
|
License-Expression: MIT
|
|
@@ -266,6 +266,84 @@ This helps with debugging and reproducibility.
|
|
|
266
266
|
|
|
267
267
|
These variables provide a convenient way to standardize behavior in CI/CD pipelines or development environments.
|
|
268
268
|
|
|
269
|
+
## Where venvs are stored and cleaning them up
|
|
270
|
+
|
|
271
|
+
Each managed environment is a plain virtual environment directory under a single
|
|
272
|
+
root. By default that root is `<system-temp-dir>/pydepinject/venvs` (overridable
|
|
273
|
+
with `PYDEPINJECT_VENV_ROOT`), and each venv is a subdirectory named either after
|
|
274
|
+
your `venv_name` or a content hash derived from the requirement set, Python
|
|
275
|
+
version, and backend.
|
|
276
|
+
|
|
277
|
+
Because the default root lives under the OS temporary directory, the operating
|
|
278
|
+
system usually reclaims it for you — temp-file cleaners and reboots clear stale
|
|
279
|
+
entries — so in the common case there is **nothing to clean up manually**.
|
|
280
|
+
|
|
281
|
+
For finer control:
|
|
282
|
+
|
|
283
|
+
- **Avoid persistence per call**: pass `ephemeral=True` so the venv is deleted as
|
|
284
|
+
soon as the decorated function or `with` block finishes.
|
|
285
|
+
- **Prune a persistent root you manage**: the venvs are just directories, so
|
|
286
|
+
`rm -rf "$PYDEPINJECT_VENV_ROOT"` removes everything, or delete individual
|
|
287
|
+
subdirectories to reclaim space selectively.
|
|
288
|
+
- **Script age-based cleanup**: each venv contains `.pydepinject-*.json` metadata
|
|
289
|
+
files recording the install timestamp (and the packages installed), which you
|
|
290
|
+
can use to find and remove environments older than a chosen age.
|
|
291
|
+
|
|
292
|
+
## Limitations and thread-safety
|
|
293
|
+
|
|
294
|
+
`pydepinject` works by mutating **process-global interpreter state** for the
|
|
295
|
+
duration of the managed scope (the decorated call or `with` block). When a
|
|
296
|
+
requirement is not already importable, it:
|
|
297
|
+
|
|
298
|
+
- inserts the venv's `site-packages` at the front of `sys.path`,
|
|
299
|
+
- prepends to the `PYTHONPATH` and `PATH` environment variables,
|
|
300
|
+
- evicts conflicting entries from `sys.modules` so fresh imports resolve to the venv.
|
|
301
|
+
|
|
302
|
+
The original `sys.path`, environment variables, and `sys.modules` state are
|
|
303
|
+
restored when the scope exits. If every requested package is already satisfied in
|
|
304
|
+
the current environment, **no global state is changed at all** — activation is a
|
|
305
|
+
no-op fast path.
|
|
306
|
+
|
|
307
|
+
Because this state is global to the interpreter, please keep the following in mind:
|
|
308
|
+
|
|
309
|
+
- **Not thread-safe and not safe under `asyncio`/concurrent use.** Two managed
|
|
310
|
+
scopes active at the same time in different threads (or interleaved coroutines)
|
|
311
|
+
will race on `sys.path`, the environment variables, and `sys.modules`, and can
|
|
312
|
+
corrupt each other's save/restore. Use `pydepinject` from a single thread of
|
|
313
|
+
control. If you need dependencies isolated across concurrent workers, give each
|
|
314
|
+
worker its own **process**.
|
|
315
|
+
- **Already-imported modules are handled best-effort.** If a package was already
|
|
316
|
+
imported earlier in the process from a different location, swapping it for the
|
|
317
|
+
venv's copy within the scope cannot be guaranteed — code holding a reference to
|
|
318
|
+
the previously imported module keeps using it.
|
|
319
|
+
- **It mutates the running interpreter, by design.** This makes it ideal for
|
|
320
|
+
scripts, notebooks, plugins, and ad-hoc tooling, but it is **not** a substitute
|
|
321
|
+
for a properly provisioned environment or a lockfile in a production
|
|
322
|
+
application. For those, prefer real environment/dependency management.
|
|
323
|
+
|
|
324
|
+
## When NOT to use this
|
|
325
|
+
|
|
326
|
+
`pydepinject` is built for **scripts, notebooks, plugins, and ad-hoc tooling or
|
|
327
|
+
CI glue** — cases where "make sure X is importable, install it if not" is the
|
|
328
|
+
whole job. It is the wrong tool, and you should reach for a real
|
|
329
|
+
environment plus a lockfile (`uv`, `pip-tools`, Poetry, etc.) instead, when:
|
|
330
|
+
|
|
331
|
+
- You are building a **deployable application or service** where dependency
|
|
332
|
+
reproducibility matters.
|
|
333
|
+
- Your code runs under **threads, `asyncio`, or parallel workers** — activation
|
|
334
|
+
mutates global interpreter state and is not safe to interleave (see
|
|
335
|
+
[Limitations and thread-safety](#limitations-and-thread-safety); give each
|
|
336
|
+
worker its own process).
|
|
337
|
+
- You need **pinned, audited, reproducible** dependency sets across machines and
|
|
338
|
+
CI runs.
|
|
339
|
+
- Packages are **already imported** earlier in the process and must be swapped
|
|
340
|
+
mid-run — `pydepinject` can only do this best-effort.
|
|
341
|
+
- You run a **long-lived production process** where persistent venvs accumulating
|
|
342
|
+
under the temp root would be a problem.
|
|
343
|
+
|
|
344
|
+
For the mechanics behind these caveats, see
|
|
345
|
+
[Limitations and thread-safety](#limitations-and-thread-safety) above.
|
|
346
|
+
|
|
269
347
|
## Unit Tests
|
|
270
348
|
|
|
271
349
|
Unit tests are provided to verify the functionality of the `requires`. The tests use `pytest` and cover various scenarios including decorator usage, context manager usage, ephemeral environments, and more.
|
|
@@ -233,6 +233,84 @@ This helps with debugging and reproducibility.
|
|
|
233
233
|
|
|
234
234
|
These variables provide a convenient way to standardize behavior in CI/CD pipelines or development environments.
|
|
235
235
|
|
|
236
|
+
## Where venvs are stored and cleaning them up
|
|
237
|
+
|
|
238
|
+
Each managed environment is a plain virtual environment directory under a single
|
|
239
|
+
root. By default that root is `<system-temp-dir>/pydepinject/venvs` (overridable
|
|
240
|
+
with `PYDEPINJECT_VENV_ROOT`), and each venv is a subdirectory named either after
|
|
241
|
+
your `venv_name` or a content hash derived from the requirement set, Python
|
|
242
|
+
version, and backend.
|
|
243
|
+
|
|
244
|
+
Because the default root lives under the OS temporary directory, the operating
|
|
245
|
+
system usually reclaims it for you — temp-file cleaners and reboots clear stale
|
|
246
|
+
entries — so in the common case there is **nothing to clean up manually**.
|
|
247
|
+
|
|
248
|
+
For finer control:
|
|
249
|
+
|
|
250
|
+
- **Avoid persistence per call**: pass `ephemeral=True` so the venv is deleted as
|
|
251
|
+
soon as the decorated function or `with` block finishes.
|
|
252
|
+
- **Prune a persistent root you manage**: the venvs are just directories, so
|
|
253
|
+
`rm -rf "$PYDEPINJECT_VENV_ROOT"` removes everything, or delete individual
|
|
254
|
+
subdirectories to reclaim space selectively.
|
|
255
|
+
- **Script age-based cleanup**: each venv contains `.pydepinject-*.json` metadata
|
|
256
|
+
files recording the install timestamp (and the packages installed), which you
|
|
257
|
+
can use to find and remove environments older than a chosen age.
|
|
258
|
+
|
|
259
|
+
## Limitations and thread-safety
|
|
260
|
+
|
|
261
|
+
`pydepinject` works by mutating **process-global interpreter state** for the
|
|
262
|
+
duration of the managed scope (the decorated call or `with` block). When a
|
|
263
|
+
requirement is not already importable, it:
|
|
264
|
+
|
|
265
|
+
- inserts the venv's `site-packages` at the front of `sys.path`,
|
|
266
|
+
- prepends to the `PYTHONPATH` and `PATH` environment variables,
|
|
267
|
+
- evicts conflicting entries from `sys.modules` so fresh imports resolve to the venv.
|
|
268
|
+
|
|
269
|
+
The original `sys.path`, environment variables, and `sys.modules` state are
|
|
270
|
+
restored when the scope exits. If every requested package is already satisfied in
|
|
271
|
+
the current environment, **no global state is changed at all** — activation is a
|
|
272
|
+
no-op fast path.
|
|
273
|
+
|
|
274
|
+
Because this state is global to the interpreter, please keep the following in mind:
|
|
275
|
+
|
|
276
|
+
- **Not thread-safe and not safe under `asyncio`/concurrent use.** Two managed
|
|
277
|
+
scopes active at the same time in different threads (or interleaved coroutines)
|
|
278
|
+
will race on `sys.path`, the environment variables, and `sys.modules`, and can
|
|
279
|
+
corrupt each other's save/restore. Use `pydepinject` from a single thread of
|
|
280
|
+
control. If you need dependencies isolated across concurrent workers, give each
|
|
281
|
+
worker its own **process**.
|
|
282
|
+
- **Already-imported modules are handled best-effort.** If a package was already
|
|
283
|
+
imported earlier in the process from a different location, swapping it for the
|
|
284
|
+
venv's copy within the scope cannot be guaranteed — code holding a reference to
|
|
285
|
+
the previously imported module keeps using it.
|
|
286
|
+
- **It mutates the running interpreter, by design.** This makes it ideal for
|
|
287
|
+
scripts, notebooks, plugins, and ad-hoc tooling, but it is **not** a substitute
|
|
288
|
+
for a properly provisioned environment or a lockfile in a production
|
|
289
|
+
application. For those, prefer real environment/dependency management.
|
|
290
|
+
|
|
291
|
+
## When NOT to use this
|
|
292
|
+
|
|
293
|
+
`pydepinject` is built for **scripts, notebooks, plugins, and ad-hoc tooling or
|
|
294
|
+
CI glue** — cases where "make sure X is importable, install it if not" is the
|
|
295
|
+
whole job. It is the wrong tool, and you should reach for a real
|
|
296
|
+
environment plus a lockfile (`uv`, `pip-tools`, Poetry, etc.) instead, when:
|
|
297
|
+
|
|
298
|
+
- You are building a **deployable application or service** where dependency
|
|
299
|
+
reproducibility matters.
|
|
300
|
+
- Your code runs under **threads, `asyncio`, or parallel workers** — activation
|
|
301
|
+
mutates global interpreter state and is not safe to interleave (see
|
|
302
|
+
[Limitations and thread-safety](#limitations-and-thread-safety); give each
|
|
303
|
+
worker its own process).
|
|
304
|
+
- You need **pinned, audited, reproducible** dependency sets across machines and
|
|
305
|
+
CI runs.
|
|
306
|
+
- Packages are **already imported** earlier in the process and must be swapped
|
|
307
|
+
mid-run — `pydepinject` can only do this best-effort.
|
|
308
|
+
- You run a **long-lived production process** where persistent venvs accumulating
|
|
309
|
+
under the temp root would be a problem.
|
|
310
|
+
|
|
311
|
+
For the mechanics behind these caveats, see
|
|
312
|
+
[Limitations and thread-safety](#limitations-and-thread-safety) above.
|
|
313
|
+
|
|
236
314
|
## Unit Tests
|
|
237
315
|
|
|
238
316
|
Unit tests are provided to verify the functionality of the `requires`. The tests use `pytest` and cover various scenarios including decorator usage, context manager usage, ephemeral environments, and more.
|
|
@@ -23,7 +23,7 @@ from .backends import VenvBackendRegistry
|
|
|
23
23
|
P = typing.ParamSpec("P")
|
|
24
24
|
R = typing.TypeVar("R")
|
|
25
25
|
|
|
26
|
-
VERSION = "0.0.
|
|
26
|
+
VERSION = "0.0.5dev0"
|
|
27
27
|
__version__ = VERSION
|
|
28
28
|
|
|
29
29
|
logger = logging.getLogger(__name__)
|
|
@@ -278,8 +278,29 @@ class RequirementManager:
|
|
|
278
278
|
self.original_path = os.environ.get("PATH", "")
|
|
279
279
|
self.original_syspath = sys.path.copy()
|
|
280
280
|
|
|
281
|
+
# Once we start mutating global interpreter state, any failure (of any
|
|
282
|
+
# exception type) must restore it before propagating; otherwise the host
|
|
283
|
+
# process is left with a polluted sys.path/PATH/PYTHONPATH.
|
|
284
|
+
try:
|
|
285
|
+
return self._mutate_and_install()
|
|
286
|
+
except BaseException:
|
|
287
|
+
logger.exception("Failed to activate venv %s", self.venv_path)
|
|
288
|
+
self._deactivate_venv()
|
|
289
|
+
raise
|
|
290
|
+
|
|
291
|
+
def _mutate_and_install(self):
|
|
292
|
+
"""Mutate interpreter state to point at the venv and install packages.
|
|
293
|
+
|
|
294
|
+
Assumes ``original_*`` state has already been captured by the caller so
|
|
295
|
+
that a failure here can be rolled back.
|
|
296
|
+
"""
|
|
281
297
|
self._create_virtualenv()
|
|
282
298
|
|
|
299
|
+
# Mark activated before mutating any global state, so an interrupt in the
|
|
300
|
+
# middle of the mutations below still triggers a full restore via
|
|
301
|
+
# _deactivate_venv (which is gated on this flag).
|
|
302
|
+
self._activated = True
|
|
303
|
+
|
|
283
304
|
bin_dir = _bin_dir(self.venv_path)
|
|
284
305
|
venv_site_packages = _site_packages_dir(self.venv_path)
|
|
285
306
|
os.environ["PYTHONPATH"] = str(venv_site_packages) + (
|
|
@@ -287,7 +308,6 @@ class RequirementManager:
|
|
|
287
308
|
)
|
|
288
309
|
os.environ["PATH"] = str(bin_dir) + os.pathsep + self.original_path
|
|
289
310
|
sys.path.insert(0, str(venv_site_packages))
|
|
290
|
-
self._activated = True
|
|
291
311
|
if is_requirements_satisfied(*self.packages):
|
|
292
312
|
logger.debug(
|
|
293
313
|
"Requirements %s already satisfied within %s",
|
|
@@ -301,6 +321,7 @@ class RequirementManager:
|
|
|
301
321
|
logger.debug(
|
|
302
322
|
"Purged modules from venv %s: %s", self.venv_path, purged_modules
|
|
303
323
|
)
|
|
324
|
+
return self
|
|
304
325
|
|
|
305
326
|
def _deactivate_venv(self):
|
|
306
327
|
if not self._activated:
|
|
@@ -316,13 +337,9 @@ class RequirementManager:
|
|
|
316
337
|
shutil.rmtree(self.venv_path)
|
|
317
338
|
|
|
318
339
|
def __enter__(self):
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
logger.exception("Failed to activate venv")
|
|
323
|
-
if self.ephemeral:
|
|
324
|
-
self._deactivate_venv()
|
|
325
|
-
raise
|
|
340
|
+
# _activate_venv restores global state and cleans up on any failure, so
|
|
341
|
+
# we simply let the original exception propagate.
|
|
342
|
+
self._activate_venv()
|
|
326
343
|
return self
|
|
327
344
|
|
|
328
345
|
def __exit__(
|
|
@@ -336,13 +353,7 @@ class RequirementManager:
|
|
|
336
353
|
|
|
337
354
|
def __call__(self, func: Callable[P, R] | None = None) -> Callable[P, R] | None:
|
|
338
355
|
if func is None:
|
|
339
|
-
|
|
340
|
-
self._activate_venv()
|
|
341
|
-
except RuntimeError:
|
|
342
|
-
logger.exception("Failed to activate venv")
|
|
343
|
-
if self.ephemeral:
|
|
344
|
-
self._deactivate_venv()
|
|
345
|
-
raise
|
|
356
|
+
self._activate_venv()
|
|
346
357
|
return None
|
|
347
358
|
|
|
348
359
|
@functools.wraps(func)
|
|
@@ -38,6 +38,27 @@ def _venv_python(venv_path: pathlib.Path) -> pathlib.Path:
|
|
|
38
38
|
return _bin_dir(venv_path) / name
|
|
39
39
|
|
|
40
40
|
|
|
41
|
+
def _tail(output: str, *, max_lines: int = 20, max_chars: int = 4000) -> str:
|
|
42
|
+
"""Return the trailing portion of command output for an error message.
|
|
43
|
+
|
|
44
|
+
Installers print the actual failure at the end of their output, so the tail
|
|
45
|
+
is the diagnostic part. The full output is logged separately; this keeps the
|
|
46
|
+
exception message readable.
|
|
47
|
+
"""
|
|
48
|
+
output = output.strip()
|
|
49
|
+
if not output:
|
|
50
|
+
return "(no output captured)"
|
|
51
|
+
lines = output.splitlines()
|
|
52
|
+
truncated = len(lines) > max_lines
|
|
53
|
+
text = "\n".join(lines[-max_lines:])
|
|
54
|
+
if len(text) > max_chars:
|
|
55
|
+
text = text[-max_chars:]
|
|
56
|
+
truncated = True
|
|
57
|
+
if truncated:
|
|
58
|
+
text = f"(truncated; see logs for full output)\n{text}"
|
|
59
|
+
return text
|
|
60
|
+
|
|
61
|
+
|
|
41
62
|
class VenvBackend:
|
|
42
63
|
"""Abstract base class for virtual environment backends."""
|
|
43
64
|
|
|
@@ -70,17 +91,36 @@ class VenvBackend:
|
|
|
70
91
|
"""Check if the backend is supported on the current system."""
|
|
71
92
|
|
|
72
93
|
def _run_command(self, cmd: list[str]) -> None: # noqa: PLR6301
|
|
73
|
-
"""Run a command in the virtual environment.
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
94
|
+
"""Run a command in the virtual environment.
|
|
95
|
+
|
|
96
|
+
Output is captured (stdout and stderr merged) and stays quiet on success.
|
|
97
|
+
On failure the full output is logged and a trailing excerpt is included in
|
|
98
|
+
the raised ``RuntimeError`` for debuggability.
|
|
99
|
+
"""
|
|
100
|
+
cmd_str = " ".join(cmd)
|
|
101
|
+
logger.debug("Running command: %s", cmd_str)
|
|
102
|
+
result = subprocess.run(
|
|
103
|
+
cmd,
|
|
104
|
+
stdout=subprocess.PIPE,
|
|
105
|
+
stderr=subprocess.STDOUT,
|
|
106
|
+
text=True,
|
|
107
|
+
check=False,
|
|
108
|
+
)
|
|
109
|
+
output = result.stdout or ""
|
|
110
|
+
if result.returncode != 0:
|
|
111
|
+
logger.error(
|
|
112
|
+
"Command failed with exit code %d: %s\nOutput:\n%s",
|
|
113
|
+
result.returncode,
|
|
114
|
+
cmd_str,
|
|
115
|
+
output,
|
|
80
116
|
)
|
|
81
|
-
|
|
82
|
-
f"Command failed with exit code {
|
|
83
|
-
|
|
117
|
+
message = "\n".join([
|
|
118
|
+
f"Command failed with exit code {result.returncode}: {cmd_str}",
|
|
119
|
+
"Output:",
|
|
120
|
+
_tail(output),
|
|
121
|
+
])
|
|
122
|
+
raise RuntimeError(message)
|
|
123
|
+
logger.debug("Command output:\n%s", output)
|
|
84
124
|
|
|
85
125
|
|
|
86
126
|
class VenvBackendRegistry:
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pydepinject
|
|
3
|
-
Version: 0.0.
|
|
3
|
+
Version: 0.0.5.dev0
|
|
4
4
|
Summary: A package to dynamically inject requirements into a virtual environment.
|
|
5
5
|
Author: pydepinject
|
|
6
6
|
License-Expression: MIT
|
|
@@ -266,6 +266,84 @@ This helps with debugging and reproducibility.
|
|
|
266
266
|
|
|
267
267
|
These variables provide a convenient way to standardize behavior in CI/CD pipelines or development environments.
|
|
268
268
|
|
|
269
|
+
## Where venvs are stored and cleaning them up
|
|
270
|
+
|
|
271
|
+
Each managed environment is a plain virtual environment directory under a single
|
|
272
|
+
root. By default that root is `<system-temp-dir>/pydepinject/venvs` (overridable
|
|
273
|
+
with `PYDEPINJECT_VENV_ROOT`), and each venv is a subdirectory named either after
|
|
274
|
+
your `venv_name` or a content hash derived from the requirement set, Python
|
|
275
|
+
version, and backend.
|
|
276
|
+
|
|
277
|
+
Because the default root lives under the OS temporary directory, the operating
|
|
278
|
+
system usually reclaims it for you — temp-file cleaners and reboots clear stale
|
|
279
|
+
entries — so in the common case there is **nothing to clean up manually**.
|
|
280
|
+
|
|
281
|
+
For finer control:
|
|
282
|
+
|
|
283
|
+
- **Avoid persistence per call**: pass `ephemeral=True` so the venv is deleted as
|
|
284
|
+
soon as the decorated function or `with` block finishes.
|
|
285
|
+
- **Prune a persistent root you manage**: the venvs are just directories, so
|
|
286
|
+
`rm -rf "$PYDEPINJECT_VENV_ROOT"` removes everything, or delete individual
|
|
287
|
+
subdirectories to reclaim space selectively.
|
|
288
|
+
- **Script age-based cleanup**: each venv contains `.pydepinject-*.json` metadata
|
|
289
|
+
files recording the install timestamp (and the packages installed), which you
|
|
290
|
+
can use to find and remove environments older than a chosen age.
|
|
291
|
+
|
|
292
|
+
## Limitations and thread-safety
|
|
293
|
+
|
|
294
|
+
`pydepinject` works by mutating **process-global interpreter state** for the
|
|
295
|
+
duration of the managed scope (the decorated call or `with` block). When a
|
|
296
|
+
requirement is not already importable, it:
|
|
297
|
+
|
|
298
|
+
- inserts the venv's `site-packages` at the front of `sys.path`,
|
|
299
|
+
- prepends to the `PYTHONPATH` and `PATH` environment variables,
|
|
300
|
+
- evicts conflicting entries from `sys.modules` so fresh imports resolve to the venv.
|
|
301
|
+
|
|
302
|
+
The original `sys.path`, environment variables, and `sys.modules` state are
|
|
303
|
+
restored when the scope exits. If every requested package is already satisfied in
|
|
304
|
+
the current environment, **no global state is changed at all** — activation is a
|
|
305
|
+
no-op fast path.
|
|
306
|
+
|
|
307
|
+
Because this state is global to the interpreter, please keep the following in mind:
|
|
308
|
+
|
|
309
|
+
- **Not thread-safe and not safe under `asyncio`/concurrent use.** Two managed
|
|
310
|
+
scopes active at the same time in different threads (or interleaved coroutines)
|
|
311
|
+
will race on `sys.path`, the environment variables, and `sys.modules`, and can
|
|
312
|
+
corrupt each other's save/restore. Use `pydepinject` from a single thread of
|
|
313
|
+
control. If you need dependencies isolated across concurrent workers, give each
|
|
314
|
+
worker its own **process**.
|
|
315
|
+
- **Already-imported modules are handled best-effort.** If a package was already
|
|
316
|
+
imported earlier in the process from a different location, swapping it for the
|
|
317
|
+
venv's copy within the scope cannot be guaranteed — code holding a reference to
|
|
318
|
+
the previously imported module keeps using it.
|
|
319
|
+
- **It mutates the running interpreter, by design.** This makes it ideal for
|
|
320
|
+
scripts, notebooks, plugins, and ad-hoc tooling, but it is **not** a substitute
|
|
321
|
+
for a properly provisioned environment or a lockfile in a production
|
|
322
|
+
application. For those, prefer real environment/dependency management.
|
|
323
|
+
|
|
324
|
+
## When NOT to use this
|
|
325
|
+
|
|
326
|
+
`pydepinject` is built for **scripts, notebooks, plugins, and ad-hoc tooling or
|
|
327
|
+
CI glue** — cases where "make sure X is importable, install it if not" is the
|
|
328
|
+
whole job. It is the wrong tool, and you should reach for a real
|
|
329
|
+
environment plus a lockfile (`uv`, `pip-tools`, Poetry, etc.) instead, when:
|
|
330
|
+
|
|
331
|
+
- You are building a **deployable application or service** where dependency
|
|
332
|
+
reproducibility matters.
|
|
333
|
+
- Your code runs under **threads, `asyncio`, or parallel workers** — activation
|
|
334
|
+
mutates global interpreter state and is not safe to interleave (see
|
|
335
|
+
[Limitations and thread-safety](#limitations-and-thread-safety); give each
|
|
336
|
+
worker its own process).
|
|
337
|
+
- You need **pinned, audited, reproducible** dependency sets across machines and
|
|
338
|
+
CI runs.
|
|
339
|
+
- Packages are **already imported** earlier in the process and must be swapped
|
|
340
|
+
mid-run — `pydepinject` can only do this best-effort.
|
|
341
|
+
- You run a **long-lived production process** where persistent venvs accumulating
|
|
342
|
+
under the temp root would be a problem.
|
|
343
|
+
|
|
344
|
+
For the mechanics behind these caveats, see
|
|
345
|
+
[Limitations and thread-safety](#limitations-and-thread-safety) above.
|
|
346
|
+
|
|
269
347
|
## Unit Tests
|
|
270
348
|
|
|
271
349
|
Unit tests are provided to verify the functionality of the `requires`. The tests use `pytest` and cover various scenarios including decorator usage, context manager usage, ephemeral environments, and more.
|
|
@@ -9,10 +9,5 @@ src/pydepinject.egg-info/SOURCES.txt
|
|
|
9
9
|
src/pydepinject.egg-info/dependency_links.txt
|
|
10
10
|
src/pydepinject.egg-info/requires.txt
|
|
11
11
|
src/pydepinject.egg-info/top_level.txt
|
|
12
|
-
src/requirementmanager.egg-info/PKG-INFO
|
|
13
|
-
src/requirementmanager.egg-info/SOURCES.txt
|
|
14
|
-
src/requirementmanager.egg-info/dependency_links.txt
|
|
15
|
-
src/requirementmanager.egg-info/requires.txt
|
|
16
|
-
src/requirementmanager.egg-info/top_level.txt
|
|
17
12
|
tests/conftest.py
|
|
18
13
|
tests/test_pydepinject.py
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
from __future__ import annotations
|
|
2
2
|
|
|
3
3
|
import functools
|
|
4
|
+
import os
|
|
5
|
+
import sys
|
|
4
6
|
|
|
5
7
|
import pytest
|
|
6
8
|
|
|
@@ -376,6 +378,121 @@ def test_invalid_requirements_install_raises_error_decorator(
|
|
|
376
378
|
assert len(list(venv_root.iterdir())) == 1
|
|
377
379
|
|
|
378
380
|
|
|
381
|
+
@pytest.mark.parametrize(
|
|
382
|
+
"exc",
|
|
383
|
+
[RuntimeError("install boom"), FileNotFoundError("missing binary")],
|
|
384
|
+
ids=["runtimeerror", "filenotfounderror"],
|
|
385
|
+
)
|
|
386
|
+
@pytest.mark.parametrize("ephemeral", [True, False], ids=["ephemeral", "non-ephemeral"])
|
|
387
|
+
def test_failed_activation_restores_global_state(
|
|
388
|
+
venv_root, monkeypatch, ephemeral, exc
|
|
389
|
+
):
|
|
390
|
+
"""A failure during installation must restore sys.path and env vars and
|
|
391
|
+
propagate the original exception type, regardless of ephemeral.
|
|
392
|
+
|
|
393
|
+
The venv is created (so global state has been mutated) before install runs,
|
|
394
|
+
so a raising install exercises the failure-cleanup path.
|
|
395
|
+
"""
|
|
396
|
+
with pytest.raises(ImportError):
|
|
397
|
+
import attrs
|
|
398
|
+
|
|
399
|
+
def _boom(*args, **kwargs): # noqa: ARG001 - stub replacing backend.install
|
|
400
|
+
raise exc
|
|
401
|
+
|
|
402
|
+
from pydepinject import backends
|
|
403
|
+
|
|
404
|
+
monkeypatch.setattr(backends.VenvBackendVenv, "install", _boom, raising=True)
|
|
405
|
+
|
|
406
|
+
# Concrete sentinel for PYTHONPATH so restoration is unambiguous; PATH is left
|
|
407
|
+
# at its real value because venv creation shells out and needs it.
|
|
408
|
+
monkeypatch.setenv("PYTHONPATH", "/pdi-sentinel")
|
|
409
|
+
pythonpath_before = os.environ["PYTHONPATH"]
|
|
410
|
+
path_before = os.environ["PATH"]
|
|
411
|
+
syspath_before = sys.path.copy()
|
|
412
|
+
|
|
413
|
+
mgr = requires(
|
|
414
|
+
"attrs", venv_root=venv_root, ephemeral=ephemeral, venv_backend="venv"
|
|
415
|
+
)
|
|
416
|
+
|
|
417
|
+
with pytest.raises(type(exc)):
|
|
418
|
+
mgr()
|
|
419
|
+
|
|
420
|
+
assert sys.path == syspath_before
|
|
421
|
+
assert os.environ["PATH"] == path_before
|
|
422
|
+
assert os.environ["PYTHONPATH"] == pythonpath_before
|
|
423
|
+
|
|
424
|
+
# The import should still fail in the current interpreter.
|
|
425
|
+
with pytest.raises(ImportError):
|
|
426
|
+
import attrs
|
|
427
|
+
|
|
428
|
+
|
|
429
|
+
def test_interrupt_mid_activation_restores_env(venv_root, monkeypatch):
|
|
430
|
+
"""An interrupt landing after env mutation but before activation completes
|
|
431
|
+
must still restore PATH/PYTHONPATH.
|
|
432
|
+
|
|
433
|
+
The env vars are set before ``sys.path.insert``; making ``insert`` raise
|
|
434
|
+
reproduces an interrupt in that narrow window. Cleanup must not depend on the
|
|
435
|
+
activation having reached its final ``_activated = True``.
|
|
436
|
+
"""
|
|
437
|
+
with pytest.raises(ImportError):
|
|
438
|
+
import attrs
|
|
439
|
+
|
|
440
|
+
monkeypatch.setenv("PYTHONPATH", "/pdi-sentinel")
|
|
441
|
+
pythonpath_before = os.environ["PYTHONPATH"]
|
|
442
|
+
path_before = os.environ["PATH"]
|
|
443
|
+
|
|
444
|
+
# Must be a real list subclass: sys.path has to stay a plain list for
|
|
445
|
+
# CPython's import machinery, so UserList is not an option here.
|
|
446
|
+
class _RaisingInsertList(list): # noqa: FURB189
|
|
447
|
+
def insert(self, *args, **kwargs): # noqa: ARG002, PLR6301 - stub
|
|
448
|
+
raise KeyboardInterrupt("simulated interrupt mid-activation")
|
|
449
|
+
|
|
450
|
+
monkeypatch.setattr(sys, "path", _RaisingInsertList(sys.path))
|
|
451
|
+
|
|
452
|
+
mgr = requires("attrs", venv_root=venv_root, venv_backend="venv")
|
|
453
|
+
|
|
454
|
+
with pytest.raises(KeyboardInterrupt):
|
|
455
|
+
mgr()
|
|
456
|
+
|
|
457
|
+
assert os.environ["PYTHONPATH"] == pythonpath_before
|
|
458
|
+
assert os.environ["PATH"] == path_before
|
|
459
|
+
|
|
460
|
+
|
|
461
|
+
def test_run_command_surfaces_output_on_failure(tmp_path):
|
|
462
|
+
"""A failing command must surface its captured output in the raised error.
|
|
463
|
+
|
|
464
|
+
The script is written to a file (rather than ``-c``) so the marker strings
|
|
465
|
+
appear only in the subprocess output, never in the command line itself.
|
|
466
|
+
"""
|
|
467
|
+
from pydepinject import backends
|
|
468
|
+
|
|
469
|
+
script_path = tmp_path / "boom.py"
|
|
470
|
+
script_path.write_text(
|
|
471
|
+
"import sys\n"
|
|
472
|
+
"sys.stdout.write('hello-from-stdout\\n')\n"
|
|
473
|
+
"sys.stderr.write('boom-from-stderr\\n')\n"
|
|
474
|
+
"sys.exit(3)\n"
|
|
475
|
+
)
|
|
476
|
+
|
|
477
|
+
backend = backends.VenvBackendVenv(tmp_path)
|
|
478
|
+
|
|
479
|
+
with pytest.raises(RuntimeError) as excinfo:
|
|
480
|
+
backend._run_command([sys.executable, str(script_path)])
|
|
481
|
+
|
|
482
|
+
message = str(excinfo.value)
|
|
483
|
+
assert "boom-from-stderr" in message
|
|
484
|
+
assert "hello-from-stdout" in message
|
|
485
|
+
assert "exit code 3" in message
|
|
486
|
+
|
|
487
|
+
|
|
488
|
+
def test_run_command_success_returns_without_error(tmp_path):
|
|
489
|
+
"""A successful command must not raise even though output is now captured."""
|
|
490
|
+
from pydepinject import backends
|
|
491
|
+
|
|
492
|
+
backend = backends.VenvBackendVenv(tmp_path)
|
|
493
|
+
backend._run_command([sys.executable, "-c", "print('ok')"])
|
|
494
|
+
|
|
495
|
+
|
|
379
496
|
def test_invalid_backend_value_raises_valueerror(venv_root):
|
|
380
497
|
# Supplying an unknown backend should raise ValueError during initialization
|
|
381
498
|
with pytest.raises(ValueError, match="Invalid venv_backend: invalid"):
|
|
@@ -1,32 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: requirementmanager
|
|
3
|
-
Version: 0.0.1.dev0
|
|
4
|
-
Summary: A package to dynamically inject requirements into a virtual environment.
|
|
5
|
-
Author-email: Your Name <your.email@example.com>
|
|
6
|
-
Maintainer-email: Your Name <your.email@example.com>
|
|
7
|
-
License: MIT
|
|
8
|
-
Project-URL: homepage, https://github.com/yourusername/requirementmanager
|
|
9
|
-
Project-URL: documentation, https://github.com/yourusername/requirementmanager
|
|
10
|
-
Project-URL: repository, https://github.com/yourusername/requirementmanager
|
|
11
|
-
Keywords: virtualenv,requirements,dependency management
|
|
12
|
-
Classifier: Development Status :: 4 - Beta
|
|
13
|
-
Classifier: Intended Audience :: Developers
|
|
14
|
-
Classifier: Topic :: Software Development :: Libraries
|
|
15
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
16
|
-
Classifier: Programming Language :: Python :: 3
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.9
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
-
Classifier: Typing :: Typed
|
|
22
|
-
Description-Content-Type: text/x-rst
|
|
23
|
-
Provides-Extra: dev
|
|
24
|
-
Requires-Dist: ruff; extra == "dev"
|
|
25
|
-
Requires-Dist: pytest; extra == "dev"
|
|
26
|
-
Requires-Dist: flake8; extra == "dev"
|
|
27
|
-
Requires-Dist: black; extra == "dev"
|
|
28
|
-
Requires-Dist: pyright; extra == "dev"
|
|
29
|
-
Requires-Dist: twine; extra == "dev"
|
|
30
|
-
Requires-Dist: wheel; extra == "dev"
|
|
31
|
-
Requires-Dist: setuptools; extra == "dev"
|
|
32
|
-
Requires-Dist: pre-commit; extra == "dev"
|
|
@@ -1,7 +0,0 @@
|
|
|
1
|
-
pyproject.toml
|
|
2
|
-
src/pydepinject/__init__.py
|
|
3
|
-
src/requirementmanager.egg-info/PKG-INFO
|
|
4
|
-
src/requirementmanager.egg-info/SOURCES.txt
|
|
5
|
-
src/requirementmanager.egg-info/dependency_links.txt
|
|
6
|
-
src/requirementmanager.egg-info/requires.txt
|
|
7
|
-
src/requirementmanager.egg-info/top_level.txt
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
pydepinject
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{pydepinject-0.0.4.dev0 → pydepinject-0.0.5.dev0}/src/pydepinject.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|