staticer 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,170 @@
1
+ Metadata-Version: 2.4
2
+ Name: staticer
3
+ Version: 0.0.1
4
+ Summary: Embed dependencies in dynamic-executables
5
+ Home-page: https://github.com/mon231/staticer/
6
+ Author: Ariel Tubul
7
+ Requires-Python: >=3.9
8
+ Description-Content-Type: text/markdown
9
+ Requires-Dist: firstclass<0.3
10
+ Requires-Dist: pefile
11
+ Requires-Dist: pyelftools
12
+ Dynamic: author
13
+ Dynamic: description
14
+ Dynamic: description-content-type
15
+ Dynamic: home-page
16
+ Dynamic: requires-dist
17
+ Dynamic: requires-python
18
+ Dynamic: summary
19
+
20
+ # staticer
21
+ Know it when you create a dynamiclly-linked executable, <br />
22
+ And try to execute it on other machines, but it fails to find expected symbols? <br />
23
+ Using your linux-executable on other machines with different libc, and can't find expected GLIBC function versions? <br />
24
+ Compiling a Windows-program and failing to run it on other machines? Or simply want your project to be published as onefile? <br />
25
+ Wait no more as `staticer` is here to the rescue! <br />
26
+ No more execution failures or long-taking compilation adaptions, <br />
27
+ Just to simply support same-kernel different-OS / OS versioning!
28
+
29
+ **NOTE** you should use this tool for legit-development / debugging / educational purposes only!
30
+
31
+ ## Installation
32
+ Simply run:
33
+ > pip install staticer
34
+
35
+ Make sure to have python scripts folder in your path, <br/>
36
+ And use the correct version of pip for python3.
37
+
38
+ ## Usage
39
+ ```
40
+ staticer <input> -o <output> [--exclude NAME[,NAME...]] [--lib-dir PATH]
41
+ ```
42
+
43
+ **NOTE** `-o`/`--output` overwrites an existing file at that path without warning or backup.
44
+
45
+ ## Goals & Parctice
46
+ We want to get en executable-file and embed its dependencies recursively. <br />
47
+ We need to patch the executable-file's import-table and prevent it from loading the dynamic libraries. <br />
48
+ We need to patch the executable-file's import-table and have the resolved symbol addresses in it, <br />
49
+ And make the loader ignore that import-table in order not to clash, as it is invalidated. <br />
50
+ We need to embed the code/data/... sections of each embedded dependency, <br />
51
+ Patch their import-table the same way, and call their initializers (i.e. `DllMain` / `Constructor`)
52
+
53
+ In order to intercept explicit dynamic-loading (from the executed code itself), <br />
54
+ We'll have to hook `dlopen`+`dlsym` / `LoadLibraryW`+`GetProcAddress` and make them return addresses to embedded resources, <br />
55
+ Or make them call the original functions, if they try to access dynamic-libraries that weren't embedded.
56
+
57
+ **NOTE** OS-based / C++ `exception` unwinding works across embedded-dependency boundaries (unwind metadata is preserved); a self-contained embedded module that both throws and catches internally may still hit compiler-specific personality-routine details beyond the documented ABI
58
+ **NOTE** we currently do NOT support invoking `elf-destructor` / `DllMain` process-detach
59
+ **NOTE** we might never support `DllMain` invoking for thread attach/detach reasons
60
+
61
+ ## Features & How They're Implemented
62
+
63
+ Everything below operates on the real on-disk binary format directly (`pefile`
64
+ for PE, `pyelftools` for reading ELF + hand-packed structs for writing it back,
65
+ since `pyelftools` is read-only) — no linker, compiler, or native toolchain is
66
+ invoked by the installed package itself.
67
+
68
+ - **Dependency graph & embed order.** `staticer` recursively resolves a
69
+ binary's imports (PE Import Directory / ELF `DT_NEEDED`) into a dependency
70
+ graph, then topologically sorts it so each embedded dependency's own
71
+ initializer runs before anything that depends on it, mirroring normal
72
+ loader behavior. Exclusion (`--exclude`, or a dependency found outside
73
+ `--lib-dir`) propagates *transitively*: anything an excluded dependency
74
+ itself imports is also left dynamic, so the same code never ends up both
75
+ embedded and dynamically loaded as two independent copies. Cycles among
76
+ embeddable dependencies are detected and rejected with a clear error naming
77
+ the cycle, rather than looping forever or silently picking an order.
78
+
79
+ - **PIC / relocation-bearing dependencies (the common case).** A
80
+ dependency's code/data sections are appended into the host as new
81
+ sections/segments, its relocations (PE `.reloc`, ELF `.rela`/`.rel`) are
82
+ re-applied against the new placement, and the host's import-table entries
83
+ pointing at it are patched to the embedded address and removed from the
84
+ import table / `DT_NEEDED`. On ELF, new content extends the host's own
85
+ existing loadable segment in place rather than adding a new one — adding a
86
+ genuinely new segment and relocating the program-header table into it
87
+ breaks the kernel's `AT_PHDR` calculation (its skew assumption only holds
88
+ for a segment that shares the *original* image's `vaddr - offset` skew),
89
+ silently breaking TLS/RELRO/stack-flag setup. This took three real,
90
+ execution-verified fix rounds to get right.
91
+
92
+ - **Fixed-base / non-relocatable dependencies.** A dependency with *no*
93
+ usable relocation data (as opposed to one that merely prefers a fixed
94
+ base but still carries relocations — those always take the normal path
95
+ above) is placed at its own declared preferred address instead, with
96
+ every such reservation checked against the host's own image, every other
97
+ reserved dependency, and any bridging filler sections, failing loudly on
98
+ a genuine address conflict rather than producing a silently-malformed
99
+ binary. On PE, embedding any such dependency also clears
100
+ `DYNAMIC_BASE`/`HIGH_ENTROPY_VA` on the output, since a fixed-base
101
+ dependency's untracked absolute references are only valid if the whole
102
+ combined image never gets relocated as a whole. ELF has no equivalent
103
+ per-image flag to clear, so embedding a fixed-base dependency into a PIE
104
+ host fails loudly instead.
105
+
106
+ - **Exception / unwind metadata.** PE `.pdata`/`.xdata` (the
107
+ `RUNTIME_FUNCTION`/`UNWIND_INFO` exception directory) and ELF
108
+ `.eh_frame`/`.eh_frame_hdr` entries are merged into the host rather than
109
+ discarded, with every RVA/offset they reference patched to the relocated
110
+ address — so a C++ exception, or a Windows structured exception, can
111
+ unwind correctly across the boundary between the host and an embedded
112
+ dependency. A fully self-contained module that both throws *and* catches
113
+ entirely within itself can still hit compiler-internal personality-routine
114
+ details beyond the documented unwind ABI (see the note above) — this is a
115
+ real limit of the public unwind data format, not something this project's
116
+ implementation chose to skip.
117
+
118
+ - **Generated entrypoint stub.** The output's entrypoint is replaced with a
119
+ small generated stub that calls each embedded dependency's initializer
120
+ (`DllMain` process-attach / ELF constructor) in dependency order, then
121
+ jumps to the original entrypoint. On PE, the stub correctly distinguishes
122
+ real process-attach from thread-attach/detach when the output itself is a
123
+ DLL (so embedded initializers run exactly once, not on every thread
124
+ event). On ELF, a `.so` input can't rely on `e_entry` at all — the
125
+ dynamic linker never calls it for a shared object — so its init sequence
126
+ is injected as an additional `DT_INIT_ARRAY` entry instead, which *does*
127
+ run via the normal constructor-calling mechanism.
128
+
129
+ - **`--exclude` / `--lib-dir` scoping.** `--exclude NAME[,NAME...]` keeps
130
+ specific dependencies dynamic by name (case-insensitively on PE, matching
131
+ Windows' own case-insensitive filenames; case-sensitively on ELF, matching
132
+ real filesystem semantics on Linux). `--lib-dir PATH` restricts embedding
133
+ to dependencies actually found inside that folder — anything resolved from
134
+ elsewhere (including system directories) stays a normal dynamic import.
135
+ Dependency names come straight from the binary being processed, so they're
136
+ treated as untrusted input: a crafted name with path-traversal components
137
+ or an absolute path is rejected rather than allowed to escape the intended
138
+ search directory.
139
+
140
+ - **Real-world binaries.** Tested against actual system binaries
141
+ (`cmd.exe`/`hostname.exe` on Windows, `bash`/`grep` on Linux), both as a
142
+ curated subset (via `--lib-dir`/`--exclude`) and as closer-to-full
143
+ transitive closures. A handful of real-world-only bugs only ever
144
+ surfaced against genuine system DLLs/`.so`s — forwarded exports,
145
+ ordinal-only imports, forwarder import cycles, stale `DT_VERNEED`
146
+ entries, `R_*_COPY` relocations for a dependency's own globals, and weak
147
+ symbols from the C runtime's own startup boilerplate — all fixed and
148
+ covered by regression tests built from the same binaries that found them.
149
+ Also tried against a genuine real-world *instrumentation library*
150
+ (Frida's own `frida-gadget`, downloaded for real from its GitHub release
151
+ assets — see `tests/windows/test_pe_frida_gadget.py` /
152
+ `tests/linux/test_elf_frida_gadget.py`): it carries its own TLS
153
+ directory on both PE and ELF, so it currently hits the
154
+ already-documented TLS limitation below rather than embedding
155
+ end-to-end — confirmed by reading frida-core's own config-resolution
156
+ source that, were TLS merging supported, frida-gadget's config-file
157
+ lookup would resolve relative to the *host's* own path once embedded
158
+ (since there's no longer a separately-loaded module for it to resolve
159
+ against), so the config would need to be named after the host binary's
160
+ stem, not the gadget's.
161
+
162
+ ## Limitations
163
+ - Explicit dynamic-loading from the executed code itself (`dlopen`+`dlsym` / `LoadLibraryW`+`GetProcAddress`) is not hooked yet; only dependencies resolved through the input's own import/dynamic-section table are embedded.
164
+ - A "full closure" embed on ELF still bottoms out at glibc's own dynamic linker/loader internals, which stay dynamically linked rather than embedded.
165
+ - Process-detach cleanup (`elf-destructor` / `DllMain` `DLL_PROCESS_DETACH`) is not invoked for embedded dependencies, and thread attach/detach notifications may never be supported.
166
+ - Because `DLL_PROCESS_DETACH` is never delivered to an embedded dependency (see above), calling `FreeLibrary` on a staticer-produced DLL *before* process exit can crash the process: a dependency built against a dynamically-linked CRT registers process-wide TLS/FLS state during its (real, CRT-startup) `DLL_PROCESS_ATTACH` that nothing ever tears down, and the CRT's own exit-time bookkeeping later dereferences it after the freed library's memory is gone. Confirmed reproducible. Safe as long as the output DLL stays loaded for the life of the process (the common case); avoid `FreeLibrary`-ing it otherwise.
167
+ - `-o`/`--output` silently overwrites an existing file at that path (see Usage above).
168
+ - A dependency using PE Thread Local Storage (`__declspec(thread)`) is rejected with a clear error rather than embedded incorrectly — TLS-directory merging isn't implemented yet.
169
+ - ELF embedding of a dependency whose own further imports aren't themselves embedded or excluded (an "in-between" import) isn't supported; such a dependency fails loudly rather than silently losing that further dependency.
170
+ - A COPY-relocated ELF symbol larger than one pointer, containing its own interior pointer fields, doesn't get relocation entries for those interior pointers — a genuine architectural limit, since ELF carries no type/layout metadata for a COPY-relocated symbol beyond a byte count.
@@ -0,0 +1,151 @@
1
+ # staticer
2
+ Know it when you create a dynamiclly-linked executable, <br />
3
+ And try to execute it on other machines, but it fails to find expected symbols? <br />
4
+ Using your linux-executable on other machines with different libc, and can't find expected GLIBC function versions? <br />
5
+ Compiling a Windows-program and failing to run it on other machines? Or simply want your project to be published as onefile? <br />
6
+ Wait no more as `staticer` is here to the rescue! <br />
7
+ No more execution failures or long-taking compilation adaptions, <br />
8
+ Just to simply support same-kernel different-OS / OS versioning!
9
+
10
+ **NOTE** you should use this tool for legit-development / debugging / educational purposes only!
11
+
12
+ ## Installation
13
+ Simply run:
14
+ > pip install staticer
15
+
16
+ Make sure to have python scripts folder in your path, <br/>
17
+ And use the correct version of pip for python3.
18
+
19
+ ## Usage
20
+ ```
21
+ staticer <input> -o <output> [--exclude NAME[,NAME...]] [--lib-dir PATH]
22
+ ```
23
+
24
+ **NOTE** `-o`/`--output` overwrites an existing file at that path without warning or backup.
25
+
26
+ ## Goals & Parctice
27
+ We want to get en executable-file and embed its dependencies recursively. <br />
28
+ We need to patch the executable-file's import-table and prevent it from loading the dynamic libraries. <br />
29
+ We need to patch the executable-file's import-table and have the resolved symbol addresses in it, <br />
30
+ And make the loader ignore that import-table in order not to clash, as it is invalidated. <br />
31
+ We need to embed the code/data/... sections of each embedded dependency, <br />
32
+ Patch their import-table the same way, and call their initializers (i.e. `DllMain` / `Constructor`)
33
+
34
+ In order to intercept explicit dynamic-loading (from the executed code itself), <br />
35
+ We'll have to hook `dlopen`+`dlsym` / `LoadLibraryW`+`GetProcAddress` and make them return addresses to embedded resources, <br />
36
+ Or make them call the original functions, if they try to access dynamic-libraries that weren't embedded.
37
+
38
+ **NOTE** OS-based / C++ `exception` unwinding works across embedded-dependency boundaries (unwind metadata is preserved); a self-contained embedded module that both throws and catches internally may still hit compiler-specific personality-routine details beyond the documented ABI
39
+ **NOTE** we currently do NOT support invoking `elf-destructor` / `DllMain` process-detach
40
+ **NOTE** we might never support `DllMain` invoking for thread attach/detach reasons
41
+
42
+ ## Features & How They're Implemented
43
+
44
+ Everything below operates on the real on-disk binary format directly (`pefile`
45
+ for PE, `pyelftools` for reading ELF + hand-packed structs for writing it back,
46
+ since `pyelftools` is read-only) — no linker, compiler, or native toolchain is
47
+ invoked by the installed package itself.
48
+
49
+ - **Dependency graph & embed order.** `staticer` recursively resolves a
50
+ binary's imports (PE Import Directory / ELF `DT_NEEDED`) into a dependency
51
+ graph, then topologically sorts it so each embedded dependency's own
52
+ initializer runs before anything that depends on it, mirroring normal
53
+ loader behavior. Exclusion (`--exclude`, or a dependency found outside
54
+ `--lib-dir`) propagates *transitively*: anything an excluded dependency
55
+ itself imports is also left dynamic, so the same code never ends up both
56
+ embedded and dynamically loaded as two independent copies. Cycles among
57
+ embeddable dependencies are detected and rejected with a clear error naming
58
+ the cycle, rather than looping forever or silently picking an order.
59
+
60
+ - **PIC / relocation-bearing dependencies (the common case).** A
61
+ dependency's code/data sections are appended into the host as new
62
+ sections/segments, its relocations (PE `.reloc`, ELF `.rela`/`.rel`) are
63
+ re-applied against the new placement, and the host's import-table entries
64
+ pointing at it are patched to the embedded address and removed from the
65
+ import table / `DT_NEEDED`. On ELF, new content extends the host's own
66
+ existing loadable segment in place rather than adding a new one — adding a
67
+ genuinely new segment and relocating the program-header table into it
68
+ breaks the kernel's `AT_PHDR` calculation (its skew assumption only holds
69
+ for a segment that shares the *original* image's `vaddr - offset` skew),
70
+ silently breaking TLS/RELRO/stack-flag setup. This took three real,
71
+ execution-verified fix rounds to get right.
72
+
73
+ - **Fixed-base / non-relocatable dependencies.** A dependency with *no*
74
+ usable relocation data (as opposed to one that merely prefers a fixed
75
+ base but still carries relocations — those always take the normal path
76
+ above) is placed at its own declared preferred address instead, with
77
+ every such reservation checked against the host's own image, every other
78
+ reserved dependency, and any bridging filler sections, failing loudly on
79
+ a genuine address conflict rather than producing a silently-malformed
80
+ binary. On PE, embedding any such dependency also clears
81
+ `DYNAMIC_BASE`/`HIGH_ENTROPY_VA` on the output, since a fixed-base
82
+ dependency's untracked absolute references are only valid if the whole
83
+ combined image never gets relocated as a whole. ELF has no equivalent
84
+ per-image flag to clear, so embedding a fixed-base dependency into a PIE
85
+ host fails loudly instead.
86
+
87
+ - **Exception / unwind metadata.** PE `.pdata`/`.xdata` (the
88
+ `RUNTIME_FUNCTION`/`UNWIND_INFO` exception directory) and ELF
89
+ `.eh_frame`/`.eh_frame_hdr` entries are merged into the host rather than
90
+ discarded, with every RVA/offset they reference patched to the relocated
91
+ address — so a C++ exception, or a Windows structured exception, can
92
+ unwind correctly across the boundary between the host and an embedded
93
+ dependency. A fully self-contained module that both throws *and* catches
94
+ entirely within itself can still hit compiler-internal personality-routine
95
+ details beyond the documented unwind ABI (see the note above) — this is a
96
+ real limit of the public unwind data format, not something this project's
97
+ implementation chose to skip.
98
+
99
+ - **Generated entrypoint stub.** The output's entrypoint is replaced with a
100
+ small generated stub that calls each embedded dependency's initializer
101
+ (`DllMain` process-attach / ELF constructor) in dependency order, then
102
+ jumps to the original entrypoint. On PE, the stub correctly distinguishes
103
+ real process-attach from thread-attach/detach when the output itself is a
104
+ DLL (so embedded initializers run exactly once, not on every thread
105
+ event). On ELF, a `.so` input can't rely on `e_entry` at all — the
106
+ dynamic linker never calls it for a shared object — so its init sequence
107
+ is injected as an additional `DT_INIT_ARRAY` entry instead, which *does*
108
+ run via the normal constructor-calling mechanism.
109
+
110
+ - **`--exclude` / `--lib-dir` scoping.** `--exclude NAME[,NAME...]` keeps
111
+ specific dependencies dynamic by name (case-insensitively on PE, matching
112
+ Windows' own case-insensitive filenames; case-sensitively on ELF, matching
113
+ real filesystem semantics on Linux). `--lib-dir PATH` restricts embedding
114
+ to dependencies actually found inside that folder — anything resolved from
115
+ elsewhere (including system directories) stays a normal dynamic import.
116
+ Dependency names come straight from the binary being processed, so they're
117
+ treated as untrusted input: a crafted name with path-traversal components
118
+ or an absolute path is rejected rather than allowed to escape the intended
119
+ search directory.
120
+
121
+ - **Real-world binaries.** Tested against actual system binaries
122
+ (`cmd.exe`/`hostname.exe` on Windows, `bash`/`grep` on Linux), both as a
123
+ curated subset (via `--lib-dir`/`--exclude`) and as closer-to-full
124
+ transitive closures. A handful of real-world-only bugs only ever
125
+ surfaced against genuine system DLLs/`.so`s — forwarded exports,
126
+ ordinal-only imports, forwarder import cycles, stale `DT_VERNEED`
127
+ entries, `R_*_COPY` relocations for a dependency's own globals, and weak
128
+ symbols from the C runtime's own startup boilerplate — all fixed and
129
+ covered by regression tests built from the same binaries that found them.
130
+ Also tried against a genuine real-world *instrumentation library*
131
+ (Frida's own `frida-gadget`, downloaded for real from its GitHub release
132
+ assets — see `tests/windows/test_pe_frida_gadget.py` /
133
+ `tests/linux/test_elf_frida_gadget.py`): it carries its own TLS
134
+ directory on both PE and ELF, so it currently hits the
135
+ already-documented TLS limitation below rather than embedding
136
+ end-to-end — confirmed by reading frida-core's own config-resolution
137
+ source that, were TLS merging supported, frida-gadget's config-file
138
+ lookup would resolve relative to the *host's* own path once embedded
139
+ (since there's no longer a separately-loaded module for it to resolve
140
+ against), so the config would need to be named after the host binary's
141
+ stem, not the gadget's.
142
+
143
+ ## Limitations
144
+ - Explicit dynamic-loading from the executed code itself (`dlopen`+`dlsym` / `LoadLibraryW`+`GetProcAddress`) is not hooked yet; only dependencies resolved through the input's own import/dynamic-section table are embedded.
145
+ - A "full closure" embed on ELF still bottoms out at glibc's own dynamic linker/loader internals, which stay dynamically linked rather than embedded.
146
+ - Process-detach cleanup (`elf-destructor` / `DllMain` `DLL_PROCESS_DETACH`) is not invoked for embedded dependencies, and thread attach/detach notifications may never be supported.
147
+ - Because `DLL_PROCESS_DETACH` is never delivered to an embedded dependency (see above), calling `FreeLibrary` on a staticer-produced DLL *before* process exit can crash the process: a dependency built against a dynamically-linked CRT registers process-wide TLS/FLS state during its (real, CRT-startup) `DLL_PROCESS_ATTACH` that nothing ever tears down, and the CRT's own exit-time bookkeeping later dereferences it after the freed library's memory is gone. Confirmed reproducible. Safe as long as the output DLL stays loaded for the life of the process (the common case); avoid `FreeLibrary`-ing it otherwise.
148
+ - `-o`/`--output` silently overwrites an existing file at that path (see Usage above).
149
+ - A dependency using PE Thread Local Storage (`__declspec(thread)`) is rejected with a clear error rather than embedded incorrectly — TLS-directory merging isn't implemented yet.
150
+ - ELF embedding of a dependency whose own further imports aren't themselves embedded or excluded (an "in-between" import) isn't supported; such a dependency fails loudly rather than silently losing that further dependency.
151
+ - A COPY-relocated ELF symbol larger than one pointer, containing its own interior pointer fields, doesn't get relocation entries for those interior pointers — a genuine architectural limit, since ELF carries no type/layout metadata for a COPY-relocated symbol beyond a byte count.
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,26 @@
1
+ import setuptools
2
+ from pathlib import Path
3
+
4
+
5
+ CURRENT_FOLDER = Path(__file__).parent
6
+ README_PATH = CURRENT_FOLDER / 'README.md'
7
+
8
+
9
+ def main():
10
+ setuptools.setup(
11
+ name = "staticer",
12
+ version = "0.0.1",
13
+ author = "Ariel Tubul",
14
+ description = "Embed dependencies in dynamic-executables",
15
+ packages = setuptools.find_packages(),
16
+ long_description = README_PATH.read_text(),
17
+ url = "https://github.com/mon231/staticer/",
18
+ long_description_content_type='text/markdown',
19
+ entry_points = {'console_scripts': ['staticer=staticer.staticer:main']},
20
+ python_requires = '>=3.9',
21
+ install_requires = ['firstclass<0.3', 'pefile', 'pyelftools']
22
+ )
23
+
24
+
25
+ if __name__ == '__main__':
26
+ main()
@@ -0,0 +1 @@
1
+ from .staticer import main
File without changes
@@ -0,0 +1,116 @@
1
+ """Format-agnostic dependency graph: resolution, transitive exclusion, and
2
+ dependency-first init order, driven entirely by caller-supplied
3
+ ``list_imports``/``resolve`` callables.
4
+
5
+ This module knows nothing about PE or ELF; those formats supply their own
6
+ ``list_imports``/``resolve`` implementations (see ``pe_embedder``/
7
+ ``elf_embedder``) and call into ``build_dependency_graph``/``embed_order``.
8
+ """
9
+
10
+ from dataclasses import field
11
+ from pathlib import Path
12
+ from typing import Callable, Dict, List, Optional, Set
13
+
14
+ from firstclass import firstclass
15
+
16
+
17
+ @firstclass
18
+ class DependencyNode:
19
+ name: str
20
+ path: Path
21
+ embeddable: bool
22
+ # Names of this node's own resolved imports (edges), keyed the same
23
+ # way as the graph dict. Populated by build_dependency_graph;
24
+ # embed_order walks it to produce a dependencies-first order. Not
25
+ # part of the original interface's three fields, but required for
26
+ # embed_order to have any edge information to sort over, since the
27
+ # graph dict is its only input.
28
+ imports: List[str] = field(default_factory=list)
29
+
30
+
31
+ class CyclicDependencyError(Exception):
32
+ pass
33
+
34
+
35
+ def build_dependency_graph(
36
+ root_path: Path,
37
+ list_imports: Callable[[Path], List[str]],
38
+ resolve: Callable[[str, List[Path]], Optional[Path]],
39
+ search_dirs: List[Path],
40
+ lib_dir: Optional[Path],
41
+ exclude: Set[str],
42
+ ) -> Dict[str, DependencyNode]:
43
+ nodes: Dict[str, DependencyNode] = {}
44
+
45
+ def in_scope(path: Path) -> bool:
46
+ return lib_dir is None or path.is_relative_to(lib_dir)
47
+
48
+ def visit(name: str, path: Path, force_excluded: bool) -> None:
49
+ is_excluded = force_excluded or (name in exclude) or not in_scope(path)
50
+ if name in nodes:
51
+ node = nodes[name]
52
+ # Only cascade when this visit newly excludes the node (guards
53
+ # against both redundant re-cascading on repeat revisits and,
54
+ # critically, infinite recursion around cycles: once a node is
55
+ # already False, re-walking its recorded imports again is a
56
+ # no-op).
57
+ if is_excluded and node.embeddable:
58
+ node.embeddable = False
59
+ for child_name in node.imports:
60
+ child = nodes.get(child_name)
61
+ if child is not None:
62
+ visit(child_name, child.path, force_excluded=True)
63
+ return
64
+ node = DependencyNode(name=name, path=path, embeddable=not is_excluded)
65
+ nodes[name] = node
66
+ for imp_name in list_imports(path):
67
+ resolved = resolve(imp_name, search_dirs)
68
+ if resolved is None:
69
+ continue
70
+ node.imports.append(imp_name)
71
+ visit(imp_name, resolved, force_excluded=is_excluded)
72
+
73
+ for imp_name in list_imports(root_path):
74
+ resolved = resolve(imp_name, search_dirs)
75
+ if resolved is not None:
76
+ visit(imp_name, resolved, force_excluded=False)
77
+ return nodes
78
+
79
+
80
+ def embed_order(graph: Dict[str, DependencyNode]) -> List[DependencyNode]:
81
+ """Dependencies-first topological order over the embeddable subset of
82
+ ``graph``. Raises CyclicDependencyError, naming the cycle, if a cycle
83
+ exists among embeddable nodes."""
84
+ order: List[DependencyNode] = []
85
+ visited: Set[str] = set()
86
+ in_progress: List[str] = []
87
+ in_progress_set: Set[str] = set()
88
+
89
+ def visit(name: str) -> None:
90
+ if name in visited:
91
+ return
92
+ if name in in_progress_set:
93
+ cycle = in_progress[in_progress.index(name):] + [name]
94
+ raise CyclicDependencyError(
95
+ "cyclic dependency detected: " + " -> ".join(cycle)
96
+ )
97
+
98
+ node = graph[name]
99
+ in_progress.append(name)
100
+ in_progress_set.add(name)
101
+
102
+ for imp_name in node.imports:
103
+ dep = graph.get(imp_name)
104
+ if dep is not None and dep.embeddable:
105
+ visit(imp_name)
106
+
107
+ in_progress.pop()
108
+ in_progress_set.discard(name)
109
+ visited.add(name)
110
+ order.append(node)
111
+
112
+ for name, node in graph.items():
113
+ if node.embeddable:
114
+ visit(name)
115
+
116
+ return order