codehound 1.0.3__tar.gz → 1.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: codehound
3
- Version: 1.0.3
3
+ Version: 1.2.0
4
4
  Summary: An AST-based static analyzer that hunts real correctness and async-safety bugs in Python code.
5
5
  Project-URL: Homepage, https://github.com/kratos0718/codehound
6
6
  Project-URL: Issues, https://github.com/kratos0718/codehound/issues
@@ -18,9 +18,13 @@ Provides-Extra: dev
18
18
  Requires-Dist: pytest>=7; extra == 'dev'
19
19
  Description-Content-Type: text/markdown
20
20
 
21
- # 🐕 codehound
21
+ <p align="center">
22
+ <img src="assets/logo.png" alt="codehound" width="220">
23
+ </p>
22
24
 
23
- **An AST-based static analyzer that hunts *real* bugs in large Python codebases — every rule is backed by a bug that was actually found and merged into a major open-source AI framework.**
25
+ <h1 align="center">codehound</h1>
26
+
27
+ **An AST-based static analyzer that hunts *real* bugs in large Python codebases — seven of the ten rules are backed by a bug that was actually found and merged into a major open-source AI framework; the other three are hardening rules verified against real false positives instead.**
24
28
 
25
29
  [![CI](https://github.com/kratos0718/codehound/actions/workflows/ci.yml/badge.svg)](https://github.com/kratos0718/codehound/actions/workflows/ci.yml)
26
30
  [![PyPI](https://img.shields.io/pypi/v/codehound.svg)](https://pypi.org/project/codehound/)
@@ -116,9 +120,66 @@ codehound list
116
120
  | **CH004** | `deprecated-get-event-loop` | `asyncio.get_event_loop()` outside a running loop — deprecated since 3.10. | crewAI structured-tool / Snowflake search tool |
117
121
  | **CH005** | `unclosed-file-handle` | `f = open(...)` with no `with` and no matching `.close()` — leaks descriptors until `RLIMIT_NOFILE` is exhausted. | agno `OpenAITools.transcribe_audio` |
118
122
  | **CH006** | `floating-task` | `asyncio.create_task(...)` whose result is discarded — the loop keeps only a *weak* reference, so the task can be GC'd before it finishes. (Ruff RUF006) | hardening rule — the most under-caught async bug |
123
+ | **CH007** | `unawaited-coroutine-call` | `foo()` where `foo` is `async def`, called as a bare statement — no `await`, no scheduling. The coroutine object is created and dropped; the body **never runs at all**. | hardening rule — see below |
124
+ | **CH008** | `asyncio-run-in-running-loop` | `asyncio.run(...)` called from inside an `async def` — always raises `RuntimeError`, immediately, every time. | hardening rule — zero corpus hits (see below) |
125
+ | **CH009** | `floating-thread` | A non-daemon `threading.Thread` that's `.start()`ed but never `.join()`ed — the thread analog of CH006. | hardening rule — see below |
126
+ | **CH010** | `loop-closure-capture` | A `lambda` inside a `for` loop (or comprehension) that's *stored* (appended, assigned, returned) and captures the loop variable by reference — every stored instance ends up sharing the loop's **final** value. | **accelerate** (HuggingFace) — `MegatronEngine.get_module_config`'s `param_sync_func` list, PR #4273 |
119
127
 
120
128
  `codehound list` prints this from the source of truth.
121
129
 
130
+ CH007-CH010 don't have found-and-merged bugs behind all of them the way
131
+ CH001-CH006 do - three are hardening rules for well-known Python
132
+ correctness gotchas rather than something this project personally
133
+ tracked down first. CH010 is the exception: it found a genuine, serious
134
+ bug on its own, in HuggingFace's `accelerate` - see below. Building all
135
+ four surfaced real false positives, each one fixed before shipping:
136
+
137
+ - **CH007** (agno): a bare `self.foo()` call matched against an unrelated
138
+ same-named `async def foo` on a *different* class (agno's own
139
+ sync/async "twin method" convention, e.g. `ZepTools`/`ZepAsyncTools`),
140
+ and a plain callable parameter shadowed by an unrelated same-named
141
+ async function hundreds of lines away in the same file.
142
+ - **CH009** (llama_index): a thread handed off through a *different*
143
+ object's attribute, not `self` - `chat_response.write_response_to_history_thread
144
+ = thread`, with `chat_response` itself returned and the thread joined
145
+ later once the caller finishes consuming the stream.
146
+ - **CH010** (marimo): `sorted(rows, key=lambda row: row[sort_arg.by])`
147
+ inside `for sort_arg in ...` - the lambda references the loop variable,
148
+ but `sorted()` calls it *immediately*, synchronously, before the next
149
+ iteration moves `sort_arg` on. Nothing outlives the iteration. This
150
+ reshaped the check entirely: it now only fires when a lambda is
151
+ directly *stored* (`.append(...)`, assignment, `return`), not merely
152
+ passed as a callback argument to something that consumes it on the spot.
153
+
154
+ **The `accelerate` find (CH010):** `MegatronEngine.get_module_config`
155
+ builds one callback per model chunk for distributed-training parameter
156
+ sync: `[lambda x: self.optimizer.finish_param_sync(model_index, x) for
157
+ model_index in range(len(self.module))]`. Every lambda captures
158
+ `model_index` by reference; by the time any of them actually runs, the
159
+ comprehension has finished and `model_index` holds its final value for
160
+ *all* of them - whichever chunk's callback fires, it reports the
161
+ *last* chunk's index. Fixed with the standard default-argument capture
162
+ (`model_index=model_index`) and a regression test that fails on the
163
+ pre-fix code (all three callbacks report index 2) and passes on the fix.
164
+ PR: [huggingface/accelerate#4273](https://github.com/huggingface/accelerate/pull/4273).
165
+
166
+ Scanning ~20 major Python AI/ML frameworks with the fixed CH007/CH008/CH009
167
+ turned up zero further real instances beyond the ones above - itself a
168
+ result, not a null: CH008's bug fails immediately and unconditionally, so
169
+ it's very unlikely to survive basic testing; CH007 and CH009 both only
170
+ match same-file names by design, and most real cases of either are
171
+ plausibly cross-module.
172
+
173
+ **One check we built and did not ship: CH011 `exception-chaining`**
174
+ (`except X as e: raise Y(...)` with no `from e`, discarding the real
175
+ traceback - overlaps flake8-bugbear B904). It worked exactly as designed,
176
+ but at a scale that says more about how common the pattern is than about
177
+ anything worth flagging: **1,911 hits across the same ~20-framework
178
+ corpus**. Shipping a check that fires that often would make every scan
179
+ result mostly CH011 noise, undermining the "a finding must be defensible"
180
+ standard the rest of this tool holds itself to. Built, measured, and
181
+ deliberately left out - a real decision, not an oversight.
182
+
122
183
  ---
123
184
 
124
185
  ## How it works
@@ -135,12 +196,16 @@ codehound/
135
196
  ├── datetime_utcnow.py (CH003)
136
197
  ├── get_event_loop.py (CH004)
137
198
  ├── resource_leak.py (CH005)
138
- └── floating_task.py (CH006)
199
+ ├── floating_task.py (CH006)
200
+ ├── unawaited_coroutine.py (CH007)
201
+ ├── asyncio_run_in_loop.py (CH008)
202
+ ├── floating_thread.py (CH009)
203
+ └── loop_closure_capture.py (CH010)
139
204
  ```
140
205
 
141
206
  Each check receives a parsed `ast` tree plus the precomputed parent map and returns `Finding`s. Adding a rule is one file + one registry line + a test. See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for a full walkthrough of the engine, the parent map, and the design decisions.
142
207
 
143
- **False-positive discipline is a feature.** CH005 won't flag a handle that's `return`ed (the caller owns it) or explicitly `.close()`d. CH006 won't flag `TaskGroup.create_task` (the group holds the reference). CH001 only fires when the *enclosing* function is `async`. The test suite asserts both "bad code is flagged" and "correct code is not."
208
+ **False-positive discipline is a feature.** CH005 won't flag a handle that's `return`ed (the caller owns it) or explicitly `.close()`d. CH006 won't flag `TaskGroup.create_task` (the group holds the reference). CH001 only fires when the *enclosing* function is `async`. CH007 scopes `self.foo()` matches to async methods on the *same* class as the call site, and bare `foo()` matches to module-level async functions that aren't shadowed by a same-named parameter. CH009 doesn't flag a thread handed off as *any* object's attribute, not just `self`. CH010 only fires when a lambda is directly stored (appended, assigned, returned), not merely passed as a callback argument that gets consumed on the spot. All four of those guards exist because of real false positives caught while building the checks (see above). The test suite asserts both "bad code is flagged" and "correct code is not."
144
209
 
145
210
  ---
146
211
 
@@ -157,10 +222,15 @@ Every check has paired tests: the buggy pattern *is* flagged, and the idiomatic
157
222
 
158
223
  ## Roadmap
159
224
 
160
- - [ ] `await` on a non-awaited coroutine (missing-await detection)
225
+ - [x] `await` on a non-awaited coroutine (missing-await detection) — CH007
226
+ - [x] PyPI release — `pip install codehound`
227
+ - [x] `asyncio.run()` inside a running loop — CH008
228
+ - [x] Non-daemon thread started without a join — CH009 (the thread analog of CH006)
229
+ - [x] Loop-variable closure capture in lambdas — CH010
230
+ - [ ] Cross-module resolution for CH007/CH009 (currently same-file only)
161
231
  - [ ] Sync HTTP clients constructed inside async request handlers
162
232
  - [ ] `--fix` for the mechanical rules (CH002, CH003, CH004)
163
- - [ ] Pre-commit hook + PyPI release
233
+ - [ ] Pre-commit hook
164
234
 
165
235
  ---
166
236
 
@@ -1,6 +1,10 @@
1
- # 🐕 codehound
1
+ <p align="center">
2
+ <img src="assets/logo.png" alt="codehound" width="220">
3
+ </p>
2
4
 
3
- **An AST-based static analyzer that hunts *real* bugs in large Python codebases — every rule is backed by a bug that was actually found and merged into a major open-source AI framework.**
5
+ <h1 align="center">codehound</h1>
6
+
7
+ **An AST-based static analyzer that hunts *real* bugs in large Python codebases — seven of the ten rules are backed by a bug that was actually found and merged into a major open-source AI framework; the other three are hardening rules verified against real false positives instead.**
4
8
 
5
9
  [![CI](https://github.com/kratos0718/codehound/actions/workflows/ci.yml/badge.svg)](https://github.com/kratos0718/codehound/actions/workflows/ci.yml)
6
10
  [![PyPI](https://img.shields.io/pypi/v/codehound.svg)](https://pypi.org/project/codehound/)
@@ -96,9 +100,66 @@ codehound list
96
100
  | **CH004** | `deprecated-get-event-loop` | `asyncio.get_event_loop()` outside a running loop — deprecated since 3.10. | crewAI structured-tool / Snowflake search tool |
97
101
  | **CH005** | `unclosed-file-handle` | `f = open(...)` with no `with` and no matching `.close()` — leaks descriptors until `RLIMIT_NOFILE` is exhausted. | agno `OpenAITools.transcribe_audio` |
98
102
  | **CH006** | `floating-task` | `asyncio.create_task(...)` whose result is discarded — the loop keeps only a *weak* reference, so the task can be GC'd before it finishes. (Ruff RUF006) | hardening rule — the most under-caught async bug |
103
+ | **CH007** | `unawaited-coroutine-call` | `foo()` where `foo` is `async def`, called as a bare statement — no `await`, no scheduling. The coroutine object is created and dropped; the body **never runs at all**. | hardening rule — see below |
104
+ | **CH008** | `asyncio-run-in-running-loop` | `asyncio.run(...)` called from inside an `async def` — always raises `RuntimeError`, immediately, every time. | hardening rule — zero corpus hits (see below) |
105
+ | **CH009** | `floating-thread` | A non-daemon `threading.Thread` that's `.start()`ed but never `.join()`ed — the thread analog of CH006. | hardening rule — see below |
106
+ | **CH010** | `loop-closure-capture` | A `lambda` inside a `for` loop (or comprehension) that's *stored* (appended, assigned, returned) and captures the loop variable by reference — every stored instance ends up sharing the loop's **final** value. | **accelerate** (HuggingFace) — `MegatronEngine.get_module_config`'s `param_sync_func` list, PR #4273 |
99
107
 
100
108
  `codehound list` prints this from the source of truth.
101
109
 
110
+ CH007-CH010 don't have found-and-merged bugs behind all of them the way
111
+ CH001-CH006 do - three are hardening rules for well-known Python
112
+ correctness gotchas rather than something this project personally
113
+ tracked down first. CH010 is the exception: it found a genuine, serious
114
+ bug on its own, in HuggingFace's `accelerate` - see below. Building all
115
+ four surfaced real false positives, each one fixed before shipping:
116
+
117
+ - **CH007** (agno): a bare `self.foo()` call matched against an unrelated
118
+ same-named `async def foo` on a *different* class (agno's own
119
+ sync/async "twin method" convention, e.g. `ZepTools`/`ZepAsyncTools`),
120
+ and a plain callable parameter shadowed by an unrelated same-named
121
+ async function hundreds of lines away in the same file.
122
+ - **CH009** (llama_index): a thread handed off through a *different*
123
+ object's attribute, not `self` - `chat_response.write_response_to_history_thread
124
+ = thread`, with `chat_response` itself returned and the thread joined
125
+ later once the caller finishes consuming the stream.
126
+ - **CH010** (marimo): `sorted(rows, key=lambda row: row[sort_arg.by])`
127
+ inside `for sort_arg in ...` - the lambda references the loop variable,
128
+ but `sorted()` calls it *immediately*, synchronously, before the next
129
+ iteration moves `sort_arg` on. Nothing outlives the iteration. This
130
+ reshaped the check entirely: it now only fires when a lambda is
131
+ directly *stored* (`.append(...)`, assignment, `return`), not merely
132
+ passed as a callback argument to something that consumes it on the spot.
133
+
134
+ **The `accelerate` find (CH010):** `MegatronEngine.get_module_config`
135
+ builds one callback per model chunk for distributed-training parameter
136
+ sync: `[lambda x: self.optimizer.finish_param_sync(model_index, x) for
137
+ model_index in range(len(self.module))]`. Every lambda captures
138
+ `model_index` by reference; by the time any of them actually runs, the
139
+ comprehension has finished and `model_index` holds its final value for
140
+ *all* of them - whichever chunk's callback fires, it reports the
141
+ *last* chunk's index. Fixed with the standard default-argument capture
142
+ (`model_index=model_index`) and a regression test that fails on the
143
+ pre-fix code (all three callbacks report index 2) and passes on the fix.
144
+ PR: [huggingface/accelerate#4273](https://github.com/huggingface/accelerate/pull/4273).
145
+
146
+ Scanning ~20 major Python AI/ML frameworks with the fixed CH007/CH008/CH009
147
+ turned up zero further real instances beyond the ones above - itself a
148
+ result, not a null: CH008's bug fails immediately and unconditionally, so
149
+ it's very unlikely to survive basic testing; CH007 and CH009 both only
150
+ match same-file names by design, and most real cases of either are
151
+ plausibly cross-module.
152
+
153
+ **One check we built and did not ship: CH011 `exception-chaining`**
154
+ (`except X as e: raise Y(...)` with no `from e`, discarding the real
155
+ traceback - overlaps flake8-bugbear B904). It worked exactly as designed,
156
+ but at a scale that says more about how common the pattern is than about
157
+ anything worth flagging: **1,911 hits across the same ~20-framework
158
+ corpus**. Shipping a check that fires that often would make every scan
159
+ result mostly CH011 noise, undermining the "a finding must be defensible"
160
+ standard the rest of this tool holds itself to. Built, measured, and
161
+ deliberately left out - a real decision, not an oversight.
162
+
102
163
  ---
103
164
 
104
165
  ## How it works
@@ -115,12 +176,16 @@ codehound/
115
176
  ├── datetime_utcnow.py (CH003)
116
177
  ├── get_event_loop.py (CH004)
117
178
  ├── resource_leak.py (CH005)
118
- └── floating_task.py (CH006)
179
+ ├── floating_task.py (CH006)
180
+ ├── unawaited_coroutine.py (CH007)
181
+ ├── asyncio_run_in_loop.py (CH008)
182
+ ├── floating_thread.py (CH009)
183
+ └── loop_closure_capture.py (CH010)
119
184
  ```
120
185
 
121
186
  Each check receives a parsed `ast` tree plus the precomputed parent map and returns `Finding`s. Adding a rule is one file + one registry line + a test. See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for a full walkthrough of the engine, the parent map, and the design decisions.
122
187
 
123
- **False-positive discipline is a feature.** CH005 won't flag a handle that's `return`ed (the caller owns it) or explicitly `.close()`d. CH006 won't flag `TaskGroup.create_task` (the group holds the reference). CH001 only fires when the *enclosing* function is `async`. The test suite asserts both "bad code is flagged" and "correct code is not."
188
+ **False-positive discipline is a feature.** CH005 won't flag a handle that's `return`ed (the caller owns it) or explicitly `.close()`d. CH006 won't flag `TaskGroup.create_task` (the group holds the reference). CH001 only fires when the *enclosing* function is `async`. CH007 scopes `self.foo()` matches to async methods on the *same* class as the call site, and bare `foo()` matches to module-level async functions that aren't shadowed by a same-named parameter. CH009 doesn't flag a thread handed off as *any* object's attribute, not just `self`. CH010 only fires when a lambda is directly stored (appended, assigned, returned), not merely passed as a callback argument that gets consumed on the spot. All four of those guards exist because of real false positives caught while building the checks (see above). The test suite asserts both "bad code is flagged" and "correct code is not."
124
189
 
125
190
  ---
126
191
 
@@ -137,10 +202,15 @@ Every check has paired tests: the buggy pattern *is* flagged, and the idiomatic
137
202
 
138
203
  ## Roadmap
139
204
 
140
- - [ ] `await` on a non-awaited coroutine (missing-await detection)
205
+ - [x] `await` on a non-awaited coroutine (missing-await detection) — CH007
206
+ - [x] PyPI release — `pip install codehound`
207
+ - [x] `asyncio.run()` inside a running loop — CH008
208
+ - [x] Non-daemon thread started without a join — CH009 (the thread analog of CH006)
209
+ - [x] Loop-variable closure capture in lambdas — CH010
210
+ - [ ] Cross-module resolution for CH007/CH009 (currently same-file only)
141
211
  - [ ] Sync HTTP clients constructed inside async request handlers
142
212
  - [ ] `--fix` for the mechanical rules (CH002, CH003, CH004)
143
- - [ ] Pre-commit hook + PyPI release
213
+ - [ ] Pre-commit hook
144
214
 
145
215
  ---
146
216
 
@@ -1,7 +1,9 @@
1
1
  """codehound - an AST-based static analyzer that hunts real bugs in Python code.
2
2
 
3
- Six checks, each backed by a bug that was actually found and fixed in a popular
4
- open-source AI framework (agno, crewAI, mem0, huggingface_hub).
3
+ Ten checks. Seven are each backed by a bug that was actually found and
4
+ fixed in a popular open-source AI framework (agno, crewAI, mem0,
5
+ llama_index, accelerate). The other three (CH007-CH009) are hardening
6
+ rules verified against real false positives instead - see docs/FINDINGS.md.
5
7
  """
6
8
 
7
9
  from __future__ import annotations
@@ -9,7 +11,7 @@ from __future__ import annotations
9
11
  from codehound.checks import ALL_CHECKS, get_checks
10
12
  from codehound.core import Check, Finding, scan_file, scan_path
11
13
 
12
- __version__ = "1.0.3"
14
+ __version__ = "1.2.0"
13
15
 
14
16
  __all__ = [
15
17
  "ALL_CHECKS",
@@ -2,12 +2,16 @@
2
2
 
3
3
  from __future__ import annotations
4
4
 
5
+ from codehound.checks.asyncio_run_in_loop import AsyncioRunInRunningLoop
5
6
  from codehound.checks.blocking_async import BlockingCallInAsync
6
7
  from codehound.checks.datetime_utcnow import DeprecatedDatetimeUtcnow
7
8
  from codehound.checks.floating_task import FloatingTask
9
+ from codehound.checks.floating_thread import FloatingThread
8
10
  from codehound.checks.get_event_loop import DeprecatedGetEventLoop
11
+ from codehound.checks.loop_closure_capture import LoopClosureCapture
9
12
  from codehound.checks.mutable_defaults import MutableDefaultArgument
10
13
  from codehound.checks.resource_leak import UnclosedFileHandle
14
+ from codehound.checks.unawaited_coroutine import UnawaitedCoroutineCall
11
15
  from codehound.core import Check
12
16
 
13
17
  ALL_CHECKS: list[type[Check]] = [
@@ -17,6 +21,10 @@ ALL_CHECKS: list[type[Check]] = [
17
21
  DeprecatedGetEventLoop,
18
22
  UnclosedFileHandle,
19
23
  FloatingTask,
24
+ UnawaitedCoroutineCall,
25
+ AsyncioRunInRunningLoop,
26
+ FloatingThread,
27
+ LoopClosureCapture,
20
28
  ]
21
29
 
22
30
 
@@ -0,0 +1,62 @@
1
+ """CH008 - ``asyncio.run()`` called while a loop is already running.
2
+
3
+ ``asyncio.run()`` creates a new event loop and refuses to run if one is
4
+ already active on the current thread - calling it from inside a coroutine
5
+ raises ``RuntimeError: asyncio.run() cannot be called from a running event
6
+ loop`` immediately, every time, unconditionally. Unlike most of the other
7
+ checks here, this isn't a subtle production-only failure; it fails the
8
+ first time the code path executes. It shows up anyway, usually from a
9
+ function that was written and tested as a synchronous entry point, then
10
+ got called from newly-async'd calling code without anyone changing its
11
+ body.
12
+
13
+ Scoped to the *immediate* enclosing function only, matching how the other
14
+ checks avoid cross-function analysis: `async def outer(): def inner():
15
+ asyncio.run(...)` does not flag, since `inner` is nested but still
16
+ synchronous itself - whether calling it while a loop is running is
17
+ actually safe depends on what thread it runs on, which this checker has
18
+ no way to know.
19
+ """
20
+
21
+ from __future__ import annotations
22
+
23
+ import ast
24
+
25
+ from codehound.core import Check, Finding, enclosing_function
26
+
27
+
28
+ class AsyncioRunInRunningLoop(Check):
29
+ code = "CH008"
30
+ name = "asyncio-run-in-running-loop"
31
+ description = "asyncio.run() called from inside an async function; always raises RuntimeError."
32
+
33
+ def run(self, tree: ast.AST, parents: dict, path: str) -> list[Finding]:
34
+ findings: list[Finding] = []
35
+ for node in ast.walk(tree):
36
+ if not isinstance(node, ast.Call):
37
+ continue
38
+ func = node.func
39
+ if not (
40
+ isinstance(func, ast.Attribute)
41
+ and func.attr == "run"
42
+ and isinstance(func.value, ast.Name)
43
+ and func.value.id == "asyncio"
44
+ ):
45
+ continue
46
+ fn = enclosing_function(node, parents)
47
+ if not isinstance(fn, ast.AsyncFunctionDef):
48
+ continue
49
+ findings.append(
50
+ Finding(
51
+ path=path,
52
+ line=node.lineno,
53
+ col=node.col_offset,
54
+ code=self.code,
55
+ message=(
56
+ f"`asyncio.run(...)` inside async function `{fn.name}` always raises "
57
+ f"RuntimeError - a loop is already running here; `await` the coroutine "
58
+ f"directly instead."
59
+ ),
60
+ )
61
+ )
62
+ return findings
@@ -0,0 +1,159 @@
1
+ """CH009 - Non-daemon ``threading.Thread`` started without a matching ``join()``.
2
+
3
+ The thread analog of CH006's floating-task: a plain ``threading.Thread``
4
+ that gets ``.start()``ed but never joined keeps running independently of
5
+ whoever created it. Unlike a daemon thread (an explicit, intentional
6
+ "let this die with the process" choice), a *non*-daemon thread that's
7
+ never joined can outlive the function that spawned it, hold the
8
+ interpreter open past when the caller expects the program to exit, or
9
+ race with cleanup code that assumed the work was done because the
10
+ function that started it returned.
11
+
12
+ Two shapes:
13
+ - Chained: ``threading.Thread(target=f).start()`` - the thread object is
14
+ never even captured, so joining it later is structurally impossible.
15
+ - Assigned: ``t = threading.Thread(target=f); t.start()`` - captured, but
16
+ no matching ``t.join()`` (and not returned, not stored as an attribute
17
+ of anything, not marked ``daemon=True``) before the enclosing function
18
+ returns.
19
+
20
+ A real false positive found while building this: llama_index's chat
21
+ engines create the thread, then do ``chat_response.write_response_to_history_thread
22
+ = thread`` and return ``chat_response`` - the thread isn't lost, it's
23
+ handed off through a *different* object's attribute (not ``self``), and
24
+ gets joined later once the caller finishes consuming the stream
25
+ (``chat_engine/types.py``). The escape check below treats a thread
26
+ assigned as *any* object's attribute as a deliberate hand-off, not just
27
+ ``self.<attr>``.
28
+ """
29
+
30
+ from __future__ import annotations
31
+
32
+ import ast
33
+
34
+ from codehound.core import Check, Finding, enclosing_function
35
+
36
+
37
+ def _is_thread_call(node: ast.expr) -> bool:
38
+ if not isinstance(node, ast.Call):
39
+ return False
40
+ func = node.func
41
+ if isinstance(func, ast.Name):
42
+ return func.id == "Thread"
43
+ if isinstance(func, ast.Attribute):
44
+ return func.attr == "Thread" and isinstance(func.value, ast.Name) and func.value.id == "threading"
45
+ return False
46
+
47
+
48
+ def _has_daemon_true_kwarg(call: ast.Call) -> bool:
49
+ for kw in call.keywords:
50
+ if kw.arg == "daemon" and isinstance(kw.value, ast.Constant) and kw.value.value is True:
51
+ return True
52
+ return False
53
+
54
+
55
+ class FloatingThread(Check):
56
+ code = "CH009"
57
+ name = "floating-thread"
58
+ description = "Non-daemon threading.Thread started without a matching join()."
59
+
60
+ def run(self, tree: ast.AST, parents: dict, path: str) -> list[Finding]:
61
+ findings: list[Finding] = []
62
+
63
+ # Chained: threading.Thread(...).start() with no assignment at all.
64
+ for node in ast.walk(tree):
65
+ if not isinstance(node, ast.Expr) or not isinstance(node.value, ast.Call):
66
+ continue
67
+ call = node.value
68
+ if not (isinstance(call.func, ast.Attribute) and call.func.attr == "start"):
69
+ continue
70
+ receiver = call.func.value
71
+ if not _is_thread_call(receiver):
72
+ continue
73
+ if _has_daemon_true_kwarg(receiver):
74
+ continue
75
+ findings.append(
76
+ Finding(
77
+ path=path,
78
+ line=node.lineno,
79
+ col=node.col_offset,
80
+ code=self.code,
81
+ message=(
82
+ "`threading.Thread(...).start()` - the thread is never captured, so it "
83
+ "can never be joined; either keep a reference and join() it, or pass "
84
+ "`daemon=True` if letting it outlive this scope is intentional."
85
+ ),
86
+ )
87
+ )
88
+
89
+ # Assigned: t = threading.Thread(...); ... t.start() ... [no t.join()]
90
+ for node in ast.walk(tree):
91
+ if not isinstance(node, ast.Assign) or not _is_thread_call(node.value):
92
+ continue
93
+ if len(node.targets) != 1 or not isinstance(node.targets[0], ast.Name):
94
+ continue
95
+ name = node.targets[0].id
96
+ call = node.value
97
+ if _has_daemon_true_kwarg(call):
98
+ continue
99
+
100
+ fn = enclosing_function(node, parents)
101
+ if fn is None:
102
+ continue
103
+
104
+ started = False
105
+ joined = False
106
+ escapes = False
107
+ for n in ast.walk(fn):
108
+ if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute) and isinstance(
109
+ n.func.value, ast.Name
110
+ ) and n.func.value.id == name:
111
+ if n.func.attr == "start":
112
+ started = True
113
+ elif n.func.attr == "join":
114
+ joined = True
115
+ elif isinstance(n, ast.Assign):
116
+ for tgt in n.targets:
117
+ if (
118
+ isinstance(tgt, ast.Attribute)
119
+ and tgt.attr != "daemon"
120
+ and isinstance(n.value, ast.Name)
121
+ and n.value.id == name
122
+ ):
123
+ # Stored as an attribute of *any* object (not just
124
+ # self) - e.g. a response object that hands the
125
+ # thread off to its own caller for later joining.
126
+ # Real pattern: llama_index's chat_engine classes
127
+ # stash the thread on the StreamingAgentChatResponse
128
+ # they return, which joins it once the stream is
129
+ # fully consumed (chat_engine/types.py).
130
+ escapes = True
131
+ if (
132
+ isinstance(tgt, ast.Attribute)
133
+ and isinstance(tgt.value, ast.Name)
134
+ and tgt.value.id == name
135
+ and tgt.attr == "daemon"
136
+ and isinstance(n.value, ast.Constant)
137
+ and n.value.value is True
138
+ ):
139
+ escapes = True # daemon set post-construction
140
+ elif isinstance(n, ast.Return) and isinstance(n.value, ast.Name) and n.value.id == name:
141
+ escapes = True
142
+
143
+ if not started or joined or escapes:
144
+ continue
145
+
146
+ findings.append(
147
+ Finding(
148
+ path=path,
149
+ line=node.lineno,
150
+ col=node.col_offset,
151
+ code=self.code,
152
+ message=(
153
+ f"`{name} = threading.Thread(...)` in `{fn.name}` is started but never "
154
+ f"joined; it can outlive this function. Call `{name}.join()`, return it "
155
+ f"to the caller, or pass `daemon=True` if that's intentional."
156
+ ),
157
+ )
158
+ )
159
+ return findings
@@ -0,0 +1,123 @@
1
+ """CH010 - A lambda captures a ``for``-loop variable by reference, not value.
2
+
3
+ Python closures capture *variables*, not their values at creation time.
4
+ ``[lambda: i for i in range(3)]`` creates three lambdas that all read the
5
+ same cell - by the time any of them is called, the loop has finished and
6
+ ``i`` is ``2`` in all three. The fix is to bind the value as a default
7
+ argument (``lambda i=i: i``), which evaluates ``i`` at lambda-creation
8
+ time instead of call time.
9
+
10
+ This only bites when the lambda is called *after* the loop variable has
11
+ moved on: stored in a list and used later, returned, handed to a
12
+ callback registry. The overwhelmingly common way a lambda appears inside
13
+ a loop is the *opposite* of that - passed straight into a higher-order
14
+ function that calls it immediately and returns, all within the same
15
+ iteration (``sorted(rows, key=lambda r: r[field])``, ``filter(...)``,
16
+ ``max(..., key=...)``). That shape is completely safe: nothing outlives
17
+ the iteration. A real false positive of exactly this kind was found
18
+ scanning marimo's table sorter (``sorted(key=lambda row: row[sort_arg.by])``
19
+ inside ``for sort_arg in ...``) before this check was scoped correctly.
20
+
21
+ So this only fires when the lambda is directly *stored* rather than
22
+ passed as a callback: the argument to ``.append(...)``/``.add(...)``, the
23
+ value of an assignment (`` x = lambda: ...``, `` d[k] = lambda: ...``), or
24
+ returned/yielded. A lambda passed as an argument to anything else -
25
+ ``sorted``, ``filter``, ``map``, a comprehension's own condition - is not
26
+ flagged, because nothing has captured it past this iteration.
27
+
28
+ Comprehensions (`` [lambda: i for i in range(3)] ``) are the one
29
+ exception: the ``elt``/``key``/``value`` position *is* inherently the
30
+ "produce and store" position, so no extra storage-context check is
31
+ needed there.
32
+ """
33
+
34
+ from __future__ import annotations
35
+
36
+ import ast
37
+
38
+ from codehound.core import Check, Finding
39
+
40
+ _STORAGE_METHODS = {"append", "add"}
41
+
42
+
43
+ def _references_name_unshadowed(lam: ast.Lambda, name: str) -> bool:
44
+ param_names = {a.arg for a in (lam.args.posonlyargs + lam.args.args + lam.args.kwonlyargs)}
45
+ if lam.args.vararg:
46
+ param_names.add(lam.args.vararg.arg)
47
+ if lam.args.kwarg:
48
+ param_names.add(lam.args.kwarg.arg)
49
+ if name in param_names:
50
+ return False
51
+ for node in ast.walk(lam.body):
52
+ if isinstance(node, ast.Name) and node.id == name and isinstance(node.ctx, ast.Load):
53
+ return True
54
+ return False
55
+
56
+
57
+ def _is_stored(lam: ast.Lambda, parents: dict) -> bool:
58
+ """True only if the lambda is directly stored somewhere that outlives
59
+ this loop iteration - not merely passed as a callback argument to a
60
+ function (sorted/filter/map/...) that consumes it on the spot."""
61
+ parent = parents.get(id(lam))
62
+ if isinstance(parent, (ast.Assign, ast.AnnAssign)):
63
+ return True
64
+ if isinstance(parent, (ast.Return, ast.Yield)):
65
+ return True
66
+ if (
67
+ isinstance(parent, ast.Call)
68
+ and isinstance(parent.func, ast.Attribute)
69
+ and parent.func.attr in _STORAGE_METHODS
70
+ and lam in parent.args
71
+ ):
72
+ return True
73
+ return False
74
+
75
+
76
+ class LoopClosureCapture(Check):
77
+ code = "CH010"
78
+ name = "loop-closure-capture"
79
+ description = "Lambda captures a for-loop variable by reference; all instances see the final value."
80
+
81
+ def run(self, tree: ast.AST, parents: dict, path: str) -> list[Finding]:
82
+ findings: list[Finding] = []
83
+ seen: set[int] = set()
84
+
85
+ for loop in ast.walk(tree):
86
+ targets: list[str] = []
87
+ body_nodes: list[ast.AST] = []
88
+ require_storage = False
89
+ if isinstance(loop, ast.For) and isinstance(loop.target, ast.Name):
90
+ targets = [loop.target.id]
91
+ body_nodes = loop.body
92
+ require_storage = True
93
+ elif isinstance(loop, (ast.ListComp, ast.SetComp, ast.GeneratorExp, ast.DictComp)):
94
+ gens = [g for g in loop.generators if isinstance(g.target, ast.Name)]
95
+ targets = [g.target.id for g in gens]
96
+ body_nodes = [loop.elt] if not isinstance(loop, ast.DictComp) else [loop.key, loop.value]
97
+ else:
98
+ continue
99
+
100
+ for container in body_nodes:
101
+ for node in ast.walk(container):
102
+ if not isinstance(node, ast.Lambda) or id(node) in seen:
103
+ continue
104
+ if require_storage and not _is_stored(node, parents):
105
+ continue
106
+ for var in targets:
107
+ if _references_name_unshadowed(node, var):
108
+ seen.add(id(node))
109
+ findings.append(
110
+ Finding(
111
+ path=path,
112
+ line=node.lineno,
113
+ col=node.col_offset,
114
+ code=self.code,
115
+ message=(
116
+ f"lambda captures loop variable `{var}` by reference; if "
117
+ f"called after the loop moves on, every instance sees the "
118
+ f"same final value. Bind it explicitly: `lambda {var}={var}: ...`."
119
+ ),
120
+ )
121
+ )
122
+ break
123
+ return findings