typedframes 0.4.0__tar.gz → 0.4.1__tar.gz

{typedframes-0.4.0 → typedframes-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: typedframes
-Version: 0.4.0
+Version: 0.4.1
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
@@ -42,7 +42,7 @@ Project-URL: Repository, https://github.com/w-martin/typedframes
 
 > ⚠️ **Project Status: Proof of Concept**
 >
-> `typedframes` (v0.4.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
+> `typedframes` (v0.4.1) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
 > integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
 > column checking over production-grade stability.
 >
@@ -591,17 +591,17 @@ Notes:
 Fast feedback reduces development time. The typedframes Rust binary provides near-instant column checking.
 
 **Benchmark results** (20 runs, 3 warmup, caches cleared between runs):
-*2026-08-12 · Darwin 25.6.0 · arm · CPython 3.14.4 · 64GiB RAM · Great Expectations pinned @ 1.9.3*
+*2026-08-18 · Darwin 25.6.0 · arm · CPython 3.14.4 · 64GiB RAM · Great Expectations pinned @ 1.20.0*
 
-| Tool | Version | What it does | typedframes (13 files) | great_expectations (482 files) |
+| Tool | Version | What it does | typedframes (13 files) | great_expectations (485 files) |
 |------|---------|--------------|------------------------|--------------------------------|
-| typedframes | 0.4.0 | DataFrame column checker | 47ms ±998µs (IQR 1ms) | 183ms ±2ms (IQR 2ms) |
-| ruff | 0.16.2 | Linter (no type checking) | 29ms ±677µs (IQR 729µs) | 208ms ±2ms (IQR 3ms) |
-| ty | 0.0.69 | Type checker | 74ms ±900µs (IQR 1ms) | 941ms ±14ms (IQR 18ms) |
-| pyrefly | 1.2.0 | Type checker | 96ms ±2ms (IQR 2ms) | 275ms ±7ms (IQR 14ms) |
-| mypy | 2.3.0 | Type checker (no plugin) | 2.78s ±47ms (IQR 84ms) | 4.25s ±48ms (IQR 76ms) |
-| mypy + typedframes | 2.3.0 | Type checker + column checker | 2.74s ±29ms (IQR 34ms) | 4.44s ±54ms (IQR 105ms) |
-| pyright | 1.1.411 | Type checker | 808ms ±24ms (IQR 21ms) | 3.45s ±93ms (IQR 138ms) |
+| typedframes | 0.4.1 | DataFrame column checker | 51ms ±918µs (IQR 1ms) | 219ms ±2ms (IQR 4ms) |
+| ruff | 0.16.3 | Linter (no type checking) | 30ms ±764µs (IQR 922µs) | 233ms ±3ms (IQR 4ms) |
+| ty | 0.0.72 | Type checker | 73ms ±1ms (IQR 2ms) | 810ms ±10ms (IQR 13ms) |
+| pyrefly | 1.2.0 | Type checker | 104ms ±2ms (IQR 2ms) | 276ms ±9ms (IQR 14ms) |
+| mypy | 2.3.1 | Type checker (no plugin) | 3.07s ±17ms (IQR 25ms) | 4.56s ±29ms (IQR 57ms) |
+| mypy + typedframes | 2.3.1 | Type checker + column checker | 3.07s ±13ms (IQR 17ms) | 4.84s ±20ms (IQR 23ms) |
+| pyright | 1.1.411 | Type checker | 781ms ±5ms (IQR 6ms) | 3.46s ±25ms (IQR 41ms) |
 
 *Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
 
@@ -815,7 +815,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 
 | Feature                         | typedframes            | Pandera     | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame      | narwhals | dataframely      | patito           |
 |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|------------------|------------------|
-| **Version tested**              | 0.4.0                  | 0.32.1      | 1.19.1             | 0.3.7                 | 3.0.5        | 0.1.2       | 1.1.3              | 5.0.0            | 2.24.0   | 3.0.0            | 0.8.6            |
+| **Version tested**              | 0.4.1                  | 0.32.1      | 1.20.0             | 0.3.7                 | 3.0.5        | 0.1.2       | 1.1.3              | 5.1.1            | 2.24.0   | 3.0.0            | 0.8.6            |
 | **Analysis Type**               |
 | When errors are caught          | **Static (lint-time)** | Runtime     | Runtime            | Runtime               | Static       | Runtime     | Runtime            | Runtime          | Runtime  | Runtime          | Runtime          |
 | **Static Analysis (our focus)** |
@@ -860,7 +860,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 - **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
   static analysis.
 
-- **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.0.0): Alternative immutable DataFrame library.
+- **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.1.1): Alternative immutable DataFrame library.
   Not compatible with pandas/polars — requires a full rewrite to StaticFrame's own API. Column access is still
   string-based; mypy does not catch column name typos. Type safety comes from immutability guarantees, not schema checking.
 
@@ -869,7 +869,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
   type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
   below.
 
-- **[Great Expectations](https://greatexpectations.io/)** (v1.19.1): Comprehensive data quality framework. Defines
+- **[Great Expectations](https://greatexpectations.io/)** (v1.20.0): Comprehensive data quality framework. Defines
   "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
   validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
   code. Supports pandas, Spark, and SQL backends.
@@ -892,10 +892,10 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
 typedframes for comprehensive type checking:
 
-- **[mypy](https://mypy-lang.org/)** (v2.3.0): The original Python type checker. typedframes provides a mypy plugin for
+- **[mypy](https://mypy-lang.org/)** (v2.3.1): The original Python type checker. typedframes provides a mypy plugin for
   column checking. See [performance benchmarks](#static-analysis-performance).
 
-- **[ty](https://github.com/astral-sh/ty)** (v0.0.69, Astral): New Rust-based type checker, faster than mypy on
+- **[ty](https://github.com/astral-sh/ty)** (v0.0.72, Astral): New Rust-based type checker, faster than mypy on
   large codebases. Does not support mypy plugins—use typedframes standalone checker.
 
 - **[pyrefly](https://pyrefly.org/)** (v1.2.0, Meta): Rust-based type checker from Meta, replacement for Pyre. Fast,

{typedframes-0.4.0 → typedframes-0.4.1}/README.md

@@ -8,7 +8,7 @@
 
 > ⚠️ **Project Status: Proof of Concept**
 >
-> `typedframes` (v0.4.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
+> `typedframes` (v0.4.1) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
 > integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
 > column checking over production-grade stability.
 >
@@ -557,17 +557,17 @@ Notes:
 Fast feedback reduces development time. The typedframes Rust binary provides near-instant column checking.
 
 **Benchmark results** (20 runs, 3 warmup, caches cleared between runs):
-*2026-08-12 · Darwin 25.6.0 · arm · CPython 3.14.4 · 64GiB RAM · Great Expectations pinned @ 1.9.3*
+*2026-08-18 · Darwin 25.6.0 · arm · CPython 3.14.4 · 64GiB RAM · Great Expectations pinned @ 1.20.0*
 
-| Tool | Version | What it does | typedframes (13 files) | great_expectations (482 files) |
+| Tool | Version | What it does | typedframes (13 files) | great_expectations (485 files) |
 |------|---------|--------------|------------------------|--------------------------------|
-| typedframes | 0.4.0 | DataFrame column checker | 47ms ±998µs (IQR 1ms) | 183ms ±2ms (IQR 2ms) |
-| ruff | 0.16.2 | Linter (no type checking) | 29ms ±677µs (IQR 729µs) | 208ms ±2ms (IQR 3ms) |
-| ty | 0.0.69 | Type checker | 74ms ±900µs (IQR 1ms) | 941ms ±14ms (IQR 18ms) |
-| pyrefly | 1.2.0 | Type checker | 96ms ±2ms (IQR 2ms) | 275ms ±7ms (IQR 14ms) |
-| mypy | 2.3.0 | Type checker (no plugin) | 2.78s ±47ms (IQR 84ms) | 4.25s ±48ms (IQR 76ms) |
-| mypy + typedframes | 2.3.0 | Type checker + column checker | 2.74s ±29ms (IQR 34ms) | 4.44s ±54ms (IQR 105ms) |
-| pyright | 1.1.411 | Type checker | 808ms ±24ms (IQR 21ms) | 3.45s ±93ms (IQR 138ms) |
+| typedframes | 0.4.1 | DataFrame column checker | 51ms ±918µs (IQR 1ms) | 219ms ±2ms (IQR 4ms) |
+| ruff | 0.16.3 | Linter (no type checking) | 30ms ±764µs (IQR 922µs) | 233ms ±3ms (IQR 4ms) |
+| ty | 0.0.72 | Type checker | 73ms ±1ms (IQR 2ms) | 810ms ±10ms (IQR 13ms) |
+| pyrefly | 1.2.0 | Type checker | 104ms ±2ms (IQR 2ms) | 276ms ±9ms (IQR 14ms) |
+| mypy | 2.3.1 | Type checker (no plugin) | 3.07s ±17ms (IQR 25ms) | 4.56s ±29ms (IQR 57ms) |
+| mypy + typedframes | 2.3.1 | Type checker + column checker | 3.07s ±13ms (IQR 17ms) | 4.84s ±20ms (IQR 23ms) |
+| pyright | 1.1.411 | Type checker | 781ms ±5ms (IQR 6ms) | 3.46s ±25ms (IQR 41ms) |
 
 *Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
 
@@ -781,7 +781,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 
 | Feature                         | typedframes            | Pandera     | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame      | narwhals | dataframely      | patito           |
 |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|------------------|------------------|
-| **Version tested**              | 0.4.0                  | 0.32.1      | 1.19.1             | 0.3.7                 | 3.0.5        | 0.1.2       | 1.1.3              | 5.0.0            | 2.24.0   | 3.0.0            | 0.8.6            |
+| **Version tested**              | 0.4.1                  | 0.32.1      | 1.20.0             | 0.3.7                 | 3.0.5        | 0.1.2       | 1.1.3              | 5.1.1            | 2.24.0   | 3.0.0            | 0.8.6            |
 | **Analysis Type**               |
 | When errors are caught          | **Static (lint-time)** | Runtime     | Runtime            | Runtime               | Static       | Runtime     | Runtime            | Runtime          | Runtime  | Runtime          | Runtime          |
 | **Static Analysis (our focus)** |
@@ -826,7 +826,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 - **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
   static analysis.
 
-- **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.0.0): Alternative immutable DataFrame library.
+- **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.1.1): Alternative immutable DataFrame library.
   Not compatible with pandas/polars — requires a full rewrite to StaticFrame's own API. Column access is still
   string-based; mypy does not catch column name typos. Type safety comes from immutability guarantees, not schema checking.
 
@@ -835,7 +835,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
   type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
   below.
 
-- **[Great Expectations](https://greatexpectations.io/)** (v1.19.1): Comprehensive data quality framework. Defines
+- **[Great Expectations](https://greatexpectations.io/)** (v1.20.0): Comprehensive data quality framework. Defines
   "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
   validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
   code. Supports pandas, Spark, and SQL backends.
@@ -858,10 +858,10 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
 typedframes for comprehensive type checking:
 
-- **[mypy](https://mypy-lang.org/)** (v2.3.0): The original Python type checker. typedframes provides a mypy plugin for
+- **[mypy](https://mypy-lang.org/)** (v2.3.1): The original Python type checker. typedframes provides a mypy plugin for
   column checking. See [performance benchmarks](#static-analysis-performance).
 
-- **[ty](https://github.com/astral-sh/ty)** (v0.0.69, Astral): New Rust-based type checker, faster than mypy on
+- **[ty](https://github.com/astral-sh/ty)** (v0.0.72, Astral): New Rust-based type checker, faster than mypy on
   large codebases. Does not support mypy plugins—use typedframes standalone checker.
 
 - **[pyrefly](https://pyrefly.org/)** (v1.2.0, Meta): Rust-based type checker from Meta, replacement for Pyre. Fast,

{typedframes-0.4.0 → typedframes-0.4.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "typedframes"
-version = "0.4.0"
+version = "0.4.1"
 description = "Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime."
 keywords = ["pandas", "polars", "type-checking", "static-analysis", "dataframe", "linter", "mypy-plugin"]
 readme = "README.md"
@@ -138,14 +138,14 @@ dev = [
     "pytest>=9.1.1",
     "pytest-cov>=7.1.0",
     "pytest-xdist>=3.8.0",
-    "ruff>=0.16.2",
-    "ty>=0.0.69",
+    "ruff>=0.16.3",
+    "ty>=0.0.72",
     "openpyxl>=3.1.5",
-    "pyarrow>=25.0.0",
+    "pyarrow>=25.0.1",
     "xlsxwriter>=3.2.9",
     "fastexcel>=0.20.2",
     "pandas-stubs>=3.0.5.260730",
-    "mypy>=2.3.0",
+    "mypy>=2.3.1",
 ]
 
 [tool.bandit]

{typedframes-0.4.0 → typedframes-0.4.1}/rust/Cargo.lock

@@ -1347,7 +1347,7 @@ checksum = "7d56353a2a665ad0f41a421187180aab746c8c325620617ad883a99a1cbe66d2"
 
 [[package]]
 name = "typedframes_checker"
-version = "0.4.0"
+version = "0.4.1"
 dependencies = [
  "anyhow",
  "criterion",

{typedframes-0.4.0 → typedframes-0.4.1}/rust/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "typedframes_checker"
-version = "0.4.0"
+version = "0.4.1"
 edition = "2021"
 description = "A high-performance Rust-based checker for typedframes DataFrame column schemas"
 license = "MIT"

{typedframes-0.4.0 → typedframes-0.4.1}/rust/src/config.rs

@@ -36,6 +36,14 @@ pub struct LinterConfig {
     // rather than indexing all of site-packages, which would be both expensive and a
     // much larger, unbounded trust surface.
     pub(crate) trace_external_packages: Option<Vec<String>>,
+    // Opt-out counterpart to the default-on candidate tracing (see
+    // `discover_external_package_candidates` in index.rs): a package named here is
+    // never traced, even if the project's own code calls it in a way that looks
+    // DataFrame-shaped. Does NOT affect `trace_external_packages` -- an explicit
+    // force-include there always wins, since naming a package in both lists is a
+    // contradiction the user should resolve, not one this checker should silently
+    // arbitrate.
+    pub(crate) excluded_external_packages: Option<Vec<String>>,
     // Directory names to prune when collecting `.py` files. When set, REPLACES the
     // built-in default set (`DEFAULT_EXCLUDED_DIRS` -- `.git`, `.venv`, `node_modules`,
     // `__pycache__`, `.claude`, etc.) entirely rather than adding to it, matching
@@ -54,6 +62,7 @@ impl LinterConfig {
         warnings: None,
         sql_dialect: None,
         trace_external_packages: None,
+        excluded_external_packages: None,
         exclude: None,
     };
 }
@@ -196,4 +205,27 @@ mod tests {
             Some(vec![".claude".to_string(), "vendor".to_string()])
         );
     }
+
+    #[test]
+    fn test_load_linter_config_reads_excluded_external_packages() {
+        let temp = tempfile::tempdir().unwrap();
+        let root = temp.path();
+
+        fs::write(
+            root.join("pyproject.toml"),
+            "[tool.typedframes]\nenabled = true",
+        )
+        .unwrap();
+        assert_eq!(load_linter_config(root).excluded_external_packages, None);
+
+        fs::write(
+            root.join("pyproject.toml"),
+            "[tool.typedframes]\nexcluded_external_packages = [\"huge_untrusted_pkg\"]",
+        )
+        .unwrap();
+        assert_eq!(
+            load_linter_config(root).excluded_external_packages,
+            Some(vec!["huge_untrusted_pkg".to_string()])
+        );
+    }
 }

{typedframes-0.4.0 → typedframes-0.4.1}/rust/src/index.rs

@@ -262,18 +262,17 @@ pub(crate) fn find_site_packages_dir_uncached(project_root: &Path) -> Option<Pat
     None
 }
 
-// Collect `.py` files for each explicitly allowlisted external package (see
-// LinterConfig::trace_external_packages), resolved from the project's own
-// site-packages directory. Deliberately narrow: only the allowlisted packages' own
-// directories are walked — never the whole site-packages tree, which would be both
-// expensive and a far larger, unbounded trust surface than an explicit opt-in list.
+// Collect `.py` files for each named external package, resolved from the project's
+// own site-packages directory. `packages` is the already-merged, already-filtered set
+// -- see `resolve_traced_external_packages` for how `trace_external_packages`
+// (explicit force-include), auto-discovered candidates, and `excluded_external_packages`
+// combine into it. Deliberately narrow: only the named packages' own directories are
+// walked — never the whole site-packages tree, which would be both expensive and a far
+// larger, unbounded trust surface than a bounded, explainable package list.
 pub(crate) fn collect_external_package_files(
     project_root: &Path,
-    config: &LinterConfig,
+    packages: &[String],
 ) -> Vec<PathBuf> {
-    let Some(packages) = config.trace_external_packages.as_ref() else {
-        return Vec::new();
-    };
     if packages.is_empty() {
         return Vec::new();
     }
@@ -305,11 +304,18 @@ pub(crate) fn collect_external_package_files(
 // Runs the linter in index mode (diagnostics discarded) to collect schemas and
 // functions, then separately parses `__all__` assignments and `from X import Y`
 // statements for wildcard-import support and delegate-target resolution respectively.
+// Returns the file's `IndexEntry` plus the top-level package names of any external
+// (non-first-party) imports it called in an unresolved, DataFrame-shaped way (see
+// `Linter::unresolved_dataframe_shaped_calls`) -- candidates for on-demand external
+// package tracing, resolved by `discover_external_package_candidates` in
+// `build_index_internal`. This file's own imports/module_aliases are exactly what's
+// needed to resolve each candidate's raw call-site name to a module, so it's done
+// here rather than requiring a second pass over the same file.
 pub(crate) fn index_file(
     path: &Path,
     project_root: &Path,
     config: &LinterConfig,
-) -> Option<IndexEntry> {
+) -> Option<(IndexEntry, Vec<String>)> {
     let source = fs::read_to_string(path).ok()?;
 
     let mut linter = Linter::new();
@@ -420,17 +426,97 @@ pub(crate) fn index_file(
         }
     }
 
-    Some(IndexEntry {
-        schemas,
-        schema_locations,
-        functions,
-        exports,
-        imports,
-        module_aliases,
-    })
+    // Resolve each recorded unresolved-call name (an attribute-call receiver or a
+    // bare-call callee) to the module it was imported from, then take that module's
+    // top-level package name -- matching the same dotted-import-binds-first-segment
+    // convention as the `Stmt::Import` handling above. A name that isn't a plain
+    // import at all (a local variable, a class instance, a closure) simply has no
+    // entry in `imports`/`module_aliases` and is silently dropped here, same as any
+    // other unresolvable name elsewhere in this module.
+    let external_package_candidates: Vec<String> = linter
+        .unresolved_dataframe_shaped_calls
+        .iter()
+        .filter_map(|call| {
+            let module_name = if call.is_attribute_call {
+                module_aliases.get(&call.name)
+            } else {
+                imports.get(&call.name)
+            }?;
+            module_name.split('.').next().map(str::to_string)
+        })
+        .collect();
+
+    Some((
+        IndexEntry {
+            schemas,
+            schema_locations,
+            functions,
+            exports,
+            imports,
+            module_aliases,
+        },
+        external_package_candidates,
+    ))
+}
+
+// Ceiling on how many AUTO-discovered candidate packages (see
+// `discover_external_package_candidates`) get traced by default. Never applies to
+// `trace_external_packages` -- an explicit force-include is never capped, since the
+// user opted into it deliberately and by name. Bounds the worst case for a project
+// whose first-party code happens to call many different external packages in
+// DataFrame-shaped ways, in the same spirit as the MAX_SQL_FILE_READS_PER_FILE /
+// MAX_SQL_FILE_BYTES guardrails in linter.rs's own `read_sql_file`.
+pub(crate) const MAX_AUTO_TRACED_PACKAGES: usize = 20;
+
+// Merge `trace_external_packages` (forced, uncapped, exempt from
+// `excluded_external_packages`), auto-discovered candidates (capped at
+// MAX_AUTO_TRACED_PACKAGES, alphabetically for determinism, filtered by
+// `excluded_external_packages` and by not already being a first-party module), into
+// the final list of package names to trace.
+pub(crate) fn resolve_traced_external_packages(
+    config: &LinterConfig,
+    discovered_candidates: std::collections::HashSet<String>,
+    project_root: &Path,
+    first_party_files: &HashMap<String, IndexEntry>,
+) -> Vec<String> {
+    let forced: std::collections::HashSet<String> = config
+        .trace_external_packages
+        .clone()
+        .unwrap_or_default()
+        .into_iter()
+        .collect();
+    let excluded: std::collections::HashSet<String> = config
+        .excluded_external_packages
+        .clone()
+        .unwrap_or_default()
+        .into_iter()
+        .collect();
+
+    let mut auto_candidates: Vec<String> = discovered_candidates
+        .into_iter()
+        .filter(|name| !forced.contains(name))
+        .filter(|name| !excluded.contains(name))
+        // A candidate that resolves to a first-party module isn't external at all --
+        // e.g. a plain `import utils` inside a `src/` layout project can look exactly
+        // like an unresolved external call before cross-file symbols are loaded.
+        .filter(|name| resolve_module_file(name, project_root, first_party_files).is_none())
+        .collect();
+    auto_candidates.sort();
+    auto_candidates.truncate(MAX_AUTO_TRACED_PACKAGES);
+
+    let mut result: Vec<String> = forced.into_iter().collect();
+    result.extend(auto_candidates);
+    result.sort();
+    result.dedup();
+    result
 }
 
-// Build a ProjectIndex by indexing every `.py` file under `project_root`.
+// Build a ProjectIndex by indexing every `.py` file under `project_root`. Two-phase:
+// first-party files are indexed on their own (collecting external-package candidates
+// along the way, see `index_file`), THEN the resolved external-package list is indexed
+// and merged in — see `resolve_traced_external_packages`. This ordering is required,
+// not just convenient: deciding which external packages are worth tracing depends on
+// having already seen every first-party file's own imports and DataFrame-shaped usage.
 pub(crate) fn build_index_internal(project_root: &Path) -> ProjectIndex {
     // Loaded once and threaded into every per-file `Linter`, so index-time inference
     // (this function) and check-time inference (`check_file`) agree on `sql_dialect` —
@@ -446,16 +532,30 @@ pub(crate) fn build_index_internal(project_root: &Path) -> ProjectIndex {
             .map(|s| s.to_string())
             .collect(),
     };
-    let mut py_files = collect_py_files(project_root, &resolved_excludes);
-    py_files.extend(collect_external_package_files(project_root, &config));
+    let py_files = collect_py_files(project_root, &resolved_excludes);
+
     let mut files = HashMap::new();
-    for file_path in py_files {
-        if let Some(entry) = index_file(&file_path, project_root, &config) {
+    let mut discovered_candidates: std::collections::HashSet<String> =
+        std::collections::HashSet::new();
+    for file_path in &py_files {
+        if let Some((entry, candidates)) = index_file(file_path, project_root, &config) {
+            discovered_candidates.extend(candidates);
+            if let Some(path_str) = file_path.to_str() {
+                files.insert(path_str.to_string(), entry);
+            }
+        }
+    }
+
+    let traced_packages =
+        resolve_traced_external_packages(&config, discovered_candidates, project_root, &files);
+    for file_path in collect_external_package_files(project_root, &traced_packages) {
+        if let Some((entry, _candidates)) = index_file(&file_path, project_root, &config) {
             if let Some(path_str) = file_path.to_str() {
                 files.insert(path_str.to_string(), entry);
             }
         }
     }
+
     let all_schemas = compute_all_schemas(&files);
     let all_schema_locations = compute_all_schema_locations(&files);
     resolve_param_schema_requires(&mut files, &all_schemas);
@@ -1131,6 +1231,97 @@ mod tests {
     use crate::config::LinterConfig;
     use std::path::Path;
 
+    #[test]
+    fn test_should_cap_auto_discovered_candidates_but_never_force_included_ones() {
+        // arrange: 25 auto-discovered candidates (more than MAX_AUTO_TRACED_PACKAGES),
+        // plus one package named in trace_external_packages that would sort dead last
+        // alphabetically -- it must still appear in the result, uncapped.
+        let config = LinterConfig {
+            enabled: None,
+            warnings: None,
+            sql_dialect: None,
+            trace_external_packages: Some(vec!["zzz_forced_pkg".to_string()]),
+            excluded_external_packages: None,
+            exclude: None,
+        };
+        let candidates: std::collections::HashSet<String> =
+            (0..25).map(|i| format!("pkg_{i:02}")).collect();
+        let dir = tempfile::tempdir().unwrap();
+
+        // act
+        let result =
+            resolve_traced_external_packages(&config, candidates, dir.path(), &HashMap::new());
+
+        // assert: forced package present regardless of the cap ...
+        assert!(result.contains(&"zzz_forced_pkg".to_string()));
+        // ... and the auto-discovered portion is capped at MAX_AUTO_TRACED_PACKAGES,
+        // keeping the alphabetically-first candidates for determinism.
+        let auto_portion: Vec<&String> = result
+            .iter()
+            .filter(|name| *name != "zzz_forced_pkg")
+            .collect();
+        assert_eq!(auto_portion.len(), MAX_AUTO_TRACED_PACKAGES);
+        assert!(auto_portion.contains(&&"pkg_00".to_string()));
+        assert!(!auto_portion.contains(&&format!("pkg_{:02}", 24)));
+    }
+
+    #[test]
+    fn test_should_respect_excluded_external_packages_in_resolve_traced_external_packages() {
+        // arrange
+        let config = LinterConfig {
+            enabled: None,
+            warnings: None,
+            sql_dialect: None,
+            trace_external_packages: None,
+            excluded_external_packages: Some(vec!["untrusted_pkg".to_string()]),
+            exclude: None,
+        };
+        let candidates: std::collections::HashSet<String> =
+            ["untrusted_pkg".to_string(), "fine_pkg".to_string()]
+                .into_iter()
+                .collect();
+        let dir = tempfile::tempdir().unwrap();
+
+        // act
+        let result =
+            resolve_traced_external_packages(&config, candidates, dir.path(), &HashMap::new());
+
+        // assert
+        assert!(!result.contains(&"untrusted_pkg".to_string()));
+        assert!(result.contains(&"fine_pkg".to_string()));
+    }
+
+    #[test]
+    fn test_should_never_treat_a_first_party_module_as_an_external_candidate() {
+        // arrange: "utils" resolves to a real first-party file already in the index --
+        // a bare `import utils` looking momentarily unresolved (before cross-file
+        // symbols are loaded) must never be treated as an external-tracing candidate.
+        let dir = tempfile::tempdir().unwrap();
+        let root = dir.path();
+        let config = LinterConfig::EMPTY;
+        let mut first_party_files = HashMap::new();
+        first_party_files.insert(
+            root.join("utils.py").to_str().unwrap().to_string(),
+            IndexEntry {
+                schemas: HashMap::new(),
+                schema_locations: HashMap::new(),
+                functions: HashMap::new(),
+                exports: Vec::new(),
+                imports: HashMap::new(),
+                module_aliases: HashMap::new(),
+            },
+        );
+        let candidates: std::collections::HashSet<String> =
+            ["utils".to_string()].into_iter().collect();
+
+        // act
+        let result =
+            resolve_traced_external_packages(&config, candidates, root, &first_party_files);
+
+        // assert
+        assert!(result.is_empty());
+    }
+
     #[test]
     fn test_should_not_hang_on_mutually_delegating_functions() {
         // arrange: f and g delegate to each other — a cycle in the delegate graph.