codemble 0.4.0__tar.gz → 0.5.0__tar.gz

{codemble-0.4.0 → codemble-0.5.0}/CHANGELOG.md

@@ -5,6 +5,82 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) · Versioning:
 
 ## [Unreleased]
 
+## [0.5.0] - 2026-07-20
+
+### Added
+- Large projects now show a staged loading screen instead of a frozen tab.
+  Parsing runs on a worker thread; the app polls `GET /api/picker/progress` and
+  names the stage it is in — discovering, parsing, resolving, checks, layout —
+  with a real file count while files are read, and a narrated sub-step
+  (resolving imports, resolving calls, building the galaxy map, composing the
+  project) once the parse moves past counting files. A failed parse reports
+  the parser's own message and offers an in-app retry; cancelling stops the
+  parse at the next file boundary. A realistic 1,000-file Python project
+  parses and renders an interactive galaxy end to end; the parse itself is
+  1.56x faster after removing a parser hotspot (below), and the one-pass check
+  index builds a full 1,000-region suite in under a second.
+- The over-cap prompt is actionable in-app: the busiest subdirectories are
+  buttons that navigate the picker, and a typed path field accepts any folder
+  inside your home directory. Outside-home paths are still refused.
+- A **Clear this project's progress** control on the star chart, behind a
+  confirmation, scoped to the open project only.
+- A non-interactive `codemble` run now prints the busiest-scope suggestions
+  with the scale refusal instead of a bare message.
+
+### Changed
+- The scale cap moved from 300 to 1,000 supported source files. LOD and
+  clustering remain Phase 2 work.
+- `POST /api/picker/select` returns `202 {"state": "parsing"}` instead of
+  blocking until the graph is ready.
+- `CODEMBLE_DATA_DIR` now relocates **everything** Codemble keeps under your home
+  directory — saved progress, the narration cache, and the `config` file — rather
+  than progress alone. All three resolve through one helper, so pointing the
+  variable somewhere else moves them together. Unset, the default is unchanged:
+  `~/.codemble`.
+- Language focus now also projects onto the **2D Map**, matching the galaxy: a
+  focused language keeps its boxes, rows, and edges while the others are dropped,
+  and the survivors keep their backend-computed coordinates. The projection
+  happens in the frontend and never re-lays-out or mutates the immutable graph,
+  so focus finally means the same thing on both layers.
+- The no-WebGL message now points the learner to the Map/Diagram layer, which
+  draws the same parser evidence as plain SVG and is one switch away, instead of
+  only stating that the galaxy needs WebGL.
+
+### Fixed
+- Check generation walked every graph edge up to four times per region, and
+  rebuilt the node lookup and the option pool per region — an O(regions × graph)
+  freeze at bind. One index is now built per bind in a single pass, measured
+  ~16x faster at 1,000 files. A committed golden fixture proves the generated
+  suites and answers are byte-identical for the Python and mixed fixtures.
+- `GET /api/graph` and `GET /api/map` re-hydrated progress and re-sorted every
+  node and edge on every request. The serialized document is cached and
+  invalidated on region light-up, Home selection, and project bind or reset —
+  measured ~25-26x faster warm.
+- The Python adapter resolved a node's owning module with an O(definitions ×
+  modules) scan; it now walks the node id's own dotted prefixes in O(depth),
+  a further 1.56x on total parse wall-clock. Output is byte-identical.
+- The test suite no longer reads the developer's `~/.codemble/config` or their
+  `ANTHROPIC_API_KEY` / `OPENAI_API_KEY`. Every server test that built an app
+  without an explicit study service inherited whatever the machine had
+  configured, so the same test could construct a live provider locally and none
+  in CI. The two tests that request `/explanation` would then make a real, billed
+  API call and cache the reply under the developer's home directory.
+- First-run onboarding assumed the learner was on the galaxy, but Easy — the
+  default audience — opens on the 2D Map. The coach-marks and the footer control
+  hint now key on the active layer, so a learner on the Map reads map guidance
+  ("click a box or row to study, switch tabs") instead of galaxy scroll/camera
+  controls that are not on screen.
+- The audience-mode gate and the first-run coach-marks no longer open as two
+  stacked modals on a genuine first run; the coach-marks wait until a mode has
+  been chosen, so the gate resolves first.
+- With no parser-recognisable entrypoint the "Change Home" control is not
+  rendered, so both Map tabs now state that reason instead of pointing at a
+  button that is not there. The has-candidates-but-unselected copy is unchanged.
+- A project bind slow enough to outlast a parse cancellation could commit after
+  the picker had already been reset, resurrecting the just-released project or
+  clobbering the next selection. The commit now re-checks cancellation under the
+  same lock that performs it, so a stale bind can never rebind a released project.
+
 ## [0.4.0] - 2026-07-20
 
 The galaxy read as flat spheres on a plain background, connections were hard to

{codemble-0.4.0 → codemble-0.5.0}/CLAUDE.md

@@ -194,9 +194,43 @@ switching, then the "living cosmos" visual overhaul with a 2D Map layer, then
 the approved Non-Goal and binding relaxations. Phase A (the narration/mode/
 connections surface and project switching) and Phase B (M12: call-depth
 orbits, the 2D Map layer, and the living-cosmos visual overhaul) have both
-since shipped. Phase C (~1,000-file scale with staged parse progress) has not
-started — there is still no threaded parse or parse-progress screen, and the
-scale cap remains ~300 files.
+since shipped, and were released together as **v0.4.0** (tag `v0.4.0`, published
+to PyPI, verified end to end from a clean `uvx codemble==0.4.0` install: the
+wheel's SPA bundle is byte-identical to the tag, all 27 regions draw unclipped,
+and the galaxy renders deep space with no console errors). Phase C (M13:
+~1,000-file scale with staged parse progress) has since shipped from that
+plan: parsing now runs on a worker thread behind a `202`-accepted picker
+select and a polled `GET /api/picker/progress` through five honest stages,
+cancellation is checked between files and a crashed worker reports as an
+in-app error rather than a hung server, the scale cap moved 300 → 1,000 with
+the over-cap prompt offering clickable busiest scopes plus a home-jailed typed
+path, a one-pass check index replaced the per-region edge scans
+(byte-identical suites, pinned by a golden fixture before the refactor),
+`/api/graph` and `/api/map` responses are now cached with invalidation on
+light-up, Home change, and binding, and a Clear this project's progress
+control was added to the star chart. A dedicated verification pass at a
+realistic ~1,000-file project then found the `resolving` stage — the slowest
+one — showed no moving signal for most of the wait; the fix narrates its real
+sub-steps instead of leaving the screen static, and a parser hotspot found
+alongside it (an O(definitions × modules) module-resolution scan in the Python
+adapter) was fixed too, together taking real parse wall-clock on a 1,000-file
+Python project from roughly 11.5s to roughly 7.5s with byte-identical output.
+Suite hermeticity was also closed on the read side: `CODEMBLE_DATA_DIR` now
+relocates the narration cache and the `config` file as well as saved progress
+through one `codemble/paths.py` helper, and the test suite clears every
+provider variable `from_environment` reads, so a server test can no longer make
+a real billed API call against a developer's exported key. A pre-release
+re-audit then closed a cluster of first-run gaps that converged on the
+Easy-default learner (who lands on the 2D Map): the coach-marks and footer now
+teach the layer the learner is actually on, the audience gate and coach-marks
+no longer stack as two modals, the no-entrypoint Map tabs stop pointing at a
+Change Home button that isn't shown, language focus now filters the Map as a
+frontend projection, and a parse `bind` that outlasts a cancel can no longer
+rebind a released project. Phase C plus that gap-fix wave shipped as **v0.5.0**;
+the parse work collided with an independent implementation of the same three
+foundational commits on `main`, reconciled by taking the branch's verified
+superset while preserving main's unique `CODEMBLE_DATA_DIR`/config-isolation
+fix, which lived in files the branch never touched.
 
 ### M0 — Repo, docs & website scaffold ✅ (2026-07-19)
 - [x] Root: README, LICENSE (Apache-2.0), CoC, SECURITY, CONTRIBUTING,
@@ -344,6 +378,24 @@ hash file content, never coordinates, so the orbit relayout did not re-dim any
 region; reduced motion always yields the finished lit state with zero
 animation.
 
+### M13 — Galaxy UX Phase C: scale ✅ (2026-07-20)
+- [x] Threaded parse behind `202` select, `GET /api/picker/progress`, and a
+      staged loading screen with real file counts
+- [x] Cancellation checked between files; a crashed worker becomes an error
+      state, never a hung server
+- [x] Scale cap 300 → 1,000; clickable busiest scopes plus a jailed path field;
+      suggestions in the non-TTY CLI refusal
+- [x] One per-bind check index replacing the per-region edge scans, pinned by a
+      golden suite fixture
+- [x] Cached `/api/graph` and `/api/map` documents invalidated on light-up,
+      Home, and binding
+- [x] Terminal stage lines for `codemble <path>`; reset-progress control
+
+**Acceptance:** a ~1,000-file project parses with live progress and reaches an
+interactive galaxy; re-fetching the graph does not re-sort the world; the scale
+prompt is actionable entirely in-app; generated check suites are byte-identical
+to before the index change.
+
 ## Decision Log **[AGENT-MAINTAINED — append only]**
 
 | Date | Decision | Why |
@@ -406,6 +458,10 @@ animation.
 | 2026-07-20 | Bloom resolution is capped with `composer.setPixelRatio(1)`, not the `UnrealBloomPass` constructor | `EffectComposer.setSize` forwards the canvas size to every pass on resize, overwriting the constructor's `resolution`. The pixel ratio is the cap that survives |
 | 2026-07-20 | **Corrects the row above**: bloom is capped by wrapping the bloom pass's own `setSize`, and the composer keeps the renderer's pixel ratio | The pixel ratio *did* cap bloom, but `EffectComposer.setSize` multiplies it into `renderTarget1/2` and every pass, so the whole scene rendered at 1x and upscaled — measured 1280x611 scene passes on a 2560x1221 buffer at dpr 2. Wrapping the one pass caps the one expensive thing: scene now 2560x1221, bloom mip 0 800x382 (1280x611 uncapped), `?benchmark` at 951 nodes unchanged at 928.8 → 961.5 fps median |
 | 2026-07-20 | **Corrects "binding is one-shot"** (2026-07-19 picker row): binding is one-*at-a-time*. `serve_project` attaches `PickerConfig(browse_root=Path.home())` too, so a `codemble <path>` run also exposes the picker endpoints after a reset, and browse then enumerates non-hidden directories under `$HOME` | The Switch project control has to work without a process restart, which is what that config is for — but the earlier row still claimed a permanent 409 for the path-opened flow, and this file is the source of truth. The home jail and the Host-header allowlist are unchanged; only the "one-shot" claim was false. An app built with no `PickerConfig` at all remains genuinely one-shot and refuses reset |
+| 2026-07-20 | `CODEMBLE_DATA_DIR` owns every home-directory path — progress, the narration cache, and the `config` file — through one `codemble/paths.py` helper; the test suite additionally clears every provider variable `StudyService.from_environment` reads | The variable redirected progress only, while `StudyService` hardcoded `Path.home()` for the other two, so `create_app`'s default study service read the developer's real config and `ANTHROPIC_API_KEY`. Two server tests GET `/explanation` and assert only that a `status` key came back — true of `no_key`, `ready`, and `error` alike — so on a machine with a key they made a real billed API call and cached the reply under the developer's home while still passing. Redirecting the directory does nothing about the process environment, which is why the suite must clear the keys as well. No new variable is introduced, the default stays `~/.codemble`, and explicit `environ`/`config_path`/`cache_root` arguments still win over both channels |
+| 2026-07-20 | Progress reporting is a thread-scoped per-file hook (`note_file_parsed`) bound by `ProjectParser`, not a new `LanguageAdapter` parameter | The public adapter seam must stay unchanged for Phase 2 languages; one hook site per adapter also gives cancellation its exact "between files" meaning |
+| 2026-07-20 | Phase C adds `DELETE /api/progress`, the `CLEAR_PROGRESS` session event, and a `clearProgress` adapter method beyond the shared contract's Phase C rows | The contract's Phase C rows covered parse progress only, while the no-reset-progress-control gap is mapped to Phase C by the spec; recorded here rather than silently widened |
+| 2026-07-20 | Generated check suites are pinned by a committed golden fixture before any performance work touches `checks/service.py` | The Correctness Contract makes suite drift top-severity, and a refactor that changes an answer is invisible without a byte-level pin |
 
 ## Non-Goals — do NOT build (point here when asked)
 
@@ -446,7 +502,7 @@ animation.
 
 ## Edge cases & limits
 
-- >~300 supported source files → prompt to scope to a subdirectory (LOD arrives Phase 2)
+- >~1,000 supported source files → prompt to scope to a subdirectory (LOD arrives Phase 2)
 - No clear entrypoint → ranked candidates; user picks Home
 - Syntax errors / partial parses → parse what you can, flag the rest, never crash
 - Missing/invalid key → galaxy + structure + checks work; explanations show "add your key"

{codemble-0.4.0 → codemble-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codemble
-Version: 0.4.0
+Version: 0.5.0
 Summary: A learning game that turns the code AI wrote for you into a galaxy you light up by understanding it.
 Project-URL: Homepage, https://udhawan97.github.io/Codemble/
 Project-URL: Repository, https://github.com/udhawan97/Codemble
@@ -164,6 +164,17 @@ of your own code becomes a sky you understand.
   and this one has no model configured at all.
 </sub></p>
 
+<p align="center">
+  <img src="https://github.com/udhawan97/Codemble/raw/main/docs-site/public/shots/loading.png" alt="Codemble's staged loading screen mapping a large project, with five named stages — finding source files, reading each file, connecting imports and calls, building graph-only checks, placing your galaxy — a live count reading 13 of 900 files, and a cancel button" width="900">
+</p>
+
+<p align="center"><sub>
+  Large projects (up to roughly 1,000 source files) parse in the background with
+  visible staged progress — a real file count while reading, named steps while
+  resolving — so a big project never looks like a frozen tab. Cancel any time to
+  pick another.
+</sub></p>
+
 ## Two layers over one graph
 
 Codemble draws the same parser evidence two ways, switchable in the header. The
@@ -259,9 +270,9 @@ game; only the optional prose narration is unavailable.
 
 - **Supported source:** Python 3.11+, JavaScript/JSX, TypeScript/TSX, and mixed
   projects. Unsupported languages stay outside the graph rather than being guessed.
-- **Scale:** above roughly 300 supported source files, choose a subdirectory —
-  the in-app picker prompts for the scope, or pass
-  `codemble --path ./project/subdirectory`.
+- **Scale:** above roughly 1,000 supported source files, choose a subdirectory —
+  the in-app picker offers the busiest scopes as buttons and accepts a typed
+  path, or pass `codemble --path ./project/subdirectory`.
 - **Ambiguous Home:** choose a parser-ranked entrypoint in the app or pass
   `--entrypoint NODE_ID`.
 - **Broken source:** syntax errors remain visible; Codemble maps safe partial

{codemble-0.4.0 → codemble-0.5.0}/README.md

@@ -132,6 +132,17 @@ of your own code becomes a sky you understand.
   and this one has no model configured at all.
 </sub></p>
 
+<p align="center">
+  <img src="https://github.com/udhawan97/Codemble/raw/main/docs-site/public/shots/loading.png" alt="Codemble's staged loading screen mapping a large project, with five named stages — finding source files, reading each file, connecting imports and calls, building graph-only checks, placing your galaxy — a live count reading 13 of 900 files, and a cancel button" width="900">
+</p>
+
+<p align="center"><sub>
+  Large projects (up to roughly 1,000 source files) parse in the background with
+  visible staged progress — a real file count while reading, named steps while
+  resolving — so a big project never looks like a frozen tab. Cancel any time to
+  pick another.
+</sub></p>
+
 ## Two layers over one graph
 
 Codemble draws the same parser evidence two ways, switchable in the header. The
@@ -227,9 +238,9 @@ game; only the optional prose narration is unavailable.
 
 - **Supported source:** Python 3.11+, JavaScript/JSX, TypeScript/TSX, and mixed
   projects. Unsupported languages stay outside the graph rather than being guessed.
-- **Scale:** above roughly 300 supported source files, choose a subdirectory —
-  the in-app picker prompts for the scope, or pass
-  `codemble --path ./project/subdirectory`.
+- **Scale:** above roughly 1,000 supported source files, choose a subdirectory —
+  the in-app picker offers the busiest scopes as buttons and accepts a typed
+  path, or pass `codemble --path ./project/subdirectory`.
 - **Ambiguous Home:** choose a parser-ranked entrypoint in the app or pass
   `--entrypoint NODE_ID`.
 - **Broken source:** syntax errors remain visible; Codemble maps safe partial

{codemble-0.4.0 → codemble-0.5.0}/TESTING.md

@@ -8,8 +8,8 @@ happened.
 
 ## Protect your project
 
-- Use a local project with at most 300 supported source files, or choose a smaller
-  subdirectory when Codemble prompts (`--path` from the CLI).
+- Use a local project with at most 1,000 supported source files, or choose a
+  smaller subdirectory when Codemble prompts (`--path` from the CLI).
 - Do not paste source code, API keys, credentials, or private project names into
   feedback. Codemble itself runs locally and has no telemetry.
 - An LLM key is optional. The galaxy, source, Lens, checks, and lighting work

{codemble-0.4.0 → codemble-0.5.0}/codemble/__init__.py

@@ -1,4 +1,4 @@
 """Codemble — a learning game that turns the code AI wrote for you into a galaxy
 you light up by understanding it. See CLAUDE.md for the agent brief and roadmap."""
 
-__version__ = "0.4.0"
+__version__ = "0.5.0"

codemble-0.5.0/codemble/adapters/parse_progress.py

@@ -0,0 +1,99 @@
+"""Per-file parse progress and cancellation, without widening the adapter seam.
+
+``LanguageAdapter`` keeps its four public methods exactly as they are.  A
+parse that wants progress binds a hook for its own thread with
+``reporting_files``; each adapter's private per-file helper calls
+``note_file_parsed`` once per source file it finishes reading.  That single
+call site is also the only cancellation check point, so "between files" means
+exactly one place in each adapter.
+"""
+
+from __future__ import annotations
+
+import threading
+from collections.abc import Callable, Iterator
+from contextlib import contextmanager
+from typing import Protocol, runtime_checkable
+
+_local = threading.local()
+
+
+class ParseCancelled(Exception):
+    """A running parse was cancelled; no graph will be produced."""
+
+
+@runtime_checkable
+class ParseProgress(Protocol):
+    """The reporting surface ``ProjectParser`` writes one parse's state to."""
+
+    def stage(self, stage: str) -> None:
+        """Report the stage now running."""
+
+    def files_total(self, total: int) -> None:
+        """Report how many source files this parse will read."""
+
+    def file_parsed(self) -> None:
+        """Report one finished file; raise ``ParseCancelled`` to stop."""
+
+    def detail(self, detail: str) -> None:
+        """Report the sub-step now running within a stage.
+
+        The ``resolving`` stage is one label over several seconds of cross-file
+        work; a sub-step keeps the screen honestly changing instead of frozen.
+        It is advisory only: unlike ``file_parsed`` it never cancels, so a
+        reporter that ignores it loses nothing but the finer copy.
+        """
+
+
+@contextmanager
+def reporting_files(on_file: Callable[[], None] | None) -> Iterator[None]:
+    """Bind ``on_file`` for this thread for the duration of one parse."""
+
+    previous = getattr(_local, "on_file", None)
+    _local.on_file = on_file
+    try:
+        yield
+    finally:
+        _local.on_file = previous
+
+
+def note_file_parsed() -> None:
+    """Report one finished source file; a no-op when nobody is listening."""
+
+    on_file = getattr(_local, "on_file", None)
+    if on_file is not None:
+        on_file()
+
+
+@contextmanager
+def reporting_detail(on_detail: Callable[[str], None] | None) -> Iterator[None]:
+    """Bind ``on_detail`` for this thread for the duration of one parse.
+
+    Separate from ``reporting_files`` because resolving sub-steps outlive the
+    file-reading loop: they run through cross-file resolution and composition.
+    """
+
+    previous = getattr(_local, "on_detail", None)
+    _local.on_detail = on_detail
+    try:
+        yield
+    finally:
+        _local.on_detail = previous
+
+
+def note_detail(detail: str) -> None:
+    """Report the resolving sub-step now running; a no-op with no listener."""
+
+    on_detail = getattr(_local, "on_detail", None)
+    if on_detail is not None:
+        on_detail(detail)
+
+
+__all__ = [
+    "ParseCancelled",
+    "ParseProgress",
+    "note_detail",
+    "note_file_parsed",
+    "reporting_detail",
+    "reporting_files",
+]

{codemble-0.4.0 → codemble-0.5.0}/codemble/adapters/project.py

@@ -13,6 +13,12 @@ from codemble.adapters.discovery import (
     SourceOwnership,
     discover_project_sources,
 )
+from codemble.adapters.parse_progress import (
+    ParseProgress,
+    note_detail,
+    reporting_detail,
+    reporting_files,
+)
 from codemble.graph.finalize import GraphFinalizationError, finalize_graph
 
 
@@ -26,10 +32,14 @@ class ProjectScaleError(ProjectParseError):
     def __init__(self, intake: ProjectIntake, scale_cap: int) -> None:
         self.intake = intake
         self.scale_cap = scale_cap
+        scopes = ", ".join(
+            f"{directory} ({count})" for directory, count in intake.scope_counts()[:6]
+        )
+        suggestion = f" Busiest scopes: {scopes}." if scopes else ""
         super().__init__(
             f"found {len(intake.files)} supported source files; Codemble is capped at "
             f"{scale_cap}. Re-run with `codemble --path PATH` to choose a project "
-            "subdirectory."
+            f"subdirectory.{suggestion}"
         )
 
 
@@ -85,7 +95,9 @@ class ProjectParser:
 
         return tuple(adapter.language for adapter in self._adapters)
 
-    scale_cap = 300
+    # Raised from 300 with the Phase C threaded parse and staged loading
+    # screen; LOD and clustering remain Phase 2.
+    scale_cap = 1000
 
     def intake(self, path: Path, *, explicit: bool = False) -> ProjectIntake:
         """Resolve one project scope and every adapter's owned files."""
@@ -133,30 +145,52 @@ class ProjectParser:
         *,
         entrypoint: str | None = None,
         explicit: bool = False,
+        progress: ParseProgress | None = None,
     ) -> Graph:
         """Parse every detected language and return one deterministic graph."""
 
-        intake = (
-            source
-            if isinstance(source, ProjectIntake)
-            else self.intake(source, explicit=explicit)
-        )
+        if isinstance(source, ProjectIntake):
+            intake = source
+        else:
+            if progress is not None:
+                progress.stage("discovering")
+            intake = self.intake(source, explicit=explicit)
+        owned = {
+            adapter.language: intake._files_for(adapter.language)
+            for adapter in self._adapters
+        }
+        if progress is not None:
+            # The counter totals the files adapters will actually read, which
+            # is what ``note_file_parsed`` counts.  ``intake.files`` is the
+            # deduplicated union and would drift if two adapters ever shared
+            # an extension.
+            progress.files_total(sum(len(files) for files in owned.values()))
+            progress.stage("parsing")
         graphs: list[Graph] = []
-        for adapter in self._adapters:
-            files = intake._files_for(adapter.language)
-            if not files:
-                continue
-            try:
-                graphs.append(adapter.parse_files(intake.root, files))
-            except AdapterParseError as error:
-                raise ProjectParseError(str(error)) from error
-        return _compose_graphs(tuple(graphs), intake.root, entrypoint)
+        on_file = progress.file_parsed if progress is not None else None
+        # ``detail`` outlives the file-read loop: the adapters narrate their
+        # cross-file passes and composition narrates the merge, all under the
+        # single ``resolving`` stage the design spec fixes.
+        on_detail = getattr(progress, "detail", None) if progress is not None else None
+        with reporting_detail(on_detail), reporting_files(on_file):
+            for adapter in self._adapters:
+                files = owned[adapter.language]
+                if not files:
+                    continue
+                try:
+                    graphs.append(adapter.parse_files(intake.root, files))
+                except AdapterParseError as error:
+                    raise ProjectParseError(str(error)) from error
+            if progress is not None:
+                progress.stage("resolving")
+            return _compose_graphs(tuple(graphs), intake.root, entrypoint)
 
 def _compose_graphs(
     graphs: tuple[Graph, ...],
     project_root: Path,
     entrypoint: str | None,
 ) -> Graph:
+    note_detail("Composing your project")
     nodes: list[Node] = []
     edges = []
     annotations = []

{codemble-0.4.0 → codemble-0.5.0}/codemble/adapters/python_ast.py

@@ -12,6 +12,7 @@ from pathlib import Path
 
 from codemble.adapters.base import AdapterParseError, ConceptAnnotation, Edge, Graph, Node
 from codemble.adapters.discovery import SourceDiscoveryError, discover_source_files
+from codemble.adapters.parse_progress import note_detail, note_file_parsed
 from codemble.graph.finalize import GraphFinalizationError, finalize_graph
 
 _APP_FACTORIES = {"FastAPI", "Flask", "Typer"}
@@ -350,6 +351,10 @@ class PythonAstAdapter:
         module_bindings: dict[str, list[_ImportBinding]] = defaultdict(list)
         scope_bindings: dict[str, list[_ImportBinding]] = defaultdict(list)
 
+        # The stage counter is full once the last file is read; from here the
+        # resolving stage does its cross-file work, so it narrates the real
+        # passes rather than sitting on one frozen label.
+        note_detail("Resolving imports")
         for parsed in parsed_files:
             if parsed.tree is None:
                 continue
@@ -371,6 +376,7 @@ class PythonAstAdapter:
                 _, bindings = _resolve_import(parsed, syntax, modules, node_by_id)
                 scope_bindings[definition.node_id].extend(bindings)
 
+        note_detail("Resolving calls")
         call_edges: list[Edge] = []
         children_by_parent: dict[str, list[Node]] = defaultdict(list)
         for definition in definitions:
@@ -401,6 +407,7 @@ class PythonAstAdapter:
                     )
                 )
 
+        note_detail("Reading language concepts")
         all_edges = [*import_edges, *call_edges]
         annotations: list[ConceptAnnotation] = []
         for parsed in parsed_files:
@@ -430,6 +437,7 @@ class PythonAstAdapter:
                 parsed.relative_path for parsed in parsed_files if parsed.tree is None
             ),
         )
+        note_detail("Building the galaxy map")
         try:
             return finalize_graph(draft, entrypoint=entrypoint)
         except GraphFinalizationError as error:
@@ -488,7 +496,7 @@ def _parse_file(path: Path, project_root: Path) -> _ParsedFile:
     except (SyntaxError, UnicodeDecodeError):
         source = raw.decode("utf-8", errors="replace")
         tree = None
-    return _ParsedFile(
+    parsed = _ParsedFile(
         path=path,
         relative_path=relative.as_posix(),
         module=_module_name(relative, project_root),
@@ -496,6 +504,8 @@ def _parse_file(path: Path, project_root: Path) -> _ParsedFile:
         digest=digest,
         tree=tree,
     )
+    note_file_parsed()
+    return parsed
 
 
 def _module_name(relative: Path, project_root: Path) -> str:
@@ -757,10 +767,18 @@ def _call_leaf_name(expression: ast.expr) -> str | None:
 
 
 def _module_from_node_id(node_id: str, modules: set[str]) -> str:
-    return max(
-        (module for module in modules if node_id == module or node_id.startswith(f"{module}.")),
-        key=len,
-    )
+    # Longest module that is a dotted prefix of ``node_id``. Walking the id's
+    # own prefixes (longest first) is byte-identical to scanning every module,
+    # but O(id depth) set lookups instead of O(modules) startswith calls -- the
+    # difference between 47M string scans and a few thousand on a 1k-file parse.
+    candidate = node_id
+    while True:
+        if candidate in modules:
+            return candidate
+        cut = candidate.rfind(".")
+        if cut == -1:
+            raise ValueError(f"no module owns node id: {node_id}")
+        candidate = candidate[:cut]
 
 
 __all__ = ["PythonAstAdapter", "PythonParseError"]

{codemble-0.4.0 → codemble-0.5.0}/codemble/adapters/typescript_tree_sitter.py

@@ -20,6 +20,7 @@ from codemble.adapters.base import (
     Node,
 )
 from codemble.adapters.discovery import SourceDiscoveryError, discover_source_files
+from codemble.adapters.parse_progress import note_file_parsed
 from codemble.graph.finalize import GraphFinalizationError, finalize_graph
 
 _JAVASCRIPT_EXTENSIONS = frozenset({".js", ".jsx", ".mjs", ".cjs"})
@@ -344,7 +345,7 @@ def _parse_file(path: Path, project_root: Path) -> _ParsedFile:
     relative = path.relative_to(project_root).as_posix()
     language = "javascript" if path.suffix.lower() in _JAVASCRIPT_EXTENSIONS else "typescript"
     parser = Parser(_language_for(path.suffix.lower()))
-    return _ParsedFile(
+    parsed = _ParsedFile(
         path=path,
         project_root=project_root,
         relative_path=relative,
@@ -355,6 +356,8 @@ def _parse_file(path: Path, project_root: Path) -> _ParsedFile:
         digest=hashlib.sha256(raw).hexdigest(),
         tree=parser.parse(raw),
     )
+    note_file_parsed()
+    return parsed
 
 
 def _language_for(extension: str) -> Language: