stackthread 0.1.0.dev2__tar.gz → 0.1.0.dev3__tar.gz

stackthread-0.1.0.dev3/PKG-INFO

@@ -0,0 +1,532 @@
+Metadata-Version: 2.5
+Name: stackthread
+Version: 0.1.0.dev3
+Summary: Click any element in your running web app and follow it through your code: component, handler, request, endpoint, database.
+Project-URL: Vite plugin, https://www.npmjs.com/package/@stackthread/vite
+Author: The stackthread authors
+License-Expression: MIT
+License-File: LICENSE
+Keywords: code-navigation,developer-tools,django,fastapi,react,static-analysis,vite
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Web Environment
+Classifier: Framework :: Django
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development
+Classifier: Topic :: Software Development :: Debuggers
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# stackthread
+
+[![PyPI](https://img.shields.io/pypi/v/stackthread?include_prereleases&label=PyPI)](https://pypi.org/project/stackthread/)
+[![npm](https://img.shields.io/npm/v/@stackthread/vite/next?label=npm%20%40stackthread%2Fvite)](https://www.npmjs.com/package/@stackthread/vite)
+[![Python](https://img.shields.io/pypi/pyversions/stackthread)](https://pypi.org/project/stackthread/)
+[![Licence: MIT](https://img.shields.io/badge/licence-MIT-blue)](#licence)
+
+**Click any element in your running web app and follow it through your code**: the component that drew it, the
+handler it runs, the request it sends, the endpoint that answers, the functions behind it and the tables they
+touch. Open any of them at the right line, edit, save. The map rebuilds as you type.
+
+stackthread is for the moment every developer knows: you are looking at a button in the browser and you need the
+code behind it, all of it, across the frontend and the backend.
+
+```
+<button> Mark as paid                    OrderPage.tsx:22
+  payNow()            onClick handler     OrderPage.tsx:12
+  shop.pay()          used by payNow      shop.ts:25
+POST /api/orders/{}/pay/                  sent by shop.pay
+POST /api/orders/{pk}/pay/   drf endpoint urls.py:8
+  OrderViewSet.pay()                      views.py:27
+    services.pay()                        services.py:28
+Order (shop_orders)  written by pay       models.py:13
+```
+
+- **Works on your machine only.** No account, no telemetry, no network calls beyond `127.0.0.1`.
+- **Reads code, never runs it.** Your backend does not need to be running, and nothing is imported or executed.
+- **Honest about certainty.** Every link is marked *exact*, *inferred* or *heuristic*.
+- **No Python dependencies.** Node.js reads the frontend with a bundled parser.
+
+> **Status: early preview (0.1 development releases).** Install with `pip install --pre stackthread` and
+> `npm install --save-dev @stackthread/vite@next` until 0.1.0 is released.
+
+## Contents
+
+- [Supported stacks](#supported-stacks)
+- [Install](#install)
+- [Quick start](#quick-start)
+- [Using the workbench](#using-the-workbench)
+- [Command line](#command-line)
+- [Configuration](#configuration)
+- [Project layouts](#project-layouts)
+- [Teaching stackthread your domain](#teaching-stackthread-your-domain)
+- [Troubleshooting](#troubleshooting)
+- [Security and privacy](#security-and-privacy)
+- [Limitations](#limitations)
+
+## Supported stacks
+
+| Layer | Supported |
+|---|---|
+| Backend | FastAPI, Django (URLconf, function and class-based views), Django REST framework, django-ninja |
+| Data | Django ORM, SQLAlchemy 2, SQLModel, Flask-SQLAlchemy queries, MongoDB collections (heuristic) |
+| Frontend | React (JSX and TSX) on Vite 5 or later |
+| Routing | React Router (route objects and `<Routes>`), TanStack Router (code-based and file-based) |
+| HTTP | `fetch`, axios and axios instances, ky, ofetch, openapi-fetch, and your own request wrappers |
+
+What each framework detector understands, in detail:
+
+- **FastAPI:** `FastAPI()` and `APIRouter(prefix=...)` wherever they are created; `get`, `post`, `put`, `patch`,
+  `delete`, `options`, `head` and `api_route`; `include_router` nested to any depth and `add_api_route`;
+  `Depends` and `Security`, shown as "injected into"; `response_model=` and parameter and return types.
+- **Django:** `ROOT_URLCONF` followed through `include()`; `path()` with converters (`<int:pk>` becomes `{pk}`)
+  and `re_path()` with named groups; function views and class-based views through `as_view()`, one endpoint per
+  HTTP method, including the generic views.
+- **Django REST framework:** `SimpleRouter` and `DefaultRouter`; `ViewSet`, `GenericViewSet`, `ModelViewSet`,
+  `ReadOnlyModelViewSet` and the mixins; `@action(detail=..., methods=[...], url_path=...)` and `lookup_field`;
+  `APIView` methods and `@api_view`.
+- **django-ninja:** `NinjaAPI()` mounted with `api.urls`; `Router()` with `add_router(prefix, router)`, as an
+  object or an import string; route decorators and `api_operation`.
+- **Data:** model classes and their tables, and which functions read or write them: `Model.objects` chains,
+  `get_object_or_404`, `save()` and `delete()`; SQLAlchemy `select`, `insert`, `update`, `delete` and session
+  calls; Flask-SQLAlchemy `Model.query`; MongoDB collection methods.
+- **React:** components (functions, `memo`, `forwardRef`, higher-order components, classes), hooks, props,
+  `useState`, `useReducer`, `useMemo`, `useCallback`, `useRef`, context; every JSX element with its handlers,
+  shown values, render condition and the list it is rendered from. A submit button follows its form's `onSubmit`;
+  a prop is followed to the parent that passed it.
+- **HTTP:** literal, template and concatenated URLs; `axios.create({ baseURL })` instances defined anywhere;
+  typed `openapi-fetch` paths; wrapper functions that pass their URL on; import aliases from `tsconfig.json`,
+  `jsconfig.json` and `vite.config.*`; Vite `server.proxy` keys tell stackthread which requests reach the backend.
+
+Python is parsed with the standard library `ast` (Python 3.14 syntax accepted on older interpreters).
+JavaScript and TypeScript are parsed with a bundled Babel, so your project's compiler versions do not matter.
+
+## Install
+
+| You need | Version | Why |
+|---|---|---|
+| Python | 3.11 or later | runs the indexer and the local server |
+| Node.js | 22.18 or later | reads JavaScript and TypeScript files |
+| Vite | 5 or later | makes the elements of your app clickable |
+
+Install the Python package where your backend's tools live: in the backend's virtual environment, as a dev
+dependency, or globally.
+
+```sh
+pip install --pre stackthread          # or: uv add --dev --prerelease allow stackthread
+                                       # or: pipx install --pip-args=--pre stackthread
+npm install --save-dev @stackthread/vite@next
+```
+
+## Quick start
+
+**1. Add the Vite plugin**, before the React plugin:
+
+```ts
+// vite.config.ts
+import stackthread from '@stackthread/vite'
+import react from '@vitejs/plugin-react'
+import { defineConfig } from 'vite'
+
+export default defineConfig({
+  plugins: [stackthread(), react()],
+})
+```
+
+In `vite dev` it marks every JSX element with its source location and serves the workbench. In `vite build` it
+does nothing: production bundles contain nothing from stackthread.
+
+**2. Describe the project.** From the folder that holds both your backend and your frontend:
+
+```sh
+stackthread init
+```
+
+`init` finds Python backends (by `manage.py`, or a `pyproject.toml`, `requirements.txt`, `setup.cfg` or
+`Pipfile` that names a supported framework) and React-on-Vite frontends (a `package.json` with `react` and
+`vite`), up to four folders deep, and writes `stackthread.toml`:
+
+```toml
+[project]
+name = "shop"
+
+[[sources]]
+name = "api"
+lang = "python"
+root = "backend"
+exclude = ["**/migrations/**"]
+
+[[sources]]
+name = "web"
+lang = "typescript"
+root = "web"
+include = ["src/**/*"]
+```
+
+The same settings can live under `[tool.stackthread]` in `pyproject.toml`.
+
+**3. Run it**, next to your usual dev servers:
+
+```sh
+stackthread serve      # reads the code, watches it, serves the workbench on 127.0.0.1
+npm run dev            # your app, as usual
+```
+
+**4. Open the workbench** at `http://localhost:5173/__stackthread/` (use your Vite port), press **Inspect** and
+click anything in your app.
+
+## Using the workbench
+
+The workbench shows your running app on the left and its code on the right. Its top bar switches between
+**Inspector**, **Map** and **Findings**.
+
+### Inspector
+
+Turn on **Inspect** (or press **Alt+Shift+S**) and click a button, a field, a heading or any text. The panel
+shows its thread in five layers:
+
+| Layer | What it lists |
+|---|---|
+| On screen | the element and the components that render it |
+| In the browser | the handlers and state it uses, and the functions they call |
+| Over the wire | the requests those functions send |
+| On the server | the endpoint that answers, and the functions and dependencies behind it |
+| Data | the models and tables those functions read or write |
+
+Each row names a real definition with its file and line, and why it is there ("onClick handler", "sent by
+shop.pay", "written by pay"). Rows marked *reads* are data the element shows; the rest are actions. From the panel:
+
+- **Show it in the page** highlights the element again in the app.
+- **Show in the map** opens the focus view around a row.
+- **Copy** puts the whole thread on the clipboard as text, for a ticket, a review or an AI assistant.
+
+The left sidebar lists the **pages** found in your router and the **views** you defined (see
+[Configuration](#configuration)); click one to open it in the app.
+
+You can also inspect without the workbench: inside your app, **Alt+Shift+S** turns on inspect mode and a panel
+docks on the right side of the page.
+
+### Code pane
+
+Every row opens its file at the line in the code pane, with syntax highlighting for Python, TypeScript and
+JavaScript. **Ctrl+S** (or **Cmd+S**) saves. stackthread:
+
+- refuses the save if the file changed on disk since you opened it, so your editor's work is never overwritten;
+- never writes secrets or lock files (`.env*`, `*.pem`, `*.key`, lock files, `.git/` and more);
+- can run your formatter after each save (`server.format`).
+
+**Show the thread of the definition at this line** answers "who reaches this function?" from any line. The
+button named after your editor opens the file there instead (VS Code, Cursor, Windsurf, Zed, IntelliJ IDEA,
+PyCharm or WebStorm; set `server.editor`).
+
+### Map
+
+| Tab | What it shows |
+|---|---|
+| Overview | counts, and every page, component, endpoint and model grouped by source |
+| Focus | one definition with everything it calls and everything that reaches it |
+| Endpoints | every endpoint with its method, handler and framework, and where the frontend calls it |
+| Pages | every route, the component it renders and where the route is defined |
+
+### Search
+
+**Ctrl+K** (or **Cmd+K**) finds components, functions, endpoints, pages and files. **Enter** opens the result in
+the code pane, **Alt+Enter** in the map.
+
+### Findings
+
+The **Findings** tab lists what `stackthread check` reports: requests that reach no endpoint, imports that break
+your layer rules, files that could not be parsed.
+
+### Status and problem reports
+
+The status in the top bar always says what stackthread is doing:
+
+| Status | Meaning |
+|---|---|
+| Indexing 48/132 files | the first full read, with a progress line under the top bar |
+| Re-indexing OrderPage.tsx | a file changed and is being read again |
+| Live · rev 3 | up to date; the number goes up with every change |
+| Offline, reconnecting | the browser lost the stackthread server; it retries by itself |
+
+Click it for the activity panel: recent events, and a **Problems** tab with every warning and error.
+**Report a problem**, there or in the status bar, makes a zip of the logs and diagnostics to attach when you ask
+for help. It never contains source code.
+
+### Keyboard
+
+| Keys | Action |
+|---|---|
+| Alt+Shift+S | inspect mode on or off, in the workbench and inside your app |
+| Ctrl+K / Cmd+K | search |
+| Enter, Alt+Enter | in search: open in the code pane, open in the map |
+| Ctrl+S / Cmd+S | save in the code pane |
+| Esc | close search, a dialog, or inspect mode in your app |
+
+The theme button in the top bar switches between your system theme, light and dark.
+
+## Command line
+
+```
+stackthread [--config PATH] [--log-level LEVEL | --verbose] <command> [options]
+```
+
+| Command | What it does |
+|---|---|
+| `init [--force]` | detect backends and frontends and write `stackthread.toml` |
+| `serve` | serve the workbench on `127.0.0.1`, read the code in the background, follow changes |
+| `scan` | index once and print a summary; `--json FILE` writes the whole graph |
+| `explain FILE:LINE` | the thread through a code location, as text or `--json` |
+| `check` | findings for CI; exits with 1 at or above `--fail-on` (default `error`) |
+| `doctor` | check Node.js, the bundled parser, the UI and the config |
+| `logs` | print, filter and follow the log |
+| `report` | make a problem report zip |
+
+`serve` options: `--port N` (default 7341; the next nine are tried when busy), `--no-open`, `--no-watch`,
+`--read-only`.
+
+`explain` shows who reaches a function from the screen, which requests lead to it and what data it touches:
+
+```
+$ stackthread explain backend/shop/services.py:21
+reserve()  backend/shop/services.py:21
+
+In the browser
+  shop.placeOrder()  web/src/lib/shop.ts:23  calls POST /api/orders/
+
+Over the wire
+  POST /api/orders/  web/src/lib/shop.ts:24  sent by shop.placeOrder
+
+On the server
+  reserve()  backend/shop/services.py:21  this definition
+  POST /api/orders/  backend/shop/urls.py:8  reaches this definition
+
+Data
+  Product (shop_product)  backend/shop/models.py:4  written by reserve
+```
+
+### In CI
+
+```sh
+pip install --pre stackthread
+stackthread check                     # --fail-on warning is stricter; --json for machines
+```
+
+| Rule | Severity | Meaning |
+|---|---|---|
+| `unmatched-request` | warning | a frontend request matches no endpoint |
+| `layers` | error | an import breaks a `[[rules.layers]]` order |
+| `parse-error` | error | a file could not be parsed |
+| `node-unavailable` | error | frontend sources exist but Node.js 22.18+ was not found |
+| `adapter` | error | an adapter failed to load or raised |
+
+Exit codes: 0 success, 1 findings at the threshold, 2 invalid configuration, 70 unexpected error (in the log),
+130 interrupted.
+
+## Configuration
+
+stackthread reads `stackthread.toml` from the current folder or the nearest parent, or `[tool.stackthread]` in
+`pyproject.toml`. Every key is optional. Unknown keys and wrong types are reported together, with suggestions for
+misspellings. Paths are relative to the project root; `*` matches within a folder and `**` across folders.
+`.gitignore`, `node_modules`, virtual environments and build output are always skipped.
+
+```toml
+[project]
+name = "shop"                       # shown in the workbench title
+
+[[sources]]                         # one per backend or frontend
+name = "api"
+lang = "python"                     # python | typescript | javascript
+root = "backend"                    # the folder that contains your top-level package
+exclude = ["**/migrations/**"]
+# include = ["**/*.py"]            # default: every file of the language
+# frameworks = ["drf", "django-orm"]  # default: detect everything
+
+[[sources]]
+name = "web"
+lang = "typescript"
+root = "web"
+include = ["src/**/*"]
+# alias = { "@" = "src" }           # default: from tsconfig.json and vite.config.*
+
+[http]
+# Requests from "web" starting with /api reach routes of "api" with the prefix replaced:
+links = [{ from = "web", to = "api", client_prefix = "/api", server_prefix = "" }]
+clients = ["request"]               # extra functions to treat as HTTP clients
+
+[pages]
+params = { orderId = "1" }          # sample values, so /orders/$orderId opens as /orders/1
+
+[[views]]                           # named states your router does not express
+group = "Checkout"
+name = "Payment step"
+path = "/checkout?step=payment"
+
+[server]
+port = 7341
+editor = "vscode"                   # vscode | cursor | windsurf | zed | idea | pycharm | webstorm | none
+edit = true                         # allow saving; `serve --read-only` overrides it
+edit_deny = ["config/*.yaml"]       # added to the built-in list of files that are never written
+format = { python = ["ruff", "format", "{file}"], typescript = ["npx", "prettier", "--write", "{file}"] }
+open_browser = true
+
+[[rules.layers]]                    # for `stackthread check`: lower layers never import higher ones
+name = "backend layers"
+order = ["shop.core", "shop.services", "shop.api"]
+
+[log]
+level = "debug"                     # what the log file records
+size_kb = 1024                      # size of one file before it is rotated
+keep = 5                            # rotated files kept
+
+[adapters]
+use = ["tools.stackthread_adapter:Orders"]
+```
+
+Python test files (`test_*.py`, `*_test.py`, `tests/`) are indexed and tagged: they appear in search but never in a
+server thread.
+
+## Project layouts
+
+- **Monorepo with `backend/` and `web/`:** run `init` and `serve` at the root. The plugin finds the root by
+  looking upwards for `stackthread.toml`.
+- **Several frontends:** one `[[sources]]` per frontend, and the plugin in each Vite app.
+- **Backend and frontend in different repositories:** put both under one folder, or run stackthread in the
+  frontend repository with a source `root = "../api"`.
+- **Without the Vite plugin:** `http://127.0.0.1:7341/__stackthread/` serves the same workbench; the map, search,
+  code pane and reverse threads work, but there is no live app to click. Django projects that render templates
+  get the map, endpoints, models, `explain` and `check`.
+
+## Teaching stackthread your domain
+
+Every application has meaning no framework detector can see: order stages kept in a database, feature flags,
+translation keys. There are two ways to add it.
+
+### Concepts: no code
+
+A concept links a `data-*` attribute in your markup to where its value is defined:
+
+```tsx
+<button data-i18n="pay_button" onClick={pay}>{t('pay_button')}</button>
+```
+
+```toml
+[[concepts]]
+attribute = "data-i18n"
+label = "Text {value}"
+find = [{ file = "web/src/locales/en.json", pattern = '"{value}":' }]
+mentions = true                     # also list up to 40 source lines that mention the value
+```
+
+Clicking the button, or anything inside an element with the attribute, adds a layer "Text pay_button" with the
+line that defines it and the lines that use it.
+
+### Adapters: a little Python
+
+An adapter is a class in your own code; every method is optional.
+
+```python
+# tools/stackthread_adapter.py
+from stackthread.adapters import Adapter, Context, View
+from stackthread.model import Finding, Graph, Node
+
+
+class Orders(Adapter):
+    name = "orders"
+
+    def views(self, ctx: Context) -> list[View]:
+        """Named states for the sidebar: here, one per order stage."""
+        stages = ["packing", "shipping", "delivered"]
+        return [View(group="Order stages", name=s.title(), path=f"/orders/1?stage={s}") for s in stages]
+
+    def extend(self, ctx: Context, graph: Graph) -> None:
+        """Add nodes and edges after the built-in detectors ran."""
+        graph.add_node(Node(id="stage:packing", kind="store", name="Packing stage", lang="py", source="api"))
+
+    def lineage(self, ctx: Context, selection: dict, layers: list[dict]) -> None:
+        """Add or change rows of a thread before it is shown."""
+        layers.append({"id": "stage", "title": "Stage", "note": "from the stage table",
+                       "rows": [{"label": "Packing", "kind": "stage"}]})
+
+    def checks(self, ctx: Context, graph: Graph) -> list[Finding]:
+        """Extra findings for `stackthread check` and the Findings tab."""
+        return []
+```
+
+Enable it with `[adapters] use = ["tools.stackthread_adapter:Orders"]`, or publish it from a package through the
+`stackthread.adapters` entry-point group:
+
+```toml
+[project.entry-points."stackthread.adapters"]
+orders = "my_package.stackthread:Orders"
+```
+
+An adapter receives `ctx.config` (the parsed settings), `ctx.project` (with the graph and the facts of every file),
+`graph` (nodes, edges, and `out`, `into`, `of_kind`, `at(file, line)`), `selection` (what was clicked) and `layers`
+(the thread as JSON). An adapter that fails becomes an `adapter` finding; indexing carries on without it. Adapters
+run with your permissions, like any dev dependency.
+
+## Troubleshooting
+
+Start with `stackthread doctor`: it checks Node.js, the bundled parser, the UI and the configuration, and says
+where the logs are.
+
+Every run writes JSON lines to `.stackthread/logs/stackthread.log` in your project (the folder ignores itself in
+git). Read them with:
+
+```sh
+stackthread logs                    # the last 50 entries at info and above
+stackthread logs --level warning    # only problems
+stackthread logs --area http -f     # one part, following new entries
+```
+
+Areas: `cli`, `index`, `worker` (the Node process), `server`, `http`, `edit`, `adapters` and `client` (errors in
+the workbench, the in-app panel and the Vite plugin). Use `--verbose` on any command for more on the console.
+
+`stackthread report --note "what went wrong"` (or **Report a problem** in the workbench) makes a zip with the
+logs, versions, `doctor` checks and summaries of the settings and the project. It never contains source code;
+paths under your home folder are shown as `~`. Nothing is sent anywhere; you decide where the file goes.
+
+**A request is not linked to its endpoint.** `stackthread check` lists it as `unmatched-request` with the URL it
+saw. Usually the frontend calls `/api/...` while backend routes start at `/`; add an `[http] links` entry.
+
+**Frontend files are missing.** Check Node.js 22.18 or later with `stackthread doctor`; the log area `worker`
+shows what Node printed.
+
+**The workbench says the server is not running.** Start `stackthread serve` in the project and reload.
+
+## Security and privacy
+
+stackthread is for one developer, on their own machine, with their own project.
+
+- **Nothing leaves your machine.** No telemetry, no update checks, no accounts.
+- **Loopback only.** The server binds `127.0.0.1`, with no option to change it.
+- **Other websites cannot use it.** Requests must carry a loopback `Host`, a loopback `Origin` when present, and a
+  random 256-bit session token created at start-up.
+- **The Vite plugin adds no new door.** It only proxies same-origin requests from this machine, only in
+  `vite dev`.
+- **Edits are fenced.** Only existing files inside your sources; never secrets or lock files; a save is refused if
+  the file changed on disk; files over 2 MB or not UTF-8 are refused; writes are atomic and keep line endings.
+  `--read-only` turns saving off.
+- **The workbench is locked down** with a strict Content-Security-Policy.
+- **Logs stay local** and never contain file contents or the session token.
+
+## Limitations
+
+stackthread reads code; where a value only exists at run time (a URL built from a variable, a router registered
+conditionally) it shows what it can, and `check` lists what it could not link.
+
+Not supported yet: Next.js, Remix framework mode and other server-rendered React frameworks; Vue, Svelte,
+Angular; Flask, Starlette and Litestar routes; GraphQL and tRPC; webpack and Rspack.
+
+## Licence
+
+MIT. The workbench bundles Preact and CodeMirror, and the parser bundles Babel, all under the MIT licence; their
+notices ship in the package.