django-explain-errors 0.10.1__tar.gz → 0.11.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. {django_explain_errors-0.10.1/django_explain_errors.egg-info → django_explain_errors-0.11.1}/PKG-INFO +269 -34
  2. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/README.md +266 -33
  3. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1/django_explain_errors.egg-info}/PKG-INFO +269 -34
  4. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/django_explain_errors.egg-info/SOURCES.txt +2 -0
  5. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/django_explain_errors.egg-info/requires.txt +3 -0
  6. django_explain_errors-0.11.1/explain_errors/client.py +220 -0
  7. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/management/commands/build_error_index.py +4 -0
  8. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/middleware.py +61 -7
  9. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/rag/indexer.py +27 -5
  10. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/rag/retriever.py +73 -5
  11. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/rag/store.py +35 -0
  12. django_explain_errors-0.11.1/explain_errors/sanitize.py +105 -0
  13. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/setup.py +4 -1
  14. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_build_error_index.py +21 -0
  15. django_explain_errors-0.11.1/tests/test_client.py +294 -0
  16. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_eval_judge.py +158 -0
  17. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_eval_run.py +183 -2
  18. django_explain_errors-0.11.1/tests/test_local_embeddings.py +505 -0
  19. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_rag.py +10 -11
  20. django_explain_errors-0.11.1/tests/test_sanitize.py +215 -0
  21. django_explain_errors-0.11.1/tests/test_token_param_fallback.py +249 -0
  22. django_explain_errors-0.10.1/explain_errors/client.py +0 -38
  23. django_explain_errors-0.10.1/explain_errors/sanitize.py +0 -44
  24. django_explain_errors-0.10.1/tests/test_client.py +0 -115
  25. django_explain_errors-0.10.1/tests/test_sanitize.py +0 -74
  26. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/LICENSE +0 -0
  27. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/MANIFEST.in +0 -0
  28. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/django_explain_errors.egg-info/dependency_links.txt +0 -0
  29. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/django_explain_errors.egg-info/top_level.txt +0 -0
  30. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/__init__.py +0 -0
  31. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/debug_page.py +0 -0
  32. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/management/__init__.py +0 -0
  33. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/management/commands/__init__.py +0 -0
  34. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/rag/__init__.py +0 -0
  35. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/throttle.py +0 -0
  36. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/explain_errors/tracebacks.py +0 -0
  37. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/setup.cfg +0 -0
  38. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_async_marking.py +0 -0
  39. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_debug_page.py +0 -0
  40. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_dedup.py +0 -0
  41. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_eval_fixtures.py +0 -0
  42. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_language.py +0 -0
  43. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_middleware.py +0 -0
  44. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_print_stdout.py +0 -0
  45. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_process_exception_split.py +0 -0
  46. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_signals.py +0 -0
  47. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_throttle.py +0 -0
  48. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_tracebacks.py +0 -0
  49. {django_explain_errors-0.10.1 → django_explain_errors-0.11.1}/tests/test_truncation.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: django-explain-errors
3
- Version: 0.10.1
3
+ Version: 0.11.1
4
4
  Summary: Django middleware that explains unhandled exceptions in DEBUG using an LLM, optionally grounded in your own project source via a local vector index. Works with OpenAI, Claude, or any OpenAI-compatible endpoint including local models.
5
5
  Home-page: https://github.com/topunix/django-explain-errors
6
6
  Author: topunix
@@ -35,6 +35,8 @@ Requires-Dist: python-dotenv>=1.0
35
35
  Requires-Dist: asgiref>=3.6
36
36
  Provides-Extra: rag
37
37
  Requires-Dist: sqlite-vec>=0.1.0; extra == "rag"
38
+ Provides-Extra: local-embeddings
39
+ Requires-Dist: sentence-transformers>=2.2; extra == "local-embeddings"
38
40
  Dynamic: author
39
41
  Dynamic: author-email
40
42
  Dynamic: classifier
@@ -91,20 +93,22 @@ better. This package is not trying to compete there.
91
93
  - Captures Django errors and exceptions
92
94
  - Shows the explanation on Django's debug page, directly under the
93
95
  exception headline (`EXPLAIN_ERRORS_INJECT_DEBUG_PAGE`, on by default)
96
+ - Codebase-aware explanations via RAG over a local sqlite-vec index
97
+ (recommended; see "Codebase-aware explanations" below)
98
+ - Local embeddings via an optional extra: your codebase is indexed on your machine and never uploaded to an embeddings provider (only the traceback and a few relevant snippets reach your chat model; see [Privacy](#privacy))
99
+ - Separate embeddings provider config, so Anthropic users can use RAG
100
+ - Explains errors using OpenAI out of the box, Anthropic's Claude models, or any other
101
+ OpenAI-compatible endpoint (Ollama, LM Studio, Azure, gateways) via `OPENAI_BASE_URL`,
102
+ including fully local models, so no code has to leave your machine
103
+ - Explanations in your language via `EXPLAIN_ERRORS_LANGUAGE`, with exception names,
104
+ identifiers, and code kept in English
94
105
  - Copy button on each code block in the explanation, so a suggested fix
95
106
  copies in one click
96
107
  - Prints the explanation to stdout, for terminal workflows and logs
97
108
  (`EXPLAIN_ERRORS_PRINT_STDOUT`, on by default)
98
109
  - Optional JSON 500 response instead of the debug page
99
110
  (`EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE = False`)
100
- - Explains errors using OpenAI out of the box, Anthropic's Claude models, or any other
101
- OpenAI-compatible endpoint (Ollama, LM Studio, Azure, gateways) via `OPENAI_BASE_URL`,
102
- including fully local models, so no code has to leave your machine
103
- - Codebase-aware explanations via RAG over a local sqlite-vec index
104
- (recommended; see "Codebase-aware explanations" below)
105
- - Explanations in your language via `EXPLAIN_ERRORS_LANGUAGE`, with exception names,
106
- identifiers, and code kept in English
107
- - Redacts secrets, tokens, and emails from tracebacks before sending
111
+ - Redacts common secret patterns, tokens, and emails (see [Privacy](#privacy))
108
112
  - Rate limits API calls with a configurable sliding window
109
113
  - Works with both sync (WSGI) and async (ASGI) views
110
114
  - Manages the API key using environment variables
@@ -153,8 +157,9 @@ pip install django-explain-errors
153
157
  ## Production Safety
154
158
 
155
159
  This middleware is intended for local development only. When active, it sends exception
156
- tracebacks to the configured LLM provider, which may include source code, file paths, and
157
- local variable values. Keep it out of production.
160
+ tracebacks to the configured LLM provider, which include file paths, the source line of each
161
+ frame, and the exception message (which can contain values). Local variables are not sent.
162
+ Keep it out of production.
158
163
 
159
164
  **Primary safeguard:** register the middleware only when `DEBUG` is on, or only in your
160
165
  development settings module:
@@ -195,15 +200,6 @@ python manage.py check --deploy --fail-level WARNING
195
200
  The view raised a ValueError because ...
196
201
  ```
197
202
 
198
- ## Async Support
199
-
200
- The middleware exposes both `sync_capable = True` and `async_capable = True`. At initialization it inspects `get_response` to decide whether it is part of a sync or async chain:
201
-
202
- - Under WSGI (for example `runserver` with sync views), requests flow through the synchronous handler.
203
- - Under ASGI (for example with async views), requests are awaited through the async handler. The blocking model API call is offloaded with `asgiref.sync.sync_to_async` so the event loop is not blocked.
204
-
205
- No additional settings are needed. See Installation above for where to place the middleware in `MIDDLEWARE`.
206
-
207
203
  ## Debug page injection
208
204
 
209
205
  When Django renders its debug page for a 500, `explain_errors` also inserts a banner
@@ -243,6 +239,15 @@ what Django's debug page looks like: a new section appears above the request met
243
239
  table. If you rely on the debug page's exact markup (a scraper, a screenshot test), account
244
240
  for this.
245
241
 
242
+ ## Async Support
243
+
244
+ The middleware exposes both `sync_capable = True` and `async_capable = True`. At initialization it inspects `get_response` to decide whether it is part of a sync or async chain:
245
+
246
+ - Under WSGI (for example `runserver` with sync views), requests flow through the synchronous handler.
247
+ - Under ASGI (for example with async views), requests are awaited through the async handler. The blocking model API call is offloaded with `asgiref.sync.sync_to_async` so the event loop is not blocked.
248
+
249
+ No additional settings are needed. See Installation above for where to place the middleware in `MIDDLEWARE`.
250
+
246
251
  ## Compatibility
247
252
 
248
253
  How this middleware interacts with other error-handling and debugging tools, in the default
@@ -250,16 +255,14 @@ preserve mode and with `EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE=False`:
250
255
 
251
256
  | Package | Preserve mode (default) | `EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE=False` |
252
257
  | ------- | ------------------------ | ------------------------------------------- |
253
- | Django Debug Toolbar | Works — Django renders its debug page (with the explanation banner, unless `EXPLAIN_ERRORS_INJECT_DEBUG_PAGE = False`), and Debug Toolbar injects into it. | Broken — the JSON 500 response has no HTML to inject into. |
258
+ | Django Debug Toolbar | Works. Django renders its debug page (with the explanation banner, unless `EXPLAIN_ERRORS_INJECT_DEBUG_PAGE = False`), and Debug Toolbar injects into it. Verified with the banner present, with the toolbar middleware before or after this one, on WSGI and ASGI. | Broken. The JSON 500 response has no HTML to inject into. |
254
259
  | Sentry, Rollbar | Work — the exception propagates and Django re-raises it, so `got_request_exception` fires. | Broken — returning a response ends exception handling before `got_request_exception` fires. |
255
260
  | Django REST Framework | Partial — only exceptions DRF does not already handle itself reach this middleware. | Partial, same reason. |
256
261
  | Silk and other profiling panels | Timings are inflated by the model API call, since `process_exception` blocks the request path. | Same. |
257
262
  | CORS, GZip, WhiteNoise | No interaction. | No interaction. |
258
263
  | `runserver_plus` / Werkzeug debugger | Works — returning `None` re-raises the original exception, and `django-extensions` replaces Django's debug-page renderer with one that re-raises instead, so Werkzeug's WSGI wrapper catches it and shows the interactive debugger. | Broken — the JSON 500 response ends exception handling before it reaches `runserver_plus`'s exception hook, so the Werkzeug debugger never appears. |
259
264
 
260
- Debug Toolbar, Sentry, and Werkzeug were verified empirically, on both Django 4.2 and Django
261
- 6.1, in both modes. DRF, Silk, and CORS/GZip/WhiteNoise are reasoned from the mechanism
262
- rather than tested.
265
+ Debug Toolbar, Sentry, and Werkzeug were verified empirically, on both Django 4.2 and Django 6.1, in both modes. The Debug Toolbar check was repeated with the explanation banner present (default preserve mode) on Django 4.2 and 6.1, under WSGI and ASGI, for sync and async views, with the toolbar middleware before and after this one. Both orderings work. Visual layout and toolbar JavaScript behavior were not checked. DRF, Silk, and CORS/GZip/WhiteNoise are reasoned from the mechanism rather than tested.
263
266
 
264
267
  One additional behavior worth knowing, also verified on both Django versions: when the
265
268
  explanation call itself fails (a bad key, a timeout, an unreachable endpoint), a
@@ -269,6 +272,10 @@ catches the exception internally, so it never becomes an unhandled exception, bu
269
272
  than relying on `got_request_exception`. That event is unrelated to whatever error the
270
273
  developer is actually investigating.
271
274
 
275
+ OpenAI reasoning models are supported. They reject `max_tokens`, so when the API rejects it
276
+ the middleware retries once with `max_completion_tokens` (same value, `OPENAI_MAX_TOKENS`)
277
+ and keeps using that for later calls. Other providers keep receiving `max_tokens`.
278
+
272
279
  ## Configuration
273
280
 
274
281
  | Setting / variable | Required | Description |
@@ -285,8 +292,8 @@ developer is actually investigating.
285
292
  | `OPENAI_BASE_URL` (env or settings) | No | Base URL for any OpenAI-compatible API (for example Ollama at `http://localhost:11434/v1`). When set, a missing API key is replaced with a placeholder since local servers do not require one. |
286
293
  | `EXPLAIN_ERRORS_MAX_CALLS` | No | Together with `EXPLAIN_ERRORS_WINDOW_SECONDS`, caps API spend to at most this many explanations within a rolling window; once the cap is hit, further errors in that window are not sent for explanation until an earlier call ages out. Defaults to `5`. |
287
294
  | `EXPLAIN_ERRORS_WINDOW_SECONDS` | No | Length in seconds of the rolling window `EXPLAIN_ERRORS_MAX_CALLS` is measured against. Defaults to `60` (with the defaults, at most 5 explanations per 60-second window). |
288
- | `EXPLAIN_ERRORS_REDACT_PATTERNS` | No | Extra regex pattern strings (each passed to `re.compile`) applied to the traceback, appended after the built-in secret/token/email patterns. An invalid pattern is skipped with a warning rather than raising. Defaults to `[]`. |
289
- | `EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS` | No | When `True`, skips the built-in secret/token/email redaction patterns entirely and redacts only what `EXPLAIN_ERRORS_REDACT_PATTERNS` specifies. Turning this on removes the default protection against leaking secrets and PII in tracebacks. Defaults to `False`. |
295
+ | `EXPLAIN_ERRORS_REDACT_PATTERNS` | No | Extra regex pattern strings (each passed to `re.compile`) applied to the traceback, the RAG query, and indexed chunks (see [Redaction](#redaction)), appended after the built-in secret/token/email patterns. An invalid pattern is skipped with a warning rather than raising. Defaults to `[]`. |
296
+ | `EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS` | No | When `True`, skips the built-in secret/token/email redaction patterns entirely and redacts only what `EXPLAIN_ERRORS_REDACT_PATTERNS` specifies. Turning this on removes the default protection against leaking secrets and PII in tracebacks, RAG queries, and indexed chunks (see [Redaction](#redaction)). Defaults to `False`. |
290
297
  | `EXPLAIN_ERRORS_REDACT_REPLACEMENT` | No | Replacement string substituted for anything matched by the redaction patterns. Defaults to `"[REDACTED]"`. |
291
298
  | `EXPLAIN_ERRORS_LANGUAGE` | No | Language the explanation prose is written in, as a plain name or code (for example `"Spanish"` or `"es"`). Defaults to `None`, meaning English. Exception names, identifiers, code, file paths, and tracebacks always stay in English regardless of this setting. |
292
299
  | `EXPLAIN_ERRORS_DEDUP` | No | When `True` (the default), a repeat of an error already explained in this process replays the earlier explanation instead of calling the model or using a throttle slot. Two errors are identical when they have the same exception type, the same exception message, and the same innermost project line (file, line number, and that line's source text); editing the line makes it a new error. Keeps the last 128 distinct errors per worker (least recently used entries are evicted first) and entries never expire on a timer. Set to `False` to explain every error. |
@@ -334,6 +341,59 @@ Reasoning models behind `OPENAI_BASE_URL` spend part of the token budget on inte
334
341
  before producing visible output. At a low `OPENAI_MAX_TOKENS`, the budget can be used up by
335
342
  reasoning alone, and the explanation comes back empty. Raise `OPENAI_MAX_TOKENS` if you see this.
336
343
 
344
+ ### RAG does not work with Anthropic alone
345
+
346
+ The [RAG layer](#codebase-aware-explanations-rag) needs an embeddings endpoint, and
347
+ Anthropic's OpenAI-compatible API has none: `POST /v1/embeddings` returns HTTP 404
348
+ (`not_found_error`), and Anthropic does not offer its own embedding model. With only the
349
+ config above, `build_error_index` fails with an error saying so, and retrieval at request
350
+ time falls back to the traceback-only prompt.
351
+
352
+ To use RAG with Claude, keep Anthropic for chat and point embeddings at an OpenAI-compatible
353
+ provider with the `EXPLAIN_ERRORS_RAG_EMBED_*` settings:
354
+
355
+ ```python
356
+ # Chat: Anthropic
357
+ OPENAI_BASE_URL = "https://api.anthropic.com/v1/"
358
+ OPENAI_API_KEY = "your_anthropic_api_key_here"
359
+ OPENAI_MODEL = "..."
360
+
361
+ # Embeddings: any OpenAI-compatible embeddings provider (OpenAI shown)
362
+ EXPLAIN_ERRORS_RAG_ENABLED = True
363
+ EXPLAIN_ERRORS_RAG_EMBED_BASE_URL = "https://api.openai.com/v1/"
364
+ EXPLAIN_ERRORS_RAG_EMBED_API_KEY = "your_openai_api_key_here"
365
+ EXPLAIN_ERRORS_RAG_EMBED_MODEL = "text-embedding-3-small"
366
+ ```
367
+
368
+ Each of the three settings falls back to the chat value when unset, with one exception for
369
+ safety: if `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` differs from the chat base URL,
370
+ `EXPLAIN_ERRORS_RAG_EMBED_API_KEY` is required, and `ImproperlyConfigured` is raised without
371
+ it. The chat key is never sent to a different provider. A local OpenAI-compatible
372
+ embeddings server that ignores keys still needs a dummy value, for example
373
+ `"sk-no-key-required"`.
374
+
375
+ The embed settings are also read from environment variables of the same name, which take
376
+ precedence over Django settings. Indexed source chunks are sent to the embeddings provider
377
+ (after redaction), not to Anthropic.
378
+
379
+ ### No second key: local embeddings
380
+
381
+ If you would rather not use a second provider, run the embeddings on your own machine
382
+ instead. Chat stays on Anthropic and no embeddings API key is needed:
383
+
384
+ ```python
385
+ # Chat: Anthropic
386
+ OPENAI_BASE_URL = "https://api.anthropic.com/v1/"
387
+ OPENAI_API_KEY = "your_anthropic_api_key_here"
388
+ OPENAI_MODEL = "..."
389
+
390
+ # Embeddings: on this machine
391
+ EXPLAIN_ERRORS_RAG_ENABLED = True
392
+ EXPLAIN_ERRORS_RAG_EMBED_BACKEND = "local"
393
+ ```
394
+
395
+ See [Local embeddings](#local-embeddings) for the install command and what to expect.
396
+
337
397
  ### A note on API keys and 401s
338
398
 
339
399
  `explain_errors` reads `OPENAI_API_KEY` (see Configuration above); it does not read
@@ -345,6 +405,133 @@ and rejected, so a missing `OPENAI_API_KEY` shows up as an opaque `401 Unauthori
345
405
  than a clear configuration error. If you see a 401 with `OPENAI_BASE_URL` pointed at a remote
346
406
  provider, check that `OPENAI_API_KEY` — not a provider-specific variable — is actually set.
347
407
 
408
+ ## Privacy
409
+
410
+ This section describes what leaves your machine, when, and to which endpoint. It describes
411
+ this package's behavior only. It makes no claims about what a provider does with data after
412
+ receiving it; check your provider's own terms.
413
+
414
+ ### What is sent to the chat provider
415
+
416
+ On each error that triggers an explanation, one request goes to the endpoint set by
417
+ `OPENAI_BASE_URL` (OpenAI by default). Its contents:
418
+
419
+ - A fixed system prompt, plus the language name if `EXPLAIN_ERRORS_LANGUAGE` is set.
420
+ - The traceback: exception type and message, and for each frame the file path, line number,
421
+ and source line. Library frames are condensed to fit `OPENAI_MAX_TRACEBACK_CHARS`
422
+ (default 3000), and the result is redacted. Local variable values are not included, but the
423
+ exception message can contain values.
424
+ - With RAG enabled, up to `EXPLAIN_ERRORS_RAG_TOP_K` (default 4) retrieved code chunks, each
425
+ under a header with its file path and line range. The traceback and chunks together are
426
+ capped at `EXPLAIN_ERRORS_RAG_MAX_PROMPT_CHARS` (default 6000).
427
+ - The model name and token limit as request parameters, and your API key as the credential.
428
+
429
+ Nothing else is added: not the request path, headers, body, session, or user, and not your
430
+ Django settings. The request object is never passed to the prompt builder.
431
+
432
+ Not every error causes a request. A repeat of an already explained error replays the cached
433
+ explanation (`EXPLAIN_ERRORS_DEDUP`, on by default), and calls are limited per worker process
434
+ to 5 per 60 seconds by default.
435
+
436
+ ### What is sent to the embeddings provider
437
+
438
+ Only when RAG is enabled and the embeddings backend is not `local`:
439
+
440
+ - **When you run `build_error_index`:** every chunk of every `.py`, `.html`, and `.txt` file
441
+ under `EXPLAIN_ERRORS_RAG_INCLUDE` (default: `BASE_DIR`), after redaction. Directories named
442
+ in `EXPLAIN_ERRORS_RAG_EXCLUDE` are skipped. The default is `migrations`, `venv`, `.venv`,
443
+ `env`, `node_modules`, `static`, `media`, and `.git`. Setting it replaces that list rather
444
+ than extending it, and it matches directory names only, not individual files. File paths
445
+ are not part of the embedded text.
446
+ - **On each error (RAG enabled):** a query made of the exception type and message plus about
447
+ five lines of source on each side of the innermost project frame, after redaction.
448
+
449
+ The index file itself stays on disk. It holds the redacted chunk text.
450
+
451
+ ### Summary by setup
452
+
453
+ | Setup | Indexing (your codebase) | Each error (traceback + snippets) |
454
+ |---|---|---|
455
+ | OpenAI chat + OpenAI embeddings | Redacted chunks go to OpenAI embeddings | Query goes to OpenAI embeddings; redacted traceback and retrieved snippets go to OpenAI chat |
456
+ | Anthropic chat + separate embeddings provider | Redacted chunks go to the embeddings provider only, not to Anthropic | Query goes to the embeddings provider; redacted traceback and retrieved snippets go to Anthropic |
457
+ | Any chat + local embeddings | Stays on your machine | Query is embedded locally; redacted traceback and retrieved snippets go to your chat provider |
458
+ | Local chat (e.g. Ollama) + local embeddings | Nothing leaves the machine* | Nothing leaves the machine* |
459
+ | No RAG | Not applicable, no index is built | Redacted traceback goes to your chat provider |
460
+
461
+ \* Assumes the chat endpoint is genuinely local; the package sends to whatever
462
+ `OPENAI_BASE_URL` points at. The local embedding model is downloaded on first use (see
463
+ [Local embeddings](#local-embeddings)), which transfers the model to you and none of your code.
464
+
465
+ ### Redaction
466
+
467
+ Before any text is sent or stored, it passes through a regex-based redactor. By default it
468
+ replaces the following with `[REDACTED]` (`EXPLAIN_ERRORS_REDACT_REPLACEMENT`):
469
+
470
+ - **Secret-like assignments.** A name that starts with `secret`, `password`, `passwd`,
471
+ `token`, `api_key` (or `apikey`), or `auth`, followed by `=` or `:`, together with its
472
+ value. The name may continue after the keyword with `_` or `-` and more word characters,
473
+ or with a capitalized suffix, so `SECRET_KEY`, `AWS_SECRET_ACCESS_KEY`, `secret-key`, and
474
+ `apiKeyId` all match. A quote after the name is allowed, so dict keys such as
475
+ `'PASSWORD': 'x'` match. A quoted value is redacted whole, spaces included. Text before the
476
+ keyword stays: `AWS_SECRET_ACCESS_KEY = "x"` becomes `AWS_[REDACTED]`.
477
+ - **Credentials in URLs.** In `scheme://user:pw@host`, the `user:pw` part. The scheme and
478
+ host stay.
479
+ - **Known key shapes**, whatever the variable is called: `sk-` (OpenAI, and Anthropic's
480
+ `sk-ant-`), Stripe `sk_live_` and `sk_test_`, GitHub `ghp_`, `gho_`, `ghu_`, `ghs_`,
481
+ `ghr_`, and `github_pat_`, Slack `xoxb-`, `xoxa-`, `xoxp-`, `xoxr-`, and `xoxs-`, AWS
482
+ `AKIA`, and Google `AIza`. Each needs a minimum run of characters after the prefix (for
483
+ example 20 after `sk-`), so short strings do not match.
484
+ - `Authorization` header values and `Bearer` tokens.
485
+ - Email addresses.
486
+
487
+ Five Django settings start with a keyword but hold no secret, so they are never redacted:
488
+ `AUTH_USER_MODEL`, `AUTHENTICATION_BACKENDS`, `AUTH_PASSWORD_VALIDATORS`, `PASSWORD_HASHERS`,
489
+ and `PASSWORD_RESET_TIMEOUT`. The match is on the exact name.
490
+
491
+ It is applied to the traceback (after truncation), to the RAG query text, and to every chunk
492
+ at index time, before the chunk is embedded or written to the index. It is not applied to file
493
+ paths, which appear in tracebacks and in chunk headers.
494
+
495
+ Add your own regexes with `EXPLAIN_ERRORS_REDACT_PATTERNS`. They run after the defaults.
496
+ `EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS = True` turns the defaults off.
497
+
498
+ **Limits.** This is pattern matching, not a secret scanner. Treat it as a safety net.
499
+
500
+ - A secret is only caught if its name contains one of the keywords above or its value has a
501
+ known shape. A bare `key`, `private_key`, or `credentials` is not redacted, so
502
+ `private_key = "abc123"` passes through. A bare `key` is left alone on purpose, because redacting it
503
+ would also hide ordinary code such as dict keys and cache keys.
504
+ - Keys with an unknown prefix, or shorter than the length each shape requires, are not
505
+ caught unless the name matches.
506
+ - An unquoted value with spaces is redacted up to the first space only, and a value that
507
+ spans lines (a triple-quoted string, a PEM block) is not redacted past its first line.
508
+ - Some harmless names are redacted too. Any name that starts with a keyword followed by `_`,
509
+ `-`, or a capital letter matches, so `token_count` and `PASSWORD_FIELD` lose their values
510
+ even though they are not secrets. Names that only begin with the same letters, such as
511
+ `author`, `authors`, and `tokenizer`, are left alone.
512
+ - Names, IP addresses, phone numbers, other personal data, and file paths (which can contain
513
+ your username) are not redacted. Neither are values inside exception messages unless a
514
+ pattern happens to match them.
515
+ - `settings.py` normally sits under `BASE_DIR`, so it is indexed by default. Its
516
+ `SECRET_KEY` and `DATABASES` password are caught, but a secret stored under a name with no
517
+ keyword may not be. Add a pattern for your own naming, for example
518
+ `EXPLAIN_ERRORS_REDACT_PATTERNS = [r"(?i)(?:private_key|credentials)\w*\s*=\s*\S+"]`, leave
519
+ the settings directory out of `EXPLAIN_ERRORS_RAG_INCLUDE`, or use local embeddings.
520
+
521
+ ### DEBUG only
522
+
523
+ The middleware does nothing when `DEBUG` is `False`: no client is created and no request is
524
+ made. One thing is not tied to `DEBUG`: `manage.py build_error_index` is a command, so it
525
+ sends chunks to a non-local embeddings provider whenever you run it. For keeping the
526
+ middleware out of production, see [Production Safety](#production-safety).
527
+
528
+ ### Telemetry
529
+
530
+ This package's own code contains no telemetry or analytics and makes no network requests
531
+ other than to the chat and embeddings endpoints you configure. Nothing is sent to the
532
+ maintainer. Its dependencies (`openai`, and optionally `sqlite-vec` and
533
+ `sentence-transformers`) are separate projects and are not covered by this statement.
534
+
348
535
  ## Explanation language
349
536
 
350
537
  By default, explanations are written in English. Set `EXPLAIN_ERRORS_LANGUAGE` to
@@ -396,6 +583,11 @@ also retrieves the most relevant chunks of your own project's source code
396
583
  from a local vector index and includes them in the prompt, so explanations
397
584
  can reference your actual functions and classes instead of guessing at them.
398
585
 
586
+ RAG needs a provider with an embeddings endpoint. Anthropic's OpenAI-compatible API has
587
+ none, so Claude users must configure a separate embeddings provider or use
588
+ [local embeddings](#local-embeddings); see
589
+ [RAG does not work with Anthropic alone](#rag-does-not-work-with-anthropic-alone).
590
+
399
591
  Recommended for most projects. It is opt-in only because setup takes three
400
592
  steps (install the extra, build the index, enable it), and it adds no
401
593
  dependencies or behavior until you do.
@@ -420,8 +612,8 @@ python manage.py build_error_index
420
612
  ```
421
613
 
422
614
  This walks your project, chunks Python files by top-level function/class
423
- (and other text files by fixed-size line windows), embeds each chunk with
424
- the configured endpoint's embeddings API, and writes them to a local index file. Re-run it
615
+ (and other text files by fixed-size line windows), embeds each chunk through the configured embeddings provider (or on this machine with the
616
+ [local backend](#local-embeddings)), and writes them to a local index file. Re-run it
425
617
  whenever your source changes meaningfully. Indexing is not automatic.
426
618
  Rebuilding is idempotent: it builds into a temp file and atomically replaces
427
619
  the previous index.
@@ -441,16 +633,15 @@ EXPLAIN_ERRORS_RAG_ENABLED = True
441
633
  | `EXPLAIN_ERRORS_RAG_ENABLED` | `False` | Master switch for the RAG layer. |
442
634
  | `EXPLAIN_ERRORS_RAG_INDEX_PATH` | `<BASE_DIR>/.explain_errors_index.db` | Path to the local vector index file. |
443
635
  | `EXPLAIN_ERRORS_RAG_TOP_K` | `4` | Number of chunks retrieved and injected into the prompt. |
444
- | `EXPLAIN_ERRORS_RAG_EMBED_MODEL` | `"text-embedding-3-small"` | Embedding model used for indexing and retrieval. Must exist on the configured endpoint. |
636
+ | `EXPLAIN_ERRORS_RAG_EMBED_BACKEND` | `"openai"` | `"openai"` embeds through an OpenAI-compatible endpoint. `"local"` embeds on this machine with sentence-transformers and needs no API key (see [Local embeddings](#local-embeddings)). Also read from the environment, which takes precedence. |
637
+ | `EXPLAIN_ERRORS_RAG_EMBED_MODEL` | `"text-embedding-3-small"` (`"all-MiniLM-L6-v2"` with the `local` backend) | Embedding model used for indexing and retrieval. Must exist on the configured endpoint, or be a sentence-transformers model name for the `local` backend. |
638
+ | `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` | chat `OPENAI_BASE_URL` | Base URL of the embeddings provider. Ignored by the `local` backend. Also read from the environment. Set it when chat and embeddings use different providers (see the Anthropic section). |
639
+ | `EXPLAIN_ERRORS_RAG_EMBED_API_KEY` | chat `OPENAI_API_KEY` | Key for the embeddings provider. Ignored by the `local` backend. Also read from the environment. Required when the embed base URL differs from the chat one. |
445
640
  | `EXPLAIN_ERRORS_RAG_INCLUDE` | `None` (defaults to `BASE_DIR`) | List of directories to index. |
446
641
  | `EXPLAIN_ERRORS_RAG_EXCLUDE` | migrations, venvs, `node_modules`, static, media, `.git` | Directory names to skip while indexing. |
447
642
  | `EXPLAIN_ERRORS_RAG_MAX_PROMPT_CHARS` | `6000` | Combined character budget for the traceback + retrieved source sections of the prompt. |
448
643
 
449
- Every chunk of source code and every retrieval query is passed through the
450
- same `sanitize_traceback()` redaction used for tracebacks, which strips
451
- patterns that look like secrets, tokens, and emails before anything is sent to the model API or written to the index. It's a pattern-based filter, not a
452
- guarantee: it catches recognizable secret shapes, not arbitrary sensitive
453
- data that doesn't match one.
644
+ Chunks and queries are redacted before they are sent or written to the index; see [Privacy](#privacy) for what is and isn't caught.
454
645
 
455
646
  If RAG is enabled but the index is missing, `sqlite-vec` isn't installed, or
456
647
  retrieval fails for any reason, the middleware logs a warning and falls back
@@ -458,6 +649,50 @@ to the traceback-only prompt. It never breaks error reporting.
458
649
 
459
650
  RAG-grounded explanations tend to be longer than traceback-only ones. The default `OPENAI_MAX_TOKENS` already leaves generous headroom for this, but if you've lowered it, raise it back up when RAG is enabled so explanations are not truncated.
460
651
 
652
+ ### Local embeddings
653
+
654
+ Embeddings can run on your own machine, with no embeddings provider and no second API key.
655
+ This also works with Anthropic for chat (see [Using Anthropic](#using-anthropic-claude-models)).
656
+
657
+ Install the extra alongside `rag`:
658
+
659
+ ```bash
660
+ pip install django-explain-errors[rag,local-embeddings]
661
+ ```
662
+
663
+ Then set one setting:
664
+
665
+ ```python
666
+ EXPLAIN_ERRORS_RAG_ENABLED = True
667
+ EXPLAIN_ERRORS_RAG_EMBED_BACKEND = "local"
668
+ ```
669
+
670
+ Then build the index as usual with `python manage.py build_error_index`.
671
+
672
+ What to expect:
673
+
674
+ - **Install size.** The extra pulls in `sentence-transformers` and PyTorch. With the default
675
+ PyTorch wheels on Linux that is roughly 1 to 2 GB; a CPU-only PyTorch build is much smaller.
676
+ The core install and the `rag` extra are unchanged.
677
+ - **First index build.** The default model, `all-MiniLM-L6-v2`, is downloaded the first time
678
+ it is used (about 90 MB), then cached by `sentence-transformers`. Set
679
+ `EXPLAIN_ERRORS_RAG_EMBED_MODEL` to use a different sentence-transformers model.
680
+ - **First explanation after server start.** The model is loaded once per process, so the first
681
+ explanation that uses RAG after the dev server starts pays a one-time load. Later ones do not.
682
+ - **Rebuild the index when you switch.** Vectors from different models are not comparable. The
683
+ index records its backend, model, and vector size. If the current settings do not match, the
684
+ middleware logs one warning per process telling you to rerun `build_error_index` and falls
685
+ back to the traceback-only prompt. It never queries a mismatched index. An index built before
686
+ this feature existed counts as an `openai` index.
687
+ - **A leftover OpenAI model name.** If `EXPLAIN_ERRORS_RAG_EMBED_MODEL` is still set to an
688
+ OpenAI model such as `text-embedding-3-small`, the `local` backend raises
689
+ `ImproperlyConfigured`. Unset it, or name a local model.
690
+ - `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` and `EXPLAIN_ERRORS_RAG_EMBED_API_KEY` are ignored by the
691
+ local backend.
692
+
693
+ **Privacy.** Local embeddings keep your source off any embeddings API, but the traceback and
694
+ retrieved snippets still go to your chat provider; see [Privacy](#privacy).
695
+
461
696
  ### .gitignore
462
697
 
463
698
  The index file is a local build artifact, not something to commit. Add it