django-explain-errors 0.10.0__tar.gz → 0.11.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {django_explain_errors-0.10.0/django_explain_errors.egg-info → django_explain_errors-0.11.0}/PKG-INFO +265 -34
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/README.md +262 -33
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0/django_explain_errors.egg-info}/PKG-INFO +265 -34
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/django_explain_errors.egg-info/SOURCES.txt +2 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/django_explain_errors.egg-info/requires.txt +3 -0
- django_explain_errors-0.11.0/explain_errors/client.py +220 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/management/commands/build_error_index.py +4 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/middleware.py +13 -2
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/rag/indexer.py +27 -5
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/rag/retriever.py +73 -5
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/rag/store.py +35 -0
- django_explain_errors-0.11.0/explain_errors/sanitize.py +105 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/setup.py +4 -1
- django_explain_errors-0.11.0/tests/test_async_marking.py +82 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_build_error_index.py +21 -0
- django_explain_errors-0.11.0/tests/test_client.py +294 -0
- django_explain_errors-0.11.0/tests/test_local_embeddings.py +505 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_rag.py +10 -11
- django_explain_errors-0.11.0/tests/test_sanitize.py +215 -0
- django_explain_errors-0.10.0/explain_errors/client.py +0 -38
- django_explain_errors-0.10.0/explain_errors/sanitize.py +0 -44
- django_explain_errors-0.10.0/tests/test_client.py +0 -115
- django_explain_errors-0.10.0/tests/test_sanitize.py +0 -74
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/LICENSE +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/MANIFEST.in +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/django_explain_errors.egg-info/dependency_links.txt +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/django_explain_errors.egg-info/top_level.txt +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/__init__.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/debug_page.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/management/__init__.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/management/commands/__init__.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/rag/__init__.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/throttle.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/explain_errors/tracebacks.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/setup.cfg +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_debug_page.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_dedup.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_eval_fixtures.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_eval_judge.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_eval_run.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_language.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_middleware.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_print_stdout.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_process_exception_split.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_signals.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_throttle.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_tracebacks.py +0 -0
- {django_explain_errors-0.10.0 → django_explain_errors-0.11.0}/tests/test_truncation.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: django-explain-errors
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.11.0
|
|
4
4
|
Summary: Django middleware that explains unhandled exceptions in DEBUG using an LLM, optionally grounded in your own project source via a local vector index. Works with OpenAI, Claude, or any OpenAI-compatible endpoint including local models.
|
|
5
5
|
Home-page: https://github.com/topunix/django-explain-errors
|
|
6
6
|
Author: topunix
|
|
@@ -35,6 +35,8 @@ Requires-Dist: python-dotenv>=1.0
|
|
|
35
35
|
Requires-Dist: asgiref>=3.6
|
|
36
36
|
Provides-Extra: rag
|
|
37
37
|
Requires-Dist: sqlite-vec>=0.1.0; extra == "rag"
|
|
38
|
+
Provides-Extra: local-embeddings
|
|
39
|
+
Requires-Dist: sentence-transformers>=2.2; extra == "local-embeddings"
|
|
38
40
|
Dynamic: author
|
|
39
41
|
Dynamic: author-email
|
|
40
42
|
Dynamic: classifier
|
|
@@ -91,20 +93,22 @@ better. This package is not trying to compete there.
|
|
|
91
93
|
- Captures Django errors and exceptions
|
|
92
94
|
- Shows the explanation on Django's debug page, directly under the
|
|
93
95
|
exception headline (`EXPLAIN_ERRORS_INJECT_DEBUG_PAGE`, on by default)
|
|
96
|
+
- Codebase-aware explanations via RAG over a local sqlite-vec index
|
|
97
|
+
(recommended; see "Codebase-aware explanations" below)
|
|
98
|
+
- Local embeddings via an optional extra: your codebase is indexed on your machine and never uploaded to an embeddings provider (only the traceback and a few relevant snippets reach your chat model; see [Privacy](#privacy))
|
|
99
|
+
- Separate embeddings provider config, so Anthropic users can use RAG
|
|
100
|
+
- Explains errors using OpenAI out of the box, Anthropic's Claude models, or any other
|
|
101
|
+
OpenAI-compatible endpoint (Ollama, LM Studio, Azure, gateways) via `OPENAI_BASE_URL`,
|
|
102
|
+
including fully local models, so no code has to leave your machine
|
|
103
|
+
- Explanations in your language via `EXPLAIN_ERRORS_LANGUAGE`, with exception names,
|
|
104
|
+
identifiers, and code kept in English
|
|
94
105
|
- Copy button on each code block in the explanation, so a suggested fix
|
|
95
106
|
copies in one click
|
|
96
107
|
- Prints the explanation to stdout, for terminal workflows and logs
|
|
97
108
|
(`EXPLAIN_ERRORS_PRINT_STDOUT`, on by default)
|
|
98
109
|
- Optional JSON 500 response instead of the debug page
|
|
99
110
|
(`EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE = False`)
|
|
100
|
-
-
|
|
101
|
-
OpenAI-compatible endpoint (Ollama, LM Studio, Azure, gateways) via `OPENAI_BASE_URL`,
|
|
102
|
-
including fully local models, so no code has to leave your machine
|
|
103
|
-
- Codebase-aware explanations via RAG over a local sqlite-vec index
|
|
104
|
-
(recommended; see "Codebase-aware explanations" below)
|
|
105
|
-
- Explanations in your language via `EXPLAIN_ERRORS_LANGUAGE`, with exception names,
|
|
106
|
-
identifiers, and code kept in English
|
|
107
|
-
- Redacts secrets, tokens, and emails from tracebacks before sending
|
|
111
|
+
- Redacts common secret patterns, tokens, and emails (see [Privacy](#privacy))
|
|
108
112
|
- Rate limits API calls with a configurable sliding window
|
|
109
113
|
- Works with both sync (WSGI) and async (ASGI) views
|
|
110
114
|
- Manages the API key using environment variables
|
|
@@ -153,8 +157,9 @@ pip install django-explain-errors
|
|
|
153
157
|
## Production Safety
|
|
154
158
|
|
|
155
159
|
This middleware is intended for local development only. When active, it sends exception
|
|
156
|
-
tracebacks to the configured LLM provider, which
|
|
157
|
-
|
|
160
|
+
tracebacks to the configured LLM provider, which include file paths, the source line of each
|
|
161
|
+
frame, and the exception message (which can contain values). Local variables are not sent.
|
|
162
|
+
Keep it out of production.
|
|
158
163
|
|
|
159
164
|
**Primary safeguard:** register the middleware only when `DEBUG` is on, or only in your
|
|
160
165
|
development settings module:
|
|
@@ -195,15 +200,6 @@ python manage.py check --deploy --fail-level WARNING
|
|
|
195
200
|
The view raised a ValueError because ...
|
|
196
201
|
```
|
|
197
202
|
|
|
198
|
-
## Async Support
|
|
199
|
-
|
|
200
|
-
The middleware exposes both `sync_capable = True` and `async_capable = True`. At initialization it inspects `get_response` to decide whether it is part of a sync or async chain:
|
|
201
|
-
|
|
202
|
-
- Under WSGI (for example `runserver` with sync views), requests flow through the synchronous handler.
|
|
203
|
-
- Under ASGI (for example with async views), requests are awaited through the async handler. The blocking model API call is offloaded with `asgiref.sync.sync_to_async` so the event loop is not blocked.
|
|
204
|
-
|
|
205
|
-
No additional settings are needed. See Installation above for where to place the middleware in `MIDDLEWARE`.
|
|
206
|
-
|
|
207
203
|
## Debug page injection
|
|
208
204
|
|
|
209
205
|
When Django renders its debug page for a 500, `explain_errors` also inserts a banner
|
|
@@ -243,6 +239,15 @@ what Django's debug page looks like: a new section appears above the request met
|
|
|
243
239
|
table. If you rely on the debug page's exact markup (a scraper, a screenshot test), account
|
|
244
240
|
for this.
|
|
245
241
|
|
|
242
|
+
## Async Support
|
|
243
|
+
|
|
244
|
+
The middleware exposes both `sync_capable = True` and `async_capable = True`. At initialization it inspects `get_response` to decide whether it is part of a sync or async chain:
|
|
245
|
+
|
|
246
|
+
- Under WSGI (for example `runserver` with sync views), requests flow through the synchronous handler.
|
|
247
|
+
- Under ASGI (for example with async views), requests are awaited through the async handler. The blocking model API call is offloaded with `asgiref.sync.sync_to_async` so the event loop is not blocked.
|
|
248
|
+
|
|
249
|
+
No additional settings are needed. See Installation above for where to place the middleware in `MIDDLEWARE`.
|
|
250
|
+
|
|
246
251
|
## Compatibility
|
|
247
252
|
|
|
248
253
|
How this middleware interacts with other error-handling and debugging tools, in the default
|
|
@@ -250,16 +255,14 @@ preserve mode and with `EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE=False`:
|
|
|
250
255
|
|
|
251
256
|
| Package | Preserve mode (default) | `EXPLAIN_ERRORS_PRESERVE_DEBUG_PAGE=False` |
|
|
252
257
|
| ------- | ------------------------ | ------------------------------------------- |
|
|
253
|
-
| Django Debug Toolbar | Works
|
|
258
|
+
| Django Debug Toolbar | Works. Django renders its debug page (with the explanation banner, unless `EXPLAIN_ERRORS_INJECT_DEBUG_PAGE = False`), and Debug Toolbar injects into it. Verified with the banner present, with the toolbar middleware before or after this one, on WSGI and ASGI. | Broken. The JSON 500 response has no HTML to inject into. |
|
|
254
259
|
| Sentry, Rollbar | Work — the exception propagates and Django re-raises it, so `got_request_exception` fires. | Broken — returning a response ends exception handling before `got_request_exception` fires. |
|
|
255
260
|
| Django REST Framework | Partial — only exceptions DRF does not already handle itself reach this middleware. | Partial, same reason. |
|
|
256
261
|
| Silk and other profiling panels | Timings are inflated by the model API call, since `process_exception` blocks the request path. | Same. |
|
|
257
262
|
| CORS, GZip, WhiteNoise | No interaction. | No interaction. |
|
|
258
263
|
| `runserver_plus` / Werkzeug debugger | Works — returning `None` re-raises the original exception, and `django-extensions` replaces Django's debug-page renderer with one that re-raises instead, so Werkzeug's WSGI wrapper catches it and shows the interactive debugger. | Broken — the JSON 500 response ends exception handling before it reaches `runserver_plus`'s exception hook, so the Werkzeug debugger never appears. |
|
|
259
264
|
|
|
260
|
-
Debug Toolbar, Sentry, and Werkzeug were verified empirically, on both Django 4.2 and Django
|
|
261
|
-
6.1, in both modes. DRF, Silk, and CORS/GZip/WhiteNoise are reasoned from the mechanism
|
|
262
|
-
rather than tested.
|
|
265
|
+
Debug Toolbar, Sentry, and Werkzeug were verified empirically, on both Django 4.2 and Django 6.1, in both modes. The Debug Toolbar check was repeated with the explanation banner present (default preserve mode) on Django 4.2 and 6.1, under WSGI and ASGI, for sync and async views, with the toolbar middleware before and after this one. Both orderings work. Visual layout and toolbar JavaScript behavior were not checked. DRF, Silk, and CORS/GZip/WhiteNoise are reasoned from the mechanism rather than tested.
|
|
263
266
|
|
|
264
267
|
One additional behavior worth knowing, also verified on both Django versions: when the
|
|
265
268
|
explanation call itself fails (a bad key, a timeout, an unreachable endpoint), a
|
|
@@ -285,8 +288,8 @@ developer is actually investigating.
|
|
|
285
288
|
| `OPENAI_BASE_URL` (env or settings) | No | Base URL for any OpenAI-compatible API (for example Ollama at `http://localhost:11434/v1`). When set, a missing API key is replaced with a placeholder since local servers do not require one. |
|
|
286
289
|
| `EXPLAIN_ERRORS_MAX_CALLS` | No | Together with `EXPLAIN_ERRORS_WINDOW_SECONDS`, caps API spend to at most this many explanations within a rolling window; once the cap is hit, further errors in that window are not sent for explanation until an earlier call ages out. Defaults to `5`. |
|
|
287
290
|
| `EXPLAIN_ERRORS_WINDOW_SECONDS` | No | Length in seconds of the rolling window `EXPLAIN_ERRORS_MAX_CALLS` is measured against. Defaults to `60` (with the defaults, at most 5 explanations per 60-second window). |
|
|
288
|
-
| `EXPLAIN_ERRORS_REDACT_PATTERNS` | No | Extra regex pattern strings (each passed to `re.compile`) applied to the traceback, appended after the built-in secret/token/email patterns. An invalid pattern is skipped with a warning rather than raising. Defaults to `[]`. |
|
|
289
|
-
| `EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS` | No | When `True`, skips the built-in secret/token/email redaction patterns entirely and redacts only what `EXPLAIN_ERRORS_REDACT_PATTERNS` specifies. Turning this on removes the default protection against leaking secrets and PII in tracebacks. Defaults to `False`. |
|
|
291
|
+
| `EXPLAIN_ERRORS_REDACT_PATTERNS` | No | Extra regex pattern strings (each passed to `re.compile`) applied to the traceback, the RAG query, and indexed chunks (see [Redaction](#redaction)), appended after the built-in secret/token/email patterns. An invalid pattern is skipped with a warning rather than raising. Defaults to `[]`. |
|
|
292
|
+
| `EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS` | No | When `True`, skips the built-in secret/token/email redaction patterns entirely and redacts only what `EXPLAIN_ERRORS_REDACT_PATTERNS` specifies. Turning this on removes the default protection against leaking secrets and PII in tracebacks, RAG queries, and indexed chunks (see [Redaction](#redaction)). Defaults to `False`. |
|
|
290
293
|
| `EXPLAIN_ERRORS_REDACT_REPLACEMENT` | No | Replacement string substituted for anything matched by the redaction patterns. Defaults to `"[REDACTED]"`. |
|
|
291
294
|
| `EXPLAIN_ERRORS_LANGUAGE` | No | Language the explanation prose is written in, as a plain name or code (for example `"Spanish"` or `"es"`). Defaults to `None`, meaning English. Exception names, identifiers, code, file paths, and tracebacks always stay in English regardless of this setting. |
|
|
292
295
|
| `EXPLAIN_ERRORS_DEDUP` | No | When `True` (the default), a repeat of an error already explained in this process replays the earlier explanation instead of calling the model or using a throttle slot. Two errors are identical when they have the same exception type, the same exception message, and the same innermost project line (file, line number, and that line's source text); editing the line makes it a new error. Keeps the last 128 distinct errors per worker (least recently used entries are evicted first) and entries never expire on a timer. Set to `False` to explain every error. |
|
|
@@ -334,6 +337,59 @@ Reasoning models behind `OPENAI_BASE_URL` spend part of the token budget on inte
|
|
|
334
337
|
before producing visible output. At a low `OPENAI_MAX_TOKENS`, the budget can be used up by
|
|
335
338
|
reasoning alone, and the explanation comes back empty. Raise `OPENAI_MAX_TOKENS` if you see this.
|
|
336
339
|
|
|
340
|
+
### RAG does not work with Anthropic alone
|
|
341
|
+
|
|
342
|
+
The [RAG layer](#codebase-aware-explanations-rag) needs an embeddings endpoint, and
|
|
343
|
+
Anthropic's OpenAI-compatible API has none: `POST /v1/embeddings` returns HTTP 404
|
|
344
|
+
(`not_found_error`), and Anthropic does not offer its own embedding model. With only the
|
|
345
|
+
config above, `build_error_index` fails with an error saying so, and retrieval at request
|
|
346
|
+
time falls back to the traceback-only prompt.
|
|
347
|
+
|
|
348
|
+
To use RAG with Claude, keep Anthropic for chat and point embeddings at an OpenAI-compatible
|
|
349
|
+
provider with the `EXPLAIN_ERRORS_RAG_EMBED_*` settings:
|
|
350
|
+
|
|
351
|
+
```python
|
|
352
|
+
# Chat: Anthropic
|
|
353
|
+
OPENAI_BASE_URL = "https://api.anthropic.com/v1/"
|
|
354
|
+
OPENAI_API_KEY = "your_anthropic_api_key_here"
|
|
355
|
+
OPENAI_MODEL = "..."
|
|
356
|
+
|
|
357
|
+
# Embeddings: any OpenAI-compatible embeddings provider (OpenAI shown)
|
|
358
|
+
EXPLAIN_ERRORS_RAG_ENABLED = True
|
|
359
|
+
EXPLAIN_ERRORS_RAG_EMBED_BASE_URL = "https://api.openai.com/v1/"
|
|
360
|
+
EXPLAIN_ERRORS_RAG_EMBED_API_KEY = "your_openai_api_key_here"
|
|
361
|
+
EXPLAIN_ERRORS_RAG_EMBED_MODEL = "text-embedding-3-small"
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
Each of the three settings falls back to the chat value when unset, with one exception for
|
|
365
|
+
safety: if `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` differs from the chat base URL,
|
|
366
|
+
`EXPLAIN_ERRORS_RAG_EMBED_API_KEY` is required, and `ImproperlyConfigured` is raised without
|
|
367
|
+
it. The chat key is never sent to a different provider. A local OpenAI-compatible
|
|
368
|
+
embeddings server that ignores keys still needs a dummy value, for example
|
|
369
|
+
`"sk-no-key-required"`.
|
|
370
|
+
|
|
371
|
+
The embed settings are also read from environment variables of the same name, which take
|
|
372
|
+
precedence over Django settings. Indexed source chunks are sent to the embeddings provider
|
|
373
|
+
(after redaction), not to Anthropic.
|
|
374
|
+
|
|
375
|
+
### No second key: local embeddings
|
|
376
|
+
|
|
377
|
+
If you would rather not use a second provider, run the embeddings on your own machine
|
|
378
|
+
instead. Chat stays on Anthropic and no embeddings API key is needed:
|
|
379
|
+
|
|
380
|
+
```python
|
|
381
|
+
# Chat: Anthropic
|
|
382
|
+
OPENAI_BASE_URL = "https://api.anthropic.com/v1/"
|
|
383
|
+
OPENAI_API_KEY = "your_anthropic_api_key_here"
|
|
384
|
+
OPENAI_MODEL = "..."
|
|
385
|
+
|
|
386
|
+
# Embeddings: on this machine
|
|
387
|
+
EXPLAIN_ERRORS_RAG_ENABLED = True
|
|
388
|
+
EXPLAIN_ERRORS_RAG_EMBED_BACKEND = "local"
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
See [Local embeddings](#local-embeddings) for the install command and what to expect.
|
|
392
|
+
|
|
337
393
|
### A note on API keys and 401s
|
|
338
394
|
|
|
339
395
|
`explain_errors` reads `OPENAI_API_KEY` (see Configuration above); it does not read
|
|
@@ -345,6 +401,133 @@ and rejected, so a missing `OPENAI_API_KEY` shows up as an opaque `401 Unauthori
|
|
|
345
401
|
than a clear configuration error. If you see a 401 with `OPENAI_BASE_URL` pointed at a remote
|
|
346
402
|
provider, check that `OPENAI_API_KEY` — not a provider-specific variable — is actually set.
|
|
347
403
|
|
|
404
|
+
## Privacy
|
|
405
|
+
|
|
406
|
+
This section describes what leaves your machine, when, and to which endpoint. It describes
|
|
407
|
+
this package's behavior only. It makes no claims about what a provider does with data after
|
|
408
|
+
receiving it; check your provider's own terms.
|
|
409
|
+
|
|
410
|
+
### What is sent to the chat provider
|
|
411
|
+
|
|
412
|
+
On each error that triggers an explanation, one request goes to the endpoint set by
|
|
413
|
+
`OPENAI_BASE_URL` (OpenAI by default). Its contents:
|
|
414
|
+
|
|
415
|
+
- A fixed system prompt, plus the language name if `EXPLAIN_ERRORS_LANGUAGE` is set.
|
|
416
|
+
- The traceback: exception type and message, and for each frame the file path, line number,
|
|
417
|
+
and source line. Library frames are condensed to fit `OPENAI_MAX_TRACEBACK_CHARS`
|
|
418
|
+
(default 3000), and the result is redacted. Local variable values are not included, but the
|
|
419
|
+
exception message can contain values.
|
|
420
|
+
- With RAG enabled, up to `EXPLAIN_ERRORS_RAG_TOP_K` (default 4) retrieved code chunks, each
|
|
421
|
+
under a header with its file path and line range. The traceback and chunks together are
|
|
422
|
+
capped at `EXPLAIN_ERRORS_RAG_MAX_PROMPT_CHARS` (default 6000).
|
|
423
|
+
- The model name and token limit as request parameters, and your API key as the credential.
|
|
424
|
+
|
|
425
|
+
Nothing else is added: not the request path, headers, body, session, or user, and not your
|
|
426
|
+
Django settings. The request object is never passed to the prompt builder.
|
|
427
|
+
|
|
428
|
+
Not every error causes a request. A repeat of an already explained error replays the cached
|
|
429
|
+
explanation (`EXPLAIN_ERRORS_DEDUP`, on by default), and calls are limited per worker process
|
|
430
|
+
to 5 per 60 seconds by default.
|
|
431
|
+
|
|
432
|
+
### What is sent to the embeddings provider
|
|
433
|
+
|
|
434
|
+
Only when RAG is enabled and the embeddings backend is not `local`:
|
|
435
|
+
|
|
436
|
+
- **When you run `build_error_index`:** every chunk of every `.py`, `.html`, and `.txt` file
|
|
437
|
+
under `EXPLAIN_ERRORS_RAG_INCLUDE` (default: `BASE_DIR`), after redaction. Directories named
|
|
438
|
+
in `EXPLAIN_ERRORS_RAG_EXCLUDE` are skipped. The default is `migrations`, `venv`, `.venv`,
|
|
439
|
+
`env`, `node_modules`, `static`, `media`, and `.git`. Setting it replaces that list rather
|
|
440
|
+
than extending it, and it matches directory names only, not individual files. File paths
|
|
441
|
+
are not part of the embedded text.
|
|
442
|
+
- **On each error (RAG enabled):** a query made of the exception type and message plus about
|
|
443
|
+
five lines of source on each side of the innermost project frame, after redaction.
|
|
444
|
+
|
|
445
|
+
The index file itself stays on disk. It holds the redacted chunk text.
|
|
446
|
+
|
|
447
|
+
### Summary by setup
|
|
448
|
+
|
|
449
|
+
| Setup | Indexing (your codebase) | Each error (traceback + snippets) |
|
|
450
|
+
|---|---|---|
|
|
451
|
+
| OpenAI chat + OpenAI embeddings | Redacted chunks go to OpenAI embeddings | Query goes to OpenAI embeddings; redacted traceback and retrieved snippets go to OpenAI chat |
|
|
452
|
+
| Anthropic chat + separate embeddings provider | Redacted chunks go to the embeddings provider only, not to Anthropic | Query goes to the embeddings provider; redacted traceback and retrieved snippets go to Anthropic |
|
|
453
|
+
| Any chat + local embeddings | Stays on your machine | Query is embedded locally; redacted traceback and retrieved snippets go to your chat provider |
|
|
454
|
+
| Local chat (e.g. Ollama) + local embeddings | Nothing leaves the machine* | Nothing leaves the machine* |
|
|
455
|
+
| No RAG | Not applicable, no index is built | Redacted traceback goes to your chat provider |
|
|
456
|
+
|
|
457
|
+
\* Assumes the chat endpoint is genuinely local; the package sends to whatever
|
|
458
|
+
`OPENAI_BASE_URL` points at. The local embedding model is downloaded on first use (see
|
|
459
|
+
[Local embeddings](#local-embeddings)), which transfers the model to you and none of your code.
|
|
460
|
+
|
|
461
|
+
### Redaction
|
|
462
|
+
|
|
463
|
+
Before any text is sent or stored, it passes through a regex-based redactor. By default it
|
|
464
|
+
replaces the following with `[REDACTED]` (`EXPLAIN_ERRORS_REDACT_REPLACEMENT`):
|
|
465
|
+
|
|
466
|
+
- **Secret-like assignments.** A name that starts with `secret`, `password`, `passwd`,
|
|
467
|
+
`token`, `api_key` (or `apikey`), or `auth`, followed by `=` or `:`, together with its
|
|
468
|
+
value. The name may continue after the keyword with `_` or `-` and more word characters,
|
|
469
|
+
or with a capitalized suffix, so `SECRET_KEY`, `AWS_SECRET_ACCESS_KEY`, `secret-key`, and
|
|
470
|
+
`apiKeyId` all match. A quote after the name is allowed, so dict keys such as
|
|
471
|
+
`'PASSWORD': 'x'` match. A quoted value is redacted whole, spaces included. Text before the
|
|
472
|
+
keyword stays: `AWS_SECRET_ACCESS_KEY = "x"` becomes `AWS_[REDACTED]`.
|
|
473
|
+
- **Credentials in URLs.** In `scheme://user:pw@host`, the `user:pw` part. The scheme and
|
|
474
|
+
host stay.
|
|
475
|
+
- **Known key shapes**, whatever the variable is called: `sk-` (OpenAI, and Anthropic's
|
|
476
|
+
`sk-ant-`), Stripe `sk_live_` and `sk_test_`, GitHub `ghp_`, `gho_`, `ghu_`, `ghs_`,
|
|
477
|
+
`ghr_`, and `github_pat_`, Slack `xoxb-`, `xoxa-`, `xoxp-`, `xoxr-`, and `xoxs-`, AWS
|
|
478
|
+
`AKIA`, and Google `AIza`. Each needs a minimum run of characters after the prefix (for
|
|
479
|
+
example 20 after `sk-`), so short strings do not match.
|
|
480
|
+
- `Authorization` header values and `Bearer` tokens.
|
|
481
|
+
- Email addresses.
|
|
482
|
+
|
|
483
|
+
Five Django settings start with a keyword but hold no secret, so they are never redacted:
|
|
484
|
+
`AUTH_USER_MODEL`, `AUTHENTICATION_BACKENDS`, `AUTH_PASSWORD_VALIDATORS`, `PASSWORD_HASHERS`,
|
|
485
|
+
and `PASSWORD_RESET_TIMEOUT`. The match is on the exact name.
|
|
486
|
+
|
|
487
|
+
It is applied to the traceback (after truncation), to the RAG query text, and to every chunk
|
|
488
|
+
at index time, before the chunk is embedded or written to the index. It is not applied to file
|
|
489
|
+
paths, which appear in tracebacks and in chunk headers.
|
|
490
|
+
|
|
491
|
+
Add your own regexes with `EXPLAIN_ERRORS_REDACT_PATTERNS`. They run after the defaults.
|
|
492
|
+
`EXPLAIN_ERRORS_REDACT_DISABLE_DEFAULTS = True` turns the defaults off.
|
|
493
|
+
|
|
494
|
+
**Limits.** This is pattern matching, not a secret scanner. Treat it as a safety net.
|
|
495
|
+
|
|
496
|
+
- A secret is only caught if its name contains one of the keywords above or its value has a
|
|
497
|
+
known shape. A bare `key`, `private_key`, or `credentials` is not redacted, so
|
|
498
|
+
`private_key = "abc123"` passes through. A bare `key` is left alone on purpose, because redacting it
|
|
499
|
+
would also hide ordinary code such as dict keys and cache keys.
|
|
500
|
+
- Keys with an unknown prefix, or shorter than the length each shape requires, are not
|
|
501
|
+
caught unless the name matches.
|
|
502
|
+
- An unquoted value with spaces is redacted up to the first space only, and a value that
|
|
503
|
+
spans lines (a triple-quoted string, a PEM block) is not redacted past its first line.
|
|
504
|
+
- Some harmless names are redacted too. Any name that starts with a keyword followed by `_`,
|
|
505
|
+
`-`, or a capital letter matches, so `token_count` and `PASSWORD_FIELD` lose their values
|
|
506
|
+
even though they are not secrets. Names that only begin with the same letters, such as
|
|
507
|
+
`author`, `authors`, and `tokenizer`, are left alone.
|
|
508
|
+
- Names, IP addresses, phone numbers, other personal data, and file paths (which can contain
|
|
509
|
+
your username) are not redacted. Neither are values inside exception messages unless a
|
|
510
|
+
pattern happens to match them.
|
|
511
|
+
- `settings.py` normally sits under `BASE_DIR`, so it is indexed by default. Its
|
|
512
|
+
`SECRET_KEY` and `DATABASES` password are caught, but a secret stored under a name with no
|
|
513
|
+
keyword may not be. Add a pattern for your own naming, for example
|
|
514
|
+
`EXPLAIN_ERRORS_REDACT_PATTERNS = [r"(?i)(?:private_key|credentials)\w*\s*=\s*\S+"]`, leave
|
|
515
|
+
the settings directory out of `EXPLAIN_ERRORS_RAG_INCLUDE`, or use local embeddings.
|
|
516
|
+
|
|
517
|
+
### DEBUG only
|
|
518
|
+
|
|
519
|
+
The middleware does nothing when `DEBUG` is `False`: no client is created and no request is
|
|
520
|
+
made. One thing is not tied to `DEBUG`: `manage.py build_error_index` is a command, so it
|
|
521
|
+
sends chunks to a non-local embeddings provider whenever you run it. For keeping the
|
|
522
|
+
middleware out of production, see [Production Safety](#production-safety).
|
|
523
|
+
|
|
524
|
+
### Telemetry
|
|
525
|
+
|
|
526
|
+
This package's own code contains no telemetry or analytics and makes no network requests
|
|
527
|
+
other than to the chat and embeddings endpoints you configure. Nothing is sent to the
|
|
528
|
+
maintainer. Its dependencies (`openai`, and optionally `sqlite-vec` and
|
|
529
|
+
`sentence-transformers`) are separate projects and are not covered by this statement.
|
|
530
|
+
|
|
348
531
|
## Explanation language
|
|
349
532
|
|
|
350
533
|
By default, explanations are written in English. Set `EXPLAIN_ERRORS_LANGUAGE` to
|
|
@@ -396,6 +579,11 @@ also retrieves the most relevant chunks of your own project's source code
|
|
|
396
579
|
from a local vector index and includes them in the prompt, so explanations
|
|
397
580
|
can reference your actual functions and classes instead of guessing at them.
|
|
398
581
|
|
|
582
|
+
RAG needs a provider with an embeddings endpoint. Anthropic's OpenAI-compatible API has
|
|
583
|
+
none, so Claude users must configure a separate embeddings provider or use
|
|
584
|
+
[local embeddings](#local-embeddings); see
|
|
585
|
+
[RAG does not work with Anthropic alone](#rag-does-not-work-with-anthropic-alone).
|
|
586
|
+
|
|
399
587
|
Recommended for most projects. It is opt-in only because setup takes three
|
|
400
588
|
steps (install the extra, build the index, enable it), and it adds no
|
|
401
589
|
dependencies or behavior until you do.
|
|
@@ -420,8 +608,8 @@ python manage.py build_error_index
|
|
|
420
608
|
```
|
|
421
609
|
|
|
422
610
|
This walks your project, chunks Python files by top-level function/class
|
|
423
|
-
(and other text files by fixed-size line windows), embeds each chunk with
|
|
424
|
-
|
|
611
|
+
(and other text files by fixed-size line windows), embeds each chunk through the configured embeddings provider (or on this machine with the
|
|
612
|
+
[local backend](#local-embeddings)), and writes them to a local index file. Re-run it
|
|
425
613
|
whenever your source changes meaningfully. Indexing is not automatic.
|
|
426
614
|
Rebuilding is idempotent: it builds into a temp file and atomically replaces
|
|
427
615
|
the previous index.
|
|
@@ -441,16 +629,15 @@ EXPLAIN_ERRORS_RAG_ENABLED = True
|
|
|
441
629
|
| `EXPLAIN_ERRORS_RAG_ENABLED` | `False` | Master switch for the RAG layer. |
|
|
442
630
|
| `EXPLAIN_ERRORS_RAG_INDEX_PATH` | `<BASE_DIR>/.explain_errors_index.db` | Path to the local vector index file. |
|
|
443
631
|
| `EXPLAIN_ERRORS_RAG_TOP_K` | `4` | Number of chunks retrieved and injected into the prompt. |
|
|
444
|
-
| `
|
|
632
|
+
| `EXPLAIN_ERRORS_RAG_EMBED_BACKEND` | `"openai"` | `"openai"` embeds through an OpenAI-compatible endpoint. `"local"` embeds on this machine with sentence-transformers and needs no API key (see [Local embeddings](#local-embeddings)). Also read from the environment, which takes precedence. |
|
|
633
|
+
| `EXPLAIN_ERRORS_RAG_EMBED_MODEL` | `"text-embedding-3-small"` (`"all-MiniLM-L6-v2"` with the `local` backend) | Embedding model used for indexing and retrieval. Must exist on the configured endpoint, or be a sentence-transformers model name for the `local` backend. |
|
|
634
|
+
| `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` | chat `OPENAI_BASE_URL` | Base URL of the embeddings provider. Ignored by the `local` backend. Also read from the environment. Set it when chat and embeddings use different providers (see the Anthropic section). |
|
|
635
|
+
| `EXPLAIN_ERRORS_RAG_EMBED_API_KEY` | chat `OPENAI_API_KEY` | Key for the embeddings provider. Ignored by the `local` backend. Also read from the environment. Required when the embed base URL differs from the chat one. |
|
|
445
636
|
| `EXPLAIN_ERRORS_RAG_INCLUDE` | `None` (defaults to `BASE_DIR`) | List of directories to index. |
|
|
446
637
|
| `EXPLAIN_ERRORS_RAG_EXCLUDE` | migrations, venvs, `node_modules`, static, media, `.git` | Directory names to skip while indexing. |
|
|
447
638
|
| `EXPLAIN_ERRORS_RAG_MAX_PROMPT_CHARS` | `6000` | Combined character budget for the traceback + retrieved source sections of the prompt. |
|
|
448
639
|
|
|
449
|
-
|
|
450
|
-
same `sanitize_traceback()` redaction used for tracebacks, which strips
|
|
451
|
-
patterns that look like secrets, tokens, and emails before anything is sent to the model API or written to the index. It's a pattern-based filter, not a
|
|
452
|
-
guarantee: it catches recognizable secret shapes, not arbitrary sensitive
|
|
453
|
-
data that doesn't match one.
|
|
640
|
+
Chunks and queries are redacted before they are sent or written to the index; see [Privacy](#privacy) for what is and isn't caught.
|
|
454
641
|
|
|
455
642
|
If RAG is enabled but the index is missing, `sqlite-vec` isn't installed, or
|
|
456
643
|
retrieval fails for any reason, the middleware logs a warning and falls back
|
|
@@ -458,6 +645,50 @@ to the traceback-only prompt. It never breaks error reporting.
|
|
|
458
645
|
|
|
459
646
|
RAG-grounded explanations tend to be longer than traceback-only ones. The default `OPENAI_MAX_TOKENS` already leaves generous headroom for this, but if you've lowered it, raise it back up when RAG is enabled so explanations are not truncated.
|
|
460
647
|
|
|
648
|
+
### Local embeddings
|
|
649
|
+
|
|
650
|
+
Embeddings can run on your own machine, with no embeddings provider and no second API key.
|
|
651
|
+
This also works with Anthropic for chat (see [Using Anthropic](#using-anthropic-claude-models)).
|
|
652
|
+
|
|
653
|
+
Install the extra alongside `rag`:
|
|
654
|
+
|
|
655
|
+
```bash
|
|
656
|
+
pip install django-explain-errors[rag,local-embeddings]
|
|
657
|
+
```
|
|
658
|
+
|
|
659
|
+
Then set one setting:
|
|
660
|
+
|
|
661
|
+
```python
|
|
662
|
+
EXPLAIN_ERRORS_RAG_ENABLED = True
|
|
663
|
+
EXPLAIN_ERRORS_RAG_EMBED_BACKEND = "local"
|
|
664
|
+
```
|
|
665
|
+
|
|
666
|
+
Then build the index as usual with `python manage.py build_error_index`.
|
|
667
|
+
|
|
668
|
+
What to expect:
|
|
669
|
+
|
|
670
|
+
- **Install size.** The extra pulls in `sentence-transformers` and PyTorch. With the default
|
|
671
|
+
PyTorch wheels on Linux that is roughly 1 to 2 GB; a CPU-only PyTorch build is much smaller.
|
|
672
|
+
The core install and the `rag` extra are unchanged.
|
|
673
|
+
- **First index build.** The default model, `all-MiniLM-L6-v2`, is downloaded the first time
|
|
674
|
+
it is used (about 90 MB), then cached by `sentence-transformers`. Set
|
|
675
|
+
`EXPLAIN_ERRORS_RAG_EMBED_MODEL` to use a different sentence-transformers model.
|
|
676
|
+
- **First explanation after server start.** The model is loaded once per process, so the first
|
|
677
|
+
explanation that uses RAG after the dev server starts pays a one-time load. Later ones do not.
|
|
678
|
+
- **Rebuild the index when you switch.** Vectors from different models are not comparable. The
|
|
679
|
+
index records its backend, model, and vector size. If the current settings do not match, the
|
|
680
|
+
middleware logs one warning per process telling you to rerun `build_error_index` and falls
|
|
681
|
+
back to the traceback-only prompt. It never queries a mismatched index. An index built before
|
|
682
|
+
this feature existed counts as an `openai` index.
|
|
683
|
+
- **A leftover OpenAI model name.** If `EXPLAIN_ERRORS_RAG_EMBED_MODEL` is still set to an
|
|
684
|
+
OpenAI model such as `text-embedding-3-small`, the `local` backend raises
|
|
685
|
+
`ImproperlyConfigured`. Unset it, or name a local model.
|
|
686
|
+
- `EXPLAIN_ERRORS_RAG_EMBED_BASE_URL` and `EXPLAIN_ERRORS_RAG_EMBED_API_KEY` are ignored by the
|
|
687
|
+
local backend.
|
|
688
|
+
|
|
689
|
+
**Privacy.** Local embeddings keep your source off any embeddings API, but the traceback and
|
|
690
|
+
retrieved snippets still go to your chat provider; see [Privacy](#privacy).
|
|
691
|
+
|
|
461
692
|
### .gitignore
|
|
462
693
|
|
|
463
694
|
The index file is a local build artifact, not something to commit. Add it
|