django-template-compiler 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,8 @@
1
+ {
2
+ "permissions": {
3
+ "allow": [
4
+ "Bash(git -C /home/sam/django-template-compiler show --stat HEAD)",
5
+ "Bash(python3 -c \"import django; print\\(django.__version__\\)\")"
6
+ ]
7
+ }
8
+ }
@@ -0,0 +1,63 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+
8
+ jobs:
9
+ test:
10
+ runs-on: ubuntu-latest
11
+ strategy:
12
+ fail-fast: false
13
+ matrix:
14
+ python-version: ["3.10", "3.11", "3.12", "3.13"]
15
+ django-version: ["4.2", "5.1", "5.2"]
16
+ steps:
17
+ - uses: actions/checkout@v5
18
+ - uses: actions/setup-python@v6
19
+ with:
20
+ python-version: ${{ matrix.python-version }}
21
+ - name: Install
22
+ run: |
23
+ pip install "django~=${{ matrix.django-version }}.0" pytest
24
+ pip install -e .
25
+ - name: Test
26
+ run: pytest -q
27
+
28
+ # Compatibility oracle: Django's own template_tests suite, run with dtc's
29
+ # compiled render path patched in (scripts/run_django_suite.py).
30
+ django-suite:
31
+ runs-on: ubuntu-latest
32
+ strategy:
33
+ fail-fast: false
34
+ matrix:
35
+ django-version: ["4.2", "5.1", "5.2"]
36
+ steps:
37
+ - uses: actions/checkout@v5
38
+ - uses: actions/setup-python@v6
39
+ with:
40
+ python-version: "3.12"
41
+ - name: Install
42
+ run: |
43
+ pip install "django~=${{ matrix.django-version }}.0"
44
+ pip install -e .
45
+ - name: Clone Django source at installed version
46
+ run: |
47
+ VERSION=$(python -c "import django; print(django.__version__)")
48
+ git clone --depth 1 --branch "$VERSION" https://github.com/django/django.git /tmp/django-src
49
+ - name: Run Django template_tests against dtc
50
+ run: python scripts/run_django_suite.py /tmp/django-src
51
+
52
+ build:
53
+ runs-on: ubuntu-latest
54
+ steps:
55
+ - uses: actions/checkout@v5
56
+ - uses: actions/setup-python@v6
57
+ with:
58
+ python-version: "3.12"
59
+ - name: Build and check distribution
60
+ run: |
61
+ pip install build twine
62
+ python -m build
63
+ twine check dist/*
@@ -0,0 +1,218 @@
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[codz]
4
+ *$py.class
5
+
6
+ # C extensions
7
+ *.so
8
+
9
+ # Distribution / packaging
10
+ .Python
11
+ build/
12
+ develop-eggs/
13
+ dist/
14
+ downloads/
15
+ eggs/
16
+ .eggs/
17
+ lib/
18
+ lib64/
19
+ parts/
20
+ sdist/
21
+ var/
22
+ wheels/
23
+ share/python-wheels/
24
+ *.egg-info/
25
+ .installed.cfg
26
+ *.egg
27
+ MANIFEST
28
+
29
+ # PyInstaller
30
+ # Usually these files are written by a python script from a template
31
+ # before PyInstaller builds the exe, so as to inject date/other infos into it.
32
+ *.manifest
33
+ *.spec
34
+
35
+ # Installer logs
36
+ pip-log.txt
37
+ pip-delete-this-directory.txt
38
+
39
+ # Unit test / coverage reports
40
+ htmlcov/
41
+ .tox/
42
+ .nox/
43
+ .coverage
44
+ .coverage.*
45
+ .cache
46
+ nosetests.xml
47
+ coverage.xml
48
+ *.cover
49
+ *.py.cover
50
+ .hypothesis/
51
+ .pytest_cache/
52
+ cover/
53
+
54
+ # Translations
55
+ *.mo
56
+ *.pot
57
+
58
+ # Django stuff:
59
+ *.log
60
+ local_settings.py
61
+ db.sqlite3
62
+ db.sqlite3-journal
63
+
64
+ # Flask stuff:
65
+ instance/
66
+ .webassets-cache
67
+
68
+ # Scrapy stuff:
69
+ .scrapy
70
+
71
+ # Sphinx documentation
72
+ docs/_build/
73
+
74
+ # PyBuilder
75
+ .pybuilder/
76
+ target/
77
+
78
+ # Jupyter Notebook
79
+ .ipynb_checkpoints
80
+
81
+ # IPython
82
+ profile_default/
83
+ ipython_config.py
84
+
85
+ # pyenv
86
+ # For a library or package, you might want to ignore these files since the code is
87
+ # intended to run in multiple environments; otherwise, check them in:
88
+ # .python-version
89
+
90
+ # pipenv
91
+ # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92
+ # However, in case of collaboration, if having platform-specific dependencies or dependencies
93
+ # having no cross-platform support, pipenv may install dependencies that don't work, or not
94
+ # install all needed dependencies.
95
+ # Pipfile.lock
96
+
97
+ # UV
98
+ # Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
99
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
100
+ # commonly ignored for libraries.
101
+ # uv.lock
102
+
103
+ # poetry
104
+ # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
106
+ # commonly ignored for libraries.
107
+ # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108
+ # poetry.lock
109
+ # poetry.toml
110
+
111
+ # pdm
112
+ # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
113
+ # pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
114
+ # https://pdm-project.org/en/latest/usage/project/#working-with-version-control
115
+ # pdm.lock
116
+ # pdm.toml
117
+ .pdm-python
118
+ .pdm-build/
119
+
120
+ # pixi
121
+ # Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
122
+ # pixi.lock
123
+ # Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
124
+ # in the .venv directory. It is recommended not to include this directory in version control.
125
+ .pixi
126
+
127
+ # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
128
+ __pypackages__/
129
+
130
+ # Celery stuff
131
+ celerybeat-schedule
132
+ celerybeat.pid
133
+
134
+ # Redis
135
+ *.rdb
136
+ *.aof
137
+ *.pid
138
+
139
+ # RabbitMQ
140
+ mnesia/
141
+ rabbitmq/
142
+ rabbitmq-data/
143
+
144
+ # ActiveMQ
145
+ activemq-data/
146
+
147
+ # SageMath parsed files
148
+ *.sage.py
149
+
150
+ # Environments
151
+ .env
152
+ .envrc
153
+ .venv
154
+ env/
155
+ venv/
156
+ ENV/
157
+ env.bak/
158
+ venv.bak/
159
+
160
+ # Spyder project settings
161
+ .spyderproject
162
+ .spyproject
163
+
164
+ # Rope project settings
165
+ .ropeproject
166
+
167
+ # mkdocs documentation
168
+ /site
169
+
170
+ # mypy
171
+ .mypy_cache/
172
+ .dmypy.json
173
+ dmypy.json
174
+
175
+ # Pyre type checker
176
+ .pyre/
177
+
178
+ # pytype static type analyzer
179
+ .pytype/
180
+
181
+ # Cython debug symbols
182
+ cython_debug/
183
+
184
+ # PyCharm
185
+ # JetBrains specific template is maintained in a separate JetBrains.gitignore that can
186
+ # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
187
+ # and can be added to the global gitignore or merged into this file. For a more nuclear
188
+ # option (not recommended) you can uncomment the following to ignore the entire idea folder.
189
+ # .idea/
190
+
191
+ # Abstra
192
+ # Abstra is an AI-powered process automation framework.
193
+ # Ignore directories containing user credentials, local state, and settings.
194
+ # Learn more at https://abstra.io/docs
195
+ .abstra/
196
+
197
+ # Visual Studio Code
198
+ # Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
199
+ # that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
200
+ # and can be added to the global gitignore or merged into this file. However, if you prefer,
201
+ # you could uncomment the following to ignore the entire vscode folder
202
+ # .vscode/
203
+ # Temporary file for partial code execution
204
+ tempCodeRunnerFile.py
205
+
206
+ # Ruff stuff:
207
+ .ruff_cache/
208
+
209
+ # PyPI configuration file
210
+ .pypirc
211
+
212
+ # Marimo
213
+ marimo/_static/
214
+ marimo/_lsp/
215
+ __marimo__/
216
+
217
+ # Streamlit
218
+ .streamlit/secrets.toml
@@ -0,0 +1,58 @@
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ ## Project goal
6
+
7
+ A drop-in replacement for Django's template engine that compiles templates to Python for speed, with **100% compatibility** as a hard requirement — including third-party custom tags and filters. Output must be byte-identical to Django's stock engine in all cases. When compatibility and speed conflict, compatibility wins: anything the compiler can't handle correctly must fall back to Django's interpreted renderer rather than approximate it.
8
+
9
+ Work proceeded in phases — see `ROADMAP.md` for the full record, including where the design diverged from the plan. All eight phases' engineering is complete (scope locals, context flattening, int fast path, the disk cache, and the literal-include fast path included); what remains before 0.1.0 is release work: a trial against a real application and the PyPI release.
10
+
11
+ Correctness invariants to preserve when changing the compiler:
12
+ - Oracle suites run **strict** (`DTC_STRICT`): compiler errors fail CI rather than falling back. Keep it that way.
13
+ - Compiled shortcuts must honor a patched `Template._render` (test instrumentation, autopatch, third-party hooks) — see `runtime._render_is_patched`.
14
+ - Templates whose compiled function embeds bridged or identity-keyed-state nodes are non-shareable across parses (`__dtc_shareable__`).
15
+ - `tests/test_fuzz.py` fuzzes with a fresh seed every run; a CI fuzz failure is a real bug — reproduce with the printed `DTC_FUZZ_SEED`.
16
+
17
+ ## Naming
18
+
19
+ The PyPI distribution name is `django-template-compiler`; the import name is `dtc` (the bare name `dtc` is taken on PyPI). Source lives in `src/dtc/`.
20
+
21
+ ## Commands
22
+
23
+ ```bash
24
+ python3 -m venv .venv && .venv/bin/pip install -e '.[dev]' # one-time setup
25
+ .venv/bin/pytest # run tests
26
+ .venv/bin/pytest tests/test_backend.py -k differential # run a subset
27
+ .venv/bin/python -m build && .venv/bin/twine check dist/* # build + validate dists
28
+ .venv/bin/python benchmarks/bench.py # speed vs stock engine + cold-start report
29
+ .venv/bin/python scripts/run_django_suite.py <django-src> # Django's own template_tests vs dtc (strict)
30
+ DTC_FUZZ_ITERS=10000 .venv/bin/pytest tests/test_fuzz.py # extended differential fuzzing
31
+ ```
32
+
33
+ For the last command, `<django-src>` is a Django source checkout whose tag matches the installed version (the script verifies): `git clone --depth 1 --branch $(python -c 'import django; print(django.__version__)') https://github.com/django/django.git`. It works by installing `dtc.autopatch`, which patches `Template._render` engine-wide (and `django.test.utils.instrumented_test_render`, preserving the `template_rendered` signal) — that's what routes Django's engine-level test templates through the compiler. It prints compiled/fallback stats at exit; a run where `renders_compiled` is 0 proved nothing.
34
+
35
+ Version is single-sourced from `__version__` in `src/dtc/__init__.py` (hatchling dynamic version). CI (`.github/workflows/ci.yml`) tests Python 3.10–3.13 × Django 4.2/5.1/5.2 — code must work across that whole matrix.
36
+
37
+ ## Architecture
38
+
39
+ The design is **hybrid: compile what we can, bridge what we can't.**
40
+
41
+ - Templates are parsed by Django's own `Lexer`/`Parser` (never reimplement parsing — reusing it is a deliberate compatibility decision). Only the render path is replaced.
42
+ - `src/dtc/backend.py` — `DTCTemplates`, a `BACKENDS` engine that subclasses Django's stock `DjangoTemplates` backend, inheriting all configuration/loader/OPTIONS handling unchanged (the one dtc-specific option, `dtc_disk_cache`, is popped before Django's `Engine` sees it). Its `Template` proxy renders through `runtime.template_render`, which takes the compiled path when available and otherwise Django's own `Template.render`; either way `Template.render`'s observable bookkeeping (`render_context.push_state`, `bind_template` — which runs context processors and exposes `context.template.engine`) is reproduced exactly.
43
+ - `src/dtc/compiler.py` — the compiler. Contract: `compile_template(template) -> callable(Context) -> str | None`, where the callable replaces `Template._render`. `None` now means only "debug engine" (by policy) or "internal compiler error" (fail-open, logged to logger `dtc`, counted in `stats["templates_error"]`; raises instead under `dtc.compiler.STRICT` / `DTC_STRICT=1`). **Every parseable template compiles.** Dedicated codegen covers text, variables (filters call the registered functions directly, with `is_safe`/`needs_autoescape`/`expects_localtime` specialization decided at compile time — custom filters compile natively), `{% if %}`, `{% for %}`, `{% with %}`, `{% autoescape %}`, `{% comment %}`, `{% verbatim %}`, `{% load %}`, inheritance (`{% block %}`/`{% extends %}`/`{% include %}`), `@simple_tag`, `@inclusion_tag`, and the container tags (`{% spaceless %}`, `{% filter %}`, `{% ifchanged %}`, `{% localize %}`, `{% localtime %}`, `{% timezone %}`, `{% language %}` — containers matter because bridging one forces its subtree to render interpreted); any other node bridges as `render_annotated(context)`, exact because compiled code performs *real* `context.push()/pop()/__setitem__` operations. Leaf tags (`url`, `csrf_token`, `static`, ...) stay bridged deliberately: their render is the work itself. Literal `{% include "name" %}` sites get a fast path: `construct_relative_path` folds at compile time, the target and its compiled function resolve once per top-level render (cached on `render_context`, the same lifetime as `IncludeNode.render`'s own per-node cache, so loader reload semantics match stock), and each call reproduces `IncludeNode.render` inline — real `context.push()`/`new()`, inlined `push_state` — before calling the target's compiled function directly; a patched `Template._render` (anything not in `runtime._transparent_renders`) or an uncompiled target falls back to the runtime mirror per call. Speed layers, each gated by the analysis pass: exact-type output fast paths (`str`/`SafeString`/`int`); the `forloop` dict elided when provably unreferenced (bridged nodes, blocks, non-isolated includes, and `takes_context` tags inside a loop disable elision — isolated `{% include ... only %}` renders against `context.new()` and doesn't — unseen content may reference `forloop`, and `IfChangedNode` uses the dict as its state frame); scope locals for loop/`with`-bound names (disabled per scope when its body contains an opaque bridge or a `takes_context` tag call, either of which can rebind names behind the locals); and a flattened read snapshot for names the unit never writes (disabled template-wide by any opaque bridge or `takes_context` tag, threshold-gated by weighted read count; the snapshot excludes the root context layer — `runtime.flatten_tail` — so a `context.dicts[0]` write from any nested template is never served stale: root-resolved reads miss and replay through the live walk). A tag declared context-safe (`dtc_context_safe` attribute / `dtc.declare_safe`) or writes-declared (`dtc_context_writes` naming the instance attributes that hold its written key names / `dtc.declare_writes`; both verified at render time under `DTC_CHECK_DECLARATIONS=1`) waives the write-opacity — it keeps the snapshot (declared writes join the unit's written set), scope locals (a shadowed local resyncs from the live context right after the bridge), and shareability; it still bridges via `render_annotated` and still forces `forloop` in loops (declarations don't enumerate reads); its `child_nodelists` are analyzed normally, so nested writers still gate. Compiled functions embedding bridged or identity-keyed-state nodes are marked non-shareable (`__dtc_shareable__`) — stateful nodes key state by node identity, so per-parse functions must stay per-parse (declared-safe bridges are exempt: the declaration promises no identity-keyed state).
44
+ - `src/dtc/diskcache.py` — opt-in cold-start cache (`OPTIONS["dtc_disk_cache"]` / `DTC_DISK_CACHE`): marshaled code objects keyed by SHA-256 of the generated source (self-validating — codegen still runs per process to rebuild the namespace from live parse objects; only `compile()` is skipped). Fail-open on every read; loaded code is exec'd, so the directory must be trusted.
45
+ - `src/dtc/runtime.py` — render-time support: `compiled_for()` (per-instance + per-(engine, name, source) compile caching — the latter prevents recompiling per render under non-cached loaders; only `__dtc_shareable__` functions enter the source cache), the pristine-`_render` check (`_render_is_patched`) that routes around the compiled path when anything has patched `Template._render` (plus `_transparent_renders`, the list of render functions the literal-include fast path may route around — pristine `_render` and autopatch's stats-only replacement), `resolve_include` (per-render resolution for literal include sites), the `stats` counters, and mirrors of `BlockNode`/`ExtendsNode`/`IncludeNode.render` with the template-render call made compiled-aware. Block bodies compile to standalone functions attached to BlockNodes as `_dtc_body`, linked at render time through Django's own `BlockContext`, so mixed compiled/interpreted inheritance chains work in both directions. The mirrors are verified byte-identical across Django 4.2–5.2; the CI oracle suites police upstream drift.
46
+ - `src/dtc/autopatch.py` — opt-in engine-wide hook (patches `Template._render`); used by the oracle suite runner. The supported integration is the backend.
47
+
48
+ The central exactness strategy (originally the "context escape hatch" problem, dissolved in phase 3): **the live `Context` is always maintained and always authoritative** — compiled code performs real `push()/pop()/__setitem__` operations, which is what makes bridging arbitrary third-party nodes exact. Scope locals and the flattened snapshot are read-only accelerations layered on top, gated off by analysis wherever an opaque node could write behind them. The gating is per-template, which leaves one documented compatibility boundary (README "Limitation"): a third-party tag that mutates an *intermediate* context layer (`context.dicts[i]` indexing, `Context.set_upward`, deletions, unbalanced push) from inside an included/extended template can stale an enclosing template's snapshot or scope locals — cross-template effects are assumed scope-limited or root-layer (the snapshot excludes `dicts[0]` so root writes stay exact). Same-template use of such tags is exact; `test_known_limitation_intermediate_layer_writes` pins the boundary. The hoisted `_autoescape` local is resynced after every site that hands the live context to foreign code (bridged renders, slow-path node replays, `takes_context` calls, the block/include mirrors) — such code may set `context.autoescape`, which stock rendering reads live. Generated fast paths inline only the provably-identical happy paths of `Variable._resolve_lookup`/`render_value_in_context`/`FilterExpression.resolve`; any deviation (lookup failure, callable, unusual type) replays through the *original node*, so every slow path is Django's own code. Preserve both halves of this when changing codegen.
49
+
50
+ ## Testing policy
51
+
52
+ Three oracle layers, all in CI:
53
+
54
+ 1. Differential tests: `tests/test_backend.py` and `tests/test_compiler.py` render each case through both dtc and Django's stock backend and assert identical output. Differential tests for compiled features must also assert `template._compiled is not None` — otherwise they silently compare Django with Django (and the test settings must keep `DEBUG=False`, since debug engines never compile). Any new compiler capability needs differential cases covering it, including edge semantics (autoescape/`mark_safe`, silent `VariableDoesNotExist`, context push/pop scoping, `forloop`/`parentloop`, `{{ block.super }}`).
55
+ 2. Django's own `template_tests` suite runs against dtc in CI on every supported Django version, in strict mode, via `scripts/run_django_suite.py` — it has caught real bugs every time coverage grew. The stats it prints must show a large `renders_compiled` and `templates_error: 0`.
56
+ 3. `tests/test_fuzz.py` — grammar-based differential fuzzer, fresh seed per run (`DTC_FUZZ_SEED=<printed seed>` to reproduce, `DTC_FUZZ_ITERS` to scale up; run ≥10k iterations locally after codegen changes).
57
+
58
+ `benchmarks/bench.py` measures speedup per scenario plus a cold-start report (parse vs. compile vs. disk-cache-warm); run it when touching codegen — it has caught performance regressions the tests can't (recompile-per-render, fast-path misses).
@@ -0,0 +1,28 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2026, Sam Tregar
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice, this
9
+ list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright notice,
12
+ this list of conditions and the following disclaimer in the documentation
13
+ and/or other materials provided with the distribution.
14
+
15
+ 3. Neither the name of the copyright holder nor the names of its
16
+ contributors may be used to endorse or promote products derived from
17
+ this software without specific prior written permission.
18
+
19
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,171 @@
1
+ Metadata-Version: 2.4
2
+ Name: django-template-compiler
3
+ Version: 0.0.1
4
+ Summary: A drop-in replacement for Django's template engine, 100% compatible including custom tags and filters, but much faster
5
+ Project-URL: Homepage, https://github.com/samtregar/django-template-compiler
6
+ Project-URL: Repository, https://github.com/samtregar/django-template-compiler
7
+ Project-URL: Issues, https://github.com/samtregar/django-template-compiler/issues
8
+ Author-email: Sam Tregar <sam@tregar.com>
9
+ License-Expression: BSD-3-Clause
10
+ License-File: LICENSE
11
+ Keywords: compiler,django,performance,templates
12
+ Classifier: Development Status :: 2 - Pre-Alpha
13
+ Classifier: Framework :: Django
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.10
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
22
+ Classifier: Topic :: Text Processing :: Markup :: HTML
23
+ Requires-Python: >=3.10
24
+ Requires-Dist: django>=4.2
25
+ Provides-Extra: dev
26
+ Requires-Dist: build; extra == 'dev'
27
+ Requires-Dist: pytest; extra == 'dev'
28
+ Requires-Dist: twine; extra == 'dev'
29
+ Description-Content-Type: text/markdown
30
+
31
+ # django-template-compiler
32
+
33
+ A drop-in replacement for Django's template engine, 100% compatible including custom tags and filters, but much faster.
34
+
35
+ **Status: pre-alpha, but substantially complete.** Every parseable template compiles: dedicated code generation for the core template language (variables, filters, control flow, inheritance, `simple_tag`/`inclusion_tag`, container tags), with anything else — arbitrary third-party tags included — running as-is against the live context. dtc passes Django's own template test suite (Django 4.2–5.2) in CI, plus a differential fuzzer. Typical speedups: 1.6–2.1x on template-bound rendering, with a ~1.0x floor when a template is dominated by bridged tags. Not yet exercised by production traffic — try it and report.
36
+
37
+ Two behaviors worth knowing:
38
+
39
+ - **`DEBUG=True` disables compilation** (per engine): Django's debug error page and exception annotation need the interpreted render path. Production configs get the compiled path; development keeps perfect debugging.
40
+ - **Django test instrumentation is honored**: when `setup_test_environment()` (the test runner / `assertTemplateUsed`) patches template rendering, dtc detects the patch and routes through it, so the `template_rendered` signal fires exactly as with stock Django.
41
+
42
+ ## How it works
43
+
44
+ Templates are parsed with Django's own lexer and parser, then compiled to Python code — a `{% for %}` loop becomes a real Python `for` loop, variable lookups become direct attribute/key access. Anything the compiler can't handle yet (including arbitrary custom tags) falls back to Django's interpreted render path, so output is always exactly what Django would produce.
45
+
46
+ ## Benchmarks
47
+
48
+ `benchmarks/bench.py`, Python 3.11, Django 5.2 (µs per render; higher speedup is better):
49
+
50
+ | scenario | django | dtc | speedup |
51
+ |---|---:|---:|---:|
52
+ | 40 plain variables | 52.2 | 28.6 | 1.8x |
53
+ | 100-row loop | 165.7 | 73.2 | 2.3x |
54
+ | 100-row loop with `forloop.counter` | 679.4 | 126.9 | **5.4x** |
55
+ | 50×4 table (nested loop + if) | 516.3 | 271.9 | 1.9x |
56
+ | with/if scopes | 206.8 | 91.2 | 2.3x |
57
+ | spaceless-wrapped table | 248.2 | 114.6 | 2.2x |
58
+ | inheritance + include in loop | 151.4 | 93.5 | 1.6x |
59
+ | bridged unknown tag (worst case) | 26.3 | 21.0 | 1.3x |
60
+
61
+ For reference, Jinja2 renders the table scenario in ~80µs — dtc closes about half the gap to Jinja2 while producing byte-identical Django output. The remaining distance is the price of Django's semantics themselves (silent variable failures, callable auto-invocation, the context stack), which dtc preserves exactly and Jinja2 deliberately dropped.
62
+
63
+ ## Installation
64
+
65
+ ```bash
66
+ pip install django-template-compiler
67
+ ```
68
+
69
+ The import name is `dtc`.
70
+
71
+ ## Usage
72
+
73
+ Change one line in your `TEMPLATES` setting:
74
+
75
+ ```python
76
+ TEMPLATES = [
77
+ {
78
+ "BACKEND": "dtc.backend.DTCTemplates", # was django.template.backends.django.DjangoTemplates
79
+ "DIRS": [BASE_DIR / "templates"],
80
+ "APP_DIRS": True,
81
+ "OPTIONS": {
82
+ # all DjangoTemplates options work unchanged
83
+ "context_processors": [...],
84
+ },
85
+ },
86
+ ]
87
+ ```
88
+
89
+ Everything else — template syntax, custom tag libraries, context processors, `{% load %}`, filters — works unchanged.
90
+
91
+ ### Cold starts and the disk cache
92
+
93
+ Compiling costs roughly 9x Django's parse per template, paid once per process. If your deployment restarts processes often (serverless, aggressive autoscaling), enable the disk cache, which persists compiled code objects across processes and cuts that overhead by ~70%:
94
+
95
+ ```python
96
+ "OPTIONS": {
97
+ "dtc_disk_cache": True, # ~/.cache/dtc/..., or pass an explicit path
98
+ },
99
+ ```
100
+
101
+ Cache entries are keyed by a hash of the generated code, so stale entries are impossible by construction; corrupt or version-mismatched entries are silently recompiled. Point it only at a directory you trust — cached code is executed.
102
+
103
+ ### Declaring custom tags context-safe
104
+
105
+ A custom tag without dedicated codegen renders through its own `render()` against the live context, which is always exact — but because the compiler can't see what that `render()` does, one such tag disables the read optimizations around it: the flattened read snapshot (template-wide), scope locals (in every enclosing `{% for %}`/`{% with %}`), and compiled-function sharing across template instances (which matters without a cached loader). `takes_context` simple/inclusion tags pay the first two as well.
106
+
107
+ Most tags never write the context. If yours is one of them, declare it:
108
+
109
+ ```python
110
+ class BreadcrumbNode(Node):
111
+ dtc_context_safe = True # stock Django ignores this; dtc keeps its
112
+ ... # optimizations around the tag
113
+
114
+ # takes_context tags declare the *function*:
115
+ @register.simple_tag(takes_context=True)
116
+ def current_section(context):
117
+ return context.get("section", "home")
118
+ current_section.dtc_context_safe = True
119
+
120
+ # third-party tags you can't edit, e.g. in settings or AppConfig.ready():
121
+ import dtc
122
+ dtc.declare_safe(SomeThirdPartyNode)
123
+ ```
124
+
125
+ The declaration is a promise about every `render()` call: the context stack and its mappings are left exactly as found (balanced push/pop inside is fine); no state keyed on the node's identity (Django's `CycleNode`/`IfChangedNode` pattern); behavior depends only on the parsed source. *Reading* the context is always fine, as is setting `context.autoescape`. A container tag may render nested writers freely, provided every nodelist it renders is listed in the standard `child_nodelists` attribute — the compiler analyzes those children itself; a rendered-but-unlisted nodelist is the one thing that can silently break output. Subclasses inherit the declaration with the `render()` it describes (`dtc_context_safe = False` opts back out). See `help(dtc.declare_safe)` for the precise contract.
126
+
127
+ Tags that *do* write the context can declare **what** they write instead, as long as the target names are fixed at parse time — the common capture/setter shape:
128
+
129
+ ```python
130
+ class CaptureNode(Node):
131
+ # names the instance attributes holding the written context keys
132
+ dtc_context_writes = ("target",)
133
+
134
+ def __init__(self, nodelist, target):
135
+ self.nodelist = nodelist
136
+ self.target = target # {% capture NAME %}...{% endcapture %}
137
+
138
+ def render(self, context):
139
+ context[self.target] = self.nodelist.render(context)
140
+ return ""
141
+
142
+ # or for classes you can't edit:
143
+ dtc.declare_writes(SomeVendorSetterNode, "dest")
144
+ ```
145
+
146
+ The compiler routes reads of the declared names through the live context and keeps every optimization on for everything else — including scope locals: if a declared write shadows a `{% for %}`/`{% with %}` name, the generated code re-reads that local right after the tag runs. The rest of the contract matches `dtc_context_safe`; the declared keys may be *set* only (no deletions), and an attribute holding `None` means an optional target unused at that site. See `help(dtc.declare_writes)`.
147
+
148
+ Declared writes may target the normal top-of-stack (`context[key] = value`) **or** the root layer (`context.dicts[0][key] = value` — the pattern used by tags that persist a value across template boundaries, past every scope pop). Root-written names are never served from the read snapshot (it excludes the root layer by design), so they stay exact across template boundaries, includes, and re-writes.
149
+
150
+ A wrong declaration produces wrong output silently — so verify it: run your test suite with `DTC_CHECK_DECLARATIONS=1` and dtc checks every declared render, raising `dtc.ContextSafeViolation` on any write outside the declaration. (Containers wrapping legitimate writers are skipped by the checker; the source-determinism clause isn't mechanically checkable.)
151
+
152
+ Tags that just compute a value from their arguments — a formatter, a calculator, a lookup — are better rewritten as `@register.simple_tag`: those compile natively, declaration-free, with argument resolution inlined.
153
+
154
+ ### Limitation: tags that rewrite enclosing context layers
155
+
156
+ Within a single template, *any* custom tag is rendered exactly — the compiler disables its read optimizations around every tag it doesn't recognize. Across template boundaries there is one assumption: a tag's context effects that outlive an `{% include %}`/`{% block %}`/`{% extends %}` are either **scope-limited** (ordinary `context[key] = value` writes and balanced push/pop, which die with the layers that the include/block machinery pops) or **root-layer** (`context.dicts[0][key] = value`, which dtc handles as described above). Every Django built-in and every `simple_tag`/`inclusion_tag` satisfies this.
157
+
158
+ A tag that mutates an *intermediate* layer of the caller's stack — indexing `context.dicts[1]`, calling `Context.set_upward()`, deleting keys from enclosing layers, or leaking an unbalanced `push()` — from inside an included or extended template **can produce output that differs from stock Django**: the enclosing template was compiled without knowledge of that tag, and its read snapshot or scope locals may serve the pre-mutation value. `DTC_CHECK_DECLARATIONS` cannot catch this (the tag carries no declaration, and the effect surfaces in a different template than the tag).
159
+
160
+ If you have such a tag, the supported paths are: write the root layer instead (`dicts[0]` — fully supported and declarable), write the top of the stack, or confine the mutation to the template that renders the tag. Note that intermediate-layer writes are fragile under stock Django too — what `dicts[1]` *is* depends on the stack depth at the call site.
161
+
162
+ ## Development
163
+
164
+ ```bash
165
+ pip install -e .[dev]
166
+ pytest
167
+ ```
168
+
169
+ ## License
170
+
171
+ BSD 3-Clause. See [LICENSE](LICENSE).