django-template-compiler 0.0.1__py3-none-any.whl

django_template_compiler-0.0.1.dist-info/METADATA

@@ -0,0 +1,171 @@
+Metadata-Version: 2.4
+Name: django-template-compiler
+Version: 0.0.1
+Summary: A drop-in replacement for Django's template engine, 100% compatible including custom tags and filters, but much faster
+Project-URL: Homepage, https://github.com/samtregar/django-template-compiler
+Project-URL: Repository, https://github.com/samtregar/django-template-compiler
+Project-URL: Issues, https://github.com/samtregar/django-template-compiler/issues
+Author-email: Sam Tregar <sam@tregar.com>
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Keywords: compiler,django,performance,templates
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Framework :: Django
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Classifier: Topic :: Text Processing :: Markup :: HTML
+Requires-Python: >=3.10
+Requires-Dist: django>=4.2
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: twine; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# django-template-compiler
+
+A drop-in replacement for Django's template engine, 100% compatible including custom tags and filters, but much faster.
+
+**Status: pre-alpha, but substantially complete.** Every parseable template compiles: dedicated code generation for the core template language (variables, filters, control flow, inheritance, `simple_tag`/`inclusion_tag`, container tags), with anything else — arbitrary third-party tags included — running as-is against the live context. dtc passes Django's own template test suite (Django 4.2–5.2) in CI, plus a differential fuzzer. Typical speedups: 1.6–2.1x on template-bound rendering, with a ~1.0x floor when a template is dominated by bridged tags. Not yet exercised by production traffic — try it and report.
+
+Two behaviors worth knowing:
+
+- **`DEBUG=True` disables compilation** (per engine): Django's debug error page and exception annotation need the interpreted render path. Production configs get the compiled path; development keeps perfect debugging.
+- **Django test instrumentation is honored**: when `setup_test_environment()` (the test runner / `assertTemplateUsed`) patches template rendering, dtc detects the patch and routes through it, so the `template_rendered` signal fires exactly as with stock Django.
+
+## How it works
+
+Templates are parsed with Django's own lexer and parser, then compiled to Python code — a `{% for %}` loop becomes a real Python `for` loop, variable lookups become direct attribute/key access. Anything the compiler can't handle yet (including arbitrary custom tags) falls back to Django's interpreted render path, so output is always exactly what Django would produce.
+
+## Benchmarks
+
+`benchmarks/bench.py`, Python 3.11, Django 5.2 (µs per render; higher speedup is better):
+
+| scenario | django | dtc | speedup |
+|---|---:|---:|---:|
+| 40 plain variables | 52.2 | 28.6 | 1.8x |
+| 100-row loop | 165.7 | 73.2 | 2.3x |
+| 100-row loop with `forloop.counter` | 679.4 | 126.9 | **5.4x** |
+| 50×4 table (nested loop + if) | 516.3 | 271.9 | 1.9x |
+| with/if scopes | 206.8 | 91.2 | 2.3x |
+| spaceless-wrapped table | 248.2 | 114.6 | 2.2x |
+| inheritance + include in loop | 151.4 | 93.5 | 1.6x |
+| bridged unknown tag (worst case) | 26.3 | 21.0 | 1.3x |
+
+For reference, Jinja2 renders the table scenario in ~80µs — dtc closes about half the gap to Jinja2 while producing byte-identical Django output. The remaining distance is the price of Django's semantics themselves (silent variable failures, callable auto-invocation, the context stack), which dtc preserves exactly and Jinja2 deliberately dropped.
+
+## Installation
+
+```bash
+pip install django-template-compiler
+```
+
+The import name is `dtc`.
+
+## Usage
+
+Change one line in your `TEMPLATES` setting:
+
+```python
+TEMPLATES = [
+    {
+        "BACKEND": "dtc.backend.DTCTemplates",  # was django.template.backends.django.DjangoTemplates
+        "DIRS": [BASE_DIR / "templates"],
+        "APP_DIRS": True,
+        "OPTIONS": {
+            # all DjangoTemplates options work unchanged
+            "context_processors": [...],
+        },
+    },
+]
+```
+
+Everything else — template syntax, custom tag libraries, context processors, `{% load %}`, filters — works unchanged.
+
+### Cold starts and the disk cache
+
+Compiling costs roughly 9x Django's parse per template, paid once per process. If your deployment restarts processes often (serverless, aggressive autoscaling), enable the disk cache, which persists compiled code objects across processes and cuts that overhead by ~70%:
+
+```python
+"OPTIONS": {
+    "dtc_disk_cache": True,  # ~/.cache/dtc/..., or pass an explicit path
+},
+```
+
+Cache entries are keyed by a hash of the generated code, so stale entries are impossible by construction; corrupt or version-mismatched entries are silently recompiled. Point it only at a directory you trust — cached code is executed.
+
+### Declaring custom tags context-safe
+
+A custom tag without dedicated codegen renders through its own `render()` against the live context, which is always exact — but because the compiler can't see what that `render()` does, one such tag disables the read optimizations around it: the flattened read snapshot (template-wide), scope locals (in every enclosing `{% for %}`/`{% with %}`), and compiled-function sharing across template instances (which matters without a cached loader). `takes_context` simple/inclusion tags pay the first two as well.
+
+Most tags never write the context. If yours is one of them, declare it:
+
+```python
+class BreadcrumbNode(Node):
+    dtc_context_safe = True   # stock Django ignores this; dtc keeps its
+    ...                       # optimizations around the tag
+
+# takes_context tags declare the *function*:
+@register.simple_tag(takes_context=True)
+def current_section(context):
+    return context.get("section", "home")
+current_section.dtc_context_safe = True
+
+# third-party tags you can't edit, e.g. in settings or AppConfig.ready():
+import dtc
+dtc.declare_safe(SomeThirdPartyNode)
+```
+
+The declaration is a promise about every `render()` call: the context stack and its mappings are left exactly as found (balanced push/pop inside is fine); no state keyed on the node's identity (Django's `CycleNode`/`IfChangedNode` pattern); behavior depends only on the parsed source. *Reading* the context is always fine, as is setting `context.autoescape`. A container tag may render nested writers freely, provided every nodelist it renders is listed in the standard `child_nodelists` attribute — the compiler analyzes those children itself; a rendered-but-unlisted nodelist is the one thing that can silently break output. Subclasses inherit the declaration with the `render()` it describes (`dtc_context_safe = False` opts back out). See `help(dtc.declare_safe)` for the precise contract.
+
+Tags that *do* write the context can declare **what** they write instead, as long as the target names are fixed at parse time — the common capture/setter shape:
+
+```python
+class CaptureNode(Node):
+    # names the instance attributes holding the written context keys
+    dtc_context_writes = ("target",)
+
+    def __init__(self, nodelist, target):
+        self.nodelist = nodelist
+        self.target = target            # {% capture NAME %}...{% endcapture %}
+
+    def render(self, context):
+        context[self.target] = self.nodelist.render(context)
+        return ""
+
+# or for classes you can't edit:
+dtc.declare_writes(SomeVendorSetterNode, "dest")
+```
+
+The compiler routes reads of the declared names through the live context and keeps every optimization on for everything else — including scope locals: if a declared write shadows a `{% for %}`/`{% with %}` name, the generated code re-reads that local right after the tag runs. The rest of the contract matches `dtc_context_safe`; the declared keys may be *set* only (no deletions), and an attribute holding `None` means an optional target unused at that site. See `help(dtc.declare_writes)`.
+
+Declared writes may target the normal top-of-stack (`context[key] = value`) **or** the root layer (`context.dicts[0][key] = value` — the pattern used by tags that persist a value across template boundaries, past every scope pop). Root-written names are never served from the read snapshot (it excludes the root layer by design), so they stay exact across template boundaries, includes, and re-writes.
+
+A wrong declaration produces wrong output silently — so verify it: run your test suite with `DTC_CHECK_DECLARATIONS=1` and dtc checks every declared render, raising `dtc.ContextSafeViolation` on any write outside the declaration. (Containers wrapping legitimate writers are skipped by the checker; the source-determinism clause isn't mechanically checkable.)
+
+Tags that just compute a value from their arguments — a formatter, a calculator, a lookup — are better rewritten as `@register.simple_tag`: those compile natively, declaration-free, with argument resolution inlined.
+
+### Limitation: tags that rewrite enclosing context layers
+
+Within a single template, *any* custom tag is rendered exactly — the compiler disables its read optimizations around every tag it doesn't recognize. Across template boundaries there is one assumption: a tag's context effects that outlive an `{% include %}`/`{% block %}`/`{% extends %}` are either **scope-limited** (ordinary `context[key] = value` writes and balanced push/pop, which die with the layers that the include/block machinery pops) or **root-layer** (`context.dicts[0][key] = value`, which dtc handles as described above). Every Django built-in and every `simple_tag`/`inclusion_tag` satisfies this.
+
+A tag that mutates an *intermediate* layer of the caller's stack — indexing `context.dicts[1]`, calling `Context.set_upward()`, deleting keys from enclosing layers, or leaking an unbalanced `push()` — from inside an included or extended template **can produce output that differs from stock Django**: the enclosing template was compiled without knowledge of that tag, and its read snapshot or scope locals may serve the pre-mutation value. `DTC_CHECK_DECLARATIONS` cannot catch this (the tag carries no declaration, and the effect surfaces in a different template than the tag).
+
+If you have such a tag, the supported paths are: write the root layer instead (`dicts[0]` — fully supported and declarable), write the top of the stack, or confine the mutation to the template that renders the tag. Note that intermediate-layer writes are fragile under stock Django too — what `dicts[1]` *is* depends on the stack depth at the call site.
+
+## Development
+
+```bash
+pip install -e .[dev]
+pytest
+```
+
+## License
+
+BSD 3-Clause. See [LICENSE](LICENSE).

django_template_compiler-0.0.1.dist-info/RECORD

@@ -0,0 +1,10 @@
+dtc/__init__.py,sha256=SCu3XWMtuAIIWEHaSYtHpQUThdzYFVaP-Le4Md2VYrc,5620
+dtc/autopatch.py,sha256=CqoP3e92fVAiK_8sNH00CWlgdBqVZeEypYcYPNh3JOE,2512
+dtc/backend.py,sha256=_vuCIpBC-A4_ztG7QHdMj2r3Dx6_YKARVrAXwIKVks4,2791
+dtc/compiler.py,sha256=0bjPwlqUzK5i9jkFI0hg_x3M7FwjvjT2nNU3i8xxHPA,65272
+dtc/diskcache.py,sha256=p0WZclYTZF7iOCDx0WndcdHyooHTXpygwPBES4jSq_g,3541
+dtc/runtime.py,sha256=30MFZXSM6Gz-ZsLtVmNRLJT0mWRW2tvbdf9WUlUzZ44,15408
+django_template_compiler-0.0.1.dist-info/METADATA,sha256=4Usc0xiDupcO_5snK03OQmWD6k4cgift6EqVcTg3ujE,11008
+django_template_compiler-0.0.1.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+django_template_compiler-0.0.1.dist-info/licenses/LICENSE,sha256=n74JfQDQIZ-eBz4adf7AMuwOqLwzu9PYy2cO3HaueBI,1497
+django_template_compiler-0.0.1.dist-info/RECORD,,

django_template_compiler-0.0.1.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

django_template_compiler-0.0.1.dist-info/licenses/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Sam Tregar
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

dtc/__init__.py

@@ -0,0 +1,124 @@
+"""dtc -- a drop-in, compiling replacement for Django's template engine."""
+
+__version__ = "0.0.1"
+
+__all__ = ["ContextSafeViolation", "declare_safe", "declare_writes", "__version__"]
+
+
+class ContextSafeViolation(Exception):
+    """A node declared ``dtc_context_safe`` broke the declaration's contract.
+
+    Raised only under ``DTC_CHECK_DECLARATIONS=1`` (see ``declare_safe``);
+    without it a wrong declaration silently produces wrong output.
+    """
+
+
+def declare_safe(obj):
+    """Declare a custom template tag context-safe, letting the compiler keep
+    its read optimizations around it.
+
+    ``obj`` is either a ``django.template.Node`` subclass (a raw
+    ``register.tag`` tag) or the function registered with
+    ``@simple_tag(takes_context=True)`` / ``@inclusion_tag(takes_context=True)``.
+    Sets ``obj.dtc_context_safe = True`` and returns ``obj``, so it works as a
+    decorator. Tags you own can skip this helper and set the class attribute
+    directly — stock Django ignores it, so no dtc import is needed.
+
+    The declaration is a promise about every ``render()`` call (for a
+    function, every call):
+
+    (a) The context stack and every mapping on it are left exactly as found:
+        no ``push``/``pop``/``__setitem__``/``del`` visible after return
+        (balanced internal push/pop is fine). Effects of rendering child
+        nodelists are exempt — see (d).
+    (b) No per-render state keyed by the node's identity — nothing like
+        ``context.render_context[self]`` or mutable state on ``self``
+        (Django's CycleNode/IfChangedNode pattern). Caches derived purely
+        from the parsed arguments are fine.
+    (c) Behavior depends only on the parsed source: node instances parsed
+        from identical source are interchangeable (dtc may render other
+        same-source template instances through one parse's node objects).
+    (d) Any nodelist the node renders is listed in Django's
+        ``child_nodelists`` attribute. The compiler analyzes those children
+        itself, so their effects (including context writes by nested tags)
+        are exempt from (a). A nodelist rendered but *not* listed hides
+        nested writers from the compiler and produces wrong output — this
+        is the one part of the contract nothing can check.
+
+    Reading the context is always fine, as is setting
+    ``context.autoescape`` (compiled code re-reads it after every bridged
+    call). Clauses (b) and (c) do not apply to takes_context functions —
+    only (a) does.
+
+    Subclasses inherit the declaration along with the ``render()`` it
+    covers; a subclass whose ``render()`` no longer qualifies must set
+    ``dtc_context_safe = False``.
+
+    Run your test suite with ``DTC_CHECK_DECLARATIONS=1`` to verify
+    declarations: declared-safe renders are then checked against clauses
+    (a) and (b) and raise ``ContextSafeViolation`` on violation.
+    """
+    if isinstance(obj, type):
+        from django.template.base import Node
+
+        if not issubclass(obj, Node):
+            raise TypeError(
+                f"declare_safe() expects a django.template.Node subclass or a "
+                f"takes_context tag function, got the class {obj!r}"
+            )
+    elif not callable(obj):
+        raise TypeError(
+            f"declare_safe() expects a django.template.Node subclass or a "
+            f"takes_context tag function, got {obj!r}"
+        )
+    obj.dtc_context_safe = True
+    return obj
+
+
+def declare_writes(node_class, *attr_names):
+    """Declare that a Node subclass's ``render()`` writes exactly the
+    context keys held in the named *instance attributes* — and follows the
+    rest of the ``declare_safe`` contract for everything else.
+
+    The attributes are named (rather than the keys themselves) because
+    write targets are parse-time data::
+
+        class CaptureNode(Node):
+            def __init__(self, nodelist, target):
+                self.nodelist = nodelist
+                self.target = target     # {% capture NAME %}...{% endcapture %}
+            def render(self, context):
+                context[self.target] = self.nodelist.render(context)
+                return ""
+
+        dtc.declare_writes(CaptureNode, "target")
+        # or, on a class you own, equivalently:
+        #     dtc_context_writes = ("target",)
+
+    The promise, per ``render()`` call: the only context mutations are
+    setting the named keys — on the effective top of the stack
+    (``context[k] = value``) or on the root layer
+    (``context.dicts[0][k] = value``, the pattern of tags that persist a
+    value across template boundaries) — no other writes, no intermediate
+    layers, no
+    deletions, no net push/pop, and clauses (b)–(d) of the
+    ``declare_safe`` contract. An attribute holding None means an optional
+    target unused at that site. With no attributes this is equivalent to
+    ``declare_safe``. Only Node subclasses can declare writes (a
+    ``takes_context`` function's targets aren't inspectable); subclasses
+    inherit the declaration.
+
+    Verify with ``DTC_CHECK_DECLARATIONS=1``: writes outside the declared
+    keys raise ``ContextSafeViolation``.
+    """
+    from django.template.base import Node
+
+    if not (isinstance(node_class, type) and issubclass(node_class, Node)):
+        raise TypeError(
+            f"declare_writes() expects a django.template.Node subclass, "
+            f"got {node_class!r}"
+        )
+    if not all(isinstance(a, str) for a in attr_names):
+        raise TypeError("declare_writes() attribute names must be strings")
+    node_class.dtc_context_writes = attr_names
+    return node_class

dtc/autopatch.py

@@ -0,0 +1,65 @@
+"""Engine-level instrumentation: compile every ``django.template.base.Template``.
+
+``install()`` patches ``Template._render`` to try the dtc-compiled path with
+lazy per-instance compilation (shared with the rest of dtc via
+``runtime.compiled_for``), falling back to Django's interpreted renderer.
+This hooks the *engine* level rather than the BACKENDS proxy, so it also
+covers templates constructed directly — which is how Django's own
+``template_tests`` suite builds them (see ``scripts/run_django_suite.py``).
+
+It also substitutes ``django.test.utils.instrumented_test_render`` so that
+when ``setup_test_environment()`` re-patches ``_render`` for test
+instrumentation, the replacement still takes the compiled path: the
+``template_rendered`` signal is sent exactly as stock Django sends it, then
+rendering proceeds compiled-or-fallback.
+
+Experimental; the supported integration point is ``dtc.backend.DTCTemplates``.
+"""
+
+from __future__ import annotations
+
+from . import runtime
+from .runtime import compiled_for, stats
+
+_installed = False
+
+
+def install():
+    global _installed
+    if _installed:
+        return
+    _installed = True
+
+    import django.test.utils as test_utils
+    from django.template.base import Template
+    from django.test.signals import template_rendered
+
+    orig_render = Template._render
+
+    def _render(self, context):
+        compiled = compiled_for(self)
+        if compiled is None:
+            stats["renders_fallback"] += 1
+            return orig_render(self, context)
+        stats["renders_compiled"] += 1
+        return compiled(context)
+
+    Template._render = _render
+    # This replacement is compiled-aware (it only adds stats counting), so
+    # the literal-include fast path may route around it. The
+    # instrumented_test_render below must NOT be registered: it sends the
+    # template_rendered signal, which routing around would silence.
+    runtime._transparent_renders.append(_render)
+
+    def instrumented_test_render(self, context):
+        # Byte-for-byte what django.test.utils.instrumented_test_render does,
+        # with the nodelist render swapped for compiled-or-fallback.
+        template_rendered.send(sender=self, template=self, context=context)
+        compiled = compiled_for(self)
+        if compiled is None:
+            stats["renders_fallback"] += 1
+            return self.nodelist.render(context)
+        stats["renders_compiled"] += 1
+        return compiled(context)
+
+    test_utils.instrumented_test_render = instrumented_test_render

dtc/backend.py

@@ -0,0 +1,80 @@
+"""Django BACKENDS engine for dtc.
+
+Usage in settings.py -- change only the BACKEND line of an existing
+DjangoTemplates configuration:
+
+    TEMPLATES = [
+        {
+            "BACKEND": "dtc.backend.DTCTemplates",
+            "DIRS": [...],
+            "APP_DIRS": True,
+            "OPTIONS": {...},
+        },
+    ]
+
+All OPTIONS accepted by django.template.backends.django.DjangoTemplates
+(context_processors, libraries, builtins, autoescape, debug, loaders,
+string_if_invalid, ...) are accepted here with identical meaning.
+"""
+
+from __future__ import annotations
+
+from django.template import TemplateDoesNotExist
+from django.template.backends.django import (
+    DjangoTemplates,
+    Template as DjangoTemplateProxy,
+    reraise,
+)
+from django.template.context import make_context
+
+from .runtime import compiled_for, template_render
+
+
+class DTCTemplates(DjangoTemplates):
+    """Drop-in replacement for the DjangoTemplates backend.
+
+    Parsing, template loading, and configuration are inherited unchanged
+    from Django; only the render path differs. Templates that the compiler
+    can handle render through generated Python code, everything else falls
+    back to Django's interpreted renderer.
+    """
+
+    def __init__(self, params):
+        params = params.copy()
+        options = params.get("OPTIONS", {}).copy()
+        # dtc's own option must not reach Django's Engine (unknown kwargs
+        # raise TypeError there).
+        disk_cache = options.pop("dtc_disk_cache", None)
+        params["OPTIONS"] = options
+        super().__init__(params)
+        if disk_cache:
+            from .diskcache import resolve_dir
+
+            self.engine._dtc_disk_cache = resolve_dir(disk_cache)
+
+    def from_string(self, template_code):
+        return Template(self.engine.from_string(template_code), self)
+
+    def get_template(self, template_name):
+        try:
+            return Template(self.engine.get_template(template_name), self)
+        except TemplateDoesNotExist as exc:
+            reraise(exc, self)
+
+
+class Template(DjangoTemplateProxy):
+    """Backend template proxy that prefers the compiled render path."""
+
+    def __init__(self, template, backend):
+        super().__init__(template, backend)
+        self._compiled = compiled_for(template)  # instance-cached
+
+    def render(self, context=None, request=None):
+        context = make_context(
+            context, request, autoescape=self.backend.engine.autoescape
+        )
+        # template_render reproduces django.template.base.Template.render
+        # exactly around the compiled body (render_context push, template
+        # binding — which runs context processors on RequestContext), and
+        # falls back to Django's own render when not compiled.
+        return template_render(self.template, context)