xvcl 2.5.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of xvcl might be problematic. Click here for more details.

@@ -0,0 +1,1534 @@
1
+ Metadata-Version: 2.4
2
+ Name: xvcl
3
+ Version: 2.5.0
4
+ Summary: Extended VCL compiler with metaprogramming features for Fastly VCL
5
+ Author-email: Frank Denis <github@pureftpd.org>
6
+ License: MIT
7
+ License-File: LICENSE
8
+ Keywords: compiler,fastly,preprocessor,varnish,vcl
9
+ Classifier: Development Status :: 4 - Beta
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3.9
14
+ Classifier: Programming Language :: Python :: 3.10
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Topic :: Software Development :: Compilers
18
+ Requires-Python: >=3.9
19
+ Provides-Extra: dev
20
+ Requires-Dist: mypy>=1.0.0; extra == 'dev'
21
+ Requires-Dist: pytest>=7.0.0; extra == 'dev'
22
+ Requires-Dist: ruff>=0.1.0; extra == 'dev'
23
+ Description-Content-Type: text/markdown
24
+
25
+ # xvcl
26
+
27
+ Supercharge your Fastly VCL with programming constructs like loops, functions, constants, and more.
28
+
29
+ **📖 [Quick Reference Guide](xvcl-quick-reference.md)** - One-page syntax reference for all xvcl features
30
+
31
+ ## Table of Contents
32
+
33
+ - [Introduction](#introduction)
34
+ - [Why Use xvcl?](#why-use-xvcl)
35
+ - [Installation](#installation)
36
+ - [Quick Start](#quick-start)
37
+ - [Core Features](#core-features)
38
+ - [Constants](#constants)
39
+ - [Template Expressions](#template-expressions)
40
+ - [For Loops](#for-loops)
41
+ - [Conditionals](#conditionals)
42
+ - [Variables](#variables)
43
+ - [File Includes](#file-includes)
44
+ - [Advanced Features](#advanced-features)
45
+ - [Inline Macros](#inline-macros)
46
+ - [Functions](#functions)
47
+ - [Command-Line Usage](#command-line-usage)
48
+ - [Integration with Falco](#integration-with-falco)
49
+ - [Best Practices](#best-practices)
50
+ - [Troubleshooting](#troubleshooting)
51
+
52
+ ## Introduction
53
+
54
+ xvcl is VCL transpiler that extends Fastly VCL with programming constructs that generate standard VCL code.
55
+
56
+ Think of it as a build step for your VCL: write enhanced VCL source files, run xvcl, and get clean, valid VCL output.
57
+
58
+ > **💡 Tip:** For a quick syntax reference, see the [xvcl Quick Reference Guide](xvcl-quick-reference.md).
59
+
60
+ **What you can do:**
61
+
62
+ - Define constants once, use them everywhere
63
+ - Generate repetitive code with for loops
64
+ - Create reusable functions with return values
65
+ - Build zero-overhead macros for common patterns
66
+ - Conditionally compile code for different environments
67
+ - Split large VCL files into modular includes
68
+
69
+ **What you get:**
70
+
71
+ - Standard VCL output that works with Fastly
72
+ - Compile-time safety and error checking
73
+ - Reduced code duplication
74
+ - Better maintainability
75
+
76
+ ## Why Use xvcl?
77
+
78
+ VCL is powerful but limited by design. You can't define functions with return values, you can't use loops, and managing constants means find-and-replace. This leads to:
79
+
80
+ - **Copy-paste errors:** Similar backends? Copy, paste, modify, repeat, make mistakes
81
+ - **Magic numbers:** Hardcoded values scattered throughout your code
82
+ - **Duplication:** Same logic repeated in multiple subroutines
83
+ - **Poor maintainability:** Change one thing, update it in 20 places
84
+
85
+ xvcl solves these problems by adding programming constructs that compile down to clean VCL.
86
+
87
+ ### Real-World Example
88
+
89
+ **Without xvcl (manual, error-prone):**
90
+
91
+ ```vcl
92
+ backend web1 {
93
+ .host = "web1.example.com";
94
+ .port = "80";
95
+ }
96
+
97
+ backend web2 {
98
+ .host = "web2.example.com";
99
+ .port = "80";
100
+ }
101
+
102
+ backend web3 {
103
+ .host = "web3.example.com";
104
+ .port = "80";
105
+ }
106
+
107
+ sub vcl_recv {
108
+ if (req.http.Host == "web1.example.com") {
109
+ set req.backend = web1;
110
+ }
111
+ if (req.http.Host == "web2.example.com") {
112
+ set req.backend = web2;
113
+ }
114
+ if (req.http.Host == "web3.example.com") {
115
+ set req.backend = web3;
116
+ }
117
+ }
118
+ ```
119
+
120
+ **With xvcl (clean, maintainable):**
121
+
122
+ ```vcl
123
+ #const BACKENDS = ["web1", "web2", "web3"]
124
+
125
+ #for backend in BACKENDS
126
+ backend {{backend}} {
127
+ .host = "{{backend}}.example.com";
128
+ .port = "80";
129
+ }
130
+ #endfor
131
+
132
+ sub vcl_recv {
133
+ #for backend in BACKENDS
134
+ if (req.http.Host == "{{backend}}.example.com") {
135
+ set req.backend = {{backend}};
136
+ }
137
+ #endfor
138
+ }
139
+ ```
140
+
141
+ Adding a new backend? Just update the list. xvcl generates all the code.
142
+
143
+ ## Installation
144
+
145
+ xvcl is a Python package. You need Python 3.9 or later.
146
+
147
+ ```bash
148
+ # Using pip
149
+ pip install xvcl
150
+
151
+ # Or install from source
152
+ pip install .
153
+
154
+ # Development installation with dev dependencies
155
+ pip install -e ".[dev]"
156
+
157
+ # Using uv (recommended for faster installation)
158
+ uv pip install xvcl
159
+ ```
160
+
161
+ After installation, the `xvcl` command is available globally.
162
+
163
+ **No external dependencies** - uses only Python standard library.
164
+
165
+ ## Quick Start
166
+
167
+ Create an xvcl source file (use `.xvcl` extension by convention):
168
+
169
+ **`hello.xvcl`:**
170
+
171
+ ```vcl
172
+ #const MESSAGE = "Hello from xvcl!"
173
+
174
+ sub vcl_recv {
175
+ set req.http.X-Message = "{{MESSAGE}}";
176
+ }
177
+ ```
178
+
179
+ Run xvcl:
180
+
181
+ ```bash
182
+ xvcl hello.xvcl -o hello.vcl
183
+ ```
184
+
185
+ Output **`hello.vcl`:**
186
+
187
+ ```vcl
188
+ sub vcl_recv {
189
+ set req.http.X-Message = "Hello from xvcl!";
190
+ }
191
+ ```
192
+
193
+ Validate with Falco:
194
+
195
+ ```bash
196
+ falco lint hello.vcl
197
+ ```
198
+
199
+ ## Core Features
200
+
201
+ ### Constants
202
+
203
+ Define named constants with type checking. Constants are evaluated at preprocessing time and substituted into your code.
204
+
205
+ **Syntax:**
206
+
207
+ ```vcl
208
+ #const NAME TYPE = value
209
+ ```
210
+
211
+ **Supported types:**
212
+ - `INTEGER` - Whole numbers
213
+ - `STRING` - Text strings
214
+ - `FLOAT` - Decimal numbers
215
+ - `BOOL` - True/False
216
+
217
+ **Example:**
218
+
219
+ ```vcl
220
+ #const MAX_AGE INTEGER = 3600
221
+ #const ORIGIN STRING = "origin.example.com"
222
+ #const PRODUCTION BOOL = True
223
+ #const CACHE_VERSION FLOAT = 1.5
224
+ ```
225
+
226
+ **Using constants in templates:**
227
+
228
+ ```vcl
229
+ #const TTL = 300
230
+ #const BACKEND_HOST = "api.example.com"
231
+
232
+ backend F_api {
233
+ .host = "{{BACKEND_HOST}}";
234
+ .port = "443";
235
+ }
236
+
237
+ sub vcl_fetch {
238
+ set beresp.ttl = {{TTL}}s;
239
+ }
240
+ ```
241
+
242
+ **Why use constants?**
243
+
244
+ - Single source of truth for configuration values
245
+ - Easy to update across entire VCL
246
+ - Type safety prevents mistakes
247
+ - Self-documenting code
248
+
249
+ ### Template Expressions
250
+
251
+ Embed Python expressions in your VCL using `{{expression}}` syntax. Expressions are evaluated at preprocessing time.
252
+
253
+ **Example:**
254
+
255
+ ```vcl
256
+ #const PORT = 8080
257
+
258
+ sub vcl_recv {
259
+ set req.http.X-Port = "{{PORT}}";
260
+ set req.http.X-Double = "{{PORT * 2}}";
261
+ set req.http.X-Hex = "{{hex(PORT)}}";
262
+ }
263
+ ```
264
+
265
+ **Output:**
266
+
267
+ ```vcl
268
+ sub vcl_recv {
269
+ set req.http.X-Port = "8080";
270
+ set req.http.X-Double = "16160";
271
+ set req.http.X-Hex = "0x1f90";
272
+ }
273
+ ```
274
+
275
+ **Available functions:**
276
+
277
+ - `range(n)` - Generate number sequences
278
+ - `len(list)` - Get list length
279
+ - `str(x)`, `int(x)` - Type conversions
280
+ - `hex(n)` - Hexadecimal conversion
281
+ - `format(x, fmt)` - Format values
282
+ - `enumerate(iterable)` - Enumerate with indices
283
+ - `min(...)`, `max(...)` - Min/max values
284
+ - `abs(n)` - Absolute value
285
+
286
+ **String formatting:**
287
+
288
+ ```vcl
289
+ #const REGION = "us-east"
290
+ #const INDEX = 1
291
+
292
+ set req.backend = F_backend_{{REGION}}_{{INDEX}};
293
+ ```
294
+
295
+ ### For Loops
296
+
297
+ Generate repetitive VCL code by iterating over ranges or lists.
298
+
299
+ **Syntax:**
300
+
301
+ ```vcl
302
+ #for variable in iterable
303
+ // Code to repeat
304
+ #endfor
305
+ ```
306
+
307
+ **Example 1: Range-based loop**
308
+
309
+ ```vcl
310
+ #for i in range(5)
311
+ backend web{{i}} {
312
+ .host = "web{{i}}.example.com";
313
+ .port = "80";
314
+ }
315
+ #endfor
316
+ ```
317
+
318
+ **Output:**
319
+
320
+ ```vcl
321
+ backend web0 {
322
+ .host = "web0.example.com";
323
+ .port = "80";
324
+ }
325
+ backend web1 {
326
+ .host = "web1.example.com";
327
+ .port = "80";
328
+ }
329
+ // ... continues through web4
330
+ ```
331
+
332
+ **Example 2: List iteration**
333
+
334
+ ```vcl
335
+ #const REGIONS = ["us-east", "us-west", "eu-west"]
336
+
337
+ #for region in REGIONS
338
+ backend F_{{region}} {
339
+ .host = "{{region}}.example.com";
340
+ .port = "443";
341
+ .ssl = true;
342
+ }
343
+ #endfor
344
+ ```
345
+
346
+ **Example 3: Nested loops**
347
+
348
+ ```vcl
349
+ #const REGIONS = ["us", "eu"]
350
+ #const ENVS = ["prod", "staging"]
351
+
352
+ #for region in REGIONS
353
+ #for env in ENVS
354
+ backend {{region}}_{{env}} {
355
+ .host = "{{env}}.{{region}}.example.com";
356
+ .port = "443";
357
+ }
358
+ #endfor
359
+ #endfor
360
+ ```
361
+
362
+ **Why use for loops?**
363
+
364
+ - Generate multiple similar backends
365
+ - Create ACL entries from lists
366
+ - Build routing logic programmatically
367
+ - Reduce copy-paste errors
368
+
369
+ ### Conditionals
370
+
371
+ Conditionally include or exclude code based on compile-time conditions.
372
+
373
+ **Syntax:**
374
+
375
+ ```vcl
376
+ #if condition
377
+ // Code when true
378
+ #else
379
+ // Code when false (optional)
380
+ #endif
381
+ ```
382
+
383
+ **Example 1: Environment-specific configuration**
384
+
385
+ ```vcl
386
+ #const PRODUCTION = True
387
+ #const DEBUG = False
388
+
389
+ sub vcl_recv {
390
+ #if PRODUCTION
391
+ set req.http.X-Environment = "production";
392
+ unset req.http.X-Debug-Info;
393
+ #else
394
+ set req.http.X-Environment = "development";
395
+ set req.http.X-Debug-Info = "Enabled";
396
+ #endif
397
+
398
+ #if DEBUG
399
+ set req.http.X-Request-ID = randomstr(16, "0123456789abcdef");
400
+ #endif
401
+ }
402
+ ```
403
+
404
+ **Example 2: Feature flags**
405
+
406
+ ```vcl
407
+ #const ENABLE_NEW_ROUTING = True
408
+ #const ENABLE_RATE_LIMITING = False
409
+
410
+ sub vcl_recv {
411
+ #if ENABLE_NEW_ROUTING
412
+ call new_routing_logic;
413
+ #else
414
+ call legacy_routing_logic;
415
+ #endif
416
+
417
+ #if ENABLE_RATE_LIMITING
418
+ if (ratelimit.check_rate("client_" + client.ip, 1, 100, 60s, 1000s)) {
419
+ error 429 "Too Many Requests";
420
+ }
421
+ #endif
422
+ }
423
+ ```
424
+
425
+ **Why use conditionals?**
426
+
427
+ - Single codebase for multiple environments
428
+ - Easy feature flag management
429
+ - Dead code elimination (code in false branches isn't generated)
430
+ - Compile-time optimization
431
+
432
+ ### Variables
433
+
434
+ Declare and initialize local variables in one step.
435
+
436
+ **Syntax:**
437
+
438
+ ```vcl
439
+ #let name TYPE = expression;
440
+ ```
441
+
442
+ **Example:**
443
+
444
+ ```vcl
445
+ sub vcl_recv {
446
+ #let timestamp STRING = std.time(now, now);
447
+ #let cache_key STRING = req.url.path + req.http.Host;
448
+
449
+ set req.http.X-Timestamp = var.timestamp;
450
+ set req.hash = var.cache_key;
451
+ }
452
+ ```
453
+
454
+ **Expands to:**
455
+
456
+ ```vcl
457
+ sub vcl_recv {
458
+ declare local var.timestamp STRING;
459
+ set var.timestamp = std.time(now, now);
460
+ declare local var.cache_key STRING;
461
+ set var.cache_key = req.url.path + req.http.Host;
462
+
463
+ set req.http.X-Timestamp = var.timestamp;
464
+ set req.hash = var.cache_key;
465
+ }
466
+ ```
467
+
468
+ **Why use #let?**
469
+
470
+ - Shorter syntax than separate declare + set
471
+ - Clear initialization point
472
+ - Reduces boilerplate
473
+
474
+ ### File Includes
475
+
476
+ Split large VCL files into modular, reusable components.
477
+
478
+ **Syntax:**
479
+
480
+ ```vcl
481
+ #include "path/to/file.xvcl"
482
+ ```
483
+
484
+ **Example project structure:**
485
+
486
+ ```
487
+ vcl/
488
+ ├── main.xvcl
489
+ ├── includes/
490
+ │ ├── backends.xvcl
491
+ │ ├── security.xvcl
492
+ │ └── routing.xvcl
493
+ ```
494
+
495
+ **`main.xvcl`:**
496
+
497
+ ```vcl
498
+ #include "includes/backends.xvcl"
499
+ #include "includes/security.xvcl"
500
+ #include "includes/routing.xvcl"
501
+
502
+ sub vcl_recv {
503
+ call security_checks;
504
+ call routing_logic;
505
+ }
506
+ ```
507
+
508
+ **`includes/backends.xvcl`:**
509
+
510
+ ```vcl
511
+ #const BACKENDS = ["web1", "web2", "web3"]
512
+
513
+ #for backend in BACKENDS
514
+ backend F_{{backend}} {
515
+ .host = "{{backend}}.example.com";
516
+ .port = "443";
517
+ }
518
+ #endfor
519
+ ```
520
+
521
+ **Include path resolution:**
522
+
523
+ 1. Relative to the current file
524
+ 2. Relative to include paths specified with `-I`
525
+
526
+ **Run with include paths:**
527
+
528
+ ```bash
529
+ xvcl main.xvcl -o main.vcl -I ./vcl/includes
530
+ ```
531
+
532
+ **Features:**
533
+
534
+ - **Include-once semantics:** Files are only included once even if referenced multiple times
535
+ - **Cycle detection:** Prevents circular includes
536
+ - **Shared constants:** Constants defined in included files are available to the parent
537
+
538
+ **Why use includes?**
539
+
540
+ - Organize large VCL projects
541
+ - Share common configurations across multiple VCL files
542
+ - Team collaboration (different files for different concerns)
543
+ - Reusable components library
544
+
545
+ ## Advanced Features
546
+
547
+ ### Inline Macros
548
+
549
+ Create zero-overhead text substitution macros. Unlike functions, macros are expanded inline at compile time with no runtime cost.
550
+
551
+ **Syntax:**
552
+
553
+ ```vcl
554
+ #inline macro_name(param1, param2, ...)
555
+ expression
556
+ #endinline
557
+ ```
558
+
559
+ **Example 1: String concatenation**
560
+
561
+ ```vcl
562
+ #inline add_prefix(s)
563
+ "prefix-" + s
564
+ #endinline
565
+
566
+ #inline add_suffix(s)
567
+ s + "-suffix"
568
+ #endinline
569
+
570
+ sub vcl_recv {
571
+ set req.http.X-Modified = add_prefix("test");
572
+ set req.http.X-Both = add_prefix(add_suffix("middle"));
573
+ }
574
+ ```
575
+
576
+ **Output:**
577
+
578
+ ```vcl
579
+ sub vcl_recv {
580
+ set req.http.X-Modified = "prefix-" + "test";
581
+ set req.http.X-Both = "prefix-" + "middle" + "-suffix";
582
+ }
583
+ ```
584
+
585
+ **Example 2: Common patterns**
586
+
587
+ ```vcl
588
+ #inline normalize_host(host)
589
+ std.tolower(regsub(host, "^www\.", ""))
590
+ #endinline
591
+
592
+ #inline cache_key(url, host)
593
+ digest.hash_md5(url + "|" + host)
594
+ #endinline
595
+
596
+ sub vcl_recv {
597
+ set req.http.X-Normalized = normalize_host(req.http.Host);
598
+ set req.hash = cache_key(req.url, req.http.Host);
599
+ }
600
+ ```
601
+
602
+ **Output:**
603
+
604
+ ```vcl
605
+ sub vcl_recv {
606
+ set req.http.X-Normalized = std.tolower(regsub(req.http.Host, "^www\.", ""));
607
+ set req.hash = digest.hash_md5(req.url + "|" + req.http.Host);
608
+ }
609
+ ```
610
+
611
+ **Example 3: Operator precedence handling**
612
+
613
+ xvcl automatically handles operator precedence:
614
+
615
+ ```vcl
616
+ #inline double(x)
617
+ x + x
618
+ #endinline
619
+
620
+ sub vcl_recv {
621
+ declare local var.result INTEGER;
622
+ set var.result = double(5) * 10; // Correctly expands to (5 + 5) * 10
623
+ }
624
+ ```
625
+
626
+ **Macros vs Functions:**
627
+
628
+ | Feature | Macros | Functions |
629
+ | ------------- | ------------------- | ----------------------------- |
630
+ | Expansion | Compile-time inline | Runtime subroutine call |
631
+ | Overhead | None | Subroutine call + global vars |
632
+ | Return values | Expression only | Single or tuple |
633
+ | Use case | Simple expressions | Complex logic |
634
+
635
+ **When to use macros:**
636
+
637
+ - String manipulation patterns
638
+ - Simple calculations
639
+ - Common expressions repeated throughout code
640
+ - When you need zero runtime overhead
641
+
642
+ **When to use functions:**
643
+
644
+ - Complex logic with multiple statements
645
+ - Need to return multiple values
646
+ - Conditional logic or loops inside the reusable code
647
+
648
+ ### Functions
649
+
650
+ Define reusable functions with parameters and return values. Functions are compiled into VCL subroutines.
651
+
652
+ **Syntax:**
653
+
654
+ ```vcl
655
+ #def function_name(param1 TYPE, param2 TYPE, ...) -> RETURN_TYPE
656
+ // Function body
657
+ return value;
658
+ #enddef
659
+ ```
660
+
661
+ **Example 1: Simple function**
662
+
663
+ ```vcl
664
+ #def add(a INTEGER, b INTEGER) -> INTEGER
665
+ declare local var.sum INTEGER;
666
+ set var.sum = a + b;
667
+ return var.sum;
668
+ #enddef
669
+
670
+ sub vcl_recv {
671
+ declare local var.result INTEGER;
672
+ set var.result = add(5, 10);
673
+ set req.http.X-Sum = var.result;
674
+ }
675
+ ```
676
+
677
+ **Example 2: String processing**
678
+
679
+ ```vcl
680
+ #def normalize_path(path STRING) -> STRING
681
+ declare local var.result STRING;
682
+ set var.result = std.tolower(path);
683
+ set var.result = regsub(var.result, "/$", "");
684
+ return var.result;
685
+ #enddef
686
+
687
+ sub vcl_recv {
688
+ declare local var.clean_path STRING;
689
+ set var.clean_path = normalize_path(req.url.path);
690
+ set req.url = var.clean_path;
691
+ }
692
+ ```
693
+
694
+ **Example 3: Functions with conditionals**
695
+
696
+ ```vcl
697
+ #def should_cache(url STRING) -> BOOL
698
+ declare local var.cacheable BOOL;
699
+
700
+ if (url ~ "^/api/") {
701
+ set var.cacheable = false;
702
+ } else if (url ~ "\.(jpg|png|css|js)$") {
703
+ set var.cacheable = true;
704
+ } else {
705
+ set var.cacheable = false;
706
+ }
707
+
708
+ return var.cacheable;
709
+ #enddef
710
+
711
+ sub vcl_recv {
712
+ declare local var.can_cache BOOL;
713
+ set var.can_cache = should_cache(req.url.path);
714
+
715
+ if (var.can_cache) {
716
+ return(lookup);
717
+ } else {
718
+ return(pass);
719
+ }
720
+ }
721
+ ```
722
+
723
+ **Example 4: Tuple returns (multiple values)**
724
+
725
+ ```vcl
726
+ #def parse_user_agent(ua STRING) -> (STRING, STRING)
727
+ declare local var.browser STRING;
728
+ declare local var.os STRING;
729
+
730
+ if (ua ~ "Chrome") {
731
+ set var.browser = "chrome";
732
+ } else if (ua ~ "Firefox") {
733
+ set var.browser = "firefox";
734
+ } else {
735
+ set var.browser = "other";
736
+ }
737
+
738
+ if (ua ~ "Windows") {
739
+ set var.os = "windows";
740
+ } else if (ua ~ "Mac") {
741
+ set var.os = "macos";
742
+ } else {
743
+ set var.os = "other";
744
+ }
745
+
746
+ return var.browser, var.os;
747
+ #enddef
748
+
749
+ sub vcl_recv {
750
+ declare local var.browser STRING;
751
+ declare local var.os STRING;
752
+
753
+ set var.browser, var.os = parse_user_agent(req.http.User-Agent);
754
+
755
+ set req.http.X-Browser = var.browser;
756
+ set req.http.X-OS = var.os;
757
+ }
758
+ ```
759
+
760
+ **Behind the scenes:**
761
+
762
+ Functions are compiled into VCL subroutines using global headers for parameter passing:
763
+
764
+ ```vcl
765
+ // Your code:
766
+ set var.result = add(5, 10);
767
+
768
+ // Becomes:
769
+ set req.http.X-Func-add-a = std.itoa(5);
770
+ set req.http.X-Func-add-b = std.itoa(10);
771
+ call add;
772
+ set var.result = std.atoi(req.http.X-Func-add-Return);
773
+ ```
774
+
775
+ xvcl generates the `sub add { ... }` implementation and handles all type conversions automatically.
776
+
777
+ **Function features:**
778
+
779
+ - **Type safety:** Parameters and returns are type-checked
780
+ - **Multiple returns:** Use tuple syntax to return multiple values
781
+ - **Automatic conversions:** INTEGER/FLOAT/BOOL are converted to/from STRING automatically
782
+ - **Scope annotations:** Generated subroutines work in all VCL scopes
783
+
784
+ **Why use functions?**
785
+
786
+ - Reusable complex logic
787
+ - Reduce code duplication
788
+ - Easier testing (test the function once)
789
+ - Better code organization
790
+
791
+ ## Command-Line Usage
792
+
793
+ **Basic usage:**
794
+
795
+ ```bash
796
+ xvcl input.xvcl -o output.vcl
797
+ ```
798
+
799
+ **Options:**
800
+
801
+ | Option | Description |
802
+ | --------------- | --------------------------------------------------- |
803
+ | `input` | Input xvcl source file (required) |
804
+ | `-o, --output` | Output VCL file (default: replaces .xvcl with .vcl) |
805
+ | `-I, --include` | Add include search path (repeatable) |
806
+ | `--debug` | Enable debug output with expansion traces |
807
+ | `--source-maps` | Add source map comments to output |
808
+ | `-v, --verbose` | Verbose output (alias for --debug) |
809
+
810
+ **Examples:**
811
+
812
+ ```bash
813
+ # Basic compilation
814
+ xvcl main.xvcl -o main.vcl
815
+
816
+ # With include paths
817
+ xvcl main.xvcl -o main.vcl \
818
+ -I ./includes \
819
+ -I ./shared
820
+
821
+ # Debug mode (see expansion traces)
822
+ xvcl main.xvcl -o main.vcl --debug
823
+
824
+ # With source maps (track generated code origin)
825
+ xvcl main.xvcl -o main.vcl --source-maps
826
+ ```
827
+
828
+ **Automatic output naming:**
829
+
830
+ If you don't specify `-o`, xvcl replaces `.xvcl` with `.vcl`:
831
+
832
+ ```bash
833
+ # These are equivalent:
834
+ xvcl main.xvcl
835
+ xvcl main.xvcl -o main.vcl
836
+ ```
837
+
838
+ **Debug mode output:**
839
+
840
+ ```bash
841
+ $ xvcl example.xvcl --debug
842
+
843
+ [DEBUG] Processing file: example.xvcl
844
+ [DEBUG] Pass 1: Extracting constants
845
+ [DEBUG] Defined constant: MAX_AGE = 3600
846
+ [DEBUG] Pass 2: Processing includes
847
+ [DEBUG] Pass 3: Extracting inline macros
848
+ [DEBUG] Defined macro: add_prefix(s)
849
+ [DEBUG] Pass 4: Extracting functions
850
+ [DEBUG] Pass 5: Processing directives and generating code
851
+ [DEBUG] Processing #for at line 10
852
+ [DEBUG] Loop iterating 3 times
853
+ [DEBUG] Iteration 0: backend = web1
854
+ [DEBUG] Iteration 1: backend = web2
855
+ [DEBUG] Iteration 2: backend = web3
856
+ [DEBUG] Pass 6: Generating function subroutines
857
+ ✓ Compiled example.xvcl -> example.vcl
858
+ Constants: 1
859
+ Macros: 1 (add_prefix)
860
+ Functions: 0
861
+ ```
862
+
863
+ ## Integration with Falco
864
+
865
+ xvcl generates standard VCL that you can use with Falco's full toolset.
866
+
867
+ **Recommended workflow:**
868
+
869
+ ```bash
870
+ # 1. Write your xvcl source
871
+ vim main.xvcl
872
+
873
+ # 2. Compile with xvcl
874
+ xvcl main.xvcl -o main.vcl
875
+
876
+ # 3. Lint with Falco
877
+ falco lint main.vcl
878
+
879
+ # 4. Test with Falco
880
+ falco test main.vcl
881
+
882
+ # 5. Simulate with Falco
883
+ falco simulate main.vcl
884
+ ```
885
+
886
+ **Makefile integration:**
887
+
888
+ ```makefile
889
+ # Makefile
890
+ .PHONY: build lint test clean
891
+
892
+ XVCL = xvcl
893
+ SOURCES = $(wildcard *.xvcl)
894
+ OUTPUTS = $(SOURCES:.xvcl=.vcl)
895
+
896
+ build: $(OUTPUTS)
897
+
898
+ %.vcl: %.xvcl
899
+ $(XVCL) $< -o $@ -I ./includes
900
+
901
+ lint: build
902
+ falco lint *.vcl
903
+
904
+ test: build
905
+ falco test *.vcl
906
+
907
+ clean:
908
+ rm -f $(OUTPUTS)
909
+ ```
910
+
911
+ **CI/CD integration:**
912
+
913
+ ```yaml
914
+ # .github/workflows/vcl.yml
915
+ name: VCL CI
916
+
917
+ on: [push, pull_request]
918
+
919
+ jobs:
920
+ test:
921
+ runs-on: ubuntu-latest
922
+ steps:
923
+ - uses: actions/checkout@v3
924
+
925
+ - name: Set up Python
926
+ uses: actions/setup-python@v4
927
+ with:
928
+ python-version: '3.10'
929
+
930
+ - name: Install Falco
931
+ run: |
932
+ wget https://github.com/ysugimoto/falco/releases/latest/download/falco_linux_amd64
933
+ chmod +x falco_linux_amd64
934
+ sudo mv falco_linux_amd64 /usr/local/bin/falco
935
+
936
+ - name: Compile xvcl
937
+ run: |
938
+ xvcl main.xvcl -o main.vcl
939
+
940
+ - name: Lint VCL
941
+ run: falco lint main.vcl
942
+
943
+ - name: Test VCL
944
+ run: falco test main.vcl
945
+ ```
946
+
947
+ **Testing compiled VCL:**
948
+
949
+ You can write Falco tests for your generated VCL:
950
+
951
+ **`main.test.vcl`:**
952
+
953
+ ```vcl
954
+ // @suite: Backend routing tests
955
+
956
+ // @test: Should route to correct backend
957
+ sub test_backend_routing {
958
+ set req.http.Host = "web1.example.com";
959
+ call vcl_recv;
960
+
961
+ assert.equal(req.backend, "web1");
962
+ }
963
+ ```
964
+
965
+ Run tests after compilation:
966
+
967
+ ```bash
968
+ xvcl main.xvcl -o main.vcl
969
+ falco test main.vcl
970
+ ```
971
+
972
+ ## Best Practices
973
+
974
+ ### 1. Use the `.xvcl` extension
975
+
976
+ Makes it clear which files are xvcl source files:
977
+
978
+ ```
979
+ ✓ main.xvcl → main.vcl
980
+ ✗ main.vcl → main.vcl.processed
981
+ ```
982
+
983
+ ### 2. Keep constants at the top
984
+
985
+ ```vcl
986
+ // Good: Constants first, easy to find
987
+ #const MAX_BACKENDS = 10
988
+ #const PRODUCTION = True
989
+
990
+ #for i in range(MAX_BACKENDS)
991
+ // ... use constant
992
+ #endfor
993
+ ```
994
+
995
+ ### 3. Use descriptive constant names
996
+
997
+ ```vcl
998
+ // Good
999
+ #const CACHE_TTL_SECONDS = 3600
1000
+ #const API_BACKEND_HOST = "api.example.com"
1001
+
1002
+ // Bad
1003
+ #const X = 3600
1004
+ #const B = "api.example.com"
1005
+ ```
1006
+
1007
+ ### 4. Comment your macros and functions
1008
+
1009
+ ```vcl
1010
+ // Normalizes a hostname by removing www prefix and converting to lowercase
1011
+ #inline normalize_host(host)
1012
+ std.tolower(regsub(host, "^www\.", ""))
1013
+ #endinline
1014
+
1015
+ // Parses User-Agent and returns (browser, os) tuple
1016
+ #def parse_user_agent(ua STRING) -> (STRING, STRING)
1017
+ // ...
1018
+ #enddef
1019
+ ```
1020
+
1021
+ ### 5. Prefer macros for simple expressions, functions for complex logic
1022
+
1023
+ ```vcl
1024
+ // Good: Simple expression = macro
1025
+ #inline cache_key(url, host)
1026
+ digest.hash_md5(url + "|" + host)
1027
+ #endinline
1028
+
1029
+ // Good: Complex logic = function
1030
+ #def should_cache(url STRING, method STRING) -> BOOL
1031
+ declare local var.result BOOL;
1032
+ if (method != "GET" && method != "HEAD") {
1033
+ set var.result = false;
1034
+ } else if (url ~ "^/api/") {
1035
+ set var.result = false;
1036
+ } else {
1037
+ set var.result = true;
1038
+ }
1039
+ return var.result;
1040
+ #enddef
1041
+ ```
1042
+
1043
+ ### 6. Use includes for organization
1044
+
1045
+ ```
1046
+ vcl/
1047
+ ├── main.xvcl # Main entry point
1048
+ ├── config.xvcl # Constants and configuration
1049
+ ├── includes/
1050
+ │ ├── backends.xvcl
1051
+ │ ├── security.xvcl
1052
+ │ ├── routing.xvcl
1053
+ │ └── caching.xvcl
1054
+ ```
1055
+
1056
+ ### 7. Version control both source and output
1057
+
1058
+ ```gitignore
1059
+ # Include both in git
1060
+ *.xvcl
1061
+ *.vcl
1062
+
1063
+ # But gitignore generated files in CI
1064
+ # (if you regenerate on deploy)
1065
+ ```
1066
+
1067
+ **Or** only version control source files and regenerate on deployment:
1068
+
1069
+ ```gitignore
1070
+ # Version control xvcl source only
1071
+ *.xvcl
1072
+
1073
+ # Ignore generated VCL
1074
+ *.vcl
1075
+ ```
1076
+
1077
+ Choose based on your deployment process.
1078
+
1079
+ ### 8. Add source maps in development
1080
+
1081
+ ```bash
1082
+ # Development: easier debugging
1083
+ xvcl main.xvcl -o main.vcl --source-maps
1084
+
1085
+ # Production: cleaner output
1086
+ xvcl main.xvcl -o main.vcl
1087
+ ```
1088
+
1089
+ Source maps add comments like:
1090
+
1091
+ ```vcl
1092
+ // BEGIN INCLUDE: includes/backends.xvcl
1093
+ backend F_web1 { ... }
1094
+ // END INCLUDE: includes/backends.xvcl
1095
+ ```
1096
+
1097
+ ### 9. Test incrementally
1098
+
1099
+ Don't write a massive source file and compile once. Test as you go:
1100
+
1101
+ ```bash
1102
+ # Write a bit
1103
+ vim main.xvcl
1104
+
1105
+ # Compile
1106
+ xvcl main.xvcl
1107
+
1108
+ # Check output
1109
+ cat main.vcl
1110
+
1111
+ # Lint
1112
+ falco lint main.vcl
1113
+
1114
+ # Repeat
1115
+ ```
1116
+
1117
+ ### 10. Use debug mode when things go wrong
1118
+
1119
+ ```bash
1120
+ xvcl main.xvcl --debug
1121
+ ```
1122
+
1123
+ Shows exactly what xvcl is doing.
1124
+
1125
+ ## Troubleshooting
1126
+
1127
+ ### Error: "Name 'X' is not defined"
1128
+
1129
+ **Problem:** You're using a variable or constant that doesn't exist.
1130
+
1131
+ ```vcl
1132
+ #const PORT = 8080
1133
+
1134
+ set req.http.X-Value = "{{PROT}}"; // Typo!
1135
+ ```
1136
+
1137
+ **Error:**
1138
+
1139
+ ```
1140
+ Error at main.xvcl:3:
1141
+ Name 'PROT' is not defined
1142
+ Did you mean: PORT?
1143
+ ```
1144
+
1145
+ **Solution:** Check spelling. xvcl suggests similar names.
1146
+
1147
+ ### Error: "Invalid #const syntax"
1148
+
1149
+ **Problem:** Malformed constant declaration.
1150
+
1151
+ ```vcl
1152
+ #const PORT 8080 // Missing = sign
1153
+ #const = 8080 // Missing name
1154
+ #const PORT = STRING // Missing value
1155
+ ```
1156
+
1157
+ **Solution:** Use correct syntax:
1158
+
1159
+ ```vcl
1160
+ #const PORT INTEGER = 8080
1161
+ ```
1162
+
1163
+ ### Error: "No matching #endfor for #for"
1164
+
1165
+ **Problem:** Missing closing keyword.
1166
+
1167
+ ```vcl
1168
+ #for i in range(10)
1169
+ backend web{{i}} { ... }
1170
+ // Missing #endfor
1171
+ ```
1172
+
1173
+ **Solution:** Add the closing keyword:
1174
+
1175
+ ```vcl
1176
+ #for i in range(10)
1177
+ backend web{{i}} { ... }
1178
+ #endfor
1179
+ ```
1180
+
1181
+ ### Error: "Circular include detected"
1182
+
1183
+ **Problem:** File A includes file B which includes file A.
1184
+
1185
+ ```
1186
+ main.xvcl includes util.xvcl
1187
+ util.xvcl includes main.xvcl
1188
+ ```
1189
+
1190
+ **Solution:** Restructure your includes. Create a shared file:
1191
+
1192
+ ```
1193
+ main.xvcl includes shared.xvcl
1194
+ util.xvcl includes shared.xvcl
1195
+ ```
1196
+
1197
+ ### Error: "Cannot find included file"
1198
+
1199
+ **Problem:** Include path is wrong or file doesn't exist.
1200
+
1201
+ ```vcl
1202
+ #include "includes/backends.xvcl"
1203
+ ```
1204
+
1205
+ **Solution:** Check path and use `-I` flag:
1206
+
1207
+ ```bash
1208
+ xvcl main.xvcl -o main.vcl -I ./includes
1209
+ ```
1210
+
1211
+ ### Generated VCL has syntax errors
1212
+
1213
+ **Problem:** xvcl generated invalid VCL.
1214
+
1215
+ **Solution:**
1216
+
1217
+ 1. Check the generated output:
1218
+ ```bash
1219
+ cat main.vcl
1220
+ ```
1221
+
1222
+ 2. Find the problematic section
1223
+
1224
+ 3. Trace back to source with `--source-maps`:
1225
+
1226
+ ```bash
1227
+ xvcl main.xvcl -o main.vcl --source-maps
1228
+ ```
1229
+
1230
+ 4. Fix the source file
1231
+
1232
+ ### Macro expansion issues
1233
+
1234
+ **Problem:** Macro expands incorrectly.
1235
+
1236
+ ```vcl
1237
+ #inline double(x)
1238
+ x + x
1239
+ #endinline
1240
+
1241
+ set var.result = double(1 + 2);
1242
+ // Expands to: (1 + 2) + (1 + 2) ✓ Correct
1243
+ ```
1244
+
1245
+ xvcl automatically adds parentheses when needed.
1246
+
1247
+ **If you see issues:** Check operator precedence in your macro definition.
1248
+
1249
+ ### Function calls not working
1250
+
1251
+ **Problem:** Function call doesn't get replaced.
1252
+
1253
+ ```vcl
1254
+ #def add(a INTEGER, b INTEGER) -> INTEGER
1255
+ return a + b;
1256
+ #enddef
1257
+
1258
+ set var.result = add(5, 10);
1259
+ ```
1260
+
1261
+ **Common causes:**
1262
+
1263
+ 1. **Missing semicolon:** Function calls must end with `;`
1264
+ ```vcl
1265
+ set var.result = add(5, 10); // ✓ Correct
1266
+ set var.result = add(5, 10) // ✗ Won't match
1267
+ ```
1268
+
1269
+ 2. **Wrong number of arguments:**
1270
+ ```vcl
1271
+ set var.result = add(5); // ✗ Expects 2 args
1272
+ ```
1273
+
1274
+ 3. **Typo in function name:**
1275
+ ```vcl
1276
+ set var.result = addr(5, 10); // ✗ Function 'addr' not defined
1277
+ ```
1278
+
1279
+ ### Performance issues
1280
+
1281
+ **Problem:** Compilation is slow.
1282
+
1283
+ **Common causes:**
1284
+
1285
+ 1. **Large loops:** `#for i in range(10000)` generates 10,000 copies
1286
+ 2. **Deep nesting:** Multiple nested loops or includes
1287
+ 3. **Complex macros:** Heavily nested macro expansions
1288
+
1289
+ **Solutions:**
1290
+
1291
+ 1. Reduce loop iterations if possible
1292
+ 2. Use functions instead of generating everything inline
1293
+ 3. Split into multiple source files
1294
+ 4. Profile with `--debug` to see what's slow
1295
+
1296
+ ### Getting help
1297
+
1298
+ **Check the error context:**
1299
+
1300
+ Errors show surrounding lines:
1301
+
1302
+ ```
1303
+ Error at main.xvcl:15:
1304
+ Invalid #for syntax: #for in range(10)
1305
+
1306
+ Context:
1307
+ 13: sub vcl_recv {
1308
+ 14: // Generate backends
1309
+ → 15: #for in range(10)
1310
+ 16: backend web{{i}} { ... }
1311
+ 17: #endfor
1312
+ 18: }
1313
+ ```
1314
+
1315
+ **Enable debug mode:**
1316
+
1317
+ ```bash
1318
+ xvcl main.xvcl --debug
1319
+ ```
1320
+
1321
+ **Validate generated VCL:**
1322
+
1323
+ ```bash
1324
+ falco lint main.vcl -vv
1325
+ ```
1326
+
1327
+ The `-vv` flag shows detailed Falco errors.
1328
+
1329
+ ---
1330
+
1331
+ ## Examples Gallery
1332
+
1333
+ Here are complete, working examples you can use as starting points.
1334
+
1335
+ ### Example 1: Multi-region backends
1336
+
1337
+ **`multi-region.xvcl`:**
1338
+
1339
+ ```vcl
1340
+ #const REGIONS = ["us-east", "us-west", "eu-west", "ap-south"]
1341
+ #const DEFAULT_REGION = "us-east"
1342
+
1343
+ #for region in REGIONS
1344
+ backend F_origin_{{region}} {
1345
+ .host = "origin-{{region}}.example.com";
1346
+ .port = "443";
1347
+ .ssl = true;
1348
+ .connect_timeout = 5s;
1349
+ .first_byte_timeout = 30s;
1350
+ .between_bytes_timeout = 10s;
1351
+ }
1352
+ #endfor
1353
+
1354
+ sub vcl_recv {
1355
+ declare local var.region STRING;
1356
+
1357
+ // Detect region from client IP or header
1358
+ if (req.http.X-Region) {
1359
+ set var.region = req.http.X-Region;
1360
+ } else {
1361
+ set var.region = "{{DEFAULT_REGION}}";
1362
+ }
1363
+
1364
+ // Route to appropriate backend
1365
+ #for region in REGIONS
1366
+ if (var.region == "{{region}}") {
1367
+ set req.backend = F_origin_{{region}};
1368
+ }
1369
+ #endfor
1370
+ }
1371
+ ```
1372
+
1373
+ ### Example 2: Feature flag system
1374
+
1375
+ **`feature-flags.xvcl`:**
1376
+
1377
+ ```vcl
1378
+ #const ENABLE_NEW_CACHE_POLICY = True
1379
+ #const ENABLE_WEBP_CONVERSION = True
1380
+ #const ENABLE_ANALYTICS = False
1381
+ #const ENABLE_DEBUG_HEADERS = False
1382
+
1383
+ sub vcl_recv {
1384
+ #if ENABLE_NEW_CACHE_POLICY
1385
+ // New cache policy with fine-grained control
1386
+ if (req.url.path ~ "\.(jpg|png|gif|css|js)$") {
1387
+ set req.http.X-Cache-Policy = "static";
1388
+ } else {
1389
+ set req.http.X-Cache-Policy = "dynamic";
1390
+ }
1391
+ #else
1392
+ // Legacy cache policy
1393
+ set req.http.X-Cache-Policy = "default";
1394
+ #endif
1395
+
1396
+ #if ENABLE_WEBP_CONVERSION
1397
+ if (req.http.Accept ~ "image/webp") {
1398
+ set req.http.X-Image-Format = "webp";
1399
+ }
1400
+ #endif
1401
+
1402
+ #if ENABLE_ANALYTICS
1403
+ set req.http.X-Analytics-ID = uuid.generate();
1404
+ #endif
1405
+ }
1406
+
1407
+ sub vcl_deliver {
1408
+ #if ENABLE_DEBUG_HEADERS
1409
+ set resp.http.X-Cache-Status = resp.http.X-Cache;
1410
+ set resp.http.X-Backend = req.backend;
1411
+ set resp.http.X-Region = req.http.X-Region;
1412
+ #endif
1413
+ }
1414
+ ```
1415
+
1416
+ ### Example 3: URL normalization library
1417
+
1418
+ **`url-utils.xvcl`:**
1419
+
1420
+ ```vcl
1421
+ // Inline macros for common URL operations
1422
+ #inline strip_www(host)
1423
+ regsub(host, "^www\.", "")
1424
+ #endinline
1425
+
1426
+ #inline lowercase_host(host)
1427
+ std.tolower(host)
1428
+ #endinline
1429
+
1430
+ #inline normalize_host(host)
1431
+ lowercase_host(strip_www(host))
1432
+ #endinline
1433
+
1434
+ #inline remove_trailing_slash(path)
1435
+ regsub(path, "/$", "")
1436
+ #endinline
1437
+
1438
+ #inline remove_query_string(url)
1439
+ regsub(url, "\?.*$", "")
1440
+ #endinline
1441
+
1442
+ // Function for complex normalization
1443
+ #def normalize_url(url STRING, host STRING) -> STRING
1444
+ declare local var.result STRING;
1445
+ declare local var.clean_host STRING;
1446
+ declare local var.clean_path STRING;
1447
+
1448
+ set var.clean_host = normalize_host(host);
1449
+ set var.clean_path = remove_trailing_slash(url);
1450
+ set var.result = "https://" + var.clean_host + var.clean_path;
1451
+
1452
+ return var.result;
1453
+ #enddef
1454
+
1455
+ sub vcl_recv {
1456
+ declare local var.canonical_url STRING;
1457
+ set var.canonical_url = normalize_url(req.url.path, req.http.Host);
1458
+ set req.http.X-Canonical-URL = var.canonical_url;
1459
+ }
1460
+ ```
1461
+
1462
+ ### Example 4: A/B testing framework
1463
+
1464
+ **`ab-testing.xvcl`:**
1465
+
1466
+ ```vcl
1467
+ #const EXPERIMENTS = [
1468
+ ("homepage_hero", 50),
1469
+ ("checkout_flow", 30),
1470
+ ("pricing_page", 25)
1471
+ ]
1472
+
1473
+ #def assign_experiment(exp_id STRING, percentage INTEGER) -> BOOL
1474
+ declare local var.hash STRING;
1475
+ declare local var.value INTEGER;
1476
+ declare local var.assigned BOOL;
1477
+
1478
+ set var.hash = digest.hash_md5(client.ip + exp_id);
1479
+ set var.value = std.atoi(substr(var.hash, 0, 2)) % 100;
1480
+ set var.assigned = (var.value < percentage);
1481
+
1482
+ return var.assigned;
1483
+ #enddef
1484
+
1485
+ sub vcl_recv {
1486
+ declare local var.in_experiment BOOL;
1487
+
1488
+ #for exp_id, percentage in EXPERIMENTS
1489
+ set var.in_experiment = assign_experiment("{{exp_id}}", {{percentage}});
1490
+ if (var.in_experiment) {
1491
+ set req.http.X-Experiment-{{exp_id}} = "variant";
1492
+ } else {
1493
+ set req.http.X-Experiment-{{exp_id}} = "control";
1494
+ }
1495
+ #endfor
1496
+ }
1497
+ ```
1498
+
1499
+ ---
1500
+
1501
+ ## Summary
1502
+
1503
+ xvcl extends Fastly VCL with powerful programming constructs:
1504
+
1505
+ **Core features:**
1506
+
1507
+ - Constants - Single source of truth for configuration
1508
+ - Template expressions - Dynamic value substitution
1509
+ - For loops - Generate repetitive code
1510
+ - Conditionals - Environment-specific builds
1511
+ - Variables - Cleaner local variable syntax
1512
+ - Includes - Modular code organization
1513
+
1514
+ **Advanced features:**
1515
+
1516
+ - Inline macros - Zero-overhead text substitution
1517
+ - Functions - Reusable logic with return values
1518
+
1519
+ **Benefits:**
1520
+
1521
+ - Less code duplication
1522
+ - Fewer copy-paste errors
1523
+ - Better maintainability
1524
+ - Easier testing
1525
+ - Faster development
1526
+
1527
+ **Integration:**
1528
+
1529
+ - Works with Falco's full toolset
1530
+ - Standard VCL output
1531
+ - No runtime overhead
1532
+ - Easy CI/CD integration
1533
+
1534
+ Start simple, add complexity as needed. xvcl grows with your VCL projects.