@redpanda-data/docs-extensions-and-macros 4.11.1 → 4.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (31) hide show
  1. package/bin/doc-tools.js +201 -10
  2. package/package.json +3 -1
  3. package/tools/property-extractor/COMPUTED_CONSTANTS.md +173 -0
  4. package/tools/property-extractor/Makefile +12 -1
  5. package/tools/property-extractor/README.adoc +828 -97
  6. package/tools/property-extractor/compare-properties.js +38 -13
  7. package/tools/property-extractor/constant_resolver.py +610 -0
  8. package/tools/property-extractor/file_pair.py +42 -0
  9. package/tools/property-extractor/generate-handlebars-docs.js +41 -8
  10. package/tools/property-extractor/helpers/gt.js +9 -0
  11. package/tools/property-extractor/helpers/includes.js +17 -0
  12. package/tools/property-extractor/helpers/index.js +3 -0
  13. package/tools/property-extractor/helpers/isEnterpriseEnum.js +24 -0
  14. package/tools/property-extractor/helpers/renderPropertyExample.js +6 -5
  15. package/tools/property-extractor/overrides.json +248 -0
  16. package/tools/property-extractor/parser.py +254 -32
  17. package/tools/property-extractor/property_bag.py +40 -0
  18. package/tools/property-extractor/property_extractor.py +1417 -430
  19. package/tools/property-extractor/requirements.txt +1 -0
  20. package/tools/property-extractor/templates/property-backup.hbs +161 -0
  21. package/tools/property-extractor/templates/property.hbs +104 -49
  22. package/tools/property-extractor/templates/topic-property-backup.hbs +148 -0
  23. package/tools/property-extractor/templates/topic-property.hbs +72 -34
  24. package/tools/property-extractor/tests/test_known_values.py +617 -0
  25. package/tools/property-extractor/tests/transformers_test.py +81 -6
  26. package/tools/property-extractor/topic_property_extractor.py +23 -10
  27. package/tools/property-extractor/transformers.py +2191 -369
  28. package/tools/property-extractor/type_definition_extractor.py +669 -0
  29. package/tools/redpanda-connect/helpers/renderConnectFields.js +33 -1
  30. package/tools/redpanda-connect/report-delta.js +132 -9
  31. package/tools/property-extractor/definitions.json +0 -245
@@ -1,7 +1,317 @@
1
+ #!/usr/bin/env python3
2
+ """
3
+ Redpanda Property Transformers - Configuration Property Processing Pipeline
4
+
5
+ This module contains a comprehensive set of transformers that process C++ configuration property
6
+ declarations extracted from Redpanda's source code and convert them into structured JSON schema
7
+ definitions suitable for documentation generation.
8
+
9
+ ================================================================================
10
+ OVERVIEW & ARCHITECTURE
11
+ ================================================================================
12
+
13
+ The transformation pipeline converts raw C++ property declarations into standardized JSON objects
14
+ that can be consumed by documentation generators, validation systems, and other downstream tools.
15
+
16
+ TRANSFORMATION PIPELINE FLOW:
17
+ 1. Tree-sitter parses C++ source → Raw AST nodes
18
+ 2. Parser extracts property declarations → Structured info dicts
19
+ 3. Transformers process info dicts → Normalized PropertyBag objects
20
+ 4. PropertyBags serialized → Final JSON schema output
21
+
22
+ INPUT FORMAT (from parser):
23
+ - info["declaration"]: Full C++ type declaration
24
+ - info["params"]: List of parsed constructor parameters
25
+ - info["name_in_file"]: C++ variable name
26
+ - info["type"]: Property template type (e.g., "property", "enterprise_property")
27
+
28
+ OUTPUT FORMAT (PropertyBag):
29
+ - Complete JSON schema-compatible property definition
30
+ - Normalized types, defaults, bounds, metadata
31
+ - Ready for handlebars template consumption
32
+
33
+ ================================================================================
34
+ TRANSFORMER EXECUTION ORDER & DEPENDENCIES
35
+ ================================================================================
36
+
37
+ Transformers are applied in a specific order to ensure dependencies are resolved correctly:
38
+
39
+ 1. ParamNormalizerTransformer - Standardizes parameter ordering for enterprise properties
40
+ 2. BasicInfoTransformer - Extracts basic name, description, file location
41
+ 3. MetaParamTransformer - Parses C++ meta{} initializers into structured data
42
+ 4. NeedsRestartTransformer - Extracts restart requirements from meta
43
+ 5. GetsRestoredTransformer - Extracts backup/restore flags from meta
44
+ 6. IsSecretTransformer - Identifies secret/sensitive properties from meta
45
+ 7. VisibilityTransformer - Determines property visibility (user/tunable/deprecated) from meta
46
+ 8. IsNullableTransformer - Determines if property can be null/unset
47
+ 9. IsArrayTransformer - Identifies array types (std::vector, one_or_many_property)
48
+ 10. TypeTransformer - Maps C++ types to JSON Schema types
49
+ 11. DeprecatedTransformer - Marks deprecated properties from meta
50
+ 12. NumericBoundsTransformer - Calculates min/max bounds for integer types
51
+ 13. DurationBoundsTransformer - Calculates bounds for std::chrono duration types
52
+ 14. SimpleDefaultValuesTransformer - Extracts simple default values
53
+ 15. FriendlyDefaultTransformer - Converts C++ defaults to human-readable format
54
+ 16. ExperimentalTransformer - Marks experimental features from meta
55
+ 17. AliasTransformer - Extracts property aliases from meta
56
+ 18. EnterpriseTransformer - Handles enterprise-only feature restrictions
57
+
58
+ ================================================================================
59
+ KEY CONCEPTS & DATA STRUCTURES
60
+ ================================================================================
61
+
62
+ PROPERTY TYPES HANDLED:
63
+ - property<T> - Standard Redpanda config property
64
+ - enterprise_property<T> - Enterprise edition only property
65
+ - deprecated_property<T> - Deprecated property (generates warnings)
66
+ - one_or_many_property<T> - Accepts single value OR array of values
67
+
68
+ SPECIAL C++ PATTERNS PROCESSED:
69
+ - std::optional<T> - Nullable properties
70
+ - std::vector<T> - Array properties
71
+ - std::chrono::duration types - Time duration properties with bounds
72
+ - Integer types (int32_t, etc.) - Numeric properties with automatic bounds
73
+ - meta{.key = value, ...} - Redpanda metadata initializers
74
+
75
+ ================================================================================
76
+ PROPERTY ARITIES - UNDERSTANDING CONSTRUCTOR PARAMETER PATTERNS
77
+ ================================================================================
78
+
79
+ Redpanda configuration properties are C++ objects with constructor signatures that vary
80
+ based on feature requirements. Understanding these "arities" (parameter counts) is crucial
81
+ for correctly extracting property metadata.
82
+
83
+ BASIC PROPERTY PATTERNS:
84
+ ┌─────────────────────────────────────────────────────────────────────────────
85
+ │ 2-PARAMETER: property<T>(name, description)
86
+ │ Example: property<bool>(*this, "enable_feature", "Enable the feature")
87
+ │ Used for: Simple properties with no metadata or custom defaults
88
+ │ Extraction: [0] = name, [1] = description
89
+ └─────────────────────────────────────────────────────────────────────────────
90
+ ┌─────────────────────────────────────────────────────────────────────────────
91
+ │ 3-PARAMETER: property<T>(name, description, default)
92
+ │ Example: property<int>(*this, "port", "Server port", 9092)
93
+ │ Used for: Properties with simple custom default values
94
+ │ Extraction: [0] = name, [1] = description, [2] = default
95
+ └─────────────────────────────────────────────────────────────────────────────
96
+ ┌─────────────────────────────────────────────────────────────────────────────
97
+ │ 4-PARAMETER: property<T>(name, description, meta, default)
98
+ │ Example: property<bool>(*this, "flag", "Description", meta{.needs_restart=yes}, true)
99
+ │ Used for: Properties with metadata (restart requirements, visibility, etc.)
100
+ │ Extraction: [0] = name, [1] = description, [2] = meta{}, [3] = default
101
+ └─────────────────────────────────────────────────────────────────────────────
102
+
103
+ ENTERPRISE PROPERTY PATTERNS (More Complex):
104
+ ┌─────────────────────────────────────────────────────────────────────────────
105
+ │ 3-PARAMETER ENTERPRISE: enterprise_property<T>(name, description, default)
106
+ │ Example: enterprise_property<bool>(*this, "audit_enabled", "Enable auditing", false)
107
+ │ Used for: Enterprise features with simple restriction (all enterprise values)
108
+ │ Extraction: [0] = name, [1] = description, [2] = default
109
+ │ Note: No explicit restriction vector means "any enterprise value"
110
+ └─────────────────────────────────────────────────────────────────────────────
111
+ ┌─────────────────────────────────────────────────────────────────────────────
112
+ │ 4-PARAMETER ENTERPRISE: enterprise_property<T>(name, description, meta, default)
113
+ │ Example: enterprise_property<int>(*this, "limit", "Limit", meta{.secret=yes}, 100)
114
+ │ Used for: Enterprise features with metadata but no specific value restrictions
115
+ │ Extraction: [0] = name, [1] = description, [2] = meta{}, [3] = default
116
+ └─────────────────────────────────────────────────────────────────────────────
117
+ ┌─────────────────────────────────────────────────────────────────────────────
118
+ │ 5-PARAMETER ENTERPRISE WITH RESTRICTIONS:
119
+ │ Pattern A: enterprise_property<T>(restrictions, name, description, meta, default)
120
+ │ Pattern B: enterprise_property<T>(name, restrictions, description, meta, default)
121
+
122
+ │ Example A: enterprise_property<string>(
123
+ │ std::vector<ss::sstring>{"value1", "value2"},
124
+ │ *this, "feature_mode", "Operating mode", meta{}, "value1"
125
+ │ )
126
+
127
+ │ Example B: enterprise_property<string>(
128
+ │ *this, "feature_mode",
129
+ │ std::vector<ss::sstring>{"value1", "value2"},
130
+ │ "Operating mode", meta{}, "value1"
131
+ │ )
132
+
133
+ │ Used for: Enterprise features with specific allowed values per license tier
134
+ │ Extraction: ParamNormalizerTransformer detects and skips restriction vectors
135
+ │ After normalization: [0] = name, [1] = description, [2] = meta{}, [3] = default
136
+
137
+ │ Detection: Check for "std::vector" in params[0] or params[1] value string
138
+ │ Processing: EnterpriseTransformer extracts restriction vector to populate enterprise_value
139
+ └─────────────────────────────────────────────────────────────────────────────
140
+
141
+ PARAMETER POSITION VARIATIONS - WHY NORMALIZATION IS NEEDED:
142
+ The same semantic information appears at different parameter indices depending on:
143
+ 1. Whether property is standard vs enterprise (affects parameter count)
144
+ 2. Whether enterprise restrictions are present (shifts all subsequent parameters)
145
+ 3. Whether metadata is included (adds meta{} parameter)
146
+ 4. Constructor pattern evolution (C++ codebase changes over time)
147
+
148
+ EXAMPLE OF POSITION VARIATION:
149
+ ┌─────────────────────────────────────────────────────────────────────────────
150
+ │ Standard property: params[0]=name, params[1]=desc, params[2]=meta, params[3]=default
151
+ │ Enterprise (no restrict): params[0]=name, params[1]=desc, params[2]=meta, params[3]=default
152
+ │ Enterprise (restrict@0): params[0]=RESTRICTIONS, params[1]=name, params[2]=desc, params[3]=meta, params[4]=default
153
+ │ Enterprise (restrict@1): params[0]=name, params[1]=RESTRICTIONS, params[2]=desc, params[3]=meta, params[4]=default
154
+
155
+ │ Solution: ParamNormalizerTransformer shifts params to create consistent layout
156
+ │ After normalization, ALL properties have: [0]=name, [1]=desc, [2]=meta, [3]=default
157
+ └─────────────────────────────────────────────────────────────────────────────
158
+
159
+ VALIDATORS AND SKIPPED PARAMETERS:
160
+ Some properties include validator lambdas or callable parameters that must be skipped:
161
+ ┌─────────────────────────────────────────────────────────────────────────────
162
+ │ property<T>([](const T& v) { return v > 0; }, name, description, default)
163
+ │ └──────────────────────────────┘
164
+ │ Lambda validator
165
+
166
+ │ BasicInfoTransformer skips these to find the first string literal (the name)
167
+ │ Detection: params with type "lambda_expression" or validator-like structure
168
+ └─────────────────────────────────────────────────────────────────────────────
169
+
170
+ HOW TO ADD SUPPORT FOR NEW PARAMETER PATTERNS:
171
+ 1. Identify the parameter count and positions of name/description/meta/default
172
+ 2. Update ParamNormalizerTransformer if new enterprise patterns are added
173
+ 3. Update BasicInfoTransformer if name/description extraction needs adjustment
174
+ 4. Update EnterpriseTransformer if new restriction patterns are introduced
175
+ 5. Test with properties following the new pattern to verify extraction
176
+ 6. Document the new pattern in this section for future maintainers
177
+
178
+ DOWNSTREAM CONSUMPTION:
179
+ The transformed PropertyBag objects are consumed by:
180
+ - generate-handlebars-docs.js - Documentation generation
181
+ - property_extractor.py - Final JSON schema assembly
182
+
183
+ ================================================================================
184
+ DEBUGGING & MAINTENANCE
185
+ ================================================================================
186
+
187
+ DEBUG SYSTEM:
188
+ - @debug_transformer decorator logs before/after state for each transformer
189
+ - DEBUG_TRANSFORMERS flag enables/disables debugging globally
190
+ - DEBUG_FILTER narrows logging to specific property names
191
+ - Full parameter and property state logged for troubleshooting
192
+
193
+ ADDING NEW TRANSFORMERS:
194
+ 1. Inherit from transformer pattern (accepts() + parse() methods)
195
+ 2. Add @debug_transformer decorator for debugging support
196
+ 3. Insert in proper execution order in main pipeline
197
+ 4. Document expected inputs/outputs and downstream dependencies
198
+ 5. Add comprehensive docstrings following this module's patterns
199
+
200
+ MAINTENANCE NOTES:
201
+ - Transformers depend on Tree-sitter AST structure - changes to parser may require updates
202
+ - C++ code patterns change over time - watch for new constructor patterns
203
+ - JSON Schema evolution may require type mapping updates
204
+ - Enterprise feature detection logic may need updates for new licensing models
205
+
206
+ ================================================================================
207
+ """
208
+
1
209
  import re
2
210
  import logging
3
211
  from property_bag import PropertyBag
4
212
  from parser import normalize_string
213
+ import pprint
214
+
215
+ # Compiled regex patterns for performance optimization
216
+ DOT_ASSIGNMENT_PATTERN = re.compile(r"\.([A-Za-z_]+)\s*=\s*([A-Za-z0-9_:]+)")
217
+ NAMESPACE_STRIP_PATTERN = re.compile(r"^.*::")
218
+ NUMERIC_PATTERN = re.compile(r"^-?\d+(\.\d+)?$")
219
+ PROPERTY_TEMPLATE_PATTERN = re.compile(r"^.*property<(.+)>.*")
220
+ OPTIONAL_TEMPLATE_PATTERN = re.compile(r".*std::optional<(.+)>.*")
221
+ VECTOR_TEMPLATE_PATTERN = re.compile(r".*std::vector<(.+)>.*")
222
+ ONE_OR_MANY_PATTERN = re.compile(r".*one_or_many_property<(.+)>.*")
223
+ DEPRECATED_PROPERTY_PATTERN = re.compile(r".*deprecated_property<(.+)>.*")
224
+ NUMERIC_TYPE_PATTERN = re.compile(r"^(unsigned|u?int(8|16|32|64)?(_t)?)")
225
+ CHRONO_DECLARATION_PATTERN = re.compile(r"std::chrono::")
226
+ LEGACY_DEFAULT_PATTERN = re.compile(r"legacy_default<[^>]+>\(([^,]+)")
227
+ BRACED_CONTENT_PATTERN = re.compile(r"^\{.*\}$")
228
+ BRACED_EXTRACT_PATTERN = re.compile(r"^\{(.*)\}$")
229
+ FLOAT_PATTERN = re.compile(r"^-?\d+\.\d+$")
230
+ INT_PATTERN = re.compile(r"^-?\d+$")
231
+ SIZE_SUFFIX_PATTERN = re.compile(r"(\d+)_([KMGTP])iB")
232
+
233
+ # Computed C++ constant definitions that require evaluation
234
+ # These are constants defined with complex expressions that can't be easily parsed
235
+ # Values are computed from the C++ definitions
236
+ COMPUTED_CONSTANTS = {
237
+ # From src/v/serde/rw/chrono.h:20
238
+ # inline constexpr auto max_serializable_ms = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::nanoseconds::max());
239
+ # Calculation: std::numeric_limits<int64_t>::max() / 1,000,000 = 9223372036854775807 / 1000000 = 9223372036854 ms
240
+ "max_serializable_ms": 9223372036854, # ~292 years in milliseconds
241
+ "serde::max_serializable_ms": 9223372036854, # Namespace-qualified version
242
+ }
243
+
244
+ # Debug configuration - useful for development and troubleshooting
245
+ DEBUG_TRANSFORMERS = False # Master switch for transformer debugging
246
+ DEBUG_FILTER = None # Filter to specific property name (or None for all)
247
+
248
+ def debug_transformer(cls):
249
+ """
250
+ Decorator that wraps transformer parse() methods to provide detailed debugging information.
251
+
252
+ This decorator is essential for debugging the transformation pipeline. It logs the complete
253
+ state of parameters and properties before and after each transformer executes, making it
254
+ easy to trace how properties are being processed and identify issues.
255
+
256
+ DEBUGGING OUTPUT INCLUDES:
257
+ - Transformer class name being executed
258
+ - Property name being processed
259
+ - Full parameter list before transformation
260
+ - Full parameter list after transformation
261
+ - Updated property keys after transformation
262
+ - Error details if transformation fails
263
+
264
+ USAGE:
265
+ Apply @debug_transformer decorator to any transformer class:
266
+
267
+ @debug_transformer
268
+ class MyTransformer:
269
+ def accepts(self, info, file_pair): ...
270
+ def parse(self, property, info, file_pair): ...
271
+
272
+ FILTERING:
273
+ Use DEBUG_FILTER to narrow logging to specific properties:
274
+ DEBUG_FILTER = "kafka_api" # Only log kafka_api property transformations
275
+ DEBUG_FILTER = None # Log all property transformations
276
+
277
+ Args:
278
+ cls: The transformer class to wrap with debugging capabilities
279
+
280
+ Returns:
281
+ The wrapped transformer class with debug logging in parse() method
282
+ """
283
+ orig_parse = cls.parse
284
+
285
+ def wrapped_parse(self, property, info, file_pair):
286
+ if not DEBUG_TRANSFORMERS:
287
+ return orig_parse(self, property, info, file_pair)
288
+
289
+ name = property.get("name") or info.get("name_in_file") or "?"
290
+ if DEBUG_FILTER and DEBUG_FILTER not in str(name):
291
+ return orig_parse(self, property, info, file_pair)
292
+
293
+ print("\n" + "=" * 80)
294
+ print(f"🔍 Transformer: {cls.__name__}")
295
+ print(f"Property: {name}")
296
+ print(f"Params BEFORE ({len(info.get('params', []))}):")
297
+ pprint.pp(info.get("params", []))
298
+
299
+ try:
300
+ result = orig_parse(self, property, info, file_pair)
301
+ except Exception as e:
302
+ print(f"❌ ERROR in {cls.__name__}: {e}")
303
+ raise
304
+
305
+ print(f"Params AFTER ({len(info.get('params', []))}):")
306
+ pprint.pp(info.get("params", []))
307
+ print(f"Updated property keys: {list(property.keys())}")
308
+ print("=" * 80 + "\n")
309
+
310
+ return result
311
+
312
+ cls.parse = wrapped_parse
313
+ return cls
314
+
5
315
 
6
316
  # Get logger for this module
7
317
  logger = logging.getLogger(__name__)
@@ -12,12 +322,30 @@ logger = logging.getLogger(__name__)
12
322
  # the centralized enterprise value processing logic without creating import cycles.
13
323
  def get_process_enterprise_value():
14
324
  """
15
- Lazily load the centralized process_enterprise_value function from property_extractor.
325
+ Lazily import the centralized enterprise value processing function to avoid circular imports.
326
+
327
+ The property_extractor module imports transformers.py, so we cannot import property_extractor
328
+ at module level. This lazy loading pattern allows EnterpriseTransformer to access the
329
+ centralized enterprise value processing logic without creating circular dependencies.
16
330
 
17
- Attempts to import and return the `process_enterprise_value` callable; logs an error and returns `None` if the import fails.
331
+ ENTERPRISE VALUE PROCESSING:
332
+ The process_enterprise_value function converts enterprise-specific C++ expressions into
333
+ JSON-compatible values for restricted feature handling. Examples include:
334
+ - Converting enterprise feature flags to boolean values
335
+ - Processing enterprise license restriction lists
336
+ - Normalizing enterprise property default values
337
+
338
+ CIRCULAR IMPORT AVOIDANCE:
339
+ - property_extractor.py imports transformers.py (needs transformer classes)
340
+ - transformers.py needs process_enterprise_value from property_extractor.py
341
+ - Solution: Lazy import at function call time, not at module import time
18
342
 
19
343
  Returns:
20
- The `process_enterprise_value` callable if available, `None` otherwise.
344
+ callable or None: The process_enterprise_value function if successfully imported,
345
+ None if import fails (with error logged)
346
+
347
+ Raises:
348
+ Does not raise - logs ImportError and returns None on failure
21
349
  """
22
350
  try:
23
351
  from property_extractor import process_enterprise_value
@@ -27,14 +355,200 @@ def get_process_enterprise_value():
27
355
  return None
28
356
 
29
357
 
358
+ def is_validator_param(p):
359
+ """
360
+ Determine if a parameter represents a validator function or callable.
361
+
362
+ Redpanda properties often include validator functions to ensure configuration values
363
+ are valid. These validators are C++ callables that are not relevant for documentation
364
+ but need to be identified and filtered out during parameter processing.
365
+
366
+ VALIDATOR DETECTION CRITERIA:
367
+ 1. Contains validator-related keywords ("validator", "validate_", "checker")
368
+ 2. Has type indicating it's a function identifier ("qualified_identifier", "unresolved_identifier")
369
+ 3. Appears to be a simple identifier without complex syntax (no braces/commas)
370
+
371
+ EXAMPLES OF VALIDATOR PARAMETERS:
372
+ - validate_memory_size (function name)
373
+ - config::memory_validator (qualified validator)
374
+ - cluster_size_checker (validation callable)
375
+
376
+ NON-VALIDATOR PARAMETERS:
377
+ - {.needs_restart = no} (meta initializer)
378
+ - "default_value" (string literal)
379
+ - 42 (numeric literal)
380
+
381
+ Args:
382
+ p (dict): Parameter dictionary with 'value' and 'type' keys
383
+
384
+ Returns:
385
+ bool: True if parameter appears to be a validator function, False otherwise
386
+ """
387
+ if not isinstance(p, dict):
388
+ return False
389
+ val = str(p.get("value", "")).strip()
390
+ typ = p.get("type", "")
391
+
392
+ # String literals and lambda expressions are never validators
393
+ if typ in ("string_literal", "lambda_expression"):
394
+ return False
395
+
396
+ # Check for validator-related keywords
397
+ if any(x in val for x in ("validator", "validate_", "checker")):
398
+ return True
399
+
400
+ # Check for identifier types that typically represent functions
401
+ if typ in ("qualified_identifier", "unresolved_identifier"):
402
+ return True
403
+
404
+ # Simple identifiers without complex syntax are often validators
405
+ if not ("{" in val or "," in val or "}" in val):
406
+ return True
407
+
408
+ return False
409
+
410
+
411
+ def find_meta_dict(info):
412
+ """
413
+ Locate and extract Redpanda property metadata from C++ meta initializers.
414
+
415
+ Redpanda uses a special C++ pattern for specifying property metadata through
416
+ meta{} initializers. This function finds and parses these metadata blocks
417
+ from the raw parameter data extracted by the Tree-sitter parser.
418
+
419
+ SUPPORTED META SYNTAX PATTERNS:
420
+ 1. Explicit meta wrapper:
421
+ meta{ .needs_restart = needs_restart::no, .visibility = visibility::user }
422
+
423
+ 2. Bare initializer (detected by presence of metadata keys):
424
+ {.needs_restart = needs_restart::no, .visibility = visibility::tunable}
425
+
426
+ METADATA KEYS RECOGNIZED:
427
+ - needs_restart: Whether changing this property requires broker restart
428
+ - visibility: Property visibility level (user, tunable, deprecated)
429
+ - deprecated: Deprecation status and reason
430
+ - secret: Whether property contains sensitive data
431
+ - experimental: Experimental feature flag
432
+ - gets_restored: Whether property is included in backup/restore
433
+ - example: Example values for documentation
434
+
435
+ PROCESSING STATES:
436
+ 1. Raw string form: Before MetaParamTransformer processes initializers
437
+ 2. Parsed dict form: After MetaParamTransformer converts to structured data
438
+
439
+ This function handles both states to support early metadata access before
440
+ the full transformation pipeline completes.
441
+
442
+ Args:
443
+ info (dict): Property info dictionary containing 'params' list
444
+
445
+ Returns:
446
+ dict or None: Parsed metadata dictionary with normalized keys,
447
+ None if no metadata found
448
+
449
+ Examples:
450
+ Raw input: "meta{ .visibility = visibility::user }"
451
+ Output: {"visibility": "user"}
452
+
453
+ Parsed input: {"needs_restart": "needs_restart::no", "type": "initializer_list"}
454
+ Output: {"needs_restart": "needs_restart::no"}
455
+ """
456
+ for p in info.get("params", []):
457
+ val = p.get("value")
458
+
459
+ # Case 1: Already parsed dict
460
+ if isinstance(val, dict) and any(
461
+ k in val for k in ("needs_restart", "visibility", "deprecated", "secret", "experimental")
462
+ ):
463
+ return val
464
+
465
+ # Case 2: Raw string form (early lookup before MetaParamTransformer)
466
+ if isinstance(val, str) and ("meta{" in val or val.strip().startswith("{")):
467
+ # Minimal regex extraction for .key = value pairs
468
+ matches = DOT_ASSIGNMENT_PATTERN.findall(val)
469
+ if matches:
470
+ return {k: v for k, v in matches}
471
+ return None
472
+
473
+ def get_meta_value(info, key, default=None):
474
+ """
475
+ Extract and normalize a specific metadata value from property metadata.
476
+
477
+ This is a convenience function that combines metadata discovery with value
478
+ normalization. It handles the common pattern of extracting metadata values
479
+ and stripping C++ namespace qualifiers to get clean, usable values.
480
+
481
+ VALUE NORMALIZATION:
482
+ C++ metadata often uses qualified identifiers like "needs_restart::no" or
483
+ "visibility::user". This function strips the namespace portion (everything
484
+ before "::") to get the clean value ("no", "user").
485
+
486
+ COMMON USAGE PATTERNS:
487
+ - get_meta_value(info, "needs_restart", "no") → "no" or "yes"
488
+ - get_meta_value(info, "visibility", "user") → "user", "tunable", "deprecated"
489
+ - get_meta_value(info, "deprecated") → None or deprecation reason
490
+
491
+ Args:
492
+ info (dict): Property info dictionary to search for metadata
493
+ key (str): Metadata key to extract (e.g., "needs_restart", "visibility")
494
+ default (any): Default value if key not found or metadata missing
495
+
496
+ Returns:
497
+ any: Normalized metadata value with C++ qualifiers stripped,
498
+ or default if key not found
499
+
500
+ Examples:
501
+ Input meta: {"needs_restart": "needs_restart::yes"}
502
+ get_meta_value(info, "needs_restart") → "yes"
503
+
504
+ Input meta: {"visibility": "visibility::tunable"}
505
+ get_meta_value(info, "visibility") → "tunable"
506
+
507
+ Missing key:
508
+ get_meta_value(info, "nonexistent", "default") → "default"
509
+ """
510
+ meta = find_meta_dict(info)
511
+ if not meta or key not in meta:
512
+ return default
513
+ val = meta[key]
514
+ # Strip C++ namespace qualifiers like "needs_restart::no" → "no"
515
+ if isinstance(val, str):
516
+ return NAMESPACE_STRIP_PATTERN.sub("", val)
517
+ return val
518
+
519
+
30
520
  def get_resolve_constexpr_identifier():
31
521
  """
32
- Lazily import and return the `resolve_constexpr_identifier` function from `property_extractor`.
522
+ Lazily import constexpr identifier resolution function to avoid circular imports.
523
+
524
+ This function provides access to the constexpr identifier resolution logic from
525
+ property_extractor.py while avoiding circular import issues. The constexpr resolver
526
+ looks up C++ constexpr variable definitions in source code to get their literal values.
33
527
 
34
- Attempts to import `resolve_constexpr_identifier` and return it to avoid circular-import issues.
528
+ CONSTEXPR RESOLUTION PROCESS:
529
+ 1. C++ property defaults often reference constexpr variables by name
530
+ 2. The resolver searches Redpanda source files for constexpr definitions
531
+ 3. It extracts the literal value assigned to the constexpr variable
532
+ 4. The literal value is used as the actual property default
533
+
534
+ EXAMPLES OF CONSTEXPR RESOLUTION:
535
+ - C++ default: "default_client_id" → Searches for: constexpr auto default_client_id = "redpanda";
536
+ - Resolved to: "redpanda"
537
+
538
+ - C++ default: "kafka_group_topic" → Searches for: constexpr string_view kafka_group_topic = "__consumer_offsets";
539
+ - Resolved to: "__consumer_offsets"
540
+
541
+ CIRCULAR IMPORT HANDLING:
542
+ - property_extractor.py imports transformers.py for transformer classes
543
+ - FriendlyDefaultTransformer needs resolve_constexpr_identifier from property_extractor.py
544
+ - Solution: Lazy import at call time prevents circular dependency
35
545
 
36
546
  Returns:
37
- Callable or None: The `resolve_constexpr_identifier` callable when available, otherwise `None`.
547
+ callable or None: The resolve_constexpr_identifier function if import succeeds,
548
+ None if import fails (with exception logged)
549
+
550
+ Raises:
551
+ Does not raise - logs exceptions and returns None on import failure
38
552
  """
39
553
  try:
40
554
  from property_extractor import resolve_constexpr_identifier
@@ -43,52 +557,274 @@ def get_resolve_constexpr_identifier():
43
557
  logger.exception("Cannot import resolve_constexpr_identifier from property_extractor: %s", e)
44
558
  return None
45
559
 
46
-
560
+ @debug_transformer
47
561
  class BasicInfoTransformer:
562
+ """
563
+ Extract fundamental property information: name, description, and source file location.
564
+
565
+ This is typically the first transformer in the pipeline and establishes the basic
566
+ property identity that other transformers build upon. It handles the core property
567
+ metadata that appears in virtually all property declarations.
568
+
569
+ PROCESSING RESPONSIBILITIES:
570
+ 1. Property name resolution with robust fallback logic
571
+ 2. Source file path normalization
572
+ 3. Description extraction from constructor parameters
573
+
574
+ NAME RESOLUTION PRIORITY:
575
+ 1. Pre-existing property["name"] (from previous processing)
576
+ 2. info["name_in_file"] (C++ variable name from Tree-sitter)
577
+ 3. First parameter value if it looks like an identifier
578
+
579
+ DESCRIPTION EXTRACTION LOGIC:
580
+ 1. If first parameter is a string and differs from name → use as description
581
+ 2. Otherwise, try second parameter if it's a string
582
+ 3. Handles both quoted and unquoted string values
583
+
584
+ SOURCE FILE NORMALIZATION:
585
+ Converts absolute paths to relative paths starting with "src/" for consistent
586
+ documentation references regardless of build environment.
587
+
588
+ EXPECTED FINAL RESULT:
589
+ {
590
+ "name": "property_name", # Clean identifier for JSON keys
591
+ "description": "Human readable...", # Description for documentation
592
+ "defined_in": "src/v/config/..." # Normalized file path
593
+ }
594
+
595
+ DOWNSTREAM USAGE:
596
+ - generate-handlebars-docs.js uses "name" for property identification
597
+ - Documentation templates use "description" for user-facing text
598
+ - Source links use "defined_in" for GitHub integration
599
+ """
600
+
48
601
  def accepts(self, info, file_pair):
49
- """
50
- Always accepts the provided info and file_pair.
51
-
52
- Parameters:
53
- info (dict): Parsed metadata for a property (annotation/params/declaration).
54
- file_pair (object): Pair of source/implementation file metadata used by transformers.
55
-
56
- Returns:
57
- bool: Always returns True, indicating this transformer should be applied.
58
- """
59
602
  return True
60
603
 
61
604
  def parse(self, property, info, file_pair):
62
- if not info.get("params") or len(info["params"]) == 0:
605
+ params = info.get("params") or []
606
+ if not params:
63
607
  return property
64
-
65
- property["name"] = info["params"][0]["value"]
608
+
609
+ # --- Step 1: find the "real" start of the property definition ---
610
+ # Skip lambdas, validators, and non-string literals at the start
611
+ start_idx = 0
612
+ for i, p in enumerate(params):
613
+ val = str(p.get("value", ""))
614
+ typ = p.get("type", "")
615
+ if is_validator_param(p):
616
+ continue
617
+ if typ in ("lambda_expression", "qualified_identifier", "unresolved_identifier"):
618
+ continue
619
+ if not (val.startswith('"') and val.endswith('"')):
620
+ continue
621
+ # First string literal we hit is the name
622
+ start_idx = i
623
+ break
624
+
625
+ # --- Step 2: extract name and description robustly ---
626
+ name = property.get("name") or info.get("name_in_file")
627
+ if not name and len(params) > start_idx:
628
+ name = params[start_idx].get("value", "").strip('"')
629
+ property["name"] = name
630
+
631
+ desc = None
632
+ if len(params) > start_idx + 1:
633
+ v0 = params[start_idx].get("value")
634
+ v1 = params[start_idx + 1].get("value")
635
+ if isinstance(v1, str) and len(v1) > 10 and " " in v1:
636
+ desc = v1
637
+ elif isinstance(v0, str) and len(v0) > 10 and " " in v0:
638
+ desc = v0
639
+ property["description"] = desc
640
+
641
+ # --- Step 3: defined_in ---
66
642
  property["defined_in"] = re.sub(
67
643
  r"^.*src/", "src/", str(file_pair.implementation)
68
644
  )
69
- property["description"] = (
70
- info["params"][1]["value"] if len(info["params"]) > 1 else None
71
- )
645
+ return property
646
+
647
+ @debug_transformer
648
+ class ParamNormalizerTransformer:
649
+ """
650
+ Normalize parameter ordering for enterprise properties to enable consistent downstream parsing.
651
+
652
+ Enterprise properties in Redpanda have more complex constructor signatures than regular
653
+ properties because they include license restriction information. This transformer
654
+ standardizes the parameter ordering so other transformers can rely on predictable
655
+ parameter positions.
656
+
657
+ ENTERPRISE PROPERTY CONSTRUCTOR PATTERNS:
658
+ 1. enterprise_property<T>(restricted_values, name, description, meta, default)
659
+ 2. enterprise_property<T>(name, restricted_values, description, meta, default)
660
+ 3. enterprise_property<T>(name, description, meta, default) [no restrictions]
661
+
662
+ TARGET STANDARDIZED LAYOUT AFTER NORMALIZATION:
663
+ [0] name - Property identifier
664
+ [1] description - Human-readable description
665
+ [2] meta - Metadata initializer (meta{...})
666
+ [3] default - Default value
667
+
668
+ NORMALIZATION LOGIC:
669
+ 1. Detect presence of std::vector restricted values parameter
670
+ 2. Shift parameter array to skip restriction parameters
671
+ 3. Result: consistent [name, description, meta, default] layout
672
+
673
+ RESTRICTION PARAMETER DETECTION:
674
+ Enterprise properties may have std::vector<T> parameters containing lists of
675
+ allowed values for license-restricted features. These parameters contain
676
+ "std::vector" in their string representation and need to be skipped.
677
+
678
+ SKIP CONDITIONS:
679
+ - Properties starting with simple literals (true/false/numbers) are not normalized
680
+ - These typically represent different constructor patterns that don't need adjustment
681
+
682
+ EXPECTED FINAL RESULT:
683
+ Consistent parameter ordering that allows other transformers to find:
684
+ - params[0]: Property name
685
+ - params[1]: Property description
686
+ - params[2]: Meta information
687
+ - params[3]: Default value
688
+
689
+ DOWNSTREAM DEPENDENCIES:
690
+ - BasicInfoTransformer relies on predictable name/description positions
691
+ - SimpleDefaultValuesTransformer expects defaults in consistent locations
692
+ - EnterpriseTransformer needs to find restriction parameters
693
+ """
694
+
695
+ def accepts(self, info, file_pair):
696
+ """
697
+ Only process enterprise property declarations.
698
+
699
+ Args:
700
+ info (dict): Property declaration info with 'type' field
701
+ file_pair (FilePair): Source file pair (unused)
702
+
703
+ Returns:
704
+ bool: True if this is an enterprise property needing normalization
705
+ """
706
+ return bool(info.get("type") and "enterprise" in info["type"])
707
+
708
+ def parse(self, property, info, file_pair):
709
+ """
710
+ Normalize enterprise property parameter ordering by skipping restriction parameters.
711
+
712
+ Args:
713
+ property (PropertyBag): Property object (returned unchanged)
714
+ info (dict): Property declaration info to normalize in-place
715
+ file_pair (FilePair): Source file pair (unused)
716
+
717
+ Returns:
718
+ PropertyBag: Unchanged property object (normalization modifies info dict)
719
+ """
720
+ params = info.get("params", [])
721
+ if not params:
722
+ return property
723
+
724
+ first_val = str(params[0].get("value", ""))
725
+ second_val = str(params[1].get("value", "")) if len(params) > 1 else ""
726
+
727
+ # Skip normalization for simple literal values - different constructor pattern
728
+ if first_val in ("true", "false") or NUMERIC_PATTERN.match(first_val):
729
+ return property
730
+
731
+ # Pattern 1: Restriction vector in position 0
732
+ # enterprise_property<T>(restricted_values, name, description, meta, default)
733
+ if len(params) >= 4 and "std::vector" in first_val:
734
+ info["params"] = params[1:]
735
+ logger.debug("[ParamNormalizerTransformer] Shifted enterprise params by 1 (restrictions in pos 0)")
736
+
737
+ # Pattern 2: Restriction vector in position 1
738
+ # enterprise_property<T>(name, restricted_values, description, meta, default)
739
+ elif len(params) >= 5 and "std::vector" in second_val:
740
+ info["params"] = params[2:]
741
+ logger.debug("[ParamNormalizerTransformer] Shifted enterprise params by 2 (restrictions in pos 1)")
742
+
743
+ return property
72
744
 
73
745
 
746
+ @debug_transformer
74
747
  class IsNullableTransformer:
748
+ """
749
+ Determine if a property can have null/unset values based on C++ type and metadata.
750
+
751
+ Nullability is a critical JSON schema property that affects validation and documentation.
752
+ This transformer analyzes both the C++ type system and explicit metadata to determine
753
+ if a property can be null or must always have a value.
754
+
755
+ NULLABILITY DETECTION METHODS:
756
+ 1. Explicit "required" metadata in meta{} block (highest priority)
757
+ 2. std::optional<T> wrapper type detection (automatic nullability)
758
+ 3. Default assumption: non-nullable unless evidence suggests otherwise
759
+
760
+ C++ OPTIONAL TYPE HANDLING:
761
+ Properties declared as std::optional<T> are automatically nullable since the
762
+ C++ optional type explicitly models the concept of "value or no value".
763
+
764
+ EXPLICIT REQUIRED METADATA:
765
+ Properties can specify required = required::yes/no in their meta block:
766
+ meta{ .required = required::no } → nullable = true
767
+ meta{ .required = required::yes } → nullable = false
768
+
769
+ DOWNSTREAM IMPACT:
770
+ - JSON Schema validation: nullable=false properties cannot be null/undefined
771
+ - Documentation: nullable properties show "optional" indicators
772
+ - Configuration validation: nullable properties allow omission from config files
773
+
774
+ EXPECTED FINAL RESULT:
775
+ {
776
+ "nullable": true # Property can be null/unset
777
+ }
778
+ OR
779
+ {
780
+ "nullable": false # Property must have a value
781
+ }
782
+
783
+ USAGE IN GENERATED DOCS:
784
+ - Nullable properties show "(optional)" in parameter lists
785
+ - Non-nullable properties show validation requirements
786
+ - Schema validators use nullable flag for validation rules
787
+ """
788
+
75
789
  def accepts(self, info, file_pair):
790
+ """
791
+ Process all properties - nullability determination is universal.
792
+
793
+ Returns:
794
+ bool: Always True - all properties need nullability determination
795
+ """
76
796
  return True
77
797
 
78
798
  def parse(self, property, info, file_pair):
799
+ """
800
+ Analyze property declaration to determine nullability.
801
+
802
+ Args:
803
+ property (PropertyBag): Property to set nullable flag on
804
+ info (dict): Property declaration info with params and declaration
805
+ file_pair (FilePair): Source file pair (unused)
806
+
807
+ Returns:
808
+ PropertyBag: Property with nullable field set
809
+ """
810
+ # Method 1: Check explicit "required" metadata (highest priority)
79
811
  if len(info["params"]) > 2 and "required" in info["params"][2]["value"]:
80
812
  is_required = (
81
813
  re.sub(r"^.*::", "", info["params"][2]["value"]["required"]) == "yes"
82
814
  )
83
815
  property["nullable"] = not is_required
816
+
817
+ # Method 2: Detect std::optional<T> wrapper type
84
818
  elif "std::optional" in info["declaration"]:
85
819
  property["nullable"] = True
820
+
821
+ # Method 3: Default to non-nullable
86
822
  else:
87
823
  property["nullable"] = False
88
824
 
89
825
  return property
90
826
 
91
-
827
+ @debug_transformer
92
828
  class IsArrayTransformer:
93
829
  """
94
830
  Detects properties that should be treated as arrays based on their C++ type declarations.
@@ -142,63 +878,295 @@ class IsArrayTransformer:
142
878
  info["declaration"]
143
879
  )
144
880
 
145
-
881
+ @debug_transformer
146
882
  class NeedsRestartTransformer:
883
+ """
884
+ Extract restart requirements from property metadata for operational documentation.
885
+
886
+ This transformer identifies properties that require a broker restart when modified.
887
+ Restart requirements are crucial operational information that must be prominently
888
+ displayed in documentation to prevent accidental service disruptions.
889
+
890
+ RESTART REQUIREMENT SOURCES:
891
+ Properties specify restart needs via meta{} blocks:
892
+ meta{ .needs_restart = needs_restart::yes } → requires restart
893
+ meta{ .needs_restart = needs_restart::no } → live reconfiguration
894
+ MISSING meta .needs_restart → defaults to requires restart (true)
895
+
896
+ OPERATIONAL SIGNIFICANCE:
897
+ - Properties requiring restart cannot be changed without downtime
898
+ - Live-configurable properties allow zero-downtime configuration updates
899
+ - Critical for production change management and deployment planning
900
+ - Default to requiring restart for safety when metadata is missing
901
+
902
+ EXPECTED FINAL RESULT:
903
+ {
904
+ "needs_restart": true # Restart required for changes (default if missing)
905
+ }
906
+ OR
907
+ {
908
+ "needs_restart": false # Live reconfiguration supported (explicitly set)
909
+ }
910
+
911
+ DOWNSTREAM USAGE:
912
+ - Documentation shows restart warnings for affected properties
913
+ - Configuration management tools use this for change validation
914
+ - Operations teams use this for maintenance planning
915
+ """
916
+
147
917
  def accepts(self, info, file_pair):
148
- # Only accept when the params blob exists and contains a needs_restart entry
149
- return (
150
- len(info.get("params", [])) > 2
151
- and isinstance(info["params"][2].get("value"), dict)
152
- and "needs_restart" in info["params"][2]["value"]
153
- )
918
+ """Process all properties - assign default restart requirement if missing."""
919
+ return True
154
920
 
155
921
  def parse(self, property, info, file_pair):
156
- # We only get here if accepts(...) returned True, so the metadata blob is present
157
- raw = info["params"][2]["value"]["needs_restart"]
158
- flag = re.sub(r"^.*::", "", raw)
159
- # Store as boolean; do not set any default when metadata is absent
160
- property["needs_restart"] = (flag != "no")
922
+ """Extract restart requirement, defaulting based on property type if not specified."""
923
+ val = get_meta_value(info, "needs_restart")
924
+ if val is None:
925
+ # Check if this is a topic property
926
+ is_topic_property = (
927
+ property.get("is_topic_property", False) or
928
+ property.get("config_scope") == "topic"
929
+ )
930
+
931
+ if is_topic_property:
932
+ # Topic properties default to not requiring restart
933
+ property["needs_restart"] = False
934
+ else:
935
+ # Cluster and broker properties default to requiring restart for safety
936
+ property["needs_restart"] = True
937
+ else:
938
+ property["needs_restart"] = (val != "no")
939
+ return property
161
940
 
941
+ @debug_transformer
162
942
  class GetsRestoredTransformer:
943
+ """
944
+ Extract backup/restore inclusion flags for disaster recovery documentation.
945
+
946
+ This transformer identifies properties that are included in or excluded from
947
+ Redpanda's backup and restore operations. This information is essential for
948
+ disaster recovery planning and data migration procedures.
949
+
950
+ RESTORATION METADATA PATTERNS:
951
+ Properties can use either naming convention:
952
+ meta{ .gets_restored = restored::yes } → included in backups
953
+ meta{ .restored = restored::no } → excluded from backups
954
+
955
+ BACKUP/RESTORE BEHAVIOR:
956
+ - gets_restored = true: Property value saved in backups and restored
957
+ - gets_restored = false: Property reset to default during restore
958
+
959
+ TYPICAL EXCLUSIONS:
960
+ - Temporary operational state (cache sizes, connection limits)
961
+ - Environment-specific settings (hostnames, paths)
962
+ - Credentials and secrets (handled separately for security)
963
+
964
+ EXPECTED FINAL RESULT:
965
+ {
966
+ "gets_restored": true # Included in backup/restore operations
967
+ }
968
+ OR
969
+ {
970
+ "gets_restored": false # Excluded from backup/restore
971
+ }
972
+
973
+ DOWNSTREAM USAGE:
974
+ - Disaster recovery documentation lists which settings persist
975
+ - Backup tools use this for selective property restoration
976
+ - Migration guides indicate what needs manual reconfiguration
977
+ """
978
+
163
979
  def accepts(self, info, file_pair):
164
- # only run if the third param blob exists and has our flag
165
- return (
166
- len(info.get("params", [])) > 2
167
- and isinstance(info["params"][2].get("value"), dict)
168
- and "gets_restored" in info["params"][2]["value"]
169
- )
980
+ """Process properties with backup/restore metadata."""
981
+ return (get_meta_value(info, "gets_restored") is not None or
982
+ get_meta_value(info, "restored") is not None)
983
+
984
+ def parse(self, property, info, file_pair):
985
+ """Extract restoration flag from either naming convention."""
986
+ val = get_meta_value(info, "gets_restored") or get_meta_value(info, "restored", "no")
987
+ property["gets_restored"] = (val != "no")
988
+ return property
989
+
990
+
991
+ @debug_transformer
992
+ class IsSecretTransformer:
993
+ """
994
+ Identify properties containing sensitive data for security documentation.
995
+
996
+ This transformer marks properties that contain sensitive information such as
997
+ passwords, API keys, certificates, or other confidential data. This enables
998
+ appropriate security warnings and handling in documentation.
999
+
1000
+ SECRET DETECTION:
1001
+ Properties explicitly mark sensitive data:
1002
+ meta{ .secret = secret::yes } → contains sensitive data
1003
+ meta{ .secret = secret::no } → safe to display
1004
+
1005
+ SECURITY IMPLICATIONS:
1006
+ - Secret properties should never appear in logs or debug output
1007
+ - Documentation must warn about secure storage requirements
1008
+ - Configuration examples use placeholder values
1009
+ - Backup/restore may require special handling
1010
+
1011
+ EXPECTED FINAL RESULT:
1012
+ {
1013
+ "is_secret": true # Contains sensitive data - handle with care
1014
+ }
1015
+ OR
1016
+ {
1017
+ "is_secret": false # Safe to display (default)
1018
+ }
1019
+
1020
+ DOWNSTREAM USAGE:
1021
+ - Documentation generators hide or mask secret property values
1022
+ - Configuration validators warn about insecure secret storage
1023
+ - Logging systems exclude secret properties from output
1024
+ """
1025
+
1026
+ def accepts(self, info, file_pair):
1027
+ """Process properties with secret metadata."""
1028
+ return get_meta_value(info, "secret") is not None
170
1029
 
171
1030
  def parse(self, property, info, file_pair):
172
- raw = info["params"][2]["value"]["gets_restored"]
173
- # strip off e.g. "gets_restored::no" "no"
174
- flag = re.sub(r"^.*::", "", raw)
175
- # store as boolean
176
- property["gets_restored"] = (flag != "no")
1031
+ """Extract and normalize secret flag."""
1032
+ val = get_meta_value(info, "secret", "no")
1033
+ property["is_secret"] = (val == "yes")
1034
+ return property
177
1035
 
178
1036
 
1037
+ @debug_transformer
179
1038
  class VisibilityTransformer:
1039
+ """
1040
+ Classify property visibility levels for appropriate documentation targeting.
1041
+
1042
+ This transformer categorizes properties by their intended audience and usage
1043
+ complexity. Visibility levels determine where properties appear in documentation
1044
+ and how prominently they are featured.
1045
+
1046
+ VISIBILITY LEVELS:
1047
+ - user: End-user configurable properties (appear in user guides)
1048
+ - tunable: Advanced/performance tuning properties (expert documentation)
1049
+ - deprecated: Legacy properties (migration guides only)
1050
+
1051
+ VISIBILITY METADATA:
1052
+ Properties specify their visibility:
1053
+ meta{ .visibility = visibility::user } → user-facing documentation
1054
+ meta{ .visibility = visibility::tunable } → advanced/tuning guides
1055
+ meta{ .visibility = visibility::deprecated} → migration documentation only
1056
+
1057
+ DOCUMENTATION IMPACT:
1058
+ - 'user' properties: Featured in getting-started and configuration guides
1059
+ - 'tunable' properties: Advanced sections, performance documentation
1060
+ - 'deprecated' properties: Migration guides with replacement information
1061
+
1062
+ EXPECTED FINAL RESULT:
1063
+ {
1064
+ "visibility": "user" # Primary user documentation
1065
+ }
1066
+ OR
1067
+ {
1068
+ "visibility": "tunable" # Advanced/expert documentation
1069
+ }
1070
+ OR
1071
+ {
1072
+ "visibility": "deprecated" # Migration documentation only
1073
+ }
1074
+
1075
+ DOWNSTREAM USAGE:
1076
+ - generate-handlebars-docs.js filters properties by visibility
1077
+ - Documentation templates show user properties prominently
1078
+ - Advanced guides include tunable properties for optimization
1079
+ """
1080
+
180
1081
  def accepts(self, info, file_pair):
181
- return (
182
- True
183
- if len(info["params"]) > 2 and "visibility" in info["params"][2]["value"]
184
- else False
185
- )
1082
+ """Process properties with visibility metadata."""
1083
+ return get_meta_value(info, "visibility") is not None
186
1084
 
187
1085
  def parse(self, property, info, file_pair):
188
- property["visibility"] = re.sub(
189
- r"^.*::", "", info["params"][2]["value"]["visibility"]
190
- )
1086
+ """Extract visibility level with user as default."""
1087
+ vis = get_meta_value(info, "visibility", "user")
1088
+ property["visibility"] = vis
1089
+
1090
+ # Mark as deprecated if visibility is deprecated
1091
+ if vis == "deprecated":
1092
+ property["is_deprecated"] = True
191
1093
 
1094
+ return property
192
1095
 
1096
+ @debug_transformer
193
1097
  class TypeTransformer:
1098
+ """
1099
+ Map C++ property types to JSON Schema types for documentation and validation.
1100
+
1101
+ This is one of the most critical transformers in the pipeline. It bridges the gap
1102
+ between C++ type system and JSON Schema by analyzing complex C++ template
1103
+ declarations and converting them to standardized JSON types that can be consumed
1104
+ by documentation generators and validation systems.
1105
+
1106
+ TYPE MAPPING RESPONSIBILITIES:
1107
+ 1. Parse complex nested C++ template declarations
1108
+ 2. Extract inner types from wrappers (property<T>, std::optional<T>, etc.)
1109
+ 3. Map C++ types to JSON Schema primitives (string, integer, boolean, object, array)
1110
+ 4. Handle deprecated property type detection and mapping
1111
+ 5. Support Redpanda-specific type patterns (one_or_many_property, enterprise types)
1112
+
1113
+ C++ TO JSON SCHEMA TYPE MAPPINGS:
1114
+ - std::string, string_view → "string"
1115
+ - int32_t, uint64_t, size_t → "integer"
1116
+ - bool → "boolean"
1117
+ - double, float → "number"
1118
+ - std::chrono::* durations → "integer" (with bounds from DurationBoundsTransformer)
1119
+ - Custom model/config classes → "object" (with $ref to definitions)
1120
+ - enum classes → "string" (with enum constraints)
1121
+
1122
+ COMPLEX TYPE UNWRAPPING:
1123
+ Handles nested template patterns like:
1124
+ - property<std::vector<model::broker_endpoint>> → array of objects
1125
+ - enterprise_property<std::optional<std::string>> → nullable string
1126
+ - one_or_many_property<config::endpoint_tls_config> → array of objects
1127
+ - deprecated_property<size_t> → deprecated integer
1128
+
1129
+ TEMPLATE PARSING ALGORITHM:
1130
+ Uses sophisticated bracket counting to handle arbitrarily nested templates:
1131
+ 1. Find template name position in declaration
1132
+ 2. Track opening/closing angle brackets with proper nesting
1133
+ 3. Extract content between matching brackets
1134
+ 4. Recursively process nested templates
1135
+
1136
+ EXPECTED FINAL RESULT:
1137
+ {
1138
+ "type": "string" # JSON Schema primitive type
1139
+ }
1140
+ OR
1141
+ {
1142
+ "type": "object" # Complex type referencing definitions
1143
+ }
1144
+ OR
1145
+ {
1146
+ "type": "array", # Array type with items specification
1147
+ "items": {"type": "object"}
1148
+ }
1149
+
1150
+ DOWNSTREAM DEPENDENCIES:
1151
+ - IsArrayTransformer: Depends on type extraction for array item types
1152
+ - NumericBoundsTransformer: Uses C++ type info for bounds calculation
1153
+ - resolve_type_and_default(): Uses type info for default value processing
1154
+ - JSON Schema validators: Use type info for validation rules
1155
+ """
194
1156
 
195
1157
  # Class-level constants for type pattern matching
196
1158
  # Shared with IsArrayTransformer for consistency
197
1159
  ARRAY_PATTERN_STD_VECTOR = "std::vector"
198
- ARRAY_PATTERN_ONE_OR_MANY = "one_or_many_property"
1160
+ ARRAY_PATTERN_ONE_OR_MANY = "one_or_many_property"
199
1161
  OPTIONAL_PATTERN = "std::optional"
200
1162
 
201
1163
  def accepts(self, info, file_pair):
1164
+ """
1165
+ Process all properties - type mapping is universally required.
1166
+
1167
+ Returns:
1168
+ bool: Always True - every property needs type information
1169
+ """
202
1170
  return True
203
1171
 
204
1172
  def get_cpp_type_from_declaration(self, declaration):
@@ -252,23 +1220,23 @@ class TypeTransformer:
252
1220
  else:
253
1221
  # Fallback to original regex for simpler cases
254
1222
  raw_type = (
255
- re.sub(r"^.*property<(.+)>.*", "\\1", one_line_declaration)
1223
+ PROPERTY_TEMPLATE_PATTERN.sub(r"\1", one_line_declaration)
256
1224
  .split()[0]
257
1225
  .replace(",", "")
258
1226
  )
259
1227
 
260
1228
  if self.OPTIONAL_PATTERN in raw_type:
261
- raw_type = re.sub(".*std::optional<(.+)>.*", "\\1", raw_type)
1229
+ raw_type = OPTIONAL_TEMPLATE_PATTERN.sub(r"\1", raw_type)
262
1230
 
263
1231
  if self.ARRAY_PATTERN_STD_VECTOR in raw_type:
264
- raw_type = re.sub(".*std::vector<(.+)>.*", "\\1", raw_type)
1232
+ raw_type = VECTOR_TEMPLATE_PATTERN.sub(r"\1", raw_type)
265
1233
 
266
1234
  # Handle one_or_many_property<T> - extract the inner type T
267
1235
  # This is essential for Redpanda's flexible configuration properties
268
1236
  # that can accept either single values or arrays
269
1237
  # Check and extract from raw_type for consistency with other type extractors
270
1238
  if self.ARRAY_PATTERN_ONE_OR_MANY in raw_type:
271
- raw_type = re.sub(".*one_or_many_property<(.+)>.*", "\\1", raw_type)
1239
+ raw_type = ONE_OR_MANY_PATTERN.sub(r"\1", raw_type)
272
1240
  raw_type = raw_type.split()[0].replace(",", "")
273
1241
 
274
1242
  return raw_type
@@ -313,46 +1281,51 @@ class TypeTransformer:
313
1281
 
314
1282
  def parse(self, property, info, file_pair):
315
1283
  """
316
- Set the property's "type" field to the JSON schema type derived from the C++ declaration.
317
-
318
- Parameters:
319
- property (dict): Mutable property bag to be updated.
320
- info (dict): Parsed property metadata; its "declaration" field is used to determine the type.
321
- file_pair: Unused here; present for transformer interface compatibility.
322
-
323
- Returns:
324
- property (dict): The same property bag with "type" set to the derived type string.
1284
+ Set the property's 'type' field to the JSON schema type derived from the C++ declaration.
1285
+ Always sets is_deprecated explicitly.
1286
+ Keeps the inner (real) type even for deprecated_property<T>.
1287
+ Also captures the original C++ type in c_type field for debugging and type lookup.
325
1288
  """
326
- property["type"] = self.get_type_from_declaration(info["declaration"])
327
- return property
1289
+ declaration = info.get("declaration", "") or ""
328
1290
 
1291
+ # --- detect deprecation from declaration ---
1292
+ is_deprecated = "deprecated_property" in declaration
1293
+ property["is_deprecated"] = is_deprecated
329
1294
 
330
- class DeprecatedTransformer:
331
- def accepts(self, info, file_pair):
332
- return "deprecated_property" in info["declaration"] or (
333
- len(info["params"]) > 2
334
- and "visibility" in info["params"][2]["value"]
335
- and "deprecated" in info["params"][2]["value"]["visibility"]
336
- )
1295
+ # --- unwrap deprecated_property<T> to extract real base type ---
1296
+ if is_deprecated:
1297
+ inner_decl = re.sub(r".*deprecated_property<(.+)>.*", r"\1", declaration)
1298
+ else:
1299
+ inner_decl = declaration
337
1300
 
338
- def parse(self, property, info, file_pair):
339
- property["is_deprecated"] = True
340
- property["type"] = None
1301
+ # --- capture the original C++ type for debugging and definition lookup ---
1302
+ cpp_type = self.get_cpp_type_from_declaration(inner_decl)
1303
+ if cpp_type:
1304
+ property["c_type"] = cpp_type
341
1305
 
1306
+ # --- derive the JSON schema type from the inner declaration ---
1307
+ derived_type = self.get_type_from_declaration(inner_decl)
1308
+ property["type"] = derived_type
342
1309
 
343
- class IsSecretTransformer:
1310
+ return property
1311
+
1312
+ @debug_transformer
1313
+ class DeprecatedTransformer:
1314
+ """
1315
+ Marks the property as deprecated if 'deprecated' appears in meta.
1316
+ """
344
1317
  def accepts(self, info, file_pair):
345
- return (
346
- True
347
- if len(info["params"]) > 2 and "secret" in info["params"][2]["value"]
348
- else False
349
- )
1318
+ return get_meta_value(info, "deprecated") is not None
350
1319
 
351
1320
  def parse(self, property, info, file_pair):
352
- is_secret = re.sub(r"^.*::", "", info["params"][2]["value"]["secret"])
353
- property["is_secret"] = is_secret == "yes"
1321
+ val = get_meta_value(info, "deprecated")
1322
+ property["is_deprecated"] = True
1323
+ if val and val not in ("yes", "true"):
1324
+ property["deprecated_reason"] = val
1325
+ return property
354
1326
 
355
1327
 
1328
+ @debug_transformer
356
1329
  class NumericBoundsTransformer:
357
1330
  def __init__(self, type_transformer):
358
1331
  self.type_transformer = type_transformer
@@ -379,7 +1352,7 @@ class NumericBoundsTransformer:
379
1352
  property["minimum"] = type_mapping[type_str][0]
380
1353
  property["maximum"] = type_mapping[type_str][1]
381
1354
 
382
-
1355
+ @debug_transformer
383
1356
  class DurationBoundsTransformer:
384
1357
  def __init__(self, type_transformer):
385
1358
  self.type_transformer = type_transformer
@@ -407,249 +1380,473 @@ class DurationBoundsTransformer:
407
1380
  property["minimum"] = type_mapping[duration_type][0]
408
1381
  property["maximum"] = type_mapping[duration_type][1]
409
1382
 
410
-
1383
+ @debug_transformer
411
1384
  class SimpleDefaultValuesTransformer:
412
1385
  def accepts(self, info, file_pair):
413
- # The default value is the 4th parameter.
414
- return info["params"] and len(info["params"]) > 3
1386
+ return bool(info.get("params") and len(info["params"]) > 3)
415
1387
 
416
1388
  def parse(self, property, info, file_pair):
417
- default = info["params"][3]["value"]
1389
+ params = info.get("params", [])
1390
+ if not params:
1391
+ return property
418
1392
 
419
- # Handle simple cases.
420
- if default == "std::nullopt":
421
- property["default"] = None
422
- elif default == "{}":
423
- pass
424
- elif isinstance(default, PropertyBag):
425
- property["default"] = default
426
- elif re.search(r"^-?[0-9][0-9']*$", default): # integers (allow digit group separators)
427
- property["default"] = int(re.sub(r"[^0-9-]", "", default))
428
- elif re.search(r"^-?[0-9]+(\.[0-9]+)?$", default): # floats
429
- property["default"] = float(re.sub(r"[^0-9.\-]", "", default))
430
- elif re.search("^(true|false)$", default): # booleans
431
- property["default"] = True if default == "true" else False
432
- elif re.search(r"^\{[^:]+\}$", default): # string lists
433
- property["default"] = [
434
- normalize_string(s)
435
- for s in re.sub(r"{([^}]+)}", r"\1", default).split(",")
436
- ]
1393
+ # Find where the meta{} param is
1394
+ meta_index = next(
1395
+ (i for i, p in enumerate(params)
1396
+ if isinstance(p.get("value"), (dict, str))
1397
+ and ("meta{" in str(p["value"]) or
1398
+ (isinstance(p["value"], dict) and "needs_restart" in p["value"]))),
1399
+ None,
1400
+ )
1401
+
1402
+ # Default comes immediately after meta
1403
+ if meta_index is None:
1404
+ default_index = 3 if len(params) > 3 else None
437
1405
  else:
438
- # File sizes.
439
- matches = re.search("^([0-9]+)_(.)iB$", default)
440
- if matches:
441
- size = int(matches.group(1))
442
- unit = matches.group(2)
443
- if unit == "K":
444
- size = size * 1024
445
- elif unit == "M":
446
- size = size * 1024**2
447
- elif unit == "G":
448
- size = size * 1024**3
449
- elif unit == "T":
450
- size = size * 1024**4
451
- elif unit == "P":
452
- size = size * 1024**5
453
- property["default"] = size
454
- elif re.search("^(https|/[^/])", default): # URLs and paths
455
- property["default"] = default
456
- else:
457
- # For durations, enums, or other default initializations.
458
- if not re.search("([0-9]|::|\\()", default):
459
- property["default"] = default
460
- else:
461
- property["default"] = default
1406
+ default_index = meta_index + 1 if len(params) > meta_index + 1 else None
462
1407
 
1408
+ if default_index is None or default_index >= len(params):
1409
+ return property
463
1410
 
464
- class FriendlyDefaultTransformer:
465
- """
466
- Transforms C++ default expressions into a more user-friendly format for docs.
467
- Handles cases like:
468
- - std::numeric_limits<uint64_t>::max()
469
- - std::chrono::seconds(15min)
470
- - std::vector<ss::sstring>{"basic"}
471
- - std::chrono::milliseconds(10)
472
- - std::nullopt
473
- """
474
-
475
- # Class-level constants for pattern matching in default values
476
- ARRAY_PATTERN_STD_VECTOR = "std::vector"
477
- SSTRING_CONSTRUCTOR_PATTERN = r'ss::sstring\{([a-zA-Z_][a-zA-Z0-9_]*)\}'
478
- VECTOR_INITIALIZER_PATTERN = r'std::vector<[^>]+>\s*\{(.*)\}$'
479
- CHRONO_PATTERN = r"std::chrono::(\w+)\(([^)]+)\)"
480
-
481
- def __init__(self):
482
- """
483
- Initialize the transformer and set up a placeholder for a lazily-loaded resolver.
484
-
485
- Sets self._resolver to None to indicate the resolver has not been loaded yet.
486
- """
1411
+ # Candidate default param
1412
+ default_param = params[default_index]
1413
+ default = default_param.get("value")
1414
+
1415
+ # Skip obvious validator params
1416
+ if is_validator_param(default_param):
1417
+ return property
1418
+
1419
+ # std::nullopt means "no default"
1420
+ if isinstance(default, str) and "std::nullopt" in default:
1421
+ property["default"] = None
1422
+ return property
1423
+
1424
+ # legacy_default<T>(value, legacy_version{N})
1425
+ if isinstance(default, str) and "legacy_default" in default:
1426
+ match = re.search(r"legacy_default<[^>]+>\(([^,]+)", default)
1427
+ if match:
1428
+ default = match.group(1).strip()
1429
+
1430
+ # {a, b, c} initializer list
1431
+ if isinstance(default, str) and re.match(r"^\{.*\}$", default):
1432
+ inner = re.sub(r"^\{(.*)\}$", r"\1", default).strip()
1433
+ if inner:
1434
+ items = [normalize_string(x.strip().strip('"')) for x in inner.split(",")]
1435
+ property["default"] = items
1436
+ else:
1437
+ property["default"] = []
1438
+ return property
1439
+
1440
+ # Simple booleans, numerics, or size literals
1441
+ if isinstance(default, str):
1442
+ if default in ("true", "false"):
1443
+ property["default"] = (default == "true")
1444
+ return property
1445
+ if re.match(r"^-?\d+\.\d+$", default):
1446
+ property["default"] = float(default)
1447
+ return property
1448
+ if re.match(r"^-?\d+$", default):
1449
+ property["default"] = int(default)
1450
+ return property
1451
+ # e.g. 20_GiB, 256_MiB
1452
+ size_match = re.match(r"(\d+)_([KMGTP])iB", default)
1453
+ if size_match:
1454
+ num, unit = int(size_match.group(1)), size_match.group(2)
1455
+ mult = {"K": 1024, "M": 1024**2, "G": 1024**3, "T": 1024**4, "P": 1024**5}[unit]
1456
+ property["default"] = num * mult
1457
+ return property
1458
+
1459
+ # Fallback — plain string
1460
+ default_value = normalize_string(str(default)).replace("std::", "")
1461
+
1462
+ # Handle known type constructors that tree-sitter extracted
1463
+ # These are C++ default constructors where we know the resulting value
1464
+ if default_value in ["leaders_preference", "config::leaders_preference", "leaders_preference{}", "config::leaders_preference{}"]:
1465
+ default_value = "none" # config::leaders_preference{} defaults to "none"
1466
+ elif default_value in ["data_directory_path", "config::data_directory_path", "data_directory_path{}", "config::data_directory_path{}"]:
1467
+ default_value = "" # config::data_directory_path{} defaults to empty string
1468
+
1469
+ property["default"] = default_value
1470
+ return property
1471
+
1472
+
1473
+ @debug_transformer
1474
+ class FriendlyDefaultTransformer:
1475
+ """
1476
+ Transforms complex C++ default expressions into human-readable JSON-friendly values.
1477
+ Handles patterns such as:
1478
+ - std::chrono::<unit>{<value>}
1479
+ - chrono::<unit>{<value>}
1480
+ - net::unresolved_address("127.0.0.1", 9092)
1481
+ - ss::sstring{CONSTEXPR}
1482
+ - { "a", "b" } and std::vector<...>{...}
1483
+ - std::nullopt and legacy_default<...>
1484
+ - Computed C++ constants (max_serializable_ms, etc.)
1485
+ - Complex symbolic constants
1486
+
1487
+ COMPUTED CONSTANTS:
1488
+ Some C++ constants involve complex compile-time expressions that cannot be easily parsed.
1489
+ These are pre-computed and stored in the COMPUTED_CONSTANTS dictionary. For example:
1490
+ - max_serializable_ms: std::chrono::duration_cast<std::chrono::milliseconds>(
1491
+ std::chrono::nanoseconds::max()) = 9223372036854 ms
1492
+ """
1493
+
1494
+ ARRAY_PATTERN_STD_VECTOR = r"std::vector<[^>]+>\s*\{(.*)\}$"
1495
+ CHRONO_PATTERN = r"(?:std::)?chrono::(\w+)\s*\{\s*([^}]+)\s*\}"
1496
+ UNRESOLVED_ADDRESS_PATTERN = r'net::unresolved_address\s*\(\s*"?([^",]+)"?\s*,\s*([^)]+)\)'
1497
+ SSTRING_PATTERN = r'ss::sstring\s*\{\s*([^}]+)\s*\}'
1498
+ NUMERIC_LIMITS_PATTERN = r"std::numeric_limits<[^>]+>::max\(\)"
1499
+ LEGACY_DEFAULT_PATTERN = r"legacy_default<[^>]+>\(([^,]+)"
1500
+ BRACED_LIST_PATTERN = r"^\{(.*)\}$"
1501
+
1502
+ def __init__(self):
487
1503
  self._resolver = None
488
-
1504
+
489
1505
  def accepts(self, info, file_pair):
490
- """
491
- Determine whether the transformer should run for the given property info by checking for a fourth parameter.
492
-
493
- Parameters:
494
- info (dict): Parsed property metadata; expects a "params" list when present.
495
- file_pair (tuple): Source/implementation file pair (unused by this check).
496
-
497
- Returns:
498
- `true` if `info["params"]` exists and contains at least four items, `false` otherwise.
499
- """
500
- return info.get("params") and len(info["params"]) > 3
1506
+ return bool(info.get("params") and len(info["params"]) > 2)
501
1507
 
502
1508
  def _get_resolver(self):
503
- """
504
- Lazily load and cache the constexpr identifier resolver.
505
-
506
- Returns:
507
- callable or None: The resolver function if available, or `None` if it could not be loaded.
508
- """
509
1509
  if self._resolver is None:
510
1510
  resolver = get_resolve_constexpr_identifier()
511
1511
  self._resolver = resolver if resolver else False
512
1512
  return self._resolver if self._resolver is not False else None
513
1513
 
514
1514
  def _resolve_identifier(self, identifier):
515
- """
516
- Resolve a constexpr identifier to its corresponding string value.
517
-
518
- Parameters:
519
- identifier (str): Identifier to resolve (for example, "scram" or "gssapi").
520
-
521
- Returns:
522
- str or None: The resolved string value if successful, `None` when the identifier is invalid or cannot be resolved.
523
- """
524
- if not identifier or not isinstance(identifier, str):
525
- logger.warning(f"Invalid identifier for resolution: {identifier}")
526
- return None
527
-
528
1515
  resolver = self._get_resolver()
529
1516
  if resolver:
530
1517
  try:
531
1518
  return resolver(identifier)
532
- except (AttributeError, TypeError, ValueError) as e:
533
- logger.debug(f"Failed to resolve identifier '{identifier}': {e}")
534
- except Exception as e:
535
- logger.exception(f"Unexpected error resolving identifier '{identifier}': {e}")
536
-
537
- return None
538
-
539
- def _process_sstring_constructor(self, item):
540
- """
541
- Convert an ss::sstring{identifier} constructor string to its resolved value when possible.
542
-
543
- If the input matches the ss::sstring{...} pattern, attempts to resolve the enclosed identifier and returns the resolved string. If resolution fails, returns the raw identifier. If the input does not match the pattern or is falsy, returns it unchanged.
544
-
545
- Parameters:
546
- item (str): The constructor expression or string to process.
547
-
548
- Returns:
549
- str: The resolved string when resolution succeeds, the extracted identifier if resolution fails, or the original input if it does not match.
550
- """
551
- if not item:
552
- return item
553
-
554
- match = re.match(self.SSTRING_CONSTRUCTOR_PATTERN, item)
555
- if not match:
556
- return item
557
-
558
- identifier = match.group(1)
559
- resolved = self._resolve_identifier(identifier)
560
-
561
- if resolved:
562
- logger.debug(f"Resolved ss::sstring{{{identifier}}} -> '{resolved}'")
563
- return resolved
564
-
565
- # Log warning but continue with original identifier
566
- logger.warning(f"Could not resolve identifier '{identifier}' in ss::sstring constructor")
1519
+ except Exception:
1520
+ return identifier
567
1521
  return identifier
568
1522
 
569
- def _parse_vector_contents(self, contents):
570
- """
571
- Parse a comma-separated std::vector initializer string into a list of cleaned, processed items.
572
-
573
- Parameters:
574
- contents (str): The inner contents of a vector initializer (e.g. '\"a\", ss::sstring{ID}, \"b\"'); may be empty or None.
575
-
576
- Returns:
577
- list: Ordered list of processed, unquoted items with empty entries omitted.
578
- """
579
- if not contents:
1523
+ def _parse_initializer_list(self, text):
1524
+ """Handle braced lists like {"a", "b"}."""
1525
+ inner = re.sub(self.BRACED_LIST_PATTERN, r"\1", text.strip()).strip()
1526
+ if not inner:
580
1527
  return []
581
-
582
- # Split by comma and process each item
583
- raw_items = [contents] if ',' not in contents else contents.split(',')
584
-
585
- processed_items = []
586
- for item in raw_items:
587
- item = item.strip(' "\'')
588
- if item: # Skip empty items
589
- processed_item = self._process_sstring_constructor(item)
590
- processed_items.append(processed_item)
591
-
592
- return processed_items
1528
+ parts = [p.strip().strip('"\'') for p in inner.split(",") if p.strip()]
1529
+ return [normalize_string(p) for p in parts]
1530
+
1531
+ def _parse_vector_initializer(self, text):
1532
+ """Parse std::vector<T>{a, b, c} → ["a", "b", "c"]"""
1533
+ match = re.search(self.ARRAY_PATTERN_STD_VECTOR, text)
1534
+ if not match:
1535
+ return []
1536
+ contents = match.group(1).strip()
1537
+ return self._parse_initializer_list(f"{{{contents}}}")
593
1538
 
594
1539
  def parse(self, property, info, file_pair):
595
- """
596
- Convert a C++ default expression into a JSON-friendly value and store it on the property under the "default" key.
597
-
598
- Parameters:
599
- property (dict): Property dictionary to modify; updated in place with a "default" entry.
600
- info (dict): Parsed property information; the default expression is expected at info["params"][3]["value"].
601
- file_pair: File pair context (ignored by this function).
602
-
603
- Returns:
604
- dict: The modified property dictionary with a normalized "default" value.
605
- """
606
- default = info["params"][3]["value"]
607
-
608
- # Handle null/empty defaults
609
- if not default:
1540
+ params = info.get("params", [])
1541
+ if not params:
1542
+ return property
1543
+
1544
+ # find meta param index
1545
+ meta_index = next(
1546
+ (i for i, p in enumerate(params)
1547
+ if isinstance(p.get("value"), (dict, str))
1548
+ and ("meta{" in str(p["value"])
1549
+ or (isinstance(p["value"], dict) and "needs_restart" in p["value"]))),
1550
+ None,
1551
+ )
1552
+
1553
+ # default param follows meta
1554
+ default_index = meta_index + 1 if meta_index is not None and len(params) > meta_index + 1 else 3
1555
+ if default_index >= len(params):
1556
+ return property
1557
+
1558
+ default = params[default_index].get("value")
1559
+ if not isinstance(default, str):
610
1560
  return property
611
1561
 
612
- # Transform std::nullopt into None
613
- if "std::nullopt" in default:
1562
+ d = default.strip()
1563
+
1564
+ # ------------------------------------------------------------------
1565
+ # Handle empty / nullopt / none cases
1566
+ # ------------------------------------------------------------------
1567
+ if "std::nullopt" in d or d in ("std::nullopt", "nullopt"):
614
1568
  property["default"] = None
615
1569
  return property
616
1570
 
617
- # Transform std::numeric_limits expressions
618
- if "std::numeric_limits" in default:
1571
+ # ------------------------------------------------------------------
1572
+ # Handle legacy_default<T>(value, ...)
1573
+ # ------------------------------------------------------------------
1574
+ legacy_match = re.search(self.LEGACY_DEFAULT_PATTERN, d)
1575
+ if legacy_match:
1576
+ d = legacy_match.group(1).strip()
1577
+
1578
+ # ------------------------------------------------------------------
1579
+ # std::numeric_limits<T>::max()
1580
+ # ------------------------------------------------------------------
1581
+ if re.search(self.NUMERIC_LIMITS_PATTERN, d):
619
1582
  property["default"] = "Maximum value"
620
1583
  return property
621
1584
 
622
- # Transform std::chrono durations
623
- if "std::chrono" in default:
624
- match = re.search(self.CHRONO_PATTERN, default)
1585
+ # ------------------------------------------------------------------
1586
+ # chrono::duration forms
1587
+ # ------------------------------------------------------------------
1588
+ # ------------------------------------------------------------------
1589
+ # chrono::duration forms
1590
+ # - chrono::minutes{5} -> "5 minutes"
1591
+ # - std::chrono::weeks{2} -> "2 weeks"
1592
+ # - chrono::milliseconds{1min} -> "1 minute"
1593
+ # ------------------------------------------------------------------
1594
+ chrono_match = re.search(self.CHRONO_PATTERN, d)
1595
+ if chrono_match:
1596
+ unit, value = chrono_match.groups()
1597
+ value = value.strip()
1598
+
1599
+ # First handle inner literals like 1min, 30s, 100ms, 2h
1600
+ lit_match = re.match(r"(\d+)\s*(min|s|ms|h)", value)
1601
+ if lit_match:
1602
+ num, suffix = lit_match.groups()
1603
+ inner_unit_map = {
1604
+ "min": "minute",
1605
+ "s": "second",
1606
+ "ms": "millisecond",
1607
+ "h": "hour",
1608
+ }
1609
+ base = inner_unit_map.get(suffix, suffix)
1610
+
1611
+ # pluralize
1612
+ if num != "1" and not base.endswith("s"):
1613
+ base = base + "s"
1614
+
1615
+ property["default"] = f"{num} {base}"
1616
+ # don't forcibly override property["type"] here; leave it as-is
1617
+ return property
1618
+
1619
+ # Otherwise it's something like chrono::minutes{5} or chrono::weeks{2}
1620
+ # Use the outer chrono unit name.
1621
+ human_unit = unit
1622
+ # Simple pluralization fix if needed
1623
+ if value == "1" and human_unit.endswith("s"):
1624
+ human_unit = human_unit[:-1]
1625
+
1626
+ property["default"] = f"{value} {human_unit}"
1627
+ return property
1628
+
1629
+
1630
+ # ------------------------------------------------------------------
1631
+ # net::unresolved_address("127.0.0.1", 9092)
1632
+ # ------------------------------------------------------------------
1633
+ if "net::unresolved_address" in d:
1634
+ match = re.search(self.UNRESOLVED_ADDRESS_PATTERN, d)
625
1635
  if match:
626
- unit = match.group(1)
627
- value = match.group(2).strip()
628
- property["default"] = f"{value} {unit}"
1636
+ addr, port = match.groups()
1637
+ try:
1638
+ port_val = int(port)
1639
+ except ValueError:
1640
+ port_val = normalize_string(port)
1641
+ property["default"] = {
1642
+ "address": addr.strip(),
1643
+ "port": port_val,
1644
+ }
1645
+ property["type"] = "object"
1646
+ property["$ref"] = "#/definitions/net::unresolved_address"
629
1647
  return property
630
1648
 
631
- # Transform std::vector defaults
632
- if self.ARRAY_PATTERN_STD_VECTOR in default:
633
- vector_match = re.search(self.VECTOR_INITIALIZER_PATTERN, default)
634
- if vector_match:
635
- contents = vector_match.group(1).strip()
636
- items = self._parse_vector_contents(contents)
637
- property["default"] = items
1649
+ # ------------------------------------------------------------------
1650
+ # ss::sstring{CONSTEXPR}
1651
+ # ------------------------------------------------------------------
1652
+ sstr_match = re.search(self.SSTRING_PATTERN, d)
1653
+ if sstr_match:
1654
+ ident = sstr_match.group(1).strip()
1655
+ resolved = self._resolve_identifier(ident)
1656
+ property["default"] = resolved or ident
1657
+ return property
1658
+
1659
+ # ------------------------------------------------------------------
1660
+ # std::vector initializer
1661
+ # ------------------------------------------------------------------
1662
+ if "std::vector" in d:
1663
+ property["default"] = self._parse_vector_initializer(d)
1664
+ property["type"] = "array"
1665
+ return property
1666
+
1667
+ # ------------------------------------------------------------------
1668
+ # Plain braced list { ... }
1669
+ # ------------------------------------------------------------------
1670
+ if re.match(self.BRACED_LIST_PATTERN, d):
1671
+ property["default"] = self._parse_initializer_list(d)
1672
+ property["type"] = "array"
1673
+ return property
1674
+
1675
+ # ------------------------------------------------------------------
1676
+ # Plain bool / numeric literals
1677
+ # ------------------------------------------------------------------
1678
+ if d in ("true", "false"):
1679
+ property["default"] = (d == "true")
1680
+ return property
1681
+ if re.match(r"^-?\d+\.\d+[fFlL]*$", d):
1682
+ # Strip C++ floating point suffixes (f, F, l, L)
1683
+ property["default"] = float(re.sub(r"[fFlL]+$", "", d))
1684
+ return property
1685
+ # Strip C++ integer suffixes (u, U, l, L, ll, LL, ul, UL, etc.)
1686
+ int_match = re.match(r"^(-?\d+)([uUlL]+)?$", d)
1687
+ if int_match:
1688
+ property["default"] = int(int_match.group(1))
1689
+ return property
1690
+
1691
+ # ------------------------------------------------------------------
1692
+ # Size literals like 20_GiB
1693
+ # ------------------------------------------------------------------
1694
+ size_match = re.match(r"(\d+)_([KMGTP])iB", d)
1695
+ if size_match:
1696
+ num, unit = int(size_match.group(1)), size_match.group(2)
1697
+ mult = {"K": 1024, "M": 1024**2, "G": 1024**3, "T": 1024**4, "P": 1024**5}[unit]
1698
+ property["default"] = num * mult
1699
+ return property
1700
+
1701
+ # ------------------------------------------------------------------
1702
+ # Computed C++ constants (max_serializable_ms, serde::max_serializable_ms, etc.)
1703
+ # Check this BEFORE generic namespace identifier check to ensure computed constants are resolved
1704
+ # ------------------------------------------------------------------
1705
+ if d in COMPUTED_CONSTANTS:
1706
+ property["default"] = COMPUTED_CONSTANTS[d]
1707
+ return property
1708
+
1709
+ # ------------------------------------------------------------------
1710
+ # Constant-like or symbolic identifiers (model::..., net::..., etc.)
1711
+ # ------------------------------------------------------------------
1712
+ if "::" in d and not d.startswith("std::"):
1713
+ # Try to resolve to string enum if possible
1714
+ resolved = self._resolve_identifier(d)
1715
+ property["default"] = resolved or d
1716
+ return property
1717
+
1718
+ # ------------------------------------------------------------------
1719
+ # Fallback — normalized string
1720
+ # ------------------------------------------------------------------
1721
+ property["default"] = normalize_string(d).replace("std::", "")
1722
+ return property
1723
+
1724
+
1725
+ @debug_transformer
1726
+ class EnumDefaultMappingTransformer:
1727
+ """
1728
+ Maps enum default values using enum string mappings.
1729
+
1730
+ When an enum type has a _to_string() conversion function, the type definitions
1731
+ include enum_string_mappings. This transformer applies those mappings to the
1732
+ property's default value.
1733
+
1734
+ Example:
1735
+ Property: write_caching_default
1736
+ Type: model::write_caching_mode
1737
+ Default: "default_false" (raw enum value)
1738
+
1739
+ After mapping:
1740
+ Default: "false" (user-facing string)
1741
+ """
1742
+
1743
+
1744
+ def accepts(self, info, file_pair):
1745
+ # Accept properties that have enum constraints
1746
+ if "caching" in info.get("name", ""):
1747
+ logger.warning(f"[EnumDefaultMapping.accepts] {info.get('name')}: enum={info.get('enum')}, result={bool(info.get('enum'))}")
1748
+ return bool(info.get("enum"))
1749
+
1750
+ def parse(self, property, info, file_pair):
1751
+ # Get default from property dict (may have been transformed by previous transformers)
1752
+ default = property.get("default")
1753
+
1754
+ # Get enum values from BOTH property (transformed) and info (raw)
1755
+ # Prefer property's enum if it exists (already transformed), otherwise use info's
1756
+ enum_values = property.get("enum", info.get("enum", []))
1757
+
1758
+ prop_name = property.get("name", info.get("name", "unknown"))
1759
+
1760
+ # DEBUG logging for write_caching_default
1761
+ if prop_name == "write_caching_default":
1762
+ logger.warning(f"[EnumDefaultMapping] {prop_name}: default={default}, enum_values={enum_values}")
1763
+ logger.warning(f" property keys: {list(property.keys())}")
1764
+ logger.warning(f" info.enum: {info.get('enum')}")
1765
+
1766
+ # Skip if no default
1767
+ if not default or not isinstance(default, str):
1768
+ if prop_name == "write_caching_default":
1769
+ logger.warning(f"[EnumDefaultMapping] Skipping: no default or not string")
1770
+ return property
1771
+
1772
+ # Skip if default is already one of the mapped enum values
1773
+ if default in enum_values:
1774
+ if prop_name == "write_caching_default":
1775
+ logger.warning(f"[EnumDefaultMapping] Skipping: default already in enum values")
1776
+ return property
1777
+
1778
+ # Check if we have a type definition with mappings
1779
+ type_defs = get_type_definitions()
1780
+ if not type_defs:
1781
+ if prop_name == "write_caching_default":
1782
+ logger.warning(f"[EnumDefaultMapping] No type definitions available")
1783
+ return property
1784
+
1785
+ # Try to find the enum type definition
1786
+ # The enum might be stored under various names, try to match
1787
+ for type_name, type_def in type_defs.items():
1788
+ if type_def.get("type") != "enum":
1789
+ continue
1790
+
1791
+ mappings = type_def.get("enum_string_mappings")
1792
+ if not mappings:
1793
+ continue
1794
+
1795
+ # If this default value exists in the mappings, apply it
1796
+ if default in mappings:
1797
+ mapped_value = mappings[default]
1798
+ property["default"] = mapped_value
1799
+ logger.debug(f"Mapped enum default for {prop_name}: {default} -> {mapped_value}")
638
1800
  return property
639
1801
 
640
- # For all other cases, leave the default as-is
641
- property["default"] = default
642
1802
  return property
643
1803
 
644
1804
 
1805
+ def get_type_definitions():
1806
+ """
1807
+ Lazily import type definitions to avoid circular imports.
1808
+ Returns the type definitions dictionary from property_extractor.
1809
+ """
1810
+ try:
1811
+ import property_extractor
1812
+ # Access the module-level variable directly
1813
+ if hasattr(property_extractor, '_type_definitions_cache'):
1814
+ cache = property_extractor._type_definitions_cache
1815
+ if cache: # Check it's not empty
1816
+ return cache
1817
+ return None
1818
+ except (ImportError, AttributeError) as e:
1819
+ logger.debug(f"Could not import type definitions: {e}")
1820
+ return None
1821
+
1822
+
1823
+ @debug_transformer
645
1824
  class ExperimentalTransformer:
1825
+ """
1826
+ Marks a property as experimental if flagged in meta or hidden_when_default.
1827
+ """
646
1828
  def accepts(self, info, file_pair):
647
- return info.get("type") is not None and info["type"].startswith(("development_", "hidden_when_default_"))
1829
+ return (
1830
+ get_meta_value(info, "experimental") is not None
1831
+ or (info.get("type") and info["type"].startswith(("development_", "hidden_when_default_")))
1832
+ )
1833
+
648
1834
  def parse(self, property, info, file_pair):
649
- property["is_experimental_property"] = True
1835
+ # Check if type indicates experimental (development_ or hidden_when_default_)
1836
+ is_experimental_type = (
1837
+ info.get("type") and
1838
+ info["type"].startswith(("development_", "hidden_when_default_"))
1839
+ )
1840
+
1841
+ # Get meta value, but default to "yes" if type is experimental
1842
+ default_val = "yes" if is_experimental_type else "no"
1843
+ val = get_meta_value(info, "experimental", default_val)
1844
+
1845
+ property["is_experimental_property"] = (val != "no")
650
1846
  return property
651
1847
 
652
1848
 
1849
+ @debug_transformer
653
1850
  class AliasTransformer:
654
1851
  def accepts(self, info, file_pair):
655
1852
  if 'params' in info:
@@ -670,108 +1867,733 @@ class AliasTransformer:
670
1867
  aliases.extend(alias.strip('{}"') for alias in aliases_dict.values())
671
1868
  property['aliases'] = aliases
672
1869
 
673
-
1870
+ @debug_transformer
674
1871
  class EnterpriseTransformer:
675
1872
  """
676
- Transforms enterprise property values from C++ expressions to user-friendly JSON.
1873
+ Detect and extract enterprise-only constructor parameters.
1874
+
1875
+ Relies on the parser's header metadata (`is_enterprise` / `is_enterprise_wrapper`)
1876
+ rather than heuristics. If a property is not marked as enterprise in its
1877
+ declaration type (e.g. `enterprise<property<T>>`), this transformer skips it.
677
1878
 
678
- This transformer processes enterprise values by delegating to the
679
- centralized process_enterprise_value function which handles the full range of
680
- C++ expression types found in enterprise property definitions.
1879
+ Supported patterns:
1880
+ - restricted_only: Enterprise config with restricted values only
1881
+ - sanctioned_only: Enterprise config with sanctioned values only
1882
+ - restricted_with_sanctioned: both restricted + sanctioned values
681
1883
  """
1884
+
682
1885
  def accepts(self, info, file_pair):
683
- """
684
- Return True if the provided info indicates an enterprise-only property.
685
-
686
- Parameters:
687
- info (dict): The metadata dictionary for a property. This function checks for a 'type' key whose string contains 'enterprise'.
688
- file_pair: Unused; present for transformer interface compatibility.
689
-
690
- Returns:
691
- bool: True when info contains a 'type' that includes 'enterprise', otherwise False.
692
- """
693
- return bool(info.get('type') and 'enterprise' in info['type'])
1886
+ """Run for all properties to set is_enterprise flag."""
1887
+ # Always run for all properties so we can set is_enterprise flag
1888
+ return True
694
1889
 
695
1890
  def parse(self, property, info, file_pair):
696
- """
697
- Mark a property as enterprise-only and attach its enterprise value.
698
-
699
- If an enterprise value is present in info['params'][0]['value'], this method attempts to process it using the shared
700
- process_enterprise_value helper (loaded via get_process_enterprise_value()). If the processor is unavailable or raises
701
- an exception, the raw enterprise value is used.
702
-
703
- Side effects:
704
- - Sets property["enterprise_value"] to the processed or raw value.
705
- - Sets property["is_enterprise"] = True.
706
- - Removes the first element from info['params'].
707
-
708
- Parameters:
709
- property (dict): Property bag to modify and return.
710
- info (dict): Parsed metadata; must have a non-None 'params' list for processing.
711
- file_pair: Unused here but accepted for transformer API compatibility.
712
-
713
- Returns:
714
- dict: The updated property bag.
715
- """
716
- if info['params'] is not None:
717
- enterpriseValue = info['params'][0]['value']
718
-
719
- # Get the processing function
720
- process_enterprise_value = get_process_enterprise_value()
721
- if process_enterprise_value is None:
722
- property["enterprise_value"] = enterpriseValue
723
- property['is_enterprise'] = True
724
- del info['params'][0]
725
- return property
726
-
727
- try:
728
- processed_value = process_enterprise_value(enterpriseValue)
729
- property["enterprise_value"] = processed_value
730
- except Exception:
731
- # Fallback to raw value if processing fails
732
- property["enterprise_value"] = enterpriseValue
1891
+ """Identify and record enterprise constructor values, if applicable."""
1892
+ if not self.accepts(info, file_pair):
1893
+ return property
733
1894
 
734
- property['is_enterprise'] = True
735
- del info['params'][0]
1895
+ # Check if this is actually an enterprise property
1896
+ is_enterprise_property = info.get("is_enterprise") or info.get("is_enterprise_wrapper")
1897
+
1898
+ params = info.get("params", [])
1899
+ if not params or len(params) < 3:
1900
+ # Not enough params to be an enterprise constructor
1901
+ property["is_enterprise"] = bool(is_enterprise_property)
1902
+ return property
1903
+
1904
+ # If not marked as enterprise in the source, set to False and return
1905
+ if not is_enterprise_property:
1906
+ property["is_enterprise"] = False
1907
+ return property
1908
+
1909
+ enterprise_constructor = None
1910
+ restricted_vals, sanctioned_vals = None, None
1911
+
1912
+ # --- Check for vector at both start and end ---
1913
+ if len(params) >= 6 and all("std::vector" in str(p["value"]) for p in [params[0], params[-1]]):
1914
+ restricted_vals = self._extract_list_items(params[0]["value"])
1915
+ last_vector_vals = self._extract_list_items(params[-1]["value"])
1916
+
1917
+ # If last vector is a superset of first vector, it's the enum definition, not sanctioned values
1918
+ # Pattern: restricted_only with enum defined in last parameter
1919
+ if set(restricted_vals).issubset(set(last_vector_vals)) and len(last_vector_vals) > len(restricted_vals):
1920
+ enterprise_constructor = "restricted_only"
1921
+ info["params"] = params[1:-1] # Keep last param for enum extraction by TypeTransformer
1922
+ else:
1923
+ # True restricted_with_sanctioned: last vector is sanctioned values, not enum
1924
+ enterprise_constructor = "restricted_with_sanctioned"
1925
+ sanctioned_vals = last_vector_vals
1926
+ info["params"] = params[1:-1]
1927
+
1928
+ # --- restricted_only (vector form) ---
1929
+ elif len(params) >= 5 and "std::vector" in str(params[0]["value"]):
1930
+ enterprise_constructor = "restricted_only"
1931
+ restricted_vals = self._extract_list_items(params[0]["value"])
1932
+ info["params"] = params[1:]
1933
+
1934
+ # --- sanctioned_only (vector form) ---
1935
+ elif len(params) >= 5 and "std::vector" in str(params[-1]["value"]):
1936
+ enterprise_constructor = "sanctioned_only"
1937
+ sanctioned_vals = self._extract_list_items(params[-1]["value"])
1938
+ info["params"] = params[:-1]
1939
+
1940
+ # --- restricted_with_sanctioned (scalar form) ---
1941
+ elif (
1942
+ len(params) >= 6
1943
+ and all(p["type"] in ("true", "false", "integer_literal", "string_literal", "qualified_identifier") for p in params[:2])
1944
+ ):
1945
+ enterprise_constructor = "restricted_with_sanctioned"
1946
+ restricted_vals = [self._clean_value(params[0]["value"])]
1947
+ sanctioned_vals = [self._clean_value(params[1]["value"])]
1948
+ info["params"] = params[2:]
1949
+
1950
+ # --- restricted_only (scalar form) ---
1951
+ elif len(params) >= 5 and params[0]["type"] in (
1952
+ "true", "false", "integer_literal", "string_literal", "qualified_identifier"
1953
+ ):
1954
+ enterprise_constructor = "restricted_only"
1955
+ restricted_vals = [self._clean_value(params[0]["value"])]
1956
+ info["params"] = params[1:]
1957
+
1958
+ # --- simple enterprise property (lambda validator pattern) ---
1959
+ elif (len(params) >= 3 and
1960
+ params[0].get("type") == "lambda_expression" and
1961
+ params[1].get("type") == "string_literal"):
1962
+ enterprise_constructor = "simple"
1963
+ # Don't modify params for simple enterprise properties - they have normal structure
1964
+ # Remove the lambda validator from parameters as it's not needed for documentation
1965
+ info["params"] = params[1:]
1966
+
1967
+ if not enterprise_constructor:
1968
+ # Not an enterprise property - explicitly set to False
1969
+ property["is_enterprise"] = False
736
1970
  return property
737
1971
 
1972
+ # Record enterprise attributes
1973
+ property["is_enterprise"] = True
1974
+ property["enterprise_constructor"] = enterprise_constructor
1975
+
1976
+ if restricted_vals is not None:
1977
+ property["enterprise_restricted_value"] = restricted_vals
1978
+ property["enterprise_value"] = restricted_vals # backward compat
1979
+
1980
+ if sanctioned_vals is not None:
1981
+ property["enterprise_sanctioned_value"] = sanctioned_vals
1982
+
1983
+ # Add friendly description (values are already cleaned by _clean_value)
1984
+ if enterprise_constructor == "restricted_with_sanctioned":
1985
+ r = restricted_vals[0]
1986
+ s = sanctioned_vals[0]
1987
+ property["enterprise_default_description"] = (
1988
+ f"Default: `{s}` (Community) or `{r}` (Enterprise)"
1989
+ )
1990
+ elif enterprise_constructor == "restricted_only":
1991
+ if len(restricted_vals) > 1:
1992
+ vals = ", ".join(f"`{v}`" for v in restricted_vals)
1993
+ property["enterprise_default_description"] = (
1994
+ f"Available only with Enterprise license: {vals}"
1995
+ )
1996
+ else:
1997
+ property["enterprise_default_description"] = (
1998
+ f"Available only with Enterprise license: `{restricted_vals[0]}`"
1999
+ )
2000
+ elif enterprise_constructor == "sanctioned_only":
2001
+ property["enterprise_default_description"] = (
2002
+ f"Community-only configuration. Sanctioned value: `{sanctioned_vals[0]}`"
2003
+ )
2004
+
2005
+ return property
2006
+
2007
+ # --- Helper: clean literal/identifier values ---
2008
+ def _clean_value(self, val):
2009
+ if not isinstance(val, str):
2010
+ return val
2011
+ val = val.strip()
2012
+ # remove surrounding quotes for string literals
2013
+ if len(val) >= 2 and val[0] == '"' and val[-1] == '"':
2014
+ val = val[1:-1]
2015
+ # Strip C++ namespace qualifiers from enum values
2016
+ # e.g., model::partition_autobalancing_mode::continuous → continuous
2017
+ if '::' in val:
2018
+ val = val.split('::')[-1]
2019
+ return val
2020
+
2021
+ # --- Helper: extract list elements from std::vector{...} ---
2022
+ def _extract_list_items(self, vector_text):
2023
+ match = re.search(r"\{([^}]*)\}", vector_text)
2024
+ if not match:
2025
+ return []
2026
+ contents = match.group(1)
2027
+ items = [self._clean_value(v.strip()) for v in contents.split(",") if v.strip()]
2028
+ return items
2029
+
738
2030
 
2031
+ @debug_transformer
739
2032
  class MetaParamTransformer:
2033
+ """
2034
+ Converts Redpanda meta initializer strings (meta{...} or {...})
2035
+ into structured dictionaries usable by downstream transformers.
2036
+
2037
+ Handles all Redpanda meta fields:
2038
+ - .needs_restart
2039
+ - .visibility
2040
+ - .example
2041
+ - .deprecated
2042
+ - .secret
2043
+ - .experimental
2044
+ - .gets_restored / .restored
2045
+
2046
+ Works with both explicit and implicit meta wrappers:
2047
+ meta{ .visibility = visibility::user }
2048
+ {.needs_restart = needs_restart::no, .example = "123"}
2049
+
2050
+ Skips non-meta initializer lists like {"a", "b"}.
2051
+ """
2052
+
2053
+ def accepts(self, info, file_pair):
2054
+ # Accept if any param looks like a meta initializer
2055
+ return any(
2056
+ isinstance(p.get("value"), str)
2057
+ and (
2058
+ # explicit meta{...}
2059
+ p["value"].strip().startswith("meta{")
2060
+ # or bare {...} with at least one known meta key
2061
+ or (
2062
+ p["value"].strip().startswith("{")
2063
+ and re.search(
2064
+ r"\.(needs_restart|visibility|example|deprecated|secret|experimental|gets_restored|restored)\s*=",
2065
+ p["value"]
2066
+ )
2067
+ )
2068
+ )
2069
+ for p in info.get("params", [])
2070
+ )
2071
+
2072
+ def parse(self, property, info, file_pair):
2073
+ params = info.get("params", [])
2074
+ for p in params:
2075
+ val = p.get("value")
2076
+ if not isinstance(val, str):
2077
+ continue
2078
+
2079
+ stripped = val.strip()
2080
+
2081
+ # Only treat as meta if it matches known meta key patterns
2082
+ if not (
2083
+ stripped.startswith("meta{")
2084
+ or (
2085
+ stripped.startswith("{")
2086
+ and re.search(
2087
+ r"\.(needs_restart|visibility|example|deprecated|secret|experimental|gets_restored|restored)\s*=",
2088
+ stripped
2089
+ )
2090
+ )
2091
+ ):
2092
+ continue
2093
+
2094
+ # Extract content between first '{' and matching '}'
2095
+ match = re.search(r"\{(.*)\}", stripped, re.DOTALL)
2096
+ if not match:
2097
+ continue
2098
+
2099
+ inner = match.group(1).strip()
2100
+ meta_dict = {}
2101
+
2102
+ # Split fields like `.needs_restart = needs_restart::no,`
2103
+ for field in re.split(r",\s*(?=\.)", inner):
2104
+ field = field.strip().lstrip(".")
2105
+ if not field or "=" not in field:
2106
+ continue
2107
+
2108
+ key, value = [s.strip() for s in field.split("=", 1)]
2109
+ clean_key = key.replace(".", "")
2110
+ meta_dict[clean_key] = value
2111
+
2112
+ # 🔹 Inline special handlers for known meta keys
2113
+ # ----------------------------------------------
2114
+
2115
+ # Example values
2116
+ if clean_key == "example":
2117
+ val_clean = value.strip().strip('"')
2118
+ # Try to coerce to int or float, else leave as string
2119
+ if re.fullmatch(r"-?\d+", val_clean):
2120
+ property["example"] = int(val_clean)
2121
+ elif re.fullmatch(r"-?\d+\.\d+", val_clean):
2122
+ property["example"] = float(val_clean)
2123
+ else:
2124
+ property["example"] = normalize_string(val_clean)
2125
+
2126
+ # Gets_restored / restored flags
2127
+ elif clean_key in ("gets_restored", "restored"):
2128
+ val_clean = re.sub(r"^.*::", "", value)
2129
+ property["gets_restored"] = (val_clean != "no")
2130
+
2131
+ # Secret flags
2132
+ elif clean_key == "secret":
2133
+ val_clean = re.sub(r"^.*::", "", value)
2134
+ property["is_secret"] = (val_clean == "yes")
2135
+
2136
+ # Visibility normalization (user, tunable, deprecated)
2137
+ elif clean_key == "visibility":
2138
+ val_clean = re.sub(r"^.*::", "", value)
2139
+ property["visibility"] = val_clean
2140
+
2141
+ # Needs restart normalization
2142
+ elif clean_key == "needs_restart":
2143
+ val_clean = re.sub(r"^.*::", "", value)
2144
+ property["needs_restart"] = (val_clean != "no")
2145
+
2146
+ # Tag and attach parsed meta info
2147
+ meta_dict["type"] = "initializer_list"
2148
+ p["value"] = meta_dict
2149
+
2150
+ return property
2151
+
2152
+
2153
+ @debug_transformer
2154
+ class ExampleTransformer:
2155
+ """
2156
+ ExampleTransformer - Extracts example values from C++ property meta parameters
2157
+
2158
+ RESPONSIBILITY:
2159
+ Processes the `.example` field from C++ meta{} initializers and adds the example
2160
+ value to the PropertyBag for documentation generation.
2161
+
2162
+ PROCESSING:
2163
+ 1. Checks parameters for dictionary format meta data (already parsed by parser)
2164
+ 2. Searches for `example` key in initializer_list parameters
2165
+ 3. Attempts to parse example value as integer, float, or string
2166
+ 4. Adds parsed example to property as "example" field
2167
+
2168
+ DOWNSTREAM USAGE:
2169
+ - Documentation generators use example values in property descriptions
2170
+ - Configuration examples use placeholder values
2171
+ - API documentation shows realistic usage patterns
2172
+
2173
+ INPUT FORMAT (parsed parameters):
2174
+ [
2175
+ {'value': 'property_name', 'type': 'string_literal'},
2176
+ {'value': 'Description...', 'type': 'string_literal'},
2177
+ {'value': {'needs_restart': 'needs_restart::yes', 'example': '"1073741824"', 'visibility': 'visibility::tunable'}, 'type': 'initializer_list'},
2178
+ ...
2179
+ ]
2180
+
2181
+ OUTPUT:
2182
+ Adds to PropertyBag: {"example": "1073741824"}
2183
+ """
2184
+
740
2185
  def accepts(self, info, file_pair):
2186
+ """Accept properties that have meta parameters with example values in dictionary format"""
2187
+ if not info.get("params"):
2188
+ return False
2189
+
2190
+ # Look for initializer_list parameters that contain example keys
2191
+ for param in info["params"]:
2192
+ if param.get("type") == "initializer_list" and isinstance(param.get("value"), dict):
2193
+ if "example" in param["value"]:
2194
+ return True
2195
+ return False
2196
+
2197
+ def parse(self, property, info, file_pair):
2198
+ """Extract example value from parsed meta parameters"""
2199
+ if not self.accepts(info, file_pair):
2200
+ return property
2201
+
2202
+ for param in info["params"]:
2203
+ if param.get("type") == "initializer_list" and isinstance(param.get("value"), dict):
2204
+ meta_dict = param["value"]
2205
+ if "example" in meta_dict:
2206
+ example_val = meta_dict["example"]
2207
+
2208
+ # Clean up the value (remove quotes, etc.)
2209
+ if isinstance(example_val, str):
2210
+ example_val = example_val.strip().strip('"\'')
2211
+
2212
+ # Try to coerce to appropriate type
2213
+ if isinstance(example_val, str):
2214
+ if re.fullmatch(r"-?\d+", example_val):
2215
+ property["example"] = int(example_val)
2216
+ elif re.fullmatch(r"-?\d+\.\d+", example_val):
2217
+ property["example"] = float(example_val)
2218
+ else:
2219
+ property["example"] = example_val
2220
+ else:
2221
+ property["example"] = example_val
2222
+ break
2223
+
2224
+ return property
2225
+
2226
+
2227
+ @debug_transformer
2228
+ class ValidatorEnumExtractor:
2229
+ """
2230
+ ValidatorEnumExtractor - Extracts enum constraints from validator functions for array properties
2231
+
2232
+ RESPONSIBILITY:
2233
+ Analyzes validator functions to extract enum constraints for array-typed properties.
2234
+ For example, if sasl_mechanisms uses validate_sasl_mechanisms, this transformer:
2235
+ 1. Finds the validator function in validators.cc
2236
+ 2. Identifies the constraint array (e.g., supported_sasl_mechanisms)
2237
+ 3. Resolves that array to get the actual enum values
2238
+ 4. Adds them to property['items']['enum']
2239
+
2240
+ PROCESSING:
2241
+ 1. Detects array properties (type="array") with validator parameters
2242
+ 2. Extracts validator function name from params
2243
+ 3. Parses validator to find constraint array
2244
+ 4. Resolves array to get enum values (e.g., ["SCRAM", "GSSAPI", "OAUTHBEARER", "PLAIN"])
2245
+ 5. Sets property['items']['enum'] with the discovered values
2246
+
2247
+ DOWNSTREAM USAGE:
2248
+ - JSON Schema generators use items.enum for validation rules
2249
+ - Documentation shows accepted values for array properties
2250
+ - API clients use enum values for input validation
2251
+
2252
+ EXAMPLE:
2253
+ Input property with validator:
2254
+ sasl_mechanisms(..., validate_sasl_mechanisms)
2255
+
2256
+ Output property with enum:
2257
+ {
2258
+ "type": "array",
2259
+ "items": {
2260
+ "type": "string",
2261
+ "enum": ["SCRAM", "GSSAPI", "OAUTHBEARER", "PLAIN"]
2262
+ }
2263
+ }
2264
+ """
2265
+ def __init__(self, constant_resolver):
741
2266
  """
742
- Check if the given info contains parameters that include a meta{...} value.
2267
+ Args:
2268
+ constant_resolver: ConstantResolver instance for resolving C++ constants
743
2269
  """
744
- if 'params' in info:
745
- for param in info['params']:
746
- if isinstance(param, dict) and 'value' in param:
747
- value = param['value']
748
- if isinstance(value, str) and value.startswith("meta{"):
749
- return True
750
- return False
2270
+ self.constant_resolver = constant_resolver
2271
+
2272
+ def accepts(self, info, file_pair):
2273
+ """Only process array properties that have validator params."""
2274
+ params = info.get("params", [])
2275
+ if not params:
2276
+ return False
2277
+
2278
+ # Must be an array type
2279
+ base_type = info.get("base_property_type", "")
2280
+ if not base_type or "vector" not in base_type:
2281
+ return False
2282
+
2283
+ # Must have a validator parameter (usually the last param)
2284
+ # Validator is typically after: name, desc, meta, default
2285
+ return len(params) >= 5
751
2286
 
752
2287
  def parse(self, property, info, file_pair):
2288
+ """Extract enum constraint from validator function."""
2289
+ if not self.accepts(info, file_pair):
2290
+ return property
2291
+
2292
+ params = info.get("params", [])
2293
+
2294
+ # Look for validator in the last few parameters
2295
+ # Typically: params[0]=name, [1]=desc, [2]=meta, [3]=default, [4]=validator
2296
+ validator_param = None
2297
+ validator_name = None
2298
+
2299
+ for i in range(min(4, len(params)), len(params)):
2300
+ param = params[i]
2301
+ param_val = param.get("value", "")
2302
+ param_type = param.get("type", "")
2303
+
2304
+ # Skip if it's not a validator-like identifier
2305
+ if param_type not in ("identifier", "qualified_identifier", "call_expression"):
2306
+ continue
2307
+
2308
+ # Check if it looks like a validator function name
2309
+ if isinstance(param_val, str) and ("validate" in param_val or "validator" in param_val):
2310
+ validator_name = param_val.replace("&", "").strip()
2311
+ validator_param = param
2312
+ break
2313
+
2314
+ if not validator_name:
2315
+ return property
2316
+
2317
+ # Use constant_resolver to extract enum constraint from validator
2318
+ from constant_resolver import resolve_validator_enum_constraint
2319
+
2320
+ enum_results = resolve_validator_enum_constraint(validator_name, self.constant_resolver)
2321
+
2322
+ if enum_results:
2323
+ # Add enum to items
2324
+ if "items" not in property:
2325
+ property["items"] = {}
2326
+
2327
+ # Extract just the values for the enum field
2328
+ property["items"]["enum"] = [result["value"] for result in enum_results]
2329
+
2330
+ # Add metadata about which enum values are enterprise-only
2331
+ enum_metadata = {}
2332
+ for result in enum_results:
2333
+ enum_metadata[result["value"]] = {
2334
+ "is_enterprise": result["is_enterprise"]
2335
+ }
2336
+
2337
+ # Only add x-enum-metadata if there are enterprise values
2338
+ if any(result["is_enterprise"] for result in enum_results):
2339
+ property["items"]["x-enum-metadata"] = enum_metadata
2340
+
2341
+ logger.info(f"✓ Extracted enum constraint for {property.get('name', 'unknown')} from validator {validator_name}: {property['items']['enum']}")
2342
+
2343
+ return property
2344
+
2345
+
2346
+ @debug_transformer
2347
+ class RuntimeValidationEnumExtractor:
2348
+ """
2349
+ RuntimeValidationEnumExtractor - Extracts enum constraints from runtime validation functions
2350
+
2351
+ RESPONSIBILITY:
2352
+ For string properties without constructor validators, searches for runtime validation
2353
+ functions that compare the property value against constants and extracts those as enum values.
2354
+
2355
+ PROCESSING:
2356
+ 1. Detects string properties (not arrays) without validator parameters
2357
+ 2. Searches the source file for validation functions that reference the property
2358
+ 3. Parses comparison patterns (e.g., property != constant1 && property != constant2)
2359
+ 4. Resolves constants to actual string values
2360
+ 5. Sets property['enum'] with discovered values
2361
+
2362
+ EXAMPLE:
2363
+ Input property without constructor validator:
2364
+ sasl_mechanism(*this, "sasl_mechanism", "Description...", {}, "")
2365
+
2366
+ Runtime validation function:
2367
+ void validate_sasl_properties(..., std::string_view mechanism, ...) {
2368
+ if (mechanism != security::scram_sha256_authenticator::name
2369
+ && mechanism != security::scram_sha512_authenticator::name
2370
+ && mechanism != security::oidc::sasl_authenticator::name) {
2371
+ throw std::invalid_argument("Invalid mechanism");
2372
+ }
2373
+ }
2374
+
2375
+ Output property with enum:
2376
+ {
2377
+ "type": "string",
2378
+ "enum": ["SCRAM-SHA-256", "SCRAM-SHA-512", "OAUTHBEARER"]
2379
+ }
2380
+ """
2381
+ def __init__(self, constant_resolver):
753
2382
  """
754
- Transform into a structured dictionary.
2383
+ Args:
2384
+ constant_resolver: ConstantResolver instance for resolving C++ constants
755
2385
  """
756
- if 'params' not in info or info['params'] is None:
757
- return property
758
-
759
- iterable_params = info['params']
760
- for param in iterable_params:
761
- if isinstance(param['value'], str) and param['value'].startswith("meta{"):
762
- # Extract content between meta{ and } using explicit slicing
763
- param_value = param['value']
764
- if param_value.endswith('}'):
765
- meta_content = param_value[5:-1].strip() # Remove "meta{" and "}"
766
- else:
767
- # Handle malformed meta{ without closing }
768
- meta_content = param_value[5:].strip() # Remove "meta{" only
2386
+ self.constant_resolver = constant_resolver
2387
+
2388
+ def accepts(self, info, file_pair):
2389
+ """Only process string properties without constructor validators."""
2390
+ params = info.get("params", [])
2391
+ if not params:
2392
+ return False
2393
+
2394
+ # Must be a string type (not array)
2395
+ base_type = info.get("base_property_type", "")
2396
+ if not base_type:
2397
+ return False
2398
+ if "vector" in base_type or "array" in base_type:
2399
+ return False
2400
+
2401
+ # Should be ss::sstring or std::string
2402
+ if "sstring" not in base_type and "string" not in base_type:
2403
+ return False
2404
+
2405
+ # Should NOT have a validator parameter (those are handled by ValidatorEnumExtractor)
2406
+ # Check if any param looks like a validator
2407
+ for param in params[4:]: # Skip name, desc, meta, default
2408
+ param_val = param.get("value", "")
2409
+ if isinstance(param_val, str) and ("validate" in param_val or "validator" in param_val):
2410
+ return False
2411
+
2412
+ return True
2413
+
2414
+ def parse(self, property, info, file_pair):
2415
+ """Extract enum constraint from runtime validation function."""
2416
+ if not self.accepts(info, file_pair):
2417
+ return property
2418
+
2419
+ property_name = property.get("name")
2420
+ defined_in = property.get("defined_in")
2421
+
2422
+ if not property_name or not defined_in:
2423
+ return property
2424
+
2425
+ # Use constant_resolver to extract enum constraint from runtime validation
2426
+ from constant_resolver import resolve_runtime_validation_enum_constraint
2427
+
2428
+ enum_results = resolve_runtime_validation_enum_constraint(
2429
+ property_name, defined_in, self.constant_resolver
2430
+ )
2431
+
2432
+ if enum_results:
2433
+ # Extract just the values for the enum field
2434
+ property["enum"] = [result["value"] for result in enum_results]
2435
+
2436
+ # Add metadata about which enum values are enterprise-only
2437
+ enum_metadata = {}
2438
+ for result in enum_results:
2439
+ enum_metadata[result["value"]] = {
2440
+ "is_enterprise": result["is_enterprise"]
2441
+ }
2442
+
2443
+ # Only add x-enum-metadata if there are enterprise values
2444
+ if any(result["is_enterprise"] for result in enum_results):
2445
+ property["x-enum-metadata"] = enum_metadata
2446
+
2447
+ logger.info(f"✓ Extracted runtime validation enum for {property_name}: {property['enum']}")
2448
+
2449
+ return property
2450
+
2451
+
2452
+
2453
+ ################################################################################
2454
+ # TRANSFORMER SYSTEM USAGE AND EXTENSION GUIDE
2455
+ ################################################################################
2456
+
2457
+ """
2458
+ USING THE TRANSFORMER SYSTEM:
2459
+
2460
+ The transformers are designed to be used as part of the property extraction pipeline
2461
+ in property_extractor.py. The typical usage pattern is:
2462
+
2463
+ transformers = [
2464
+ ParamNormalizerTransformer(),
2465
+ BasicInfoTransformer(),
2466
+ MetaParamTransformer(),
2467
+ NeedsRestartTransformer(),
2468
+ GetsRestoredTransformer(),
2469
+ IsSecretTransformer(),
2470
+ VisibilityTransformer(),
2471
+ IsNullableTransformer(),
2472
+ IsArrayTransformer(type_transformer),
2473
+ TypeTransformer(),
2474
+ DeprecatedTransformer(),
2475
+ NumericBoundsTransformer(type_transformer),
2476
+ DurationBoundsTransformer(type_transformer),
2477
+ SimpleDefaultValuesTransformer(),
2478
+ FriendlyDefaultTransformer(),
2479
+ ExperimentalTransformer(),
2480
+ AliasTransformer(),
2481
+ EnterpriseTransformer(),
2482
+ ]
2483
+
2484
+ for name, property_info in extracted_properties.items():
2485
+ property_bag = PropertyBag()
2486
+
2487
+ for transformer in transformers:
2488
+ if transformer.accepts(property_info, file_pair):
2489
+ transformer.parse(property_bag, property_info, file_pair)
2490
+
2491
+ final_properties[name] = property_bag
2492
+
2493
+ CREATING NEW TRANSFORMERS:
2494
+
2495
+ To add a new transformer, follow this pattern:
2496
+
2497
+ @debug_transformer
2498
+ class MyNewTransformer:
2499
+ '''
2500
+ Brief description of what this transformer does.
2501
+
2502
+ Include:
2503
+ - What it detects/extracts
2504
+ - Expected input format
2505
+ - Expected output format
2506
+ - Downstream dependencies
2507
+ - Usage examples
2508
+ '''
2509
+
2510
+ def accepts(self, info, file_pair):
2511
+ '''
2512
+ Return True if this transformer should process the given property.
2513
+
2514
+ Args:
2515
+ info (dict): Raw property info from Tree-sitter parser
2516
+ file_pair (FilePair): Source file locations
769
2517
 
770
- meta_dict = {}
771
- for item in meta_content.split(','):
772
- item = item.strip()
773
- if '=' in item:
774
- key, value = item.split('=')
775
- meta_dict[key.strip().replace('.', '')] = value.strip()
776
- meta_dict['type'] = 'initializer_list' # Enforce required type
777
- param['value'] = meta_dict
2518
+ Returns:
2519
+ bool: True if transformer should process this property
2520
+ '''
2521
+ # Add your detection logic here
2522
+ return some_condition_check(info)
2523
+
2524
+ def parse(self, property, info, file_pair):
2525
+ '''
2526
+ Extract/transform information from raw property info.
2527
+
2528
+ Args:
2529
+ property (PropertyBag): Property to populate/modify
2530
+ info (dict): Raw property info to extract from
2531
+ file_pair (FilePair): Source file locations
2532
+
2533
+ Returns:
2534
+ PropertyBag: The modified property (typically the same object)
2535
+ '''
2536
+ # Add your extraction/transformation logic here
2537
+ property["my_new_field"] = extract_my_data(info)
2538
+ return property
2539
+
2540
+ TRANSFORMER DESIGN PRINCIPLES:
2541
+
2542
+ 1. SINGLE RESPONSIBILITY: Each transformer should handle one specific aspect
2543
+ of property processing (types, defaults, metadata, etc.)
2544
+
2545
+ 2. DEFENSIVE PROGRAMMING: Always check for expected data structures and
2546
+ handle missing/malformed data gracefully
2547
+
2548
+ 3. IMMUTABLE INPUTS: Never modify the 'info' dict unless explicitly designed
2549
+ to normalize it (like ParamNormalizerTransformer)
2550
+
2551
+ 4. COMPREHENSIVE LOGGING: Use @debug_transformer decorator and add detailed
2552
+ docstrings for debugging and maintenance
2553
+
2554
+ 5. DEPENDENCY AWARENESS: Understand which transformers depend on your output
2555
+ and ensure proper ordering in the pipeline
2556
+
2557
+ 6. TYPE SAFETY: Handle different parameter types robustly (strings, dicts, lists)
2558
+
2559
+ 7. PATTERN CONSISTENCY: Follow established patterns for metadata extraction,
2560
+ type mapping, and error handling
2561
+
2562
+ DEBUGGING TRANSFORMER ISSUES:
2563
+
2564
+ 1. Enable DEBUG_TRANSFORMERS = True at module level
2565
+ 2. Set DEBUG_FILTER to focus on specific properties
2566
+ 3. Use @debug_transformer decorator on new transformers
2567
+ 4. Check transformer execution order - dependencies must run first
2568
+ 5. Verify accepts() logic correctly identifies target properties
2569
+ 6. Test with various C++ property declaration patterns
2570
+
2571
+ COMMON GOTCHAS:
2572
+
2573
+ 1. Parameter ordering varies between property types (enterprise vs regular)
2574
+ 2. Tree-sitter parsing can produce different structures for similar C++ code
2575
+ 3. Meta information may be in different formats (raw strings vs parsed dicts)
2576
+ 4. Type extraction must handle arbitrarily nested template patterns
2577
+ 5. Default values may reference constants that need separate resolution
2578
+ 6. Circular imports between transformers.py and property_extractor.py
2579
+
2580
+ MAINTENANCE CHECKLIST:
2581
+
2582
+ When Redpanda C++ code patterns change:
2583
+ □ Update type extraction patterns for new C++ constructs
2584
+ □ Add new metadata keys to find_meta_dict() recognition
2585
+ □ Update enterprise property parameter patterns
2586
+ □ Test with sample properties from new Redpanda versions
2587
+ □ Update documentation examples to reflect current usage
2588
+ □ Verify backwards compatibility with existing property patterns
2589
+
2590
+ When JSON Schema requirements change:
2591
+ □ Update type mapping in TypeTransformer
2592
+ □ Add new validation constraints as transformer outputs
2593
+ □ Update downstream schema consumption in documentation generators
2594
+ □ Test schema validation with updated property definitions
2595
+
2596
+ The transformer system is designed to be extensible and maintainable. Follow
2597
+ these patterns and your additions will integrate smoothly with the existing
2598
+ pipeline while maintaining reliability and debuggability.
2599
+ """