@redpanda-data/docs-extensions-and-macros 4.12.5 → 4.13.0

package/macros/USER_GUIDE.adoc

@@ -0,0 +1,220 @@
+= AsciiDoc Macros User Guide
+:toc:
+:toclevels: 3
+
+Complete guide to using AsciiDoc macros in your documentation.
+
+== Getting started
+
+=== Installation
+
+Install the package:
+
+[,bash]
+----
+npm i @redpanda-data/docs-extensions-and-macros
+----
+
+=== Configuration
+
+Register macros in your Antora playbook under `asciidoc.extensions`:
+
+[,yaml]
+----
+asciidoc:
+  extensions:
+    - '@redpanda-data/docs-extensions-and-macros/macros/glossary'
+    - '@redpanda-data/docs-extensions-and-macros/macros/config-ref'
+    - '@redpanda-data/docs-extensions-and-macros/macros/helm-ref'
+----
+
+IMPORTANT: Use `asciidoc.extensions` for macros, not `antora.extensions`.
+
+== Using macros
+
+=== glossterm - Glossary terms
+
+Reference glossary terms with tooltips:
+
+[,asciidoc]
+----
+The glossterm:partition[partition] stores data.
+glossterm:broker[broker,A server that handles requests]
+----
+
+*Syntax:* `glossterm:term[definition]`
+
+* `term` - The term to reference (required)
+* `definition` - Custom definition (optional, uses global glossary by default)
+
+=== config_ref - Configuration references
+
+Link to configuration properties:
+
+[,asciidoc]
+----
+config_ref:log.retention.ms,true,tunable-properties[]
+config_ref:cleanup.policy,false[]
+----
+
+*Syntax:* `config_ref:property,isLink,path[]`
+
+* `property` - Configuration property name (required)
+* `isLink` - Whether to create a link (true/false, required)
+* `path` - Document path for link (required if isLink is true)
+
+=== helm_ref - Helm value references
+
+Link to Helm chart values:
+
+[,asciidoc]
+----
+helm_ref:storage.tieredConfig.cloud_storage_enabled[]
+helm_ref:[]
+----
+
+*Syntax:* `helm_ref:value[]`
+
+* `value` - Helm value path (optional, links to values.yaml overview if omitted)
+
+=== components_by_category - Component display
+
+Display Redpanda Connect components by category:
+
+[,asciidoc]
+----
+components_by_category::inputs[]
+components_by_category::outputs[]
+components_by_category::processors[]
+----
+
+*Syntax:* `components_by_category::[type]`
+
+* `type` - Component type (inputs, outputs, processors, etc.)
+
+=== component_table - Component table
+
+Generate searchable component table:
+
+[,asciidoc]
+----
+component_table::[]
+----
+
+Generates a searchable table of all components with filters.
+
+=== component_type_dropdown - Type switcher
+
+Create dropdown for switching between component types:
+
+[,asciidoc]
+----
+component_type_dropdown::[]
+----
+
+Generates a dropdown to switch between different implementations of the same component.
+
+== Configuration options
+
+=== Glossary configuration
+
+Configure glossary behavior in playbook:
+
+[,yaml]
+----
+asciidoc:
+  attributes:
+    glossary-tooltip: data-tooltip  # Enable tooltips
+    glossary-page: glossary.adoc    # Link target
+    glossary-links: true             # Enable links
+  extensions:
+    - '@redpanda-data/docs-extensions-and-macros/macros/glossary'
+----
+
+*Options:*
+
+* `glossary-tooltip` - Tooltip implementation (title, true, or data-attribute-name)
+* `glossary-page` - Page to link glossary terms to
+* `glossary-links` - Enable/disable links (default: true)
+* `glossary-term-role` - CSS role for terms (default: glossary-term)
+
+== Troubleshooting
+
+=== Macro not rendering
+
+*Problem:* Macro appears as plain text
+
+*Solutions:*
+
+* Verify macro is registered under `asciidoc.extensions`
+* Check macro syntax is correct
+* Ensure package is installed
+* Try rebuilding: `npx antora playbook.yml`
+
+=== Glossary terms not found
+
+*Problem:* Glossary definitions missing
+
+*Solutions:*
+
+* Define terms in shared component or local `antora.yml`
+* Check term spelling matches exactly
+* Provide inline definition as fallback
+
+=== Links not working
+
+*Problem:* config_ref or helm_ref links broken
+
+*Solutions:*
+
+* Verify target page exists
+* Check path is relative to reference module
+* Ensure property/value name is correct
+
+=== Component macros empty
+
+*Problem:* components_by_category shows no components
+
+*Solutions:*
+
+* Ensure component aggregator extension is enabled
+* Check component data is generated
+* Verify component type is correct
+
+== Best practices
+
+=== Use glossary terms consistently
+
+[,asciidoc]
+----
+✓ Good: glossterm:partition[]
+✗ Bad:  partition (without macro)
+----
+
+=== Provide fallback definitions
+
+[,asciidoc]
+----
+glossterm:custom-term[A definition in case term isn't in glossary]
+----
+
+=== Link to configuration when relevant
+
+[,asciidoc]
+----
+To configure retention, set config_ref:log.retention.ms,true,tunable-properties[].
+----
+
+=== Keep macro usage readable
+
+[,asciidoc]
+----
+✓ Good: Configure config_ref:retention.ms,true,topic-properties[].
+✗ Bad:  config_ref:retention.ms,true,topic-properties[]config_ref:cleanup.policy,true,topic-properties[].
+----
+
+== Related documentation
+
+* link:REFERENCE.adoc[Macros reference] - Complete macro documentation
+* link:DEVELOPMENT.adoc[Development guide] - How to create new macros
+* https://docs.asciidoctor.org/asciidoc/latest/macros/[AsciiDoc Macro Syntax]

package/macros/rp-connect-components.js

@@ -273,10 +273,10 @@ module.exports.register = function (registry, context) {
    *
    * @returns {Object} - An object with two properties:
    *   - `certified`: An array of SQL drivers with 'certified' support level. Each driver contains:
-   *     - `commercialName`: The trimmed commercial name of the driver (e.g., 'PostgreSQL').
+   *     - `commercialName`: The trimmed commercial name of the driver (for example, 'PostgreSQL').
    *     - `isCloudSupported`: A boolean indicating whether the driver supports cloud.
    *   - `community`: An array of SQL drivers with 'community' support level. Each driver contains:
-   *     - `commercialName`: The trimmed commercial name of the driver (e.g., 'Trino').
+   *     - `commercialName`: The trimmed commercial name of the driver (for example, 'Trino').
    *     - `isCloudSupported`: A boolean indicating whether the driver supports cloud.
    *
    * Example return structure:
@@ -330,7 +330,7 @@ module.exports.register = function (registry, context) {
    * @param {Object} connectors - An object containing the connector data, where each key is a connector name and
    *   each value contains details about its types, licensing, and cloud support.
    *   {
-   *     types: Map - A map of connector types (e.g., Input, Output, Processor), with associated commercial names.
+   *     types: Map - A map of connector types (for example, Input, Output, Processor), with associated commercial names.
    *     isLicensed: 'Yes' or 'No' - Indicates if the connector requires an enterprise license.
    *     isCloudConnectorSupported: true or false - Indicates if any type for this connector supports Redpanda Cloud.
    *   }
@@ -340,7 +340,7 @@ module.exports.register = function (registry, context) {
    *     community: Array<{ commercialName: string, isCloudSupported: boolean }>
    *   }
    * @param {boolean} isCloud - A flag indicating whether to filter by cloud support. If true, only cloud-supported connectors are shown.
-   * @param {boolean} showAllInfo - A flag indicating whether to show all information or limit the data displayed (e.g., for cloud-only views).
+   * @param {boolean} showAllInfo - A flag indicating whether to show all information or limit the data displayed (for example, for cloud-only views).
    *
    * @returns {string} - A string containing the generated HTML for the connectors table rows.
    *   The output is a string of HTML rows with the following columns:
@@ -831,7 +831,7 @@ module.exports.register = function (registry, context) {
               // Generic fallback for words ending in 's Selected'
               return pluralText.replace(/s Selected$/, ' Selected');
             } else if (pluralText.endsWith('ies Selected')) {
-              // Handle words ending in 'ies' (e.g., "Categories Selected" -> "Category Selected")
+              // Handle words ending in 'ies' (for example, "Categories Selected" -> "Category Selected")
               return pluralText.replace(/ies Selected$/, 'y Selected');
             } else {
               // If no pattern matches, return as-is
@@ -984,7 +984,7 @@ module.exports.register = function (registry, context) {
     self.named('component_type_dropdown');
     self.process((parent, target, attrs) => {
       const attributes = parent.getDocument().getAttributes();
-      const component = attributes['page-component-title'];  // Current component (e.g., 'Redpanda Cloud' or 'Redpanda Connect')
+      const component = attributes['page-component-title'];  // Current component (for example, 'Redpanda Cloud' or 'Redpanda Connect')
       const name = attributes['doctitle'];
       const type = attributes['type'];
       if (!name || !type) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@redpanda-data/docs-extensions-and-macros",
-  "version": "4.12.5",
+  "version": "4.13.0",
   "description": "Antora extensions and macros developed for Redpanda documentation.",
   "keywords": [
     "antora",
@@ -13,7 +13,8 @@
     "name": "Redpanda Docs Team"
   },
   "bin": {
-    "doc-tools": "./bin/doc-tools.js"
+    "doc-tools": "./bin/doc-tools.js",
+    "doc-tools-mcp": "./bin/doc-tools-mcp.js"
   },
   "scripts": {
     "install-test-dependencies": "doc-tools install-test-dependencies",
@@ -22,8 +23,12 @@
     "build": "antora --to-dir docs --fetch local-antora-playbook.yml",
     "serve": "wds --node-resolve --open preview/test/ --watch --root-dir docs",
     "test": "jest",
+    "test:mcp": "jest __tests__/mcp/",
+    "test:mcp:integration": "jest __tests__/mcp/integration.test.js",
+    "test:mcp:contract": "jest __tests__/mcp/cli-contract.test.js",
     "test:python": "./__tests__/tools/property-extractor/setup-and-test.sh",
     "test:all": "npm run test && npm run test:python",
+    "generate:cli-docs": "node tools/generate-cli-docs.js",
     "bundle:admin": "doc-tools generate bundle-openapi --surface admin",
     "bundle:connect": "doc-tools generate bundle-openapi --surface connect",
     "bundle:both": "doc-tools generate bundle-openapi --surface both"
@@ -85,14 +90,17 @@
   "dependencies": {
     "@asciidoctor/tabs": "^1.0.0-beta.6",
     "@bufbuild/buf": "^1.28.1",
+    "@modelcontextprotocol/sdk": "^1.0.4",
     "@octokit/core": "^6.1.2",
     "@octokit/plugin-retry": "^7.1.1",
     "@octokit/rest": "^21.0.1",
     "@redocly/cli": "^2.2.0",
+    "ajv": "^8.12.0",
     "algoliasearch": "^4.17.0",
     "chalk": "4.1.2",
     "cheerio": "^1.1.2",
     "commander": "^14.0.0",
+    "glob": "^11.0.0",
     "gulp": "^4.0.2",
     "gulp-connect": "^5.7.0",
     "handlebars": "^4.7.8",
@@ -117,6 +125,7 @@
     "@antora/cli": "3.1.4",
     "@antora/site-generator": "3.1.4",
     "@web/dev-server": "^0.2.5",
-    "jest": "^29.7.0"
+    "jest": "^29.7.0",
+    "jest-junit": "^16.0.0"
   }
 }

package/tools/bundle-openapi.js

@@ -12,8 +12,8 @@ const yaml = require('yaml');
  * and validates that the result matches MAJOR.MINOR.PATCH with optional pre-release/build metadata.
  * Throws if the input is not a non-empty string or does not conform to the expected version format.
  *
- * @param {string} tag - Git tag (e.g., 'v25.1.1', '25.1.1', or 'dev').
- * @returns {string} Normalized version (e.g., '25.1.1' or 'dev').
+ * @param {string} tag - Git tag (for example, 'v25.1.1', '25.1.1', or 'dev').
+ * @returns {string} Normalized version (for example, '25.1.1' or 'dev').
  * @throws {Error} If `tag` is not a non-empty string or does not match the semantic version pattern.
  */
 function normalizeTag(tag) {
@@ -44,8 +44,8 @@ function normalizeTag(tag) {
  *
  * Accepts a semantic version like `25.1.1` and yields `25.1`. The special value
  * `'dev'` is returned unchanged.
- * @param {string} version - Semantic version (e.g., `'25.1.1'`) or `'dev'`.
- * @returns {string} The `major.minor` string (e.g., `'25.1'`) or `'dev'`.
+ * @param {string} version - Semantic version (for example, `'25.1.1'`) or `'dev'`.
+ * @returns {string} The `major.minor` string (for example, `'25.1'`) or `'dev'`.
  * @throws {Error} If `version` is not a non-empty string, lacks major/minor parts, or if major/minor are not numeric.
  */
 function getMajorMinor(version) {
@@ -554,13 +554,13 @@ function postProcessBundle(filePath, options, quiet = false) {
  * Bundle OpenAPI fragments for the specified API surface(s) from a repository tag and write the resulting bundled YAML files to disk.
  *
  * @param {Object} options - Configuration options.
- * @param {string} options.tag - Git tag to checkout (e.g., 'v25.1.1').
+ * @param {string} options.tag - Git tag to checkout (for example, 'v25.1.1').
  * @param {'admin'|'connect'|'both'} options.surface - API surface to process.
  * @param {string} [options.output] - Standalone output file path; when provided, used for the single output file.
  * @param {string} [options.outAdmin] - Output path for the admin API when integrating with doc-tools mode.
  * @param {string} [options.outConnect] - Output path for the connect API when integrating with doc-tools mode.
  * @param {string} [options.repo] - Repository URL to clone (defaults to https://github.com/redpanda-data/redpanda.git).
- * @param {string} [options.adminMajor] - Admin API major version string used for metadata (e.g., 'v2.0.0').
+ * @param {string} [options.adminMajor] - Admin API major version string used for metadata (for example, 'v2.0.0').
  * @param {boolean} [options.useAdminMajorVersion] - When true and processing the admin surface, use `adminMajor` for the bundle info.version.
  * @param {boolean} [options.quiet=false] - Suppress logging to stdout/stderr when true.
  * @returns {Object|Object[]} An object (for a single surface) or an array of objects (for both surfaces) with fields:
@@ -625,9 +625,19 @@ async function bundleOpenAPI(options) {
     if (!quiet) {
       console.log('📥 Cloning redpanda repository...');
     }
-    
-    const repositoryUrl = repo || 'https://github.com/redpanda-data/redpanda.git';
-    
+
+    const { getAuthenticatedGitHubUrl, hasGitHubToken } = require('../cli-utils/github-token');
+
+    let repositoryUrl = repo || 'https://github.com/redpanda-data/redpanda.git';
+
+    // Use token if available for better rate limits and reliability
+    if (hasGitHubToken() && repositoryUrl.includes('github.com')) {
+      repositoryUrl = getAuthenticatedGitHubUrl(repositoryUrl);
+      if (!quiet) {
+        console.log('🔑 Using authenticated clone (token provided)');
+      }
+    }
+
     try {
       execSync(`git clone --depth 1 --branch ${tag} ${repositoryUrl} redpanda`, {
         cwd: tempDir,
@@ -778,7 +788,7 @@ if (require.main === module) {
   program
     .name('bundle-openapi')
     .description('Bundle OpenAPI fragments from Redpanda repository')
-    .requiredOption('-t, --tag <tag>', 'Git tag to checkout (e.g., v25.1.1)')
+    .requiredOption('-t, --tag <tag>', 'Git tag to checkout (for example, v25.1.1)')
     .requiredOption('-s, --surface <surface>', 'API surface', (value) => {
       if (!['admin', 'connect', 'both'].includes(value)) {
         throw new Error('Invalid API surface. Must be "admin", "connect", or "both"');

package/tools/cloud-regions/generate-cloud-regions.js

@@ -203,7 +203,7 @@ function processCloudRegions(yamlText) {
  * @param {string} options.repo - GitHub repository name.
  * @param {string} options.path - Path to the YAML file within the repository.
  * @param {string} [options.ref='main'] - Git reference (branch, tag, or commit SHA).
- * @param {string} [options.format='md'] - The output format (e.g., 'md' for Markdown).
+ * @param {string} [options.format='md'] - The output format (for example, 'md' for Markdown).
  * @param {string} [options.token] - Optional GitHub token for authentication.
  * @param {string} [options.template] - Optional path to custom Handlebars template.
  * @returns {string} The rendered cloud regions output.

package/tools/fetch-from-github.js

@@ -27,20 +27,34 @@ async function saveFile(content, saveDir, filename) {
   console.log(`Saved: ${target}`);
 }
 
-async function fetchFromGithub(owner, repo, remotePath, saveDir, customFilename) {
+/**
+ * Fetch file or directory from GitHub repository
+ * @param {string} owner - Repository owner (for example, "redpanda-data")
+ * @param {string} repo - Repository name (for example, "redpanda-operator")
+ * @param {string} remotePath - Path to file or directory in repository
+ * @param {string} saveDir - Local directory to save files to
+ * @param {string} [customFilename] - Optional custom filename (for single files)
+ * @param {string} [ref=null] - Git ref (branch, tag, or commit SHA). Defaults to repository's default branch if null
+ * @returns {Promise<void>}
+ */
+async function fetchFromGithub(owner, repo, remotePath, saveDir, customFilename, ref = null) {
   const octokit = await loadOctokit();
 
   try {
-    const resp = await octokit.repos.getContent({ owner, repo, path: remotePath });
+    const params = { owner, repo, path: remotePath };
+    if (ref) {
+      params.ref = ref;
+    }
+    const resp = await octokit.repos.getContent(params);
     if (Array.isArray(resp.data)) {
       // directory
       for (const item of resp.data) {
         if (item.type === 'file') {
-          await fetchFromGithub(owner, repo, item.path, saveDir, customFilename);
+          await fetchFromGithub(owner, repo, item.path, saveDir, customFilename, ref);
         } else if (item.type === 'dir') {
           // For directories, maintain the directory structure
           const nestedDir = path.join(saveDir, path.basename(item.path));
-          await fetchFromGithub(owner, repo, item.path, nestedDir);
+          await fetchFromGithub(owner, repo, item.path, nestedDir, null, ref);
         }
       }
     } else {

package/tools/gen-rpk-ascii.py

@@ -18,7 +18,9 @@ suggestedReadings = """
 
 
 cmd_dict = {}
-basic_commands_docker = ["docker","exec","-it"]
+# Use 'docker exec' without -it flags for non-interactive/background execution
+# -it flags require a TTY and fail in CI/background jobs
+basic_commands_docker = ["docker", "exec"]
 basic_commands_docker.append("redpanda-1")
 rpk_basic_command = ["rpk"]