@redpanda-data/docs-extensions-and-macros 2.5.1 → 3.0.0

package/README.adoc

@@ -1,5 +1,7 @@
 = Antora Extensions and Macros for Redpanda Docs
-:url-project: https://github.com/JakeSCahill/antora-extensions-and-macros
+:url-org: https://github.com/redpanda-data
+:url-project: {url-org}/docs-extensions-and-macros
+:url-playbook: {url-org}/docs-site
 :url-git: https://git-scm.com
 :url-git-dl: {url-git}/downloads
 :url-nodejs: https://nodejs.org
@@ -33,7 +35,7 @@ When you have Node.js installed, use the following command to install the `antor
 
 [,bash]
 ----
-npm i antora-extensions-and-macros
+npm i @redpanda-data/docs-extensions-and-macros
 ----
 
 To use the development version instead, refer to the <<development-quickstart,Development Quickstart>>.
@@ -70,7 +72,7 @@ Whether to index all versions or just the latest version of a component.
 ```yaml
 antora:
   extensions:
-  - require: 'node_modules/antora-extensions-and-macros/extensions/algolia-indexer/index.js'
+  - require: '@redpanda-data/docs-extensions-and-macros/extensions/algolia-indexer/index'
     excludes: ['.thumbs','script', '.page-versions','.feedback-section','.banner-container']
     index-latest-only: true
 ```
@@ -92,48 +94,19 @@ NOTE: If you don't set the environment variable, the latest versions may not be
 ```yaml
 antora:
   extensions:
-  - 'node_modules/antora-extensions-and-macros/extensions/version-fetcher/set-latest-version.js'
+  - '@redpanda-data/docs-extensions-and-macros/extensions/version-fetcher/set-latest-version'
 ```
 
 === Global attributes
 
-This extension fetches the content of all YAML files in a GitHub directory and merges the contents with the `asciidoc.attributes` object in the Antora playbook. This allows you to define Asciidoc attributes in an external repository and automatically include them in your documentation.
-
-[IMPORTANT]
-====
-- The GitHub directory that contains the global attributes must be named `global-attributes`.
-- Only YAML files are supported. Other types of files are ignored.
-- If a key is present in both the global attributes and the playbook's `asciidoc.attributes`, the value in the playbook takes precedence.
-====
-
-==== Environment variables
-
-- `GLOBAL_ATTRIBUTES_GITHUB_TOKEN` (optional): A Personal access token (PAT) that has `repo` permissions for the GitHub organization defined in `org`.
-
-NOTE: If you don't set the environment variable, global attributes may not be made available to your build. When the environment variable is not set, the extension sends unauthenticated requests to GitHub. Unauthenticated requests may result in hitting the API rate limit and cause GitHub to reject the request.
-
-==== Configuration options
-
-The extension accepts the following configuration options:
-
-org (required)::
-The GitHub organization that owns the repository.
-
-repo (required)::
-The name of the repository.
-
-branch (required)::
-The branch in the repository where the global attributes are located.
+This extension collects Asciidoc attributes from the {url-playbook}[`shared` component] and makes them available to all component versions. Having global attributes is useful for consistent configuration of local and production builds.
 
 ==== Registration example
 
 ```yaml
 antora:
   extensions:
-  - require: 'node_modules/antora-extensions-and-macros/extensions/add-global-attributes.js'
-    org: example
-    repo: test
-    branch: main
+  - require: '@redpanda-data/docs-extensions-and-macros/extensions/add-global-attributes'
 ```
 
 === Replace attributes in attachments
@@ -152,7 +125,28 @@ This extension replaces AsciiDoc attribute placeholders with their respective va
 ```yaml
 antora:
   extensions:
-  - 'node_modules/antora-extensions-and-macros/extensions/replace-attributes-in-attachments.js'
+  - '@redpanda-data/docs-extensions-and-macros/extensions/replace-attributes-in-attachments'
+```
+
+=== Aggregate terms
+
+This extension aggregates all term pages from the {url-playbook}[`shared` component] and does the following:
+
+- Makes all `term-name` and `hover-text` attributes available to the <<glossterm-macro,`glossterm` macro>>.
+- Looks for glossary pages named `reference:glossary.adoc` in all versions of the Redpanda docs and appends the contents of each term file to the glossary in alphabetical order.
+- If a glossary page is found, sets the `glossary-page` attribute of the `glossterm` macro to `reference:glossary.adoc`.
+
+[IMPORTANT]
+====
+By default, this extension processes glossary pages for the `ROOT` (redpanda) component only. This behavior is hardcoded and cannot be changed in the configuration.
+====
+
+==== Registration example
+
+```yaml
+antora:
+  extensions:
+  - '@redpanda-data/docs-extensions-and-macros/extensions/aggregate-terms'
 ```
 
 === Unlisted pages
@@ -176,7 +170,7 @@ The heading under which to list the unlisted pages in the navigation. The defaul
 ```yaml
 antora:
   extensions:
-  - require: 'node_modules/antora-extensions-and-macros/extensions/unlisted-pages.js'
+  - require: '@redpanda-data/docs-extensions-and-macros/extensions/unlisted-pages'
     addToNavigation: true
     unlistedPagesHeading: 'Additional Resources'
 ```
@@ -228,33 +222,22 @@ config_ref:example_config,true,tunable-properties[]
 ----
 asciidoc:
   extensions:
-    - 'node_modules/antora-extensions-and-macros/macros/config-ref.js'
+    - '@redpanda-data/docs-extensions-and-macros/macros/config-ref'
 ----
 
-=== glossary and glossterm
+=== glossterm
 
-The glossary module provides a way to define and reference glossary terms in your AsciiDoc documents.
+The `glossterm` inline macro provides a way to define and reference glossary terms in your AsciiDoc documents.
 
-This module consists of two parts: a block macro (`glossary`) and an inline macro (`glossterm`).
-
-NOTE: This macro is a customized version of https://gitlab.com/djencks/asciidoctor-glossary[`asciidoctor-glossary`]. We added the ability to define terms globally in <<global-attributes, global attributes>> as well as link to external URLs.
+NOTE: This macro is a customized version of https://gitlab.com/djencks/asciidoctor-glossary[`asciidoctor-glossary`].
 
 ==== Usage
 
-To insert a glossary dlist, use the glossary block macro.
-
-[,asciidoc]
-----
-glossary::[]
-----
-
-Glossary terms defined in the `glossterm` inline macro before the `glossary` macro is used appear as a definition list, sorted by term.
-
-The `glossterm` inline macro is used to reference a term within the text of the document:
+Use the `glossterm` inline macro to reference a term within the text of the document:
 
 [,asciidoc]
 ----
-glossterm:myTerm[myDefinition]
+glossterm:my term[myDefinition]
 ----
 
 It takes two parameters:
@@ -263,7 +246,7 @@ term::
 The term to be defined.
 
 definition (optional)::
-The definition of the term. If the term is defined in the <<global-attributes, global attributes>>, you can omit the definition as it will always be replaced by the definition in the global attributes.
+The definition of the term. If the term is defined in the {url-playbook}[`shared` component] or the `local-terms` object of the `antora.yml` file, you can omit the definition as it will always be replaced by those definitions.
 
 ==== Configuration options
 
@@ -290,22 +273,13 @@ Whether to enable tooltips for the defined terms. Valid values are:
 
 The last two options are intended to support js/css tooltip solutions such as tippy.js.
 
-[IMPORTANT]
-.Multi-page use
-====
-In Antora, a glossary is constructed for each component-version.
-When the `glossary` block macro is evaluated, only terms known as of the rendering can be included.
-Therefore, it is necessary that the page containing this macro in a component-version be rendered last.
-It may be possible to arrange this by naming the page starting with a lot of 'z’s, such as `zzzzzz-glossary.adoc`.
-====
-
 ==== Registration example
 
 [,yaml]
 ----
 asciidoc:
   extensions:
-    - 'node_modules/antora-extensions-and-macros/macros/glossary.js'
+    - '@redpanda-data/docs-extensions-and-macros/macros/glossary'
 ----
 
 === helm_ref
@@ -347,7 +321,7 @@ If you do not specify a Helm reference value, the macro generates a link without
 ----
 asciidoc:
   extensions:
-    - 'node_modules/antora-extensions-and-macros/macros/helm-ref.js'
+    - '@redpanda-data/docs-extensions-and-macros/macros/helm-ref'
 ----
 
 == Development quickstart
@@ -418,5 +392,5 @@ If you want to use the project locally before it is published, you can specify t
 asciidoc:
   attributes:
   extensions:
-  - '<path-to-local-project>/antora-extensions-and-macros/extensions/<extension-name>'
-----
+  - '<path-to-local-project>/docs-extensions-and-macros/extensions/<extension-name>'
+----

package/extensions/add-global-attributes.js

@@ -2,76 +2,43 @@
 * antora:
     extensions:
  *    - require: ./extensions/add-global-attributes.js
-       org: example
-       repo: test
-       branch: main
 */
 
-const yaml = require('js-yaml');
-const { Octokit } = require("@octokit/rest");
-const { retry } = require("@octokit/plugin-retry");
-const OctokitWithRetries = Octokit.plugin(retry);
-const chalk = require('chalk')
-
-
 module.exports.register = function ({ config }) {
-  const logger = this.getLogger('global-attributes-extension')
-
-  this
-    .on('playbookBuilt', async ({ playbook }) => {
-
-      const globalAttributesUrl = `/repos/${config.org}/${config.repo}/contents/global-attributes?ref=${config.branch}`;
-
-      let githubOptions = {
-        userAgent: 'Redpanda Docs',
-        baseUrl: 'https://api.github.com',
-      };
-
-      if(process.env.GLOBAL_ATTRIBUTES_GITHUB_TOKEN){
-        githubOptions.auth = process.env.GLOBAL_ATTRIBUTES_GITHUB_TOKEN;
-      } else {
-        logger.warn('GLOBAL_ATTRIBUTES_GITHUB_TOKEN environment variable not set. Attempting unauthenticated request.')
-      }
-
-      const github = new OctokitWithRetries(githubOptions);
-
-      try {
-        // Request the contents of the directory from the GitHub API
-        const response = await github.request('GET ' + globalAttributesUrl);
-
-        if (response.status === 200) {
-          const directoryContents = response.data;
-
-          // Filter out only YAML files
-          const yamlFiles = directoryContents.filter(file => file.name.endsWith('.yaml') || file.name.endsWith('.yml'));
-
-          let globalAttributes = {};
-          for (let file of yamlFiles) {
-            const fileResponse = await github.request('GET ' + file.download_url);
-            const fileData = yaml.load(fileResponse.data);
-            globalAttributes = {...globalAttributes, ...fileData};
+  const yaml = require('js-yaml');
+  const _ = require('lodash');
+  const logger = this.getLogger('global-attributes-extension');
+  const chalk = require('chalk')
+
+  this.on('contentAggregated', ({ siteCatalog, contentAggregate }) => {
+    let attributeFile;
+    try {
+      for (const component of contentAggregate) {
+        if (component.name === 'shared') {
+          attributeFile = component.files.find(file => file.path.includes('attributes.yml'));
+          if (!attributeFile) {
+            logger.warn("No attributes.yml file found in 'shared' component.");
+          } else {
+            siteCatalog.attributeFile = yaml.load(attributeFile.contents.toString('utf8'));
+            console.log(chalk.green('Loaded attributes from shared component.'));
           }
-
-          let mergedAttributes = {
-            ...globalAttributes,
-            ...playbook.asciidoc.attributes
-          };
-
-          playbook.asciidoc.attributes = mergedAttributes;
-
-          console.log(chalk.green('Merged global attributes into playbook'));
-
-        } else {
-          logger.warn(`Could not fetch global attributes: ${response.statusText}`);
-          return null
-        }
-      } catch(error) {
-        if (error.response.url) {
-          logger.warn(error.status + ' Could not get ' + error.response.url)
-        }
-        else {
-          logger.warn(error)
+          break
         }
       }
-    })
-}
+    } catch (error) {
+      logger.error(`Error loading attributes: ${error.message}`);
+    }
+  })
+
+  .on('contentClassified', ({ siteCatalog, contentCatalog }) => {
+    const components = contentCatalog.getComponents()
+    try {
+      components.forEach(({asciidoc}) => {
+        asciidoc.attributes = _.merge(asciidoc.attributes, siteCatalog.attributeFile);
+      });
+      console.log(chalk.green('Merged global attributes into each component.'));
+    } catch (error) {
+      logger.error(`Error merging attributes: ${error.message}`);
+    }
+  });
+}

package/extensions/aggregate-terms.js

@@ -0,0 +1,60 @@
+/* Example use in the playbook
+* antora:
+    extensions:
+ *    - require: ./extensions/aggregate-terms.js
+*/
+
+module.exports.register = function ({ config }) {
+  const logger = this.getLogger('term-aggregation-extension');
+  const chalk = require('chalk')
+
+  this.on('contentAggregated', ({ siteCatalog, contentAggregate }) => {
+    try {
+      for (const component of contentAggregate) {
+        if (component.name === 'shared') {
+          const termFiles = component.files.filter(file => file.path.includes('modules/terms/partials/'));
+          siteCatalog.terms = {}
+          termFiles.forEach(file => {
+            const termContent = file.contents.toString('utf8');
+            siteCatalog.terms[file.basename] = termContent;
+          });
+          console.log(chalk.green('Loaded terms from shared component.'));
+          break
+        }
+      }
+    } catch (error) {
+      logger.error(`Error loading terms: ${error.message}`);
+    }
+  })
+
+  .on('contentClassified', ({ siteCatalog, contentCatalog }) => {
+    const components = contentCatalog.getComponents()
+    try {
+      for (const { versions } of components) {
+        for (const { name: component, version, asciidoc, title } of versions) {
+          // TODO: will need a better way to filter when we have more content components
+          if (component !== 'ROOT') continue;
+          const glossaryPage = contentCatalog.resolvePage(`${version?version +'@':''}${component}:reference:glossary.adoc`);
+          if (glossaryPage) {
+            asciidoc.attributes['glossary-page'] = 'reference:glossary.adoc'
+            const glossaryContent = glossaryPage.contents.toString('utf8');
+            let newContent = glossaryContent;
+            const sortedTerms = Object.keys(siteCatalog.terms).sort((a, b) => a.toLowerCase().localeCompare(b.toLowerCase()));
+            for (i = 0; i < sortedTerms.length; i++) {
+              const termName = sortedTerms[i];
+              const termContent = siteCatalog.terms[termName];
+              newContent += '\n\n' + termContent;
+            }
+            glossaryPage.contents = Buffer.from(newContent, 'utf8');
+
+            console.log(chalk.green(`Transposed terms into glossary for ${component}.`));
+          } else {
+            logger.warn(`Skipping ${title} ${version} - No glossary page (reference:glossary.adoc) found`)
+          }
+        }
+      }
+    } catch (error) {
+      logger.error(`Error transposing terms: ${error.message}`);
+    }
+  });
+}

package/extensions/replace-attributes-in-attachments.js

@@ -7,6 +7,7 @@ module.exports.register = function ({ config }) {
   this.on('documentsConverted', ({playbook, contentCatalog}) => {
     for (const { versions } of contentCatalog.getComponents()) {
       for (const { name: component, version } of versions) {
+        // TODO: will need a better way to filter when we have more content components
         if (component !== 'ROOT') continue;
         const attachments = contentCatalog.findBy({ component, version, family });
         for (const attachment of attachments) {

package/macros/glossary.js

@@ -12,16 +12,47 @@ module.exports.register = function (registry, config = {}) {
     function getKey (src) {
       return `${src.version}@${src.component}`
     }
-
     const contentCatalog = config.contentCatalog
     if (!contentCatalog[$glossaryContexts]) contentCatalog[$glossaryContexts] = {}
     const glossaryContexts = contentCatalog[$glossaryContexts]
+    // Check if the terms have been cached
+    const sharedKey = 'sharedTerms'
+    if (!glossaryContexts[sharedKey]) {
+      // Get the term files from the 'shared' component
+      const termFiles = contentCatalog.findBy({ component: 'shared', module: 'terms', family: 'partial' })
+      // Extract the term definitions from the files
+      const ATTRIBUTE_REGEX = /^:([a-zA-Z0-9_-]+):[ \t]*(.*)$/gm
+
+      const terms = termFiles.map(file => {
+        const content = file.contents.toString()
+        const attributes = {}
+
+        let match
+        while ((match = ATTRIBUTE_REGEX.exec(content)) !== null) {
+          const [ , name, value ] = match
+          attributes[name] = value
+        }
+
+        if (!attributes['term-name'] || !attributes['hover-text']) {
+          console.warn(`Skipping file ${file.path} due to missing 'term-name' and/or 'hover-text'.`)
+          return null
+        }
+
+        return {
+          term: attributes['term-name'],
+          def: attributes['hover-text'],
+          content
+        }
+      }).filter(Boolean)
+
+      // Store the terms in the cache
+      glossaryContexts[sharedKey] = terms
+    }
     const key = getKey(config.file.src)
     if (!glossaryContexts[key]) {
       glossaryContexts[key] = {
-        gloss: [],
+        gloss: glossaryContexts[sharedKey],
         self: undefined,
-        dlist: undefined,
       }
     }
     const context = glossaryContexts[key]
@@ -34,35 +65,7 @@ module.exports.register = function (registry, config = {}) {
   const IDRX = /[/ _.-]+/g
 
   function termId (term) {
-    return '_glossterm_' + term.toLowerCase().replace(IDRX, '_')
-  }
-
-  function dlistItem (context, term, def) {
-    const id = termId(term)
-    term = `anchor:${id}[${term}]${term}`
-    const termItem = context.self.createListItem(context.dlist, term)
-    const defItem = context.self.createListItem(context.dlist, def)
-    return [[termItem], defItem]
-  }
-
-  function glossaryBlockMacro () {
-    return function () {
-      const self = this
-      self.named('glossary')
-      self.$option('format', 'short') //no target between glossary:: and [params]
-      // self.positionalAttributes(['name', 'parameters'])
-      self.process(function (parent, target, attributes) {
-        const context = vfs.getContext()
-        const dlist = self.createList(parent, 'dlist')
-        context.self = self
-        context.dlist = dlist
-        context.gloss
-          .forEach(({ term, def }) => {
-            dlist.blocks.push(dlistItem(context, term, def))
-          })
-        return dlist
-      })
-    }
+    return term.toLowerCase().replace(IDRX, '-')
   }
 
   const TRX = /(<[a-z]+)([^>]*>.*)/
@@ -78,23 +81,9 @@ module.exports.register = function (registry, config = {}) {
         const term = attributes.term || target
         const document = parent.document
         const context = vfs.getContext()
-        // See if a predefined list of terms is available
-        const globalTerms = document.getAttribute("terms") || [];
         const localTerms = document.getAttribute("local-terms") || [];
-        let mergedTerms;
-        if(globalTerms.length > 0 && localTerms.length > 0){
-            // Convert globalTerms to a Map for fast look up
-            let globalTermsMap = new Map(globalTerms.map(i => [i.term, i]));
-
-            // Create a new array based on localTerms, but replace items if they exist in globalTerms
-            mergedTerms = localTerms.map(item => globalTermsMap.has(item.term) ? globalTermsMap.get(item.term) : item);
-
-            // Update localTerms with the mergedTerms
-            document.setAttribute("terms", mergedTerms);
-            console.log(chalk.green('Merged global terms with local terms'))
-        }
-        const termData = (mergedTerms || []).find((t) => t.term === term) || {};
-        const customLink = termData.link;
+        const localTermData = (localTerms || []).find((t) => t.term === term) || {};
+        const customLink = localTermData.link;
         var tooltip = document.getAttribute('glossary-tooltip')
         if (tooltip === 'true') tooltip = 'data-glossary-tooltip'
         if (tooltip && tooltip !== 'title' && !tooltip.startsWith('data-')) {
@@ -102,13 +91,17 @@ module.exports.register = function (registry, config = {}) {
           tooltip = undefined
         }
         const logTerms = document.hasAttribute('glossary-log-terms')
-        var definition = termData.definition || attributes.definition
+        var definition;
+        const index = context.gloss.findIndex((candidate) => candidate.term === term)
+        if (index >= 0) {
+          definition = context.gloss[index].def
+        } else {
+          definition = localTermData.definition || attributes.definition;
+        }
         if (definition) {
           logTerms && console.log(`${term}::  ${definition}`)
-          addItem(context, term, definition)
         } else if (tooltip) {
-          const index = context.gloss.findIndex((candidate) => candidate.term === term)
-          definition = ~index ? context.gloss[index].def : `${term} not yet defined`
+          definition = `${term} not yet defined`
         }
         const links = document.getAttribute('glossary-links', 'true') === 'true'
         var glossaryPage = document.getAttribute('glossary-page', '')
@@ -120,11 +113,15 @@ module.exports.register = function (registry, config = {}) {
         }
         const glossaryTermRole = document.getAttribute('glossary-term-role', 'glossary-term')
         const attrs = glossaryTermRole ? { role: glossaryTermRole } : {}
-        const inline = links
-          ? customLink
+        var inline;
+        const termExistsInContext = context.gloss.some((candidate) => candidate.term === term);
+        if (termExistsInContext && links) {
+          inline = customLink
             ? self.createInline(parent, 'anchor', target, { type: 'link', target: customLink, attributes: attrs })
             : self.createInline(parent, 'anchor', target, { type: 'xref', target: `${glossaryPage}#${termId(term)}`, reftext: target, attributes: attrs })
-          : self.createInline(parent, 'quoted', target, { attributes: attrs })
+        } else {
+          inline = self.createInline(parent, 'quoted', target, { attributes: attrs })
+        }
         if (tooltip) {
           const a = inline.convert()
           const matches = a.match(TRX)
@@ -139,27 +136,7 @@ module.exports.register = function (registry, config = {}) {
     }
   }
 
-  function addItem (context, term, def) {
-    let i = 0
-    let comp = -1
-    for (; i < context.gloss.length && ((comp = term.localeCompare(context.gloss[i].term)) > 0); i++) {
-    }
-    if (comp < 0) {
-      context.gloss.splice(i, 0, { term, def })
-      if (context.self && context.dlist) {
-        context.dlist.blocks.splice(i, 0, dlistItem(context, term, def))
-      }
-    } else {
-      console.log(`duplicate glossary term ${term}`)
-    }
-  }
-
   function doRegister (registry) {
-    if (typeof registry.blockMacro === 'function') {
-      registry.blockMacro(glossaryBlockMacro())
-    } else {
-      console.warn('no \'blockMacro\' method on alleged registry')
-    }
     if (typeof registry.inlineMacro === 'function') {
       registry.inlineMacro(glossaryInlineMacro())
     } else {

package/package.json

@@ -1,8 +1,14 @@
 {
   "name": "@redpanda-data/docs-extensions-and-macros",
-  "version": "2.5.1",
+  "version": "3.0.0",
   "description": "Antora extensions and macros developed for Redpanda documentation.",
-  "keywords": ["antora", "extension", "macro", "documentation", "redpanda"],
+  "keywords": [
+    "antora",
+    "extension",
+    "macro",
+    "documentation",
+    "redpanda"
+  ],
   "author": {
     "name": "Redpanda Docs Team"
   },
@@ -20,6 +26,7 @@
     "./extensions/add-global-attributes": "./extensions/add-global-attributes.js",
     "./extensions/version-fetcher/set-latest-version": "./extensions/version-fetcher/set-latest-version.js",
     "./extensions/algolia-indexer/index": "./extensions/algolia-indexer/index.js",
+    "./extensions/aggregate-terms": "./extensions/aggregate-terms.js",
     "./macros/glossary": "./macros/glossary.js",
     "./macros/config-ref": "./macros/config-ref.js",
     "./macros/helm-ref": "./macros/helm-ref.js"
@@ -39,10 +46,11 @@
     "@octokit/rest": "^19.0.7",
     "algoliasearch": "^4.17.0",
     "chalk": "4.1.2",
-    "js-yaml": "^4.1.0",
     "gulp": "^4.0.2",
     "gulp-connect": "^5.7.0",
     "html-entities": "2.3",
+    "js-yaml": "^4.1.0",
+    "lodash": "^4.17.21",
     "node-html-parser": "5.4.2-0"
   }
 }