@git.zone/tsdoc 1.4.5 → 1.5.1

package/readme.md

@@ -1,316 +1,730 @@
-# @git.zone/tsdoc
-
+# @git.zone/tsdoc  
 An advanced TypeScript documentation tool using AI to generate and enhance documentation for TypeScript projects.
 
 ## Install
 
-To install `@git.zone/tsdoc`, you can either install it globally or use it with `npx`.
+To install @git.zone/tsdoc, you have two options. You can install it globally so that the CLI commands are available throughout your system, or you can use it with npx if you prefer to keep the installation local to your project.
 
 ### Global Installation
 
-You can install `@git.zone/tsdoc` globally on your system using npm:
+To install the tool globally, run the following command in your terminal:
 
 ```bash
 npm install -g @git.zone/tsdoc
 ```
 
+Installing globally ensures that the CLI commands (such as tsdoc, tsdoc typedoc, tsdoc aidoc, etc.) are available anywhere on your machine without the need to refer to the local node_modules folder.
+
 ### Usage with npx
 
-If you prefer not to install it globally, you can use it with `npx`:
+If you prefer not to install the tool globally, you can invoke it using npx directly from your project. This method works well if you intend to use the tool on a per-project basis:
 
 ```bash
 npx @git.zone/tsdoc <command>
 ```
 
+In the commands below, you will see how to use the various functionalities that @git.zone/tsdoc provides for generating intricate and enhanced documentation for your TypeScript projects.
+
 ## Usage
 
-`@git.zone/tsdoc` provides a comprehensive CLI tool and advanced AI-enhanced features to generate and enhance documentation for your TypeScript projects.
+The @git.zone/tsdoc module provides a very rich and interactive CLI interface together with a set of programmable classes that let you integrate documentation generation into your build processes or workflows. This section will walk you through every aspect of the module—from its basic CLI commands to its advanced internal API usage. All examples provided below use ESM syntax with TypeScript and are designed to be comprehensive. Every code snippet is written so you can easily copy, paste, and adapt to your project. The following guide is divided into several sections covering every major feature, tool integration, and customization options available in the module.
+
+-------------------------------------------------------------------
+### Overview and Core Concepts
+
+At its heart, @git.zone/tsdoc is a CLI tool that blends classic documentation generation (using libraries such as TypeDoc) with AI-enhanced techniques. The tool reads your project files, uses a context builder to optimize file content based on token usage and configurable trimming strategies, and then leverages an AI engine to generate enhanced documentation. This complete solution is designed to integrate smoothly into your project pipeline.
 
-### Using the CLI
+Key features include:
+- **Auto-detection of documentation format**: The CLI attempts to determine the best documentation strategy for your project.
+- **Support for TypeDoc generation**: Build TypeDoc-compatible documentation directly.
+- **AI-Enhanced Documentation (AiDoc)**: Generate a README and project description using artificial intelligence that analyzes your project’s code and context.
+- **Plugin Integration**: The module leverages a variety of plugins (smartfile, smartgit, smartcli, smartai, etc.) to streamline tasks such as file manipulation, CLI interaction, shell command execution, and logging.
+- **Context Trimming and Optimization**: To manage token usage (especially for AI input), the module includes advanced context-building strategies that trim and summarize code files intelligently.
+- **Robust Internal API**: While the primary user interface is through the CLI, the underlying classes (AiDoc, TypeDoc, Readme, etc.) can be used to build custom integrations or extend the tool’s functionality.
 
-The primary interface for `@git.zone/tsdoc` is through its command-line tool. Below, we'll explore the different commands available and provide examples of how to use them.
+Below, you will find detailed explanations along with ESM/TypeScript code examples for all core use cases.
 
-### Commands
+-------------------------------------------------------------------
+### Command-Line Interface (CLI) Usage
 
-#### `tsdoc` - Auto-detect Documentation Format
+The most common way to interact with @git.zone/tsdoc is via its command-line interface (CLI). The CLI is designed to auto-detect your project’s context and trigger the appropriate commands based on your needs. Below is a guide on how to use the CLI commands.
 
-The standard command will automatically detect the documentation format of your project and generate the appropriate documentation.
+#### Basic Invocation
+
+When you run the command without any arguments, the tool attempts to determine the appropriate documentation generation mode:
 
 ```bash
 tsdoc
 ```
 
-### Example
+This will scan the project directory and attempt to detect whether your project follows a TypeDoc convention or if it would benefit from an AI-enhanced documentation build. The auto-detection logic uses the project context (for example, the presence of a ts directory or specific configuration files) to decide the best course of action.
 
-```typescript
-// In a TypeScript project, run the above command.
+##### Example Scenario
+
+Imagine you have a TypeScript project with the following structure:
+
+├── package.json  
+├── ts/  
+│   └── index.ts  
+└── readme.hints.md  
+
+When you execute:
+
+```bash
+tsdoc
 ```
 
-#### `tsdoc typedoc` - Generate Documentation using TypeDoc
+The tool will analyze the project directory, recognizing the ts/ folder, and it will route the command to the appropriate documentation generator, such as the TypeDoc generator if it detects valid structure.
 
-The `typedoc` command will generate documentation compliant with the TypeDoc format.
+#### TypeDoc Command
+
+For projects that require a traditional documentation format, you can explicitly generate documentation using the TypeDoc integration:
 
 ```bash
 tsdoc typedoc --publicSubdir docs
 ```
 
-### Example
+This command instructs the module to generate HTML documentation using TypeDoc, placing the output into a public directory (or a custom subdirectory as specified).
+
+**Inside a TypeScript file, this command can be mirrored by calling the TypeDoc class directly:**
 
 ```typescript
-import * as plugins from '@push.rocks/smartfile';
+import { TypeDoc } from '@git.zone/tsdoc';
+import * as path from 'path';
+
+const cwd = process.cwd();
+const typeDocInstance = new TypeDoc(cwd);
 
-const tsconfigPath = plugins.path.join(__dirname, 'tsconfig.json');
-const outputPath = plugins.path.join(__dirname, 'docs');
+const compileDocumentation = async (): Promise<void> => {
+  try {
+    // Specify the output subdirectory for documentation
+    await typeDocInstance.compile({
+      publicSubdir: 'docs'
+    });
+    console.log('Documentation successfully generated using TypeDoc.');
+  } catch (error) {
+    console.error('Error generating documentation with TypeDoc:', error);
+  }
+};
 
-await plugins.smartshellInstance.exec(
-  `typedoc --tsconfig ${tsconfigPath} --out ${outputPath} index.ts`,
-);
+compileDocumentation();
 ```
 
-#### `tsdoc aidoc` - Generate AI-Enhanced Documentation
+In this example, the script creates an instance of the TypeDoc class passing the current working directory. The compile method is then called with an options object, indicating that the public subdirectory should be named “docs.” The method spawns a shell command using the smart shell plugin to execute the TypeDoc binary.
+
+#### AI-Enhanced Documentation Command
 
-The `aidoc` command will use AI to generate an enhanced README and update your project description.
+One of the standout features of this module is its AI-enhanced documentation capabilities. The `aidoc` command integrates with an OpenAI interface to produce a more contextual and detailed README and project description. This is particularly useful when your project codebase has evolved and requires documentation updates based on the current source code.
+
+To run the AI-enhanced documentation generation:
 
 ```bash
 tsdoc aidoc
 ```
 
-### Example
+In an ESM/TypeScript project, you can use the AiDoc class to programmatically run the same functionality:
 
 ```typescript
 import { AiDoc } from '@git.zone/tsdoc';
 
-(async () => {
-  const aidoc = new AiDoc();
-  await aidoc.start();
-  await aidoc.buildReadme('./');
-  await aidoc.buildDescription('./');
-})();
+const buildEnhancedDocs = async (): Promise<void> => {
+  const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+  try {
+    // Start the AI interface; this internally checks if the token is valid and persists it
+    await aiDoc.start();
+
+    // Build the README file for the project directory
+    console.log('Generating README file using AI...');
+    await aiDoc.buildReadme(process.cwd());
+
+    // Build a new project description based on the codebase
+    console.log('Generating updated project description...');
+    await aiDoc.buildDescription(process.cwd());
+
+    console.log('AI-enhanced documentation generated successfully.');
+  } catch (error) {
+    console.error('Failed to generate AI-enhanced documentation:', error);
+  }
+};
+
+buildEnhancedDocs();
 ```
 
-#### `tsdoc test` - Test Your Documentation Setup
+In the above snippet, we import the AiDoc class and create an instance with an OpenAI token. The methods start(), buildReadme(), and buildDescription() streamline the process of generating enhanced documentation by leveraging the underlying AI engine. This code example should serve as a blueprint for those wishing to integrate AI-driven documentation updates as part of their CI/CD pipelines.
+
+#### Testing Your Documentation Setup
 
-The `test` command will test your current documentation setup, ensuring everything is configured correctly.
+Before you commit changes to your project documentation, it is often worthwhile to run tests to ensure that your documentation generation process is behaving as expected. The module includes a `test` command:
 
 ```bash
 tsdoc test
 ```
 
-### Example
+This command verifies that all components (CLI commands, TypeDoc compilation, AI integration, etc.) are properly configured.
+
+Here is an example test script written in TypeScript using a test bundle:
 
 ```typescript
-import * as plugins from '@git.zone/tsdoc';
+import { expect, tap } from '@push.rocks/tapbundle';
+import { AiDoc } from '@git.zone/tsdoc';
 
-await plugins.runCli().catch((err) => {
-  console.error(err);
-  process.exit(1);
+tap.test('AiDoc instance creation', async () => {
+  const aidoc = new AiDoc({ OPENAI_TOKEN: 'dummy-token' });
+  expect(aidoc).toBeInstanceOf(AiDoc);
 });
-```
 
-## Features in Depth
+tap.test('Running AI documentation generation', async () => {
+  const aidoc = new AiDoc({ OPENAI_TOKEN: 'dummy-token' });
+  await aidoc.start();
+  
+  // Attempt buildReadme and buildDescription synchronously for test coverage
+  await aidoc.buildReadme(process.cwd());
+  await aidoc.buildDescription(process.cwd());
+  
+  // If no errors are thrown, we assume the process works as expected
+  expect(true).toBe(true);
+});
+
+tap.start();
+```
 
-### Using Plugins
+This test script demonstrates how to automate the validation process by using the provided AiDoc class. Using a testing framework like tap ensures that your documentation generation remains robust even as new features are added or as the project evolves.
 
-`@git.zone/tsdoc` extensively uses plugins to extend its capabilities.
+-------------------------------------------------------------------
+### Advanced Usage Scenarios
 
-### Available Plugins
+Beyond using the CLI, @git.zone/tsdoc provides various classes and plugins that allow you to deeply integrate documentation generation within your project. The following sections document advanced usage scenarios where you programmatically interact with different components.
 
-- **npmextra**: Manage npm project-related configurations.
-- **qenv**: Environment variable management.
-- **smartai**: AI integration.
-- **smartcli**: CLI helper.
-- **smartdelay**: Simple delay utility.
-- **smartfile**: File system utilities.
-- **smartinteract**: Interaction helper.
-- **smartlog**: Logging utility.
-- **smartlogDestinationLocal**: Local file destination for logging.
-- **smartpath**: Path utilities.
-- **smartshell**: Shell command execution.
-- **typedoc**: Documentation generation.
+#### 1. Deep Dive into AiDoc Functionality
 
-### Example Usage of Plugins
+The AiDoc class is the core of the AI-enhanced documentation generation. It manages interactions with the OpenAI API, handles token validations, and integrates with project-specific configurations.
 
-#### Path Management
+Consider the following advanced usage example:
 
 ```typescript
+import { AiDoc } from '@git.zone/tsdoc';
 import * as path from 'path';
 
-const packageDir = path.join(__dirname, '../');
-const cwd = process.cwd();
-const binDir = path.join(packageDir, './node_modules/.bin');
-const assetsDir = path.join(packageDir, './assets');
-const publicDir = path.join(cwd, './public');
-const tsDir = path.join(cwd, './ts');
-const tsconfigFile = path.join(assetsDir, './tsconfig.json');
-const typedocOptionsFile = path.join(assetsDir, './typedoc.json');
+const generateProjectDocs = async () => {
+  // Create an instance of the AiDoc class with a configuration object 
+  // that includes your OpenAI token. This token will be used to query the AI.
+  const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+
+  // Initialize the AI documentation system.
+  await aiDoc.start();
+
+  // Build the README file for the current project.
+  console.log('Building README for the project...');
+  await aiDoc.buildReadme(process.cwd());
+
+  // Build an updated project description based on the analysis of the source files.
+  console.log('Building project description...');
+  await aiDoc.buildDescription(process.cwd());
+
+  // You can also generate a commit message based on code changes by using the next commit object generation.
+  try {
+    console.log('Generating commit message based on your project changes...');
+    const nextCommit = await aiDoc.buildNextCommitObject(process.cwd());
+    console.log('Next commit message object:', nextCommit);
+  } catch (error) {
+    console.error('Error generating commit message:', error);
+  }
+};
+
+generateProjectDocs();
 ```
 
-#### Logging
+In this example, the AiDoc class handles multiple tasks:
+
+- It starts by validating and printing the sanitized token.
+- It generates and writes the README file based on dynamic analysis.
+- It updates the project description stored in your configuration files.
+- It even integrates with Git to produce a suggested commit message that factors in the current state of the project directory.
+
+Internally, methods such as buildReadme() interact with the ProjectContext class to gather files and determine the relevant context. This context is trimmed and processed based on token budgets, thus ensuring that the AI interface only receives the information it can effectively process.
+
+#### 2. Interacting with the TypeDoc Class Programmatically
+
+The TypeDoc class does not merely wrap the standard TypeDoc tool. It adds a layer of automation by preparing the TypeScript environment, generating a temporary tsconfig file, and invoking TypeDoc with the proper configuration. You can use this functionality to conditionally generate documentation or integrate it into your build steps.
+
+Below is another example demonstrating the integration:
 
 ```typescript
-import * as plugins from './plugins';
-
-const logger = new plugins.smartlog.Smartlog({
-  logContext: {
-    company: 'Some Company',
-    companyunit: 'Some CompanyUnit',
-    containerName: 'Some Containername',
-    environment: 'local',
-    runtime: 'node',
-    zone: 'gitzone',
-  },
-  minimumLogLevel: 'silly',
-});
+import { TypeDoc } from '@git.zone/tsdoc';
+import * as path from 'path';
 
-logger.addLogDestination(new plugins.smartlogDestinationLocal.DestinationLocal());
+const generateTypeDocDocs = async () => {
+  // Assume you are in the root directory of your TypeScript project
+  const cwd = process.cwd();
+
+  // Create an instance of the TypeDoc class
+  const typeDocInstance = new TypeDoc(cwd);
+
+  // Prepare additional options if necessary (e.g., setting a public subdirectory for docs)
+  const options = { publicSubdir: 'documentation' };
+
+  // Compile your TypeScript project documentation.
+  // The compile method handles creating the tsconfig file, running the shell command, and cleaning up afterward.
+  try {
+    console.log('Compiling TypeScript documentation using TypeDoc...');
+    await typeDocInstance.compile(options);
+    console.log('Documentation generated at:', path.join(cwd, 'public', 'documentation'));
+  } catch (error) {
+    console.error('Error compiling TypeDoc documentation:', error);
+  }
+};
+
+generateTypeDocDocs();
 ```
 
-## Advanced Usage
+This script clearly demonstrates how TypeDoc automation is structured inside the module. By invoking the compile() method, the class takes care of setting directory paths, preparing command arguments, and executing the underlying TypeDoc binary using the smart shell plugin.
+
+#### 3. Customizing Context Building
 
-### Using `TypeDoc` Class
+One of the critical functionalities within @git.zone/tsdoc is its ability to build a smart context for documentation generation. The module not only collects file content from your project (like package.json, readme.hints.md, and other source files) but also intelligently trims and summarizes these contents to fit within token limits for AI processing.
 
-The `TypeDoc` class provides utility methods to compile TypeScript documentation.
+Consider the following deep-dive example into the context building process:
 
 ```typescript
-import { TypeDoc } from '@git.zone/tsdoc/classes.typedoc';
+import { EnhancedContext } from '@git.zone/tsdoc';
+import { ConfigManager } from '@git.zone/tsdoc/dist/ts/context/config-manager.js';
 
-const typeDocInstance = new TypeDoc(paths.cwd);
+const buildProjectContext = async () => {
+  const projectDir = process.cwd();
 
-await typeDocInstance.compile({
-  publicSubdir: 'docs',
-});
+  // Create an instance of the EnhancedContext class to optimize file content for AI use.
+  const enhancedContext = new EnhancedContext(projectDir);
+
+  // Initialize the context builder. This ensures that configuration (e.g., token budgets, trimming options) is loaded.
+  await enhancedContext.initialize();
+
+  // Optionally, you can choose to set a custom token budget and context mode.
+  enhancedContext.setTokenBudget(100000); // for example, limit tokens to 100K
+  enhancedContext.setContextMode('trimmed');
+
+  // Build the context string from selected files in your project.
+  const contextResult = await enhancedContext.buildContext('readme');
+
+  console.log('Context generated with token count:', contextResult.tokenCount);
+  console.log('Token savings due to trimming:', contextResult.tokenSavings);
+
+  // The context string includes file boundaries and token information.
+  console.log('Generated Context:', contextResult.context);
+};
+
+buildProjectContext();
 ```
 
-### Using `AiDoc` Class
+In this example:
+- The EnhancedContext class is initialized with the project directory.
+- Configuration is loaded via the ConfigManager, which reads parameters from npmextra.json.
+- The context builder then gathers files (such as package.json, readme hints, TypeScript sources, etc.), trims unnecessary content, and builds a context string.
+- Finally, it prints the overall token counts and savings, giving you valuable feedback on how the context was optimized for the AI input.
+
+This detailed context-building mechanism is essential for managing large TypeScript projects. It ensures that the AI engine can process relevant code data without being overwhelmed by too many tokens.
+
+#### 4. Working with the Readme Class for Automatic README Generation
 
-The `AiDoc` class integrates with AI services to enhance your documentation.
+The Readme class in @git.zone/tsdoc takes the AI-enhanced documentation further by not only generating a project-level README but also iterating over submodules within your project. This ensures that every published module has its own complete, AI-generated README.
 
-#### Initializing and Using AI
+Here is an advanced example demonstrating how to trigger README generation for the main project and its submodules:
 
 ```typescript
 import { AiDoc } from '@git.zone/tsdoc';
+import * as path from 'path';
+import { logger } from '@git.zone/tsdoc/dist/ts/logging.js';
+import { readFileSync } from 'fs';
+
+const buildProjectReadme = async () => {
+  const projectDir = process.cwd();
+
+  // Create an instance of the AiDoc class to handle AI-enhanced docs generation
+  const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+
+  // Start the AI interface
+  await aiDoc.start();
 
-const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+  // Build the primary README for the project directory
+  console.log('Generating primary README...');
+  await aiDoc.buildReadme(projectDir);
 
-await aiDoc.start();
-await aiDoc.buildReadme('./');
-await aiDoc.buildDescription('./');
+  // Logging function to verify submodule processing
+  logger.log('info', `Primary README generated in ${projectDir}`);
+
+  // Assume that submodules are organized in distinct directories. 
+  // Here we simulate the process of scanning subdirectories and triggering README generation for each.
+  const subModules = ['submodule1', 'submodule2'];
+
+  // Loop through each submodule directory to generate its README.
+  for (const subModule of subModules) {
+    const subModuleDir = path.join(projectDir, subModule);
+    logger.log('info', `Generating README for submodule: ${subModule}`);
+
+    // Each submodule README is generated independently.
+    await aiDoc.buildReadme(subModuleDir);
+
+    // Optionally, read the generated README content for verification.
+    const readmePath = path.join(subModuleDir, 'readme.md');
+    try {
+      const readmeContent = readFileSync(readmePath, 'utf8');
+      logger.log('info', `Generated README for ${subModule}:\n${readmeContent.substring(0, 200)}...`);
+    } catch (error) {
+      logger.log('error', `Failed to read README for ${subModule}: ${error}`);
+    }
+  }
+};
+
+buildProjectReadme();
 ```
 
-#### Retrieving AI Tokens
+In this example, the script:
+- Starts by building the AI-enhanced README for the entire project.
+- Then iterates over a list of submodule directories and generates READMEs for each.
+- Uses the logging utility to provide immediate feedback on the generation process.
+- Optionally, reads back a snippet of the generated file to verify successful documentation generation.
+
+This approach ensures that projects with multiple submodules or packages maintain a consistent and high-quality documentation standard across every component.
+
+-------------------------------------------------------------------
+### Plugin-Based Architecture and Integrations
+
+Under the hood, @git.zone/tsdoc leverages a number of smaller, focused plugins that extend its functionality. These plugins facilitate file system operations, shell command execution, environment variable management, and logging. The modular design makes it easy to extend or customize the tool according to your needs.
+
+The relevant plugins include:
+- smartai: Provides the API integration with OpenAI.
+- smartcli: Handles CLI input parsing and command setup.
+- smartdelay: Manages asynchronous delays and debouncing.
+- smartfile: Offers an abstraction over file I/O operations.
+- smartgit: Facilitates integration with git repositories (e.g., retrieving diffs, commit status).
+- smartinteract: Eases interaction with the user (prompting for tokens, confirming actions).
+- smartlog and smartlogDestinationLocal: Provide comprehensive logging mechanisms.
+- smartpath, smartshell, and smarttime: Manage file paths, execute shell commands, and process time data respectively.
+
+Below is a sample snippet illustrating how you might directly interact with a few of these plugins to, for example, run a custom shell command or log events:
 
 ```typescript
-import * as plugins from '@git.zone/tsdoc/plugins';
+import * as plugins from '@git.zone/tsdoc/dist/ts/plugins.js';
+import { logger } from '@git.zone/tsdoc/dist/ts/logging.js';
+
+const runCustomCommand = async () => {
+  // Create an instance of the smart shell utility
+  const smartshellInstance = new plugins.smartshell.Smartshell({
+    executor: 'bash',
+    pathDirectories: [plugins.smartpath.join(process.cwd(), 'node_modules/.bin')]
+  });
 
-const qenv = new plugins.qenv.Qenv();
-const openaiToken = await qenv.getEnvVarOnDemand('OPENAI_TOKEN');
+  try {
+    // Execute a sample shell command, e.g., listing files in the current directory
+    const output = await smartshellInstance.exec('ls -la');
+    logger.log('info', `Shell command output:\n${output}`);
+  } catch (error) {
+    logger.log('error', 'Error executing custom shell command:', error);
+  }
+};
+
+runCustomCommand();
 ```
 
-### Testing
+This example shows how to:
+- Import the necessary plugins.
+- Set up a smartshell instance by specifying the shell executor and the directories where executables are located.
+- Execute a command (in this case, listing directory contents) and log the results using the provided logging plugin.
+
+Such examples demonstrate the flexibility provided by the module’s internal API. They also illustrate that even if you choose not to use the CLI, you can still leverage the @git.zone/tsdoc functionality programmatically in a highly integrated fashion.
+
+-------------------------------------------------------------------
+### Handling Git Commit Messages with AI
 
-The provided tests demonstrate how to verify the functionality of the `@git.zone/tsdoc` tool.
+A unique feature of this tool is its capacity to assist with creating smart commit messages based on code changes. The Commit class (found within the aidocs_classes directory) ties together output from smartgit and AiDoc to suggest commit messages that are both descriptive and formatted according to conventional commit guidelines.
 
-#### Example Test Script
+Consider this example where you generate a commit message based on the diff from your git repository:
 
 ```typescript
-import { expect, tap } from '@push.rocks/tapbundle';
-import * as tsdoc from '../ts/index';
+import { AiDoc } from '@git.zone/tsdoc';
 
-tap.test('should create AiDoc instance', async () => {
-  const aidoc = new tsdoc.AiDoc({ OPENAI_TOKEN: 'dummy-token' });
-  expect(aidoc).toBeInstanceOf(tsdoc.AiDoc);
-});
+const generateCommitMessage = async () => {
+  // Create an instance of AiDoc
+  const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+
+  // Initialize the AI service
+  await aiDoc.start();
+
+  try {
+    // Generate and retrieve the next commit object based on the uncommitted changes in the repository.
+    const commitObject = await aiDoc.buildNextCommitObject(process.cwd());
+    console.log('Recommended commit object:', commitObject);
+
+    // The commit object is structured with the following fields:
+    // - recommendedNextVersionLevel: Indicates whether the commit is a fix, feature, or breaking change.
+    // - recommendedNextVersionScope: The scope of changes.
+    // - recommendedNextVersionMessage: A short commit message.
+    // - recommendedNextVersionDetails: A list of details explaining the changes.
+    // - recommendedNextVersion: A computed version string.
+  } catch (error) {
+    console.error('Error generating commit message:', error);
+  }
+};
 
-tap.test('should start AiDoc', async () => {
-  const aidoc = new tsdoc.AiDoc({ OPENAI_TOKEN: 'dummy-token' });
-  await aidoc.start();
-  await aidoc.buildReadme('./');
-  await aidoc.buildDescription('./');
-});
+generateCommitMessage();
+```
 
-tap.start();
+The process works as follows:
+1. The AiDoc instance is created and started.
+2. The tool uses the smartgit plugin to fetch uncommitted changes from the repository.
+3. It then builds a context string incorporating file diffs and project metadata.
+4. Finally, the OpenAI API is queried to produce a commit message formatted as JSON. This JSON object is parsed and can be used directly in your git workflow.
+
+This advanced integration assists teams in maintaining consistent commit message standards while reducing the manual burden of summarizing code changes.
+
+-------------------------------------------------------------------
+### Detailed Explanation of Internal Mechanics
+
+If you are curious about the intricate inner workings of @git.zone/tsdoc and wish to extend or debug its behavior, here is an in-depth explanation of some internal mechanisms.
+
+#### Context Trimming Strategy
+
+Managing token count is critical when interfacing with APIs that have strict limits. The module uses a multi-step process:
+- It gathers various files (such as package.json, ts files, readme hints).
+- It sorts the files and calculates the token count using the GPT tokenizer.
+- It applies trimming strategies such as removing function implementations or comments in TypeScript files, based on a configurable set of parameters.
+- Finally, it constructs a unified context string that includes file boundaries for clarity.
+
+For example, the ContextTrimmer class carries out these transformations:
+
+```typescript
+import { ContextTrimmer } from '@git.zone/tsdoc/dist/ts/context/context-trimmer.js';
+
+const trimFileContent = (filePath: string, content: string): string => {
+  // Create an instance of ContextTrimmer with default configuration
+  const trimmer = new ContextTrimmer({
+    removeImplementations: true,
+    preserveJSDoc: true,
+    maxFunctionLines: 5,
+    removeComments: true,
+    removeBlankLines: true
+  });
+  
+  // Trim the content based on the file type and configured options
+  const trimmedContent = trimmer.trimFile(filePath, content, 'trimmed');
+  return trimmedContent;
+};
+
+// Example usage with a TypeScript file
+const tsFileContent = `
+/**
+ * This function calculates the sum of two numbers.
+ */
+export const add = (a: number, b: number): number => {
+  // Calculation logic
+  return a + b;
+};
+`;
+
+const trimmedTSContent = trimFileContent('src/math.ts', tsFileContent);
+console.log('Trimmed TypeScript File Content:\n', trimmedTSContent);
 ```
 
-### Internals
+This process helps in reducing the number of tokens before sending the data to the AI API while preserving the essential context needed for documentation generation.
 
-The `@git.zone/tsdoc` consists of several internal classes and utilities that streamline its functionality.
+#### Dynamic Configuration Management
 
-#### File Paths Management
+The module’s configuration is stored in the npmextra.json file and includes settings for context building, trimming strategies, and task-specific options. The ConfigManager class reads these settings and merges them with default values. This dynamic configuration system ensures that the behavior of the documentation tool can be easily adjusted without altering the source code.
 
-Located in `ts/paths.ts`, the file defines various directories and file paths used by the tool.
+```typescript
+import { ConfigManager } from '@git.zone/tsdoc/dist/ts/context/config-manager.js';
+
+const updateDocumentationConfig = async () => {
+  const projectDir = process.cwd();
+  const configManager = ConfigManager.getInstance();
+  
+  // Initialize the configuration manager with the project directory
+  await configManager.initialize(projectDir);
+  
+  // Retrieve the current configuration
+  let currentConfig = configManager.getConfig();
+  console.log('Current context configuration:', currentConfig);
+  
+  // If you want to change some parameters (e.g., maxTokens), update and then save the new configuration
+  const newConfig = { maxTokens: 150000 };
+  await configManager.updateConfig(newConfig);
+  
+  console.log('Configuration updated successfully.');
+};
+
+updateDocumentationConfig();
+```
+
+In this snippet, the ConfigManager:
+- Loads current configuration from npmextra.json.
+- Allows updates to specific keys (such as token limits).
+- Persists these changes back to the file system using the smartfile plugin.
+
+#### Logging and Diagnostic Output
+
+Throughout its execution, @git.zone/tsdoc logs important information such as token counts, file statistics, and shell command outputs. This logging is accomplished through a combination of the smartlog and smartlogDestinationLocal plugins. The following example illustrates how logging can help diagnose execution issues:
 
 ```typescript
-import * as plugins from './plugins';
-
-export const packageDir = plugins.path.join(
-  plugins.smartpath.get.dirnameFromImportMetaUrl(import.meta.url),
-  '../',
-);
-export const cwd = process.cwd();
-export const binDir = plugins.path.join(packageDir, './node_modules/.bin');
-export const assetsDir = plugins.path.join(packageDir, './assets');
-export const publicDir = plugins.path.join(cwd, './public');
-export const tsDir = plugins.path.join(cwd, './ts');
-export const tsconfigFile = plugins.path.join(assetsDir, './tsconfig.json');
-export const typedocOptionsFile = plugins.path.join(assetsDir, './typedoc.json');
+import { logger } from '@git.zone/tsdoc/dist/ts/logging.js';
+
+const logDiagnosticInfo = () => {
+  logger.log('info', 'Starting documentation generation process...');
+  
+  // Log additional contextual information
+  logger.log('debug', 'Project directory:', process.cwd());
+  logger.log('debug', 'Token budget set for context building:', 150000);
+  
+  // Simulate a long-running process
+  setTimeout(() => {
+    logger.log('info', 'Documentation generation process completed successfully.');
+  }, 2000);
+};
+
+logDiagnosticInfo();
 ```
 
-#### Utility Commands
+Using comprehensive logging, the tool provides feedback not only during normal execution but also in error scenarios, allowing developers to troubleshoot and optimize their documentation generation workflow.
 
-Define utility commands that streamline various processes.
+-------------------------------------------------------------------
+### Integrating @git.zone/tsdoc into a Continuous Integration Pipeline
 
-##### Example: SmartCLI Usage
+For teams looking to integrate documentation generation into their CI processes, @git.zone/tsdoc can be harnessed by scripting the CLI commands or by embedding the class-based API directly into your build scripts. Here’s an example of a CI script written in TypeScript that runs as part of a GitHub Action or similar workflow:
 
 ```typescript
-// Import required modules and plugins
-import * as plugins from './plugins';
-import * as paths from './paths';
-
-// TypeDoc and AiDoc classes
-import { TypeDoc } from './classes.typedoc';
-import { AiDoc } from './classes.aidoc';
-
-// Export a run function
-export const run = async () => {
-  const tsdocCli = new plugins.smartcli.Smartcli();
-
-  tsdocCli.standardCommand().subscribe(async (argvArg) => {
-    switch (true) {
-      case await TypeDoc.isTypeDocDir(paths.cwd):
-        tsdocCli.triggerCommand('typedoc', argvArg);
-        break;
-      default:
-        console.error(`Cannot determine docs format at ${paths.cwd}`);
-    }
-  });
+import { runCli } from '@git.zone/tsdoc';
+import { logger } from '@git.zone/tsdoc/dist/ts/logging.js';
+
+const runDocumentationPipeline = async () => {
+  try {
+    logger.log('info', 'Starting the documentation pipeline...');
+    
+    // Run the CLI which automatically detects the project context and generates docs.
+    await runCli();
+    
+    logger.log('info', 'Documentation pipeline completed successfully.');
+  } catch (error) {
+    logger.log('error', 'Documentation pipeline encountered an error:', error);
+    process.exit(1);
+  }
+};
 
-  tsdocCli.addCommand('typedoc').subscribe(async (argvArg) => {
-    const typeDocInstance = new TypeDoc(paths.cwd);
-    await typeDocInstance.compile({
-      publicSubdir: argvArg.publicSubdir,
-    });
-  });
+runDocumentationPipeline();
+```
 
-  tsdocCli.addCommand('aidoc').subscribe(async (argvArg) => {
-    const aidocInstance = new AiDoc(argvArg);
-    await aidocInstance.start();
-    await aidocInstance.buildReadme(paths.cwd);
-    await aidocInstance.buildDescription(paths.cwd);
-  });
+In a CI environment, you can invoke this script to ensure that documentation is generated or updated as part of your deployment process. The process includes building the README, updating project descriptions, and generating TypeDoc documentation if the project structure warrants it.
+
+-------------------------------------------------------------------
+### Comprehensive Workflow Example
 
-  tsdocCli.startParse();
+Below is a full-fledged example that combines many of the above functionalities into a single workflow. This script is intended to be run as part of your build process or as a standalone command, and it demonstrates how to initialize all parts of the module, generate documentation for the main project and its submodules, update configuration, and log key diagnostics.
+
+```typescript
+import { AiDoc } from '@git.zone/tsdoc';
+import { TypeDoc } from '@git.zone/tsdoc';
+import { ConfigManager } from '@git.zone/tsdoc/dist/ts/context/config-manager.js';
+import { EnhancedContext } from '@git.zone/tsdoc/dist/ts/context/enhanced-context.js';
+import * as path from 'path';
+import { logger } from '@git.zone/tsdoc/dist/ts/logging.js';
+
+const runFullDocumentationWorkflow = async () => {
+  const projectDir = process.cwd();
+  
+  // Initialize configuration management
+  const configManager = ConfigManager.getInstance();
+  await configManager.initialize(projectDir);
+  logger.log('info', `Loaded configuration for project at ${projectDir}`);
+  
+  // Step 1: Generate conventional TypeDoc documentation
+  const typeDocInstance = new TypeDoc(projectDir);
+  try {
+    logger.log('info', 'Starting TypeDoc documentation generation...');
+    await typeDocInstance.compile({ publicSubdir: 'docs' });
+    logger.log('info', `TypeDoc documentation generated in ${path.join(projectDir, 'public', 'docs')}`);
+  } catch (error) {
+    logger.log('error', 'Error during TypeDoc generation:', error);
+  }
+  
+  // Step 2: Run AI-enhanced documentation generation
+  const aiDoc = new AiDoc({ OPENAI_TOKEN: 'your-openai-token' });
+  await aiDoc.start();
+  
+  // Generate main README and updated project description
+  try {
+    logger.log('info', 'Generating main README via AI-enhanced documentation...');
+    await aiDoc.buildReadme(projectDir);
+    logger.log('info', 'Main README generated successfully.');
+
+    logger.log('info', 'Generating updated project description...');
+    await aiDoc.buildDescription(projectDir);
+    logger.log('info', 'Project description updated successfully.');
+  } catch (error) {
+    logger.log('error', 'Error generating AI-enhanced documentation:', error);
+  }
+  
+  // Step 3: Generate contextual data using EnhancedContext
+  const enhancedContext = new EnhancedContext(projectDir);
+  await enhancedContext.initialize();
+  enhancedContext.setContextMode('trimmed');
+  enhancedContext.setTokenBudget(150000);
+
+  const contextResult = await enhancedContext.buildContext('readme');
+  logger.log('info', `Context built successfully. Total tokens: ${contextResult.tokenCount}. Savings: ${contextResult.tokenSavings}`);
+
+  // Step 4: Process submodules (if any) and generate READMEs
+  const subModules = ['submodule1', 'submodule2'];
+  for (const subModule of subModules) {
+    const subModuleDir = path.join(projectDir, subModule);
+    logger.log('info', `Processing submodule: ${subModule}`);
+    try {
+      await aiDoc.buildReadme(subModuleDir);
+      logger.log('info', `Submodule README generated for ${subModule}`);
+    } catch (error) {
+      logger.log('error', `Failed to generate README for ${subModule}:`, error);
+    }
+  }
+  
+  // Optional: Generate a commit message suggestion based on current changes
+  try {
+    logger.log('info', 'Generating commit message suggestion...');
+    const commitObject = await aiDoc.buildNextCommitObject(projectDir);
+    logger.log('info', 'Suggested commit message object:', commitObject);
+  } catch (error) {
+    logger.log('error', 'Error generating commit message suggestion:', error);
+  }
+  
+  logger.log('info', 'Full documentation workflow completed successfully.');
 };
+
+runFullDocumentationWorkflow();
 ```
 
-By leveraging these functionalities, you can configure and extend `@git.zone/tsdoc` to fit your specific documentation generation needs.
+This comprehensive workflow showcases the integration of various facets of the @git.zone/tsdoc module:
+- Loading and updating configuration via the ConfigManager.
+- Generating static documentation using TypeDoc.
+- Enhancing documentation with AI through the AiDoc class.
+- Optimizing project context with the EnhancedContext class.
+- Iterating over submodules to ensure all parts of your project are documented.
+- Providing useful diagnostic logging for every step.
+
+-------------------------------------------------------------------
+### Wrapping Up the Usage Guide
+
+The examples provided above demonstrate that @git.zone/tsdoc is not simply a CLI tool—it is a complete documentation framework designed to adapt to your workflow. Whether you are a developer looking to automate documentation updates in your CI pipeline or a team seeking an AI-powered enhancement for your project metadata, this module offers a wide range of interfaces and hooks for you to leverage.
+
+Key takeaways:
+- The CLI handles most routine tasks automatically while also exposing commands for specific documentation generation strategies.
+- Programmatic usage allows deep integration with your project’s build and commit processes.
+- The internal architecture—built on plugins, context optimization, and extensive logging—ensures that the tool can scale with project complexity.
+- Advanced users can customize context trimming, file inclusion rules, and even modify AI queries to better suit their project’s needs.
+
+Each code example provided here is written using modern ESM syntax and TypeScript to ensure compatibility with current development practices. Since the module is designed with extensibility in mind, developers are encouraged to explore the source code (especially the classes in the ts/ and ts/aidocs_classes directories) for further customization opportunities.
 
-### Further Enhancements
+By integrating @git.zone/tsdoc into your workflow, you ensure that your project documentation remains accurate, comprehensive, and reflective of your latest code changes—whether you are generating a simple README or a complex API documentation set enhanced by AI insights.
 
-The `@git.zone/tsdoc` tool is designed to be extensible. Explore the source files and plugins to add more functionality or integrate with other tools as needed. This README provides an extensive overview of the commands and features but it's always beneficial to dive into the source code to understand the intricacies and potential customizations. Happy documenting!
+Happy documenting!
 
 ## License and Legal Information
 
-This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
+This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 
 
 **Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.