@git.zone/tsdoc 1.4.5 → 1.5.0

package/readme.plan.md

@@ -0,0 +1,314 @@
+# TSDocs Context Optimization Plan
+
+## Problem Statement
+
+For large TypeScript projects, the context generated for AI-based documentation creation becomes too large, potentially exceeding even o4-mini's 200K token limit. This affects the ability to effectively generate:
+
+- Project documentation (README.md)
+- API descriptions and keywords
+- Commit messages and changelogs
+
+Current implementation simply includes all TypeScript files and key project files, but lacks intelligent selection, prioritization, or content reduction mechanisms.
+
+## Analysis of Approaches
+
+### 1. Smart Content Selection
+
+**Description:** Intelligently select only files that are necessary for the specific task being performed, using heuristic rules.
+
+**Advantages:**
+- Simple to implement
+- Predictable behavior
+- Can be fine-tuned for different operations
+
+**Disadvantages:**
+- Requires manual tuning of rules
+- May miss important context in complex projects
+- Static approach lacks adaptability
+
+**Implementation Complexity:** Medium
+
+### 2. File Prioritization
+
+**Description:** Rank files by relevance using git history, file size, import/export analysis, and relationship to the current task.
+
+**Advantages:**
+- Adaptively includes the most relevant files first
+- Maintains context for frequently changed or central files
+- Can leverage git history for additional signals
+
+**Disadvantages:**
+- Complexity in determining accurate relevance scores
+- Requires analyzing project structure
+- May require scanning imports/exports for dependency analysis
+
+**Implementation Complexity:** High
+
+### 3. Chunking Strategy
+
+**Description:** Process the project in logical segments, generating intermediate results that are then combined to create the final output.
+
+**Advantages:**
+- Can handle projects of any size
+- Focused context for each specific part
+- May improve quality by focusing on specific areas deeply
+
+**Disadvantages:**
+- Complex orchestration of multiple AI calls
+- Challenge in maintaining consistency across chunks
+- May increase time and cost for processing
+
+**Implementation Complexity:** High
+
+### 4. Dynamic Context Trimming
+
+**Description:** Automatically reduce context by removing non-essential code while preserving structure. Techniques include:
+- Removing implementation details but keeping interfaces and type definitions
+- Truncating large functions while keeping signatures
+- Removing comments and whitespace (except JSDoc)
+- Keeping only imports/exports for context files
+
+**Advantages:**
+- Preserves full project structure
+- Flexible token usage based on importance
+- Good balance between completeness and token efficiency
+
+**Disadvantages:**
+- Potential to remove important implementation details
+- Risk of missing context needed for specific tasks
+- Complex rules for what to trim vs keep
+
+**Implementation Complexity:** Medium
+
+### 5. Embeddings-Based Retrieval
+
+**Description:** Create vector embeddings of project files and retrieve only the most relevant ones for a specific task using semantic similarity.
+
+**Advantages:**
+- Highly adaptive to different types of requests
+- Leverages semantic understanding of content
+- Can scale to extremely large projects
+
+**Disadvantages:**
+- Requires setting up and managing embeddings database
+- Added complexity of running vector similarity searches
+- Higher resource requirements for maintaining embeddings
+
+**Implementation Complexity:** Very High
+
+### 6. Task-Specific Contexts
+
+**Description:** Create separate optimized contexts for different tasks (readme, commit messages, etc.) with distinct file selection and processing strategies.
+
+**Advantages:**
+- Highly optimized for each specific task
+- Efficient token usage for each operation
+- Improved quality through task-focused contexts
+
+**Disadvantages:**
+- Maintenance of multiple context building strategies
+- More complex configuration
+- Potential duplication in implementation
+
+**Implementation Complexity:** Medium
+
+### 7. Recursive Summarization
+
+**Description:** Summarize larger files first, then include these summaries in the final context along with smaller files included in full.
+
+**Advantages:**
+- Can handle arbitrary project sizes
+- Preserves essential information from all files
+- Balanced approach to token usage
+
+**Disadvantages:**
+- Quality loss from summarization
+- Increased processing time from multiple AI calls
+- Complex orchestration logic
+
+**Implementation Complexity:** High
+
+## Implementation Strategy
+
+We propose a phased implementation approach, starting with the most impactful and straightforward approaches, then building toward more complex solutions as needed:
+
+### Phase 1: Foundation (1-2 weeks)
+
+1. **Implement Dynamic Context Trimming**
+   - Create a `ContextProcessor` class that takes SmartFile objects and applies trimming rules
+   - Implement configurable trimming rules (remove implementations, keep signatures)
+   - Add a configuration option to control trimming aggressiveness
+   - Support preserving JSDoc comments while removing other comments
+
+2. **Enhance Token Monitoring**
+   - Track token usage per file to identify problematic files
+   - Implement token budgeting to stay within limits
+   - Add detailed token reporting for optimization
+
+### Phase 2: Smart Selection (2-3 weeks)
+
+3. **Implement Task-Specific Contexts**
+   - Create specialized context builders for readme, commit messages, and descriptions
+   - Customize file selection rules for each task
+   - Add configuration options for task-specific settings
+
+4. **Add Smart Content Selection**
+   - Implement heuristic rules for file importance
+   - Create configuration for inclusion/exclusion patterns
+   - Add ability to focus on specific directories or modules
+
+### Phase 3: Advanced Techniques (3-4 weeks)
+
+5. **Implement File Prioritization**
+   - Add git history analysis to identify frequently changed files
+   - Implement dependency analysis to identify central files
+   - Create a scoring system for file relevance
+
+6. **Add Optional Recursive Summarization**
+   - Implement file summarization for large files
+   - Create a hybrid approach that mixes full files and summaries
+   - Add configuration to control summarization thresholds
+
+### Phase 4: Research-Based Approaches (Future Consideration)
+
+7. **Research and Evaluate Embeddings-Based Retrieval**
+   - Prototype embeddings creation for TypeScript files
+   - Evaluate performance and accuracy
+   - Implement if benefits justify the complexity
+
+8. **Explore Chunking Strategies**
+   - Research effective chunking approaches for documentation
+   - Prototype and evaluate performance
+   - Implement if benefits justify the complexity
+
+## Technical Design
+
+### Core Components
+
+1. **ContextBuilder** - Enhanced version of current ProjectContext
+   ```typescript
+   interface IContextBuilder {
+     buildContext(): Promise<string>;
+     getTokenCount(): number;
+     setContextMode(mode: 'normal' | 'trimmed' | 'summarized'): void;
+     setTokenBudget(maxTokens: number): void;
+     setPrioritizationStrategy(strategy: IPrioritizationStrategy): void;
+   }
+   ```
+
+2. **FileProcessor** - Handles per-file processing and trimming
+   ```typescript
+   interface IFileProcessor {
+     processFile(file: SmartFile): Promise<string>;
+     setProcessingMode(mode: 'full' | 'trim' | 'summarize'): void;
+     getTokenCount(): number;
+   }
+   ```
+
+3. **PrioritizationStrategy** - Ranks files by importance
+   ```typescript
+   interface IPrioritizationStrategy {
+     rankFiles(files: SmartFile[], context: string): Promise<SmartFile[]>;
+     setImportanceMetrics(metrics: IImportanceMetrics): void;
+   }
+   ```
+
+4. **TaskContextFactory** - Creates optimized contexts for specific tasks
+   ```typescript
+   interface ITaskContextFactory {
+     createContextForReadme(projectDir: string): Promise<string>;
+     createContextForCommit(projectDir: string, diff: string): Promise<string>;
+     createContextForDescription(projectDir: string): Promise<string>;
+   }
+   ```
+
+### Configuration Options
+
+The system will support configuration via a new section in `npmextra.json`:
+
+```json
+{
+  "tsdoc": {
+    "context": {
+      "maxTokens": 190000,
+      "defaultMode": "dynamic",
+      "taskSpecificSettings": {
+        "readme": {
+          "mode": "full",
+          "includePaths": ["src/", "lib/"],
+          "excludePaths": ["test/", "examples/"]
+        },
+        "commit": {
+          "mode": "trimmed",
+          "focusOnChangedFiles": true
+        },
+        "description": {
+          "mode": "summarized",
+          "includePackageInfo": true
+        }
+      },
+      "trimming": {
+        "removeImplementations": true,
+        "preserveInterfaces": true,
+        "preserveTypeDefs": true,
+        "preserveJSDoc": true,
+        "maxFunctionLines": 5
+      }
+    }
+  }
+}
+```
+
+## Cost-Benefit Analysis
+
+### Cost Considerations
+
+1. **Development costs**
+   - Initial implementation of foundational components (~30-40 hours)
+   - Testing and validation across different project sizes (~10-15 hours)
+   - Documentation and configuration examples (~5 hours)
+
+2. **Operational costs**
+   - Potential increased processing time for context preparation
+   - Additional API calls for summarization or embeddings approaches
+   - Monitoring and maintenance of the system
+
+### Benefits
+
+1. **Scalability**
+   - Support for projects of any size, up to and beyond o4-mini's 200K token limit
+   - Future-proof design that can adapt to different models and token limits
+
+2. **Quality improvements**
+   - More focused contexts lead to better AI outputs
+   - Task-specific optimization improves relevance
+   - Consistent performance regardless of project size
+
+3. **User experience**
+   - Predictable behavior for all project sizes
+   - Transparent token usage reporting
+   - Configuration options for different usage patterns
+
+## First Deliverable
+
+For immediate improvements, we recommend implementing Dynamic Context Trimming and Task-Specific Contexts first, as these offer the best balance of impact and implementation complexity.
+
+### Implementation Plan for Dynamic Context Trimming
+
+1. Create a basic `ContextTrimmer` class that processes TypeScript files:
+   - Remove function bodies but keep signatures
+   - Preserve interface and type definitions
+   - Keep imports and exports
+   - Preserve JSDoc comments
+
+2. Integrate with the existing ProjectContext class:
+   - Add a trimming mode option
+   - Apply trimming during the context building process
+   - Track and report token savings
+
+3. Modify the CLI to support trimming options:
+   - Add a `--trim` flag to enable trimming
+   - Add a `--trim-level` option for controlling aggressiveness
+   - Show token usage with and without trimming
+
+This approach could reduce token usage by 40-70% while preserving the essential structure of the codebase, making it suitable for large projects while maintaining high-quality AI outputs.

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@git.zone/tsdoc',
-  version: '1.4.5',
-  description: 'An advanced TypeScript documentation tool using AI to generate and enhance documentation for TypeScript projects.'
+  version: '1.5.0',
+  description: 'A comprehensive TypeScript documentation tool that leverages AI to generate and enhance project documentation, including dynamic README creation, API docs via TypeDoc, and smart commit message generation.'
 }

package/ts/aidocs_classes/commit.ts

@@ -31,15 +31,27 @@ export class Commit {
       'pnpm-lock.yaml',
       'package-lock.json',
     ]);
-    const projectContext = new ProjectContext(this.projectDir);
-    let contextString = await projectContext.update();
-    contextString = `
-${contextString}
-
-Below is the diff of the uncommitted changes. If nothing is changed, there are no changes:
-
-${diffStringArray[0] ? diffStringArray.join('\n\n') : 'No changes.'}
-    `;
+    // Use the new TaskContextFactory for optimized context
+    const taskContextFactory = new (await import('../context/index.js')).TaskContextFactory(this.projectDir);
+    await taskContextFactory.initialize();
+    
+    // Generate context specifically for commit task
+    const contextResult = await taskContextFactory.createContextForCommit(
+      diffStringArray[0] ? diffStringArray.join('\n\n') : 'No changes.'
+    );
+    
+    // Get the optimized context string
+    let contextString = contextResult.context;
+    
+    // Log token usage statistics
+    console.log(`Token usage - Context: ${contextResult.tokenCount}, Files: ${contextResult.includedFiles.length + contextResult.trimmedFiles.length}, Savings: ${contextResult.tokenSavings}`);
+    
+    // Check for token overflow against model limits
+    const MODEL_TOKEN_LIMIT = 200000; // o4-mini
+    if (contextResult.tokenCount > MODEL_TOKEN_LIMIT * 0.9) {
+      console.log(`⚠️ Warning: Context size (${contextResult.tokenCount} tokens) is close to or exceeds model limit (${MODEL_TOKEN_LIMIT} tokens).`);
+      console.log(`The model may not be able to process all information effectively.`);
+    }
 
     let result = await this.aiDocsRef.openaiInstance.chat({
       systemMessage: `

package/ts/aidocs_classes/description.ts

@@ -18,9 +18,16 @@ export class Description {
   }
 
   public async build() {
-    // we can now assemble the directory structure.
-    const projectContext = new ProjectContext(this.projectDir);
-    const contextString = await projectContext.update();
+    // Use the new TaskContextFactory for optimized context
+    const taskContextFactory = new (await import('../context/index.js')).TaskContextFactory(this.projectDir);
+    await taskContextFactory.initialize();
+    
+    // Generate context specifically for description task
+    const contextResult = await taskContextFactory.createContextForDescription();
+    const contextString = contextResult.context;
+    
+    // Log token usage statistics
+    console.log(`Token usage - Context: ${contextResult.tokenCount}, Files: ${contextResult.includedFiles.length + contextResult.trimmedFiles.length}, Savings: ${contextResult.tokenSavings}`);
 
     let result = await this.aiDocsRef.openaiInstance.chat({
       systemMessage: `
@@ -48,7 +55,11 @@ Don't wrap the JSON in three ticks json!!!
       result.message.replace('```json', '').replace('```', ''),
     );
 
-    const npmextraJson = (await projectContext.gatherFiles()).smartfilesNpmextraJSON;
+    // Create a standard ProjectContext instance for file operations
+    const projectContext = new ProjectContext(this.projectDir);
+    const files = await projectContext.gatherFiles();
+    
+    const npmextraJson = files.smartfilesNpmextraJSON;
     const npmextraJsonContent = JSON.parse(npmextraJson.contents.toString());
 
     npmextraJsonContent.gitzone.module.description = resultObject.description;
@@ -58,7 +69,7 @@ Don't wrap the JSON in three ticks json!!!
     await npmextraJson.write();
 
     // do the same with packageJson
-    const packageJson = (await projectContext.gatherFiles()).smartfilePackageJSON;
+    const packageJson = files.smartfilePackageJSON;
     const packageJsonContent = JSON.parse(packageJson.contents.toString());
     packageJsonContent.description = resultObject.description;
     packageJsonContent.keywords = resultObject.keywords;

package/ts/aidocs_classes/projectcontext.ts

@@ -5,6 +5,8 @@ export class ProjectContext {
 
   // INSTANCE
   public projectDir: string;
+  private tokenCount: number = 0;
+  private contextString: string = '';
 
   constructor(projectDirArg: string) {
     this.projectDir = projectDirArg;
@@ -63,6 +65,24 @@ ${smartfile.contents.toString()}
       .join('\n');
   }
 
+  /**
+   * Calculate the token count for a string using the GPT tokenizer
+   * @param text The text to count tokens for
+   * @param model The model to use for token counting (default: gpt-3.5-turbo)
+   * @returns The number of tokens in the text
+   */
+  public countTokens(text: string, model: string = 'gpt-3.5-turbo'): number {
+    try {
+      // Use the gpt-tokenizer library to count tokens
+      const tokens = plugins.gptTokenizer.encode(text);
+      return tokens.length;
+    } catch (error) {
+      console.error('Error counting tokens:', error);
+      // Provide a rough estimate (4 chars per token) if tokenization fails
+      return Math.ceil(text.length / 4);
+    }
+  }
+
   private async buildContext(dirArg: string) {
     const files = await this.gatherFiles();
     let context = await this.convertFilesToContext([
@@ -73,10 +93,33 @@ ${smartfile.contents.toString()}
       ...files.smartfilesMod,
       ...files.smartfilesTest,
     ]);
+    // Count tokens in the context
+    this.contextString = context;
+    this.tokenCount = this.countTokens(context);
+    
     // console.log(context);
     return context;
   }
 
+  /**
+   * Get the token count for the current context
+   * @returns The number of tokens in the context
+   */
+  public getTokenCount(): number {
+    return this.tokenCount;
+  }
+
+  /**
+   * Get both the context string and its token count
+   * @returns An object containing the context string and token count
+   */
+  public getContextWithTokenCount(): { context: string; tokenCount: number } {
+    return {
+      context: this.contextString,
+      tokenCount: this.tokenCount
+    };
+  }
+
   public async update() {
     const result = await this.buildContext(this.projectDir);
     return result;

package/ts/aidocs_classes/readme.ts

@@ -17,11 +17,19 @@ export class Readme {
   public async build() {
     let finalReadmeString = ``;
 
-    // we can now assemble the directory structure.
-    const projectContext = new ProjectContext(this.projectDir);
-    const contextString = await projectContext.update();
+    // Use the new TaskContextFactory for optimized context
+    const taskContextFactory = new (await import('../context/index.js')).TaskContextFactory(this.projectDir);
+    await taskContextFactory.initialize();
+    
+    // Generate context specifically for readme task
+    const contextResult = await taskContextFactory.createContextForReadme();
+    const contextString = contextResult.context;
+    
+    // Log token usage statistics
+    console.log(`Token usage - Context: ${contextResult.tokenCount}, Files: ${contextResult.includedFiles.length + contextResult.trimmedFiles.length}, Savings: ${contextResult.tokenSavings}`);
 
     // lets first check legal before introducung any cost
+    const projectContext = new ProjectContext(this.projectDir);
     const npmExtraJson = JSON.parse(
       (await projectContext.gatherFiles()).smartfilesNpmextraJSON.contents.toString()
     );

package/ts/classes.aidoc.ts

@@ -94,4 +94,37 @@ export class AiDoc {
     const projectContextInstance = new aiDocsClasses.ProjectContext(projectDirArg);
     return await projectContextInstance.gatherFiles();
   }
+  
+  /**
+   * Get the context with token count information
+   * @param projectDirArg The path to the project directory
+   * @returns An object containing the context string and its token count
+   */
+  public async getProjectContextWithTokenCount(projectDirArg: string) {
+    const projectContextInstance = new aiDocsClasses.ProjectContext(projectDirArg);
+    await projectContextInstance.update();
+    return projectContextInstance.getContextWithTokenCount();
+  }
+  
+  /**
+   * Get just the token count for a project's context
+   * @param projectDirArg The path to the project directory
+   * @returns The number of tokens in the project context
+   */
+  public async getProjectContextTokenCount(projectDirArg: string) {
+    const projectContextInstance = new aiDocsClasses.ProjectContext(projectDirArg);
+    await projectContextInstance.update();
+    return projectContextInstance.getTokenCount();
+  }
+  
+  /**
+   * Count tokens in a text string using GPT tokenizer
+   * @param text The text to count tokens for
+   * @param model The model to use for tokenization (default: gpt-3.5-turbo)
+   * @returns The number of tokens in the text
+   */
+  public countTokens(text: string, model: string = 'gpt-3.5-turbo'): number {
+    const projectContextInstance = new aiDocsClasses.ProjectContext('');
+    return projectContextInstance.countTokens(text, model);
+  }
 }

package/ts/cli.ts

@@ -4,6 +4,7 @@ import { logger } from './logging.js';
 
 import { TypeDoc } from './classes.typedoc.js';
 import { AiDoc } from './classes.aidoc.js';
+import * as context from './context/index.js';
 
 export const run = async () => {
   const tsdocCli = new plugins.smartcli.Smartcli();
@@ -30,6 +31,18 @@ export const run = async () => {
   tsdocCli.addCommand('aidoc').subscribe(async (argvArg) => {
     const aidocInstance = new AiDoc();
     await aidocInstance.start();
+    
+    // Get context token count if requested
+    if (argvArg.tokens || argvArg.showTokens) {
+      logger.log('info', `Calculating context token count...`);
+      const tokenCount = await aidocInstance.getProjectContextTokenCount(paths.cwd);
+      logger.log('ok', `Total context token count: ${tokenCount}`);
+      
+      if (argvArg.tokensOnly) {
+        return; // Exit early if we only want token count
+      }
+    }
+    
     logger.log('info', `Generating new readme...`);
     logger.log('info', `This may take some time...`);
     await aidocInstance.buildReadme(paths.cwd);
@@ -38,6 +51,121 @@ export const run = async () => {
     await aidocInstance.buildDescription(paths.cwd);
   });
 
+  tsdocCli.addCommand('tokens').subscribe(async (argvArg) => {
+    const aidocInstance = new AiDoc();
+    await aidocInstance.start();
+    
+    logger.log('info', `Calculating context token count...`);
+    
+    // Determine context mode based on args
+    let contextMode: context.ContextMode = 'full';
+    if (argvArg.trim || argvArg.trimmed) {
+      contextMode = 'trimmed';
+    } else if (argvArg.summarize || argvArg.summarized) {
+      contextMode = 'summarized';
+    }
+    
+    // Get task type if specified
+    let taskType: context.TaskType | undefined = undefined;
+    if (argvArg.task) {
+      if (['readme', 'commit', 'description'].includes(argvArg.task)) {
+        taskType = argvArg.task as context.TaskType;
+      } else {
+        logger.log('warn', `Unknown task type: ${argvArg.task}. Using default context.`);
+      }
+    }
+    
+    // Use enhanced context
+    const taskFactory = new context.TaskContextFactory(paths.cwd);
+    await taskFactory.initialize();
+    
+    let contextResult: context.IContextResult;
+    
+    if (argvArg.all) {
+      // Show stats for all task types
+      const stats = await taskFactory.getTokenStats();
+      
+      logger.log('ok', 'Token statistics by task:');
+      for (const [task, data] of Object.entries(stats)) {
+        logger.log('info', `\n${task.toUpperCase()}:`);
+        logger.log('info', `  Tokens: ${data.tokenCount}`);
+        logger.log('info', `  Token savings: ${data.savings}`);
+        logger.log('info', `  Files: ${data.includedFiles} included, ${data.trimmedFiles} trimmed, ${data.excludedFiles} excluded`);
+        
+        // Calculate percentage of model context
+        const o4MiniPercentage = (data.tokenCount / 200000 * 100).toFixed(2);
+        logger.log('info', `  Context usage: ${o4MiniPercentage}% of o4-mini (200K tokens)`);
+      }
+      
+      return;
+    }
+    
+    if (taskType) {
+      // Get context for specific task
+      contextResult = await taskFactory.createContextForTask(taskType);
+    } else {
+      // Get generic context with specified mode
+      const enhancedContext = new context.EnhancedContext(paths.cwd);
+      await enhancedContext.initialize();
+      enhancedContext.setContextMode(contextMode);
+      
+      if (argvArg.maxTokens) {
+        enhancedContext.setTokenBudget(parseInt(argvArg.maxTokens, 10));
+      }
+      
+      contextResult = await enhancedContext.buildContext();
+    }
+    
+    // Display results
+    logger.log('ok', `Total context token count: ${contextResult.tokenCount}`);
+    logger.log('info', `Files included: ${contextResult.includedFiles.length}`);
+    logger.log('info', `Files trimmed: ${contextResult.trimmedFiles.length}`);
+    logger.log('info', `Files excluded: ${contextResult.excludedFiles.length}`);
+    logger.log('info', `Token savings: ${contextResult.tokenSavings}`);
+    
+    if (argvArg.detailed) {
+      // Show more detailed info about the context and token usage
+      const o4MiniPercentage = (contextResult.tokenCount / 200000 * 100).toFixed(2);
+      logger.log('info', `Token usage: ${o4MiniPercentage}% of o4-mini 200K token context window`);
+      
+      if (argvArg.model) {
+        // Show percentages for different models
+        if (argvArg.model === 'gpt4') {
+          const gpt4Percentage = (contextResult.tokenCount / 8192 * 100).toFixed(2);
+          logger.log('info', `Token usage (GPT-4): ${gpt4Percentage}% of 8192 token context window`);
+        } else if (argvArg.model === 'gpt35') {
+          const gpt35Percentage = (contextResult.tokenCount / 4096 * 100).toFixed(2);
+          logger.log('info', `Token usage (GPT-3.5): ${gpt35Percentage}% of 4096 token context window`);
+        }
+      }
+      
+      // Estimate cost (approximate values)
+      const o4MiniInputCost = 0.00005; // per 1K tokens for o4-mini
+      const estimatedCost = (contextResult.tokenCount / 1000 * o4MiniInputCost).toFixed(6);
+      logger.log('info', `Estimated input cost: $${estimatedCost} (o4-mini)`);
+      
+      if (argvArg.listFiles) {
+        // List files included in context
+        logger.log('info', '\nIncluded files:');
+        contextResult.includedFiles.forEach(file => {
+          logger.log('info', `  ${file.relativePath} (${file.tokenCount} tokens)`);
+        });
+        
+        logger.log('info', '\nTrimmed files:');
+        contextResult.trimmedFiles.forEach(file => {
+          logger.log('info', `  ${file.relativePath} (${file.tokenCount} tokens)`);
+        });
+        
+        if (contextResult.excludedFiles.length > 0) {
+          logger.log('info', '\nExcluded files:');
+          contextResult.excludedFiles.forEach(file => {
+            logger.log('info', `  ${file.relativePath} (${file.tokenCount} tokens)`);
+          });
+        }
+      }
+    }
+  });
+
   tsdocCli.addCommand('test').subscribe((argvArg) => {
     tsdocCli.triggerCommand('typedoc', argvArg);
     process.on('exit', async () => {