@git.zone/tsdoc 1.12.0 → 2.0.0

package/readme.md

@@ -1,4 +1,4 @@
-# @git.zone/tsdoc 🚀
+# @git.zone/tsdoc
 
 AI-Powered Documentation for TypeScript Projects
 
@@ -6,402 +6,226 @@ AI-Powered Documentation for TypeScript Projects
 
 For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.
 
-## What is tsdoc?
-
-`@git.zone/tsdoc` is a next-generation documentation CLI tool that combines traditional TypeDoc generation with cutting-edge AI to create comprehensive, intelligent documentation for your TypeScript projects. It reads your code, understands it, and writes documentation that actually makes sense.
-
-### ✨ Key Features
-
-- **🤖 AI-Enhanced Documentation** - Leverages AI to generate contextual READMEs
-- **🧠 Smart Context Building** - Intelligent file prioritization with dependency analysis and caching
-- **📚 TypeDoc Integration** - Classic API documentation generation when you need it
-- **💬 Smart Commit Messages** - AI analyzes your changes and suggests meaningful commit messages
-- **🎯 Context Optimization** - Advanced token management with 40-60% reduction in usage
-- **⚡ Performance Optimized** - 3-5x faster with lazy loading and parallel processing
-- **📦 Zero Config** - Works out of the box with sensible defaults
-- **🔧 Highly Configurable** - Customize every aspect when needed
-
-## Installation
+## Install
 
 ```bash
-# Global installation (recommended)
+# Global installation (recommended for CLI usage)
 pnpm add -g @git.zone/tsdoc
 
 # Or use with npx
 npx @git.zone/tsdoc
+
+# Or install locally as a dependency
+pnpm add @git.zone/tsdoc
 ```
 
-## Quick Start
+## Usage
+
+`@git.zone/tsdoc` is a comprehensive TypeScript documentation tool that combines traditional TypeDoc API documentation generation with AI-powered documentation workflows. It uses OpenAI models via the Vercel AI SDK to generate READMEs, project descriptions, keywords, and semantic commit messages by intelligently exploring your project with agentic tool use.
+
+### CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `tsdoc` | Auto-detects project type and runs TypeDoc |
+| `tsdoc aidoc` | Generates AI-powered README + description/keywords |
+| `tsdoc readme` | Generates AI-powered README only |
+| `tsdoc description` | Generates AI-powered description and keywords only |
+| `tsdoc commit` | Generates a semantic commit message from uncommitted changes |
+| `tsdoc typedoc` | Generates traditional TypeDoc API documentation |
 
-### Generate AI-Powered Documentation
+### Generating AI-Powered Documentation
+
+The `aidoc` command combines README generation and description/keyword generation in one step:
 
 ```bash
 # In your project root
 tsdoc aidoc
 ```
 
-That's it! tsdoc will analyze your entire codebase and generate:
-- A comprehensive README.md
-- Updated package.json description and keywords
-- Smart documentation based on your actual code structure
+This will:
+1. Analyze your codebase using an AI agent with filesystem access
+2. Generate a comprehensive `readme.md`
+3. Update `package.json` and `npmextra.json` with an AI-generated description and keywords
 
-### Generate Traditional TypeDoc
+You can also run these separately:
 
 ```bash
-tsdoc typedoc --publicSubdir docs
-```
+# Generate only the README
+tsdoc readme
 
-### Get Smart Commit Messages
-
-```bash
-tsdoc commit
+# Generate only the description and keywords
+tsdoc description
 ```
 
-## CLI Commands
-
-| Command | Description |
-|---------|-------------|
-| `tsdoc` | Auto-detects and runs appropriate documentation |
-| `tsdoc aidoc` | Generate AI-enhanced documentation |
-| `tsdoc typedoc` | Generate TypeDoc documentation |
-| `tsdoc commit` | Generate smart commit message |
-| `tsdoc tokens` | Analyze token usage for AI context |
-
-### Token Analysis
+### Generating Commit Messages
 
-Understanding token usage helps optimize AI costs:
+The `commit` command analyzes your uncommitted changes and produces a structured commit object:
 
 ```bash
-# Show token count for current project
-tsdoc tokens
+tsdoc commit
+```
 
-# Show detailed stats for all task types
-tsdoc tokens --all
+Output is a JSON object following conventional commits:
 
-# Show detailed breakdown with file listing
-tsdoc tokens --detailed --listFiles
+```json
+{
+  "recommendedNextVersionLevel": "feat",
+  "recommendedNextVersionScope": "core",
+  "recommendedNextVersionMessage": "add new feature for better documentation",
+  "recommendedNextVersionDetails": [
+    "implemented X",
+    "refactored Y"
+  ],
+  "recommendedNextVersion": "1.13.0",
+  "changelog": "# Changelog\n\n## 2026-03-11 - 1.13.0 - core\n..."
+}
 ```
 
-### Command Options
+The commit command includes intelligent diff processing that:
+- Excludes lock files, build artifacts, IDE directories, and caches from the diff
+- Prioritizes source files over build artifacts
+- Samples large diffs with head/tail extraction to stay within token budgets
+- Automatically generates or updates the changelog
 
-#### tsdoc aidoc
-- `--tokens` / `--showTokens` - Show token count before generating
-- `--tokensOnly` - Only show token count, don't generate
+### Generating TypeDoc
 
-#### tsdoc typedoc
-- `--publicSubdir <dir>` - Output subdirectory within public folder
+For traditional API documentation via TypeDoc:
 
-#### tsdoc tokens
-- `--task <type>` - Specify task type: `readme`, `commit`, or `description`
-- `--all` - Show stats for all task types
-- `--detailed` - Show detailed token usage and costs
-- `--listFiles` - List all files included in context
-- `--model <name>` - Show usage for specific model (`gpt4`, `gpt35`)
+```bash
+# Generate to default ./public directory
+tsdoc typedoc
 
-## Configuration
+# Generate to a specific subdirectory
+tsdoc typedoc --publicSubdir docs
+```
 
-Configure tsdoc via `npmextra.json`:
+### Monorepo Support
 
-```json
-{
-  "@git.zone/tsdoc": {
-    "legal": "## License and Legal Information\n\n...",
-    "context": {
-      "maxTokens": 190000,
-      "defaultMode": "trimmed",
-      "cache": {
-        "enabled": true,
-        "ttl": 3600,
-        "maxSize": 100
-      },
-      "analyzer": {
-        "useAIRefinement": false
-      },
-      "prioritization": {
-        "dependencyWeight": 0.3,
-        "relevanceWeight": 0.4,
-        "efficiencyWeight": 0.2,
-        "recencyWeight": 0.1
-      },
-      "tiers": {
-        "essential": { "minScore": 0.8, "trimLevel": "none" },
-        "important": { "minScore": 0.5, "trimLevel": "light" },
-        "optional": { "minScore": 0.2, "trimLevel": "aggressive" }
-      },
-      "taskSpecificSettings": {
-        "readme": {
-          "mode": "trimmed",
-          "includePaths": ["ts/", "src/"],
-          "excludePaths": ["test/", "node_modules/"]
-        },
-        "commit": {
-          "mode": "trimmed",
-          "focusOnChangedFiles": true
-        }
-      },
-      "trimming": {
-        "removeImplementations": true,
-        "preserveInterfaces": true,
-        "preserveJSDoc": true,
-        "maxFunctionLines": 5,
-        "removeComments": true
-      }
-    }
-  }
-}
-```
+When generating READMEs, tsdoc automatically detects monorepo submodules via `@git.zone/tspublish` conventions. Each submodule directory with a `tspublish.json` gets its own generated README.
 
-### Configuration Options
-
-#### Context Settings
-- **maxTokens** - Maximum tokens for AI context (default: 190000)
-- **defaultMode** - Default context mode: 'full', 'trimmed', or 'summarized'
-- **cache** - Caching configuration for improved performance
-- **analyzer** - Smart file analysis and prioritization settings
-- **prioritization** - Weights for file importance scoring
-- **tiers** - Tier thresholds and trimming levels
-
-#### Cache Configuration
-- **enabled** - Enable/disable file caching (default: true)
-- **ttl** - Time-to-live in seconds (default: 3600)
-- **maxSize** - Maximum cache size in MB (default: 100)
-- **directory** - Cache directory path (default: .nogit/context-cache)
-
-## How It Works
-
-### 🚀 Smart Context Building Pipeline
-
-1. **📊 Fast Metadata Scanning** - Lazy loading scans files without reading contents
-2. **🧬 Dependency Analysis** - Builds dependency graph from import statements
-3. **🎯 Intelligent Scoring** - Multi-factor importance scoring:
-   - **Relevance**: Task-specific file importance (e.g., index.ts for READMEs)
-   - **Centrality**: How many files depend on this file
-   - **Efficiency**: Information density (tokens vs. value)
-   - **Recency**: Recently changed files (for commits)
-4. **🏆 Smart Prioritization** - Files sorted by combined importance score
-5. **🎭 Tier-Based Trimming** - Adaptive trimming based on importance:
-   - **Essential** (score ≥ 0.8): No trimming
-   - **Important** (score ≥ 0.5): Light trimming
-   - **Optional** (score ≥ 0.2): Aggressive trimming
-6. **💾 Intelligent Caching** - Cache results with file change detection
-7. **🧠 AI Processing** - Send optimized context to AI for documentation
-
-### Context Optimization Benefits
-
-The smart context system delivers significant improvements:
-
-| Metric | Before | After | Improvement |
-|--------|--------|-------|-------------|
-| **Token Usage** | ~190k (limit) | ~110-130k | ⬇️ 40-60% reduction |
-| **Build Time** | 4-6 seconds | 1-2 seconds | ⚡ 3-5x faster |
-| **Memory Usage** | All files loaded | Metadata + selected | 📉 80%+ reduction |
-| **Relevance** | Alphabetical sorting | Smart scoring | 🎯 90%+ relevant |
-| **Cache Hits** | None | 70-80% | 🚀 Major speedup |
-
-## Environment Variables
-
-| Variable | Description |
-|----------|-------------|
-| `OPENAI_TOKEN` | Your OpenAI API key for AI features (required) |
-
-The token can also be provided interactively on first run - it will be persisted in `~/.npmextra/kv/@git.zone/tsdoc.json`.
-
-## Use Cases
-
-### 🚀 Continuous Integration
-
-```yaml
-# .github/workflows/docs.yml
-name: Documentation
-on: [push]
-jobs:
-  docs:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v3
-      - name: Setup Node
-        uses: actions/setup-node@v3
-        with:
-          node-version: '18'
-      - name: Generate Documentation
-        env:
-          OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }}
-        run: |
-          npm install -g @git.zone/tsdoc
-          tsdoc aidoc
-      - name: Commit Changes
-        run: |
-          git config --local user.email "action@github.com"
-          git config --local user.name "GitHub Action"
-          git add readme.md package.json
-          git commit -m "docs: update documentation [skip ci]" || exit 0
-          git push
-```
+### Programmatic API
 
-### 🔄 Pre-Commit Hooks
+You can also use tsdoc programmatically:
 
-```bash
-# .git/hooks/prepare-commit-msg
-#!/bin/bash
-tsdoc commit > .git/COMMIT_EDITMSG
-```
+```typescript
+import { AiDoc } from '@git.zone/tsdoc';
 
-### 📦 Package Publishing
+const aidoc = new AiDoc();
+await aidoc.start(); // Initializes the AI model (prompts for OpenAI token if needed)
 
-```json
-{
-  "scripts": {
-    "prepublishOnly": "tsdoc aidoc",
-    "version": "tsdoc aidoc && git add readme.md"
-  }
-}
-```
+// Generate a README
+await aidoc.buildReadme('/path/to/project');
 
-## Requirements
+// Generate description and keywords
+await aidoc.buildDescription('/path/to/project');
 
-- **Node.js** >= 18.0.0
-- **TypeScript** project
-- **OpenAI API key** (for AI features)
+// Generate a commit message object
+const commitObj = await aidoc.buildNextCommitObject('/path/to/project');
+console.log(commitObj);
 
-## Troubleshooting
+// Get project context information
+const context = await aidoc.getProjectContext('/path/to/project');
 
-### Token Limit Exceeded
+// Get token count for a project
+const tokenCount = await aidoc.getProjectContextTokenCount('/path/to/project');
 
-If you hit token limits, try:
+// Estimate tokens in arbitrary text
+const tokens = aidoc.countTokens('some text here');
 
-```bash
-# Check token usage details
-tsdoc tokens --all --detailed
+await aidoc.stop();
 ```
 
-Or configure stricter limits in `npmextra.json`:
+## Configuration
 
-```json
-{
-  "@git.zone/tsdoc": {
-    "context": {
-      "maxTokens": 100000,
-      "tiers": {
-        "essential": { "minScore": 0.9, "trimLevel": "none" },
-        "important": { "minScore": 0.7, "trimLevel": "aggressive" },
-        "optional": { "minScore": 0.5, "trimLevel": "aggressive" }
-      }
-    }
-  }
-}
-```
+### OpenAI Token
 
-### Missing API Key
+An OpenAI API key is required for all AI features. It can be provided in three ways:
 
-Set your OpenAI key:
+1. **Environment variable**: `OPENAI_TOKEN`
+2. **Interactive prompt**: On first run, tsdoc will prompt for the token and persist it
+3. **Constructor argument**: Pass `{ OPENAI_TOKEN: 'sk-...' }` to `new AiDoc()`
 
-```bash
-export OPENAI_TOKEN="your-key-here"
-tsdoc aidoc
-```
+The token is persisted at `~/.npmextra/kv/@git.zone/tsdoc.json` for subsequent runs.
 
-### Slow Performance
+### npmextra.json
 
-Enable caching and adjust settings in `npmextra.json`:
+tsdoc uses `npmextra.json` for project metadata. The `tsdoc` key holds legal information that gets appended to generated READMEs:
 
 ```json
 {
-  "@git.zone/tsdoc": {
-    "context": {
-      "cache": {
-        "enabled": true,
-        "ttl": 7200,
-        "maxSize": 200
-      }
+  "tsdoc": {
+    "legal": "\n## License and Legal Information\n\n..."
+  },
+  "gitzone": {
+    "module": {
+      "githost": "gitlab.com",
+      "gitscope": "gitzone",
+      "gitrepo": "tsdoc",
+      "npmPackagename": "@git.zone/tsdoc",
+      "description": "...",
+      "keywords": ["..."]
     }
   }
 }
 ```
 
-### Cache Issues
+## Architecture
 
-Clear the cache if needed:
+### Core Components
 
-```bash
-rm -rf .nogit/context-cache
+```
+@git.zone/tsdoc
+├── AiDoc                 # Main orchestrator - manages AI model and delegates to task classes
+├── TypeDoc               # Traditional TypeDoc API documentation generation
+├── ProjectContext        # Gathers project files for context (package.json, ts/, test/)
+├── DiffProcessor         # Intelligent git diff processing with prioritization and sampling
+├── Readme                # AI-powered README generation using runAgent with filesystem tools
+├── Commit                # AI-powered commit message generation with diff analysis
+├── Description           # AI-powered description and keyword generation
+└── CLI                   # Command-line interface built on @push.rocks/smartcli
 ```
 
-## Why tsdoc?
-
-### 🎯 Actually Understands Your Code
-Not just parsing, but real comprehension through AI. The smart context system ensures AI sees the most relevant parts of your codebase.
-
-### ⏱️ Saves Hours
-Generate complete, accurate documentation in seconds. The intelligent caching system makes subsequent runs even faster.
+### AI Agent Architecture
 
-### 🔄 Always Up-to-Date
-Regenerate documentation with every change. Smart dependency analysis ensures nothing important is missed.
+tsdoc uses `@push.rocks/smartagent`'s `runAgent()` function with `@push.rocks/smartai`'s `getModel()` for all AI tasks. Each documentation task (readme, commit, description) runs an autonomous AI agent that:
 
-### 🎨 Beautiful Output
-Clean, professional documentation every time. AI understands your code's purpose and explains it clearly.
+1. Receives a system prompt defining its role and constraints
+2. Gets access to scoped filesystem tools (read-only, limited to project directory)
+3. Explores the project structure autonomously using tool calls
+4. Produces the final output (README markdown, commit JSON, or description JSON)
 
-### 💰 Cost-Effective
-Smart context optimization reduces AI API costs by 40-60% without sacrificing quality.
+### Diff Processing
 
-## Architecture
+The `DiffProcessor` class handles large git diffs intelligently:
 
-### Core Components
+- **Small files** (< 300 lines): Included in full
+- **Medium files** (< 800 lines): Head/tail sampling with context
+- **Large files**: Metadata only (filepath, lines added/removed)
+- Files are prioritized by importance: source > test > config > docs > build artifacts
+- Token budget is enforced dynamically based on OpenAI's context limits
 
-```
-@git.zone/tsdoc
-├── AiDoc              # Main AI documentation orchestrator
-├── TypeDoc            # Traditional TypeDoc integration
-├── Context System     # Smart context building
-│   ├── EnhancedContext      # Main context builder
-│   ├── LazyFileLoader       # Efficient file loading
-│   ├── ContextCache         # Performance caching
-│   ├── ContextAnalyzer      # Intelligent file analysis
-│   ├── ContextTrimmer       # Adaptive code trimming
-│   ├── DiffProcessor        # Git diff optimization
-│   ├── ConfigManager        # Configuration management
-│   └── TaskContextFactory   # Task-specific contexts
-└── CLI                # Command-line interface
-```
-
-### Data Flow
+## Requirements
 
-```
-Project Files
-    ↓
-LazyFileLoader (metadata scan)
-    ↓
-ContextAnalyzer (scoring & prioritization)
-    ↓
-ContextCache (check cache)
-    ↓
-File Loading (parallel, on-demand)
-    ↓
-ContextTrimmer (tier-based)
-    ↓
-Token Budget (enforcement)
-    ↓
-AI Model
-    ↓
-Generated Documentation
-```
+- **Node.js** >= 18.0.0
+- **TypeScript** project with `ts/` source directory
+- **OpenAI API key** (for AI features)
 
 ## License and Legal Information
 
-This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
+This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
 
 **Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
 
 ### Trademarks
 
-This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
-
-Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
+This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
 
 ### Company Information
 
 Task Venture Capital GmbH
-Registered at District Court Bremen HRB 35230 HB, Germany
+Registered at District court Bremen HRB 35230 HB, Germany
 
-For any legal inquiries or further information, please contact us via email at hello@task.vc.
+For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
 
 By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@git.zone/tsdoc',
-  version: '1.12.0',
+  version: '2.0.0',
   description: 'A comprehensive TypeScript documentation tool that leverages AI to generate and enhance project documentation, including dynamic README creation, API docs via TypeDoc, and smart commit message generation.'
 }