@promptbook/core 0.103.0-9 → 0.104.0-0

package/README.md

@@ -9,8 +9,8 @@ Turn your company's scattered knowledge into AI ready Books
 
 
 
-[![NPM Version of ![Promptbook logo - cube with letters P and B](./design/logo-h1.png) Promptbook](https://badge.fury.io/js/promptbook.svg)](https://www.npmjs.com/package/promptbook)
-[![Quality of package ![Promptbook logo - cube with letters P and B](./design/logo-h1.png) Promptbook](https://packagequality.com/shield/promptbook.svg)](https://packagequality.com/#?package=promptbook)
+[![NPM Version of ![Promptbook logo](./design/logo-h1.png) Promptbook](https://badge.fury.io/js/promptbook.svg)](https://www.npmjs.com/package/promptbook)
+[![Quality of package ![Promptbook logo](./design/logo-h1.png) Promptbook](https://packagequality.com/shield/promptbook.svg)](https://packagequality.com/#?package=promptbook)
 [![Known Vulnerabilities](https://snyk.io/test/github/webgptorg/promptbook/badge.svg)](https://snyk.io/test/github/webgptorg/promptbook)
 [![🧪 Test Books](https://github.com/webgptorg/promptbook/actions/workflows/test-books.yml/badge.svg)](https://github.com/webgptorg/promptbook/actions/workflows/test-books.yml)
 [![🧪 Test build](https://github.com/webgptorg/promptbook/actions/workflows/test-build.yml/badge.svg)](https://github.com/webgptorg/promptbook/actions/workflows/test-build.yml)
@@ -23,11 +23,7 @@ Turn your company's scattered knowledge into AI ready Books
 
 ## 🌟 New Features
 
--   🚀 **GPT-5 Support** - Now includes OpenAI's most advanced language model with unprecedented reasoning capabilities and 200K context window
--   💡 VS Code support for `.book` files with syntax highlighting and IntelliSense
--   🐳 Official Docker image (`hejny/promptbook`) for seamless containerized usage
--   🔥 Native support for OpenAI `o3-mini`, GPT-4 and other leading LLMs
--   🔍 DeepSeek integration for advanced knowledge search
+-   **Gemini 3 Support**
 
 
 
@@ -59,237 +55,260 @@ The core package serves as the foundation of the Promptbook ecosystem. It abstra
 ## 🔧 High-Level Functionality
 
 This package orchestrates the entire promptbook execution lifecycle:
-- **Pipeline Management**: Parse, validate, and compile promptbook definitions
-- **Execution Engine**: Create and manage pipeline executors with comprehensive error handling
-- **LLM Integration**: Unified interface for multiple LLM providers (OpenAI, Anthropic, Google, etc.)
-- **Parameter Processing**: Template parameter substitution and validation
-- **Knowledge Management**: Handle knowledge sources and scraping
-- **Storage Abstraction**: Flexible storage backends for caching and persistence
-- **Format Support**: Parse and validate various data formats (JSON, CSV, XML)
+
+-   **Pipeline Management**: Parse, validate, and compile promptbook definitions
+-   **Execution Engine**: Create and manage pipeline executors with comprehensive error handling
+-   **LLM Integration**: Unified interface for multiple LLM providers (OpenAI, Anthropic, Google, etc.)
+-   **Parameter Processing**: Template parameter substitution and validation
+-   **Knowledge Management**: Handle knowledge sources and scraping
+-   **Storage Abstraction**: Flexible storage backends for caching and persistence
+-   **Format Support**: Parse and validate various data formats (JSON, CSV, XML)
 
 ## ✨ Key Features
 
-- 🚀 **Universal Pipeline Executor** - Execute promptbooks with any supported LLM provider
-- 🔄 **Multi-Provider Support** - Seamlessly switch between OpenAI, Anthropic, Google, and other providers
-- 📊 **Comprehensive Validation** - Validate promptbooks, parameters, and execution results
-- 🎯 **Expectation Checking** - Built-in validation for output format, length, and content expectations
-- 🧠 **Knowledge Integration** - Scrape and process knowledge from various sources
-- 💾 **Flexible Storage** - Memory, filesystem, and custom storage backends
-- 🔧 **Error Handling** - Detailed error types for debugging and monitoring
-- 📈 **Usage Tracking** - Monitor token usage, costs, and performance metrics
-- 🎨 **Format Parsers** - Support for JSON, CSV, XML, and text formats
-- 🔀 **Pipeline Migration** - Upgrade and migrate pipeline definitions
+-   🚀 **Universal Pipeline Executor** - Execute promptbooks with any supported LLM provider
+-   🔄 **Multi-Provider Support** - Seamlessly switch between OpenAI, Anthropic, Google, and other providers
+-   📊 **Comprehensive Validation** - Validate promptbooks, parameters, and execution results
+-   🎯 **Expectation Checking** - Built-in validation for output format, length, and content expectations
+-   🧠 **Knowledge Integration** - Scrape and process knowledge from various sources
+-   💾 **Flexible Storage** - Memory, filesystem, and custom storage backends
+-   🔧 **Error Handling** - Detailed error types for debugging and monitoring
+-   📈 **Usage Tracking** - Monitor token usage, costs, and performance metrics
+-   🎨 **Format Parsers** - Support for JSON, CSV, XML, and text formats
+-   🔀 **Pipeline Migration** - Upgrade and migrate pipeline definitions
 
 ## 📦 Exported Entities
 
 ### Version Information
-- `BOOK_LANGUAGE_VERSION` - Current book language version
-- `PROMPTBOOK_ENGINE_VERSION` - Current engine version
+
+-   `BOOK_LANGUAGE_VERSION` - Current book language version
+-   `PROMPTBOOK_ENGINE_VERSION` - Current engine version
 
 ### Agent and Book Management
-- `createAgentModelRequirements` - Create model requirements for agents
-- `parseAgentSource` - Parse agent source code
-- `isValidBook` - Validate book format
-- `validateBook` - Comprehensive book validation
-- `DEFAULT_BOOK` - Default book template
+
+-   `createAgentModelRequirements` - Create model requirements for agents
+-   `parseAgentSource` - Parse agent source code
+-   `isValidBook` - Validate book format
+-   `validateBook` - Comprehensive book validation
+-   `DEFAULT_BOOK` - Default book template
 
 ### Commitment System
-- `createEmptyAgentModelRequirements` - Create empty model requirements
-- `createBasicAgentModelRequirements` - Create basic model requirements
-- `NotYetImplementedCommitmentDefinition` - Placeholder for future commitments
-- `getCommitmentDefinition` - Get specific commitment definition
-- `getAllCommitmentDefinitions` - Get all available commitment definitions
-- `getAllCommitmentTypes` - Get all commitment types
-- `isCommitmentSupported` - Check if commitment is supported
+
+-   `createEmptyAgentModelRequirements` - Create empty model requirements
+-   `createBasicAgentModelRequirements` - Create basic model requirements
+-   `NotYetImplementedCommitmentDefinition` - Placeholder for future commitments
+-   `getCommitmentDefinition` - Get specific commitment definition
+-   `getAllCommitmentDefinitions` - Get all available commitment definitions
+-   `getAllCommitmentTypes` - Get all commitment types
+-   `isCommitmentSupported` - Check if commitment is supported
 
 ### Collection Management
-- `collectionToJson` - Convert collection to JSON
-- `createCollectionFromJson` - Create collection from JSON data
-- `createCollectionFromPromise` - Create collection from async source
-- `createCollectionFromUrl` - Create collection from URL
-- `createSubcollection` - Create filtered subcollection
+
+-   `pipelineCollectionToJson` - Convert collection to JSON
+-   `createPipelineCollectionFromJson` - Create collection from JSON data
+-   `createPipelineCollectionFromPromise` - Create collection from async source
+-   `createPipelineCollectionFromUrl` - Create collection from URL
+-   `createPipelineSubcollection` - Create filtered subcollection
 
 ### Configuration Constants
-- `NAME` - Project name
-- `ADMIN_EMAIL` - Administrator email
-- `ADMIN_GITHUB_NAME` - GitHub username
-- `CLAIM` - Project claim/tagline
-- `DEFAULT_BOOK_TITLE` - Default book title
-- `DEFAULT_TASK_TITLE` - Default task title
-- `DEFAULT_PROMPT_TASK_TITLE` - Default prompt task title
-- `DEFAULT_BOOK_OUTPUT_PARAMETER_NAME` - Default output parameter name
-- `DEFAULT_MAX_FILE_SIZE` - Maximum file size limit
-- `BIG_DATASET_TRESHOLD` - Threshold for large datasets
-- `FAILED_VALUE_PLACEHOLDER` - Placeholder for failed values
-- `PENDING_VALUE_PLACEHOLDER` - Placeholder for pending values
-- `MAX_FILENAME_LENGTH` - Maximum filename length
-- `DEFAULT_INTERMEDIATE_FILES_STRATEGY` - Strategy for intermediate files
-- `DEFAULT_MAX_PARALLEL_COUNT` - Maximum parallel executions
-- `DEFAULT_MAX_EXECUTION_ATTEMPTS` - Maximum execution attempts
-- `DEFAULT_MAX_KNOWLEDGE_SOURCES_SCRAPING_DEPTH` - Knowledge scraping depth limit
-- `DEFAULT_MAX_KNOWLEDGE_SOURCES_SCRAPING_TOTAL` - Knowledge scraping total limit
-- `DEFAULT_BOOKS_DIRNAME` - Default books directory name
-- `DEFAULT_DOWNLOAD_CACHE_DIRNAME` - Default download cache directory
-- `DEFAULT_EXECUTION_CACHE_DIRNAME` - Default execution cache directory
-- `DEFAULT_SCRAPE_CACHE_DIRNAME` - Default scrape cache directory
-- `CLI_APP_ID` - CLI application identifier
-- `PLAYGROUND_APP_ID` - Playground application identifier
-- `DEFAULT_PIPELINE_COLLECTION_BASE_FILENAME` - Default collection filename
-- `DEFAULT_REMOTE_SERVER_URL` - Default remote server URL
-- `DEFAULT_CSV_SETTINGS` - Default CSV parsing settings
-- `DEFAULT_IS_VERBOSE` - Default verbosity setting
-- `SET_IS_VERBOSE` - Verbosity setter
-- `DEFAULT_IS_AUTO_INSTALLED` - Default auto-install setting
-- `DEFAULT_TASK_SIMULATED_DURATION_MS` - Default task simulation duration
-- `DEFAULT_GET_PIPELINE_COLLECTION_FUNCTION_NAME` - Default collection function name
-- `DEFAULT_MAX_REQUESTS_PER_MINUTE` - Rate limiting configuration
-- `API_REQUEST_TIMEOUT` - API request timeout
-- `PROMPTBOOK_LOGO_URL` - Official logo URL
+
+-   `NAME` - Project name
+-   `ADMIN_EMAIL` - Administrator email
+-   `ADMIN_GITHUB_NAME` - GitHub username
+-   `CLAIM` - Project claim/tagline
+-   `DEFAULT_BOOK_TITLE` - Default book title
+-   `DEFAULT_TASK_TITLE` - Default task title
+-   `DEFAULT_PROMPT_TASK_TITLE` - Default prompt task title
+-   `DEFAULT_BOOK_OUTPUT_PARAMETER_NAME` - Default output parameter name
+-   `DEFAULT_MAX_FILE_SIZE` - Maximum file size limit
+-   `BIG_DATASET_TRESHOLD` - Threshold for large datasets
+-   `FAILED_VALUE_PLACEHOLDER` - Placeholder for failed values
+-   `PENDING_VALUE_PLACEHOLDER` - Placeholder for pending values
+-   `MAX_FILENAME_LENGTH` - Maximum filename length
+-   `DEFAULT_INTERMEDIATE_FILES_STRATEGY` - Strategy for intermediate files
+-   `DEFAULT_MAX_PARALLEL_COUNT` - Maximum parallel executions
+-   `DEFAULT_MAX_EXECUTION_ATTEMPTS` - Maximum execution attempts
+-   `DEFAULT_MAX_KNOWLEDGE_SOURCES_SCRAPING_DEPTH` - Knowledge scraping depth limit
+-   `DEFAULT_MAX_KNOWLEDGE_SOURCES_SCRAPING_TOTAL` - Knowledge scraping total limit
+-   `DEFAULT_BOOKS_DIRNAME` - Default books directory name
+-   `DEFAULT_DOWNLOAD_CACHE_DIRNAME` - Default download cache directory
+-   `DEFAULT_EXECUTION_CACHE_DIRNAME` - Default execution cache directory
+-   `DEFAULT_SCRAPE_CACHE_DIRNAME` - Default scrape cache directory
+-   `CLI_APP_ID` - CLI application identifier
+-   `PLAYGROUND_APP_ID` - Playground application identifier
+-   `DEFAULT_PIPELINE_COLLECTION_BASE_FILENAME` - Default collection filename
+-   `DEFAULT_REMOTE_SERVER_URL` - Default remote server URL
+-   `DEFAULT_CSV_SETTINGS` - Default CSV parsing settings
+-   `DEFAULT_IS_VERBOSE` - Default verbosity setting
+-   `SET_IS_VERBOSE` - Verbosity setter
+-   `DEFAULT_IS_AUTO_INSTALLED` - Default auto-install setting
+-   `DEFAULT_TASK_SIMULATED_DURATION_MS` - Default task simulation duration
+-   `DEFAULT_GET_PIPELINE_COLLECTION_FUNCTION_NAME` - Default collection function name
+-   `DEFAULT_MAX_REQUESTS_PER_MINUTE` - Rate limiting configuration
+-   `API_REQUEST_TIMEOUT` - API request timeout
+-   `PROMPTBOOK_LOGO_URL` - Official logo URL
 
 ### Model and Provider Constants
-- `MODEL_TRUST_LEVELS` - Trust levels for different models
-- `MODEL_ORDERS` - Ordering preferences for models
-- `ORDER_OF_PIPELINE_JSON` - JSON property ordering
-- `RESERVED_PARAMETER_NAMES` - Reserved parameter names
+
+-   `MODEL_TRUST_LEVELS` - Trust levels for different models
+-   `MODEL_ORDERS` - Ordering preferences for models
+-   `ORDER_OF_PIPELINE_JSON` - JSON property ordering
+-   `RESERVED_PARAMETER_NAMES` - Reserved parameter names
 
 ### Pipeline Processing
-- `compilePipeline` - Compile pipeline from source
-- `parsePipeline` - Parse pipeline definition
-- `pipelineJsonToString` - Convert pipeline JSON to string
-- `prettifyPipelineString` - Format pipeline string
-- `extractParameterNamesFromTask` - Extract parameter names
-- `validatePipeline` - Validate pipeline structure
+
+-   `compilePipeline` - Compile pipeline from source
+-   `parsePipeline` - Parse pipeline definition
+-   `pipelineJsonToString` - Convert pipeline JSON to string
+-   `prettifyPipelineString` - Format pipeline string
+-   `extractParameterNamesFromTask` - Extract parameter names
+-   `validatePipeline` - Validate pipeline structure
 
 ### Dialog and Interface Tools
-- `CallbackInterfaceTools` - Callback-based interface tools
-- `CallbackInterfaceToolsOptions` - Options for callback tools (type)
+
+-   `CallbackInterfaceTools` - Callback-based interface tools
+-   `CallbackInterfaceToolsOptions` - Options for callback tools (type)
 
 ### Error Handling
-- `BoilerplateError` - Base error class
-- `PROMPTBOOK_ERRORS` - All error types registry
-- `AbstractFormatError` - Abstract format validation error
-- `AuthenticationError` - Authentication failure error
-- `CollectionError` - Collection-related error
-- `EnvironmentMismatchError` - Environment compatibility error
-- `ExpectError` - Expectation validation error
-- `KnowledgeScrapeError` - Knowledge scraping error
-- `LimitReachedError` - Resource limit error
-- `MissingToolsError` - Missing tools error
-- `NotFoundError` - Resource not found error
-- `NotYetImplementedError` - Feature not implemented error
-- `ParseError` - Parsing error
-- `PipelineExecutionError` - Pipeline execution error
-- `PipelineLogicError` - Pipeline logic error
-- `PipelineUrlError` - Pipeline URL error
-- `PromptbookFetchError` - Fetch operation error
-- `UnexpectedError` - Unexpected error
-- `WrappedError` - Wrapped error container
+
+-   `BoilerplateError` - Base error class
+-   `PROMPTBOOK_ERRORS` - All error types registry
+-   `AbstractFormatError` - Abstract format validation error
+-   `AuthenticationError` - Authentication failure error
+-   `CollectionError` - Collection-related error
+-   `EnvironmentMismatchError` - Environment compatibility error
+-   `ExpectError` - Expectation validation error
+-   `KnowledgeScrapeError` - Knowledge scraping error
+-   `LimitReachedError` - Resource limit error
+-   `MissingToolsError` - Missing tools error
+-   `NotFoundError` - Resource not found error
+-   `NotYetImplementedError` - Feature not implemented error
+-   `ParseError` - Parsing error
+-   `PipelineExecutionError` - Pipeline execution error
+-   `PipelineLogicError` - Pipeline logic error
+-   `PipelineUrlError` - Pipeline URL error
+-   `PromptbookFetchError` - Fetch operation error
+-   `UnexpectedError` - Unexpected error
+-   `WrappedError` - Wrapped error container
 
 ### Execution Engine
-- `createPipelineExecutor` - Create pipeline executor
-- `computeCosineSimilarity` - Compute cosine similarity for embeddings
-- `embeddingVectorToString` - Convert embedding vector to string
-- `executionReportJsonToString` - Convert execution report to string
-- `ExecutionReportStringOptions` - Report formatting options (type)
-- `ExecutionReportStringOptionsDefaults` - Default report options
+
+-   `createPipelineExecutor` - Create pipeline executor
+-   `computeCosineSimilarity` - Compute cosine similarity for embeddings
+-   `embeddingVectorToString` - Convert embedding vector to string
+-   `executionReportJsonToString` - Convert execution report to string
+-   `ExecutionReportStringOptions` - Report formatting options (type)
+-   `ExecutionReportStringOptionsDefaults` - Default report options
 
 ### Usage and Metrics
-- `addUsage` - Add usage metrics
-- `isPassingExpectations` - Check if expectations are met
-- `ZERO_VALUE` - Zero usage value constant
-- `UNCERTAIN_ZERO_VALUE` - Uncertain zero value constant
-- `ZERO_USAGE` - Zero usage object
-- `UNCERTAIN_USAGE` - Uncertain usage object
-- `usageToHuman` - Convert usage to human-readable format
-- `usageToWorktime` - Convert usage to work time estimate
+
+-   `addUsage` - Add usage metrics
+-   `isPassingExpectations` - Check if expectations are met
+-   `ZERO_VALUE` - Zero usage value constant
+-   `UNCERTAIN_ZERO_VALUE` - Uncertain zero value constant
+-   `ZERO_USAGE` - Zero usage object
+-   `UNCERTAIN_USAGE` - Uncertain usage object
+-   `usageToHuman` - Convert usage to human-readable format
+-   `usageToWorktime` - Convert usage to work time estimate
 
 ### Format Parsers
-- `CsvFormatError` - CSV format error
-- `CsvFormatParser` - CSV format parser
-- `MANDATORY_CSV_SETTINGS` - Required CSV settings
-- `TextFormatParser` - Text format parser
+
+-   `CsvFormatError` - CSV format error
+-   `CsvFormatParser` - CSV format parser
+-   `MANDATORY_CSV_SETTINGS` - Required CSV settings
+-   `TextFormatParser` - Text format parser
 
 ### Form Factor Definitions
-- `BoilerplateFormfactorDefinition` - Boilerplate form factor
-- `ChatbotFormfactorDefinition` - Chatbot form factor
-- `CompletionFormfactorDefinition` - Completion form factor
-- `GeneratorFormfactorDefinition` - Generator form factor
-- `GenericFormfactorDefinition` - Generic form factor
-- `ImageGeneratorFormfactorDefinition` - Image generator form factor
-- `FORMFACTOR_DEFINITIONS` - All form factor definitions
-- `MatcherFormfactorDefinition` - Matcher form factor
-- `SheetsFormfactorDefinition` - Sheets form factor
-- `TranslatorFormfactorDefinition` - Translator form factor
+
+-   `BoilerplateFormfactorDefinition` - Boilerplate form factor
+-   `ChatbotFormfactorDefinition` - Chatbot form factor
+-   `CompletionFormfactorDefinition` - Completion form factor
+-   `GeneratorFormfactorDefinition` - Generator form factor
+-   `GenericFormfactorDefinition` - Generic form factor
+-   `ImageGeneratorFormfactorDefinition` - Image generator form factor
+-   `FORMFACTOR_DEFINITIONS` - All form factor definitions
+-   `MatcherFormfactorDefinition` - Matcher form factor
+-   `SheetsFormfactorDefinition` - Sheets form factor
+-   `TranslatorFormfactorDefinition` - Translator form factor
 
 ### LLM Provider Integration
-- `filterModels` - Filter available models
-- `$llmToolsMetadataRegister` - LLM tools metadata registry
-- `$llmToolsRegister` - LLM tools registry
-- `createLlmToolsFromConfiguration` - Create tools from config
-- `cacheLlmTools` - Cache LLM tools
-- `countUsage` - Count total usage
-- `limitTotalUsage` - Limit total usage
-- `joinLlmExecutionTools` - Join multiple LLM tools
-- `MultipleLlmExecutionTools` - Multiple LLM tools container
+
+-   `filterModels` - Filter available models
+-   `$llmToolsMetadataRegister` - LLM tools metadata registry
+-   `$llmToolsRegister` - LLM tools registry
+-   `createLlmToolsFromConfiguration` - Create tools from config
+-   `cacheLlmTools` - Cache LLM tools
+-   `countUsage` - Count total usage
+-   `limitTotalUsage` - Limit total usage
+-   `joinLlmExecutionTools` - Join multiple LLM tools
+-   `MultipleLlmExecutionTools` - Multiple LLM tools container
 
 ### Provider Registrations
-- `_AnthropicClaudeMetadataRegistration` - Anthropic Claude registration
-- `_AzureOpenAiMetadataRegistration` - Azure OpenAI registration
-- `_DeepseekMetadataRegistration` - Deepseek registration
-- `_GoogleMetadataRegistration` - Google registration
-- `_OllamaMetadataRegistration` - Ollama registration
-- `_OpenAiMetadataRegistration` - OpenAI registration
-- `_OpenAiAssistantMetadataRegistration` - OpenAI Assistant registration
-- `_OpenAiCompatibleMetadataRegistration` - OpenAI Compatible registration
+
+-   `_AnthropicClaudeMetadataRegistration` - Anthropic Claude registration
+-   `_AzureOpenAiMetadataRegistration` - Azure OpenAI registration
+-   `_DeepseekMetadataRegistration` - Deepseek registration
+-   `_GoogleMetadataRegistration` - Google registration
+-   `_OllamaMetadataRegistration` - Ollama registration
+-   `_OpenAiMetadataRegistration` - OpenAI registration
+-   `_OpenAiAssistantMetadataRegistration` - OpenAI Assistant registration
+-   `_OpenAiCompatibleMetadataRegistration` - OpenAI Compatible registration
 
 ### Pipeline Management
-- `migratePipeline` - Migrate pipeline to newer version
-- `preparePersona` - Prepare persona for execution
-- `book` - Book notation utilities
-- `isValidPipelineString` - Validate pipeline string
-- `GENERIC_PIPELINE_INTERFACE` - Generic pipeline interface
-- `getPipelineInterface` - Get pipeline interface
-- `isPipelineImplementingInterface` - Check interface implementation
-- `isPipelineInterfacesEqual` - Compare pipeline interfaces
-- `EXPECTATION_UNITS` - Units for expectations
-- `validatePipelineString` - Validate pipeline string format
+
+-   `migratePipeline` - Migrate pipeline to newer version
+-   `preparePersona` - Prepare persona for execution
+-   `book` - Book notation utilities
+-   `isValidPipelineString` - Validate pipeline string
+-   `GENERIC_PIPELINE_INTERFACE` - Generic pipeline interface
+-   `getPipelineInterface` - Get pipeline interface
+-   `isPipelineImplementingInterface` - Check interface implementation
+-   `isPipelineInterfacesEqual` - Compare pipeline interfaces
+-   `EXPECTATION_UNITS` - Units for expectations
+-   `validatePipelineString` - Validate pipeline string format
 
 ### Pipeline Preparation
-- `isPipelinePrepared` - Check if pipeline is prepared
-- `preparePipeline` - Prepare pipeline for execution
-- `unpreparePipeline` - Unprepare pipeline
+
+-   `isPipelinePrepared` - Check if pipeline is prepared
+-   `preparePipeline` - Prepare pipeline for execution
+-   `unpreparePipeline` - Unprepare pipeline
 
 ### Remote Server Integration
-- `identificationToPromptbookToken` - Convert ID to token
-- `promptbookTokenToIdentification` - Convert token to ID
+
+-   `identificationToPromptbookToken` - Convert ID to token
+-   `promptbookTokenToIdentification` - Convert token to ID
 
 ### Knowledge Scraping
-- `_BoilerplateScraperMetadataRegistration` - Boilerplate scraper registration
-- `prepareKnowledgePieces` - Prepare knowledge pieces
-- `$scrapersMetadataRegister` - Scrapers metadata registry
-- `$scrapersRegister` - Scrapers registry
-- `makeKnowledgeSourceHandler` - Create knowledge source handler
-- `promptbookFetch` - Fetch with promptbook context
-- `_LegacyDocumentScraperMetadataRegistration` - Legacy document scraper
-- `_DocumentScraperMetadataRegistration` - Document scraper registration
-- `_MarkdownScraperMetadataRegistration` - Markdown scraper registration
-- `_MarkitdownScraperMetadataRegistration` - Markitdown scraper registration
-- `_PdfScraperMetadataRegistration` - PDF scraper registration
-- `_WebsiteScraperMetadataRegistration` - Website scraper registration
+
+-   `_BoilerplateScraperMetadataRegistration` - Boilerplate scraper registration
+-   `prepareKnowledgePieces` - Prepare knowledge pieces
+-   `$scrapersMetadataRegister` - Scrapers metadata registry
+-   `$scrapersRegister` - Scrapers registry
+-   `makeKnowledgeSourceHandler` - Create knowledge source handler
+-   `promptbookFetch` - Fetch with promptbook context
+-   `_LegacyDocumentScraperMetadataRegistration` - Legacy document scraper
+-   `_DocumentScraperMetadataRegistration` - Document scraper registration
+-   `_MarkdownScraperMetadataRegistration` - Markdown scraper registration
+-   `_MarkitdownScraperMetadataRegistration` - Markitdown scraper registration
+-   `_PdfScraperMetadataRegistration` - PDF scraper registration
+-   `_WebsiteScraperMetadataRegistration` - Website scraper registration
 
 ### Storage Backends
-- `BlackholeStorage` - Blackhole storage (discards data)
-- `MemoryStorage` - In-memory storage
-- `PrefixStorage` - Prefixed storage wrapper
+
+-   `BlackholeStorage` - Blackhole storage (discards data)
+-   `MemoryStorage` - In-memory storage
+-   `PrefixStorage` - Prefixed storage wrapper
 
 ### Type Definitions
-- `MODEL_VARIANTS` - Available model variants
-- `NonTaskSectionTypes` - Non-task section types
-- `SectionTypes` - All section types
-- `TaskTypes` - Task types
+
+-   `MODEL_VARIANTS` - Available model variants
+-   `NonTaskSectionTypes` - Non-task section types
+-   `SectionTypes` - All section types
+-   `TaskTypes` - Task types
 
 ### Server Configuration
-- `REMOTE_SERVER_URLS` - Remote server URLs
+
+-   `REMOTE_SERVER_URLS` - Remote server URLs
 
 > 💡 This package does not make sense on its own, look at [all promptbook packages](#-packages) or just install all by `npm i ptbk`
 
@@ -309,11 +328,15 @@ The main challenge is to narrow it down, constrain it, set the proper **context,
 
 Promptbook takes the best from both worlds. You are defining your AI behavior by simple **books**, which are very explicit. They are automatically enforced, but they are very easy to understand, very easy to write, and very reliable and portable.
 
+<table style="border: 1px solid #777; border-radius: 10px;"><tr><td>
 
-<img
-    alt="Paul Smith & Associés Book"
-    src="https://promptbook.studio/embed/book-preview.png?book=Paul%20Smith%20%26%20Associ%C3%A9s%0A%20%20%20%20%20%20%7C%20PERSONA%20You%20are%20a%20company%20lawyer.%0A%20%20%20%20%20%20%7C%20Your%20job%20is%20to%20provide%20legal%20advice%20and%20support%20to%20the%20company%20and%20its%20employees.%0A%20%20%20%20%20%20%7C%20You%20are%20knowledgeable%2C%20professional%2C%20and%20detail-oriented.&width=800&height=450&nonce=0"
-/>
+**<ins>Paul Smith & Associés</ins>**<br/>
+<br/>
+**PERSONA** You are a company lawyer.<br/>
+Your job is to provide legal advice and support to the company and its employees.<br/>
+You are knowledgeable, professional, and detail-oriented.<br/>
+
+</td></tr></table>
 
 <div style="page-break-after: always;"></div>
 
@@ -327,11 +350,15 @@ You can look at it as prompting (or writing a system message), but decorated by
 
 Personas define the character of your AI persona, its role, and how it should interact with users. It sets the tone and style of communication.
 
+<table style="border: 1px solid #777; border-radius: 10px;"><tr><td>
+
+**<ins>Paul Smith & Associés</ins>**<br/>
+<br/>
+**PERSONA** You are a company lawyer.<br/>
+Your job is to provide legal advice and support to the company and its employees.<br/>
+You are knowledgeable, professional, and detail-oriented.<br/>
 
-<img
-    alt="Paul Smith & Associés Book"
-    src="https://promptbook.studio/embed/book-preview.png?book=Paul%20Smith%20%26%20Associ%C3%A9s%0A%20%20%20%20%20%20%7C%20PERSONA%20You%20are%20a%20company%20lawyer.%0A%20%20%20%20%20%20%7C%20Your%20job%20is%20to%20provide%20legal%20advice%20and%20support%20to%20the%20company%20and%20its%20employees.%0A%20%20%20%20%20%20%7C%20You%20are%20knowledgeable%2C%20professional%2C%20and%20detail-oriented.&width=800&height=450&nonce=0"
-/>
+</td></tr></table>
 
 #### `Knowledge` commitment
 
@@ -341,11 +368,18 @@ This can include domain-specific knowledge, company policies, or any other relev
 
 Promptbook Engine will automatically enforce this knowledge during interactions. When the knowledge is short enough, it will be included in the prompt. When it is too long, it will be stored in vector databases and RAG retrieved when needed. But you don't need to care about it.
 
+<table style="border: 1px solid #777; border-radius: 10px;"><tr><td>
 
-<img
-    alt="Paul Smith & Associés Book"
-    src="https://promptbook.studio/embed/book-preview.png?book=Paul%20Smith%20%26%20Associ%C3%A9s%0A%20%20%20%20%20%20%7C%20PERSONA%20You%20are%20a%20company%20lawyer.%0A%20%20%20%20%20%20%7C%20Your%20job%20is%20to%20provide%20legal%20advice%20and%20support%20to%20the%20company%20and%20its%20employees.%0A%20%20%20%20%20%20%7C%20You%20are%20knowledgeable%2C%20professional%2C%20and%20detail-oriented.%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20%20https%3A%2F%2Fcompany.com%2Fcompany-policies.pdf%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20https%3A%2F%2Fcompany.com%2Finternal-documents%2Femployee-handbook.docx&width=800&height=450&nonce=0"
-/>
+**<ins>Paul Smith & Associés</ins>**<br/>
+<br/>
+**PERSONA** You are a company lawyer.<br/>
+Your job is to provide legal advice and support to the company and its employees.<br/>
+You are knowledgeable, professional, and detail-oriented.<br/>
+<br/>
+**KNOWLEDGE** https://company.com/company-policies.pdf<br/>
+**KNOWLEDGE** https://company.com/internal-documents/employee-handbook.docx<br/>
+
+</td></tr></table>
 
 #### `Rule` commitment
 
@@ -353,21 +387,42 @@ Rules will enforce specific behaviors or constraints on the AI's responses. This
 
 Depending on rule strictness, Promptbook will either propagate it to the prompt or use other techniques, like adversary agent, to enforce it.
 
+<table style="border: 1px solid #777; border-radius: 10px;"><tr><td>
+
+**<ins>Paul Smith & Associés</ins>**<br/>
+<br/>
+**PERSONA** You are a company lawyer.<br/>
+Your job is to provide legal advice and support to the company and its employees.<br/>
+You are knowledgeable, professional, and detail-oriented.<br/>
+<br/>
+**RULE** Always ensure compliance with laws and regulations.<br/>
+**RULE** Never provide legal advice outside your area of expertise.<br/>
+**RULE** Never provide legal advice about criminal law.<br/>
+**KNOWLEDGE** https://company.com/company-policies.pdf<br/>
+**KNOWLEDGE** https://company.com/internal-documents/employee-handbook.docx<br/>
 
-<img
-    alt="Paul Smith & Associés Book"
-    src="https://promptbook.studio/embed/book-preview.png?book=Paul%20Smith%20%26%20Associ%C3%A9s%0A%20%20%20%20%20%20%7C%20PERSONA%20You%20are%20a%20company%20lawyer.%0A%20%20%20%20%20%20%7C%20Your%20job%20is%20to%20provide%20legal%20advice%20and%20support%20to%20the%20company%20and%20its%20employees.%0A%20%20%20%20%20%20%7C%20You%20are%20knowledgeable%2C%20professional%2C%20and%20detail-oriented.%0A%20%20%20%20%20%20%7C%20RULE%20Always%20ensure%20compliance%20with%20laws%20and%20regulations.%0A%20%20%20%20%20%20%7C%20RULE%20Never%20provide%20legal%20advice%20outside%20your%20area%20of%20expertise.%0A%20%20%20%20%20%20%7C%20RULE%20Never%20provide%20legal%20advice%20about%20criminal%20law.%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20%20https%3A%2F%2Fcompany.com%2Fcompany-policies.pdf%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20https%3A%2F%2Fcompany.com%2Finternal-documents%2Femployee-handbook.docx&width=800&height=450&nonce=0"
-/>
+</td></tr></table>
 
 #### `Action` commitment
 
 Action Commitment allows you to define specific actions that the AI can take during interactions. This can include things like posting on a social media platform, sending emails, creating calendar events, or interacting with your internal systems.
 
+<table style="border: 1px solid #777; border-radius: 10px;"><tr><td>
 
-<img
-    alt="Paul Smith & Associés Book"
-    src="https://promptbook.studio/embed/book-preview.png?book=Paul%20Smith%20%26%20Associ%C3%A9s%0A%20%20%20%20%20%20%7C%20PERSONA%20You%20are%20a%20company%20lawyer.%0A%20%20%20%20%20%20%7C%20Your%20job%20is%20to%20provide%20legal%20advice%20and%20support%20to%20the%20company%20and%20its%20employees.%0A%20%20%20%20%20%20%7C%20You%20are%20knowledgeable%2C%20professional%2C%20and%20detail-oriented.%0A%20%20%20%20%20%20%7C%20RULE%20Always%20ensure%20compliance%20with%20laws%20and%20regulations.%0A%20%20%20%20%20%20%7C%20RULE%20Never%20provide%20legal%20advice%20outside%20your%20area%20of%20expertise.%0A%20%20%20%20%20%20%7C%20RULE%20Never%20provide%20legal%20advice%20about%20criminal%20law.%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20%20https%3A%2F%2Fcompany.com%2Fcompany-policies.pdf%0A%20%20%20%20%20%20%7C%20KNOWLEDGE%20https%3A%2F%2Fcompany.com%2Finternal-documents%2Femployee-handbook.docx%0A%20%20%20%20%20%20%7C%20ACTION%20When%20a%20user%20asks%20about%20an%20issue%20that%20could%20be%20treated%20as%20a%20crime%2C%20notify%20legal%40company.com.&width=800&height=450&nonce=0"
-/>
+**<ins>Paul Smith & Associés</ins>**<br/>
+<br/>
+**PERSONA** You are a company lawyer.<br/>
+Your job is to provide legal advice and support to the company and its employees.<br/>
+You are knowledgeable, professional, and detail-oriented.<br/>
+<br/>
+**RULE** Always ensure compliance with laws and regulations.<br/>
+**RULE** Never provide legal advice outside your area of expertise.<br/>
+**RULE** Never provide legal advice about criminal law.<br/>
+**KNOWLEDGE** https://company.com/company-policies.pdf<br/>
+**KNOWLEDGE** https://company.com/internal-documents/employee-handbook.docx<br/>
+**ACTION** When a user asks about an issue that could be treated as a crime, notify legal@company.com.<br/>
+
+</td></tr></table>
 
 [Read more about the language](./BLUEPRINT.md)
 
@@ -414,14 +469,10 @@ Now you want to use it. There are several ways how to write your first book:
 
 We have written ai asistant in book who can help you with writing your first book.
 
-
-
 #### Your AI twin
 
 Copy your own behavior, personality, and knowledge into book and create your AI twin. It can help you with your work, personal life, or any other task.
 
-
-
 #### AI persona workpool
 
 Or you can pick from our library of pre-written books for various roles and tasks. You can find books for customer support, coding, marketing, sales, HR, legal, and many other roles.
@@ -541,8 +592,6 @@ Join our growing community of developers and users:
 
 
 
-
-
 ## 📚 Documentation
 
 See detailed guides and API reference in the [docs](https://github.com/webgptorg/promptbook/discussions/categories/concepts) or [online](https://discord.gg/x3QWNaa89N).
@@ -619,8 +668,6 @@ The following glossary is used to clarify certain concepts:
 
 _Note: This section is not a complete dictionary, more list of general AI / LLM terms that has connection with Promptbook_
 
-
-
 ### 💯 Core concepts
 
 -   [📚 Collection of pipelines](https://github.com/webgptorg/promptbook/discussions/65)