@promptbook/openai 0.44.0-2 → 0.44.0-21

package/README.md

@@ -1,5 +1,719 @@
-# 📖 Promptbook
+# ![Promptbook logo - cube with letters P and B](./other/design/logo-h1.png) Promptbook
 
 Library to supercharge your use of large language models
 
-[Read the manual](https://github.com/webgptorg/promptbook)
+<!--Badges-->
+<!--⚠️WARNING: This section was generated by https://github.com/hejny/batch-project-editor/blob/main/src/workflows/800-badges/badges.ts so every manual change will be overwritten.-->
+
+[![License of 📖 Prompt book](https://img.shields.io/github/license/webgptorg/promptbook.svg?style=flat)](https://github.com/webgptorg/promptbook/blob/main/LICENSE)
+[![Known Vulnerabilities](https://snyk.io/test/github/webgptorg/promptbook/badge.svg)](https://snyk.io/test/github/webgptorg/promptbook)
+[![Issues](https://img.shields.io/github/issues/webgptorg/promptbook.svg?style=flat)](https://github.com/webgptorg/promptbook/issues)
+[![Socket Badge](https://socket.dev/api/badge/npm/package/@promptbook/openai)](https://socket.dev/npm/package/@promptbook/openai)
+
+<!--/Badges-->
+
+## 📦 Package `@promptbook/openai`
+
+- Promptbooks are [divided into several](#-packages) packages, all are published from [single monorepo](https://github.com/webgptorg/promptbook).
+- This package `@promptbook/openai` is one part of the promptbook ecosystem.
+
+To install this package, run:
+
+```bash
+npm i @promptbook/openai
+```
+
+Wrapper around [OpenAI's SDK](https://www.npmjs.com/package/openai) to make it easier to use inside Promptbooks.
+
+<!--!!! Simillar wrappers-->
+
+
+---
+
+Rest of the documentation is common for entire promptbook ecosystem:
+
+<!--
+TODO: Probbably remove this section only in packages
+> ⚠ Warning: This library is still in early development.
+-->
+
+## 🤍 Whitepaper
+
+When you have a simple, single prompt for ChatGPT, GPT-4, Anthropic Claude, Google Gemini, Llama 2, or whatever, it doesn't matter how it is integrated. Whether it's the direct calling of a REST API, using the SDK, hardcoding the prompt in the source code, or importing a text file, the process remains the same.
+
+If you need something more advanced or want to extend the capabilities of LLMs, you generally have three ways to proceed:
+
+1. **Fine-tune** the model to your specifications or even train your own.
+2. **Prompt-engineer** the prompt to the best shape you can achieve.
+3. Use **multiple prompts** in a pipeline to get the best result.
+
+In any of these situations, but especially in (3), the Promptbook library can make your life easier and make **orchestraror for your prompts**.
+
+-   **Separation of concerns** between prompt engineer and programmer; between code files and prompt files; and between prompts, templates, templating pipelines, and their execution logic.
+-   Set up a **common format** for prompts that is interchangeable between projects and language/technology stacks.
+-   **Preprocessing** and cleaning the input data from the user.
+-   Use default values - **Jokers** to bypass some parts of the pipeline.
+-   **Expect** some specific output from the model.
+-   **Retry** mismatched outputs.
+-   **Combine** multiple models together.
+-   Interactive **User interaction** with the model and the user.
+-   Leverage **external** sources (like ChatGPT plugins or OpenAI's GPTs).
+-   Simplify your code to be **DRY** and not repeat all the boilerplate code for each prompt.
+-   **Versioning** of promptbooks
+-   **Reuse** parts of promptbooks in/between projects.
+-   Run the LLM **optimally** in parallel, with the best _cost/quality_ ratio or _speed/quality_ ratio.
+-   **Execution report** to see what happened during the execution.
+-   **Logging** the results of the promptbooks.
+-   _(Not ready yet)_ **Caching** calls to LLMs to save money and time.
+-   _(Not ready yet)_ Extend one prompt book from another one.
+-   _(Not ready yet)_ Leverage the **streaming** to make super cool UI/UX.
+-   _(Not ready yet)_ **A/B testing** to determine which prompt works best for the job.
+
+![WebGPT](./other/screencasts/screencast-fiabciakcmgepblmdkmemdbbkilneeeh-2023.10.26-21_46_17.gif)
+
+## 🧔 Promptbook _(for prompt-engeneers)_
+
+**P**romp**t** **b**oo**k** markdown file (**PTBK** for short, or `.ptbk.md`) is document that describes a series of prompts that are chained together to form somewhat reciepe for transforming natural language input. Inside a PTBK you can use chat prompts, completion prompts, scripting or trigger interaction with user to ask for additional information.
+
+-   Multiple promptbooks forms a library which will become a **part of your application codebase**.
+-   Theese promptbooks are designed such as they **can be written by non-programmers**.
+
+<!-- TODO: [🧠] Make some more clear escaping -->
+
+### Sample:
+
+File `write-website-content.ptbk.md`:
+
+<!------------------------[ Sample: ]------------------------>
+
+> # 🌍 Create website content
+>
+> Instructions for creating web page content.
+>
+> -   PROMPTBOOK URL https://promptbook.webgpt.com/en/write-website-content.ptbk.md@v0.1.0
+> -   PROMPTBOOK VERSION 0.0.1
+> -   INPUT  PARAM `{rawTitle}` Automatically suggested a site name or empty text
+> -   INPUT  PARAM `{rawAssigment}` Automatically generated site entry from image recognition
+> -   OUTPUT PARAM `{content}` Web content
+> -   OUTPUT PARAM `{keywords}` Keywords
+>
+> ## 👤 Specifying the assigment
+>
+> What is your web about?
+>
+> -   PROMPT DIALOG
+>
+> ```
+> {rawAssigment}
+> ```
+>
+> `-> {assigment}` Website assignment and specification
+>
+> ## ✨ Improving the title
+>
+> -   MODEL VARIANT Chat
+> -   MODEL NAME `gpt-4`
+> -   POSTPROCESSING `unwrapResult`
+>
+> ```
+> As an experienced marketing specialist, you have been entrusted with improving the name of your client's business.
+>
+> A suggested name from a client:
+> "{rawTitle}"
+>
+> Assignment from customer:
+>
+> > {assigment}
+>
+> ## Instructions:
+>
+> -   Write only one name suggestion
+> -   The name will be used on the website, business cards, visuals, etc.
+> ```
+>
+> `-> {enhancedTitle}` Enhanced title
+>
+> ## 👤 Website title approval
+>
+> Is the title for your website okay?
+>
+> -   PROMPT DIALOG
+>
+> ```
+> {enhancedTitle}
+> ```
+>
+> `-> {title}` Title for the website
+>
+> ## 🐰 Cunning subtitle
+>
+> -   MODEL VARIANT Chat
+> -   MODEL NAME `gpt-4`
+> -   POSTPROCESSING `unwrapResult`
+>
+> ```
+> As an experienced copywriter, you have been entrusted with creating a claim for the "{title}" web page.
+>
+> A website assignment from a customer:
+>
+> > {assigment}
+>
+> ## Instructions:
+>
+> -   Write only one name suggestion
+> -   Claim will be used on website, business cards, visuals, etc.
+> -   Claim should be punchy, funny, original
+> ```
+>
+> `-> {claim}` Claim for the web
+>
+> ## 🚦 Keyword analysis
+>
+> -   MODEL VARIANT Chat
+> -   MODEL NAME `gpt-4`
+>
+> ```
+> As an experienced SEO specialist, you have been entrusted with creating keywords for the website "{title}".
+>
+> Website assignment from the customer:
+>
+> > {assigment}
+>
+> ## Instructions:
+>
+> -   Write a list of keywords
+> -   Keywords are in basic form
+>
+> ## Example:
+>
+> -   Ice cream
+> -   Olomouc
+> -   Quality
+> -   Family
+> -   Tradition
+> -   Italy
+> -   Craft
+>
+> ```
+>
+> `-> {keywords}` Keywords
+>
+> ## 🔗 Combine the beginning
+>
+> -   SIMPLE TEMPLATE
+>
+> ```
+>
+> # {title}
+>
+> > {claim}
+>
+> ```
+>
+> `-> {contentBeginning}` Beginning of web content
+>
+> ## 🖋 Write the content
+>
+> -   MODEL VARIANT Completion
+> -   MODEL NAME `gpt-3.5-turbo-instruct`
+>
+> ```
+> As an experienced copywriter and web designer, you have been entrusted with creating text for a new website {title}.
+>
+> A website assignment from a customer:
+>
+> > {assigment}
+>
+> ## Instructions:
+>
+> -   Text formatting is in Markdown
+> -   Be concise and to the point
+> -   Use keywords, but they should be naturally in the text
+> -   This is the complete content of the page, so don't forget all the important information and elements the page should contain
+> -   Use headings, bullets, text formatting
+>
+> ## Keywords:
+>
+> {keywords}
+>
+> ## Web Content:
+>
+> {contentBeginning}
+> ```
+>
+> `-> {contentBody}` Middle of the web content
+>
+> ## 🔗 Combine the content
+>
+> -   SIMPLE TEMPLATE
+>
+> ```markdown
+> {contentBeginning}
+>
+> {contentBody}
+> ```
+>
+> `-> {content}`
+
+<!------------------------[ /Sample ]------------------------>
+
+Following is the scheme how the promptbook above is executed:
+
+```mermaid
+%% 🔮 Tip: Open this on GitHub or in the VSCode website to see the Mermaid graph visually
+
+flowchart LR
+  subgraph "🌍 Create website content"
+
+      direction TB
+
+      input((Input)):::input
+      templateSpecifyingTheAssigment(👤 Specifying the assigment)
+      input--"{rawAssigment}"-->templateSpecifyingTheAssigment
+      templateImprovingTheTitle(✨ Improving the title)
+      input--"{rawTitle}"-->templateImprovingTheTitle
+      templateSpecifyingTheAssigment--"{assigment}"-->templateImprovingTheTitle
+      templateWebsiteTitleApproval(👤 Website title approval)
+      templateImprovingTheTitle--"{enhancedTitle}"-->templateWebsiteTitleApproval
+      templateCunningSubtitle(🐰 Cunning subtitle)
+      templateWebsiteTitleApproval--"{title}"-->templateCunningSubtitle
+      templateSpecifyingTheAssigment--"{assigment}"-->templateCunningSubtitle
+      templateKeywordAnalysis(🚦 Keyword analysis)
+      templateWebsiteTitleApproval--"{title}"-->templateKeywordAnalysis
+      templateSpecifyingTheAssigment--"{assigment}"-->templateKeywordAnalysis
+      templateCombineTheBeginning(🔗 Combine the beginning)
+      templateWebsiteTitleApproval--"{title}"-->templateCombineTheBeginning
+      templateCunningSubtitle--"{claim}"-->templateCombineTheBeginning
+      templateWriteTheContent(🖋 Write the content)
+      templateWebsiteTitleApproval--"{title}"-->templateWriteTheContent
+      templateSpecifyingTheAssigment--"{assigment}"-->templateWriteTheContent
+      templateKeywordAnalysis--"{keywords}"-->templateWriteTheContent
+      templateCombineTheBeginning--"{contentBeginning}"-->templateWriteTheContent
+      templateCombineTheContent(🔗 Combine the content)
+      templateCombineTheBeginning--"{contentBeginning}"-->templateCombineTheContent
+      templateWriteTheContent--"{contentBody}"-->templateCombineTheContent
+
+      templateCombineTheContent--"{content}"-->output
+      output((Output)):::output
+
+      classDef input color: grey;
+      classDef output color: grey;
+
+  end;
+```
+
+[More template samples](./samples/templates/)
+
+_Note: We are using [postprocessing functions](#postprocessing-functions) like `unwrapResult` that can be used to postprocess the result._
+
+## 📦 Packages
+
+This library is divided into several packages, all are published from [single monorepo](https://github.com/webgptorg/promptbook):
+
+<!--[🔠]-->
+
+-   **[@promptbook/core](https://www.npmjs.com/package/@promptbook/core)** - Core of the library, it contains the main logic for promptbooks
+-   **[@promptbook/utils](https://www.npmjs.com/package/@promptbook/utils)** - Utility functions used in the library but also useful for individual use in preprocessing and postprocessing LLM inputs and outputs
+-   _(Not finished)_ **[@promptbook/wizzard](https://www.npmjs.com/package/@promptbook/wizzard)** - Wizard for creating+running promptbooks in single line
+-   **[@promptbook/execute-javascript](https://www.npmjs.com/package/@promptbook/execute-javascript)** - Execution tools for javascript inside promptbooks
+-   **[@promptbook/openai](https://www.npmjs.com/package/@promptbook/openai)** - Execution tools for OpenAI API, wrapper around OpenAI SDK
+-   **[@promptbook/remote-client](https://www.npmjs.com/package/@promptbook/remote-client)** - Remote client for remote execution of promptbooks
+-   **[@promptbook/remote-server](https://www.npmjs.com/package/@promptbook/remote-server)** - Remote server for remote execution of promptbooks
+-   **[@promptbook/types](https://www.npmjs.com/package/@promptbook/types)** - Just typescript types used in the library
+-   **[@promptbook/cli](https://www.npmjs.com/package/@promptbook/cli)** - Command line interface utilities for promptbooks
+
+## 📚 Dictionary
+
+The following glossary is used to clarify certain basic concepts:
+
+### Prompt
+
+Prompt in a text along with model requirements, but without any execution or templating logic.
+
+For example:
+
+```json
+{
+    "request": "Which sound does a cat make?",
+    "modelRequirements": {
+        "variant": "CHAT"
+    }
+}
+```
+
+```json
+{
+    "request": "I am a cat.\nI like to eat fish.\nI like to sleep.\nI like to play with a ball.\nI l",
+    "modelRequirements": {
+        "variant": "COMPLETION"
+    }
+}
+```
+
+### Prompt Template
+
+Similar concept to Prompt, but with templating logic.
+
+For example:
+
+```json
+{
+    "request": "Which sound does a {animalName} make?",
+    "modelRequirements": {
+        "variant": "CHAT"
+    }
+}
+```
+
+### Model Requirements
+
+Abstract way to specify the LLM.
+It does not specify the LLM with concrete version itself, only the requirements for the LLM.
+_NOT chatgpt-3.5-turbo BUT CHAT variant of GPT-3.5._
+
+For example:
+
+```json
+{
+    "variant": "CHAT",
+    "version": "GPT-3.5",
+    "temperature": 0.7
+}
+```
+
+### Execution type
+
+Each block of promptbook can have a different execution type.
+It is specified in list of requirements for the block.
+By default, it is `Prompt template`
+
+-   _(default)_ `Prompt template` The block is a prompt template and is executed by LLM (OpenAI, Azure,...)
+-   `SIMPLE TEMPLATE` The block is a simple text template which is just filled with parameters
+-   `Script` The block is a script that is executed by some script runtime, the runtime is determined by block type, currently only `javascript` is supported but we plan to add `python` and `typescript` in the future.
+-   `PROMPT DIALOG` Ask user for input
+
+### Parameters
+
+Parameters that are placed in the prompt template and replaced to create the prompt.
+It is a simple key-value object.
+
+```json
+{
+    "animalName": "cat",
+    "animalSound": "Meow!"
+}
+```
+
+There are three types of template parameters, depending on how they are used in the promptbook:
+
+-   **INPUT PARAMETER**s are required to execute the promptbook.
+-   **Intermediate parameters** are used internally in the promptbook.
+-   **OUTPUT PARAMETER**s are explicitelly marked and they are returned as the result of the promptbook execution.
+
+_Note: Parameter can be both intermedite and output at the same time._
+
+### Promptbook
+
+Promptbook is **core concept of this library**.
+It represents a series of prompt templates chained together to form a **pipeline** / one big prompt template with input and result parameters.
+
+Internally it can have multiple formats:
+
+-   **.ptbk.md file** in custom markdown format described above
+-   _(concept)_ **.ptbk** format, custom fileextension based on markdown
+-   _(internal)_ **JSON** format, parsed from the .ptbk.md file
+
+### Promptbook **Library**
+
+Library of all promptbooks used in your application.
+
+<!-- TODO: !!! Write more -->
+
+### Prompt Result
+
+Prompt result is the simplest concept of execution.
+It is the result of executing one prompt _(NOT a template)_.
+
+For example:
+
+```json
+{
+    "response": "Meow!",
+    "model": "chatgpt-3.5-turbo"
+}
+```
+
+### Execution Tools
+
+`ExecutionTools` is an interface which contains all the tools needed to execute prompts.
+It contais 3 subtools:
+
+-   `NaturalExecutionTools`
+-   `ScriptExecutionTools`
+-   `UserInterfaceTools`
+
+Which are described below:
+
+#### Natural Execution Tools
+
+`NaturalExecutionTools` is a container for all the tools needed to execute prompts to large language models like GPT-4.
+On its interface it exposes common methods for prompt execution.
+Internally it calls OpenAI, Azure, GPU, proxy, cache, logging,...
+
+`NaturalExecutionTools` an abstract interface that is implemented by concrete execution tools:
+
+-   `OpenAiExecutionTools`
+-   _(Not implemented yet)_ `AnthropicClaudeExecutionTools`
+-   _(Not implemented yet)_ `AzureOpenAiExecutionTools`
+-   _(Not implemented yet)_ `BardExecutionTools`
+-   _(Not implemented yet)_ `LamaExecutionTools`
+-   _(Not implemented yet)_ `GpuExecutionTools`
+-   And a special case are `RemoteNaturalExecutionTools` that connect to a remote server and run one of the above execution tools on that server.
+-   The second special case is `MockedEchoNaturalExecutionTools` that is used for testing and mocking.
+-   The third special case is `LogNaturalExecutionToolsWrapper` that is technically also an execution tools but it is more proxy wrapper around other execution tools that logs all calls to execution tools.
+
+#### Script Execution Tools
+
+`ScriptExecutionTools` is an abstract container that represents all the tools needed to EXECUTE SCRIPTs. It is implemented by concrete execution tools:
+
+-   `JavascriptExecutionTools` is a wrapper around `vm2` module that executes javascript code in a sandbox.
+-   `JavascriptEvalExecutionTools` is wrapper around `eval` function that executes javascript. It is used for testing and mocking **NOT intended to use in the production** due to its unsafe nature, use `JavascriptExecutionTools` instead.
+-   _(Not implemented yet)_ `TypescriptExecutionTools` executes typescript code in a sandbox.
+-   _(Not implemented yet)_ `PythonExecutionTools` executes python code in a sandbox.
+
+There are [postprocessing functions](#postprocessing-functions) that can be used to postprocess the result.
+
+#### User Interface Tools
+
+`UserInterfaceTools` is an abstract container that represents all the tools needed to interact with the user. It is implemented by concrete execution tools:
+
+-   _(Not implemented yet)_ `ConsoleInterfaceTools` is a wrapper around `readline` module that interacts with the user via console.
+-   `SimplePromptInterfaceTools` is a wrapper around `window.prompt` synchronous function that interacts with the user via browser prompt. It is used for testing and mocking **NOT intended to use in the production** due to its synchronous nature.
+-   `CallbackInterfaceTools` delagates the user interaction to a async callback function. You need to provide your own implementation of this callback function and its bind to UI. <!-- <- TODO: Provide here a way how to do it with some our plugin -->
+
+### Executor
+
+Executor is a simple async function that takes **input parameters** and returns **output parameters**.
+It is constructed by combining execution tools and promptbook to execute together.
+
+### 🃏 Jokers
+
+Joker is a previously defined parameter that is used to bypass some parts of the pipeline.
+If the joker is present in the template, it is checked to see if it meets the requirements (without postprocessing), and if so, it is used instead of executing that prompt template. There can be multiple wildcards in a prompt template, if so they are checked in order and the first one that meets the requirements is used.
+
+If none of the jokers meet the requirements, the prompt template is executed as usual.
+
+This can be useful, for example, if you want to use some predefined data, or if you want to use some data from the user, but you are not sure if it is suitable form.
+
+When using wildcards, you must have at least one minimum expectation. If you do not have a minimum expectation, the joker will always fulfil the expectation because it has none, so it makes no logical sense.
+
+Look at [jokers.ptbk.md](samples/templates/41-jokers.ptbk.md) sample.
+
+### Postprocessing functions
+
+You can define postprocessing functions when creating `JavascriptEvalExecutionTools`:
+
+```
+
+```
+
+Additionally there are some usefull string-manipulation build-in functions, which are [listed here](src/execution/plugins/script-execution-tools/javascript/JavascriptEvalExecutionTools.ts).
+
+### Expectations
+
+`Expect` command describes the desired output of the prompt template (after post-processing)
+It can set limits for the maximum/minimum length of the output, measured in characters, words, sentences, paragraphs,...
+
+_Note: LLMs work with tokens, not characters, but in Promptbooks we want to use some human-recognisable and cross-model interoperable units._
+
+```markdown
+# ✨ Sample: Expectations
+
+-   PROMPTBOOK URL https://promptbook.example.com/samples/postprocessing-2.ptbk.md@v1
+-   PROMPTBOOK VERSION 1.0.0
+-   INPUT  PARAMETER {yourName} Name of the hero
+
+## 💬 Question
+
+-   EXPECT MAX 30 CHARACTERS
+-   EXPECT MIN 2 CHARACTERS
+-   EXPECT MAX 3 WORDS
+-   EXPECT EXACTLY 1 SENTENCE
+-   EXPECT EXACTLY 1 LINE
+
+...
+```
+
+There are two types of expectations which are not strictly symmetrical:
+
+#### Minimal expectations
+
+-   `EXPECT MIN 0 ...` is not valid minimal expectation. It makes no sense.
+-   `EXPECT JSON` is both minimal and maximal expectation
+-   When you are using `JOKER` in same prompt template, you need to have at least one minimal expectation
+
+#### Maximal expectations
+
+-   `EXPECT MAX 0 ...` is valid maximal expectation. For example, you can expect 0 pages and 2 sentences.
+-   `EXPECT JSON` is both minimal and maximal expectation
+
+Look at [expectations.ptbk.md](samples/templates/45-expectations.ptbk.md) and [expect-json.ptbk.md](samples/templates/45-expect-json.ptbk.md) samples for more.
+
+<!--
+### New
+[🥻] Insert here when making new command
+-->
+
+### Execution report
+
+Execution report is a simple object or markdown that contains information about the execution of the promptbook.
+
+[See the example of such a report](/samples/templates/50-advanced.report.md)
+
+<!-- TODO: Write more -->
+
+### Remote server
+
+Remote server is a proxy server that uses its execution tools internally and exposes the executor interface externally.
+
+You can simply use `RemoteExecutionTools` on client-side javascript and connect to your remote server.
+This is useful to make all logic on browser side but not expose your API keys or no need to use customer's GPU.
+
+## 👨‍💻 Usage and integration _(for developers)_
+
+<!--
+
+TODO: [🧙‍♂️]
+
+### 🧙‍♂️ Using wizzard
+
+First you need to install this library:
+
+```bash
+npm install --save @promptbook/wizzard
+```
+
+> TODO: !! Write the Wizzard sample
+
+[Usage samples](./samples/usage/)
+
+-->
+
+### 🔌 Usage in Typescript / Javascript
+
+-   [Simple usage](./samples/usage/simple-script)
+-   [Usage with client and remote server](./samples/usage/remote)
+
+## ❔ FAQ
+
+If you have a question [start a discussion](https://github.com/webgptorg/promptbook/discussions/), [open an issue](https://github.com/webgptorg/promptbook/issues) or [write me an email](https://www.pavolhejny.com/contact).
+
+### Why not just use the OpenAI SDK / Anthropic Claude SDK / ...?
+
+Different levels of abstraction. OpenAI library is for direct use of OpenAI API. This library is for a higher level of abstraction. It is for creating prompt templates and promptbooks that are independent of the underlying library, LLM model, or even LLM provider.
+
+### How is it different from the Langchain library?
+
+Langchain is primarily aimed at ML developers working in Python. This library is for developers working in javascript/typescript and creating applications for end users.
+
+We are considering creating a bridge/converter between these two libraries.
+
+<!--
+
+==========
+Include:
+- Langchain is the python library and JavaScript is on second place
+- Langchain primarily focused on making templates, not on combining templates into larger structures
+- at the language level it distinguishes between chat and completion, I need to mix the two into one template pipeline
+- for a non-programmer it's quite hard to work with such a thing and write templates - I would much prefer a system that allows non-technical people to write templates (of which there are many more on the market than free pythonists)
+- The focus of promptbooks is primarily on building user applications, not the data processing, training or autogpt.
+-->
+
+### Promptbooks vs. OpenAI`s GPTs
+
+GPTs are chat assistants that can be assigned to specific tasks and materials. But they are still chat assistants. Promptbooks are a way to orchestrate many more predefined tasks to have much tighter control over the process. Promptbooks are not a good technology for creating human-like chatbots, GPTs are not a good technology for creating outputs with specific requirements.
+
+<!--
+TODO:!!!
+### Promptbooks vs. Semantic Kernel
+
+
+-->
+
+<!--
+TODO:
+### Promptbooks vs. Langtail
+
+
+-->
+
+<!--
+TODO:
+### Promptbooks vs. Evidentally AI
+
+Logging and monitoring
+
+-->
+
+### Where should I store my promptbooks?
+
+If you use raw SDKs, you just put prompts in the sourcecode, mixed in with typescript, javascript, python or whatever programming language you use.
+
+If you use promptbooks, you can store them in several places, each with its own advantages and disadvantages:
+
+1. As **source code**, typically git-committed. In this case you can use the versioning system and the promptbooks will be tightly coupled with the version of the application. You still get the power of promptbooks, as you separate the concerns of the prompt-engineer and the programmer.
+
+2. As data in a **database** In this case, promptbooks are like posts / articles on the blog. They can be modified independently of the application. You don't need to redeploy the application to change the promptbooks. You can have multiple versions of promptbooks for each user. You can have a web interface for non-programmers to create and modify promptbooks. But you lose the versioning system and you still have to consider the interface between the promptbooks and the application _(= input and output parameters)_.
+
+3. In a **configuration** in environment variables. This is a good way to store promptbooks if you have an application with multiple deployments and you want to have different but simple promptbooks for each deployment and you don't need to change them often.
+
+### What should I do when I need same promptbook in multiple human languages?
+
+A single promptbook can be written for several _(human)_ languages at once. However, we recommend that you have separate promptbooks for each language.
+
+In large language models, you will get better results if you have prompts in the same language as the user input.
+
+The best way to manage this is to have suffixed promptbooks like `write-website-content.en.ptbk.md` and `write-website-content.cs.ptbk.md` for each supported language.
+
+<!--
+TODO: (Maybe)
+### Why you need to explicitly specify input and output parameters?
+-->
+
+<!--
+
+
+
+!!!!
+
+
+
+unit testing
+
+escaping
+
+how i get block into prompt
+
+
+## 🚷 Limitations
+
+function calling
+system message
+iterations
+
+-->
+
+## ⌚ Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md)
+
+<!--Contributing-->
+<!--⚠️WARNING: This section was generated by https://github.com/hejny/batch-project-editor/blob/main/src/workflows/810-contributing/contributing.ts so every manual change will be overwritten.-->
+
+## 🖋️ Contributing
+
+I am open to pull requests, feedback, and suggestions. Or if you like this utility, you can [☕ buy me a coffee](https://www.buymeacoffee.com/hejny) or [donate via cryptocurrencies](https://github.com/hejny/hejny/blob/main/documents/crypto.md).
+
+You can also ⭐ star the promptbook package, [follow me on GitHub](https://github.com/hejny) or [various other social networks](https://www.pavolhejny.com/contact/).
+
+<!--/Contributing-->

package/esm/typings/_packages/utils.index.d.ts

@@ -1,6 +1,7 @@
 import { prettifyPromptbookString } from '../conversion/prettify/prettifyPromptbookString';
 import { parseNumber } from '../conversion/utils/parseNumber';
 import { assertsExecutionSuccessful } from '../execution/assertsExecutionSuccessful';
+import { checkExpectations, isPassingExpectations } from '../execution/utils/checkExpectations';
 import { replaceParameters } from '../execution/utils/replaceParameters';
 import { executionReportJsonToString } from '../types/execution-report/executionReportJsonToString';
 import { ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults } from '../types/execution-report/ExecutionReportStringOptions';
@@ -40,20 +41,21 @@ import { removeQuotes } from '../utils/removeQuotes';
 import { trimCodeBlock } from '../utils/trimCodeBlock';
 import { trimEndOfCodeBlock } from '../utils/trimEndOfCodeBlock';
 import { unwrapResult } from '../utils/unwrapResult';
-export { assertsExecutionSuccessful, executionReportJsonToString, ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults, extractAllBlocksFromMarkdown, // <- [🌻]
+export { assertsExecutionSuccessful, checkExpectations, executionReportJsonToString, ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults, extractAllBlocksFromMarkdown, // <- [🌻]
 extractAllListItemsFromMarkdown, extractBlock, // <- [🌻]
-extractOneBlockFromMarkdown, isValidJsonString, parseNumber, // <- [🌻]
+extractOneBlockFromMarkdown, isPassingExpectations, isValidJsonString, parseNumber, // <- [🌻]
 prettifyPromptbookString, removeContentComments, removeEmojis, removeMarkdownFormatting, removeQuotes, replaceParameters, trimCodeBlock, trimEndOfCodeBlock, unwrapResult, };
 export { countCharacters, countLines, countPages, countParagraphs, countSentences, CountUtils, countWords };
 export { splitIntoSentences };
 export declare const normalizeTo: {
     camelCase: typeof normalizeTo_camelCase;
     PascalCase: typeof normalizeTo_PascalCase;
-    'SCREAMING-CASE': typeof normalizeTo_SCREAMING_CASE;
+    SCREAMING_CASE: typeof normalizeTo_SCREAMING_CASE;
     snake_case: typeof normalizeTo_snake_case;
     'kebab-case': typeof normalizeToKebabCase;
 };
 export { capitalize, decapitalize, DIACRITIC_VARIANTS_LETTERS, IKeywords, isValidKeyword, nameToUriPart, nameToUriParts, normalizeTo_camelCase, normalizeTo_PascalCase, normalizeTo_SCREAMING_CASE, normalizeTo_snake_case, normalizeToKebabCase, normalizeWhitespaces, parseKeywords, parseKeywordsFromString, removeDiacritics, searchKeywords, string_keyword, };
 /**
  * TODO: [🧠] Maybe create some indipendent package like `markdown-tools` from both here exported and @private utilities
+ * Note: [🕙] It does not make sence to have simple lower / UPPER case normalization
  */

package/esm/typings/config.d.ts

@@ -2,3 +2,7 @@
  * The maximum number of iterations for a loops
  */
 export declare const LOOP_LIMIT = 1000;
+/**
+ * The maximum number of iterations for a loops which adds characters one by one
+ */
+export declare const CHARACTER_LOOP_LIMIT = 100000;

package/esm/typings/execution/plugins/natural-execution-tools/mocked/fakeTextToExpectations.d.ts

@@ -1,7 +1,15 @@
 import type { Expectations } from '../../../../types/PromptbookJson/PromptTemplateJson';
+import { PostprocessingFunction } from '../../script-execution-tools/javascript/JavascriptExecutionToolsOptions';
 /**
  * Gets the expectations and creates a fake text that meets the expectations
  *
+ * Note: You can provide postprocessing functions to modify the text before checking the expectations
+ *       The result will be the text BEFORE the postprocessing
+ *
  * @private internal util for MockedFackedNaturalExecutionTools
  */
-export declare function $fakeTextToExpectations(expectations: Expectations): string;
+export declare function $fakeTextToExpectations(expectations: Expectations, postprocessing?: Array<PostprocessingFunction>): Promise<string>;
+/**
+ * TODO: Implement better - create FakeLLM from this
+ * TODO: [💝] Unite object for expecting amount and format - use here also a format
+ */

package/esm/typings/execution/plugins/natural-execution-tools/mocked/fakeTextToExpectations.test.d.ts

@@ -0,0 +1 @@
+export {};

package/esm/typings/execution/plugins/natural-execution-tools/mocked/faked-completion.test.d.ts

@@ -0,0 +1 @@
+export {};

package/esm/typings/execution/plugins/script-execution-tools/javascript/JavascriptExecutionToolsOptions.d.ts

@@ -15,8 +15,12 @@ export type JavascriptExecutionToolsOptions = CommonExecutionToolsOptions & {
      * Note: There are also some built-in functions available:
      *      @see ./JavascriptEvalExecutionTools.ts
      */
-    functions?: Record<string_javascript_name, ((value: string) => Promisable<string>) | Function>;
+    functions?: Record<string_javascript_name, PostprocessingFunction>;
 };
+/**
+ * Function that can be used to postprocess the output of the LLM
+ */
+export type PostprocessingFunction = ((value: string) => Promisable<string>) | Function;
 /**
  * TODO: [🧠][💙] Distinct between options passed into ExecutionTools and to ExecutionTools.execute
  */

package/esm/typings/execution/plugins/script-execution-tools/javascript/utils/unknownToString.d.ts

@@ -0,0 +1,7 @@
+/**
+ * Converts anything to string that can be used for debugging and logging
+ *
+ * @param value String value for logging
+ * @private Internal util
+ */
+export declare function unknownToString(value: unknown): string;

package/esm/typings/execution/utils/checkExpectations.d.ts

@@ -0,0 +1,25 @@
+import type { Expectations } from '../../types/PromptbookJson/PromptTemplateJson';
+/**
+ * Function checkExpectations will check if the expectations on given value are met
+ *
+ * Note: There are two simmilar functions:
+ * - `checkExpectations` which throws an error if the expectations are not met
+ * - `isPassingExpectations` which returns a boolean
+ *
+ * @throws {ExpectError} if the expectations are not met
+ * @returns {void} Nothing
+ */
+export declare function checkExpectations(expectations: Expectations, value: string): void;
+/**
+ * Function checkExpectations will check if the expectations on given value are met
+ *
+ * Note: There are two simmilar functions:
+ * - `checkExpectations` which throws an error if the expectations are not met
+ * - `isPassingExpectations` which returns a boolean
+ *
+ * @returns {boolean} True if the expectations are met
+ */
+export declare function isPassingExpectations(expectations: Expectations, value: string): boolean;
+/**
+ * TODO: [💝] Unite object for expecting amount and format
+ */

package/esm/typings/execution/utils/checkExpectations.test.d.ts

@@ -0,0 +1 @@
+export {};

package/esm/typings/types/Prompt.d.ts

@@ -1,4 +1,5 @@
 import type { string_name, string_prompt, string_promptbook_url_with_hashtemplate, string_title } from '.././types/typeAliases';
+import { PostprocessingFunction } from '../execution/plugins/script-execution-tools/javascript/JavascriptExecutionToolsOptions';
 import type { ModelRequirements } from './ModelRequirements';
 import type { Expectations } from './PromptbookJson/PromptTemplateJson';
 /**
@@ -24,6 +25,10 @@ export type Prompt = {
      * Requirements for the model
      */
     readonly modelRequirements: ModelRequirements;
+    /**
+     * List of postprocessing steps that are executed after the prompt
+     */
+    readonly postprocessing?: Array<PostprocessingFunction>;
     /**
      * Expectations for the answer
      *

package/esm/typings/types/PromptbookJson/PromptTemplateJson.d.ts

@@ -121,6 +121,7 @@ interface PromptTemplateJsonCommon {
      * Expect this format of the answer
      *
      * Note: Expectations are performed after all postprocessing steps
+     * @deprecated [💝]
      */
     readonly expectFormat?: ExpectFormatCommand['format'];
     /**
@@ -130,5 +131,6 @@ interface PromptTemplateJsonCommon {
 }
 export {};
 /**
+ * TODO: [💝] Unite object for expecting amount and format - remove expectFormat
  * TODO: use one helper type> (string_prompt | string_javascript | string_markdown) & string_template
  */

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@promptbook/openai",
-    "version": "0.44.0-2",
+    "version": "0.44.0-21",
     "description": "Library to supercharge your use of large language models",
     "private": false,
     "sideEffects": false,
@@ -38,7 +38,7 @@
         "openai": "4.2.0"
     },
     "peerDependencies": {
-        "@promptbook/core": "0.44.0-2"
+        "@promptbook/core": "0.44.0-21"
     },
     "main": "./umd/index.umd.js",
     "module": "./esm/index.es.js",

package/umd/typings/_packages/utils.index.d.ts

@@ -1,6 +1,7 @@
 import { prettifyPromptbookString } from '../conversion/prettify/prettifyPromptbookString';
 import { parseNumber } from '../conversion/utils/parseNumber';
 import { assertsExecutionSuccessful } from '../execution/assertsExecutionSuccessful';
+import { checkExpectations, isPassingExpectations } from '../execution/utils/checkExpectations';
 import { replaceParameters } from '../execution/utils/replaceParameters';
 import { executionReportJsonToString } from '../types/execution-report/executionReportJsonToString';
 import { ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults } from '../types/execution-report/ExecutionReportStringOptions';
@@ -40,20 +41,21 @@ import { removeQuotes } from '../utils/removeQuotes';
 import { trimCodeBlock } from '../utils/trimCodeBlock';
 import { trimEndOfCodeBlock } from '../utils/trimEndOfCodeBlock';
 import { unwrapResult } from '../utils/unwrapResult';
-export { assertsExecutionSuccessful, executionReportJsonToString, ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults, extractAllBlocksFromMarkdown, // <- [🌻]
+export { assertsExecutionSuccessful, checkExpectations, executionReportJsonToString, ExecutionReportStringOptions, ExecutionReportStringOptionsDefaults, extractAllBlocksFromMarkdown, // <- [🌻]
 extractAllListItemsFromMarkdown, extractBlock, // <- [🌻]
-extractOneBlockFromMarkdown, isValidJsonString, parseNumber, // <- [🌻]
+extractOneBlockFromMarkdown, isPassingExpectations, isValidJsonString, parseNumber, // <- [🌻]
 prettifyPromptbookString, removeContentComments, removeEmojis, removeMarkdownFormatting, removeQuotes, replaceParameters, trimCodeBlock, trimEndOfCodeBlock, unwrapResult, };
 export { countCharacters, countLines, countPages, countParagraphs, countSentences, CountUtils, countWords };
 export { splitIntoSentences };
 export declare const normalizeTo: {
     camelCase: typeof normalizeTo_camelCase;
     PascalCase: typeof normalizeTo_PascalCase;
-    'SCREAMING-CASE': typeof normalizeTo_SCREAMING_CASE;
+    SCREAMING_CASE: typeof normalizeTo_SCREAMING_CASE;
     snake_case: typeof normalizeTo_snake_case;
     'kebab-case': typeof normalizeToKebabCase;
 };
 export { capitalize, decapitalize, DIACRITIC_VARIANTS_LETTERS, IKeywords, isValidKeyword, nameToUriPart, nameToUriParts, normalizeTo_camelCase, normalizeTo_PascalCase, normalizeTo_SCREAMING_CASE, normalizeTo_snake_case, normalizeToKebabCase, normalizeWhitespaces, parseKeywords, parseKeywordsFromString, removeDiacritics, searchKeywords, string_keyword, };
 /**
  * TODO: [🧠] Maybe create some indipendent package like `markdown-tools` from both here exported and @private utilities
+ * Note: [🕙] It does not make sence to have simple lower / UPPER case normalization
  */

package/umd/typings/config.d.ts

@@ -2,3 +2,7 @@
  * The maximum number of iterations for a loops
  */
 export declare const LOOP_LIMIT = 1000;
+/**
+ * The maximum number of iterations for a loops which adds characters one by one
+ */
+export declare const CHARACTER_LOOP_LIMIT = 100000;

package/umd/typings/execution/plugins/natural-execution-tools/mocked/fakeTextToExpectations.d.ts

@@ -1,7 +1,15 @@
 import type { Expectations } from '../../../../types/PromptbookJson/PromptTemplateJson';
+import { PostprocessingFunction } from '../../script-execution-tools/javascript/JavascriptExecutionToolsOptions';
 /**
  * Gets the expectations and creates a fake text that meets the expectations
  *
+ * Note: You can provide postprocessing functions to modify the text before checking the expectations
+ *       The result will be the text BEFORE the postprocessing
+ *
  * @private internal util for MockedFackedNaturalExecutionTools
  */
-export declare function $fakeTextToExpectations(expectations: Expectations): string;
+export declare function $fakeTextToExpectations(expectations: Expectations, postprocessing?: Array<PostprocessingFunction>): Promise<string>;
+/**
+ * TODO: Implement better - create FakeLLM from this
+ * TODO: [💝] Unite object for expecting amount and format - use here also a format
+ */

package/umd/typings/execution/plugins/natural-execution-tools/mocked/fakeTextToExpectations.test.d.ts

@@ -0,0 +1 @@
+export {};

package/umd/typings/execution/plugins/natural-execution-tools/mocked/faked-completion.test.d.ts

@@ -0,0 +1 @@
+export {};

package/umd/typings/execution/plugins/script-execution-tools/javascript/JavascriptExecutionToolsOptions.d.ts

@@ -15,8 +15,12 @@ export type JavascriptExecutionToolsOptions = CommonExecutionToolsOptions & {
      * Note: There are also some built-in functions available:
      *      @see ./JavascriptEvalExecutionTools.ts
      */
-    functions?: Record<string_javascript_name, ((value: string) => Promisable<string>) | Function>;
+    functions?: Record<string_javascript_name, PostprocessingFunction>;
 };
+/**
+ * Function that can be used to postprocess the output of the LLM
+ */
+export type PostprocessingFunction = ((value: string) => Promisable<string>) | Function;
 /**
  * TODO: [🧠][💙] Distinct between options passed into ExecutionTools and to ExecutionTools.execute
  */

package/umd/typings/execution/plugins/script-execution-tools/javascript/utils/unknownToString.d.ts

@@ -0,0 +1,7 @@
+/**
+ * Converts anything to string that can be used for debugging and logging
+ *
+ * @param value String value for logging
+ * @private Internal util
+ */
+export declare function unknownToString(value: unknown): string;

package/umd/typings/execution/utils/checkExpectations.d.ts

@@ -0,0 +1,25 @@
+import type { Expectations } from '../../types/PromptbookJson/PromptTemplateJson';
+/**
+ * Function checkExpectations will check if the expectations on given value are met
+ *
+ * Note: There are two simmilar functions:
+ * - `checkExpectations` which throws an error if the expectations are not met
+ * - `isPassingExpectations` which returns a boolean
+ *
+ * @throws {ExpectError} if the expectations are not met
+ * @returns {void} Nothing
+ */
+export declare function checkExpectations(expectations: Expectations, value: string): void;
+/**
+ * Function checkExpectations will check if the expectations on given value are met
+ *
+ * Note: There are two simmilar functions:
+ * - `checkExpectations` which throws an error if the expectations are not met
+ * - `isPassingExpectations` which returns a boolean
+ *
+ * @returns {boolean} True if the expectations are met
+ */
+export declare function isPassingExpectations(expectations: Expectations, value: string): boolean;
+/**
+ * TODO: [💝] Unite object for expecting amount and format
+ */

package/umd/typings/execution/utils/checkExpectations.test.d.ts

@@ -0,0 +1 @@
+export {};

package/umd/typings/types/Prompt.d.ts

@@ -1,4 +1,5 @@
 import type { string_name, string_prompt, string_promptbook_url_with_hashtemplate, string_title } from '.././types/typeAliases';
+import { PostprocessingFunction } from '../execution/plugins/script-execution-tools/javascript/JavascriptExecutionToolsOptions';
 import type { ModelRequirements } from './ModelRequirements';
 import type { Expectations } from './PromptbookJson/PromptTemplateJson';
 /**
@@ -24,6 +25,10 @@ export type Prompt = {
      * Requirements for the model
      */
     readonly modelRequirements: ModelRequirements;
+    /**
+     * List of postprocessing steps that are executed after the prompt
+     */
+    readonly postprocessing?: Array<PostprocessingFunction>;
     /**
      * Expectations for the answer
      *

package/umd/typings/types/PromptbookJson/PromptTemplateJson.d.ts

@@ -121,6 +121,7 @@ interface PromptTemplateJsonCommon {
      * Expect this format of the answer
      *
      * Note: Expectations are performed after all postprocessing steps
+     * @deprecated [💝]
      */
     readonly expectFormat?: ExpectFormatCommand['format'];
     /**
@@ -130,5 +131,6 @@ interface PromptTemplateJsonCommon {
 }
 export {};
 /**
+ * TODO: [💝] Unite object for expecting amount and format - remove expectFormat
  * TODO: use one helper type> (string_prompt | string_javascript | string_markdown) & string_template
  */