promptbook 0.72.0 → 0.74.0-0

package/README.md

@@ -18,6 +18,8 @@ Build responsible, controlled and transparent applications on top of LLM models!
 
 ## ✨ New Features
 
+-   💙 Working on [the **Book** language v1](https://github.com/webgptorg/book)
+-   📚 Support of `.docx`, `.doc` and `.pdf` documents
 -   ✨ **Support of [OpenAI o1 model](https://openai.com/o1/)**
 
 
@@ -30,7 +32,7 @@ Build responsible, controlled and transparent applications on top of LLM models!
 
 If you have a simple, single prompt for ChatGPT, GPT-4, Anthropic Claude, Google Gemini, Llama 3, or whatever, it doesn't matter how you integrate it. Whether it's calling a REST API directly, using the SDK, hardcoding the prompt into the source code, or importing a text file, the process remains the same.
 
-But often you will struggle with the **limitations of LLMs**, such as **hallucinations, off-topic responses, poor quality output, language and prompt drift, word repetition repetition repetition repetition or misuse, lack of context, or just plain w𝒆𝐢rd responses**. When this happens, you generally have three options:
+But often you will struggle with the **limitations of LLMs**, such as **hallucinations, off-topic responses, poor quality output, language and prompt drift, word repetition repetition repetition repetition or misuse, lack of context, or just plain w𝒆𝐢rd resp0nses**. When this happens, you generally have three options:
 
 1. **Fine-tune** the model to your specifications or even train your own.
 2. **Prompt-engineer** the prompt to the best shape you can achieve.
@@ -38,248 +40,87 @@ But often you will struggle with the **limitations of LLMs**, such as **hallucin
 
 In all of these situations, but especially in 3., the **✨ Promptbook can make your life waaaaaaaaaay easier**.
 
--   [**Separates concerns**](https://github.com/webgptorg/promptbook/discussions/32) between prompt-engineer and programmer, between code files and prompt files, and between prompts and their execution logic.
--   Establishes a [**common format `.ptbk.md`**](https://github.com/webgptorg/promptbook/discussions/85) that can be used to describe your prompt business logic without having to write code or deal with the technicalities of LLMs.
--   **Forget** about **low-level details** like choosing the right model, tokens, context size, temperature, top-k, top-p, or kernel sampling. **Just write your intent** and [**persona**](https://github.com/webgptorg/promptbook/discussions/22) who should be responsible for the task and let the library do the rest.
--   Has built-in **orchestration** of [pipeline](https://github.com/webgptorg/promptbook/discussions/64) execution and many tools to make the process easier, more reliable, and more efficient, such as caching, [compilation+preparation](https://github.com/webgptorg/promptbook/discussions/78), [just-in-time fine-tuning](https://github.com/webgptorg/promptbook/discussions/33), [expectation-aware generation](https://github.com/webgptorg/promptbook/discussions/37), [agent adversary expectations](https://github.com/webgptorg/promptbook/discussions/39), and more.
+-   [**Separates concerns**](https://github.com/webgptorg/promptbook/discussions/32) between prompt-engineer and programmer, between code files and prompt files, and between prompts and their execution logic. For this purpose, it introduces a new language called [the **💙 Book**](https://github.com/webgptorg/book).
+-   Book allows you to **focus on the business** logic without having to write code or deal with the technicalities of LLMs.
+-   **Forget** about **low-level details** like choosing the right model, tokens, context size, `temperature`, `top-k`, `top-p`, or kernel sampling. **Just write your intent** and [**persona**](https://github.com/webgptorg/promptbook/discussions/22) who should be responsible for the task and let the library do the rest.
+-   We have built-in **orchestration** of [pipeline](https://github.com/webgptorg/promptbook/discussions/64) execution and many tools to make the process easier, more reliable, and more efficient, such as caching, [compilation+preparation](https://github.com/webgptorg/promptbook/discussions/78), [just-in-time fine-tuning](https://github.com/webgptorg/promptbook/discussions/33), [expectation-aware generation](https://github.com/webgptorg/promptbook/discussions/37), [agent adversary expectations](https://github.com/webgptorg/promptbook/discussions/39), and more.
 -   Sometimes even the best prompts with the best framework like Promptbook `:)` can't avoid the problems. In this case, the library has built-in **[anomaly detection](https://github.com/webgptorg/promptbook/discussions/40) and logging** to help you find and fix the problems.
--   Promptbook has built in versioning. You can test multiple **A/B versions** of pipelines and see which one works best.
--   Promptbook is designed to do [**RAG** (Retrieval-Augmented Generation)](https://github.com/webgptorg/promptbook/discussions/41) and other advanced techniques. You can use **knowledge** to improve the quality of the output.
-
-
-
-## 🧔 Pipeline _(for prompt-engeneers)_
-
-**P**romp**t** **b**oo**k** markdown file (or `.ptbk.md` file) is document that describes a **pipeline** - a series of prompts that are chained together to form somewhat reciepe for transforming natural language input.
-
--   Multiple pipelines forms a **collection** which will handle core **know-how of your LLM application**.
--   Theese pipelines are designed such as they **can be written by non-programmers**.
-
-
-
-### Sample:
-
-File `write-website-content.ptbk.md`:
-
-
-
-
-
-> # 🌍 Create website content
->
-> Instructions for creating web page content.
->
-> -   PIPELINE URL https://promptbook.studio/webgpt/write-website-content.ptbk.md
-> -   INPUT  PARAM `{rawTitle}` Automatically suggested a site name or empty text
-> -   INPUT  PARAM `{rawAssigment}` Automatically generated site entry from image recognition
-> -   OUTPUT PARAM `{websiteContent}` Web content
-> -   OUTPUT PARAM `{keywords}` Keywords
->
-> ## 👤 Specifying the assigment
->
-> What is your web about?
->
-> -   DIALOG TEMPLATE
->
-> ```
-> {rawAssigment}
-> ```
->
-> `-> {assigment}` Website assignment and specification
->
-> ## ✨ Improving the title
->
-> -   PERSONA Jane, Copywriter and Marketing Specialist.
->
-> ```
-> As an experienced marketing specialist, you have been entrusted with improving the name of your client's business.
->
-> A suggested name from a client:
-> "{rawTitle}"
->
-> Assignment from customer:
->
-> > {assigment}
->
-> ## Instructions:
->
-> -   Write only one name suggestion
-> -   The name will be used on the website, business cards, visuals, etc.
-> ```
->
-> `-> {enhancedTitle}` Enhanced title
->
-> ## 👤 Website title approval
->
-> Is the title for your website okay?
->
-> -   DIALOG TEMPLATE
->
-> ```
-> {enhancedTitle}
-> ```
->
-> `-> {title}` Title for the website
->
-> ## 🐰 Cunning subtitle
->
-> -   PERSONA Josh, a copywriter, tasked with creating a claim for the website.
->
-> ```
-> As an experienced copywriter, you have been entrusted with creating a claim for the "{title}" web page.
->
-> A website assignment from a customer:
->
-> > {assigment}
->
-> ## Instructions:
->
-> -   Write only one name suggestion
-> -   Claim will be used on website, business cards, visuals, etc.
-> -   Claim should be punchy, funny, original
-> ```
->
-> `-> {claim}` Claim for the web
->
-> ## 🚦 Keyword analysis
->
-> -   PERSONA Paul, extremely creative SEO specialist.
->
-> ```
-> As an experienced SEO specialist, you have been entrusted with creating keywords for the website "{title}".
->
-> Website assignment from the customer:
->
-> > {assigment}
->
-> ## Instructions:
->
-> -   Write a list of keywords
-> -   Keywords are in basic form
->
-> ## Example:
->
-> -   Ice cream
-> -   Olomouc
-> -   Quality
-> -   Family
-> -   Tradition
-> -   Italy
-> -   Craft
->
-> ```
->
-> `-> {keywords}` Keywords
->
-> ## 🔗 Combine the beginning
->
-> -   SIMPLE TEMPLATE
->
-> ```
->
-> # {title}
->
-> > {claim}
->
-> ```
->
-> `-> {contentBeginning}` Beginning of web content
->
-> ## 🖋 Write the content
->
-> -   PERSONA Jane
->
-> ```
-> As an experienced copywriter and web designer, you have been entrusted with creating text for a new website {title}.
->
-> A website assignment from a customer:
->
-> > {assigment}
->
-> ## Instructions:
->
-> -   Text formatting is in Markdown
-> -   Be concise and to the point
-> -   Use keywords, but they should be naturally in the text
-> -   This is the complete content of the page, so don't forget all the important information and elements the page should contain
-> -   Use headings, bullets, text formatting
->
-> ## Keywords:
->
-> {keywords}
->
-> ## Web Content:
->
-> {contentBeginning}
-> ```
->
-> `-> {contentBody}` Middle of the web content
->
-> ## 🔗 Combine the content
->
-> -   SIMPLE TEMPLATE
->
-> ```markdown
-> {contentBeginning}
->
-> {contentBody}
-> ```
->
-> `-> {websiteContent}`
-
-
-
-Following is the scheme how the promptbook above is executed:
-
-```mermaid
-%% 🔮 Tip: Open this on GitHub or in the VSCode website to see the Mermaid graph visually
-
-flowchart LR
-  subgraph "🌍 Create website content"
-
-      direction TB
-
-      input((Input)):::input
-      templateSpecifyingTheAssigment(👤 Specifying the assigment)
-      input--"{rawAssigment}"-->templateSpecifyingTheAssigment
-      templateImprovingTheTitle(✨ Improving the title)
-      input--"{rawTitle}"-->templateImprovingTheTitle
-      templateSpecifyingTheAssigment--"{assigment}"-->templateImprovingTheTitle
-      templateWebsiteTitleApproval(👤 Website title approval)
-      templateImprovingTheTitle--"{enhancedTitle}"-->templateWebsiteTitleApproval
-      templateCunningSubtitle(🐰 Cunning subtitle)
-      templateWebsiteTitleApproval--"{title}"-->templateCunningSubtitle
-      templateSpecifyingTheAssigment--"{assigment}"-->templateCunningSubtitle
-      templateKeywordAnalysis(🚦 Keyword analysis)
-      templateWebsiteTitleApproval--"{title}"-->templateKeywordAnalysis
-      templateSpecifyingTheAssigment--"{assigment}"-->templateKeywordAnalysis
-      templateCombineTheBeginning(🔗 Combine the beginning)
-      templateWebsiteTitleApproval--"{title}"-->templateCombineTheBeginning
-      templateCunningSubtitle--"{claim}"-->templateCombineTheBeginning
-      templateWriteTheContent(🖋 Write the content)
-      templateWebsiteTitleApproval--"{title}"-->templateWriteTheContent
-      templateSpecifyingTheAssigment--"{assigment}"-->templateWriteTheContent
-      templateKeywordAnalysis--"{keywords}"-->templateWriteTheContent
-      templateCombineTheBeginning--"{contentBeginning}"-->templateWriteTheContent
-      templateCombineTheContent(🔗 Combine the content)
-      templateCombineTheBeginning--"{contentBeginning}"-->templateCombineTheContent
-      templateWriteTheContent--"{contentBody}"-->templateCombineTheContent
-
-      templateCombineTheContent--"{websiteContent}"-->output
-      output((Output)):::output
-
-      classDef input color: grey;
-      classDef output color: grey;
-
-  end;
-```
+-   Versioning is build in. You can test multiple **A/B versions** of pipelines and see which one works best.
+-   Promptbook is designed to use [**RAG** (Retrieval-Augmented Generation)](https://github.com/webgptorg/promptbook/discussions/41) and other advanced techniques to bring the context of your business to generic LLM. You can use **knowledge** to improve the quality of the output.
+
+
+
+## 💜 The Promptbook Project
+
+
+
+<table>
+  <tbody>
+    <tr>
+      <td>Promptbook whitepaper</td>
+      <td>Basic motivations and problems which we are trying to solve</td>
+      <td rowspan=3>https://github.com/webgptorg/book</td>
+    </tr>
+    <tr>
+      <td>Promptbook <i>(system)</i></td>
+      <td>Promptbook ...</td>
+    </tr>
+    <tr>
+      <td>Book language</td>
+      <td>
+          Book is a markdown-like language to define projects, pipelines, knowledge,... in the Promptbook system. It is designed to be understandable by non-programmers and non-technical people
+      </td>
+    </tr>
+    <tr>
+      <td>Promptbook typescript project</td>
+      <td>Implementation of Promptbook in TypeScript published into multiple packages to NPM</td>
+      <td>https://github.com/webgptorg/promptbook</td>
+    </tr>
+    <tr>
+      <td>Promptbook studio</td>
+      <td>Promptbook studio</td>
+      <td rowspan=2>https://github.com/hejny/promptbook-studio</td>
+    </tr>
+    <tr>
+      <td>Promptbook miniapps</td>
+      <td>Promptbook miniapps</td>
+    </tr>
+  </tbody>
+</table>
+
+## 💙 Book language _(for prompt-engineer)_
+
+Promptbook [pipelines](https://github.com/webgptorg/promptbook/discussions/64) are written in markdown-like language called [Book](https://github.com/webgptorg/book). It is designed to be understandable by non-programmers and non-technical people.
+
 
--   [More template samples](./samples/pipelines/)
--   [Read more about `.ptbk.md` file format here](https://github.com/webgptorg/promptbook/discussions/categories/concepts?discussions_q=is%3Aopen+label%3A.ptbk.md+category%3AConcepts)
 
-_Note: We are using [postprocessing functions](#postprocessing-functions) like `unwrapResult` that can be used to postprocess the result._
+```markdown
+# 🌟 My first Book
 
-## 📦 Packages
+-   INPUT PARAMETER {subject}
+-   OUTPUT PARAMETER {article}
+
+## Sample subject
+
+> Promptbook
+
+-> {subject}
+
+## Write an article
+
+-   PERSONA Jane, marketing specialist with prior experience in writing articles about technology and artificial intelligence
+-   KNOWLEDGE https://ptbk.io
+-   KNOWLEDGE ./promptbook.pdf
+-   EXPECT MIN 1 Sentence
+-   EXPECT MAX 1 Paragraph
+
+> Write an article about the future of artificial intelligence in the next 10 years and how metalanguages will change the way AI is used in the world.
+> Look specifically at the impact of {subject} on the AI industry.
+
+-> {article}
+```
+
+## 📦 Packages _(for developers)_
 
 This library is divided into several packages, all are published from [single monorepo](https://github.com/webgptorg/promptbook).
 You can install all of them at once:
@@ -321,6 +162,8 @@ Or you can install them separately:
 
 The following glossary is used to clarify certain concepts:
 
+### Basic terms
+
 
 
 ### Core concepts
@@ -353,8 +196,8 @@ The following glossary is used to clarify certain concepts:
 
 ## 🔌 Usage in Typescript / Javascript
 
--   [Simple usage](./samples/usage/simple-script)
--   [Usage with client and remote server](./samples/usage/remote)
+-   [Simple usage](./examples/usage/simple-script)
+-   [Usage with client and remote server](./examples/usage/remote)
 
 ## ➕➖ When to use Promptbook?
 

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "promptbook",
-    "version": "0.72.0",
+    "version": "0.74.0-0",
     "description": "Supercharge your use of large language models",
     "private": false,
     "sideEffects": false,
@@ -31,7 +31,8 @@
         "o1",
         "o1-mini",
         "o1-preview",
-        "anthropic"
+        "anthropic",
+        "LLMOps"
     ],
     "license": "CC-BY-4.0",
     "bugs": {
@@ -49,27 +50,27 @@
         }
     ],
     "peerDependencies": {
-        "@promptbook/core": "0.72.0"
+        "@promptbook/core": "0.74.0-0"
     },
     "dependencies": {
-        "@promptbook/anthropic-claude": "0.72.0",
-        "@promptbook/azure-openai": "0.72.0",
-        "@promptbook/browser": "0.72.0",
-        "@promptbook/cli": "0.72.0",
-        "@promptbook/core": "0.72.0",
-        "@promptbook/documents": "0.72.0",
-        "@promptbook/execute-javascript": "0.72.0",
-        "@promptbook/fake-llm": "0.72.0",
-        "@promptbook/langtail": "0.72.0",
-        "@promptbook/legacy-documents": "0.72.0",
-        "@promptbook/markdown-utils": "0.72.0",
-        "@promptbook/node": "0.72.0",
-        "@promptbook/openai": "0.72.0",
-        "@promptbook/pdf": "0.72.0",
-        "@promptbook/remote-client": "0.72.0",
-        "@promptbook/remote-server": "0.72.0",
-        "@promptbook/types": "0.72.0",
-        "@promptbook/utils": "0.72.0",
-        "@promptbook/website-crawler": "0.72.0"
+        "@promptbook/anthropic-claude": "0.74.0-0",
+        "@promptbook/azure-openai": "0.74.0-0",
+        "@promptbook/browser": "0.74.0-0",
+        "@promptbook/cli": "0.74.0-0",
+        "@promptbook/core": "0.74.0-0",
+        "@promptbook/documents": "0.74.0-0",
+        "@promptbook/execute-javascript": "0.74.0-0",
+        "@promptbook/fake-llm": "0.74.0-0",
+        "@promptbook/langtail": "0.74.0-0",
+        "@promptbook/legacy-documents": "0.74.0-0",
+        "@promptbook/markdown-utils": "0.74.0-0",
+        "@promptbook/node": "0.74.0-0",
+        "@promptbook/openai": "0.74.0-0",
+        "@promptbook/pdf": "0.74.0-0",
+        "@promptbook/remote-client": "0.74.0-0",
+        "@promptbook/remote-server": "0.74.0-0",
+        "@promptbook/types": "0.74.0-0",
+        "@promptbook/utils": "0.74.0-0",
+        "@promptbook/website-crawler": "0.74.0-0"
     }
 }