pygpt-net 2.1.37__py3-none-any.whl → 2.4.48__py3-none-any.whl

{pygpt_net-2.1.37.dist-info → pygpt_net-2.4.48.dist-info}/METADATA

@@ -1,66 +1,88 @@
 Metadata-Version: 2.1
 Name: pygpt-net
-Version: 2.1.37
-Summary: Desktop AI Assistant powered by GPT-4, GPT-4V, GPT-3.5, DALL-E 3, Langchain LLMs, Llama-index, Whisper with chatbot, assistant, text completion, vision and image generation, internet access, chat with files, commands and code execution, file upload and download and more
-Home-page: https://github.com/szczyglis-dev/py-gpt
+Version: 2.4.48
+Summary: Desktop AI Assistant powered by models: OpenAI o1, GPT-4o, GPT-4, GPT-4 Vision, GPT-3.5, DALL-E 3, Llama 3, Mistral, Gemini, Claude, Bielik, and other models supported by Langchain, Llama Index, and Ollama. Features include chatbot, text completion, image generation, vision analysis, speech-to-text, internet access, file handling, command execution and more.
+Home-page: https://pygpt.net
 License: MIT
-Keywords: py_gpt,py-gpt,pygpt,desktop,app,gpt,gpt4,gpt4-v,gpt3.5,gpt-4,gpt-4V,gpt-3.5,tts,whisper,vision,chatgpt,dall-e,chat,chatbot,assistant,text completion,image generation,ai,api,openai,api key,langchain,llama-index,presets,ui,qt,pyside
+Keywords: py_gpt,py-gpt,pygpt,desktop,app,o1,gpt,gpt4,gpt-4o,gpt-4v,gpt3.5,gpt-4,gpt-4-vision,gpt-3.5,llama3,mistral,gemini,bielik,claude,tts,whisper,vision,chatgpt,dall-e,chat,chatbot,assistant,text completion,image generation,ai,api,openai,api key,langchain,llama-index,ollama,presets,ui,qt,pyside
 Author: Marcin Szczyglinski
 Author-email: info@pygpt.net
-Requires-Python: >=3.10,<3.12
+Requires-Python: >=3.10,<3.13
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Utilities
 Requires-Dist: EbookLib (>=0.18,<0.19)
-Requires-Dist: Markdown (>=3.5.1,<4.0.0)
+Requires-Dist: Markdown (>=3.7,<4.0)
 Requires-Dist: PyAudio (>=0.2.14,<0.3.0)
-Requires-Dist: PySide6 (==6.4.2)
-Requires-Dist: PySide6-Addons (==6.4.2)
-Requires-Dist: PySide6-Essentials (==6.4.2)
-Requires-Dist: SQLAlchemy (>=2.0.23,<3.0.0)
-Requires-Dist: SpeechRecognition (>=3.10.0,<4.0.0)
-Requires-Dist: beautifulsoup4 (>=4.12.2,<5.0.0)
-Requires-Dist: chromadb (>=0.4.23,<0.5.0)
+Requires-Dist: PyAutoGUI (>=0.9.54,<0.10.0)
+Requires-Dist: PySide6 (==6.6.1)
+Requires-Dist: Pygments (>=2.18.0,<3.0.0)
+Requires-Dist: SQLAlchemy (>=2.0.27,<3.0.0)
+Requires-Dist: SpeechRecognition (>=3.10.1,<4.0.0)
+Requires-Dist: beautifulsoup4 (>=4.12.3,<5.0.0)
+Requires-Dist: chromadb (>=0.5.17,<0.6.0)
 Requires-Dist: croniter (>=2.0.1,<3.0.0)
 Requires-Dist: docker (>=7.0.0,<8.0.0)
 Requires-Dist: docx2txt (>=0.8,<0.9)
-Requires-Dist: langchain (>=0.1.9,<0.2.0)
-Requires-Dist: langchain-community (>=0.0.24,<0.0.25)
-Requires-Dist: langchain-experimental (>=0.0.52,<0.0.53)
-Requires-Dist: langchain-openai (>=0.0.2.post1,<0.0.3)
-Requires-Dist: llama-index (>=0.10.13.post1,<0.11.0)
-Requires-Dist: llama-index-embeddings-azure-openai (>=0.1.6,<0.2.0)
-Requires-Dist: llama-index-llms-azure-openai (>=0.1.5,<0.2.0)
-Requires-Dist: llama-index-readers-chatgpt-plugin (>=0.1.3,<0.2.0)
-Requires-Dist: llama-index-readers-database (>=0.1.3,<0.2.0)
-Requires-Dist: llama-index-readers-file (>=0.1.6,<0.2.0)
-Requires-Dist: llama-index-readers-github (>=0.1.7,<0.2.0)
-Requires-Dist: llama-index-readers-google (>=0.1.4,<0.2.0)
-Requires-Dist: llama-index-readers-microsoft-onedrive (>=0.1.3,<0.2.0)
-Requires-Dist: llama-index-readers-twitter (>=0.1.3,<0.2.0)
-Requires-Dist: llama-index-readers-web (>=0.1.6,<0.2.0)
-Requires-Dist: llama-index-vector-stores-chroma (>=0.1.4,<0.2.0)
-Requires-Dist: llama-index-vector-stores-elasticsearch (>=0.1.4,<0.2.0)
-Requires-Dist: llama-index-vector-stores-pinecone (>=0.1.3,<0.2.0)
-Requires-Dist: llama-index-vector-stores-redis (>=0.1.2,<0.2.0)
+Requires-Dist: google-generativeai (>=0.8.3,<0.9.0)
+Requires-Dist: httpx (>=0.27.2,<0.28.0)
+Requires-Dist: httpx-socks (>=0.9.2,<0.10.0)
+Requires-Dist: ipykernel (>=6.29.5,<7.0.0)
+Requires-Dist: jupyter_client (>=8.6.3,<9.0.0)
+Requires-Dist: langchain (>=0.2.14,<0.3.0)
+Requires-Dist: langchain-community (>=0.2.12,<0.3.0)
+Requires-Dist: langchain-experimental (>=0.0.64,<0.0.65)
+Requires-Dist: langchain-openai (>=0.1.22,<0.2.0)
+Requires-Dist: llama-index (>=0.12.11,<0.13.0)
+Requires-Dist: llama-index-agent-openai (>=0.4.2,<0.5.0)
+Requires-Dist: llama-index-core (==0.12.11)
+Requires-Dist: llama-index-embeddings-azure-openai (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-embeddings-gemini (>=0.3.1,<0.4.0)
+Requires-Dist: llama-index-embeddings-huggingface-api (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-embeddings-ollama (>=0.5.0,<0.6.0)
+Requires-Dist: llama-index-embeddings-openai (>=0.3.1,<0.4.0)
+Requires-Dist: llama-index-llms-anthropic (>=0.6.3,<0.7.0)
+Requires-Dist: llama-index-llms-azure-openai (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-llms-gemini (>=0.4.3,<0.5.0)
+Requires-Dist: llama-index-llms-huggingface-api (>=0.3.1,<0.4.0)
+Requires-Dist: llama-index-llms-ollama (>=0.5.0,<0.6.0)
+Requires-Dist: llama-index-llms-openai (>=0.3.13,<0.4.0)
+Requires-Dist: llama-index-llms-openai-like (>=0.3.3,<0.4.0)
+Requires-Dist: llama-index-multi-modal-llms-openai (>=0.4.2,<0.5.0)
+Requires-Dist: llama-index-readers-chatgpt-plugin (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-readers-database (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-readers-file (>=0.4.3,<0.5.0)
+Requires-Dist: llama-index-readers-github (>=0.5.0,<0.6.0)
+Requires-Dist: llama-index-readers-google (>=0.5.0,<0.6.0)
+Requires-Dist: llama-index-readers-microsoft-onedrive (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-readers-twitter (>=0.3.0,<0.4.0)
+Requires-Dist: llama-index-readers-web (>=0.3.3,<0.4.0)
+Requires-Dist: llama-index-vector-stores-chroma (>=0.4.1,<0.5.0)
+Requires-Dist: llama-index-vector-stores-elasticsearch (>=0.4.0,<0.5.0)
+Requires-Dist: llama-index-vector-stores-pinecone (>=0.4.2,<0.5.0)
+Requires-Dist: llama-index-vector-stores-redis (>=0.4.0,<0.5.0)
+Requires-Dist: mss (>=9.0.2,<10.0.0)
 Requires-Dist: nbconvert (>=7.16.1,<8.0.0)
-Requires-Dist: openai (>=1.12.0,<2.0.0)
-Requires-Dist: opencv-python (>=4.8.1.78,<5.0.0.0)
+Requires-Dist: openai (>=1.55.1,<1.60.0)
+Requires-Dist: opencv-python (>=4.9.0.80,<5.0.0.0)
 Requires-Dist: packaging (>=23.2,<24.0)
-Requires-Dist: pandas (>=2.1.4,<3.0.0)
+Requires-Dist: pandas (>=2.2.0,<3.0.0)
 Requires-Dist: pillow (>=10.2.0,<11.0.0)
 Requires-Dist: pinecone-client (>=3.1.0,<4.0.0)
 Requires-Dist: pydub (>=0.25.1,<0.26.0)
 Requires-Dist: pygame (>=2.5.2,<3.0.0)
-Requires-Dist: pypdf (>=4.0.2,<5.0.0)
+Requires-Dist: pynput (>=1.7.7,<2.0.0)
+Requires-Dist: pypdf (>=5.1.0,<6.0.0)
 Requires-Dist: pyserial (>=3.5,<4.0)
+Requires-Dist: python-markdown-math (>=0.8,<0.9)
 Requires-Dist: qt-material (>=2.14,<3.0)
 Requires-Dist: redis (>=5.0.1,<6.0.0)
 Requires-Dist: show-in-file-manager (>=1.1.4,<2.0.0)
-Requires-Dist: tiktoken (>=0.5.2,<0.6.0)
+Requires-Dist: tiktoken (>=0.7.0,<0.8.0)
 Requires-Dist: wikipedia (>=1.4.0,<2.0.0)
 Requires-Dist: youtube-transcript-api (>=0.6.2,<0.7.0)
 Project-URL: Documentation, https://pygpt.readthedocs.io/
@@ -71,33 +93,35 @@ Description-Content-Type: text/markdown
 
 [![pygpt](https://snapcraft.io/pygpt/badge.svg)](https://snapcraft.io/pygpt)
 
-Release: **2.1.37** | build: **2024.03.19** | Python: **>=3.10, <3.12**
+Release: **2.4.48** | build: **2025.01.16** | Python: **>=3.10, <3.13**
 
-Official website: https://pygpt.net | Documentation: https://pygpt.readthedocs.io
-
-Snap Store: https://snapcraft.io/pygpt | PyPi: https://pypi.org/project/pygpt-net
-
-Compiled version for Linux (`tar.gz`) and Windows 10/11 (`msi`) 64-bit: https://pygpt.net/#download
+> Official website: https://pygpt.net | Documentation: https://pygpt.readthedocs.io
+> 
+> Discord: https://pygpt.net/discord | Snap: https://snapcraft.io/pygpt | PyPi: https://pypi.org/project/pygpt-net
+> 
+> Compiled version for Linux (`zip`) and Windows 10/11 (`msi`) 64-bit: https://pygpt.net/#download
+> 
+> ❤️ Donate: https://www.buymeacoffee.com/szczyglis | https://github.com/sponsors/szczyglis-dev
 
 ## Overview
 
-**PyGPT** is **all-in-one** Desktop AI Assistant that provides direct interaction with OpenAI language models, including `GPT-4`, `GPT-4 Vision`, and `GPT-3.5`, through the `OpenAI API`. The application also integrates with alternative LLMs, like those available on `HuggingFace`, by utilizing `Langchain`.
+**PyGPT** is **all-in-one** Desktop AI Assistant that provides direct interaction with OpenAI language models, including `o1`, `gpt-4o`, `gpt-4`, `gpt-4 Vision`, and `gpt-3.5`, through the `OpenAI API`. By utilizing `LangChain` and `LlamaIndex`, the application also supports alternative LLMs, like those available on `HuggingFace`, locally available models (like `Llama 3`,`Mistral` or `Bielik`), `Google Gemini` and `Anthropic Claude`.
 
-This assistant offers multiple modes of operation such as chat, assistants, completions, and image-related tasks using `DALL-E 3` for generation and `GPT-4 Vision` for image analysis. **PyGPT** has filesystem capabilities for file I/O, can generate and run Python code, execute system commands, execute custom commands and manage file transfers. It also allows models to perform web searches with the `Google` and `Microsoft Bing`.
+This assistant offers multiple modes of operation such as chat, assistants, completions, and image-related tasks using `DALL-E 3` for generation and `gpt-4 Vision` for image analysis. **PyGPT** has filesystem capabilities for file I/O, can generate and run Python code, execute system commands, execute custom commands and manage file transfers. It also allows models to perform web searches with the `Google` and `Microsoft Bing`.
 
 For audio interactions, **PyGPT** includes speech synthesis using the `Microsoft Azure`, `Google`, `Eleven Labs` and `OpenAI` Text-To-Speech services. Additionally, it features speech recognition capabilities provided by `OpenAI Whisper`, `Google` and `Bing` enabling the application to understand spoken commands and transcribe audio inputs into text. It features context memory with save and load functionality, enabling users to resume interactions from predefined points in the conversation. Prompt creation and management are streamlined through an intuitive preset system.
 
 **PyGPT**'s functionality extends through plugin support, allowing for custom enhancements. Its multi-modal capabilities make it an adaptable tool for a range of AI-assisted operations, such as text-based interactions, system automation, daily assisting, vision applications, natural language processing, code generation and image creation.
 
-Multiple operation modes are included, such as chat, text completion, assistant, vision, Langchain, Chat with files (via `Llama-index`), commands execution, external API calls and image generation, making **PyGPT** a multi-tool for many AI-driven tasks.
+Multiple operation modes are included, such as chat, text completion, assistant, vision, LangChain, Chat with Files (via `LlamaIndex`), commands execution, external API calls and image generation, making **PyGPT** a multi-tool for many AI-driven tasks.
 
-**Video** (mp4, version `2.0.153`, build `2024-02-18`):
+**Video** (mp4, version `2.4.35`, build `2024-11-28`):
 
-https://github.com/szczyglis-dev/py-gpt/assets/61396542/996db435-fea3-4836-85b5-4f93505df6c4
+https://github.com/user-attachments/assets/5751a003-950f-40e7-a655-d098bbf27b0c
 
-**Screenshot** (version `2.0.153` build `2024-02-18`):
+**Screenshot** (version `2.4.35`, build `2024-11-28`):
 
-![v2_main](https://github.com/szczyglis-dev/py-gpt/assets/61396542/3a7fe49f-b6c4-4daa-b3ba-521affd5af28)
+![v2_main](https://github.com/user-attachments/assets/5d1b0da4-f8b3-437f-af07-764798315253)
 
 You can download compiled 64-bit versions for Windows and Linux here: https://pygpt.net/#download
 
@@ -105,55 +129,59 @@ You can download compiled 64-bit versions for Windows and Linux here: https://py
 
 - Desktop AI Assistant for `Linux`, `Windows` and `Mac`, written in Python.
 - Works similarly to `ChatGPT`, but locally (on a desktop computer).
-- 8 modes of operation: Chat, Vision, Completion, Assistant, Image generation, Langchain, Chat with files and Agent (autonomous).
-- Supports multiple models: `GPT-4`, `GPT-3.5`, and any model accessible through `Langchain`.
-- Handles and stores the full context of conversations (short-term memory).
-- Real-time video camera capture in Vision mode.
+- 11 modes of operation: Chat, Vision, Completion, Assistant, Image generation, LangChain, Chat with Files, Chat with Audio, Experts, Autonomous Mode and Agents.
+- Supports multiple models: `o1`, `GPT-4o`, `GPT-4`, `GPT-3.5`, and any model accessible through `LangChain`, `LlamaIndex` and `Ollama` such as `Llama 3`, `Mistral`, `Google Gemini`, `Anthropic Claude`, `Bielik`, etc.
+- Chat with your own Files: integrated `LlamaIndex` support: chat with data such as: `txt`, `pdf`, `csv`, `html`, `md`, `docx`, `json`, `epub`, `xlsx`, `xml`, webpages, `Google`, `GitHub`, video/audio, images and other data types, or use conversation history as additional context provided to the model.
+- Built-in vector databases support and automated files and data embedding.
+- Included support features for individuals with disabilities: customizable keyboard shortcuts, voice control, and translation of on-screen actions into audio via speech synthesis.
+- Handles and stores the full context of conversations (short and long-term memory).
 - Internet access via `Google` and `Microsoft Bing`.
 - Speech synthesis via `Microsoft Azure`, `Google`, `Eleven Labs` and `OpenAI` Text-To-Speech services.
-- Speech recognition via `OpenAI Whisper`, `Google`, `Google Cloud` and `Microsoft Bing`.
-- Image analysis via `GPT-4 Vision`.
-- Crontab / Task scheduler included.
-- Integrated `Langchain` support (you can connect to any LLM, e.g., on `HuggingFace`).
-- Integrated `Llama-index` support: chat with `txt`, `pdf`, `csv`, `html`, `md`, `docx`, `json`, `epub`, `xlsx`, `xml`, webpages, `Google`, `GitHub`, video/audio, images and other data types, or use conversation history as additional context provided to the model.
+- Speech recognition via `OpenAI Whisper`, `Google` and `Microsoft Speech Recognition`.
+- Real-time video camera capture in Vision mode.
+- Image analysis via `GPT-4 Vision` and `GPT-4o`.
+- Integrated `LangChain` support (you can connect to any LLM, e.g., on `HuggingFace`).
 - Integrated calendar, day notes and search in contexts by selected date.
-- Commands execution (via plugins: access to the local filesystem, Python code interpreter, system commands execution).
+- Tools and commands execution (via plugins: access to the local filesystem, Python Code Interpreter, system commands execution, and more).
 - Custom commands creation and execution.
+- Crontab / Task scheduler included.
 - Manages files and attachments with options to upload, download, and organize.
 - Context history with the capability to revert to previous contexts (long-term memory).
 - Allows you to easily manage prompts with handy editable presets.
 - Provides an intuitive operation and interface.
 - Includes a notepad.
 - Includes simple painter / drawing tool.
-- Includes optional Autonomous Mode (Agents).
 - Supports multiple languages.
-- Enables the use of all the powerful features of `GPT-4`, `GPT-4V`, and `GPT-3.5`.
 - Requires no previous knowledge of using AI models.
-- Simplifies image generation using `DALL-E 3` and `DALL-E 2`.
-- Possesses the potential to support future OpenAI models.
+- Simplifies image generation using `DALL-E`.
 - Fully configurable.
 - Themes support.
+- Real-time code syntax highlighting.
 - Plugins support.
 - Built-in token usage calculation.
-- It's open source; source code is available on `GitHub`.
+- Possesses the potential to support future OpenAI models.
+- **Open source**; source code is available on `GitHub`.
 - Utilizes the user's own API key.
+- and many more.
 
 The application is free, open-source, and runs on PCs with `Linux`, `Windows 10`, `Windows 11` and `Mac`. 
 Full Python source code is available on `GitHub`.
 
-**PyGPT uses the user's API key  -  to use the application, 
-you must have a registered OpenAI account and your own API key.**
+**PyGPT uses the user's API key  -  to use the GPT models, 
+you must have a registered OpenAI account and your own API key. Local models do not require any API keys.**
 
-You can also use built-it Langchain support to connect to other Large Language Models (LLMs), 
+You can also use built-it LangChain support to connect to other Large Language Models (LLMs), 
 such as those on HuggingFace. Additional API keys may be required.
 
 # Installation
 
-## Compiled versions (Linux, Windows 10 and 11)
+## Binaries (Linux, Windows 10 and 11)
 
-You can download compiled versions for `Linux` and `Windows` (10/11). 
+You can download compiled binary versions for `Linux` and `Windows` (10/11). 
 
-Download the `.msi` or `tar.gz` for the appropriate OS from the download page at https://pygpt.net and then extract files from the archive and run the application. 64-bit only.
+**PyGPT** binaries require a PC with Windows 10, 11, or Linux. Simply download the installer or the archive with the appropriate version from the download page at https://pygpt.net, extract it, or install it, and then run the application. A binary version for Mac is not available, so you must run PyGPT from PyPi or from the source code on Mac. Currently, only 64-bit binaries are available.
+
+Linux version requires `GLIBC` >= `2.35`.
 
 ## Snap Store
 
@@ -183,6 +211,18 @@ sudo snap connect pygpt:camera
 sudo snap connect pygpt:audio-record :audio-record
 ```
 
+**Connecting IPython in Docker in Snap version**:
+
+To use IPython in the Snap version, you must connect PyGPT to the Docker daemon:
+
+```commandline
+sudo snap connect pygpt:docker-executables docker:docker-executables
+```
+
+````commandline
+sudo snap connect pygpt:docker docker:docker-daemon
+````
+
 ## PyPi (pip)
 
 The application can also be installed from `PyPi` using `pip install`:
@@ -206,11 +246,11 @@ pip install pygpt-net
 pygpt
 ```
 
-## Source Code
+## Running from GitHub source code
 
-An alternative method is to download the source code from `GitHub` and execute the application using the Python interpreter (>=3.10, <3.12). 
+An alternative method is to download the source code from `GitHub` and execute the application using the Python interpreter (`>=3.10`, `<3.13`). 
 
-### Running from GitHub source code
+### Install with pip
 
 1. Clone git repository or download .zip file:
 
@@ -219,7 +259,7 @@ git clone https://github.com/szczyglis-dev/py-gpt.git
 cd py-gpt
 ```
 
-2. Create virtual environment:
+2. Create a new virtual environment:
 
 ```commandline
 python3 -m venv venv
@@ -238,7 +278,7 @@ pip install -r requirements.txt
 python3 run.py
 ```
 
-**Install with Poetry**
+### Install with Poetry
 
 1. Clone git repository or download .zip file:
 
@@ -303,9 +343,17 @@ sudo apt install libasound2-data
 sudo apt install libasound2-plugins
 ```
 
+**Problems with GLIBC on Linux**
+
+If you encounter error: 
+
+```commandline
+Error loading Python lib libpython3.10.so.1.0: dlopen: /lib/x86_64-linux-gnu/libm.so.6: version GLIBC_2.35 not found (required by libpython3.10.so.1.0)
+```
+when trying to run the compiled version for Linux, try updating GLIBC to version `2.35`, or use a newer operating system that has at least version `2.35` of GLIBC.
+
 **Access to camera in Snap version:**
 
-To use camera in Vision mode in Snap version you must connect the camera with:
 
 ```commandline
 sudo snap connect pygpt:camera
@@ -330,33 +378,54 @@ The absence of the installed libraries may cause display errors or completely pr
 
 It may also be necessary to add the path `C:\path\to\venv\Lib\python3.x\site-packages\PySide6` to the `PATH` variable.
 
+**WebEngine/Chromium renderer and OpenGL problems**
+
+If you have a problems with `WebEngine / Chromium` renderer you can force the legacy mode by launching the app with command line arguments:
+
+``` ini
+python3 run.py --legacy=1
+```
+
+and to force disable OpenGL hardware acceleration:
+
+``` ini
+python3 run.py --disable-gpu=1
+```
+
+You can also manualy enable legacy mode by editing config file - open the `%WORKDIR%/config.json` config file in editor and set the following options:
+
+``` json
+"render.engine": "legacy",
+"render.open_gl": false,
+```
+
 ## Other requirements
 
 For operation, an internet connection is needed (for API connectivity), a registered OpenAI account, 
-and an active API key that must be input into the program.
+and an active API key that must be input into the program. Local models, such as `Llama3` do not require OpenAI account and any API keys.
 
 ## Debugging and logging
 
-**Tip:** Go to `Debugging and Logging` section for instructions on how to log and diagnose issues in a more detailed manner.
+Please go to `Debugging and Logging` section for instructions on how to log and diagnose issues in a more detailed manner.
 
 
 # Quick Start
 
 ## Setting-up OpenAI API KEY
 
+**Tip:** The API key is required to work with the OpenAI API. If you wish to use custom API endpoints or local API that do not require API keys, simply enter anything into the API key field to avoid a prompt about the API key being empty.
+
 During the initial launch, you must configure your API key within the application.
 
 To do so, navigate to the menu:
 
 ``` ini
-Config -> Settings...
+Config -> Settings -> API Keys
 ```
 
 and then paste the API key into the `OpenAI API KEY` field.
 
-## Overview
-
-![v2_settings](https://github.com/szczyglis-dev/py-gpt/assets/61396542/21d7e43d-858f-4bc7-a06f-ec848338e7a9)
+![v2_api_keys](https://github.com/user-attachments/assets/8564add8-364b-471d-80d5-7e99ae77e129)
 
 The API key can be obtained by registering on the OpenAI website:
 
@@ -368,126 +437,196 @@ Your API keys will be available here:
 
 **Note:** The ability to use models within the application depends on the API user's access to a given model!
 
-# Chat, completion, assistants and vision (GPT-4, GPT-3.5, Langchain)
+# Working modes
 
 ## Chat
 
-**+ inline Vision and Image generation**
+**+ Inline Vision and Image generation**
 
-This mode in **PyGPT** mirrors `ChatGPT`, allowing you to chat with models such as `GPT-4`, `GPT-4 Turbo` and `GPT-3.5`. It's easy to switch models whenever you want. It works by using the `ChatCompletion API`.
+This mode in **PyGPT** mirrors `ChatGPT`, allowing you to chat with models such as `o1`, `GPT-4`, `GPT-4o` and `GPT-3.5`. It works by using the `ChatCompletion` OpenAI API.
 
-The main part of the interface is a chat window where conversations appear. Right below that is where you type your messages. On the right side of the screen, there's a section to set up or change your system prompts. You can also save these setups as presets to quickly switch between different models or tasks.
+**Tip: This mode directly uses the OpenAI API. If you want to use models other than GPT (such as Gemini, Claude, or Llama3), use `Chat with Files` mode.**
 
-Above where you type your messages, the interface shows you the number of tokens your message will use up as you type it – this helps to keep track of usage. There's also a feature to upload files in this area. Go to the `Files` tab to manage your uploads or add attachments to send to the OpenAI API (but this makes effect only in `Assisant` and `Vision` modes).
+The main part of the interface is a chat window where you see your conversations. Below it is a message box for typing. On the right side, you can set up or change the model and system prompt. You can also save these settings as presets to easily switch between models or tasks.
 
-![v2_mode_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/931a07f3-9fd5-40b3-a446-1540d2587899)
+Above where you type your messages, the interface shows you the number of tokens your message will use up as you type it – this helps to keep track of usage. There is also a feature to attach and upload files in this area. Go to the `Files and Attachments` section for more information on how to use attachments.
 
-**Vision:** If you want to send photos or image from camera to analysis you must enable plugin **GPT-4 Vision Inline** in the Plugins menu.
-Plugin allows you to send photos or image from camera to analysis in any Chat mode:
+![v2_mode_chat](https://github.com/user-attachments/assets/ed9a0290-1dcc-42e7-9585-078ad06f2e28)
 
-![v3_vision_plugins](https://github.com/szczyglis-dev/py-gpt/assets/61396542/104b0a80-7cf8-4a02-aa74-27e89ad2e409)
+**Vision:** If you want to send photos from your disk or images from your camera for analysis, and the selected model does not support Vision, you must enable the `GPT-4 Vision (inline)` plugin in the Plugins menu. This plugin allows you to send photos or images from your camera for analysis in any Chat mode.
+
+![v3_vision_plugins](https://github.com/user-attachments/assets/7d16f5f3-71b0-4c87-8f52-77b42ac9ded8)
 
 With this plugin, you can capture an image with your camera or attach an image and send it for analysis to discuss the photograph:
 
-![v3_vision_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/93d3928c-f86a-4313-b645-6277c26a39b9)
+![v3_vision_chat](https://github.com/user-attachments/assets/928d1aed-2689-44e1-b32a-68f02f83fb55)
 
-**Image generation:** If you want to generate images (using DALL-E) directly in chat you must enable plugin **DALL-E 3 Inline** in the Plugins menu.
+**Image generation:** If you want to generate images (using DALL-E) directly in chat you must enable plugin `DALL-E 3 (inline)` in the Plugins menu.
 Plugin allows you to generate images in Chat mode:
 
-![v3_img_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/52715d3e-725b-4e8c-b62d-669bd5da595d)
+![v3_img_chat](https://github.com/user-attachments/assets/1af65452-1ed1-43ec-8d78-21b0e61f0ec3)
+
+
+## Chat with Audio
+
+2024-11-26: currently in beta.
+
+This mode works like the Chat mode but with native support for audio input and output using a multimodal model - `gpt-4o-audio`. In this mode, audio input and output are directed to and from the model directly, without the use of external plugins. This enables faster and better audio communication.
+
+More info: https://platform.openai.com/docs/guides/audio/quickstart
+
+Currently, in beta. Tool and function calls are not enabled in this mode.
+
+**INFO:** The execution of commands and tools in this mode is temporarily unavailable.
 
 ## Completion
 
-This mode provides in-depth access to a broader range of capabilities offered by Large Language Models (LLMs). While it maintains a chat-like interface for user interaction, it introduces additional settings and functional richness beyond typical chat exchanges. Users can leverage this mode to prompt models for complex text completions, role-play dialogues between different characters, perform text analysis, and execute a variety of other sophisticated tasks. It supports any model provided by the OpenAI API as well as other models through `Langchain`.
+An older mode of operation that allows working in the standard text completion mode. However, it allows for a bit more flexibility with the text by enabling you to initiate the entire discussion in any way you like.
 
 Similar to chat mode, on the right-hand side of the interface, there are convenient presets. These allow you to fine-tune instructions and swiftly transition between varied configurations and pre-made prompt templates.
 
 Additionally, this mode offers options for labeling the AI and the user, making it possible to simulate dialogues between specific characters - for example, you could create a conversation between Batman and the Joker, as predefined in the prompt. This feature presents a range of creative possibilities for setting up different conversational scenarios in an engaging and exploratory manner.
 
-![v2_mode_completion](https://github.com/szczyglis-dev/py-gpt/assets/61396542/56e45c45-067c-4d63-9f41-f26dbbf08660)
-
 From version `2.0.107` the `davinci` models are deprecated and has been replaced with `gpt-3.5-turbo-instruct` model in Completion mode.
 
-## Assistants
+## Image generation (DALL-E)
 
-This mode uses the new OpenAI's **Assistants API**.
+### DALL-E 3
 
-This mode expands on the basic chat functionality by including additional external tools like a `Code Interpreter` for executing code, `Retrieval Files` for accessing files, and custom `Functions` for enhanced interaction and integration with other APIs or services. In this mode, you can easily upload and download files. **PyGPT** streamlines file management, enabling you to quickly upload documents and manage files created by the model.
+**PyGPT** enables quick and easy image creation with `DALL-E 3`. 
+The older model version, `DALL-E 2`, is also accessible. Generating images is akin to a chat conversation  -  a user's prompt triggers the generation, followed by downloading, saving to the computer, 
+and displaying the image onscreen. You can send raw prompt to `DALL-E` in `Image generation` mode or ask the model for the best prompt.
 
-Setting up new assistants is simple - a single click is all it takes, and they instantly sync with the `OpenAI API`. Importing assistants you've previously created with OpenAI into **PyGPT** is also a seamless process.
+![v3_img](https://github.com/user-attachments/assets/d7521cd1-3162-4425-89df-f7a43881574f)
 
-![v2_mode_assistant](https://github.com/szczyglis-dev/py-gpt/assets/61396542/e0c2c248-82c4-41d2-a0f5-5fa595911745)
+Image generation using DALL-E is available in every mode via plugin `DALL-E 3 Image Generation (inline)`. Just ask any model, in any mode, like e.g. GPT-4 to generate an image and it will do it inline, without need to mode change.
 
-In Assistant mode you are allowed to storage your files (per Assistant) and manage them easily from app:
+If you want to generate images (using DALL-E) directly in chat you must enable plugin **DALL-E 3 Inline** in the Plugins menu.
+Plugin allows you to generate images in Chat mode:
 
-![v2_mode_assistant_upload](https://github.com/szczyglis-dev/py-gpt/assets/61396542/0a25c34d-e989-4d00-b049-6acf8c271606)
+![v3_img_chat](https://github.com/user-attachments/assets/1af65452-1ed1-43ec-8d78-21b0e61f0ec3)
+
+### Multiple variants
+
+You can generate up to **4 different variants** (DALL-E 2) for a given prompt in one session. DALL-E 3 allows one image.
+To select the desired number of variants to create, use the slider located in the right-hand corner at 
+the bottom of the screen. This replaces the conversation temperature slider when you switch to image generation mode.
+
+### Raw mode
+
+There is an option for switching prompt generation mode.
+
+If **Raw Mode** is enabled, DALL-E will receive the prompt exactly as you have provided it.
+If **Raw Mode** is disabled, GPT will generate the best prompt for you based on your instructions.
+
+![v2_dalle2](https://github.com/szczyglis-dev/py-gpt/assets/61396542/e1c30292-8ed0-4346-8b85-6d7a02d65fb6)
+
+### Image storage
+
+Once you've generated an image, you can easily save it anywhere on your disk by right-clicking on it. 
+You also have the options to delete it or view it in full size in your web browser.
+
+**Tip:** Use presets to save your prepared prompts. 
+This lets you quickly use them again for generating new images later on.
+
+The app keeps a history of all your prompts, allowing you to revisit any session and reuse previous 
+prompts for creating new images.
+
+Images are stored in ``img`` directory in **PyGPT** user data folder.
 
-Please note that token usage calculation is unavailable in this mode. Nonetheless, file (attachment) 
-uploads are supported. Simply navigate to the `Files` tab to effortlessly manage files and attachments which 
-can be sent to the OpenAI API.
 
 ## Vision (GPT-4 Vision)
 
-This mode enables image analysis using the `GPT-4 Vision` model. Functioning much like the chat mode, 
+This mode enables image analysis using the `gpt-4o` and `gpt-4-vision` models. Functioning much like the chat mode, 
 it also allows you to upload images or provide URLs to images. The vision feature can analyze both local 
 images and those found online. 
 
-**From version 2.0.68** - Vision is integrated into any chat mode via plugin `GPT-4 Vision (inline)`. Just enable the plugin and use Vision in standard modes.
-
-**From version 2.0.14** - Vision mode also includes real-time video capture from camera. To enable capture check the option `Camera` on the right-bottom corner. It will enable real-time capturing from your camera. To capture image from camera and append it to chat just click on video at left side. You can also enable `Auto capture` - image will be captured and appended to chat message every time you send message.
+Vision is also integrated into any chat mode via plugin `GPT-4 Vision (inline)`. Just enable the plugin and use Vision in other work modes, such as Chat or Chat with Files.
 
-![v2_capture_enable](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c40ce0b4-57c8-4643-9982-25d15e68377e)
+Vision mode also includes real-time video capture from camera. To capture image from camera and append it to chat just click on video at left side. You can also enable `Auto capture` - image will be captured and appended to chat message every time you send message.
 
 **1) Video camera real-time image capture**
 
-![v2_capture1](https://github.com/szczyglis-dev/py-gpt/assets/61396542/d6d26a81-c559-470f-ba28-8d2d836dc138)
+![v2_capture1](https://github.com/szczyglis-dev/py-gpt/assets/61396542/477bb7fa-4639-42bb-8466-937e88e4a835)
 
-![v3_vision_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/93d3928c-f86a-4313-b645-6277c26a39b9)
+![v3_vision_chat](https://github.com/user-attachments/assets/928d1aed-2689-44e1-b32a-68f02f83fb55)
 
 **2) you can also provide an image URL**
 
-![v2_mode_vision](https://github.com/szczyglis-dev/py-gpt/assets/61396542/6c8bbec7-6f67-46d9-bb5f-e833c015b39c)
+![v2_mode_vision](https://github.com/szczyglis-dev/py-gpt/assets/61396542/d1b68225-bf7f-4aa5-9562-b973211b57d7)
 
 **3) or you can just upload your local images or use the inline Vision in the standard chat mode:**
 
-![v2_mode_vision_upload](https://github.com/szczyglis-dev/py-gpt/assets/61396542/68d26c32-9c7e-4068-b7d2-6c00e27a1d80)
+![v2_mode_vision_upload](https://github.com/szczyglis-dev/py-gpt/assets/61396542/7885d7d0-072e-4053-a81b-6374711fd348)
 
-**Tip:** When using `Vision (inline)` by utilizing a plugin in standard mode, such as `Chat` (not `Vision` mode), the `+ Vision` special checkbox will appear at the bottom of the Chat window. It will be automatically enabled any time you provide content for analysis (like an uploaded photo). When the checkbox is enabled, the vision model is used. If you wish to exit the vision model after image analysis, simply uncheck the checkbox. It will activate again automatically when the next image content for analysis is provided.
+**Tip:** When using `Vision (inline)` by utilizing a plugin in standard mode, such as `Chat` (not `Vision` mode), the `+ Vision` label will appear at the bottom of the Chat window.
+
+## Assistants
+
+This mode uses the OpenAI's **Assistants API**.
+
+This mode expands on the basic chat functionality by including additional external tools like a `Code Interpreter` for executing code, `Retrieval Files` for accessing files, and custom `Functions` for enhanced interaction and integration with other APIs or services. In this mode, you can easily upload and download files. **PyGPT** streamlines file management, enabling you to quickly upload documents and manage files created by the model.
+
+Setting up new assistants is simple - a single click is all it takes, and they instantly sync with the `OpenAI API`. Importing assistants you've previously created with OpenAI into **PyGPT** is also a seamless process.
+
+![v2_mode_assistant](https://github.com/user-attachments/assets/726d31f8-9120-47af-9811-269c1178f506)
+
+In Assistant mode you are allowed to storage your files in remote vector store (per Assistant) and manage them easily from app:
+
+![v2_mode_assistant_upload](https://github.com/szczyglis-dev/py-gpt/assets/61396542/b2c835ea-2816-4b85-bb6f-e08874e758f7)
+
+Please note that token usage calculation is unavailable in this mode. Nonetheless, file (attachment) 
+uploads are supported. Simply navigate to the `Files` tab to effortlessly manage files and attachments which 
+can be sent to the OpenAI API.
+
+### Vector stores (via Assistants API)
+
+Assistant mode supports the use of external vector databases offered by the OpenAI API. This feature allows you to store your files in a database and then search them using the Assistant's API. Each assistant can be linked to one vector database—if a database is linked, all files uploaded in this mode will be stored in the linked vector database. If an assistant does not have a linked vector database, a temporary database is automatically created during the file upload, which is accessible only in the current thread. Files from temporary databases are automatically deleted after 7 days.
+
+To enable the use of vector stores, enable the `Chat with Files` checkbox in the Assistant settings. This enables the `File search` tool in Assistants API.
+
+To manage external vector databases, click the DB icon next to the vector database selection list in the Assistant creation and editing window (screen below). In this management window, you can create a new vector database, edit an existing one, or import a list of all existing databases from the OpenAI server:
+
+![v2_assistant_stores](https://github.com/szczyglis-dev/py-gpt/assets/61396542/2f605326-5bf5-4c82-8dfd-cb1c0edf6724)
+
+You can define, using `Expire days`, how long files should be automatically kept in the database before deletion (as storing files on OpenAI incurs costs). If the value is set to 0, files will not be automatically deleted.
+
+The vector database in use will be displayed in the list of uploaded files, on the field to the right—if a file is stored in a database, the name of the database will be displayed there; if not, information will be shown indicating that the file is only accessible within the thread:
 
-## Langchain
+![v2_assistant_stores_upload](https://github.com/szczyglis-dev/py-gpt/assets/61396542/8f13c2eb-f961-4eae-b08b-0b4937f06ca9)
 
-This mode enables you to work with models that are supported by `Langchain`. The Langchain support is integrated 
+## LangChain
+
+This mode enables you to work with models that are supported by `LangChain`. The LangChain support is integrated 
 into the application, allowing you to interact with any LLM by simply supplying a configuration 
 file for the specific model. You can add as many models as you like; just list them in the configuration 
 file named `models.json`.
 
-Available LLMs providers supported by **PyGPT**:
+Available LLMs providers supported by **PyGPT**, in `LangChain` and `Chat with Files (LlamaIndex)` modes:
 
 ```
 - OpenAI
 - Azure OpenAI
+- Google (Gemini, etc.)
 - HuggingFace
 - Anthropic
-- Llama 2
-- Ollama
+- Ollama (Llama3, Mistral, etc.)
 ```
 
-![v2_mode_langchain](https://github.com/szczyglis-dev/py-gpt/assets/61396542/85e95c38-86db-4c75-b994-ba42521c278b)
-
 You have the ability to add custom model wrappers for models that are not available by default in **PyGPT**. 
 To integrate a new model, you can create your own wrapper and register it with the application. 
-Detailed instructions for this process are provided in the section titled `Managing models / Adding models via Langchain`.
+Detailed instructions for this process are provided in the section titled `Managing models / Adding models via LangChain`.
 
-##  Chat with files (Llama-index)
+##  Chat with Files (LlamaIndex)
 
 This mode enables chat interaction with your documents and entire context history through conversation. 
-It seamlessly incorporates `Llama-index` into the chat interface, allowing for immediate querying of your indexed documents.
+It seamlessly incorporates `LlamaIndex` into the chat interface, allowing for immediate querying of your indexed documents.
 
 **Querying single files**
 
-From version `2.1.8`, you can also query individual files "on the fly" using the `query_file` command from the `Files I/O` plugin. This allows you to query any file by simply asking a question about that file. A temporary index will be created in memory for the file being queried, and an answer will be returned from it. From version `2.1.9` similar command is available for querying web and external content: `Directly query web content with Llama-index`.
+You can also query individual files "on the fly" using the `query_file` command from the `Files I/O` plugin. This allows you to query any file by simply asking a question about that file. A temporary index will be created in memory for the file being queried, and an answer will be returned from it. From version `2.1.9` similar command is available for querying web and external content: `Directly query web content with LlamaIndex`.
 
-For example:
+**For example:**
 
 If you have a file: `data/my_cars.txt` with content `My car is red.`
 
@@ -495,9 +634,9 @@ You can ask for: `Query the file my_cars.txt about what color my car is.`
 
 And you will receive the response: `Red`.
 
-Note: this command indexes the file only for the current query and does not persist it in the database. To store queried files also in the standard index you must enable the option "Auto-index readed files" in plugin settings. Remember to enable "Execute commands" checkbox to allow usage of query commands. 
+Note: this command indexes the file only for the current query and does not persist it in the database. To store queried files also in the standard index you must enable the option `Auto-index readed files` in plugin settings. Remember to enable `+ Tools` checkbox to allow usage of tools and commands from plugins. 
 
-**Using Chat with files mode**
+**Using Chat with Files mode**
 
 In this mode, you are querying the whole index, stored in a vector store database.
 To start, you need to index (embed) the files you want to use as additional context.
@@ -511,13 +650,13 @@ For a visualization from OpenAI's page, see this picture:
 
 Source: https://cdn.openai.com/new-and-improved-embedding-model/draft-20221214a/vectors-3.svg
 
-To index your files, simply copy or upload them  into the `data` directory and initiate indexing (embedding) by clicking the `Index all` button, or right-click on a file and select `Index...`. Additionally, you have the option to utilize data from indexed files in any Chat mode by activating the `Chat with files (Llama-index, inline)` plugin.
+To index your files, simply copy or upload them  into the `data` directory and initiate indexing (embedding) by clicking the `Index all` button, or right-click on a file and select `Index...`. Additionally, you have the option to utilize data from indexed files in any Chat mode by activating the `Chat with Files (LlamaIndex, inline)` plugin.
 
 ![v2_idx1](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c3dfbc89-cbfe-4ae3-b7e7-821401d755cd)
 
 After the file(s) are indexed (embedded in vector store), you can use context from them in chat mode:
 
-![v2_idx2](https://github.com/szczyglis-dev/py-gpt/assets/61396542/29c89de9-ba3c-49ca-97b3-e6397fd648ad)
+![v2_idx2](https://github.com/szczyglis-dev/py-gpt/assets/61396542/70c9ab66-82d9-4f61-81ed-268743bfa6b4)
 
 Built-in file loaders: 
 
@@ -557,20 +696,19 @@ Built-in file loaders:
 - Webpages (crawling any webpage content)
 - YouTube (transcriptions)
 
-You can configure data loaders in `Settings / Llama-index / Data Loaders` by providing list of keyword arguments for specified loaders.
+You can configure data loaders in `Settings / Indexes (LlamaIndex) / Data Loaders` by providing list of keyword arguments for specified loaders.
 You can also develop and provide your own custom loader and register it within the application.
 
-**From version `2.0.100` Llama-index is also integrated with context database - you can use data from database (your context history) as additional context in discussion. 
-Options for indexing existing context history or enabling real-time indexing new ones (from database) are available in `Settings / Llama-index` section.**
+LlamaIndex is also integrated with context database - you can use data from database (your context history) as additional context in discussion. 
+Options for indexing existing context history or enabling real-time indexing new ones (from database) are available in `Settings / Indexes (LlamaIndex)` section.
 
 **WARNING:** remember that when indexing content, API calls to the embedding model are used. Each indexing consumes additional tokens. Always control the number of tokens used on the OpenAI page.
 
-**Tip:** when using `Chat with files` you are using additional context from db data and files indexed from `data` directory, not the files sending via `Attachments` tab. 
-Attachments tab in `Chat with files` mode can be used to provide images to `Vision (inline)` plugin only.
+**Tip:** Using the Chat with Files mode, you have default access to files manually indexed from the /data directory. However, you can use additional context by attaching a file - such additional context from the attachment does not land in the main index, but only in a temporary one, available only for the given conversation.
 
-**Token limit:** When you use `Chat with files` in non-query mode, Llama-index adds extra context to the system prompt. If you use a plugins (which also adds more instructions to system prompt), you might go over the maximum number of tokens allowed. If you get a warning that says you've used too many tokens, turn off plugins you're not using or turn off the "Execute commands" option to reduce the number of tokens used by the system prompt.
+**Token limit:** When you use `Chat with Files` in non-query mode, LlamaIndex adds extra context to the system prompt. If you use a plugins (which also adds more instructions to system prompt), you might go over the maximum number of tokens allowed. If you get a warning that says you've used too many tokens, turn off plugins you're not using or turn off the "+ Tools" option to reduce the number of tokens used by the system prompt.
 
-**Available vector stores** (provided by `Llama-index`):
+**Available vector stores** (provided by `LlamaIndex`):
 
 ```
 - ChromaVectorStore
@@ -580,722 +718,473 @@ Attachments tab in `Chat with files` mode can be used to provide images to `Visi
 - SimpleVectorStore
 ```
 
-You can configure selected vector store by providing config options like `api_key`, etc. in `Settings -> Llama-index` window. 
-Arguments provided here (on list: `Vector Store (**kwargs)` in `Advanced settings` will be passed to selected vector store provider. 
-You can check keyword arguments needed by selected provider on Llama-index API reference page: 
+You can configure selected vector store by providing config options like `api_key`, etc. in `Settings -> LlamaIndex` window. See the section: `Configuration / Vector stores` for configuration reference.
 
-https://docs.llamaindex.ai/en/stable/api_reference/storage/vector_store.html
 
-Which keyword arguments are passed to providers?
+**Configuring data loaders**
 
-For `ChromaVectorStore` and `SimpleVectorStore` all arguments are set by PyGPT and passed internally (you do not need to configure anything).
-For other providers you can provide these arguments:
+In the `Settings -> LlamaIndex -> Data loaders` section you can define the additional keyword arguments to pass into data loader instance. See the section: `Configuration / Data Loaders` for configuration reference.
 
-**ElasticsearchStore**
 
-Keyword arguments for ElasticsearchStore(`**kwargs`):
+##  Agent (LlamaIndex) 
 
-- `index_name` (default: current index ID, already set, not required)
-- any other keyword arguments provided on list
+**Currently in beta version** -- introduced in `2.4.10` (2024-11-14)
 
-**PinecodeVectorStore**
+Mode that allows the use of agents offered by `LlamaIndex`.
 
-Keyword arguments for Pinecone(`**kwargs`):
+Includes built-in agents:
 
-- `api_key`
-- index_name (default: current index ID, already set, not required)
+- OpenAI
+- ReAct
+- Structured Planner (sub-tasks)
 
-**RedisVectorStore**
+In the future, the list of built-in agents will be expanded.
 
-Keyword arguments for RedisVectorStore(`**kwargs`):
+You can also create your own agent by creating a new provider that inherits from `pygpt_net.provider.agents.base`.
 
-- `index_name` (default: current index ID, already set, not required)
-- any other keyword arguments provided on list
+**Tools and Plugins**  
 
-You can extend list of available providers by creating custom provider and registering it on app launch.
+In this mode, all commands from active plugins are available (commands from plugins are automatically converted into tools for the agent on-the-fly).
 
-By default, you are using chat-based mode when using `Chat with files`.
-If you want to only query index (without chat) you can enable `Query index only (without chat)` option.
+**RAG - using indexes**  
 
-### Adding custom vector stores and data loaders
+If an index is selected in the agent preset, a tool for reading data from the index is automatically added to the agent, creating a RAG automatically.
 
-You can create a custom vector store provider or data loader for your data and develop a custom launcher for the application. To register your custom vector store provider or data loader, simply register it by passing the vector store provider instance to `vector_stores` keyword argument and loader instance in the `loaders` keyword argument:
+Multimodality is currently unavailable, only text is supported. Vision support will be added in the future.
 
-```python
+**Loop / Evaluate Mode**
 
-# custom_launcher.py
+You can run the agent in autonomous mode, in a loop, and with evaluation of the current output. When you enable the `Loop / Evaluate` checkbox, after the final response is given, the quality of the answer will be rated on a percentage scale of `0% to 100%` by another agent. If the response receives a score lower than the one expected (set using a slider at the bottom right corner of the screen, with a default value `75%`), a prompt will be sent to the agent requesting improvements and enhancements to the response.
 
-from pygpt_net.app import run
-from plugins import CustomPlugin, OtherCustomPlugin
-from llms import CustomLLM
-from vector_stores import CustomVectorStore
-from loaders import CustomLoader
+Setting the expected (required) score to `0%` means that the response will be evaluated every time the agent produces a result, and it will always be prompted to self-improve its answer. This way, you can put the agent in an autonomous loop, where it will continue to operate until it succeeds.
 
-plugins = [
-    CustomPlugin(),
-    OtherCustomPlugin(),
-]
-llms = [
-    CustomLLM(),
-]
-vector_stores = [
-    CustomVectorStore(),
-]
-loaders = [
-    CustomLoader(),
-]
+You can set the limit of steps in such a loop by going to `Settings -> Agents and experts -> LlamaIndex agents -> Max evaluation steps `. The default value is `3`, meaning the agent will only make three attempts to improve or correct its answer. If you set the limit to zero, there will be no limit, and the agent can operate in this mode indefinitely (watch out for tokens!).
 
-run(
-    plugins=plugins,
-    llms=llms,
-    vector_stores=vector_stores,  # <--- list with custom vector store providers
-    loaders=loaders  # <--- list with custom data loaders
-)
-```
-The vector store provider must be an instance of `pygpt_net.provider.vector_stores.base.BaseStore`. 
-You can review the code of the built-in providers in `pygpt_net.provider.vector_stores` and use them as examples when creating a custom provider.
+You can change the prompt used for evaluating the response in `Settings -> Prompts -> Agent: evaluation prompt in loop`. Here, you can adjust it to suit your needs, for example, by defining more or less critical feedback for the responses received.
 
-The data loader must be an instance of `pygpt_net.provider.loaders.base.BaseLoader`. 
-You can review the code of the built-in loaders in `pygpt_net.provider.loaders` and use them as examples when creating a custom loader.
+##  Agent (Autonomous) 
 
-**Configuring data loaders**
+This is an older version of the Agent mode, still available as legacy. However, it is recommended to use the newer mode: `Agent (LlamaIndex)`.
 
-In the `Settings -> Llama-index -> Data loaders` section you can define the additional keyword arguments to pass into data loader instance.
+**WARNING: Please use this mode with caution** - autonomous mode, when connected with other plugins, may produce unexpected results!
 
-In most cases, an internal Llama-index loaders are used internally. 
-You can check these base loaders e.g. here:
+The mode activates autonomous mode, where AI begins a conversation with itself. 
+You can set this loop to run for any number of iterations. Throughout this sequence, the model will engage
+in self-dialogue, answering his own questions and comments, in order to find the best possible solution, subjecting previously generated steps to criticism.
 
-File: https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/readers/llama-index-readers-file/llama_index/readers/file
+**WARNING:** Setting the number of run steps (iterations) to `0` activates an infinite loop which can generate a large number of requests and cause very high token consumption, so use this option with caution! Confirmation will be displayed every time you run the infinite loop.
 
-Web: https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/readers/llama-index-readers-web
+This mode is similar to `Auto-GPT` - it can be used to create more advanced inferences and to solve problems by breaking them down into subtasks that the model will autonomously perform one after another until the goal is achieved.
 
-**Tip:** to index an external data or data from the Web just ask for it, by using `Command: Web Search` plugin, e.g. you can ask the model with `Please index the youtube video: URL to video`, etc. Data loader for a specified content will be choosen automatically.
+You can create presets with custom instructions for multiple agents, incorporating various workflows, instructions, and goals to achieve.
 
-Allowed additional keyword arguments for built-in data loaders (files):
+All plugins are available for agents, so you can enable features such as file access, command execution, web searching, image generation, vision analysis, etc., for your agents. Connecting agents with plugins can create a fully autonomous, self-sufficient system. All currently enabled plugins are automatically available to the Agent.
 
-**CSV Files**  (file_csv)
+When the `Auto-stop` option is enabled, the agent will attempt to stop once the goal has been reached.
 
-- `concat_rows` - bool, default: `True`
-- `encoding` - str, default: `utf-8`
+In opposition to `Auto-stop`, when the `Always continue...` option is enabled, the agent will use the "always continue" prompt to generate additional reasoning and automatically proceed to the next step, even if it appears that the task has been completed.
 
-**HTML Files**  (file_html)
+**Options**
 
-- `tag` - str, default: `section`
-- `ignore_no_id` - bool, default: `False`
+The agent is essentially a **virtual** mode that internally sequences the execution of a selected underlying mode. 
+You can choose which internal mode the agent should use in the settings:
 
-**Image (vision)**  (file_image_vision)
+```Settings / Agent (autonomous) / Sub-mode to use```
 
-This loader can operate in two modes: local model and API.
-If the local mode is enabled, then the local model will be used. The local mode requires a Python/PyPi version of the application and is not available in the compiled or Snap versions.
-If the API mode (default) is selected, then the OpenAI API and the standard vision model will be used. 
+Available choices include: `chat`, `completion`, `langchain`, `vision`, `llama_index` (Chat with Files).
 
-**Note:** Usage of API mode consumes additional tokens in OpenAI API (for `GPT-4 Vision` model)!
+Default is: `chat`.
 
-Local mode requires `torch`, `transformers`, `sentencepiece` and `Pillow` to be installed and uses the `Salesforce/blip2-opt-2.7b` model to describing images.
+If you want to use the LlamaIndex mode when running the agent, you can also specify which index `LlamaIndex` should use with the option:
 
-- `keep_image` - bool, default: `False`
-- `local_prompt` - str, default: `Question: describe what you see in this image. Answer:`
-- `api_prompt` - str, default: `Describe what you see in this image` - Prompt to use in API
-- `api_model` - str, default: `gpt-4-vision-preview` - Model to use in API
-- `api_tokens` - int, default: `1000` - Max output tokens in API
+```Settings / Agents and experts / Index to use```
 
-**IPYNB Notebook files**  (file_ipynb)
+![v2_agent_settings](https://github.com/user-attachments/assets/5e4658dc-4488-415b-8be4-51bb5b45d0bd)
 
-- `parser_config` - dict, default: `None`
-- `concatenate` - bool, default: `False`
 
-**Markdown files**  (file_md)
+##  Experts (co-op, co-operation mode)
 
-- `remove_hyperlinks` - bool, default: `True`
-- `remove_images` - bool, default: `True`
+**This mode is experimental.**
 
-**PDF documents**  (file_pdf)
+Expert mode allows for the creation of experts (using presets) and then consulting them during a conversation. In this mode, a primary base context is created for conducting the conversation. From within this context, the model can make requests to an expert to perform a task and return the results to the main thread. When an expert is called in the background, a separate context is created for them with their own memory. This means that each expert, during the life of one main context, also has access to their own memory via their separate, isolated context.
 
-- `return_full_document` - bool, default: `False`
+**In simple terms - you can imagine an expert as a separate, additional instance of the model running in the background, which can be called at any moment for assistance, with its own context and memory, as well as its own specialized instructions in a given subject.**
 
-**Video/Audio**  (file_video_audio)
+Experts do not share contexts with one another, and the only point of contact between them is the main conversation thread. In this main thread, the model acts as a manager of experts, who can exchange data between them as needed.
 
-This loader can operate in two modes: local model and API.
-If the local mode is enabled, then the local `Whisper` model will be used. The local mode requires a Python/PyPi version of the application and is not available in the compiled or Snap versions.
-If the API mode (default) is selected, then the currently selected provider in `Audio Input` plugin will be used. If the `OpenAI Whisper` is chosen then the OpenAI API and the API Whisper model will be used. 
+An expert is selected based on the name in the presets; for example, naming your expert as: ID = python_expert, name = "Python programmer" will create an expert whom the model will attempt to invoke for matters related to Python programming. You can also manually request to refer to a given expert:
 
-**Note:** Usage of Whisper via API consumes additional tokens in OpenAI API (for `Whisper` model)!
+```bash
+Call the Python expert to generate some code.
+```
 
-Local mode requires `torch` and `openai-whisper` to be installed and uses the `Whisper` model locally to transcribing video and audio.
+Experts can be activated or deactivated - to enable or disable use RMB context menu to select the `Enable/Disable` options from the presets list. Only enabled experts are available to use in the thread.
 
-- `model_version` - str, default: `base` - Whisper model to use, available models: https://github.com/openai/whisper
+Experts can also be used in `Agent (autonomous)` mode - by creating a new agent using a preset. Simply move the appropriate experts to the active list to automatically make them available for use by the agent.
 
-**XML files**  (file_xml)
+You can also use experts in "inline" mode - by activating the `Experts (inline)` plugin. This allows for the use of experts in any mode, such as normal chat.
 
-- `tree_level_split` - int, default: `0`
+Expert mode, like agent mode, is a "virtual" mode - you need to select a target mode of operation for it, which can be done in the settings at `Settings / Agent (autonomous) / Sub-mode for experts`.
 
-Allowed additional keyword arguments for built-in data loaders (Web and external content):
+You can also ask for a list of active experts at any time:
 
-**Bitbucket**  (web_bitbucket)
+```bash
+Give me a list of active experts.
+```
 
-- `username` - str, default: `None`
-- `api_key` - str, default: `None`
-- `extensions_to_skip` - list, default: `[]`
 
-**ChatGPT Retrieval**  (web_chatgpt_retrieval)
+# Context and memory
 
-- `endpoint_url` - str, default: `None`
-- `bearer_token` - str, default: `None`
-- `retries` - int, default: `None`
-- `batch_size` - int, default: `100`
+## Short and long-term memory
 
-**Google Calendar** (web_google_calendar)
+**PyGPT** features a continuous chat mode that maintains a long context of the ongoing dialogue. It preserves the entire conversation history and automatically appends it to each new message (prompt) you send to the AI. Additionally, you have the flexibility to revisit past conversations whenever you choose. The application keeps a record of your chat history, allowing you to resume discussions from the exact point you stopped.
 
-- `credentials_path` - str, default: `credentials.json`
-- `token_path` - str, default: `token.json`
+## Handling multiple contexts
 
-**Google Docs** (web_google_docs)
+On the left side of the application interface, there is a panel that displays a list of saved conversations. You can save numerous contexts and switch between them with ease. This feature allows you to revisit and continue from any point in a previous conversation. **PyGPT** automatically generates a summary for each context, akin to the way `ChatGPT` operates and gives you the option to modify these titles itself.
 
-- `credentials_path` - str, default: `credentials.json`
-- `token_path` - str, default: `token.json`
+![v2_context_list](https://github.com/user-attachments/assets/75d1eb9d-da85-422b-8b54-d5f3e95ba059)
 
-**Google Drive** (web_google_drive)
+You can disable context support in the settings by using the following option:
 
-- `credentials_path` - str, default: `credentials.json`
-- `token_path` - str, default: `token.json`
-- `pydrive_creds_path` - str, default: `creds.txt`
+``` ini
+Config -> Settings -> Use context 
+```
 
-**Google Gmail** (web_google_gmail)
+## Clearing history
 
-- `credentials_path` - str, default: `credentials.json`
-- `token_path` - str, default: `token.json`
-- `use_iterative_parser` - bool, default: `False`
-- `max_results` - int, default: `10`
-- `results_per_page` - int, default: `None`
+You can clear the entire memory (all contexts) by selecting the menu option:
 
-**Google Keep** (web_google_keep)
+``` ini
+File -> Clear history...
+```
 
-- `credentials_path` - str, default: `keep_credentials.json`
+## Context storage
 
-**Google Sheets** (web_google_sheets)
+On the application side, the context is stored in the `SQLite` database located in the working directory (`db.sqlite`).
+In addition, all history is also saved to `.txt` files for easy reading.
 
-- `credentials_path` - str, default: `credentials.json`
-- `token_path` - str, default: `token.json`
+Once a conversation begins, a title for the chat is generated and displayed on the list to the left. This process is similar to `ChatGPT`, where the subject of the conversation is summarized, and a title for the thread is created based on that summary. You can change the name of the thread at any time.
 
-**GitHub Issues**  (web_github_issues)
+# Files And Attachments
 
-- `token` - str, default: `None`
-- `verbose` - bool, default: `False`
+## Uploading attachments
 
-**GitHub Repository**  (web_github_repository)
+**Using Your Own Files as Additional Context in Conversations**
 
-- `token` - str, default: `None`
-- `verbose` - bool, default: `False`
-- `concurrent_requests` - int, default: `5`
-- `timeout` - int, default: `5`
-- `retries` - int, default: `0`
-- `filter_dirs_include` - list, default: `None`
-- `filter_dirs_exclude` - list, default: `None`
-- `filter_file_ext_include` - list, default: `None`
-- `filter_file_ext_exclude` - list, default: `None`
+You can use your own files (for example, to analyze them) during any conversation. You can do this in two ways: by indexing (embedding) your files in a vector database, which makes them available all the time during a "Chat with Files" session, or by adding a file attachment (the attachment file will only be available during the conversation in which it was uploaded).
 
-**Microsoft OneDrive**  (web_microsoft_onedrive)
+**Attachments**
 
-- `client_id` - str, default: `None`
-- `client_secret` - str, default: `None`
-- `tenant_id` - str, default: `consumers`
+**PyGPT** makes it simple for users to upload files and send them to the model for tasks like analysis, similar to attaching files in `ChatGPT`. There's a separate `Attachments` tab next to the text input area specifically for managing file uploads. 
 
-**Sitemap (XML)**  (web_sitemap)
+**Tip: Attachments uploaded in group are available in all contexts in group**.
 
-- `html_to_text` - bool, default: `False`
-- `limit` - int, default: `10`
+![v2_file_input](https://github.com/user-attachments/assets/db8467b6-2d07-4e20-a795-430fc09443a7)
 
-**SQL Database**  (web_database)
+You can use attachments to provide additional context to the conversation. Uploaded files will be converted into text using loaders from LlamaIndex, and then embedded into the vector store. You can upload any file format supported by the application through LlamaIndex. Supported formats include:
 
-- `engine` - str, default: `None`
-- `uri` - str, default: `None`
-- `scheme` - str, default: `None`
-- `host` - str, default: `None`
-- `port` - str, default: `None`
-- `user` - str, default: `None`
-- `password` - str, default: `None`
-- `dbname` - str, default: `None`
+Text-based types:
 
-**Twitter/X posts**  (web_twitter)
+- CSV files (csv)
+- Epub files (epub)
+- Excel .xlsx spreadsheets (xlsx)
+- HTML files (html, htm)
+- IPYNB Notebook files (ipynb)
+- JSON files (json)
+- Markdown files (md)
+- PDF documents (pdf)
+- Plain-text files (txt and etc.)
+- Word .docx documents (docx)
+- XML files (xml)
 
-- `bearer_token` - str, default: `None`
-- `num_tweets` - int, default: `100`
+Media-types:
 
-##  Agent (autonomous) 
+- Image (using vision) (jpg, jpeg, png, gif, bmp, tiff, webp)
+- Video/audio (mp4, avi, mov, mkv, webm, mp3, mpeg, mpga, m4a, wav)
 
-This mode is experimental.
+Archives:
 
-**WARNING: Please use this mode with caution!** - autonomous mode, when connected with other plugins, may produce unexpected results!
+- zip
+- tar, tar.gz, tar.bz2
 
-The mode activates autonomous mode, where AI begins a conversation with itself. 
-You can set this loop to run for any number of iterations. Throughout this sequence, the model will engage
-in self-dialogue, answering his own questions and comments, in order to find the best possible solution, subjecting previously generated steps to criticism.
+The content from the uploaded attachments will be used in the current conversation and will be available throughout (per context). There are 3 modes available for working with additional context from attachments:
 
-![v2_agent_toolbox](https://github.com/szczyglis-dev/py-gpt/assets/61396542/a0ae5d13-942e-4a18-9c53-33e7ad1886ff)
+- `Full context`: Provides best results. This mode attaches the entire content of the read file to the user's prompt. This process happens in the background and may require a large number of tokens if you uploaded extensive content.
 
-**WARNING:** Setting the number of run steps (iterations) to `0` activates an infinite loop which can generate a large number of requests and cause very high token consumption, so use this option with caution! Confirmation will be displayed every time you run the infinite loop.
+- `RAG`: The indexed attachment will only be queried in real-time using LlamaIndex. This operation does not require any additional tokens, but it may not provide access to the full content of the file 1:1.
 
-This mode is similar to `Auto-GPT` - it can be used to create more advanced inferences and to solve problems by breaking them down into subtasks that the model will autonomously perform one after another until the goal is achieved.
+- `Summary`: When queried, an additional query will be generated in the background and executed by a separate model to summarize the content of the attachment and return the required information to the main model. You can change the model used for summarization in the settings under the `Files and attachments` section.
 
-You can create presets with custom instructions for multiple agents, incorporating various workflows, instructions, and goals to achieve.
+In the `RAG` and `Summary` mode, you can enable an additional setting by going to `Settings -> Files and attachments -> Use history in RAG query`. This allows for better preparation of queries for RAG. When this option is turned on, the entire conversation context is considered, rather than just the user's last query. This allows for better searching of the index for additional context. In the `RAG limit` option, you can set a limit on how many recent entries in a discussion should be considered (`0 = no limit, default: 3`).
 
-All plugins are available for agents, so you can enable features such as file access, command execution, web searching, image generation, vision analysis, etc., for your agents. Connecting agents with plugins can create a fully autonomous, self-sufficient system. All currently enabled plugins are automatically available to the Agent.
+**Important**: When using `Full context` mode, the entire content of the file is included in the prompt, which can result in high token usage each time. If you want to reduce the number of tokens used, instead use the `RAG` option, which will only query the indexed attachment in the vector database to provide additional context.
 
-When the `Auto-stop` option is enabled, the agent will attempt to stop once the goal has been reached.
+**Images as Additional Context**
 
-**Options**
+Files such as jpg, png, and similar images are a special case. By default, images are not used as additional context; they are analyzed in real-time using a vision model. If you want to use them as additional context instead, you must enable the "Allow images as additional context" option in the settings: `Files and attachments -> Allow images as additional context`.
 
-The agent is essentially a **virtual** mode that internally sequences the execution of a selected underlying mode. 
-You can choose which internal mode the agent should use in the settings:
+**Uploading larger files and auto-index**
 
-```Settings / Agent (autonomous) / Sub-mode to use```
+To use the `RAG` mode, the file must be indexed in the vector database. This occurs automatically at the time of upload if the `Auto-index on upload` option in the `Attachments` tab is enabled. When uploading large files, such indexing might take a while - therefore, if you are using the `Full context` option, which does not use the index, you can disable the `Auto-index` option to speed up the upload of the attachment. In this case, it will only be indexed when the `RAG` option is called for the first time, and until then, attachment will be available in the form of `Full context` and `Summary`.
 
-Available choices include: `chat`, `completion`, `langchain`, `vision`, `llama_index` (Chat with files).
+## Downloading files
 
-Default is: `chat`.
+**PyGPT** enables the automatic download and saving of files created by the model. This is carried out in the background, with the files being saved to an `data` folder located within the user's working directory. To view or manage these files, users can navigate to the `Files` tab which features a file browser for this specific directory. Here, users have the interface to handle all files sent by the AI.
 
-If you want to use the Llama-index mode when running the agent, you can also specify which index `Llama-index` should use with the option:
+This `data` directory is also where the application stores files that are generated locally by the AI, such as code files or any other data requested from the model. Users have the option to execute code directly from the stored files and read their contents, with the results fed back to the AI. This hands-off process is managed by the built-in plugin system and model-triggered commands. You can also indexing files from this directory (using integrated `LlamaIndex`) and use it's contents as additional context provided to discussion.
 
-```Settings / Agent (autonomous) / Index to use```
+The `Files I/O` plugin takes care of file operations in the `data` directory, while the `Code Interpreter` plugin allows for the execution of code from these files.
 
-![v2_agent_settings](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c577d219-eb25-4f0e-9ea5-adf20a6b6b81)
+![v2_file_output](https://github.com/szczyglis-dev/py-gpt/assets/61396542/2ada219d-68c9-45e3-96af-86ac5bc73593)
 
+To allow the model to manage files or python code execution, the `+ Tools` option must be active, along with the above-mentioned plugins:
 
-# Files and attachments
+![v2_code_execute](https://github.com/user-attachments/assets/0b96b362-52ca-4928-9675-a39038d787a1)
 
-## Input attachments (upload)
+# Presets
 
-**PyGPT** makes it simple for users to upload files to the server and send them to the model for tasks like analysis, similar to attaching files in `ChatGPT`. There's a separate `Files` tab next to the text input area specifically for managing file uploads. Users can opt to have files automatically deleted after each upload or keep them on the list for repeated use.
+## What is preset?
 
-![v2_file_input](https://github.com/szczyglis-dev/py-gpt/assets/61396542/00263f3d-ec62-4daa-85b3-5a5ce178dce0)
+Presets in **PyGPT** are essentially templates used to store and quickly apply different configurations. Each preset includes settings for the mode you want to use (such as chat, completion, or image generation), an initial system prompt, an assigned name for the AI, a username for the session, and the desired "temperature" for the conversation. A warmer "temperature" setting allows the AI to provide more creative responses, while a cooler setting encourages more predictable replies. These presets can be used across various modes and with models accessed via the `OpenAI API` or `LangChain`.
 
-The attachment feature is available in both the `Assistant` and `Vision` modes at default.
-In `Assistant` mode, you can send documents and files to analyze, while in `Vision` mode, you can send images.
-In other modes, you can enable attachments by activating the `Vision (inline)` plugin (for providing images only).
+The application lets you create as many presets as needed and easily switch among them. Additionally, you can clone an existing preset, which is useful for creating variations based on previously set configurations and experimentation.
 
-## Files (download, code generation)
+![v2_preset](https://github.com/user-attachments/assets/15d142c7-6669-4e0a-9a21-e57555a4cb83)
 
-**PyGPT** enables the automatic download and saving of files created by the model. This is carried out in the background, with the files being saved to an `data` folder located within the user's working directory. To view or manage these files, users can navigate to the `Files` tab which features a file browser for this specific directory. Here, users have the interface to handle all files sent by the AI.
+## Example usage
 
-This `data` directory is also where the application stores files that are generated locally by the AI, such as code files or any other data requested from the model. Users have the option to execute code directly from the stored files and read their contents, with the results fed back to the AI. This hands-off process is managed by the built-in plugin system and model-triggered commands. You can also indexing files from this directory (using integrated `Llama-index`) and use it's contents as additional context provided to discussion.
+The application includes several sample presets that help you become acquainted with the mechanism of their use.
 
-The `Command: Files I/O` plugin takes care of file operations in the `data` directory, while the `Command: Code Interpreter` plugin allows for the execution of code from these files.
+# Profiles
+
+You can create multiple profiles for an app and switch between them. Each profile uses its own configuration, settings, context history, and a separate folder for user files. This allows you to set up different environments and quickly switch between them, changing the entire setup with just one click.
+
+The app lets you create new profiles, edit existing ones, and duplicate current ones.
+
+To create a new profile, select the option from the menu: `Config -> Profile -> New Profile...`
+
+To edit saved profiles, choose the option from the menu: `Config -> Profile -> Edit Profiles...`
+
+To switch to a created profile, pick the profile from the menu: `Config -> Profile -> [Profile Name]`
+
+Each profile uses its own user directory (workdir). You can link a newly created or edited profile to an existing workdir with its configuration.
+
+The name of the currently active profile is shown as (Profile Name) in the window title.
+
+# Models
+
+## Built-in models
+
+PyGPT has built-in support for models (as of 2024-11-27):
+
+- `bielik-11b-v2.2-instruct:Q4_K_M`
+- `chatgpt-4o-latest`
+- `claude-3-5-sonnet-20240620`
+- `claude-3-opus-20240229`
+- `codellama`
+- `dall-e-2`
+- `dall-e-3`
+- `gemini-1.5-flash`
+- `gemini-1.5-pro`
+- `gpt-3.5-turbo`
+- `gpt-3.5-turbo-1106`
+- `gpt-3.5-turbo-16k`
+- `gpt-3.5-turbo-instruct`
+- `gpt-4`
+- `gpt-4-0125-preview`
+- `gpt-4-1106-preview`
+- `gpt-4-32k`
+- `gpt-4-turbo`
+- `gpt-4-turbo-2024-04-09`
+- `gpt-4-turbo-preview`
+- `gpt-4-vision-preview`
+- `gpt-4o`
+- `gpt-4o-2024-11-20`
+- `gpt-4o-audio-preview`
+- `gpt-4o-mini`
+- `llama2-uncensored`
+- `llama3.1`
+- `llama3.1:405b`
+- `llama3.1:70b`
+- `mistral`
+- `mistral-large`
+- `o1-mini`
+- `o1-preview`
 
-![v2_file_output](https://github.com/szczyglis-dev/py-gpt/assets/61396542/2ada219d-68c9-45e3-96af-86ac5bc73593)
+All models are specified in the configuration file `models.json`, which you can customize. 
+This file is located in your working directory. You can add new models provided directly by `OpenAI API`
+and those supported by `LlamaIndex` or `LangChain` to this file. Configuration for LangChain wrapper is placed in `langchain` key, configuration for LlamaIndex in `llama_index` key.
 
-To allow the model to manage files or python code execution, the `Execute commands` option must be active, along with the above-mentioned plugins:
+## Adding a custom model
 
-![v2_code_execute](https://github.com/szczyglis-dev/py-gpt/assets/61396542/d5181eeb-6ab4-426f-93f0-037d256cb078)
+You can add your own models. See the section `Extending PyGPT / Adding a new model` for more info.
 
-# Draw (paint)
+There is built-in support for those LLM providers:
 
-Using the `Draw` tool, you can create quick sketches and submit them to the model for analysis. You can also edit opened from disk or captured from camera images, for example, by adding elements like arrows or outlines to objects. Additionally, you can capture screenshots from the system - the captured image is placed in the drawing tool and attached to the query being sent.
+- OpenAI (openai)
+- Azure OpenAI (azure_openai)
+- Google (google)
+- HuggingFace (huggingface)
+- Anthropic (anthropic)
+- Ollama (ollama)
 
-![v2_draw](https://github.com/szczyglis-dev/py-gpt/assets/61396542/09c1de36-1241-4330-9fd7-67c6e09888fa)
+## How to use local or non-GPT models
 
-To capture the screenshot just click on the `Ask with screenshot` option in a tray-icon dropdown:
+### Llama 3, Mistral, and other local models
 
-![v2_screenshot](https://github.com/szczyglis-dev/py-gpt/assets/61396542/7305a814-ca76-4f8f-8908-47f6a9496fa5)
+How to use locally installed Llama 3 or Mistral models:
 
-# Calendar
+1) Choose a working mode: `Chat with Files` or `LangChain`.
 
-Using the calendar, you can go back to selected conversations from a specific day and add daily notes. After adding a note, it will be marked on the list, and you can change the color of its label by right-clicking and selecting `Set label color`. By clicking on a particular day of the week, conversations from that day will be displayed.
+2) On the models list - select, edit, or add a new model (with `ollama` provider). You can edit the model settings through the menu `Config -> Models`, then configure the model parameters in the `advanced` section.
 
-![v2_calendar](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c7d17375-b61f-452c-81bc-62a7d466fc18)
+3) Download and install Ollama from here: https://github.com/ollama/ollama
 
-# Context and memory
+For example, on Linux:
 
-## Short and long-term memory
+```curl -fsSL https://ollama.com/install.sh | sh```
 
-**PyGPT** features a continuous chat mode that maintains a long context of the ongoing dialogue. It preserves the entire conversation history and automatically appends it to each new message (prompt) you send to the AI. Additionally, you have the flexibility to revisit past conversations whenever you choose. The application keeps a record of your chat history, allowing you to resume discussions from the exact point you stopped.
+4) Run the model (e.g. Llama 3) locally on your machine. For example, on Linux:
 
-## Handling multiple contexts
+```ollama run llama3.1```
 
-On the left side of the application interface, there is a panel that displays a list of saved conversations. You can save numerous contexts and switch between them with ease. This feature allows you to revisit and continue from any point in a previous conversation. **PyGPT** automatically generates a summary for each context, akin to the way `ChatGPT` operates and gives you the option to modify these titles itself.
+5) Return to PyGPT and select the correct model from models list to chat with selected model using Ollama running locally.
 
-![v2_context_list](https://github.com/szczyglis-dev/py-gpt/assets/61396542/9228ea4c-f30c-4b02-ba85-da10b4e2eb7b)
+**Example available models**
 
-You can disable context support in the settings by using the following option:
+- `llama3.1`
+- `codellama`
+- `mistral`
+- `llama2-uncensored`
 
-``` ini
-Config -> Settings -> Use context 
-```
+You can add more models by editing the models list.
 
-## Clearing history
+**List of all models supported by Ollama**
 
-You can clear the entire memory (all contexts) by selecting the menu option:
+https://ollama.com/library
 
-``` ini
-File -> Clear history...
-```
+https://github.com/ollama/ollama
 
-## Context storage
+**IMPORTANT:** Remember to define the correct model name in the **kwargs list in the model settings.
 
-On the application side, the context is stored in the `SQLite` database located in the working directory (`db.sqlite`).
-In addition, all history is also saved to `.txt` files for easy reading.
+**Using local embeddings**
 
-Once a conversation begins, a title for the chat is generated and displayed on the list to the left. This process is similar to `ChatGPT`, where the subject of the conversation is summarized, and a title for the thread is created based on that summary. You can change the name of the thread at any time.
+Refer to: https://docs.llamaindex.ai/en/stable/examples/embeddings/ollama_embedding/
 
-# Presets
+You can use an Ollama instance for embeddings. Simply select the `ollama` provider in:
 
-## What is preset?
+```Config -> Settings -> Indexes (LlamaIndex) -> Embeddings -> Embeddings provider```
 
-Presets in **PyGPT** are essentially templates used to store and quickly apply different configurations. Each preset includes settings for the mode you want to use (such as chat, completion, or image generation), an initial system message, an assigned name for the AI, a username for the session, and the desired "temperature" for the conversation. A warmer "temperature" setting allows the AI to provide more creative responses, while a cooler setting encourages more predictable replies. These presets can be used across various modes and with models accessed via the `OpenAI API` or `Langchain`.
+Define parameters like model name and Ollama base URL in the Embeddings provider **kwargs list, e.g.:
 
-The system lets you create as many presets as needed and easily switch among them. Additionally, you can clone an existing preset, which is useful for creating variations based on previously set configurations and experimentation.
+- name: `model_name`, value: `llama3.1`, type: `str`
 
-![v2_preset](https://github.com/szczyglis-dev/py-gpt/assets/61396542/88167631-feb6-45ca-a006-25a21ec2339e)
+- name: `base_url`, value: `http://localhost:11434`, type: `str`
 
-## Example usage
+### Google Gemini and Anthropic Claude
 
-The application includes several sample presets that help you become acquainted with the mechanism of their use.
+To use `Gemini` or `Claude` models, select the `Chat with Files` mode in PyGPT and select a predefined model.
+Remember to configure the required parameters like API keys in the model ENV config fields.
 
+**Google Gemini**
 
-# Image generation (DALL-E 3 and DALL-E 2)
+Required ENV:
 
-## DALL-E 3
+- GOOGLE_API_KEY
 
-**PyGPT** enables quick and easy image creation with `DALL-E 3`. 
-The older model version, `DALL-E 2`, is also accessible. Generating images is akin to a chat conversation  -  a user's prompt triggers the generation, followed by downloading, saving to the computer, 
-and displaying the image onscreen. You can send raw prompt to `DALL-E` in `Image generation` mode or ask the model for the best prompt.
+Required **kwargs:
 
-**From version 2.0.68 (released 2023-12-31) image generation using DALL-E is available in every mode via plugin `DALL-E 3 Image Generation (inline)`. Just ask any model, in any mode, like e.g. GPT-4 to generate an image and it will do it inline, without need to mode change.**
+- model
 
-If you want to generate images (using DALL-E) directly in chat you must enable plugin **DALL-E 3 Inline** in the Plugins menu.
-Plugin allows you to generate images in Chat mode:
+**Anthropic Claude**
 
-![v3_img_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/52715d3e-725b-4e8c-b62d-669bd5da595d)
+Required ENV:
 
-## Multiple variants
+- ANTHROPIC_API_KEY
 
-You can generate up to **4 different variants** (DALL-E 2) for a given prompt in one session. DALL-E 3 allows one image.
-To select the desired number of variants to create, use the slider located in the right-hand corner at 
-the bottom of the screen. This replaces the conversation temperature slider when you switch to image generation mode.
+Required **kwargs:
 
-## Raw mode
+- model
 
-There is an option for switching prompt generation mode.
 
-If **Raw Mode** is enabled, DALL-E will receive the prompt exactly as you have provided it.
-If **Raw Mode** is disabled, GPT will generate the best prompt for you based on your instructions.
+# Plugins
 
-![v2_dalle2](https://github.com/szczyglis-dev/py-gpt/assets/61396542/e1c30292-8ed0-4346-8b85-6d7a02d65fb6)
+## Overview
 
-## Image storage
+**PyGPT** can be enhanced with plugins to add new features.
 
-Once you've generated an image, you can easily save it anywhere on your disk by right-clicking on it. 
-You also have the options to delete it or view it in full size in your web browser.
+**Tip:** Plugins works best with GPT-4 models.
 
-**Tip:** Use presets to save your prepared prompts. 
-This lets you quickly use them again for generating new images later on.
+The following plugins are currently available, and model can use them instantly:
 
-The app keeps a history of all your prompts, allowing you to revisit any session and reuse previous 
-prompts for creating new images.
+- `Audio Input` - provides speech recognition.
 
-Images are stored in ``img`` directory in **PyGPT** user data folder.
+- `Audio Output` - provides voice synthesis.
 
-# Managing models
+- `Autonomous Agent (inline)` - enables autonomous conversation (AI to AI), manages loop, and connects output back to input. This is the inline Agent mode.
 
-All models are specified in the configuration file `models.json`, which you can customize. 
-This file is located in your working directory. You can add new models provided directly by `OpenAI API`
-and those supported by `Langchain` to this file. Configuration for Langchain wrapper is placed in `langchain` key.
+- `Chat with Files (LlamaIndex, inline)` - plugin integrates `LlamaIndex` storage in any chat and provides additional knowledge into context (from indexed files and previous context from database).
 
-## Adding custom LLMs via Langchain
+- `API calls` - plugin lets you connect the model to the external services using custom defined API calls.
 
-To add a new model using the Langchain wrapper in **PyGPT**, insert the model's configuration details into the `models.json` file. This should include the model's name, its supported modes (either `chat`, `completion`, or both), the LLM provider (which can be either e.g. `OpenAI` or `HuggingFace`), and, if you are using a `HuggingFace` model, an optional `API KEY`.
+- `Code Interpreter` - responsible for generating and executing Python code, functioning much like 
+the Code Interpreter on ChatGPT, but locally. This means GPT can interface with any script, application, or code. 
+Plugins can work in conjunction to perform sequential tasks; for example, the `Files` plugin can write generated 
+Python code to a file, which the `Code Interpreter` can execute it and return its result to GPT.
 
-Example of models configuration - `models.json`:
+- `Custom Commands` - allows you to create and execute custom commands on your system.
 
-```
-"gpt-3.5-turbo": {
-    "id": "gpt-3.5-turbo",
-    "name": "gpt-3.5-turbo",
-    "mode": [
-        "chat",
-        "assistant",
-        "langchain",
-        "llama_index"
-    ],
-    "langchain": {
-        "provider": "openai",
-        "mode": [
-            "chat"
-        ],
-        "args": [
-            {
-                "name": "model_name",
-                "value": "gpt-3.5-turbo",
-                "type": "str"
-            }
-        ],
-        "env": [
-            {
-                "name": "OPENAI_API_KEY",
-                "value": "{api_key}"
-            }
-        ]
-    },
-    "llama_index": {
-        "provider": "openai",
-        "mode": [
-            "chat"
-        ],
-        "args": [
-            {
-                "name": "model",
-                "value": "gpt-3.5-turbo",
-                "type": "str"
-            }
-        ],
-        "env": [
-            {
-                "name": "OPENAI_API_KEY",
-                "value": "{api_key}"
-            }
-        ]
-    },
-    "ctx": 4096,
-    "tokens": 4096,
-    "default": false
-},
-```
+- `Files I/O` - provides access to the local filesystem, enabling GPT to read and write files, 
+as well as list and create directories.
 
-There is bult-in support for those LLMs providers:
+- `System (OS)` - allows you to create and execute custom commands on your system.
 
-```
-- OpenAI (openai)
-- Azure OpenAI (azure_openai)
-- HuggingFace (huggingface)
-- Anthropic (anthropic)
-- Llama 2 (llama2)
-- Ollama (ollama)
-```
+- `Mouse and Keyboard` - provides the ability to control the mouse and keyboard by the model.
 
-## Adding custom LLM providers
+- `Web Search` - provides the ability to connect to the Web, search web pages for current data, and index external content using LlamaIndex data loaders.
 
-Handling LLMs with Langchain is implemented through separated wrappers. This allows for the addition of support for any provider and model available via Langchain. All built-in wrappers for the models and its providers are placed in the `pygpt_net.provider.llms`.
+- `Serial port / USB` - plugin provides commands for reading and sending data to USB ports.
 
-These wrappers are loaded into the application during startup using `launcher.add_llm()` method:
+- `Context history (calendar, inline)` - provides access to context history database.
 
-```python
-# app.py
+- `Crontab / Task scheduler` - plugin provides cron-based job scheduling - you can schedule tasks/prompts to be sent at any time using cron-based syntax for task setup.
 
-from pygpt_net.provider.llms.openai import OpenAILLM
-from pygpt_net.provider.llms.azure_openai import AzureOpenAILLM
-from pygpt_net.provider.llms.anthropic import AnthropicLLM
-from pygpt_net.provider.llms.hugging_face import HuggingFaceLLM
-from pygpt_net.provider.llms.llama import Llama2LLM
-from pygpt_net.provider.llms.ollama import OllamaLLM
+- `DALL-E 3: Image Generation (inline)` - integrates DALL-E 3 image generation with any chat and mode. Just enable and ask for image in Chat mode, using standard model like GPT-4. The plugin does not require the `+ Tools` option to be enabled.
 
+- `Experts (inline)` - allows calling experts in any chat mode. This is the inline Experts (co-op) mode.
 
-def run(**kwargs):
-    """Runs the app."""
-    # Initialize the app
-    launcher = Launcher()
-    launcher.init()
+- `GPT-4 Vision (inline)` - integrates Vision capabilities with any chat mode, not just Vision mode. When the plugin is enabled, the model temporarily switches to vision in the background when an image attachment or vision capture is provided.
 
-    # Register plugins
-    ...
+- `Real Time` - automatically appends the current date and time to the system prompt, informing the model about current time.
 
-    # Register langchain LLMs wrappers
-    launcher.add_llm(OpenAILLM())
-    launcher.add_llm(AzureOpenAILLM())
-    launcher.add_llm(AnthropicLLM())
-    launcher.add_llm(HuggingFaceLLM())
-    launcher.add_llm(Llama2LLM())
-    launcher.add_llm(OllamaLLM())
+- `System Prompt Extra (append)` - appends additional system prompts (extra data) from a list to every current system prompt. You can enhance every system prompt with extra instructions that will be automatically appended to the system prompt.
 
-    # Launch the app
-    launcher.run()
-```
+- `Voice Control (inline)` - provides voice control command execution within a conversation.
 
-To add support for providers not included by default, you can create your own wrapper that returns a custom model to the application and then pass this custom wrapper to the launcher.
+- `Mailer` - Provides the ability to send, receive and read emails.
 
-Extending **PyGPT** with custom plugins and LLM wrappers is straightforward:
+## Audio Input
 
-- Pass instances of custom plugins and LLM wrappers directly to the launcher.
+The plugin facilitates speech recognition (by default using the `Whisper` model from OpenAI, `Google` and `Bing` are also available). It allows for voice commands to be relayed to the AI using your own voice. Whisper doesn't require any extra API keys or additional configurations; it uses the main OpenAI key. In the plugin's configuration options, you should adjust the volume level (min energy) at which the plugin will respond to your microphone. Once the plugin is activated, a new `Speak` option will appear at the bottom near the `Send` button  -  when this is enabled, the application will respond to the voice received from the microphone.
 
-To register custom LLM wrappers:
+The plugin can be extended with other speech recognition providers.
 
-- Provide a list of LLM wrapper instances as `llms` keyword argument.
+Options:
 
-**Example:**
+- `Provider` *provider*
 
+Choose the provider. *Default:* `Whisper`
 
-```python
-# launcher.py
+Available providers:
 
-from pygpt_net.app import run
-from plugins import CustomPlugin, OtherCustomPlugin
-from llms import CustomLLM
+- Whisper (via `OpenAI API`)
+- Whisper (local model) - not available in compiled and Snap versions, only Python/PyPi version
+- Google (via `SpeechRecognition` library)
+- Google Cloud (via `SpeechRecognition` library)
+- Microsoft Bing (via `SpeechRecognition` library)
 
-plugins = [
-    CustomPlugin(),
-    OtherCustomPlugin(),
-]
-llms = [
-    CustomLLM(),
-]
-vector_stores = []
-
-run(
-    plugins=plugins, 
-    llms=llms, 
-    vector_stores=vector_stores
-)
-```
-
-**Examples (tutorial files)** 
-
-See the `examples` directory in this repository with examples of custom launcher, plugin, vector store, LLM (Langchain and Llama-index) provider and data loader:
-
-- `examples/custom_launcher.py`
-
-- `examples/example_audio_input.py`
-
-- `examples/example_audio_output.py`
-
-- `examples/example_data_loader.py`
-
-- `examples/example_llm.py`  <-- use it as an example
-
-- `examples/example_plugin.py`
-
-- `examples/example_vector_store.py`
-
-- `examples/example_web_search.py`
-
-These example files can be used as a starting point for creating your own extensions for **PyGPT**.
-
-To integrate your own model or provider into **PyGPT**, you can also reference the classes located in the `pygpt_net.provider.llms`. These samples can act as an more complex example for your custom class. Ensure that your custom wrapper class includes two essential methods: `chat` and `completion`. These methods should return the respective objects required for the model to operate in `chat` and `completion` modes.
-
-
-## Adding custom Vector Store providers
-
-**From version 2.0.114 you can also register your own Vector Store provider**:
-
-```python
-# app.py
-
-# vector stores
-from pygpt_net.provider.vector_stores.chroma import ChromaProvider
-from pygpt_net.provider.vector_stores.elasticsearch import ElasticsearchProvider
-from pygpt_net.provider.vector_stores.pinecode import PinecodeProvider
-from pygpt_net.provider.vector_stores.redis import RedisProvider
-from pygpt_net.provider.vector_stores.simple import SimpleProvider
-
-def run(**kwargs):
-    # ...
-    # register base vector store providers (llama-index)
-    launcher.add_vector_store(ChromaProvider())
-    launcher.add_vector_store(ElasticsearchProvider())
-    launcher.add_vector_store(PinecodeProvider())
-    launcher.add_vector_store(RedisProvider())
-    launcher.add_vector_store(SimpleProvider())
-
-    # register custom vector store providers (llama-index)
-    vector_stores = kwargs.get('vector_stores', None)
-    if isinstance(vector_stores, list):
-        for store in vector_stores:
-            launcher.add_vector_store(store)
-
-    # ...
-```
-
-To register your custom vector store provider just register it by passing provider instance in `vector_stores` keyword argument:
-
-```python
-
-# custom_launcher.py
-
-from pygpt_net.app import run
-from plugins import CustomPlugin, OtherCustomPlugin
-from llms import CustomLLM
-from vector_stores import CustomVectorStore
-
-plugins = [
-    CustomPlugin(),
-    OtherCustomPlugin(),
-]
-llms = [
-    CustomLLM(),
-]
-vector_stores = [
-    CustomVectorStore(),
-]
-
-run(
-    plugins=plugins,
-    llms=llms,
-    vector_stores=vector_stores
-)
-```
-
-# Plugins
-
-**PyGPT** can be enhanced with plugins to add new features.
-
-The following plugins are currently available, and model can use them instantly:
-
-- `Audio Input` - provides speech recognition.
-
-- `Audio Output` - provides voice synthesis.
-
-- `Autonomous Agent (inline)` - enables autonomous conversation (AI to AI), manages loop, and connects output back to input. This is the inline Agent mode.
-
-- `Chat with files (Llama-index, inline)` - plugin integrates `Llama-index` storage in any chat and provides additional knowledge into context (from indexed files and previous context from database).
-
-- `Command: API calls` - plugin lets you connect the model to the external services using custom defined API calls.
-
-- `Command: Code Interpreter` - responsible for generating and executing Python code, functioning much like 
-the Code Interpreter on ChatGPT, but locally. This means GPT can interface with any script, application, or code. 
-The plugin can also execute system commands, allowing GPT to integrate with your operating system. 
-Plugins can work in conjunction to perform sequential tasks; for example, the `Files` plugin can write generated 
-Python code to a file, which the `Code Interpreter` can execute it and return its result to GPT.
-
-- `Command: Custom Commands` - allows you to create and execute custom commands on your system.
-
-- `Command: Files I/O` - provides access to the local filesystem, enabling GPT to read and write files, 
-as well as list and create directories.
-
-- `Command: Web Search` - provides the ability to connect to the Web, search web pages for current data, and index external content using Llama-index data loaders.
-
-- `Command: Serial port / USB` - plugin provides commands for reading and sending data to USB ports.
-
-- `Context history (calendar, inline)` - provides access to context history database.
-
-- `Crontab / Task scheduler` - plugin provides cron-based job scheduling - you can schedule tasks/prompts to be sent at any time using cron-based syntax for task setup.
-
-- `DALL-E 3: Image Generation (inline)` - integrates DALL-E 3 image generation with any chat and mode. Just enable and ask for image in Chat mode, using standard model like GPT-4. The plugin does not require the `Execute commands` option to be enabled.
-
-- `GPT-4 Vision (inline)` - integrates Vision capabilities with any chat mode, not just Vision mode. When the plugin is enabled, the model temporarily switches to vision in the background when an image attachment or vision capture is provided.
-
-- `Real Time` - automatically appends the current date and time to the system prompt, informing the model about current time.
-
-- `System Prompt Extra (append)` - appends additional system prompts (extra data) from a list to every current system prompt. You can enhance every system prompt with extra instructions that will be automatically appended to the system prompt.
-
-
-## Audio Input
-
-The plugin facilitates speech recognition (by default using the `Whisper` model from OpenAI, `Google` and `Bing` are also available). It allows for voice commands to be relayed to the AI using your own voice. Whisper doesn't require any extra API keys or additional configurations; it uses the main OpenAI key. In the plugin's configuration options, you should adjust the volume level (min energy) at which the plugin will respond to your microphone. Once the plugin is activated, a new `Speak` option will appear at the bottom near the `Send` button  -  when this is enabled, the application will respond to the voice received from the microphone.
-
-The plugin can be extended with other speech recognition providers.
-
-Options:
-
-- `Provider` *provider*
-
-Choose the provider. *Default:* `Whisper`
-
-Available providers:
-
-- Whisper (via `OpenAI API`)
-- Whisper (local model) - not available in compiled and Snap versions, only Python/PyPi version
-- Google (via `SpeechRecognition` library)
-- Google Cloud (via `SpeechRecognition` library)
-- Microsoft Bing (via `SpeechRecognition` library)
-
-**Whisper (API)**
+**Whisper (API)**
 
 - `Model` *whisper_model*
 
@@ -1429,7 +1318,7 @@ Options reference: https://pypi.org/project/SpeechRecognition/1.3.1/
 The plugin lets you turn text into speech using the TTS model from OpenAI or other services like ``Microsoft Azure``, ``Google``, and ``Eleven Labs``. You can add more text-to-speech providers to it too. `OpenAI TTS` does not require any additional API keys or extra configuration; it utilizes the main OpenAI key. 
 Microsoft Azure requires to have an Azure API Key. Before using speech synthesis via `Microsoft Azure`, `Google` or `Eleven Labs`, you must configure the audio plugin with your API keys, regions and voices if required.
 
-![v2_azure](https://github.com/szczyglis-dev/py-gpt/assets/61396542/8035e9a5-5a01-44a1-85da-6e44c52459e4)
+![v2_azure](https://github.com/user-attachments/assets/475108c1-5ea8-4f43-8cd5-effcd5ef352c)
 
 Through the available options, you can select the voice that you want the model to use. More voice synthesis providers coming soon.
 
@@ -1555,30 +1444,30 @@ If enabled, plugin will stop after goal is reached." *Default:* `True`
 
 - `Reverse roles between iterations` *reverse_roles*
 
-Only for Completion/Langchain modes. 
+Only for Completion/LangChain modes. 
 If enabled, this option reverses the roles (AI <> user) with each iteration. For example, 
 if in the previous iteration the response was generated for "Batman," the next iteration will use that 
 response to generate an input for "Joker." *Default:* `True`
 
-## Chat with files (Llama-index, inline)
+## Chat with Files (LlamaIndex, inline)
 
-Plugin integrates `Llama-index` storage in any chat and provides additional knowledge into context.
+Plugin integrates `LlamaIndex` storage in any chat and provides additional knowledge into context.
 
-- `Ask Llama-index first` *ask_llama_first*
+- `Ask LlamaIndex first` *ask_llama_first*
 
-When enabled, then `Llama-index` will be asked first, and response will be used as additional knowledge in prompt. When disabled, then `Llama-index` will be asked only when needed. **INFO: Disabled in autonomous mode (via plugin)!** *Default:* `False`
+When enabled, then `LlamaIndex` will be asked first, and response will be used as additional knowledge in prompt. When disabled, then `LlamaIndex` will be asked only when needed. **INFO: Disabled in autonomous mode (via plugin)!** *Default:* `False`
 
-- `Auto-prepare question before asking Llama-index first` *prepare_question*
+- `Auto-prepare question before asking LlamaIndex first` *prepare_question*
 
-When enabled, then question will be prepared before asking Llama-index first to create best query. *Default:* `False`
+When enabled, then question will be prepared before asking LlamaIndex first to create best query. *Default:* `False`
 
 - `Model for question preparation` *model_prepare_question*
 
-Model used to prepare question before asking Llama-index. *Default:* `gpt-3.5-turbo`
+Model used to prepare question before asking LlamaIndex. *Default:* `gpt-3.5-turbo`
 
 - `Max output tokens for question preparation` *prepare_question_max_tokens*
 
-Max tokens in output when preparing question before asking Llama-index. *Default:* `500`
+Max tokens in output when preparing question before asking LlamaIndex. *Default:* `500`
 
 - `Prompt for question preparation` *syntax_prepare_question*
 
@@ -1586,26 +1475,26 @@ System prompt for question preparation.
 
 - `Max characters in question` *max_question_chars*
 
-Max characters in question when querying Llama-index, 0 = no limit. *Default:* `1000`
+Max characters in question when querying LlamaIndex, 0 = no limit. *Default:* `1000`
 
 - `Append metadata to context` *append_meta*
 
-If enabled, then metadata from Llama-index will be appended to additional context. *Default:* `False`
+If enabled, then metadata from LlamaIndex will be appended to additional context. *Default:* `False`
 
 - `Model` *model_query*
 
-Model used for querying `Llama-index`. *Default:* `gpt-3.5-turbo`
+Model used for querying `LlamaIndex`. *Default:* `gpt-3.5-turbo`
 
 - `Indexes IDs` *idx*
 
 Indexes to use. If you want to use multiple indexes at once then separate them by comma. *Default:* `base`
 
 
-## Command: API calls
+## API calls
 
 **PyGPT** lets you connect the model to the external services using custom defined API calls.
 
-To activate this feature, turn on the `Command: API calls` plugin found in the `Plugins` menu.
+To activate this feature, turn on the `API calls` plugin found in the `Plugins` menu.
 
 In this plugin you can provide list of allowed API calls, their parameters and request types. The model will replace provided placeholders with required params and make API call to external service.
 
@@ -1658,58 +1547,162 @@ Connection timeout (seconds). *Default:* `5`
 User agent to use when making requests. *Default:* `Mozilla/5.0`
 
 
-## Command: Code Interpreter
+## Code Interpreter
 
 ### Executing Code
 
-The plugin operates similarly to the `Code Interpreter` in `ChatGPT`, with the key difference that it works locally on the user's system. It allows for the execution of any Python code on the computer that the model may generate. When combined with the `Command: Files I/O` plugin, it facilitates running code from files saved in the `data` directory. You can also prepare your own code files and enable the model to use them or add your own plugin for this purpose. You can execute commands and code on the host machine or in Docker container.
+From version `2.4.13` with built-in `IPython`.
 
-**Real-time code interpreter:** From version `2.1.29`, a real-time Python code interpreter is included. Click the `<>` icon to open the interpreter window. Both the input and output of the interpreter are connected to the plugin. Any output generated by the executed code will be displayed in the interpreter. Additionally, you can request the model to retrieve contents from the interpreter window output.
+The plugin operates similarly to the `Code Interpreter` in `ChatGPT`, with the key difference that it works locally on the user's system. It allows for the execution of any Python code on the computer that the model may generate. When combined with the `Files I/O` plugin, it facilitates running code from files saved in the `data` directory. You can also prepare your own code files and enable the model to use them or add your own plugin for this purpose. You can execute commands and code on the host machine or in Docker container.
 
-![v2_python](https://github.com/szczyglis-dev/py-gpt/assets/61396542/793e554c-7619-402a-8370-ab89c7464fec)
+**IPython:** Starting from version `2.4.13`, it is highly recommended to adopt the new option: `IPython`, which offers significant improvements over previous workflows. IPython provides a robust environment for executing code within a kernel, allowing you to maintain the state of your session by preserving the results of previous commands. This feature is particularly useful for iterative development and data analysis, as it enables you to build upon prior computations without starting from scratch. Moreover, IPython supports the use of magic commands, such as `!pip install <package_name>`, which facilitate the installation of new packages directly within the session. This capability streamlines the process of managing dependencies and enhances the flexibility of your development environment. Overall, IPython offers a more efficient and user-friendly experience for executing and managing code.
 
-### Executing system commands
+To use IPython in sandbox mode, Docker must be installed on your system. 
 
-Another feature is the ability to execute system commands and return their results. With this functionality, the plugin can run any system command, retrieve the output, and then feed the result back to the model. When used with other features, this provides extensive integration capabilities with the system.
+You can find the installation instructions here: https://docs.docker.com/engine/install/
 
+**Tip: connecting IPython in Docker in Snap version**:
 
+To use IPython in the Snap version, you must connect PyGPT to the Docker daemon:
+
+```commandline
+sudo snap connect pygpt:docker-executables docker:docker-executables
+```
+
+````commandline
+sudo snap connect pygpt:docker docker:docker-daemon
+````
+
+
+**Code interpreter:** a real-time Python Code Interpreter is built-in. Click the `<>` icon to open the interpreter window. Both the input and output of the interpreter are connected to the plugin. Any output generated by the executed code will be displayed in the interpreter. Additionally, you can request the model to retrieve contents from the interpreter window output.
+
+![v2_python](https://github.com/szczyglis-dev/py-gpt/assets/61396542/793e554c-7619-402a-8370-ab89c7464fec)
 
-**Tip:** always remember to enable the `Execute commands` option to allow execute commands from the plugins.
+
+**Tip:** always remember to enable the `+ Tools` option to allow execute commands from the plugins.
 
 
 **Options:**
 
+**General**
+
+- `Connect to the Python Code Interpreter window` *attach_output*
+
+Automatically attach code input/output to the Python Code Interpreter window. *Default:* `True`
+
+- `Tool: get_python_output` *cmd.get_python_output*
+
+Allows `get_python_output` command execution. If enabled, it allows retrieval of the output from the Python Code Interpreter window. *Default:* `True`
+
+- `Tool: get_python_input` *cmd.get_python_input*
+
+Allows `get_python_input` command execution. If enabled, it allows retrieval all input code (from edit section) from the Python Code Interpreter window. *Default:* `True`
+
+- `Tool: clear_python_output` *cmd.clear_python_output*
+
+Allows `clear_python_output` command execution. If enabled, it allows clear the output of the Python Code Interpreter window. *Default:* `True`
+
+
+**IPython**
+
+- `Sandbox (docker container)` *sandbox_ipython*
+
+Executes IPython in sandbox (docker container). Docker must be installed and running.
+
+- `Dockerfile` *ipython_dockerfile*
+
+You can customize the Dockerfile for the image used by IPython by editing the configuration above and rebuilding the image via Tools -> Rebuild IPython Docker Image.
+
+- `Session Key` *ipython_session_key*
+
+It must match the key provided in the Dockerfile.
+
+- `Docker image name` *ipython_image_name*
+
+Custom image name
+
+- `Docker container name` *ipython_container_name*
+
+Custom container name
+
+- `Connection address` *ipython_conn_addr*
+
+Default: 127.0.0.1
+
+- `Port: shell` *ipython_port_shell*
+
+Default: 5555
+
+- `Port: iopub` *ipython_port_iopub*
+
+Default: 5556
+
+- `Port: stdin` *ipython_port_stdin*
+
+Default: 5557
+
+- `Port: control` *ipython_port_control*
+
+Default: 5558
+
+- `Port: hb` *ipython_port_hb*
+
+Default: 5559
+
+
+- `Tool: ipython_execute` *cmd.ipython_execute*
+
+Allows Python code execution in IPython interpreter (in current kernel). *Default:* `True`
+
+- `Tool: python_kernel_restart` *cmd.ipython_kernel_restart*
+
+Allows to restart IPython kernel. *Default:* `True`
+
+
+**Python (legacy)**
+
+- `Sandbox (docker container)` *sandbox_docker*
+
+Executes commands in sandbox (docker container). Docker must be installed and running.
+
 - `Python command template` *python_cmd_tpl*
 
 Python command template (use {filename} as path to file placeholder). *Default:* `python3 {filename}`
 
-- `Enable: code_execute` *cmd.code_execute*
+- `Dockerfile` *dockerfile*
+
+You can customize the Dockerfile for the image used by legacy Python by editing the configuration above and rebuilding the image via Tools -> Rebuild Python (Legacy) Docker Image.
+
+- `Docker image name` *image_name*
+
+Custom Docker image name
+
+- `Docker container name` *container_name*
+
+Custom Docker container name
+
+- `Tool: code_execute` *cmd.code_execute*
 
 Allows `code_execute` command execution. If enabled, provides Python code execution (generate and execute from file). *Default:* `True`
 
-- `Enable: code_execute_all` *cmd.code_execute_all*
+- `Tool: code_execute_all` *cmd.code_execute_all*
 
 Allows `code_execute_all` command execution. If enabled, provides execution of all the Python code in interpreter window. *Default:* `True`
 
-- `Enable: code_execute_file` *cmd.code_execute_file*
+- `Tool: code_execute_file` *cmd.code_execute_file*
 
 Allows `code_execute_file` command execution. If enabled, provides Python code execution from existing .py file. *Default:* `True`
- 
-- `Enable: sys_exec` *cmd.sys_exec*
-
-Allows `sys_exec` command execution. If enabled, provides system commands execution. *Default:* `True`
 
-- `Enable: get_python_output` *cmd.get_python_output*
 
-Allows `get_python_output` command execution. If enabled, it allows retrieval of the output from the Python code interpreter window. *Default:* `True`
+**HTML Canvas**
 
-- `Enable: get_python_input` *cmd.get_python_input*
+- `Tool: render_html_output` *cmd.render_html_output*
 
-Allows `get_python_input` command execution. If enabled, it allows retrieval all input code (from edit section) from the Python code interpreter window. *Default:* `True`
+Allows `render_html_output` command execution. If enabled, it allows to render HTML/JS code in built-it HTML/JS browser (HTML Canvas). *Default:* `True`
 
-- `Enable: clear_python_output` *cmd.clear_python_output*
+- `Tool: get_html_output` *cmd.get_html_output*
 
-Allows `clear_python_output` command execution. If enabled, it allows clear the output of the Python code interpreter window. *Default:* `True`
+Allows `get_html_output` command execution. If enabled, it allows retrieval current output from HTML Canvas. *Default:* `True`
 
 - `Sandbox (docker container)` *sandbox_docker*
 
@@ -1719,20 +1712,13 @@ Execute commands in sandbox (docker container). Docker must be installed and run
 
 Docker image to use for sandbox *Default:* `python:3.8-alpine`
 
-- `Auto-append CWD to sys_exec` *auto_cwd*
-
-Automatically append current working directory to `sys_exec` command. *Default:* `True`
-
-- `Connect to the Python code interpreter window` *attach_output*
-
-Automatically attach code input/output to the Python code interpreter window. *Default:* `True`
 
 
-## Command: Custom Commands
+## Custom Commands
 
 With the `Custom Commands` plugin, you can integrate **PyGPT** with your operating system and scripts or applications. You can define an unlimited number of custom commands and instruct GPT on when and how to execute them. Configuration is straightforward, and **PyGPT** includes a simple tutorial command for testing and learning how it works:
 
-![v2_custom_cmd](https://github.com/szczyglis-dev/py-gpt/assets/61396542/b30b8724-9ca1-44b1-abc7-78241588e1f6)
+![v2_custom_cmd](https://github.com/user-attachments/assets/e1d803e8-9452-4507-a9a6-7a43b83b897d)
 
 To add a new custom command, click the **ADD** button and then:
 
@@ -1788,9 +1774,9 @@ With the setup above, every time you ask GPT to generate a song for you and save
 
 ```> please execute tutorial test command```
 
-![v2_custom_cmd_example](https://github.com/szczyglis-dev/py-gpt/assets/61396542/64a95a9c-634c-4d71-ad7d-4eabe8b2509b)
+![v2_custom_cmd_example](https://github.com/szczyglis-dev/py-gpt/assets/61396542/97cbc5b9-0dd9-487e-9182-d9873dea42ab)
 
-## Command: Files I/O
+## Files I/O
 
 The plugin allows for file management within the local filesystem. It enables the model to create, read, write and query files located in the `data` directory, which can be found in the user's work directory. With this plugin, the AI can also generate Python code files and thereafter execute that code within the user's system.
 
@@ -1807,8 +1793,8 @@ Plugin capabilities include:
 - Copying files and directories
 - Moving (renaming) files and directories
 - Reading file info
-- Indexing files and directories using Llama-index
-- Querying files using Llama-index
+- Indexing files and directories using LlamaIndex
+- Querying files using LlamaIndex
 - Searching for files and directories
 
 If a file being created (with the same name) already exists, a prefix including the date and time is added to the file name.
@@ -1817,89 +1803,89 @@ If a file being created (with the same name) already exists, a prefix including
 
 **General**
 
-- `Enable: send (upload) file as attachment` *cmd.send_file*
+- `Tool: send (upload) file as attachment` *cmd.send_file*
 
 Allows `cmd.send_file` command execution. *Default:* `True`
 
-- `Enable: read file` *cmd.read_file*
+- `Tool: read file` *cmd.read_file*
 
 Allows `read_file` command execution. *Default:* `True`
 
-- `Enable: append to file` *cmd.append_file*
+- `Tool: append to file` *cmd.append_file*
 
 Allows `append_file` command execution. Text-based files only (plain text, JSON, CSV, etc.) *Default:* `True`
 
-- `Enable: save file` *cmd.save_file*
+- `Tool: save file` *cmd.save_file*
 
 Allows `save_file` command execution. Text-based files only (plain text, JSON, CSV, etc.) *Default:* `True`
 
-- `Enable: delete file` *cmd.delete_file*
+- `Tool: delete file` *cmd.delete_file*
 
 Allows `delete_file` command execution. *Default:* `True`
 
-- `Enable: list files (ls)` *cmd.list_files*
+- `Tool: list files (ls)` *cmd.list_files*
 
 Allows `list_dir` command execution. *Default:* `True`
 
-- `Enable: list files in dirs in directory (ls)` *cmd.list_dir*
+- `Tool: list files in dirs in directory (ls)` *cmd.list_dir*
 
 Allows `mkdir` command execution. *Default:* `True`
 
-- `Enable: downloading files` *cmd.download_file*
+- `Tool: downloading files` *cmd.download_file*
 
 Allows `download_file` command execution. *Default:* `True`
 
-- `Enable: removing directories` *cmd.rmdir*
+- `Tool: removing directories` *cmd.rmdir*
 
 Allows `rmdir` command execution. *Default:* `True`
 
-- `Enable: copying files` *cmd.copy_file*
+- `Tool: copying files` *cmd.copy_file*
 
 Allows `copy_file` command execution. *Default:* `True`
 
-- `Enable: copying directories (recursive)` *cmd.copy_dir*
+- `Tool: copying directories (recursive)` *cmd.copy_dir*
 
 Allows `copy_dir` command execution. *Default:* `True`
 
-- `Enable: move files and directories (rename)` *cmd.move*
+- `Tool: move files and directories (rename)` *cmd.move*
 
 Allows `move` command execution. *Default:* `True`
 
-- `Enable: check if path is directory` *cmd.is_dir*
+- `Tool: check if path is directory` *cmd.is_dir*
 
 Allows `is_dir` command execution. *Default:* `True`
 
-- `Enable: check if path is file` *cmd.is_file*
+- `Tool: check if path is file` *cmd.is_file*
 
 Allows `is_file` command execution. *Default:* `True`
 
-- `Enable: check if file or directory exists` *cmd.file_exists*
+- `Tool: check if file or directory exists` *cmd.file_exists*
 
 Allows `file_exists` command execution. *Default:* `True`
 
-- `Enable: get file size` *cmd.file_size*
+- `Tool: get file size` *cmd.file_size*
 
 Allows `file_size` command execution. *Default:* `True`
 
-- `Enable: get file info` *cmd.file_info*
+- `Tool: get file info` *cmd.file_info*
 
 Allows `file_info` command execution. *Default:* `True`
 
-- `Enable: find file or directory` *cmd.find*
+- `Tool: find file or directory` *cmd.find*
 
 Allows `find` command execution. *Default:* `True`
 
-- `Enable: get current working directory` *cmd.cwd*
+- `Tool: get current working directory` *cmd.cwd*
 
 Allows `cwd` command execution. *Default:* `True`
 
 - `Use data loaders` *use_loaders*
 
-Use data loaders from Llama-index for file reading (`read_file` command). *Default:* `True`
+Use data loaders from LlamaIndex for file reading (`read_file` command). *Default:* `True`
 
 **Indexing**
 
-- `Enable: quick query the file with Llama-index` *cmd.query_file*
+- `Tool: quick query the file with LlamaIndex` *cmd.query_file*
 
 Allows `query_file` command execution (in-memory index). If enabled, model will be able to quick index file into memory and query it for data (in-memory index) *Default:* `True`
 
@@ -1907,9 +1893,9 @@ Allows `query_file` command execution (in-memory index). If enabled, model will
 
 Model used for query temporary index for `query_file` command (in-memory index). *Default:* `gpt-3.5-turbo`
 
-- `Enable: indexing files to persistent index` *cmd.file_index*
+- `Tool: indexing files to persistent index` *cmd.file_index*
 
-Allows `file_index` command execution. If enabled, model will be able to index file or directory using Llama-index (persistent index). *Default:* `True`
+Allows `file_index` command execution. If enabled, model will be able to index file or directory using LlamaIndex (persistent index). *Default:* `True`
 
 - `Index to use when indexing files` *idx*
 
@@ -1923,83 +1909,183 @@ If enabled, every time file is read, it will be automatically indexed (persisten
 
 If enabled, file will be indexed without return its content on file read (persistent index). *Default:* `False`
 
+## System (OS)
 
-## Command: Web Search
+The plugin provides access to the operating system and executes system commands.
 
-**PyGPT** lets you connect GPT to the internet and carry out web searches in real time as you make queries.
+**Options:**
 
-To activate this feature, turn on the `Command: Web Search` plugin found in the `Plugins` menu.
+**General**
 
-Web searches are provided by `Google Custom Search Engine` and `Microsoft Bing` APIs and can be extended with other search engine providers. 
+- `Auto-append CWD to sys_exec` *auto_cwd*
 
-**Options**
+Automatically append current working directory to `sys_exec` command. *Default:* `True`
 
-- `Provider` *provider*
+- `Tool: sys_exec` *cmd.sys_exec*
 
-Choose the provider. *Default:* `Google`
+Allows `sys_exec` command execution. If enabled, provides system commands execution. *Default:* `True`
 
-Available providers:
 
-- Google
-- Microsoft Bing
+## Mouse And Keyboard
 
-**Google**
+Introduced in version: `2.4.4` (2024-11-09)
 
-To use this provider, you need an API key, which you can obtain by registering an account at:
+**WARNING: Use this plugin with caution - allowing all options gives the model full control over the mouse and keyboard**
 
-https://developers.google.com/custom-search/v1/overview
+The plugin allows for controlling the mouse and keyboard by the model. With this plugin, you can send a task to the model, e.g., "open notepad, type something in it" or "open web browser, do search, find something."
 
-After registering an account, create a new project and select it from the list of available projects:
+Plugin capabilities include:
 
-https://programmablesearchengine.google.com/controlpanel/all
+- Get mouse cursor position
+- Control mouse cursor position
+- Control mouse clicks
+- Control mouse scroll
+- Control the keyboard (pressing keys, typing text)
+- Making screenshots
 
-After selecting your project, you need to enable the `Whole Internet Search` option in its settings. 
-Then, copy the following two items into **PyGPT**:
+The `+ Tools` option must be enabled to use this plugin.
 
-- `Api Key`
-- `CX ID`
+**Options:**
 
-These data must be configured in the appropriate fields in the `Plugins / Settings...` menu:
+**General**
 
-![v2_plugin_google](https://github.com/szczyglis-dev/py-gpt/assets/61396542/f2e0df62-caaa-40ef-9b1e-239b2f912ec8)
+- `Prompt` *prompt*
 
-- `Google Custom Search API KEY` *google_api_key*
+Prompt used to instruct how to control the mouse and keyboard.
 
-You can obtain your own API key at https://developers.google.com/custom-search/v1/overview
+- `Enable: Allow mouse movement` *allow_mouse_move*
 
-- `Google Custom Search CX ID` *google_api_cx*
+Allows mouse movement. *Default:* `True`
 
-You will find your CX ID at https://programmablesearchengine.google.com/controlpanel/all - remember to enable "Search on ALL internet pages" option in project settings.
+- `Enable: Allow mouse click` *allow_mouse_click*
 
-**Microsoft Bing**
+Allows mouse click. *Default:* `True`
 
-- `Bing Search API KEY` *bing_api_key*
+- `Enable: Allow mouse scroll` *allow_mouse_scroll*
 
-You can obtain your own API key at https://www.microsoft.com/en-us/bing/apis/bing-web-search-api
+Allows mouse scroll. *Default:* `True`
 
-- `Bing Search API endpoint` *bing_endpoint*
+- `Enable: Allow keyboard key press` *allow_keyboard*
 
-API endpoint for Bing Search API, default: https://api.bing.microsoft.com/v7.0/search
+Allows keyboard typing. *Default:* `True`
 
-**General options**
+- `Enable: Allow making screenshots` *allow_screenshot*
 
-- `Number of pages to search` *num_pages*
+Allows making screenshots. *Default:* `True`
 
-Number of max pages to search per query. *Default:* `10`
+- `Tool: mouse_get_pos` *cmd.mouse_get_pos*
 
-- `Max content characters` *max_page_content_length*
+Allows `mouse_get_pos` command execution. *Default:* `True`
 
-Max characters of page content to get (0 = unlimited). *Default:* `0`
+- `Tool: mouse_set_pos` *cmd.mouse_set_pos*
 
-- `Per-page content chunk size` *chunk_size*
+Allows `mouse_set_pos` command execution. *Default:* `True`
 
-Per-page content chunk size (max characters per chunk). *Default:* `20000`
+- `Tool: make_screenshot` *cmd.make_screenshot*
 
-- `Disable SSL verify` *disable_ssl*
+Allows `make_screenshot` command execution. *Default:* `True`
 
-Disables SSL verification when crawling web pages. *Default:* `False`
+- `Tool: mouse_click` *cmd.mouse_click*
 
-- `Timeout` *timeout*
+Allows `mouse_click` command execution. *Default:* `True`
+
+- `Tool: mouse_move` *cmd.mouse_move*
+
+Allows `mouse_move` command execution. *Default:* `True`
+
+- `Tool: mouse_scroll` *cmd.mouse_scroll*
+
+Allows `mouse_scroll` command execution. *Default:* `True`
+
+- `Tool: keyboard_key` *cmd.keyboard_key*
+
+Allows `keyboard_key` command execution. *Default:* `True`
+
+- `Tool: keyboard_type` *cmd.keyboard_type*
+
+Allows `keyboard_type` command execution. *Default:* `True`
+
+
+## Web Search
+
+**PyGPT** lets you connect GPT to the internet and carry out web searches in real time as you make queries.
+
+To activate this feature, turn on the `Web Search` plugin found in the `Plugins` menu.
+
+Web searches are provided by `Google Custom Search Engine` and `Microsoft Bing` APIs and can be extended with other search engine providers. 
+
+**Options**
+
+- `Provider` *provider*
+
+Choose the provider. *Default:* `Google`
+
+Available providers:
+
+- Google
+- Microsoft Bing
+
+**Google**
+
+To use this provider, you need an API key, which you can obtain by registering an account at:
+
+https://developers.google.com/custom-search/v1/overview
+
+After registering an account, create a new project and select it from the list of available projects:
+
+https://programmablesearchengine.google.com/controlpanel/all
+
+After selecting your project, you need to enable the `Whole Internet Search` option in its settings. 
+Then, copy the following two items into **PyGPT**:
+
+- `Api Key`
+- `CX ID`
+
+These data must be configured in the appropriate fields in the `Plugins / Settings...` menu:
+
+![v2_plugin_google](https://github.com/user-attachments/assets/6a9ed44d-7a1e-45f7-a9c9-afd2b66baccc)
+
+- `Google Custom Search API KEY` *google_api_key*
+
+You can obtain your own API key at https://developers.google.com/custom-search/v1/overview
+
+- `Google Custom Search CX ID` *google_api_cx*
+
+You will find your CX ID at https://programmablesearchengine.google.com/controlpanel/all - remember to enable "Search on ALL internet pages" option in project settings.
+
+**Microsoft Bing**
+
+- `Bing Search API KEY` *bing_api_key*
+
+You can obtain your own API key at https://www.microsoft.com/en-us/bing/apis/bing-web-search-api
+
+- `Bing Search API endpoint` *bing_endpoint*
+
+API endpoint for Bing Search API, default: https://api.bing.microsoft.com/v7.0/search
+
+**General options**
+
+- `Number of pages to search` *num_pages*
+
+Number of max pages to search per query. *Default:* `10`
+
+- `Max content characters` *max_page_content_length*
+
+Max characters of page content to get (0 = unlimited). *Default:* `0`
+
+- `Per-page content chunk size` *chunk_size*
+
+Per-page content chunk size (max characters per chunk). *Default:* `20000`
+
+- `Disable SSL verify` *disable_ssl*
+
+Disables SSL verification when crawling web pages. *Default:* `False`
+
+- `Use raw content (without summarization)` *raw*
+
+Return raw content from web search instead of summarized content. Provides more data but consumes more tokens. *Default:* `True`
+
+- `Timeout` *timeout*
 
 Connection timeout (seconds). *Default:* `5`
 
@@ -2009,47 +2095,41 @@ User agent to use when making requests. *Default:* `Mozilla/5.0`.
 
 - `Max result length` *max_result_length*
 
-Max length of summarized result (characters). *Default:* `1500`
+Max length of the summarized or raw result (characters). *Default:* `50000`
 
 - `Max summary tokens` *summary_max_tokens*
 
 Max tokens in output when generating summary. *Default:* `1500`
 
-- `Enable: search the Web` *cmd.web_search*
+- `Tool: web_search` *cmd.web_search*
 
 Allows `web_search` command execution. If enabled, model will be able to search the Web. *Default:* `True`
 
-- `Enable: opening URLs` *cmd.web_url_open*
+- `Tool: web_url_open` *cmd.web_url_open*
 
 Allows `web_url_open` command execution. If enabled, model will be able to open specified URL and summarize content. *Default:* `True`
 
-- `Enable: reading the raw content from URLs` *cmd.web_url_raw*
+- `Tool: web_url_raw` *cmd.web_url_raw*
 
 Allows `web_url_raw` command execution. If enabled, model will be able to open specified URL and get the raw content. *Default:* `True`
 
-- `Enable: getting a list of URLs from search results` *cmd.web_urls*
-
-Allows `web_urls` command execution. If enabled, model will be able to search the Web and get founded URLs list. *Default:* `True`
-
-- `Enable: indexing web and external content` *cmd.web_index*
-
-Allows `web_index` command execution. If enabled, model will be able to index pages and external content using Llama-index (persistent index). *Default:* `True`
+- `Tool: web_request` *cmd.web_request*
 
-- `Enable: quick query the web and external content` *cmd.web_index_query*
+Allows `web_request` command execution. If enabled, model will be able to send any HTTP request to specified URL or API endpoint. *Default:* `True`
 
-Allows `web_index_query` command execution. If enabled, model will be able to quick index and query web content using Llama-index (in-memory index). *Default:* `True`
+- `Tool: web_extract_links` *cmd.web_extract_links*
 
-- `Auto-index all used URLs using Llama-index` *auto_index*
+Allows `web_extract_links` command execution. If enabled, model will be able to open URL and get list of all links from it. *Default:* `True`
 
-If enabled, every URL used by the model will be automatically indexed using Llama-index (persistent index). *Default:* `False`
+- `Tool: web_extract_images` *cmd.web_extract_images*
 
-- `Index to use` *idx*
+Allows `web_extract_images` command execution. If enabled, model will be able to open URL and get list of all images from it.. *Default:* `True`
 
-ID of index to use for web page indexing (persistent index). *Default:* `base`
+**Advanced**
 
 - `Model used for web page summarize` *summary_model*
 
-Model used for web page summarize. *Default:* `gpt-3.5-turbo-1106`
+Model used for web page summarize. *Default:* `gpt-4o-mini`
 
 - `Summarize prompt` *prompt_summarize*
 
@@ -2059,7 +2139,25 @@ Prompt used for web search results summarize, use {query} as a placeholder for s
 
 Prompt used for specified URL page summarize.
 
-## Command: Serial port / USB
+**Indexing**
+
+- `Tool: web_index` *cmd.web_index*
+
+Allows `web_index` command execution. If enabled, model will be able to index pages and external content using LlamaIndex (persistent index). *Default:* `True`
+
+- `Tool: web_index_query` *cmd.web_index_query*
+
+Allows `web_index_query` command execution. If enabled, model will be able to quick index and query web content using LlamaIndex (in-memory index). *Default:* `True`
+
+- `Auto-index all used URLs using LlamaIndex` *auto_index*
+
+If enabled, every URL used by the model will be automatically indexed using LlamaIndex (persistent index). *Default:* `False`
+
+- `Index to use` *idx*
+
+ID of index to use for web page indexing (persistent index). *Default:* `base`
+
+## Serial port / USB
 
 Provides commands for reading and sending data to USB ports.
 
@@ -2106,15 +2204,15 @@ Timeout in seconds. *Default:* `1`
 
 Sleep in seconds after connection *Default:* `2`
 
-- `Enable: Send text commands to USB port` *cmd.serial_send*
+- `Tool: Send text commands to USB port` *cmd.serial_send*
 
 Allows `serial_send` command execution. *Default:* `True`
 
-- `Enable: Send raw bytes to USB port` *cmd.serial_send_bytes*
+- `Tool: Send raw bytes to USB port` *cmd.serial_send_bytes*
 
 Allows `serial_send_bytes` command execution. *Default:* `True`
 
-- `Enable: Read data from USB port` *cmd.serial_read*
+- `Tool: Read data from USB port` *cmd.serial_read*
 
 Allows `serial_read` command execution. *Default:* `True`
 
@@ -2137,7 +2235,7 @@ Examples of use, you can ask e.g. for the following:
 
 etc.
 
-From version `2.0.147` it is possible to use `@` ID tags to automatically use summary of previous contexts in current discussion.
+You can also use `@` ID tags to automatically use summary of previous contexts in current discussion.
 To use context from previous discussion with specified ID use following syntax in your query:
 
 ```@123```
@@ -2153,31 +2251,31 @@ Where `123` is the ID of previous context (conversation) in database, example of
 
 When enabled, it allows to automatically retrieve context history using @ tags, e.g. use @123 in question to use summary of context with ID 123 as additional context. *Default:* `False`
 
-- `Enable: get date range context list` *cmd.get_ctx_list_in_date_range*
+- `Tool: get date range context list` *cmd.get_ctx_list_in_date_range*
 
 Allows `get_ctx_list_in_date_range` command execution. If enabled, it allows getting the list of context history (previous conversations). *Default:* `True
 
-- `Enable: get context content by ID` *cmd.get_ctx_content_by_id*
+- `Tool: get context content by ID` *cmd.get_ctx_content_by_id*
 
 Allows `get_ctx_content_by_id` command execution. If enabled, it allows getting summarized content of context with defined ID. *Default:* `True`
 
-- `Enable: count contexts in date range` *cmd.count_ctx_in_date*
+- `Tool: count contexts in date range` *cmd.count_ctx_in_date*
 
 Allows `count_ctx_in_date` command execution. If enabled, it allows counting contexts in date range. *Default:* `True`
 
-- `Enable: get day note` *cmd.get_day_note*
+- `Tool: get day note` *cmd.get_day_note*
 
 Allows `get_day_note` command execution. If enabled, it allows retrieving day note for specific date. *Default:* `True`
 
-- `Enable: add day note` *cmd.add_day_note*
+- `Tool: add day note` *cmd.add_day_note*
 
 Allows `add_day_note` command execution. If enabled, it allows adding day note for specific date. *Default:* `True`
 
-- `Enable: update day note` *cmd.update_day_note*
+- `Tool: update day note` *cmd.update_day_note*
 
 Allows `update_day_note` command execution. If enabled, it allows updating day note for specific date. *Default:* `True`
 
-- `Enable: remove day note` *cmd.remove_day_note*
+- `Tool: remove day note` *cmd.remove_day_note*
 
 Allows `remove_day_note` command execution. If enabled, it allows removing day note for specific date. *Default:* `True`
 
@@ -2235,7 +2333,7 @@ If enabled, then a tray notification will be shown on every run of the job. *Def
 
 ## DALL-E 3: Image Generation (inline)
 
-The plugin integrates `DALL-E 3` image generation with any chat mode. Simply enable it and request an image in Chat mode, using a standard model such as `GPT-4`. The plugin does not require the `Execute commands` option to be enabled.
+The plugin integrates `DALL-E 3` image generation with any chat mode. Simply enable it and request an image in Chat mode, using a standard model such as `GPT-4`. The plugin does not require the `+ Tools` option to be enabled.
 
 **Options**
 
@@ -2243,6 +2341,12 @@ The plugin integrates `DALL-E 3` image generation with any chat mode. Simply ena
 
 The prompt is used to generate a query for the `DALL-E` image generation model, which runs in the background.
 
+##  Experts (inline)
+
+The plugin allows calling experts in any chat mode. This is the inline Experts (co-op) mode.
+
+See the `Work modes -> Experts` section for more details.
+
 ## GPT-4 Vision (inline)
 
 The plugin integrates vision capabilities across all chat modes, not just Vision mode. Once enabled, it allows the model to seamlessly switch to vision processing in the background whenever an image attachment or vision capture is detected.
@@ -2263,13 +2367,56 @@ The prompt used for vision mode. It will append or replace current system prompt
 
 Replace whole system prompt with vision prompt against appending it to the current prompt. *Default:* `False`
 
-- `Enable: capturing images from camera` *cmd.camera_capture*
+- `Tool: capturing images from camera` *cmd.camera_capture*
+
+Allows `capture` command execution. If enabled, model will be able to capture images from camera itself. The `+ Tools` option must be enabled. *Default:* `False`
+
+- `Tool: making screenshots` *cmd.make_screenshot*
+
+Allows `screenshot` command execution. If enabled, model will be able to making screenshots itself. The `+ Tools` option must be enabled. *Default:* `False`
+
+## Mailer
+
+Enables the sending, receiving, and reading of emails from the inbox. Currently, only SMTP is supported. More options coming soon.
+
+**Options**
+
+- `From (email)` *from_email*
+
+From (email), e.g. me@domain.com
+
+- `Tool: send_mail` *cmd.send_mail*
+
+Allows `send_mail` command execution. If enabled, model will be able to sending emails.
+
+- `Tool: receive_emails` *cmd.receive_emails*
+
+Allows `receive_emails` command execution. If enabled, model will be able to receive emails from the server.
+
+- `Tool: get_email_body` *cmd.get_email_body*
+
+Allows `get_email_body` command execution. If enabled, model will be able to receive message body from the server.
+
+- `SMTP Host` *smtp_host*
+
+SMTP Host, e.g. smtp.domain.com
+
+- `SMTP Port (Inbox)` *smtp_port_inbox*
+
+SMTP Port, default: 995
+
+- `SMTP Port (Outbox)` *smtp_port_outbox*
+
+SMTP Port, default: 465
+
+- `SMTP User` *smtp_user*
+
+SMTP User, e.g. user@domain.com
 
-Allows `capture` command execution. If enabled, model will be able to capture images from camera itself. The `Execute commands` option must be enabled. *Default:* `False`
+- `SMTP Password` *smtp_password*
 
-- `Enable: making screenshots` *cmd.make_screenshot*
+SMTP Password.
 
-Allows `screenshot` command execution. If enabled, model will be able to making screenshots itself. The `Execute commands` option must be enabled. *Default:* `False`
 
 ## Real Time
 
@@ -2306,325 +2453,354 @@ List of extra prompts - prompts that will be appended to system prompt.
 All active extra prompts defined on list will be appended to the system prompt in the order they are listed here.
 
 
+## Voice Control (inline)
+
+The plugin provides voice control command execution within a conversation.
+
+See the ``Accessibility`` section for more details.
+
+
 # Creating Your Own Plugins
 
 You can create your own plugin for **PyGPT** at any time. The plugin can be written in Python and then registered with the application just before launching it. All plugins included with the app are stored in the `plugin` directory - you can use them as coding examples for your own plugins.
 
 PyGPT can be extended with:
 
-- Custom plugins
+- custom models
 
-- Custom LLMs wrappers
+- custom plugins
 
-- Custom vector store providers
+- custom LLMs wrappers
 
-- Custom data loaders
+- custom vector store providers
 
-- Custom audio input providers
+- custom data loaders
 
-- Custom audio output providers
+- custom audio input providers
 
-- Custom web search engine providers
+- custom audio output providers
 
+- custom web search engine providers
 
-**Examples (tutorial files)** 
 
-See the `examples` directory in this repository with examples of custom launcher, plugin, vector store, LLM (Langchain and Llama-index) provider and data loader:
+See the section `Extending PyGPT / Adding a custom plugin` for more details.
 
-- `examples/custom_launcher.py`
+# Functions and commands execution
 
-- `examples/example_audio_input.py`
+**Tip** remember to enable the `+ Tools` checkbox to enable execution of tools and commands from plugins.
 
-- `examples/example_audio_output.py`
+From version `2.2.20` PyGPT uses native API function calls by default. You can go back to internal syntax (described below) by switching off option `Config -> Settings -> Prompts -> Use native API function calls`. Native API function calls are available in Chat, Completion and Assistant modes only (using OpenAI API).
 
-- `examples/example_data_loader.py`
+In background, **PyGPT** uses an internal syntax to define commands and their parameters, which can then be used by the model and executed on the application side or even directly in the system. This syntax looks as follows (example command below):
 
-- `examples/example_llm.py`
+```~###~{"cmd": "send_email", "params": {"quote": "Why don't skeletons fight each other? They don't have the guts!"}}~###~```
 
-- `examples/example_plugin.py`
+It is a JSON object wrapped between `~###~`. The application extracts the JSON object from such formatted text and executes the appropriate function based on the provided parameters and command name. Many of these types of commands are defined in plugins (e.g., those used for file operations or internet searches). You can also define your own commands using the `Custom Commands` plugin, or simply by creating your own plugin and adding it to the application.
 
-- `examples/example_vector_store.py`
+**Tip:** The `+ Tools` option checkbox must be enabled to allow the execution of commands from plugins. Disable the option if you do not want to use commands, to prevent additional token usage (as the command execution system prompt consumes additional tokens).
 
-- `examples/example_web_search.py`
+![v2_code_execute](https://github.com/user-attachments/assets/0b96b362-52ca-4928-9675-a39038d787a1)
 
-These example files can be used as a starting point for creating your own extensions for **PyGPT**.
+When native API function calls are disabled, a special system prompt responsible for invoking commands is added to the main system prompt if the `+ Tools` option is active.
 
-Extending PyGPT with custom plugins, LLMs wrappers and vector stores:
+However, there is an additional possibility to define your own commands and execute them with the help of GPT.
+These are functions - defined on the OpenAI API side and described using JSON objects. You can find a complete guide on how to define functions here:
 
-- You can pass custom plugin instances, LLMs wrappers and vector store providers to the launcher.
+https://platform.openai.com/docs/guides/function-calling
 
-- This is useful if you want to extend PyGPT with your own plugins, vectors storage and LLMs.
+https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models
 
-To register custom plugins:
+PyGPT offers compatibility of these functions with commands used in the application. All you need to do is define the appropriate functions using the syntax required by OpenAI, and PyGPT will do the rest, translating such syntax on the fly into its own internal format.
 
-- Pass a list with the plugin instances as `plugins` keyword argument.
+You can define functions for modes: `Chat` and `Assistants`.
+Note that - in Chat mode, they should be defined in `Presets`, and for Assistants, in the `Assistant` settings.
 
-To register custom LLMs wrappers:
+**Example of usage:**
 
-- Pass a list with the LLMs wrappers instances as `llms` keyword argument.
+1) Chat
 
-To register custom vector store providers:
+Create a new Preset, open the Preset edit dialog and add a new function using `+ Function` button with the following content:
 
-- Pass a list with the vector store provider instances as `vector_stores` keyword argument.
+**Name:** `send_email`
 
-To register custom data loaders:
+**Description:** `Sends a quote using email`
 
-- Pass a list with the data loader instances as `loaders` keyword argument.
+**Params (JSON):**
 
-To register custom audio input providers:
+```json
+{
+        "type": "object",
+        "properties": {
+            "quote": {
+                "type": "string",
+                "description": "A generated funny quote"
+            }
+        },
+        "required": [
+            "quote"
+        ]
+}
+```
 
-- Pass a list with the audio input provider instances as `audio_input` keyword argument.
+Then, in the `Custom Commands` plugin, create a new command with the same name and the same parameters:
 
-To register custom audio output providers:
+**Command name:** `send_email`
 
-- Pass a list with the audio output provider instances as `audio_output` keyword argument.
+**Instruction/prompt:** `send mail` *(don't needed, because it will be called on OpenAI side)*
 
-To register custom web providers:
+**Params list:** `quote`
 
-- Pass a list with the web provider instances as `web` keyword argument.
+**Command to execute:** `echo "OK. Email sent: {quote}"`
 
-**Example:**
+At next, enable the `+ Tools` option and enable the plugin.
 
+Ask GPT in Chat mode:
 
-```python
-# custom_launcher.py
+```Create a funny quote and email it```
 
-from pygpt_net.app import run
-from plugins import CustomPlugin, OtherCustomPlugin
-from llms import CustomLLM
-from vector_stores import CustomVectorStore
+In response you will receive prepared command, like this:
 
-plugins = [
-    CustomPlugin(),
-    OtherCustomPlugin(),
-]
-llms = [
-    CustomLLM(),
-]
-vector_stores = [
-    CustomVectorStore(),
-]
+```~###~{"cmd": "send_email", "params": {"quote": "Why do we tell actors to 'break a leg?' Because every play has a cast!"}}~###~```
 
-run(
-    plugins=plugins,
-    llms=llms,
-    vector_stores=vector_stores
-)
-```
+After receiving this, PyGPT will execute the system `echo` command with params given from `params` field and replacing `{quote}` placeholder with `quote` param value.
 
-## Handling events
+As a result, response like this will be sent to the model:
 
-In the plugin, you can receive and modify dispatched events.
-To do this, create a method named `handle(self, event, *args, **kwargs)` and handle the received events like here:
+```[{"request": {"cmd": "send_email"}, "result": "OK. Email sent: Why do we tell actors to 'break a leg?' Because every play has a cast!"}]```
 
-```python
-# custom_plugin.py
 
-from pygpt_net.core.dispatcher import Event
+2) Assistant
 
+In this mode (via Assistants API), it should be done similarly, with the difference that here the functions should be defined in the assistant's settings.
 
-def handle(self, event: Event, *args, **kwargs):
-    """
-    Handle dispatched events
+With this flow you can use both forms - OpenAI and PyGPT - to define and execute commands and functions in the application. They will cooperate with each other and you can use them interchangeably.
 
-    :param event: event object
-    """
-    name = event.name
-    data = event.data
-    ctx = event.ctx
+# Tools
 
-    if name == Event.INPUT_BEFORE:
-        self.some_method(data['value'])
-    elif name == Event.CTX_BEGIN:
-        self.some_other_method(ctx)
-    else:
-    	# ...
-```
+PyGPT features several useful tools, including:
 
-**List of Events**
+- Notepad
+- Painter
+- Calendar
+- Indexer
+- Media Player
+- Image viewer
+- Text editor
+- Transcribe audio/video files
+- Python Code Interpreter
+- HTML/JS Canvas (built-in HTML renderer)
 
-Event names are defined in `Event` class in `pygpt_net.core.dispatcher.Event`.
+![v2_tool_menu](https://github.com/user-attachments/assets/c8041cdc-64fd-41a5-b1af-8c987b06e5f0)
 
-Syntax: `event name` - triggered on, `event data` *(data type)*:
 
-- `AI_NAME` - when preparing an AI name, `data['value']` *(string, name of the AI assistant)*
+## Notepad
 
-- `AUDIO_INPUT_STOP` - force stop audio input
+The application has a built-in notepad, divided into several tabs. This can be useful for storing information in a convenient way, without the need to open an external text editor. The content of the notepad is automatically saved whenever the content changes.
 
-- `AUDIO_INPUT_TOGGLE` - when speech input is enabled or disabled, `data['value']` *(bool, True/False)*
+![v2_notepad](https://github.com/szczyglis-dev/py-gpt/assets/61396542/f6aa0126-bad1-4e6c-ace6-72e979186433)
 
-- `AUDIO_OUTPUT_STOP` - force stop audio output
+## Painter
 
-- `AUDIO_OUTPUT_TOGGLE` - when speech output is enabled or disabled, `data['value']` *(bool, True/False)*
+Using the `Painter` tool, you can create quick sketches and submit them to the model for analysis. You can also edit opened from disk or captured from camera images, for example, by adding elements like arrows or outlines to objects. Additionally, you can capture screenshots from the system - the captured image is placed in the drawing tool and attached to the query being sent.
 
-- `AUDIO_READ_TEXT` - on text read with speech synthesis, `data['value']` *(str)*
+![v2_draw](https://github.com/szczyglis-dev/py-gpt/assets/61396542/09c1de36-1241-4330-9fd7-67c6e09888fa)
 
-- `CMD_EXECUTE` - when a command is executed, `data['commands']` *(list, commands and arguments)*
+To capture the screenshot just click on the `Ask with screenshot` option in a tray-icon dropdown:
 
-- `CMD_INLINE` - when an inline command is executed, `data['commands']` *(list, commands and arguments)*
+![v2_screenshot](https://github.com/szczyglis-dev/py-gpt/assets/61396542/7305a814-ca76-4f8f-8908-47f6a9496fa5)
 
-- `CMD_SYNTAX` - when appending syntax for commands, `data['prompt'], data['syntax']` *(string, list, prompt and list with commands usage syntax)*
+## Calendar
 
-- `CMD_SYNTAX_INLINE` - when appending syntax for commands (inline mode), `data['prompt'], data['syntax']` *(string, list, prompt and list with commands usage syntax)*
-
-- `CTX_AFTER` - after the context item is sent, `ctx`
-
-- `CTX_BEFORE` - before the context item is sent, `ctx`
-
-- `CTX_BEGIN` - when context item create, `ctx`
-
-- `CTX_END` - when context item handling is finished, `ctx`
-
-- `CTX_SELECT` - when context is selected on list, `data['value']` *(int, ctx meta ID)*
-
-- `DISABLE` - when the plugin is disabled, `data['value']` *(string, plugin ID)*
-
-- `ENABLE` - when the plugin is enabled, `data['value']` *(string, plugin ID)*
+Using the calendar, you can go back to selected conversations from a specific day and add daily notes. After adding a note, it will be marked on the list, and you can change the color of its label by right-clicking and selecting `Set label color`. By clicking on a particular day of the week, conversations from that day will be displayed.
 
-- `FORCE_STOP` - on force stop plugins
+![v2_calendar](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c7d17375-b61f-452c-81bc-62a7d466fc18)
 
-- `INPUT_BEFORE` - upon receiving input from the textarea, `data['value']` *(string, text to be sent)*
 
-- `MODE_BEFORE` - before the mode is selected `data['value'], data['prompt']` *(string, string, mode ID)*
+## Indexer
 
-- `MODE_SELECT` - on mode select `data['value']` *(string, mode ID)*
 
-- `MODEL_BEFORE` - before the model is selected `data['value']` *(string, model ID)*
+This tool allows indexing of local files or directories and external web content to a vector database, which can then be used with the `Chat with Files` mode. Using this tool, you can manage local indexes and add new data with built-in `LlamaIndex` integration.
 
-- `MODEL_SELECT` - on model select `data['value']` *(string, model ID)*
+![v2_tool_indexer](https://github.com/szczyglis-dev/py-gpt/assets/61396542/1caeab6e-6119-44e2-a7cb-ed34f8fe9e30)
 
-- `PLUGIN_SETTINGS_CHANGED` - on plugin settings update
+## Media Player
 
-- `PLUGIN_OPTION_GET` - on request for plugin option value `data['name'], data['value']` *(string, any, name of requested option, value)*
 
-- `POST_PROMPT` - after preparing a system prompt, `data['value']` *(string, system prompt)*
+A simple video/audio player that allows you to play video files directly from within the app.
 
-- `PRE_PROMPT` - before preparing a system prompt, `data['value']` *(string, system prompt)*
 
-- `SYSTEM_PROMPT` - when preparing a system prompt, `data['value']` *(string, system prompt)*
+## Image Viewer
 
-- `UI_ATTACHMENTS` - when the attachment upload elements are rendered, `data['value']` *(bool, show True/False)*
 
-- `UI_VISION` - when the vision elements are rendered, `data['value']` *(bool, show True/False)*
+A simple image browser that lets you preview images directly within the app.
 
-- `USER_NAME` - when preparing a user's name, `data['value']` *(string, name of the user)*
 
-- `USER_SEND` - just before the input text is sent, `data['value']` *(string, input text)*
+## Text Editor
 
 
-You can stop the propagation of a received event at any time by setting `stop` to `True`:
+A simple text editor that enables you to edit text files directly within the app.
 
-```
-event.stop = True
-```
 
-# Functions and commands execute
+## Transcribe Audio/Video Files
 
-**Tip:** `gpt-4-1106-preview` is the best model to use for command handling, The `gpt-4-turbo-preview` model can sometimes refuse to execute commands.
 
-**PyGPT** uses an internal syntax to define commands and their parameters, which can then be used by the model and executed on the application side or even directly in the system. This syntax looks as follows (example command below):
+An audio transcription tool with which you can prepare a transcript from a video or audio file. It will use a speech recognition plugin to generate the text from the file.
 
-```~###~{"cmd": "send_email", "params": {"quote": "Why don't skeletons fight each other? They don't have the guts!"}}~###~```
 
-It is JSON wrapped between `~###~`. The application extracts the JSON object from such formatted text and executes the appropriate function based on the provided parameters and command name. Many of these types of commands are defined in plugins (e.g., those used for file operations or internet searches). You can also define your own commands using the `Custom Commands` plugin, or simply by creating your own plugin and adding it to the application.
+## Python Code Interpreter
 
-**Tip:** The `Execute commands` option checkbox must be enabled to allow the execution of commands from plugins. Disable the option if you do not want to use commands, to prevent additional token usage (as the command execution system prompt consumes additional tokens).
 
-![v2_code_execute](https://github.com/szczyglis-dev/py-gpt/assets/61396542/d5181eeb-6ab4-426f-93f0-037d256cb078)
+This tool allows you to run Python code directly from within the app. It is integrated with the `Code Interpreter` plugin, ensuring that code generated by the model is automatically available from the interpreter. In the plugin settings, you can enable the execution of code in a Docker environment.
 
-A special system prompt responsible for invoking commands is added to the main system prompt if the `Execute commands` option is active.
+## HTML/JS Canvas
 
-However, there is an additional possibility to define your own commands and execute them with the help of GPT.
-These are functions - defined on the OpenAI API side and described using JSON objects. You can find a complete guide on how to define functions here:
+Allows to render HTML/JS code in HTML Canvas (built-in renderer based on Chromium). To use it, just ask the model to render the HTML/JS code in built-in browser (HTML Canvas). Tool is integrated with the `Code Interpreter` plugin.
 
-https://platform.openai.com/docs/guides/function-calling
 
-https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models
+# Token usage calculation
 
+## Input tokens
 
-From version **2.0.150**, PyGPT offers compatibility of these functions with commands used in the application. All you need to do is define the appropriate functions using the syntax required by OpenAI, and PyGPT will do the rest, translating such syntax on the fly into its own internal format.
+The application features a token calculator. It attempts to forecast the number of tokens that 
+a particular query will consume and displays this estimate in real time. This gives you improved 
+control over your token usage. The app provides detailed information about the tokens used for the user's prompt, 
+the system prompt, any additional data, and those used within the context (the memory of previous entries).
 
-You can define functions for modes: `Chat` and `Assistants`.
-Note that - in Chat mode, they should be defined in `Presets`, and for Assistants, in the `Assistant` settings.
+**Remember that these are only approximate calculations and do not include, for example, the number of tokens consumed by some plugins. You can find the exact number of tokens used on the OpenAI website.**
 
-**Example of usage:**
+![v2_tokens1](https://github.com/user-attachments/assets/e131a880-986d-4014-b5fd-9820516c6a10)
 
-1) Chat
+## Total tokens
 
-Create a new Preset, open the Preset edit dialog and add a new function using `+ Function` button with the following content:
+After receiving a response from the model, the application displays the actual total number of tokens used for the query (received from the API).
 
-**Name:** `send_email`
+![v2_tokens2](https://github.com/user-attachments/assets/52cd355f-c7b0-432b-9ff9-31c8a6a60b89)
 
-**Description:** `Sends a quote using email`
 
-**Params (JSON):**
+# Accessibility
 
-```json
-{
-        "type": "object",
-        "properties": {
-            "quote": {
-                "type": "string",
-                "description": "A generated funny quote"
-            }
-        },
-        "required": [
-            "quote"
-        ]
-}
-```
+Since version `2.2.8`, PyGPT has added beta support for disabled people and voice control. This may be very useful for blind people.
 
-Then, in the `Custom Commands` plugin, create a new command with the same name and the same parameters:
+In the `Config / Accessibility` menu, you can turn on accessibility features such as:
 
-**Command name:** `send_email`
 
-**Instruction/prompt:** `send mail` *(don't needed, because it will be called on OpenAI side)*
+- activating voice control
 
-**Params list:** `quote`
+- translating actions and events on the screen with audio speech
 
-**Command to execute:** `echo "OK. Email sent: {quote}"`
+- setting up keyboard shortcuts for actions.
 
-At next, enable the `Execute commands` option and enable the plugin.
 
-Ask GPT in Chat mode:
+**Using voice control**
 
-```Create a funny quote and email it```
+Voice control can be turned on in two ways: globally, through settings in `Config -> Accessibility`, and by using the `Voice control (inline)` plugin. Both options let you use the same voice commands, but they work a bit differently - the global option allows you to run commands outside of a conversation, anywhere, while the plugin option lets you execute commands directly during a conversation – allowing you to interact with the model and execute commands at the same time, within the conversation.
 
-In response you will receive prepared command, like this:
+In the plugin (inline) option, you can also turn on a special trigger word that will be needed for content to be recognized as a voice command. You can set this up by going to `Plugins -> Settings -> Voice Control (inline)`:
 
-```~###~{"cmd": "send_email", "params": {"quote": "Why do we tell actors to 'break a leg?' Because every play has a cast!"}}~###~```
+```bash
+Magic prefix for voice commands
+```
 
-After receiving this, PyGPT will execute the system `echo` command with params given from `params` field and replacing `{quote}` placeholder with `quote` param value.
+**Tip:** When the voice control is enabled via a plugin, simply provide commands while providing the content of the conversation by using the standard `Microphone` button.
 
-As a result, response like this will be sent to the model:
 
-```[{"request": {"cmd": "send_email"}, "result": "OK. Email sent: Why do we tell actors to 'break a leg?' Because every play has a cast!"}]```
+**Enabling voice control globally**
 
 
-2) Assistant
+Turn on the voice control option in `Config / Accessibility`:
 
-In this mode (via Assistants API), it should be done similarly, with the difference that here the functions should be defined in the assistant's settings.
 
-With this flow you can use both forms - OpenAI and PyGPT - to define and execute commands and functions in the application. They will cooperate with each other and you can use them interchangeably.
+```bash
+Enable voice control (using microphone)
+```
 
-# Token usage calculation
+Once you enable this option, an `Voice Control` button will appear at the bottom right corner of the window. When you click on this button, the microphone will start listening; clicking it again stops listening and starts recognizing the voice command you said. You can cancel voice recording at any time with the `ESC` key. You can also set a keyboard shortcut to turn voice recording on/off.
+
+
+Voice command recognition works based on a model, so you don't have to worry about saying things perfectly.
+
+
+**Here's a list of commands you can ask for by voice:**
+
+- Get the current application status
+- Exit the application
+- Enable audio output
+- Disable audio output
+- Enable audio input
+- Disable audio input
+- Add a memo to the calendar
+- Clear memos from calendar
+- Read the calendar memos
+- Enable the camera
+- Disable the camera
+- Capture image from camera
+- Create a new context
+- Go to the previous context
+- Go to the next context
+- Go to the latest context
+- Focus on the input
+- Send the input
+- Clear the input
+- Get current conversation info
+- Get available commands list
+- Stop executing current action
+- Clear the attachments
+- Read the last conversation entry
+- Read the whole conversation
+- Rename current context
+- Search for a conversation
+- Clear the search results
+- Send the message to input
+- Append message to current input without sending it
+- Switch to chat mode
+- Switch to chat with files (llama-index) mode
+- Switch to the next mode
+- Switch to the previous mode
+- Switch to the next model
+- Switch to the previous model
+- Add note to notepad
+- Clear notepad contents
+- Read current notepad contents
+- Switch to the next preset
+- Switch to the previous preset
+- Switch to the chat tab
+- Switch to the calendar tab
+- Switch to the draw (painter) tab
+- Switch to the files tab
+- Switch to the notepad tab
+- Switch to the next tab
+- Switch to the previous tab
+- Start listening for voice input
+- Stop listening for voice input
+- Toggle listening for voice input
+
+More commands coming soon.
+
+Just ask for an action that matches one of the descriptions above. These descriptions are also known to the model, and relevant commands are assigned to them. When you voice a command that fits one of those patterns, the model will trigger the appropriate action.
+
+
+For convenience, you can enable a short sound to play when voice recording starts and stops. To do this, turn on the option:
+
+
+```bash
+Audio notify microphone listening start/stop
+```
 
-## Input tokens
+To enable a sound notification when a voice command is recognized and command execution begins, turn on the option:
 
-The application features a token calculator. It attempts to forecast the number of tokens that 
-a particular query will consume and displays this estimate in real time. This gives you improved 
-control over your token usage. The app provides detailed information about the tokens used for the user's prompt, 
-the system prompt, any additional data, and those used within the context (the memory of previous entries).
 
-**Remember that these are only approximate calculations and do not include, for example, the number of tokens consumed by some plugins. You can find the exact number of tokens used on the OpenAI website.**
+```bash
+Audio notify voice command execution
+```
 
-![v2_tokens1](https://github.com/szczyglis-dev/py-gpt/assets/61396542/29b610be-9e96-41cc-84f0-1b946886f801)
+For voice translation of on-screen events and information about completed commands via speech synthesis, you can turn on the option:
 
-## Total tokens
+```bash
+Use voice synthesis to describe events on the screen.
+```
 
-After receiving a response from the model, the application displays the actual total number of tokens used for the query (received from the API).
+![v2_access](https://github.com/szczyglis-dev/py-gpt/assets/61396542/02dd161b-6fb1-48f9-9217-40c658888833)
 
-![v2_tokens2](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c81e95b5-7c33-41a6-8910-21d674db37e5)
 
 # Configuration
 
@@ -2636,23 +2812,47 @@ The following basic options can be modified directly within the application:
 Config -> Settings...
 ```
 
-![v2_settings](https://github.com/szczyglis-dev/py-gpt/assets/61396542/21d7e43d-858f-4bc7-a06f-ec848338e7a9)
+![v2_settings](https://github.com/user-attachments/assets/003b0f86-8225-4478-8525-fb9324ac5c88)
 
 **General**
 
-- `OpenAI API KEY`: The personal API key you'll need to enter into the application for it to function.
+- `Minimize to tray on exit`: Minimize to tray icon on exit. Tray icon enabled is required for this option to work. Default: False.
+
+- `Render engine`: chat output render engine: `WebEngine / Chromium` - for full HTML/CSS and `Legacy (markdown)` for legacy, simple markdown CSS output. Default: WebEngine / Chromium.
+
+- `OpenGL hardware acceleration`: enables hardware acceleration in `WebEngine / Chromium` renderer.  Default: False.
+
+- `Application environment (os.environ)`: Additional environment vars to set on application start.
+
+**API Keys**
+
+- `OpenAI API KEY`: Required for the OpenAI API. If you wish to use custom endpoints or local APIs, then you may enter any value here.
 
 - `OpenAI ORGANIZATION KEY`: The organization's API key, which is optional for use within the application.
 
 - `API Endpoint`: OpenAI API endpoint URL, default: https://api.openai.com/v1.
 
-- `Number of notepads`: Number of notepad tabs. Restart of the application is required for this option to take effect.
+- `Proxy address`: Proxy address to be used for connection; supports HTTP/SOCKS.
 
-- `Minimize to tray on exit`: Minimize to tray icon on exit. Tray icon enabled is required for this option to work. Default: False.
+- `Google API KEY`: Required for the Google API and Gemini models.
+
+- `Anthropic API KEY`: Required for the Anthropic API and Claude models.
+
+- `HuggingFace API KEY`: Required for the HuggingFace API.
+
+- `OpenAI API version`: Azure OpenAI API version, e.g. 2023-07-01-preview
+
+- `Azure OpenAI API endpoint`: Azure OpenAI API endpoint, https://<your-resource-name>.openai.azure.com/
 
 **Layout**
 
-- `Font Size (chat window)`: Adjusts the font size in the chat window.
+- `Zoom`: Adjusts the zoom in chat window (web render view). `WebEngine / Chromium` render mode only.
+
+- `Style (chat)`: Chat style (Blocks, or ChatGPT-like, or ChatGPT-like Wide. `WebEngine / Chromium` render mode only.
+
+- `Code syntax highlight`: Syntax highlight theme in code blocks. `WebEngine / Chromium` render mode only.
+
+- `Font Size (chat window)`: Adjusts the font size in the chat window (plain-text) and notepads.
 
 - `Font Size (input)`: Adjusts the font size in the input window.
 
@@ -2672,8 +2872,6 @@ Config -> Settings...
 
 - `Use theme colors in chat window`: Use color theme in chat window, Default: True.
 
-- `Disable markdown formatting in output`: Enables plain-text display in output window, Default: False.
-
 **Files and attachments**
 
 - `Store attachments in the workdir upload directory`: Enable to store a local copy of uploaded attachments for future use. Default: True
@@ -2682,12 +2880,30 @@ Config -> Settings...
 
 - `Directory for file downloads`: Subdirectory for downloaded files, e.g. in Assistants mode, inside "data". Default: "download"
 
+- `Verbose mode`: Enabled verbose mode when using attachment as additional context.
+
+- `Model for querying index`: Model to use for preparing query and querying the index when the RAG option is selected.
+
+- `Model for attachment content summary`: Model to use when generating a summary for the content of a file when the Summary option is selected.
+
+- `Use history in RAG query`: When enabled, the content of the entire conversation will be used when preparing a query if mode is RAG or Summary.
+
+- `RAG limit`: Only if the option `Use history in RAG query` is enabled. Specify the limit of how many recent entries in the conversation will be used when generating a query for RAG. 0 = no limit.
+
 **Context**
 
 - `Context Threshold`: Sets the number of tokens reserved for the model to respond to the next prompt.
 
 - `Limit of last contexts on list to show  (0 = unlimited)`: Limit of the last contexts on list, default: 0 (unlimited)
 
+- `Show context groups on top of the context list`: Display groups on top, default: False
+
+- `Show date separators on the context list`: Show date periods, default: True
+
+- `Show date separators in groups on the context list`: Show date periods in groups, default: True
+
+- `Show date separators in pinned on the context list`: Show date periods in pinned items, default: False
+
 - `Use Context`: Toggles the use of conversation context (memory of previous inputs).
 
 - `Store History`: Toggles conversation history store.
@@ -2700,12 +2916,14 @@ Config -> Settings...
 
 - `Search also in conversation content, not only in titles`: When enabled, context search will also consider the content of conversations, not just the titles of conversations.
 
-- `Show Llama-index sources`: If enabled, sources utilized will be displayed in the response (if available, it will not work in streamed chat).
+- `Show LlamaIndex sources`: If enabled, sources utilized will be displayed in the response (if available, it will not work in streamed chat).
 
-- `Always show audio icon`: If enabled, then read with audio icon will be always displayed.
+- `Show code interpreter output`: If enabled, output from the code interpreter in the Assistant API will be displayed in real-time (in stream mode), Default: True.
 
 - `Use extra context output`: If enabled, plain text output (if available) from command results will be displayed alongside the JSON output, Default: True.
 
+- `Convert lists to paragraphs`: If enabled, lists (ul, ol) will be converted to paragraphs (p), Default: True.
+
 - `Model used for auto-summary`: Model used for context auto-summary (default: *gpt-3.5-turbo-1106*).
 
 **Models**
@@ -2726,6 +2944,8 @@ Config -> Settings...
 
 **Prompts**
 
+- `Use native API function calls`: Use API function calls to run commands from plugins instead of using command prompts - Chat and Assistants modes ONLY, default: True
+
 - `Command execute: instruction`: Prompt for appending command execution instructions. Placeholders: {schema}, {extra}
 
 - `Command execute: extra footer (non-Assistant modes)`: Extra footer to append after commands JSON schema.
@@ -2736,9 +2956,17 @@ Config -> Settings...
 
 - `Context: auto-summary (user message)`: User message for context auto-summary. Placeholders: {input}, {output}
 
-- `Agent: continue`: Prompt sent to automatically continue the conversation. Default: `continue...`
+- `Agent: evaluation prompt in loop (LlamaIndex)`: Prompt used for evaluating the response in Agents (LlamaIndex) mode.
+
+- `Agent: system instruction (Legacy)`: Prompt to instruct how to handle autonomous mode.
+
+- `Agent: continue (Legacy)`: Prompt sent to automatically continue the conversation.
 
-- `Agent: goal update`: Prompt to instruct how to update current goal status.
+- `Agent: continue (always, more steps) (Legacy)`: Prompt sent to always automatically continue the conversation (more reasoning - "Always continue..." option).
+
+- `Agent: goal update (Legacy)`: Prompt to instruct how to update current goal status.
+
+- `Experts: Master prompt`: Prompt to instruct how to handle experts.
 
 - `DALL-E: image generate`: Prompt for generating prompts for DALL-E (if raw-mode is disabled).
 
@@ -2754,206 +2982,1052 @@ Config -> Settings...
 
 **Vision**
 
+- `Vision: Camera Input Device`: Video capture camera index (index of the camera, default: 0).
+
 - `Vision: Camera capture width (px)`: Video capture resolution (width).
 
 - `Vision: Camera capture height (px)`: Video capture resolution (height).
 
-- `Vision: Camera IDX (number)`: Video capture camera index (number of camera).
+- `Vision: Image capture quality`: Video capture image JPEG quality (%).
+
+**Audio**
+
+- `Audio Input Device`: Selects the audio device for Microphone input.
+
+- `Channels`: Input channels, default: 1
+
+- `Sampling Rate`: Sampling rate, default: 44100
+
+**Indexes (LlamaIndex)**
+
+- `Indexes`: List of created indexes.
+
+- `Vector Store`: Vector store to use (vector database provided by LlamaIndex).
+
+- `Vector Store (**kwargs)`: Keyword arguments for vector store provider (api_key, index_name, etc.).
+
+- `Embeddings provider`: Embeddings provider.
+
+- `Embeddings provider (ENV)`: ENV vars to embeddings provider (API keys, etc.).
+
+- `Embeddings provider (**kwargs)`: Keyword arguments for embeddings provider (model name, etc.).
+
+- `RPM limit for embeddings API calls`: Specify the limit of maximum requests per minute (RPM), 0 = no limit.
+
+- `Recursive directory indexing`: Enables recursive directory indexing, default is False.
+
+- `Replace old document versions in the index during re-indexing`: If enabled, previous versions of documents will be deleted from the index when the newest versions are indexed, default is True.
+
+- `Excluded file extensions`: File extensions to exclude if no data loader for this extension, separated by comma.
+
+- `Force exclude files`: If enabled, the exclusion list will be applied even when the data loader for the extension is active. Default: False.
+
+- `Stop indexing on error`: If enabled, indexing will stop whenever an error occurs Default: True.
+
+- `Custom metadata to append/replace to indexed documents (file)`: Define custom metadata key => value fields for specified file extensions, separate extensions by comma.\nAllowed placeholders: {path}, {relative_path} {filename}, {dirname}, {relative_dir} {ext}, {size}, {mtime}, {date}, {date_time}, {time}, {timestamp}. Use * (asterisk) as extension if you want to apply field to all files. Set empty value to remove field with specified key from metadata.
+
+- `Custom metadata to append/replace to indexed documents (web)`: Define custom metadata key => value fields for specified external data loaders.\nAllowed placeholders: {date}, {date_time}, {time}, {timestamp} + {data loader args}
+
+- `Additional keyword arguments (**kwargs) for data loaders`: Additional keyword arguments, such as settings, API keys, for the data loader. These arguments will be passed to the loader; please refer to the LlamaIndex or LlamaHub loaders reference for a list of allowed arguments for the specified data loader.
+
+- `Use local models in Video/Audio and Image (vision) loaders`: Enables usage of local models in Video/Audio and Image (vision) loaders. If disabled then API models will be used (GPT-4 Vision and Whisper). Note: local models will work only in Python version (not compiled/Snap). Default: False.
+
+- `Auto-index DB in real time`: Enables conversation context auto-indexing in defined modes.
+
+- `ID of index for auto-indexing`: Index to use if auto-indexing of conversation context is enabled.
+
+- `Enable auto-index in modes`: List of modes with enabled context auto-index, separated by comma.
+
+- `DB (ALL), DB (UPDATE), FILES (ALL)`: Index the data – batch indexing is available here.
+
+- `Chat mode`: chat mode for use in query engine, default: context
+
+**Agent and experts**
+
+**General**
+
+- `Display a tray notification when the goal is achieved.`: If enabled, a notification will be displayed after goal achieved / finished run.
+
+**LlamaIndex Agents**
+
+- `Max steps (per iteration)` - Max steps is one iteration before goal achieved
+
+- `Max evaluation steps in loop` - Maximum evaluation steps to achieve the final result, set 0 to infinity
+
+- `Append and compare previous evaluation prompt in next evaluation` - If enabled, previous improvement prompt will be checked in next eval in loop, default: False
+
+- `Verbose` - enables verbose mode.
+
+**Legacy**
+
+- `Sub-mode for agents`: Sub-mode to use in Agent mode (chat, completion, langchain, llama_index, etc.). Default: chat.
+
+- `Sub-mode for experts`: Sub-mode to use in Experts mode (chat, completion, langchain, llama_index, etc.). Default: chat.
+
+- `Index to use`: Only if sub-mode is llama_index (Chat with Files), choose the index to use in both Agent and Expert modes.
+
+**Accessibility**
+
+- `Enable voice control (using microphone)`: enables voice control (using microphone and defined commands).
+
+- `Model`: model used for voice command recognition.
+
+- `Use voice synthesis to describe events on the screen.`: enables audio description of on-screen events.
+
+- `Use audio output cache`: If enabled, all static audio outputs will be cached on the disk instead of being generated every time. Default: True.
+
+- `Audio notify microphone listening start/stop`: enables audio "tick" notify when microphone listening started/ended.
+
+- `Audio notify voice command execution`: enables audio "tick" notify when voice command is executed.
+
+- `Control shortcut keys`: configuration for keyboard shortcuts for a specified actions.
+
+- `Blacklist for voice synthesis events describe (ignored events)`: list of muted events for 'Use voice synthesis to describe event' option.
+
+- `Voice control actions blacklist`: Disable actions in voice control; add actions to the blacklist to prevent execution through voice commands.
+
+**Updates**
+
+- `Check for updates on start`: Enables checking for updates on start. Default: True.
+
+- `Check for updates in background`: Enables checking for updates in background (checking every 5 minutes). Default: True.
+
+**Developer**
+
+- `Show debug menu`: Enables debug (developer) menu.
+
+- `Log and debug context`: Enables logging of context input/output.
+
+- `Log and debug events`: Enables logging of event dispatch.
+
+- `Log plugin usage to console`: Enables logging of plugin usage to console.
+
+- `Log DALL-E usage to console`: Enables logging of DALL-E usage to console.
+
+- `Log LlamaIndex usage to console`: Enables logging of LlamaIndex usage to console.
+
+- `Log Assistants usage to console`: Enables logging of Assistants API usage to console.
+
+- `Log level`: toggle log level (ERROR|WARNING|INFO|DEBUG)
+
+
+## JSON files
+
+The configuration is stored in JSON files for easy manual modification outside of the application. 
+These configuration files are located in the user's work directory within the following subdirectory:
+
+``` ini
+{HOME_DIR}/.config/pygpt-net/
+```
+
+## Manual configuration
+
+You can manually edit the configuration files in this directory (this is your work directory):
+
+``` ini
+{HOME_DIR}/.config/pygpt-net/
+```
+
+- `assistants.json` - stores the list of assistants.
+- `attachments.json` - stores the list of current attachments.
+- `config.json` - stores the main configuration settings.
+- `models.json` - stores models configurations.
+- `cache` - a directory for audio cache.
+- `capture` - a directory for captured images from camera and screenshots
+- `css` - a directory for CSS stylesheets (user override)
+- `history` - a directory for context history in `.txt` format.
+- `idx` - `LlamaIndex` indexes
+- `img` - a directory for images generated with `DALL-E 3` and `DALL-E 2`, saved as `.png` files.
+- `locale` - a directory for locales (user override)
+- `data` - a directory for data files and files downloaded/generated by GPT.
+- `presets` - a directory for presets stored as `.json` files.
+- `upload` - a directory for local copies of attachments coming from outside the workdir
+- `db.sqlite` - a database with contexts, notepads and indexes data records
+- `app.log` - a file with error and debug log
+
+---
+
+## Setting the Working Directory Using Command Line Arguments
+
+To set the current working directory using a command-line argument, use:
+
+```
+python3 ./run.py --workdir="/path/to/workdir"
+```
+or, for the binary version:
+
+```
+pygpt.exe --workdir="/path/to/workdir"
+```
+
+
+## Translations / Locale
+
+Locale `.ini` files are located in the app directory:
+
+``` ini
+./data/locale
+```
+
+This directory is automatically scanned when the application launches. To add a new translation, 
+create and save the file with the appropriate name, for example:
+
+``` ini
+locale.es.ini   
+```
+
+This will add Spanish as a selectable language in the application's language menu.
+
+**Overwriting CSS and locales with Your Own Files:**
+
+You can also overwrite files in the `locale` and `css` app directories with your own files in the user directory. 
+This allows you to overwrite language files or CSS styles in a very simple way - by just creating files in your working directory.
+
+
+``` ini
+{HOME_DIR}/.config/pygpt-net/
+```
+
+- `locale` - a directory for locales in `.ini` format.
+- `css` - a directory for CSS styles in `.css` format.
+
+**Adding Your Own Fonts**
+
+You can add your own fonts and use them in CSS files. To load your own fonts, you should place them in the `%workdir%/fonts` directory. Supported font types include: `otf`, `ttf`.
+You can see the list of loaded fonts in `Debug / Config`.
+
+**Example:**
+
+```
+%workdir%
+|_css
+|_data
+|_fonts
+   |_MyFont
+     |_MyFont-Regular.ttf
+     |_MyFont-Bold.ttf
+     |...
+```
+
+```css
+pre {{
+    font-family: 'MyFont';
+}}
+```
+
+## Data Loaders
+
+**Configuring data loaders**
+
+In the `Settings -> LlamaIndex -> Data loaders` section you can define the additional keyword arguments to pass into data loader instance.
+
+In most cases, an internal LlamaIndex loaders are used internally. 
+You can check these base loaders e.g. here:
+
+File: https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/readers/llama-index-readers-file/llama_index/readers/file
+
+Web: https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/readers/llama-index-readers-web
+
+**Tip:** to index an external data or data from the Web just ask for it, by using `Web Search` plugin, e.g. you can ask the model with `Please index the youtube video: URL to video`, etc. Data loader for a specified content will be choosen automatically.
+
+Allowed additional keyword arguments for built-in data loaders (files):
+
+**CSV Files**  (file_csv)
+
+- `concat_rows` - bool, default: `True`
+- `encoding` - str, default: `utf-8`
+
+**HTML Files**  (file_html)
+
+- `tag` - str, default: `section`
+- `ignore_no_id` - bool, default: `False`
+
+**Image (vision)**  (file_image_vision)
+
+This loader can operate in two modes: local model and API.
+If the local mode is enabled, then the local model will be used. The local mode requires a Python/PyPi version of the application and is not available in the compiled or Snap versions.
+If the API mode (default) is selected, then the OpenAI API and the standard vision model will be used. 
+
+**Note:** Usage of API mode consumes additional tokens in OpenAI API (for `GPT-4 Vision` model)!
+
+Local mode requires `torch`, `transformers`, `sentencepiece` and `Pillow` to be installed and uses the `Salesforce/blip2-opt-2.7b` model to describing images.
+
+- `keep_image` - bool, default: `False`
+- `local_prompt` - str, default: `Question: describe what you see in this image. Answer:`
+- `api_prompt` - str, default: `Describe what you see in this image` - Prompt to use in API
+- `api_model` - str, default: `gpt-4-vision-preview` - Model to use in API
+- `api_tokens` - int, default: `1000` - Max output tokens in API
+
+**IPYNB Notebook files**  (file_ipynb)
+
+- `parser_config` - dict, default: `None`
+- `concatenate` - bool, default: `False`
+
+**Markdown files**  (file_md)
+
+- `remove_hyperlinks` - bool, default: `True`
+- `remove_images` - bool, default: `True`
+
+**PDF documents**  (file_pdf)
+
+- `return_full_document` - bool, default: `False`
+
+**Video/Audio**  (file_video_audio)
+
+This loader can operate in two modes: local model and API.
+If the local mode is enabled, then the local `Whisper` model will be used. The local mode requires a Python/PyPi version of the application and is not available in the compiled or Snap versions.
+If the API mode (default) is selected, then the currently selected provider in `Audio Input` plugin will be used. If the `OpenAI Whisper` is chosen then the OpenAI API and the API Whisper model will be used. 
+
+**Note:** Usage of Whisper via API consumes additional tokens in OpenAI API (for `Whisper` model)!
+
+Local mode requires `torch` and `openai-whisper` to be installed and uses the `Whisper` model locally to transcribing video and audio.
+
+- `model_version` - str, default: `base` - Whisper model to use, available models: https://github.com/openai/whisper
+
+**XML files**  (file_xml)
+
+- `tree_level_split` - int, default: `0`
+
+Allowed additional keyword arguments for built-in data loaders (Web and external content):
+
+**Bitbucket**  (web_bitbucket)
+
+- `username` - str, default: `None`
+- `api_key` - str, default: `None`
+- `extensions_to_skip` - list, default: `[]`
+
+**ChatGPT Retrieval**  (web_chatgpt_retrieval)
+
+- `endpoint_url` - str, default: `None`
+- `bearer_token` - str, default: `None`
+- `retries` - int, default: `None`
+- `batch_size` - int, default: `100`
+
+**Google Calendar** (web_google_calendar)
+
+- `credentials_path` - str, default: `credentials.json`
+- `token_path` - str, default: `token.json`
+
+**Google Docs** (web_google_docs)
+
+- `credentials_path` - str, default: `credentials.json`
+- `token_path` - str, default: `token.json`
+
+**Google Drive** (web_google_drive)
+
+- `credentials_path` - str, default: `credentials.json`
+- `token_path` - str, default: `token.json`
+- `pydrive_creds_path` - str, default: `creds.txt`
+- `client_config` - dict, default: `{}`
+
+**Google Gmail** (web_google_gmail)
+
+- `credentials_path` - str, default: `credentials.json`
+- `token_path` - str, default: `token.json`
+- `use_iterative_parser` - bool, default: `False`
+- `max_results` - int, default: `10`
+- `results_per_page` - int, default: `None`
+
+**Google Keep** (web_google_keep)
+
+- `credentials_path` - str, default: `keep_credentials.json`
+
+**Google Sheets** (web_google_sheets)
+
+- `credentials_path` - str, default: `credentials.json`
+- `token_path` - str, default: `token.json`
+
+**GitHub Issues**  (web_github_issues)
+
+- `token` - str, default: `None`
+- `verbose` - bool, default: `False`
+
+**GitHub Repository**  (web_github_repository)
+
+- `token` - str, default: `None`
+- `verbose` - bool, default: `False`
+- `concurrent_requests` - int, default: `5`
+- `timeout` - int, default: `5`
+- `retries` - int, default: `0`
+- `filter_dirs_include` - list, default: `None`
+- `filter_dirs_exclude` - list, default: `None`
+- `filter_file_ext_include` - list, default: `None`
+- `filter_file_ext_exclude` - list, default: `None`
+
+**Microsoft OneDrive**  (web_microsoft_onedrive)
+
+- `client_id` - str, default: `None`
+- `client_secret` - str, default: `None`
+- `tenant_id` - str, default: `consumers`
+
+**Sitemap (XML)**  (web_sitemap)
+
+- `html_to_text` - bool, default: `False`
+- `limit` - int, default: `10`
+
+**SQL Database**  (web_database)
+
+- `uri` - str, default: `None`
+
+You can provide a single URI in the form of: `{scheme}://{user}:{password}@{host}:{port}/{dbname}`, or you can provide each field manually:
+
+- `scheme` - str, default: `None`
+- `host` - str, default: `None`
+- `port` - str, default: `None`
+- `user` - str, default: `None`
+- `password` - str, default: `None`
+- `dbname` - str, default: `None`
+
+**Twitter/X posts**  (web_twitter)
+
+- `bearer_token` - str, default: `None`
+- `num_tweets` - int, default: `100`
+
+## Vector stores
+
+**Available vector stores** (provided by `LlamaIndex`):
+
+```
+- ChromaVectorStore
+- ElasticsearchStore
+- PinecodeVectorStore
+- RedisVectorStore
+- SimpleVectorStore
+```
+
+You can configure selected vector store by providing config options like `api_key`, etc. in `Settings -> LlamaIndex` window. 
+
+Arguments provided here (on list: `Vector Store (**kwargs)` in `Advanced settings` will be passed to selected vector store provider. You can check keyword arguments needed by selected provider on LlamaIndex API reference page: 
+
+https://docs.llamaindex.ai/en/stable/api_reference/storage/vector_store.html
+
+Which keyword arguments are passed to providers?
+
+For `ChromaVectorStore` and `SimpleVectorStore` all arguments are set by PyGPT and passed internally (you do not need to configure anything).
+
+For other providers you can provide these arguments:
+
+**ElasticsearchStore**
+
+Keyword arguments for ElasticsearchStore(`**kwargs`):
+
+- `index_name` (default: current index ID, already set, not required)
+- any other keyword arguments provided on list
+
+**PinecodeVectorStore**
+
+Keyword arguments for Pinecone(`**kwargs`):
+
+- `api_key`
+- index_name (default: current index ID, already set, not required)
+
+**RedisVectorStore**
+
+Keyword arguments for RedisVectorStore(`**kwargs`):
+
+- `index_name` (default: current index ID, already set, not required)
+- any other keyword arguments provided on list
+
+You can extend list of available providers by creating custom provider and registering it on app launch.
+
+By default, you are using chat-based mode when using `Chat with Files`.
+If you want to only query index (without chat) you can enable `Query index only (without chat)` option.
+
+### Adding custom vector stores and data loaders
+
+You can create a custom vector store provider or data loader for your data and develop a custom launcher for the application. 
+
+See the section `Extending PyGPT / Adding a custom Vector Store provider` for more details.
+
+# Updates
+
+### Updating PyGPT
+
+**PyGPT** comes with an integrated update notification system. When a new version with additional features is released, you'll receive an alert within the app. 
+
+To get the new version, simply download it and start using it in place of the old one. All your custom settings like configuration, presets, indexes, and past conversations will be kept and ready to use right away in the new version.
+
+# Debugging and Logging
+
+In `Settings -> Developer` dialog, you can enable the `Show debug menu` option to turn on the debugging menu. The menu allows you to inspect the status of application elements. In the debugging menu, there is a `Logger` option that opens a log window. In the window, the program's operation is displayed in real-time.
+
+**Logging levels**:
+
+By default, all errors and exceptions are logged to the file:
+
+```ini
+{HOME_DIR}/.config/pygpt-net/app.log
+```
+
+To increase the logging level (`ERROR` level is default), run the application with `--debug` argument:
+
+``` ini
+python3 run.py --debug=1
+```
+
+or
+
+```ini
+python3 run.py --debug=2
+```
+
+The value `1` enables the `INFO`logging level.
+
+The value `2` enables the `DEBUG` logging level (most information).
+
+**Compatibility (legacy) mode**
+
+If you have a problems with `WebEngine / Chromium` renderer you can force the legacy mode by launching the app with command line arguments:
+
+``` ini
+python3 run.py --legacy=1
+```
+
+and to force disable OpenGL hardware acceleration:
+
+``` ini
+python3 run.py --disable-gpu=1
+```
+
+You can also manualy enable legacy mode by editing config file - open the `%WORKDIR%/config.json` config file in editor and set the following options:
+
+``` json
+"render.engine": "legacy",
+"render.open_gl": false,
+```
+
+# Extending PyGPT
+
+## Quick start
+
+You can create your own extensions for **PyGPT** at any time.
+
+PyGPT can be extended with:
+
+- custom models
+
+- custom plugins
+
+- custom LLM wrappers
+
+- custom vector store providers
+
+- custom data loaders
+
+- custom audio input providers
+
+- custom audio output providers
+
+- custom web search engine providers
+
+**Examples (tutorial files)** 
+
+See the `examples` directory in this repository with examples of custom launcher, plugin, vector store, LLM (LangChain and LlamaIndex) provider and data loader:
+
+- `examples/custom_launcher.py`
+
+- `examples/example_audio_input.py`
+
+- `examples/example_audio_output.py`
+
+- `examples/example_data_loader.py`
+
+- `examples/example_llm.py`
+
+- `examples/example_plugin.py`
+
+- `examples/example_vector_store.py`
+
+- `examples/example_web_search.py`
+
+These example files can be used as a starting point for creating your own extensions for **PyGPT**.
+
+Extending PyGPT with custom plugins, LLMs wrappers and vector stores:
+
+- You can pass custom plugin instances, LLMs wrappers and vector store providers to the launcher.
+
+- This is useful if you want to extend PyGPT with your own plugins, vectors storage and LLMs.
+
+To register custom plugins:
+
+- Pass a list with the plugin instances as `plugins` keyword argument.
+
+To register custom LLMs wrappers:
+
+- Pass a list with the LLMs wrappers instances as `llms` keyword argument.
+
+To register custom vector store providers:
+
+- Pass a list with the vector store provider instances as `vector_stores` keyword argument.
+
+To register custom data loaders:
+
+- Pass a list with the data loader instances as `loaders` keyword argument.
+
+To register custom audio input providers:
+
+- Pass a list with the audio input provider instances as `audio_input` keyword argument.
+
+To register custom audio output providers:
+
+- Pass a list with the audio output provider instances as `audio_output` keyword argument.
+
+To register custom web providers:
+
+- Pass a list with the web provider instances as `web` keyword argument.
+
+## Adding a custom model
+
+To add a new model using the OpenAI API, LangChain, or LlamaIndex wrapper, use the editor in `Config -> Models` or manually edit the `models.json` file by inserting the model's configuration details. If you are adding a model via LangChain or LlamaIndex, ensure to include the model's name, its supported modes (either `chat`, `completion`, or both), the LLM provider (such as `OpenAI` or `HuggingFace`), and, if you are using an external API-based model, an optional `API KEY` along with any other necessary environment settings.
+
+Example of models configuration - `%WORKDIR%/models.json`:
+
+```
+"gpt-3.5-turbo": {
+    "id": "gpt-3.5-turbo",
+    "name": "gpt-3.5-turbo",
+    "mode": [
+        "chat",
+        "assistant",
+        "langchain",
+        "llama_index"
+    ],
+    "langchain": {
+        "provider": "openai",
+        "mode": [
+            "chat"
+        ],
+        "args": [
+            {
+                "name": "model_name",
+                "value": "gpt-3.5-turbo",
+                "type": "str"
+            }
+        ],
+        "env": [
+            {
+                "name": "OPENAI_API_KEY",
+                "value": "{api_key}"
+            }
+        ]
+    },
+    "llama_index": {
+        "provider": "openai",
+        "mode": [
+            "chat"
+        ],
+        "args": [
+            {
+                "name": "model",
+                "value": "gpt-3.5-turbo",
+                "type": "str"
+            }
+        ],
+        "env": [
+            {
+                "name": "OPENAI_API_KEY",
+                "value": "{api_key}"
+            }
+        ]
+    },
+    "ctx": 4096,
+    "tokens": 4096,
+    "default": false
+},
+```
+
+There is built-in support for those LLM providers:
+
+```
+- `OpenAI` (openai)
+- `Azure OpenAI` (azure_openai)
+- `Google` (google)
+- `HuggingFace API` (huggingface_api)
+- `Anthropic` (anthropic)
+- `Ollama` (ollama)
+```
+
+**Tip**: `{api_key}` in `models.json` is a placeholder for the main OpenAI API KEY from the settings. It will be replaced by the configured key value.
+
+## Adding a custom plugin
+
+### Creating Your Own Plugin
+
+You can create your own plugin for **PyGPT**. The plugin can be written in Python and then registered with the application just before launching it. All plugins included with the app are stored in the `plugin` directory - you can use them as coding examples for your own plugins.
+
+**Examples (tutorial files)** 
+
+See the example plugin in this `examples` directory:
+
+- `examples/example_plugin.py`
+
+These example file can be used as a starting point for creating your own plugin for **PyGPT**.
+
+To register a custom plugin:
+
+- Create a custom launcher for the app.
+
+- Pass a list with the custom plugin instances as `plugins` keyword argument.
+
+**Example of a custom launcher:**
+
+
+```python
+# custom_launcher.py
+
+from pygpt_net.app import run
+from plugins import CustomPlugin, OtherCustomPlugin
+from llms import CustomLLM
+from vector_stores import CustomVectorStore
+
+plugins = [
+    CustomPlugin(),
+    OtherCustomPlugin(),
+]
+llms = [
+    CustomLLM(),
+]
+vector_stores = [
+    CustomVectorStore(),
+]
+
+run(
+    plugins=plugins,
+    llms=llms,
+    vector_stores=vector_stores,
+)
+```
+
+### Handling events
+
+In the plugin, you can receive and modify dispatched events.
+To do this, create a method named `handle(self, event, *args, **kwargs)` and handle the received events like here:
+
+```python
+# custom_plugin.py
+
+from pygpt_net.core.events import Event
+
+
+def handle(self, event: Event, *args, **kwargs):
+    """
+    Handle dispatched events
+
+    :param event: event object
+    """
+    name = event.name
+    data = event.data
+    ctx = event.ctx
+
+    if name == Event.INPUT_BEFORE:
+        self.some_method(data['value'])
+    elif name == Event.CTX_BEGIN:
+        self.some_other_method(ctx)
+    else:
+    	# ...
+```
+
+### List of Events
+
+Event names are defined in `Event` class in `pygpt_net.core.events`.
+
+Syntax: `event name` - triggered on, `event data` *(data type)*:
+
+- `AI_NAME` - when preparing an AI name, `data['value']` *(string, name of the AI assistant)*
+
+- `AGENT_PROMPT` - on agent prompt in eval mode, `data['value']` *(string, prompt)*
+
+- `AUDIO_INPUT_RECORD_START` - start audio input recording
+
+- `AUDIO_INPUT_RECORD_STOP` -  stop audio input recording
+
+- `AUDIO_INPUT_RECORD_TOGGLE` - toggle audio input recording
+
+- `AUDIO_INPUT_TRANSCRIBE` - on audio file transcribe, `data['path']` *(string, path to audio file)*
+
+- `AUDIO_INPUT_STOP` - force stop audio input
+
+- `AUDIO_INPUT_TOGGLE` - when speech input is enabled or disabled, `data['value']` *(bool, True/False)*
+
+- `AUDIO_OUTPUT_STOP` - force stop audio output
+
+- `AUDIO_OUTPUT_TOGGLE` - when speech output is enabled or disabled, `data['value']` *(bool, True/False)*
+
+- `AUDIO_READ_TEXT` - on text read using speech synthesis, `data['text']` *(str, text to read)*
+
+- `CMD_EXECUTE` - when a command is executed, `data['commands']` *(list, commands and arguments)*
+
+- `CMD_INLINE` - when an inline command is executed, `data['commands']` *(list, commands and arguments)*
+
+- `CMD_SYNTAX` - when appending syntax for commands, `data['prompt'], data['syntax']` *(string, list, prompt and list with commands usage syntax)*
+
+- `CMD_SYNTAX_INLINE` - when appending syntax for commands (inline mode), `data['prompt'], data['syntax']` *(string, list, prompt and list with commands usage syntax)*
+
+- `CTX_AFTER` - after the context item is sent, `ctx`
+
+- `CTX_BEFORE` - before the context item is sent, `ctx`
+
+- `CTX_BEGIN` - when context item create, `ctx`
+
+- `CTX_END` - when context item handling is finished, `ctx`
+
+- `CTX_SELECT` - when context is selected on list, `data['value']` *(int, ctx meta ID)*
+
+- `DISABLE` - when the plugin is disabled, `data['value']` *(string, plugin ID)*
+
+- `ENABLE` - when the plugin is enabled, `data['value']` *(string, plugin ID)*
+
+- `FORCE_STOP` - on force stop plugins
+
+- `INPUT_BEFORE` - upon receiving input from the textarea, `data['value']` *(string, text to be sent)*
 
-- `Vision: Image capture quality`: Video capture image JPEG quality (%).
+- `MODE_BEFORE` - before the mode is selected `data['value'], data['prompt']` *(string, string, mode ID)*
 
-**Indexes (Llama-index)**
+- `MODE_SELECT` - on mode select `data['value']` *(string, mode ID)*
 
-- `Indexes`: List of created indexes.
+- `MODEL_BEFORE` - before the model is selected `data['value']` *(string, model ID)*
 
-- `Vector Store`: Vector store to use (vector database provided by Llama-index).
+- `MODEL_SELECT` - on model select `data['value']` *(string, model ID)*
 
-- `Vector Store (**kwargs)`: Keyword arguments for vector store provider (api_key, index_name, etc.).
+- `PLUGIN_SETTINGS_CHANGED` - on plugin settings update (saving settings)
 
-- `Embeddings provider`: Embeddings provider.
+- `PLUGIN_OPTION_GET` - on request for plugin option value `data['name'], data['value']` *(string, any, name of requested option, value)*
 
-- `Embeddings provider (ENV)`: ENV vars to embeddings provider (API keys, etc.).
+- `POST_PROMPT` - after preparing a system prompt, `data['value']` *(string, system prompt)*
 
-- `Embeddings provider (**kwargs)`: Keyword arguments for embeddings provider (model name, etc.).
+- `POST_PROMPT_ASYNC` - after preparing a system prompt, just before request in async thread, `data['value']` *(string, system prompt)*
 
-- `RPM limit for embeddings API calls`: Specify the limit of maximum requests per minute (RPM), 0 = no limit.
+- `POST_PROMPT_END` - after preparing a system prompt, just before request in async thread, at the very end `data['value']` *(string, system prompt)*
 
-- `Recursive directory indexing`: Enables recursive directory indexing, default is False.
+- `PRE_PROMPT` - before preparing a system prompt, `data['value']` *(string, system prompt)*
 
-- `Replace old document versions in the index during re-indexing`: If enabled, previous versions of documents will be deleted from the index when the newest versions are indexed, default is True.
+- `SYSTEM_PROMPT` - when preparing a system prompt, `data['value']` *(string, system prompt)*
 
-- `Excluded file extensions`: File extensions to exclude if no data loader for this extension, separated by comma.
+- `TOOL_OUTPUT_RENDER` - when rendering extra content from tools from plugins, `data['content']` *(string, content)*
 
-- `Force exclude files`: If enabled, the exclusion list will be applied even when the data loader for the extension is active. Default: False.
+- `UI_ATTACHMENTS` - when the attachment upload elements are rendered, `data['value']` *(bool, show True/False)*
 
-- `Custom metadata to append/replace to indexed documents (file)`: Define custom metadata key => value fields for specified file extensions, separate extensions by comma.\nAllowed placeholders: {path}, {relative_path} {filename}, {dirname}, {relative_dir} {ext}, {size}, {mtime}, {date}, {date_time}, {time}, {timestamp}. Use * (asterisk) as extension if you want to apply field to all files. Set empty value to remove field with specified key from metadata.
+- `UI_VISION` - when the vision elements are rendered, `data['value']` *(bool, show True/False)*
 
-- `Custom metadata to append/replace to indexed documents (web)`: Define custom metadata key => value fields for specified external data loaders.\nAllowed placeholders: {date}, {date_time}, {time}, {timestamp} + {data loader args}
+- `USER_NAME` - when preparing a user's name, `data['value']` *(string, name of the user)*
 
-- `Additional keyword arguments (**kwargs) for data loaders`: Additional keyword arguments, such as settings, API keys, for the data loader. These arguments will be passed to the loader; please refer to the Llama-index or LlamaHub loaders reference for a list of allowed arguments for the specified data loader.
+- `USER_SEND` - just before the input text is sent, `data['value']` *(string, input text)*
 
-- `Use local models in Video/Audio and Image (vision) loaders`: Enables usage of local models in Video/Audio and Image (vision) loaders. If disabled then API models will be used (GPT-4 Vision and Whisper). Note: local models will work only in Python version (not compiled/Snap). Default: False.
 
-- `Auto-index DB in real time`: Enables conversation context auto-indexing in defined modes.
+You can stop the propagation of a received event at any time by setting `stop` to `True`:
 
-- `ID of index for auto-indexing`: Index to use if auto-indexing of conversation context is enabled.
+```
+event.stop = True
+```
 
-- `Enable auto-index in modes`: List of modes with enabled context auto-index, separated by comma.
+Events flow can be debugged by enabling the option `Config -> Settings -> Developer -> Log and debug events`.
 
-- `DB (ALL), DB (UPDATE), FILES (ALL)`: Index the data – batch indexing is available here.
+## Adding a custom LLM provider
 
-**Agent (autonomous)**
+Handling LLMs with LangChain and LlamaIndex is implemented through separated wrappers. This allows for the addition of support for any provider and model available via LangChain or LlamaIndex. All built-in wrappers for the models and its providers are placed in the `pygpt_net.provider.llms`.
 
-- `Sub-mode to use`: Sub-mode to use in Agent mode (chat, completion, langchain, llama_index, etc.). Default: chat.
+These wrappers are loaded into the application during startup using `launcher.add_llm()` method:
 
-- `Index to use`: Only if sub-mode is llama_index (Chat with files), choose the index to use in Agent mode.
+```python
+# app.py
 
-- `Display a tray notification when the goal is achieved.`: If enabled, a notification will be displayed after goal achieved / finished run.
+from pygpt_net.provider.llms.openai import OpenAILLM
+from pygpt_net.provider.llms.azure_openai import AzureOpenAILLM
+from pygpt_net.provider.llms.anthropic import AnthropicLLM
+from pygpt_net.provider.llms.hugging_face import HuggingFaceLLM
+from pygpt_net.provider.llms.ollama import OllamaLLM
+from pygpt_net.provider.llms.google import GoogleLLM
 
-**Updates**
 
-- `Check for updates on start`: Enables checking for updates on start. Default: True.
+def run(**kwargs):
+    """Runs the app."""
+    # Initialize the app
+    launcher = Launcher()
+    launcher.init()
 
-- `Check for updates in background`: Enables checking for updates in background (checking every 5 minutes). Default: True.
+    # Register plugins
+    ...
 
-**Developer**
+    # Register langchain and llama-index LLMs wrappers
+    launcher.add_llm(OpenAILLM())
+    launcher.add_llm(AzureOpenAILLM())
+    launcher.add_llm(AnthropicLLM())
+    launcher.add_llm(HuggingFaceLLM())
+    launcher.add_llm(OllamaLLM())
+    launcher.add_llm(GoogleLLM())
 
-- `Show debug menu`: Enables debug (developer) menu.
+    # Launch the app
+    launcher.run()
+```
 
-- `Log and debug context`: Enables logging of context input/output.
+To add support for providers not included by default, you can create your own wrapper that returns a custom model to the application and then pass this custom wrapper to the launcher.
 
-- `Log and debug events`: Enables logging of event dispatch.
+Extending **PyGPT** with custom plugins and LLM wrappers is straightforward:
 
-- `Log plugin usage to console`: Enables logging of plugin usage to console.
+- Pass instances of custom plugins and LLM wrappers directly to the launcher.
 
-- `Log DALL-E usage to console`: Enables logging of DALL-E usage to console.
+To register custom LLM wrappers:
 
-- `Log Llama-index usage to console`: Enables logging of Llama-index usage to console.
+- Provide a list of LLM wrapper instances as `llms` keyword argument.
 
-- `Log Assistants usage to console`: Enables logging of Assistants API usage to console.
+**Example:**
 
-- `Log level`: toggle log level (ERROR|WARNING|INFO|DEBUG)
 
+```python
+# launcher.py
 
-## JSON files
+from pygpt_net.app import run
+from plugins import CustomPlugin, OtherCustomPlugin
+from llms import CustomLLM
 
-The configuration is stored in JSON files for easy manual modification outside of the application. 
-These configuration files are located in the user's work directory within the following subdirectory:
+plugins = [
+    CustomPlugin(),
+    OtherCustomPlugin(),
+]
+llms = [
+    CustomLLM(),  # <--- custom LLM provider (wrapper)
+]
+vector_stores = []
 
-``` ini
-{HOME_DIR}/.config/pygpt-net/
+run(
+    plugins=plugins, 
+    llms=llms, 
+    vector_stores=vector_stores,
+)
 ```
 
-# Notepad
-
-The application has a built-in notepad, divided into several tabs. This can be useful for storing information in a convenient way, without the need to open an external text editor. The content of the notepad is automatically saved whenever the content changes.
+**Examples (tutorial files)** 
 
-![v2_notepad](https://github.com/szczyglis-dev/py-gpt/assets/61396542/f6aa0126-bad1-4e6c-ace6-72e979186433)
+See the `examples` directory in this repository with examples of custom launcher, plugin, vector store, LLM (LangChain and LlamaIndex) provider and data loader:
 
-# Advanced configuration
+- `examples/custom_launcher.py`
 
-## Manual configuration
+- `examples/example_audio_input.py`
 
+- `examples/example_audio_output.py`
 
-You can manually edit the configuration files in this directory (this is your work directory):
+- `examples/example_data_loader.py`
 
-``` ini
-{HOME_DIR}/.config/pygpt-net/
-```
+- `examples/example_llm.py`  <-- use it as an example
 
-- `assistants.json` - stores the list of assistants.
-- `attachments.json` - stores the list of current attachments.
-- `config.json` - stores the main configuration settings.
-- `models.json` - stores models configurations.
-- `capture` - a directory for captured images from camera and screenshots
-- `css` - a directory for CSS stylesheets (user override)
-- `history` - a directory for context history in `.txt` format.
-- `idx` - `Llama-index` indexes
-- `img` - a directory for images generated with `DALL-E 3` and `DALL-E 2`, saved as `.png` files.
-- `locale` - a directory for locales (user override)
-- `data` - a directory for data files and files downloaded/generated by GPT.
-- `presets` - a directory for presets stored as `.json` files.
-- `upload` - a directory for local copies of attachments coming from outside the workdir
-- `db.sqlite` - a database with contexts, notepads and indexes data records
-- `app.log` - a file with error and debug log
+- `examples/example_plugin.py`
 
----
+- `examples/example_vector_store.py`
 
-## Translations / Locale
+- `examples/example_web_search.py`
 
-Locale `.ini` files are located in the app directory:
+These example files can be used as a starting point for creating your own extensions for **PyGPT**.
 
-``` ini
-./data/locale
-```
+To integrate your own model or provider into **PyGPT**, you can also reference the classes located in the `pygpt_net.provider.llms`. These samples can act as an more complex example for your custom class. Ensure that your custom wrapper class includes two essential methods: `chat` and `completion`. These methods should return the respective objects required for the model to operate in `chat` and `completion` modes.
 
-This directory is automatically scanned when the application launches. To add a new translation, 
-create and save the file with the appropriate name, for example:
+Every single LLM provider (wrapper) inherits from `BaseLLM` class and can provide 3 components: provider for LangChain, provider for LlamaIndex, and provider for Embeddings.
 
-``` ini
-locale.es.ini   
-```
 
-This will add Spanish as a selectable language in the application's language menu.
+## Adding a custom vector store provider
 
-**Overwriting CSS and locales with your own files:**
+You can create a custom vector store provider or data loader for your data and develop a custom launcher for the application. To register your custom vector store provider or data loader, simply register it by passing the vector store provider instance to `vector_stores` keyword argument and loader instance in the `loaders` keyword argument:
 
-You can also overwrite files in the `locale` and `css` app directories with your own files in the user directory. 
-This allows you to overwrite language files or CSS styles in a very simple way - by just creating files in your working directory.
 
+```python
+# app.py
 
-``` ini
-{HOME_DIR}/.config/pygpt-net/
-```
+# vector stores
+from pygpt_net.provider.vector_stores.chroma import ChromaProvider
+from pygpt_net.provider.vector_stores.elasticsearch import ElasticsearchProvider
+from pygpt_net.provider.vector_stores.pinecode import PinecodeProvider
+from pygpt_net.provider.vector_stores.redis import RedisProvider
+from pygpt_net.provider.vector_stores.simple import SimpleProvider
 
-- `locale` - a directory for locales in `.ini` format.
-- `css` - a directory for CSS styles in `.css` format.
+def run(**kwargs):
+    # ...
+    # register base vector store providers (llama-index)
+    launcher.add_vector_store(ChromaProvider())
+    launcher.add_vector_store(ElasticsearchProvider())
+    launcher.add_vector_store(PinecodeProvider())
+    launcher.add_vector_store(RedisProvider())
+    launcher.add_vector_store(SimpleProvider())
 
+    # register custom vector store providers (llama-index)
+    vector_stores = kwargs.get('vector_stores', None)
+    if isinstance(vector_stores, list):
+        for store in vector_stores:
+            launcher.add_vector_store(store)
 
-## Debugging and Logging
+    # ...
+```
 
-In `Settings -> Developer` dialog, you can enable the `Show debug menu` option to turn on the debugging menu. The menu allows you to inspect the status of application elements. In the debugging menu, there is a `Logger` option that opens a log window. In the window, the program's operation is displayed in real-time.
+To register your custom vector store provider just register it by passing provider instance in `vector_stores` keyword argument:
 
-**Logging levels**:
+```python
 
-By default, all errors and exceptions are logged to the file:
+# custom_launcher.py
 
-```ini
-{HOME_DIR}/.config/pygpt-net/app.log
-```
+from pygpt_net.app import run
+from plugins import CustomPlugin, OtherCustomPlugin
+from llms import CustomLLM
+from vector_stores import CustomVectorStore
 
-To increase the logging level (`ERROR` level is default), run the application with `--debug` argument:
+plugins = [
+    CustomPlugin(),
+    OtherCustomPlugin(),
+]
+llms = [
+    CustomLLM(),
+]
+vector_stores = [
+    CustomVectorStore(),  # <--- custom vector store provider
+]
 
-``` ini
-python3 run.py --debug=1
+run(
+    plugins=plugins,
+    llms=llms,
+    vector_stores=vector_stores,
+)
 ```
 
-or
-
-```ini
-python3 run.py --debug=2
-```
+The vector store provider must be an instance of `pygpt_net.provider.vector_stores.base.BaseStore`. 
+You can review the code of the built-in providers in `pygpt_net.provider.vector_stores` and use them as examples when creating a custom provider.
 
-The value `1` enables the `INFO`logging level.
+### Adding a custom data loader
 
-The value `2` enables the `DEBUG` logging level (most information).
 
-## Updates
+```python
 
-### Updating PyGPT
+# custom_launcher.py
 
-**PyGPT** comes with an integrated update notification system. When a new version with additional features is released, you'll receive an alert within the app. 
+from pygpt_net.app import run
+from plugins import CustomPlugin, OtherCustomPlugin
+from llms import CustomLLM
+from vector_stores import CustomVectorStore
+from loaders import CustomLoader
 
-To get the new version, simply download it and start using it in place of the old one. All your custom settings like configuration, presets, indexes, and past conversations will be kept and ready to use right away in the new version.
+plugins = [
+    CustomPlugin(),
+    OtherCustomPlugin(),
+]
+llms = [
+    CustomLLM(),
+]
+vector_stores = [
+    CustomVectorStore(),
+]
+loaders = [
+    CustomLoader(),  # <---- custom data loader
+]
 
+run(
+    plugins=plugins,
+    llms=llms,
+    vector_stores=vector_stores,  # <--- list with custom vector store providers
+    loaders=loaders  # <--- list with custom data loaders
+)
+```
 
-## Coming soon
+The data loader must be an instance of `pygpt_net.provider.loaders.base.BaseLoader`. 
+You can review the code of the built-in loaders in `pygpt_net.provider.loaders` and use them as examples when creating a custom loader.
 
-- Enhanced integration with Langchain
-- More vector databases support
-- Development of autonomous agents
 
-## DISCLAIMER
+# DISCLAIMER
 
 This application is not officially associated with OpenAI. The author shall not be held liable for any damages 
 resulting from the use of this application. It is provided "as is," without any form of warranty. 
@@ -2969,61 +4043,85 @@ may consume additional tokens that are not displayed in the main window.
 
 ## Recent changes:
 
-# 2.1.37 (2024-03-19)
+**2.4.48 (2025-01-16)**
+
+- Fix: parsing lists in data loaders configuration.
+- Fix: crash on Windows on PySide6 v6.6.0.
+- Added Gemini embeddings to LlamaIndex settings.
+- LlamaIndex upgraded to 0.12.11.
+- Security updates.
 
-- Added generation of audio transcriptions from audio/video files.
+**2.4.47 (2025-01-14)**
 
-# 2.1.36 (2024-03-19)
+- Added support for Python 3.12.
+- Added a new model to Chat with Files: gemini-2.0-flash-exp.
+- PySide6 upgraded to 6.6.0.
 
-- Added split screen view (output/edit) to code interpreter.
+**2.4.46 (2024-12-16)**
 
-# 2.1.35 (2024-03-19)
+- Added a new tab in Settings: "API Keys", where the API keys configuration for Google and Anthropic models has been relocated.
+- Introduced a new mode in "Chat with Files": "Retrieve Only", which allows for retrieving raw documents from the index.
+- Fixed a bug related to tool calls in the Gemini provider when using Chat with Files mode.
 
-- Added text file editor and image viewer.
-- Improved video player and code interpreter.
+**2.4.45 (2024-12-16)**
 
-# 2.1.34 (2024-03-18)
+- Enhanced web data loaders UI.
 
-- Added video player.
-- Fixed audio input message append in agent mode.
+**2.4.44 (2024-12-16)**
 
-# 2.1.33 (2024-03-18)
+- Enhanced web data loaders.
+- Web loaders have been added to attachments, allowing external web content to be attached to context via the "+Web" button in the Attachments tab.
+- Improved handling of attachments in groups and added an attachment icon when a group contains attachments.
 
-- Code interpreter enabled even if interpreter plugin is disabled.
-- Added "Connect to the Python code interpreter window" config option in Code interpreter plugin.
-- Added Monospace font support - issue #37.
-- Improved raw JSON command outputs.
+**2.4.43 (2024-12-15)**
 
-# 2.1.32 (2024-03-17)
+- Fix: Bug on attachment upload.
+- Added: Attachments uploaded in groups are now available for all contexts in the group (beta).
 
-- Global prompts config moved to a separated settings section "Prompts".
-- Added auto-clear option in code interpreter.
-- Added "Use extra context output" config option in "Settings -> Context".
+**2.4.42 (2024-12-15)**
 
-# 2.1.31 (2024-03-17)
+- Added Mailer plugin, which allows sending and retrieving emails from the server, and reading them. It currently supports only SMTP.
+- Added 'web_request' command to the Web Search plugin, enabling GET/POST/PUT and other connections to any address and API endpoint. It also supports sending POST data, files, headers, cookies, and more.
+- Improved audio output.
+- Enhanced visibility of the Video menu.
+- Other fixes.
 
-- Improved code interpreter integration with sandbox/docker.
-- Added display of hidden files in Files explorer.
-- Added "Use as image" context menu action in Files explorer.
+**2.4.41 (2024-12-14)**
 
-# 2.1.30 (2024-03-16)
+- Improved switching between columns on a split screen.
+- Added visual identification of the active column.
 
-- Improved real-time Python code interpreter: added edit mode and whole code execution (history).
+**2.4.40 (2024-12-13)**
 
-# 2.1.29 (2024-03-16)
+- Enhanced Split Screen mode, now promoted from beta to stable.
+- Python Code Interpreter tool added to the Tabs.
+- HTML/JS Canvas tool added to the Tabs.
+- Added attachment icon to the context list if context has attachments.
+- Improved audio playback.
+- Improved web search.
+- Added a thumbnail image to web search results.
+- Added a new commands to web search: "extract_images" and "extract_links".
+- Added the option "Use raw content (without summarization)" to the web search plugin, which provides a more detailed result to the main model.
+- Extended the default maximum result characters to 50,000 in the web search plugin.
 
-- Added real-time Python code interpreter (<> icon), connected with the Code Interpreter plugin's input and output.
-- Improved plugin command execution.
+**2.4.39 (2024-12-09)**
 
-# 2.1.28 (2024-03-15)
+- Added "Split Screen" mode (accessible via the switch in the bottom-right corner of the screen), which allows you to work in two windows simultaneously. It is currently experimental (beta). Future updates will include Code Interpreter and Canvas running in tabs.
 
-- Fixed local commands handling in Assistant API.
-- Fixed system prompt replace after mode changed on app start.
-- Improved related links rendering in chat output.
-- Added RPM limit config option for embeddings API.
-- UI improvements.
+- Fixed: Language switch.
 
-The full changelog is located in the **[CHANGELOG.md](CHANGELOG.md)** file in the main folder of this repository.
+**2.4.38 (2024-12-08)**
+
+- Added the ability to select a style for chat display between: Blocks, ChatGPT-like, and ChatGPT-like Wide. New option in the menu: Config -> Theme -> Style...
+- Added configuration options for audio input in Settings -> Audio -> Audio Input Device, Channels, and Sampling rate.
+
+**2.4.37 (2024-11-30)**
+
+- The `Query only` mode in `Uploaded` tab has been renamed to `RAG`.
+- New options have been added under `Settings -> Files and Attachments`: 
+  - `Use history in RAG query`: When enabled, the content of the entire conversation will be used when preparing a query if the mode is set to RAG or Summary.
+  - `RAG limit`: This option is applicable only if 'Use history in RAG query' is enabled. It specifies the limit on how many recent entries in the conversation will be used when generating a query for RAG. A value of 0 indicates no limit.
+- Cache: dynamic parts of the system prompt (from plugins) have been moved to the very end of the prompt stack to enable the use of prompt cache mechanisms in OpenAI.
 
 
 # Credits and links
@@ -3032,8 +4130,12 @@ The full changelog is located in the **[CHANGELOG.md](CHANGELOG.md)** file in th
 
 **Documentation:** <https://pygpt.readthedocs.io>
 
+**Support and donate:** <https://pygpt.net/#donate>
+
 **GitHub:** <https://github.com/szczyglis-dev/py-gpt>
 
+**Discord:** <https://pygpt.net/discord>
+
 **Snap Store:** <https://snapcraft.io/pygpt>
 
 **PyPI:** <https://pypi.org/project/pygpt-net>
@@ -3046,23 +4148,27 @@ The full changelog is located in the **[CHANGELOG.md](CHANGELOG.md)** file in th
 
 # Special thanks
 
-GitHub community:
+GitHub's community:
+
+- [@BillionShields](https://github.com/BillionShields)
 
-- **gfsysa**
+- [@gfsysa](https://github.com/gfsysa)
 
-- **kaneda2004**
+- [@glinkot](https://github.com/glinkot)
 
-- **linnflux**
+- [@kaneda2004](https://github.com/kaneda2004)
 
-- **moritz-t-w**
+- [@linnflux](https://github.com/linnflux)
 
-- **oleksii-honchar**
+- [@moritz-t-w](https://github.com/moritz-t-w)
 
-- **yf007**
+- [@oleksii-honchar](https://github.com/oleksii-honchar)
+
+- [@yf007](https://github.com/yf007)
 
 ## Third-party libraries
 
-Full list of external libraries used in this project is located in the **[requirements.txt](requirements.txt)** file in the main folder of this repository.
+Full list of external libraries used in this project is located in the [requirements.txt](https://github.com/szczyglis-dev/py-gpt/blob/master/requirements.txt) file in the main folder of the repository.
 
 All used SVG icons are from `Material Design Icons` provided by Google:
 
@@ -3070,4 +4176,13 @@ https://github.com/google/material-design-icons
 
 https://fonts.google.com/icons
 
-Code of the Llama-index offline loaders integrated into app is taken from LlamaHub: https://llamahub.ai
+Monaspace fonts provided by GitHub: https://github.com/githubnext/monaspace
+
+Code of the LlamaIndex offline loaders integrated into app is taken from LlamaHub: https://llamahub.ai
+
+Awesome ChatGPT Prompts (used in templates): https://github.com/f/awesome-chatgpt-prompts/
+
+Code syntax highlight powered by: https://highlightjs.org
+
+LaTeX support by: https://katex.org and https://github.com/mitya57/python-markdown-math
+