pygpt-net 2.4.30__py3-none-any.whl → 2.4.34__py3-none-any.whl

CHANGELOG.md

@@ -1,5 +1,29 @@
 # CHANGELOG
 
+## 2.4.34 (2024-11-26)
+
+- Added a new mode: `Chat with Audio`, with built-in multimodal support for audio input/output. Currently in beta, the execution of commands and tools in this mode is temporarily unavailable.
+- Added new models: `gpt-4o-audio-preview`, `gpt-4o-2024-11-20`, `chatgpt-4o-latest`.
+- Force disabled integration with the native system menu.
+
+## 2.4.33 (2024-11-26)
+
+- Improved CSS and rendering of file and image lists.
+- Added displaying of used attachments in the chat window.
+
+## 2.4.32 (2024-11-26)
+
+- The "Add URL" option added to the "Attachments" tab allows users to include content from a given website as additional context. Currently, it only supports standard web pages and video transcription for YouTube links. More "web" options will be added in the future.
+- Added UTF-8 as default in attachments content text read/write.
+
+## 2.4.31 (2024-11-25)
+
+- Added an option checkbox `Auto-index on upload` in the `Attachments` tab: 
+
+**Tip:** To use the `Query only` mode, the file must be indexed in the vector database. This occurs automatically at the time of upload if the `Auto-index on upload` option in the `Attachments` tab is enabled. When uploading large files, such indexing might take a while - therefore, if you are using the `Full context` option, which does not use the index, you can disable the `Auto-index` option to speed up the upload of the attachment. In this case, it will only be indexed when the `Query only` option is called for the first time, and until then, attachment will be available in the form of `Full context` and `Summary`.
+
+- Added context menu options in `Uploaded attachments` tab: `Open`, `Open Source directory` and `Open Storage directory`.
+
 ## 2.4.30 (2024-11-25)
 
 - Added instruction to model about mapped data directory in both legacy and IPython code interepreter.

README.md

@@ -2,7 +2,7 @@
 
 [![pygpt](https://snapcraft.io/pygpt/badge.svg)](https://snapcraft.io/pygpt)
 
-Release: **2.4.30** | build: **2024.11.25** | Python: **>=3.10, <3.12**
+Release: **2.4.34** | build: **2024.11.26** | Python: **>=3.10, <3.12**
 
 > Official website: https://pygpt.net | Documentation: https://pygpt.readthedocs.io
 > 
@@ -155,11 +155,11 @@ pip install pygpt-net
 pygpt
 ```
 
-## Source Code
+## Running from GitHub source code
 
 An alternative method is to download the source code from `GitHub` and execute the application using the Python interpreter (>=3.10, <3.12). 
 
-### Running from GitHub source code
+### Install with pip
 
 1. Clone git repository or download .zip file:
 
@@ -187,7 +187,7 @@ pip install -r requirements.txt
 python3 run.py
 ```
 
-**Install with Poetry**
+### Install with Poetry
 
 1. Clone git repository or download .zip file:
 
@@ -356,7 +356,7 @@ This mode in **PyGPT** mirrors `ChatGPT`, allowing you to chat with models such
 
 The main part of the interface is a chat window where conversations appear. Right below that is where you type your messages. On the right side of the screen, there's a section to set up or change your system prompts. You can also save these setups as presets to quickly switch between different models or tasks.
 
-Above where you type your messages, the interface shows you the number of tokens your message will use up as you type it – this helps to keep track of usage. There's also a feature to upload files in this area. Go to the `Files` tab to manage your uploads or add attachments to send to the OpenAI API (but this makes effect only in `Assisant` and `Vision` modes).
+Above where you type your messages, the interface shows you the number of tokens your message will use up as you type it – this helps to keep track of usage. There's also a feature to upload files in this area. Go to the `Attachments` tab to manage your uploads or add attachments to send to the OpenAI API (but this makes effect only in `Assisant` and `Vision` modes).
 
 ![v2_mode_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/f573ee22-8539-4259-b180-f97e54bc0d94)
 
@@ -374,6 +374,17 @@ Plugin allows you to generate images in Chat mode:
 
 ![v3_img_chat](https://github.com/szczyglis-dev/py-gpt/assets/61396542/c288a4b3-c932-4201-b5a3-8452aea49817)
 
+
+## Chat with Audio
+
+2024-11-26: currently in beta.
+
+This mode works like the Chat mode but with native support for audio input and output using a multimodal model - `gpt-4o-audio`. In this mode, audio input and output are directed to and from the model directly, without the use of external plugins. This enables faster and better audio communication.
+
+More info: https://platform.openai.com/docs/guides/audio/quickstart
+
+**INFO:** The execution of commands and tools in this mode is temporarily unavailable.
+
 ## Completion
 
 This mode provides in-depth access to a broader range of capabilities offered by Large Language Models (LLMs). While it maintains a chat-like interface for user interaction, it introduces additional settings and functional richness beyond typical chat exchanges. Users can leverage this mode to prompt models for complex text completions, role-play dialogues between different characters, perform text analysis, and execute a variety of other sophisticated tasks. It supports any model provided by the OpenAI API as well as other models through `Langchain`.
@@ -1097,6 +1108,10 @@ The content from the uploaded attachments will be used in the current conversati
 
 **Note:** Only text files from the list above are included in the additional context. Images will not be included in the context but will be used by the vision model in real-time. Adding image files to the context will be available in future versions.
 
+**Uploading larger files and auto-index:**
+
+To use the `Query only` mode, the file must be indexed in the vector database. This occurs automatically at the time of upload if the `Auto-index on upload` option in the `Attachments` tab is enabled. When uploading large files, such indexing might take a while - therefore, if you are using the `Full context` option, which does not use the index, you can disable the `Auto-index` option to speed up the upload of the attachment. In this case, it will only be indexed when the `Query only` option is called for the first time, and until then, attachment will be available in the form of `Full context` and `Summary`.
+
 ## Files (download, code generation)
 
 **PyGPT** enables the automatic download and saving of files created by the model. This is carried out in the background, with the files being saved to an `data` folder located within the user's working directory. To view or manage these files, users can navigate to the `Files` tab which features a file browser for this specific directory. Here, users have the interface to handle all files sent by the AI.
@@ -1565,6 +1580,8 @@ Python code to a file, which the `Code Interpreter` can execute it and return it
 - `Files I/O` - provides access to the local filesystem, enabling GPT to read and write files, 
 as well as list and create directories.
 
+- `System (OS)` - allows you to create and execute custom commands on your system.
+
 - `Mouse and Keyboard` - provides the ability to control the mouse and keyboard by the model.
 
 - `Web Search` - provides the ability to connect to the Web, search web pages for current data, and index external content using Llama-index data loaders.
@@ -3652,6 +3669,30 @@ may consume additional tokens that are not displayed in the main window.
 
 ## Recent changes:
 
+**2.4.34 (2024-11-26)**
+
+- Added a new mode: `Chat with Audio`, with built-in multimodal support for audio input/output. Currently in `beta`, the execution of commands and tools in this mode is temporarily unavailable.
+- Added new models: `gpt-4o-audio-preview`, `gpt-4o-2024-11-20`, `chatgpt-4o-latest`.
+- Force disabled integration with the native system menu.
+
+**2.4.33 (2024-11-26)**
+
+- Improved CSS and rendering of file and image lists.
+- Added displaying of used attachments in the chat window.
+
+**2.4.32 (2024-11-26)**
+
+- The "Add URL" option added to the "Attachments" tab allows users to include content from a given website as additional context. Currently, it only supports standard web pages and video transcription for YouTube links. More "web" options will be added in the future.
+- Added UTF-8 as default in attachments content text read/write.
+
+**2.4.31 (2024-11-25)**
+
+- Added an option checkbox `Auto-index on upload` in the `Attachments` tab: 
+
+**Tip:** To use the `Query only` mode, the file must be indexed in the vector database. This occurs automatically at the time of upload if the `Auto-index on upload` option in the `Attachments` tab is enabled. When uploading large files, such indexing might take a while - therefore, if you are using the `Full context` option, which does not use the index, you can disable the `Auto-index` option to speed up the upload of the attachment. In this case, it will only be indexed when the `Query only` option is called for the first time, and until then, attachment will be available in the form of `Full context` and `Summary`.
+
+- Added context menu options in `Uploaded attachments` tab: `Open`, `Open Source directory` and `Open Storage directory`.
+
 **2.4.30 (2024-11-25)**
 
 - Added instruction to model about mapped data directory in both legacy and IPython code interepreter.

pygpt_net/CHANGELOG.txt

@@ -1,3 +1,27 @@
+2.4.34 (2024-11-26)
+
+- Added a new mode: Chat with Audio, with built-in multimodal support for audio input/output. Currently in beta, the execution of commands and tools in this mode is temporarily unavailable.
+- Added new models: gpt-4o-audio-preview, gpt-4o-2024-11-20, chatgpt-4o-latest.
+- Force disabled integration with the native system menu.
+
+2.4.33 (2024-11-26)
+
+- Improved CSS and rendering of file and image lists.
+- Added displaying of used attachments in the chat window.
+
+2.4.32 (2024-11-26)
+
+- The "Add URL" option added to the "Attachments" tab allows users to include content from a given website as additional context. Currently, it only supports standard web pages and video transcription for YouTube links. More "web" options will be added in the future.
+- Added UTF-8 as default in attachments content text read/write.
+
+2.4.31 (2024-11-25)
+
+- Added an option checkbox `Auto-index on upload` in the `Attachments` tab: 
+
+Tip: To use the `Query only` mode, the file must be indexed in the vector database. This occurs automatically at the time of upload if the `Auto-index on upload` option in the `Attachments` tab is enabled. When uploading large files, such indexing might take a while - therefore, if you are using the `Full context` option, which does not use the index, you can disable the `Auto-index` option to speed up the upload of the attachment. In this case, it will only be indexed when the `Query only` option is called for the first time, and until then, attachment will be available in the form of `Full context` and `Summary`.
+
+- Added context menu options in `Uploaded attachments` tab: `Open`, `Open Source directory` and `Open Storage directory`.
+
 2.4.30 (2024-11-25)
 
 - Added instruction to model about mapped data directory in both legacy and IPython code interepreter.

pygpt_net/__init__.py

@@ -6,15 +6,15 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.25 01:00:00                  #
+# Updated Date: 2024.11.26 21:00:00                  #
 # ================================================== #
 
 __author__ = "Marcin Szczygliński"
 __copyright__ = "Copyright 2024, Marcin Szczygliński"
 __credits__ = ["Marcin Szczygliński"]
 __license__ = "MIT"
-__version__ = "2.4.30"
-__build__ = "2024.11.25"
+__version__ = "2.4.34"
+__build__ = "2024.11.26"
 __maintainer__ = "Marcin Szczygliński"
 __github__ = "https://github.com/szczyglis-dev/py-gpt"
 __website__ = "https://pygpt.net"

pygpt_net/controller/access/__init__.py

@@ -6,7 +6,7 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.20 03:00:00                  #
+# Updated Date: 2024.11.26 19:00:00                  #
 # ================================================== #
 
 from pygpt_net.core.events import BaseEvent, ControlEvent, AppEvent
@@ -73,9 +73,9 @@ class Access:
             self.window.core.plugins.get("audio_input").handler_simple.stop_recording(timeout=True)
 
         # stop audio output if playing
-        if self.window.controller.audio.is_playing():
-            self.window.controller.audio.stop_output()
+        #if self.window.controller.audio.is_playing():
+        self.window.controller.audio.stop_output()
 
         # stop generating if active
-        if self.window.controller.chat.input.generating:
-            self.window.controller.kernel.stop()
+        #if self.window.controller.chat.input.generating:
+        self.window.controller.kernel.stop()

pygpt_net/controller/access/control.py

@@ -6,7 +6,7 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.21 20:00:00                  #
+# Updated Date: 2024.11.26 19:00:00                  #
 # ================================================== #
 
 import re
@@ -22,7 +22,8 @@ from pygpt_net.core.types import (
     MODE_EXPERT,
     MODE_LANGCHAIN,
     MODE_LLAMA_INDEX,
-    MODE_VISION, MODE_IMAGE,
+    MODE_VISION, 
+    MODE_IMAGE,
 )
 from pygpt_net.core.tabs.tab import Tab
 from pygpt_net.core.events import ControlEvent

pygpt_net/controller/attachment.py

@@ -6,11 +6,12 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.23 00:00:00                  #
+# Updated Date: 2024.11.26 02:00:00                  #
 # ================================================== #
 
 import os
 from datetime import datetime
+from urllib.parse import urlparse
 
 from PySide6.QtGui import QImage
 from PySide6.QtWidgets import QFileDialog, QApplication
@@ -51,6 +52,13 @@ class Attachment:
         else:
             self.window.ui.nodes['attachments.capture_clear'].setChecked(False)
 
+        #  auto-index
+        if self.window.core.config.has('attachments_auto_index') \
+                and self.window.core.config.get('attachments_auto_index'):
+            self.window.ui.nodes['attachments.auto_index'].setChecked(True)
+        else:
+            self.window.ui.nodes['attachments.auto_index'].setChecked(False)
+
         self.window.core.attachments.load()
         self.update()
 
@@ -268,6 +276,38 @@ class Attachment:
             self.window.core.attachments.save()
             self.update()
 
+    def open_add_url(self):
+        """Open add attachment URL dialog"""
+        self.window.ui.dialog['url'].id = "attachment"
+        self.window.ui.dialog['url'].input.setText("")
+        self.window.ui.dialog['url'].current = ""
+        self.window.ui.dialog['url'].show()
+        self.window.ui.dialog['url'].input.setFocus()
+
+    def add_url(self, url: str):
+        """
+        Add URL
+
+        :param url: URL
+        """
+        if not url:
+            return
+        mode = self.window.core.config.get('mode')
+        try:
+            domain = urlparse(url).netloc
+        except Exception as e:
+            domain = os.path.basename(url)
+        attachment = self.window.core.attachments.new(
+            mode=mode,
+            name=domain,
+            path=url,
+            auto_save=False,
+            type=AttachmentItem.TYPE_URL,
+        )
+        self.window.core.attachments.save()
+        self.update()
+        self.window.ui.dialog['url'].close()
+
     def open_dir(self, mode: str, idx: int):
         """
         Open in directory
@@ -321,6 +361,24 @@ class Attachment:
             return ''
         return data.path
 
+    def get_by_idx(self, mode: str, idx: int) -> str:
+        """
+        Get attachment by index
+
+        :param mode: mode
+        :param idx: index
+        :return: path
+        """
+        file_id = self.window.core.attachments.get_id_by_idx(
+            mode=mode,
+            idx=idx,
+        )
+        data = self.window.core.attachments.get_by_id(
+            mode=mode,
+            id=file_id,
+        )
+        return data
+
     def has(self, mode: str) -> bool:
         """
         Return True if current mode has attachments
@@ -411,6 +469,14 @@ class Attachment:
         """
         self.window.core.config.set('attachments_capture_clear', value)
 
+    def toggle_auto_index(self, value: bool):
+        """
+        Toggle auto index
+
+        :param value: value of the checkbox
+        """
+        self.window.core.config.set('attachments_auto_index', value)
+
     def is_capture_clear(self) -> bool:
         """
         Return True if capture clear is enabled

pygpt_net/controller/audio/__init__.py

@@ -6,13 +6,12 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.20 21:00:00                  #
+# Updated Date: 2024.11.26 19:00:00                  #
 # ================================================== #
 
 import os
 
 from pygpt_net.core.events import Event, BaseEvent
-from pygpt_net.plugin.audio_output.worker import PlayWorker
 from pygpt_net.item.ctx import CtxItem
 from pygpt_net.utils import trans
 
@@ -64,6 +63,12 @@ class Audio:
         self.window.core.config.save()
         self.update()
 
+    def enable_input(self):
+        """Enable audio input"""
+        self.window.controller.plugins.enable('audio_input')
+        self.window.core.config.save()
+        self.update()
+
     def disable_input(self, update: bool = True):
         """
         Disable audio input
@@ -104,6 +109,16 @@ class Audio:
             return True
         return False
 
+    def is_input_enabled(self) -> bool:
+        """
+        Check if any audio input is enabled
+
+        :return: True if enabled
+        """
+        if self.window.controller.plugins.is_enabled('audio_input'):
+            return True
+        return False
+
     def update_listeners(self):
         """Update audio listeners"""
         is_output = False
@@ -161,16 +176,29 @@ class Audio:
         }
         self.window.dispatch(event, all=all)
 
+    def play_chat_audio(self, path: str):
+        """
+        Play audio file (chat multimodal response)
+
+        :param path: audio file path
+        """
+        if not self.is_output_enabled():
+            return
+        self.play_audio(path)
+
     def play_audio(self, path: str):
         """
         Play audio file
 
         :param path: audio file path
         """
-        worker = PlayWorker()
-        worker.window = self.window
-        worker.path = path
-        self.window.threadpool.start(worker)
+        ctx = CtxItem()
+        event = Event(Event.AUDIO_PLAYBACK)
+        event.ctx = ctx
+        event.data = {
+            'audio_file': path,
+        }
+        self.window.dispatch(event, all=True)
 
     def play_sound(self, filename: str):
         """

pygpt_net/controller/chat/__init__.py

@@ -6,13 +6,14 @@
 # GitHub:  https://github.com/szczyglis-dev/py-gpt   #
 # MIT License                                        #
 # Created By  : Marcin Szczygliński                  #
-# Updated Date: 2024.11.23 00:00:00                  #
+# Updated Date: 2024.11.26 19:00:00                  #
 # ================================================== #
 
 from pygpt_net.core.events import AppEvent
 from pygpt_net.item.ctx import CtxItem
 
 from .attachment import Attachment
+from .audio import Audio
 from .command import Command
 from .common import Common
 from .files import Files
@@ -35,6 +36,7 @@ class Chat:
         """
         self.window = window
         self.attachment = Attachment(window)
+        self.audio = Audio(window)
         self.command = Command(window)
         self.common = Common(window)
         self.files = Files(window)