GameSentenceMiner 2.1.2.post1__py3-none-any.whl → 2.2.1__py3-none-any.whl

GameSentenceMiner-2.2.1.dist-info/METADATA

@@ -0,0 +1,229 @@
+Metadata-Version: 2.2
+Name: GameSentenceMiner
+Version: 2.2.1
+Summary: A tool for mining sentences from games. Update: Use ffprobe from GSM Directory
+Author-email: Beangate <bpwhelan95@gmail.com>
+License: MIT License
+Project-URL: Homepage, https://github.com/bpwhelan/GameSentenceMiner
+Project-URL: Repository, https://github.com/bpwhelan/GameSentenceMiner
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: requests~=2.32.3
+Requires-Dist: watchdog~=5.0.2
+Requires-Dist: DateTime~=5.5
+Requires-Dist: pyperclip~=1.9.0
+Requires-Dist: vosk~=0.3.45
+Requires-Dist: soundfile~=0.12.1
+Requires-Dist: toml~=0.10.2
+Requires-Dist: psutil~=6.0.0
+Requires-Dist: rapidfuzz~=3.9.7
+Requires-Dist: obs-websocket-py~=1.0
+Requires-Dist: plyer~=2.1.0
+Requires-Dist: keyboard~=0.13.5
+Requires-Dist: websockets~=13.0.1
+Requires-Dist: stable-ts~=2.17.5
+Requires-Dist: silero-vad~=5.1.2
+Requires-Dist: ttkbootstrap~=1.10.1
+Requires-Dist: dataclasses_json~=0.6.7
+Requires-Dist: numpy
+Requires-Dist: pystray
+Requires-Dist: pywin32
+
+# Game Sentence Miner
+
+This project automates the recording of game sentence audio to help with Anki Card Creation.
+
+This allows us to create cards from texthooker/yomitan, and automatically get screenshot and sentence audio from the
+game we are playing.
+
+Short Demo (Watch this first): https://www.youtube.com/watch?v=J2At52oWieU
+
+Installation: https://www.youtube.com/watch?v=b-L4g9tA508
+
+## Features:
+
+- **OBS Replay Buffer**: Constantly records the last X seconds of gameplay.
+- **Voice Activity Detection**: Automatically cuts the end of the clip to the exact moment the voice ended.
+- **Clipboard Interaction**: Automatically monitors the clipboard for dialogue events.
+- **Websocket Listening**: Listens to a websocket uri for text-events from stuff like Agent/Textractor.
+- **Hotkey Automation**: Single hotkey to trigger video recording, screenshot, and transcription.
+- **1-Click Card Creation**: Monitors anki for new cards from Yomitan, and automatically gets audio from games.
+
+## Prerequisites
+
+- [Python 3.11+](https://www.python.org/downloads/release/python-3119/)
+- Important: 3.13 is [NOT supported](https://stackoverflow.com/questions/79175945/keyerror-version-installing-openai-whisper-on-python-3-13).
+
+---
+
+## 1. Installing and Running the Script
+
+https://pypi.org/project/GameSentenceMiner/
+
+Python + pip needs to be installed, make sure you install 3.11 or higher, since older versions may not be supported.
+
+Install:
+```commandline
+pip install gamesentenceminer
+```
+
+Run:
+```commandline
+gamesentenceminer
+```
+
+On first run, this will download OBS and FFMPEG, and do some basic configuration. You will need to edit the GSM config for your specific Anki fields and whatnot.
+
+---
+
+## 2. Setting Up OBS Replay Buffer
+
+1. Go to Settings > Output > Replay Buffer, and make sure OBS Replay Buffer is enabled, I recommend setting it to 60 seconds, but shorter and longer buffers should also work.
+2. Set Scene/Source. I recommend using "Game Capture" with "Capture Audio" Enabled. And then mute Desktop/microphone
+    1. If "Game Capture" Does not work, use "Window Capture".
+    2. I recommend having a Scene PER Game, with the name of the scene labeled as the game, this makes it easier for the
+       script to know (with a config option) what game you are playing.
+3. In Output Settings, set "Recording Format" to mkv, and "Audio Encoder" to Opus. Alternate Audio Encoder settings are supported, but will be re-encoded to Opus by default.
+
+Here are the Settings I use in OBS. Make sure the recordings folder is the same as the "folder_to_watch" in the config.
+![image](https://github.com/user-attachments/assets/0056816d-af3c-4a3c-bc6a-4aff5c28cadb)
+![image](https://github.com/user-attachments/assets/dd2f95a6-f546-41d9-8136-de7b1b035a5d)
+
+---
+
+## 3. Configuring the App.
+
+### Configuration GUI
+
+The `GameSentenceMiner` project now includes a graphical interface to simplify configuration. With default values
+already set, this GUI lets you adjust settings as needed. Here’s how to get started:
+
+#### Running the Configuration GUI
+
+To open the GUI, you have two options:
+
+1. **Tray Icon**: Right Click the Tray Icon and Click `Open Settings`
+
+#### Default Settings and Customization
+
+The GUI loads with default values for each setting, so if you’re just getting started, you may only need to change options
+in the "path" config. If you make changes, remember to click **Save Settings** to apply them.
+Please take a second to look through the config to see what is available, there is a lot of extra functionality hidden
+behind config options.
+
+![image](https://github.com/user-attachments/assets/ffac9888-de0a-412b-817f-e22a55ce7b55)![image](https://github.com/user-attachments/assets/981c112a-1ddc-4e07-9c39-57fe46644ff5)![image](https://github.com/user-attachments/assets/29470a97-6013-4ca8-9059-48af735eb3a8)![image](https://github.com/user-attachments/assets/8e9c8f03-dc43-4822-a3c5-43f36ca65364)
+
+
+---
+
+
+## 4. One Click Card Creation
+
+This is the flagship feature of this script, so here is a section explaining it. It is possible to do full 1-click card
+creation with this tool + Yomitan/JL. The relevant settings are located in `Features` and `OBS` section in the config.
+
+Demo: https://www.youtube.com/watch?v=9dmmXO2CGNw
+
+Screenshots to help with setup (THIS IS ALREADY DONE FOR YOU IN NEWER VERSIONS):
+
+![image](https://github.com/user-attachments/assets/7de031e9-ce28-42eb-a8fd-0e60ef70dc3d)
+
+![image](https://github.com/user-attachments/assets/b0c70a1a-65b5-4fe7-a7e4-ccb0b9a5b249)
+
+![image](https://github.com/user-attachments/assets/4cf492eb-12a2-429f-aa0e-f87fc0fa6270)
+
+## 4. Example Process
+
+1. Start script: `gamesentenceminer`
+2. Start game
+3. Hook Game with Agent (or textractor) with clipboard enabled
+4. Create Anki Card with target word (through a texthooker page/Yomitan)
+5. When finished gaming, End script (not required)
+
+Once the Anki card is created:
+
+1. **OBS** will save the last X seconds of gameplay.
+2. The Python script will trim the audio based on last clipboard event, and the end of voice line detected in VAD if
+   enabled.
+3. Will attempt to update the LAST anki card created.
+
+---
+
+## How to Update the Script
+
+### PIP Install (Preferred)
+
+If you installed the script via pip, you can update it with the following command:
+```bash
+pip install --upgrade gamesentenceminer
+```
+
+
+### Source (For if you want to make edits to the script)
+
+I will probably remove this section at a later date. If you want to make edits to the script, you should know how to do this.
+
+To ensure you always have the latest version of this script, you can use `git pull` to update your local repository with
+the latest changes from the remote repository.
+
+#### Step-by-Step Instructions
+
+1. Open your terminal and navigate to the directory where you cloned the repository:
+    ```bash
+    cd path/to/script
+    ```
+
+2. Run the following command to fetch and integrate the latest changes:
+    ```bash
+    git pull origin main
+    ```
+
+    - **`origin`** refers to the remote repository from which you cloned the code.
+    - **`main`** refers to the main branch of the repository. I~~~~f your default branch has a different name (
+      e.g., `master` or `dev`), replace `main` with that branch name.
+
+3. The `git pull` command will download and apply any updates from the remote repository to your local version.
+
+### Example:
+
+```bash
+$ cd path/to/script
+$ git pull origin master
+```
+
+---
+
+## Disclaimer/Troubleshooting
+
+Every game/hook is different, so it's really impossible that any script can get it perfect every time. Also OBS is
+sometimes a bit finicky if running for too long. If the audio timing is off, please first try some troubleshooting
+steps before making an issue:
+
+- Try Restarting OBS
+- Make sure your hook is the best you can find. (Preferably it gives you the text RIGHT when the voice line starts)
+- Try Adjusting Offset Configuration in `config.toml` to better match your situation. (i.e. if the hook is late, add a
+  negative beginning offset)
+- Try using "Trim beginning" in `VAD` settings.
+
+### Setup Troubleshooting
+
+Just going to continuously update this with issues that I have helped users with. Look here first if you have issues
+setting it up.
+
+- Make sure folder_to_watch is the same as your recordings path in OBS. It defaults to ~/Videos, but I recommend setting
+  it to ~/Videos/GSM.
+- If using clipboard, make sure Agent/Textractor sending to clipboard is enabled.
+- If using websocket, make sure the websocket server is running and the uri is correct in both GSM AND agent/textractor. Textractor uses a default of 6677, and I would recommend changing Agent to use 6677 as well.
+
+
+## Contact
+
+If you run into issues ask in my [Discord](https://discord.gg/yP8Qse6bb8), or make an issue here.
+
+## Donations
+
+If you've benefited from this or any of my other projects, please consider supporting my work
+via [Github Sponsors](https://github.com/sponsors/bpwhelan) or [Ko-fi.](https://ko-fi.com/beangate)

GameSentenceMiner-2.2.1.dist-info/RECORD

@@ -0,0 +1,24 @@
+src/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+src/anki.py,sha256=FfOxvozC4LBzhexiyY_uDSH3wgxzf4Uqvf8DZyyGlsQ,9138
+src/config_gui.py,sha256=oY397FO5WYHluPNOtfed3Eakw3TMpVStCaoEU60jw00,48963
+src/configuration.py,sha256=Cw5vUF74ZKcSinbciNMiLKpMO8M6sf-0THokHO_31nE,14010
+src/ffmpeg.py,sha256=-P0ysDrBefYpu8JOV6-tu52zqg2wDMwVqSVeO5iFQxc,10836
+src/gametext.py,sha256=EGFGzNrkLX_YdUWbksE7bf_4P7O-zcbdhsgubpUt5xw,4025
+src/gsm.py,sha256=DTVkA-MxFq3zsxH1wl1TJPz-sQIrjZdxCtzMsE9YYW0,16250
+src/model.py,sha256=oh8VVT8T1UKekbmP6MGNgQ8jIuQ_7Rg4GPzDCn2kJo8,1999
+src/notification.py,sha256=4lteRvzizUb081t_87DpophOQceBnNerNq3ppq5POH8,1996
+src/obs.py,sha256=SAP-_JB-37ZjZIbpWs9-MYPJHtT0BpKLD-KQuvneHtg,6538
+src/package_updater.py,sha256=Yel-KWBHr5kgIJpHM1A1w5tcx_gDqrLrK0pqXRhinlI,1472
+src/util.py,sha256=QRJMSEtnLaT4c2avQ0gZNn1oPtDfBbqcE8zuFHTzbd8,4569
+src/downloader/Untitled_json.py,sha256=RUUl2bbbCpUDUUS0fP0tdvf5FngZ7ILdA_J5TFYAXUQ,15272
+src/downloader/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+src/downloader/download_tools.py,sha256=m1UV8EvWJdNn2cz11BCUI2mth46RhbVdVHXYYC318V8,6378
+src/vad/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+src/vad/silero_trim.py,sha256=rcHzSNjAm_liiKBWsdywpIC1oUrM00I7kxmomXCQ-Io,1521
+src/vad/vosk_helper.py,sha256=4oaI7uesINu1sO7PA0-M4qSOmiEneRRN4Ae8ztmYCBE,5798
+src/vad/whisper_helper.py,sha256=N8-FZMIjLb5CiFeyY8pz050v7RguU39B6tnvsPTbSmc,3383
+GameSentenceMiner-2.2.1.dist-info/METADATA,sha256=QZzBzJ6OpsODRoGaY52g3VQOQEfhKLhrlsmNmdKpVqs,9157
+GameSentenceMiner-2.2.1.dist-info/WHEEL,sha256=In9FTNxeP60KnTkGw7wk6mJPYd_dQSjEZmXdBdMCI-8,91
+GameSentenceMiner-2.2.1.dist-info/entry_points.txt,sha256=hzWrryQzGBdqbVqzUIBbVZvvh7MBTUYvWiIs95QrfHE,51
+GameSentenceMiner-2.2.1.dist-info/top_level.txt,sha256=74rtVfumQlgAPzR5_2CgYN24MB0XARCg0t-gzk6gTrM,4
+GameSentenceMiner-2.2.1.dist-info/RECORD,,

GameSentenceMiner-2.2.1.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+gamesentenceminer = src.gsm:main

GameSentenceMiner-2.2.1.dist-info/top_level.txt

@@ -0,0 +1 @@
+src

{GameSentenceMiner → src}/anki.py

@@ -6,15 +6,12 @@ import urllib.request
 
 import requests as req
 
-from . import util
-from . import ffmpeg
-from . import notification
-from . import obs
+from src import obs, util, notification, ffmpeg
 
-from .configuration import *
-from .configuration import get_config
-from .gametext import get_last_two_sentences
-from .obs import get_current_game
+from src.configuration import *
+from src.configuration import get_config
+from src.gametext import get_last_two_sentences
+from src.obs import get_current_game
 
 audio_in_anki = None
 screenshot_in_anki = None
@@ -171,15 +168,15 @@ def get_cards_by_sentence(sentence):
     card_ids = invoke("findCards", query=query)
 
     if not card_ids:
-        print(f"Didn't find any cards matching query:\n{query}")
+        logger.warning(f"Didn't find any cards matching query:\n{query}")
         return {}
     if len(card_ids) > 1:
-        print(f'Found more than 1, and not updating cards for query: \n{query}')
+        logger.warning(f'Found more than 1, and not updating cards for query: \n{query}')
         return {}
 
     last_notes = invoke('notesInfo', notes=[card_ids[0]])[0]
 
-    print(f"Found Card to backfill!: {card_ids[0]}")
+    logger.info(f"Found Card to backfill!: {card_ids[0]}")
 
     return last_notes
 
@@ -191,7 +188,7 @@ def check_for_new_cards():
     try:
         current_note_ids = get_note_ids()
     except Exception as e:
-        print(f"Error fetching Anki notes: {e}")
+        logger.error(f"Error fetching Anki notes: {e}")
         return
     new_card_ids = current_note_ids - previous_note_ids
     if new_card_ids and not first_run:
@@ -208,16 +205,16 @@ def update_new_card():
 
     use_prev_audio = util.use_previous_audio
     if util.lock.locked():
-        print("Audio still being Trimmed, Card Queued!")
+        logger.info("Audio still being Trimmed, Card Queued!")
         use_prev_audio = True
     with util.lock:
-        print(f"use previous audio: {use_prev_audio}")
+        logger.info(f"use previous audio: {use_prev_audio}")
         if get_config().obs.get_game_from_scene:
             obs.update_current_game()
         if use_prev_audio:
             update_anki_card(last_card, note=get_initial_card_info(last_card), reuse_audio=True)
         else:
-            print("New card(s) detected!")
+            logger.info("New card(s) detected!")
             obs.save_replay_buffer()
 
 

{GameSentenceMiner → src}/config_gui.py

@@ -3,10 +3,9 @@ from tkinter import filedialog, messagebox, simpledialog
 
 import ttkbootstrap as ttk
 
-from .package_updater import check_for_updates, get_latest_version, update, get_current_version
-from . import configuration
-from . import obs
-from .configuration import *
+from src.package_updater import check_for_updates, get_latest_version, update, get_current_version
+from src import obs, configuration
+from src.configuration import *
 
 settings_saved = False
 on_save = []

{GameSentenceMiner → src}/configuration.py

@@ -46,9 +46,9 @@ class General:
 @dataclass_json
 @dataclass
 class Paths:
-    folder_to_watch: str = expanduser("~/Videos/OBS")
-    audio_destination: str = expanduser("~/Videos/OBS/Audio/")
-    screenshot_destination: str = expanduser("~/Videos/OBS/SS/")
+    folder_to_watch: str = expanduser("~/Videos/GSM")
+    audio_destination: str = expanduser("~/Videos/GSM/Audio/")
+    screenshot_destination: str = expanduser("~/Videos/GSM/SS/")
     remove_video: bool = True
     remove_audio: bool = False
     remove_screenshot: bool = False
@@ -305,13 +305,11 @@ def load_config():
                     config = ProfileConfig.from_dict(config_file)
                     new_config = Config(configs = {DEFAULT_CONFIG : config}, current_profile=DEFAULT_CONFIG)
 
-                    print(new_config)
-
                     with open(config_path, 'w') as file:
                         json.dump(new_config.to_dict(), file, indent=4)
                     return new_config
         except json.JSONDecodeError as e:
-            print(f"Error parsing config.json: {e}")
+            logger.error(f"Error parsing config.json: {e}")
             return None
     elif os.path.exists('config.toml'):
         config = ProfileConfig().load_from_toml('config.toml')
@@ -334,7 +332,7 @@ def get_config():
         config = config_instance.get_config()
 
         if config.features.backfill_audio and config.features.full_auto:
-            print("Cannot have backfill_audio and obs_full_auto_mode turned on at the same time!")
+            logger.error("Cannot have backfill_audio and obs_full_auto_mode turned on at the same time!")
             exit(1)
 
     # print(config_instance.get_config())
@@ -369,7 +367,7 @@ console_handler = logging.StreamHandler()
 console_handler.setLevel(logging.INFO)
 
 # Create rotating file handler with level DEBUG
-file_handler = RotatingFileHandler(get_log_path(), maxBytes=10_000_000, backupCount=2, encoding='utf-8')
+file_handler = RotatingFileHandler(get_log_path(), maxBytes=10 * 1024 * 1024, backupCount=0, encoding='utf-8')
 file_handler.setLevel(logging.DEBUG)
 
 # Create a formatter