warp-beacon 1.0.2__tar.gz → 1.0.4__tar.gz

{warp_beacon-1.0.2/warp_beacon.egg-info → warp_beacon-1.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: warp_beacon
-Version: 1.0.2
+Version: 1.0.4
 Summary: Telegram bot for expanding external media links
 Home-page: https://github.com/sb0y/warp_beacon
 Author: Andrey Bagrintsev
@@ -231,14 +231,14 @@ Requires-Dist: pillow
 Requires-Dist: pymongo
 Requires-Dist: instagrapi==2.0.0
 
-# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) 
+# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) [![Build DEB package](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml)
 
 Telegram bot for external social networks media scrapling.
 Works with links in personal messages and also with group chats.
 
 Just send to bot media link and get video.
 
-All bot configuration stored in [variables.env](https://github.com/sb0y/warp_beacon/blob/main/variables.env) file.
+All bot configuration stored in [warp_beacon.conf](https://github.com/sb0y/warp_beacon/blob/main/etc/warp_beacon.conf) file.
 
 ## Configuration example ##
 
@@ -261,9 +261,21 @@ WORKERS_POOL_SIZE=3
 Any Linux machine will suit in.
 
 Install docker and git
-```
-apt update
-apt install docker-compose git
+```bash
+sudo apt update
+# uninstall old docker packages
+for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
+sudo apt install ca-certificates curl git
+sudo install -m 0755 -d /etc/apt/keyrings
+sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+sudo chmod a+r /etc/apt/keyrings/docker.asc
+# Add the repository to Apt sources:
+echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+sudo apt update
+sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin git
 ```
 
 Download sources
@@ -277,10 +289,65 @@ cd warp_beacon
 
 Run app
 ```bash
-docker-compose up -d
+sudo docker compose up -d
 ```
 
 Check logs
 ```bash
-docker-compose logs warp_beacon -f
+sudo docker compose logs warp_beacon -f
+```
+
+## How to install from PIP ##
+
+```bash
+sudo apt update
+sudo apt install python3-pip
+sudo pip install warp-beacon
+```
+
+Your configuration file will be located at `/usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf`.
+For convenience, we will copy it to a common directory:
+
+```bash
+mkdir /etc/warp_beacon
+cp /usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf /etc/warp_beacon/warp_beacon.conf
+```
+
+Run the app
+
+```bash
+source /etc/warp_beacon/warp_beacon.conf && /usr/local/bin/warp_beacon
+```
+
+Most likely you will need a systemd service so that you don't have to start the service manually and don't have to worry about service start on server reboot.
+
+```bash
+cp /usr/local/lib/python3.10/dist-packages/lib/systemd/system/warp_beacon.service /lib/systemd/system
+systemctl unmask warp_beacon.service
+systemctl enable warp_beacon.service
+# start the service app
+systemctl start warp_beacon.service
+```
+
+## How to build Python whl package ##
+```bash
+sudo apt install python3-pip python3-build python3-virtualenv dh-virtualenv
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+python3 -m build
+```
+
+## How to build Ubuntu deb package ##
+
+```bash
+sudo apt update
+sudo apt install debhelper python3-pip python3-build python3-virtualenv dh-virtualenv dh-python
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+```
+
+build deb file
+
+```bash
+dpkg-buildpackage -us -uc -b
 ```

warp_beacon-1.0.4/README.md

@@ -0,0 +1,120 @@
+# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) [![Build DEB package](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml)
+
+Telegram bot for external social networks media scrapling.
+Works with links in personal messages and also with group chats.
+
+Just send to bot media link and get video.
+
+All bot configuration stored in [warp_beacon.conf](https://github.com/sb0y/warp_beacon/blob/main/etc/warp_beacon.conf) file.
+
+## Configuration example ##
+
+```env
+TG_TOKEN="you telegram token received from @BotFather"
+INSTAGRAM_LOGIN="instagram login (email or cell phone)"
+INSTAGRAM_PASSWORD="instgram password"
+INSTAGRAM_VERIFICATION_CODE="instagram 2FA if required, default empty"
+MONGODB_HOST="mongodb"
+MONGODB_PORT="27017"
+MONGODB_USER="root"
+MONGODB_PASSWORD="changeme"
+VIDEO_STORAGE_DIR="/var/warp_beacon/videos"
+WORKERS_POOL_SIZE=3
+```
+## Deployed example bot ##
+[Try it 🚀](https://t.me/anus_sebe_zablokiruy_bot)
+
+## How to run ##
+Any Linux machine will suit in.
+
+Install docker and git
+```bash
+sudo apt update
+# uninstall old docker packages
+for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
+sudo apt install ca-certificates curl git
+sudo install -m 0755 -d /etc/apt/keyrings
+sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+sudo chmod a+r /etc/apt/keyrings/docker.asc
+# Add the repository to Apt sources:
+echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+sudo apt update
+sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin git
+```
+
+Download sources
+```bash
+git clone https://github.com/sb0y/warp_beacon.git
+```
+Go to sources directory
+```bash
+cd warp_beacon
+```
+
+Run app
+```bash
+sudo docker compose up -d
+```
+
+Check logs
+```bash
+sudo docker compose logs warp_beacon -f
+```
+
+## How to install from PIP ##
+
+```bash
+sudo apt update
+sudo apt install python3-pip
+sudo pip install warp-beacon
+```
+
+Your configuration file will be located at `/usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf`.
+For convenience, we will copy it to a common directory:
+
+```bash
+mkdir /etc/warp_beacon
+cp /usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf /etc/warp_beacon/warp_beacon.conf
+```
+
+Run the app
+
+```bash
+source /etc/warp_beacon/warp_beacon.conf && /usr/local/bin/warp_beacon
+```
+
+Most likely you will need a systemd service so that you don't have to start the service manually and don't have to worry about service start on server reboot.
+
+```bash
+cp /usr/local/lib/python3.10/dist-packages/lib/systemd/system/warp_beacon.service /lib/systemd/system
+systemctl unmask warp_beacon.service
+systemctl enable warp_beacon.service
+# start the service app
+systemctl start warp_beacon.service
+```
+
+## How to build Python whl package ##
+```bash
+sudo apt install python3-pip python3-build python3-virtualenv dh-virtualenv
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+python3 -m build
+```
+
+## How to build Ubuntu deb package ##
+
+```bash
+sudo apt update
+sudo apt install debhelper python3-pip python3-build python3-virtualenv dh-virtualenv dh-python
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+```
+
+build deb file
+
+```bash
+dpkg-buildpackage -us -uc -b
+```

warp_beacon-1.0.4/etc/.gitignore

@@ -0,0 +1,2 @@
+warp_beacon.conf
+etc/warp_beacon.conf

warp_beacon-1.0.4/warp_beacon/__version__.py

@@ -0,0 +1,2 @@
+__version__ = "1.0.4"
+

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon/jobs/abstract.py

@@ -49,6 +49,14 @@ class AbstractJob(ABC):
 	def __repr__(self) -> str:
 		return str(self.to_dict())
 
+	def is_empty(self) -> bool:
+		if self.media_type == "collection":
+			if not self.media_collection:
+				return True
+		elif not self.local_media_path:
+			return True
+		return False
+
 	def to_dict(self) -> dict:
 		d = {}
 		for key in dir(self.__class__):

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon/jobs/download_job.py

@@ -20,4 +20,4 @@ class DownloadJob(AbstractJob):
 		if "media_collection" in d:
 			for k, v in enumerate(d["media_collection"]):
 				d["media_collection"][k] = UploadJob.build(**v)
-		return UploadJob.build(**d)
+		return UploadJob.build(**d)

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon/jobs/upload_job.py

@@ -23,4 +23,4 @@ class UploadJob(AbstractJob):
 		if key in self.__dict__:
 			self.__dict__[key] = value
 
-		return self
+		return self

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon/scrapler/__init__.py

@@ -3,7 +3,7 @@ import multiprocessing
 import time
 import logging
 from requests.exceptions import ConnectTimeout, HTTPError
-from instagrapi.exceptions import MediaNotFound, UnknownError
+from instagrapi.exceptions import MediaNotFound, UnknownError, ClientNotFoundError, UserNotFound
 
 from warp_beacon.mediainfo.video import VideoInfo
 from warp_beacon.uploader import AsyncUploader
@@ -70,8 +70,8 @@ class AsyncDownloader(object):
 										logging.error("ConnectTimeout download error!")
 										logging.exception(e)
 										time.sleep(2)
-									except MediaNotFound as e:
-										logging.warning("MediaNotFound occurred!")
+									except (MediaNotFound, ClientNotFoundError, UserNotFound) as e:
+										logging.warning("Not found error occurred!")
 										logging.exception(e)
 										self.uploader.queue_task(job.to_upload_job(
 											job_failed=True,
@@ -118,8 +118,17 @@ class AsyncDownloader(object):
 										else:
 											job_args["local_media_path"] = item["local_media_path"]
 
+										logging.debug("local_media_path: '%s'", job_args.get("local_media_path", ""))
+										logging.debug("media_collection: '%s'", str(job_args.get("media_collection", {})))
 										upload_job = job.to_upload_job(**job_args)
-										self.uploader.queue_task(upload_job)
+										if upload_job.is_empty():
+											logging.info("Upload job is empty. Nothing to do here!")
+											self.uploader.queue_task(job.to_upload_job(
+												job_failed=True,
+												job_failed_msg="Seems like this link doesn't contains any media.")
+											)
+										else:
+											self.uploader.queue_task(upload_job)
 							else:
 								logging.info("Job already in work in parallel worker. Redirecting job to upload worker.")
 								self.uploader.queue_task(job.to_upload_job())

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon/scrapler/instagram.py

@@ -6,6 +6,7 @@ from typing import Callable, Optional, Union
 
 import requests
 import urllib3
+from urllib.parse import urljoin, urlparse
 import logging
 
 from instagrapi.mixins.story import Story
@@ -51,7 +52,10 @@ class InstagramScrapler(ScraplerAbstract):
 		self.load_session()
 		def _scrap() -> tuple[str]:
 			if "stories" in url:
-				url_last_part = list(filter(None, url.split('/')))[-1]
+				# remove URL options
+				_url = urljoin(url, urlparse(url).path)
+				url_last_part = list(filter(None, _url.split('/')))[-1]
+				logging.debug("url last part: '%s'", url_last_part)
 				if url_last_part.isnumeric():
 					return "story", self.scrap_story(url)
 				else:

{warp_beacon-1.0.2 → warp_beacon-1.0.4/warp_beacon.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: warp_beacon
-Version: 1.0.2
+Version: 1.0.4
 Summary: Telegram bot for expanding external media links
 Home-page: https://github.com/sb0y/warp_beacon
 Author: Andrey Bagrintsev
@@ -231,14 +231,14 @@ Requires-Dist: pillow
 Requires-Dist: pymongo
 Requires-Dist: instagrapi==2.0.0
 
-# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) 
+# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) [![Build DEB package](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/build-deb.yml)
 
 Telegram bot for external social networks media scrapling.
 Works with links in personal messages and also with group chats.
 
 Just send to bot media link and get video.
 
-All bot configuration stored in [variables.env](https://github.com/sb0y/warp_beacon/blob/main/variables.env) file.
+All bot configuration stored in [warp_beacon.conf](https://github.com/sb0y/warp_beacon/blob/main/etc/warp_beacon.conf) file.
 
 ## Configuration example ##
 
@@ -261,9 +261,21 @@ WORKERS_POOL_SIZE=3
 Any Linux machine will suit in.
 
 Install docker and git
-```
-apt update
-apt install docker-compose git
+```bash
+sudo apt update
+# uninstall old docker packages
+for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
+sudo apt install ca-certificates curl git
+sudo install -m 0755 -d /etc/apt/keyrings
+sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+sudo chmod a+r /etc/apt/keyrings/docker.asc
+# Add the repository to Apt sources:
+echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+sudo apt update
+sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin git
 ```
 
 Download sources
@@ -277,10 +289,65 @@ cd warp_beacon
 
 Run app
 ```bash
-docker-compose up -d
+sudo docker compose up -d
 ```
 
 Check logs
 ```bash
-docker-compose logs warp_beacon -f
+sudo docker compose logs warp_beacon -f
+```
+
+## How to install from PIP ##
+
+```bash
+sudo apt update
+sudo apt install python3-pip
+sudo pip install warp-beacon
+```
+
+Your configuration file will be located at `/usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf`.
+For convenience, we will copy it to a common directory:
+
+```bash
+mkdir /etc/warp_beacon
+cp /usr/local/lib/python3.10/dist-packages/etc/warp_beacon/warp_beacon.conf /etc/warp_beacon/warp_beacon.conf
+```
+
+Run the app
+
+```bash
+source /etc/warp_beacon/warp_beacon.conf && /usr/local/bin/warp_beacon
+```
+
+Most likely you will need a systemd service so that you don't have to start the service manually and don't have to worry about service start on server reboot.
+
+```bash
+cp /usr/local/lib/python3.10/dist-packages/lib/systemd/system/warp_beacon.service /lib/systemd/system
+systemctl unmask warp_beacon.service
+systemctl enable warp_beacon.service
+# start the service app
+systemctl start warp_beacon.service
+```
+
+## How to build Python whl package ##
+```bash
+sudo apt install python3-pip python3-build python3-virtualenv dh-virtualenv
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+python3 -m build
+```
+
+## How to build Ubuntu deb package ##
+
+```bash
+sudo apt update
+sudo apt install debhelper python3-pip python3-build python3-virtualenv dh-virtualenv dh-python
+# If you are getting build errors you probably need the latest version of python3-build
+sudo python3 -m pip install --upgrade build
+```
+
+build deb file
+
+```bash
+dpkg-buildpackage -us -uc -b
 ```

{warp_beacon-1.0.2 → warp_beacon-1.0.4}/warp_beacon.egg-info/SOURCES.txt

@@ -4,6 +4,7 @@ README.md
 pyproject.toml
 setup.cfg
 setup.py
+etc/.gitignore
 etc/warp_beacon.conf
 etc/warp_beacon.service
 warp_beacon/__init__.py

warp_beacon-1.0.2/README.md

@@ -1,53 +0,0 @@
-# warp_beacon [![Upload Python Package](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/python-publish.yml) [![Docker Image CI](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml/badge.svg)](https://github.com/sb0y/warp_beacon/actions/workflows/docker-image.yml) 
-
-Telegram bot for external social networks media scrapling.
-Works with links in personal messages and also with group chats.
-
-Just send to bot media link and get video.
-
-All bot configuration stored in [variables.env](https://github.com/sb0y/warp_beacon/blob/main/variables.env) file.
-
-## Configuration example ##
-
-```env
-TG_TOKEN="you telegram token received from @BotFather"
-INSTAGRAM_LOGIN="instagram login (email or cell phone)"
-INSTAGRAM_PASSWORD="instgram password"
-INSTAGRAM_VERIFICATION_CODE="instagram 2FA if required, default empty"
-MONGODB_HOST="mongodb"
-MONGODB_PORT="27017"
-MONGODB_USER="root"
-MONGODB_PASSWORD="changeme"
-VIDEO_STORAGE_DIR="/var/warp_beacon/videos"
-WORKERS_POOL_SIZE=3
-```
-## Deployed example bot ##
-[Try it 🚀](https://t.me/anus_sebe_zablokiruy_bot)
-
-## How to run ##
-Any Linux machine will suit in.
-
-Install docker and git
-```
-apt update
-apt install docker-compose git
-```
-
-Download sources
-```bash
-git clone https://github.com/sb0y/warp_beacon.git
-```
-Go to sources directory
-```bash
-cd warp_beacon
-```
-
-Run app
-```bash
-docker-compose up -d
-```
-
-Check logs
-```bash
-docker-compose logs warp_beacon -f
-```

warp_beacon-1.0.2/warp_beacon/__version__.py

@@ -1,2 +0,0 @@
-__version__ = "1.0.2"
-