starwarden 1.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,188 @@
1
+ Metadata-Version: 2.3
2
+ Name: starwarden
3
+ Version: 1.5.0
4
+ Summary: Sync GitHub Stars to Linkwarden
5
+ Requires-Dist: apprise==2.2.0
6
+ Requires-Dist: pygithub==2.8.1
7
+ Requires-Dist: python-dotenv==1.2.4
8
+ Requires-Dist: requests==2.33.0
9
+ Requires-Dist: rich==15.0.0
10
+ Requires-Dist: tqdm==4.70.1
11
+ Requires-Python: >=3.13
12
+ Description-Content-Type: text/markdown
13
+
14
+ ![Docker Pulls](https://img.shields.io/docker/pulls/rtuszik/starwarden) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/rtuszik/starwarden/docker.yml) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/rtuszik/starwarden/lint.yml)
15
+
16
+ # StarWarden
17
+
18
+ StarWarden allows you to export GitHub stars to Linkwarden.
19
+
20
+ ![screenshot](https://github.com/rtuszik/starwarden/blob/main/assets/screenshot.png?raw=true)
21
+
22
+ ## Features
23
+
24
+ - Export GitHub stars to Linkwarden
25
+ - Export to existing collection or create new colletion
26
+ - Keep Linkwarden collection up-to-date with your GitHub stars using the docker image.
27
+
28
+ ## Installation
29
+
30
+ StarWarden requires Python 3.13 or later.
31
+
32
+ ### From PyPI
33
+
34
+ Install with [uv](https://docs.astral.sh/uv/getting-started/installation/):
35
+
36
+ ```bash
37
+ uv tool install starwarden
38
+ ```
39
+
40
+ Or with [mise](https://mise.jdx.dev/):
41
+
42
+ ```bash
43
+ mise use -g pypi:starwarden
44
+ ```
45
+
46
+ Set the required environment variables in your shell:
47
+
48
+ ```bash
49
+ export GITHUB_TOKEN=your_github_token
50
+ export GITHUB_USERNAME=your_github_username
51
+ export LINKWARDEN_URL=your_linkwarden_instance_url
52
+ export LINKWARDEN_TOKEN=your_linkwarden_api_token
53
+ ```
54
+
55
+ You can also put these variables in a `.env` file in the directory where you run `starwarden`. See [Environment Variables](#environment-variables) for all options.
56
+
57
+ Run:
58
+
59
+ ```bash
60
+ starwarden
61
+ ```
62
+
63
+ To update an existing collection without the interactive menu:
64
+
65
+ ```bash
66
+ starwarden -id YOUR_COLLECTION_ID
67
+ ```
68
+
69
+ To upgrade:
70
+
71
+ ```bash
72
+ uv tool upgrade starwarden
73
+ # or
74
+ mise upgrade pypi:starwarden
75
+ ```
76
+
77
+ ### From Source
78
+
79
+ 1. Clone the repository
80
+ 2. Install requirements:
81
+
82
+ ```bash
83
+ uv sync --locked
84
+ ```
85
+
86
+ 3. Create a `.env` file with:
87
+
88
+ ```bash
89
+ GITHUB_TOKEN=your_github_token
90
+ GITHUB_USERNAME=your_github_username
91
+ LINKWARDEN_URL=your_linkwarden_instance_url
92
+ LINKWARDEN_TOKEN=your_linkwarden_api_token
93
+ APPRISE_URLS=apprise_url_1,apprise_url_2
94
+ OPT_TAG=true
95
+ OPT_TAG_GITHUB=true
96
+ OPT_TAG_GITHUBSTARS=true
97
+ OPT_TAG_LANGUAGE=false
98
+ OPT_TAG_USERNAME=false
99
+ ```
100
+
101
+ #### Usage
102
+
103
+ Run:
104
+
105
+ ```bash
106
+ uv run starwarden.py
107
+ ```
108
+
109
+ To directly update an existing collection without an interactive menu, run:
110
+
111
+ ```bash
112
+ uv run starwarden.py -id YOUR_COLLECTION_ID
113
+ ```
114
+
115
+ ## Environment Variables
116
+
117
+ | Name | Default | Description |
118
+ | :------------------ | :----------: | :------------------------------------------------------------------------------------------------------------- |
119
+ | GITHUB_TOKEN | | GitHub API token |
120
+ | GITHUB_USERNAME | | GitHub username |
121
+ | LINKWARDEN_URL | | Linkwarden URL <https://your-linkwarden-instance.com> |
122
+ | LINKWARDEN_TOKEN | | Linkwarden API token |
123
+ | COLLECTION_ID | | Linkwarden Collection ID to update (Number) /collections/499 |
124
+ | CRON_SCHEDULE | 0 6 \* \* \* | Cron Schedule (default is daily at 6am) |
125
+ | APPRISE_URLS | | Apprise URL for push notifications, separated by commas, see [Apprise](https://github.com/caronc/apprise/wiki) |
126
+ | OPT_TAG | true | Enable/Disable all Tagging |
127
+ | OPT_TAG_GITHUB | true | Tag Link with "GitHub" |
128
+ | OPT_TAG_GITHUBSTARS | true | Tag Link with "GitHub Stars" |
129
+ | OPT_TAG_LANGUAGE | false | Tag Link with Language of repo (e.g. Python or JavaScript) |
130
+ | OPT_TAG_USERNAME | false | Tag GitHub username |
131
+ | OPT_TAG_CUSTOM | | Add custom tags, separated by commas (e.g. tag1,tag2) |
132
+
133
+ ## Unsupervised Updates
134
+
135
+ For automated, unsupervised updates, you can use Docker with the provided docker-compose.yml file.
136
+
137
+ 1. Make sure you have Docker and Docker Compose installed on your system.
138
+
139
+ 2. Create a `.env` file in the project root with the following variables:
140
+
141
+ ```bash
142
+ GITHUB_TOKEN=your_github_token
143
+ GITHUB_USERNAME=your_github_username
144
+ LINKWARDEN_URL=your_linkwarden_instance_url
145
+ LINKWARDEN_TOKEN=your_linkwarden_api_token
146
+ COLLECTION_ID=your_linkwarden_collection_id
147
+ CRON_SCHEDULE=0 0 * * * # Run daily at midnight, adjust as needed
148
+ APPRISE_URLS=apprise_url_1,apprise_url_2
149
+ OPT_TAG=true
150
+ OPT_TAG_GITHUB=true
151
+ OPT_TAG_GITHUBSTARS=true
152
+ OPT_TAG_LANGUAGE=false
153
+ OPT_TAG_USERNAME=false
154
+ ```
155
+
156
+ 3. Use the following `docker-compose.yml` file:
157
+
158
+ ```yaml
159
+ version: "3"
160
+ services:
161
+ starwarden:
162
+ image: rtuszik/starwarden:latest
163
+ env_file: .env
164
+ volumes:
165
+ - ./starwarden.log:/app/starwarden.log
166
+ ```
167
+
168
+ 4. Run the following command to start the Docker container:
169
+
170
+ ```bash
171
+ docker compose up -d
172
+ ```
173
+
174
+ The container will now automatically run StarWarden on the specified schedule without any manual intervention.
175
+
176
+ To manually trigger an update, you can run:
177
+
178
+ ```bash
179
+ docker compose exec starwarden python /app/starwarden.py -id $COLLECTION_ID
180
+ ```
181
+
182
+ ## Contributing
183
+
184
+ Contributions are welcome! Please feel free to submit a Pull Request.
185
+
186
+ ## License
187
+
188
+ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
@@ -0,0 +1,175 @@
1
+ ![Docker Pulls](https://img.shields.io/docker/pulls/rtuszik/starwarden) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/rtuszik/starwarden/docker.yml) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/rtuszik/starwarden/lint.yml)
2
+
3
+ # StarWarden
4
+
5
+ StarWarden allows you to export GitHub stars to Linkwarden.
6
+
7
+ ![screenshot](https://github.com/rtuszik/starwarden/blob/main/assets/screenshot.png?raw=true)
8
+
9
+ ## Features
10
+
11
+ - Export GitHub stars to Linkwarden
12
+ - Export to existing collection or create new colletion
13
+ - Keep Linkwarden collection up-to-date with your GitHub stars using the docker image.
14
+
15
+ ## Installation
16
+
17
+ StarWarden requires Python 3.13 or later.
18
+
19
+ ### From PyPI
20
+
21
+ Install with [uv](https://docs.astral.sh/uv/getting-started/installation/):
22
+
23
+ ```bash
24
+ uv tool install starwarden
25
+ ```
26
+
27
+ Or with [mise](https://mise.jdx.dev/):
28
+
29
+ ```bash
30
+ mise use -g pypi:starwarden
31
+ ```
32
+
33
+ Set the required environment variables in your shell:
34
+
35
+ ```bash
36
+ export GITHUB_TOKEN=your_github_token
37
+ export GITHUB_USERNAME=your_github_username
38
+ export LINKWARDEN_URL=your_linkwarden_instance_url
39
+ export LINKWARDEN_TOKEN=your_linkwarden_api_token
40
+ ```
41
+
42
+ You can also put these variables in a `.env` file in the directory where you run `starwarden`. See [Environment Variables](#environment-variables) for all options.
43
+
44
+ Run:
45
+
46
+ ```bash
47
+ starwarden
48
+ ```
49
+
50
+ To update an existing collection without the interactive menu:
51
+
52
+ ```bash
53
+ starwarden -id YOUR_COLLECTION_ID
54
+ ```
55
+
56
+ To upgrade:
57
+
58
+ ```bash
59
+ uv tool upgrade starwarden
60
+ # or
61
+ mise upgrade pypi:starwarden
62
+ ```
63
+
64
+ ### From Source
65
+
66
+ 1. Clone the repository
67
+ 2. Install requirements:
68
+
69
+ ```bash
70
+ uv sync --locked
71
+ ```
72
+
73
+ 3. Create a `.env` file with:
74
+
75
+ ```bash
76
+ GITHUB_TOKEN=your_github_token
77
+ GITHUB_USERNAME=your_github_username
78
+ LINKWARDEN_URL=your_linkwarden_instance_url
79
+ LINKWARDEN_TOKEN=your_linkwarden_api_token
80
+ APPRISE_URLS=apprise_url_1,apprise_url_2
81
+ OPT_TAG=true
82
+ OPT_TAG_GITHUB=true
83
+ OPT_TAG_GITHUBSTARS=true
84
+ OPT_TAG_LANGUAGE=false
85
+ OPT_TAG_USERNAME=false
86
+ ```
87
+
88
+ #### Usage
89
+
90
+ Run:
91
+
92
+ ```bash
93
+ uv run starwarden.py
94
+ ```
95
+
96
+ To directly update an existing collection without an interactive menu, run:
97
+
98
+ ```bash
99
+ uv run starwarden.py -id YOUR_COLLECTION_ID
100
+ ```
101
+
102
+ ## Environment Variables
103
+
104
+ | Name | Default | Description |
105
+ | :------------------ | :----------: | :------------------------------------------------------------------------------------------------------------- |
106
+ | GITHUB_TOKEN | | GitHub API token |
107
+ | GITHUB_USERNAME | | GitHub username |
108
+ | LINKWARDEN_URL | | Linkwarden URL <https://your-linkwarden-instance.com> |
109
+ | LINKWARDEN_TOKEN | | Linkwarden API token |
110
+ | COLLECTION_ID | | Linkwarden Collection ID to update (Number) /collections/499 |
111
+ | CRON_SCHEDULE | 0 6 \* \* \* | Cron Schedule (default is daily at 6am) |
112
+ | APPRISE_URLS | | Apprise URL for push notifications, separated by commas, see [Apprise](https://github.com/caronc/apprise/wiki) |
113
+ | OPT_TAG | true | Enable/Disable all Tagging |
114
+ | OPT_TAG_GITHUB | true | Tag Link with "GitHub" |
115
+ | OPT_TAG_GITHUBSTARS | true | Tag Link with "GitHub Stars" |
116
+ | OPT_TAG_LANGUAGE | false | Tag Link with Language of repo (e.g. Python or JavaScript) |
117
+ | OPT_TAG_USERNAME | false | Tag GitHub username |
118
+ | OPT_TAG_CUSTOM | | Add custom tags, separated by commas (e.g. tag1,tag2) |
119
+
120
+ ## Unsupervised Updates
121
+
122
+ For automated, unsupervised updates, you can use Docker with the provided docker-compose.yml file.
123
+
124
+ 1. Make sure you have Docker and Docker Compose installed on your system.
125
+
126
+ 2. Create a `.env` file in the project root with the following variables:
127
+
128
+ ```bash
129
+ GITHUB_TOKEN=your_github_token
130
+ GITHUB_USERNAME=your_github_username
131
+ LINKWARDEN_URL=your_linkwarden_instance_url
132
+ LINKWARDEN_TOKEN=your_linkwarden_api_token
133
+ COLLECTION_ID=your_linkwarden_collection_id
134
+ CRON_SCHEDULE=0 0 * * * # Run daily at midnight, adjust as needed
135
+ APPRISE_URLS=apprise_url_1,apprise_url_2
136
+ OPT_TAG=true
137
+ OPT_TAG_GITHUB=true
138
+ OPT_TAG_GITHUBSTARS=true
139
+ OPT_TAG_LANGUAGE=false
140
+ OPT_TAG_USERNAME=false
141
+ ```
142
+
143
+ 3. Use the following `docker-compose.yml` file:
144
+
145
+ ```yaml
146
+ version: "3"
147
+ services:
148
+ starwarden:
149
+ image: rtuszik/starwarden:latest
150
+ env_file: .env
151
+ volumes:
152
+ - ./starwarden.log:/app/starwarden.log
153
+ ```
154
+
155
+ 4. Run the following command to start the Docker container:
156
+
157
+ ```bash
158
+ docker compose up -d
159
+ ```
160
+
161
+ The container will now automatically run StarWarden on the specified schedule without any manual intervention.
162
+
163
+ To manually trigger an update, you can run:
164
+
165
+ ```bash
166
+ docker compose exec starwarden python /app/starwarden.py -id $COLLECTION_ID
167
+ ```
168
+
169
+ ## Contributing
170
+
171
+ Contributions are welcome! Please feel free to submit a Pull Request.
172
+
173
+ ## License
174
+
175
+ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
@@ -0,0 +1,100 @@
1
+ [project]
2
+ name = "starwarden"
3
+ version = "1.5.0"
4
+ description = "Sync GitHub Stars to Linkwarden"
5
+ readme = "README.md"
6
+ requires-python = ">=3.13"
7
+ dependencies = [
8
+ "apprise==2.2.0",
9
+ "pygithub==2.8.1",
10
+ "python-dotenv==1.2.4",
11
+ "requests==2.33.0",
12
+ "rich==15.0.0",
13
+ "tqdm==4.70.1",
14
+ ]
15
+
16
+ [project.scripts]
17
+ starwarden = "starwarden.main:main"
18
+
19
+ [dependency-groups]
20
+ dev = [
21
+ "pytest>=9.0.2",
22
+ "ruff>=0.12.12",
23
+ "ty>=0.0.1a20",
24
+ "vulture>=2.14",
25
+ ]
26
+
27
+ [build-system]
28
+ requires = ["uv_build>=0.9.16"]
29
+ build-backend = "uv_build"
30
+
31
+ [tool.pytest.ini_options]
32
+ testpaths = ["tests"]
33
+
34
+ [tool.ruff]
35
+ indent-width = 4
36
+ line-length = 120
37
+
38
+ [tool.ruff.lint]
39
+ ignore = [
40
+ "ANN",
41
+ "COM",
42
+ "C90",
43
+ "DJ",
44
+ "EXE",
45
+ "BLE",
46
+ "PTH",
47
+ "T10",
48
+ "TID",
49
+ "D100",
50
+ "D101",
51
+ "D102",
52
+ "D103",
53
+ "D104",
54
+ "D105",
55
+ "D106",
56
+ "D107",
57
+ "D101",
58
+ "D107",
59
+ "D102",
60
+ "D104",
61
+ "D213",
62
+ "D203",
63
+ "D400",
64
+ "D415",
65
+ "G004",
66
+ "PLR2004",
67
+ "E501",
68
+ "TRY",
69
+ "SIM105",
70
+ ]
71
+ extend-select = [
72
+ "B",
73
+ "S",
74
+ "SIM",
75
+ "T20",
76
+ "C901",
77
+ ]
78
+ fixable = ["ALL"]
79
+ unfixable = []
80
+ dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
81
+
82
+ [tool.ruff.lint.per-file-ignores]
83
+ "tests/**/*.py" = ["S101"]
84
+
85
+ [tool.ruff.format]
86
+ quote-style = "double"
87
+ indent-style = "space"
88
+ line-ending = "auto"
89
+ docstring-code-format = false
90
+
91
+ [tool.uv.build-backend]
92
+ module-name = "starwarden"
93
+ module-root = ""
94
+
95
+ [tool.vulture]
96
+ min_confidence = 100
97
+ exclude = [
98
+ ".venv",
99
+ "tests",
100
+ ]
@@ -0,0 +1,96 @@
1
+ [project]
2
+ name = "starwarden"
3
+ version = "1.5.0"
4
+ description = "Sync GitHub Stars to Linkwarden"
5
+ readme = "README.md"
6
+ requires-python = ">=3.13"
7
+ dependencies = [
8
+ "apprise==2.2.0",
9
+ "pygithub==2.8.1",
10
+ "python-dotenv==1.2.4",
11
+ "requests==2.33.0",
12
+ "rich==15.0.0",
13
+ "tqdm==4.70.1",
14
+ ]
15
+
16
+ [project.scripts]
17
+ starwarden = "starwarden.main:main"
18
+
19
+ [dependency-groups]
20
+ dev = [
21
+ "pytest>=9.0.2",
22
+ "ruff>=0.12.12",
23
+ "ty>=0.0.1a20",
24
+ "vulture>=2.14",
25
+ ]
26
+
27
+ [build-system]
28
+ requires = ["uv_build>=0.9.16"]
29
+ build-backend = "uv_build"
30
+
31
+ [tool.pytest.ini_options]
32
+ testpaths = ["tests"]
33
+
34
+ [tool.ruff]
35
+ indent-width = 4
36
+ line-length = 120
37
+
38
+ [tool.ruff.lint]
39
+ ignore = [
40
+ "ANN", # flake8-annotations
41
+ "COM", # flake8-commas
42
+ "C90", # mccabe complexity
43
+ "DJ", # django
44
+ "EXE", # flake8-executable
45
+ "BLE", # blind except
46
+ "PTH", # flake8-pathlib
47
+ "T10", # debugger
48
+ "TID", # flake8-tidy-imports
49
+ "D100",
50
+ "D101",
51
+ "D102",
52
+ "D103",
53
+ "D104",
54
+ "D105",
55
+ "D106",
56
+ "D107",
57
+ "D101",
58
+ "D107", # missing docstring in public module
59
+ "D102", # missing docstring in public class
60
+ "D104", # missing docstring in public package
61
+ "D213",
62
+ "D203",
63
+ "D400",
64
+ "D415",
65
+ "G004",
66
+ "PLR2004",
67
+ "E501", # line too long
68
+ "TRY",
69
+ "SIM105", # faster without contextlib
70
+ ]
71
+
72
+ extend-select = ["B", "S", "SIM", "T20", "C901"]
73
+
74
+ fixable = ["ALL"]
75
+ unfixable = []
76
+
77
+ # Allow unused variables when underscore-prefixed.
78
+ dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
79
+
80
+ [tool.ruff.lint.per-file-ignores]
81
+ "tests/**/*.py" = ["S101"]
82
+
83
+ [tool.ruff.format]
84
+ quote-style = "double"
85
+ indent-style = "space"
86
+ line-ending = "auto"
87
+
88
+ docstring-code-format = false
89
+
90
+ [tool.uv.build-backend]
91
+ module-name = "starwarden"
92
+ module-root = ""
93
+
94
+ [tool.vulture]
95
+ min_confidence = 100
96
+ exclude = [".venv", "tests"]
File without changes
@@ -0,0 +1,45 @@
1
+ import argparse
2
+ import os
3
+ import sys
4
+
5
+ from dotenv import find_dotenv, load_dotenv
6
+
7
+ from starwarden.utils.logger import get_logger
8
+
9
+ logger = get_logger()
10
+
11
+
12
+ def parse_args():
13
+ parser = argparse.ArgumentParser(description="Export GitHub starred repositories as individual links to Linkwarden")
14
+ parser.add_argument("-id", type=int, help="Specify the collection ID to sync")
15
+ parser.add_argument(
16
+ "-d",
17
+ "--debug",
18
+ action="store_true",
19
+ help="Enable debug logging",
20
+ )
21
+ return parser.parse_args()
22
+
23
+
24
+ def load_env():
25
+ load_dotenv(find_dotenv(usecwd=True))
26
+ config = {
27
+ "github_token": os.getenv("GITHUB_TOKEN"),
28
+ "github_username": os.getenv("GITHUB_USERNAME"),
29
+ "linkwarden_url": os.getenv("LINKWARDEN_URL"),
30
+ "linkwarden_token": os.getenv("LINKWARDEN_TOKEN"),
31
+ "opt_tag": os.getenv("OPT_TAG", "false").lower() in ("true", "1"),
32
+ "opt_tag_github": os.getenv("OPT_TAG_GITHUB", "false").lower() in ("true", "1"),
33
+ "opt_tag_githubStars": os.getenv("OPT_TAG_GITHUBSTARS", "false").lower() in ("true", "1"),
34
+ "opt_tag_language": os.getenv("OPT_TAG_LANGUAGE", "false").lower() in ("true", "1"),
35
+ "opt_tag_username": os.getenv("OPT_TAG_USERNAME", "false").lower() in ("true", "1"),
36
+ "opt_tag_custom": os.getenv("OPT_TAG_CUSTOM", ""),
37
+ "APPRISE_URLS": os.getenv("APPRISE_URLS"),
38
+ "DOCKERIZED": os.getenv("DOCKERIZED", "false").lower() in ("true", "1"),
39
+ }
40
+
41
+ if not all([config["github_username"], config["linkwarden_url"], config["linkwarden_token"]]):
42
+ logger.error("Missing required environment variables. Please check your .env file.")
43
+ sys.exit(1)
44
+
45
+ return config
@@ -0,0 +1,50 @@
1
+ import time
2
+ from math import ceil
3
+
4
+ from github import Github, GithubException, RateLimitExceededException
5
+ from urllib3 import Retry
6
+
7
+ from starwarden.utils.logger import get_logger
8
+
9
+ logger = get_logger()
10
+
11
+
12
+ def config_retry(backoff_factor=1.0, total=8):
13
+ Retry.DEFAULT_BACKOFF_MAX = backoff_factor * 2 ** (total - 1)
14
+ return Retry(total=total, backoff_factor=backoff_factor)
15
+
16
+
17
+ def get_starred_repos(github_token, github_username):
18
+ gh = Github(github_token, retry=config_retry()) if github_token else Github(retry=config_retry())
19
+ user = gh.get_user(github_username)
20
+ starred = user.get_starred()
21
+ total_pages = ceil(starred.totalCount / 30)
22
+
23
+ for page_num in range(total_pages):
24
+ while True:
25
+ try:
26
+ yield from starred.get_page(page_num)
27
+ break
28
+ except RateLimitExceededException as e:
29
+ handle_rate_limit(e)
30
+ except GithubException as e:
31
+ if e.status == 403 and "rate limit" in str(e).lower():
32
+ handle_rate_limit(e)
33
+ else:
34
+ logger.error(f"GitHub API error: {str(e)}")
35
+ raise
36
+
37
+
38
+ def handle_rate_limit(e, retry_after=None):
39
+ if retry_after is None:
40
+ retry_after = e.headers.get("Retry-After", 60)
41
+
42
+ retry_after = int(retry_after)
43
+ logger.warning(f"Rate limit exceeded. Waiting for {retry_after} seconds before retrying.")
44
+ time.sleep(retry_after)
45
+
46
+
47
+ def get_total_starred_repos(github_token, github_username):
48
+ gh = Github(github_token, retry=config_retry()) if github_token else Github(retry=config_retry())
49
+ user = gh.get_user(github_username)
50
+ return user.get_starred().totalCount
@@ -0,0 +1,187 @@
1
+ import json
2
+
3
+ import requests
4
+ from requests.adapters import HTTPAdapter
5
+ from requests.exceptions import Timeout
6
+ from urllib3.util.retry import Retry
7
+
8
+ from starwarden.utils.logger import get_logger
9
+
10
+ logger = get_logger()
11
+
12
+ LINK_EXISTS_GLOBALLY = object()
13
+
14
+
15
+ class ExistingLinksError(Exception):
16
+ """Existing links could not be read completely."""
17
+
18
+
19
+ def _retrying_session():
20
+ session = requests.Session()
21
+ retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504], allowed_methods=["GET"])
22
+ session.mount("http://", HTTPAdapter(max_retries=retry))
23
+ session.mount("https://", HTTPAdapter(max_retries=retry))
24
+ return session
25
+
26
+
27
+ def get_existing_links(linkwarden_url, linkwarden_token, collection_id):
28
+ url = f"{linkwarden_url.rstrip('/')}/api/v1/links"
29
+ headers = {
30
+ "Authorization": f"Bearer {linkwarden_token}",
31
+ "Content-Type": "application/json",
32
+ }
33
+ cursor = 0
34
+ seen_links = set()
35
+ with _retrying_session() as session:
36
+ while True:
37
+ try:
38
+ logger.debug(f"Fetching links from cursor {cursor} for collection {collection_id}")
39
+ response = session.get(
40
+ url,
41
+ params={"collectionId": collection_id, "cursor": cursor, "sort": 1},
42
+ headers=headers,
43
+ timeout=30,
44
+ )
45
+ response.raise_for_status()
46
+ data = response.json()
47
+ except requests.RequestException as e:
48
+ logger.error(f"Error fetching links from cursor {cursor}: {e!s}")
49
+ if e.response is not None:
50
+ logger.error(f"Response status code: {e.response.status_code}")
51
+ logger.error(f"Response content: {e.response.text}")
52
+ raise ExistingLinksError(f"Could not fetch existing links from cursor {cursor}") from e
53
+ except ValueError as e:
54
+ logger.error(f"Invalid JSON from cursor {cursor}: {e!s}")
55
+ raise ExistingLinksError(f"Invalid response from cursor {cursor}") from e
56
+
57
+ links = data.get("response") if isinstance(data, dict) else None
58
+ if not isinstance(links, list) or not all(isinstance(link, dict) and "url" in link for link in links):
59
+ raise ExistingLinksError(f"Unexpected response shape from cursor {cursor}")
60
+ logger.debug(f"Fetched {len(links)} links from cursor {cursor}")
61
+
62
+ # empty page = end; duplicate-only page != end
63
+ if not links:
64
+ break
65
+
66
+ for link in links:
67
+ if link["url"] not in seen_links:
68
+ seen_links.add(link["url"])
69
+ yield link["url"]
70
+
71
+ # sort=1 is id asc, so cursor must grow
72
+ next_cursor = links[-1].get("id")
73
+ if not isinstance(next_cursor, int) or next_cursor <= cursor:
74
+ raise ExistingLinksError(f"Pagination did not advance past cursor {cursor}")
75
+ cursor = next_cursor
76
+
77
+
78
+ def get_collections(linkwarden_url, linkwarden_token):
79
+ url = f"{linkwarden_url.rstrip('/')}/api/v1/collections"
80
+ headers = {
81
+ "Authorization": f"Bearer {linkwarden_token}",
82
+ "Content-Type": "application/json",
83
+ }
84
+ try:
85
+ response = requests.get(url, headers=headers, timeout=30)
86
+ response.raise_for_status()
87
+ data = response.json()
88
+ collections = data.get("response", [])
89
+ logger.debug(f"Fetched collections: {json.dumps(collections, indent=2)}")
90
+ return collections
91
+ except requests.RequestException as e:
92
+ raise Exception(f"Error fetching collections from Linkwarden: {e!s}") from e
93
+
94
+
95
+ def create_collection(linkwarden_url, linkwarden_token, name, description=""):
96
+ url = f"{linkwarden_url.rstrip('/')}/api/v1/collections"
97
+ headers = {
98
+ "Authorization": f"Bearer {linkwarden_token}",
99
+ "Content-Type": "application/json",
100
+ }
101
+ data = {"name": name, "description": description}
102
+ try:
103
+ logger.debug(f"Attempting to create collection: {name}")
104
+ response = requests.post(
105
+ url,
106
+ headers=headers,
107
+ json=data,
108
+ timeout=30,
109
+ )
110
+ logger.debug(f"Response status code: {response.status_code}")
111
+ logger.debug(f"Response content: {response.text}")
112
+ response.raise_for_status()
113
+ created_collection = response.json().get("response", {})
114
+ logger.info(f"Created collection: {created_collection}")
115
+ return created_collection
116
+ except Timeout:
117
+ logger.error("Request timed out while creating new collection in Linkwarden")
118
+ return None
119
+ except requests.RequestException as e:
120
+ logger.error(f"Error creating new collection in Linkwarden: {e!s}")
121
+ if hasattr(e, "response") and e.response is not None:
122
+ logger.error(f"Response status code: {e.response.status_code}")
123
+ logger.error(f"Response content: {e.response.text}")
124
+ return None
125
+
126
+
127
+ def upload_link(linkwarden_url, linkwarden_token, collection_id, repo, tags):
128
+ description = repo.description or ""
129
+ if len(description) > 2048:
130
+ description = f"{description[:2045]}..."
131
+ link_data = {
132
+ "url": repo.html_url,
133
+ "title": repo.full_name,
134
+ "description": description,
135
+ "collection": {"id": collection_id},
136
+ }
137
+ if tags:
138
+ link_data["tags"] = tags
139
+
140
+ url = f"{linkwarden_url.rstrip('/')}/api/v1/links"
141
+ headers = {
142
+ "Authorization": f"Bearer {linkwarden_token}",
143
+ "Content-Type": "application/json",
144
+ }
145
+ logger.debug(f"Sending link data to Linkwarden: {json.dumps(link_data, indent=2)}")
146
+
147
+ try:
148
+ response = requests.post(
149
+ url,
150
+ headers=headers,
151
+ json=link_data,
152
+ timeout=30,
153
+ )
154
+ # Check for 409 Conflict
155
+ if response.status_code == 409:
156
+ response_json = response.json()
157
+ if response_json.get("response") == "Link already exists":
158
+ logger.info(f"Link for {repo.full_name} already exists in Linkwarden (status 409).")
159
+ return LINK_EXISTS_GLOBALLY
160
+
161
+ response.raise_for_status()
162
+ response_json = response.json()
163
+
164
+ logger.debug(f"API response for {repo.full_name}: {json.dumps(response_json, indent=2)}")
165
+
166
+ created_link = response_json.get("response", {})
167
+ if created_link and "id" in created_link:
168
+ returned_collection_id = created_link.get("collectionId")
169
+ if returned_collection_id != collection_id:
170
+ logger.warning(
171
+ f"Link created in collection {returned_collection_id} instead of requested collection {collection_id}"
172
+ )
173
+ logger.info(
174
+ f"Successfully created link for {repo.full_name}. Link ID: {created_link['id']}, Collection ID: {returned_collection_id}"
175
+ )
176
+ return created_link["id"]
177
+ else:
178
+ logger.error(f"Unexpected response format for {repo.full_name}. Response: {response_json}")
179
+ return None
180
+
181
+ except requests.RequestException as e:
182
+ logger.error(f"Error uploading {repo.full_name} to Linkwarden: {e!s}")
183
+ if hasattr(e, "response") and e.response is not None:
184
+ logger.error(f"Response status code: {e.response.status_code}")
185
+ logger.error(f"Response content: {e.response.text}")
186
+
187
+ return None
@@ -0,0 +1,185 @@
1
+ import sys
2
+ from http.client import HTTPConnection
3
+
4
+ import requests
5
+ from rich.progress import Progress
6
+
7
+ from starwarden import config, github_api, linkwarden_api, tui
8
+ from starwarden.linkwarden_api import LINK_EXISTS_GLOBALLY
9
+ from starwarden.utils import notify
10
+ from starwarden.utils.logger import get_logger
11
+
12
+ logger = get_logger()
13
+
14
+
15
+ def build_tags(config_data, repo):
16
+ if not config_data["opt_tag"]:
17
+ return []
18
+
19
+ tags = []
20
+
21
+ if config_data["opt_tag_github"]:
22
+ tags.append({"name": "GitHub"})
23
+ if config_data["opt_tag_githubStars"]:
24
+ tags.append({"name": "GitHub Stars"})
25
+ if config_data["opt_tag_language"] and repo.language:
26
+ tags.append({"name": repo.language})
27
+ if config_data["opt_tag_username"]:
28
+ tags.append({"name": config_data["github_username"]})
29
+ if config_data["opt_tag_custom"]:
30
+ for tag in config_data["opt_tag_custom"].split(","):
31
+ if len(tag) > 0:
32
+ tags.append({"name": tag.strip()})
33
+
34
+ return tags
35
+
36
+
37
+ def run_update(config_data, collection_id):
38
+ logger.debug(f"Collection ID: {collection_id}")
39
+ try:
40
+ existing_links = set(
41
+ linkwarden_api.get_existing_links(
42
+ config_data["linkwarden_url"], config_data["linkwarden_token"], collection_id
43
+ )
44
+ )
45
+ except linkwarden_api.ExistingLinksError as e:
46
+ # incomplete list would re-upload missing links
47
+ message = f"{e}. Skipping this run to avoid creating duplicates."
48
+ logger.error(message)
49
+ tui.console.print(message, style="danger")
50
+ notify.send_notification(config_data, message=message, title="Starwarden Run Skipped")
51
+ return
52
+ logger.info(f"Found {len(existing_links)} existing links in the collection.")
53
+ tui.console.print(
54
+ f"Found {len(existing_links)} existing links in the collection.",
55
+ style="info",
56
+ )
57
+
58
+ logger.info("Fetching starred repositories from GitHub...")
59
+ tui.console.print("Fetching starred repositories from GitHub...", style="info")
60
+ total_repos = github_api.get_total_starred_repos(config_data["github_token"], config_data["github_username"])
61
+ logger.info(f"Total starred repositories: {total_repos}")
62
+ tui.console.print(f"Total starred repositories: {total_repos}", style="info")
63
+
64
+ successful_uploads = 0
65
+ failed_uploads = 0
66
+ skipped_uploads = 0
67
+
68
+ with Progress() as progress:
69
+ task = progress.add_task("Processing starred repos...", total=total_repos, style="info")
70
+ for repo in github_api.get_starred_repos(config_data["github_token"], config_data["github_username"]):
71
+ if repo.html_url in existing_links:
72
+ logger.info(f"Skipping {repo.full_name} as it already exists in the collection")
73
+ skipped_uploads += 1
74
+ progress.update(task, advance=1)
75
+ continue
76
+
77
+ max_retries = 3
78
+ retry_count = 0
79
+ tags = build_tags(config_data, repo)
80
+
81
+ while retry_count < max_retries:
82
+ try:
83
+ logger.info(f"Processing repository: {repo.full_name}")
84
+ link_id = linkwarden_api.upload_link(
85
+ config_data["linkwarden_url"], config_data["linkwarden_token"], collection_id, repo, tags
86
+ )
87
+
88
+ if link_id is LINK_EXISTS_GLOBALLY:
89
+ logger.info(f"Skipping {repo.full_name} as it already exists globally in Linkwarden.")
90
+ skipped_uploads += 1
91
+ elif link_id:
92
+ logger.info(f"Successfully processed {repo.full_name}. Link ID: {link_id}")
93
+ successful_uploads += 1
94
+ else:
95
+ logger.warning(f"Failed to upload {repo.full_name}")
96
+ failed_uploads += 1
97
+ break
98
+ except requests.exceptions.RequestException as e:
99
+ if e.response is not None and e.response.status_code == 429:
100
+ retry_after = e.response.headers.get("Retry-After", 60)
101
+ github_api.handle_rate_limit(e, retry_after)
102
+ retry_count += 1
103
+ else:
104
+ logger.error(f"Error uploading {repo.full_name} to Linkwarden: {e!s}")
105
+ failed_uploads += 1
106
+ break
107
+ except Exception as e:
108
+ logger.error(f"Unexpected error processing {repo.full_name}: {e!s}")
109
+ failed_uploads += 1
110
+ break
111
+ else:
112
+ logger.error(f"Max retries reached for {repo.full_name}. Moving to next repository.")
113
+ failed_uploads += 1
114
+
115
+ progress.update(task, advance=1)
116
+
117
+ logger.info("Finished processing all starred repositories.")
118
+ logger.info(f"Successful uploads: {successful_uploads}")
119
+ logger.info(f"Failed uploads: {failed_uploads}")
120
+ logger.info(f"Skipped uploads: {skipped_uploads}")
121
+
122
+ tui.console.print("\nFinished processing all starred repositories.", style="info")
123
+ tui.console.print(f"Successful uploads: {successful_uploads}", style="info")
124
+ tui.console.print(f"Failed uploads: {failed_uploads}", style="warning")
125
+ tui.console.print(f"Skipped uploads: {skipped_uploads}", style="info")
126
+ notify.send_notification(
127
+ config_data,
128
+ message=f"Successful uploads: {successful_uploads}, failed uploads: {failed_uploads}, skipped_uploads: {skipped_uploads}",
129
+ title="Starwarden Status",
130
+ )
131
+
132
+
133
+ def main():
134
+ args = config.parse_args()
135
+ config_data = config.load_env()
136
+ if args.debug:
137
+ HTTPConnection.debuglevel = 1
138
+
139
+ if not args.id:
140
+ tui.display_welcome()
141
+ flavor = tui.main_menu()
142
+ if flavor == 1:
143
+ tui.console.print("Fetching existing collections...", style="info")
144
+ collections = linkwarden_api.get_collections(config_data["linkwarden_url"], config_data["linkwarden_token"])
145
+ collection_id = tui.select_collection(collections)
146
+ if not collection_id:
147
+ logger.error("Failed to select a collection. Exiting.")
148
+ tui.console.print("Failed to select a collection. Exiting.", style="danger")
149
+ return
150
+ run_update(config_data, collection_id)
151
+
152
+ elif flavor == 2:
153
+ collection_name = tui.create_collection_prompt()
154
+ logger.info(f"Creating new collection: {collection_name}")
155
+ tui.console.print(f"Creating new collection: {collection_name}", style="info")
156
+ collection = linkwarden_api.create_collection(
157
+ config_data["linkwarden_url"], config_data["linkwarden_token"], collection_name
158
+ )
159
+ if not collection:
160
+ logger.error("Failed to create a new collection. Exiting.")
161
+ tui.console.print("Failed to create a new collection. Exiting.", style="danger")
162
+ return
163
+ collection_id = collection.get("id")
164
+ logger.info(f"Created new collection with ID: {collection_id}")
165
+ tui.console.print(f"Created new collection with ID: {collection_id}", style="info")
166
+ run_update(config_data, collection_id)
167
+ else:
168
+ logger.info("Exiting StarWarden.")
169
+ tui.console.print("Exiting StarWarden. Goodbye!", style="info")
170
+ return
171
+ else:
172
+ run_update(config_data, args.id)
173
+
174
+
175
+ if __name__ == "__main__":
176
+ config_data = config.load_env()
177
+ try:
178
+ main()
179
+ except Exception as e:
180
+ tui.console.print_exception(show_locals=True)
181
+ logger.exception(f"Unexpected error: {e!s}")
182
+ tui.console.print(f"Unexpected error: {e!s}", style="danger")
183
+ error_message = f"Starwarden process failed with a critical error: {e}"
184
+ notify.send_notification(config_data, message=error_message, title="Starwarden Process Failure")
185
+ sys.exit(1)
@@ -0,0 +1,93 @@
1
+ from rich import box
2
+ from rich.console import Console
3
+ from rich.panel import Panel
4
+ from rich.prompt import Prompt
5
+ from rich.table import Column, Table
6
+ from rich.theme import Theme
7
+
8
+ theme = Theme({"info": "#74c7ec", "prompt": "#cba6f7", "warning": "#fab387", "danger": "#f38ba8"})
9
+ console = Console(theme=theme)
10
+
11
+
12
+ def display_welcome():
13
+ welcome_text = r"""
14
+ _______.___________. ___ .______ ____ __ ____ ___ .______ _______ _______ .__ __.
15
+ / | | / \ | _ \ \ \ / \ / / / \ | _ \ | \ | ____|| \ | |
16
+ | (----`---| |----` / ^ \ | |_) | \ \/ \/ / / ^ \ | |_) | | .--. || |__ | \| |
17
+ \ \ | | / /_\ \ | / \ / / /_\ \ | / | | | || __| | . ` |
18
+ .----) | | | / _____ \ | |\ \----. \ /\ / / _____ \ | |\ \----.| '--' || |____ | |\ |
19
+ |_______/ |__| /__/ \__\ | _| `._____| \__/ \__/ /__/ \__\ | _| `._____||_______/ |_______||__| \__|
20
+
21
+ """
22
+ console.print(
23
+ Panel(welcome_text, expand=True, border_style="#fab387"),
24
+ justify="center",
25
+ style="info",
26
+ )
27
+
28
+
29
+ def main_menu():
30
+ options = [
31
+ "Update existing GitHub Stars collection",
32
+ "Create a new GitHub Stars collection",
33
+ "Exit",
34
+ ]
35
+ console.print("Choose an option:", style="info")
36
+ for i, option in enumerate(options, 1):
37
+ console.print(f" {i}. {option}", style="info")
38
+
39
+ choice = Prompt.ask("Enter your choice", choices=[str(i) for i in range(1, len(options) + 1)])
40
+ return int(choice)
41
+
42
+
43
+ def select_collection(collections):
44
+ if collections is None or not collections:
45
+ console.print("No collections found or failed to fetch collections.", style="danger")
46
+ return None
47
+
48
+ console.print("Available collections:", style="info")
49
+ ids = []
50
+ collection_table = Table(
51
+ Column(
52
+ header="Collection ID",
53
+ justify="center",
54
+ header_style="#fab387",
55
+ style="#fab387",
56
+ ),
57
+ Column(
58
+ header="Name",
59
+ justify="left",
60
+ header_style="#eba0ac",
61
+ style="#eba0ac",
62
+ no_wrap=True,
63
+ ),
64
+ Column(
65
+ header="Number of Links",
66
+ justify="center",
67
+ header_style="#cba6f7",
68
+ style="#cba6f7",
69
+ ),
70
+ title="",
71
+ box=box.SIMPLE_HEAD,
72
+ )
73
+
74
+ for _i, collection in enumerate(collections, 1):
75
+ collection_table.add_row(
76
+ f"{collection.get('id')}",
77
+ f"{collection.get('name')}",
78
+ f"{collection.get('_count', {}).get('links', 'No. of links unknown')}",
79
+ )
80
+ ids.append(str(collection.get("id")))
81
+
82
+ console.print(collection_table)
83
+
84
+ choice = Prompt.ask(
85
+ "Enter Collection ID: ",
86
+ choices=ids,
87
+ show_choices=False,
88
+ )
89
+ return int(choice)
90
+
91
+
92
+ def create_collection_prompt():
93
+ return Prompt.ask("Enter the name for the new GitHub Stars collection")
File without changes
@@ -0,0 +1,75 @@
1
+ import logging
2
+ import os
3
+ from logging.handlers import RotatingFileHandler
4
+ from typing import Optional, Union
5
+
6
+ from dotenv import find_dotenv, load_dotenv
7
+ from rich.logging import RichHandler
8
+
9
+ load_dotenv(find_dotenv(usecwd=True))
10
+ LogLevel = Union[int, str]
11
+
12
+ LOGGER_NAME = os.getenv("LOGGER_NAME", "app")
13
+ LOG_FILE = os.getenv("LOG_FILE", "starwarden.log")
14
+ ENABLE_CONSOLE_LOGGING = os.getenv("ENABLE_CONSOLE_LOGGING", "true").lower() in ("true", "1", "t")
15
+ CONSOLE_LEVEL = os.getenv("CONSOLE_LEVEL", "WARNING")
16
+ FILE_LEVEL = os.getenv("FILE_LEVEL", "INFO")
17
+ LOG_FORMAT = os.getenv("LOG_FORMAT", "%(asctime)s - %(name)s - %(levelname)s - %(message)s")
18
+ MAX_BYTES = int(os.getenv("MAX_BYTES", 10 * 1024 * 1024)) # 10MB
19
+ BACKUP_COUNT = int(os.getenv("BACKUP_COUNT", 2))
20
+
21
+ _logger: Optional[logging.Logger] = None
22
+
23
+
24
+ def setup_logging(
25
+ *, # Enforce keyword-only arguments
26
+ logger_name: str = LOGGER_NAME,
27
+ log_file: str = LOG_FILE,
28
+ enable_console_logging: bool = ENABLE_CONSOLE_LOGGING,
29
+ console_level: LogLevel = CONSOLE_LEVEL,
30
+ file_level: LogLevel = FILE_LEVEL,
31
+ log_format: str = LOG_FORMAT,
32
+ max_bytes: int = MAX_BYTES,
33
+ backup_count: int = BACKUP_COUNT,
34
+ ) -> logging.Logger:
35
+ global _logger
36
+ if _logger is not None:
37
+ return _logger
38
+
39
+ logger = logging.getLogger(logger_name)
40
+ logger.setLevel(logging.DEBUG)
41
+ formatter = logging.Formatter(log_format)
42
+
43
+ # Console handler
44
+ if enable_console_logging and not any(isinstance(h, RichHandler) for h in logger.handlers):
45
+ console_handler = RichHandler(rich_tracebacks=True)
46
+ console_handler.setLevel(console_level)
47
+ logger.addHandler(console_handler)
48
+
49
+ # Rotating File handler
50
+ try:
51
+ if not any(
52
+ isinstance(h, RotatingFileHandler) and getattr(h, "baseFilename", None) == os.path.abspath(log_file)
53
+ for h in logger.handlers
54
+ ):
55
+ file_handler = RotatingFileHandler(log_file, maxBytes=max_bytes, backupCount=backup_count, encoding="utf-8")
56
+ file_handler.setLevel(file_level)
57
+ file_handler.setFormatter(formatter)
58
+ logger.addHandler(file_handler)
59
+ except (PermissionError, OSError) as e:
60
+ import sys
61
+
62
+ # stderr as fallback
63
+ sys.stderr.write(f"Failed to create log file handler for '{log_file}': {e}\n")
64
+ logger.warning(f"Failed to create log file handler for '{log_file}': {e}", exc_info=False)
65
+
66
+ logger.propagate = False
67
+ _logger = logger
68
+ return logger
69
+
70
+
71
+ def get_logger() -> logging.Logger:
72
+ global _logger
73
+ if _logger is None:
74
+ _logger = setup_logging()
75
+ return _logger
@@ -0,0 +1,27 @@
1
+ import apprise
2
+
3
+ from starwarden.utils.logger import get_logger
4
+
5
+ logger = get_logger()
6
+
7
+
8
+ def send_notification(config: dict, message: str, title: str = "Starwarden Notification"):
9
+ apprise_urls = config.get("APPRISE_URLS")
10
+ if not apprise_urls:
11
+ logger.info("No APPRISE_URLS set, skipping notification.")
12
+ return
13
+
14
+ apobj = apprise.Apprise()
15
+
16
+ for url in apprise_urls.split(","):
17
+ if url.strip():
18
+ apobj.add(url.strip())
19
+
20
+ if len(apobj) == 0:
21
+ logger.warning("No valid Apprise URLs were found after processing the APPRISE_URLS variable.")
22
+ return
23
+
24
+ if not apobj.notify(body=message, title=title):
25
+ logger.error("Failed to send notification to one or more Apprise targets.")
26
+ else:
27
+ logger.info("Successfully sent notification.")