PgQueuer 0.3.2__tar.gz → 0.3.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pgqueuer-0.3.4/CONTRIBUTING.md +49 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/PKG-INFO +26 -5
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/README.md +25 -4
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/cli.py +9 -10
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/logconfig.py +2 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/models.py +2 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/qm.py +4 -4
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/queries.py +3 -1
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer.egg-info/PKG-INFO +26 -5
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer.egg-info/SOURCES.txt +1 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/_version.py +2 -2
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/.github/workflows/ci.yml +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/.github/workflows/linting.yml +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/.github/workflows/release.yml +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/.gitignore +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/LICENSE +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/docker-compose.yml +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/pyproject.toml +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/setup.cfg +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/__init__.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/__main__.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/_version.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/py.typed +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer/tm.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer.egg-info/dependency_links.txt +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer.egg-info/requires.txt +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/src/PgQueuer.egg-info/top_level.txt +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/conftest.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/db/Dockerfile +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/db/init_db.sh +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/test_qm.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/test_queries.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/test/test_tm.py +0 -0
- {pgqueuer-0.3.2 → pgqueuer-0.3.4}/tools/benchmark.py +0 -0
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Contributing to PgQueuer
|
|
2
|
+
|
|
3
|
+
First off, thank you for considering contributing to PgQueuer.
|
|
4
|
+
|
|
5
|
+
## How to Contribute
|
|
6
|
+
|
|
7
|
+
There are many ways you can contribute to PgQueuer, from writing tutorials or blog posts, improving the documentation, submitting bug reports and feature requests, or writing code which can be incorporated into PgQueuer itself.
|
|
8
|
+
|
|
9
|
+
Below you will find a set of guidelines to follow when contributing. These are mostly guidelines, not rules, so use your best judgment and feel free to propose changes to this document.
|
|
10
|
+
|
|
11
|
+
### Reporting Bugs
|
|
12
|
+
|
|
13
|
+
Before submitting bug reports, please check the issue list to avoid duplicating issues. When you are creating a bug report, please include as many details as possible by filling out the provided template. The more details you provide, the easier it will be for others to resolve the issue.
|
|
14
|
+
|
|
15
|
+
### Feature Requests
|
|
16
|
+
|
|
17
|
+
Feature requests are welcome. But take a moment to find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Please provide as much detail and context as possible, which includes explaining why you're requesting the feature.
|
|
18
|
+
|
|
19
|
+
### Pull Requests
|
|
20
|
+
|
|
21
|
+
Please follow these steps to have your contribution considered by the maintainers:
|
|
22
|
+
|
|
23
|
+
1. Follow all instructions in the template.
|
|
24
|
+
2. Follow the [style guides](#style-guides) for Python.
|
|
25
|
+
3. After you submit your pull request, verify that all status checks are passing.
|
|
26
|
+
|
|
27
|
+
While the prerequisites above must be satisfied prior to having your pull request reviewed, the reviewer(s) may ask you to complete additional design work, tests, or other changes before your pull request can be ultimately accepted.
|
|
28
|
+
|
|
29
|
+
## Style Guides
|
|
30
|
+
|
|
31
|
+
### Git Commit Messages
|
|
32
|
+
|
|
33
|
+
* Use the present tense ("Add feature" not "Added feature").
|
|
34
|
+
* Limit the first line to 72 characters or less.
|
|
35
|
+
* Reference issues and pull requests liberally after the first line.
|
|
36
|
+
|
|
37
|
+
### Python Styleguide
|
|
38
|
+
|
|
39
|
+
Adhere to the [PEP 8](https://pep8.org/) style guide, using `ruff` for automatic formatting. Ensure that your submissions are clean by running `ruff` before submitting a pull request. This will help maintain code consistency and readability across the project.
|
|
40
|
+
|
|
41
|
+
### Documentation Styleguide
|
|
42
|
+
|
|
43
|
+
* Use Markdown for formatting.
|
|
44
|
+
* Document new code based on existing documentation patterns.
|
|
45
|
+
* Update the README, if necessary, depending on the changes.
|
|
46
|
+
|
|
47
|
+
## Community
|
|
48
|
+
|
|
49
|
+
Discussions about PgQueuer take place on this repository's [Issues](https://github.com/janbjorge/PgQueuer/issues) and [Pull Requests](https://github.com/janbjorge/PgQueuer/pulls) sections. Anybody is welcome to join these conversations.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: PgQueuer
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.4
|
|
4
4
|
Summary: PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.
|
|
5
5
|
Author: janbjorge
|
|
6
6
|
License: MIT License
|
|
@@ -90,15 +90,36 @@ The CLI supports several flags to customize the connection settings. Use `--help
|
|
|
90
90
|
|
|
91
91
|
#### Dashboard Command
|
|
92
92
|
|
|
93
|
-
The
|
|
93
|
+
The dashboard command provides a real-time view of job processing statistics, which can be refreshed at a specified interval. This is particularly useful for monitoring the status of jobs dynamically. Below are the options available for customizing the dashboard display:
|
|
94
94
|
|
|
95
|
+
- `--interval <seconds>`: Set the refresh interval in seconds for updating the dashboard display. If not set, the dashboard will update only once and then exit.
|
|
96
|
+
- `--tail <number>`: Specify the number of the most recent log entries to display.
|
|
97
|
+
- `--table-format <format>`: Choose the format of the table used to display statistics. Supported formats include grid, plain, html, and others provided by the tabulate library.
|
|
98
|
+
|
|
99
|
+
Example command to launch the dashboard:
|
|
95
100
|
```bash
|
|
96
101
|
python -m PgQueuer dashboard --interval 10 --tail 25 --table-format grid
|
|
97
102
|
```
|
|
98
103
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
104
|
+
Example output from the dashboard:
|
|
105
|
+
```bash
|
|
106
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
107
|
+
| Created | Count | Entrypoint | Time in Queue (HH:MM:SS) | Status | Priority |
|
|
108
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
109
|
+
| 2024-05-05 16:44:26+00:00 | 49 | sync | 0:00:01 | successful | 0 |
|
|
110
|
+
| 2024-05-05 16:44:26+00:00 | 82 | async | 0:00:01 | successful | 0 |
|
|
111
|
+
| 2024-05-05 16:44:26+00:00 | 1615 | sync | 0:00:00 | successful | 0 |
|
|
112
|
+
| 2024-05-05 16:44:26+00:00 | 1586 | async | 0:00:00 | successful | 0 |
|
|
113
|
+
| 2024-05-05 16:44:25+00:00 | 198 | sync | 0:00:01 | successful | 0 |
|
|
114
|
+
| 2024-05-05 16:44:25+00:00 | 230 | async | 0:00:01 | successful | 0 |
|
|
115
|
+
| 2024-05-05 16:44:25+00:00 | 1802 | sync | 0:00:00 | successful | 0 |
|
|
116
|
+
| 2024-05-05 16:44:25+00:00 | 1778 | async | 0:00:00 | successful | 0 |
|
|
117
|
+
| 2024-05-05 16:44:24+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
118
|
+
| 2024-05-05 16:44:24+00:00 | 1506 | async | 0:00:00 | successful | 0 |
|
|
119
|
+
| 2024-05-05 16:44:23+00:00 | 1505 | async | 0:00:00 | successful | 0 |
|
|
120
|
+
| 2024-05-05 16:44:23+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
121
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
122
|
+
```
|
|
102
123
|
|
|
103
124
|
### Example Usage
|
|
104
125
|
|
|
@@ -48,15 +48,36 @@ The CLI supports several flags to customize the connection settings. Use `--help
|
|
|
48
48
|
|
|
49
49
|
#### Dashboard Command
|
|
50
50
|
|
|
51
|
-
The
|
|
51
|
+
The dashboard command provides a real-time view of job processing statistics, which can be refreshed at a specified interval. This is particularly useful for monitoring the status of jobs dynamically. Below are the options available for customizing the dashboard display:
|
|
52
52
|
|
|
53
|
+
- `--interval <seconds>`: Set the refresh interval in seconds for updating the dashboard display. If not set, the dashboard will update only once and then exit.
|
|
54
|
+
- `--tail <number>`: Specify the number of the most recent log entries to display.
|
|
55
|
+
- `--table-format <format>`: Choose the format of the table used to display statistics. Supported formats include grid, plain, html, and others provided by the tabulate library.
|
|
56
|
+
|
|
57
|
+
Example command to launch the dashboard:
|
|
53
58
|
```bash
|
|
54
59
|
python -m PgQueuer dashboard --interval 10 --tail 25 --table-format grid
|
|
55
60
|
```
|
|
56
61
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
62
|
+
Example output from the dashboard:
|
|
63
|
+
```bash
|
|
64
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
65
|
+
| Created | Count | Entrypoint | Time in Queue (HH:MM:SS) | Status | Priority |
|
|
66
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
67
|
+
| 2024-05-05 16:44:26+00:00 | 49 | sync | 0:00:01 | successful | 0 |
|
|
68
|
+
| 2024-05-05 16:44:26+00:00 | 82 | async | 0:00:01 | successful | 0 |
|
|
69
|
+
| 2024-05-05 16:44:26+00:00 | 1615 | sync | 0:00:00 | successful | 0 |
|
|
70
|
+
| 2024-05-05 16:44:26+00:00 | 1586 | async | 0:00:00 | successful | 0 |
|
|
71
|
+
| 2024-05-05 16:44:25+00:00 | 198 | sync | 0:00:01 | successful | 0 |
|
|
72
|
+
| 2024-05-05 16:44:25+00:00 | 230 | async | 0:00:01 | successful | 0 |
|
|
73
|
+
| 2024-05-05 16:44:25+00:00 | 1802 | sync | 0:00:00 | successful | 0 |
|
|
74
|
+
| 2024-05-05 16:44:25+00:00 | 1778 | async | 0:00:00 | successful | 0 |
|
|
75
|
+
| 2024-05-05 16:44:24+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
76
|
+
| 2024-05-05 16:44:24+00:00 | 1506 | async | 0:00:00 | successful | 0 |
|
|
77
|
+
| 2024-05-05 16:44:23+00:00 | 1505 | async | 0:00:00 | successful | 0 |
|
|
78
|
+
| 2024-05-05 16:44:23+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
79
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
80
|
+
```
|
|
60
81
|
|
|
61
82
|
### Example Usage
|
|
62
83
|
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
1
3
|
import argparse
|
|
2
4
|
import asyncio
|
|
3
5
|
import os
|
|
@@ -40,7 +42,7 @@ async def display_stats(
|
|
|
40
42
|
)
|
|
41
43
|
|
|
42
44
|
|
|
43
|
-
async def
|
|
45
|
+
async def fetch_and_display(
|
|
44
46
|
queries: Queries,
|
|
45
47
|
interval: timedelta | None,
|
|
46
48
|
tail: int,
|
|
@@ -49,8 +51,7 @@ async def fetch_and_dispay(
|
|
|
49
51
|
clear_and_home = "\033[2J\033[H"
|
|
50
52
|
while True:
|
|
51
53
|
print(clear_and_home, end="")
|
|
52
|
-
|
|
53
|
-
await display_stats(stats, tablefmt)
|
|
54
|
+
await display_stats(await queries.log_statistics(tail), tablefmt)
|
|
54
55
|
if interval is None:
|
|
55
56
|
return
|
|
56
57
|
await asyncio.sleep(interval.total_seconds())
|
|
@@ -219,17 +220,15 @@ async def main() -> None:
|
|
|
219
220
|
queries = Queries(pool)
|
|
220
221
|
match parsed.command:
|
|
221
222
|
case "install":
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
else:
|
|
223
|
+
print(QueryBuilder().create_install_query())
|
|
224
|
+
if not parsed.dry_run:
|
|
225
225
|
await queries.install()
|
|
226
226
|
case "uninstall":
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
else:
|
|
227
|
+
print(QueryBuilder().create_uninstall_query())
|
|
228
|
+
if not parsed.dry_run:
|
|
230
229
|
await queries.uninstall()
|
|
231
230
|
case "dashboard":
|
|
232
|
-
await
|
|
231
|
+
await fetch_and_display(
|
|
233
232
|
queries,
|
|
234
233
|
parsed.interval,
|
|
235
234
|
parsed.tail,
|
|
@@ -115,11 +115,11 @@ class QueueManager:
|
|
|
115
115
|
tasks, logs timeouts, and resets connections upon termination.
|
|
116
116
|
"""
|
|
117
117
|
async with (
|
|
118
|
-
self.pool.acquire() as
|
|
118
|
+
self.pool.acquire() as connection,
|
|
119
119
|
TaskManager() as tm,
|
|
120
120
|
):
|
|
121
121
|
listener = PGEventQueue()
|
|
122
|
-
await listener.connect(
|
|
122
|
+
await listener.connect(connection, self.channel)
|
|
123
123
|
|
|
124
124
|
while self.alive:
|
|
125
125
|
while self.alive and (job := await self.queries.dequeue()):
|
|
@@ -136,8 +136,8 @@ class QueueManager:
|
|
|
136
136
|
dequeue_timeout,
|
|
137
137
|
)
|
|
138
138
|
|
|
139
|
-
|
|
140
|
-
await
|
|
139
|
+
connection.remove_termination_listener(_critical_termination_listener)
|
|
140
|
+
await connection.reset()
|
|
141
141
|
|
|
142
142
|
async def _dispatch(self, job: Job) -> None:
|
|
143
143
|
"""
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
1
3
|
import dataclasses
|
|
2
4
|
import os
|
|
3
5
|
from typing import Final
|
|
@@ -119,7 +121,7 @@ class QueryBuilder:
|
|
|
119
121
|
status {self.settings.statistics_table_status_type} NOT NULL,
|
|
120
122
|
entrypoint TEXT NOT NULL
|
|
121
123
|
);
|
|
122
|
-
CREATE UNIQUE INDEX
|
|
124
|
+
CREATE UNIQUE INDEX {self.settings.statistics_table}_unique_count ON {self.settings.statistics_table} (
|
|
123
125
|
priority,
|
|
124
126
|
DATE_TRUNC('sec', created at time zone 'UTC'),
|
|
125
127
|
DATE_TRUNC('sec', time_in_queue),
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: PgQueuer
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.4
|
|
4
4
|
Summary: PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.
|
|
5
5
|
Author: janbjorge
|
|
6
6
|
License: MIT License
|
|
@@ -90,15 +90,36 @@ The CLI supports several flags to customize the connection settings. Use `--help
|
|
|
90
90
|
|
|
91
91
|
#### Dashboard Command
|
|
92
92
|
|
|
93
|
-
The
|
|
93
|
+
The dashboard command provides a real-time view of job processing statistics, which can be refreshed at a specified interval. This is particularly useful for monitoring the status of jobs dynamically. Below are the options available for customizing the dashboard display:
|
|
94
94
|
|
|
95
|
+
- `--interval <seconds>`: Set the refresh interval in seconds for updating the dashboard display. If not set, the dashboard will update only once and then exit.
|
|
96
|
+
- `--tail <number>`: Specify the number of the most recent log entries to display.
|
|
97
|
+
- `--table-format <format>`: Choose the format of the table used to display statistics. Supported formats include grid, plain, html, and others provided by the tabulate library.
|
|
98
|
+
|
|
99
|
+
Example command to launch the dashboard:
|
|
95
100
|
```bash
|
|
96
101
|
python -m PgQueuer dashboard --interval 10 --tail 25 --table-format grid
|
|
97
102
|
```
|
|
98
103
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
104
|
+
Example output from the dashboard:
|
|
105
|
+
```bash
|
|
106
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
107
|
+
| Created | Count | Entrypoint | Time in Queue (HH:MM:SS) | Status | Priority |
|
|
108
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
109
|
+
| 2024-05-05 16:44:26+00:00 | 49 | sync | 0:00:01 | successful | 0 |
|
|
110
|
+
| 2024-05-05 16:44:26+00:00 | 82 | async | 0:00:01 | successful | 0 |
|
|
111
|
+
| 2024-05-05 16:44:26+00:00 | 1615 | sync | 0:00:00 | successful | 0 |
|
|
112
|
+
| 2024-05-05 16:44:26+00:00 | 1586 | async | 0:00:00 | successful | 0 |
|
|
113
|
+
| 2024-05-05 16:44:25+00:00 | 198 | sync | 0:00:01 | successful | 0 |
|
|
114
|
+
| 2024-05-05 16:44:25+00:00 | 230 | async | 0:00:01 | successful | 0 |
|
|
115
|
+
| 2024-05-05 16:44:25+00:00 | 1802 | sync | 0:00:00 | successful | 0 |
|
|
116
|
+
| 2024-05-05 16:44:25+00:00 | 1778 | async | 0:00:00 | successful | 0 |
|
|
117
|
+
| 2024-05-05 16:44:24+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
118
|
+
| 2024-05-05 16:44:24+00:00 | 1506 | async | 0:00:00 | successful | 0 |
|
|
119
|
+
| 2024-05-05 16:44:23+00:00 | 1505 | async | 0:00:00 | successful | 0 |
|
|
120
|
+
| 2024-05-05 16:44:23+00:00 | 1500 | sync | 0:00:00 | successful | 0 |
|
|
121
|
+
+---------------------------+-------+------------+--------------------------+------------+----------+
|
|
122
|
+
```
|
|
102
123
|
|
|
103
124
|
### Example Usage
|
|
104
125
|
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|