cs-progress 20250412__tar.gz → 20250528__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: cs-progress
|
|
3
|
-
Version:
|
|
3
|
+
Version: 20250528
|
|
4
4
|
Summary: A progress tracker with methods for throughput, ETA and update notification; also a compound progress meter composed from other progress meters.
|
|
5
5
|
Keywords: python2,python3
|
|
6
6
|
Author-email: Cameron Simpson <cs@cskk.id.au>
|
|
@@ -12,14 +12,14 @@ Classifier: Intended Audience :: Developers
|
|
|
12
12
|
Classifier: Operating System :: OS Independent
|
|
13
13
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
14
14
|
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
|
|
15
|
-
Requires-Dist: cs.deco>=
|
|
15
|
+
Requires-Dist: cs.deco>=20250513
|
|
16
16
|
Requires-Dist: cs.logutils>=20250323
|
|
17
17
|
Requires-Dist: cs.py.func>=20240630
|
|
18
|
+
Requires-Dist: cs.queues>=20250426
|
|
18
19
|
Requires-Dist: cs.resources>=20250325
|
|
19
20
|
Requires-Dist: cs.seq>=20250306
|
|
20
|
-
Requires-Dist: cs.
|
|
21
|
-
Requires-Dist: cs.
|
|
22
|
-
Requires-Dist: cs.upd>=20240630
|
|
21
|
+
Requires-Dist: cs.units>=20250528
|
|
22
|
+
Requires-Dist: cs.upd>=20250426
|
|
23
23
|
Requires-Dist: icontract
|
|
24
24
|
Requires-Dist: typeguard
|
|
25
25
|
Project-URL: MonoRepo Commits, https://bitbucket.org/cameron_simpson/css/commits/branch/main
|
|
@@ -30,15 +30,36 @@ Project-URL: Source, https://github.com/cameron-simpson/css/blob/main/lib/python
|
|
|
30
30
|
A progress tracker with methods for throughput, ETA and update notification;
|
|
31
31
|
also a compound progress meter composed from other progress meters.
|
|
32
32
|
|
|
33
|
-
*Latest release
|
|
34
|
-
|
|
33
|
+
*Latest release 20250528*:
|
|
34
|
+
progressbar: also run the progress bar if report_print is true in order to get the report (was optimised out when the Upd was disabled).
|
|
35
|
+
|
|
36
|
+
This contains the follow main items:
|
|
37
|
+
* `progressbar`: a wrapper for an iterable presenting a progress
|
|
38
|
+
bar in the terminal
|
|
39
|
+
* `Progress`: a progress tracking class
|
|
40
|
+
* `OverProgress`: a progress tracking class which tracks the
|
|
41
|
+
aggregate of multiple `Progress` instances
|
|
42
|
+
|
|
43
|
+
Example:
|
|
44
|
+
|
|
45
|
+
for item in progressbar(items, "task name"):
|
|
46
|
+
....
|
|
47
|
+
|
|
48
|
+
Short summary:
|
|
49
|
+
* `auto_progressbar`: Decorator for a function accepting an optional `progress` keyword parameter. If `progress` is not `None` and the default `Upd` is not disabled, run the function with a progress bar.
|
|
50
|
+
* `BaseProgress`: The base class for `Progress` and `OverProcess` with various common methods.
|
|
51
|
+
* `CheckPoint`: CheckPoint(time, position).
|
|
52
|
+
* `OverProgress`: A `Progress`-like class computed from a set of subsidiary `Progress`es.
|
|
53
|
+
* `Progress`: A progress counter to track task completion with various utility methods.
|
|
54
|
+
* `progressbar`: Convenience function to construct and run a `Progress.iterbar` wrapping the iterable `it`, issuing and withdrawing a progress bar during the iteration. If there is no current `Upd` instance or it is disabled, this returns `it` directly.
|
|
55
|
+
* `selftest`: Exercise some of the functionality.
|
|
35
56
|
|
|
36
57
|
Module contents:
|
|
37
58
|
- <a name="auto_progressbar"></a>`auto_progressbar(*da, **dkw)`: Decorator for a function accepting an optional `progress`
|
|
38
59
|
keyword parameter.
|
|
39
60
|
If `progress` is not `None` and the default `Upd` is not disabled,
|
|
40
61
|
run the function with a progress bar.
|
|
41
|
-
- <a name="BaseProgress"></a>`Class `BaseProgress
|
|
62
|
+
- <a name="BaseProgress"></a>`Class `BaseProgress``: The base class for `Progress` and `OverProcess`
|
|
42
63
|
with various common methods.
|
|
43
64
|
|
|
44
65
|
Note that durations are in seconds
|
|
@@ -78,18 +99,19 @@ if its position is less than `int(other)`.
|
|
|
78
99
|
Construct a progress arrow representing completion
|
|
79
100
|
to fit in the specified `width`.
|
|
80
101
|
|
|
81
|
-
*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at
|
|
102
|
+
*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x1070dcea0>, **kw)`*:
|
|
82
103
|
A context manager to create and withdraw a progress bar.
|
|
83
104
|
It returns the `UpdProxy` which displays the progress bar.
|
|
84
105
|
|
|
85
106
|
Parameters:
|
|
86
|
-
* `label`:
|
|
107
|
+
* `label`: an optional label for the progress bar,
|
|
87
108
|
default from `self.name`.
|
|
88
|
-
* `
|
|
89
|
-
|
|
90
|
-
* `
|
|
91
|
-
|
|
92
|
-
|
|
109
|
+
* `insert_pos`: where to insert the progress bar within the `cs.Upd`,
|
|
110
|
+
default `1`
|
|
111
|
+
* `poll`: an optional callable which will receive `self`,
|
|
112
|
+
which can be used to update the progress state before
|
|
113
|
+
updating the progress bar display; useful if the progress
|
|
114
|
+
should be updates from some other programme state
|
|
93
115
|
* `recent_window`: optional timeframe to define "recent" in seconds;
|
|
94
116
|
if the default `statusfunc` (`Progress.status`) is used
|
|
95
117
|
this is passed to it
|
|
@@ -98,11 +120,15 @@ Parameters:
|
|
|
98
120
|
this may also be a `bool`, which if true will use `Upd.print`
|
|
99
121
|
in order to interoperate with `Upd`.
|
|
100
122
|
* `stalled`: optional string to replace the word `'stalled'`
|
|
101
|
-
in the status line; for a
|
|
102
|
-
* `
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
123
|
+
in the status line; for a worker this might be better as `'idle'`
|
|
124
|
+
* `statusfunc`: an optional function to compute the progress bar text
|
|
125
|
+
accepting `(self,label,width)`; default `Progress.status`
|
|
126
|
+
* `update_period`: an optional frequency with which to update the display,
|
|
127
|
+
default from `DEFAULT_UPDATE_PERIOD` (0.3s);
|
|
128
|
+
if set to `0` then the display is updated whenever `self` is updated
|
|
129
|
+
* `width`: an optional width expressing how wide the progress bar
|
|
130
|
+
text may be.
|
|
131
|
+
The default comes from the `proxy.width` property.
|
|
106
132
|
|
|
107
133
|
Example use:
|
|
108
134
|
|
|
@@ -125,7 +151,7 @@ If `remaining_time` is `None`, this is also `None`.
|
|
|
125
151
|
Format `value` accoridng to `scale` and `max_parts`
|
|
126
152
|
using `cs.units.transcribe`.
|
|
127
153
|
|
|
128
|
-
*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, cancelled=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at
|
|
154
|
+
*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, cancelled=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at 0x1070dd300>, **bar_kw)`*:
|
|
129
155
|
An iterable progress bar: a generator yielding values
|
|
130
156
|
from the iterable `it` while updating a progress bar.
|
|
131
157
|
|
|
@@ -149,7 +175,7 @@ Parameters:
|
|
|
149
175
|
* `cancelled`: an optional callable to test for iteration cancellation
|
|
150
176
|
Other parameters are passed to `Progress.bar`.
|
|
151
177
|
|
|
152
|
-
Example
|
|
178
|
+
Example uses:
|
|
153
179
|
|
|
154
180
|
from cs.units import DECIMAL_SCALE
|
|
155
181
|
rows = [some list of data]
|
|
@@ -157,23 +183,30 @@ Example use:
|
|
|
157
183
|
for row in P.iterbar(rows, incfirst=True):
|
|
158
184
|
... do something with each row ...
|
|
159
185
|
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
186
|
+
with open(data_filename, 'rb') as f:
|
|
187
|
+
datalen = os.stat(f).st_size
|
|
188
|
+
def readfrom(f):
|
|
189
|
+
while True:
|
|
190
|
+
bs = f.read(65536)
|
|
191
|
+
if not bs:
|
|
192
|
+
break
|
|
193
|
+
yield bs
|
|
194
|
+
P = Progress(total=datalen)
|
|
195
|
+
for bs in P.iterbar(readfrom(f), itemlenfunc=len):
|
|
196
|
+
... process the file data in bs ...
|
|
171
197
|
|
|
172
198
|
*`BaseProgress.qbar(self, label=None, **iterbar_kw) -> cs.queues.QueueIterator`*:
|
|
173
|
-
Set up a progress bar, return a `
|
|
174
|
-
This is a shim for `Progress.iterbar`
|
|
175
|
-
worker to iterate
|
|
176
|
-
|
|
199
|
+
Set up a progress bar, return a closeable `Queue`-like object
|
|
200
|
+
for receiving items. This is a shim for `Progress.iterbar`
|
|
201
|
+
which dispatches a worker to iterate items put onto a queue.
|
|
202
|
+
|
|
203
|
+
Example:
|
|
204
|
+
|
|
205
|
+
Q = Progress.qbar("label")
|
|
206
|
+
try:
|
|
207
|
+
... do work, calling Q.put(item) ...
|
|
208
|
+
finally:
|
|
209
|
+
Q.close()
|
|
177
210
|
|
|
178
211
|
*`BaseProgress.ratio`*:
|
|
179
212
|
The fraction of progress completed: `(position-start)/(total-start)`.
|
|
@@ -238,7 +271,7 @@ during `elapsed_time`.
|
|
|
238
271
|
|
|
239
272
|
*`BaseProgress.throughput_recent(self, time_window)`*:
|
|
240
273
|
The recent throughput. Implemented by subclasses.
|
|
241
|
-
- <a name="CheckPoint"></a>`Class `CheckPoint(builtins.tuple)
|
|
274
|
+
- <a name="CheckPoint"></a>`Class `CheckPoint(builtins.tuple)``: CheckPoint(time, position)
|
|
242
275
|
|
|
243
276
|
*`CheckPoint.__replace__(self, /, **kwds)`*:
|
|
244
277
|
Return a new CheckPoint object replacing specified fields with new values
|
|
@@ -248,7 +281,7 @@ Alias for field number 1
|
|
|
248
281
|
|
|
249
282
|
*`CheckPoint.time`*:
|
|
250
283
|
Alias for field number 0
|
|
251
|
-
- <a name="OverProgress"></a>`Class `OverProgress(BaseProgress)
|
|
284
|
+
- <a name="OverProgress"></a>`Class `OverProgress(BaseProgress)``: A `Progress`-like class computed from a set of subsidiary `Progress`es.
|
|
252
285
|
|
|
253
286
|
AN OverProgress instance has an attribute ``notify_update`` which
|
|
254
287
|
is a set of callables.
|
|
@@ -309,7 +342,7 @@ The `throughput_recent` is the sum of the subsidiary throughput_recentss.
|
|
|
309
342
|
|
|
310
343
|
*`OverProgress.total`*:
|
|
311
344
|
The `total` is the sum of the subsidiary totals.
|
|
312
|
-
- <a name="Progress"></a>`Class `Progress(BaseProgress)
|
|
345
|
+
- <a name="Progress"></a>`Class `Progress(BaseProgress)``: A progress counter to track task completion with various utility methods.
|
|
313
346
|
|
|
314
347
|
Example:
|
|
315
348
|
|
|
@@ -433,9 +466,11 @@ Record more progress.
|
|
|
433
466
|
>>> P.update(12)
|
|
434
467
|
>>> P.position
|
|
435
468
|
12
|
|
436
|
-
- <a name="progressbar"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at
|
|
469
|
+
- <a name="progressbar"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x1070ded40>, **kw)`: Convenience function to construct and run a `Progress.iterbar`
|
|
437
470
|
wrapping the iterable `it`,
|
|
438
471
|
issuing and withdrawing a progress bar during the iteration.
|
|
472
|
+
If there is no current `Upd` instance or it is disabled, this
|
|
473
|
+
returns `it` directly.
|
|
439
474
|
|
|
440
475
|
Parameters:
|
|
441
476
|
* `it`: the iterable to consume
|
|
@@ -461,6 +496,9 @@ Record more progress.
|
|
|
461
496
|
|
|
462
497
|
|
|
463
498
|
|
|
499
|
+
*Release 20250528*:
|
|
500
|
+
progressbar: also run the progress bar if report_print is true in order to get the report (was optimised out when the Upd was disabled).
|
|
501
|
+
|
|
464
502
|
*Release 20250412*:
|
|
465
503
|
Bugfix progressbar: fix early return if no active Upd.
|
|
466
504
|
|
|
@@ -9,14 +9,14 @@ keywords = [
|
|
|
9
9
|
"python3",
|
|
10
10
|
]
|
|
11
11
|
dependencies = [
|
|
12
|
-
"cs.deco>=
|
|
12
|
+
"cs.deco>=20250513",
|
|
13
13
|
"cs.logutils>=20250323",
|
|
14
14
|
"cs.py.func>=20240630",
|
|
15
|
+
"cs.queues>=20250426",
|
|
15
16
|
"cs.resources>=20250325",
|
|
16
17
|
"cs.seq>=20250306",
|
|
17
|
-
"cs.
|
|
18
|
-
"cs.
|
|
19
|
-
"cs.upd>=20240630",
|
|
18
|
+
"cs.units>=20250528",
|
|
19
|
+
"cs.upd>=20250426",
|
|
20
20
|
"icontract",
|
|
21
21
|
"typeguard",
|
|
22
22
|
]
|
|
@@ -29,7 +29,7 @@ classifiers = [
|
|
|
29
29
|
"Topic :: Software Development :: Libraries :: Python Modules",
|
|
30
30
|
"License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
|
|
31
31
|
]
|
|
32
|
-
version = "
|
|
32
|
+
version = "20250528"
|
|
33
33
|
|
|
34
34
|
[project.license]
|
|
35
35
|
text = "GNU General Public License v3 or later (GPLv3+)"
|
|
@@ -45,15 +45,36 @@ text = """
|
|
|
45
45
|
A progress tracker with methods for throughput, ETA and update notification;
|
|
46
46
|
also a compound progress meter composed from other progress meters.
|
|
47
47
|
|
|
48
|
-
*Latest release
|
|
49
|
-
|
|
48
|
+
*Latest release 20250528*:
|
|
49
|
+
progressbar: also run the progress bar if report_print is true in order to get the report (was optimised out when the Upd was disabled).
|
|
50
|
+
|
|
51
|
+
This contains the follow main items:
|
|
52
|
+
* `progressbar`: a wrapper for an iterable presenting a progress
|
|
53
|
+
bar in the terminal
|
|
54
|
+
* `Progress`: a progress tracking class
|
|
55
|
+
* `OverProgress`: a progress tracking class which tracks the
|
|
56
|
+
aggregate of multiple `Progress` instances
|
|
57
|
+
|
|
58
|
+
Example:
|
|
59
|
+
|
|
60
|
+
for item in progressbar(items, \"task name\"):
|
|
61
|
+
....
|
|
62
|
+
|
|
63
|
+
Short summary:
|
|
64
|
+
* `auto_progressbar`: Decorator for a function accepting an optional `progress` keyword parameter. If `progress` is not `None` and the default `Upd` is not disabled, run the function with a progress bar.
|
|
65
|
+
* `BaseProgress`: The base class for `Progress` and `OverProcess` with various common methods.
|
|
66
|
+
* `CheckPoint`: CheckPoint(time, position).
|
|
67
|
+
* `OverProgress`: A `Progress`-like class computed from a set of subsidiary `Progress`es.
|
|
68
|
+
* `Progress`: A progress counter to track task completion with various utility methods.
|
|
69
|
+
* `progressbar`: Convenience function to construct and run a `Progress.iterbar` wrapping the iterable `it`, issuing and withdrawing a progress bar during the iteration. If there is no current `Upd` instance or it is disabled, this returns `it` directly.
|
|
70
|
+
* `selftest`: Exercise some of the functionality.
|
|
50
71
|
|
|
51
72
|
Module contents:
|
|
52
73
|
- <a name=\"auto_progressbar\"></a>`auto_progressbar(*da, **dkw)`: Decorator for a function accepting an optional `progress`
|
|
53
74
|
keyword parameter.
|
|
54
75
|
If `progress` is not `None` and the default `Upd` is not disabled,
|
|
55
76
|
run the function with a progress bar.
|
|
56
|
-
- <a name=\"BaseProgress\"></a>`Class `BaseProgress
|
|
77
|
+
- <a name=\"BaseProgress\"></a>`Class `BaseProgress``: The base class for `Progress` and `OverProcess`
|
|
57
78
|
with various common methods.
|
|
58
79
|
|
|
59
80
|
Note that durations are in seconds
|
|
@@ -93,18 +114,19 @@ if its position is less than `int(other)`.
|
|
|
93
114
|
Construct a progress arrow representing completion
|
|
94
115
|
to fit in the specified `width`.
|
|
95
116
|
|
|
96
|
-
*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at
|
|
117
|
+
*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x1070dcea0>, **kw)`*:
|
|
97
118
|
A context manager to create and withdraw a progress bar.
|
|
98
119
|
It returns the `UpdProxy` which displays the progress bar.
|
|
99
120
|
|
|
100
121
|
Parameters:
|
|
101
|
-
* `label`:
|
|
122
|
+
* `label`: an optional label for the progress bar,
|
|
102
123
|
default from `self.name`.
|
|
103
|
-
* `
|
|
104
|
-
|
|
105
|
-
* `
|
|
106
|
-
|
|
107
|
-
|
|
124
|
+
* `insert_pos`: where to insert the progress bar within the `cs.Upd`,
|
|
125
|
+
default `1`
|
|
126
|
+
* `poll`: an optional callable which will receive `self`,
|
|
127
|
+
which can be used to update the progress state before
|
|
128
|
+
updating the progress bar display; useful if the progress
|
|
129
|
+
should be updates from some other programme state
|
|
108
130
|
* `recent_window`: optional timeframe to define \"recent\" in seconds;
|
|
109
131
|
if the default `statusfunc` (`Progress.status`) is used
|
|
110
132
|
this is passed to it
|
|
@@ -113,11 +135,15 @@ Parameters:
|
|
|
113
135
|
this may also be a `bool`, which if true will use `Upd.print`
|
|
114
136
|
in order to interoperate with `Upd`.
|
|
115
137
|
* `stalled`: optional string to replace the word `'stalled'`
|
|
116
|
-
in the status line; for a
|
|
117
|
-
* `
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
138
|
+
in the status line; for a worker this might be better as `'idle'`
|
|
139
|
+
* `statusfunc`: an optional function to compute the progress bar text
|
|
140
|
+
accepting `(self,label,width)`; default `Progress.status`
|
|
141
|
+
* `update_period`: an optional frequency with which to update the display,
|
|
142
|
+
default from `DEFAULT_UPDATE_PERIOD` (0.3s);
|
|
143
|
+
if set to `0` then the display is updated whenever `self` is updated
|
|
144
|
+
* `width`: an optional width expressing how wide the progress bar
|
|
145
|
+
text may be.
|
|
146
|
+
The default comes from the `proxy.width` property.
|
|
121
147
|
|
|
122
148
|
Example use:
|
|
123
149
|
|
|
@@ -140,7 +166,7 @@ If `remaining_time` is `None`, this is also `None`.
|
|
|
140
166
|
Format `value` accoridng to `scale` and `max_parts`
|
|
141
167
|
using `cs.units.transcribe`.
|
|
142
168
|
|
|
143
|
-
*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, cancelled=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at
|
|
169
|
+
*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, cancelled=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at 0x1070dd300>, **bar_kw)`*:
|
|
144
170
|
An iterable progress bar: a generator yielding values
|
|
145
171
|
from the iterable `it` while updating a progress bar.
|
|
146
172
|
|
|
@@ -164,7 +190,7 @@ Parameters:
|
|
|
164
190
|
* `cancelled`: an optional callable to test for iteration cancellation
|
|
165
191
|
Other parameters are passed to `Progress.bar`.
|
|
166
192
|
|
|
167
|
-
Example
|
|
193
|
+
Example uses:
|
|
168
194
|
|
|
169
195
|
from cs.units import DECIMAL_SCALE
|
|
170
196
|
rows = [some list of data]
|
|
@@ -172,23 +198,30 @@ Example use:
|
|
|
172
198
|
for row in P.iterbar(rows, incfirst=True):
|
|
173
199
|
... do something with each row ...
|
|
174
200
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
201
|
+
with open(data_filename, 'rb') as f:
|
|
202
|
+
datalen = os.stat(f).st_size
|
|
203
|
+
def readfrom(f):
|
|
204
|
+
while True:
|
|
205
|
+
bs = f.read(65536)
|
|
206
|
+
if not bs:
|
|
207
|
+
break
|
|
208
|
+
yield bs
|
|
209
|
+
P = Progress(total=datalen)
|
|
210
|
+
for bs in P.iterbar(readfrom(f), itemlenfunc=len):
|
|
211
|
+
... process the file data in bs ...
|
|
186
212
|
|
|
187
213
|
*`BaseProgress.qbar(self, label=None, **iterbar_kw) -> cs.queues.QueueIterator`*:
|
|
188
|
-
Set up a progress bar, return a `
|
|
189
|
-
This is a shim for `Progress.iterbar`
|
|
190
|
-
worker to iterate
|
|
191
|
-
|
|
214
|
+
Set up a progress bar, return a closeable `Queue`-like object
|
|
215
|
+
for receiving items. This is a shim for `Progress.iterbar`
|
|
216
|
+
which dispatches a worker to iterate items put onto a queue.
|
|
217
|
+
|
|
218
|
+
Example:
|
|
219
|
+
|
|
220
|
+
Q = Progress.qbar(\"label\")
|
|
221
|
+
try:
|
|
222
|
+
... do work, calling Q.put(item) ...
|
|
223
|
+
finally:
|
|
224
|
+
Q.close()
|
|
192
225
|
|
|
193
226
|
*`BaseProgress.ratio`*:
|
|
194
227
|
The fraction of progress completed: `(position-start)/(total-start)`.
|
|
@@ -253,7 +286,7 @@ during `elapsed_time`.
|
|
|
253
286
|
|
|
254
287
|
*`BaseProgress.throughput_recent(self, time_window)`*:
|
|
255
288
|
The recent throughput. Implemented by subclasses.
|
|
256
|
-
- <a name=\"CheckPoint\"></a>`Class `CheckPoint(builtins.tuple)
|
|
289
|
+
- <a name=\"CheckPoint\"></a>`Class `CheckPoint(builtins.tuple)``: CheckPoint(time, position)
|
|
257
290
|
|
|
258
291
|
*`CheckPoint.__replace__(self, /, **kwds)`*:
|
|
259
292
|
Return a new CheckPoint object replacing specified fields with new values
|
|
@@ -263,7 +296,7 @@ Alias for field number 1
|
|
|
263
296
|
|
|
264
297
|
*`CheckPoint.time`*:
|
|
265
298
|
Alias for field number 0
|
|
266
|
-
- <a name=\"OverProgress\"></a>`Class `OverProgress(BaseProgress)
|
|
299
|
+
- <a name=\"OverProgress\"></a>`Class `OverProgress(BaseProgress)``: A `Progress`-like class computed from a set of subsidiary `Progress`es.
|
|
267
300
|
|
|
268
301
|
AN OverProgress instance has an attribute ``notify_update`` which
|
|
269
302
|
is a set of callables.
|
|
@@ -324,7 +357,7 @@ The `throughput_recent` is the sum of the subsidiary throughput_recentss.
|
|
|
324
357
|
|
|
325
358
|
*`OverProgress.total`*:
|
|
326
359
|
The `total` is the sum of the subsidiary totals.
|
|
327
|
-
- <a name=\"Progress\"></a>`Class `Progress(BaseProgress)
|
|
360
|
+
- <a name=\"Progress\"></a>`Class `Progress(BaseProgress)``: A progress counter to track task completion with various utility methods.
|
|
328
361
|
|
|
329
362
|
Example:
|
|
330
363
|
|
|
@@ -448,9 +481,11 @@ Record more progress.
|
|
|
448
481
|
>>> P.update(12)
|
|
449
482
|
>>> P.position
|
|
450
483
|
12
|
|
451
|
-
- <a name=\"progressbar\"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at
|
|
484
|
+
- <a name=\"progressbar\"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x1070ded40>, **kw)`: Convenience function to construct and run a `Progress.iterbar`
|
|
452
485
|
wrapping the iterable `it`,
|
|
453
486
|
issuing and withdrawing a progress bar during the iteration.
|
|
487
|
+
If there is no current `Upd` instance or it is disabled, this
|
|
488
|
+
returns `it` directly.
|
|
454
489
|
|
|
455
490
|
Parameters:
|
|
456
491
|
* `it`: the iterable to consume
|
|
@@ -476,6 +511,9 @@ Record more progress.
|
|
|
476
511
|
|
|
477
512
|
|
|
478
513
|
|
|
514
|
+
*Release 20250528*:
|
|
515
|
+
progressbar: also run the progress bar if report_print is true in order to get the report (was optimised out when the Upd was disabled).
|
|
516
|
+
|
|
479
517
|
*Release 20250412*:
|
|
480
518
|
Bugfix progressbar: fix early return if no active Upd.
|
|
481
519
|
|
|
@@ -8,6 +8,18 @@
|
|
|
8
8
|
|
|
9
9
|
''' A progress tracker with methods for throughput, ETA and update notification;
|
|
10
10
|
also a compound progress meter composed from other progress meters.
|
|
11
|
+
|
|
12
|
+
This contains the follow main items:
|
|
13
|
+
* `progressbar`: a wrapper for an iterable presenting a progress
|
|
14
|
+
bar in the terminal
|
|
15
|
+
* `Progress`: a progress tracking class
|
|
16
|
+
* `OverProgress`: a progress tracking class which tracks the
|
|
17
|
+
aggregate of multiple `Progress` instances
|
|
18
|
+
|
|
19
|
+
Example:
|
|
20
|
+
|
|
21
|
+
for item in progressbar(items, "task name"):
|
|
22
|
+
....
|
|
11
23
|
'''
|
|
12
24
|
|
|
13
25
|
from collections import namedtuple
|
|
@@ -21,13 +33,12 @@ from typing import Callable, Optional
|
|
|
21
33
|
from icontract import ensure
|
|
22
34
|
from typeguard import typechecked
|
|
23
35
|
|
|
24
|
-
from cs.deco import decorator, uses_quiet
|
|
36
|
+
from cs.deco import decorator, fmtdoc, uses_quiet
|
|
25
37
|
from cs.logutils import debug, exception
|
|
26
38
|
from cs.py.func import funcname
|
|
27
39
|
from cs.queues import IterableQueue, QueueIterator
|
|
28
40
|
from cs.resources import RunState, uses_runstate
|
|
29
41
|
from cs.seq import seq
|
|
30
|
-
from cs.threads import bg
|
|
31
42
|
from cs.units import (
|
|
32
43
|
transcribe_time,
|
|
33
44
|
transcribe as transcribe_units,
|
|
@@ -38,7 +49,7 @@ from cs.units import (
|
|
|
38
49
|
)
|
|
39
50
|
from cs.upd import Upd, uses_upd, print # pylint: disable=redefined-builtin
|
|
40
51
|
|
|
41
|
-
__version__ = '
|
|
52
|
+
__version__ = '20250528'
|
|
42
53
|
|
|
43
54
|
DISTINFO = {
|
|
44
55
|
'keywords': ["python2", "python3"],
|
|
@@ -50,9 +61,9 @@ DISTINFO = {
|
|
|
50
61
|
'cs.deco',
|
|
51
62
|
'cs.logutils',
|
|
52
63
|
'cs.py.func',
|
|
64
|
+
'cs.queues',
|
|
53
65
|
'cs.resources',
|
|
54
66
|
'cs.seq',
|
|
55
|
-
'cs.threads',
|
|
56
67
|
'cs.units',
|
|
57
68
|
'cs.upd',
|
|
58
69
|
'icontract',
|
|
@@ -67,7 +78,7 @@ DEFAULT_THROUGHPUT_WINDOW = 5
|
|
|
67
78
|
DEFAULT_UPDATE_PERIOD = 0.3
|
|
68
79
|
|
|
69
80
|
@functools.total_ordering
|
|
70
|
-
class BaseProgress
|
|
81
|
+
class BaseProgress:
|
|
71
82
|
''' The base class for `Progress` and `OverProcess`
|
|
72
83
|
with various common methods.
|
|
73
84
|
|
|
@@ -378,6 +389,7 @@ class BaseProgress(object):
|
|
|
378
389
|
@contextmanager
|
|
379
390
|
@uses_quiet
|
|
380
391
|
@uses_upd
|
|
392
|
+
@fmtdoc
|
|
381
393
|
def bar(
|
|
382
394
|
self,
|
|
383
395
|
label=None,
|
|
@@ -397,13 +409,14 @@ class BaseProgress(object):
|
|
|
397
409
|
It returns the `UpdProxy` which displays the progress bar.
|
|
398
410
|
|
|
399
411
|
Parameters:
|
|
400
|
-
* `label`:
|
|
412
|
+
* `label`: an optional label for the progress bar,
|
|
401
413
|
default from `self.name`.
|
|
402
|
-
* `
|
|
403
|
-
|
|
404
|
-
* `
|
|
405
|
-
|
|
406
|
-
|
|
414
|
+
* `insert_pos`: where to insert the progress bar within the `cs.Upd`,
|
|
415
|
+
default `1`
|
|
416
|
+
* `poll`: an optional callable which will receive `self`,
|
|
417
|
+
which can be used to update the progress state before
|
|
418
|
+
updating the progress bar display; useful if the progress
|
|
419
|
+
should be updates from some other programme state
|
|
407
420
|
* `recent_window`: optional timeframe to define "recent" in seconds;
|
|
408
421
|
if the default `statusfunc` (`Progress.status`) is used
|
|
409
422
|
this is passed to it
|
|
@@ -412,11 +425,15 @@ class BaseProgress(object):
|
|
|
412
425
|
this may also be a `bool`, which if true will use `Upd.print`
|
|
413
426
|
in order to interoperate with `Upd`.
|
|
414
427
|
* `stalled`: optional string to replace the word `'stalled'`
|
|
415
|
-
in the status line; for a
|
|
416
|
-
* `
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
428
|
+
in the status line; for a worker this might be better as `'idle'`
|
|
429
|
+
* `statusfunc`: an optional function to compute the progress bar text
|
|
430
|
+
accepting `(self,label,width)`; default `Progress.status`
|
|
431
|
+
* `update_period`: an optional frequency with which to update the display,
|
|
432
|
+
default from `DEFAULT_UPDATE_PERIOD` ({DEFAULT_UPDATE_PERIOD}s);
|
|
433
|
+
if set to `0` then the display is updated whenever `self` is updated
|
|
434
|
+
* `width`: an optional width expressing how wide the progress bar
|
|
435
|
+
text may be.
|
|
436
|
+
The default comes from the `proxy.width` property.
|
|
420
437
|
|
|
421
438
|
Example use:
|
|
422
439
|
|
|
@@ -459,7 +476,7 @@ class BaseProgress(object):
|
|
|
459
476
|
|
|
460
477
|
cancel_ticker = False
|
|
461
478
|
|
|
462
|
-
def
|
|
479
|
+
def _ticker():
|
|
463
480
|
''' Worker to update the progress bar every `update_period` seconds.
|
|
464
481
|
'''
|
|
465
482
|
time.sleep(update_period)
|
|
@@ -467,8 +484,6 @@ class BaseProgress(object):
|
|
|
467
484
|
update(self, None)
|
|
468
485
|
time.sleep(update_period)
|
|
469
486
|
|
|
470
|
-
if update_period == 0:
|
|
471
|
-
self.notify_update.add(update)
|
|
472
487
|
try:
|
|
473
488
|
start_pos = self.position
|
|
474
489
|
with upd.insert(
|
|
@@ -477,8 +492,12 @@ class BaseProgress(object):
|
|
|
477
492
|
text_auto=text_auto,
|
|
478
493
|
) as proxy:
|
|
479
494
|
update(self, None)
|
|
480
|
-
if update_period
|
|
481
|
-
|
|
495
|
+
if update_period == 0:
|
|
496
|
+
# update every time the Progress is updated
|
|
497
|
+
self.notify_update.add(update)
|
|
498
|
+
elif update_period > 0:
|
|
499
|
+
# update every update_period seconds
|
|
500
|
+
Thread(target=_ticker, name=f'{label}-ticker', daemon=True).start()
|
|
482
501
|
yield proxy
|
|
483
502
|
finally:
|
|
484
503
|
cancel_ticker = True
|
|
@@ -531,7 +550,7 @@ class BaseProgress(object):
|
|
|
531
550
|
* `cancelled`: an optional callable to test for iteration cancellation
|
|
532
551
|
Other parameters are passed to `Progress.bar`.
|
|
533
552
|
|
|
534
|
-
Example
|
|
553
|
+
Example uses:
|
|
535
554
|
|
|
536
555
|
from cs.units import DECIMAL_SCALE
|
|
537
556
|
rows = [some list of data]
|
|
@@ -539,17 +558,17 @@ class BaseProgress(object):
|
|
|
539
558
|
for row in P.iterbar(rows, incfirst=True):
|
|
540
559
|
... do something with each row ...
|
|
541
560
|
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
561
|
+
with open(data_filename, 'rb') as f:
|
|
562
|
+
datalen = os.stat(f).st_size
|
|
563
|
+
def readfrom(f):
|
|
564
|
+
while True:
|
|
565
|
+
bs = f.read(65536)
|
|
566
|
+
if not bs:
|
|
567
|
+
break
|
|
568
|
+
yield bs
|
|
569
|
+
P = Progress(total=datalen)
|
|
570
|
+
for bs in P.iterbar(readfrom(f), itemlenfunc=len):
|
|
571
|
+
... process the file data in bs ...
|
|
553
572
|
'''
|
|
554
573
|
if cancelled is None:
|
|
555
574
|
cancelled = lambda: runstate.cancelled
|
|
@@ -570,14 +589,23 @@ class BaseProgress(object):
|
|
|
570
589
|
proxy.text = None
|
|
571
590
|
|
|
572
591
|
def qbar(self, label=None, **iterbar_kw) -> QueueIterator:
|
|
573
|
-
''' Set up a progress bar, return a `
|
|
574
|
-
This is a shim for `Progress.iterbar`
|
|
575
|
-
worker to iterate
|
|
576
|
-
|
|
592
|
+
''' Set up a progress bar, return a closeable `Queue`-like object
|
|
593
|
+
for receiving items. This is a shim for `Progress.iterbar`
|
|
594
|
+
which dispatches a worker to iterate items put onto a queue.
|
|
595
|
+
|
|
596
|
+
Example:
|
|
597
|
+
|
|
598
|
+
Q = Progress.qbar("label")
|
|
599
|
+
try:
|
|
600
|
+
... do work, calling Q.put(item) ...
|
|
601
|
+
finally:
|
|
602
|
+
Q.close()
|
|
577
603
|
'''
|
|
578
604
|
Q = IterableQueue(name=label)
|
|
579
605
|
|
|
580
606
|
def qbar_worker():
|
|
607
|
+
''' Consume the items from `Q`, updating the progress bar.
|
|
608
|
+
'''
|
|
581
609
|
for _ in self.iterbar(Q, label=label, **iterbar_kw):
|
|
582
610
|
pass
|
|
583
611
|
|
|
@@ -808,7 +836,7 @@ class Progress(BaseProgress):
|
|
|
808
836
|
positions = self._positions
|
|
809
837
|
# scan for first item still in time window,
|
|
810
838
|
# never discard the last 2 positions
|
|
811
|
-
for ndx in range(
|
|
839
|
+
for ndx in range(len(positions) - 1):
|
|
812
840
|
posn = positions[ndx]
|
|
813
841
|
if posn.time >= oldest:
|
|
814
842
|
# this is the first element to keep, discard preceeding (if any)
|
|
@@ -849,8 +877,7 @@ class Progress(BaseProgress):
|
|
|
849
877
|
return None
|
|
850
878
|
now = time.time()
|
|
851
879
|
time0 = now - time_window
|
|
852
|
-
|
|
853
|
-
time0 = self.start_time
|
|
880
|
+
time0 = max(time0, self.start_time)
|
|
854
881
|
# lowest time and position
|
|
855
882
|
# low_time will be time0
|
|
856
883
|
# low_pos will be the matching position, probably interpolated
|
|
@@ -1045,11 +1072,14 @@ def progressbar(
|
|
|
1045
1072
|
total=None,
|
|
1046
1073
|
units_scale=UNSCALED_SCALE,
|
|
1047
1074
|
upd: Upd,
|
|
1075
|
+
report_print=None,
|
|
1048
1076
|
**iterbar_kw
|
|
1049
1077
|
):
|
|
1050
1078
|
''' Convenience function to construct and run a `Progress.iterbar`
|
|
1051
1079
|
wrapping the iterable `it`,
|
|
1052
1080
|
issuing and withdrawing a progress bar during the iteration.
|
|
1081
|
+
If there is no current `Upd` instance or it is disabled, this
|
|
1082
|
+
returns `it` directly.
|
|
1053
1083
|
|
|
1054
1084
|
Parameters:
|
|
1055
1085
|
* `it`: the iterable to consume
|
|
@@ -1070,17 +1100,18 @@ def progressbar(
|
|
|
1070
1100
|
for row in progressbar(rows):
|
|
1071
1101
|
... do something with row ...
|
|
1072
1102
|
'''
|
|
1073
|
-
if upd is None or upd.disabled:
|
|
1074
|
-
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
1103
|
+
if not report_print and (upd is None or upd.disabled):
|
|
1104
|
+
return it
|
|
1105
|
+
if total is None:
|
|
1106
|
+
try:
|
|
1107
|
+
total = len(it)
|
|
1108
|
+
except TypeError:
|
|
1109
|
+
total = None
|
|
1110
|
+
return Progress(
|
|
1111
|
+
name=label, position=position, total=total, units_scale=units_scale
|
|
1112
|
+
).iterbar(
|
|
1113
|
+
it, report_print=report_print, **iterbar_kw
|
|
1114
|
+
)
|
|
1084
1115
|
|
|
1085
1116
|
@decorator
|
|
1086
1117
|
def auto_progressbar(func, label=None, report_print=False):
|