python-backoff 2.2.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,15 @@
1
+ __pycache__/
2
+ *.pyc
3
+ .coverage
4
+ virtualenv
5
+ MANIFEST
6
+ build/
7
+ dist/
8
+ .cache/
9
+ *.egg-info
10
+ .vscode
11
+ .tox/
12
+ coverage.xml
13
+
14
+ # mkdocs
15
+ site/
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2014 litl, LLC.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
@@ -0,0 +1,417 @@
1
+ Metadata-Version: 2.4
2
+ Name: python-backoff
3
+ Version: 2.2.2
4
+ Summary: Function decoration for backoff and retry
5
+ Project-URL: Documentation, https://python-backoff.github.io/backoff
6
+ Project-URL: Repository, https://github.com/python-backoff/backoff
7
+ Author-email: Bob Green <rgreen@aquent.com>
8
+ Maintainer-email: Edgar Ramírez-Mondragón <edgarrm358@gmail.com>
9
+ License-Expression: MIT
10
+ License-File: LICENSE
11
+ Keywords: backoff,decorators,retry
12
+ Classifier: Development Status :: 5 - Production/Stable
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Natural Language :: English
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python
18
+ Classifier: Programming Language :: Python :: 3
19
+ Classifier: Programming Language :: Python :: 3.7
20
+ Classifier: Programming Language :: Python :: 3.8
21
+ Classifier: Programming Language :: Python :: 3.9
22
+ Classifier: Programming Language :: Python :: 3.10
23
+ Classifier: Programming Language :: Python :: 3.11
24
+ Classifier: Programming Language :: Python :: 3.12
25
+ Classifier: Programming Language :: Python :: 3.13
26
+ Classifier: Programming Language :: Python :: 3.14
27
+ Classifier: Programming Language :: Python :: 3.15
28
+ Classifier: Topic :: Internet :: WWW/HTTP
29
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
30
+ Classifier: Topic :: Utilities
31
+ Requires-Python: >=3.7
32
+ Description-Content-Type: text/x-rst
33
+
34
+ backoff
35
+ =======
36
+
37
+ .. image:: https://github.com/python-backoff/backoff/actions/workflows/ci.yaml/badge.svg
38
+ :target: https://github.com/python-backoff/backoff/actions/workflows/ci.yaml
39
+ .. image:: https://github.com/python-backoff/backoff/workflows/CodeQL/badge.svg
40
+ :target: https://github.com/python-backoff/backoff/actions/workflows/codeql-analysis.yml
41
+ .. image:: https://img.shields.io/pypi/v/python-backoff.svg
42
+ :target: https://pypi.python.org/pypi/python-backoff
43
+ .. image:: https://img.shields.io/github/license/python-backoff/backoff
44
+ :target: https://github.com/python-backoff/backoff/blob/main/LICENSE
45
+
46
+ **Function decoration for backoff and retry**
47
+
48
+ This module provides function decorators which can be used to wrap a
49
+ function such that it will be retried until some condition is met. It
50
+ is meant to be of use when accessing unreliable resources with the
51
+ potential for intermittent failures i.e. network resources and external
52
+ APIs. Somewhat more generally, it may also be of use for dynamically
53
+ polling resources for externally generated content.
54
+
55
+ Decorators support both regular functions for synchronous code and
56
+ `asyncio <https://docs.python.org/3/library/asyncio.html>`__'s coroutines
57
+ for asynchronous code.
58
+
59
+ Examples
60
+ ========
61
+
62
+ Since Kenneth Reitz's `requests <http://python-requests.org>`_ module
63
+ has become a defacto standard for synchronous HTTP clients in Python,
64
+ networking examples below are written using it, but it is in no way required
65
+ by the backoff module.
66
+
67
+ @backoff.on_exception
68
+ ---------------------
69
+
70
+ The ``on_exception`` decorator is used to retry when a specified exception
71
+ is raised. Here's an example using exponential backoff when any
72
+ ``requests`` exception is raised:
73
+
74
+ .. code-block:: python
75
+
76
+ @backoff.on_exception(backoff.expo,
77
+ requests.exceptions.RequestException)
78
+ def get_url(url):
79
+ return requests.get(url)
80
+
81
+ The decorator will also accept a tuple of exceptions for cases where
82
+ the same backoff behavior is desired for more than one exception type:
83
+
84
+ .. code-block:: python
85
+
86
+ @backoff.on_exception(backoff.expo,
87
+ (requests.exceptions.Timeout,
88
+ requests.exceptions.ConnectionError))
89
+ def get_url(url):
90
+ return requests.get(url)
91
+
92
+ **Give Up Conditions**
93
+
94
+ Optional keyword arguments can specify conditions under which to give
95
+ up.
96
+
97
+ The keyword argument ``max_time`` specifies the maximum amount
98
+ of total time in seconds that can elapse before giving up.
99
+
100
+ .. code-block:: python
101
+
102
+ @backoff.on_exception(backoff.expo,
103
+ requests.exceptions.RequestException,
104
+ max_time=60)
105
+ def get_url(url):
106
+ return requests.get(url)
107
+
108
+
109
+ Keyword argument ``max_tries`` specifies the maximum number of calls
110
+ to make to the target function before giving up.
111
+
112
+ .. code-block:: python
113
+
114
+ @backoff.on_exception(backoff.expo,
115
+ requests.exceptions.RequestException,
116
+ max_tries=8,
117
+ jitter=None)
118
+ def get_url(url):
119
+ return requests.get(url)
120
+
121
+
122
+ In some cases the raised exception instance itself may need to be
123
+ inspected in order to determine if it is a retryable condition. The
124
+ ``giveup`` keyword arg can be used to specify a function which accepts
125
+ the exception and returns a truthy value if the exception should not
126
+ be retried:
127
+
128
+ .. code-block:: python
129
+
130
+ def fatal_code(e):
131
+ return 400 <= e.response.status_code < 500
132
+
133
+ @backoff.on_exception(backoff.expo,
134
+ requests.exceptions.RequestException,
135
+ max_time=300,
136
+ giveup=fatal_code)
137
+ def get_url(url):
138
+ return requests.get(url)
139
+
140
+ By default, when a give up event occurs, the exception in question is reraised
141
+ and so code calling an `on_exception`-decorated function may still
142
+ need to do exception handling. This behavior can optionally be disabled
143
+ using the `raise_on_giveup` keyword argument.
144
+
145
+ In the code below, `requests.exceptions.RequestException` will not be raised
146
+ when giveup occurs. Note that the decorated function will return `None` in this
147
+ case, regardless of the logic in the `on_exception` handler.
148
+
149
+ .. code-block:: python
150
+
151
+ def fatal_code(e):
152
+ return 400 <= e.response.status_code < 500
153
+
154
+ @backoff.on_exception(backoff.expo,
155
+ requests.exceptions.RequestException,
156
+ max_time=300,
157
+ raise_on_giveup=False,
158
+ giveup=fatal_code)
159
+ def get_url(url):
160
+ return requests.get(url)
161
+
162
+ This is useful for non-mission critical code where you still wish to retry
163
+ the code inside of `backoff.on_exception` but wish to proceed with execution
164
+ even if all retries fail.
165
+
166
+ @backoff.on_predicate
167
+ ---------------------
168
+
169
+ The ``on_predicate`` decorator is used to retry when a particular
170
+ condition is true of the return value of the target function. This may
171
+ be useful when polling a resource for externally generated content.
172
+
173
+ Here's an example which uses a fibonacci sequence backoff when the
174
+ return value of the target function is the empty list:
175
+
176
+ .. code-block:: python
177
+
178
+ @backoff.on_predicate(backoff.fibo, lambda x: x == [], max_value=13)
179
+ def poll_for_messages(queue):
180
+ return queue.get()
181
+
182
+ Extra keyword arguments are passed when initializing the
183
+ wait generator, so the ``max_value`` param above is passed as a keyword
184
+ arg when initializing the fibo generator.
185
+
186
+ When not specified, the predicate param defaults to the falsey test,
187
+ so the above can more concisely be written:
188
+
189
+ .. code-block:: python
190
+
191
+ @backoff.on_predicate(backoff.fibo, max_value=13)
192
+ def poll_for_message(queue):
193
+ return queue.get()
194
+
195
+ More simply, a function which continues polling every second until it
196
+ gets a non-falsey result could be defined like like this:
197
+
198
+ .. code-block:: python
199
+
200
+ @backoff.on_predicate(backoff.constant, jitter=None, interval=1)
201
+ def poll_for_message(queue):
202
+ return queue.get()
203
+
204
+ The jitter is disabled in order to keep the polling frequency fixed.
205
+
206
+ @backoff.runtime
207
+ ----------------
208
+
209
+ You can also use the ``backoff.runtime`` generator to make use of the
210
+ return value or thrown exception of the decorated method.
211
+
212
+ For example, to use the value in the ``Retry-After`` header of the response:
213
+
214
+ .. code-block:: python
215
+
216
+ @backoff.on_predicate(
217
+ backoff.runtime,
218
+ predicate=lambda r: r.status_code == 429,
219
+ value=lambda r: int(r.headers.get("Retry-After")),
220
+ jitter=None,
221
+ )
222
+ def get_url():
223
+ return requests.get(url)
224
+
225
+ Jitter
226
+ ------
227
+
228
+ A jitter algorithm can be supplied with the ``jitter`` keyword arg to
229
+ either of the backoff decorators. This argument should be a function
230
+ accepting the original unadulterated backoff value and returning it's
231
+ jittered counterpart.
232
+
233
+ As of version 1.2, the default jitter function ``backoff.full_jitter``
234
+ implements the 'Full Jitter' algorithm as defined in the AWS
235
+ Architecture Blog's `Exponential Backoff And Jitter
236
+ <https://www.awsarchitectureblog.com/2015/03/backoff.html>`_ post.
237
+ Note that with this algorithm, the time yielded by the wait generator
238
+ is actually the *maximum* amount of time to wait.
239
+
240
+ Previous versions of backoff defaulted to adding some random number of
241
+ milliseconds (up to 1s) to the raw sleep value. If desired, this
242
+ behavior is now available as ``backoff.random_jitter``.
243
+
244
+ Using multiple decorators
245
+ -------------------------
246
+
247
+ The backoff decorators may also be combined to specify different
248
+ backoff behavior for different cases:
249
+
250
+ .. code-block:: python
251
+
252
+ @backoff.on_predicate(backoff.fibo, max_value=13)
253
+ @backoff.on_exception(backoff.expo,
254
+ requests.exceptions.HTTPError,
255
+ max_time=60)
256
+ @backoff.on_exception(backoff.expo,
257
+ requests.exceptions.Timeout,
258
+ max_time=300)
259
+ def poll_for_message(queue):
260
+ return queue.get()
261
+
262
+
263
+ Runtime Configuration
264
+ ---------------------
265
+
266
+ The decorator functions ``on_exception`` and ``on_predicate`` are
267
+ generally evaluated at import time. This is fine when the keyword args
268
+ are passed as constant values, but suppose we want to consult a
269
+ dictionary with configuration options that only become available at
270
+ runtime. The relevant values are not available at import time. Instead,
271
+ decorator functions can be passed callables which are evaluated at
272
+ runtime to obtain the value:
273
+
274
+ .. code-block:: python
275
+
276
+ def lookup_max_time():
277
+ # pretend we have a global reference to 'app' here
278
+ # and that it has a dictionary-like 'config' property
279
+ return app.config["BACKOFF_MAX_TIME"]
280
+
281
+ @backoff.on_exception(backoff.expo,
282
+ ValueError,
283
+ max_time=lookup_max_time)
284
+
285
+ Event handlers
286
+ --------------
287
+
288
+ Both backoff decorators optionally accept event handler functions
289
+ using the keyword arguments ``on_success``, ``on_backoff``, and ``on_giveup``.
290
+ This may be useful in reporting statistics or performing other custom
291
+ logging.
292
+
293
+ Handlers must be callables with a unary signature accepting a dict
294
+ argument. This dict contains the details of the invocation. Valid keys
295
+ include:
296
+
297
+ * *target*: reference to the function or method being invoked
298
+ * *args*: positional arguments to func
299
+ * *kwargs*: keyword arguments to func
300
+ * *tries*: number of invocation tries so far
301
+ * *elapsed*: elapsed time in seconds so far
302
+ * *wait*: seconds to wait (``on_backoff`` handler only)
303
+ * *value*: value triggering backoff (``on_predicate`` decorator only)
304
+
305
+ A handler which prints the details of the backoff event could be
306
+ implemented like so:
307
+
308
+ .. code-block:: python
309
+
310
+ def backoff_hdlr(details):
311
+ print ("Backing off {wait:0.1f} seconds after {tries} tries "
312
+ "calling function {target} with args {args} and kwargs "
313
+ "{kwargs}".format(**details))
314
+
315
+ @backoff.on_exception(backoff.expo,
316
+ requests.exceptions.RequestException,
317
+ on_backoff=backoff_hdlr)
318
+ def get_url(url):
319
+ return requests.get(url)
320
+
321
+ **Multiple handlers per event type**
322
+
323
+ In all cases, iterables of handler functions are also accepted, which
324
+ are called in turn. For example, you might provide a simple list of
325
+ handler functions as the value of the ``on_backoff`` keyword arg:
326
+
327
+ .. code-block:: python
328
+
329
+ @backoff.on_exception(backoff.expo,
330
+ requests.exceptions.RequestException,
331
+ on_backoff=[backoff_hdlr1, backoff_hdlr2])
332
+ def get_url(url):
333
+ return requests.get(url)
334
+
335
+ **Getting exception info**
336
+
337
+ In the case of the ``on_exception`` decorator, all ``on_backoff`` and
338
+ ``on_giveup`` handlers are called from within the except block for the
339
+ exception being handled. Therefore exception info is available to the
340
+ handler functions via the python standard library, specifically
341
+ ``sys.exc_info()`` or the ``traceback`` module. The exception is also
342
+ available at the *exception* key in the `details` dict passed to the
343
+ handlers.
344
+
345
+ Asynchronous code
346
+ -----------------
347
+
348
+ Backoff supports asynchronous execution in Python 3.5 and above.
349
+
350
+ To use backoff in asynchronous code based on
351
+ `asyncio <https://docs.python.org/3/library/asyncio.html>`__
352
+ you simply need to apply ``backoff.on_exception`` or ``backoff.on_predicate``
353
+ to coroutines.
354
+ You can also use coroutines for the ``on_success``, ``on_backoff``, and
355
+ ``on_giveup`` event handlers, with the interface otherwise being identical.
356
+
357
+ The following examples use `aiohttp <https://aiohttp.readthedocs.io/>`__
358
+ asynchronous HTTP client/server library.
359
+
360
+ .. code-block:: python
361
+
362
+ @backoff.on_exception(backoff.expo, aiohttp.ClientError, max_time=60)
363
+ async def get_url(url):
364
+ async with aiohttp.ClientSession(raise_for_status=True) as session:
365
+ async with session.get(url) as response:
366
+ return await response.text()
367
+
368
+ Logging configuration
369
+ ---------------------
370
+
371
+ By default, backoff and retry attempts are logged to the 'backoff'
372
+ logger. By default, this logger is configured with a NullHandler, so
373
+ there will be nothing output unless you configure a handler.
374
+ Programmatically, this might be accomplished with something as simple
375
+ as:
376
+
377
+ .. code-block:: python
378
+
379
+ logging.getLogger('backoff').addHandler(logging.StreamHandler())
380
+
381
+ The default logging level is INFO, which corresponds to logging
382
+ anytime a retry event occurs. If you would instead like to log
383
+ only when a giveup event occurs, set the logger level to ERROR.
384
+
385
+ .. code-block:: python
386
+
387
+ logging.getLogger('backoff').setLevel(logging.ERROR)
388
+
389
+ It is also possible to specify an alternate logger with the ``logger``
390
+ keyword argument. If a string value is specified the logger will be
391
+ looked up by name.
392
+
393
+ .. code-block:: python
394
+
395
+ @backoff.on_exception(backoff.expo,
396
+ requests.exceptions.RequestException,
397
+ logger='my_logger')
398
+ # ...
399
+
400
+ It is also supported to specify a Logger (or LoggerAdapter) object
401
+ directly.
402
+
403
+ .. code-block:: python
404
+
405
+ my_logger = logging.getLogger('my_logger')
406
+ my_handler = logging.StreamHandler()
407
+ my_logger.addHandler(my_handler)
408
+ my_logger.setLevel(logging.ERROR)
409
+
410
+ @backoff.on_exception(backoff.expo,
411
+ requests.exceptions.RequestException,
412
+ logger=my_logger)
413
+ # ...
414
+
415
+ Default logging can be disabled all together by specifying
416
+ ``logger=None``. In this case, if desired alternative logging behavior
417
+ could be defined by using custom event handlers.