cachier 3.1.2__tar.gz → 3.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {cachier-3.1.2/src/cachier.egg-info → cachier-3.2.1}/PKG-INFO +63 -4
- {cachier-3.1.2 → cachier-3.2.1}/README.rst +59 -1
- {cachier-3.1.2 → cachier-3.2.1}/pyproject.toml +22 -2
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/__init__.py +2 -1
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/_version.py +2 -1
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/config.py +2 -3
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/core.py +13 -2
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/cores/base.py +1 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/cores/mongo.py +3 -1
- cachier-3.2.1/src/cachier/cores/sql.py +288 -0
- cachier-3.2.1/src/cachier/version.info +1 -0
- {cachier-3.1.2 → cachier-3.2.1/src/cachier.egg-info}/PKG-INFO +63 -4
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier.egg-info/SOURCES.txt +2 -1
- cachier-3.1.2/src/cachier/version.info +0 -1
- {cachier-3.1.2 → cachier-3.2.1}/LICENSE +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/MANIFEST.in +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/setup.cfg +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/__main__.py +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/_types.py +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/cores/__init__.py +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/cores/memory.py +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/cores/pickle.py +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier/py.typed +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier.egg-info/dependency_links.txt +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier.egg-info/entry_points.txt +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier.egg-info/requires.txt +0 -0
- {cachier-3.1.2 → cachier-3.2.1}/src/cachier.egg-info/top_level.txt +0 -0
@@ -1,6 +1,6 @@
|
|
1
|
-
Metadata-Version: 2.
|
1
|
+
Metadata-Version: 2.4
|
2
2
|
Name: cachier
|
3
|
-
Version: 3.1
|
3
|
+
Version: 3.2.1
|
4
4
|
Summary: Persistent, stale-free, local and cross-machine caching for Python functions.
|
5
5
|
Author-email: Shay Palachy Affek <shay.palachy@gmail.com>
|
6
6
|
License: MIT License
|
@@ -32,11 +32,11 @@ Classifier: Intended Audience :: Developers
|
|
32
32
|
Classifier: License :: OSI Approved :: MIT License
|
33
33
|
Classifier: Programming Language :: Python
|
34
34
|
Classifier: Programming Language :: Python :: 3 :: Only
|
35
|
-
Classifier: Programming Language :: Python :: 3.8
|
36
35
|
Classifier: Programming Language :: Python :: 3.9
|
37
36
|
Classifier: Programming Language :: Python :: 3.10
|
38
37
|
Classifier: Programming Language :: Python :: 3.11
|
39
38
|
Classifier: Programming Language :: Python :: 3.12
|
39
|
+
Classifier: Programming Language :: Python :: 3.13
|
40
40
|
Classifier: Topic :: Other/Nonlisted Topic
|
41
41
|
Classifier: Topic :: Software Development :: Libraries
|
42
42
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
@@ -45,6 +45,7 @@ Description-Content-Type: text/x-rst
|
|
45
45
|
License-File: LICENSE
|
46
46
|
Requires-Dist: portalocker>=2.3.2
|
47
47
|
Requires-Dist: watchdog>=2.3.1
|
48
|
+
Dynamic: license-file
|
48
49
|
|
49
50
|
Cachier
|
50
51
|
#######
|
@@ -92,7 +93,7 @@ Features
|
|
92
93
|
========
|
93
94
|
|
94
95
|
* Pure Python.
|
95
|
-
* Compatible with Python 3.
|
96
|
+
* Compatible with Python 3.9+ (Python 2.7 was discontinued in version 1.2.8).
|
96
97
|
* Supported and `tested on Linux, OS X and Windows <https://travis-ci.org/shaypal5/cachier>`_.
|
97
98
|
* A simple interface.
|
98
99
|
* Defining "shelf life" for cached values.
|
@@ -390,6 +391,64 @@ You can set an in-memory cache by assigning the ``backend`` parameter with ``'me
|
|
390
391
|
|
391
392
|
Note, however, that ``cachier``'s in-memory core is simple, and has no monitoring or cap on cache size, and can thus lead to memory errors on large return values - it is mainly intended to be used with future multi-core functionality. As a rule, Python's built-in ``lru_cache`` is a much better stand-alone solution.
|
392
393
|
|
394
|
+
SQLAlchemy (SQL) Core
|
395
|
+
---------------------
|
396
|
+
|
397
|
+
**Note:** The SQL core requires SQLAlchemy to be installed. It is not installed by default with cachier. To use the SQL backend, run::
|
398
|
+
|
399
|
+
pip install SQLAlchemy
|
400
|
+
|
401
|
+
Cachier supports a generic SQL backend via SQLAlchemy, allowing you to use SQLite, PostgreSQL, MySQL, and other databases.
|
402
|
+
|
403
|
+
**Usage Example (SQLite in-memory):**
|
404
|
+
|
405
|
+
.. code-block:: python
|
406
|
+
|
407
|
+
from cachier import cachier
|
408
|
+
|
409
|
+
@cachier(backend="sql", sql_engine="sqlite:///:memory:")
|
410
|
+
def my_func(x):
|
411
|
+
return x * 2
|
412
|
+
|
413
|
+
**Usage Example (PostgreSQL):**
|
414
|
+
|
415
|
+
.. code-block:: python
|
416
|
+
|
417
|
+
@cachier(backend="sql", sql_engine="postgresql://user:pass@localhost/dbname")
|
418
|
+
def my_func(x):
|
419
|
+
return x * 2
|
420
|
+
|
421
|
+
**Usage Example (MySQL):**
|
422
|
+
|
423
|
+
.. code-block:: python
|
424
|
+
|
425
|
+
@cachier(backend="sql", sql_engine="mysql+pymysql://user:pass@localhost/dbname")
|
426
|
+
def my_func(x):
|
427
|
+
return x * 2
|
428
|
+
|
429
|
+
**Configuration Options:**
|
430
|
+
|
431
|
+
- ``sql_engine``: SQLAlchemy connection string, Engine, or callable returning an Engine.
|
432
|
+
- All other standard cachier options are supported.
|
433
|
+
|
434
|
+
**Table Schema:**
|
435
|
+
|
436
|
+
- ``function_id``: Unique identifier for the cached function
|
437
|
+
- ``key``: Cache key
|
438
|
+
- ``value``: Pickled result
|
439
|
+
- ``timestamp``: Datetime of cache entry
|
440
|
+
- ``stale``: Boolean, is value stale
|
441
|
+
- ``processing``: Boolean, is value being calculated
|
442
|
+
- ``completed``: Boolean, is value calculation completed
|
443
|
+
|
444
|
+
**Limitations & Notes:**
|
445
|
+
|
446
|
+
- Requires SQLAlchemy (install with ``pip install SQLAlchemy``)
|
447
|
+
- For production, use a persistent database (not ``:memory:``)
|
448
|
+
- Thread/process safety is handled via transactions and row-level locks
|
449
|
+
- Value serialization uses ``pickle``. **Warning:** `pickle` can execute arbitrary code during deserialization if the cache database is compromised. Ensure the cache is stored securely and consider using safer serialization methods like `json` if security is a concern.
|
450
|
+
- For best performance, ensure your DB supports row-level locking
|
451
|
+
|
393
452
|
|
394
453
|
Contributing
|
395
454
|
============
|
@@ -44,7 +44,7 @@ Features
|
|
44
44
|
========
|
45
45
|
|
46
46
|
* Pure Python.
|
47
|
-
* Compatible with Python 3.
|
47
|
+
* Compatible with Python 3.9+ (Python 2.7 was discontinued in version 1.2.8).
|
48
48
|
* Supported and `tested on Linux, OS X and Windows <https://travis-ci.org/shaypal5/cachier>`_.
|
49
49
|
* A simple interface.
|
50
50
|
* Defining "shelf life" for cached values.
|
@@ -342,6 +342,64 @@ You can set an in-memory cache by assigning the ``backend`` parameter with ``'me
|
|
342
342
|
|
343
343
|
Note, however, that ``cachier``'s in-memory core is simple, and has no monitoring or cap on cache size, and can thus lead to memory errors on large return values - it is mainly intended to be used with future multi-core functionality. As a rule, Python's built-in ``lru_cache`` is a much better stand-alone solution.
|
344
344
|
|
345
|
+
SQLAlchemy (SQL) Core
|
346
|
+
---------------------
|
347
|
+
|
348
|
+
**Note:** The SQL core requires SQLAlchemy to be installed. It is not installed by default with cachier. To use the SQL backend, run::
|
349
|
+
|
350
|
+
pip install SQLAlchemy
|
351
|
+
|
352
|
+
Cachier supports a generic SQL backend via SQLAlchemy, allowing you to use SQLite, PostgreSQL, MySQL, and other databases.
|
353
|
+
|
354
|
+
**Usage Example (SQLite in-memory):**
|
355
|
+
|
356
|
+
.. code-block:: python
|
357
|
+
|
358
|
+
from cachier import cachier
|
359
|
+
|
360
|
+
@cachier(backend="sql", sql_engine="sqlite:///:memory:")
|
361
|
+
def my_func(x):
|
362
|
+
return x * 2
|
363
|
+
|
364
|
+
**Usage Example (PostgreSQL):**
|
365
|
+
|
366
|
+
.. code-block:: python
|
367
|
+
|
368
|
+
@cachier(backend="sql", sql_engine="postgresql://user:pass@localhost/dbname")
|
369
|
+
def my_func(x):
|
370
|
+
return x * 2
|
371
|
+
|
372
|
+
**Usage Example (MySQL):**
|
373
|
+
|
374
|
+
.. code-block:: python
|
375
|
+
|
376
|
+
@cachier(backend="sql", sql_engine="mysql+pymysql://user:pass@localhost/dbname")
|
377
|
+
def my_func(x):
|
378
|
+
return x * 2
|
379
|
+
|
380
|
+
**Configuration Options:**
|
381
|
+
|
382
|
+
- ``sql_engine``: SQLAlchemy connection string, Engine, or callable returning an Engine.
|
383
|
+
- All other standard cachier options are supported.
|
384
|
+
|
385
|
+
**Table Schema:**
|
386
|
+
|
387
|
+
- ``function_id``: Unique identifier for the cached function
|
388
|
+
- ``key``: Cache key
|
389
|
+
- ``value``: Pickled result
|
390
|
+
- ``timestamp``: Datetime of cache entry
|
391
|
+
- ``stale``: Boolean, is value stale
|
392
|
+
- ``processing``: Boolean, is value being calculated
|
393
|
+
- ``completed``: Boolean, is value calculation completed
|
394
|
+
|
395
|
+
**Limitations & Notes:**
|
396
|
+
|
397
|
+
- Requires SQLAlchemy (install with ``pip install SQLAlchemy``)
|
398
|
+
- For production, use a persistent database (not ``:memory:``)
|
399
|
+
- Thread/process safety is handled via transactions and row-level locks
|
400
|
+
- Value serialization uses ``pickle``. **Warning:** `pickle` can execute arbitrary code during deserialization if the cache database is compromised. Ensure the cache is stored securely and consider using safer serialization methods like `json` if security is a concern.
|
401
|
+
- For best performance, ensure your DB supports row-level locking
|
402
|
+
|
345
403
|
|
346
404
|
Contributing
|
347
405
|
============
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# === Metadata & Build System ===
|
2
|
+
|
1
3
|
[build-system]
|
2
4
|
requires = [
|
3
5
|
"setuptools",
|
@@ -28,11 +30,11 @@ classifiers = [
|
|
28
30
|
"License :: OSI Approved :: MIT License",
|
29
31
|
"Programming Language :: Python",
|
30
32
|
"Programming Language :: Python :: 3 :: Only",
|
31
|
-
"Programming Language :: Python :: 3.8",
|
32
33
|
"Programming Language :: Python :: 3.9",
|
33
34
|
"Programming Language :: Python :: 3.10",
|
34
35
|
"Programming Language :: Python :: 3.11",
|
35
36
|
"Programming Language :: Python :: 3.12",
|
37
|
+
"Programming Language :: Python :: 3.13",
|
36
38
|
"Topic :: Other/Nonlisted Topic",
|
37
39
|
"Topic :: Software Development :: Libraries",
|
38
40
|
"Topic :: Software Development :: Libraries :: Python Modules",
|
@@ -46,6 +48,8 @@ dependencies = [
|
|
46
48
|
"watchdog>=2.3.1",
|
47
49
|
]
|
48
50
|
urls.Source = "https://github.com/python-cachier/cachier"
|
51
|
+
# --- setuptools ---
|
52
|
+
|
49
53
|
scripts.cachier = "cachier.__main__:cli"
|
50
54
|
|
51
55
|
[tool.setuptools]
|
@@ -63,6 +67,13 @@ include = [
|
|
63
67
|
] # package names should match these glob patterns (["*"] by default)
|
64
68
|
namespaces = false # to disable scanning PEP 420 namespaces (true by default)
|
65
69
|
|
70
|
+
# === Linting & Formatting ===
|
71
|
+
|
72
|
+
[tool.black]
|
73
|
+
line-length = 79
|
74
|
+
|
75
|
+
# --- ruff ---
|
76
|
+
|
66
77
|
[tool.ruff]
|
67
78
|
target-version = "py38"
|
68
79
|
line-length = 79
|
@@ -104,6 +115,7 @@ lint.per-file-ignores."src/**/__init__.py" = [
|
|
104
115
|
lint.per-file-ignores."src/cachier/config.py" = [
|
105
116
|
"D100",
|
106
117
|
]
|
118
|
+
lint.per-file-ignores."src/cachier/cores/sql.py" = [ "S301" ]
|
107
119
|
lint.per-file-ignores."tests/**" = [
|
108
120
|
"D100",
|
109
121
|
"D101",
|
@@ -119,6 +131,7 @@ lint.unfixable = [
|
|
119
131
|
"F401",
|
120
132
|
]
|
121
133
|
|
134
|
+
# --- flake8 ---
|
122
135
|
#[tool.ruff.pydocstyle]
|
123
136
|
## Use Google-style docstrings.
|
124
137
|
#convention = "google"
|
@@ -134,6 +147,10 @@ wrap-summaries = 79
|
|
134
147
|
wrap-descriptions = 79
|
135
148
|
blank = true
|
136
149
|
|
150
|
+
# === Testing ===
|
151
|
+
|
152
|
+
# --- pytest ---
|
153
|
+
|
137
154
|
[tool.pytest.ini_options]
|
138
155
|
testpaths = [
|
139
156
|
"cachier",
|
@@ -154,11 +171,14 @@ markers = [
|
|
154
171
|
"mongo: test the MongoDB core",
|
155
172
|
"memory: test the memory core",
|
156
173
|
"pickle: test the pickle core",
|
174
|
+
"sql: test the SQL core",
|
157
175
|
]
|
158
176
|
|
177
|
+
# --- coverage ---
|
178
|
+
|
159
179
|
[tool.coverage.run]
|
160
180
|
branch = true
|
161
|
-
dynamic_context = "test_function"
|
181
|
+
# dynamic_context = "test_function"
|
162
182
|
omit = [
|
163
183
|
"tests/*",
|
164
184
|
"cachier/_version.py",
|
@@ -18,7 +18,8 @@ with open(_PATH_VERSION) as fopen:
|
|
18
18
|
def _get_git_sha() -> str:
|
19
19
|
from subprocess import DEVNULL, check_output
|
20
20
|
|
21
|
-
|
21
|
+
args = ["git", "rev-parse", "--short", "HEAD"]
|
22
|
+
out = check_output(args, stderr=DEVNULL) # noqa: S603
|
22
23
|
return out.decode("utf-8").strip()
|
23
24
|
|
24
25
|
|
@@ -2,7 +2,6 @@ import hashlib
|
|
2
2
|
import os
|
3
3
|
import pickle
|
4
4
|
import threading
|
5
|
-
from collections.abc import Mapping
|
6
5
|
from dataclasses import dataclass, replace
|
7
6
|
from datetime import datetime, timedelta
|
8
7
|
from typing import Any, Optional, Union
|
@@ -65,7 +64,7 @@ def _update_with_defaults(
|
|
65
64
|
return param
|
66
65
|
|
67
66
|
|
68
|
-
def set_default_params(**params:
|
67
|
+
def set_default_params(**params: Any) -> None:
|
69
68
|
"""Configure default parameters applicable to all memoized functions."""
|
70
69
|
# It is kept for backwards compatibility with desperation warning
|
71
70
|
import warnings
|
@@ -79,7 +78,7 @@ def set_default_params(**params: Mapping) -> None:
|
|
79
78
|
set_global_params(**params)
|
80
79
|
|
81
80
|
|
82
|
-
def set_global_params(**params:
|
81
|
+
def set_global_params(**params: Any) -> None:
|
83
82
|
"""Configure global parameters applicable to all memoized functions.
|
84
83
|
|
85
84
|
This function takes the same keyword parameters as the ones defined in the
|
@@ -14,7 +14,7 @@ from collections import OrderedDict
|
|
14
14
|
from concurrent.futures import ThreadPoolExecutor
|
15
15
|
from datetime import datetime, timedelta
|
16
16
|
from functools import wraps
|
17
|
-
from typing import Any, Optional, Union
|
17
|
+
from typing import Any, Callable, Optional, Union
|
18
18
|
from warnings import warn
|
19
19
|
|
20
20
|
from .config import (
|
@@ -27,6 +27,7 @@ from .cores.base import RecalculationNeeded, _BaseCore
|
|
27
27
|
from .cores.memory import _MemoryCore
|
28
28
|
from .cores.mongo import _MongoCore
|
29
29
|
from .cores.pickle import _PickleCore
|
30
|
+
from .cores.sql import _SQLCore
|
30
31
|
|
31
32
|
MAX_WORKERS_ENVAR_NAME = "CACHIER_MAX_WORKERS"
|
32
33
|
DEFAULT_MAX_WORKERS = 8
|
@@ -107,6 +108,7 @@ def cachier(
|
|
107
108
|
hash_params: Optional[HashFunc] = None,
|
108
109
|
backend: Optional[Backend] = None,
|
109
110
|
mongetter: Optional[Mongetter] = None,
|
111
|
+
sql_engine: Optional[Union[str, Any, Callable[[], Any]]] = None,
|
110
112
|
stale_after: Optional[timedelta] = None,
|
111
113
|
next_time: Optional[bool] = None,
|
112
114
|
cache_dir: Optional[Union[str, os.PathLike]] = None,
|
@@ -134,13 +136,16 @@ def cachier(
|
|
134
136
|
hash_params : callable, optional
|
135
137
|
backend : str, optional
|
136
138
|
The name of the backend to use. Valid options currently include
|
137
|
-
'pickle', 'mongo' and '
|
139
|
+
'pickle', 'mongo', 'memory', and 'sql'. If not provided, defaults to
|
138
140
|
'pickle' unless the 'mongetter' argument is passed, in which
|
139
141
|
case the mongo backend is automatically selected.
|
140
142
|
mongetter : callable, optional
|
141
143
|
A callable that takes no arguments and returns a pymongo.Collection
|
142
144
|
object with writing permissions. If unset a local pickle cache is used
|
143
145
|
instead.
|
146
|
+
sql_engine : str, Engine, or callable, optional
|
147
|
+
SQLAlchemy connection string, Engine, or callable returning an Engine.
|
148
|
+
Used for the SQL backend.
|
144
149
|
stale_after : datetime.timedelta, optional
|
145
150
|
The time delta after which a cached result is considered stale. Calls
|
146
151
|
made after the result goes stale will trigger a recalculation of the
|
@@ -208,6 +213,12 @@ def cachier(
|
|
208
213
|
core = _MemoryCore(
|
209
214
|
hash_func=hash_func, wait_for_calc_timeout=wait_for_calc_timeout
|
210
215
|
)
|
216
|
+
elif backend == "sql":
|
217
|
+
core = _SQLCore(
|
218
|
+
hash_func=hash_func,
|
219
|
+
sql_engine=sql_engine,
|
220
|
+
wait_for_calc_timeout=wait_for_calc_timeout,
|
221
|
+
)
|
211
222
|
else:
|
212
223
|
raise ValueError("specified an invalid core: %s" % backend)
|
213
224
|
|
@@ -75,7 +75,9 @@ class _MongoCore(_BaseCore):
|
|
75
75
|
)
|
76
76
|
if not res:
|
77
77
|
return key, None
|
78
|
-
val =
|
78
|
+
val = None
|
79
|
+
if "value" in res:
|
80
|
+
val = pickle.loads(res["value"]) # noqa: S301
|
79
81
|
entry = CacheEntry(
|
80
82
|
value=val,
|
81
83
|
time=res.get("time", None),
|
@@ -0,0 +1,288 @@
|
|
1
|
+
"""A SQLAlchemy-based caching core for cachier."""
|
2
|
+
|
3
|
+
import pickle
|
4
|
+
import threading
|
5
|
+
from datetime import datetime
|
6
|
+
from typing import Any, Callable, Optional, Tuple, Union
|
7
|
+
|
8
|
+
try:
|
9
|
+
from sqlalchemy import (
|
10
|
+
Boolean,
|
11
|
+
Column,
|
12
|
+
DateTime,
|
13
|
+
Index,
|
14
|
+
LargeBinary,
|
15
|
+
String,
|
16
|
+
and_,
|
17
|
+
create_engine,
|
18
|
+
delete,
|
19
|
+
insert,
|
20
|
+
select,
|
21
|
+
update,
|
22
|
+
)
|
23
|
+
from sqlalchemy.engine import Engine
|
24
|
+
from sqlalchemy.orm import declarative_base, sessionmaker
|
25
|
+
|
26
|
+
SQLALCHEMY_AVAILABLE = True
|
27
|
+
except ImportError:
|
28
|
+
SQLALCHEMY_AVAILABLE = False
|
29
|
+
|
30
|
+
from .._types import HashFunc
|
31
|
+
from ..config import CacheEntry
|
32
|
+
from .base import RecalculationNeeded, _BaseCore, _get_func_str
|
33
|
+
|
34
|
+
if SQLALCHEMY_AVAILABLE:
|
35
|
+
Base = declarative_base()
|
36
|
+
|
37
|
+
class CacheTable(Base): # type: ignore[misc, valid-type]
|
38
|
+
"""SQLAlchemy model for cachier cache entries."""
|
39
|
+
|
40
|
+
__tablename__ = "cachier_cache"
|
41
|
+
id = Column(String, primary_key=True)
|
42
|
+
function_id = Column(String, index=True, nullable=False)
|
43
|
+
key = Column(String, index=True, nullable=False)
|
44
|
+
value = Column(LargeBinary, nullable=True)
|
45
|
+
timestamp = Column(DateTime, nullable=False)
|
46
|
+
stale = Column(Boolean, default=False)
|
47
|
+
processing = Column(Boolean, default=False)
|
48
|
+
completed = Column(Boolean, default=False)
|
49
|
+
__table_args__ = (
|
50
|
+
Index("ix_func_key", "function_id", "key", unique=True),
|
51
|
+
)
|
52
|
+
|
53
|
+
|
54
|
+
class _SQLCore(_BaseCore):
|
55
|
+
"""SQLAlchemy-based core for Cachier, supporting SQL-based backends.
|
56
|
+
|
57
|
+
This should work with SQLite, PostgreSQL and so on.
|
58
|
+
|
59
|
+
"""
|
60
|
+
|
61
|
+
def __init__(
|
62
|
+
self,
|
63
|
+
hash_func: Optional[HashFunc],
|
64
|
+
sql_engine: Optional[Union[str, "Engine", Callable[[], "Engine"]]],
|
65
|
+
wait_for_calc_timeout: Optional[int] = None,
|
66
|
+
):
|
67
|
+
if not SQLALCHEMY_AVAILABLE:
|
68
|
+
raise ImportError(
|
69
|
+
"SQLAlchemy is required for the SQL core. "
|
70
|
+
"Install with `pip install SQLAlchemy`."
|
71
|
+
)
|
72
|
+
super().__init__(
|
73
|
+
hash_func=hash_func, wait_for_calc_timeout=wait_for_calc_timeout
|
74
|
+
)
|
75
|
+
self._engine = self._resolve_engine(sql_engine)
|
76
|
+
self._Session = sessionmaker(bind=self._engine)
|
77
|
+
Base.metadata.create_all(self._engine)
|
78
|
+
self._lock = threading.RLock()
|
79
|
+
self._func_str = None
|
80
|
+
|
81
|
+
def _resolve_engine(self, sql_engine):
|
82
|
+
if isinstance(sql_engine, Engine):
|
83
|
+
return sql_engine
|
84
|
+
if isinstance(sql_engine, str):
|
85
|
+
return create_engine(sql_engine, future=True)
|
86
|
+
if callable(sql_engine):
|
87
|
+
return sql_engine()
|
88
|
+
raise ValueError(
|
89
|
+
"sql_engine must be a SQLAlchemy Engine, connection string, "
|
90
|
+
"or callable returning an Engine."
|
91
|
+
)
|
92
|
+
|
93
|
+
def set_func(self, func):
|
94
|
+
super().set_func(func)
|
95
|
+
self._func_str = _get_func_str(func)
|
96
|
+
|
97
|
+
def get_entry_by_key(self, key: str) -> Tuple[str, Optional[CacheEntry]]:
|
98
|
+
with self._lock, self._Session() as session:
|
99
|
+
row = session.execute(
|
100
|
+
select(CacheTable).where(
|
101
|
+
and_(
|
102
|
+
CacheTable.function_id == self._func_str,
|
103
|
+
CacheTable.key == key,
|
104
|
+
)
|
105
|
+
)
|
106
|
+
).scalar_one_or_none()
|
107
|
+
if not row:
|
108
|
+
return key, None
|
109
|
+
value = pickle.loads(row.value) if row.value is not None else None
|
110
|
+
entry = CacheEntry(
|
111
|
+
value=value,
|
112
|
+
time=row.timestamp,
|
113
|
+
stale=row.stale,
|
114
|
+
_processing=row.processing,
|
115
|
+
_completed=row.completed,
|
116
|
+
)
|
117
|
+
return key, entry
|
118
|
+
|
119
|
+
def set_entry(self, key: str, func_res: Any) -> None:
|
120
|
+
with self._lock, self._Session() as session:
|
121
|
+
thebytes = pickle.dumps(func_res)
|
122
|
+
now = datetime.now()
|
123
|
+
base_insert = insert(CacheTable)
|
124
|
+
stmt = (
|
125
|
+
base_insert.values(
|
126
|
+
id=f"{self._func_str}:{key}",
|
127
|
+
function_id=self._func_str,
|
128
|
+
key=key,
|
129
|
+
value=thebytes,
|
130
|
+
timestamp=now,
|
131
|
+
stale=False,
|
132
|
+
processing=False,
|
133
|
+
completed=True,
|
134
|
+
).on_conflict_do_update(
|
135
|
+
index_elements=[CacheTable.function_id, CacheTable.key],
|
136
|
+
set_={
|
137
|
+
"value": thebytes,
|
138
|
+
"timestamp": now,
|
139
|
+
"stale": False,
|
140
|
+
"processing": False,
|
141
|
+
"completed": True,
|
142
|
+
},
|
143
|
+
)
|
144
|
+
if hasattr(base_insert, "on_conflict_do_update")
|
145
|
+
else None
|
146
|
+
)
|
147
|
+
# Fallback for non-SQLite/Postgres: try update, else insert
|
148
|
+
if stmt:
|
149
|
+
session.execute(stmt)
|
150
|
+
else:
|
151
|
+
row = session.execute(
|
152
|
+
select(CacheTable).where(
|
153
|
+
and_(
|
154
|
+
CacheTable.function_id == self._func_str,
|
155
|
+
CacheTable.key == key,
|
156
|
+
)
|
157
|
+
)
|
158
|
+
).scalar_one_or_none()
|
159
|
+
if row:
|
160
|
+
session.execute(
|
161
|
+
update(CacheTable)
|
162
|
+
.where(
|
163
|
+
and_(
|
164
|
+
CacheTable.function_id == self._func_str,
|
165
|
+
CacheTable.key == key,
|
166
|
+
)
|
167
|
+
)
|
168
|
+
.values(
|
169
|
+
value=thebytes,
|
170
|
+
timestamp=now,
|
171
|
+
stale=False,
|
172
|
+
processing=False,
|
173
|
+
completed=True,
|
174
|
+
)
|
175
|
+
)
|
176
|
+
else:
|
177
|
+
session.add(
|
178
|
+
CacheTable(
|
179
|
+
id=f"{self._func_str}:{key}",
|
180
|
+
function_id=self._func_str,
|
181
|
+
key=key,
|
182
|
+
value=thebytes,
|
183
|
+
timestamp=now,
|
184
|
+
stale=False,
|
185
|
+
processing=False,
|
186
|
+
completed=True,
|
187
|
+
)
|
188
|
+
)
|
189
|
+
session.commit()
|
190
|
+
|
191
|
+
def mark_entry_being_calculated(self, key: str) -> None:
|
192
|
+
with self._lock, self._Session() as session:
|
193
|
+
row = session.execute(
|
194
|
+
select(CacheTable).where(
|
195
|
+
and_(
|
196
|
+
CacheTable.function_id == self._func_str,
|
197
|
+
CacheTable.key == key,
|
198
|
+
)
|
199
|
+
)
|
200
|
+
).scalar_one_or_none()
|
201
|
+
if row:
|
202
|
+
session.execute(
|
203
|
+
update(CacheTable)
|
204
|
+
.where(
|
205
|
+
and_(
|
206
|
+
CacheTable.function_id == self._func_str,
|
207
|
+
CacheTable.key == key,
|
208
|
+
)
|
209
|
+
)
|
210
|
+
.values(processing=True)
|
211
|
+
)
|
212
|
+
else:
|
213
|
+
session.add(
|
214
|
+
CacheTable(
|
215
|
+
id=f"{self._func_str}:{key}",
|
216
|
+
function_id=self._func_str,
|
217
|
+
key=key,
|
218
|
+
value=None,
|
219
|
+
timestamp=datetime.now(),
|
220
|
+
stale=False,
|
221
|
+
processing=True,
|
222
|
+
completed=False,
|
223
|
+
)
|
224
|
+
)
|
225
|
+
session.commit()
|
226
|
+
|
227
|
+
def mark_entry_not_calculated(self, key: str) -> None:
|
228
|
+
with self._lock, self._Session() as session:
|
229
|
+
session.execute(
|
230
|
+
update(CacheTable)
|
231
|
+
.where(
|
232
|
+
and_(
|
233
|
+
CacheTable.function_id == self._func_str,
|
234
|
+
CacheTable.key == key,
|
235
|
+
)
|
236
|
+
)
|
237
|
+
.values(processing=False)
|
238
|
+
)
|
239
|
+
session.commit()
|
240
|
+
|
241
|
+
def wait_on_entry_calc(self, key: str) -> Any:
|
242
|
+
import time
|
243
|
+
|
244
|
+
time_spent = 0
|
245
|
+
while True:
|
246
|
+
with self._lock, self._Session() as session:
|
247
|
+
row = session.execute(
|
248
|
+
select(CacheTable).where(
|
249
|
+
and_(
|
250
|
+
CacheTable.function_id == self._func_str,
|
251
|
+
CacheTable.key == key,
|
252
|
+
)
|
253
|
+
)
|
254
|
+
).scalar_one_or_none()
|
255
|
+
if not row:
|
256
|
+
raise RecalculationNeeded()
|
257
|
+
if not row.processing:
|
258
|
+
return (
|
259
|
+
pickle.loads(row.value)
|
260
|
+
if row.value is not None
|
261
|
+
else None
|
262
|
+
)
|
263
|
+
time.sleep(1)
|
264
|
+
time_spent += 1
|
265
|
+
self.check_calc_timeout(time_spent)
|
266
|
+
|
267
|
+
def clear_cache(self) -> None:
|
268
|
+
with self._lock, self._Session() as session:
|
269
|
+
session.execute(
|
270
|
+
delete(CacheTable).where(
|
271
|
+
CacheTable.function_id == self._func_str
|
272
|
+
)
|
273
|
+
)
|
274
|
+
session.commit()
|
275
|
+
|
276
|
+
def clear_being_calculated(self) -> None:
|
277
|
+
with self._lock, self._Session() as session:
|
278
|
+
session.execute(
|
279
|
+
update(CacheTable)
|
280
|
+
.where(
|
281
|
+
and_(
|
282
|
+
CacheTable.function_id == self._func_str,
|
283
|
+
CacheTable.processing,
|
284
|
+
)
|
285
|
+
)
|
286
|
+
.values(processing=False)
|
287
|
+
)
|
288
|
+
session.commit()
|
@@ -0,0 +1 @@
|
|
1
|
+
3.2.1
|
@@ -1,6 +1,6 @@
|
|
1
|
-
Metadata-Version: 2.
|
1
|
+
Metadata-Version: 2.4
|
2
2
|
Name: cachier
|
3
|
-
Version: 3.1
|
3
|
+
Version: 3.2.1
|
4
4
|
Summary: Persistent, stale-free, local and cross-machine caching for Python functions.
|
5
5
|
Author-email: Shay Palachy Affek <shay.palachy@gmail.com>
|
6
6
|
License: MIT License
|
@@ -32,11 +32,11 @@ Classifier: Intended Audience :: Developers
|
|
32
32
|
Classifier: License :: OSI Approved :: MIT License
|
33
33
|
Classifier: Programming Language :: Python
|
34
34
|
Classifier: Programming Language :: Python :: 3 :: Only
|
35
|
-
Classifier: Programming Language :: Python :: 3.8
|
36
35
|
Classifier: Programming Language :: Python :: 3.9
|
37
36
|
Classifier: Programming Language :: Python :: 3.10
|
38
37
|
Classifier: Programming Language :: Python :: 3.11
|
39
38
|
Classifier: Programming Language :: Python :: 3.12
|
39
|
+
Classifier: Programming Language :: Python :: 3.13
|
40
40
|
Classifier: Topic :: Other/Nonlisted Topic
|
41
41
|
Classifier: Topic :: Software Development :: Libraries
|
42
42
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
@@ -45,6 +45,7 @@ Description-Content-Type: text/x-rst
|
|
45
45
|
License-File: LICENSE
|
46
46
|
Requires-Dist: portalocker>=2.3.2
|
47
47
|
Requires-Dist: watchdog>=2.3.1
|
48
|
+
Dynamic: license-file
|
48
49
|
|
49
50
|
Cachier
|
50
51
|
#######
|
@@ -92,7 +93,7 @@ Features
|
|
92
93
|
========
|
93
94
|
|
94
95
|
* Pure Python.
|
95
|
-
* Compatible with Python 3.
|
96
|
+
* Compatible with Python 3.9+ (Python 2.7 was discontinued in version 1.2.8).
|
96
97
|
* Supported and `tested on Linux, OS X and Windows <https://travis-ci.org/shaypal5/cachier>`_.
|
97
98
|
* A simple interface.
|
98
99
|
* Defining "shelf life" for cached values.
|
@@ -390,6 +391,64 @@ You can set an in-memory cache by assigning the ``backend`` parameter with ``'me
|
|
390
391
|
|
391
392
|
Note, however, that ``cachier``'s in-memory core is simple, and has no monitoring or cap on cache size, and can thus lead to memory errors on large return values - it is mainly intended to be used with future multi-core functionality. As a rule, Python's built-in ``lru_cache`` is a much better stand-alone solution.
|
392
393
|
|
394
|
+
SQLAlchemy (SQL) Core
|
395
|
+
---------------------
|
396
|
+
|
397
|
+
**Note:** The SQL core requires SQLAlchemy to be installed. It is not installed by default with cachier. To use the SQL backend, run::
|
398
|
+
|
399
|
+
pip install SQLAlchemy
|
400
|
+
|
401
|
+
Cachier supports a generic SQL backend via SQLAlchemy, allowing you to use SQLite, PostgreSQL, MySQL, and other databases.
|
402
|
+
|
403
|
+
**Usage Example (SQLite in-memory):**
|
404
|
+
|
405
|
+
.. code-block:: python
|
406
|
+
|
407
|
+
from cachier import cachier
|
408
|
+
|
409
|
+
@cachier(backend="sql", sql_engine="sqlite:///:memory:")
|
410
|
+
def my_func(x):
|
411
|
+
return x * 2
|
412
|
+
|
413
|
+
**Usage Example (PostgreSQL):**
|
414
|
+
|
415
|
+
.. code-block:: python
|
416
|
+
|
417
|
+
@cachier(backend="sql", sql_engine="postgresql://user:pass@localhost/dbname")
|
418
|
+
def my_func(x):
|
419
|
+
return x * 2
|
420
|
+
|
421
|
+
**Usage Example (MySQL):**
|
422
|
+
|
423
|
+
.. code-block:: python
|
424
|
+
|
425
|
+
@cachier(backend="sql", sql_engine="mysql+pymysql://user:pass@localhost/dbname")
|
426
|
+
def my_func(x):
|
427
|
+
return x * 2
|
428
|
+
|
429
|
+
**Configuration Options:**
|
430
|
+
|
431
|
+
- ``sql_engine``: SQLAlchemy connection string, Engine, or callable returning an Engine.
|
432
|
+
- All other standard cachier options are supported.
|
433
|
+
|
434
|
+
**Table Schema:**
|
435
|
+
|
436
|
+
- ``function_id``: Unique identifier for the cached function
|
437
|
+
- ``key``: Cache key
|
438
|
+
- ``value``: Pickled result
|
439
|
+
- ``timestamp``: Datetime of cache entry
|
440
|
+
- ``stale``: Boolean, is value stale
|
441
|
+
- ``processing``: Boolean, is value being calculated
|
442
|
+
- ``completed``: Boolean, is value calculation completed
|
443
|
+
|
444
|
+
**Limitations & Notes:**
|
445
|
+
|
446
|
+
- Requires SQLAlchemy (install with ``pip install SQLAlchemy``)
|
447
|
+
- For production, use a persistent database (not ``:memory:``)
|
448
|
+
- Thread/process safety is handled via transactions and row-level locks
|
449
|
+
- Value serialization uses ``pickle``. **Warning:** `pickle` can execute arbitrary code during deserialization if the cache database is compromised. Ensure the cache is stored securely and consider using safer serialization methods like `json` if security is a concern.
|
450
|
+
- For best performance, ensure your DB supports row-level locking
|
451
|
+
|
393
452
|
|
394
453
|
Contributing
|
395
454
|
============
|
@@ -1 +0,0 @@
|
|
1
|
-
3.1.2
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|