sop4py 2.0.0__tar.gz → 2.0.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of sop4py might be problematic. Click here for more details.

Files changed (28) hide show
  1. {sop4py-2.0.0/sop4py.egg-info → sop4py-2.0.2}/PKG-INFO +32 -23
  2. {sop4py-2.0.0 → sop4py-2.0.2}/README.md +31 -12
  3. {sop4py-2.0.0 → sop4py-2.0.2}/pyproject.toml +3 -14
  4. {sop4py-2.0.0 → sop4py-2.0.2}/sop/btree.py +9 -11
  5. {sop4py-2.0.0 → sop4py-2.0.2}/sop/call_go.py +3 -1
  6. {sop4py-2.0.0 → sop4py-2.0.2}/sop/context.py +1 -1
  7. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64darwin.dylib +0 -0
  8. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64linux.so +0 -0
  9. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64windows.dll +0 -0
  10. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_arm64darwin.dylib +0 -0
  11. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_arm64linux.so +0 -0
  12. {sop4py-2.0.0 → sop4py-2.0.2}/sop/redis.py +1 -2
  13. {sop4py-2.0.0 → sop4py-2.0.2}/sop/transaction.py +2 -2
  14. {sop4py-2.0.0 → sop4py-2.0.2/sop4py.egg-info}/PKG-INFO +32 -23
  15. {sop4py-2.0.0 → sop4py-2.0.2}/sop4py.egg-info/SOURCES.txt +0 -1
  16. sop4py-2.0.0/sop4py.egg-info/requires.txt +0 -10
  17. {sop4py-2.0.0 → sop4py-2.0.2}/MANIFEST.in +0 -0
  18. {sop4py-2.0.0 → sop4py-2.0.2}/setup.cfg +0 -0
  19. {sop4py-2.0.0 → sop4py-2.0.2}/sop/__init__.py +0 -0
  20. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64darwin.h +0 -0
  21. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64linux.h +0 -0
  22. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_amd64windows.h +0 -0
  23. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_arm64darwin.h +0 -0
  24. {sop4py-2.0.0 → sop4py-2.0.2}/sop/libjsondb_arm64linux.h +0 -0
  25. {sop4py-2.0.0 → sop4py-2.0.2}/sop/test_btree.py +0 -0
  26. {sop4py-2.0.0 → sop4py-2.0.2}/sop/test_btree_idx.py +0 -0
  27. {sop4py-2.0.0 → sop4py-2.0.2}/sop4py.egg-info/dependency_links.txt +0 -0
  28. {sop4py-2.0.0 → sop4py-2.0.2}/sop4py.egg-info/top_level.txt +0 -0
@@ -1,27 +1,29 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sop4py
3
- Version: 2.0.0
4
- Summary: Scalable Objects Persistence for Python.
3
+ Version: 2.0.2
4
+ Summary: Scalable Objects Persistence for Python. Beta 3 Release
5
5
  Author-email: Gerardo Recinto <gerardorecinto@yahoo.com>
6
6
  Classifier: Programming Language :: Python :: 3
7
7
  Classifier: License :: OSI Approved :: MIT License
8
8
  Classifier: Operating System :: OS Independent
9
9
  Requires-Python: >=3.7
10
10
  Description-Content-Type: text/markdown
11
- Requires-Dist: libjsondb_amd64darwin.dylib
12
- Requires-Dist: libjsondb_amd64darwin.h
13
- Requires-Dist: libjsondb_arm64darwin.dylib
14
- Requires-Dist: libjsondb_arm64darwin.h
15
- Requires-Dist: libjsondb_amd64linux.so
16
- Requires-Dist: libjsondb_amd64linux.h
17
- Requires-Dist: libjsondb_arm64linux.so
18
- Requires-Dist: libjsondb_arm64linux.h
19
- Requires-Dist: libjsondb_amd64windows.dll
20
- Requires-Dist: libjsondb_amd64windows.h
21
11
 
22
12
  # What is SOP?
23
13
 
24
- Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO communications w/ disk drives. In a code library form factor.
14
+ Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO operations w/ disk drives. In a code library form factor.
15
+
16
+ Other "key advances" in database technology available in this SOP release:
17
+ * Serverless (or all servers!) operations mode in the cluster, i.e. - your applications that use SOP library are server-less
18
+ * Uses a new storage & L1/L2 caching strategy that is ground breaking in performance & efficiency
19
+ * New superfast realtime Orchestration engine fit for database transactions
20
+ * Support for small, medium to large/very large (multi-GBs/TB) data management
21
+ * Sports advanced/efficient/high tolerance software based (via Erasure Coding) replication
22
+ * Horizontal & Vertical scaling in storage and cluster processes
23
+ * New database storage strategy that uses segment files which avoids having to manage a huge file, resulting in higher IO efficiency & file system/disk drive friendly (medium sized) data files
24
+ * Built-in data caching, your application data can be cached automatically via config setting. No need to write special code to cache certain data set in Redis, for example. SOP can provide that if configured
25
+ * Host your applications across platforms, e.g. - microservice cluster running in Linux, another cluster or instance running in Windows, and all SOPs inter-operating with one another seamlessly and data stored in same set of disk drives. Freedom to use popular hardware architecture & OS of your choice(s)!
26
+ * Unlimited B-trees, limited only by your hardware resources. Each B-tree store, as the name implies, is a B-tree serving super-fast, sorted by Key (Key & Value pair) data sets
25
27
 
26
28
  # SOP supported Hardware/OS
27
29
  SOP supports popular architectures & Operating Systems such as Linux, Darwin & Microsoft Windows, in both ARM64 & AMD64 architectures. For Windows, only AMD64 is supported since it is the only architecture Windows is available in.
@@ -42,6 +44,7 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
42
44
  * Follow standard Python package import and start coding to use the SOP for Python code library for data management. Import the sop package in your python code file.
43
45
  * Specify Home base folders where Store info & Registry data files will be stored.
44
46
  * Specify Erasure Coding (EC) configuration details which will be used by SOP's EC based replication.
47
+ * Open the global Redis connection
45
48
  * Create a transaction
46
49
  * Begin a transaction
47
50
  * Create a new B-tree, or Open an existing B-tree
@@ -51,9 +54,10 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
51
54
  Below is an example code black for illustrating the above steps. For other SOP B-tree examples, you can checkout the code in the unit tests test_btree.py & test_btree_idx.py files that comes w/ the SOP package you downloaded from pypi.
52
55
 
53
56
  ```
54
- import sop.transaction
55
- import sop.btree
56
- import sop.context
57
+ from sop import transaction
58
+ from sop import btree
59
+ from sop import context
60
+ from sop import redis
57
61
 
58
62
  stores_folders = ("/disk1", "/disk2")
59
63
  ec = {
@@ -76,8 +80,8 @@ ec = {
76
80
  # Transaction Options (to).
77
81
  to = transaction.TransationOptions(
78
82
  transaction.TransactionMode.ForWriting.value,
79
- # commit timeout of 5mins
80
- 5,
83
+ # commit timeout of 15mins
84
+ 15,
81
85
  # Min Registry hash mod value is 250, you can specify higher value like 1000. A 250 hashmod
82
86
  # will use 1MB sized file segments. Good for demo, but for Prod, perhaps a bigger value is better.
83
87
  transaction.MIN_HASH_MOD_VALUE,
@@ -90,9 +94,10 @@ to = transaction.TransationOptions(
90
94
  # Context object.
91
95
  ctx = context.Context()
92
96
 
93
- # initialize/open SOP global Redis connection
94
- ro = RedisOptions()
95
- Redis.open_connection(ro)
97
+ # initialize/open SOP global Redis connection. You can specify your Redis cluster host address(es) & port, etc
98
+ # but defaults to localhost and default Redis port # if none is specified.
99
+ ro = redis.RedisOptions()
100
+ redis.Redis.open_connection(ro)
96
101
 
97
102
  t = transaction.Transaction(ctx, to)
98
103
  t.begin()
@@ -103,7 +108,9 @@ cache = btree.CacheConfig()
103
108
  bo = btree.BtreeOptions("barstoreec", True, cache_config=cache)
104
109
  bo.set_value_data_size(btree.ValueDataSize.Small)
105
110
 
106
- # create the new "barstoreec" b-tree store.
111
+ # create the new "barstoreec" b-tree store. Once created, you can just use the "open" method. Perhaps
112
+ # you can have a code set for creating B-trees like "admin only" script. Then all other app code uses
113
+ # the "open" method.
107
114
  b3 = btree.Btree.new(ctx, bo, t)
108
115
 
109
116
  # Since we've specified Native data type = True in BtreeOptions, we can use "integer" values as Key.
@@ -111,14 +118,16 @@ l = [
111
118
  btree.Item(1, "foo"),
112
119
  ]
113
120
 
114
- # Add Item to the B-tree,
121
+ # Add Item to the B-tree.
115
122
  b3.add(ctx, l)
116
123
 
117
124
  # Commit the transaction to finalize the new B-tree (store) change.
118
125
  t.commit(ctx)
126
+ print("ended.")
119
127
  ```
120
128
 
121
129
  # SOP in Github
122
130
  SOP open source project (MIT license) is in github. You can checkout the "...sop/jsondb/" package which contains the Go code enabling general purpose JSON data management & the Python wrapper, coding guideline of which, was described above.
123
131
 
124
132
  Please feel free to join the SOP project if you have the bandwidth and participate/co-own/lead! the project engineering.
133
+ SOP project link: https://github.com/sharedcode/sop
@@ -1,6 +1,18 @@
1
1
  # What is SOP?
2
2
 
3
- Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO communications w/ disk drives. In a code library form factor.
3
+ Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO operations w/ disk drives. In a code library form factor.
4
+
5
+ Other "key advances" in database technology available in this SOP release:
6
+ * Serverless (or all servers!) operations mode in the cluster, i.e. - your applications that use SOP library are server-less
7
+ * Uses a new storage & L1/L2 caching strategy that is ground breaking in performance & efficiency
8
+ * New superfast realtime Orchestration engine fit for database transactions
9
+ * Support for small, medium to large/very large (multi-GBs/TB) data management
10
+ * Sports advanced/efficient/high tolerance software based (via Erasure Coding) replication
11
+ * Horizontal & Vertical scaling in storage and cluster processes
12
+ * New database storage strategy that uses segment files which avoids having to manage a huge file, resulting in higher IO efficiency & file system/disk drive friendly (medium sized) data files
13
+ * Built-in data caching, your application data can be cached automatically via config setting. No need to write special code to cache certain data set in Redis, for example. SOP can provide that if configured
14
+ * Host your applications across platforms, e.g. - microservice cluster running in Linux, another cluster or instance running in Windows, and all SOPs inter-operating with one another seamlessly and data stored in same set of disk drives. Freedom to use popular hardware architecture & OS of your choice(s)!
15
+ * Unlimited B-trees, limited only by your hardware resources. Each B-tree store, as the name implies, is a B-tree serving super-fast, sorted by Key (Key & Value pair) data sets
4
16
 
5
17
  # SOP supported Hardware/OS
6
18
  SOP supports popular architectures & Operating Systems such as Linux, Darwin & Microsoft Windows, in both ARM64 & AMD64 architectures. For Windows, only AMD64 is supported since it is the only architecture Windows is available in.
@@ -21,6 +33,7 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
21
33
  * Follow standard Python package import and start coding to use the SOP for Python code library for data management. Import the sop package in your python code file.
22
34
  * Specify Home base folders where Store info & Registry data files will be stored.
23
35
  * Specify Erasure Coding (EC) configuration details which will be used by SOP's EC based replication.
36
+ * Open the global Redis connection
24
37
  * Create a transaction
25
38
  * Begin a transaction
26
39
  * Create a new B-tree, or Open an existing B-tree
@@ -30,9 +43,10 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
30
43
  Below is an example code black for illustrating the above steps. For other SOP B-tree examples, you can checkout the code in the unit tests test_btree.py & test_btree_idx.py files that comes w/ the SOP package you downloaded from pypi.
31
44
 
32
45
  ```
33
- import sop.transaction
34
- import sop.btree
35
- import sop.context
46
+ from sop import transaction
47
+ from sop import btree
48
+ from sop import context
49
+ from sop import redis
36
50
 
37
51
  stores_folders = ("/disk1", "/disk2")
38
52
  ec = {
@@ -55,8 +69,8 @@ ec = {
55
69
  # Transaction Options (to).
56
70
  to = transaction.TransationOptions(
57
71
  transaction.TransactionMode.ForWriting.value,
58
- # commit timeout of 5mins
59
- 5,
72
+ # commit timeout of 15mins
73
+ 15,
60
74
  # Min Registry hash mod value is 250, you can specify higher value like 1000. A 250 hashmod
61
75
  # will use 1MB sized file segments. Good for demo, but for Prod, perhaps a bigger value is better.
62
76
  transaction.MIN_HASH_MOD_VALUE,
@@ -69,9 +83,10 @@ to = transaction.TransationOptions(
69
83
  # Context object.
70
84
  ctx = context.Context()
71
85
 
72
- # initialize/open SOP global Redis connection
73
- ro = RedisOptions()
74
- Redis.open_connection(ro)
86
+ # initialize/open SOP global Redis connection. You can specify your Redis cluster host address(es) & port, etc
87
+ # but defaults to localhost and default Redis port # if none is specified.
88
+ ro = redis.RedisOptions()
89
+ redis.Redis.open_connection(ro)
75
90
 
76
91
  t = transaction.Transaction(ctx, to)
77
92
  t.begin()
@@ -82,7 +97,9 @@ cache = btree.CacheConfig()
82
97
  bo = btree.BtreeOptions("barstoreec", True, cache_config=cache)
83
98
  bo.set_value_data_size(btree.ValueDataSize.Small)
84
99
 
85
- # create the new "barstoreec" b-tree store.
100
+ # create the new "barstoreec" b-tree store. Once created, you can just use the "open" method. Perhaps
101
+ # you can have a code set for creating B-trees like "admin only" script. Then all other app code uses
102
+ # the "open" method.
86
103
  b3 = btree.Btree.new(ctx, bo, t)
87
104
 
88
105
  # Since we've specified Native data type = True in BtreeOptions, we can use "integer" values as Key.
@@ -90,14 +107,16 @@ l = [
90
107
  btree.Item(1, "foo"),
91
108
  ]
92
109
 
93
- # Add Item to the B-tree,
110
+ # Add Item to the B-tree.
94
111
  b3.add(ctx, l)
95
112
 
96
113
  # Commit the transaction to finalize the new B-tree (store) change.
97
114
  t.commit(ctx)
115
+ print("ended.")
98
116
  ```
99
117
 
100
118
  # SOP in Github
101
119
  SOP open source project (MIT license) is in github. You can checkout the "...sop/jsondb/" package which contains the Go code enabling general purpose JSON data management & the Python wrapper, coding guideline of which, was described above.
102
120
 
103
- Please feel free to join the SOP project if you have the bandwidth and participate/co-own/lead! the project engineering.
121
+ Please feel free to join the SOP project if you have the bandwidth and participate/co-own/lead! the project engineering.
122
+ SOP project link: https://github.com/sharedcode/sop
@@ -7,11 +7,11 @@ build-backend = "setuptools.build_meta"
7
7
 
8
8
  [project]
9
9
  name = "sop4py"
10
- version = "2.0.0"
10
+ version = "2.0.2"
11
11
  authors = [
12
12
  { name="Gerardo Recinto", email="gerardorecinto@yahoo.com" },
13
13
  ]
14
- description = "Scalable Objects Persistence for Python."
14
+ description = "Scalable Objects Persistence for Python. Beta 3 Release"
15
15
  readme = "README.md"
16
16
  requires-python = ">=3.7"
17
17
  classifiers = [
@@ -19,18 +19,7 @@ classifiers = [
19
19
  "License :: OSI Approved :: MIT License",
20
20
  "Operating System :: OS Independent",
21
21
  ]
22
- dependencies = [
23
- "libjsondb_amd64darwin.dylib",
24
- "libjsondb_amd64darwin.h",
25
- "libjsondb_arm64darwin.dylib",
26
- "libjsondb_arm64darwin.h",
27
- "libjsondb_amd64linux.so",
28
- "libjsondb_amd64linux.h",
29
- "libjsondb_arm64linux.so",
30
- "libjsondb_arm64linux.h",
31
- "libjsondb_amd64windows.dll",
32
- "libjsondb_amd64windows.h",
33
- ]
22
+ dependencies = []
34
23
 
35
24
  # build: python3 -m build
36
25
  # upload to pypi using twine: python3 -m twine upload dist/*
@@ -1,15 +1,15 @@
1
1
  import json
2
2
  import uuid
3
3
  import logging
4
- import call_go
5
- import context
4
+ from . import call_go
5
+ from . import context
6
6
 
7
7
  logger = logging.getLogger(__name__)
8
8
 
9
9
  from typing import TypeVar, Generic, Type
10
10
  from dataclasses import dataclass, asdict
11
11
 
12
- from transaction import Transaction, TransactionError
12
+ from . import transaction
13
13
 
14
14
  # Define TypeVars 'TK' & 'TV' to represent Key & Value generic types.
15
15
  TK = TypeVar("TK")
@@ -19,13 +19,11 @@ from datetime import timedelta
19
19
 
20
20
  from enum import Enum, auto
21
21
 
22
-
23
22
  class ValueDataSize(Enum):
24
23
  Small = 0
25
24
  Medium = 1
26
25
  Big = 2
27
26
 
28
-
29
27
  class PagingDirection(Enum):
30
28
  Forward = 0
31
29
  Backward = 1
@@ -126,8 +124,8 @@ class Item(Generic[TK, TV]):
126
124
  id: str = str(uuid.UUID(int=0))
127
125
 
128
126
 
129
- class BtreeError(TransactionError):
130
- """Exception for Btree-related errors, 'is derived from TransactionError."""
127
+ class BtreeError(transaction.TransactionError):
128
+ """Exception for Btree-related errors, 'is derived from transaction.TransactionError."""
131
129
 
132
130
  pass
133
131
 
@@ -186,7 +184,7 @@ class Btree(Generic[TK, TV]):
186
184
  cls: Type["Btree[TK,TV]"],
187
185
  ctx: context.Context,
188
186
  options: BtreeOptions,
189
- trans: Transaction,
187
+ trans: transaction.Transaction,
190
188
  index_spec: IndexSpecification = None,
191
189
  ) -> "Btree[TK,TV]":
192
190
  """Create a new B-tree store in the backend storage with the options specified then returns an instance
@@ -196,7 +194,7 @@ class Btree(Generic[TK, TV]):
196
194
  cls (Type[&quot;Btree[TK,TV]&quot;]): Supports generics for Key (TK) & Value (TV) pair.
197
195
  ctx (context.Context): context.Context object, useful for telling SOP in the backend the ID of the context for use in calls.
198
196
  options (BtreeOptions): _description_
199
- trans (Transaction): instance of a Transaction that the B-tree store to be opened belongs.
197
+ trans (transaction.Transaction): instance of a transaction.Transaction that the B-tree store to be opened belongs.
200
198
  index_spec (IndexSpecification, optional): Defaults to None.
201
199
 
202
200
  Raises:
@@ -226,14 +224,14 @@ class Btree(Generic[TK, TV]):
226
224
 
227
225
  @classmethod
228
226
  def open(
229
- cls: Type["Btree[TK,TV]"], ctx: context.Context, name: str, trans: Transaction
227
+ cls: Type["Btree[TK,TV]"], ctx: context.Context, name: str, trans: transaction.Transaction
230
228
  ) -> "Btree[TK,TV]":
231
229
  """Open an existing B-tree store on the backend and returns a B-tree manager that can allow code to do operations on it.
232
230
  Args:
233
231
  cls (Type[&quot;Btree[TK,TV]&quot;]): Supports generics for Key (TK) & Value (TV) pair.
234
232
  ctx (context.Context): context.Context object, useful for telling SOP in the backend the ID of the context for use in calls.
235
233
  name (str): Name of the B-tree store to open.
236
- trans (Transaction): instance of a Transaction that the B-tree store to be opened belongs.
234
+ trans (transaction.Transaction): instance of a transaction.Transaction that the B-tree store to be opened belongs.
237
235
 
238
236
  Raises:
239
237
  BtreeError: error message generated when calling different methods of the B-tree.
@@ -20,7 +20,9 @@ else:
20
20
 
21
21
  # Load the shared library
22
22
  try:
23
- lib = ctypes.CDLL(f"./libjsondb_{ext}")
23
+ script_dir = os.path.dirname(__file__) # Get directory of the current script
24
+ library_path = os.path.join(script_dir, f"libjsondb_{ext}") # Adjust filename
25
+ lib = ctypes.CDLL(library_path)
24
26
  except OSError as e:
25
27
  print(f"Error loading library: {e}")
26
28
  print("Ensure 'libjsondb_<arch><os>.so' (or .dll/.dylib) is in the same directory.")
@@ -1,4 +1,4 @@
1
- import call_go
1
+ from . import call_go
2
2
 
3
3
 
4
4
  class Context:
@@ -1,5 +1,4 @@
1
- import ctypes
2
- import call_go
1
+ from . import call_go
3
2
 
4
3
 
5
4
  class RedisOptions:
@@ -1,7 +1,7 @@
1
1
  import json
2
- import call_go
2
+ from . import call_go
3
3
  import uuid
4
- import context
4
+ from . import context
5
5
 
6
6
  from enum import Enum
7
7
  from dataclasses import dataclass, asdict
@@ -1,27 +1,29 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sop4py
3
- Version: 2.0.0
4
- Summary: Scalable Objects Persistence for Python.
3
+ Version: 2.0.2
4
+ Summary: Scalable Objects Persistence for Python. Beta 3 Release
5
5
  Author-email: Gerardo Recinto <gerardorecinto@yahoo.com>
6
6
  Classifier: Programming Language :: Python :: 3
7
7
  Classifier: License :: OSI Approved :: MIT License
8
8
  Classifier: Operating System :: OS Independent
9
9
  Requires-Python: >=3.7
10
10
  Description-Content-Type: text/markdown
11
- Requires-Dist: libjsondb_amd64darwin.dylib
12
- Requires-Dist: libjsondb_amd64darwin.h
13
- Requires-Dist: libjsondb_arm64darwin.dylib
14
- Requires-Dist: libjsondb_arm64darwin.h
15
- Requires-Dist: libjsondb_amd64linux.so
16
- Requires-Dist: libjsondb_amd64linux.h
17
- Requires-Dist: libjsondb_arm64linux.so
18
- Requires-Dist: libjsondb_arm64linux.h
19
- Requires-Dist: libjsondb_amd64windows.dll
20
- Requires-Dist: libjsondb_amd64windows.h
21
11
 
22
12
  # What is SOP?
23
13
 
24
- Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO communications w/ disk drives. In a code library form factor.
14
+ Scalable Objects Persistence (SOP) is a raw storage engine that bakes together a set of storage related features & algorithms in order to provide the most efficient & reliable (ACID attributes of transactions) technique (known) of storage management and rich search, as it brings to the application, the raw muscle of "raw storage", direct IO operations w/ disk drives. In a code library form factor.
15
+
16
+ Other "key advances" in database technology available in this SOP release:
17
+ * Serverless (or all servers!) operations mode in the cluster, i.e. - your applications that use SOP library are server-less
18
+ * Uses a new storage & L1/L2 caching strategy that is ground breaking in performance & efficiency
19
+ * New superfast realtime Orchestration engine fit for database transactions
20
+ * Support for small, medium to large/very large (multi-GBs/TB) data management
21
+ * Sports advanced/efficient/high tolerance software based (via Erasure Coding) replication
22
+ * Horizontal & Vertical scaling in storage and cluster processes
23
+ * New database storage strategy that uses segment files which avoids having to manage a huge file, resulting in higher IO efficiency & file system/disk drive friendly (medium sized) data files
24
+ * Built-in data caching, your application data can be cached automatically via config setting. No need to write special code to cache certain data set in Redis, for example. SOP can provide that if configured
25
+ * Host your applications across platforms, e.g. - microservice cluster running in Linux, another cluster or instance running in Windows, and all SOPs inter-operating with one another seamlessly and data stored in same set of disk drives. Freedom to use popular hardware architecture & OS of your choice(s)!
26
+ * Unlimited B-trees, limited only by your hardware resources. Each B-tree store, as the name implies, is a B-tree serving super-fast, sorted by Key (Key & Value pair) data sets
25
27
 
26
28
  # SOP supported Hardware/OS
27
29
  SOP supports popular architectures & Operating Systems such as Linux, Darwin & Microsoft Windows, in both ARM64 & AMD64 architectures. For Windows, only AMD64 is supported since it is the only architecture Windows is available in.
@@ -42,6 +44,7 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
42
44
  * Follow standard Python package import and start coding to use the SOP for Python code library for data management. Import the sop package in your python code file.
43
45
  * Specify Home base folders where Store info & Registry data files will be stored.
44
46
  * Specify Erasure Coding (EC) configuration details which will be used by SOP's EC based replication.
47
+ * Open the global Redis connection
45
48
  * Create a transaction
46
49
  * Begin a transaction
47
50
  * Create a new B-tree, or Open an existing B-tree
@@ -51,9 +54,10 @@ Following steps outlines how to use the Scalable Objects Persistence code librar
51
54
  Below is an example code black for illustrating the above steps. For other SOP B-tree examples, you can checkout the code in the unit tests test_btree.py & test_btree_idx.py files that comes w/ the SOP package you downloaded from pypi.
52
55
 
53
56
  ```
54
- import sop.transaction
55
- import sop.btree
56
- import sop.context
57
+ from sop import transaction
58
+ from sop import btree
59
+ from sop import context
60
+ from sop import redis
57
61
 
58
62
  stores_folders = ("/disk1", "/disk2")
59
63
  ec = {
@@ -76,8 +80,8 @@ ec = {
76
80
  # Transaction Options (to).
77
81
  to = transaction.TransationOptions(
78
82
  transaction.TransactionMode.ForWriting.value,
79
- # commit timeout of 5mins
80
- 5,
83
+ # commit timeout of 15mins
84
+ 15,
81
85
  # Min Registry hash mod value is 250, you can specify higher value like 1000. A 250 hashmod
82
86
  # will use 1MB sized file segments. Good for demo, but for Prod, perhaps a bigger value is better.
83
87
  transaction.MIN_HASH_MOD_VALUE,
@@ -90,9 +94,10 @@ to = transaction.TransationOptions(
90
94
  # Context object.
91
95
  ctx = context.Context()
92
96
 
93
- # initialize/open SOP global Redis connection
94
- ro = RedisOptions()
95
- Redis.open_connection(ro)
97
+ # initialize/open SOP global Redis connection. You can specify your Redis cluster host address(es) & port, etc
98
+ # but defaults to localhost and default Redis port # if none is specified.
99
+ ro = redis.RedisOptions()
100
+ redis.Redis.open_connection(ro)
96
101
 
97
102
  t = transaction.Transaction(ctx, to)
98
103
  t.begin()
@@ -103,7 +108,9 @@ cache = btree.CacheConfig()
103
108
  bo = btree.BtreeOptions("barstoreec", True, cache_config=cache)
104
109
  bo.set_value_data_size(btree.ValueDataSize.Small)
105
110
 
106
- # create the new "barstoreec" b-tree store.
111
+ # create the new "barstoreec" b-tree store. Once created, you can just use the "open" method. Perhaps
112
+ # you can have a code set for creating B-trees like "admin only" script. Then all other app code uses
113
+ # the "open" method.
107
114
  b3 = btree.Btree.new(ctx, bo, t)
108
115
 
109
116
  # Since we've specified Native data type = True in BtreeOptions, we can use "integer" values as Key.
@@ -111,14 +118,16 @@ l = [
111
118
  btree.Item(1, "foo"),
112
119
  ]
113
120
 
114
- # Add Item to the B-tree,
121
+ # Add Item to the B-tree.
115
122
  b3.add(ctx, l)
116
123
 
117
124
  # Commit the transaction to finalize the new B-tree (store) change.
118
125
  t.commit(ctx)
126
+ print("ended.")
119
127
  ```
120
128
 
121
129
  # SOP in Github
122
130
  SOP open source project (MIT license) is in github. You can checkout the "...sop/jsondb/" package which contains the Go code enabling general purpose JSON data management & the Python wrapper, coding guideline of which, was described above.
123
131
 
124
132
  Please feel free to join the SOP project if you have the bandwidth and participate/co-own/lead! the project engineering.
133
+ SOP project link: https://github.com/sharedcode/sop
@@ -22,5 +22,4 @@ sop/transaction.py
22
22
  sop4py.egg-info/PKG-INFO
23
23
  sop4py.egg-info/SOURCES.txt
24
24
  sop4py.egg-info/dependency_links.txt
25
- sop4py.egg-info/requires.txt
26
25
  sop4py.egg-info/top_level.txt
@@ -1,10 +0,0 @@
1
- libjsondb_amd64darwin.dylib
2
- libjsondb_amd64darwin.h
3
- libjsondb_arm64darwin.dylib
4
- libjsondb_arm64darwin.h
5
- libjsondb_amd64linux.so
6
- libjsondb_amd64linux.h
7
- libjsondb_arm64linux.so
8
- libjsondb_arm64linux.h
9
- libjsondb_amd64windows.dll
10
- libjsondb_amd64windows.h
File without changes
File without changes
File without changes
File without changes
File without changes