roaringbitmap 0.7.1__tar.gz → 0.7.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (16) hide show
  1. roaringbitmap-0.7.3/MANIFEST.in +3 -0
  2. roaringbitmap-0.7.3/PKG-INFO +177 -0
  3. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/README.rst +8 -2
  4. roaringbitmap-0.7.3/pyproject.toml +14 -0
  5. roaringbitmap-0.7.3/roaringbitmap.egg-info/PKG-INFO +177 -0
  6. roaringbitmap-0.7.3/roaringbitmap.egg-info/SOURCES.txt +13 -0
  7. roaringbitmap-0.7.3/roaringbitmap.egg-info/dependency_links.txt +1 -0
  8. roaringbitmap-0.7.3/roaringbitmap.egg-info/top_level.txt +1 -0
  9. roaringbitmap-0.7.3/setup.cfg +4 -0
  10. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/setup.py +4 -5
  11. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/src/roaringbitmap.c +41240 -21547
  12. roaringbitmap-0.7.1/PKG-INFO +0 -160
  13. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/LICENSE +0 -0
  14. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/src/_arrayops.h +0 -0
  15. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/src/bitcount.h +0 -0
  16. {roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/src/macros.h +0 -0
roaringbitmap-0.7.3/MANIFEST.in ADDED
@@ -0,0 +1,3 @@
1
+ include README.rst LICENSE setup.py
2
+ recursive-include src *.h *.c
3
+ recursive-exclude src *.pyx *.pxi *.pxd
roaringbitmap-0.7.3/PKG-INFO ADDED
@@ -0,0 +1,177 @@
1
+ Metadata-Version: 2.4
2
+ Name: roaringbitmap
3
+ Version: 0.7.3
4
+ Summary: Roaring Bitmap
5
+ Home-page: http://roaringbitmap.readthedocs.io
6
+ Author: Andreas van Cranenburgh
7
+ Author-email: A.W.van.Cranenburgh@rug.nl
8
+ License: GPL-2.0-or-later
9
+ Platform: Many
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: Operating System :: POSIX
13
+ Classifier: Programming Language :: Python :: 2.7
14
+ Classifier: Programming Language :: Python :: 3.3
15
+ Classifier: Programming Language :: Cython
16
+ Description-Content-Type: text/x-rst
17
+ License-File: LICENSE
18
+ Dynamic: author
19
+ Dynamic: author-email
20
+ Dynamic: classifier
21
+ Dynamic: description
22
+ Dynamic: description-content-type
23
+ Dynamic: home-page
24
+ Dynamic: license
25
+ Dynamic: license-file
26
+ Dynamic: platform
27
+ Dynamic: summary
28
+
29
+ Roaring Bitmap in Cython
30
+ ========================
31
+
32
+ A roaring bitmap is an efficient compressed datastructure to store a set
33
+ of integers. A Roaring bitmap stores a set of 32-bit integers in a series of
34
+ arrays and bitmaps, whichever takes the least space (which is always
35
+ ``2 ** 16`` bits or less).
36
+
37
+ This datastructure is useful for storing a large number of integers, e.g., for
38
+ an inverted index used by search engines and databases. In particular, it is
39
+ possible to quickly compute the intersection of a series of sets, which can be
40
+ used to implement a query as the conjunction of subqueries.
41
+
42
+ This implementation is based on the Java and C implementations at
43
+ https://github.com/lemire/RoaringBitmap
44
+ and https://github.com/lemire/CRoaring
45
+
46
+ Additional features of this implementation:
47
+
48
+ - Inverted list representation: blocks that are mostly full are stored
49
+ compactly as an array of non-members (instead of as an array of members or a
50
+ fixed-size bitmap).
51
+ - Collections of immutable roaring bitmaps can be efficiently serialized with
52
+ ``mmap`` in a single file.
53
+
54
+ Missing features w.r.t. CRoaring:
55
+
56
+ - Run-length encoded blocks
57
+ - Various AVX2 / SSE optimizations
58
+
59
+ See also PyRoaringBitmap, a Python wrapper of CRoaring:
60
+ https://github.com/Ezibenroc/PyRoaringBitMap
61
+
62
+ License, requirements
63
+ ---------------------
64
+ The code is licensed under GNU GPL v2, or any later version at your option.
65
+
66
+ - Python 2.7+/3.3+ http://www.python.org (headers required, e.g. python-dev package)
67
+ - Cython 3.0+ http://www.cython.org
68
+
69
+ Installation, usage
70
+ -------------------
71
+
72
+ ::
73
+
74
+ $ git clone https://github.com/andreasvc/roaringbitmap.git
75
+ $ cd roaringbitmap
76
+ $ make
77
+
78
+ For Python 2, build with a Cython version that supports Python 2.7::
79
+
80
+ $ python2 -m pip install 'Cython>=3,<3.1'
81
+ $ make py2
82
+
83
+ The C sources published on PyPI are generated with the current Cython release
84
+ and support Python 3 only.
85
+
86
+ A ``RoaringBitmap()`` can be used as a replacement for a normal (mutable)
87
+ Python set containing (unsigned) 32-bit integers:
88
+
89
+ .. code-block:: python
90
+
91
+ >>> from roaringbitmap import RoaringBitmap
92
+ >>> RoaringBitmap(range(10)) & RoaringBitmap(range(5, 15))
93
+ RoaringBitmap({5, 6, 7, 8, 9})
94
+
95
+ ``ImmutableRoaringBitmap`` is an immutable variant (analogous to ``frozenset``)
96
+ which is stored compactly as a contiguous block of memory.
97
+
98
+ A sequence of immutable RoaringBitmaps can be stored in a single file and
99
+ accessed efficiently with ``mmap``, without needing to copy or deserialize:
100
+
101
+ .. code-block:: python
102
+
103
+ >>> from roaringbitmap import MultiRoaringBitmap
104
+ >>> mrb = MultiRoaringBitmap([range(n, n + 5) for n in range(10)], filename='index')
105
+
106
+ >>> mrb = MultiRoaringBitmap.fromfile('index')
107
+ >>> mrb[5]
108
+ ImmutableRoaringBitmap({5, 6, 7, 8, 9})
109
+
110
+ For API documentation cf. http://roaringbitmap.readthedocs.io
111
+
112
+ Benchmarks
113
+ ----------
114
+ Output of ``$ make bench``::
115
+
116
+ small sparse set
117
+ 100 runs with sets of 200 random elements n s.t. 0 <= n < 40000
118
+ set() RoaringBitmap() ratio
119
+ init 0.000834 0.00138 0.603
120
+ initsort 0.00085 0.000394 2.16
121
+ and 0.00102 8.49e-05 12.1
122
+ or 0.00171 0.000169 10.1
123
+ xor 0.00152 0.000213 7.11
124
+ sub 0.000934 0.000197 4.74
125
+ iand 1.29e-05 2.97e-06 4.35
126
+ ior 9.7e-06 3.26e-06 2.98
127
+ ixor 8.98e-06 3.43e-06 2.62
128
+ isub 6.83e-06 3.3e-06 2.07
129
+ eq 0.000438 1.17e-05 37.6
130
+ neq 6.37e-06 7.81e-06 0.816
131
+ jaccard 0.0029 0.000126 23.1
132
+
133
+ medium load factor
134
+ 100 runs with sets of 59392 random elements n s.t. 0 <= n < 118784
135
+ set() RoaringBitmap() ratio
136
+ init 0.564 0.324 1.74
137
+ initsort 0.696 0.273 2.55
138
+ and 0.613 0.000418 1466
139
+ or 0.976 0.000292 3344
140
+ xor 0.955 0.000294 3250
141
+ sub 0.346 0.000316 1092
142
+ iand 0.00658 1.14e-05 575
143
+ ior 0.00594 1.08e-05 548
144
+ ixor 0.00434 1.12e-05 385
145
+ isub 0.00431 1.09e-05 397
146
+ eq 0.0991 0.000116 851
147
+ neq 9.62e-06 1.29e-05 0.743
148
+ jaccard 1.62 0.00025 6476
149
+
150
+ dense set / high load factor
151
+ 100 runs with sets of 39800 random elements n s.t. 0 <= n < 40000
152
+ set() RoaringBitmap() ratio
153
+ init 0.33 0.0775 4.26
154
+ initsort 0.352 0.148 2.38
155
+ and 0.24 0.000223 1078
156
+ or 0.45 0.000165 2734
157
+ xor 0.404 0.000161 2514
158
+ sub 0.169 0.000173 973
159
+ iand 0.00287 6.02e-06 477
160
+ ior 0.00179 6.34e-06 282
161
+ ixor 0.00195 5.53e-06 353
162
+ isub 0.0017 6.35e-06 267
163
+ eq 0.0486 4.65e-05 1045
164
+ neq 1.01e-05 1.13e-05 0.888
165
+ jaccard 0.722 0.000118 6136
166
+
167
+ See https://github.com/Ezibenroc/roaring_analysis/ for a performance comparison
168
+ of PyRoaringBitmap and this library.
169
+
170
+ References
171
+ ----------
172
+ - http://roaringbitmap.org/
173
+ - Chambi, S., Lemire, D., Kaser, O., & Godin, R. (2016). Better bitmap
174
+ performance with Roaring bitmaps. Software: practice and experience, 46(5),
175
+ pp. 709-719. http://arxiv.org/abs/1402.6407
176
+ - The idea of using the inverted list representation is based on
177
+ https://issues.apache.org/jira/browse/LUCENE-5983
{roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/README.rst RENAMED
@@ -36,7 +36,7 @@ License, requirements
36
36
  The code is licensed under GNU GPL v2, or any later version at your option.
37
37
 
38
38
  - Python 2.7+/3.3+ http://www.python.org (headers required, e.g. python-dev package)
39
- - Cython 0.20+ http://www.cython.org
39
+ - Cython 3.0+ http://www.cython.org
40
40
 
41
41
  Installation, usage
42
42
  -------------------
@@ -47,7 +47,13 @@ Installation, usage
47
47
  $ cd roaringbitmap
48
48
  $ make
49
49
 
50
- (or ``make py2`` for Python 2)
50
+ For Python 2, build with a Cython version that supports Python 2.7::
51
+
52
+ $ python2 -m pip install 'Cython>=3,<3.1'
53
+ $ make py2
54
+
55
+ The C sources published on PyPI are generated with the current Cython release
56
+ and support Python 3 only.
51
57
 
52
58
  A ``RoaringBitmap()`` can be used as a replacement for a normal (mutable)
53
59
  Python set containing (unsigned) 32-bit integers:
roaringbitmap-0.7.3/pyproject.toml ADDED
@@ -0,0 +1,14 @@
1
+ [build-system]
2
+ requires = [
3
+ "setuptools<45; python_version < '3'",
4
+ "setuptools>=61; python_version >= '3'",
5
+ "Cython>=3,<3.1; python_version < '3'",
6
+ "Cython>=3; python_version >= '3'",
7
+ ]
8
+ build-backend = "setuptools.build_meta"
9
+
10
+ [dependency-groups]
11
+ dev = [
12
+ "pytest>=3.0.0",
13
+ "pycodestyle",
14
+ ]
roaringbitmap-0.7.3/roaringbitmap.egg-info/PKG-INFO ADDED
@@ -0,0 +1,177 @@
1
+ Metadata-Version: 2.4
2
+ Name: roaringbitmap
3
+ Version: 0.7.3
4
+ Summary: Roaring Bitmap
5
+ Home-page: http://roaringbitmap.readthedocs.io
6
+ Author: Andreas van Cranenburgh
7
+ Author-email: A.W.van.Cranenburgh@rug.nl
8
+ License: GPL-2.0-or-later
9
+ Platform: Many
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: Operating System :: POSIX
13
+ Classifier: Programming Language :: Python :: 2.7
14
+ Classifier: Programming Language :: Python :: 3.3
15
+ Classifier: Programming Language :: Cython
16
+ Description-Content-Type: text/x-rst
17
+ License-File: LICENSE
18
+ Dynamic: author
19
+ Dynamic: author-email
20
+ Dynamic: classifier
21
+ Dynamic: description
22
+ Dynamic: description-content-type
23
+ Dynamic: home-page
24
+ Dynamic: license
25
+ Dynamic: license-file
26
+ Dynamic: platform
27
+ Dynamic: summary
28
+
29
+ Roaring Bitmap in Cython
30
+ ========================
31
+
32
+ A roaring bitmap is an efficient compressed datastructure to store a set
33
+ of integers. A Roaring bitmap stores a set of 32-bit integers in a series of
34
+ arrays and bitmaps, whichever takes the least space (which is always
35
+ ``2 ** 16`` bits or less).
36
+
37
+ This datastructure is useful for storing a large number of integers, e.g., for
38
+ an inverted index used by search engines and databases. In particular, it is
39
+ possible to quickly compute the intersection of a series of sets, which can be
40
+ used to implement a query as the conjunction of subqueries.
41
+
42
+ This implementation is based on the Java and C implementations at
43
+ https://github.com/lemire/RoaringBitmap
44
+ and https://github.com/lemire/CRoaring
45
+
46
+ Additional features of this implementation:
47
+
48
+ - Inverted list representation: blocks that are mostly full are stored
49
+ compactly as an array of non-members (instead of as an array of members or a
50
+ fixed-size bitmap).
51
+ - Collections of immutable roaring bitmaps can be efficiently serialized with
52
+ ``mmap`` in a single file.
53
+
54
+ Missing features w.r.t. CRoaring:
55
+
56
+ - Run-length encoded blocks
57
+ - Various AVX2 / SSE optimizations
58
+
59
+ See also PyRoaringBitmap, a Python wrapper of CRoaring:
60
+ https://github.com/Ezibenroc/PyRoaringBitMap
61
+
62
+ License, requirements
63
+ ---------------------
64
+ The code is licensed under GNU GPL v2, or any later version at your option.
65
+
66
+ - Python 2.7+/3.3+ http://www.python.org (headers required, e.g. python-dev package)
67
+ - Cython 3.0+ http://www.cython.org
68
+
69
+ Installation, usage
70
+ -------------------
71
+
72
+ ::
73
+
74
+ $ git clone https://github.com/andreasvc/roaringbitmap.git
75
+ $ cd roaringbitmap
76
+ $ make
77
+
78
+ For Python 2, build with a Cython version that supports Python 2.7::
79
+
80
+ $ python2 -m pip install 'Cython>=3,<3.1'
81
+ $ make py2
82
+
83
+ The C sources published on PyPI are generated with the current Cython release
84
+ and support Python 3 only.
85
+
86
+ A ``RoaringBitmap()`` can be used as a replacement for a normal (mutable)
87
+ Python set containing (unsigned) 32-bit integers:
88
+
89
+ .. code-block:: python
90
+
91
+ >>> from roaringbitmap import RoaringBitmap
92
+ >>> RoaringBitmap(range(10)) & RoaringBitmap(range(5, 15))
93
+ RoaringBitmap({5, 6, 7, 8, 9})
94
+
95
+ ``ImmutableRoaringBitmap`` is an immutable variant (analogous to ``frozenset``)
96
+ which is stored compactly as a contiguous block of memory.
97
+
98
+ A sequence of immutable RoaringBitmaps can be stored in a single file and
99
+ accessed efficiently with ``mmap``, without needing to copy or deserialize:
100
+
101
+ .. code-block:: python
102
+
103
+ >>> from roaringbitmap import MultiRoaringBitmap
104
+ >>> mrb = MultiRoaringBitmap([range(n, n + 5) for n in range(10)], filename='index')
105
+
106
+ >>> mrb = MultiRoaringBitmap.fromfile('index')
107
+ >>> mrb[5]
108
+ ImmutableRoaringBitmap({5, 6, 7, 8, 9})
109
+
110
+ For API documentation cf. http://roaringbitmap.readthedocs.io
111
+
112
+ Benchmarks
113
+ ----------
114
+ Output of ``$ make bench``::
115
+
116
+ small sparse set
117
+ 100 runs with sets of 200 random elements n s.t. 0 <= n < 40000
118
+ set() RoaringBitmap() ratio
119
+ init 0.000834 0.00138 0.603
120
+ initsort 0.00085 0.000394 2.16
121
+ and 0.00102 8.49e-05 12.1
122
+ or 0.00171 0.000169 10.1
123
+ xor 0.00152 0.000213 7.11
124
+ sub 0.000934 0.000197 4.74
125
+ iand 1.29e-05 2.97e-06 4.35
126
+ ior 9.7e-06 3.26e-06 2.98
127
+ ixor 8.98e-06 3.43e-06 2.62
128
+ isub 6.83e-06 3.3e-06 2.07
129
+ eq 0.000438 1.17e-05 37.6
130
+ neq 6.37e-06 7.81e-06 0.816
131
+ jaccard 0.0029 0.000126 23.1
132
+
133
+ medium load factor
134
+ 100 runs with sets of 59392 random elements n s.t. 0 <= n < 118784
135
+ set() RoaringBitmap() ratio
136
+ init 0.564 0.324 1.74
137
+ initsort 0.696 0.273 2.55
138
+ and 0.613 0.000418 1466
139
+ or 0.976 0.000292 3344
140
+ xor 0.955 0.000294 3250
141
+ sub 0.346 0.000316 1092
142
+ iand 0.00658 1.14e-05 575
143
+ ior 0.00594 1.08e-05 548
144
+ ixor 0.00434 1.12e-05 385
145
+ isub 0.00431 1.09e-05 397
146
+ eq 0.0991 0.000116 851
147
+ neq 9.62e-06 1.29e-05 0.743
148
+ jaccard 1.62 0.00025 6476
149
+
150
+ dense set / high load factor
151
+ 100 runs with sets of 39800 random elements n s.t. 0 <= n < 40000
152
+ set() RoaringBitmap() ratio
153
+ init 0.33 0.0775 4.26
154
+ initsort 0.352 0.148 2.38
155
+ and 0.24 0.000223 1078
156
+ or 0.45 0.000165 2734
157
+ xor 0.404 0.000161 2514
158
+ sub 0.169 0.000173 973
159
+ iand 0.00287 6.02e-06 477
160
+ ior 0.00179 6.34e-06 282
161
+ ixor 0.00195 5.53e-06 353
162
+ isub 0.0017 6.35e-06 267
163
+ eq 0.0486 4.65e-05 1045
164
+ neq 1.01e-05 1.13e-05 0.888
165
+ jaccard 0.722 0.000118 6136
166
+
167
+ See https://github.com/Ezibenroc/roaring_analysis/ for a performance comparison
168
+ of PyRoaringBitmap and this library.
169
+
170
+ References
171
+ ----------
172
+ - http://roaringbitmap.org/
173
+ - Chambi, S., Lemire, D., Kaser, O., & Godin, R. (2016). Better bitmap
174
+ performance with Roaring bitmaps. Software: practice and experience, 46(5),
175
+ pp. 709-719. http://arxiv.org/abs/1402.6407
176
+ - The idea of using the inverted list representation is based on
177
+ https://issues.apache.org/jira/browse/LUCENE-5983
roaringbitmap-0.7.3/roaringbitmap.egg-info/SOURCES.txt ADDED
@@ -0,0 +1,13 @@
1
+ LICENSE
2
+ MANIFEST.in
3
+ README.rst
4
+ pyproject.toml
5
+ setup.py
6
+ roaringbitmap.egg-info/PKG-INFO
7
+ roaringbitmap.egg-info/SOURCES.txt
8
+ roaringbitmap.egg-info/dependency_links.txt
9
+ roaringbitmap.egg-info/top_level.txt
10
+ src/_arrayops.h
11
+ src/bitcount.h
12
+ src/macros.h
13
+ src/roaringbitmap.c
roaringbitmap-0.7.3/roaringbitmap.egg-info/top_level.txt ADDED
@@ -0,0 +1 @@
1
+ roaringbitmap
roaringbitmap-0.7.3/setup.cfg ADDED
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
{roaringbitmap-0.7.1 → roaringbitmap-0.7.3}/setup.py RENAMED
@@ -1,8 +1,7 @@
1
1
  """Generic setup.py for Cython code."""
2
2
  import os
3
3
  import sys
4
- from distutils.core import setup
5
- from distutils.extension import Extension
4
+ from setuptools import Extension, setup
6
5
 
7
6
  PY2 = sys.version_info[0] == 2
8
7
 
@@ -33,18 +32,18 @@ with open('README.rst') as inp:
33
32
  README = inp.read()
34
33
 
35
34
  METADATA = dict(name='roaringbitmap',
36
- version='0.7.1',
35
+ version='0.7.3',
37
36
  description='Roaring Bitmap',
38
37
  long_description=README,
38
+ long_description_content_type='text/x-rst',
39
39
  author='Andreas van Cranenburgh',
40
40
  author_email='A.W.van.Cranenburgh@rug.nl',
41
41
  url='http://roaringbitmap.readthedocs.io',
42
- license='GPL',
42
+ license='GPL-2.0-or-later',
43
43
  platforms=['Many'],
44
44
  classifiers=[
45
45
  'Development Status :: 4 - Beta',
46
46
  'Intended Audience :: Science/Research',
47
- 'License :: OSI Approved :: GNU General Public License (GPL)',
48
47
  'Operating System :: POSIX',
49
48
  'Programming Language :: Python :: 2.7',
50
49
  'Programming Language :: Python :: 3.3',