b10-msgpack-numpy 0.4.8__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,18 @@
1
+ <!---
2
+ -*- mode:markdown -*-
3
+ vi:ft=markdown
4
+ -->
5
+
6
+ Authors
7
+ =======
8
+ This software was written and packaged by [Lev E. Givon](https://github.com/lebedov)
9
+
10
+ Special thanks are due to the following parties for their contributions:
11
+
12
+ - [Alex Ford](https://github.com/asford) - bug fix.
13
+ - [Colin Jermain](https://github.com/cjermain) - Python 3 support.
14
+ - [Etienne Wodey](https://github.com/airwoodix) - support for ndarrays with dtype='O'
15
+ - [John Tyree](https://github.com/johntyree) - support for numpy scalar booleans.
16
+ - [Mehdi Sadeghi](https://github.com/mehdisadeghi) - bug reports.
17
+ - [Sujoy Roy](https://github.com/tvkpz) - bug reports.
18
+ - [Jose Coutinho](https://github.com/tiagocoutinho) - make msgpack-numpy params consistent with msgpack-python.
@@ -0,0 +1,156 @@
1
+ <!--
2
+ -*- mode:markdown -*-
3
+ vi:ft=markdown
4
+ -->
5
+ Change Log
6
+ ==========
7
+
8
+ Release 0.4.8 (Baseten fork, November 2026)
9
+ -------------------------------------------
10
+ * SECURITY (CWE-502): object-dtype ndarrays are no longer unpickled by
11
+ default. ``decode``, ``unpackb``/``unpack`` and ``Unpacker`` now take an
12
+ ``allow_pickle`` argument (default ``False``) and raise ``PickleNotAllowed``
13
+ instead of calling ``pickle.loads`` on attacker-controlled bytes. Enable it
14
+ only for payloads from a trusted encoder.
15
+
16
+ Release 0.4.8 (April 28, 2022)
17
+ ------------------------------
18
+ * Add support for ndarrays with dtype=object (#46).
19
+
20
+ Release 0.4.7.1 (September 30, 2020)
21
+ ------------------------------------
22
+ * Fix Python 2.7 regression (#45).
23
+
24
+ Release 0.4.7 (September 16, 2020)
25
+ ----------------------------------
26
+ * Fix bug unpacking nested, structured dtypes (#42).
27
+
28
+ Release 0.4.6.1 (July 28, 2020)
29
+ -------------------------------
30
+ * Deprecate support for msgpack < 1.0.0 (#41).
31
+
32
+ Release 0.4.6 (May 25, 2020)
33
+ ----------------------------
34
+ * Set default use_bin_type and raw params of Packer and Unpacker classes the same as in msgpack (#39).
35
+ * Convert docs to Markdown.
36
+
37
+ Release 0.4.5 (April 12, 2020)
38
+ ------------------------------
39
+ * Set defaults appropriate to msgpack 1.0.0 (#38).
40
+
41
+ Release 0.4.4.3 (May 16, 2019)
42
+ ------------------------------
43
+ * Configure contiguous integration with Travis.
44
+ * Move unit tests out of msgpack_numpy module.
45
+ * Add workaround for issue handling n-dim arrays on MacOS (#35).
46
+
47
+ Release 0.4.4.2 (November 8, 2018)
48
+ ----------------------------------
49
+ * Fix regression handling noncontiguous arrays (#34).
50
+
51
+ Release 0.4.4.1 (September 26, 2018)
52
+ ------------------------------------
53
+ * Check Py version before defining encode and tostr funcs.
54
+ * Eliminate deprecation warnings, raise minimum required msgpack version (#31).
55
+ * Fix Python 2 support (#32).
56
+
57
+ Release 0.4.4 (September 25, 2018)
58
+ ----------------------------------
59
+ * Access ndarray memory view directly to slightly speed up encoding (#30).
60
+
61
+ Release 0.4.3.2 - (September 17, 2018)
62
+ --------------------------------------
63
+ * Update classifiers to list Py3 support (#29).
64
+
65
+ Release 0.4.3.1 - (July 11, 2018)
66
+ ---------------------------------
67
+ * Switch to numpy.frombuffer to avoid deprecation warning (#27).
68
+
69
+ Release 0.4.3 - (February 25, 2018)
70
+ -----------------------------------
71
+ * Change dependency name due to package name change (#25).
72
+
73
+ Release 0.4.2 - (December 2, 2017)
74
+ ----------------------------------
75
+ * Make decoding/encoding defaults identical to those of msgpack-python 0.4.* (#19).
76
+ * Fix handling of nested arrays in Py3 (#21).
77
+ * Improve decoding of complex values in Py3.
78
+ * Use numpy.testing functions to make unit test errors more informative.
79
+ * Make object_hook params consistent with those of msgack-python (#23).
80
+
81
+ Release 0.4.1 - (July 6, 2017)
82
+ ------------------------------
83
+ * Improve ability to decode data serialized with versions before 0.3.9 (#20).
84
+
85
+ Release 0.4.0 - (May 24, 2017)
86
+ ------------------------------
87
+ * Remove deprecated ez_setup.
88
+ * Add numpy >= 1.9.0 dependency (#18).
89
+
90
+ Release 0.3.9 - (February 7, 2017)
91
+ ----------------------------------
92
+ * Complex type handling fixes.
93
+ * Handle structured dtypes (#15).
94
+ * Note: data serialized with earlier versions cannot be deserialized with 0.3.9
95
+ (#17).
96
+
97
+ Release 0.3.8 - (Februrary 4, 2017)
98
+ -----------------------------------
99
+ * Fix deserialization with Python 3.5 (#13).
100
+
101
+ Release 0.3.7 - (March 12, 2016)
102
+ --------------------------------
103
+ * Add Python 3 support (#11).
104
+
105
+ Release 0.3.6 - (July 8, 2015)
106
+ ------------------------------
107
+ * Fall back to pure Python msgpack if compiled extension is unavailable (#6).
108
+
109
+ Release 0.3.5 - (February 12, 2015)
110
+ -----------------------------------
111
+ * Add support for numpy scalar booleans (#8).
112
+
113
+ Release 0.3.4 - (December 12, 2014)
114
+ -----------------------------------
115
+ * Add badges, update README.
116
+
117
+ Release 0.3.3 - (October 21, 2014)
118
+ ----------------------------------
119
+ * Add missing ez_setup.py file.
120
+
121
+ Release 0.3.2 - (May 6, 2014)
122
+ -----------------------------
123
+ * Simplify encoding/decoding of numpy scalars.
124
+
125
+ Release 0.3.1.1 - (March 2, 2014)
126
+ ---------------------------------
127
+ * Make package a simple module to fix dependency installation issues.
128
+
129
+ Release 0.3.1 - (March 2, 2014)
130
+ -------------------------------
131
+ * Switch to PEP 440 version numbering.
132
+ * Update to setuptools 2.2 ez_setup.py
133
+
134
+ Release 0.03 - (October 30, 2013)
135
+ ---------------------------------
136
+ * Add support for msgpack 0.4.0.
137
+ * Rename to msgpack-numpy.
138
+
139
+ Release 0.022 - (September 10, 2013)
140
+ ------------------------------------
141
+ * Fix decoding of string arrays (#4).
142
+ * Fix decoding of dicts containing ndarrays (#5).
143
+
144
+ Release 0.021 - (May 29, 2013)
145
+ ------------------------------
146
+ * Improve encoding/decoding performance for arrays.
147
+ * Fix numpy type support on different platforms (#3).
148
+
149
+ Release 0.02 - (February 21, 2013)
150
+ ----------------------------------
151
+ * Add support for msgpack 0.3.0 (contributed by crispamares).
152
+
153
+ Release 0.01 - (February 07, 2013)
154
+ ----------------------------------
155
+ * First public release.
156
+
@@ -0,0 +1,35 @@
1
+ <!---
2
+ -*- mode:markdown -*-
3
+ vi:ft=markdown
4
+ -->
5
+ License
6
+ =======
7
+
8
+ Copyright (c) 2013-2022, Lev E. Givon.
9
+ All rights reserved.
10
+
11
+ Redistribution and use in source and binary forms, with or without
12
+ modification, are permitted provided that the following conditions are
13
+ met:
14
+
15
+ * Redistributions of source code must retain the above copyright
16
+ notice, this list of conditions and the following disclaimer.
17
+ * Redistributions in binary form must reproduce the above
18
+ copyright notice, this list of conditions and the following
19
+ disclaimer in the documentation and/or other materials provided
20
+ with the distribution.
21
+ * Neither the name of Lev E. Givon nor the names of any
22
+ contributors may be used to endorse or promote products derived
23
+ from this software without specific prior written permission.
24
+
25
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
26
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
27
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
28
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
29
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
30
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
31
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
32
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
33
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
34
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
35
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,163 @@
1
+ Metadata-Version: 2.4
2
+ Name: b10-msgpack-numpy
3
+ Version: 0.4.8
4
+ Summary: Numpy data serialization using msgpack (Baseten fork with CWE-502 fix)
5
+ Home-page: https://github.com/basetenlabs/b10-msgpack-numpy
6
+ Author: Lev E. Givon
7
+ Author-email: lev@columbia.edu
8
+ License: BSD
9
+ Classifier: Development Status :: 3 - Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: License :: OSI Approved :: BSD License
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Programming Language :: Python
15
+ Classifier: Programming Language :: Python :: 2.7
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.4
18
+ Classifier: Programming Language :: Python :: 3.5
19
+ Classifier: Programming Language :: Python :: 3.6
20
+ Classifier: Programming Language :: Python :: 3.7
21
+ Classifier: Programming Language :: Python :: 3.8
22
+ Classifier: Topic :: Scientific/Engineering
23
+ Classifier: Topic :: Software Development
24
+ Description-Content-Type: text/markdown
25
+ License-File: LICENSE.md
26
+ License-File: AUTHORS.md
27
+ Requires-Dist: numpy>=1.9.0
28
+ Requires-Dist: msgpack>=0.5.2
29
+ Dynamic: author
30
+ Dynamic: author-email
31
+ Dynamic: classifier
32
+ Dynamic: description
33
+ Dynamic: description-content-type
34
+ Dynamic: home-page
35
+ Dynamic: license
36
+ Dynamic: license-file
37
+ Dynamic: requires-dist
38
+ Dynamic: summary
39
+
40
+ Package Description
41
+ -------------------
42
+ This package provides encoding and decoding routines that enable the
43
+ serialization and deserialization of numerical and array data types provided by
44
+ [numpy](http://www.numpy.org/) using the highly efficient
45
+ [msgpack](http://msgpack.org/) format. Serialization of Python's
46
+ native complex data types is also supported.
47
+
48
+ [![Latest Version](https://img.shields.io/pypi/v/msgpack-numpy.svg)](https://pypi.python.org/pypi/msgpack-numpy)
49
+ [![Build Status](https://travis-ci.org/lebedov/msgpack-numpy.svg?branch=master)](https://travis-ci.org/lebedov/msgpack-numpy)
50
+
51
+ Installation
52
+ ------------
53
+ msgpack-numpy requires msgpack-python and numpy. If you
54
+ have [pip](http://www.pip-installer.org/) installed on your
55
+ system, run
56
+
57
+ pip install msgpack-numpy
58
+
59
+ to install the package and all dependencies. You can also download
60
+ the source tarball, unpack it, and run
61
+
62
+ python setup.py install
63
+
64
+ from within the source directory.
65
+
66
+ Usage
67
+ -----
68
+ The easiest way to use msgpack-numpy is to call its monkey patching
69
+ function after importing the Python msgpack package:
70
+
71
+ import msgpack
72
+ import msgpack_numpy as m
73
+ m.patch()
74
+
75
+ This will automatically force all msgpack serialization and deserialization
76
+ routines (and other packages that use them) to become numpy-aware.
77
+ Of course, one can also manually pass the encoder and
78
+ decoder provided by msgpack-numpy to the msgpack routines:
79
+
80
+ import msgpack
81
+ import msgpack_numpy as m
82
+ import numpy as np
83
+
84
+ x = np.random.rand(5)
85
+ x_enc = msgpack.packb(x, default=m.encode)
86
+ x_rec = msgpack.unpackb(x_enc, object_hook=m.decode)
87
+
88
+ msgpack-numpy will try to use the binary (fast) extension in msgpack by default.
89
+ If msgpack was not compiled with Cython (or if the ``MSGPACK_PUREPYTHON``
90
+ variable is set), it will fall back to using the slower pure Python msgpack
91
+ implementation.
92
+
93
+ Notes
94
+ -----
95
+ The primary design goal of msgpack-numpy is ensuring preservation of numerical
96
+ data types during msgpack serialization and deserialization. Inclusion of type
97
+ information in the serialized data necessarily incurs some storage overhead; if
98
+ preservation of type information is not needed, one may be able to avoid some
99
+ of this overhead by writing a custom encoder/decoder pair that produces more
100
+ efficient serializations for those specific use cases.
101
+
102
+ Numpy arrays with a dtype of 'O' are serialized using pickle as a fallback
103
+ solution. Deserializing them requires unpickling, which executes arbitrary
104
+ code from the payload, so decoding is **disabled by default** (CWE-502):
105
+ ``decode``, ``unpackb``/``unpack`` and ``Unpacker`` raise
106
+ ``msgpack_numpy.PickleNotAllowed`` when they encounter an object-dtype array.
107
+ If you control both ends, opt in explicitly:
108
+
109
+ x_rec = msgpack.unpackb(x_enc, allow_pickle=True)
110
+
111
+ Never enable ``allow_pickle`` for untrusted input. As the overhead of pickle
112
+ serialization negates one of the reasons to use msgpack, consider writing a
113
+ custom encoder/decoder for your specific use case instead.
114
+
115
+ Note that numpy arrays deserialized by msgpack-numpy are read-only and must be copied
116
+ if they are to be modified.
117
+
118
+ Development
119
+ -----------
120
+ The latest source code can be obtained from
121
+ [GitHub](https://github.com/basetenlabs/b10-msgpack-numpy/). This is the Baseten
122
+ fork of [lebedov/msgpack-numpy](https://github.com/lebedov/msgpack-numpy); the
123
+ distribution is named ``b10-msgpack-numpy`` and still provides the
124
+ ``msgpack_numpy`` module.
125
+
126
+ msgpack-numpy maintains compatibility with python versions 2.7 and 3.5+.
127
+
128
+ Install [`tox`](https://tox.readthedocs.io/en/latest/) to support testing
129
+ across multiple python versions in your development environment. If you
130
+ use [`conda`](https://docs.conda.io/en/latest/) to install `python` use
131
+ [`tox-conda`](https://github.com/tox-dev/tox-conda) to automatically manage
132
+ testing across all supported python versions.
133
+
134
+ # Using a system python
135
+ pip install tox
136
+
137
+ # Additionally, using a conda-provided python
138
+ pip install tox tox-conda
139
+
140
+ Execute tests across supported python versions:
141
+
142
+ tox
143
+
144
+ Authors
145
+ -------
146
+ See the included [AUTHORS.md](https://github.com/lebedov/msgpack-numpy/blob/master/AUTHORS.md) file for
147
+ more information.
148
+
149
+ License
150
+ -------
151
+ This software is licensed under the [BSD License](http://www.opensource.org/licenses/bsd-license).
152
+ See the included [LICENSE.md](https://github.com/lebedov/msgpack-numpy/blob/master/LICENSE.md) file for
153
+ more information.
154
+
155
+ See Also
156
+ --------
157
+ [msgpack-numpy-js](https://github.com/emerald-geomodelling/msgpack-numpy-js) by
158
+ [Egil Möller](https://github.com/redhog) is a JavaScript library that can
159
+ serialize and deserialize JS typed arrays with msgpack.
160
+
161
+ [msgpack-numpy-rs](https://github.com/clearviewai/msgpack-numpy-rs) by [
162
+ Terrence Liu](https://github.com/terencezl) is a Rust crate (mostly) compatible with msgpack-numpy
163
+ that can serialize and deserialize numpy arrays with msgpack.
@@ -0,0 +1,131 @@
1
+ <!---
2
+ -*- mode:markdown -*-
3
+ vi:ft=markdown
4
+ -->
5
+ Numpy Data Type Serialization Using Msgpack
6
+ ===========================================
7
+
8
+ Package Description
9
+ -------------------
10
+ This package provides encoding and decoding routines that enable the
11
+ serialization and deserialization of numerical and array data types provided by
12
+ [numpy](http://www.numpy.org/) using the highly efficient
13
+ [msgpack](http://msgpack.org/) format. Serialization of Python's
14
+ native complex data types is also supported.
15
+
16
+ [![Latest Version](https://img.shields.io/pypi/v/msgpack-numpy.svg)](https://pypi.python.org/pypi/msgpack-numpy)
17
+ [![Build Status](https://travis-ci.org/lebedov/msgpack-numpy.svg?branch=master)](https://travis-ci.org/lebedov/msgpack-numpy)
18
+
19
+ Installation
20
+ ------------
21
+ msgpack-numpy requires msgpack-python and numpy. If you
22
+ have [pip](http://www.pip-installer.org/) installed on your
23
+ system, run
24
+
25
+ pip install msgpack-numpy
26
+
27
+ to install the package and all dependencies. You can also download
28
+ the source tarball, unpack it, and run
29
+
30
+ python setup.py install
31
+
32
+ from within the source directory.
33
+
34
+ Usage
35
+ -----
36
+ The easiest way to use msgpack-numpy is to call its monkey patching
37
+ function after importing the Python msgpack package:
38
+
39
+ import msgpack
40
+ import msgpack_numpy as m
41
+ m.patch()
42
+
43
+ This will automatically force all msgpack serialization and deserialization
44
+ routines (and other packages that use them) to become numpy-aware.
45
+ Of course, one can also manually pass the encoder and
46
+ decoder provided by msgpack-numpy to the msgpack routines:
47
+
48
+ import msgpack
49
+ import msgpack_numpy as m
50
+ import numpy as np
51
+
52
+ x = np.random.rand(5)
53
+ x_enc = msgpack.packb(x, default=m.encode)
54
+ x_rec = msgpack.unpackb(x_enc, object_hook=m.decode)
55
+
56
+ msgpack-numpy will try to use the binary (fast) extension in msgpack by default.
57
+ If msgpack was not compiled with Cython (or if the ``MSGPACK_PUREPYTHON``
58
+ variable is set), it will fall back to using the slower pure Python msgpack
59
+ implementation.
60
+
61
+ Notes
62
+ -----
63
+ The primary design goal of msgpack-numpy is ensuring preservation of numerical
64
+ data types during msgpack serialization and deserialization. Inclusion of type
65
+ information in the serialized data necessarily incurs some storage overhead; if
66
+ preservation of type information is not needed, one may be able to avoid some
67
+ of this overhead by writing a custom encoder/decoder pair that produces more
68
+ efficient serializations for those specific use cases.
69
+
70
+ Numpy arrays with a dtype of 'O' are serialized using pickle as a fallback
71
+ solution. Deserializing them requires unpickling, which executes arbitrary
72
+ code from the payload, so decoding is **disabled by default** (CWE-502):
73
+ ``decode``, ``unpackb``/``unpack`` and ``Unpacker`` raise
74
+ ``msgpack_numpy.PickleNotAllowed`` when they encounter an object-dtype array.
75
+ If you control both ends, opt in explicitly:
76
+
77
+ x_rec = msgpack.unpackb(x_enc, allow_pickle=True)
78
+
79
+ Never enable ``allow_pickle`` for untrusted input. As the overhead of pickle
80
+ serialization negates one of the reasons to use msgpack, consider writing a
81
+ custom encoder/decoder for your specific use case instead.
82
+
83
+ Note that numpy arrays deserialized by msgpack-numpy are read-only and must be copied
84
+ if they are to be modified.
85
+
86
+ Development
87
+ -----------
88
+ The latest source code can be obtained from
89
+ [GitHub](https://github.com/basetenlabs/b10-msgpack-numpy/). This is the Baseten
90
+ fork of [lebedov/msgpack-numpy](https://github.com/lebedov/msgpack-numpy); the
91
+ distribution is named ``b10-msgpack-numpy`` and still provides the
92
+ ``msgpack_numpy`` module.
93
+
94
+ msgpack-numpy maintains compatibility with python versions 2.7 and 3.5+.
95
+
96
+ Install [`tox`](https://tox.readthedocs.io/en/latest/) to support testing
97
+ across multiple python versions in your development environment. If you
98
+ use [`conda`](https://docs.conda.io/en/latest/) to install `python` use
99
+ [`tox-conda`](https://github.com/tox-dev/tox-conda) to automatically manage
100
+ testing across all supported python versions.
101
+
102
+ # Using a system python
103
+ pip install tox
104
+
105
+ # Additionally, using a conda-provided python
106
+ pip install tox tox-conda
107
+
108
+ Execute tests across supported python versions:
109
+
110
+ tox
111
+
112
+ Authors
113
+ -------
114
+ See the included [AUTHORS.md](https://github.com/lebedov/msgpack-numpy/blob/master/AUTHORS.md) file for
115
+ more information.
116
+
117
+ License
118
+ -------
119
+ This software is licensed under the [BSD License](http://www.opensource.org/licenses/bsd-license).
120
+ See the included [LICENSE.md](https://github.com/lebedov/msgpack-numpy/blob/master/LICENSE.md) file for
121
+ more information.
122
+
123
+ See Also
124
+ --------
125
+ [msgpack-numpy-js](https://github.com/emerald-geomodelling/msgpack-numpy-js) by
126
+ [Egil Möller](https://github.com/redhog) is a JavaScript library that can
127
+ serialize and deserialize JS typed arrays with msgpack.
128
+
129
+ [msgpack-numpy-rs](https://github.com/clearviewai/msgpack-numpy-rs) by [
130
+ Terrence Liu](https://github.com/terencezl) is a Rust crate (mostly) compatible with msgpack-numpy
131
+ that can serialize and deserialize numpy arrays with msgpack.
@@ -0,0 +1,163 @@
1
+ Metadata-Version: 2.4
2
+ Name: b10-msgpack-numpy
3
+ Version: 0.4.8
4
+ Summary: Numpy data serialization using msgpack (Baseten fork with CWE-502 fix)
5
+ Home-page: https://github.com/basetenlabs/b10-msgpack-numpy
6
+ Author: Lev E. Givon
7
+ Author-email: lev@columbia.edu
8
+ License: BSD
9
+ Classifier: Development Status :: 3 - Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: License :: OSI Approved :: BSD License
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Programming Language :: Python
15
+ Classifier: Programming Language :: Python :: 2.7
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.4
18
+ Classifier: Programming Language :: Python :: 3.5
19
+ Classifier: Programming Language :: Python :: 3.6
20
+ Classifier: Programming Language :: Python :: 3.7
21
+ Classifier: Programming Language :: Python :: 3.8
22
+ Classifier: Topic :: Scientific/Engineering
23
+ Classifier: Topic :: Software Development
24
+ Description-Content-Type: text/markdown
25
+ License-File: LICENSE.md
26
+ License-File: AUTHORS.md
27
+ Requires-Dist: numpy>=1.9.0
28
+ Requires-Dist: msgpack>=0.5.2
29
+ Dynamic: author
30
+ Dynamic: author-email
31
+ Dynamic: classifier
32
+ Dynamic: description
33
+ Dynamic: description-content-type
34
+ Dynamic: home-page
35
+ Dynamic: license
36
+ Dynamic: license-file
37
+ Dynamic: requires-dist
38
+ Dynamic: summary
39
+
40
+ Package Description
41
+ -------------------
42
+ This package provides encoding and decoding routines that enable the
43
+ serialization and deserialization of numerical and array data types provided by
44
+ [numpy](http://www.numpy.org/) using the highly efficient
45
+ [msgpack](http://msgpack.org/) format. Serialization of Python's
46
+ native complex data types is also supported.
47
+
48
+ [![Latest Version](https://img.shields.io/pypi/v/msgpack-numpy.svg)](https://pypi.python.org/pypi/msgpack-numpy)
49
+ [![Build Status](https://travis-ci.org/lebedov/msgpack-numpy.svg?branch=master)](https://travis-ci.org/lebedov/msgpack-numpy)
50
+
51
+ Installation
52
+ ------------
53
+ msgpack-numpy requires msgpack-python and numpy. If you
54
+ have [pip](http://www.pip-installer.org/) installed on your
55
+ system, run
56
+
57
+ pip install msgpack-numpy
58
+
59
+ to install the package and all dependencies. You can also download
60
+ the source tarball, unpack it, and run
61
+
62
+ python setup.py install
63
+
64
+ from within the source directory.
65
+
66
+ Usage
67
+ -----
68
+ The easiest way to use msgpack-numpy is to call its monkey patching
69
+ function after importing the Python msgpack package:
70
+
71
+ import msgpack
72
+ import msgpack_numpy as m
73
+ m.patch()
74
+
75
+ This will automatically force all msgpack serialization and deserialization
76
+ routines (and other packages that use them) to become numpy-aware.
77
+ Of course, one can also manually pass the encoder and
78
+ decoder provided by msgpack-numpy to the msgpack routines:
79
+
80
+ import msgpack
81
+ import msgpack_numpy as m
82
+ import numpy as np
83
+
84
+ x = np.random.rand(5)
85
+ x_enc = msgpack.packb(x, default=m.encode)
86
+ x_rec = msgpack.unpackb(x_enc, object_hook=m.decode)
87
+
88
+ msgpack-numpy will try to use the binary (fast) extension in msgpack by default.
89
+ If msgpack was not compiled with Cython (or if the ``MSGPACK_PUREPYTHON``
90
+ variable is set), it will fall back to using the slower pure Python msgpack
91
+ implementation.
92
+
93
+ Notes
94
+ -----
95
+ The primary design goal of msgpack-numpy is ensuring preservation of numerical
96
+ data types during msgpack serialization and deserialization. Inclusion of type
97
+ information in the serialized data necessarily incurs some storage overhead; if
98
+ preservation of type information is not needed, one may be able to avoid some
99
+ of this overhead by writing a custom encoder/decoder pair that produces more
100
+ efficient serializations for those specific use cases.
101
+
102
+ Numpy arrays with a dtype of 'O' are serialized using pickle as a fallback
103
+ solution. Deserializing them requires unpickling, which executes arbitrary
104
+ code from the payload, so decoding is **disabled by default** (CWE-502):
105
+ ``decode``, ``unpackb``/``unpack`` and ``Unpacker`` raise
106
+ ``msgpack_numpy.PickleNotAllowed`` when they encounter an object-dtype array.
107
+ If you control both ends, opt in explicitly:
108
+
109
+ x_rec = msgpack.unpackb(x_enc, allow_pickle=True)
110
+
111
+ Never enable ``allow_pickle`` for untrusted input. As the overhead of pickle
112
+ serialization negates one of the reasons to use msgpack, consider writing a
113
+ custom encoder/decoder for your specific use case instead.
114
+
115
+ Note that numpy arrays deserialized by msgpack-numpy are read-only and must be copied
116
+ if they are to be modified.
117
+
118
+ Development
119
+ -----------
120
+ The latest source code can be obtained from
121
+ [GitHub](https://github.com/basetenlabs/b10-msgpack-numpy/). This is the Baseten
122
+ fork of [lebedov/msgpack-numpy](https://github.com/lebedov/msgpack-numpy); the
123
+ distribution is named ``b10-msgpack-numpy`` and still provides the
124
+ ``msgpack_numpy`` module.
125
+
126
+ msgpack-numpy maintains compatibility with python versions 2.7 and 3.5+.
127
+
128
+ Install [`tox`](https://tox.readthedocs.io/en/latest/) to support testing
129
+ across multiple python versions in your development environment. If you
130
+ use [`conda`](https://docs.conda.io/en/latest/) to install `python` use
131
+ [`tox-conda`](https://github.com/tox-dev/tox-conda) to automatically manage
132
+ testing across all supported python versions.
133
+
134
+ # Using a system python
135
+ pip install tox
136
+
137
+ # Additionally, using a conda-provided python
138
+ pip install tox tox-conda
139
+
140
+ Execute tests across supported python versions:
141
+
142
+ tox
143
+
144
+ Authors
145
+ -------
146
+ See the included [AUTHORS.md](https://github.com/lebedov/msgpack-numpy/blob/master/AUTHORS.md) file for
147
+ more information.
148
+
149
+ License
150
+ -------
151
+ This software is licensed under the [BSD License](http://www.opensource.org/licenses/bsd-license).
152
+ See the included [LICENSE.md](https://github.com/lebedov/msgpack-numpy/blob/master/LICENSE.md) file for
153
+ more information.
154
+
155
+ See Also
156
+ --------
157
+ [msgpack-numpy-js](https://github.com/emerald-geomodelling/msgpack-numpy-js) by
158
+ [Egil Möller](https://github.com/redhog) is a JavaScript library that can
159
+ serialize and deserialize JS typed arrays with msgpack.
160
+
161
+ [msgpack-numpy-rs](https://github.com/clearviewai/msgpack-numpy-rs) by [
162
+ Terrence Liu](https://github.com/terencezl) is a Rust crate (mostly) compatible with msgpack-numpy
163
+ that can serialize and deserialize numpy arrays with msgpack.
@@ -0,0 +1,14 @@
1
+ AUTHORS.md
2
+ CHANGES.md
3
+ LICENSE.md
4
+ README.md
5
+ msgpack_numpy.py
6
+ pyproject.toml
7
+ setup.cfg
8
+ setup.py
9
+ tests.py
10
+ b10_msgpack_numpy.egg-info/PKG-INFO
11
+ b10_msgpack_numpy.egg-info/SOURCES.txt
12
+ b10_msgpack_numpy.egg-info/dependency_links.txt
13
+ b10_msgpack_numpy.egg-info/requires.txt
14
+ b10_msgpack_numpy.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ numpy>=1.9.0
2
+ msgpack>=0.5.2