pysz 0.0.7__tar.gz

pysz-0.0.7/PKG-INFO

@@ -0,0 +1,96 @@
+Metadata-Version: 2.1
+Name: pysz
+Version: 0.0.7
+Summary: A python wrapper for the SZ3 compression library
+Author-email: Zhuoxun Yang <jason.neptune.yang@gmail.com>
+Project-URL: Homepage, https://github.com/JasonYang60/pysz
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+
+# pysz
+A user-friendly and easy-to-maintain python wrapper for SZ3 using a python-to-c package Cython>=3.0.10.\
+Pysz is fully based on SZ3(https://github.com/szcompressor/SZ3). The pure SZ3 C++ library is incorporated into pysz interface through Cython, so the compression && decompression function is running much faster than Python code.
+
+
+## How to use pysz
+We provided a test code in `./test/test.py` as shown below:
+```python
+from pysz import sz
+
+# setup configuration of compression
+dim = (101, 203869, 3)
+compressor = sz.sz(*dim)
+filename = 'traj_xyz.dat'
+datatype = 'float'
+
+# compression
+cmpData3, cmp_ratio_2 = compressor.compress(datatype, 'sz3.config', filename)
+
+# decompression
+decmpData2 = compressor.decompress(datatype, 'sz3.config', filename + '.sz')
+
+# verification
+compressor.verify(datatype, 'sz3.config', filename, filename + '.sz')
+```
+The input data can either be the file path to the data file or numpy array contain the data.
+## Requirements
+Python>=3.10
+
+## Installation
+
+### Method 1. install from .whl
+If you are running a linux system and using x86 or x64 CPU platform, pysz can be installed directly from a pre-compiled `.whl` file(eg. `pysz-0.0.2-cp312-cp312-linux_x86_64.whl`) located in `./dist/`. 
+```bash
+git clone https://github.com/JasonYang60/pysz.git
+cd pysz
+mv ./dist/pysz-0.0.2-cp312-cp312-linux_x86_64.whl ./
+pip install pysz-0.0.2-cp312-cp312-linux_x86_64.whl
+```
+### Method 2. local compilation
+According to your system && CPU platform, if a coresponding `.whl` is absent, you should compile pysz locally.\
+First, Make sure cmake && gcc installed.\
+Pysz uses python-to-c package Cython>=3.0.10, so what `setup.py` does includes downloading c++ source code from github and automatically applying cmake && make command to finish c++ code compilation.\
+Then
+```bash
+pip install .
+```
+
+#### Generate pre-compiled .whl and .tar.gz file
+To pack this project up with .whl file, some python setup tool package (`wheel`, `build`) and are required.
+```bash
+pip install wheel
+pip install build
+```
+Run following code to generate `.whl` and `.tar.gz` in `./dist`:
+```bash
+python -m build
+```
+tip: there is still some issues remaining to be fixed. This command will also work:
+```bash
+python setup.py sdist bdist_wheel
+```
+### Method 3. install from Pypi
+```bash
+pip install pysz
+```
+
+## Launch in docker
+
+Dockerfile is provided for testing. 
+```bash
+docker build --name pysz .
+docker run -it --name pysz_container pysz /bin/bash
+```
+Then in container shell:
+```bash
+pip install .
+cd test
+python ./test.py
+```
+
+## TODO list:
+to test openMP and other utils && features \
+more data types to be supported

pysz-0.0.7/README.md

@@ -0,0 +1,84 @@
+# pysz
+A user-friendly and easy-to-maintain python wrapper for SZ3 using a python-to-c package Cython>=3.0.10.\
+Pysz is fully based on SZ3(https://github.com/szcompressor/SZ3). The pure SZ3 C++ library is incorporated into pysz interface through Cython, so the compression && decompression function is running much faster than Python code.
+
+
+## How to use pysz
+We provided a test code in `./test/test.py` as shown below:
+```python
+from pysz import sz
+
+# setup configuration of compression
+dim = (101, 203869, 3)
+compressor = sz.sz(*dim)
+filename = 'traj_xyz.dat'
+datatype = 'float'
+
+# compression
+cmpData3, cmp_ratio_2 = compressor.compress(datatype, 'sz3.config', filename)
+
+# decompression
+decmpData2 = compressor.decompress(datatype, 'sz3.config', filename + '.sz')
+
+# verification
+compressor.verify(datatype, 'sz3.config', filename, filename + '.sz')
+```
+The input data can either be the file path to the data file or numpy array contain the data.
+## Requirements
+Python>=3.10
+
+## Installation
+
+### Method 1. install from .whl
+If you are running a linux system and using x86 or x64 CPU platform, pysz can be installed directly from a pre-compiled `.whl` file(eg. `pysz-0.0.2-cp312-cp312-linux_x86_64.whl`) located in `./dist/`. 
+```bash
+git clone https://github.com/JasonYang60/pysz.git
+cd pysz
+mv ./dist/pysz-0.0.2-cp312-cp312-linux_x86_64.whl ./
+pip install pysz-0.0.2-cp312-cp312-linux_x86_64.whl
+```
+### Method 2. local compilation
+According to your system && CPU platform, if a coresponding `.whl` is absent, you should compile pysz locally.\
+First, Make sure cmake && gcc installed.\
+Pysz uses python-to-c package Cython>=3.0.10, so what `setup.py` does includes downloading c++ source code from github and automatically applying cmake && make command to finish c++ code compilation.\
+Then
+```bash
+pip install .
+```
+
+#### Generate pre-compiled .whl and .tar.gz file
+To pack this project up with .whl file, some python setup tool package (`wheel`, `build`) and are required.
+```bash
+pip install wheel
+pip install build
+```
+Run following code to generate `.whl` and `.tar.gz` in `./dist`:
+```bash
+python -m build
+```
+tip: there is still some issues remaining to be fixed. This command will also work:
+```bash
+python setup.py sdist bdist_wheel
+```
+### Method 3. install from Pypi
+```bash
+pip install pysz
+```
+
+## Launch in docker
+
+Dockerfile is provided for testing. 
+```bash
+docker build --name pysz .
+docker run -it --name pysz_container pysz /bin/bash
+```
+Then in container shell:
+```bash
+pip install .
+cd test
+python ./test.py
+```
+
+## TODO list:
+to test openMP and other utils && features \
+more data types to be supported

pysz-0.0.7/SZ3-bio/include/SZ3/api/sz.hpp

@@ -0,0 +1,145 @@
+#ifndef SZ3_SZ_HPP
+#define SZ3_SZ_HPP
+
+
+#include "SZ3/api/impl/SZImpl.hpp"
+#include "SZ3/version.hpp"
+#include <memory>
+
+/**
+ * API for compression
+ * @tparam T source data type
+ * @param config compression configuration. Please update the config with 1). data dimension and shape and 2). desired settings.
+ * @param data source data
+ * @param cmpSize compressed data size in bytes
+ * @return compressed data, remember to 'delete []' when the data is no longer needed.
+
+The compression algorithms are:
+ALGO_INTERP_LORENZO:
+ The default algorithm in SZ3. It is the implementation of our ICDE'21 paper.
+ The whole dataset will be compressed by interpolation or lorenzo predictor with auto-optimized settings.
+ALGO_INTERP:
+ The whole dataset will be compressed by interpolation predictor with default settings.
+ALGO_LORENZO_REG:
+ The whole dataset will be compressed by lorenzo and/or regression based predictors block by block with default settings.
+ The four predictors ( 1st-order lorenzo, 2nd-order lorenzo, 1st-order regression, 2nd-order regression)
+ can be enabled or disabled independently by conf settings (lorenzo, lorenzo2, regression, regression2).
+
+Interpolation+lorenzo example:
+SZ3::Config conf(100, 200, 300); // 300 is the fastest dimension
+conf.cmprAlgo = SZ3::ALGO_INTERP_LORENZO;
+conf.errorBoundMode = SZ3::EB_ABS; // refer to def.hpp for all supported error bound mode
+conf.absErrorBound = 1E-3; // absolute error bound 1e-3
+char *compressedData = SZ_compress(conf, data, outSize);
+
+Interpolation example:
+SZ3::Config conf(100, 200, 300); // 300 is the fastest dimension
+conf.cmprAlgo = SZ3::ALGO_INTERP;
+conf.errorBoundMode = SZ3::EB_REL; // refer to def.hpp for all supported error bound mode
+conf.relErrorBound = 1E-3; // value-rang-based error bound 1e-3
+char *compressedData = SZ_compress(conf, data, outSize);
+
+Lorenzo/regression example :
+SZ3::Config conf(100, 200, 300); // 300 is the fastest dimension
+conf.cmprAlgo = SZ3::ALGO_LORENZO_REG;
+conf.lorenzo = true; // only use 1st order lorenzo
+conf.lorenzo2 = false;
+conf.regression = false;
+conf.regression2 = false;
+conf.errorBoundMode = SZ3::EB_ABS; // refer to def.hpp for all supported error bound mode
+conf.absErrorBound = 1E-3; // absolute error bound 1e-3
+char *compressedData = SZ_compress(conf, data, outSize);
+ */
+template<class T>
+char *SZ_compress(const SZ3::Config &conf, const T *data, size_t &cmpSize) {
+    using namespace SZ3;
+    Config confCopy(conf);
+    char *cmpData;
+    if (conf.N == 1) {
+        cmpData = SZ_compress_impl<T, 1>(confCopy, data, cmpSize);
+    } else if (conf.N == 2) {
+        cmpData = SZ_compress_impl<T, 2>(confCopy, data, cmpSize);
+    } else if (conf.N == 3) {
+        cmpData = SZ_compress_impl<T, 3>(confCopy, data, cmpSize);
+    } else if (conf.N == 4) {
+        cmpData = SZ_compress_impl<T, 4>(confCopy, data, cmpSize);
+    } else {
+        printf("Data dimension higher than 4 is not supported.\n");
+        exit(0);
+    }
+    {
+        //save config
+        uchar *cmpDataPos = (uchar *) cmpData + cmpSize;
+        confCopy.save(cmpDataPos);
+        size_t newSize = (char *) cmpDataPos - cmpData;
+        write(int(newSize - cmpSize), cmpDataPos);
+        cmpSize = (char *) cmpDataPos - cmpData;
+    }
+    return cmpData;
+}
+
+
+/**
+ * API for decompression
+ * Similar with SZ_decompress(SZ3::Config &conf, char *cmpData, size_t cmpSize)
+ * The only difference is this one needs pre-allocated decData as input
+ * @tparam T decompressed data type
+ * @param conf configuration placeholder. It will be overwritten by the compression configuration
+ * @param cmpData compressed data
+ * @param cmpSize compressed data size in bytes
+ * @param decData pre-allocated memory space for decompressed data
+
+ example:
+ auto decData = new float[100*200*300];
+ SZ3::Config conf;
+ SZ_decompress(conf, cmpData, cmpSize, decData);
+
+ */
+template<class T>
+void SZ_decompress(SZ3::Config &conf, char *cmpData, size_t cmpSize, T *&decData) {
+    using namespace SZ3;
+    {
+        //load config
+        int confSize;
+        memcpy(&confSize, cmpData + (cmpSize - sizeof(int)), sizeof(int));
+        uchar const *cmpDataPos = (uchar *) cmpData + (cmpSize - sizeof(int) - confSize);
+        conf.load(cmpDataPos);
+    }
+    if (decData == nullptr) {
+        decData = new T[conf.num];
+    }
+    if (conf.N == 1) {
+        SZ_decompress_impl<T, 1>(conf, cmpData, cmpSize, decData);
+    } else if (conf.N == 2) {
+        SZ_decompress_impl<T, 2>(conf, cmpData, cmpSize, decData);
+    } else if (conf.N == 3) {
+        SZ_decompress_impl<T, 3>(conf, cmpData, cmpSize, decData);
+    } else if (conf.N == 4) {
+        SZ_decompress_impl<T, 4>(conf, cmpData, cmpSize, decData);
+    } else {
+        printf("Data dimension higher than 4 is not supported.\n");
+        exit(0);
+    }
+}
+
+/**
+ * API for decompression
+ * @tparam T decompressed data type
+ * @param conf configuration placeholder. It will be overwritten by the compression configuration
+ * @param cmpData compressed data
+ * @param cmpSize compressed data size in bytes
+ * @return decompressed data, remember to 'delete []' when the data is no longer needed.
+
+ example:
+ SZ3::Config conf;
+ float decompressedData = SZ_decompress(conf, cmpData, cmpSize)
+ */
+template<class T>
+T *SZ_decompress(SZ3::Config &conf, char *cmpData, size_t cmpSize) {
+    using namespace SZ3;
+    T *decData = nullptr;
+    SZ_decompress<T>(conf, cmpData, cmpSize, decData);
+    return decData;
+}
+
+#endif

pysz-0.0.7/SZ3-bio/include/SZ3/utils/Config.hpp

@@ -0,0 +1,267 @@
+//
+// Created by Kai Zhao on 4/28/20.
+//
+
+#ifndef SZ_Config_HPP
+#define SZ_Config_HPP
+
+#include <iostream>
+#include <vector>
+#include <numeric>
+#include <cstdint>
+#include "SZ3/def.hpp"
+#include "MemoryUtil.hpp"
+#include "SZ3/utils/inih/INIReader.h"
+
+#define SZ_FLOAT 0
+#define SZ_DOUBLE 1
+#define SZ_UINT8 2
+#define SZ_INT8 3
+#define SZ_UINT16 4
+#define SZ_INT16 5
+#define SZ_UINT32 6
+#define SZ_INT32 7
+#define SZ_UINT64 8
+#define SZ_INT64 9
+
+namespace SZ3 {
+    enum EB {
+        EB_ABS, EB_REL, EB_PSNR, EB_L2NORM, EB_ABS_AND_REL, EB_ABS_OR_REL
+    };
+    
+    constexpr const char *EB_STR[] = {"ABS", "REL", "PSNR", "NORM", "ABS_AND_REL", "ABS_OR_REL"};
+    constexpr EB EB_OPTIONS[] = {EB_ABS, EB_REL, EB_PSNR, EB_L2NORM, EB_ABS_AND_REL, EB_ABS_OR_REL};
+    
+    enum ALGO {
+        ALGO_LORENZO_REG, ALGO_INTERP_LORENZO, ALGO_INTERP, ALGO_BIOMD, ALGO_BIOMDXTC
+    };
+    constexpr const char *ALGO_STR[] = {
+        "ALGO_LORENZO_REG", "ALGO_INTERP_LORENZO", "ALGO_INTERP", "ALGO_BIOMD", "ALGO_BIOMDXTC"
+    };
+    constexpr const ALGO ALGO_OPTIONS[] = {
+        ALGO_LORENZO_REG, ALGO_INTERP_LORENZO, ALGO_INTERP, ALGO_BIOMD, ALGO_BIOMDXTC
+    };
+    
+    enum INTERP_ALGO {
+        INTERP_ALGO_LINEAR, INTERP_ALGO_CUBIC
+    };
+    
+    constexpr const char *INTERP_ALGO_STR[] = {"INTERP_ALGO_LINEAR", "INTERP_ALGO_CUBIC"};
+    constexpr INTERP_ALGO INTERP_ALGO_OPTIONS[] = {INTERP_ALGO_LINEAR, INTERP_ALGO_CUBIC};
+    
+    template<class T>
+    const char *enum2Str(T e) {
+        if (std::is_same<T, ALGO>::value) {
+            return ALGO_STR[e];
+        } else if (std::is_same<T, INTERP_ALGO>::value) {
+            return INTERP_ALGO_STR[e];
+        } else if (std::is_same<T, EB>::value) {
+            return EB_STR[e];
+        } else {
+            printf("invalid enum type for enum2Str()\n ");
+            exit(0);
+        }
+    }
+    
+    class Config {
+     public:
+        template<class... Dims>
+        Config(Dims... args) {
+            dims = std::vector<size_t>{static_cast<size_t>(std::forward<Dims>(args))...};
+            setDims(dims.begin(), dims.end());
+        }
+        
+        template<class Iter>
+        size_t setDims(Iter begin, Iter end) {
+            auto dims_ = std::vector<size_t>(begin, end);
+            dims.clear();
+            for (auto dim : dims_) {
+                if (dim > 1) {
+                    dims.push_back(dim);
+                }
+            }
+            if (dims.empty()) {
+                dims = {1};
+            }
+            N = dims.size();
+            num = std::accumulate(dims.begin(), dims.end(), (size_t) 1, std::multiplies<size_t>());
+            pred_dim = N;
+            blockSize = (N == 1 ? 128 : (N == 2 ? 16 : 6));
+            stride = blockSize;
+            return num;
+        }
+        
+        void loadcfg(const std::string &cfgpath) {
+            INIReader cfg(cfgpath);
+            
+            if (cfg.ParseError() != 0) {
+                std::cout << "Can't load cfg file " << cfgpath << std::endl;
+                exit(0);
+            }
+            
+            auto cmprAlgoStr = cfg.Get("GlobalSettings", "CmprAlgo", "");
+            if (cmprAlgoStr == ALGO_STR[ALGO_LORENZO_REG]) {
+                cmprAlgo = ALGO_LORENZO_REG;
+            } else if (cmprAlgoStr == ALGO_STR[ALGO_INTERP_LORENZO]) {
+                cmprAlgo = ALGO_INTERP_LORENZO;
+            } else if (cmprAlgoStr == ALGO_STR[ALGO_INTERP]) {
+                cmprAlgo = ALGO_INTERP;
+            } else if (cmprAlgoStr == ALGO_STR[ALGO_BIOMD]) {
+                cmprAlgo = ALGO_BIOMD;
+            } else if (cmprAlgoStr == ALGO_STR[ALGO_BIOMDXTC]) {
+                cmprAlgo = ALGO_BIOMDXTC;
+            }
+            auto ebModeStr = cfg.Get("GlobalSettings", "ErrorBoundMode", "");
+            if (ebModeStr == EB_STR[EB_ABS]) {
+                errorBoundMode = EB_ABS;
+            } else if (ebModeStr == EB_STR[EB_REL]) {
+                errorBoundMode = EB_REL;
+            } else if (ebModeStr == EB_STR[EB_PSNR]) {
+                errorBoundMode = EB_PSNR;
+            } else if (ebModeStr == EB_STR[EB_L2NORM]) {
+                errorBoundMode = EB_L2NORM;
+            } else if (ebModeStr == EB_STR[EB_ABS_AND_REL]) {
+                errorBoundMode = EB_ABS_AND_REL;
+            } else if (ebModeStr == EB_STR[EB_ABS_OR_REL]) {
+                errorBoundMode = EB_ABS_OR_REL;
+            }
+            absErrorBound = cfg.GetReal("GlobalSettings", "AbsErrorBound", absErrorBound);
+            relErrorBound = cfg.GetReal("GlobalSettings", "RelErrorBound", relErrorBound);
+            psnrErrorBound = cfg.GetReal("GlobalSettings", "PSNRErrorBound", psnrErrorBound);
+            l2normErrorBound = cfg.GetReal("GlobalSettings", "L2NormErrorBound", l2normErrorBound);
+            
+            openmp = cfg.GetBoolean("GlobalSettings", "OpenMP", openmp);
+            lorenzo = cfg.GetBoolean("AlgoSettings", "Lorenzo", lorenzo);
+            lorenzo2 = cfg.GetBoolean("AlgoSettings", "Lorenzo2ndOrder", lorenzo2);
+            regression = cfg.GetBoolean("AlgoSettings", "Regression", regression);
+            regression2 = cfg.GetBoolean("AlgoSettings", "Regression2ndOrder", regression2);
+            
+            auto interpAlgoStr = cfg.Get("AlgoSettings", "InterpolationAlgo", "");
+            if (interpAlgoStr == INTERP_ALGO_STR[INTERP_ALGO_LINEAR]) {
+                interpAlgo = INTERP_ALGO_LINEAR;
+            } else if (interpAlgoStr == INTERP_ALGO_STR[INTERP_ALGO_CUBIC]) {
+                interpAlgo = INTERP_ALGO_CUBIC;
+            }
+            interpDirection = cfg.GetInteger("AlgoSettings", "InterpolationDirection", interpDirection);
+            interpBlockSize = cfg.GetInteger("AlgoSettings", "InterpolationBlockSize", interpBlockSize);
+            blockSize = cfg.GetInteger("AlgoSettings", "BlockSize", blockSize);
+            quantbinCnt = cfg.GetInteger("AlgoSettings", "QuantizationBinTotal", quantbinCnt);
+        }
+        
+        void save(unsigned char *&c) {
+            write(N, c);
+            write(dims.data(), dims.size(), c);
+            write(num, c);
+            write(cmprAlgo, c);
+            write(errorBoundMode, c);
+            write(absErrorBound, c);
+            write(relErrorBound, c);
+            write(lorenzo, c);
+            write(lorenzo2, c);
+            write(regression, c);
+            write(regression2, c);
+            write(interpAlgo, c);
+            write(interpDirection, c);
+            write(interpBlockSize, c);
+            write(lossless, c);
+            write(encoder, c);
+            write(quantbinCnt, c);
+            write(blockSize, c);
+            write(stride, c);
+            write(pred_dim, c);
+            write(openmp, c);
+            write(dataType, c);
+        };
+        
+        void load(const unsigned char *&c) {
+            read(N, c);
+            dims.resize(N);
+            read(dims.data(), N, c);
+            read(num, c);
+            read(cmprAlgo, c);
+            read(errorBoundMode, c);
+            read(absErrorBound, c);
+            read(relErrorBound, c);
+            read(lorenzo, c);
+            read(lorenzo2, c);
+            read(regression, c);
+            read(regression2, c);
+            read(interpAlgo, c);
+            read(interpDirection, c);
+            read(interpBlockSize, c);
+            read(lossless, c);
+            read(encoder, c);
+            read(quantbinCnt, c);
+            read(blockSize, c);
+            read(stride, c);
+            read(pred_dim, c);
+            read(openmp, c);
+            read(dataType, c);
+        }
+        
+        void print() {
+            printf("===================== Begin SZ3 Configuration =====================\n");
+            printf("N = %d\n", N);
+            printf("dims = ");
+            for (auto dim : dims) {
+                printf("%zu ", dim);
+            }
+            printf("\nnum = %zu\n", num);
+            printf("CmprAlgo = %s\n", enum2Str((ALGO) cmprAlgo));
+            printf("ErrorBoundMode = %s\n", enum2Str((EB) errorBoundMode));
+            printf("AbsErrorBound = %f\n", absErrorBound);
+            printf("RelErrorBound = %f\n", relErrorBound);
+            printf("PSNRErrorBound = %f\n", psnrErrorBound);
+            printf("L2NormErrorBound = %f\n", l2normErrorBound);
+            printf("Lorenzo = %d\n", lorenzo);
+            printf("Lorenzo2ndOrder = %d\n", lorenzo2);
+            printf("Regression = %d\n", regression);
+            printf("Regression2ndOrder = %d\n", regression2);
+            printf("OpenMP = %d\n", openmp);
+            printf("DataType = %d\n", dataType);
+            printf("Lossless = %d\n", lossless);
+            printf("Encoder = %d\n", encoder);
+            printf("InterpolationAlgo = %s\n", enum2Str((INTERP_ALGO) interpAlgo));
+            printf("InterpolationDirection = %d\n", interpDirection);
+            printf("InterpolationBlockSize = %d\n", interpBlockSize);
+            printf("QuantizationBinTotal = %d\n", quantbinCnt);
+            printf("BlockSize = %d\n", blockSize);
+            printf("Stride = %d\n", stride);
+            printf("PredDim = %d\n", pred_dim);
+            printf("===================== End SZ3 Configuration =====================\n");
+        }
+        
+        static size_t size_est() {
+            return sizeof(Config) + sizeof(size_t) * 5 + 32; //sizeof(size_t) * 5 is for dims vector, 32 is for redundancy
+            return sizeof(size_t) * 5 + sizeof(double) * 4 + sizeof(bool) * 5 + sizeof(uint8_t) * 7 + sizeof(int) * 5 +
+                50; //50 is for redundancy
+        }
+        
+        char N;
+        std::vector<size_t> dims;
+        size_t num;
+        uint8_t cmprAlgo = ALGO_INTERP_LORENZO;
+        uint8_t errorBoundMode = EB_ABS;
+        double absErrorBound = 1e-3;
+        double relErrorBound = 0;
+        double psnrErrorBound = 0;
+        double l2normErrorBound = 0;
+        bool lorenzo = true;
+        bool lorenzo2 = false;
+        bool regression = true;
+        bool regression2 = false;
+        bool openmp = false;
+        uint8_t dataType = SZ_FLOAT; // dataType is only used in HDF5 filter
+        uint8_t lossless = 1; // 0-> skip lossless(use lossless_bypass); 1-> zstd
+        uint8_t encoder = 1; // 0-> skip encoder; 1->HuffmanEncoder; 2->ArithmeticEncoder
+        uint8_t interpAlgo = INTERP_ALGO_CUBIC;
+        uint8_t interpDirection = 0;
+        int interpBlockSize = 32;
+        int quantbinCnt = 65536;
+        int blockSize;
+        int stride; //not used now
+        int pred_dim; // not used now
+    };
+}
+
+#endif //SZ_CONFIG_HPP

pysz-0.0.7/SZ3-bio/include/SZ3/utils/FileUtil.hpp

@@ -0,0 +1,74 @@
+//
+// Created by Kai Zhao on 12/9/19.
+//
+
+#ifndef _SZ_FILE_UTIL
+#define _SZ_FILE_UTIL
+
+#include <iostream>
+#include <fstream>
+#include <iomanip>
+#include <cassert>
+#include <random>
+#include <sstream>
+#include <memory>
+
+namespace SZ3 {
+
+    template<typename Type>
+    void readfile(const char *file, const size_t num, Type *data) {
+        std::ifstream fin(file, std::ios::binary);
+        if (!fin) {
+            std::cout << " Error, Couldn't find the file: " << file << "\n";
+            exit(0);
+        }
+        fin.seekg(0, std::ios::end);
+        const size_t num_elements = fin.tellg() / sizeof(Type);
+        assert(num_elements == num && "File size is not equals to the input setting");
+        fin.seekg(0, std::ios::beg);
+        fin.read(reinterpret_cast<char *>(data), num_elements * sizeof(Type));
+        fin.close();
+    }
+
+    template<typename Type>
+    std::unique_ptr<Type[]> readfile(const char *file, size_t &num) {
+        std::ifstream fin(file, std::ios::binary);
+        if (!fin) {
+            std::cout << " Error, Couldn't find the file: " << file << std::endl;
+            exit(0);
+        }
+        fin.seekg(0, std::ios::end);
+        const size_t num_elements = fin.tellg() / sizeof(Type);
+        fin.seekg(0, std::ios::beg);
+//        auto data = SZ3::compat::make_unique<Type[]>(num_elements);
+        auto data = std::make_unique<Type[]>(num_elements);
+        fin.read(reinterpret_cast<char *>(&data[0]), num_elements * sizeof(Type));
+        fin.close();
+        num = num_elements;
+        return data;
+    }
+
+    template<typename Type>
+    void writefile(const char *file, Type *data, size_t num_elements) {
+        std::ofstream fout(file, std::ios::binary);
+        fout.write(reinterpret_cast<const char *>(&data[0]), num_elements * sizeof(Type));
+        fout.close();
+    }
+
+    template<typename Type>
+    void writeTextFile(const char *file, Type *data, size_t num_elements) {
+        std::ofstream fout(file);
+        if (fout.is_open()) {
+            for (size_t i = 0; i < num_elements; i++) {
+                fout << data[i] << std::endl;
+            }
+            fout.close();
+        } else {
+            std::cout << "Error, unable to open file for output: " << file << std::endl;
+            exit(0);
+        }
+    }
+
+}
+
+#endif