spikezoo 0.1.2__tar.gz → 0.2__tar.gz

spikezoo-0.2/MANIFEST.in

@@ -0,0 +1,5 @@
+include requirements.txt
+recursive-include spikezoo/data/base *
+include spikezoo/utils/vidar_loader.cpython-39-x86_64-linux-gnu.so
+recursive-include spikezoo/archs *
+recursive-exclude spikezoo/archs/spcsnet * 

spikezoo-0.2/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.2
+Name: spikezoo
+Version: 0.2
+Summary: A deep learning toolbox for spike-to-image models.
+Home-page: https://github.com/chenkang455/Spike-Zoo
+Author: Kang Chen
+Author-email: mrchenkang@stu.pku.edu.cn
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Requires-Dist: torch
+Requires-Dist: requests
+Requires-Dist: numpy
+Requires-Dist: tqdm
+Requires-Dist: scikit-image
+Requires-Dist: lpips
+Requires-Dist: pyiqa
+Requires-Dist: opencv-python
+Requires-Dist: thop
+Requires-Dist: pytorch-wavelets
+Requires-Dist: pytz
+Requires-Dist: PyWavelets
+Requires-Dist: pandas
+Requires-Dist: pillow
+Requires-Dist: scikit-learn
+Requires-Dist: scipy
+Requires-Dist: spikingjelly
+Requires-Dist: setuptools
+Dynamic: author
+Dynamic: author-email
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+<h2 align="center"> 
+  <a href="">Spike-Zoo: A Toolbox for Spike-to-Image Reconstruction
+  </a>
+</h2>
+
+## 📖 About
+⚡Spike-Zoo is the go-to library for state-of-the-art pretrained **spike-to-image** models for reconstructing the image from the given spike stream. Whether you're looking for a **simple inference** solution or **training** your own spike-to-image models, ⚡Spike-Zoo is a modular toolbox that supports both. 
+
+If Spike-Zoo helps your research or work, please help to ⭐ this repo or recommend it to your friends. Thanks😊
+
+## 🚩 Updates/Changelog
+* **25-02-02:** Release the `Spike-Zoo v0.2` code, which supports more methods, provide more usages.
+* **24-08-26:** Update the `SpikeFormer` and `RSIR` methods, the `UHSR` dataset and the `piqe` non-reference metric.
+
+* **24-07-19:** Release the `Spike-Zoo v0.1` base code.
+
+## 🍾 Quick Start
+### 1. Installation
+For users focused on **utilizing pretrained models for spike-to-image conversion**, we recommend installing SpikeZoo using one of the following methods:
+
+* Install the last stable version from PyPI:
+```
+pip install spikezoo
+```
+*  Install the latest developing version from the source code:
+```
+git clone https://github.com/chenkang455/Spike-Zoo
+cd Spike-Zoo
+python setup.py install
+```
+
+For users interested in **training their own spike-to-image model based on our framework**, we recommend cloning the repository and modifying the related code directly.
+
+### 2. Inference 
+Reconstructing images from the spike input is super easy with Spike-Zoo. Try the following code of the single model:
+``` python
+from spikezoo.pipeline import Pipeline, PipelineConfig
+pipeline = Pipeline(
+    cfg = PipelineConfig(save_folder="results"),
+    model_cfg="spk2imgnet",
+    dataset_cfg="base"
+)
+```
+You can also run multiple models at once by changing the pipeline:
+``` python
+from spikezoo.pipeline import EnsemblePipeline, EnsemblePipelineConfig
+pipeline = EnsemblePipeline(
+    cfg = EnsemblePipelineConfig(save_folder="results"),
+    model_cfg_list=['tfp','tfi', 'spk2imgnet', 'wgse', 'ssml', 'bsf', 'stir',  'spikeclip','spikeformer'],
+    dataset_cfg="base"
+)
+```
+* Having established the pipeline, run the following code to obtain the metric and save the reconstructed image from the given spike:
+``` python
+# 1. spike-to-image from the given dataset
+pipeline.spk2img_from_dataset(idx = 0)
+
+# 2. spike-to-image from the given .dat file
+pipeline.spk2img_from_file(file_path = 'data/scissor.dat',width = 400,height=250)
+
+# 3. spike-to-image from the given spike
+import spikezoo as sz
+spike = sz.load_vidar_dat("data/scissor.dat",width = 400,height = 250,version='cpp')
+pipeline.spk2img_from_spk(spike)
+```
+For detailed usage, welcome check [test_single.ipynb](examples/test_single.ipynb) and [test_multi.ipynb](examples/test_multi.ipynb) 😊😊😊.
+
+* Save all images of the given dataset.
+``` python
+pipeline.save_imgs_from_dataset()
+```
+
+* Calculate the metrics for the specified dataset.
+``` python
+pipeline.cal_metrics()
+```
+
+* Calculate the parameters (params,flops,latency) based on the established pipeline.
+``` python
+pipeline.cal_params()
+```
+
+### 3. Training
+We provide a user-friendly code for training our provided base model (modified from the `SpikeCLIP`) for the classic `REDS` dataset introduced in `Spk2ImgNet`:
+``` python
+from spikezoo.pipeline import TrainPipelineConfig, TrainPipeline
+from spikezoo.datasets.reds_small_dataset import REDS_Small_Config
+pipeline = TrainPipeline(
+    cfg=TrainPipelineConfig(save_folder="results", epochs = 10),
+    dataset_cfg=REDS_Small_Config(root_dir = "path/REDS_Small"),
+    model_cfg="base",
+)
+pipeline.train()
+``` 
+We finish the training with one 4090 GPU in `2 minutes`, achieving `34.7dB` in PSNR and `0.94` in SSIM.
+
+> 🌟 We encourage users to develop their models using our framework, with the tutorial being released soon.
+
+### 4. Others
+We provide a faster `load_vidar_dat` function implemented with `cpp` (by @zeal-ye):
+``` bash
+import spikezoo as sz
+spike = sz.load_vidar_dat("data/scissor.dat",width = 400,height = 250,version='cpp')
+```
+🚀 Results on [examples/test_load_dat.py](examples/test_load_dat.py) show that the `cpp` version is more than 10 times faster than the `python` version.
+
+
+## 📅 TODO
+- [ ] Provide the tutorials.
+- [ ] Support more training settings.
+- [ ] Support more spike-based image reconstruction methods and datasets. 
+- [ ] Support the overall pipeline for spike simulation. 
+
+## ✨‍ Acknowledgment
+Our code is built on the open-source projects of [SpikeCV](https://spikecv.github.io/), [IQA-Pytorch](https://github.com/chaofengc/IQA-PyTorch), [BasicSR](https://github.com/XPixelGroup/BasicSR) and [NeRFStudio](https://github.com/nerfstudio-project/nerfstudio).We appreciate the effort of the contributors to these repositories. Thanks for @ruizhao26 and @Leozhangjiyuan for their help in building this project.
+
+## 📑 Citation
+If you find our codes helpful to your research, please consider to use the following citation:
+```
+@misc{spikezoo,
+  title={{Spike-Zoo}: Spike-Zoo: A Toolbox for Spike-to-Image Reconstruction},
+  author={Kang Chen and Zhiyuan Ye},
+  year={2025},
+  howpublished = "[Online]. Available: \url{https://github.com/chenkang455/Spike-Zoo}"
+}
+```

spikezoo-0.2/README.md

@@ -0,0 +1,126 @@
+<h2 align="center"> 
+  <a href="">Spike-Zoo: A Toolbox for Spike-to-Image Reconstruction
+  </a>
+</h2>
+
+## 📖 About
+⚡Spike-Zoo is the go-to library for state-of-the-art pretrained **spike-to-image** models for reconstructing the image from the given spike stream. Whether you're looking for a **simple inference** solution or **training** your own spike-to-image models, ⚡Spike-Zoo is a modular toolbox that supports both. 
+
+If Spike-Zoo helps your research or work, please help to ⭐ this repo or recommend it to your friends. Thanks😊
+
+## 🚩 Updates/Changelog
+* **25-02-02:** Release the `Spike-Zoo v0.2` code, which supports more methods, provide more usages.
+* **24-08-26:** Update the `SpikeFormer` and `RSIR` methods, the `UHSR` dataset and the `piqe` non-reference metric.
+
+* **24-07-19:** Release the `Spike-Zoo v0.1` base code.
+
+## 🍾 Quick Start
+### 1. Installation
+For users focused on **utilizing pretrained models for spike-to-image conversion**, we recommend installing SpikeZoo using one of the following methods:
+
+* Install the last stable version from PyPI:
+```
+pip install spikezoo
+```
+*  Install the latest developing version from the source code:
+```
+git clone https://github.com/chenkang455/Spike-Zoo
+cd Spike-Zoo
+python setup.py install
+```
+
+For users interested in **training their own spike-to-image model based on our framework**, we recommend cloning the repository and modifying the related code directly.
+
+### 2. Inference 
+Reconstructing images from the spike input is super easy with Spike-Zoo. Try the following code of the single model:
+``` python
+from spikezoo.pipeline import Pipeline, PipelineConfig
+pipeline = Pipeline(
+    cfg = PipelineConfig(save_folder="results"),
+    model_cfg="spk2imgnet",
+    dataset_cfg="base"
+)
+```
+You can also run multiple models at once by changing the pipeline:
+``` python
+from spikezoo.pipeline import EnsemblePipeline, EnsemblePipelineConfig
+pipeline = EnsemblePipeline(
+    cfg = EnsemblePipelineConfig(save_folder="results"),
+    model_cfg_list=['tfp','tfi', 'spk2imgnet', 'wgse', 'ssml', 'bsf', 'stir',  'spikeclip','spikeformer'],
+    dataset_cfg="base"
+)
+```
+* Having established the pipeline, run the following code to obtain the metric and save the reconstructed image from the given spike:
+``` python
+# 1. spike-to-image from the given dataset
+pipeline.spk2img_from_dataset(idx = 0)
+
+# 2. spike-to-image from the given .dat file
+pipeline.spk2img_from_file(file_path = 'data/scissor.dat',width = 400,height=250)
+
+# 3. spike-to-image from the given spike
+import spikezoo as sz
+spike = sz.load_vidar_dat("data/scissor.dat",width = 400,height = 250,version='cpp')
+pipeline.spk2img_from_spk(spike)
+```
+For detailed usage, welcome check [test_single.ipynb](examples/test_single.ipynb) and [test_multi.ipynb](examples/test_multi.ipynb) 😊😊😊.
+
+* Save all images of the given dataset.
+``` python
+pipeline.save_imgs_from_dataset()
+```
+
+* Calculate the metrics for the specified dataset.
+``` python
+pipeline.cal_metrics()
+```
+
+* Calculate the parameters (params,flops,latency) based on the established pipeline.
+``` python
+pipeline.cal_params()
+```
+
+### 3. Training
+We provide a user-friendly code for training our provided base model (modified from the `SpikeCLIP`) for the classic `REDS` dataset introduced in `Spk2ImgNet`:
+``` python
+from spikezoo.pipeline import TrainPipelineConfig, TrainPipeline
+from spikezoo.datasets.reds_small_dataset import REDS_Small_Config
+pipeline = TrainPipeline(
+    cfg=TrainPipelineConfig(save_folder="results", epochs = 10),
+    dataset_cfg=REDS_Small_Config(root_dir = "path/REDS_Small"),
+    model_cfg="base",
+)
+pipeline.train()
+``` 
+We finish the training with one 4090 GPU in `2 minutes`, achieving `34.7dB` in PSNR and `0.94` in SSIM.
+
+> 🌟 We encourage users to develop their models using our framework, with the tutorial being released soon.
+
+### 4. Others
+We provide a faster `load_vidar_dat` function implemented with `cpp` (by @zeal-ye):
+``` bash
+import spikezoo as sz
+spike = sz.load_vidar_dat("data/scissor.dat",width = 400,height = 250,version='cpp')
+```
+🚀 Results on [examples/test_load_dat.py](examples/test_load_dat.py) show that the `cpp` version is more than 10 times faster than the `python` version.
+
+
+## 📅 TODO
+- [ ] Provide the tutorials.
+- [ ] Support more training settings.
+- [ ] Support more spike-based image reconstruction methods and datasets. 
+- [ ] Support the overall pipeline for spike simulation. 
+
+## ✨‍ Acknowledgment
+Our code is built on the open-source projects of [SpikeCV](https://spikecv.github.io/), [IQA-Pytorch](https://github.com/chaofengc/IQA-PyTorch), [BasicSR](https://github.com/XPixelGroup/BasicSR) and [NeRFStudio](https://github.com/nerfstudio-project/nerfstudio).We appreciate the effort of the contributors to these repositories. Thanks for @ruizhao26 and @Leozhangjiyuan for their help in building this project.
+
+## 📑 Citation
+If you find our codes helpful to your research, please consider to use the following citation:
+```
+@misc{spikezoo,
+  title={{Spike-Zoo}: Spike-Zoo: A Toolbox for Spike-to-Image Reconstruction},
+  author={Kang Chen and Zhiyuan Ye},
+  year={2025},
+  howpublished = "[Online]. Available: \url{https://github.com/chenkang455/Spike-Zoo}"
+}
+```

{spikezoo-0.1.2 → spikezoo-0.2}/setup.py

@@ -1,16 +1,16 @@
 from setuptools import find_packages
 from setuptools import setup
 
-with open("requirements.txt", "r", encoding="utf-8") as fh:
+with open("./requirements.txt", "r", encoding="utf-8") as fh:
     install_requires = fh.read()
 
-with open("README.md", "r", encoding="utf-8") as fh:
+with open("./README.md", "r", encoding="utf-8") as fh:
     long_description = fh.read()
 
 setup(
     install_requires=install_requires,
     name="spikezoo",
-    version="0.1.2",
+    version="0.2",
     author="Kang Chen",
     author_email="mrchenkang@stu.pku.edu.cn",
     description="A deep learning toolbox for spike-to-image models.",
@@ -19,5 +19,5 @@ setup(
     url="https://github.com/chenkang455/Spike-Zoo",
     packages=find_packages(),
     python_requires='>=3.7',
-    include_package_data=False,
+    include_package_data=True
 )

spikezoo-0.2/spikezoo/__init__.py

@@ -0,0 +1,13 @@
+from .utils.spike_utils import load_vidar_dat
+from .models import model_list
+from .datasets import dataset_list
+from .metrics import metric_all_names
+
+def get_datasets():
+    return dataset_list
+
+def get_models():
+    return model_list
+
+def get_metrics():
+    return metric_all_names

spikezoo-0.2/spikezoo/archs/base/nets.py

@@ -0,0 +1,34 @@
+import torch.nn as nn
+
+def conv_layer(inDim, outDim, ks, s, p, norm_layer='none'):
+    ## convolutional layer
+    conv = nn.Conv2d(inDim, outDim, kernel_size=ks, stride=s, padding=p)
+    relu = nn.ReLU(True)
+    assert norm_layer in ('batch', 'instance', 'none')
+    if norm_layer == 'none':
+        seq = nn.Sequential(*[conv, relu])
+    else:
+        if (norm_layer == 'instance'):
+            norm = nn.InstanceNorm2d(outDim, affine=False, track_running_stats=False) # instance norm
+        else:
+            momentum = 0.1
+            norm = nn.BatchNorm2d(outDim, momentum = momentum, affine=True, track_running_stats=True)
+        seq = nn.Sequential(*[conv, norm, relu])
+    return seq
+
+class BaseNet(nn.Module):
+    """Borrow the structure from the SpikeCLIP. (https://arxiv.org/abs/2501.04477)"""
+    def __init__(self, inDim=41):
+        super(BaseNet, self).__init__()
+        norm='none'
+        outDim=1
+        convBlock1 = conv_layer(inDim,64,3,1,1)
+        convBlock2 = conv_layer(64,128,3,1,1,norm)
+        convBlock3 = conv_layer(128,64,3,1,1,norm)
+        convBlock4 = conv_layer(64,16,3,1,1,norm)
+        conv = nn.Conv2d(16, outDim, 3, 1, 1) 
+        self.seq = nn.Sequential(*[convBlock1, convBlock2, convBlock3, convBlock4, conv])
+
+    def forward(self,x):
+        return self.seq(x)
+    

spikezoo-0.2/spikezoo/archs/bsf/README.md

@@ -0,0 +1,92 @@
+## [CVPR 2024] Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations
+
+<h4 align="center"> Rui Zhao<sup>1,2</sup>, Ruiqin Xiong<sup>1,2</sup>, Jing Zhao<sup>1,2</sup>, Jian Zhang<sup>3</sup>, Xiaopeng Fan<sup>4</sup>, Zhaofei Yu<sup>1,2</sup>, Tiejun Huang<sup>1,2</sup> </h4>
+<h4 align="center">1. School of Computer Science, Peking University<br>
+2. National Key Laboratory for Multimedia Information Processing, Peking University<br>
+3. School of Electronic and Computer Engineering,  Peking University<br>
+4. School of Computer Science and Technology, Harbin Institute of Technology
+</h4><br>
+
+This repository contains the official source code for our paper:
+
+Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations
+
+CVPR 2024
+
+## Environment
+
+You can choose cudatoolkit version to match your server. The code is tested on PyTorch 2.0.1+cu120.
+
+```bash
+conda create -n bsf python==3.10.9
+conda activate bsf
+# You can choose the PyTorch version you like, we recommand version >= 1.10.1
+# For example
+pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
+pip3 install -r requirements.txt
+```
+
+## Prepare the Data
+
+##### 1. Download the dataset (Approximate 50GB)
+
+[Link of the dataset (BaiduNetDisk)](https://pan.baidu.com/s/1zBp-ed1KtmhAab5Z_62ttw)  (Password: 2728) 
+
+##### 2. Deploy the dataset for training faster (Approximate <u>another</u> 125GB)
+
+firstly modify the data root and output root in `./prepare_data/crop_dataset_train.py` and `./prepare_data/crop_dataset_val.py`
+
+```shell
+cd prepare_data &&
+bash crop_train.sh $your_gpu_id &&
+bash crop_val.sh $your_gpu_id
+```
+
+## Evaluate
+
+```shell
+CUDA_VISIBLE_DEVICES=$1 python3 -W ignore main.py \
+--alpha 0.7 \
+--vis-path vis/bsf \
+-evp eval_vis/bsf \
+--logs_file_name bsf \
+--compile_model \
+--test_eval \
+--arch bsf \
+--pretrained ckpt/bsf.pth
+```
+
+## Train
+
+```shell
+CUDA_VISIBLE_DEVICES=$1 python3 -W ignore main.py \
+-bs 8 \
+-j 8 \
+-lr 1e-4 \
+--epochs 61 \
+--train-res 96 96 \
+--lr-scale-factor 0.5 \
+--milestones 10 20 30 40 50 60 70 80 90 100 \
+--alpha 0.7 \
+--vis-path vis/bsf \
+-evp eval_vis/bsf \
+--logs_file_name bsf \
+--compile_model \
+--weight_decay 0.0 \
+--eval-interval 10 \
+--half_reserve 0 \
+--arch bsf
+```
+
+## Citations
+
+If you find this code useful in your research, please consider citing our paper:
+
+```
+@inproceedings{zhao2024boosting,
+  title={Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations},
+  author={Zhao, Rui and Xiong, Ruiqin and Zhao, Jing and Zhang, Jian and Fan, Xiaopeng and Yu, Zhaofei, and Huang, Tiejun},
+  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+  year={2024}
+}
+```