foscat 3.0.14__tar.gz → 3.6.0__tar.gz

foscat-3.6.0/LICENCE

@@ -0,0 +1,13 @@
+BSD 3-Clause License
+
+Copyright (c) 2022, the Foscat developers All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

foscat-3.6.0/PKG-INFO

@@ -0,0 +1,184 @@
+Metadata-Version: 2.2
+Name: foscat
+Version: 3.6.0
+Summary: Generate synthetic Healpix or 2D data using Cross Scattering Transform
+Author-email: Jean-Marc DELOUIS <jean.marc.delouis@ifremer.fr>
+Maintainer-email: Theo Foulquier <theo.foulquier@ifremer.fr>
+License: BSD-3-Clause
+Project-URL: Repository, https://github.com/jmdelouis/FOSCAT.git
+Project-URL: Issues, https://github.com/jmdelouis/FOSCAT/issues
+Project-URL: Documentation, https://foscat-documentation.readthedocs.io/en/latest/index.html
+Keywords: scattering transform,component separation,denoising
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENCE
+Requires-Dist: imageio
+Requires-Dist: imagecodecs
+Requires-Dist: matplotlib
+Requires-Dist: numpy
+Requires-Dist: tensorflow
+Requires-Dist: healpy
+Requires-Dist: spherical
+
+# foscat
+
+[![Read the Docs](https://readthedocs.org/projects/foscat-documentation/badge/?version=latest)](https://foscat-documentation.readthedocs.io/en/latest)
+
+A python package dedicated to image component separation based on scattering transform analysis designed for high performance computing.
+
+## the concept
+
+The foscat genesis has been built to synthesise data (2D or Healpix) using Cross Scattering Transform. For a detailed method description please refer to https://arxiv.org/abs/2207.12527. This algorithm could be effectively usable for component separation (e.g. denoising).
+
+A demo package for this process can be found at https://github.com/jmdelouis/FOSCAT_DEMO.
+
+## usage
+
+# Short tutorial
+
+https://github.com/IAOCEA/demo-foscat-pangeo-eosc/blob/main/Demo_Synthesis.ipynb
+
+# FOSCAT_DEMO
+
+The python scripts _demo.py_ included in this package demonstrate how to use the foscat library to generate synthetic fields that have patterns with the same statistical properties as a specified image.
+
+# Install foscat library
+
+Before installing, make sure you have python installed in your enviroment.
+The last version of the foscat library can be installed using PyPi:
+
+```
+pip install foscat
+```
+
+Load the FOSCAT_DEMO package from github.
+
+```
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+```
+
+## Recommended installing procedures for mac users
+
+It is recomended to use python=3.9\*.
+
+```
+micromamba create -n FOSCAT
+micromamba install -n FOSCAT ‘python==3.9*’
+micromamba activate FOSCAT
+pip install foscat
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+
+```
+
+## Recommended installing procedures HPC users
+
+It is recomended to install tensorflow in advance. For [DATARMOR](https://pcdm.ifremer.fr/Equipement) for using GPU ;
+
+```
+micromamba create -n FOSCAT
+micromamba install -n FOSCAT ‘python==3.9*’
+micromamba install -n FOSCAT ‘tensorflow==2.11.0’
+micromamba activate FOSCAT
+pip install foscat
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+
+```
+
+# Spherical data example
+
+## compute a synthetic image
+
+```
+python demo.py -n=32 -k -c -s=100
+```
+
+The _demo.py_ script serves as a demonstration of the capabilities of the foscat library. It utilizes the Cross Wavelet Scattering Transform to generate a Healpix map that possesses the same characteristics as a specified input map.
+
+- `-n=32` computes map with nside=32.
+- `-k` uses 5x5 kernel.
+- `-c` uses Scattering Covariance.
+- `-l` uses LBFGS minimizer.
+- `-s=100` computes 100 steps.
+
+```
+python demo.py -n=8 [-c|--cov][-s|--steps=3000][-S=1234|--seed=1234][-k|--k5x5][-d|--data][-o|--out][-r|--orient] [-p|--path][-a|--adam]
+
+```
+
+- The "-n" option specifies the nside of the input map. The maximum nside value is 256 with the default map.
+- The "--cov" option (optional) uses scat_cov instead of scat.
+- The "--steps" option (optional) specifies the number of iterations. If not specified, the default value is 1000.
+- The "--seed" option (optional) specifies the seed of the random generator.
+- The "--path" option (optional) allows you to define the path where the output files will be written. The default path is "data".
+- The "--k5x5" option (optional) uses a 5x5 kernel instead of a 3x3.
+- The "--data" option (optional) specifies the input data file to be used. If not specified, the default file "LSS_map_nside128.npy" will be used.
+- The "--out" option (optional) specifies the output file name. If not specified, the output file will be saved in "demo".
+- The "--orient" option (optional) specifies the number of orientations. If not specified, the default value is 4.
+- The "--adam" option (optional) makes the synthesis using the ADAM optimizer instead of the L_BFGS.
+
+## plot the result
+
+The following script generates a series of plots that showcase different aspects of the synthesis process using the _demo.py_ script.
+
+> python test2D.py
+
+```
+python plotdemo.py -n=32 -c
+```
+
+# 2D field demo
+
+> python test2Dplot.py
+
+# compute a synthetic turbulent field
+
+The python scripts _demo2D.py_ included in this package demonstrate how to use the foscat library to generate a 2D synthetic fields that have patterns with the same statistical properties as a specified 2D image. In this particular case, the input field is a sea surface temperature extracted from a north atlantic ocean simulation.
+
+> python testHealpix.py
+
+```
+python demo2d.py -n=32 -k -c
+```
+
+> python testHplot.py
+
+The following script generates a series of plots that showcase different aspects of the synthesis process using the _demo2D.py_ script.
+
+```
+python plotdemo2d.py -n=32 -c
+```
+
+For more information, see the [documentation](https://foscat-documentation.readthedocs.io/en/latest/index.html).
+
+> mpirun -np 3 testHealpix_mpi.py
+
+## Authors and acknowledgment
+
+Authors: J.-M. Delouis, P. Campeti, T. Foulquier, J. Mangin, L. Mousset, T. Odaka, F. Paul, E. Allys
+
+This work is part of the R & T Deepsee project supported by CNES. The authors acknowledge the heritage of the Planck-HFI consortium regarding data, software, knowledge. This work has been supported by the Programme National de Télédétection Spatiale (PNTS, http://programmes.insu.cnrs.fr/pnts/), grant n◦ PNTS-2020-08
+
+## License
+
+BSD 3-Clause License
+
+Copyright (c) 2022, the Foscat developers All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+## Project status
+
+It is a scientific driven development. We are open to any contributing development.

foscat-3.6.0/README.md

@@ -0,0 +1,155 @@
+# foscat
+
+[![Read the Docs](https://readthedocs.org/projects/foscat-documentation/badge/?version=latest)](https://foscat-documentation.readthedocs.io/en/latest)
+
+A python package dedicated to image component separation based on scattering transform analysis designed for high performance computing.
+
+## the concept
+
+The foscat genesis has been built to synthesise data (2D or Healpix) using Cross Scattering Transform. For a detailed method description please refer to https://arxiv.org/abs/2207.12527. This algorithm could be effectively usable for component separation (e.g. denoising).
+
+A demo package for this process can be found at https://github.com/jmdelouis/FOSCAT_DEMO.
+
+## usage
+
+# Short tutorial
+
+https://github.com/IAOCEA/demo-foscat-pangeo-eosc/blob/main/Demo_Synthesis.ipynb
+
+# FOSCAT_DEMO
+
+The python scripts _demo.py_ included in this package demonstrate how to use the foscat library to generate synthetic fields that have patterns with the same statistical properties as a specified image.
+
+# Install foscat library
+
+Before installing, make sure you have python installed in your enviroment.
+The last version of the foscat library can be installed using PyPi:
+
+```
+pip install foscat
+```
+
+Load the FOSCAT_DEMO package from github.
+
+```
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+```
+
+## Recommended installing procedures for mac users
+
+It is recomended to use python=3.9\*.
+
+```
+micromamba create -n FOSCAT
+micromamba install -n FOSCAT ‘python==3.9*’
+micromamba activate FOSCAT
+pip install foscat
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+
+```
+
+## Recommended installing procedures HPC users
+
+It is recomended to install tensorflow in advance. For [DATARMOR](https://pcdm.ifremer.fr/Equipement) for using GPU ;
+
+```
+micromamba create -n FOSCAT
+micromamba install -n FOSCAT ‘python==3.9*’
+micromamba install -n FOSCAT ‘tensorflow==2.11.0’
+micromamba activate FOSCAT
+pip install foscat
+git clone https://github.com/jmdelouis/FOSCAT_DEMO.git
+
+```
+
+# Spherical data example
+
+## compute a synthetic image
+
+```
+python demo.py -n=32 -k -c -s=100
+```
+
+The _demo.py_ script serves as a demonstration of the capabilities of the foscat library. It utilizes the Cross Wavelet Scattering Transform to generate a Healpix map that possesses the same characteristics as a specified input map.
+
+- `-n=32` computes map with nside=32.
+- `-k` uses 5x5 kernel.
+- `-c` uses Scattering Covariance.
+- `-l` uses LBFGS minimizer.
+- `-s=100` computes 100 steps.
+
+```
+python demo.py -n=8 [-c|--cov][-s|--steps=3000][-S=1234|--seed=1234][-k|--k5x5][-d|--data][-o|--out][-r|--orient] [-p|--path][-a|--adam]
+
+```
+
+- The "-n" option specifies the nside of the input map. The maximum nside value is 256 with the default map.
+- The "--cov" option (optional) uses scat_cov instead of scat.
+- The "--steps" option (optional) specifies the number of iterations. If not specified, the default value is 1000.
+- The "--seed" option (optional) specifies the seed of the random generator.
+- The "--path" option (optional) allows you to define the path where the output files will be written. The default path is "data".
+- The "--k5x5" option (optional) uses a 5x5 kernel instead of a 3x3.
+- The "--data" option (optional) specifies the input data file to be used. If not specified, the default file "LSS_map_nside128.npy" will be used.
+- The "--out" option (optional) specifies the output file name. If not specified, the output file will be saved in "demo".
+- The "--orient" option (optional) specifies the number of orientations. If not specified, the default value is 4.
+- The "--adam" option (optional) makes the synthesis using the ADAM optimizer instead of the L_BFGS.
+
+## plot the result
+
+The following script generates a series of plots that showcase different aspects of the synthesis process using the _demo.py_ script.
+
+> python test2D.py
+
+```
+python plotdemo.py -n=32 -c
+```
+
+# 2D field demo
+
+> python test2Dplot.py
+
+# compute a synthetic turbulent field
+
+The python scripts _demo2D.py_ included in this package demonstrate how to use the foscat library to generate a 2D synthetic fields that have patterns with the same statistical properties as a specified 2D image. In this particular case, the input field is a sea surface temperature extracted from a north atlantic ocean simulation.
+
+> python testHealpix.py
+
+```
+python demo2d.py -n=32 -k -c
+```
+
+> python testHplot.py
+
+The following script generates a series of plots that showcase different aspects of the synthesis process using the _demo2D.py_ script.
+
+```
+python plotdemo2d.py -n=32 -c
+```
+
+For more information, see the [documentation](https://foscat-documentation.readthedocs.io/en/latest/index.html).
+
+> mpirun -np 3 testHealpix_mpi.py
+
+## Authors and acknowledgment
+
+Authors: J.-M. Delouis, P. Campeti, T. Foulquier, J. Mangin, L. Mousset, T. Odaka, F. Paul, E. Allys
+
+This work is part of the R & T Deepsee project supported by CNES. The authors acknowledge the heritage of the Planck-HFI consortium regarding data, software, knowledge. This work has been supported by the Programme National de Télédétection Spatiale (PNTS, http://programmes.insu.cnrs.fr/pnts/), grant n◦ PNTS-2020-08
+
+## License
+
+BSD 3-Clause License
+
+Copyright (c) 2022, the Foscat developers All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+## Project status
+
+It is a scientific driven development. We are open to any contributing development.

foscat-3.6.0/pyproject.toml

@@ -0,0 +1,89 @@
+[project]
+name = "foscat"
+version = "3.6.0"
+description = "Generate synthetic Healpix or 2D data using Cross Scattering Transform"
+readme = "README.md"
+license = { text = "BSD-3-Clause" }
+authors = [
+  { name = "Jean-Marc DELOUIS", email = "jean.marc.delouis@ifremer.fr" },
+]
+
+maintainers = [
+  { name = "Theo Foulquier", email = "theo.foulquier@ifremer.fr" },
+]
+
+dependencies = [
+  "imageio",
+  "imagecodecs",
+  "matplotlib",
+  "numpy",
+  "tensorflow",
+  "healpy",
+  "spherical",
+]
+
+requires-python = ">= 3.9"
+keywords = ["scattering transform", "component separation", "denoising"]
+classifiers = [
+  "Intended Audience :: Science/Research",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+]
+
+[project.urls]
+Repository = "https://github.com/jmdelouis/FOSCAT.git"
+Issues = "https://github.com/jmdelouis/FOSCAT/issues"
+Documentation = "https://foscat-documentation.readthedocs.io/en/latest/index.html"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+packages = ["foscat"]
+
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[tool.ruff]
+target-version = "py39"
+builtins = ["ellipsis"]
+exclude = [
+  ".git",
+  ".eggs",
+  "build",
+  "dist",
+  "__pycache__",
+]
+line-length = 100
+
+[tool.ruff.lint]
+ignore = [
+  "E402", # E402: module level import not at top of file
+  "E501", # E501: line too long - let black worry about that
+  "E731", # E731: do not assign a lambda expression, use a def
+  # ignore for now, fix in a later PR
+  "E722",  # bare except
+  "UP031", # percent-based string interpolation
+]
+select = [
+  "F",   # Pyflakes
+  "E",   # Pycodestyle
+  "I",   # isort
+  "UP",  # Pyupgrade
+  "TID", # flake8-tidy-imports
+  "W",
+]
+extend-safe-fixes = [
+  "TID252", # absolute imports
+]
+fixable = ["I", "TID252"]
+
+[tool.ruff.lint.isort]
+known-first-party = ["foscat"]
+
+[tool.ruff.lint.flake8-tidy-imports]
+# Disallow all relative imports.
+ban-relative-imports = "all"

foscat-3.6.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

foscat-3.6.0/src/foscat/CNN.py

@@ -0,0 +1,151 @@
+import pickle
+
+import numpy as np
+
+import foscat.scat_cov as sc
+
+
+class CNN:
+
+    def __init__(
+        self,
+        scat_operator=None,
+        nparam=1,
+        nscale=1,
+        chanlist=[],
+        in_nside=1,
+        n_chan_in=1,
+        nbatch=1,
+        SEED=1234,
+        filename=None,
+    ):
+
+        if filename is not None:
+            outlist = pickle.load(open("%s.pkl" % (filename), "rb"))
+            self.scat_operator = sc.funct(KERNELSZ=outlist[3], all_type=outlist[7])
+            self.KERNELSZ = self.scat_operator.KERNELSZ
+            self.all_type = self.scat_operator.all_type
+            self.npar = outlist[2]
+            self.nscale = outlist[5]
+            self.chanlist = outlist[0]
+            self.in_nside = outlist[4]
+            self.nbatch = outlist[1]
+            self.n_chan_in = outlist[8]
+            self.x = self.scat_operator.backend.bk_cast(outlist[6])
+            self.out_nside = self.in_nside // (2**self.nscale)
+        else:
+            self.nscale = nscale
+            self.nbatch = nbatch
+            self.npar = nparam
+            self.n_chan_in = n_chan_in
+            self.scat_operator = scat_operator
+            if len(chanlist) != nscale + 1:
+                print(
+                    "len of chanlist (here %d) should of nscale+1 (here %d)"
+                    % (len(chanlist), nscale + 1)
+                )
+                return None
+
+            self.chanlist = chanlist
+            self.KERNELSZ = scat_operator.KERNELSZ
+            self.all_type = scat_operator.all_type
+            self.in_nside = in_nside
+            self.out_nside = self.in_nside // (2**self.nscale)
+
+            np.random.seed(SEED)
+            self.x = scat_operator.backend.bk_cast(
+                np.random.randn(self.get_number_of_weights())
+                / (self.KERNELSZ * self.KERNELSZ)
+            )
+
+    def save(self, filename):
+
+        outlist = [
+            self.chanlist,
+            self.nbatch,
+            self.npar,
+            self.KERNELSZ,
+            self.in_nside,
+            self.nscale,
+            self.get_weights().numpy(),
+            self.all_type,
+            self.n_chan_in,
+        ]
+
+        myout = open("%s.pkl" % (filename), "wb")
+        pickle.dump(outlist, myout)
+        myout.close()
+
+    def get_number_of_weights(self):
+        totnchan = 0
+        for i in range(self.nscale):
+            totnchan = totnchan + self.chanlist[i] * self.chanlist[i + 1]
+        return (
+            self.npar * 12 * self.out_nside**2 * self.chanlist[self.nscale]
+            + totnchan * self.KERNELSZ * self.KERNELSZ
+            + self.KERNELSZ * self.KERNELSZ * self.n_chan_in * self.chanlist[0]
+        )
+
+    def set_weights(self, x):
+        self.x = x
+
+    def get_weights(self):
+        return self.x
+
+    def eval(self, im, indices=None, weights=None):
+
+        x = self.x
+        ww = self.scat_operator.backend.bk_reshape(
+            x[0 : self.KERNELSZ * self.KERNELSZ * self.n_chan_in * self.chanlist[0]],
+            [self.KERNELSZ * self.KERNELSZ, self.n_chan_in, self.chanlist[0]],
+        )
+        nn = self.KERNELSZ * self.KERNELSZ * self.n_chan_in * self.chanlist[0]
+
+        im = self.scat_operator.healpix_layer(im, ww)
+        im = self.scat_operator.backend.bk_relu(im)
+
+        for k in range(self.nscale):
+            ww = self.scat_operator.backend.bk_reshape(
+                x[
+                    nn : nn
+                    + self.KERNELSZ
+                    * self.KERNELSZ
+                    * self.chanlist[k]
+                    * self.chanlist[k + 1]
+                ],
+                [self.KERNELSZ * self.KERNELSZ, self.chanlist[k], self.chanlist[k + 1]],
+            )
+            nn = (
+                nn
+                + self.KERNELSZ
+                * self.KERNELSZ
+                * self.chanlist[k]
+                * self.chanlist[k + 1]
+            )
+            if indices is None:
+                im = self.scat_operator.healpix_layer(im, ww)
+            else:
+                im = self.scat_operator.healpix_layer(
+                    im, ww, indices=indices[k], weights=weights[k]
+                )
+            im = self.scat_operator.backend.bk_relu(im)
+            im = self.scat_operator.ud_grade_2(im, axis=0)
+
+        ww = self.scat_operator.backend.bk_reshape(
+            x[
+                nn : nn
+                + self.npar * 12 * self.out_nside**2 * self.chanlist[self.nscale]
+            ],
+            [12 * self.out_nside**2 * self.chanlist[self.nscale], self.npar],
+        )
+
+        im = self.scat_operator.backend.bk_matmul(
+            self.scat_operator.backend.bk_reshape(
+                im, [1, 12 * self.out_nside**2 * self.chanlist[self.nscale]]
+            ),
+            ww,
+        )
+        im = self.scat_operator.backend.bk_reshape(im, [self.npar])
+        im = self.scat_operator.backend.bk_relu(im)
+
+        return im