mmgp 1.0.5__tar.gz → 1.1.0__tar.gz

{mmgp-1.0.5 → mmgp-1.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mmgp
-Version: 1.0.5
+Version: 1.1.0
 Summary: Memory Management for the GPU Poor
 Author-email: deepbeepmeep <deepbeepmeep@yahoo.com>
 License:                     GNU GENERAL PUBLIC LICENSE
@@ -683,7 +683,11 @@ License-File: LICENSE.md
 Requires-Dist: torch>=2.1.0
 Requires-Dist: optimum-quanto
 
-**------------------ Memory Management for the GPU Poor by DeepBeepMeep ------------------**
+
+<p align="center">
+  <H2>Memory Management for the GPU Poor by DeepBeepMeep</H2>	
+</p>
+
 
 This module contains multiples optimisations so that models such as Flux (and derived), Mochi, CogView, HunyuanVideo, ...  can run smoothly on a 24 GB GPU limited card. 
 This a replacement for the accelerate library that should in theory manage offloading, but doesn't work properly with models that are loaded / unloaded several
@@ -693,31 +697,50 @@ Requirements:
 - GPU: RTX 3090/ RTX 4090 (24 GB of VRAM)
 - RAM: minimum 48 GB, recommended 64 GB 
 
+First you need to install the module in your current project with:
+```shell
+pip install mmgp
+```
+
 It is almost plug and play and just needs to be invoked from the main app just after the model pipeline has been created.
-1) First make sure that the pipeline explictly loads the models in the CPU device 
-  for instance: pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+1) First make sure that the pipeline explictly loads the models in the CPU device, for instance:
+```
+  pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+```
+
 2) Once every potential Lora has been loaded and merged, add the following lines:
 
-  *from mmgp import offload*
-  *offload.me(pipe)*
-  
-The 'transformer' model that contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
+```
+  from mmgp import offload
+  offload.me(pipe)
+```  
+The 'transformer' model in the pipe contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
 
 If you have more than 64GB RAM you may want to enable RAM pinning with the option *pinInRAM = True*. You will get in return super fast loading / unloading of models
 (this can save significant time if the same pipeline is run multiple times in a row)
 
-Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.
-
+Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.\
 For instance :
-for flux derived models: *pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }*
-for mochi: *pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }*
+
+
+- for flux derived models: 
+```
+pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }
+```
+- for mochi: 
+```
+pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }
+```
+
 
 Please note that there should be always one model whose Id is 'transformer'. It corresponds to the main image / video model which usually needs to be quantized (this is done on the fly by default when loading the model).
 
 Becareful, lots of models use the T5 XXL as a text encoder. However, quite often their corresponding pipeline configurations point at the official Google T5 XXL repository 
 where there is a huge 40GB model to download and load. It is cumbersorme as it is a 32 bits model and contains the decoder part of T5 that is not used. 
 I suggest you use instead one of the 16 bits encoder only version available around, for instance:
-*text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)*
+```
+text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)
+```
 
 Sometime just providing the pipe won't be sufficient as you will need to change the content of the core model: 
 - For instance you may need to disable an existing CPU offload logic that already exists (such as manual calls to move tensors between cuda and the cpu)
@@ -727,6 +750,6 @@ You are free to use my module for non commercial use as long you give me proper
 
 Thanks to
 ---------
-Huggingface / accelerate for the hooking examples
-Huggingface / quanto for their very useful quantizer
-gau-nernst for his Pinnig RAM samples
+- Huggingface / accelerate for the hooking examples
+- Huggingface / quanto for their very useful quantizer
+- gau-nernst for his Pinnig RAM samples

{mmgp-1.0.5 → mmgp-1.1.0}/README.md

@@ -1,4 +1,8 @@
-**------------------ Memory Management for the GPU Poor by DeepBeepMeep ------------------**
+
+<p align="center">
+  <H2>Memory Management for the GPU Poor by DeepBeepMeep</H2>	
+</p>
+
 
 This module contains multiples optimisations so that models such as Flux (and derived), Mochi, CogView, HunyuanVideo, ...  can run smoothly on a 24 GB GPU limited card. 
 This a replacement for the accelerate library that should in theory manage offloading, but doesn't work properly with models that are loaded / unloaded several
@@ -8,31 +12,50 @@ Requirements:
 - GPU: RTX 3090/ RTX 4090 (24 GB of VRAM)
 - RAM: minimum 48 GB, recommended 64 GB 
 
+First you need to install the module in your current project with:
+```shell
+pip install mmgp
+```
+
 It is almost plug and play and just needs to be invoked from the main app just after the model pipeline has been created.
-1) First make sure that the pipeline explictly loads the models in the CPU device 
-  for instance: pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+1) First make sure that the pipeline explictly loads the models in the CPU device, for instance:
+```
+  pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+```
+
 2) Once every potential Lora has been loaded and merged, add the following lines:
 
-  *from mmgp import offload*
-  *offload.me(pipe)*
-  
-The 'transformer' model that contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
+```
+  from mmgp import offload
+  offload.me(pipe)
+```  
+The 'transformer' model in the pipe contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
 
 If you have more than 64GB RAM you may want to enable RAM pinning with the option *pinInRAM = True*. You will get in return super fast loading / unloading of models
 (this can save significant time if the same pipeline is run multiple times in a row)
 
-Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.
-
+Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.\
 For instance :
-for flux derived models: *pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }*
-for mochi: *pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }*
+
+
+- for flux derived models: 
+```
+pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }
+```
+- for mochi: 
+```
+pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }
+```
+
 
 Please note that there should be always one model whose Id is 'transformer'. It corresponds to the main image / video model which usually needs to be quantized (this is done on the fly by default when loading the model).
 
 Becareful, lots of models use the T5 XXL as a text encoder. However, quite often their corresponding pipeline configurations point at the official Google T5 XXL repository 
 where there is a huge 40GB model to download and load. It is cumbersorme as it is a 32 bits model and contains the decoder part of T5 that is not used. 
 I suggest you use instead one of the 16 bits encoder only version available around, for instance:
-*text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)*
+```
+text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)
+```
 
 Sometime just providing the pipe won't be sufficient as you will need to change the content of the core model: 
 - For instance you may need to disable an existing CPU offload logic that already exists (such as manual calls to move tensors between cuda and the cpu)
@@ -42,6 +65,6 @@ You are free to use my module for non commercial use as long you give me proper
 
 Thanks to
 ---------
-Huggingface / accelerate for the hooking examples
-Huggingface / quanto for their very useful quantizer
-gau-nernst for his Pinnig RAM samples
+- Huggingface / accelerate for the hooking examples
+- Huggingface / quanto for their very useful quantizer
+- gau-nernst for his Pinnig RAM samples

mmgp-1.1.0/pyproject.toml

@@ -0,0 +1,15 @@
+[project]
+name = "mmgp"
+version = "1.1.0"
+authors = [
+  { name = "deepbeepmeep", email = "deepbeepmeep@yahoo.com" },
+]
+description = "Memory Management for the GPU Poor"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { file = "LICENSE.md" }
+dependencies = [
+  "torch >= 2.1.0",
+  "optimum-quanto",
+]
+

{mmgp-1.0.5 → mmgp-1.1.0}/src/mmgp.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mmgp
-Version: 1.0.5
+Version: 1.1.0
 Summary: Memory Management for the GPU Poor
 Author-email: deepbeepmeep <deepbeepmeep@yahoo.com>
 License:                     GNU GENERAL PUBLIC LICENSE
@@ -683,7 +683,11 @@ License-File: LICENSE.md
 Requires-Dist: torch>=2.1.0
 Requires-Dist: optimum-quanto
 
-**------------------ Memory Management for the GPU Poor by DeepBeepMeep ------------------**
+
+<p align="center">
+  <H2>Memory Management for the GPU Poor by DeepBeepMeep</H2>	
+</p>
+
 
 This module contains multiples optimisations so that models such as Flux (and derived), Mochi, CogView, HunyuanVideo, ...  can run smoothly on a 24 GB GPU limited card. 
 This a replacement for the accelerate library that should in theory manage offloading, but doesn't work properly with models that are loaded / unloaded several
@@ -693,31 +697,50 @@ Requirements:
 - GPU: RTX 3090/ RTX 4090 (24 GB of VRAM)
 - RAM: minimum 48 GB, recommended 64 GB 
 
+First you need to install the module in your current project with:
+```shell
+pip install mmgp
+```
+
 It is almost plug and play and just needs to be invoked from the main app just after the model pipeline has been created.
-1) First make sure that the pipeline explictly loads the models in the CPU device 
-  for instance: pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+1) First make sure that the pipeline explictly loads the models in the CPU device, for instance:
+```
+  pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+```
+
 2) Once every potential Lora has been loaded and merged, add the following lines:
 
-  *from mmgp import offload*
-  *offload.me(pipe)*
-  
-The 'transformer' model that contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
+```
+  from mmgp import offload
+  offload.me(pipe)
+```  
+The 'transformer' model in the pipe contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
 
 If you have more than 64GB RAM you may want to enable RAM pinning with the option *pinInRAM = True*. You will get in return super fast loading / unloading of models
 (this can save significant time if the same pipeline is run multiple times in a row)
 
-Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.
-
+Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.\
 For instance :
-for flux derived models: *pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }*
-for mochi: *pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }*
+
+
+- for flux derived models: 
+```
+pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }
+```
+- for mochi: 
+```
+pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }
+```
+
 
 Please note that there should be always one model whose Id is 'transformer'. It corresponds to the main image / video model which usually needs to be quantized (this is done on the fly by default when loading the model).
 
 Becareful, lots of models use the T5 XXL as a text encoder. However, quite often their corresponding pipeline configurations point at the official Google T5 XXL repository 
 where there is a huge 40GB model to download and load. It is cumbersorme as it is a 32 bits model and contains the decoder part of T5 that is not used. 
 I suggest you use instead one of the 16 bits encoder only version available around, for instance:
-*text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)*
+```
+text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)
+```
 
 Sometime just providing the pipe won't be sufficient as you will need to change the content of the core model: 
 - For instance you may need to disable an existing CPU offload logic that already exists (such as manual calls to move tensors between cuda and the cpu)
@@ -727,6 +750,6 @@ You are free to use my module for non commercial use as long you give me proper
 
 Thanks to
 ---------
-Huggingface / accelerate for the hooking examples
-Huggingface / quanto for their very useful quantizer
-gau-nernst for his Pinnig RAM samples
+- Huggingface / accelerate for the hooking examples
+- Huggingface / quanto for their very useful quantizer
+- gau-nernst for his Pinnig RAM samples

{mmgp-1.0.5 → mmgp-1.1.0}/src/mmgp.egg-info/SOURCES.txt

@@ -2,7 +2,6 @@ LICENSE.md
 README.md
 pyproject.toml
 src/__init__.py
-src/_version.py
 src/mmgp.py
 src/mmgp.egg-info/PKG-INFO
 src/mmgp.egg-info/SOURCES.txt

{mmgp-1.0.5 → mmgp-1.1.0}/src/mmgp.egg-info/top_level.txt

@@ -1,3 +1,2 @@
 __init__
-_version
 mmgp

mmgp-1.0.5/pyproject.toml

@@ -1,74 +0,0 @@
-[project]
-name = "mmgp"
-authors = [
-  { name = "deepbeepmeep", email = "deepbeepmeep@yahoo.com" },
-]
-description = "Memory Management for the GPU Poor"
-readme = "README.md"
-requires-python = ">=3.10"
-license = { file = "LICENSE.md" }
-dynamic = ["version"]
-dependencies = [
-  "torch >= 2.1.0",
-  "optimum-quanto",
-]
-
-[project.optional-dependencies]
-
-
-[build-system]
-build-backend = "setuptools.build_meta"
-requires = ["setuptools>=64", "wheel", "setuptools_scm>=8"]
-
-[tool.ruff]
-line-length = 110
-target-version = "py310"
-extend-exclude = ["/usr/lib/*"]
-
-[tool.ruff.lint]
-ignore = [
-  "E501", # line too long - will be fixed in format
-]
-
-[tool.ruff.format]
-quote-style = "double"
-indent-style = "space"
-line-ending = "auto"
-skip-magic-trailing-comma = false
-docstring-code-format = true
-exclude = [
-  "src/_version.py", # generated by setuptools_scm
-]
-
-[tool.ruff.lint.isort]
-combine-as-imports = true
-force-wrap-aliases = true
-known-local-folder = ["src"]
-known-first-party = ["mmgp"]
-
-[tool.pyright]
-include = ["src"]
-exclude = [
-  "**/__pycache__", # cache directories
-  "./typings",      # generated type stubs
-]
-stubPath = "./typings"
-
-[tool.tomlsort]
-in_place = true
-no_sort_tables = true
-spaces_before_inline_comment = 1
-spaces_indent_inline_array = 2
-trailing_comma_inline_array = true
-sort_first = [
-  "project",
-  "build-system",
-  "tool.setuptools",
-]
-
-# needs to be last for CI reasons
-[tool.setuptools_scm]
-write_to = "src/_version.py"
-parentdir_prefix_version = "mmgp-"
-fallback_version = "1.0.5"
-version_scheme = "post-release"

mmgp-1.0.5/src/_version.py

@@ -1,16 +0,0 @@
-# file generated by setuptools_scm
-# don't change, don't track in version control
-TYPE_CHECKING = False
-if TYPE_CHECKING:
-    from typing import Tuple, Union
-    VERSION_TUPLE = Tuple[Union[int, str], ...]
-else:
-    VERSION_TUPLE = object
-
-version: str
-__version__: str
-__version_tuple__: VERSION_TUPLE
-version_tuple: VERSION_TUPLE
-
-__version__ = version = '1.0.5'
-__version_tuple__ = version_tuple = (1, 0, 5)