compel5 0.1.0__tar.gz

compel5-0.1.0/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2023 by Damian Stewart
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

compel5-0.1.0/PKG-INFO

@@ -0,0 +1,414 @@
+Metadata-Version: 2.4
+Name: compel5
+Version: 0.1.0
+Summary: A maintained fork of Compel for modern Diffusers and Transformers versions.
+Author: Damian Stewart
+Maintainer: Christian Tannahill
+Project-URL: Homepage, https://github.com/Cstannahill/compel5
+Project-URL: Repository, https://github.com/Cstannahill/compel5.git
+Project-URL: Issues, https://github.com/Cstannahill/compel5/issues
+Project-URL: Changelog, https://github.com/Cstannahill/compel5/releases
+Keywords: diffusers,transformers,stable-diffusion,sdxl,prompt-weighting,text-embeddings
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch
+Requires-Dist: transformers<6,>=5
+Requires-Dist: diffusers>=0.35
+Requires-Dist: pyparsing<4,>=3
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Provides-Extra: examples
+Requires-Dist: notebook>=6.5.7; extra == "examples"
+Dynamic: license-file
+
+# Compel
+A text prompt weighting and blending library for transformers-type text embedding systems, by [@damian0815](https://github.com/damian0815).
+
+With a flexible and intuitive syntax, you can re-weight different parts of a prompt string and thus re-weight the different parts of the embedding tensor produced from the string.
+
+Tested and developed against Hugging Face's `StableDiffusionPipeline` but it should work with any diffusers-based system that uses an `Tokenizer` and a `Text Encoder` of some kind.  
+
+Adapted from the [InvokeAI](https://github.com/invoke-ai) prompting code (also by [@damian0815](https://github.com/damian0815)).
+
+Note that cross-attention control `.swap()` is currently ignored by Compel, but you can use it by calling `build_conditioning_tensor_for_prompt_object()` yourself, and implementing cross-attention control in your diffusion loop.
+
+## Installation
+
+`pip install compel`
+
+Current main-branch support targets:
+
+- Python `>=3.10`
+- `transformers >=5,<6`
+
+## Documentation
+
+Documentation is [here](doc/).
+
+## Demo
+
+See [compel-demo.ipynb](compel-demo.ipynb)
+
+<a target="_blank" href="https://colab.research.google.com/github/damian0815/compel/blob/main/compel-demo.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
+</a>
+
+## Quickstart
+
+With Hugging Face diffusers >=0.12:
+
+```python
+from diffusers import StableDiffusionPipeline
+from compel import CompelForSD
+
+pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
+compel = CompelForSD(pipeline)
+
+# upweight "ball"
+prompt = "a cat playing with a ball++ in the forest"
+conditioning = compel(prompt)
+
+# generate image
+images = pipeline(prompt_embeds=conditioning.embeds, num_inference_steps=20).images
+images[0].save("image.jpg")
+```
+
+#### With negative prompt:
+
+```python
+from diffusers import StableDiffusionPipeline
+from compel import CompelForSD
+
+pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
+compel = CompelForSD(pipeline)
+
+# upweight "ball"
+prompt = "a cat playing with a ball++ in the forest"
+negative_prompt = "blurry, low quality, deformed"
+conditioning = compel(prompt, negative_prompt=negative_prompt)
+
+# generate image
+images = pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=20).images
+images[0].save("image.jpg")
+```
+
+#### Batched:
+
+```python
+from diffusers import StableDiffusionPipeline
+from compel import CompelForSD
+
+pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
+compel = CompelForSD(pipeline)
+
+prompt = ["a cat playing with a ball++ in the forest", "a dog wearing a hat++"]
+negative_prompt = ["blurry, low quality, deformed", "painting"] 
+# or: negative_prompt = "blurry, low quality, deformed" # shared across all generated images
+conditioning = compel(prompt, negative_prompt=negative_prompt)
+
+# generate image
+images = pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=20).images
+images[0].save("image.jpg")
+```
+
+### SDXL:
+
+```python
+from diffusers import DiffusionPipeline
+from compel import CompelForSDXL
+import torch
+
+device = 'cuda'
+pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", 
+                                             use_safetensors=True, torch_dtype=torch.float16).to(device)
+
+prompt = "a cat playing with a ball++ in the forest"
+negative_prompt = "low quality, blurry"
+compel = CompelForSDXL(pipeline)
+conditioning = compel(prompt, negative_prompt=negative_prompt)
+
+generator = torch.Generator().manual_seed(42)
+image = pipeline(prompt_embeds=conditioning.embeds, 
+                 pooled_prompt_embeds=conditioning.pooled_embeds,
+                 negative_prompt_embeds=conditioning.negative_embeds,
+                 negative_pooled_prompt_embeds=conditioning.negative_pooled_embeds,
+                 num_inference_steps=25, width=1024, height=1024, generator=generator).images[0]
+image.save('sdxl_image.jpg')
+```
+
+### Flux
+
+```python
+from diffusers import FluxPipeline
+from compel import CompelForFlux
+import torch
+
+device = "mps"
+pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.float32).to(device) # bfloat16 causes NaN on MPS
+compel = CompelForFlux(pipe)
+
+prompt = "Astronaut---- in a jungle++++, cold color palette, muted colors, detailed, 8k"
+
+conditioning = compel(prompt)
+generator = torch.Generator().manual_seed(42)
+images = pipe(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds,
+             num_inference_steps=4, width=512, height=512, generator=generator)
+```
+
+### Style prompt
+
+Works with SDXL or Flux:
+
+```python
+from diffusers import DiffusionPipeline
+from compel import CompelForSDXL
+import torch
+
+device = 'cuda'
+pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", 
+                                             use_safetensors=True, torch_dtype=torch.float16).to(device)
+
+prompt = "a cat playing with a ball++ in the forest"
+style_prompt = "painting by van gogh, impasto, thick brush strokes"
+negative_prompt = "low quality, blurry"
+negative_style_prompt = "photography"
+compel = CompelForSDXL(pipeline) # or CompelForFlux
+conditioning = compel(prompt=prompt, negative_prompt=negative_prompt, 
+                      style_prompt=style_prompt, negative_style_prompt=negative_style_prompt)
+
+generator = torch.Generator().manual_seed(42)
+image = pipeline(prompt_embeds=conditioning.embeds, 
+                 pooled_prompt_embeds=conditioning.pooled_embeds,
+                 negative_prompt_embeds=conditioning.negative_embeds,
+                 negative_pooled_prompt_embeds=conditioning.negative_pooled_embeds,
+                 num_inference_steps=25, width=1024, height=1024, generator=generator).images[0]
+image.save('sdxl_image.jpg')
+```
+
+## Textual Inversion support
+
+If you want to have access to 🤗diffusers textual inversions, instantiate a `DiffusersTextualInversionManager` and pass it on Compel init:
+
+```python
+from diffusers import StableDiffusionPipeline
+from compel import CompelForSD, DiffusersTextualInversionManager
+pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
+textual_inversion_manager = DiffusersTextualInversionManager(pipeline)
+compel = CompelForSD(pipe=pipeline, textual_inversion_manager=textual_inversion_manager)
+```
+
+## Memory usage/VRAM leaks
+
+If you run into memory issues, please make sure you're running compel inside `with torch.no_grad():` blocks. 
+
+If this doesn't help, you could try this advice offered by @kshieh1: 
+> After image generation, you should explictly de-reference the tensor object (i.e., prompt_embeds = None) and call gc.collect()
+
+See https://github.com/damian0815/compel/issues/24 for more details. Thanks @kshieh1 !
+
+## Local Checkpoint Smoke Tests
+
+The default test suite uses tiny local diffusers components so it stays fast and CI-friendly.
+
+If you want to validate real local SDXL `.safetensors` checkpoints with `StableDiffusionXLPipeline.from_single_file(...)`,
+run:
+
+```bash
+COMPEL_RUN_LOCAL_CHECKPOINT_TESTS=1 python -m unittest test.test_diffusers_smoke -v
+```
+
+This opt-in path only uses local files and does not pull remote model weights.
+
+## Changelog
+
+#### 2.3.1 - Fix for 78 tokens / 77 tokens issue with SDXL; add `device` arg to `CompelFor*` constructors (thanks @dx2-66)
+
+Current main-branch validation also includes opt-in local SDXL `.safetensors` single-file smoke tests for
+`StableDiffusionXLPipeline.from_single_file(...)`; see `COMPEL_RUN_LOCAL_CHECKPOINT_TESTS=1` above.
+
+### 2.3.0 - Tokenization info, SplitLongTextMode CLS token handling, negative/style bugfixes
+
+* `CompelFor*` objects now return tokenization info via `conditioning.tokenization_info` dict, which contains keys for `main_positive` and (where appropriate) `main_negative`, `style_positive` and `style_negative`.
+* `SplitLongTextMode` enum has additional options for handling the CLS (EOS) token - either the CLS token can be copied from the first chunk to subsequent chunks, or all CLS tokens can be merged and the merge duplicated on all chunks. See `SplitLongTextMode` enum for details.
+* Misc edge case fixes for handling different combinations of style prompts and negative prompts with SDXL/Flux
+
+#### 2.2.1 - fix for #116
+
+### 2.2.0 - Flux support, usability improvement
+
+* Flux is now supported
+* Added `CompelForSD`, `CompelForFlux` and `CompelForSDXL` to simplify and streamline embeds vs pooled_embeds, negative prompts, and style prompts handling. See compel-demo-*.py/.ipynb for usage examples.
+* Performance improvements when working with non-weighted prompts. If a prompt is passed with no weighting, the weighting logic is completely bypassed. **This may slightly change the outputs when no weighting is applied** - revert to the old slower behaviour by calling `compel.disable_no_weights_bypass()`.
+* When working with `CompelForSD` and `CompelForSDXL` classes, non-truncation is now the default.
+
+#### 2.1.1 - expose `split_long_text_mode` to top-level Compel, default to `SENTENCES`, fix bug where splitting would fail sometimes.
+
+### 2.1.0 - add `split_long_text_mode` arg to control word splitting for long prompts when `truncate_long_prompts` is False. See `SplitLongTextMode`. Defaults to `WORDS`. 
+
+#### 2.0.3 - include contributed fixes #64, #80 and fix license in pyproject.toml/pypi
+
+#### 2.0.2 - fix for `pipeline.enable_sequential_cpu_offloading()` with SDXL models (you need to pass `device='cuda'` on compel init)
+
+#### 2.0.1 - fix for [#45](https://github.com/damian0815/compel/issues/45) padding issue with SDXL non-truncated prompts and `.and()` 
+
+### 2.0.0 - SDXL Support
+
+With big thanks to Patrick von Platen from Hugging Face for [the pull request](https://github.com/damian0815/compel/pull/41), Compel now supports SDXL. Use it like this: 
+
+```py
+from compel import Compel, ReturnedEmbeddingsType
+from diffusers import DiffusionPipeline
+import torch
+
+pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to("cuda")
+compel = Compel(tokenizer=[pipeline.tokenizer, pipeline.tokenizer_2] , text_encoder=[pipeline.text_encoder, pipeline.text_encoder_2], returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED, requires_pooled=[False, True])
+# upweight "ball"
+prompt = "a cat playing with a ball++ in the forest"
+conditioning, pooled = compel(prompt)
+# generate image
+image = pipeline(prompt_embeds=conditioning, pooled_prompt_embeds=pooled, num_inference_steps=30).images[0]
+```
+
+Please note that this is a **breaking change** if you've been using clip skip: the old boolean arg `use_penultimate_clip_layer` has been replaced with an enum `ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NORMALIZED`.
+
+
+#### 1.2.1 - actually apply `.and()` weights
+
+### 1.2.0 - Concatenate embeddings using `.and()`
+
+For Stable Diffusion 2.1 I've been experimenting with a new feature: concatenated embeddings. What I noticed, for example, is that for more complex prompts image generation quality becomes wildly better when the prompt is broken into multiple parts and fed to OpenCLIP separately.
+
+TL;DR: you can now experiment with breaking up your prompts into segments, which for SD2.1 appears to improve the generated image. The syntax is `("prompt part 1", "prompt part 2").and()`. You can have more than one part, and you can also weight them, eg `("a man eating an apple", "sitting on the roof of a car", "high quality, trending on artstation, 8K UHD").and(1, 0.5, 0.5)` which will assign weight `1` to `man eating an apple` and `0.5` to `sitting on the roof of a car` and `high quality, trending on artstation, 8K UHD`. 
+
+Here's a nonsense example from the InvokeAI discord #garbage-bin channel, created by gogurt enjoyer's incredible [nightmare prompt generator](https://huggingface.co/cactusfriend/nightmare-invokeai-prompts):
+
+```
+a moist sloppy pindlesackboy sloppy hamblin' bogomadong, Clem Fandango is pissed-off, Wario's Woods in background, making a noise like ga-woink-a
+```
+
+Plugging this straight into SD2.1 we get this, which is really not a good image:
+![](images/000075.6dfd7adf.466129594.png)
+
+However, if the prompt is broken up into chunks and fed into OpenCLIP separately as four separate prompts, and then concatenated:
+
+```
+a moist sloppy pindlesackboy sloppy hamblin' bogomadong
+
+Clem Fandango is pissed-off
+
+Wario's Woods in background
+
+making a noise like ga-woink-a
+```
+
+then output image with the same seed is *so much* better:
+![](images/000076.68b1c320.466129594.png)
+
+In the new `.and()` syntax you would prompt this as follows:
+```
+("a moist sloppy pindlesackboy sloppy hamblin' bogomadong", "Clem Fandango is pissed-off", "Wario's Woods in background", "making a noise like ga-woink-a").and()
+```
+
+The effect can be more or less subtle. Here for example is 
+```
+A dream of a distant galaxy, by Caspar David Friedrich, matte painting, trending on artstation, HQ
+```
+![](images/000129.1b33b559.2793529321.png)
+
+And the same split into two parts:
+```
+A dream of a distant galaxy, by Caspar David Friedrich, matte painting
+
+trending on artstation, HQ
+```
+![](images/000128.b5d5cd62.2793529321.png)
+
+The Compel prompt for this is: 
+```
+("A dream of a distant galaxy, by Caspar David Friedrich, matte painting", "trending on artstation, HQ").and()
+```
+
+
+
+
+#### 1.1.6 - misc small fixes
+- add `DiffusersTextualInversionManager` (thanks @pdoane)
+- fix batch embedding generation with truncated/non-truncated prompt lengths (#18, thanks @abassino)
+- add note about memory leakage (ref #24, thanks @kshieh1) 
+- fix incorrect parsing when commas are not followed by whitespace (#34, thanks @moono)
+
+#### 1.1.5 - fix for compel turning numbers into floats for text inside parentheses
+
+#### 1.1.4 - fixes for #23 (sequential offload) and InvokeAI issue #3442 (allow hyphens in LoRA names) 
+
+#### 1.1.3 - enable fetching the penultimate CLIP hidden layer (aka "clip skip")
+
+To use, pass `use_penultimate_clip_layer=True` when initializing your `Compel` instance. Note that there's no need to pass this flag for SD2.0/SD2.1 because diffusers already throws away the last hidden layer when loading the SD2.0+ text encoder.
+
+#### 1.1.2 - fix for #21 (crash when parsing long prompts with truncation enabled if there is weighted fragments beyond the truncation boundary)
+
+#### 1.1.1 - fix for #22 (issues parsing `.` characters inside parentheses)
+
+#### 1.1.0 - support for parsing `withLora`/`useLora` on `parse_prompt_string()`.
+
+* `Compel.parse_prompt_string()` now returns a `Conjunction`
+* any appearances of `withLora(name[, weight])` or `useLora(name[, weight])` anywhere in the prompt string will be parsed to `LoraWeight` instances, and returned on the outermost `Conjunction` returned by `parse_prompt_string()`.
+
+#### 1.0.5 - fix incorrect parsing when passing invalid (auto1111) syntax that has a float
+
+also fix test case for default swap parameters
+
+#### 1.0.4 - fix embeddings for empty swap target (eg `cat.swap("")`) when truncation is disabled 
+
+#### 1.0.3 - better defaults for .swap (https://github.com/damian0815/compel/issues/8)
+
+#### 1.0.2 - fix padding for non-truncated batched embeddings (https://github.com/damian0815/compel/issues/9)
+
+#### 1.0.1 - fix for InvokeAI's `--free_gpu_mem` option
+
+### 1.0.0 - new downweighting algorithm 
+
+Downweighting now works by applying an attention mask to remove the downweighted tokens, rather than literally removing them from the sequence. This behaviour is the default, but the old behaviour can be re-enabled by passing `downweight_mode=DownweightMode.REMOVE` on init of the `Compel` instance.
+
+Formerly, downweighting a token worked by both multiplying the weighting of the token's embedding, and doing an inverse-weighted blend with a copy of the token sequence that had the downweighted tokens removed. The intuition is that as weight approaches zero, the tokens being downweighted should be actually removed from the sequence. However, removing the tokens resulted in the positioning of all downstream tokens becoming messed up. The blend ended up blending a lot more than just the tokens in question. 
+
+As of v1.0.0, taking advice from @keturn and @bonlime (https://github.com/damian0815/compel/issues/7) the procedure is by default different. Downweighting still involves a blend but what is blended is a version of the token sequence with the downweighted tokens masked out, rather than removed. This correctly preserves positioning embeddings of the other tokens. 
+
+Also a bugfix: fix black images on weight 0 (https://github.com/invoke-ai/InvokeAI/issues/2832)
+
+### 0.1.10 - add support for prompts longer than the model's max token length. 
+
+To enable, initialize `Compel` with `truncate_long_prompts=False` (default is True). Prompts that are longer than the model's `max_token_length` will be chunked and padded out to an integer multiple of `max_token_length`. 
+
+Note that even if you don't use a negative prompt, you'll need to build a conditioning tensor for a negative prompt of at least `""`, and use `compel.pad_conditioning_tensors_to_same_length()`, otherwise the you'll get an error about mismatched conditioning tensor lengths:
+
+```python
+compel = Compel(..., truncate_long_prompts=False)
+prompt = "a cat playing with a ball++ in the forest, amazing, exquisite, stunning, masterpiece, skilled, powerful, incredible, amazing, trending on gregstation, greg, greggy, greggs greggson, greggy mcgregface, ..." # very long prompt
+conditioning = compel.build_conditioning_tensor(prompt)
+negative_prompt = "" # it's necessary to create an empty prompt - it can also be very long, if you want
+negative_conditioning = compel.build_conditioning_tensor(negative_prompt)
+[conditioning, negative_conditioning] = compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])
+```
+
+#### 0.1.9 - broken
+
+#### 0.1.8 - downgrade Python min version to 3.7
+
+#### 0.1.7 - InvokeAI compatibility