peft-sparta 0.0.1__tar.gz

peft_sparta-0.0.1/LICENSE

@@ -0,0 +1,203 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

peft_sparta-0.0.1/PKG-INFO

@@ -0,0 +1,375 @@
+Metadata-Version: 2.4
+Name: peft-sparta
+Version: 0.0.1
+Summary: SpaRTA adaptation wrapper. Invocation code to load and run SpaRTA adapters for inference
+Author-email: Jesus Rios <jriosal@us.ibm.com>
+Maintainer-email: Jesus Rios <jriosal@us.ibm.com>
+License-Expression: Apache-2.0
+Project-URL: Repository, https://github.com/jmriosal/peft-sparta
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch
+Requires-Dist: transformers
+Requires-Dist: safetensors
+Dynamic: license-file
+
+# PEFT-SpaRTA
+
+SpaRTA (**Spa**rse **R**andom parame**T**er **A**daptation) is a Parameter-Efficient Fine-Tuning (PEFT) alternative to traditional LoRA that reduces the number of trainable parameters by randomly selecting a very small proportion of the model parameters to train on.
+
+
+This Python package provides the invocation code necessary to load and run SpaRTA-adapted models for inference. In particular, it includes the classes 
+
+* `SpaRTAforSequenceClassification`
+* `SpaRTAforCausalLM`
+
+to load a SpaRTA adapter along its pre-trained base (transformer) model architectured, respectively, for sequence classification tasks and autoregressive text generation tasks.
+
+
+We also include the class 
+
+* `SpaRTA`
+
+to facilitate sparse random parameter adaptation of a model and train your own SpaRTA adapters. This implementation is compatible with some of the most popular trainers, as shown in [here](#how-to-train-a-sparta-adapter).
+
+
+For more details on how SpaRTA works see our [paper](https://arxiv.org/pdf/2502.15975). The original implementation of SpaRTA can be found in https://github.com/IBM/sparta.
+
+
+## Installation
+
+```bash
+pip install peft-sparta
+```
+
+## How to use it for inference
+
+### Download a SpaRTA adapter from a Hugging Face repository
+
+Let's download a [SpaRTA adapter](https://huggingface.co/jesusriosal/sparta-gemma_2b-sst2) that spacializes the *google/gemma-2b* model to do *sentiment classification* of English sentences.
+
+```bash
+
+ADAPTER_DIR='/my_sparta_adapters/sparta-gemma_2b/'
+
+mkdir -p $ADAPTER_DIR
+
+hf download jesusriosal/sparta-gemma_2b-sst2 --local-dir $ADAPTER_DIR
+
+```
+
+
+### Load the SpaRTA adapter and create the adapted model
+
+ ```python
+from peft_sparta import SpaRTAforSequenceClassification
+
+adapter_dir = '/my_sparta_adapters/sparta-gemma_2b/'
+
+model = SpaRTAforSequenceClassification(
+    adapter = adapter_dir,
+    device = 'cuda')
+
+print(model)
+```
+
+```none
+(SpaRTA)ModelForSeqClassification(
+	adapter = '/my_sparta_adapters/sparta-gemma_2b/'
+	model = 'google/gemma-2b'
+	id2label = {0: 'negative', 1: 'positive'}
+)
+```
+
+### Inputs
+Let's use our adapted model to classify a few sentences. For this adapter, the model consumes the sentences directly. No formating is needed  
+
+```python
+
+sentences = ["I enjoyed very much the movie.", 
+             "It was painful to watch.", 
+             "I couldn't enjoy more the movie.",
+             "It was a bad movie."]
+```
+### Inference 
+
+#### Probabilistic classification
+
+The adapted model can give us its estimated probabilities that each sentence (row) has negative (first column) or positive (second column) sentiment.
+
+```python
+
+class_probs = model.classify(sentences) 
+
+print(class_probs)
+```
+
+```none
+tensor([[0.1152, 0.8848],
+        [0.9497, 0.0503],
+        [0.1689, 0.8311],
+        [0.9720, 0.0280]], device='cuda:0')
+```
+
+To identify which column correspond to each class, use:
+
+```python
+print(model.id2label)
+```
+```none
+{'0': 'negative', '1': 'positive'}
+```
+
+Here are the model's estimated probabilities of positive sentiment for each sentence
+
+```python
+for sentence, pos_prob in zip(sentences, class_probs[:,1]):
+    print(f"{pos_prob.item()*100:>4.0f}%\t{sentence}")
+```
+
+```none
+ Prob   Sentence
+ ----   -----------------------------
+  88%	I enjoyed very much the movie.
+   5%	It was painful to watch.
+  83%	I couldn't enjoy more the movie.
+   3%	It was a bad movie.
+```
+
+
+
+#### Deciding the sentiment class of each sentence (deterministic classification)
+
+We have seen how the model makes probabilistic assessments of the sentiment of each sentence. If we want the model to make a definitive decison on whether the sentence has positive or negative sentiment, we can use: 
+
+```python
+classes = model.decide_class(sentences) 
+```
+
+to obtain the model's predicted class of each sentence. Basically, the model takes the most likely class as its sentiment prediction of a sentence
+```python
+for sentence, sentence_class in zip(sentences, classes):
+    print(f"'{sentence_class}':  {sentence}")
+```
+
+```none
+ Sentiment   Sentence
+-----------  -------------------------------
+'positive':  I enjoyed very much the movie.
+'negative':  It was painful to watch.
+'positive':  I couldn't enjoy more the movie.
+'negative':  It was a bad movie.
+```
+
+### Input templates
+
+Sometimes the input to the model may need to be formatted before our adapted model can processs it. 
+This is typicaly the case when using instruction-following models, for which wrapping the input within an instruction, formatted with the model's chat template, can be advantageous. In these cases, we can use the following `input_template` argument to specify the formatting over raw inputs used during training, and needed during inference. 
+
+To see this, let's use [another SpaRTA adapter](https://huggingface.co/jesusriosal/sparta-gemma_2b_it-sst2) for *sentiment classification* based on the [google/gemma-2b-it](https://huggingface.co/google/gemma-2b-it) model.  
+
+
+```bash
+hf download jesusriosal/sparta-gemma_2b-sst2 --local-dir '/my_sparta_adapters/sparta-gemma_2b_it/'
+```
+
+```python
+
+from peft_sparta import SpaRTAforSequenceClassification
+
+adapter_dir = '/my_sparta_adapters/sparta-gemma_2b_it/'
+
+model = SpaRTAforSequenceClassification(
+    adapter=adapter_dir,
+    device='cuda',
+    input_template = ("<start_of_turn>user\n"
+                      "Determine the sentiment of the following sentence about a movie. "
+                      "The sentiment can only be classified as positive or negative.\n"
+                      "Sentence: {sentence}" 
+                      "<end_of_turn>\n<start_of_turn>model\n"
+                      "The sentiment of the sentence is")
+    )
+
+print(model)
+```
+
+```none
+(SpaRTA)ModelForSeqClassification(
+	adapter = '/my_sparta_adapters/sparta-gemma_2b_it/'
+	model = 'google/gemma-2b-it'
+	id2label = {0: 'negative', 1: 'positive'}
+)
+```
+
+This SpaRTA adapter was trained formating the input sentences to be classified with the `input_template` (see `model.template` printout below), which included a task instruction. This ensures that during inference the same formatting is used on the inputs to be classified.
+
+
+```python
+print(model.template)
+```
+
+```none
+<start_of_turn>user
+Determine the sentiment of the following sentence about a movie. The sentiment can only be classified as positive or negative.
+Sentence: {sentence}<end_of_turn>
+<start_of_turn>model
+The sentiment of the sentence is
+```
+
+
+For example, the sentence 
+
+```none
+I enjoyed very much the movie.
+```
+
+is converted to 
+
+```none
+<start_of_turn>user
+Determine the sentiment of the following sentence about a movie. The sentiment can only be classified as positive or negative.
+Sentence: I enjoyed very much the movie.<end_of_turn>
+<start_of_turn>model
+The sentiment of the sentence is
+```
+before passing it to the model for classification 
+
+
+Thus, to classify the (raw, non-formatted) sentences [above](#inputs) we proceed as follows
+
+```python
+sentences = [{'sentence': sent} for sent in sentences]
+
+class_probs = model.classify(sentences)
+
+# prob of positive sentiment for each sentence 
+for sentence, pos_prob in zip(sentences, class_probs[:,1]):
+    print(f"{pos_prob.item()*100:>4.0f}%\t{sentence['sentence']}")
+```
+ 
+```none
+ 100%	I enjoyed very much the movie.
+   0%	It was painful to watch.
+ 100%	I couldn't enjoy more the movie.
+   0%	It was a bad movie.
+```
+
+```python
+classes = model.decide_class(sentences)
+
+for sentence, sentence_class in zip(sentences, classes):
+    print(f"'{sentence_class}':  {sentence['sentence']}")
+
+```
+
+```none
+ Sentiment   Sentence
+-----------  -------------------------------
+'positive':  I enjoyed very much the movie.
+'negative':  It was painful to watch.
+'positive':  I couldn't enjoy more the movie.
+'negative':  It was a bad movie.
+```
+
+
+
+### Out-of-Distribution performance evaluations 
+
+If you have a labeled dataset with English sentences and their sentiment labels, like the one below, you can evaluate the performace of these models on that dataset as follows. 
+
+Given the following dataset of new, unseen sentences and their sentiment labels:
+
+```python
+test_sentences = ["it's a charming journey. ",
+                  "bleak and desperate",
+                  "nolan is poised to embark a major career as a commercial yet inventive filmmaker.",
+                  "the acting, costumes, music, cinematography and sound are all astounding. ",
+                  "it's slow -- very, very slow. ",
+                  "the film is a refreshingly serious look at young women.",
+                  "a sometimes tedious film.",
+                  "like doing last year's taxes with your ex-wife.",
+                  "you don't have to know about music to appreciate the film. ",
+                  "in exactly 89 minutes, most of which passed as slowly as if i'd been sitting naked on an igloo, the movie sank from quirky to jerky to utter turkey."]
+
+test_labels = [1, 0, 1, 1, 0, 1, 0, 0, 1, 0]
+
+```
+where a label of 0 represents *negative* sentiment and a label of 1 *positive*.
+
+We evaluate the performance of the model on this labeled dataset as follows. We will need to first put each sentence within a dictionary with a key named `'sentence'` for the model with the `input_template`, so the sentences can be consumed by it accordingly.
+
+```pyhton
+test_sentences = [{'sentence': sent} for sent in test_sentences] # for the model with input_template
+
+model.evaluate(test_sentences, test_labels, batch_size=64)
+```
+
+```none
+loss: 0.002
+accuracy: 100%
+confusion matrix: [5, 0
+                   0, 5]
+balanced accuracy: 100% 
+MCC: 1.0
+F1-score: 1.0
+```
+
+## How to train a SpaRTA adapter
+
+Given a pre-trained model, we prepare it for fine-tuning with SpaRTA by 
+
+```python
+from peft_sparta import SpaRTA
+
+model = SpaRTA(model, sparsity=0.99)
+```
+
+This adds the adapter to the pre-trained model. The adapter consists of non-trainable randomly sampled *indices* and trainable *deltas*, representing the changes to the original model parameters for those indices. Note that in this case we have chosen a *sparsity* level of 99%, meaning that we target to keep only 1% of the model parameters to be trainable.
+
+
+Our `SpaRTA` wrapper class supports the following arguments:
+
+* `model (nn.Module)` Pre-trained model to be adapted.
+
+* `sparsity (float)` Target fraction of the total number of model parameters to make non-trainable. Must be *0 < sparsity < 1*.
+
+* `frozen_modules (list[str], optional)` List of layers name substrings to make entirely frozen (non-trainable). Classification heads ("*score*") will always be fully-trainable by default. Defaults to [*"embed_tokens", "self_attn.q", "self_attn.k", "mlp", "norm"*].
+
+* `trainable_tokens (list[int], optional)` List of (unique) token ids whose embeddings should be fully-trainable. Useful for newly added (special) tokens to the vocabulary. Defaults to *None*.
+
+* `dropout (float, optional)` Dropout probability applied to the trainable parameters during training. Must be *0 <= dropout < 1*. Defaults to *0*.
+
+
+The following notebooks illustrate examples of how to train a SpaRTA adapter with several popular trainers. 
+
+1. [Linear regression](training_examples/linear_regression.ipynb)
+2. [Sequence classification](training_examples/classification.ipynb)
+3. [Text generation](training_examples/text_generation.ipynb)
+
+
+
+## Citation
+
+```bibtex
+@article{rios2025sparsity,
+  title={Sparsity may be all you need: Sparse random parameter adaptation},
+  author={Rios, Jesus and Dognin, Pierre and Luss, Ronny and Ramamurthy, Karthikeyan N},
+  journal={arXiv preprint arXiv:2502.15975},
+  year={2025}
+}
+```
+
+```bibtex
+@software{rios2025sparta,
+  title   = {{PEFT-SpaRTA}},
+  author  = {Rios, Jesus},
+  url     = {https://github.com/jmriosal/peft-sparta}
+}
+
+
+```
+
+
+
+