tasksource 0.0.39__tar.gz → 0.0.41__tar.gz

{tasksource-0.0.39/src/tasksource.egg-info → tasksource-0.0.41}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tasksource
-Version: 0.0.39
+Version: 0.0.41
 Summary: Preprocessings to prepare datasets for a task
 Home-page: https://github.com/sileod/tasksource/
 Classifier: Programming Language :: Python :: 3
@@ -9,8 +9,17 @@ Classifier: Intended Audience :: Developers
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
-
-## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for effortless scaling of multi-task learning and evaluation.
+Requires-Dist: dotwiz
+Requires-Dist: funcy
+Requires-Dist: datasets
+Requires-Dist: exrex
+Requires-Dist: magicattr
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: sorcery
+
+## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for instant and interchangeable use
 
 Huggingface Datasets is an excellent library, but it lacks standardization, and datasets often require preprocessing work to be used interchangeably.
 `tasksource` streamlines interchangeable datasets usage to scale evaluation or multi-task learning.
@@ -21,7 +30,7 @@ Each dataset is standardized to a `MultipleChoice`, `Classification`, or `TokenC
 `pip install tasksource`
 ```python
 from tasksource import list_tasks, load_task
-df = list_tasks() # takes some time
+df = list_tasks(multilingual=False) # takes some time
 
 for id in df[df.task_type=="MultipleChoice"].id:
     dataset = load_task(id) # all yielded datasets can be used interchangeably
@@ -29,11 +38,18 @@ for id in df[df.task_type=="MultipleChoice"].id:
 
 Browse the 500+ curated tasks in tasks.md (200+ MultipleChoice tasks, 200+ Classification tasks), and feel free to request a new task. Datasets are downloaded to $HF_DATASETS_CACHE (like any Hugging Face dataset), so ensure you have more than 100GB of space available.
 
-### Pretrained model:
+You can now also use:
+```python
+load_dataset("tasksource/data", "glue/rte",max_rows=30_000)
+```
+
+### Pretrained models:
 
 Text encoder pretrained on tasksource reached state-of-the-art results: [🤗/deberta-v3-base-tasksource-nli](https://hf.co/sileod/deberta-v3-base-tasksource-nli)
 
-Tasksource pretraining is notably helpful for RLHF reward modeling.
+Tasksource pretraining is notably helpful for RLHF reward modeling or any kind of classification, including zero-shot. You can also find a large, and a multilingual version.
+
+[u/conceptofmind](https://github.com/conceptofmind) also trained Llama on tasksource: [🤗/conceptofmind/Tasksource-Open-Llama-13b](https://hf.co/conceptofmind/Tasksource-Open-Llama-13b)
 
 ### tasksource-instruct
 
@@ -43,7 +59,7 @@ The repo also contains some recasting code to convert tasksource datasets to ins
 
 ### tasksource-label-nli
 
-We also recast all classification tasks into entailment detection, to improve entailment-based zero-shot classification detection:
+We also recast all classification tasks as natural language inference, to improve entailment-based zero-shot classification detection:
 [🤗/zero-shot-label-nli](https://huggingface.co/datasets/tasksource/zero-shot-label-nli)
 
 ### Write and use custom preprocessings

tasksource-0.0.39/PKG-INFO → tasksource-0.0.41/README.md

@@ -1,16 +1,4 @@
-Metadata-Version: 2.1
-Name: tasksource
-Version: 0.0.39
-Summary: Preprocessings to prepare datasets for a task
-Home-page: https://github.com/sileod/tasksource/
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: BSD License
-Classifier: Intended Audience :: Developers
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-License-File: LICENSE
-
-## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for effortless scaling of multi-task learning and evaluation.
+## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for instant and interchangeable use
 
 Huggingface Datasets is an excellent library, but it lacks standardization, and datasets often require preprocessing work to be used interchangeably.
 `tasksource` streamlines interchangeable datasets usage to scale evaluation or multi-task learning.
@@ -21,7 +9,7 @@ Each dataset is standardized to a `MultipleChoice`, `Classification`, or `TokenC
 `pip install tasksource`
 ```python
 from tasksource import list_tasks, load_task
-df = list_tasks() # takes some time
+df = list_tasks(multilingual=False) # takes some time
 
 for id in df[df.task_type=="MultipleChoice"].id:
     dataset = load_task(id) # all yielded datasets can be used interchangeably
@@ -29,11 +17,18 @@ for id in df[df.task_type=="MultipleChoice"].id:
 
 Browse the 500+ curated tasks in tasks.md (200+ MultipleChoice tasks, 200+ Classification tasks), and feel free to request a new task. Datasets are downloaded to $HF_DATASETS_CACHE (like any Hugging Face dataset), so ensure you have more than 100GB of space available.
 
-### Pretrained model:
+You can now also use:
+```python
+load_dataset("tasksource/data", "glue/rte",max_rows=30_000)
+```
+
+### Pretrained models:
 
 Text encoder pretrained on tasksource reached state-of-the-art results: [🤗/deberta-v3-base-tasksource-nli](https://hf.co/sileod/deberta-v3-base-tasksource-nli)
 
-Tasksource pretraining is notably helpful for RLHF reward modeling.
+Tasksource pretraining is notably helpful for RLHF reward modeling or any kind of classification, including zero-shot. You can also find a large, and a multilingual version.
+
+[u/conceptofmind](https://github.com/conceptofmind) also trained Llama on tasksource: [🤗/conceptofmind/Tasksource-Open-Llama-13b](https://hf.co/conceptofmind/Tasksource-Open-Llama-13b)
 
 ### tasksource-instruct
 
@@ -43,7 +38,7 @@ The repo also contains some recasting code to convert tasksource datasets to ins
 
 ### tasksource-label-nli
 
-We also recast all classification tasks into entailment detection, to improve entailment-based zero-shot classification detection:
+We also recast all classification tasks as natural language inference, to improve entailment-based zero-shot classification detection:
 [🤗/zero-shot-label-nli](https://huggingface.co/datasets/tasksource/zero-shot-label-nli)
 
 ### Write and use custom preprocessings