tasksource 0.0.40__tar.gz → 0.0.41__tar.gz

{tasksource-0.0.40 → tasksource-0.0.41}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tasksource
-Version: 0.0.40
+Version: 0.0.41
 Summary: Preprocessings to prepare datasets for a task
 Home-page: https://github.com/sileod/tasksource/
 Classifier: Programming Language :: Python :: 3
@@ -9,6 +9,15 @@ Classifier: Intended Audience :: Developers
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: dotwiz
+Requires-Dist: funcy
+Requires-Dist: datasets
+Requires-Dist: exrex
+Requires-Dist: magicattr
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: sorcery
 
 ## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for instant and interchangeable use
 
@@ -29,7 +38,12 @@ for id in df[df.task_type=="MultipleChoice"].id:
 
 Browse the 500+ curated tasks in tasks.md (200+ MultipleChoice tasks, 200+ Classification tasks), and feel free to request a new task. Datasets are downloaded to $HF_DATASETS_CACHE (like any Hugging Face dataset), so ensure you have more than 100GB of space available.
 
-### Pretrained model:
+You can now also use:
+```python
+load_dataset("tasksource/data", "glue/rte",max_rows=30_000)
+```
+
+### Pretrained models:
 
 Text encoder pretrained on tasksource reached state-of-the-art results: [🤗/deberta-v3-base-tasksource-nli](https://hf.co/sileod/deberta-v3-base-tasksource-nli)
 

{tasksource-0.0.40 → tasksource-0.0.41}/README.md

@@ -17,7 +17,12 @@ for id in df[df.task_type=="MultipleChoice"].id:
 
 Browse the 500+ curated tasks in tasks.md (200+ MultipleChoice tasks, 200+ Classification tasks), and feel free to request a new task. Datasets are downloaded to $HF_DATASETS_CACHE (like any Hugging Face dataset), so ensure you have more than 100GB of space available.
 
-### Pretrained model:
+You can now also use:
+```python
+load_dataset("tasksource/data", "glue/rte",max_rows=30_000)
+```
+
+### Pretrained models:
 
 Text encoder pretrained on tasksource reached state-of-the-art results: [🤗/deberta-v3-base-tasksource-nli](https://hf.co/sileod/deberta-v3-base-tasksource-nli)
 

tasksource-0.0.41/src/tasksource/metadata/original.txt

@@ -0,0 +1,250 @@
+WANLI
+recast/recast_verbnet
+recast/recast_verbcorner
+recast/recast_ner
+recast/recast_sentiment
+recast/recast_puns
+recast/recast_factuality
+recast/recast_megaveridicality
+probability_words_nli/reasoning_1hop
+probability_words_nli/usnli
+probability_words_nli/reasoning_2hop
+nan-nli/joey234--nan-nli
+nli_fever
+breaking_nli
+conj_nli
+fracas
+dialogue_nli
+mpe
+dnc
+recast_white/fnplus
+recast_white/sprl
+recast_white/dpr
+robust_nli/IS_CS
+robust_nli/LI_LI
+robust_nli/ST_WO
+robust_nli/PI_SP
+robust_nli/PI_CD
+robust_nli/ST_SE
+robust_nli/ST_NE
+robust_nli/ST_LM
+robust_nli_is_sd
+robust_nli_li_ts
+gen_debiased_nli/snli_seq_z
+gen_debiased_nli/snli_z_aug
+gen_debiased_nli/snli_par_z
+gen_debiased_nli/mnli_par_z
+gen_debiased_nli/mnli_z_aug
+gen_debiased_nli/mnli_seq_z
+add_one_rte
+hlgd
+conll2003/pos_tags
+conll2003/chunk_tags
+conll2003/ner_tags
+hh-rlhf
+model-written-evals
+fig-qa
+social_i_qa
+balanced-copa
+e-CARE
+insincere-questions
+TuringBench
+vitaminc/tals--vitaminc
+rumoureval_2019/RumourEval2019
+tweet_eval/irony
+tweet_eval/stance_abortion
+tweet_eval/hate
+tweet_eval/stance_atheism
+tweet_eval/stance_climate
+tweet_eval/emoji
+tweet_eval/offensive
+tweet_eval/sentiment
+tweet_eval/emotion
+tweet_eval/stance_feminist
+tweet_eval/stance_hillary
+discovery/discovery
+pragmeval/verifiability
+pragmeval/mrda
+pragmeval/switchboard
+pragmeval/emergent
+pragmeval/gum
+pragmeval/sarcasm
+pragmeval/stac
+pragmeval/pdtb
+silicone/dyda_e
+silicone/oasis
+silicone/meld_s
+silicone/meld_e
+silicone/maptask
+silicone/dyda_da
+silicone/sem
+silicone/iemocap
+lex_glue/scotus
+lex_glue/ledgar
+language-identification
+rotten_tomatoes
+hate_speech18
+sms_spam
+snips_built_in_intents
+hate_speech_offensive
+hyperpartisan_news
+sciie
+citation_intent
+scicite
+lexical_relation_classification/ROOT09
+lexical_relation_classification/CogALexV
+lexical_relation_classification/K&H+N
+lexical_relation_classification/BLESS
+lexical_relation_classification/EVALution
+crowdflower/political-media-bias
+crowdflower/tweet_global_warming
+crowdflower/text_emotion
+crowdflower/political-media-message
+crowdflower/political-media-audience
+crowdflower/economic-news
+crowdflower/corporate-messaging
+crowdflower/airline-sentiment
+crowdflower/sentiment_nuclear_power
+ethics/commonsense
+ethics/deontology
+ethics/justice
+ethics/virtue
+tweets_hate_speech_detection
+wnut_17/wnut_17
+ncbi_disease/ncbi_disease
+acronym_identification
+jnlpba/jnlpba
+ontonotes_english/SpeedOfMagic--ontonotes_english
+blog_authorship_corpus/gender
+blog_authorship_corpus/horoscope
+blog_authorship_corpus/job
+open_question_type
+mc_taco
+discosense
+EffectiveFeedbackStudentWriting
+phrase_similarity
+scientific-exaggeration-detection
+fever-evidence-related/mwong--fever-related
+dynasent/dynabench.dynasent.r1.all/r1
+dynasent/dynabench.dynasent.r2.all/r2
+sem_eval_2010_task_8
+medmcqa
+logiqa
+cycic_classification
+cycic_multiplechoice
+commonsense_qa_2.0
+lingnli
+monotonicity-entailment
+arct
+scinli
+naturallogic
+onestop_qa
+moral_stories/full
+prost
+dynahate
+syntactic-augmentation-nli
+autotnli
+CONDAQA
+webgpt_comparisons
+synthetic-instruct-gptj-pairwise
+scruples
+wouldyourather
+attempto-nli
+defeasible-nli/snli
+defeasible-nli/atomic
+help-nli
+nli-veridicality-transitivity
+natural-language-satisfiability
+lonli
+dadc-limit-nli
+FLUTE
+summarize_from_feedback/comparisons
+folio
+tomi-nli
+avicenna
+SHP
+MedQA-USMLE-4-options-hf
+wikimedqa/medwiki
+cicero
+mutual
+NeQA
+quote-repetition
+redefine-math
+puzzte
+implicatures
+race-c
+spartqa-yn
+spartqa-mchoice
+temporal-nli
+riddle_sense
+clcd-english
+twentyquestions
+reclor
+counterfactually-augmented-imdb
+counterfactually-augmented-snli
+cnli
+boolq-natural-perturbations
+equate
+ScienceQA_text_only
+ekar_english
+implicit-hate-stg1
+logiqa-2.0-nli
+PARARULE-Plus
+mindgames
+universal_dependencies/en_partut/deprel
+universal_dependencies/en_lines/deprel
+universal_dependencies/en_gum/deprel
+universal_dependencies/en_ewt/deprel
+ambient
+path-naturalness-prediction
+cloth
+dgen
+oasst1_pairwise_rlhf_reward
+I2D2
+args_me
+Touche23-ValueEval
+starcon
+banking77
+ruletaker
+lsat_qa/all
+ConTRoL-nli
+tracie
+sherliic
+sen-making/1
+sen-making/2
+mbib-base/cognitive-bias
+mbib-base/fake-news
+mbib-base/gender-bias
+mbib-base/hate-speech
+mbib-base/linguistic-bias
+mbib-base/political-bias
+mbib-base/racial-bias
+mbib-base/text-level-bias
+robustLR
+v1/gen_train234_test2to10
+logical-fallacy
+parade
+cladder
+subjectivity
+MOH
+VUAC
+TroFi
+sharc_modified/mod
+conceptrules_v2
+disrpt/eng.dep.scidtb
+conll2000
+few-nerd/supervised
+com2sense
+scone
+winodict
+fool-me-twice
+monli
+corr2cause
+apt
+twitter-financial-news-sentiment
+SpaceNLI
+propsegment/nli
+HatemojiBuild
+regset
+esci
+dnd_style_intents

{tasksource-0.0.40 → tasksource-0.0.41}/src/tasksource/recast.py

@@ -3,8 +3,8 @@ from datasets import DatasetDict, Dataset
 from sorcery import dict_of
 import string
 
-improper_labels = ['recast/recast_kg_relations','linguisticprobing',"lexglue/scotus","pragmeval/squinky","pragmeval/emobank",'pragmeval/persuasiveness']
-improper_labels += ['glue/stsb', 'sick/relatedness', 'joci', 'utilitarianism', 'amazon_counterfactual/en', 'toxic_conversations', 'ethos/multilabel', 'lex_glue/eurlex','lex_glue/scotus', 'lex_glue/unfair_tos', 'app_reviews', 'humicroedit/subtask-1', 'stackoverflow-questions', 'go_emotions/simplified', 'google_wellformed_query', 'has_part', 'blog_authorship_corpus/age', 'promptCoherence', 'Sarcasm_News_Headline', 'auditor_review/demo-org--auditor_review', 'Dynasent_Disagreement', 'Politeness_Disagreement', 'SBIC_Disagreement', 'SChem_Disagreement', 'Dilemmas_Disagreement', 'sts-companion', 'acceptability-prediction', 'chaos-mnli-ambiguity', 'headline_cause/en_simple', 'oasst1_dense_flat', 'civil_comments']
+improper_labels =['recast/recast_kg_relations','linguisticprobing',"lex_glue/scotus","pragmeval/squinky","pragmeval/emobank",'pragmeval/persuasiveness']
+improper_labels += ['glue/stsb', 'sick/relatedness', 'joci', 'utilitarianism', 'amazon_counterfactual/en', 'toxic_conversations', 'ethos/multilabel', 'lex_glue/eurlex', 'lex_glue/unfair_tos', 'app_reviews', 'humicroedit/subtask-1', 'stackoverflow-questions', 'go_emotions/simplified', 'google_wellformed_query', 'has_part', 'blog_authorship_corpus/age', 'promptCoherence', 'Sarcasm_News_Headline', 'auditor_review/demo-org--auditor_review', 'Dynasent_Disagreement', 'Politeness_Disagreement', 'SBIC_Disagreement', 'SChem_Disagreement', 'Dilemmas_Disagreement', 'sts-companion', 'acceptability-prediction', 'chaos-mnli-ambiguity', 'headline_cause/en_simple', 'oasst1_dense_flat', 'civil_comments']
 improper_labels += ['stsb_multi_mt','MLMA_hate_speech','icl-symbol-tuning-instruct']
 
 def render_options(options):

{tasksource-0.0.40 → tasksource-0.0.41}/src/tasksource/tasks.py

@@ -394,7 +394,25 @@ ethos___multilabel = Classification(
     splits=["train", None, None]
 )
 
-tweet_eval = Classification(sentence1="text", labels="label", config_name=["emoji", "emotion", "hate", "irony", "offensive", "sentiment", "stance_abortion", "stance_atheism", "stance_climate", "stance_feminist", "stance_hillary"])
+tweet_eval = Classification(sentence1="text", labels="label",
+    config_name=["emoji", "emotion", "hate", "irony", "offensive", "sentiment"])
+
+def stance_kwargs(topic):
+    return {
+        "sentence1": constant(f'Topic: {topic}. \n Opinion:\n'), 
+        "sentence2": "text", 
+        "labels": "label", 
+        "config_name": f"stance_{topic.lower()}",
+        "dataset_name": "tweet_eval"
+    }
+
+tweet_eval_abortion = Classification(**stance_kwargs("abortion"))
+tweet_eval_atheism  = Classification(**stance_kwargs("atheism"))
+tweet_eval_climate  = Classification(**stance_kwargs("climate"))
+tweet_eval_feminist = Classification(**stance_kwargs("feminist"))
+tweet_eval_hillary  = Classification(**stance_kwargs("Hillary"))
+
+    
 
 discovery = Classification("sentence1", "sentence2", labels="label", config_name=["discovery"])
 
@@ -489,7 +507,7 @@ go_emotions___simplified = Classification(sentence1="text", labels="labels")
 #ecthr_cases___violation_prediction = Classification(labels="labels", dataset_name="ecthr_cases", config_name="violation-prediction")
 #   too long
 
-scicite = Classification(sentence1="string", labels="label")
+scicite = Classification(sentence1="string", labels="label",dataset_name="allenai/scicite")
 
 liar = Classification(sentence1="statement", labels="label")
 
@@ -557,7 +575,7 @@ SpeedOfMagic_ontonotes_english = TokenClassification(tokens="tokens", labels="ne
 
 blog_authorship_corpus__gender    = Classification(sentence1="text",labels="gender")
 blog_authorship_corpus__age       = Classification(sentence1="text",labels="age")
-blog_authorship_corpus__horoscope = Classification(sentence1="text",labels="horoscope")
+#blog_authorship_corpus__horoscope = Classification(sentence1="text",labels="horoscope")
 blog_authorship_corpus__job       = Classification(sentence1="text",labels="job")
 
 launch_open_question_type = Classification(sentence1="question", labels="resolve_type", dataset_name="launch/open_question_type")
@@ -596,7 +614,9 @@ circa = Classification(
 #    constant(""), choices=["buggy","fixed"], labels=constant(0),
 #    config_name="medium")
 
-effective_feedback_student_writing = Classification("discourse_text", labels="discourse_effectiveness",dataset_name="YaHi/EffectiveFeedbackStudentWriting")
+#effective_feedback_student_writing = Classification("discourse_text", 
+#labels="discourse_effectiveness",dataset_name="YaHi/EffectiveFeedbackStudentWriting")
+# discontinued /!\
 
 #promptSentiment = Classification("text",labels="label",dataset_name="Ericwang/promptSentiment")
 #promptNLI = Classification("premise","hypothesis",labels="label",dataset_name="Ericwang/promptNLI")
@@ -988,7 +1008,7 @@ sharc_classification = Classification("snippet", lambda x:f'{x["scenario"]}\n{x[
 
 conceptrules_v2 = Classification("context", "text", "label", dataset_name="tasksource/conceptrules_v2")
 
-scidtb = Classification("unit1_txt","unit2_txt","label", dataset_name="metaeval/disrpt",config_name='eng.dep.scidtb')
+scidtb = Classification("unit1_txt","unit2_txt","label", dataset_name="metaeval/disrpt",config_name='eng.dep.scidtb.rels')
 
 chunking = TokenClassification("tokens","chunk_tags", dataset_name="conll2000")
 
@@ -1017,7 +1037,8 @@ apt = Classification('text_a','text_b',name('labels',['not_paraphrase','paraphra
 
 #xsum_factuality = Classification("summary",labels="is_factual")
 
-financial_sentiment = Classification("text",labels="label",dataset_name="zeroshot/twitter-financial-news-sentiment")
+financial_sentiment = Classification("text",labels=name('label',['Bearish','Bullish','Neutral']),
+    dataset_name="zeroshot/twitter-financial-news-sentiment")
 
 def _icl_rand(x):
     import random
@@ -1064,4 +1085,22 @@ chatbot_arena = MultipleChoice(constant(""),
     pre_process=_preprocess_chatbot_arena)
 
 dnd_intent = Classification("examples",labels="label_names",
-    dataset_name='neurae/dnd_style_intents')
+    dataset_name='neurae/dnd_style_intents')
+
+fld = Classification("context","hypothesis", "proof_label",
+    dataset_name="hitachi-nlp/FLD.v2")
+
+sdoh_nli = Classification("premise","hypothesis",labels=lambda x:{True:"entailment",False:"not-entailment"}[x['label']],
+    dataset_name="tasksource/SDOH-NLI")
+
+scifact_entailment = Classification(lambda x:"\n".join(x["abstract"]),"claim",
+    labels=lambda x:x['verdict'].replace('NEI','NEUTRAL'),
+    dataset_name="allenai/scifact_entailment")
+
+feasibilityQA = Classification(cat(['knowledge','premise']),'hypothesis','binary_classification_label',
+    dataset_name="tasksource/feasibilityQA")
+                               
+simple_pair = Classification("premise","hypothesis","label", dataset_name="tasksource/simple_pair")
+adjective_scale_probe = Classification("premise","hypothesis","label", dataset_name="tasksource/AdjectiveScaleProbe-nli")
+repectively_nli = Classification("premise","hypothesis","label",dataset_name="tasksource/resnli")
+

{tasksource-0.0.40 → tasksource-0.0.41}/src/tasksource.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tasksource
-Version: 0.0.40
+Version: 0.0.41
 Summary: Preprocessings to prepare datasets for a task
 Home-page: https://github.com/sileod/tasksource/
 Classifier: Programming Language :: Python :: 3
@@ -9,6 +9,15 @@ Classifier: Intended Audience :: Developers
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: dotwiz
+Requires-Dist: funcy
+Requires-Dist: datasets
+Requires-Dist: exrex
+Requires-Dist: magicattr
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: sorcery
 
 ## tasksource ![](https://aeiljuispo.cloudimg.io/v7/https://s3.amazonaws.com/moonup/production/uploads/5fc0bcb41160c47d1d43856b/j06-U5e2Tifi2xOnTudqS.jpeg?w=20&h=20&f=face) 500+ curated datasets and preprocessings for instant and interchangeable use
 
@@ -29,7 +38,12 @@ for id in df[df.task_type=="MultipleChoice"].id:
 
 Browse the 500+ curated tasks in tasks.md (200+ MultipleChoice tasks, 200+ Classification tasks), and feel free to request a new task. Datasets are downloaded to $HF_DATASETS_CACHE (like any Hugging Face dataset), so ensure you have more than 100GB of space available.
 
-### Pretrained model:
+You can now also use:
+```python
+load_dataset("tasksource/data", "glue/rte",max_rows=30_000)
+```
+
+### Pretrained models:
 
 Text encoder pretrained on tasksource reached state-of-the-art results: [🤗/deberta-v3-base-tasksource-nli](https://hf.co/sileod/deberta-v3-base-tasksource-nli)
 

{tasksource-0.0.40 → tasksource-0.0.41}/src/tasksource.egg-info/SOURCES.txt

@@ -23,4 +23,5 @@ src/tasksource.egg-info/top_level.txt
 src/tasksource/metadata/__init__.py
 src/tasksource/metadata/bigbench_groups.py
 src/tasksource/metadata/blimp_groups.py
+src/tasksource/metadata/original.txt
 src/tasksource/metadata/popularity.py