SinaTools 0.1.25__tar.gz → 0.1.27__tar.gz

SinaTools-0.1.27/PKG-INFO

@@ -0,0 +1,50 @@
+Metadata-Version: 2.1
+Name: SinaTools
+Version: 0.1.27
+Summary: Open-source Python toolkit for Arabic Natural Understanding, allowing people to integrate it in their system workflow.
+Home-page: https://github.com/SinaLab/sinatools
+License: MIT license
+Description: SinaTools
+        ======================
+        Open Source Toolkit for Arabic NLP and NLU developed by [SinaLab](http://sina.birzeit.edu/) at Birzeit University. SinaTools is available through Python APIs, command lines, colabs, and online demos.
+        
+        See the full list of [Available Packages](https://sina.birzeit.edu/sinatools/), which include: (1) [Morphology Tagging](https://sina.birzeit.edu/sinatools/index.html#morph), (2) [Named Entity Recognition (NER)](https://sina.birzeit.edu/sinatools/index.html#ner), (3) [Word Sense Disambiguation (WSD)](https://sina.birzeit.edu/sinatools/index.html#wsd), (4) [Semantic Relatedness](https://sina.birzeit.edu/sinatools/index.html#sr), (5) [Synonymy Extraction and Evaluation](https://sina.birzeit.edu/sinatools/index.html#se), (6) [Relation Extraction](https://sina.birzeit.edu/sinatools/index.html), (7) [Utilities](https://sina.birzeit.edu/sinatools/index.html#u) (diacritic-based word matching, Jaccard similarly, parser, tokenizers, corpora processing, transliteration, etc).
+        
+        See [Demo Pages](https://sina.birzeit.edu/sinatools/).
+        
+        See the [benchmarking](https://www.jarrar.info/publications/HJK24.pdf), which shows that SinaTools outperformed all related toolkits. 
+        
+        Installation 
+        --------
+        To install SinaTools, ensure you are using Python version 3.10.8, then clone the [GitHub](git://github.com/SinaLab/SinaTools) repository.
+        
+        Alternatively, you can execute the following command:
+        
+        ```bash
+        pip install sinatools
+        ```
+        
+        Installing Models and Data Files
+        --------
+        Some modules in SinaTools require some data files and fine-tuned models to be downloaded. To download these models, please consult the [DataDownload](https://sina.birzeit.edu/sinatools/documentation/cli_tools/DataDownload/DataDownload.html).
+        
+        Documentation
+        --------
+        For information, please refer to the [main page](https://sina.birzeit.edu/sinatools) or the [online domuementation](https://sina.birzeit.edu/sinatools/documentation). 
+        
+        Citation
+        -------
+        Tymaa Hammouda, Mustafa Jarrar, Mohammed Khalilia: [SinaTools: Open Source Toolkit for Arabic Natural Language Understanding](http://www.jarrar.info/publications/HJK24.pdf). In Proceedings of the 2024 AI in Computational Linguistics (ACLing 2024), Procedia Computer Science, Dubai. ELSEVIER.
+        
+        License
+        --------
+        SinaTools is available under the MIT License. See the [LICENSE](https://github.com/SinaLab/sinatools/blob/main/LICENSE) file for more information.
+        
+        Reporting Issues
+        --------
+        To report any issues or bugs, please contact us at "sina.institute.bzu@gmail.com" or visit [SinaTools Issues](https://github.com/SinaLab/sinatools/issues).
+        
+        
+Keywords: sinatools
+Platform: UNKNOWN
+Description-Content-Type: text/markdown

SinaTools-0.1.27/README.rst

@@ -0,0 +1,39 @@
+SinaTools
+======================
+Open Source Toolkit for Arabic NLP and NLU developed by [SinaLab](http://sina.birzeit.edu/) at Birzeit University. SinaTools is available through Python APIs, command lines, colabs, and online demos.
+
+See the full list of [Available Packages](https://sina.birzeit.edu/sinatools/), which include: (1) [Morphology Tagging](https://sina.birzeit.edu/sinatools/index.html#morph), (2) [Named Entity Recognition (NER)](https://sina.birzeit.edu/sinatools/index.html#ner), (3) [Word Sense Disambiguation (WSD)](https://sina.birzeit.edu/sinatools/index.html#wsd), (4) [Semantic Relatedness](https://sina.birzeit.edu/sinatools/index.html#sr), (5) [Synonymy Extraction and Evaluation](https://sina.birzeit.edu/sinatools/index.html#se), (6) [Relation Extraction](https://sina.birzeit.edu/sinatools/index.html), (7) [Utilities](https://sina.birzeit.edu/sinatools/index.html#u) (diacritic-based word matching, Jaccard similarly, parser, tokenizers, corpora processing, transliteration, etc).
+
+See [Demo Pages](https://sina.birzeit.edu/sinatools/).
+
+See the [benchmarking](https://www.jarrar.info/publications/HJK24.pdf), which shows that SinaTools outperformed all related toolkits. 
+
+Installation 
+--------
+To install SinaTools, ensure you are using Python version 3.10.8, then clone the [GitHub](git://github.com/SinaLab/SinaTools) repository.
+
+Alternatively, you can execute the following command:
+
+```bash
+pip install sinatools
+```
+
+Installing Models and Data Files
+--------
+Some modules in SinaTools require some data files and fine-tuned models to be downloaded. To download these models, please consult the [DataDownload](https://sina.birzeit.edu/sinatools/documentation/cli_tools/DataDownload/DataDownload.html).
+
+Documentation
+--------
+For information, please refer to the [main page](https://sina.birzeit.edu/sinatools) or the [online domuementation](https://sina.birzeit.edu/sinatools/documentation). 
+
+Citation
+-------
+Tymaa Hammouda, Mustafa Jarrar, Mohammed Khalilia: [SinaTools: Open Source Toolkit for Arabic Natural Language Understanding](http://www.jarrar.info/publications/HJK24.pdf). In Proceedings of the 2024 AI in Computational Linguistics (ACLing 2024), Procedia Computer Science, Dubai. ELSEVIER.
+
+License
+--------
+SinaTools is available under the MIT License. See the [LICENSE](https://github.com/SinaLab/sinatools/blob/main/LICENSE) file for more information.
+
+Reporting Issues
+--------
+To report any issues or bugs, please contact us at "sina.institute.bzu@gmail.com" or visit [SinaTools Issues](https://github.com/SinaLab/sinatools/issues).

SinaTools-0.1.27/SinaTools.egg-info/PKG-INFO

@@ -0,0 +1,50 @@
+Metadata-Version: 2.1
+Name: SinaTools
+Version: 0.1.27
+Summary: Open-source Python toolkit for Arabic Natural Understanding, allowing people to integrate it in their system workflow.
+Home-page: https://github.com/SinaLab/sinatools
+License: MIT license
+Description: SinaTools
+        ======================
+        Open Source Toolkit for Arabic NLP and NLU developed by [SinaLab](http://sina.birzeit.edu/) at Birzeit University. SinaTools is available through Python APIs, command lines, colabs, and online demos.
+        
+        See the full list of [Available Packages](https://sina.birzeit.edu/sinatools/), which include: (1) [Morphology Tagging](https://sina.birzeit.edu/sinatools/index.html#morph), (2) [Named Entity Recognition (NER)](https://sina.birzeit.edu/sinatools/index.html#ner), (3) [Word Sense Disambiguation (WSD)](https://sina.birzeit.edu/sinatools/index.html#wsd), (4) [Semantic Relatedness](https://sina.birzeit.edu/sinatools/index.html#sr), (5) [Synonymy Extraction and Evaluation](https://sina.birzeit.edu/sinatools/index.html#se), (6) [Relation Extraction](https://sina.birzeit.edu/sinatools/index.html), (7) [Utilities](https://sina.birzeit.edu/sinatools/index.html#u) (diacritic-based word matching, Jaccard similarly, parser, tokenizers, corpora processing, transliteration, etc).
+        
+        See [Demo Pages](https://sina.birzeit.edu/sinatools/).
+        
+        See the [benchmarking](https://www.jarrar.info/publications/HJK24.pdf), which shows that SinaTools outperformed all related toolkits. 
+        
+        Installation 
+        --------
+        To install SinaTools, ensure you are using Python version 3.10.8, then clone the [GitHub](git://github.com/SinaLab/SinaTools) repository.
+        
+        Alternatively, you can execute the following command:
+        
+        ```bash
+        pip install sinatools
+        ```
+        
+        Installing Models and Data Files
+        --------
+        Some modules in SinaTools require some data files and fine-tuned models to be downloaded. To download these models, please consult the [DataDownload](https://sina.birzeit.edu/sinatools/documentation/cli_tools/DataDownload/DataDownload.html).
+        
+        Documentation
+        --------
+        For information, please refer to the [main page](https://sina.birzeit.edu/sinatools) or the [online domuementation](https://sina.birzeit.edu/sinatools/documentation). 
+        
+        Citation
+        -------
+        Tymaa Hammouda, Mustafa Jarrar, Mohammed Khalilia: [SinaTools: Open Source Toolkit for Arabic Natural Language Understanding](http://www.jarrar.info/publications/HJK24.pdf). In Proceedings of the 2024 AI in Computational Linguistics (ACLing 2024), Procedia Computer Science, Dubai. ELSEVIER.
+        
+        License
+        --------
+        SinaTools is available under the MIT License. See the [LICENSE](https://github.com/SinaLab/sinatools/blob/main/LICENSE) file for more information.
+        
+        Reporting Issues
+        --------
+        To report any issues or bugs, please contact us at "sina.institute.bzu@gmail.com" or visit [SinaTools Issues](https://github.com/SinaLab/sinatools/issues).
+        
+        
+Keywords: sinatools
+Platform: UNKNOWN
+Description-Content-Type: text/markdown

{SinaTools-0.1.25 → SinaTools-0.1.27}/SinaTools.egg-info/SOURCES.txt

@@ -155,6 +155,7 @@ sinatools/ner/datasets.py
 sinatools/ner/entity_extractor.py
 sinatools/ner/helpers.py
 sinatools/ner/metrics.py
+sinatools/ner/relation_extractor.py
 sinatools/ner/transforms.py
 sinatools/ner/data/__init__.py
 sinatools/ner/data/datasets.py
@@ -174,9 +175,9 @@ sinatools/synonyms/synonyms_generator.py
 sinatools/utils/__init__.py
 sinatools/utils/charsets.py
 sinatools/utils/implication.py
-sinatools/utils/jaccard.py
 sinatools/utils/parser.py
 sinatools/utils/readfile.py
+sinatools/utils/similarity.py
 sinatools/utils/text_dublication_detector.py
 sinatools/utils/text_transliteration.py
 sinatools/utils/tokenizer.py

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/CLI/DataDownload/download_files.py

@@ -40,7 +40,7 @@ from sinatools.DataDownload.downloader import urls
 
 def main():
     parser = argparse.ArgumentParser(description="Download files from specified URLs.")
-    parser.add_argument('-f', '--files', nargs="*", choices=urls.keys(),
+    parser.add_argument('-f', '--files', nargs="*",
                         help="Names of the files to download. Available files are: "
                              f"{', '.join(urls.keys())}. If no file is specified, all files will be downloaded.")
     
@@ -50,8 +50,23 @@ def main():
 
     if args.files:
         for file in args.files:
-            url = urls[file]
-            download_file(url)
+            print("file: ", file)
+            if file == "wsd":
+                download_file(urls["morph"])
+                download_file(urls["ner"])
+                download_file(urls["wsd_model"])
+                download_file(urls["wsd_tokenizer"])
+                download_file(urls["one_gram"])
+                download_file(urls["five_grams"])
+                download_file(urls["four_grams"])
+                download_file(urls["three_grams"])
+                download_file(urls["two_grams"])
+            elif file == "synonyms":
+                download_file(urls["graph_l2"])
+                download_file(urls["graph_l3"])
+            else:
+               url = urls[file]
+               download_file(url)
     else:
         download_files()
 

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/CLI/ner/corpus_entity_extractor.py

@@ -20,7 +20,7 @@ def jsons_to_list_of_lists(json_list):
     return [[d['token'], d['tags']] for d in json_list]
 
 def combine_tags(sentence):
-    output = jsons_to_list_of_lists(extract(sentence))
+    output = jsons_to_list_of_lists(extract(sentence, "nested"))
     return [word[1] for word in output]
 
 

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/CLI/ner/entity_extractor.py

@@ -46,7 +46,7 @@ def jsons_to_list_of_lists(json_list):
     return [[d['token'], d['tags']] for d in json_list]
 
 def combine_tags(sentence):
-    output = jsons_to_list_of_lists(extract(sentence))
+    output = jsons_to_list_of_lists(extract(sentence, "nested"))
     return [word[1] for word in output]
 
 

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/CLI/utils/jaccard.py

@@ -46,7 +46,7 @@ Examples:
 """
 
 import argparse
-from sinatools.utils.jaccard import jaccard
+from sinatools.utils.similarity import get_jaccard
 from sinatools.utils.readfile import read_file
 
 
@@ -76,7 +76,7 @@ def main():
         print("Either --file1 and --file2 arguments or both --set1 and --set2 arguments must be provided.")
         return
 
-    similarity = jaccard(args.delimiter, set1, set2, args.selection, args.ignoreAllDiacriticsButNotShadda, args.ignoreShaddaDiacritic)
+    similarity = get_jaccard(args.delimiter, set1, set2, args.selection, args.ignoreAllDiacriticsButNotShadda, args.ignoreShaddaDiacritic)
     
     print("Jaccard Result:", similarity)
 

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/DataDownload/downloader.py

@@ -10,13 +10,13 @@ urls = {
     'ner': 'https://sina.birzeit.edu/Wj27012000.tar.gz',
     'wsd_model': 'https://sina.birzeit.edu/bert-base-arabertv02_22_May_2021_00h_allglosses_unused01.zip',
     'wsd_tokenizer': 'https://sina.birzeit.edu/bert-base-arabertv02.zip',
-    'glosses_dic': 'https://sina.birzeit.edu/glosses_dic.pickle',
+    'one_gram': 'https://sina.birzeit.edu/one_gram.pickle',
     'five_grams': 'https://sina.birzeit.edu/five_grams.pickle',
     'four_grams':'https://sina.birzeit.edu/four_grams.pickle',
     'three_grams':'https://sina.birzeit.edu/three_grams.pickle',
     'two_grams':'https://sina.birzeit.edu/two_grams.pickle',
-    'synonyms_level2':'https://sina.birzeit.edu/synonyms_level2.pkl',
-    'synonyms_level3':'https://sina.birzeit.edu/synonyms_level3.pkl'
+    'graph_l2':'https://sina.birzeit.edu/graph_l2.pkl',
+    'graph_l3':'https://sina.birzeit.edu/graph_l3.pkl'
 }
 
 def get_appdatadir():

SinaTools-0.1.27/sinatools/VERSION

@@ -0,0 +1 @@
+0.1.27

{SinaTools-0.1.25 → SinaTools-0.1.27}/sinatools/ner/__init__.py

@@ -7,6 +7,8 @@ import torch
 import pickle
 import json
 from argparse import Namespace
+from transformers import pipeline
+#from transformers import AutoModelForSequenceClassification
 
 tagger = None
 tag_vocab = None
@@ -35,4 +37,7 @@ if torch.cuda.is_available():
 
 train_config.trainer_config["kwargs"]["model"] = model
 tagger = load_object(train_config.trainer_config["fn"], train_config.trainer_config["kwargs"])
-tagger.load(os.path.join(model_path,"checkpoints"))
+tagger.load(os.path.join(model_path,"checkpoints"))
+
+pipe = pipeline("sentiment-analysis", model= os.path.join(path, "best_model"), return_all_scores =True, max_length=128, truncation=True)
+#pipe = AutoModelForSequenceClassification.from_pretrained(os.path.join(path, "best_model"))

SinaTools-0.1.27/sinatools/ner/entity_extractor.py

@@ -0,0 +1,63 @@
+import os
+from collections import namedtuple
+from sinatools.ner.data_format import get_dataloaders, text2segments
+from . import tagger, tag_vocab, train_config
+
+
+def convert_nested_to_flat(nested_tags):
+    flat_tags = []
+    
+    for entry in nested_tags:
+        word = entry['token']
+        tags = entry['tags'].split()
+        
+        # Initialize with the first tag in the sequence
+        flat_tag = tags[0]
+        
+        for tag in tags[1:]:
+            # Check if the tag is an "I-" tag, indicating continuation of an entity
+            if tag.startswith('I-'):
+                flat_tag = tag
+                break
+        
+        flat_tags.append({
+            'token': word,
+            'tags': flat_tag
+        })
+    
+    return flat_tags
+
+def extract(text, ner_method="nested"):
+    
+    dataset, token_vocab = text2segments(text)
+
+    vocabs = namedtuple("Vocab", ["tags", "tokens"])
+    vocab = vocabs(tokens=token_vocab, tags=tag_vocab)
+
+    dataloader = get_dataloaders(
+        (dataset,),
+        vocab,
+        train_config.data_config,
+        batch_size=32,
+        shuffle=(False,),
+    )[0]
+
+
+    segments = tagger.infer(dataloader)
+    segments_lists = []
+    
+    for segment in segments:
+        for token in segment:
+            segments_list = {}
+            segments_list["token"] = token.text
+            list_of_tags = [t['tag'] for t in token.pred_tag]
+            list_of_tags = [i for i in list_of_tags if i not in('O',' ','')]
+            if list_of_tags == []:
+               segments_list["tags"] = ' '.join(['O'])
+            else:
+               segments_list["tags"] = ' '.join(list_of_tags)
+            segments_lists.append(segments_list)  
+    
+    if ner_method == "flat":
+      segments_lists = convert_nested_to_flat(segments_lists)          
+    return segments_lists

SinaTools-0.1.27/sinatools/ner/relation_extractor.py

@@ -0,0 +1,201 @@
+import torch
+import json
+from urllib.request import Request, urlopen
+from sinatools.ner.entity_extractor import extract
+from . import pipe
+
+
+# ============================ Extract entities and their types ========================
+def jsons_to_list_of_lists(json_list):
+    return [[d['token'], d['tags']] for d in json_list]
+
+def entities_and_types(sentence):
+    output_list = jsons_to_list_of_lists(extract(sentence))
+    json_short = distill_entities(output_list)
+
+    entities = {}
+    for entity in json_short:
+        name = entity[0]
+        entity_type = entity[1]
+        entities[name] = entity_type
+
+    return entities
+
+def distill_entities(entities):
+    # This is list that we put the output what we need
+    list_output = list()
+
+    # This line go to sort function and save the output to temp_entities
+    temp_entities = sortTags(entities)
+
+    # This list help us to make the output,
+    temp_list = list()
+
+    # initlize the temp_list
+    temp_list.append(["", "", 0, 0])
+    word_position = 0
+
+    # For each entity, convert ibo to distllir list.
+    for entity in temp_entities:
+        # This is counter tag of this entity
+        counter_tag = 0
+        # For each tag
+        for tag in str(entity[1]).split():
+            # If the counter tag greater than or equal to lenght of templist, if yes then we will append the empty value in templist
+            if counter_tag >= len(temp_list):
+                temp_list.append(["", "", 0, 0])
+
+            # If tag equal O and word postion of this tag is not equal zero then it will add all
+            # not empty eliment of temp list in output list
+            if "O" == tag and word_position != 0:
+                for j in range(0, len(temp_list)):
+                    if temp_list[j][1] != "":
+                        list_output.append([temp_list[j][0].strip(), temp_list[j][1], temp_list[j][2], temp_list[j][3]])
+                        temp_list[j][0] = ""
+                        temp_list[j][1] = ""
+                        temp_list[j][2] = word_position
+                        temp_list[j][3] = word_position
+            # if this tag not equal O, and split by '-' the tag and check the lenght equals two and if the first eliment
+            # of the split its B
+            elif "O" != tag and len(tag.split("-")) == 2 and tag.split("-")[0] == "B":
+                # if the temp_list of counter is not empty then it will append in output list and hten it will
+                # initilize by new string and tag in templist of counter
+                if temp_list[counter_tag][1] != "":
+                    list_output.append([temp_list[counter_tag][0].strip(), temp_list[counter_tag][1], temp_list[counter_tag][2], temp_list[counter_tag][3]])
+                temp_list[counter_tag][0] = str(entity[0]) + " "
+                temp_list[counter_tag][1] = str(tag).split("-")[1]
+                temp_list[counter_tag][2] = word_position
+                temp_list[counter_tag][3] = word_position
+
+            # if this tag not equal O, and split by '-' the tag and check the lenght equals two and if the first eliment
+            # of the split its O
+            elif "O" != tag and len(tag.split("-")) == 2 and tag.split("-")[0] == "I" and word_position != 0:
+                # For each of temp_list, check if in this counter tag of templist is same tag with this.tag
+                # then will complete if not it will save in output list and cheak another
+                for j in range(counter_tag,len(temp_list)):
+                    if temp_list[j][1] == tag[2:] and temp_list[j][3] != word_position:
+                        temp_list[j][0] += str(entity[0]) + " "
+                        temp_list[j][3] += 1
+                        break
+                    else:
+                        if temp_list[j][1] != "":
+                            list_output.append([temp_list[j][0].strip(), temp_list[j][1], temp_list[j][2], temp_list[j][3]])
+                            temp_list[j][0] = ""
+                            temp_list[j][1] = ""
+                            temp_list[j][2] = word_position
+                            temp_list[j][3] = word_position
+            counter_tag += 1
+        word_position += 1
+    # For each temp_list, at the end of the previous loop, there will be some
+    # values in this list, we should save it to the output list
+    for j in range(0, len(temp_list)):
+        if temp_list[j][1] != "":
+            list_output.append([temp_list[j][0].strip(), temp_list[j][1], temp_list[j][2], temp_list[j][3]])
+    return sorted(list_output, key=lambda x: (x[2]))
+
+def sortTags(entities):
+    temp_entities = entities
+    temp_counter = 0
+    # For each entity, this loop will sort each tag of entitiy, first it will check if the
+    # previous tags has same count of this tag, second will sort the tags and check if this tags is correct
+    for entity in temp_entities:
+        tags = entity[1].split()
+        for tag in tags:
+            # if the counter is not 0 then, will complete
+            if temp_counter != 0:
+                # Check if this tag is equal I-, if yes then it will count how many tag in this tags and
+                # count how many tag in previous tags
+                if "I-" == tag[0:2]:
+                    counter_of_this_tag = 0
+                    counter_of_previous_tag = 0
+                    for word in tags:
+                        if tag.split("-")[1] in word:
+                            counter_of_this_tag+=1
+                    for word in temp_entities[temp_counter-1][1].split():
+                        if tag.split("-")[1] in word:
+                            counter_of_previous_tag+=1
+                    # if the counter of previous tag is bigger than counter of this tag, then we
+                    # need to add I-tag in this tags
+                    if counter_of_previous_tag > counter_of_this_tag:
+                        tags.append("I-"+tag.split("-")[1])
+        # Sort the tags
+        tags.sort()
+        # Need to revers the tags because it should begins with I
+        tags.reverse()
+        # If the counter is not 0 then we can complete
+        if temp_counter != 0:
+            this_tags = tags
+            previous_tags = temp_entities[temp_counter - 1][1].split()
+            sorted_tags = list()
+
+            # Check if the this tag is not O and previous tags is not O, then will complete,
+            # if not then it will ignor this tag
+            if "O" not in this_tags and "O" not in previous_tags:
+                index = 0
+                #For each previous tags, need sort this tag by previous tags if its I, B we can ignor
+                for i in previous_tags:
+                    j = 0
+                    while this_tags and j < len(this_tags):
+                        if this_tags[j][0:2] == "I-" and this_tags[j][2:] == i[2:]:
+                            sorted_tags.insert(index, this_tags.pop(j))
+                            break
+                        elif this_tags[j][0:2] == "B-":
+                            break
+                        j += 1
+                    index += 1
+            sorted_tags += this_tags
+            tags = sorted_tags
+        str_tag = " "
+        str_tag = str_tag.join(tags)
+        str_tag = str_tag.strip()
+        temp_entities[temp_counter][1] = str_tag
+        temp_counter += 1
+    return temp_entities
+
+# ============= Prepare Templates and Catergorize Extracted Entities ================
+temp03={'location':'مكان حدوث','agent':'أحد المتأثرين في','happened at':'تاريخ حدوث'}
+categories = {
+    'agent': ['PERS', 'NORP', 'OCC', 'ORG'],
+    'location': ['LOC', 'FAC', 'GPE'],
+    'happened at': ['DATE', 'TIME']
+    }
+
+def get_entity_category(entity_type, categories):
+    for category, types in categories.items():
+        if entity_type in types:
+            return category
+    return None
+
+
+# ============ Extract entities, their types and categorize them ===============
+def relation_extraction(sentence):
+    #test_sentence="صورة إعتقال طفل فلسطيني خلال انتفاضة الأقصى ."
+    entities=entities_and_types(sentence)
+
+    event_indices = [i for i, (_, entity_type) in enumerate(entities.items()) if entity_type == 'EVENT']
+    arg_event_indices = [i for i, (_, entity_type) in enumerate(entities.items()) if entity_type != 'EVENT']
+
+    output_list=[]
+
+    for i in event_indices:
+        event_entity=list(entities.keys())[i]
+        for j in arg_event_indices:
+            arg_name= list(entities.keys())[j]
+            arg_type=entities[arg_name]
+            category = get_entity_category(arg_type, categories)
+            
+            if category in temp03:
+                relation_sentence=f"[CLS] {sentence} [SEP] {event_entity} {temp03[category]} {arg_name}"
+                predicted_relation=pipe(relation_sentence)
+                score = predicted_relation[0][0]['score']  
+                if score > 0.50:
+                    #print(f"Event:{event_entity} Relation:{category} Argument:{arg_name}\n")
+                    #output_list.append([{event_entity} ,{category}, {arg_name}])
+                    output_list.append(f"Event:{event_entity}, Relation:{category}, Argument:{arg_name}")
+
+                else:
+                    #print(f"Event:{event_entity} Relation:No relation Argument:{arg_name}\n")
+                    #output_list.append([{event_entity} ,'No relation', {arg_name}])
+                    output_list.append(f"Event:{event_entity}, Relation:No relation, Argument:{arg_name}")
+    
+    return output_list