DEMNE 1.3.0__py3-none-any.whl

demne/REST_interface/README.md

@@ -0,0 +1,49 @@
+
+![Logo](logo.png)
+
+
+# REST-interface
+
+REST is a jupyter interface that helps the user to assess the application of rules-based method, regarding entities of interest highlighted in a corpus. 
+
+In information extraction tasks, both machine learning and rules are available methods. Although the two processes are well-known and used, there is no obvious way or good practice to decide which one is the most appropriate for each project. 
+
+Using highlights from the BRAT annotation tool, REST is part of an approach to evaluate a rules-based system quickly.
+
+
+
+
+
+
+## Video presentation
+
+An explanation of REST-interface is available here :
+[Video presentation](https://vimeo.com/1005460293?share=copy)
+
+
+## Getting started
+
+### 1. Clone the Repository
+
+First, clone the repository to your local machine:
+```bash
+git clone https://github.com/GuiGuiBazin/REST-interface.git
+```
+### 2. Install Dependencies
+Install the required packages listed in requirements.txt:
+```bash
+pip install -r requirements.txt
+```
+### 3. Download the spaCy Model
+Download the spaCy model required by the project:
+```bash
+python -m spacy download fr_core_news_sm
+```
+
+### Notes: 
+- Spacy model download is absolutly necessary for the current version of REST. More models for other languages will be integrated in later updates. 
+- If you encounter any issues with dependencies, check that your environment matches the versions specified in requirements.txt.
+## Acknowledgements
+
+I would like to thank Emmanuelle Kempf and Xavier Tannier for supervising this project. 
+

demne/REST_interface/REST.ipynb

@@ -0,0 +1,153 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "f3f05c88",
+   "metadata": {},
+   "source": [
+    "# REST Interface\n",
+    "## Introduction\n",
+    "This notebook launches the REST interface, which evaluates a method system based on BRAT annotations.\n",
+    "\n",
+    "### Prerequisites\n",
+    "- Clone the repository:\n",
+    "  ```bash\n",
+    "  git clone https://github.com/GuiGuiBazin/REST-interface.git\n",
+    "  ```\n",
+    "- Install the dependencies:\n",
+    "  ```bash\n",
+    "  pip install -r requirements.txt\n",
+    "  ```\n",
+    "- Download the required spaCy model:\n",
+    "  ```bash\n",
+    "  python -m spacy download fr_core_news_sm\n",
+    "  ```\n",
+    "\n",
+    "This notebook assumes that dependencies are correctly installed and that the modules are accessible in the `REST_modules` folder."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a5a0d25",
+   "metadata": {},
+   "source": [
+    "## I - Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "f0b2d71a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "D:\\1_CLEM\\ESIEE SCHOOL\\PARCOURS RECHERCHE\\Le juste usage des LLM et méthode NLP en cancélorlogie\\DEMNE-Determination-of-Extraction-Methode-for-Named-Entity\\.venv\\Lib\\site-packages\\torch\\cuda\\__init__.py:61: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.\n",
+      "  import pynvml  # type: ignore[import]\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import necessary modules\n",
+    "%load_ext autoreload\n",
+    "%autoreload 2\n",
+    "from REST_modules import *"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3c3e4f99",
+   "metadata": {},
+   "source": [
+    "## II - Launch the REST Interface"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "58776351",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "DEBUG: Running updated calculate_df_metrics\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<style>.widget-dropdown select, .widget-dropdown .widget-readout { font-size: 14px !important; }</style>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "DEBUG: Running updated calculate_df_metrics\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "801d849a820c4a968c50d26499bfd373",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "VBox(children=(HTML(value='', layout=Layout(height='10px')), HTML(value='', layout=Layout(height='10px')), HBo…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Launch the REST interface\n",
+    "launch_rest()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cce1e551",
+   "metadata": {},
+   "source": [
+    "## III - Additional Notes\n",
+    "- If you encounter any issues, verify the dependency versions in the `requirements.txt` file.\n",
+    "- Ensure that the spaCy model is correctly downloaded (see the Introduction section).\n",
+    "- For further questions, refer to the documentation in the GitHub repository's README.md."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.12"
+  },
+  "toc-showmarkdowntxt": false
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

demne/REST_interface/REST_modules/__init__.py

@@ -0,0 +1,5 @@
+from .categorization import *
+from .initialization import *
+from .loading import *
+from .ui import *
+from .visualization import *

demne/REST_interface/REST_modules/calculs/__init__.py

@@ -0,0 +1,8 @@
+from .bootstrap import *
+from .concordancer import *
+from .metrics import *
+from .ngram import *
+from .recommendations import *
+from .regex import *
+from .results import *
+from .tfidf import *

demne/REST_interface/REST_modules/calculs/bootstrap.py

@@ -0,0 +1,140 @@
+import random
+
+import numpy as np
+
+
+def retrieve_bootstrap_data(df, _current_entity, df_metrics_locations):
+    """
+    Retrieve the number of TP, FP and FN present in each file.
+
+    Parameters :
+    df (dataframe) : Dataframe containing the annotated words, their occurrences, places and related entity.
+    current_entity (str) : current working entity selected.
+    df_metrics_locations (dataframe) : contains the locations of each categorized word, and whether they are true positive, false positive or false negative.
+
+    Return :
+    (dict) : dictionnary containing each file and their associated TP, FP and FN.
+    """
+    bootstrap_data = {}
+    # Retrieve of TP, TP(corr) and FP
+    for file in df_metrics_locations["file"].unique():
+        tp = df_metrics_locations[
+            (df_metrics_locations["file"] == file) & (df_metrics_locations["result"] == "TP")
+        ].shape[0]
+        tpcorr = df_metrics_locations[
+            (df_metrics_locations["file"] == file) & (df_metrics_locations["result"] == "TP(corr)")
+        ].shape[0]
+        fp = df_metrics_locations[
+            (df_metrics_locations["file"] == file) & (df_metrics_locations["result"] == "FP")
+        ].shape[0]
+        bootstrap_data[file] = {"TP": (tp + tpcorr), "FP": fp, "FN": 0}
+
+    # Retrieve of FN
+    for _index, row in df.iterrows():
+        if "category?" in row["category"]:
+            for place in row["places"]:
+                file = place[0] + ".txt"
+                if file in bootstrap_data:
+                    bootstrap_data[file]["FN"] += 1
+                else:
+                    bootstrap_data[file] = {"TP": 0, "FP": 0, "FN": 1}
+
+    return bootstrap_data
+
+
+def calculate_metrics(tp, fp, fn):
+    """
+    Calculate the precision, recall and f1 from the metrics.
+
+    Parameters :
+    TP (int) : number of true positive (TP).
+    FP (int) : number of false positive (FP).
+    FN (int) : number of false negative (FN).
+
+    Returns :
+    (list) : list containg the calculated precision, recall and f1 values.
+
+    """
+    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
+    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
+    f1 = (2 * precision * recall) / (precision + recall) if (precision + recall) else 0
+    return precision, recall, f1
+
+
+def estimate_confidence_intervals_bootstrap(
+    df, current_entity, df_metrics_locations, draw_number=1000, alpha=5.0
+):
+    """
+    Calculate the confidence intervals for each entity's metrics (precision and recall) by bootstrapping on the files.
+
+    Parameters :
+    df (dataframe) : Dataframe containing the annotated words, their occurrences, places and related entity.
+    current_entity (str) : current working entity selected.
+    df_metrics_locations (dataframe) : contains the locations of each categorized word, and whether they are true positive, false positive or false negative.
+    draw_number (int) : number of times that we draw files.
+    alpha (int) : represents the percentage of the distribution that falls outside the confidence interval, with a default value of 5.0
+
+    Return :
+    bootstrap_results (dict) : dictionnary of the confidence intervals of the metrics (precision and recall) calculated by bootstrap
+    """
+
+    bootstrap_data = retrieve_bootstrap_data(df, current_entity, df_metrics_locations)
+    files = list(bootstrap_data.keys())
+    precisions = []
+    recalls = []
+    f1s = []
+
+    for _ in range(draw_number):
+        sample_files = random.choices(files, k=len(files))
+        total_tp = 0
+        total_fp = 0
+        total_fn = 0
+
+        for file in sample_files:
+            total_tp += bootstrap_data[file]["TP"]
+            total_fp += bootstrap_data[file]["FP"]
+            total_fn += bootstrap_data[file]["FN"]
+
+        precision, recall, f1 = calculate_metrics(total_tp, total_fp, total_fn)
+        precisions.append(precision)
+        recalls.append(recall)
+        f1s.append(f1)
+
+    lower_p = alpha / 2
+    upper_p = 100 - (alpha / 2)
+
+    precision_conf_interval = (
+        np.percentile(precisions, lower_p),
+        np.percentile(precisions, upper_p),
+    )
+    precision_mean = np.mean(precisions)
+    precision_median = np.median(precisions)
+    recall_conf_interval = (
+        np.percentile(recalls, lower_p),
+        np.percentile(recalls, upper_p),
+    )
+    recall_mean = np.mean(recalls)
+    recall_median = np.median(recalls)
+    f1_conf_interval = (np.percentile(f1s, lower_p), np.percentile(f1s, upper_p))
+    f1_mean = np.mean(f1s)
+    f1_median = np.median(f1s)
+
+    bootstrap_results = {
+        "precision": {
+            "precision_conf_interval": precision_conf_interval,
+            "mean": precision_mean,
+            "median": precision_median,
+        },
+        "recall": {
+            "recall_conf_interval": recall_conf_interval,
+            "mean": recall_mean,
+            "median": recall_median,
+        },
+        "f1": {
+            "f1_conf_interval": f1_conf_interval,
+            "mean": f1_mean,
+            "median": f1_median,
+        },
+    }
+
+    return bootstrap_results

demne/REST_interface/REST_modules/calculs/concordancer.py

@@ -0,0 +1,138 @@
+import os
+import re
+
+import pandas as pd
+from ipydatagrid import DataGrid, Expr, TextRenderer
+
+
+def get_annotations(docs):
+    """
+    Retrieve the annotations from the different files.
+
+    Parameters :
+    docs (generator) : contains all the informations extracted from the .ann files.
+
+    Return :
+    (list) : contains the informations related to each annotation (text, place, document,entity).
+    """
+    annotations = []
+    for doc in docs:
+        for ent in doc["entities"]:
+            annotations.append(
+                (
+                    doc["num_ann"] + ".txt",
+                    ent["label"],
+                    ent["text"],
+                    ent["fragments"][0]["begin"],
+                    ent["fragments"][0]["end"],
+                )
+            )
+    return annotations
+
+
+def get_matches(path, pattern):
+    """
+    Check the occurrences of a pattern in all the .txt files from the given path.
+
+    Parameters :
+    path (string) : path of the current working directory.
+    pattern (string) : word(s) that are searched in the texts.
+
+    Return :
+    (list) : contains the sentences in which the pattern is present, as well as the filename and the place in the text.
+    """
+    matches = []
+    for file_name in os.listdir(path):
+        if file_name.endswith(".txt"):
+            with open(os.path.join(path, file_name), newline="", encoding="utf-8") as file:
+                text = file.read().lower()
+                for match in re.finditer(pattern, text):
+                    place_start = match.start()
+                    place_end = match.end()
+                    sentence = "..." + text[match.start() - 70 : match.end() + 60] + "..."
+                    matches.append((pattern, sentence, file_name, place_start, place_end))
+    return matches
+
+
+def cut_sentence(word, sentence):
+    """
+    Split the sentence in 3 parts, before and after the word of interest.
+
+    Parameters :
+    word (string) : word where the sentence needs to be split.
+    sentence (string) : sentence to split in 3 parts.
+
+    Return :
+    (strings) : the text before the word of interest, the word of interest, and the text after the word of interest.
+    """
+    cut = sentence.lower().split(word)
+    words_before = cut[0][-70:].rjust(70)
+    if len(cut[0]) > 70:
+        words_before = "..." + words_before
+    try:
+        words_after = cut[1].strip()
+    except IndexError:
+        words_after = ""
+    return words_before, word, words_after
+
+
+def background_color_concordancer(cell):
+    if cell.value != "Not annotated":
+        return "#006400"
+    else:
+        return "#db8519"
+
+
+def calculate_concordancer(pattern, current_entity, path, docs):
+    """
+    Creates a concordancer (datagrid) where the occurrences of a word in the text files are searched, and check if these occurrences corresponds to existing annotations.
+
+    Parameters :
+    pattern (string) : word(s) that are searched in the texts.
+    current_entity (string) : the current selected entity.
+    path (string) : path of the current working directory.
+    docs (generator) : contains all the informations extracted from the .ann files.
+
+    Return :
+    (datagrid) : datagrid containing the result of the concordancer.
+    """
+
+    matches = get_matches(path, pattern)
+    annotations = get_annotations(docs)
+
+    res_concordancier = []
+    for match in matches:
+        res = cut_sentence(match[0], match[1])
+        in_no_annotation = True
+        for annotation in annotations:
+            if match[2] == annotation[0]:
+                if annotation[3] <= match[3] and match[4] <= annotation[4]:
+                    if annotation[1].lower() == current_entity:
+                        res_concordancier.append((res[0], res[1], res[2], current_entity))
+                    else:
+                        res_concordancier.append((res[0], res[1], res[2], annotation[1]))
+                    in_no_annotation = False
+                    break
+        if in_no_annotation:
+            res_concordancier.append((res[0], res[1], res[2], "Not annotated"))
+
+    dg_res_concordancer = DataGrid(
+        pd.DataFrame(res_concordancier, columns=["words before", "word", "words after", "entity"]),
+        layout={"height": "350px", "width": "1165px"},
+        base_row_size=25,
+        column_widths={
+            "words before": 440,
+            "word": 110,
+            "words after": 380,
+            "entity": 150,
+        },
+    )
+    renderers = {
+        "words before": TextRenderer(horizontal_alignment="right"),
+        "word": TextRenderer(horizontal_alignment="center"),
+        "entity": TextRenderer(background_color=Expr(background_color_concordancer)),
+    }
+
+    dg_res_concordancer.renderers = renderers
+
+    return dg_res_concordancer