llm-ie 0.2.1__py3-none-any.whl → 0.3.0__py3-none-any.whl

llm_ie/asset/PromptEditor_prompts/chat.txt

@@ -0,0 +1,5 @@
+# Task description
+Chat with the user following the prompt guideline below. 
+
+# Prompt guideline
+{{prompt_guideline}}

llm_ie/asset/PromptEditor_prompts/rewrite.txt

@@ -1,4 +1,6 @@
-This is a prompt rewriting task. Rewrite the draft prompt following the prompt guideline below. DO NOT explain your answer. 
+# Task description
+Rewrite the draft prompt following the prompt guideline below. 
+DO NOT explain your answer. 
 
 # Prompt guideline
 {{prompt_guideline}}

llm_ie/asset/PromptEditor_prompts/system.txt

@@ -0,0 +1 @@
+You are an AI assistant specializing in prompt writing and improvement. Your role is to help users refine, rewrite, and generate effective prompts based on guidelines provided. You are highly knowledgeable in extracting key information and adhering to structured formats. During interactions, you will engage in clear, insightful, and context-aware conversations, providing thoughtful responses to assist the user. Maintain a polite, professional tone and ensure each response adds value to the conversation, promoting clarity and creativity in the user's prompts. If users ask about irrelevant topics (not related to prompt development), you will politely decline to answer and guide the conversation back to prompt development.

llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt

@@ -1,11 +1,27 @@
-Prompt template design:
-    1. Task description
-    2. Schema definition
-    3. Output format definition
-    4. Additional hints
-    5. Input placeholder
+Prompt Template Design:
 
-Example:
+1. Task Description:  
+   Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition).
+
+2. Schema Definition:  
+   List the key concepts that should be extracted, and provide clear definitions for each one.
+
+3. Output Format Definition:  
+   The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). Each dictionary must include a key that holds the entity text. This key can be named "entity_text" or anything else depend on the context. The attributes can either be flat (e.g., {"entity_text": "<entity_text>", "attr1": "<attr1>", "attr2": "<attr2>"}) or nested (e.g., {"entity_text": "<entity_text>", "attributes": {"attr1": "<attr1>", "attr2": "<attr2>"}}).
+
+4. Optional: Hints:  
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. Optional: Examples:  
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
+
+6. Input Placeholder:  
+   The template must include a placeholder in the format {{<placeholder_name>}} for the input text. The placeholder name can be customized as needed.
+
+
+Example 1 (single entity type with attributes):
 
     # Task description
     The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.
@@ -33,3 +49,97 @@ Example:
     Below is the Adverse reactions section:
     {{input}}
 
+
+Example 2 (multiple entity types):
+
+    # Task description
+    This is a named entity recognition task. Given medical note, annotate the Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, and Duration.
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "Drug", "Form", "Strength", "Frequency", "Route", "Dosage", "Reason", "ADE", or "Duration".
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the entity mentions: Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, or Duration:
+        [{"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"},
+        {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"}]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: Acetaminophen 650 mg PO BID 5.
+    Output: [{"entity_text": "Acetaminophen", "entity_type": "Drug"}, {"entity_text": "650 mg", "entity_type": "Strength"}, {"entity_text": "PO", "entity_type": "Route"}, {"entity_text": "BID", "entity_type": "Frequency"}]
+
+    Input: Mesalamine DR 1200 mg PO BID 2.
+    Output: [{"entity_text": "Mesalamine DR", "entity_type": "Drug"}, {"entity_text": "1200 mg", "entity_type": "Strength"}, {"entity_text": "BID", "entity_type": "Frequency"}, {"entity_text": "PO", "entity_type": "Route"}]
+
+
+    # Input placeholder
+    Below is the medical note:
+    "{{input}}"
+
+
+Example 3 (multiple entity types with corresponding attributes):
+
+    # Task description
+    This is a named entity recognition task. Given a medical note, annotate the events (EVENT) and time expressions (TIMEX3):
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "EVENT" or "TIMEX3".
+        if entity_type is "EVENT",
+            "type": the event type as one of the "TEST", "PROBLEM", "TREATMENT", "CLINICAL_DEPT", "EVIDENTIAL", or "OCCURRENCE".
+            "polarity": whether an EVENT is positive ("POS") or negative ("NAG"). For example, in “the patient reports headache, and denies chills”, the EVENT [headache] is positive in its polarity, and the EVENT [chills] is negative in its polarity.
+            "modality": whether an EVENT actually occurred or not. Must be one of the "FACTUAL", "CONDITIONAL", "POSSIBLE", or "PROPOSED".
+
+        if entity_type is "TIMEX3",
+            "type": the type as one of the "DATE", "TIME", "DURATION", or "FREQUENCY".
+            "val": the numeric value 1) DATE: [YYYY]-[MM]-[DD], 2) TIME: [hh]:[mm]:[ss], 3) DURATION: P[n][Y/M/W/D]. So, “for eleven days” will be 
+    represented as “P11D”, meaning a period of 11 days. 4)  R[n][duration], where n denotes the number of repeats. When the n is omitted, the expression denotes an unspecified amount of repeats. For example, “once a day for 3 days” is “R3P1D” (repeat the time interval of 1 day (P1D) for 3 times (R3)), twice every day is “RP12H” (repeat every 12 hours)
+            "mod": additional information regarding the temporal value of a time expression. Must be one of the:
+                “NA”: the default value, no relevant modifier is present;  
+                “MORE”, means “more than”, e.g. over 2 days (val = P2D, mod = MORE);  
+                “LESS”, means “less than”, e.g. almost 2 months (val = P2M, mod=LESS); 
+                “APPROX”, means “approximate”, e.g. nearly a week (val = P1W, mod=APPROX);  
+                “START”, describes the beginning of a period of time, e.g.  Christmas morning, 2005 (val= 2005-12-25, mod= START).  
+                “END”, describes the end of a period of time, e.g. late last year, (val = 2010, mod = END)
+                “MIDDLE”, describes the middle of a period of time, e.g. mid-September 2001 (val = 2001-09, mod = MIDDLE) 
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the EVENT or TIMEX3 entity mentions:
+        [
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "EVENT", "type": "<event type>", "polarity": "<event polarity>", "modality": "<event modality>"},
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "TIMEX3", "type": "<TIMEX3 type>", "val": "<time value>", "mod": "<additional information>"}
+            ...
+        ]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: At 9/7/93 , 1:00 a.m. , intravenous fluids rate was decreased to 50 cc&apos;s per hour , total fluids given during the first 24 hours were 140 to 150 cc&apos;s per kilo per day .
+    Output: [{"entity_text": "intravenous fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "decreased", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "total fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}, 
+            {"entity_text": "9/7/93 , 1:00 a.m.", "entity_type": "TIMEX3", "type": "TIME", "val": "1993-09-07T01:00", "mod": "NA"},
+            {"entity_text": "24 hours", "entity_type": "TIMEX3", "type": "DURATION", "val": "PT24H", "mod": "NA"}]
+
+    Input: At that time it appeared well adhered to the underlying skin .
+    Output: [{"entity_text": "it", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "well adhered", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"}]
+
+
+    # Input placeholder
+    Below is the entire medical note:
+    "{{input}}"

llm_ie/asset/prompt_guide/BinaryRelationExtractor_prompt_guide.txt

@@ -1,9 +1,29 @@
-Prompt template design:
-    1. Task description (mention binary relation extraction and ROI)
-    2. Schema definition (defines relation)
-    3. Output format definition (must use the key "Relation")
-    4. Hints
-    5. Input placeholders (must include "roi_text", "frame_1", and "frame_2" placeholders)
+Prompt Template Design:
+
+1. Task description:
+   Provide a detailed description of the task, including the background and the type of task (e.g., binary relation extraction). Mention the region of interest (ROI) text. 
+2. Schema definition: 
+   List the criterion for relation (True) and for no relation (False).
+
+3. Output format definition:
+   The ouptut must be a dictionary with a key "Relation" (i.e., {"Relation": "<True or False>"}).
+
+4. (optional) Hints:
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. (optional) Examples:
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
+
+6. Entity 1 full information:
+   Include a placeholder in the format {{<frame_1>}}
+
+7. Entity 2 full information:
+   Include a placeholder in the format {{<frame_2>}}
+
+8. Input placeholders:
+   The template must include a placeholder "{{roi_text}}" for the ROI text.
 
 
 Example:
@@ -27,12 +47,12 @@ Example:
         3. If the strength or frequency is for another medication, output False. 
         4. If the strength or frequency is for the same medication but at a different location (span), output False.
 
+    # Entity 1 full information:
+    {{frame_1}}
+
+    # Entity 2 full information:
+    {{frame_2}}
+
     # Input placeholders
     ROI Text with the two entities annotated with <entity_1> and <entity_2>:
     "{{roi_text}}"
-
-    Entity 1 full information:
-    {{frame_1}}
-
-    Entity 2 full information:
-    {{frame_2}}

llm_ie/asset/prompt_guide/MultiClassRelationExtractor_prompt_guide.txt

@@ -1,8 +1,31 @@
-Prompt template design:
-    1. Task description (mention multi-class relation extraction and ROI)
-    2. Schema definition (defines relation types)
-    3. Output format definition (must use the key "RelationType")
-    4. Input placeholders (must include "roi_text", "frame_1", and "frame_2" placeholders)
+Prompt Template Design:
+
+1. Task description:
+   Provide a detailed description of the task, including the background and the type of task (e.g., binary relation extraction). Mention the region of interest (ROI) text. 
+2. Schema definition: 
+   List the criterion for relation (True) and for no relation (False).
+
+3. Output format definition:
+   This section must include a placeholder "{{pos_rel_types}}" for the possible relation types.
+   The ouptut must be a dictionary with a key "RelationType" (i.e., {"RelationType": "<relation type or No Relation>"}).
+
+4. (optional) Hints:
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. (optional) Examples:
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
+
+6. Entity 1 full information:
+   Include a placeholder in the format {{<frame_1>}}
+
+7. Entity 2 full information:
+   Include a placeholder in the format {{<frame_2>}}
+
+8. Input placeholders:
+   The template must include a placeholder "{{roi_text}}" for the ROI text.
+
 
 
 Example:
@@ -35,12 +58,12 @@ Example:
         3. If the strength or frequency is for another medication, output "No Relation". 
         4. If the strength or frequency is for the same medication but at a different location (span), output "No Relation".
 
-    # Input placeholders
-    ROI Text with the two entities annotated with <entity_1> and <entity_2>:
-    "{{roi_text}}"
-
-    Entity 1 full information:
+    # Entity 1 full information:
     {{frame_1}}
 
-    Entity 2 full information:
-    {{frame_2}}
+    # Entity 2 full information:
+    {{frame_2}}
+
+    # Input placeholders
+    ROI Text with the two entities annotated with <entity_1> and <entity_2>:
+    "{{roi_text}}"

llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt

@@ -1,11 +1,27 @@
-Prompt template design:
-    1. Task description
-    2. Schema definition
-    3. Output format definition
-    4. Additional hints
-    5. Input placeholder
+Prompt Template Design:
 
-Example:
+1. Task Description:  
+   Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition).
+
+2. Schema Definition:  
+   List the key concepts that should be extracted, and provide clear definitions for each one.
+
+3. Output Format Definition:  
+   The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). Each dictionary must include a key that holds the entity text. This key can be named "entity_text" or anything else depend on the context. The attributes can either be flat (e.g., {"entity_text": "<entity_text>", "attr1": "<attr1>", "attr2": "<attr2>"}) or nested (e.g., {"entity_text": "<entity_text>", "attributes": {"attr1": "<attr1>", "attr2": "<attr2>"}}).
+
+4. Optional: Hints:  
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. Optional: Examples:  
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
+
+6. Input Placeholder:  
+   The template must include a placeholder in the format {{<placeholder_name>}} for the input text. The placeholder name can be customized as needed.
+
+
+Example 1 (single entity type with attributes):
 
     # Task description
     The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.
@@ -33,3 +49,97 @@ Example:
     Below is the Adverse reactions section:
     {{input}}
 
+
+Example 2 (multiple entity types):
+
+    # Task description
+    This is a named entity recognition task. Given medical note, annotate the Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, and Duration.
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "Drug", "Form", "Strength", "Frequency", "Route", "Dosage", "Reason", "ADE", or "Duration".
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the entity mentions: Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, or Duration:
+        [{"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"},
+        {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"}]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: Acetaminophen 650 mg PO BID 5.
+    Output: [{"entity_text": "Acetaminophen", "entity_type": "Drug"}, {"entity_text": "650 mg", "entity_type": "Strength"}, {"entity_text": "PO", "entity_type": "Route"}, {"entity_text": "BID", "entity_type": "Frequency"}]
+
+    Input: Mesalamine DR 1200 mg PO BID 2.
+    Output: [{"entity_text": "Mesalamine DR", "entity_type": "Drug"}, {"entity_text": "1200 mg", "entity_type": "Strength"}, {"entity_text": "BID", "entity_type": "Frequency"}, {"entity_text": "PO", "entity_type": "Route"}]
+
+
+    # Input placeholder
+    Below is the medical note:
+    "{{input}}"
+
+
+Example 3 (multiple entity types with corresponding attributes):
+
+    # Task description
+    This is a named entity recognition task. Given a medical note, annotate the events (EVENT) and time expressions (TIMEX3):
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "EVENT" or "TIMEX3".
+        if entity_type is "EVENT",
+            "type": the event type as one of the "TEST", "PROBLEM", "TREATMENT", "CLINICAL_DEPT", "EVIDENTIAL", or "OCCURRENCE".
+            "polarity": whether an EVENT is positive ("POS") or negative ("NAG"). For example, in “the patient reports headache, and denies chills”, the EVENT [headache] is positive in its polarity, and the EVENT [chills] is negative in its polarity.
+            "modality": whether an EVENT actually occurred or not. Must be one of the "FACTUAL", "CONDITIONAL", "POSSIBLE", or "PROPOSED".
+
+        if entity_type is "TIMEX3",
+            "type": the type as one of the "DATE", "TIME", "DURATION", or "FREQUENCY".
+            "val": the numeric value 1) DATE: [YYYY]-[MM]-[DD], 2) TIME: [hh]:[mm]:[ss], 3) DURATION: P[n][Y/M/W/D]. So, “for eleven days” will be 
+    represented as “P11D”, meaning a period of 11 days. 4)  R[n][duration], where n denotes the number of repeats. When the n is omitted, the expression denotes an unspecified amount of repeats. For example, “once a day for 3 days” is “R3P1D” (repeat the time interval of 1 day (P1D) for 3 times (R3)), twice every day is “RP12H” (repeat every 12 hours)
+            "mod": additional information regarding the temporal value of a time expression. Must be one of the:
+                “NA”: the default value, no relevant modifier is present;  
+                “MORE”, means “more than”, e.g. over 2 days (val = P2D, mod = MORE);  
+                “LESS”, means “less than”, e.g. almost 2 months (val = P2M, mod=LESS); 
+                “APPROX”, means “approximate”, e.g. nearly a week (val = P1W, mod=APPROX);  
+                “START”, describes the beginning of a period of time, e.g.  Christmas morning, 2005 (val= 2005-12-25, mod= START).  
+                “END”, describes the end of a period of time, e.g. late last year, (val = 2010, mod = END)
+                “MIDDLE”, describes the middle of a period of time, e.g. mid-September 2001 (val = 2001-09, mod = MIDDLE) 
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the EVENT or TIMEX3 entity mentions:
+        [
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "EVENT", "type": "<event type>", "polarity": "<event polarity>", "modality": "<event modality>"},
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "TIMEX3", "type": "<TIMEX3 type>", "val": "<time value>", "mod": "<additional information>"}
+            ...
+        ]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: At 9/7/93 , 1:00 a.m. , intravenous fluids rate was decreased to 50 cc&apos;s per hour , total fluids given during the first 24 hours were 140 to 150 cc&apos;s per kilo per day .
+    Output: [{"entity_text": "intravenous fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "decreased", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "total fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}, 
+            {"entity_text": "9/7/93 , 1:00 a.m.", "entity_type": "TIMEX3", "type": "TIME", "val": "1993-09-07T01:00", "mod": "NA"},
+            {"entity_text": "24 hours", "entity_type": "TIMEX3", "type": "DURATION", "val": "PT24H", "mod": "NA"}]
+
+    Input: At that time it appeared well adhered to the underlying skin .
+    Output: [{"entity_text": "it", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "well adhered", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"}]
+
+
+    # Input placeholder
+    Below is the entire medical note:
+    "{{input}}"

llm_ie/asset/prompt_guide/SentenceFrameExtractor_prompt_guide.txt

@@ -1,40 +1,145 @@
-Prompt template design:
-    1. Task description (mention the task is to extract information from sentences)
-    2. Schema definition
-    3. Output format definition
-    4. Additional hints
-    5. Input placeholder (mention user will feed sentence by sentence)
+Prompt Template Design:
 
-Example:
+1. Task Description:  
+   Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition).
+
+2. Schema Definition:  
+   List the key concepts that should be extracted, and provide clear definitions for each one.
+
+3. Output Format Definition:  
+   The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). Each dictionary must include a key that holds the entity text. This key can be named "entity_text" or anything else depend on the context. The attributes can either be flat (e.g., {"entity_text": "<entity_text>", "attr1": "<attr1>", "attr2": "<attr2>"}) or nested (e.g., {"entity_text": "<entity_text>", "attributes": {"attr1": "<attr1>", "attr2": "<attr2>"}}).
+
+4. Optional: Hints:  
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. Optional: Examples:  
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
+
+6. Input Placeholder:  
+   The template must include a placeholder in the format {{<placeholder_name>}} for the input text. The placeholder name can be customized as needed.
+   
+
+Example 1 (single entity type with attributes):
 
     # Task description
-    The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse Reactions section. Your task is to extract the adverse reactions in a given sentence (provided by user at a time). Note that adverse reactions can be nested under a clinical trial and potentially an arm. Your output should consider that.
+    The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.
 
     # Schema definition
     Your output should contain: 
-        If applicable, "ClinicalTrial" which is the name of the trial, 
+        "ClinicalTrial" which is the name of the trial, 
         If applicable, "Arm" which is the arm within the clinical trial, 
-        Must have, "AdverseReaction" which is the name of the adverse reaction spelled exactly as in the source document,
-        If applicable, "Percentage" which is the occurrence of the adverse reaction within the trial and arm,
+        "AdverseReaction" which is the name of the adverse reaction,
+        If applicable, "Percentage" which is the occurance of the adverse reaction within the trial and arm,
+        "Evidence" which is the EXACT sentence in the text where you found the AdverseReaction from
 
     # Output format definition
-    Your output should follow JSON format, 
-    if there are adverse reaction mentions:
-        [{"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>"},
-        {"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>"}]
-    if there is no adverse reaction in the given sentence, just output an empty list:
-        []
+    Your output should follow JSON format, for example:
+    [
+        {"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"},
+        {"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"} 
+    ]
 
     # Additional hints
     Your output should be 100% based on the provided content. DO NOT output fake numbers. 
-    If there is no specific trial or arm, just omit the "ClinicalTrial" or "Arm" key. If the percentage is not reported, just omit the "Percentage" key. 
-    I am only interested in the content in JSON format. Do NOT generate explanation. 
+    If there is no specific arm, just omit the "Arm" key. If the percentage is not reported, just omit the "Percentage" key. The "Evidence" should always be provided.
+
+    # Input placeholder
+    Below is the Adverse reactions section for your reference. I will feed you with sentences from it one by one. 
+    {{input}}
+
+
+Example 2 (multiple entity types):
+
+    # Task description
+    This is a named entity recognition task. Given a sentence from a medical note, annotate the Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, and Duration.
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "Drug", "Form", "Strength", "Frequency", "Route", "Dosage", "Reason", "ADE", or "Duration".
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the entity mentions: Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, or Duration:
+        [{"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"},
+        {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "<entity type as listed above>"}]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: Acetaminophen 650 mg PO BID 5.
+    Output: [{"entity_text": "Acetaminophen", "entity_type": "Drug"}, {"entity_text": "650 mg", "entity_type": "Strength"}, {"entity_text": "PO", "entity_type": "Route"}, {"entity_text": "BID", "entity_type": "Frequency"}]
+
+    Input: Mesalamine DR 1200 mg PO BID 2.
+    Output: [{"entity_text": "Mesalamine DR", "entity_type": "Drug"}, {"entity_text": "1200 mg", "entity_type": "Strength"}, {"entity_text": "BID", "entity_type": "Frequency"}, {"entity_text": "PO", "entity_type": "Route"}]
+
+
+    # Input placeholder
+    Below is the medical note for your reference. I will feed you with sentences from it one by one.
+    "{{input}}"
+
+
+Example 3 (multiple entity types with corresponding attributes):
+
+    # Task description
+    This is a named entity recognition task. Given a sentence from a medical note, annotate the events (EVENT) and time expressions (TIMEX3):
+
+    # Schema definition
+    Your output should contain: 
+        "entity_text": the exact wording as mentioned in the note.
+        "entity_type": type of the entity. It should be one of the "EVENT" or "TIMEX3".
+        if entity_type is "EVENT",
+            "type": the event type as one of the "TEST", "PROBLEM", "TREATMENT", "CLINICAL_DEPT", "EVIDENTIAL", or "OCCURRENCE".
+            "polarity": whether an EVENT is positive ("POS") or negative ("NAG"). For example, in “the patient reports headache, and denies chills”, the EVENT [headache] is positive in its polarity, and the EVENT [chills] is negative in its polarity.
+            "modality": whether an EVENT actually occurred or not. Must be one of the "FACTUAL", "CONDITIONAL", "POSSIBLE", or "PROPOSED".
+
+        if entity_type is "TIMEX3",
+            "type": the type as one of the "DATE", "TIME", "DURATION", or "FREQUENCY".
+            "val": the numeric value 1) DATE: [YYYY]-[MM]-[DD], 2) TIME: [hh]:[mm]:[ss], 3) DURATION: P[n][Y/M/W/D]. So, “for eleven days” will be 
+    represented as “P11D”, meaning a period of 11 days. 4)  R[n][duration], where n denotes the number of repeats. When the n is omitted, the expression denotes an unspecified amount of repeats. For example, “once a day for 3 days” is “R3P1D” (repeat the time interval of 1 day (P1D) for 3 times (R3)), twice every day is “RP12H” (repeat every 12 hours)
+            "mod": additional information regarding the temporal value of a time expression. Must be one of the:
+                “NA”: the default value, no relevant modifier is present;  
+                “MORE”, means “more than”, e.g. over 2 days (val = P2D, mod = MORE);  
+                “LESS”, means “less than”, e.g. almost 2 months (val = P2M, mod=LESS); 
+                “APPROX”, means “approximate”, e.g. nearly a week (val = P1W, mod=APPROX);  
+                “START”, describes the beginning of a period of time, e.g.  Christmas morning, 2005 (val= 2005-12-25, mod= START).  
+                “END”, describes the end of a period of time, e.g. late last year, (val = 2010, mod = END)
+                “MIDDLE”, describes the middle of a period of time, e.g. mid-September 2001 (val = 2001-09, mod = MIDDLE) 
+
+    # Output format definition
+    Your output should follow JSON format, 
+    if there are one of the EVENT or TIMEX3 entity mentions:
+        [
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "EVENT", "type": "<event type>", "polarity": "<event polarity>", "modality": "<event modality>"},
+            {"entity_text": "<Exact entity mentions as in the note>", "entity_type": "TIMEX3", "type": "<TIMEX3 type>", "val": "<time value>", "mod": "<additional information>"}
+            ...
+        ]
+    if there is no entity mentioned in the given note, just output an empty list:
+        []
+
+    I am only interested in the extracted contents in []. Do NOT explain your answer.
+
+    # Examples
+    Below are some examples:
+
+    Input: At 9/7/93 , 1:00 a.m. , intravenous fluids rate was decreased to 50 cc&apos;s per hour , total fluids given during the first 24 hours were 140 to 150 cc&apos;s per kilo per day .
+    Output: [{"entity_text": "intravenous fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "decreased", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "total fluids", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}, 
+            {"entity_text": "9/7/93 , 1:00 a.m.", "entity_type": "TIMEX3", "type": "TIME", "val": "1993-09-07T01:00", "mod": "NA"},
+            {"entity_text": "24 hours", "entity_type": "TIMEX3", "type": "DURATION", "val": "PT24H", "mod": "NA"}]
+
+    Input: At that time it appeared well adhered to the underlying skin .
+    Output: [{"entity_text": "it", "entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"},
+            {"entity_text": "well adhered", "entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"}]
 
-    The Adverse reactions section often has a sentence in the first paragraph:
-        "The following clinically significant adverse reactions are described elsewhere in the labeling:..." Make sure to extract those adverse reaction mentions. 
-    The Adverse reactions section often has summary sentences like:
-        "The most common adverse reactions were ...". Make sure to extract those adverse reaction mentions. 
 
     # Input placeholder
-    Below is the entire Adverse reactions section for your reference. I will feed you with sentences from it one by one. 
+    Below is the entire medical note for your reference. I will feed you with sentences from it one by one.
     "{{input}}"

llm_ie/data_types.py

@@ -1,4 +1,4 @@
-from typing import List, Dict, Iterable, Callable
+from typing import List, Dict, Tuple, Iterable, Callable
 import importlib.util
 import json
 
@@ -283,15 +283,12 @@ class LLMInformationExtractionDocument:
             json_file.flush()
             
 
-    def serve_viz(self, host: str = '0.0.0.0', port: int = 3000, theme:str = "light", 
-                  color_attr_key:str=None, color_map_func:Callable=None):
+    def _viz_preprocess(self) -> Tuple:
         """
-        This method serves a visualization of the document.
+        This method preprocesses the entities and relations for visualization.
         """
         if importlib.util.find_spec("ie_viz") is None:
             raise ImportError("ie_viz not found. Please install ie_viz (```pip install ie-viz```).")
-        
-        from ie_viz import serve
 
         if self.has_frame():
             entities = [{"entity_id": frame.frame_id, "start": frame.start, "end": frame.end, "attr": frame.attr} for frame in self.frames]
@@ -306,6 +303,31 @@ class LLMInformationExtractionDocument:
         else:
             relations = None
 
+        return entities, relations
+
+
+    def viz_serve(self, host: str = '0.0.0.0', port: int = 5000, theme:str = "light", 
+                  color_attr_key:str=None, color_map_func:Callable=None):
+        """
+        This method serves a visualization App of the document.
+
+        Parameters:
+        -----------
+        host : str, Optional
+            The host address to run the server on.
+        port : int, Optional
+            The port number to run the server on.
+        theme : str, Optional
+            The theme of the visualization. Must be either "light" or "dark".
+        color_attr_key : str, Optional
+            The attribute key to be used for coloring the entities.
+        color_map_func : Callable, Optional
+            The function to be used for mapping the entity attributes to colors. When provided, the color_attr_key and 
+            theme will be overwritten. The function must take an entity dictionary as input and return a color string (hex).
+        """
+        entities, relations = self._viz_preprocess()
+        from ie_viz import serve
+
         serve(text=self.text,
               entities=entities,
               relations=relations,
@@ -314,4 +336,28 @@ class LLMInformationExtractionDocument:
               theme=theme,
               color_attr_key=color_attr_key,
               color_map_func=color_map_func)
-        
+        
+    
+    def viz_render(self, theme:str = "light", color_attr_key:str=None, color_map_func:Callable=None) -> str:
+        """
+        This method renders visualization html of the document.
+
+        Parameters:
+        -----------
+        theme : str, Optional
+            The theme of the visualization. Must be either "light" or "dark".
+        color_attr_key : str, Optional
+            The attribute key to be used for coloring the entities.
+        color_map_func : Callable, Optional
+            The function to be used for mapping the entity attributes to colors. When provided, the color_attr_key and 
+            theme will be overwritten. The function must take an entity dictionary as input and return a color string (hex).
+        """
+        entities, relations = self._viz_preprocess()
+        from ie_viz import render
+
+        return render(text=self.text,
+                      entities=entities,
+                      relations=relations,
+                      theme=theme,
+                      color_attr_key=color_attr_key,
+                      color_map_func=color_map_func)

llm_ie/engines.py

@@ -243,7 +243,7 @@ class OpenAIInferenceEngine(InferenceEngine):
             for chunk in response:
                 if chunk.choices[0].delta.content is not None:
                     res += chunk.choices[0].delta.content
-                    print(chunk.choices[0].delta.content, end="")
+                    print(chunk.choices[0].delta.content, end="", flush=True)
             return res
         
         return response.choices[0].message.content

llm_ie/extractors.py

@@ -73,18 +73,49 @@ class Extractor:
 
         return prompt
     
+    def _find_dict_strings(self, text: str) -> List[str]:
+        """
+        Extracts balanced JSON-like dictionaries from a string, even if nested.
+
+        Parameters:
+        -----------
+        text : str
+            the input text containing JSON-like structures.
+
+        Returns : List[str]
+            A list of valid JSON-like strings representing dictionaries.
+        """
+        open_brace = 0
+        start = -1
+        json_objects = []
+
+        for i, char in enumerate(text):
+            if char == '{':
+                if open_brace == 0:
+                    # start of a new JSON object
+                    start = i 
+                open_brace += 1
+            elif char == '}':
+                open_brace -= 1
+                if open_brace == 0 and start != -1:
+                    json_objects.append(text[start:i + 1])
+                    start = -1
+
+        return json_objects
+    
+    
     def _extract_json(self, gen_text:str) -> List[Dict[str, str]]:
         """ 
         This method inputs a generated text and output a JSON of information tuples
         """
-        pattern = r'\{.*?\}'
         out = []
-        for match in re.findall(pattern, gen_text, re.DOTALL):
+        dict_str_list = self._find_dict_strings(gen_text)
+        for dict_str in dict_str_list:
             try:
-                tup_dict = json.loads(match)
-                out.append(tup_dict)
+                dict_obj = json.loads(dict_str)
+                out.append(dict_obj)
             except json.JSONDecodeError:
-                print(f'Post-processing failed at:\n{match}')
+                print(f'Post-processing failed at:\n{dict_str}')
         return out
     
 

llm_ie/prompt_editor.py

@@ -1,8 +1,15 @@
-import os
+import sys
+from typing import Dict, Union
 import importlib.resources
 from llm_ie.engines import InferenceEngine
 from llm_ie.extractors import FrameExtractor
+import re
+import colorama
+from colorama import Fore, Style
+import ipywidgets as widgets
+from IPython.display import display, HTML
 
+    
 class PromptEditor:
     def __init__(self, inference_engine:InferenceEngine, extractor:FrameExtractor):
         """
@@ -18,16 +25,48 @@ class PromptEditor:
         self.inference_engine = inference_engine
         self.prompt_guide = extractor.get_prompt_guide()
 
+        file_path = importlib.resources.files('llm_ie.asset.PromptEditor_prompts').joinpath('system.txt')
+        with open(file_path, 'r') as f:
+            self.system_prompt =  f.read()
+        
+
+    def _apply_prompt_template(self, text_content:Dict[str,str], prompt_template:str) -> str:
+        """
+        This method applies text_content to prompt_template and returns a prompt.
+
+        Parameters
+        ----------
+        text_content : Dict[str,str]
+            the input text content to put in prompt template. 
+            all the keys must be included in the prompt template placeholder {{<placeholder name>}}.
+
+        Returns : str
+            a prompt.
+        """
+        pattern = re.compile(r'{{(.*?)}}')
+        placeholders = pattern.findall(prompt_template)
+        if len(placeholders) != len(text_content):
+            raise ValueError(f"Expect text_content ({len(text_content)}) and prompt template placeholder ({len(placeholders)}) to have equal size.")
+        if not all([k in placeholders for k, _ in text_content.items()]):
+            raise ValueError(f"All keys in text_content ({text_content.keys()}) must match placeholders in prompt template ({placeholders}).")
+
+        prompt = pattern.sub(lambda match: re.sub(r'\\', r'\\\\', text_content[match.group(1)]), prompt_template)
+
+        return prompt
+    
+
     def rewrite(self, draft:str) -> str:
         """
         This method inputs a prompt draft and rewrites it following the extractor's guideline.
         """
         file_path = importlib.resources.files('llm_ie.asset.PromptEditor_prompts').joinpath('rewrite.txt')
         with open(file_path, 'r') as f:
-            prompt = f.read()
+            rewrite_prompt_template = f.read()
 
-        prompt = prompt.replace("{{draft}}", draft).replace("{{prompt_guideline}}", self.prompt_guide)
-        messages = [{"role": "user", "content": prompt}]
+        prompt = self._apply_prompt_template(text_content={"draft": draft, "prompt_guideline": self.prompt_guide}, 
+                                             prompt_template=rewrite_prompt_template)
+        messages = [{"role": "system", "content": self.system_prompt},
+                    {"role": "user", "content": prompt}]
         res = self.inference_engine.chat(messages, stream=True)
         return res
     
@@ -37,9 +76,114 @@ class PromptEditor:
         """
         file_path = importlib.resources.files('llm_ie.asset.PromptEditor_prompts').joinpath('comment.txt')
         with open(file_path, 'r') as f:
-            prompt = f.read()
+            comment_prompt_template = f.read()
 
-        prompt = prompt.replace("{{draft}}", draft).replace("{{prompt_guideline}}", self.prompt_guide)
-        messages = [{"role": "user", "content": prompt}]
+        prompt = self._apply_prompt_template(text_content={"draft": draft, "prompt_guideline": self.prompt_guide}, 
+                                             prompt_template=comment_prompt_template)
+        messages = [{"role": "system", "content": self.system_prompt},
+                    {"role": "user", "content": prompt}]
         res = self.inference_engine.chat(messages, stream=True)
-        return res
+        return res
+    
+
+    def _terminal_chat(self):
+        """
+        This method runs an interactive chat session in the terminal to help users write prompt templates.
+        """
+        colorama.init(autoreset=True)
+        file_path = importlib.resources.files('llm_ie.asset.PromptEditor_prompts').joinpath('chat.txt')
+        with open(file_path, 'r') as f:
+            chat_prompt_template = f.read()
+
+        prompt = self._apply_prompt_template(text_content={"prompt_guideline": self.prompt_guide}, 
+                                             prompt_template=chat_prompt_template)
+
+        messages = [{"role": "system", "content": self.system_prompt},
+                    {"role": "user", "content": prompt}]
+        
+        print(f'Welcome to the interactive chat! Type "{Fore.RED}exit{Style.RESET_ALL}" or {Fore.YELLOW}control + C{Style.RESET_ALL} to end the conversation.')
+
+        while True:
+            # Get user input
+            user_input = input(f"{Fore.GREEN}\nUser: {Style.RESET_ALL}")
+            
+            # Exit condition
+            if user_input.lower() == 'exit':
+                print(f"{Fore.YELLOW}Interactive chat ended. Goodbye!{Style.RESET_ALL}")
+                break
+            
+            # Chat
+            messages.append({"role": "user", "content": user_input})
+            print(f"{Fore.BLUE}Assistant: {Style.RESET_ALL}", end="")
+            response = self.inference_engine.chat(messages, stream=True)
+            messages.append({"role": "assistant", "content": response})
+            
+
+    def _IPython_chat(self):
+        """
+        This method runs an interactive chat session in Jupyter/IPython using ipywidgets to help users write prompt templates.
+        """
+        # Load the chat prompt template from the resources
+        file_path = importlib.resources.files('llm_ie.asset.PromptEditor_prompts').joinpath('chat.txt')
+        with open(file_path, 'r') as f:
+            chat_prompt_template = f.read()
+
+        # Prepare the initial system message with the prompt guideline
+        prompt = self._apply_prompt_template(text_content={"prompt_guideline": self.prompt_guide}, 
+                                            prompt_template=chat_prompt_template)
+
+        # Initialize conversation messages
+        messages = [{"role": "system", "content": self.system_prompt},
+                    {"role": "user", "content": prompt}]
+
+        # Widgets for user input and chat output
+        input_box = widgets.Text(placeholder="Type your message here...")
+        output_area = widgets.Output()
+
+        # Display initial instructions
+        with output_area:
+            display(HTML('Welcome to the interactive chat! Type "<span style="color: red;">exit</span>" to end the conversation.'))
+
+        def handle_input(sender):
+            user_input = input_box.value
+            input_box.value = ''  # Clear the input box after submission
+
+            # Exit condition
+            if user_input.strip().lower() == 'exit':
+                with output_area:
+                    display(HTML('<p style="color: orange;">Interactive chat ended. Goodbye!</p>'))
+                input_box.disabled = True  # Disable the input box after exiting
+                return
+
+            # Append user message to conversation
+            messages.append({"role": "user", "content": user_input})
+            print(f"User: {user_input}")
+            
+            # Display the user message
+            with output_area:
+                display(HTML(f'<pre><span style="color: green;">User: </span>{user_input}</pre>'))
+
+            # Get assistant's response and append it to conversation
+            print("Assistant: ", end="")
+            response = self.inference_engine.chat(messages, stream=True)
+            messages.append({"role": "assistant", "content": response})
+
+            # Display the assistant's response
+            with output_area:
+                display(HTML(f'<pre><span style="color: blue;">Assistant: </span>{response}</pre>'))
+
+        # Bind the user input to the handle_input function
+        input_box.on_submit(handle_input)
+
+        # Display the input box and output area
+        display(input_box)
+        display(output_area)
+
+    def chat(self):
+        """
+        External method that detects the environment and calls the appropriate chat method.
+        """
+        if 'ipykernel' in sys.modules:
+            self._IPython_chat()
+        else:
+            self._terminal_chat()

{llm_ie-0.2.1.dist-info → llm_ie-0.3.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llm-ie
-Version: 0.2.1
+Version: 0.3.0
 Summary: An LLM-powered tool that transforms everyday language into robust information extraction pipelines.
 License: MIT
 Author: Enshuo (David) Hsu
@@ -20,6 +20,13 @@ Description-Content-Type: text/markdown
 
 An LLM-powered tool that transforms everyday language into robust information extraction pipelines. 
 
+| Features | Support |
+|----------|----------|
+| **LLM Agent for prompt writing** | :white_check_mark:  Interactive chat, Python functions |
+| **Named Entity Recognition (NER)** | :white_check_mark: Document-level, Sentence-level |
+| **Entity Attributes Extraction** | :white_check_mark: Flexible formats |
+| **Relation Extraction (RE)** | :white_check_mark: Binary & Multiclass relations |
+
 ## Table of Contents
 - [Overview](#overview)
 - [Prerequisite](#prerequisite)
@@ -35,12 +42,12 @@ An LLM-powered tool that transforms everyday language into robust information ex
         - [RelationExtractor](#relationextractor)
 
 ## Overview
-LLM-IE is a toolkit that provides robust information extraction utilities for frame-based information extraction. Since prompt design has a significant impact on generative information extraction with LLMs, it also provides a built-in LLM editor to help with prompt writing. The flowchart below demonstrates the workflow starting from a casual language request.
+LLM-IE is a toolkit that provides robust information extraction utilities for named entity, entity attributes, and entity relation extraction. Since prompt design has a significant impact on generative information extraction with LLMs, it has a built-in LLM agent ("editor") to help with prompt writing. The flowchart below demonstrates the workflow starting from a casual language request to output visualization.
 
 <div align="center"><img src="doc_asset/readme_img/LLM-IE flowchart.png" width=800 ></div>
 
 ## Prerequisite
-At least one LLM inference engine is required. There are built-in supports for 🦙 [Llama-cpp-python](https://github.com/abetlen/llama-cpp-python), <img src="https://avatars.githubusercontent.com/u/151674099?s=48&v=4" alt="Icon" width="20"/> [Ollama](https://github.com/ollama/ollama), 🤗 [Huggingface_hub](https://github.com/huggingface/huggingface_hub), <img src=doc_asset/readme_img/openai-logomark.png width=16 /> [OpenAI API](https://platform.openai.com/docs/api-reference/introduction), and <img src=doc_asset/readme_img/vllm-logo.png width=20 /> vLLM. For installation guides, please refer to those projects. Other inference engines can be configured through the [InferenceEngine](src/llm_ie/engines.py) abstract class. See [LLM Inference Engine](#llm-inference-engine) section below.
+At least one LLM inference engine is required. There are built-in supports for 🦙 [Llama-cpp-python](https://github.com/abetlen/llama-cpp-python), <img src="https://avatars.githubusercontent.com/u/151674099?s=48&v=4" alt="Icon" width="20"/> [Ollama](https://github.com/ollama/ollama), 🤗 [Huggingface_hub](https://github.com/huggingface/huggingface_hub), <img src=doc_asset/readme_img/openai-logomark.png width=16 /> [OpenAI API](https://platform.openai.com/docs/api-reference/introduction), and <img src=doc_asset/readme_img/vllm-logo.png width=20 /> [vLLM](https://github.com/vllm-project/vllm). For installation guides, please refer to those projects. Other inference engines can be configured through the [InferenceEngine](src/llm_ie/engines.py) abstract class. See [LLM Inference Engine](#llm-inference-engine) section below.
 
 ## Installation
 The Python package is available on PyPI. 
@@ -125,21 +132,26 @@ We start with a casual description:
 
 *"Extract diagnosis from the clinical note. Make sure to include diagnosis date and status."* 
 
-The ```PromptEditor``` rewrites it following the schema required by the ```BasicFrameExtractor```. 
-
-```python 
+Define the AI prompt editor.
+```python
+from llm_ie.engines import OllamaInferenceEngine
 from llm_ie.extractors import BasicFrameExtractor
 from llm_ie.prompt_editor import PromptEditor
 
-# Describe the task in casual language
-prompt_draft = "Extract diagnosis from the clinical note. Make sure to include diagnosis date and status."
-
-# Use LLM editor to generate a formal prompt template with standard extraction schema
+# Define a LLM inference engine
+llm = OllamaInferenceEngine(model_name="llama3.1:8b-instruct-q8_0")
+# Define LLM prompt editor
 editor = PromptEditor(llm, BasicFrameExtractor)
-prompt_template = editor.rewrite(prompt_draft)
+# Start chat
+editor.chat()
 ```
 
-The editor generates a prompt template as below:
+This opens an interactive session:
+<div align="left"><img src=doc_asset/readme_img/terminal_chat.PNG width=1000 ></div>
+
+
+The ```PromptEditor``` drafts a prompt template following the schema required by the ```BasicFrameExtractor```:
+
 ```
 # Task description
 The paragraph below contains a clinical note with diagnoses listed. Please carefully review it and extract the diagnoses, including the diagnosis date and status.
@@ -165,6 +177,8 @@ If there is no specific date or status, just omit those keys.
 Below is the clinical note:
 {{input}}
 ```
+
+
 #### Information extraction pipeline
 Now we apply the prompt template to build an information extraction pipeline.
 
@@ -202,15 +216,33 @@ from llm_ie.data_types import LLMInformationExtractionDocument
 doc = LLMInformationExtractionDocument(doc_id="Synthesized medical note",
                                        text=note_text)
 # Add frames to a document
-for frame in frames:
-    doc.add_frame(frame, valid_mode="span", create_id=True)
+doc.add_frames(frames, create_id=True)
 
 # Save document to file (.llmie)
 doc.save("<your filename>.llmie")
 ```
 
+To visualize the extracted frames, we use the ```viz_serve()``` method. 
+```python
+doc.viz_serve()
+```
+A Flask APP starts at port 5000 (default).
+```
+* Serving Flask app 'ie_viz.utilities'
+* Debug mode: off
+WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
+ * Running on all addresses (0.0.0.0)
+ * Running on http://127.0.0.1:5000
+Press CTRL+C to quit
+127.0.0.1 - - [03/Oct/2024 23:36:22] "GET / HTTP/1.1" 200 -
+```
+
+<div align="left"><img src="doc_asset/readme_img/llm-ie_demo.PNG" width=1000 ></div>
+
+
 ## Examples
-  - [Write prompt templates with AI editors](demo/prompt_template_writing.ipynb)
+  - [Interactive chat with LLM prompt editors](demo/prompt_template_writing_via_chat.ipynb)
+  - [Write prompt templates with LLM prompt editors](demo/prompt_template_writing.ipynb)
   - [NER + RE for Drug, Strength, Frequency](demo/medication_relation_extraction.ipynb)
 
 ## User Guide
@@ -435,7 +467,30 @@ print(BasicFrameExtractor.get_prompt_guide())
 ```
 
 ### Prompt Editor
-The prompt editor is an LLM agent that reviews, comments and rewrites a prompt following the defined schema of each extractor. It is recommended to use prompt editor iteratively: 
+The prompt editor is an LLM agent that help users write prompt templates following the defined schema and guideline of each extractor. Chat with the promtp editor:
+
+```python
+from llm_ie.prompt_editor import PromptEditor
+from llm_ie.extractors import BasicFrameExtractor
+from llm_ie.engines import OllamaInferenceEngine
+
+# Define an LLM inference engine
+ollama = OllamaInferenceEngine(model_name="llama3.1:8b-instruct-q8_0")
+
+# Define editor
+editor = PromptEditor(ollama, BasicFrameExtractor)
+
+editor.chat()
+```
+
+In a terminal environment, an interactive chat session will start:
+<div align="left"><img src=doc_asset/readme_img/terminal_chat.PNG width=1000 ></div>
+
+In the Jupyter/IPython environment, an ipywidgets session will start:
+<div align="left"><img src=doc_asset/readme_img/IPython_chat.PNG width=1000 ></div>
+
+
+We can also use the `rewrite()` and `comment()` methods to programmingly interact with the prompt editor: 
 1. start with a casual description of the task
 2. have the prompt editor generate a prompt template as the starting point
 3. manually revise the prompt template
@@ -581,40 +636,29 @@ print(BasicFrameExtractor.get_prompt_guide())
 ```
 
 ```
-Prompt template design:
-    1. Task description
-    2. Schema definition
-    3. Output format definition
-    4. Additional hints
-    5. Input placeholder
+Prompt Template Design:
 
-Example:
+1. Task Description:  
+   Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition).
 
-    # Task description
-    The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.
+2. Schema Definition:  
+   List the key concepts that should be extracted, and provide clear definitions for each one.
 
-    # Schema definition
-    Your output should contain: 
-        "ClinicalTrial" which is the name of the trial, 
-        If applicable, "Arm" which is the arm within the clinical trial, 
-        "AdverseReaction" which is the name of the adverse reaction,
-        If applicable, "Percentage" which is the occurance of the adverse reaction within the trial and arm,
-        "Evidence" which is the EXACT sentence in the text where you found the AdverseReaction from
+3. Output Format Definition:  
+   The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). Each dictionary must include a key that holds the entity text. This key can be named "entity_text" or anything else depend on the context. The attributes can either be flat (e.g., {"entity_text": "<entity_text>", "attr1": "<attr1>", "attr2": "<attr2>"}) or nested (e.g., {"entity_text": "<entity_text>", "attributes": {"attr1": "<attr1>", "attr2": "<attr2>"}}).
 
-    # Output format definition
-    Your output should follow JSON format, for example:
-    [
-        {"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"},
-        {"ClinicalTrial": "<Clinical trial name or number>", "Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"} 
-    ]
+4. Optional: Hints:  
+   Provide itemized hints for the information extractors to guide the extraction process.
+
+5. Optional: Examples:  
+   Include examples in the format:  
+    Input: ...  
+    Output: ...
 
-    # Additional hints
-    Your output should be 100% based on the provided content. DO NOT output fake numbers. 
-    If there is no specific arm, just omit the "Arm" key. If the percentage is not reported, just omit the "Percentage" key. The "Evidence" should always be provided.
+6. Input Placeholder:  
+   The template must include a placeholder in the format {{<placeholder_name>}} for the input text. The placeholder name can be customized as needed.
 
-    # Input placeholder
-    Below is the Adverse reactions section:
-    {{input}}
+......
 ```
 </details>
 

llm_ie-0.3.0.dist-info/RECORD

@@ -0,0 +1,17 @@
+llm_ie/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+llm_ie/asset/PromptEditor_prompts/chat.txt,sha256=Fq62voV0JQ8xBRcxS1Nmdd7DkHs1fGYb-tmNwctZZK0,118
+llm_ie/asset/PromptEditor_prompts/comment.txt,sha256=C_lxx-dlOlFJ__jkHKosZ8HsNAeV1aowh2B36nIipBY,159
+llm_ie/asset/PromptEditor_prompts/rewrite.txt,sha256=JAwY9vm1jSmKf2qcLBYUvrSmME2EJH36bALmkwZDWYQ,178
+llm_ie/asset/PromptEditor_prompts/system.txt,sha256=QwGTIJvp-5u2P8CkGt_rabttlN1puHQwIBNquUm1ZHo,730
+llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt,sha256=m7iX4Qjsf1N2V1mbjE-x4F-qPGZA2qGJbUCdpets394,9293
+llm_ie/asset/prompt_guide/BinaryRelationExtractor_prompt_guide.txt,sha256=Z6Yc2_QRqroWcJ13owNJbo78I0wpS4XXDsOjXFR-aPk,2166
+llm_ie/asset/prompt_guide/MultiClassRelationExtractor_prompt_guide.txt,sha256=EQ9Jmh0CQmlfkWqXx6_apuEZUKK3WIrdpAvfbTX2_No,3011
+llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt,sha256=m7iX4Qjsf1N2V1mbjE-x4F-qPGZA2qGJbUCdpets394,9293
+llm_ie/asset/prompt_guide/SentenceFrameExtractor_prompt_guide.txt,sha256=oKH_QeDgpw771ZdHk3L7DYz2Jvfm7OolUoTiJyMJI30,9541
+llm_ie/data_types.py,sha256=hPz3WOeAzfn2QKmb0CxHmRdQWZQ4G9zq8U-RJBVFdYk,14329
+llm_ie/engines.py,sha256=PTYs7s_iCPmI-yFUCVCPY_cMGS77ma2VGoz4rdNkODI,9308
+llm_ie/extractors.py,sha256=l0zJEtPSuy-2f_OxPQFPH3RsyLydmN9MRQJQYiRdRbY,45026
+llm_ie/prompt_editor.py,sha256=y8YI-nOdBpxSr2boQBqG_vuGhW572a9rYPG3eb0oWH0,8205
+llm_ie-0.3.0.dist-info/METADATA,sha256=s2lWsb4RvN9_95KHqxY77Eclw4aEooTEay0lDcKD6VM,41225
+llm_ie-0.3.0.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
+llm_ie-0.3.0.dist-info/RECORD,,

llm_ie-0.2.1.dist-info/RECORD

@@ -1,15 +0,0 @@
-llm_ie/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-llm_ie/asset/PromptEditor_prompts/comment.txt,sha256=C_lxx-dlOlFJ__jkHKosZ8HsNAeV1aowh2B36nIipBY,159
-llm_ie/asset/PromptEditor_prompts/rewrite.txt,sha256=bYLOix7DUBlcWv-Q0JZ5kDnZ9OEXBt_AGDN0TydLB8o,191
-llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt,sha256=XbnU8byLGGUA3A3lT0bb2Hw-ggzhcqD3ZuKzduod2ww,1944
-llm_ie/asset/prompt_guide/BinaryRelationExtractor_prompt_guide.txt,sha256=z9Xg0fdFbVVwnTYcUTcAUvEIWhF075W8qGxN-Vj7xdo,1548
-llm_ie/asset/prompt_guide/MultiClassRelationExtractor_prompt_guide.txt,sha256=D5DphUHw8SUERUVdcIjUynuTmYJa6-PwBlF7FzxNsvQ,2276
-llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt,sha256=XbnU8byLGGUA3A3lT0bb2Hw-ggzhcqD3ZuKzduod2ww,1944
-llm_ie/asset/prompt_guide/SentenceFrameExtractor_prompt_guide.txt,sha256=8nj9OLPJMtr9Soi5JU3Xk-HC7pKNoI54xA_A4u7I5j4,2620
-llm_ie/data_types.py,sha256=zAUx-n1ePTTg1aFrml0lTZPreJx241YTeWFCprRoYbU,12245
-llm_ie/engines.py,sha256=m9ytGUX61jEy9SmVHbb90mrfGMAwC6dV-v7Jke1U7Ho,9296
-llm_ie/extractors.py,sha256=EVuHqW1lW0RpGmnRNmX4ih6ppfvy2gYOAOgc8Pngfkw,44103
-llm_ie/prompt_editor.py,sha256=dbu7A3O7O7Iw2v-xCgrTFH1-wTLAGf4SHDqdeS-He2Q,1869
-llm_ie-0.2.1.dist-info/METADATA,sha256=JOjW83QHlj-rgp09upos2-XvhvvnROBeOoX4U33AZKU,40052
-llm_ie-0.2.1.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
-llm_ie-0.2.1.dist-info/RECORD,,