data-science-document-ai 1.58.0__py3-none-any.whl → 1.59.0__py3-none-any.whl

src/prompts/library/bookingConfirmation/yangming/prompt.txt

@@ -7,66 +7,54 @@ The Freight Forwarding company receives Booking Confirmation from Yangming Carri
 These Booking Confirmations contain various details related to booking, container pick up and drop off depot details, vessel details, as well as other transport Legs data.
 They may be written in different languages such as English, German, Vietnamese, Chinese, and other European languages, and can appear in a variety of formats and layouts.
 Your role is to accurately extract specific entities from these Booking Confirmations to support efficient processing and accurate record-keeping.
-<CONTEXT>
-
 
-bookingNumber: A unique identifier for the booking.
-cyCutOff: The deadline for cargo to be delivered to the Container Yard.
-gateInReference: A reference code for cargo entering the terminal.
-gateInTerminal: The specific terminal where cargo is gated in.
-mblNumber: The Master Bill of Lading number.
-pickUpReference: A reference code for cargo pickup.
-pickUpTerminal: The specific terminal for cargo pickup.
-siCutOff: The deadline for submitting shipping instructions.
-vgmCutOff: The deadline for submitting the Verified Gross Mass of the cargo.
-transportLegs:
-    eta: The estimated time of arrival for a specific leg.
-    etd: The estimated time of departure for a specific leg.
-    imoNumber: The International Maritime Organization number for a specific leg.
-    portOfDischarge: The port where cargo is unloaded for a specific leg.
-    portOfLoading: The port where cargo is loaded for a specific leg.
-    vesselName: The name of the vessel for a specific leg.
-    voyage: The journey or route taken by the vessel for a specific leg.
+To provide context on the journey of a containers for both Export and Import shipments,
+For Export shipment: An empty container is picked up from a depot (pickupDepotCode) using a pickUpReference and goods loaded into it at a warehouse. Then the loaded container / cargo is transported back to a Container Yard or gateInTerminal before the cyCutOff date for further shipping processes. Then the POL of the First TransportLeg may start from the gateInTerminal or a different POL too.
+For Import Shipment: The loaded container / cargo arrives at a port of discharge then picked up at pickUpTerminal using pickUpReference. After delivery, an empty container is returned to a depot (dropOffDepotCode).
+<CONTEXT>
 
-your task is to extract the text value of the following entities and page numbers starting from 0 where the value was found in the document:
-SCHEMA_PLACEHOLDER
+<INSTRUCTIONS>
+- Populate fields as defined in the response schema.
+- Use the data field description to understand the context of the data.
 
-Keywords for datapoints:
-- bookingNumber: Our Reference
-- cyCutOff: FCL delivery cut-off
-- gateInReference: Our Reference
-- gateInTerminal: Export terminal delivery address
-- mblNumber: BL/SWB No(s).
-- pickUpReference: Export door positioning address(es)
-- siCutOff: shipping instruction closing
-- vgmCutOff: VGM cut-off
-- eta: eta, ETA
-- etd: etd, ETD
-- imoNumber: IMO No, IMO number
-- portOfDischarge: to
-- portOfLoading: from
-- vesselName: vessel
-- voyage: Voy. no
+- gateInTerminalCode: The specific terminal where cargo is gated in. It is mentioned as Delivery Terminal. This sometimes can be the same as portOfLoading of the First transportLeg.
+- cyCutOff: The deadline for cargo to be delivered to the Container Yard. It can be found at Cargo Cut Off or FCL delivery cut-off.
 
-Table Structure to extract TransporLegs:
-- transportlegs table has following colum names: From, To, By, ETD, ETA
-- If the first leg does not have a ETD and ETA, Also By column includes following info: ["Truck", "Combined Waterway"]; skip the row, start from one row below.
+- transportLegs: Multiple Transport Legs entries may exist, capture all instances under "transportLegs". Make sure the order of the legs are important.
+    - eta: The estimated time of arrival for a specific leg.
+    - etd: The estimated time of departure for a specific leg. It can also be referred as ETS (Estimated Time of Sailing).
+    - imoNumber: The International Maritime Organization number for a specific leg.
+    - portOfDischarge: The port where cargo is unloaded for a specific leg.
+    - portOfLoading: The port where cargo is loaded for a specific leg.
+    - vesselName: The name of the vessel for a specific leg.
+    - voyage: The journey or route taken by the vessel for a specific leg.
 
-Further explanation for the transportLegs part as follows:
-- you must differentiate the Voyage from DP Voyage. Never used the value with the key 'DP Voyage'.
-- Vessel name is in 'By' column and under the key Vessel
+IMPORTANT explanation for the transportLegs part as follows:
+- There is at least one leg in each document.
 - 'eta' must be equal or later than 'etd'!
-- portOfLoading and portOfDischarge are name of the Ports. You can rely on the port names from all over the world.
-- portOfLoading and portOfDischarge distinctly denotes the name of the ports. If you find abbreviation of the port use it, if not you can use the full name of the port
-- Abbrevations most likely to be in the paranthesis like follows (DEHAM).
-
-Further explanation for datapoints except transportLegs part as follows:
-- If gateInReference is null, assign it the same value as bookingNumber.
-- If pickUpReference is null, assign it the same value as bookingNumber.
-- If mblNumber is null, assign it the same value as bookingNumber.
-
-You must apply the following rules:
-- The JSON schema must be followed during the extraction.
-- The values must only include text found in the document
-- Do not normalize any entity value.
-- Validate the JSON make sure its a valid JSON ! No extra text, no missing comma!
+- Multiple legs are possible. When there are multiple legs,
+    - Sequential Sorting: You must manually re-order legs based on etd then eta, regardless of their order in the source text.
+    - "Transhipment" indicates the presence of a multi-leg journey.
+    - Transhipment Date is usually empty therefore you may keep ETA of the first leg and ETD of the second leg as null if there is a transhipment.
+    - Transhipment Handling: Treat any mentioned "Transhipment" as the bridge between two legs (Discharge for Leg A and Loading for Leg B).
+    - The Connectivity Rule: For any sequence of legs, the Port of Discharge of the previous leg must match the Port of Loading of the following leg.
+        - First Transhipment is the Port of Discharge for the first transportLegs and Port of Loading for the second transportLegs.
+        - Second Transhipment is the Port of Discharge for the second transportLegs and Port of Loading for the third transportLegs.
+    - Timeline Integrity: Ensure a "No Time Travel" policy: The eta of a previous leg must be earlier than or equal to the etd of the following leg.
+
+
+Structure of Multiple Leg Sequence & Mapping
+ Leg 1 (Initial):
+ - `portOfLoading`: Sailing.
+ - `portOfDischarge`: Transhipment (if exists), otherwise DISCHARGE.
+ - `etd`: Sailing Date ETS.
+ - `eta`: Transhipment Date (if exists), otherwise keep it null in case of multi-legs. Else, DISCHARGE Date for single leg.
+
+
+ Leg 2 (Intermediate):  Trigger: Only if PORT OF TRANSHIPMENT exists.
+ - `portOfLoading`: portOfDischarge of Leg 1 (Transhipment).
+ - `portOfDischarge`: DISCHARGE.
+ - `etd`: Transhipment Date if exists else keep it null
+ - `eta`: DISCHARGE Date. Do not extract Destination Date here.
+
+<INSTRUCTIONS>

src/utils.py

@@ -361,10 +361,7 @@ def extract_top_pages(pdf_bytes, num_pages=4):
 
 
 async def get_tms_mappings(
-    input_list: List[str],
-    embedding_type: str,
-    llm_ports: Optional[List[str]] = None,
-    input_key: str = None,
+    input_list: List[str], embedding_type: str, llm_ports: Optional[List[str]] = None
 ) -> Dict[str, Any]:
     """Get TMS mappings for the given values.
 
@@ -373,7 +370,6 @@ async def get_tms_mappings(
         embedding_type (str): Type of embedding to use
          (e.g., "container_types", "ports", "depots", "lineitems", "terminals").
         llm_ports (list[str], optional): List of LLM ports to use. Defaults to None.
-        input_key (str, optional): Key to use for input list in payload. Defaults to None.
 
     Returns:
         dict or string: A dictionary or a string with the mapping results.
@@ -393,7 +389,7 @@ async def get_tms_mappings(
         input_list = [input_list]
 
     # Always send a dict with named keys
-    payload = {input_key or embedding_type: input_list}
+    payload = {embedding_type: input_list}
 
     if llm_ports:
         payload["llm_ports"] = llm_ports if isinstance(llm_ports, list) else [llm_ports]