sparrow-parse 0.1.9__tar.gz → 0.2.0__tar.gz

{sparrow_parse-0.1.9 → sparrow_parse-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: sparrow-parse
-Version: 0.1.9
+Version: 0.2.0
 Summary: Sparrow Parse is a Python package for parsing and extracting information from documents.
 Home-page: https://github.com/katanaml/sparrow/tree/main/sparrow-data/parse
 License: GPL-3.0
@@ -15,10 +15,13 @@ Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Topic :: Software Development
+Requires-Dist: pymupdf4llm (==0.0.5)
 Requires-Dist: rich (>=13.7.1,<14.0.0)
+Requires-Dist: sentence-transformers (==3.0.1)
 Requires-Dist: torch (==2.2.2)
-Requires-Dist: unstructured-inference (==0.7.29)
-Requires-Dist: unstructured[all-docs] (==0.13.6)
+Requires-Dist: transformers (==4.41.2)
+Requires-Dist: unstructured-inference (==0.7.33)
+Requires-Dist: unstructured[all-docs] (==0.14.5)
 Project-URL: Repository, https://github.com/katanaml/sparrow
 Description-Content-Type: text/markdown
 
@@ -26,7 +29,7 @@ Description-Content-Type: text/markdown
 
 ## Description
 
-This module implements Sparrow Parse [library](https://pypi.org/project/sparrow-parse/) with helpful methods for data pre-processing.
+This module implements Sparrow Parse [library](https://pypi.org/project/sparrow-parse/) with helpful methods for data pre-processing, parsing and extracting information.
 
 ## Install
 
@@ -34,22 +37,97 @@ This module implements Sparrow Parse [library](https://pypi.org/project/sparrow-
 pip install sparrow-parse
 ```
 
-## Use
+## Pre-processing
 
-Import
+### Unstructured
 
 ```
-from sparrow_parse.pdf.pdf_processor import PDFProcessor
+from sparrow_parse.extractor.unstructured_processor import UnstructuredProcessor
+
+processor = UnstructuredProcessor()
+
+content, table_content = processor.extract_data(
+        file_path,  # file to process
+        strategy,  # data processing strategy supported by unstructured
+        model_name,  # model supported by unstructured
+        options,  # table extraction into HTML format
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
 ```
 
-Usage
+Example:
+
+*file_path* - `/Users/andrejb/infra/shared/katana-git/sparrow/sparrow-ml/llm/data/invoice_1.pdf`
+
+*strategy* - `hi_res`
+
+*model_name* - `yolox`
+
+*options* - `['tables', 'html']`
+
+*local* - `True`
+
+*debug* - `True`
+
+### Markdown
 
 ```
-processor = PDFProcessor()
-result = processor.process_file(file_path, strategy, model_name)
+from sparrow_parse.extractor.markdown_processor import MarkdownProcessor
+
+processor = MarkdownProcessor()
+
+content, table_content = processor.extract_data(
+        file_path,  # file to process
+        options,  # table extraction into HTML format
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
 ```
 
-Build for development
+Example:
+
+*file_path* - `/Users/andrejb/infra/shared/katana-git/sparrow/sparrow-ml/llm/data/invoice_1.pdf`
+
+*options* - `['tables', 'markdown']`
+
+*local* - `True`
+
+*debug* - `True`
+
+## Parsing and extraction
+
+```
+from sparrow_parse.extractor.html_extractor import HTMLExtractor
+
+extractor = HTMLExtractor()
+
+answer, targets_unprocessed = extractor.read_data(
+        target_columns,  # list of table columns data to fetch
+        data, # list of HTML tables
+        column_keywords,  # list of valid column names, can be empty. Useful to filter junk content
+        group_by_rows,  # JSON result grouping
+        update_targets,  # Set to true, if page contains multiple tables with the same columns
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
+
+```
+
+Example:
+
+*target_columns* - `['description', 'qty', 'net_price', 'net_worth', 'vat', 'gross_worth']`
+
+*data* - `list of HTML tables`
+
+*column_keywords* - `None`
+
+*group_by_rows* - `True`
+
+*update_targets* - `True`
+
+*local* - `True`
+
+*debug* - `True`
+
+## Library build
 
 ```
 poetry build

sparrow_parse-0.2.0/README.md

@@ -0,0 +1,131 @@
+# Sparrow Parse
+
+## Description
+
+This module implements Sparrow Parse [library](https://pypi.org/project/sparrow-parse/) with helpful methods for data pre-processing, parsing and extracting information.
+
+## Install
+
+```
+pip install sparrow-parse
+```
+
+## Pre-processing
+
+### Unstructured
+
+```
+from sparrow_parse.extractor.unstructured_processor import UnstructuredProcessor
+
+processor = UnstructuredProcessor()
+
+content, table_content = processor.extract_data(
+        file_path,  # file to process
+        strategy,  # data processing strategy supported by unstructured
+        model_name,  # model supported by unstructured
+        options,  # table extraction into HTML format
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
+```
+
+Example:
+
+*file_path* - `/Users/andrejb/infra/shared/katana-git/sparrow/sparrow-ml/llm/data/invoice_1.pdf`
+
+*strategy* - `hi_res`
+
+*model_name* - `yolox`
+
+*options* - `['tables', 'html']`
+
+*local* - `True`
+
+*debug* - `True`
+
+### Markdown
+
+```
+from sparrow_parse.extractor.markdown_processor import MarkdownProcessor
+
+processor = MarkdownProcessor()
+
+content, table_content = processor.extract_data(
+        file_path,  # file to process
+        options,  # table extraction into HTML format
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
+```
+
+Example:
+
+*file_path* - `/Users/andrejb/infra/shared/katana-git/sparrow/sparrow-ml/llm/data/invoice_1.pdf`
+
+*options* - `['tables', 'markdown']`
+
+*local* - `True`
+
+*debug* - `True`
+
+## Parsing and extraction
+
+```
+from sparrow_parse.extractor.html_extractor import HTMLExtractor
+
+extractor = HTMLExtractor()
+
+answer, targets_unprocessed = extractor.read_data(
+        target_columns,  # list of table columns data to fetch
+        data, # list of HTML tables
+        column_keywords,  # list of valid column names, can be empty. Useful to filter junk content
+        group_by_rows,  # JSON result grouping
+        update_targets,  # Set to true, if page contains multiple tables with the same columns
+        local,  # True if running from CLI, or False if running from FastAPI
+        debug)  # Debug
+
+```
+
+Example:
+
+*target_columns* - `['description', 'qty', 'net_price', 'net_worth', 'vat', 'gross_worth']`
+
+*data* - `list of HTML tables`
+
+*column_keywords* - `None`
+
+*group_by_rows* - `True`
+
+*update_targets* - `True`
+
+*local* - `True`
+
+*debug* - `True`
+
+## Library build
+
+```
+poetry build
+```
+
+Publish to PyPi
+
+```
+poetry publish
+```
+
+## Commercial usage
+
+Sparrow is available under the GPL 3.0 license, promoting freedom to use, modify, and distribute the software while ensuring any modifications remain open source under the same license. This aligns with our commitment to supporting the open-source community and fostering collaboration.
+
+Additionally, we recognize the diverse needs of organizations, including small to medium-sized enterprises (SMEs). Therefore, Sparrow is also offered for free commercial use to organizations with gross revenue below $5 million USD in the past 12 months, enabling them to leverage Sparrow without the financial burden often associated with high-quality software solutions.
+
+For businesses that exceed this revenue threshold or require usage terms not accommodated by the GPL 3.0 license—such as integrating Sparrow into proprietary software without the obligation to disclose source code modifications—we offer dual licensing options. Dual licensing allows Sparrow to be used under a separate proprietary license, offering greater flexibility for commercial applications and proprietary integrations. This model supports both the project's sustainability and the business's needs for confidentiality and customization.
+
+If your organization is seeking to utilize Sparrow under a proprietary license, or if you are interested in custom workflows, consulting services, or dedicated support and maintenance options, please contact us at abaranovskis@redsamuraiconsulting.com. We're here to provide tailored solutions that meet your unique requirements, ensuring you can maximize the benefits of Sparrow for your projects and workflows.
+
+## Author
+
+[Katana ML](https://katanaml.io), [Andrej Baranovskij](https://github.com/abaranovskis-redsamurai)
+
+## License
+
+Licensed under the GPL 3.0. Copyright 2020-2024 Katana ML, Andrej Baranovskij. [Copy of the license](https://github.com/katanaml/sparrow/blob/main/LICENSE).

{sparrow_parse-0.1.9 → sparrow_parse-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "sparrow-parse"
-version = "0.1.9"
+version = "0.2.0"
 description = "Sparrow Parse is a Python package for parsing and extracting information from documents."
 authors = ["Andrej Baranovskij <andrejus.baranovskis@gmail.com>"]
 license = "GPL-3.0"
@@ -22,9 +22,12 @@ include = [
 [tool.poetry.dependencies]
 python = ">=3.9,<3.12"
 torch = {version = "2.2.2", source = "pypi"}
-unstructured = {version = "0.13.6", extras = ["all-docs"]}
-unstructured-inference = "0.7.29"
+unstructured = {version = "0.14.5", extras = ["all-docs"]}
+unstructured-inference = "0.7.33"
 rich = "^13.7.1"
+pymupdf4llm = "0.0.5"
+transformers = "4.41.2"
+sentence-transformers = "3.0.1"
 
 
 [tool.poetry.scripts]
@@ -33,4 +36,4 @@ sparrow-parse = 'sparrow_parse:main'
 
 [build-system]
 requires = ["poetry-core"]
-build-backend = "poetry.core.masonry.api"
+build-backend = "poetry.core.masonry.api"

sparrow_parse-0.2.0/sparrow_parse/__init__.py

@@ -0,0 +1 @@
+__version__ = '0.2.0'

sparrow_parse-0.2.0/sparrow_parse/data/invoice_1_table.txt

@@ -0,0 +1,9 @@
+[
+    '<table><thead><th>No.</th><th>Description</th><th>Qty</th><th>UM</th><th>Net price</th><th>Net worth</th><th>VAT [%]</th><th>Gross worth</th></thead><tr><td></td><td>Wine Glasses Goblets Pair Clear
+Glass</td><td>5,00</td><td>eacn</td><td>12,00</td><td>60,00</td><td>10%</td><td>66,00</td></tr><tr><td></td><td>With Hooks Stemware Storage Multiple Uses Iron Wine Rack Hanging
+Glass</td><td>4,00</td><td>eacn</td><td>28,08</td><td>112,32</td><td>10%</td><td>123,55</td></tr><tr><td></td><td>Replacement Corkscrew Parts Spiral Worm Wine Opener Bottle
+Houdini</td><td>1,00</td><td>eacn</td><td>7,50</td><td>7,50</td><td>10%</td><td>8,25</td></tr><tr><td></td><td>HOME ESSENTIALS GRADIENT STEMLESS WINE GLASSES SET OF 4 20 FL OZ (591 ml)
+NEW</td><td>1,00</td><td>eacn</td><td>12,99</td><td>12,99</td><td>10%</td><td>14,29</td></tr></table>',
+    '<table><thead><th>VAT</th><th>[%]</th><th>Net worth</th><th>VAT</th><th>Gross worth</th></thead><tr><td></td><td>10%</td><td>192,81</td><td>19,28</td><td>212,09</td></tr><tr><td colspan="2">Total</td><td>$ 192,81</td><td>$
+19,28</td><td>$ 212,09</td></tr></table>'
+]

sparrow_parse-0.2.0/sparrow_parse/extractor/extractor_helper.py

@@ -0,0 +1,368 @@
+from bs4 import BeautifulSoup
+from sentence_transformers import SentenceTransformer, util
+import pandas as pd
+import re
+from io import StringIO
+
+
+def merge_html_table_headers(html_table, column_keywords, debug=False):
+    soup = BeautifulSoup(html_table, 'html.parser')
+
+    # Find all thead elements
+    theads = soup.find_all('thead')
+
+    if len(theads) > 1 and column_keywords is not None:
+        html_table = update_table_header_colspan(html_table)
+        html_table = merge_table_header_thead(html_table)
+        html_table = merge_colspan_columns(html_table)
+        html_table = normalize_html_table(html_table, debug)
+        html_table = fix_rowspan_elements(html_table)
+        html_table = merge_rows_with_rowspan(html_table)
+        html_table = detect_and_remove_junk_columns(html_table, column_keywords, debug)
+    else:
+        # If there is only one thead, return the original table
+        return html_table
+
+    return html_table
+
+
+def update_table_header_colspan(html_table):
+    soup = BeautifulSoup(html_table, 'html.parser')
+    theads = soup.find_all('thead')
+
+    for thead in theads:
+        for th in thead.find_all('th'):
+            colspan = th.get('colspan')
+            if colspan and int(colspan) > 1:
+                colspan_count = int(colspan)
+                th['colspan'] = 1
+                for _ in range(colspan_count - 1):
+                    new_th = soup.new_tag('th')
+                    th.insert_after(new_th)
+
+    return str(soup)
+
+
+def merge_table_header_thead(html_table):
+    soup = BeautifulSoup(html_table, 'html.parser')
+    theads = soup.find_all('thead')
+
+    primary_thead = theads[0]
+    secondary_thead = theads[1]
+
+    primary_ths = primary_thead.find_all('th')
+    secondary_ths = secondary_thead.find_all('th')
+
+    for i, th in enumerate(primary_ths):
+        if i < len(secondary_ths):
+            primary_text = th.text.strip()
+            secondary_text = secondary_ths[i].text.strip()
+            if primary_text and secondary_text:
+                th.string = (primary_text + ' ' + secondary_text).strip()
+            elif not primary_text and secondary_text:
+                th.string = secondary_text
+        # Remove colspan and rowspan attributes
+        th.attrs.pop('colspan', None)
+        th.attrs.pop('rowspan', None)
+
+    secondary_thead.decompose()
+
+    return str(soup)
+
+
+def merge_colspan_columns(html_table):
+    # Parse the HTML
+    soup = BeautifulSoup(html_table, 'html.parser')
+
+    # Process colspan attributes by adding empty <td> elements
+    for row in soup.find_all('tr'):
+        cols = []
+        for cell in row.find_all(['th', 'td']):
+            colspan = int(cell.get('colspan', 1))
+            # Add the cell and additional empty cells if colspan is greater than 1
+            cols.append(cell)
+            for _ in range(colspan - 1):
+                new_td = soup.new_tag('td')
+                cols.append(new_td)
+            # Remove the colspan attribute
+            if cell.has_attr('colspan'):
+                del cell['colspan']
+
+        # Replace the row's children with the updated cells
+        row.clear()
+        row.extend(cols)
+
+    return str(soup)
+
+
+def normalize_html_table(html, debug = False):
+    soup = BeautifulSoup(html, 'html.parser')
+
+    # Find the header row and count the number of cells
+    header = soup.find('thead').find_all(['th', 'td'])
+    header_cell_count = len(header)
+
+    if debug:
+        # Print the number of header cells
+        print(f"Number of cells in header: {header_cell_count}")
+
+    # Find all rows in the table body
+    rows = soup.find_all('tr')
+
+    for row in rows:
+        cells = row.find_all(['td', 'th'])
+        if len(cells) > header_cell_count:
+            extra_cells = len(cells) - header_cell_count
+            for cell in cells:
+                if cell.text.strip() == '' and extra_cells > 0:
+                    cell.decompose()
+                    extra_cells -= 1
+        elif len(cells) < header_cell_count:
+            missing_cells = header_cell_count - len(cells)
+            for _ in range(missing_cells):
+                new_cell = soup.new_tag('td')
+                row.insert(0, new_cell)
+
+    return str(soup)
+
+
+def fix_rowspan_elements(html_table):
+    # Parse the HTML table
+    soup = BeautifulSoup(html_table, 'html.parser')
+
+    # Find all table rows
+    rows = soup.find_all('tr')
+
+    # Dictionary to store rows with rowspan elements
+    rowspan_dict = {}
+
+    # Iterate over each row
+    for row_index, row in enumerate(rows):
+        # Find all cells in the row
+        cells = row.find_all(['td', 'th'])
+
+        # Iterate over each cell
+        for cell_index, cell in enumerate(cells):
+            # Check if the cell has a rowspan attribute
+            if cell.has_attr('rowspan'):
+                # Store the rowspan value and cell position
+                rowspan_value = int(cell['rowspan'])
+                if row_index not in rowspan_dict:
+                    rowspan_dict[row_index] = []
+                rowspan_dict[row_index].append((cell_index, rowspan_value))
+
+    # List to store the number of rows until the next rowspan row
+    rows_below_until_next_rowspan = []
+
+    # Get the sorted row indices that have rowspan elements
+    sorted_row_indices = sorted(rowspan_dict.keys())
+
+    # Calculate rows below each rowspan row until the next rowspan row
+    for i in range(len(sorted_row_indices)):
+        current_row = sorted_row_indices[i]
+        if i < len(sorted_row_indices) - 1:
+            next_row = sorted_row_indices[i + 1]
+            rows_below = next_row - current_row - 1
+        else:
+            rows_below = len(rows) - current_row - 1
+        rows_below_until_next_rowspan.append((current_row, rows_below))
+
+    # Detect rows where rowspan value is incorrect
+    rows_with_bad_rowspan = []
+    for row_index, rows_below in rows_below_until_next_rowspan:
+        if row_index in rowspan_dict:
+            for cell_index, rowspan_value in rowspan_dict[row_index]:
+                if rowspan_value - 1 < rows_below:
+                    print(f"Row {row_index} has a large rowspan value: {rowspan_value}")
+                    rows_with_bad_rowspan.append(row_index)
+                    break
+
+    # Modify the HTML table to adjust the rowspan attributes
+    for row_index in rows_with_bad_rowspan:
+        if row_index in rowspan_dict:
+            for cell_index, rowspan_value in rowspan_dict[row_index]:
+                # Find the cell with the rowspan attribute
+                cell = rows[row_index].find_all(['td', 'th'])[cell_index]
+                # Remove the rowspan attribute
+                del cell['rowspan']
+                # Find the next row and assign the rowspan value
+                next_row_index = row_index + 1
+                if next_row_index < len(rows):
+                    next_row_cells = rows[next_row_index].find_all(['td', 'th'])
+                    if len(next_row_cells) > cell_index:
+                        next_row_cell = next_row_cells[cell_index]
+                        next_row_cell['rowspan'] = rowspan_value
+                    else:
+                        # Create a new cell if it does not exist
+                        new_cell = soup.new_tag(cell.name)
+                        new_cell['rowspan'] = rowspan_value
+                        new_cell.string = cell.string
+                        rows[next_row_index].append(new_cell)
+
+    # Return the modified HTML table
+    return str(soup)
+
+
+def merge_rows_with_rowspan(html):
+    # Parse the HTML table using BeautifulSoup
+    soup = BeautifulSoup(html, 'html.parser')
+
+    # Extract the header
+    thead = soup.find('thead')
+
+    # Find all rows
+    rows = soup.find_all('tr')
+
+    result = []
+    i = 0
+
+    while i < len(rows):
+        row = rows[i]
+        # Check if any td in the row has a rowspan attribute
+        for td in row.find_all('td'):
+            if td.has_attr('rowspan'):
+                rowspan_value = int(td['rowspan'])
+                result.append(row)
+
+                skip_concatenation = False
+                concatenation_pairs = []
+
+                # Add rows below the current row based on the rowspan number
+                for j in range(1, rowspan_value):
+                    if i + j < len(rows):
+                        below_row = rows[i + j]
+
+                        # Compare cells
+                        row_cells = row.find_all('td')
+                        below_row_cells = below_row.find_all('td')
+                        min_length = min(len(row_cells), len(below_row_cells))
+
+                        for k in range(min_length):
+                            if is_numeric(row_cells[k].get_text(strip=True)) and is_numeric(below_row_cells[k].get_text(strip=True)):
+                                skip_concatenation = True
+                                break
+                            else:
+                                concatenation_pairs.append((row_cells[k], below_row_cells[k]))
+
+                        if skip_concatenation:
+                            result.append(below_row)
+
+                if not skip_concatenation:
+                    for row_cell, below_row_cell in concatenation_pairs:
+                        concatenated_text = (row_cell.get_text(strip=True) + ' ' + below_row_cell.get_text(strip=True)).strip()
+                        row_cell.string = concatenated_text
+
+                i += rowspan_value - 1  # Skip the rows that have been added
+                break
+            else:
+                result.append(row)
+                break
+        i += 1
+
+    # Convert result list of rows back to an HTML table string
+    new_table_soup = BeautifulSoup(f'<table>{str(thead)}</table>', 'html.parser')
+    tbody = new_table_soup.new_tag('tbody')
+    new_table_soup.table.append(tbody)
+    for row in result:
+        for td in row.find_all('td'):
+            if td.has_attr('rowspan'):
+                del td['rowspan']
+        tbody.append(row)
+
+    return str(new_table_soup.table)
+
+
+def detect_and_remove_junk_columns(html_table, target_columns, debug=False):
+    # Wrap the HTML string in a StringIO object
+    html_buffer = StringIO(html_table)
+
+    # Read the HTML table
+    df = pd.read_html(html_buffer)[0]
+
+    model = SentenceTransformer('all-mpnet-base-v2')
+
+    # Get the column names of the dataframe
+    column_names = df.columns.tolist()
+
+    # Calculate the similarity of each column name to the target column names
+    target_embeddings = model.encode(target_columns)
+    column_embeddings = model.encode(column_names)
+
+    # Initialize a dictionary to store the similarity scores
+    similarity_scores = {}
+
+    # Identify junk columns based on similarity threshold
+    junk_columns = []
+    similarity_threshold = 0.5  # Adjust this threshold as needed
+
+    for idx, col_embedding in enumerate(column_embeddings):
+        similarities = util.pytorch_cos_sim(col_embedding, target_embeddings)[0]
+        max_similarity = max(similarities)
+        max_similarity_idx = similarities.argmax().item()  # Get the index of the max similarity
+        similarity_scores[column_names[idx]] = (
+        max_similarity.item(), target_columns[max_similarity_idx])  # Store similarity score and target column name
+        if max_similarity < similarity_threshold:
+            junk_columns.append(column_names[idx])
+
+    if debug:
+        # Print the similarity scores for debugging purposes
+        for column, (score, target_col) in similarity_scores.items():
+            print(f"Column: {column}, Similarity: {score:.4f}, Target Column: {target_col}")
+
+    # Handle junk columns by concatenating their values to the nearest column on the left
+    for junk_col in junk_columns:
+        junk_col_index = column_names.index(junk_col)
+        if junk_col_index > 0:
+            nearest_col = column_names[junk_col_index - 1]
+            df[nearest_col] = df.apply(
+                lambda row: str(row[junk_col]) if pd.isna(row[nearest_col]) and pd.notna(row[junk_col])
+                else (str(row[nearest_col]) + ' ' + str(row[junk_col])) if pd.notna(row[junk_col])
+                else row[nearest_col],
+                axis=1
+            )
+        df.drop(columns=[junk_col], inplace=True)
+
+    # Replace any remaining NaN values with empty strings
+    df = df.fillna('')
+
+    if debug:
+        print(f"Junk columns: {junk_columns}")
+        print(df.to_string())
+
+    # Convert the result into an HTML table
+    html_table = df.to_html(index=False)
+
+    if debug:
+        print(html_table)
+
+    return html_table
+
+
+def clean_html_table_header_names(html_table: str) -> str:
+    """
+        Cleans the headers of an HTML table by removing junk characters and returns the updated HTML as a string.
+
+        Parameters:
+        html (str): The HTML content containing the table.
+
+        Returns:
+        str: The updated HTML table with cleaned headers.
+        """
+    # Parse the HTML table
+    soup = BeautifulSoup(html_table, "html.parser")
+    table = soup.find("table")
+
+    # Extract the headers and clean them
+    headers = table.find_all("th")
+    for th in headers:
+        clean_header = re.sub(r"[^a-zA-Z0-9\s]", "", th.get_text())
+        th.string.replace_with(clean_header)
+
+    html_table = str(soup)
+
+    return html_table
+
+
+def is_numeric(value):
+    # Check if the value is numeric
+    return bool(re.match(r'^\d+(?:,\d{3})*(?:\.\d+)?$', value))