invoice2data 0.4.1__tar.gz → 0.4.3__tar.gz

{invoice2data-0.4.1/src/invoice2data.egg-info → invoice2data-0.4.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: invoice2data
-Version: 0.4.1
+Version: 0.4.3
 Summary: Python parser to extract data from pdf invoice
 Home-page: https://github.com/invoice-x/invoice2data
 Author: Manuel Riel
@@ -15,10 +15,11 @@ Classifier: Operating System :: POSIX
 Classifier: Operating System :: Unix
 Classifier: Operating System :: Microsoft :: Windows
 Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3.6
 Classifier: Programming Language :: Python :: 3.7
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Topic :: Office/Business :: Financial
 Classifier: Topic :: Office/Business :: Financial :: Accounting
 Description-Content-Type: text/markdown
@@ -35,9 +36,9 @@ A command line tool and Python library to support your accounting
 process.
 
 1. extracts text from PDF files using different techniques, like
-   `pdftotext`, `text`, `pdfminer`, `pdfplumber` or OCR -- `tesseract`, or
+   `pdftotext`, `text`, `ocrmypdf`, `pdfminer`, `pdfplumber` or OCR -- `tesseract`, or
    `gvision` (Google Cloud Vision).
-2. searches for regex in the result using a YAML-based template system
+2. searches for regex in the result using a YAML or JSON-based template system
 3. saves results as CSV, JSON or XML or renames PDF files to match the content.
 
 With the flexible template system you can:
@@ -107,6 +108,7 @@ Choose any of the following input readers:
 - tesseract `invoice2data --input-reader tesseract invoice.pdf`
 - pdfminer.six `invoice2data --input-reader pdfminer invoice.pdf`
 - pdfplumber `invoice2data --input-reader pdfplumber invoice.pdf`
+- ocrmypdf `invoice2data --input-reader ocrmypdf invoice.pdf`
 - gvision `invoice2data --input-reader gvision invoice.pdf` (needs `GOOGLE_APPLICATION_CREDENTIALS` env var)
 
 Choose any of the following output formats:
@@ -122,7 +124,7 @@ Save output file with custom name or a specific folder
 **Note:** You must specify the `output-format` in order to create
 `output-name`
 
-Specify folder with yml templates. (e.g. your suppliers)
+Specify folder with yml templates. (e.g. your suppliers)
 
 `invoice2data --template-folder ACME-templates invoice.pdf`
 
@@ -165,7 +167,7 @@ the list to add your own. If deployed by a bigger organisation, there
 should be an interface to edit templates for new suppliers. 80-20 rule.
 For a short tutorial on how to add new templates, see [TUTORIAL.md](TUTORIAL.md).
 
-Templates are based on Yaml. They define one or more keywords to find
+Templates are based on Yaml or JSON. They define one or more keywords to find
 the right template, one or more exclude_keywords to further narrow it down
 and regexp for fields to be extracted. They could also be a static value,
 like the full company name.
@@ -209,6 +211,14 @@ The lines package has multiple settings:
 - skip_line > Optional. If first_line is passed, this pattern indicates which sub-lines will be skipped and their data not recorded. This is useful if tables span multiple pages and you need to skip over page numbers or headers that appear mid-table.
 - last_line > Optional. If first_line is passed, this pattern denotes the final line of the sub-lines and is included in the output data.
 
+:warning: Invoice2data uses a yaml templating system. The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow)
+As an alternative json templates can be used. Which are natively better supported by python.
+
+The performance with yaml templates can be greatly increased **10x** by using [libyaml](https://github.com/yaml/libyaml)
+It can be installed on most distributions by:
+`sudo apt-get libyaml-dev`
+
+
 ## Development
 
 If you are interested in improving this project, have a look at our

{invoice2data-0.4.1 → invoice2data-0.4.3}/README.md

@@ -8,9 +8,9 @@ A command line tool and Python library to support your accounting
 process.
 
 1. extracts text from PDF files using different techniques, like
-   `pdftotext`, `text`, `pdfminer`, `pdfplumber` or OCR -- `tesseract`, or
+   `pdftotext`, `text`, `ocrmypdf`, `pdfminer`, `pdfplumber` or OCR -- `tesseract`, or
    `gvision` (Google Cloud Vision).
-2. searches for regex in the result using a YAML-based template system
+2. searches for regex in the result using a YAML or JSON-based template system
 3. saves results as CSV, JSON or XML or renames PDF files to match the content.
 
 With the flexible template system you can:
@@ -80,6 +80,7 @@ Choose any of the following input readers:
 - tesseract `invoice2data --input-reader tesseract invoice.pdf`
 - pdfminer.six `invoice2data --input-reader pdfminer invoice.pdf`
 - pdfplumber `invoice2data --input-reader pdfplumber invoice.pdf`
+- ocrmypdf `invoice2data --input-reader ocrmypdf invoice.pdf`
 - gvision `invoice2data --input-reader gvision invoice.pdf` (needs `GOOGLE_APPLICATION_CREDENTIALS` env var)
 
 Choose any of the following output formats:
@@ -95,7 +96,7 @@ Save output file with custom name or a specific folder
 **Note:** You must specify the `output-format` in order to create
 `output-name`
 
-Specify folder with yml templates. (e.g. your suppliers)
+Specify folder with yml templates. (e.g. your suppliers)
 
 `invoice2data --template-folder ACME-templates invoice.pdf`
 
@@ -138,7 +139,7 @@ the list to add your own. If deployed by a bigger organisation, there
 should be an interface to edit templates for new suppliers. 80-20 rule.
 For a short tutorial on how to add new templates, see [TUTORIAL.md](TUTORIAL.md).
 
-Templates are based on Yaml. They define one or more keywords to find
+Templates are based on Yaml or JSON. They define one or more keywords to find
 the right template, one or more exclude_keywords to further narrow it down
 and regexp for fields to be extracted. They could also be a static value,
 like the full company name.
@@ -182,6 +183,14 @@ The lines package has multiple settings:
 - skip_line > Optional. If first_line is passed, this pattern indicates which sub-lines will be skipped and their data not recorded. This is useful if tables span multiple pages and you need to skip over page numbers or headers that appear mid-table.
 - last_line > Optional. If first_line is passed, this pattern denotes the final line of the sub-lines and is included in the output data.
 
+:warning: Invoice2data uses a yaml templating system. The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow)
+As an alternative json templates can be used. Which are natively better supported by python.
+
+The performance with yaml templates can be greatly increased **10x** by using [libyaml](https://github.com/yaml/libyaml)
+It can be installed on most distributions by:
+`sudo apt-get libyaml-dev`
+
+
 ## Development
 
 If you are interested in improving this project, have a look at our

{invoice2data-0.4.1 → invoice2data-0.4.3}/setup.cfg

@@ -2,7 +2,7 @@
 name = invoice2data
 author = Manuel Riel
 description = Python parser to extract data from pdf invoice
-version = 0.4.1
+version = 0.4.3
 url = https://github.com/invoice-x/invoice2data
 keywords = 
 	pdf
@@ -17,10 +17,11 @@ classifiers =
 	Operating System :: Unix
 	Operating System :: Microsoft :: Windows
 	License :: OSI Approved :: MIT License
-	Programming Language :: Python :: 3.6
 	Programming Language :: Python :: 3.7
 	Programming Language :: Python :: 3.8
 	Programming Language :: Python :: 3.9
+	Programming Language :: Python :: 3.10
+	Programming Language :: Python :: 3.11
 	Topic :: Office/Business :: Financial
 	Topic :: Office/Business :: Financial :: Accounting
 long_description = file: README.md

{invoice2data-0.4.1 → invoice2data-0.4.3}/src/invoice2data/extract/invoice_template.py

@@ -7,15 +7,16 @@ Templates are initially read from .yml files and then kept as class.
 import re
 import dateparser
 import unicodedata
-import logging
+from logging import getLogger
+from pprint import pformat
 from collections import OrderedDict
 from . import parsers
 from .plugins import lines, tables
-# Area extraction is currently added for pdftotext and tesseract (which uses pdftotext)
-from ..input import pdftotext, tesseract
+# Area extraction is currently added for pdftotext, ocrmypdf and tesseract (which uses pdftotext)
+from ..input import pdftotext, ocrmypdf, tesseract
 from typing import Optional
 
-logger = logging.getLogger(__name__)
+logger = getLogger(__name__)
 
 OPTIONS_DEFAULT = {
     "remove_whitespace": False,
@@ -41,7 +42,7 @@ class InvoiceTemplate(OrderedDict):
     -------
     prepare_input(extracted_str)
         Input raw string and do transformations, as set in template file.
-    matches_input(optimized_str)
+    matches_input(extracted_str)
         See if string matches keywords set in template file
     parse_number(value)
         Parse number, remove decimal separator and add other options
@@ -59,12 +60,15 @@ class InvoiceTemplate(OrderedDict):
         # Merge template-specific options with defaults
         self.options = OPTIONS_DEFAULT.copy()
 
-        for lang in self.options["languages"]:
-            assert len(lang) == 2, "lang code must have 2 letters"
-
         if "options" in self:
             self.options.update(self["options"])
 
+        for lang in self.options["languages"]:
+            assert len(lang) == 2, (
+                "Error in Template %s lang code must have 2 letters"
+                % self["template_name"]
+            )
+
         # Set issuer, if it doesn't exist.
         if "issuer" not in self.keys():
             self["issuer"] = self["keywords"][0]
@@ -82,7 +86,7 @@ class InvoiceTemplate(OrderedDict):
 
         # Remove accents
         if self.options["remove_accents"]:
-            optimized_str = unicodedata.normalize('NFKD', optimized_str).encode('ascii', 'ignore').decode('ascii')
+            optimized_str = re.sub('[\u0300-\u0362]', '', unicodedata.normalize('NFKD', optimized_str))
 
         # Convert to lower case
         if self.options["lowercase"]:
@@ -90,40 +94,44 @@ class InvoiceTemplate(OrderedDict):
 
         # Specific replace
         for replace in self.options["replace"]:
-            assert len(replace) == 2, "A replace should be a list of exactly 2 elements."
+            assert len(replace) == 2, (
+                "Error in Template %s A replace should be a list of exactly 2 elements."
+                % self["template_name"]
+            )
             optimized_str = re.sub(replace[0], replace[1], optimized_str)
 
         return optimized_str
 
-    def matches_input(self, optimized_str: str) -> bool:
+    def matches_input(self, extracted_str: str) -> bool:
         """See if string matches all keyword patterns and no exclude_keyword patterns set in template file.
 
         Args:
-        optimized_str: String of the text from OCR of the pdf after applying options defined in the template.
+        extracted_str: String of the text from OCR of the pdf before applying options defined in the template.
 
         Return:
         Boolean
             - True if all keywords are found and none of the exclude_keywords are found.
             - False if either not all keywords are found or at least one exclude_keyword is found."""
 
-        if all([re.search(keyword, optimized_str) for keyword in self["keywords"]]):
+        if all([re.search(keyword, extracted_str) for keyword in self["keywords"]]):
             # All keyword patterns matched
             if self["exclude_keywords"]:
-                if any([re.search(exclude_keyword, optimized_str) for exclude_keyword in self["exclude_keywords"]]):
+                if any([re.search(exclude_keyword, extracted_str) for exclude_keyword in self["exclude_keywords"]]):
                     # At least one exclude_keyword matches
-                    logger.debug("Template: %s. Keywords matched. Exclude keyword found!", self["template_name"])
+                    logger.debug("Template: %s | Keywords matched. Exclude keyword found!", self["template_name"])
                     return False
             # No exclude_keywords or none match, template is good
-            logger.debug("Template: %s. Keywords matched. No exclude keywords found.", self["template_name"])
+            logger.debug("Template: %s | Keywords matched. No exclude keywords found.", self["template_name"])
             return True
         else:
-            logger.debug("Template: %s. Failed to match all keywords.", self["template_name"])
+            logger.debug("Template: %s | Failed to match all keywords.", self["template_name"])
             return False
 
     def parse_number(self, value):
-        assert (
-            value.count(self.options["decimal_separator"]) < 2
-        ), "Decimal separator cannot be present several times"
+        assert value.count(self.options["decimal_separator"]) < 2, (
+            "Error in Template %s Decimal separator cannot be present several times"
+            % self["template_name"]
+        )
         # replace decimal separator by a |
         amount_pipe = value.replace(self.options["decimal_separator"], "|")
         # remove all possible thousands separators
@@ -177,7 +185,7 @@ class InvoiceTemplate(OrderedDict):
             self.options["date_formats"],
         )
         logger.debug(
-            "Float parsing: decimal separator=%s", self.options["decimal_separator"]
+            "Float parsing: decimal separator=[%s]", self.options["decimal_separator"]
         )
         logger.debug("keywords=%s", self["keywords"])
         logger.debug(self.options)
@@ -191,7 +199,7 @@ class InvoiceTemplate(OrderedDict):
             # v is the value
             if isinstance(v, dict):
                 # Options were supplied to this field
-                if "area" in v and input_module in (pdftotext, tesseract):
+                if "area" in v and input_module in (pdftotext, ocrmypdf, tesseract):
                     # Area is currently only supported for pdftotext
                     # area is optional and re-extracts the text being searched
                     # This obviously has a performance impact, so use wisely
@@ -199,10 +207,9 @@ class InvoiceTemplate(OrderedDict):
                     logger.debug(f"Area was specified with parameters {v['area']}")
                     # Extract the text for the specified area
                     # Do NOT overwrite optimized_str. We're inside a loop and it will affect all other fields!
-                    optimized_str_area = input_module.to_text(invoice_file, v['area']).decode("utf-8")
+                    optimized_str_area = input_module.to_text(invoice_file, v['area'])
                     # Log the text
-                    logger.debug("START pdftotext area result ===========================")
-                    logger.debug(optimized_str_area)
+                    logger.debug("START pdftotext area result ===========================\n%s", optimized_str_area)
                     logger.debug("END pdftotext area result =============================")
                     optimized_str_for_parser = optimized_str_area
                 else:
@@ -217,11 +224,11 @@ class InvoiceTemplate(OrderedDict):
                         if value:
                             output[k] = value
                         else:
-                            logger.error("Failed to parse field %s with parser %s", k, v["parser"])
+                            logger.warning("Failed to parse field %s with parser %s", k, v["parser"])
                     else:
-                        logger.warning("Field %s has unknown parser %s set", k, v["parser"])
+                        logger.error("Field %s has unknown parser %s set", k, v["parser"])
                 else:
-                    logger.warning("Field %s doesn't have parser specified", k)
+                    logger.error("Field %s doesn't have parser specified", k)
             elif k.startswith("static_"):
                 logger.debug("field=%s | static value=%s", k, v)
                 output[k.replace("static_", "")] = v
@@ -261,7 +268,8 @@ class InvoiceTemplate(OrderedDict):
 
         if set(required_fields).issubset(output.keys()):
             output["desc"] = "Invoice from %s" % (self["issuer"])
-            logger.debug(output)
+            logger.debug("\n %s", pformat(output, indent=2))
+            # when python 3.7 support stops add sort_dicts=False,
             return output
         else:
             fields = list(set(output.keys()))

invoice2data-0.4.3/src/invoice2data/extract/loader.py

@@ -0,0 +1,109 @@
+"""
+This module abstracts templates for invoice providers.
+
+Templates are initially read from .yml or .json files and then kept as class.
+"""
+
+import os
+import json
+try:
+    from yaml import load, YAMLError, CSafeLoader as SafeLoader
+except ImportError:  # pragma: no cover
+    from yaml import load, SafeLoader, YAMLError
+import pkg_resources
+from logging import getLogger
+from .invoice_template import InvoiceTemplate
+import codecs
+
+logger = getLogger(__name__)
+
+
+def read_templates(folder=None):
+    """
+    Load yaml templates from template folder. Return list of dicts.
+
+    Use built-in templates if no folder is set.
+
+    Parameters
+    ----------
+    folder : str
+        user defined folder where they stores their files, if None uses built-in templates
+
+    Returns
+    -------
+    output : Instance of `InvoiceTemplate`
+        template which match based on keywords
+
+    Examples
+    --------
+
+    >>> read_template("home/duskybomb/invoice-templates/")
+    InvoiceTemplate([('issuer', 'OYO'), ('fields', {'amount': 'Grand Total\\s+Rs (\\d+)',
+    'date': 'Date:\\s(\\d{1,2}\\/\\d{1,2}\\/\\d{1,4})', 'invoice_number': '([A-Z0-9]+)\\s+Cash at Hotel'}),
+    ('keywords', ['OYO', 'Oravel', 'Stays']), ('options', {'currency': 'INR', 'decimal_separator': '.'}),
+    ('template_name', 'com.oyo.invoice.yml'), ('exclude_keywords', [])])
+
+    After reading the template you can use the result as an instance of `InvoiceTemplate` to extract fields from
+    `extract_data()`
+
+    >>> my_template = InvoiceTemplate([('issuer', 'OYO'), ('fields', {'amount': 'Grand Total\\s+Rs (\\d+)',
+    'date': 'Date:\\s(\\d{1,2}\\/\\d{1,2}\\/\\d{1,4})', 'invoice_number': '([A-Z0-9]+)\\s+Cash at Hotel'}),
+    ('keywords', ['OYO', 'Oravel', 'Stays']), ('options', {'currency': 'INR', 'decimal_separator': '.'}),
+    ('template_name', 'com.oyo.invoice.yml'), ('exclude_keywords', [])])
+    >>> extract_data("invoice2data/test/pdfs/oyo.pdf", my_template, pdftotext)
+    {'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087',
+    'currency': 'INR', 'desc': 'Invoice IBZY2087 from OYO'}
+
+    """
+
+    output = []
+
+    if folder is None:
+        folder = pkg_resources.resource_filename(__name__, "templates")
+
+    for path, subdirs, files in os.walk(folder):
+        for name in sorted(files):
+            with codecs.open(
+                os.path.join(path, name), encoding="utf-8"
+            ) as template_file:
+                if name.endswith(".yml"):
+                    try:
+                        tpl = load(template_file.read(), Loader=SafeLoader)
+                    except YAMLError as error:
+                        logger.warning("Failed to load %s template:\n%s", name, error)
+                        continue
+                else:
+                    try:
+                        tpl = json.loads(template_file.read())
+                    except ValueError as error:
+                        logger.warning("json Loader Failed to load %s template:\n%s", name, error)
+            tpl["template_name"] = name
+
+            # Test if all required fields are in template
+            if "keywords" not in tpl.keys():
+                logger.warning(
+                    "Failed to load template %s Missing mandatory 'keywords' field.",
+                    name,
+                )
+                continue
+
+            # Convert keywords to list, if only one
+            if not isinstance(tpl["keywords"], list):
+                tpl["keywords"] = [tpl["keywords"]]
+
+            # Set excluded_keywords as empty list, if not provided
+            if "exclude_keywords" not in tpl.keys():
+                tpl["exclude_keywords"] = []
+
+            # Convert excluded_keywords to list, if only one
+            if not isinstance(tpl["exclude_keywords"], list):
+                tpl["exclude_keywords"] = [tpl["exclude_keywords"]]
+
+            if "priority" not in tpl.keys():
+                tpl["priority"] = 5
+
+            output.append(InvoiceTemplate(tpl))
+
+    logger.info("Loaded %d templates from %s", len(output), folder)
+
+    return output

{invoice2data-0.4.1 → invoice2data-0.4.3}/src/invoice2data/extract/parsers/__interface__.py

@@ -1,5 +1,3 @@
-# SPDX-License-Identifier: MIT
-
 """
 Interface for fields parsers.
 

{invoice2data-0.4.1 → invoice2data-0.4.3}/src/invoice2data/extract/parsers/lines.py

@@ -5,9 +5,9 @@ Initial work and maintenance by Holger Brunn @hbrunn
 """
 
 import re
-import logging
+from logging import getLogger
 
-logger = logging.getLogger(__name__)
+logger = getLogger(__name__)
 
 DEFAULT_OPTIONS = {"line_separator": r"\n"}
 
@@ -23,8 +23,14 @@ def parse_line(patterns, line):
 
 def parse_block(template, field, settings, content):
     # Validate settings
-    assert "line" in settings, "Line regex missing"
-
+    assert "line" in settings, (
+        "Error in Template %s Line regex missing" % template["template_name"]
+    )
+
+    logger.debug(
+        "START lines block content ========================\n%s", content
+    )
+    logger.debug("END lines block content ==========================")
     lines = []
     current_row = {}
 
@@ -91,7 +97,7 @@ def parse_block(template, field, settings, content):
                     skip_line_results = [re.search(settings["skip_line"], line)]
                 if any(skip_line_results):
                     # There was at least one match to a skip_line
-                    logger.debug("skip_line match on *%s*", line)
+                    logger.debug("skip_line match on \ns*%s*", line)
                     continue
             # If none of those have continued the loop, check if this is just a normal line
             match = parse_line(settings["line"], line)
@@ -101,7 +107,7 @@ def parse_block(template, field, settings, content):
                 current_row = parse_current_row(match, current_row)
                 continue
         # If the line doesn't match anything, log and continue to next line
-        logger.debug("ignoring *%s* because it doesn't match anything", line)
+        logger.debug("The following line doesn't match anything:\n*%s*", line)
     if current_row:
         # All lines processed, so append whatever the final current_row was to output
         lines.append(current_row)
@@ -115,14 +121,18 @@ def parse_block(template, field, settings, content):
     return lines
 
 
-def parse(template, field, _settings, content):
+def parse_by_rule(template, field, rule, content):
     # First apply default options.
     settings = DEFAULT_OPTIONS.copy()
-    settings.update(_settings)
+    settings.update(rule)
 
     # Validate settings
-    assert "start" in settings, "Lines start regex missing"
-    assert "end" in settings, "Lines end regex missing"
+    assert "start" in settings, (
+        "Error in Template %s Lines start regex missing" % template["template_name"]
+    )
+    assert "end" in settings, (
+        "Error in Template %s Lines end regex missing" % template["template_name"]
+    )
 
     blocks_count = 0
     lines = []
@@ -131,12 +141,13 @@ def parse(template, field, _settings, content):
     while True:
         start = re.search(settings["start"], content)
         if not start:
+            logger.debug("Failed to find lines block start")
             break
         content = content[start.end():]
 
         end = re.search(settings["end"], content)
         if not end:
-            logger.warning("Failed to find lines block end")
+            logger.debug("Failed to find lines block end")
             break
 
         blocks_count += 1
@@ -152,6 +163,25 @@ def parse(template, field, _settings, content):
     return lines
 
 
+def parse(template, field, settings, content):
+    if "rules" in settings:
+        # One field can have multiple sets of line-parsing rules
+        rules = settings['rules']
+    else:
+        # Original syntax stored line-parsing rules in top field YAML object
+        keys = ('start', 'end', 'line', 'first_line', 'last_line', 'skip_line', 'types')
+        rules = [{k: v for k, v in settings.items() if k in keys}]
+
+    lines = []
+    for i, rule in enumerate(rules):
+        logger.debug("Testing Rules set #%s", i)
+        new_lines = parse_by_rule(template, field, rule, content)
+        if new_lines is not None:
+            lines += new_lines
+
+    return lines
+
+
 def parse_current_row(match, current_row):
     # Parse the current row data
     for field, value in match.groupdict().items():

{invoice2data-0.4.1 → invoice2data-0.4.3}/src/invoice2data/extract/parsers/regex.py

@@ -1,3 +1,5 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
 # SPDX-License-Identifier: MIT
 
 """
@@ -32,13 +34,15 @@ def parse(template, field, settings, content, legacy=False):
     for regex in regexes:
         if not isinstance(regex, str):
             logger.warning("Field \"%s\" regex is not a string (%s)", field, str(regex))
+
             continue
         matches = re.findall(regex, content)
-        logger.debug("field=%s | regex=%s | matches=%s", field, settings["regex"], matches)
+        logger.debug("field=\033[1m\033[93m%s\033[0m | regex=\033[36m%s\033[0m | matches=\033[1m\033[92m%s\033[0m"
+                     , field, settings["regex"], matches)
         if matches:
             for match in matches:
                 if isinstance(match, tuple):
-                    logger.warning("Regex can't contain multiple capturing groups (\"" + regex + "\")")
+                    logger.warning("Regex can't contain multiple capturing groups %s", regex)
                     return None
             result += matches
 
@@ -47,15 +51,28 @@ def parse(template, field, settings, content, legacy=False):
             result[k] = template.coerce_type(v, settings["type"])
 
     if "group" in settings:
+        result = list(filter(None, result))
         if settings["group"] == "sum":
             result = sum(result)
+        elif settings["group"] == "min":
+            result = min(result)
+        elif settings["group"] == "max":
+            result = max(result)
+        elif settings["group"] == "first":
+            result = result[0]
+        elif settings["group"] == "last":
+            result = result[-1]
+        elif settings["group"] == "join":
+            result = " ".join(str(v) for v in result)
         else:
-            logger.warning("Unsupported grouping method: " + settings["group"])
+            logger.warning("Unsupported grouping method: %s", settings["group"])
             return None
     else:
+
         # Remove duplicates maintaining the order by default (it's more
         # natural). Don't do that for legacy parsing to keep backward
         # compatibility.
+
         if legacy:
             result = list(set(result))
         else:

{invoice2data-0.4.1 → invoice2data-0.4.3}/src/invoice2data/extract/parsers/static.py

@@ -4,9 +4,9 @@
 Pseudo-parser returning a static (predefined) value
 """
 
-import logging
+from logging import getLogger
 
-logger = logging.getLogger(__name__)
+logger = getLogger(__name__)
 
 
 def parse(template, field, settings, content):
@@ -14,6 +14,6 @@ def parse(template, field, settings, content):
         logger.warning("Field \"%s\" doesn't have static value specified", field)
         return None
 
-    logger.debug("field=%s | value=%s", field, settings["value"])
+    logger.debug("field=%s | value=['%s']", field, settings["value"])
 
     return settings["value"]