csvpath 0.0.476__tar.gz → 0.0.478__tar.gz

{csvpath-0.0.476 → csvpath-0.0.478}/PKG-INFO

@@ -1,15 +1,18 @@
 Metadata-Version: 2.1
 Name: csvpath
-Version: 0.0.476
+Version: 0.0.478
 Summary: A declarative language for data extraction and validation of CSV files
 Author: David Kershaw
 Author-email: dk107dk@hotmail.com
-Requires-Python: >=3.12,<4.0
+Requires-Python: >=3.9,<4.0
 Classifier: Development Status :: 3 - Alpha
 Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: File Formats
 Classifier: Topic :: Office/Business :: Financial :: Spreadsheet
@@ -177,12 +180,11 @@ This is a very basic programmatic use of CsvPath.
 ```python
     path = CsvPath()
     path.parse("""
-                    $test.csv[5-25]
-                    [
-                        #0=="Frog"
-                        @lastname.onmatch="Bats"
-                        count()==2
-                    ]
+            $test.csv[5-25][
+                #firstname == "Frog"
+                @lastname.onmatch = "Bat"
+                count() == 2
+            ]
     """)
 
     for i, line in enumerate( path.next() ):
@@ -193,17 +195,16 @@ This is a very basic programmatic use of CsvPath.
 The csvpath says:
 - Open test.csv
 - Scan lines 5 through 25
-- Match the second time we see a line where the first header equals `Frog` and set the variable called  `lastname` to "Bats"
+- Match the second time we see a line where the first header equals `Frog` and set the variable called  `lastname` to "Bat"
 
 Another path that does the same thing a bit more simply might look like:
 
 ```bash
-    $test[5-25]
-        [
-            #0=="Frog"
-            @lastname.onmatch="Bats"
-            count()==2 -> print( "$.csvpath.match_count: $.csvpath.line")
-        ]
+    $test[5-25][
+        #firstname == "Frog"
+        @lastname.onmatch = "Bat"
+        count()==2 -> print( "$.csvpath.match_count: $.csvpath.line")
+    ]
 ```
 
 In this case, we're using the "when" operator, `->`, to determine when to print.
@@ -214,12 +215,14 @@ There are a small number of configuration options. Read <a href='https://github.
 
 ## The print function
 
-Before we get into the details of the scanning and matching parts of paths, including all the functions, let's look at print. The `print` function has several important uses, including:
+Before we get into the details of scanning and matching, let's look at what CsvPath can print. The `print` function has several important uses, including:
 
-- Debugging csvpaths
 - Validating CSV files
+- Debugging csvpaths
 - Creating new CSV files based on an existing file
 
+You can <a href='https://github.com/dk107dk/csvpath/blob/main/docs/printing.md'>read more about the mechanics of printing here</a>.
+
 <a name="validating"></a>
 ### Validating CSV
 
@@ -250,7 +253,12 @@ This csvpath reorders the headers of the test file at `tests/test_resources/test
 # Comments
 CsvPaths have file scanning instructions, match components, and comments. Comments exist at the top level, outside the CsvPath's brackets, as well as in the matching part of the path. Comments within the match part are covered below.
 
-Comments outside the csvpath can contribute to a collection of metadata fields associated with a csvpath. A comment starts and ends with a `~` character. Within the comment, any word that has a colon after it is considered a metadata key. The metadata value is anything following the key up till a new metadata key word is seen or the comment ends.
+As well as documentation, comments outside the csvpath can:
+- Contribute to a collection of metadata fields associated with a csvpath
+- Switch on/off certain validation settings
+- Set the identity of a csvpath within a group of csvpaths
+
+A comment starts and ends with a `~` character. Within the comment, any word that has a colon after it is considered a metadata key. The metadata value is anything following the key up till a new metadata key word is seen or the comment ends.
 
 For example, this comment says that the csvpath has the name `Order Validation`:
 
@@ -263,6 +271,7 @@ For example, this comment says that the csvpath has the name `Order Validation`:
 
 The name `Order Validation` is available in CsvPath's `metadata` property along with the developer's name. You can use any metadata keys you like. All the metadata is available during and after a run, giving you an easy way to name, describe, attribute, etc. your csvpaths.
 
+You can <a href='https://github.com/dk107dk/csvpath/blob/main/docs/comments.md'>read more about comments and metadata here</a>.
 
 <a name="scanning"></a>
 # Scanning
@@ -427,23 +436,14 @@ Because of this nuanced approach to errors, the library will tend to raise data
 <a name="examples"></a>
 ## More Examples
 
-```bash
-    [ exists(#common_name) #0=="field" @tail.onmatch=end() not(in(@tail, 'short|medium')) ]
-```
-
-In the path above, the rules applied are:
-- The exists test of `#common_name` checks if the column with the header "common_name" has a value. Headers are named for whatever values are found in the 0th row. They indicate a column in the row being checked for match.
-- `#2` means the 3rd column, counting from 0
-- Functions and column references are ANDed together
-- `@tail` creates a variable named "tail" and sets it to the value of the last column if all else matches
-- Functions can contain functions, equality tests, and/or terms.
-
 There are more examples scattered throughout the documentation. Good places to look include:
 
+- Here are a few <a href='https://github.com/dk107dk/csvpath/blob/main/docs/examples.md'>more real-looking examples</a>
+- Try the Getting Started examples on <a href="https://www.csvpath.org">https://www.csvpath.org</a>
 - The individual <a href='https://github.com/dk107dk/csvpath/blob/main/docs/functions.md'>function descriptions</a>
-- The <a href='https://github.com/dk107dk/csvpath/tree/main/tests'>unit tests</a> _(not realistic, but a good source of ideas)_
-- A few <a href='https://github.com/dk107dk/csvpath/blob/main/docs/examples.md'>more real-looking examples</a>
-- The Getting Started examples on <a href="https://www.csvpath.org">https://www.csvpath.org</a>
+- The <a href='https://github.com/dk107dk/csvpath/tree/main/tests'>unit tests</a> and <a href='https://github.com/dk107dk/csvpath/tree/main/tests/grammar/match'>their match parts</a> are not realistic, but a good source of ideas.
+
+To create example CsvPaths from your own data, try <a href='https://autogen.csvpath.org'>CsvPath AutoGen</a>. The huge caveat is that AutoGen uses AI so your results will not be perfect. You will need to adjust, polish, and test them.
 
 <a name="grammar"></a>
 ## Grammar
@@ -457,9 +457,11 @@ Visit <a href="https://www.csvpath.org">https://www.csvpath.org</a>
 
 # Sponsors
 
-<a href='https://www.atestaanalytics.com/' ><img width="25%" src="https://raw.githubusercontent.com/dk107dk/csvpath/main/docs/images/logo-wordmark-white-on-black-trimmed-padded.png" alt="Atesta Analytics"/></a>
-
-
+<a href='https://www.atestaanalytics.com/' >
+<img width="25%" src="https://raw.githubusercontent.com/dk107dk/csvpath/main/docs/images/logo-wordmark-white-on-black-trimmed-padded.png" alt="Atesta Analytics"/></a>
+    <a href='https://www.datakitchen.io/'>
+<img src="https://datakitchen.io/wp-content/uploads/2020/10/logo.svg"
+style='width:160px; position:relative;bottom:-5px;left:15px' alt="DataKitchen" id="logo" data-height-percentage="45"></a>
 
 
 

{csvpath-0.0.476 → csvpath-0.0.478}/README.md

@@ -146,12 +146,11 @@ This is a very basic programmatic use of CsvPath.
 ```python
     path = CsvPath()
     path.parse("""
-                    $test.csv[5-25]
-                    [
-                        #0=="Frog"
-                        @lastname.onmatch="Bats"
-                        count()==2
-                    ]
+            $test.csv[5-25][
+                #firstname == "Frog"
+                @lastname.onmatch = "Bat"
+                count() == 2
+            ]
     """)
 
     for i, line in enumerate( path.next() ):
@@ -162,17 +161,16 @@ This is a very basic programmatic use of CsvPath.
 The csvpath says:
 - Open test.csv
 - Scan lines 5 through 25
-- Match the second time we see a line where the first header equals `Frog` and set the variable called  `lastname` to "Bats"
+- Match the second time we see a line where the first header equals `Frog` and set the variable called  `lastname` to "Bat"
 
 Another path that does the same thing a bit more simply might look like:
 
 ```bash
-    $test[5-25]
-        [
-            #0=="Frog"
-            @lastname.onmatch="Bats"
-            count()==2 -> print( "$.csvpath.match_count: $.csvpath.line")
-        ]
+    $test[5-25][
+        #firstname == "Frog"
+        @lastname.onmatch = "Bat"
+        count()==2 -> print( "$.csvpath.match_count: $.csvpath.line")
+    ]
 ```
 
 In this case, we're using the "when" operator, `->`, to determine when to print.
@@ -183,12 +181,14 @@ There are a small number of configuration options. Read <a href='https://github.
 
 ## The print function
 
-Before we get into the details of the scanning and matching parts of paths, including all the functions, let's look at print. The `print` function has several important uses, including:
+Before we get into the details of scanning and matching, let's look at what CsvPath can print. The `print` function has several important uses, including:
 
-- Debugging csvpaths
 - Validating CSV files
+- Debugging csvpaths
 - Creating new CSV files based on an existing file
 
+You can <a href='https://github.com/dk107dk/csvpath/blob/main/docs/printing.md'>read more about the mechanics of printing here</a>.
+
 <a name="validating"></a>
 ### Validating CSV
 
@@ -219,7 +219,12 @@ This csvpath reorders the headers of the test file at `tests/test_resources/test
 # Comments
 CsvPaths have file scanning instructions, match components, and comments. Comments exist at the top level, outside the CsvPath's brackets, as well as in the matching part of the path. Comments within the match part are covered below.
 
-Comments outside the csvpath can contribute to a collection of metadata fields associated with a csvpath. A comment starts and ends with a `~` character. Within the comment, any word that has a colon after it is considered a metadata key. The metadata value is anything following the key up till a new metadata key word is seen or the comment ends.
+As well as documentation, comments outside the csvpath can:
+- Contribute to a collection of metadata fields associated with a csvpath
+- Switch on/off certain validation settings
+- Set the identity of a csvpath within a group of csvpaths
+
+A comment starts and ends with a `~` character. Within the comment, any word that has a colon after it is considered a metadata key. The metadata value is anything following the key up till a new metadata key word is seen or the comment ends.
 
 For example, this comment says that the csvpath has the name `Order Validation`:
 
@@ -232,6 +237,7 @@ For example, this comment says that the csvpath has the name `Order Validation`:
 
 The name `Order Validation` is available in CsvPath's `metadata` property along with the developer's name. You can use any metadata keys you like. All the metadata is available during and after a run, giving you an easy way to name, describe, attribute, etc. your csvpaths.
 
+You can <a href='https://github.com/dk107dk/csvpath/blob/main/docs/comments.md'>read more about comments and metadata here</a>.
 
 <a name="scanning"></a>
 # Scanning
@@ -396,23 +402,14 @@ Because of this nuanced approach to errors, the library will tend to raise data
 <a name="examples"></a>
 ## More Examples
 
-```bash
-    [ exists(#common_name) #0=="field" @tail.onmatch=end() not(in(@tail, 'short|medium')) ]
-```
-
-In the path above, the rules applied are:
-- The exists test of `#common_name` checks if the column with the header "common_name" has a value. Headers are named for whatever values are found in the 0th row. They indicate a column in the row being checked for match.
-- `#2` means the 3rd column, counting from 0
-- Functions and column references are ANDed together
-- `@tail` creates a variable named "tail" and sets it to the value of the last column if all else matches
-- Functions can contain functions, equality tests, and/or terms.
-
 There are more examples scattered throughout the documentation. Good places to look include:
 
+- Here are a few <a href='https://github.com/dk107dk/csvpath/blob/main/docs/examples.md'>more real-looking examples</a>
+- Try the Getting Started examples on <a href="https://www.csvpath.org">https://www.csvpath.org</a>
 - The individual <a href='https://github.com/dk107dk/csvpath/blob/main/docs/functions.md'>function descriptions</a>
-- The <a href='https://github.com/dk107dk/csvpath/tree/main/tests'>unit tests</a> _(not realistic, but a good source of ideas)_
-- A few <a href='https://github.com/dk107dk/csvpath/blob/main/docs/examples.md'>more real-looking examples</a>
-- The Getting Started examples on <a href="https://www.csvpath.org">https://www.csvpath.org</a>
+- The <a href='https://github.com/dk107dk/csvpath/tree/main/tests'>unit tests</a> and <a href='https://github.com/dk107dk/csvpath/tree/main/tests/grammar/match'>their match parts</a> are not realistic, but a good source of ideas.
+
+To create example CsvPaths from your own data, try <a href='https://autogen.csvpath.org'>CsvPath AutoGen</a>. The huge caveat is that AutoGen uses AI so your results will not be perfect. You will need to adjust, polish, and test them.
 
 <a name="grammar"></a>
 ## Grammar
@@ -426,9 +423,11 @@ Visit <a href="https://www.csvpath.org">https://www.csvpath.org</a>
 
 # Sponsors
 
-<a href='https://www.atestaanalytics.com/' ><img width="25%" src="https://raw.githubusercontent.com/dk107dk/csvpath/main/docs/images/logo-wordmark-white-on-black-trimmed-padded.png" alt="Atesta Analytics"/></a>
-
-
+<a href='https://www.atestaanalytics.com/' >
+<img width="25%" src="https://raw.githubusercontent.com/dk107dk/csvpath/main/docs/images/logo-wordmark-white-on-black-trimmed-padded.png" alt="Atesta Analytics"/></a>
+    <a href='https://www.datakitchen.io/'>
+<img src="https://datakitchen.io/wp-content/uploads/2020/10/logo.svg"
+style='width:160px; position:relative;bottom:-5px;left:15px' alt="DataKitchen" id="logo" data-height-percentage="45"></a>
 
 
 

{csvpath-0.0.476 → csvpath-0.0.478}/csvpath/csvpath.py

@@ -25,6 +25,7 @@ from .util.exceptions import (
     CsvPathsException,
 )
 from .matching.util.exceptions import MatchException
+from csvpath.util.printer import Printer
 
 
 class CsvPathPublic(ABC):
@@ -70,7 +71,7 @@ class CsvPathPublic(ABC):
         matching rows"""
 
 
-class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
+class CsvPath(CsvPathPublic, ErrorCollector, Printer):  # pylint: disable=R0902, R0904
     """CsvPath represents a csvpath string that contains a reference to
     a file, scanning instructions, and rules for matching lines.
     """
@@ -121,7 +122,7 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         # the middle of iterating a file using next(). probably not a
         # good idea, tho.
         #
-        self._AND = True
+        self._AND = True  # pylint: disable=C0103
         #
         # when True the lines that do not match are returned from next()
         # and collect(). this effectively switches CsvPath from being an
@@ -239,6 +240,14 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         # way.
         self._config = config
         #
+        # these settings determine how we report function args validation
+        # errors. e.g. if print(True) the validation check fails because
+        # print() expects a string.
+        #
+        self._log_validation_errors = False
+        self._print_validation_errors = True
+        self._raise_validation_errors = True
+        #
         # there are two logger components one for CsvPath and one for CsvPaths.
         # the default levels are set in config.ini. to change the levels pass LogUtility
         # your component instance and the logging level. e.g.:
@@ -276,19 +285,19 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         self._line_monitor = lm
 
     @property
-    def AND(self) -> bool:
+    def AND(self) -> bool:  # pylint: disable=C0103
         return self._AND
 
     @AND.setter
-    def AND(self, a: bool) -> bool:
+    def AND(self, a: bool) -> bool:  # pylint: disable=C0103
         self._AND = a
 
     @property
-    def OR(self) -> bool:
+    def OR(self) -> bool:  # pylint: disable=C0103
         return not self._AND
 
     @OR.setter
-    def OR(self, a: bool) -> bool:
+    def OR(self, a: bool) -> bool:  # pylint: disable=C0103
         self._AND = not a
 
     @property
@@ -345,18 +354,53 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
     def error_collector(self, error_collector) -> None:
         self._error_collector = error_collector
 
-    def collect_error(self, e: Error) -> None:  # pylint: disable=C0116
+    def collect_error(self, error: Error) -> None:  # pylint: disable=C0116
         #
         # errors must be built and handled in ErrorHandler.
         # here we're just collecting them if collect is
         # selected by our configuration
         #
         if self._error_collector is not None:
-            self._error_collector.collect_error(e)
+            self._error_collector.collect_error(error)
         else:
             if self._errors is None:
                 self._errors = []
-            self._errors.append(e)
+            self._errors.append(error)
+
+    def report_validation_errors(self, msg: str) -> None:
+        if self.print_validation_errors:
+            self.print(msg)
+        elif self.log_validation_errors:
+            self.logger.warning(msg)
+        #
+        # the args class will do the raise if it is told to by the prop
+        #
+
+    def set_validation_error_handling(self, veh) -> None:
+        if veh and veh.find("print") > -1:
+            self._print_validation_errors = True
+        else:
+            self._print_validation_errors = False
+        if veh and veh.find("log") > -1:
+            self._log_validation_errors = True
+        else:
+            self._log_validation_errors = False
+        if veh and veh.find("raise") > -1:
+            self._raise_validation_errors = True
+        else:
+            self._raise_validation_errors = False
+
+    @property
+    def print_validation_errors(self) -> bool:
+        return self._print_validation_errors
+
+    @property
+    def log_validation_errors(self) -> bool:
+        return self._log_validation_errors
+
+    @property
+    def raise_validation_errors(self) -> bool:
+        return self._raise_validation_errors
 
     def add_printer(self, printer) -> None:  # pylint: disable=C0116
         if printer not in self.printers:
@@ -365,12 +409,40 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
     def set_printers(self, printers: List) -> None:  # pylint: disable=C0116
         self.printers = printers
 
+    @property
+    def has_default_printer(self) -> bool:
+        if not self.printers:
+            self.printers = []
+        for i, p in enumerate(self.printers):
+            if isinstance(p, StdOutPrinter):
+                return True
+        return False
+
     def print(self, string: str) -> None:  # pylint: disable=C0116
         for p in self.printers:
             p.print(string)
 
+    def print_to(self, name: str, string: str) -> None:
+        for p in self.printers:
+            p.print_to(name, string)
+
+    @property
+    def last_line(self):
+        if not self.printers or len(self.printers) == 0:
+            return None
+        return self.printers[0].last_line
+
+    @property
+    def lines_printed(self) -> int:
+        if not self.printers or len(self.printers) == 0:
+            return -1
+        self.printers[0].lines_printed
+
     @property
     def is_frozen(self) -> bool:
+        """True if the instance is matching on its last row only to
+        allow last()s to run; in which case, no variable updates
+        are allowed, along with other limitations."""
         return self._freeze_path
 
     @is_frozen.setter
@@ -404,6 +476,7 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         # the comments so we won't find them again
         #
         csvpath = MetadataParser(self).extract_metadata(instance=self, csvpath=csvpath)
+        self.update_settings_from_metadata()
         #
         #
         #
@@ -425,10 +498,10 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         # atm, tho, just create a dry-run copy. in some possible
         # unit tests we may not have a parsable match part.
         #
-        matcher = None
-        if mat:
-            matcher = Matcher(csvpath=self, data=mat, line=None, headers=None)
         if disposably:
+            matcher = None
+            if mat:
+                matcher = Matcher(csvpath=self, data=mat, line=None, headers=None)
             #
             # if the matcher was requested for some reason beyond our own needs
             # we just return it and forget it existed.
@@ -437,7 +510,83 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         if self.scanner.filename is None:
             raise FileException("Cannot proceed without a filename")
         self.get_total_lines_and_headers()
-        return self.scanner
+        return self
+
+    def update_settings_from_metadata(self) -> None:
+        #
+        # settings:
+        #   - logic-mode: AND | OR
+        #   - match-mode: matches | no-matches
+        #   - print-mode: default-off | default-on
+        #   - arg-validation-mode: print | log | raise | quiet
+        #
+        self.update_logic_mode_if()
+        self.update_match_mode_if()
+        self.update_print_mode_if()
+        self.update_arg_validation_mode_if()
+
+    def update_arg_validation_mode_if(self) -> None:
+        if self.metadata and "arg-validation-mode" in self.metadata:
+            # sets arg validation reporting. one or more or none of:
+            #  - print
+            #  - log
+            #  - raise
+            #
+            validation_mode = f"{self.metadata['arg-validation-mode']}".strip()
+            if validation_mode:
+                self.set_validation_error_handling(validation_mode)
+                self.logger.info(
+                    "Setting 'arg-validation-mode': %s",
+                    self.metadata["arg-validation-mode"],
+                )
+
+    def update_logic_mode_if(self) -> None:
+        if self.metadata and "logic-mode" in self.metadata:
+            if f"{self.metadata['logic-mode']}".strip() == "AND":
+                self.AND = True
+            elif f"{self.metadata['logic-mode']}".strip() == "OR":
+                self.AND = False
+            else:
+                self.logger.warning(
+                    "Incorrect metadata field value 'logic-mode': %s",
+                    self.metadata["logic-mode"],
+                )
+
+    def update_match_mode_if(self) -> None:
+        if "match-mode" in self.metadata:
+            if f"{self.metadata['match-mode']}".strip() == "matches":
+                self.collect_when_not_matched = False
+            elif f"{self.metadata['match-mode']}".strip() == "no-matches":
+                self.collect_when_not_matched = True
+            else:
+                self.logger.warning(
+                    "Incorrect metadata field value 'match-mode': %s",
+                    self.metadata["match-mode"],
+                )
+
+    def update_print_mode_if(self) -> None:
+        if "print-mode" in self.metadata:
+            if f"{self.metadata['print-mode']}".strip() == "default-off":
+                remove = -1
+                for i, p in enumerate(self.printers):
+                    if isinstance(p, StdOutPrinter):
+                        remove = i
+                        break
+                if remove >= 0:
+                    del self.printers[remove]
+            elif f"{self.metadata['print-mode']}".strip() == "default-on":
+                done = False
+                for i, p in enumerate(self.printers):
+                    if isinstance(p, StdOutPrinter):
+                        done = True
+                        break
+                if not done:
+                    self.printers.append(StdOutPrinter())
+            else:
+                self.logger.warning(
+                    "Incorrect metadata field value 'print-mode': %s",
+                    self.metadata["print-mode"],
+                )
 
     def parse_named_path(self, name, disposably=False):
         """disposably is True when a Matcher is needed for some purpose other than
@@ -653,7 +802,7 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
                     msg = "Line cannot be None"
                     self.logger.error(msg)
                     raise MatchException(msg)
-                elif len(line) == 0:
+                if len(line) == 0:
                     msg = "Line cannot be len() == 0"
                     self.logger.error(msg)
                     raise MatchException(msg)
@@ -684,6 +833,9 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         # this exception will blow up a standalone CsvPath but should be
         # caught and handled if there is a CsvPaths.
         #
+        # but when would it happen? shouldn't we just let Python's exception
+        # handle it should it really occur?
+        #
         if self.scanner.filename is None:
             raise FileException("There is no filename")
         with open(self.scanner.filename, "r", encoding="utf-8") as file:
@@ -695,6 +847,35 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
                 yield line
         self.finalize()
 
+    """
+    # potential replacement for method above
+    # this is a proposal for having the results of one csvpath feed into another
+    # in memory. the goal being to both shape the data chain-of-responsibility-style
+    # and also to narrow the data for performance gains.
+    #
+    # we would need:
+    #   - csvpaths.chain_result_data
+    #   - named-path added to csvpath metadata early-on
+    #
+    # caching this here for now. jury is out on if it should be added.
+    #
+    def _next_line_new(self) -> List[Any]:
+        self.logger.info("beginning to scan file: %s", self.scanner.filename)
+        if self.csvpath and self.csvpaths.chain_result_data:
+            rs = csvpath.result_manager.get_named_results(self.metadata["named-paths"])
+            for line in rs[len(rs)-1].lines:
+                yield line
+        elif:
+            with open(self.scanner.filename, "r", encoding="utf-8") as file:
+                reader = csv.reader(
+                    file, delimiter=self.delimiter, quotechar=self.quotechar
+                )
+                for line in reader:
+                    self.track_line(line=line)
+                    yield line
+        self.finalize()
+    """
+
     def finalize(self) -> None:
         """clears caches, etc. this is an internal method, but not _ because
         it is part of the lifecycle and we might find a reason to call it
@@ -788,6 +969,8 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
         return False
 
     def raise_match_count_if(self):
+        """if the match count has already been raised earlier in the matching
+        process than the caller we don't raise it; otherwise, we raise."""
         if self._current_match_count == self.match_count:
             self.match_count += 1
         else:
@@ -880,7 +1063,7 @@ class CsvPath(CsvPathPublic, ErrorCollector):  # pylint: disable=R0902, R0904
             self.matcher = Matcher(
                 csvpath=self, data=self.match, line=line, headers=self.headers, myid=h
             )
-            self.matcher._AND = self._AND
+            self.matcher.AND = self._AND
         else:
             self.logger.debug("Resetting and reloading matcher")
             self.matcher.reset()