space-packet-parser 4.0.2__tar.gz → 4.2.0__tar.gz

{space_packet_parser-4.0.2 → space_packet_parser-4.2.0}/CITATION.cff

@@ -1,7 +1,7 @@
 cff-version: 1.2.0
 title: 'space_packet_parser'
 type: software
-version: '4.0.2'
+version: '4.2.0'
 description: A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.
 license: BSD-3-Clause
 abstract: The space_packet_parser Python library is a generalized, configurable packet decoding library for CCSDS telemetry
@@ -11,9 +11,9 @@ abstract: The space_packet_parser Python library is a generalized, configurable
     generator pattern. The binary string may originate from an in-memory bit stream, a buffered file reader opened in
     binary mode, or a python socket object; in every case, a small buffer is used to read chunks of data to ensure that
     memory usage is minimal. The space_packet_parser library supports robust error handling and is capable of skipping malformed
-    packet structures, unidentified packet structures, and can dynamically parse mixed APID packet streams. The
+    packet structures and unidentified packet structures and can dynamically parse mixed APID packet streams. The
     `Parser.generator` object supports keyword arguments for debugging such as the ability to return unraised parsing
-    errors, which contain
+    errors, which contain partial instances of parsed packet data to aid in debugging.
 authors:
 - email: gavin.medley@lasp.colorado.edu
   name: Gavin Medley
@@ -21,9 +21,12 @@ authors:
 - email: michael.chambliss@lasp.colorado.edu
   name: Michael Chambliss
   orcid: "0009-0003-7493-0542"
+- email: greg.lucas@lasp.colorado.edu
+  name: Greg Lucas
+  orcid: "0000-0003-1331-1863"
 maintainers:
 - email: gavin.medley@lasp.colorado.edu
   name: Gavin Medley
   orcid: "0000-0002-3520-9715"
 repository-code: "https://github.com/medley56/space_packet_parser"
-url: "TBD"
+url: "https://space-packet-parser.readthedocs.io"

space_packet_parser-4.2.0/PKG-INFO

@@ -0,0 +1,54 @@
+Metadata-Version: 2.1
+Name: space_packet_parser
+Version: 4.2.0
+Summary: A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.
+Home-page: https://space-packet-parser.readthedocs.io
+License: BSD-3-Clause
+Keywords: ccsds,xtce,space data systems,space packet protocol,packet parsing,lasp,university of colorado,data processing,data extraction,data manipulation,data transformation,data encoding,data decoding,packet inspection,binary data,python
+Author: Gavin Medley
+Author-email: gavin.medley@lasp.colorado.edu
+Maintainer: Gavin Medley
+Maintainer-email: gavin.medley@lasp.colorado.edu
+Requires-Python: >=3.8
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: bitstring (>=4.0.1)
+Project-URL: Repository, https://github.com/medley56/space_packet_parser
+Description-Content-Type: text/markdown
+
+# Space Packet Parser
+
+----------
+[![Test Status](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml/badge.svg)](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml)
+[![Doc Status](https://readthedocs.org/projects/space-packet-parser/badge/?version=latest)](https://readthedocs.org/projects/space-packet-parser/)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7735001.svg)](https://doi.org/10.5281/zenodo.7735001)
+
+Documentation: [https://space-packet-parser.readthedocs.io/en/latest/](https://space-packet-parser.readthedocs.io/en/latest/)
+
+Space Packet Parser is a package for decoding CCSDS telemetry packets according to an XTCE or CSV packet structure definition. 
+It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the 
+XTCE telemetry packet specification.
+
+Resources:
+- [XTCE (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x2g2.pdf)
+- [XTCE Element Description (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x1g2.pdf)
+- [XTCE (Blue Book - Recommended Standard)](https://public.ccsds.org/Pubs/660x0b2.pdf)
+
+## Installation
+```bash
+pip install space_packet_parser
+```
+
+## Missions using Space Packet Parser
+
+[<img src="https://imap.princeton.edu/sites/g/files/toruqf1601/files/imap-mark-hor-multicolor-dark.png" alt="IMAP" height="70"/>](https://imap.princeton.edu/)
+[<img src="https://clarreo-pathfinder.larc.nasa.gov/wp-content/uploads/sites/133/2019/08/clarreo_pathfinder_mission_patch_design_v4_decal_1_24_17.png" alt="CLARREO" height="140"/>](https://clarreo-pathfinder.larc.nasa.gov/)
+[<img src="https://lasp.colorado.edu/libera/files/2021/02/Libera-Logo-HiRes.png" alt="Libera" height="70"/>](https://lasp.colorado.edu/libera/)
+[<img src="https://lasp.colorado.edu/ctim/files/2023/01/CTIM_LOGO_350x100_centered_transparent.png" alt="CTIM-FD" height="70"/>](https://lasp.colorado.edu/ctim/)
+[<img src="https://mms.gsfc.nasa.gov/images/promotional_materials/mms_decal_rgb_4in_trabk_72dpi.png" alt="MMS-FEEPS" height="140"/>](https://lasp.colorado.edu/mms/sdc/public/)
+

space_packet_parser-4.2.0/README.md

@@ -0,0 +1,30 @@
+# Space Packet Parser
+
+----------
+[![Test Status](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml/badge.svg)](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml)
+[![Doc Status](https://readthedocs.org/projects/space-packet-parser/badge/?version=latest)](https://readthedocs.org/projects/space-packet-parser/)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7735001.svg)](https://doi.org/10.5281/zenodo.7735001)
+
+Documentation: [https://space-packet-parser.readthedocs.io/en/latest/](https://space-packet-parser.readthedocs.io/en/latest/)
+
+Space Packet Parser is a package for decoding CCSDS telemetry packets according to an XTCE or CSV packet structure definition. 
+It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the 
+XTCE telemetry packet specification.
+
+Resources:
+- [XTCE (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x2g2.pdf)
+- [XTCE Element Description (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x1g2.pdf)
+- [XTCE (Blue Book - Recommended Standard)](https://public.ccsds.org/Pubs/660x0b2.pdf)
+
+## Installation
+```bash
+pip install space_packet_parser
+```
+
+## Missions using Space Packet Parser
+
+[<img src="https://imap.princeton.edu/sites/g/files/toruqf1601/files/imap-mark-hor-multicolor-dark.png" alt="IMAP" height="70"/>](https://imap.princeton.edu/)
+[<img src="https://clarreo-pathfinder.larc.nasa.gov/wp-content/uploads/sites/133/2019/08/clarreo_pathfinder_mission_patch_design_v4_decal_1_24_17.png" alt="CLARREO" height="140"/>](https://clarreo-pathfinder.larc.nasa.gov/)
+[<img src="https://lasp.colorado.edu/libera/files/2021/02/Libera-Logo-HiRes.png" alt="Libera" height="70"/>](https://lasp.colorado.edu/libera/)
+[<img src="https://lasp.colorado.edu/ctim/files/2023/01/CTIM_LOGO_350x100_centered_transparent.png" alt="CTIM-FD" height="70"/>](https://lasp.colorado.edu/ctim/)
+[<img src="https://mms.gsfc.nasa.gov/images/promotional_materials/mms_decal_rgb_4in_trabk_72dpi.png" alt="MMS-FEEPS" height="140"/>](https://lasp.colorado.edu/mms/sdc/public/)

{space_packet_parser-4.0.2 → space_packet_parser-4.2.0}/pyproject.toml

@@ -1,19 +1,20 @@
 [tool.poetry]
 name = "space_packet_parser"
-version = "4.0.2"
+version = "4.2.0"
 description = "A CCSDS telemetry packet decoding library based on the XTCE packet format description standard."
 license = "BSD-3-Clause"
 readme = "README.md"
 include = ["LICENSE.txt", "CITATION.cff"]
 authors = [
     "Gavin Medley <gavin.medley@lasp.colorado.edu>",
-    "Michael Chambliss <michael.chambliss@lasp.colorado.edu>"
+    "Michael Chambliss <michael.chambliss@lasp.colorado.edu>",
+    "Greg Lucas <greg.lucas@lasp.colorado.edu>",
 ]
 maintainers = [
     "Gavin Medley <gavin.medley@lasp.colorado.edu>"
 ]
 repository = "https://github.com/medley56/space_packet_parser"
-homepage = ""
+homepage = "https://space-packet-parser.readthedocs.io"
 keywords = [
     "ccsds",
     "xtce",
@@ -34,19 +35,20 @@ keywords = [
 ]
 
 [tool.poetry.dependencies]
-python = ">=3.8,<3.12"
-bitstring = ">=3.0.0,<5"
+python = ">=3.8"
+bitstring = ">=4.0.1"
 
 [tool.poetry.group.dev.dependencies]
-pylint = "^2"
-pytest = "^6"
-pytest-randomly = "^3"
-pytest-cov = "^3"
-pyyaml = "^6.0"
-sphinx = "^6.1.3"
-myst-parser = "^1.0.0"
-sphinx-autoapi = "^2.0.1"
-sphinx-rtd-theme = "^1.2.0"
+pylint = "*"
+pytest = "*"
+pytest-randomly = "*"
+pytest-cov = "*"
+pyyaml = "*"
+sphinx = "*"
+myst-parser = "*"
+sphinx-autoapi = "*"
+sphinx-rtd-theme = "*"
+coverage = "*"
 
 [tool.poetry.group.examples]
 optional = true

{space_packet_parser-4.0.2 → space_packet_parser-4.2.0}/space_packet_parser/parser.py

@@ -6,7 +6,7 @@ import io
 import logging
 import socket
 import time
-from typing import BinaryIO, Tuple
+from typing import BinaryIO, Optional, Tuple
 import warnings
 # Installed
 import bitstring
@@ -15,19 +15,19 @@ from space_packet_parser import xtcedef, csvdef
 
 logger = logging.getLogger(__name__)
 
-CcsdsPacketHeaderElement = namedtuple('CcsdsPacketHeaderElement', ['name', 'format_string'])
+CcsdsPacketHeaderElement = namedtuple('CcsdsPacketHeaderElement', ['name', 'nbits'])
 
 CCSDS_HEADER_DEFINITION = [
-    CcsdsPacketHeaderElement('VERSION', 'uint:3'),
-    CcsdsPacketHeaderElement('TYPE', 'uint:1'),
-    CcsdsPacketHeaderElement('SEC_HDR_FLG', 'uint:1'),
-    CcsdsPacketHeaderElement('PKT_APID', 'uint:11'),
-    CcsdsPacketHeaderElement('SEQ_FLGS', 'uint:2'),
-    CcsdsPacketHeaderElement('SRC_SEQ_CTR', 'uint:14'),
-    CcsdsPacketHeaderElement('PKT_LEN', 'uint:16')
+    CcsdsPacketHeaderElement('VERSION', 3),
+    CcsdsPacketHeaderElement('TYPE', 1),
+    CcsdsPacketHeaderElement('SEC_HDR_FLG', 1),
+    CcsdsPacketHeaderElement('PKT_APID', 11),
+    CcsdsPacketHeaderElement('SEQ_FLGS', 2),
+    CcsdsPacketHeaderElement('SRC_SEQ_CTR', 14),
+    CcsdsPacketHeaderElement('PKT_LEN', 16)
 ]
 
-CCSDS_HEADER_LENGTH_BITS = 48
+CCSDS_HEADER_LENGTH_BYTES = 6
 
 Packet = namedtuple('Packet', ['header', 'data'])
 
@@ -35,7 +35,8 @@ Packet = namedtuple('Packet', ['header', 'data'])
 class ParsedDataItem(xtcedef.AttrComparable):
     """Representation of a parsed parameter"""
 
-    def __init__(self, name: str, raw_value: any, unit: str = None, derived_value: float or str = None):
+    def __init__(self, name: str, raw_value: any, unit: str = None, derived_value: float or str = None,
+                 short_description: str = None, long_description: str = None):
         """Constructor
 
         Parameters
@@ -48,6 +49,10 @@ class ParsedDataItem(xtcedef.AttrComparable):
             Raw representation of the parsed value. May be lots of different types but most often an integer
         derived_value : float or str
             May be a calibrated value or an enum lookup
+        short_description : str
+            Parameter short description
+        long_description : str
+            Parameter long description
         """
         if name is None or raw_value is None:
             raise ValueError("Invalid ParsedDataItem. Must define name and raw_value.")
@@ -55,6 +60,8 @@ class ParsedDataItem(xtcedef.AttrComparable):
         self.raw_value = raw_value
         self.unit = unit
         self.derived_value = derived_value
+        self.short_description = short_description
+        self.long_description = long_description
 
     def __repr__(self):
         return (f"{self.__class__.__name__}("
@@ -97,62 +104,30 @@ class PacketParser:
         self.word_size = word_size
 
     @staticmethod
-    def _parse_header(packet_data: bitstring.ConstBitStream,
-                      start_position: int = None,
-                      reset_cursor: bool = False) -> dict:
+    def _parse_header(packet_data: bytes) -> dict:
         """Parses the CCSDS standard header.
 
         Parameters
         ----------
-        packet_data : bitstring.ConstBitStream
-            Binary data stream of packet data.
-        start_position : int
-            Position from which to start parsing. If not provided, will start whenever the cursor currently is.
-        reset_cursor : bool
-            If True, upon parsing the header data, reset the cursor to the original position in the stream.
-            This still applies even if start_position is specified. start_position will be used only for parsing the
-            header and then the cursor will be returned to the location it was at before this function was called.
+        packet_data : bytes
+            6 bytes of binary data.
 
         Returns
         -------
         header : dict
             Dictionary of header items.
         """
-        original_cursor_position = packet_data.pos
-
-        if start_position:
-            packet_data.pos = start_position
-
-        header = {
-            item.name: ParsedDataItem(name=item.name, unit=None, raw_value=packet_data.read(item.format_string))
-            for item in CCSDS_HEADER_DEFINITION
-        }
-
-        if reset_cursor:
-            packet_data.pos = original_cursor_position
-
+        header = {}
+        current_bit = 0
+        for item in CCSDS_HEADER_DEFINITION:
+            header[item.name] = ParsedDataItem(name=item.name,
+                                               unit=None,
+                                               raw_value=_extract_bits(packet_data, current_bit, item.nbits))
+            current_bit += item.nbits
         return header
 
-    @staticmethod
-    def _total_packet_bits_from_pkt_len(pkt_len: int):
-        """Calculate the total length of a CCSDS packet in bits based on the PKT_LEN field in its header.
-
-        Parameters
-        ----------
-        pkt_len : int
-            PKT_LEN value from CCSDS header
-
-        Returns
-        -------
-        : int
-            Length, in bits of the packet
-
-        """
-        # 4.1.3.5.3 The length count C shall be expressed as:
-        #   C = (Total Number of Octets in the Packet Data Field) – 1
-        # We also just reparsed the CCSDS header though as well, so that's an additional 6 octets
-        return 8 * (pkt_len + 1 + 6)
-
+    # DEPRECATED! Remove in next major release along with CSV parser
+    # pylint: disable=inconsistent-return-statements
     def _determine_packet_by_restrictions(self, parsed_header: dict) -> Tuple[str, list]:
         """Examines a dictionary representation of a CCSDS header and determines which packet type applies.
         This packet type must be unique. If the header data satisfies the restrictions for more than one packet
@@ -206,6 +181,7 @@ class PacketParser:
                 "Unable to choose a packet type to parse. "
                 "Note: Restricting container inheritance based on non-header data items is not possible in a "
                 "general way and is not supported by this package.", partial_data=parsed_header)
+        # pylint: enable=inconsistent-return-statements
 
     @staticmethod
     def parse_packet(packet_data: bitstring.ConstBitStream,
@@ -237,7 +213,9 @@ class PacketParser:
                 name=p.name,
                 unit=p.parameter_type.unit,
                 raw_value=parsed_value,
-                derived_value=derived_value
+                derived_value=derived_value,
+                short_description=p.short_description,
+                long_description=p.long_description
             )
 
         def _parse_sequence_container(sc: xtcedef.SequenceContainer):
@@ -250,11 +228,7 @@ class PacketParser:
         parsed_items = {}
         current_container: xtcedef.SequenceContainer = containers[root_container_name]
         while True:
-            for entry in current_container.entry_list:
-                if isinstance(entry, xtcedef.Parameter):
-                    _parse_parameter(entry)
-                elif isinstance(entry, xtcedef.SequenceContainer):
-                    _parse_sequence_container(entry)
+            _parse_sequence_container(current_container)
 
             valid_inheritors = []
             for inheritor_name in current_container.inheritors:
@@ -320,23 +294,25 @@ class PacketParser:
                 name=parameter.name,
                 unit=parameter.parameter_type.unit,
                 raw_value=parsed_value,
-                derived_value=derived_value
+                derived_value=derived_value,
+                short_description=parameter.short_description,
+                long_description=parameter.long_description
             )
 
         return Packet(header=header, data=user_data)
 
     @staticmethod
-    def print_progress(current_bits: int, total_bits: int or None,
+    def print_progress(current_bytes: int, total_bytes: int or None,
                        start_time_ns: int, current_packets: int,
                        end: str = '\r', log: bool = False):
         """Prints a progress bar, including statistics on parsing rate.
 
         Parameters
         ----------
-        current_bits : int
-            Number of bits parsed so far.
-        total_bits : int
-            Number of total bits to parse (if known)
+        current_bytes : int
+            Number of bytes parsed so far.
+        total_bytes : int
+            Number of total bytes to parse (if known)
         current_packets : int
             Number of packets parsed so far.
         start_time_ns : int
@@ -349,26 +325,26 @@ class PacketParser:
         progress_char = "="
         bar_length = 20
 
-        if total_bits is not None:  # If we actually have an endpoint (i.e. not using a socket)
-            percentage = int((current_bits / total_bits) * 100)  # Percent Completed Calculation
-            progress = int((bar_length * current_bits) / total_bits)  # Progress Done Calculation
+        if total_bytes is not None:  # If we actually have an endpoint (i.e. not using a socket)
+            percentage = int((current_bytes / total_bytes) * 100)  # Percent Completed Calculation
+            progress = int((bar_length * current_bytes) / total_bytes)  # Progress Done Calculation
         else:
             percentage = "???"
             progress = 0
 
         elapsed_ns = time.time_ns() - start_time_ns
         delta = dt.timedelta(microseconds=elapsed_ns / 1E3)
-        kbps = int(current_bits * 1E6 / elapsed_ns)
+        kbps = int(current_bytes // 8 * 1E6 / elapsed_ns)
         pps = int(current_packets * 1E9 / elapsed_ns)
         info_str = f"[Elapsed: {delta}, " \
-                   f"Parsed {current_bits} bits ({current_packets} packets) " \
+                   f"Parsed {current_bytes} bytes ({current_packets} packets) " \
                    f"at {kbps}kb/s ({pps}pkts/s)]"
         loadbar = f"Progress: [{progress*progress_char:{bar_length}}]{percentage}% {info_str}"
         print(loadbar, end=end)
         if log is True:
             logger.info(loadbar)
 
-    def generator(self,
+    def generator(self,  # pylint: disable=too-many-branches,too-many-statements
                   binary_data: bitstring.ConstBitStream or BinaryIO or socket.socket,
                   parse_bad_pkts: bool = True,
                   skip_header_bits: int = 0,
@@ -376,7 +352,7 @@ class PacketParser:
                   ccsds_headers_only: bool = False,
                   yield_unrecognized_packet_errors: bool = False,
                   show_progress: bool = False,
-                  buffer_read_size_bytes: int = 4096):
+                  buffer_read_size_bytes: Optional[int] = None):
         """Create and return a Packet generator that reads from a ConstBitStream or a filelike object or a socket.
 
         Creating a generator object to return allows the user to create
@@ -401,6 +377,7 @@ class PacketParser:
         ccsds_headers_only : bool, Optional
             If True, only parses the packet headers (does not use the provided packet definition).
         yield_unrecognized_packet_errors : bool, Optional
+            Default False.
             If False, UnrecognizedPacketTypeErrors are caught silently and parsing continues to the next packet.
             If True, the generator will yield an UnrecognizedPacketTypeError in the event of an unrecognized
             packet. Note: These exceptions are not raised by default but are instead returned so that the generator
@@ -411,28 +388,24 @@ class PacketParser:
             ends.
         buffer_read_size_bytes : int, Optional
             Number of bytes to read from e.g. a BufferedReader or socket binary data source on each read attempt.
-            Default is 4096 bytes.
+            Default is 4096 bytes from a socket, -1 (full read) from a file.
 
         Yields
         -------
-        : Packet or UnrecognizedPacketTypeError
+        Packet or UnrecognizedPacketTypeError
             Generator yields Packet objects containing the parsed packet data for each subsequent packet.
             If yield_unrecognized_packet_errors is True, it will yield an unraised exception object,
             which can be raised or used for debugging purposes.
         """
 
-        def fill_read_buffer(source: bitstring.ConstBitStream or BinaryIO or socket.socket,
-                             buffer: bitstring.BitStream,
-                             read_size_bytes: int) -> int:
-            """Read data from a source and add it to an existing buffer (BitStream).
+        def read_bytes_from_source(source: bitstring.ConstBitStream or BinaryIO or socket.socket,
+                                   read_size_bytes: int) -> int:
+            """Read data from a source and return the bytes read.
 
             Parameters
             ----------
             source : bitstring.ConstBitStream or BinaryIO or socket.socket
                 Source of data.
-            buffer : bitstring.BitStream
-                A reference to a rotating buffer to which the new data is appended. Mutating this changes the data
-                available to the caller by reference so we don't return it.
             read_size_bytes : int
                 Max number of bytes to read from the source per read attempt. For sockets, this should be a small
                 power of 2 (e.g. 4096) due to networking and hardware conventions. For a file or ConstBitStream object
@@ -441,99 +414,120 @@ class PacketParser:
 
             Returns
             -------
-            result : int
-                Number of bits added to the buffer. Note that the buffer may still have nonzero length from previous
-                data even when this returns zero.
+            bytes
+                The bytes that were read from the source.
             """
             if isinstance(source, io.BufferedIOBase):
-                new_bytes = source.read(read_size_bytes)
-                buffer += new_bytes
-                return len(new_bytes)*8
-
+                return source.read(read_size_bytes)
             if isinstance(source, socket.socket):
-                new_bytes = source.recv(read_size_bytes)
-                buffer += new_bytes  # Append BitStream with newly read bytes
-                return len(new_bytes)*8
-
+                return source.recv(read_size_bytes)
             if isinstance(source, bitstring.ConstBitStream):
+                # This either reads read_size_bytes bytes or it just reads to the end of the data
                 new_bits = source[source.pos:source.pos + read_size_bytes * 8]
-                assert isinstance(new_bits, bitstring.ConstBitStream)
-                source.pos += len(new_bits)  # This way we never overshoot the end of the data
-                buffer += new_bits
-                return len(new_bits)
-
-            raise ValueError(f"Unrecognized data source: {source}")
+                source.pos += len(new_bits)  # Set the source.pos to exactly where we read to
+                return new_bits.tobytes()
+            if isinstance(source, io.TextIOWrapper):
+                raise IOError("Packet data file opened in TextIO mode. You must open packet data in binary mode.")
+            raise IOError(f"Unrecognized data source: {source}")
 
         # ========
         # Start of generator
         # ========
         if isinstance(binary_data, bitstring.ConstBitStream):
-            total_length_bits = len(binary_data)
+            total_length_bytes = len(binary_data) // 8
+            if buffer_read_size_bytes is None:
+                # Default to a full read of the bitstream
+                buffer_read_size_bytes = total_length_bytes
             logger.info(
-                f"Creating packet generator from pre-loaded ConstBitStream. Total length is {total_length_bits}")
+                f"Creating packet generator from pre-loaded ConstBitStream. Total length is {total_length_bytes} bytes")
         elif isinstance(binary_data, io.BufferedIOBase):
-            total_length_bits = 8 * binary_data.seek(0, io.SEEK_END)  # This is probably preferable to len
+            if buffer_read_size_bytes is None:
+                # Default to a full read of the file
+                buffer_read_size_bytes = -1
+            total_length_bytes = binary_data.seek(0, io.SEEK_END)  # This is probably preferable to len
             binary_data.seek(0, 0)
             logger.info(f"Creating packet generator from a filelike object, {binary_data}. "
-                        f"Total length is {total_length_bits}bits")
+                        f"Total length is {total_length_bytes} bytes")
         else:  # It's a socket and we don't know how much data we will get
             logger.info("Creating packet generator to read from a socket. Total length to parse is unknown.")
-            total_length_bits = None  # We don't know how long it is
+            total_length_bytes = None  # We don't know how long it is
+            if buffer_read_size_bytes is None:
+                # Default to 4096 bytes from a socket
+                buffer_read_size_bytes = 4096
 
         # ========
         # Packet loop. Each iteration of this loop yields a ParsedPacket object
         # ========
         start_time = time.time_ns()
-        n_bits_parsed = 0  # Keep track of how many bits we have parsed
+        n_bytes_parsed = 0  # Keep track of how many bytes we have parsed
         n_packets_parsed = 0  # Keep track of how many packets we have parsed
-        read_buffer = bitstring.BitStream()  # Not const because it's a rotating buffer
+        read_buffer = b""  # Empty bytes object to start
+        skip_header_bytes = skip_header_bits // 8  # Internally keep track of bytes
+        current_pos = 0  # Keep track of where we are in the buffer
         while True:
-            if total_length_bits and n_bits_parsed == total_length_bits:
+            if total_length_bytes and n_bytes_parsed == total_length_bytes:
                 break  # Exit if we know the length and we've reached it
 
             if show_progress is True:
-                self.print_progress(current_bits=n_bits_parsed, total_bits=total_length_bits,
+                self.print_progress(current_bytes=n_bytes_parsed, total_bytes=total_length_bytes,
                                     start_time_ns=start_time, current_packets=n_packets_parsed)
 
+            if current_pos > 20_000_000:
+                # Only trim the buffer after 20 MB read to prevent modifying
+                # the bitstream and trimming after every packet
+                read_buffer = read_buffer[current_pos:]
+                current_pos = 0
+
             # Fill buffer enough to parse a header
-            while len(read_buffer) < skip_header_bits + CCSDS_HEADER_LENGTH_BITS:
-                result = fill_read_buffer(binary_data, read_buffer,
-                                          read_size_bytes=buffer_read_size_bytes)
+            while len(read_buffer) - current_pos < skip_header_bytes + CCSDS_HEADER_LENGTH_BYTES:
+                result = read_bytes_from_source(binary_data, read_size_bytes=buffer_read_size_bytes)
                 if not result:  # If there is verifiably no more data to add, break
                     break
-
-            read_buffer.pos += skip_header_bits
-            header = self._parse_header(read_buffer, reset_cursor=True)
-            specified_total_packet_length_bits = self._total_packet_bits_from_pkt_len(header['PKT_LEN'].raw_value)
-            n_packets_parsed += 1  # Consider it a counted packet once we've parsed the header
+                read_buffer += result
+            # Skip the header bytes
+            current_pos += skip_header_bytes
+            header_bytes = read_buffer[current_pos:current_pos + CCSDS_HEADER_LENGTH_BYTES]
+            header = self._parse_header(header_bytes)
+
+            # per the CCSDS spec
+            # 4.1.3.5.3 The length count C shall be expressed as:
+            #   C = (Total Number of Octets in the Packet Data Field) – 1
+            n_bytes_data = header['PKT_LEN'].raw_value + 1
+            n_bytes_packet = CCSDS_HEADER_LENGTH_BYTES + n_bytes_data
+
+            # Consider it a counted packet once we've parsed the header
+            # and update the number of bits parsed
+            n_packets_parsed += 1
+            n_bytes_parsed += skip_header_bytes + n_bytes_packet
             if ccsds_headers_only is True:
-                # Trim read buffer (this also reduces memory usage over time for reading a ConstBitStream)
-                n_bits_parsed += skip_header_bits + specified_total_packet_length_bits
-                read_buffer = read_buffer[specified_total_packet_length_bits + skip_header_bits:]
+                # update the current position to the end of the packet data
+                current_pos += n_bytes_packet
                 yield Packet(header=header, data=None)
                 continue
 
             # Based on PKT_LEN fill buffer enough to read a full packet
-            while len(read_buffer) < skip_header_bits + specified_total_packet_length_bits:
-                result = fill_read_buffer(binary_data, read_buffer,
-                                          read_size_bytes=buffer_read_size_bytes)
+            while len(read_buffer) - current_pos < n_bytes_packet:
+                result = read_bytes_from_source(binary_data, read_size_bytes=buffer_read_size_bytes)
                 if not result:  # If there is verifiably no more data to add, break
                     break
-
+                read_buffer += result
+
+            # current_pos is still before the header, so we are reading the entire packet here
+            packet_bytes = read_buffer[current_pos:current_pos + n_bytes_packet]
+            current_pos += n_bytes_packet
+            # Send bitstring data to the parser for now
+            # TODO: Look into parsing the raw bytes directly
+            bitstring_packet = bitstring.ConstBitStream(packet_bytes)
             try:
                 if isinstance(self.packet_definition, xtcedef.XtcePacketDefinition):
-                    packet = self.parse_packet(read_buffer,
+                    packet = self.parse_packet(bitstring_packet,
                                                self.packet_definition.named_containers,
                                                root_container_name=root_container_name,
                                                word_size=self.word_size)
                 else:
                     _, parameter_list = self._determine_packet_by_restrictions(header)
-                    packet = self.legacy_parse_packet(read_buffer, parameter_list, word_size=self.word_size)
+                    packet = self.legacy_parse_packet(bitstring_packet, parameter_list, word_size=self.word_size)
             except UnrecognizedPacketTypeError as e:
-                # Regardless of whether we handle the error, we still want to chop the read_buffer in preparation
-                # for parsing the next packet
-                n_bits_parsed += skip_header_bits + specified_total_packet_length_bits
-                read_buffer = read_buffer[skip_header_bits + specified_total_packet_length_bits:]
                 logger.debug(f"Unrecognized error on packet with APID {header['PKT_APID'].raw_value}'")
                 if yield_unrecognized_packet_errors is True:
                     # Yield the caught exception without raising it (raising ends generator)
@@ -547,22 +541,46 @@ class PacketParser:
                                  f"{packet.header['PKT_LEN'].raw_value}. This might be because the CCSDS header is "
                                  f"incorrectly represented in your packet definition document.")
 
-            actual_length_parsed = read_buffer.pos - skip_header_bits
-
-            if actual_length_parsed != specified_total_packet_length_bits:
+            actual_length_parsed = bitstring_packet.pos // 8
+            if actual_length_parsed != n_bytes_packet:
                 logger.warning(f"Parsed packet length "
-                               f"({actual_length_parsed}b) did not match "
-                               f"length specified in header ({specified_total_packet_length_bits}b). "
-                               f"Updating bit string position to correct position "
+                               f"({actual_length_parsed}B) did not match "
+                               f"length specified in header ({n_bytes_packet}B). "
+                               f"Updating the position to the correct position "
                                "indicated by CCSDS header.")
                 if not parse_bad_pkts:
                     logger.warning("Skipping (not yielding) bad packet because parse_bad_pkts is falsy.")
                     continue
-            n_bits_parsed += skip_header_bits + specified_total_packet_length_bits
-            read_buffer = read_buffer[specified_total_packet_length_bits + skip_header_bits:]
+
             yield packet
 
         if show_progress is True:
-            self.print_progress(current_bits=n_bits_parsed, total_bits=total_length_bits,
+            self.print_progress(current_bytes=n_bytes_parsed, total_bytes=total_length_bytes,
                                 start_time_ns=start_time, current_packets=n_packets_parsed,
                                 end="\n", log=True)
+
+
+def _extract_bits(data: bytes, start_bit: int, nbits: int):
+    """Extract nbits from the data starting from the least significant end.
+    
+    If data = b"abcdefgh", start_bit = 2, nbits = 3, then the bits extracted are
+    "cde" and those are turned into a Python integer and returned.
+
+    Parameters
+    ----------
+    data : bytes
+        Data to extract bits from
+    start_bit : int
+        Starting bit location
+    nbits : int
+        Number of bits to extract
+    
+    Returns
+    -------
+    int
+        Extracted bits as an integer
+    """
+    # Shift the value to the right to get the start bit to the least significant position
+    # Then mask out the bits we want to keep
+    value = int.from_bytes(data, byteorder="big")
+    return (value >> (len(data) * 8 - start_bit - nbits)) & (2 ** nbits - 1)

{space_packet_parser-4.0.2 → space_packet_parser-4.2.0}/space_packet_parser/xtcedef.py

@@ -237,7 +237,7 @@ class Comparison(MatchCriteria):
         if isinstance(required_value, str):
             parsed_value = f"'{parsed_value}'"
             required_value = f"'{required_value}'"
-        return eval(f"{parsed_value} {operator} {required_value}")
+        return eval(f"{parsed_value} {operator} {required_value}")  # pylint: disable=eval-used
 
 
 class Condition(MatchCriteria):
@@ -392,7 +392,7 @@ class Condition(MatchCriteria):
         if isinstance(left_value, str):
             left_value = f"'{left_value}'"
             right_value = f"'{right_value}'"
-        return eval(f"{left_value} {operator} {right_value}")
+        return eval(f"{left_value} {operator} {right_value}")  # pylint: disable=eval-used
 
 
 Anded = namedtuple('Anded', ['conditions', 'ors'])
@@ -463,13 +463,15 @@ class BooleanExpression(MatchCriteria):
             return cls(expression=_parse_ored(element.find('xtce:ORedConditions', ns)))
         raise ValueError(f"Failed to parse {element}")
 
-    def evaluate(self, parsed_data: dict, **kwargs) -> bool:
+    def evaluate(self, parsed_data: dict, current_parsed_value: int or float = None) -> bool:
         """Evaluate the criteria in the BooleanExpression down to a single boolean.
 
         Parameters
         ----------
         parsed_data : dict
             Dictionary of parsed parameter data so far. Used to evaluate truthyness of the match criteria.
+        current_parsed_value : int or float, Optional
+            Current value being parsed.
 
         Returns
         -------
@@ -862,10 +864,12 @@ class ContextCalibrator(AttrComparable):
         if context_match_element.find('xtce:Comparison', ns) is not None:
             return [Comparison.from_match_criteria_xml_element(
                 context_match_element.find('xtce:Comparison', ns), ns)]
-        # TODO: Implement handling of BooleanExpression
-        raise NotImplementedError("ContextCalibrator doesn't contain Comparison on ComparisonList. This probably"
-                                  "means the match criteria is an unsupported type (BooleanExpression or "
-                                  "CustomAlgorithm).")
+        if context_match_element.find('xtce:BooleanExpression', ns) is not None:
+            return [BooleanExpression.from_match_criteria_xml_element(
+                context_match_element.find('xtce:BooleanExpression', ns), ns)]
+        raise NotImplementedError("ContextCalibrator doesn't contain Comparison, ComparisonList, or BooleanExpression. "
+                                  "This probably means the match criteria is an unsupported type "
+                                  "(CustomAlgorithm).")
 
     @classmethod
     def from_context_calibrator_xml_element(cls, element: ElementTree.Element, ns: dict):
@@ -1140,6 +1144,7 @@ class StringDataEncoding(DataEncoding):
         : int or None
             Number of bits to skip after parsing the string
         """
+        # pylint: disable=too-many-branches
         skip_bits_after = 0  # Gets modified if we have a termination character
         if self.fixed_length:
             strlen_bits = self.fixed_length
@@ -1161,10 +1166,7 @@ class StringDataEncoding(DataEncoding):
                 strlen_bits = parsed_data[self.dynamic_length_reference].raw_value
             strlen_bits = int(strlen_bits)
         elif self.termination_character is not None:
-            print(f"hex termination character: {self.termination_character}")
             termination_char_utf8_bytes = bytes.fromhex(self.termination_character)
-            print(f"bytes termination character (utf-8): {termination_char_utf8_bytes}, "
-                  f"len={len(termination_char_utf8_bytes)}")
 
             if self.encoding in ['utf-16-le', 'utf-16-be']:
                 bytes_per_char = 2
@@ -1178,12 +1180,9 @@ class StringDataEncoding(DataEncoding):
             bits_per_byte = 8
             look_ahead_n_bytes = 0
             while look_ahead_n_bytes <= len(packet_data) - packet_data.pos:
-                print(f"looking ahead {look_ahead_n_bytes} bytes")
                 look_ahead = packet_data.peek(f'bytes:{look_ahead_n_bytes}')  # Outputs UTF-8 encoded byte string
                 look_ahead = look_ahead.decode('utf-8').encode(self.encoding)  # Force specified encoding
-                print(f"string so far: {look_ahead}")
                 if termination_char_utf8_bytes in look_ahead:
-                    print('Found termination character.')
                     # Implicit assumption of one termination character in specified encoding
                     tclen_bits = bytes_per_char * bits_per_byte
                     strlen_bits = (look_ahead_n_bytes * bits_per_byte) - tclen_bits
@@ -1199,6 +1198,7 @@ class StringDataEncoding(DataEncoding):
         if self.length_linear_adjuster is not None:
             strlen_bits = self.length_linear_adjuster(strlen_bits)
         return f"bytes:{strlen_bits // 8}", skip_bits_after
+        # pylint: enable=too-many-branches
 
     def parse_value(self, packet_data: bitstring.ConstBitStream, parsed_data: dict, **kwargs):
         """Parse a value from packet data, possibly using previously parsed data items to inform parsing.
@@ -1228,7 +1228,8 @@ class StringDataEncoding(DataEncoding):
         Strings in XTCE can be described in three ways:
 
         1. Using a termination character that marks the end of the string.
-        2. Using a fixed length, which may be derived from referenced parameter either directly or via a discrete lookup table.
+        2. Using a fixed length, which may be derived from referenced parameter either directly or via a discrete
+           lookup table.
         3. Using a leading size field that describes the size of the following string.
 
         Parameters
@@ -1394,7 +1395,10 @@ class IntegerDataEncoding(NumericDataEncoding):
         : cls
         """
         size_in_bits = int(element.attrib['sizeInBits'])
-        encoding = element.attrib['encoding']
+        if 'encoding' in element.attrib:
+            encoding = element.attrib['encoding']
+        else:
+            encoding = "unsigned"
         calibrator = cls.get_default_calibrator(element, ns)
         context_calibrators = cls.get_context_calibrators(element, ns)
         return cls(size_in_bits=size_in_bits, encoding=encoding,
@@ -1431,6 +1435,9 @@ class FloatDataEncoding(NumericDataEncoding):
                              f"Must be one of {self._supported_encodings}.")
         if encoding == 'MIL-1750A':
             raise NotImplementedError("MIL-1750A encoded floats are not supported by this library yet.")
+        if encoding == 'IEEE-754' and size_in_bits not in (16, 32, 64):
+            raise ValueError(f"Invalid size_in_bits value for IEEE-754 FloatDataEncoding, {size_in_bits}. "
+                             "Must be 16, 32, or 64.")
         super().__init__(size_in_bits, encoding=encoding,
                          default_calibrator=default_calibrator, context_calibrators=context_calibrators)
 
@@ -1631,7 +1638,7 @@ class ParameterType(AttrComparable, metaclass=ABCMeta):
 
     @classmethod
     def from_parameter_type_xml_element(cls, element: ElementTree.Element, ns: dict):
-        """Create an IntegerParameterType from an <xtce:IntegerParameterType> XML element.
+        """Create a *ParameterType from an <xtce:*ParameterType> XML element.
 
         Parameters
         ----------
@@ -1642,7 +1649,7 @@ class ParameterType(AttrComparable, metaclass=ABCMeta):
 
         Returns
         -------
-        : IntegerParameterType
+        : ParameterType
         """
         name = element.attrib['name']
         unit = cls.get_units(element, ns)
@@ -1666,6 +1673,7 @@ class ParameterType(AttrComparable, metaclass=ABCMeta):
         -------
         : str or None
         """
+        # Assume we are not parsing a Time Parameter Type, which stores units differently
         units = parameter_type_element.findall('xtce:UnitSet/xtce:Unit', ns)
         # TODO: Implement multiple unit elements for compound unit definitions
         assert len(units) <= 1, f"Found {len(units)} <xtce:Unit> elements in a single <xtce:UnitSet>." \
@@ -1673,6 +1681,7 @@ class ParameterType(AttrComparable, metaclass=ABCMeta):
                                 f"and is not yet supported by this library."
         if units:
             return " ".join([u.text for u in units])
+        # Units are optional so return None if they aren't specified
         return None
 
     @staticmethod
@@ -1693,9 +1702,10 @@ class ParameterType(AttrComparable, metaclass=ABCMeta):
         """
         for data_encoding in [StringDataEncoding, IntegerDataEncoding, FloatDataEncoding, BinaryDataEncoding]:
             # Try to find each type of data encoding element. If we find one, we assume it's the only one.
-            element = parameter_type_element.find(f"xtce:{data_encoding.__name__}", ns)
+            element = parameter_type_element.find(f".//xtce:{data_encoding.__name__}", ns)
             if element is not None:
                 return data_encoding.from_data_encoding_xml_element(element, ns)
+        return None
 
     def parse_value(self, packet_data: bitstring.ConstBitStream, parsed_data: dict, **kwargs):
         """Using the parameter type definition and associated data encoding, parse a value from a bit stream starting
@@ -1868,10 +1878,222 @@ class BinaryParameterType(ParameterType):
         self.encoding = encoding
 
 
-class Parameter:
+class BooleanParameterType(ParameterType):
+    """<xtce:BooleanParameterType>"""
+
+    def __init__(self, name: str, encoding: DataEncoding, unit: str = None):
+        """Constructor that just issues a warning if the encoding is String or Binary"""
+        if isinstance(encoding, (BinaryDataEncoding, StringDataEncoding)):
+            warnings.warn(f"You are encoding a BooleanParameterType with a {type(encoding)} encoding."
+                          f"This is almost certainly a very bad idea because the behavior of string and binary "
+                          f"encoded booleans is not specified in XTCE. e.g. is the string \"0\" truthy?")
+        super().__init__(name, encoding, unit)
+
+    def parse_value(self, packet_data: bitstring.ConstBitStream, parsed_data: dict, **kwargs):
+        """Using the parameter type definition and associated data encoding, parse a value from a bit stream starting
+        at the current cursor position.
+
+        Parameters
+        ----------
+        packet_data : bitstring.ConstBitStream
+            Binary packet data with cursor at the beginning of this parameter's data field.
+        parsed_data : dict
+            Previously parsed data
+
+        Returns
+        -------
+        parsed_value : int
+            Raw encoded value
+        derived_value : str
+            Resulting boolean representation of the encoded raw value
+        """
+        raw, _ = super().parse_value(packet_data, parsed_data, **kwargs)
+        # Note: This behaves very strangely for String and Binary data encodings.
+        # Don't use those for Boolean parameters. The behavior isn't specified well in XTCE.
+        return raw, bool(raw)
+
+
+class TimeParameterType(ParameterType, metaclass=ABCMeta):
+    """Abstract class for time parameter types"""
+
+    def __init__(self, name: str, encoding: DataEncoding, unit: str = None,
+                 epoch: str = None, offset_from: str = None):
+        """Constructor
+
+        Parameters
+        ----------
+        name : str
+            Parameter type name. Usually something like 'MSN__PARAM_Type'.
+        encoding : DataEncoding
+            How the data is encoded. e.g. IntegerDataEncoding, StringDataEncoding, etc.
+        unit : str, Optional
+            String describing the unit for the stored value. Note that if a scale and offset are provided on
+            the Encoding element, the unit applies to the scaled value, not the raw value.
+        epoch : str, Optional
+            String describing the starting epoch for the date or datetime encoded in the parameter.
+            Must be xs:date, xs:dateTime, or one of the following: "TAI", "J2000", "UNIX", "POSIX", "GPS".
+        offset_from : str, Optional
+            Used to reference another time parameter by name. It allows
+            for the stringing together of several dissimilar but related time parameters.
+
+        Notes
+        -----
+        The XTCE spec is not very clear about OffsetFrom or what it is for. We parse it but don't use it for
+        anything.
+        """
+        super().__init__(name, encoding, unit=unit)
+        self.epoch = epoch
+        self.offset_from = offset_from
+
+    @classmethod
+    def from_parameter_type_xml_element(cls, element: ElementTree.Element, ns: dict):
+        """Create a *TimeParameterType from an <xtce:*TimeParameterType> XML element.
+
+        Parameters
+        ----------
+        element : ElementTree.Element
+            The XML element from which to create the object.
+        ns: dict
+            XML namespace dict
+
+        Returns
+        -------
+        : TimeParameterType
+        """
+        name = element.attrib['name']
+        unit = cls.get_units(element, ns)
+        encoding = cls.get_data_encoding(element, ns)
+        encoding_unit_scaler = cls.get_time_unit_linear_scaler(element, ns)
+        if encoding_unit_scaler:
+            encoding.default_calibrator = encoding_unit_scaler
+        epoch = cls.get_epoch(element, ns)
+        offset_from = cls.get_offset_from(element, ns)
+        return cls(name, encoding, unit, epoch, offset_from)
+
+    @staticmethod
+    def get_units(parameter_type_element: ElementTree.Element, ns: dict) -> str or None:
+        """Finds the units associated with a parameter type element and parsed them to return a unit string.
+        We assume only one <xtce:Unit> but this could be extended to support multiple units.
+        See section 4.3.2.2.4 of CCSDS 660.1-G-1
+
+        Parameters
+        ----------
+        parameter_type_element : ElementTree.Element
+            The parameter type element
+        ns : dict
+            XML namespace dictionary
+
+        Returns
+        -------
+        : str or None
+        """
+        encoding_element = parameter_type_element.find('xtce:Encoding', ns)
+        if encoding_element and "units" in encoding_element.attrib:
+            units = encoding_element.attrib["units"]
+            return units
+        # Units are optional so return None if they aren't specified
+        return None
+
+    @staticmethod
+    def get_time_unit_linear_scaler(
+            parameter_type_element: ElementTree.Element, ns: dict) -> PolynomialCalibrator or None:
+        """Finds the linear calibrator associated with the Encoding element for the parameter type element.
+        See section 4.3.2.4.8.3 of CCSDS 660.1-G-2
+
+        Parameters
+        ----------
+        parameter_type_element : ElementTree.Element
+            The parameter type element
+        ns : dict
+            XML namespace dictionary
+
+        Returns
+        -------
+        : PolynomialCalibrator or None
+        """
+        encoding_element = parameter_type_element.find('xtce:Encoding', ns)
+        coefficients = []
+
+        if "offset" in encoding_element.attrib:
+            offset = encoding_element.attrib["offset"]
+            c0 = PolynomialCoefficient(coefficient=float(offset), exponent=0)
+            coefficients.append(c0)
+
+        if "scale" in encoding_element.attrib:
+            scale = encoding_element.attrib["scale"]
+            c1 = PolynomialCoefficient(coefficient=float(scale), exponent=1)
+            coefficients.append(c1)
+        # If we have an offset but not a scale, we need to add a first order term with coefficient 1
+        elif "offset" in encoding_element.attrib:
+            c1 = PolynomialCoefficient(coefficient=1, exponent=1)
+            coefficients.append(c1)
+
+        if coefficients:
+            return PolynomialCalibrator(coefficients=coefficients)
+        # If we didn't find offset nor scale, return None (no calibrator)
+        return None
+
+    @staticmethod
+    def get_epoch(parameter_type_element: ElementTree.Element, ns: dict) -> str or None:
+        """Finds the epoch associated with a parameter type element and parses them to return an epoch string.
+        See section 4.3.2.4.9 of CCSDS 660.1-G-2
+
+        Parameters
+        ----------
+        parameter_type_element : ElementTree.Element
+            The parameter type element
+        ns : dict
+            XML namespace dictionary
+
+        Returns
+        -------
+        : str or None
+            The epoch string, which may be a datetime string or a named epoch such as TAI.
+        """
+        epoch_element = parameter_type_element.find('xtce:ReferenceTime/xtce:Epoch', ns)
+        if epoch_element is not None:
+            return epoch_element.text
+        return None
+
+    @staticmethod
+    def get_offset_from(parameter_type_element: ElementTree.Element, ns: dict) -> str or None:
+        """Finds the parameter referenced in OffsetFrom in a parameter type element and returns the name of the
+        referenced parameter (which must be of type TimeParameterType).
+        See section 4.3.2.4.9 of CCSDS 660.1-G-1
+
+        Parameters
+        ----------
+        parameter_type_element : ElementTree.Element
+            The parameter type element
+        ns : dict
+            XML namespace dictionary
+
+        Returns
+        -------
+        : str or None
+            The named of the referenced parameter.
+        """
+        offset_from_element = parameter_type_element.find('xtce:ReferenceTime/xtce:OffsetFrom', ns)
+        if offset_from_element is not None:
+            return offset_from_element.attrib['parameterRef']
+        return None
+
+
+class AbsoluteTimeParameterType(TimeParameterType):
+    """<xtce:AbsoluteTimeParameterType>"""
+    pass
+
+
+class RelativeTimeParameterType(TimeParameterType):
+    """<xtce:RelativeTimeParameterType>"""
+    pass
+
+
+class Parameter(AttrComparable):
     """<xtce:Parameter>"""
 
-    def __init__(self, name: str, parameter_type: ParameterType):
+    def __init__(self, name: str, parameter_type: ParameterType,
+                 short_description: str = None, long_description: str = None):
         """Constructor
 
         Parameters
@@ -1880,9 +2102,15 @@ class Parameter:
             Parameter name. Typically something like MSN__PARAMNAME
         parameter_type : ParameterType
             Parameter type object that describes how the parameter is stored.
+        short_description : str
+            Short description of parameter as parsed from XTCE
+        long_description : str
+            Long description of parameter as parsed from XTCE
         """
         self.name = name
         self.parameter_type = parameter_type
+        self.short_description = short_description
+        self.long_description = long_description
 
     def __repr__(self):
         module = self.__class__.__module__
@@ -1890,7 +2118,7 @@ class Parameter:
         return f"<{module}.{qualname} {self.name}>"
 
 
-class SequenceContainer:
+class SequenceContainer(AttrComparable):
     """<xtce:SequenceContainer>"""
 
     def __init__(self,
@@ -1951,6 +2179,9 @@ class XtcePacketDefinition:
         '{{{xtce}}}FloatParameterType': FloatParameterType,
         '{{{xtce}}}EnumeratedParameterType': EnumeratedParameterType,
         '{{{xtce}}}BinaryParameterType': BinaryParameterType,
+        '{{{xtce}}}BooleanParameterType': BooleanParameterType,
+        '{{{xtce}}}AbsoluteTimeParameterType': AbsoluteTimeParameterType,
+        '{{{xtce}}}RelativeTimeParameterType': RelativeTimeParameterType,
     }
 
     def __init__(self, xtce_document: str or Path, ns: dict = None):
@@ -1970,6 +2201,7 @@ class XtcePacketDefinition:
         """
         self._sequence_container_cache = {}  # Lookup for parsed sequence container objects
         self._parameter_cache = {}  # Lookup for parsed parameter objects
+        self._parameter_type_cache = {}  # Lookup for parsed parameter type objects
         self.ns = ns or self._default_namespace
         self.type_tag_to_object = {k.format(**self.ns): v for k, v in
                                    self._tag_to_type_template.items()}
@@ -2008,10 +2240,7 @@ class XtcePacketDefinition:
         try:
             base_container, restriction_criteria = self._get_container_base_container(sequence_container)
             base_sequence_container = self.parse_sequence_container_contents(base_container)
-            #base_sequence_container.restriction_criteria = restriction_criteria
             base_container_name = base_sequence_container.name
-            # Prepend the base container. This is necessary for handling multiple inheritance.
-            #entry_list.insert(0, base_sequence_container)
         except ElementNotFoundError:
             base_container_name = None
             restriction_criteria = None
@@ -2028,14 +2257,32 @@ class XtcePacketDefinition:
                 else:
                     parameter_element = self._find_parameter(parameter_name)
                     parameter_type_name = parameter_element.attrib['parameterTypeRef']
-                    parameter_type_element = self._find_parameter_type(parameter_type_name)
-                    parameter_type_class = self.type_tag_to_object[parameter_type_element.tag]
+
+                    # If we've already parsed this parameter type for a different parameter
+                    if parameter_type_name in self._parameter_type_cache:
+                        parameter_type_object = self._parameter_type_cache[parameter_type_name]
+                    else:
+                        parameter_type_element = self._find_parameter_type(parameter_type_name)
+                        parameter_type_class = self.type_tag_to_object[parameter_type_element.tag]
+                        parameter_type_object = parameter_type_class.from_parameter_type_xml_element(
+                            parameter_type_element, self.ns)
+                        self._parameter_type_cache[parameter_type_name] = parameter_type_object  # Add to cache
+
+                    parameter_short_description = parameter_element.attrib['shortDescription'] if (
+                            'shortDescription' in parameter_element.attrib
+                    ) else None
+                    parameter_long_description = parameter_element.find('xtce:LongDescription', self.ns).text if (
+                            parameter_element.find('xtce:LongDescription', self.ns) is not None
+                    ) else None
+
                     parameter_object = Parameter(
                         name=parameter_name,
-                        parameter_type=parameter_type_class.from_parameter_type_xml_element(
-                            parameter_type_element, self.ns))
+                        parameter_type=parameter_type_object,
+                        short_description=parameter_short_description,
+                        long_description=parameter_long_description
+                    )
                     entry_list.append(parameter_object)
-                    self._parameter_cache[parameter_name] = parameter_object
+                    self._parameter_cache[parameter_name] = parameter_object  # Add to cache
             elif entry.tag == '{{{xtce}}}ContainerRefEntry'.format(**self.ns):  # pylint: disable=consider-using-f-string
                 nested_container = self._find_container(name=entry.attrib['containerRef'])
                 entry_list.append(self.parse_sequence_container_contents(nested_container))
@@ -2057,9 +2304,19 @@ class XtcePacketDefinition:
 
     @property
     def named_containers(self):
-        """Property accessor that returns the dict cache of SequenceContainer objecs"""
+        """Property accessor that returns the dict cache of SequenceContainer objects"""
         return self._sequence_container_cache
 
+    @property
+    def named_parameters(self):
+        """Property accessor that returns the dict cache of Parameter objects"""
+        return self._parameter_cache
+
+    @property
+    def named_parameter_types(self):
+        """Property accessor that returns the dict cache of ParameterType objects"""
+        return self._parameter_type_cache
+
     @property
     def flattened_containers(self):
         """Accesses a flattened, generic representation of non-abstract packet definitions along with their

space_packet_parser-4.0.2/PKG-INFO

@@ -1,37 +0,0 @@
-Metadata-Version: 2.1
-Name: space-packet-parser
-Version: 4.0.2
-Summary: A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.
-Home-page: https://github.com/medley56/space_packet_parser
-License: BSD-3-Clause
-Keywords: ccsds,xtce,space data systems,space packet protocol,packet parsing,lasp,university of colorado,data processing,data extraction,data manipulation,data transformation,data encoding,data decoding,packet inspection,binary data,python
-Author: Gavin Medley
-Author-email: gavin.medley@lasp.colorado.edu
-Maintainer: Gavin Medley
-Maintainer-email: gavin.medley@lasp.colorado.edu
-Requires-Python: >=3.8,<3.12
-Classifier: License :: OSI Approved :: BSD License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Requires-Dist: bitstring (>=3.0.0,<5)
-Project-URL: Repository, https://github.com/medley56/space_packet_parser
-Description-Content-Type: text/markdown
-
-# Space Packet Parser - a configurable parser for CCSDS telemetry packets
-This is a package for decoding CCSDS telemetry packets according to an XTCE or CSV packet structure definition. 
-It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the 
-XTCE telemetry packet specification.
-
-Resources:
-- [XTCE (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x2g2.pdf)
-- [XTCE Element Description (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x1g2.pdf)
-- [XTCE (Blue Book - Recommended Standard)](https://public.ccsds.org/Pubs/660x0b2.pdf)
-
-## Installation
-```bash
-pip install space_packet_parser
-```
-

space_packet_parser-4.0.2/README.md

@@ -1,14 +0,0 @@
-# Space Packet Parser - a configurable parser for CCSDS telemetry packets
-This is a package for decoding CCSDS telemetry packets according to an XTCE or CSV packet structure definition. 
-It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the 
-XTCE telemetry packet specification.
-
-Resources:
-- [XTCE (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x2g2.pdf)
-- [XTCE Element Description (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x1g2.pdf)
-- [XTCE (Blue Book - Recommended Standard)](https://public.ccsds.org/Pubs/660x0b2.pdf)
-
-## Installation
-```bash
-pip install space_packet_parser
-```

space_packet_parser-4.0.2/setup.py

@@ -1,30 +0,0 @@
-# -*- coding: utf-8 -*-
-from setuptools import setup
-
-packages = \
-['space_packet_parser']
-
-package_data = \
-{'': ['*']}
-
-install_requires = \
-['bitstring>=3.0.0,<5']
-
-setup_kwargs = {
-    'name': 'space-packet-parser',
-    'version': '4.0.2',
-    'description': 'A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.',
-    'long_description': '# Space Packet Parser - a configurable parser for CCSDS telemetry packets\nThis is a package for decoding CCSDS telemetry packets according to an XTCE or CSV packet structure definition. \nIt is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the \nXTCE telemetry packet specification.\n\nResources:\n- [XTCE (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x2g2.pdf)\n- [XTCE Element Description (Green Book - Informational Report)](https://public.ccsds.org/Pubs/660x1g2.pdf)\n- [XTCE (Blue Book - Recommended Standard)](https://public.ccsds.org/Pubs/660x0b2.pdf)\n\n## Installation\n```bash\npip install space_packet_parser\n```\n',
-    'author': 'Gavin Medley',
-    'author_email': 'gavin.medley@lasp.colorado.edu',
-    'maintainer': 'Gavin Medley',
-    'maintainer_email': 'gavin.medley@lasp.colorado.edu',
-    'url': 'https://github.com/medley56/space_packet_parser',
-    'packages': packages,
-    'package_data': package_data,
-    'install_requires': install_requires,
-    'python_requires': '>=3.8,<3.12',
-}
-
-
-setup(**setup_kwargs)