tdlpackio 2.0.0b1__tar.gz

tdlpackio-2.0.0b1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 NOAA Meteorological Development Laboratory (MDL)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tdlpackio-2.0.0b1/MANIFEST.in

@@ -0,0 +1,15 @@
+include MANIFEST.in
+include setup.py
+include pyproject.toml
+include README.md
+include TDLPACK.md
+include LICENSE
+include VERSION
+include create_docs.sh
+include requirements.txt
+include TdlpackIO.py
+graft docs
+graft src/ext
+graft src/tdlpackio
+prune tests
+global-exclude *.py[cod] *.c *.so *.DS_Store

tdlpackio-2.0.0b1/PKG-INFO

@@ -0,0 +1,99 @@
+Metadata-Version: 2.4
+Name: tdlpackio
+Version: 2.0.0b1
+Summary: Python interface for reading and writing TDLPACK data
+Author-email: Eric Engle <eric.engle@noaa.gov>
+License: MIT
+Project-URL: Documentation, https://noaa-mdl.github.io/tdlpackio
+Project-URL: Respository, https://github.com/NOAA-MDL/tdlpackio
+Keywords: atmospheric-science,data-science,meteorology,mos,numpy,statistics,statpp,weather,weather-data
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Programming Language :: Cython
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: MacOS
+Requires-Python: <3.15,>=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Provides-Extra: xarray
+Requires-Dist: xarray; extra == "xarray"
+Dynamic: license-file
+
+# tdlpackio
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+[![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/)
+[![Python 3.11](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/release/python-3110/)
+[![Python 3.12](https://img.shields.io/badge/python-3.12-blue.svg)](https://www.python.org/downloads/release/python-3120/)
+[![Python 3.13](https://img.shields.io/badge/python-3.13-blue.svg)](https://www.python.org/downloads/release/python-3130/)
+[![Python 3.14](https://img.shields.io/badge/python-3.14-blue.svg)](https://www.python.org/downloads/release/python-3140/)
+
+![Build Linux](https://github.com/NOAA-MDL/tdlpackio/actions/workflows/build_linux.yml/badge.svg)
+![Build macOS](https://github.com/NOAA-MDL/tdlpackio/actions/workflows/build_macos.yml/badge.svg)
+
+![PyPI](https://img.shields.io/pypi/v/tdlpackio?label=pypi%20package)
+![PyPI - Downloads](https://img.shields.io/pypi/dm/tdlpackio)
+
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/version.svg)](https://anaconda.org/conda-forge/tdlpackio)
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/platforms.svg)](https://anaconda.org/conda-forge/tdlpackio)
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/downloads.svg)](https://anaconda.org/conda-forge/tdlpackio)
+
+## Introduction
+
+tdlpackio provides a Python interface for reading and writing TDLPACK files. The NOAA/NWS Meteorological Development Lab ([MDL](https://www.weather.gov/mdl/)) produces [Model Output Statistics (MOS)](https://vlab.noaa.gov/web/mdl/mos) and the [National Blend of Models (NBM)](https://vlab.noaa.gov/web/mdl/nbm). These products are generated from the MDL's in-house MOS-2000 Software System (MOS2K). The MOS2K system defines a GRIB-like data format called TDLPACK.  A brief introduction to TDLPACK files and data format can be found [here](TDLPACK.md). tdlpackio provides a Cython extension module, tdlpacklib, which provides an interface to [libtdlpack](https://github.com/NOAA-MDL/libtdlpack), a Fortran-based library containing a subset of MOS2K subroutines.
+
+## Requirements
+* Python 3.10, 3.11, 3.12, 3.13, 3.14
+* [libtdlpack](https://github.com/NOAA-MDL/libtdlpack) 1.0.0+
+* setuptools
+* cython 3.0+
+* numpy 1.26+
+* pyproj 1.9.6+
+* C compiler: GNU, Intel, and Apple Clang have been tested.
+
+## Required External Libraries
+
+### libtdlpack
+The [libtdlpack](https://github.com/NOAA-MDL/libtdlpack) library is required for tdlpackio.  You will have to build and install this yourself, but this is not difficult.  For macOS users, libtdlpack can be installed via [this Homebrew Tap](https://github.com/NOAA-MDL/homebrew-mdllibs).  If you use the *conda ecosystems, then you can install via `conda install -c conda-forge libtdlpack`.
+
+## Install
+
+Once again, this assumes that libtdlpack has been installed.  If libtdlpack has been installed into a "common" installation path, then it will be found, otherwise define environment variable `TDLPACK_DIR` with the installation path.
+
+* From [PyPI](https://pypi.python.org/pypi/tdlpackio) via pip:
+
+```
+pip install tdlpackio
+```
+* From [conda-forge](https://anaconda.org/conda-forge/tdlpackio) via conda:
+
+```
+conda install -c conda-forge tdlpackio
+```
+
+* From source:
+```shell
+pip install .
+```
+
+## Development
+
+The development evolution of tdlpackio will mainly focus on how best to serve that purpose and its primary user's -- mainly meteorologists, physical scientists, and software developers supporting the missions within NOAA's National Weather Service (NWS) and National Centers for Environmental Prediction (NCEP), and other NOAA organizations.
+
+## Disclaimer
+
+This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an 'as is' basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

tdlpackio-2.0.0b1/README.md

@@ -0,0 +1,65 @@
+# tdlpackio
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+[![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/)
+[![Python 3.11](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/release/python-3110/)
+[![Python 3.12](https://img.shields.io/badge/python-3.12-blue.svg)](https://www.python.org/downloads/release/python-3120/)
+[![Python 3.13](https://img.shields.io/badge/python-3.13-blue.svg)](https://www.python.org/downloads/release/python-3130/)
+[![Python 3.14](https://img.shields.io/badge/python-3.14-blue.svg)](https://www.python.org/downloads/release/python-3140/)
+
+![Build Linux](https://github.com/NOAA-MDL/tdlpackio/actions/workflows/build_linux.yml/badge.svg)
+![Build macOS](https://github.com/NOAA-MDL/tdlpackio/actions/workflows/build_macos.yml/badge.svg)
+
+![PyPI](https://img.shields.io/pypi/v/tdlpackio?label=pypi%20package)
+![PyPI - Downloads](https://img.shields.io/pypi/dm/tdlpackio)
+
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/version.svg)](https://anaconda.org/conda-forge/tdlpackio)
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/platforms.svg)](https://anaconda.org/conda-forge/tdlpackio)
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/tdlpackio/badges/downloads.svg)](https://anaconda.org/conda-forge/tdlpackio)
+
+## Introduction
+
+tdlpackio provides a Python interface for reading and writing TDLPACK files. The NOAA/NWS Meteorological Development Lab ([MDL](https://www.weather.gov/mdl/)) produces [Model Output Statistics (MOS)](https://vlab.noaa.gov/web/mdl/mos) and the [National Blend of Models (NBM)](https://vlab.noaa.gov/web/mdl/nbm). These products are generated from the MDL's in-house MOS-2000 Software System (MOS2K). The MOS2K system defines a GRIB-like data format called TDLPACK.  A brief introduction to TDLPACK files and data format can be found [here](TDLPACK.md). tdlpackio provides a Cython extension module, tdlpacklib, which provides an interface to [libtdlpack](https://github.com/NOAA-MDL/libtdlpack), a Fortran-based library containing a subset of MOS2K subroutines.
+
+## Requirements
+* Python 3.10, 3.11, 3.12, 3.13, 3.14
+* [libtdlpack](https://github.com/NOAA-MDL/libtdlpack) 1.0.0+
+* setuptools
+* cython 3.0+
+* numpy 1.26+
+* pyproj 1.9.6+
+* C compiler: GNU, Intel, and Apple Clang have been tested.
+
+## Required External Libraries
+
+### libtdlpack
+The [libtdlpack](https://github.com/NOAA-MDL/libtdlpack) library is required for tdlpackio.  You will have to build and install this yourself, but this is not difficult.  For macOS users, libtdlpack can be installed via [this Homebrew Tap](https://github.com/NOAA-MDL/homebrew-mdllibs).  If you use the *conda ecosystems, then you can install via `conda install -c conda-forge libtdlpack`.
+
+## Install
+
+Once again, this assumes that libtdlpack has been installed.  If libtdlpack has been installed into a "common" installation path, then it will be found, otherwise define environment variable `TDLPACK_DIR` with the installation path.
+
+* From [PyPI](https://pypi.python.org/pypi/tdlpackio) via pip:
+
+```
+pip install tdlpackio
+```
+* From [conda-forge](https://anaconda.org/conda-forge/tdlpackio) via conda:
+
+```
+conda install -c conda-forge tdlpackio
+```
+
+* From source:
+```shell
+pip install .
+```
+
+## Development
+
+The development evolution of tdlpackio will mainly focus on how best to serve that purpose and its primary user's -- mainly meteorologists, physical scientists, and software developers supporting the missions within NOAA's National Weather Service (NWS) and National Centers for Environmental Prediction (NCEP), and other NOAA organizations.
+
+## Disclaimer
+
+This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an 'as is' basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

tdlpackio-2.0.0b1/TDLPACK.md

@@ -0,0 +1,40 @@
+# TDLPACK
+
+## TDLPACK File Types
+
+Before we describe the TDLPACK data format, one first needs to know the different types of files in which a TDLPACK record can be reside in.  There a two file types: Fortran unformatted (variable record length); Fortran direct-access (fixed record length).  In MOS2K world, these are known as sequential and random access files, respectively.
+
+### Sequential Files
+
+This is a Fortran unformatted, variable record length file.  The TDLPACK record is contained within a Fortran record.  The TDLPACK record preceeded by 8-bytes that contains the record length in bytes of the TDLPACK record.  A TDLPACK sequental file can also contain 2 other type of records: station call letter record; trailer record.  Station call letters in MOS2K of of type CHARACTER*8.
+
+Station call letter record is of the following format in bytes:
+* 1 - 8: Size of the station call letters in bytes (number of stations * 8)
+* 9 - _n_: Station Call Letters
+
+Trailer record is of the following format in bytes:
+* 1 - 8: Record length
+* 9 - _n_:  Trailer record consisting of date/time information
+
+### Random Access Files
+
+This is a Fortran direct access, fixed record length file.  One can think of these files like a book in that there is a table of contents (i.e. key records) that point to where specific TDLPACK records (i.e. data records) exist within the file.  Also, there exists a Master Key Record at the beginning of the file.  In a Random Access File, a TDLPACK record is considered a logical record because it can span multiple (Fortran) physical records.  A TDLPACK random access file can also contain a station call letter record and is the same format as discussed above, however, accessing this record is different.
+
+## TDLPACK Data Format
+
+The following will attempt to briefly explain the TDLPACK format.  Please read the official documentation [here](https://www.weather.gov/media/mdl/TDL_OfficeNote00-1.pdf).  TDLPACK is a big-endian binary data format and is GRIB-like in that it contains an initial 4-character string to identify the data format "TDLP"; indentification sections (Indicator, Product, and Grid); Data section; and an End Section that contains "7777".  A more detailed description of TDLPACK sections can be found in the chapter 5 of the official documentation.  TDLPACK contains the following sections:
+
+* Section 0 - Indictator Section
+* Section 1 - Product Definition Section
+* Section 2 - Grid Definition Section (**NOTE**: Not present when data are vector)
+* Section 3 - Bitmap Section (defined, but never supported in code)
+* Section 4 - Data Section
+* Section 5 - End Section
+
+A TDLPACK record can contain 2 types of data: vector data (i.e. most likely data at "stations"); gridded data (i.e. data at regularly spaced projected gridpoints).  TDLPACK only supports the following map projections:
+
+* Lambert Conformal Conic
+* Polar Stereograhic
+* Mercator
+
+There is no support for geographic grids (i.e. latitude-longitude grids).

tdlpackio-2.0.0b1/TdlpackIO.py

@@ -0,0 +1,384 @@
+"""
+TdlpackIO is a pure Python implementation for performing IO with TDLPACK sequential files
+(i.e. Fortran unformatted files).  Instead of using Fortran for perform IO, we are using
+Python builtins.open() in binary mode.  This allows us to perform stream-based IO for TDLPACK
+files.  When a file is opened for reading, its contents (TDLPACK records) are automatically
+indexed and stored in a dictionary.  The dictionary stores the byte offset the data record;
+the size of the data record; date and lead time; and MOS-2000 ID.
+
+This indexing allow the user to access a TDLPACK sequential file in a random-access nature.
+For example if a users wants to read the 500th record in the file, the first 499 records in
+their entirety do not need to be read.
+"""
+import logging
+import numpy as np
+import os
+import pdb
+import pytdlpack
+import struct
+import sys  
+import warnings
+
+__version__ = pytdlpack.__version__ # Share the version number
+
+_IS_PYTHON3 = sys.version_info.major >= 3
+
+if _IS_PYTHON3:
+    import builtins
+else:
+    import __builtin__ as builtins
+
+ONE_MB = 1048576
+
+class open(object):
+    def __init__(self,filename,mode='r'):
+        """
+        Class Constructor
+
+        Parameters
+        ----------
+
+        **`filename : str`**
+
+        File name.
+
+        **`mode : str, optional, default = 'r'`**
+
+        File handle mode.  The default is open for reading ('r').
+        """
+        if mode == 'r' or mode == 'w':
+            mode = mode+'b'
+        elif mode == 'a':
+            mode = 'wb'
+        self._filehandle = builtins.open(filename,mode=mode,buffering=ONE_MB)
+        self._hasindex = False
+        self._index = {}
+        self.mode = mode
+        self.name = os.path.abspath(filename)
+        self.records = 0
+        self.recordnumber = 0
+        self.size = os.path.getsize(self.name)
+        # Perform indexing on read
+        if 'r' in self.mode:
+            self._get_index()
+
+    def __enter__(self):
+        """
+        """
+        return self
+
+    def __exit__(self,atype,value,traceback):
+        """
+        """
+        self.close()
+
+    def __iter__(self):
+        """
+        """
+        return self
+
+    def __next__(self):
+        """
+        """
+        if self.recordnumber < self.records:
+            return self.read(1)[0]
+        else:
+            raise StopIteration
+
+    def __repr__(self):
+        """
+        """
+        strings = []
+        keys = self.__dict__.keys()
+        for k in keys:
+            if not k.startswith('_'):
+                strings.append('%s = %s\n'%(k,self.__dict__[k]))
+        return ''.join(strings)
+
+    def __getitem__(self,key):
+        """
+        """
+        if isinstance(key,slice):
+            beg, end, inc = key.indices(self.records)
+            self.seek(beg)
+            return [self.record(i+1) for i in range(beg,end,inc)]
+        elif isinstance(key,int):
+            if key == 0: return None
+            self.seek(key)
+            return self.record(key)
+        else:
+            raise KeyError('Key must be an integer record number or a slice')
+
+    def _get_index(self):
+        """
+        Perform indexing of data records.
+        """
+        #pdb.set_trace()
+        # Initialize index dictionary
+        self._index['offset'] = []
+        self._index['size'] = []
+        self._index['type'] = []
+        self._index['date'] = []
+        self._index['lead'] = []
+        self._index['id1'] = []
+        self._index['id2'] = []
+        self._index['id3'] = []
+        self._index['id4'] = []
+        self._index['dims'] = []
+        self._index['linked_station_id_record'] = []
+        _last_station_id_record = 0
+
+        # Iterate
+        while True:
+            try:
+                # First read 4-byte Fortran record header, then read the next
+                # 44 bytes which provides enough information to catalog the
+                # data record.
+                pos = self._filehandle.tell()
+                fortran_header = struct.unpack('>i',self._filehandle.read(4))[0]
+                if fortran_header >= 132:
+                    bytes_to_read = 132
+                else:
+                    bytes_to_read = fortran_header
+                temp = np.frombuffer(self._filehandle.read(bytes_to_read),dtype='>i4')
+                _header = struct.unpack('>4s',temp[2])[0].decode()
+
+                # Check to first 4 bytes of the data record to determine the data
+                # record type.
+                if _header == 'PLDT':
+                    # TDLPACK data record
+                    # Here we create a dimension dictionary per TDLPACK record and store in
+                    # the index.
+                    _dimdict = {}
+                    _pos = 16+temp.tobytes()[16]
+                    if bool(int(bin(temp.tobytes()[17])[-1])):
+                        # Grid
+                        _dimdict['nx'] = struct.unpack('>h',temp.tobytes()[_pos+2:_pos+4])[0]
+                        _dimdict['ny'] = struct.unpack('>h',temp.tobytes()[_pos+4:_pos+6])[0]
+                    else:
+                        # Vector
+                        _dimdict['nsta'] = struct.unpack('>i',temp.tobytes()[_pos+4:_pos+8])[0]
+                    self._index['size'].append(temp[1])
+                    self._index['type'].append('data')
+                    self._index['date'].append(temp[6])
+                    self._index['lead'].append(int(str(temp[9])[-3:]))
+                    self._index['id1'].append(temp[7])
+                    self._index['id2'].append(temp[8])
+                    self._index['id3'].append(temp[9])
+                    self._index['id4'].append(temp[10])
+                    self._index['dims'].append(_dimdict)
+                    self._index['linked_station_id_record'].append(_last_station_id_record)
+                else:
+                    if temp[1] == 24 and temp[6] == 9999:
+                        # Trailer record
+                        self._index['size'].append(temp[1])
+                        self._index['type'].append('trailer')
+                        self._index['date'].append(None)
+                        self._index['lead'].append(None)
+                        self._index['id1'].append(None)
+                        self._index['id2'].append(None)
+                        self._index['id3'].append(None)
+                        self._index['id4'].append(None)
+                        self._index['dims'].append(None)
+                        self._index['linked_station_id_record'].append(_last_station_id_record)
+                    else:
+                        # Station ID record
+                        self._index['size'].append(temp[1])
+                        self._index['type'].append('station')
+                        self._index['date'].append(None)
+                        self._index['lead'].append(None)
+                        self._index['id1'].append(400001000)
+                        self._index['id2'].append(0)  
+                        self._index['id3'].append(0)
+                        self._index['id4'].append(0)
+                        self._index['dims'].append(None)
+                        self._index['linked_station_id_record'].append(_last_station_id_record)
+
+                # At this point we have successfully identified a TDLPACK record from
+                # the file. Increment self.records and position the file pointer to
+                # now read the Fortran trailer.
+                self.records += 1 # Includes trailer records
+                self._filehandle.seek(fortran_header-bytes_to_read,1)
+                fortran_trailer = struct.unpack('>i',self._filehandle.read(4))[0]
+
+                # Check Fortran header and trailer for the record.
+                if fortran_header != fortran_trailer:
+                    raise IOError('Bad Fortran record.')
+
+                # NOTE: The 'offset' key contains the byte position in the file of where
+                # data record begins. A value of 12 is added to consider a 4-byte Fortran
+                # header, 4-byte "trash", and 4-byte ioctet value (already) stored on index.
+                self._index['offset'].append(pos+12) # 4-byte header + 4-byte trash + 4-byte ioctet
+
+                # Hold the record number of the last station ID record
+                if self._index['type'][-1] == 'station':
+                    _last_station_id_record = self.records # This should be OK.
+
+            except(struct.error):
+                self._filehandle.seek(0)
+                break
+
+        self._hasindex = True
+        self.dates = tuple(sorted(set(list(filter(None,self._index['date'])))))
+        self.leadtimes = tuple(sorted(set(list(filter(None,self._index['lead'])))))
+
+    def close(self):
+        """
+        Close the file handle
+        """
+        self._filehandle.close()
+
+    def read(self,num=None,unpack=True):
+        """
+        Read num records from the current position.
+        """
+        #pdb.set_trace()
+        recs = []
+        if num == 0:
+            return recs
+        elif num == 1:
+            reclist = [self.recordnumber+1]
+        elif num > 1:
+            reclist = list(range(self.recordnumber+1,self.recordnumber+1+num))
+        for n in reclist:
+            nn = n-1 # Use this for the self._index referencing
+            kwargs = {}
+            self.seek(n)
+            kwargs['ioctet'] = self._index['size'][nn]
+            kwargs['ipack'] = np.frombuffer(self._filehandle.read(self._index['size'][nn]),dtype='>i4')
+            if self._index['type'][nn] == 'data':
+                kwargs['reference_date'] = self._index['date'][nn]
+                rec = pytdlpack.TdlpackRecord(**kwargs)
+                if unpack: rec.unpack()
+                recs.append(rec)
+            elif self._index['type'][nn] == 'station':
+                kwargs['ipack'] = kwargs['ipack'].byteswap()
+                kwargs['number_of_stations'] = np.int32(kwargs['ioctet']//pytdlpack.NCHAR)
+                rec = pytdlpack.TdlpackStationRecord(**kwargs)
+                if unpack: rec.unpack()
+                recs.append(rec)
+            elif self._index['type'][nn] == 'trailer':
+                recs.append(pytdlpack.TdlpackTrailerRecord(**kwargs))
+            self.recordnumber = n
+        return recs
+    
+    def record(self,rec,unpack=True):
+        """
+        Read the N-th record.
+        """
+        #pdb.set_trace()
+        if rec is None:
+            return None
+        if rec <= 0:
+            warnings.warn("Record numbers begin at 1.") 
+            return None
+        elif rec > self.records:
+            warnings.warn("Not that many records in the file.")
+            return None
+        else:
+            self.seek(rec) # Use the actual record number here.
+            return self.read(1,unpack=unpack)[0]
+
+    def seek(self,offset):
+        """
+        Set the position within the file in units of data records.
+        """
+        #pdb.set_trace()
+        if self._hasindex:
+            if offset == 0:
+                self._filehandle.seek(self._index['offset'][offset])
+                self.recordnumber = offset
+            elif offset > 0:
+                self._filehandle.seek(self._index['offset'][offset-1])
+                self.recordnumber = offset-1
+    
+    def fetch(self,date=None,id=None,lead=None,unpack=True):
+        """
+        Fetch TDLPACK data record by means of date, lead time, id or any combination
+        thereof.
+        """
+        #pdb.set_trace()
+        recs = []
+        idx = None
+        match_count = 0
+
+        # Match by date.
+        if type(date) is not list:
+           if date is None:
+               date = []
+           else:
+               date = [date]
+        if len(date) > 0: match_count += 1
+        for d in date:
+            if d is not None:
+                if idx is None:
+                    idx = np.where(np.array(self._index['date'])==d)[0]
+                else:
+                    idx = np.concatenate((idx,np.where(np.array(self._index['date'])==d)[0]))
+
+        # Match by ID.
+        if id is not None:
+            # Test for type
+            if type(id) is str:
+                # Need all 4 words for now....
+                id = [int(i) for i in list(filter(None,id.split(' ')))]
+                print(id)
+            # Match by MOS ID (all 4 words)
+            match_count += 4
+            allrecs = np.arange(self.records)
+            # ID1
+            if id[0] == -1:
+                idx1 = allrecs
+            elif id[0] >= 0:
+                idx1 = np.where(np.array(self._index['id1'])==id[0])[0]
+            # ID2
+            if id[1] == -1:
+                idx2 = allrecs
+            elif id[1] >= 0:
+                idx2 = np.where(np.array(self._index['id2'])==id[1])[0]
+            # ID3
+            if id[2] == -1:
+                idx3 = allrecs
+            elif id[2] >= 0:
+                idx3 = np.where(np.array(self._index['id3'])==id[2])[0]
+            # ID4
+            if id[3] == -1:
+                idx4 = allrecs
+            elif id[3] >= 0:
+                idx4 = np.where(np.array(self._index['id4'])==id[3])[0]
+
+            if idx is not None:
+                idx = np.concatenate((idx,idx1,idx2,idx3,idx4))
+            else:
+                idx = np.concatenate((idx1,idx2,idx3,idx4))
+
+        # Match by lead times(s).
+        if type(lead) is not list:
+            if lead is None:
+                lead = []
+            else:
+                lead = [lead]
+        if len(lead) > 0: match_count += 1
+        for l in lead:
+            if l is not None:
+                if idx is None:
+                    idx = np.where(np.array(self._index['lead'])==l)[0]
+                else:
+                    idx = np.concatenate((idx,np.where(np.array(self._index['lead'])==l)[0]))
+
+        # Now determine the count of unique index values.  The count needs to match the
+        # value of match_count.  Where this occurs, the index values are extracted.
+        vals,cnts = np.unique(idx,return_counts=True)
+        idx = vals[np.where(cnts==match_count)[0]]
+
+        # Now we iterate over the matching index values and build the list of
+        # records.
+        for i in idx:
+            recs.append(self.record(i+1,unpack=unpack))
+        return recs
+    
+    def tell(self):
+        """
+        Return the position in units of records.
+        """
+        return self.recordnumber

tdlpackio-2.0.0b1/VERSION

@@ -0,0 +1 @@
+2.0.0b1

tdlpackio-2.0.0b1/create_docs.sh

@@ -0,0 +1,9 @@
+#!/bin/sh
+# ----------------------------------------------------------------------------------------
+# Uses real pdoc (https://github.com/mitmproxy/pdoc), not pdoc3
+#
+# pip install pdoc
+# ----------------------------------------------------------------------------------------
+export PYTHONPATH=src
+export VERSION=$(cat ./VERSION)
+python -m pdoc -d numpy --footer-text "tdlpackio v${VERSION}" -o docs tdlpackio

tdlpackio-2.0.0b1/docs/index.html

@@ -0,0 +1,7 @@
+<!doctype html>
+<html>
+<head>
+    <meta charset="utf-8">
+    <meta http-equiv="refresh" content="0; url=./tdlpackio.html"/>
+</head>
+</html>