xradio 0.0.58__tar.gz → 0.0.60__tar.gz

{xradio-0.0.58/src/xradio.egg-info → xradio-0.0.60}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: xradio
-Version: 0.0.58
+Version: 0.0.60
 Summary:  Xarray Radio Astronomy Data IO
 Author-email: Jan-Willem Steeb <jsteeb@nrao.edu>, Federico Montesino Pouzols <pouzols@eso.edu>, Dave Mehringer <dmehring@nrao.edu>, Peter Wortmann <peter.wortmann@skao.int>
 License: BSD 3-Clause License
@@ -108,25 +108,32 @@ Xarray Radio Astronomy Data IO is still in development.
 [![Version Status](https://img.shields.io/pypi/v/xradio.svg)](https://pypi.python.org/pypi/xradio/)
 
 # Installing
-It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed:
+XRADIO can be installed in virtual environments via pip. It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed, for example:
 ```sh
 conda create --name xradio python=3.12 --no-default-packages
 conda activate xradio
 ```
+> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
+
 XRADIO can now be installed using:
 ```sh
 pip install xradio
 ```
-This will also install the minimal dependencies for XRADIO. To install the minimal dependencies and the interactive components (JupyterLab) use:
+This will also install the minimal dependencies for XRADIO.
+
+Note that if only the minimal dependencies are installed, the functionality to convert MSv2 to MSv4 will not be available.
+This requires installing `python-casacore` (also included in the `all` group, see below), or alternatively the
+`casatools` backend, as explained in the [casatools I/O backend guide](docs/source/measurement_set/guides/backends.md).
+
+To install the minimal dependencies and the interactive components (JupyterLab) use:
 ```sh
 pip install "xradio[interactive]"
 ```
+
 To enable conversion from MSv2 to MSv4 use (this only works for Linux):
 ```sh
 pip install "xradio[python-casacore]"
 ```
-> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
-
 To be able to run tests:
 ```sh
 pip install "xradio[test]"
@@ -135,3 +142,10 @@ Multiple-dependencies can be installed using:
 ```sh
 pip install "xradio[interactive,python-casacore,test]"
 ```
+
+To install a more complete set of dependencies:
+```sh
+pip install "xradio[all]"
+```
+This will include the dependencies required to run the interactive Jupyter notebooks, run tests, build documentation,
+and python-casacore to enable MSv2=>MSv4 functionality.

{xradio-0.0.58 → xradio-0.0.60}/README.md

@@ -10,25 +10,32 @@ Xarray Radio Astronomy Data IO is still in development.
 [![Version Status](https://img.shields.io/pypi/v/xradio.svg)](https://pypi.python.org/pypi/xradio/)
 
 # Installing
-It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed:
+XRADIO can be installed in virtual environments via pip. It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed, for example:
 ```sh
 conda create --name xradio python=3.12 --no-default-packages
 conda activate xradio
 ```
+> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
+
 XRADIO can now be installed using:
 ```sh
 pip install xradio
 ```
-This will also install the minimal dependencies for XRADIO. To install the minimal dependencies and the interactive components (JupyterLab) use:
+This will also install the minimal dependencies for XRADIO.
+
+Note that if only the minimal dependencies are installed, the functionality to convert MSv2 to MSv4 will not be available.
+This requires installing `python-casacore` (also included in the `all` group, see below), or alternatively the
+`casatools` backend, as explained in the [casatools I/O backend guide](docs/source/measurement_set/guides/backends.md).
+
+To install the minimal dependencies and the interactive components (JupyterLab) use:
 ```sh
 pip install "xradio[interactive]"
 ```
+
 To enable conversion from MSv2 to MSv4 use (this only works for Linux):
 ```sh
 pip install "xradio[python-casacore]"
 ```
-> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
-
 To be able to run tests:
 ```sh
 pip install "xradio[test]"
@@ -36,4 +43,11 @@ pip install "xradio[test]"
 Multiple-dependencies can be installed using:
 ```sh
 pip install "xradio[interactive,python-casacore,test]"
-```
+```
+
+To install a more complete set of dependencies:
+```sh
+pip install "xradio[all]"
+```
+This will include the dependencies required to run the interactive Jupyter notebooks, run tests, build documentation,
+and python-casacore to enable MSv2=>MSv4 functionality.

{xradio-0.0.58 → xradio-0.0.60}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "xradio"
-version = "v0.0.58"
+version = "v0.0.60"
 description = " Xarray Radio Astronomy Data IO"
 authors = [
     {name = "Jan-Willem Steeb", email="jsteeb@nrao.edu"},

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/_utils/_msv2/conversion.py

@@ -432,8 +432,49 @@ def calc_indx_for_row_split(tb_tool, taql_where):
 
 
 def create_coordinates(
-    xds, in_file, ddi, utime, interval, baseline_ant1_id, baseline_ant2_id, scan_id
-):
+    xds: xr.Dataset,
+    in_file: str,
+    ddi: int,
+    utime: np.ndarray,
+    interval: np.ndarray,
+    baseline_ant1_id: np.ndarray,
+    baseline_ant2_id: np.ndarray,
+    scan_id: np.ndarray,
+) -> tuple[xr.Dataset, int]:
+    """
+    Creates coordinates of a VisibilityXds/SpectrumXds and assigns them to the input
+    correlated dataset.
+
+    Parameters
+    ----------
+    xds :
+        dataset to add the coords to
+    in_file :
+        path to input MSv2
+    ddi :
+        DDI index (row) for this MSv4
+    utime :
+        unique times, for the time coordinate
+    interval :
+        interval col values from the MSv2, for the integration_time attribute
+        of the time coord
+    baseline_ant1_id :
+        ANTENNA1 ids to be used as coord
+    baseline_ant2_id :
+        ANTENNA2 ids to be used as coord
+    scan_id :
+        SCAN_ID values from MSv2, for the scan_name coord
+
+    Returns
+    -------
+    tuple[xr.Dataset, int]
+        A tuple of:
+        - The input dataset with coordinates added and populated with all MSv4 schema
+          attributes.
+        - The MSv2 spectral_window_id of this DDI/MSv4, which is no longer added to
+          the frequency coord but is required to create other secondary xdss (antenna,
+          gain_curve, phase_calibration, system_calibration, field_and_source).
+    """
     coords = {
         "time": utime,
         "baseline_antenna1_id": ("baseline_id", baseline_ant1_id),
@@ -487,6 +528,7 @@ def create_coordinates(
         spw_name = spw_name + "_" + str(spectral_window_id)
 
     xds.frequency.attrs["spectral_window_name"] = spw_name
+    xds.frequency.attrs["spectral_window_intent"] = "UNSPECIFIED"
     msv4_measure = column_description_casacore_to_msv4_measure(
         freq_column_description["REF_FREQUENCY"],
         ref_code=spectral_window_xds["MEAS_FREQ_REF"].data,
@@ -496,7 +538,6 @@ def create_coordinates(
         msv4_measure["units"],
         msv4_measure["observer"],
     )
-    xds.frequency.attrs["spectral_window_id"] = spectral_window_id
 
     # Add if doppler table is present
     # xds.frequency.attrs["doppler_velocity"] =
@@ -534,7 +575,7 @@ def create_coordinates(
         interval, msv4_measure["units"] if msv4_measure else "s"
     )
 
-    return xds
+    return xds, spectral_window_id
 
 
 def find_min_max_times(tb_tool: tables.table, taql_where: str) -> tuple:
@@ -963,6 +1004,7 @@ def convert_and_write_partition(
     phase_cal_interpolate: bool = False,
     sys_cal_interpolate: bool = False,
     compressor: numcodecs.abc.Codec = numcodecs.Zstd(level=2),
+    add_reshaping_indices: bool = False,
     storage_backend="zarr",
     parallel_mode: str = "none",
     overwrite: bool = False,
@@ -999,6 +1041,8 @@ def convert_and_write_partition(
         _description_, by default None
     compressor : numcodecs.abc.Codec, optional
         _description_, by default numcodecs.Zstd(level=2)
+    add_reshaping_indices : bool, optional
+        _description_, by default False
     storage_backend : str, optional
         _description_, by default "zarr"
     parallel_mode : _type_, optional
@@ -1086,7 +1130,7 @@ def convert_and_write_partition(
         scan_id[tidxs, bidxs] = tb_tool.getcol("SCAN_NUMBER")
         scan_id = np.max(scan_id, axis=1)
 
-        xds = create_coordinates(
+        xds, spectral_window_id = create_coordinates(
             xds,
             in_file,
             ddi,
@@ -1154,7 +1198,7 @@ def convert_and_write_partition(
 
         ant_xds = create_antenna_xds(
             in_file,
-            xds.frequency.attrs["spectral_window_id"],
+            spectral_window_id,
             antenna_id,
             feed_id,
             telescope_name,
@@ -1163,9 +1207,7 @@ def convert_and_write_partition(
         logger.debug("Time antenna xds  " + str(time.time() - start))
 
         start = time.time()
-        gain_curve_xds = create_gain_curve_xds(
-            in_file, xds.frequency.attrs["spectral_window_id"], ant_xds
-        )
+        gain_curve_xds = create_gain_curve_xds(in_file, spectral_window_id, ant_xds)
         logger.debug("Time gain_curve xds  " + str(time.time() - start))
 
         start = time.time()
@@ -1175,7 +1217,7 @@ def convert_and_write_partition(
             phase_cal_interp_time = None
         phase_calibration_xds = create_phase_calibration_xds(
             in_file,
-            xds.frequency.attrs["spectral_window_id"],
+            spectral_window_id,
             ant_xds,
             time_min_max,
             phase_cal_interp_time,
@@ -1190,6 +1232,7 @@ def convert_and_write_partition(
             sys_cal_interp_time = None
         system_calibration_xds = create_system_calibration_xds(
             in_file,
+            spectral_window_id,
             xds.frequency,
             ant_xds,
             sys_cal_interp_time,
@@ -1276,7 +1319,7 @@ def convert_and_write_partition(
             create_field_and_source_xds(
                 in_file,
                 field_id,
-                xds.frequency.attrs["spectral_window_id"],
+                spectral_window_id,
                 field_times,
                 is_single_dish,
                 time_min_max,
@@ -1328,6 +1371,12 @@ def convert_and_write_partition(
             else:
                 xds.attrs["type"] = "visibility"
 
+        # Add tidxs and bidxs for testing
+        if add_reshaping_indices:
+            xds["tidxs"] = tidxs
+            xds["bidxs"] = bidxs
+            xds["row_id"] = tb_tool.rownumbers()  # tb_tool.getcol("row_id")
+
         start = time.time()
         ms_v4_name = pathlib.Path(in_file).name.replace(".ms", "") + "_" + str(ms_v4_id)
         ms_xdt.ds = xds

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/_utils/_msv2/create_field_and_source_xds.py

@@ -573,7 +573,9 @@ def pad_missing_sources(
     for missing_id in missing_source_ids:
         missing_source_xds[concat_dim] = missing_id
         xdss_to_concat.append(missing_source_xds)
-    filled_source_xds = xr.concat(xdss_to_concat, concat_dim).sortby(concat_dim)
+    filled_source_xds = xr.concat(xdss_to_concat, concat_dim, join="outer").sortby(
+        concat_dim
+    )
 
     return filled_source_xds
 

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/_utils/_msv2/msv4_sub_xdss.py

@@ -591,6 +591,7 @@ def prepare_generic_sys_cal_xds(generic_sys_cal_xds: xr.Dataset) -> xr.Dataset:
 
 def create_system_calibration_xds(
     in_file: str,
+    spectral_window_id: int,
     main_xds_frequency: xr.DataArray,
     ant_xds: xr.DataArray,
     sys_cal_interp_time: Union[xr.DataArray, None] = None,
@@ -602,9 +603,11 @@ def create_system_calibration_xds(
     ----------
     in_file: str
         Input MS name.
+    spectral_window_id: int
+        SPW ID from MSv2
     main_xds_frequency: xr.DataArray
         frequency array of the main xds (MSv4), containing among other things
-        spectral_window_id and measures metadata
+        the measures metadata
     ant_xds : xr.Dataset
         The antenna_xds that has information such as names, stations, etc., for coordinates
     sys_cal_interp_time: Union[xr.DataArray, None] = None,
@@ -616,7 +619,6 @@ def create_system_calibration_xds(
         System calibration Xarray Dataset.
     """
 
-    spectral_window_id = main_xds_frequency.attrs["spectral_window_id"]
     try:
         generic_sys_cal_xds = load_generic_table(
             in_file,

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/convert_msv2_to_processing_set.py

@@ -62,6 +62,7 @@ def convert_msv2_to_processing_set(
     sys_cal_interpolate: bool = False,
     use_table_iter: bool = False,
     compressor: numcodecs.abc.Codec = numcodecs.Zstd(level=2),
+    add_reshaping_indices: bool = False,
     storage_backend: str = "zarr",
     parallel_mode: str = "none",
     overwrite: bool = False,
@@ -98,6 +99,8 @@ def convert_msv2_to_processing_set(
         Whether to use the table iterator to read the main table of the MS v2. This should be set to True when reading datasets with large number of rows and few partitions, by default False.
     compressor : numcodecs.abc.Codec, optional
         The Blosc compressor to use when saving the converted data to disk using Zarr, by default numcodecs.Zstd(level=2).
+    add_reshaping_indices : bool, optional
+        Whether to add the tidxs, bidxs and row_id variables to each partition of the main dataset. These can be used to reshape the data back to the original ordering in the MS v2. This is mainly intended for testing and debugging, by default False.
     storage_backend : {"zarr", "netcdf"}, optional
         The on-disk format to use. "netcdf" is not yet implemented.
     parallel_mode : {"none", "partition", "time"}, optional
@@ -178,6 +181,7 @@ def convert_msv2_to_processing_set(
                     ephemeris_interpolate=ephemeris_interpolate,
                     phase_cal_interpolate=phase_cal_interpolate,
                     sys_cal_interpolate=sys_cal_interpolate,
+                    add_reshaping_indices=add_reshaping_indices,
                     compressor=compressor,
                     parallel_mode=parallel_mode,
                     overwrite=overwrite,
@@ -198,6 +202,7 @@ def convert_msv2_to_processing_set(
                 ephemeris_interpolate=ephemeris_interpolate,
                 phase_cal_interpolate=phase_cal_interpolate,
                 sys_cal_interpolate=sys_cal_interpolate,
+                add_reshaping_indices=add_reshaping_indices,
                 compressor=compressor,
                 parallel_mode=parallel_mode,
                 overwrite=overwrite,

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/measurement_set_xdt.py

@@ -203,8 +203,14 @@ class MeasurementSetXdt:
         else:
             line_name = []
 
+        if "spectral_window_intent" not in self._xdt.frequency.attrs:
+            spw_intent = "UNSPECIFIED"
+        else:
+            spw_intent = self._xdt.frequency.attrs["spectral_window_intent"]
+
         partition_info = {
             "spectral_window_name": self._xdt.frequency.attrs["spectral_window_name"],
+            "spectral_window_intent": spw_intent,
             "field_name": to_list(np.unique(field_and_source_xds.field_name.values)),
             "polarization_setup": to_list(self._xdt.polarization.values),
             "scan_name": to_list(np.unique(self._xdt.scan_name.values)),
@@ -228,7 +234,8 @@ class MeasurementSetXdt:
         description: str = None,
         data_group_dv_shared_with: str = None,
     ) -> xr.DataTree:
-        """_summary_
+        """Adds a data group to the MSv4 DataTree, grouping the given data, weight, flag, etc. variables
+        and field_and_source_xds.
 
         Parameters
         ----------
@@ -254,7 +261,7 @@ class MeasurementSetXdt:
         Returns
         -------
         xr.DataTree
-            _description_
+            MSv4 DataTree with the new group added
         """
 
         if data_group_dv_shared_with is None:

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/processing_set_xdt.py

@@ -142,7 +142,7 @@ class ProcessingSetXdt:
         if "freq_axis" in self.meta:
             return self.meta["freq_axis"]
         else:
-            spw_ids = []
+            spw_names = []
             freq_axis_list = []
             frame = self._xdt[next(iter(self._xdt.children))].frequency.attrs[
                 "observer"
@@ -151,11 +151,13 @@ class ProcessingSetXdt:
                 assert (
                     frame == ms_xdt.frequency.attrs["observer"]
                 ), "Frequency reference frame not consistent in Processing Set."
-                if ms_xdt.frequency.attrs["spectral_window_id"] not in spw_ids:
-                    spw_ids.append(ms_xdt.frequency.attrs["spectral_window_id"])
+                if ms_xdt.frequency.attrs["spectral_window_name"] not in spw_names:
+                    spw_names.append(ms_xdt.frequency.attrs["spectral_window_name"])
                     freq_axis_list.append(ms_xdt.frequency)
 
-            freq_axis = xr.concat(freq_axis_list, dim="frequency").sortby("frequency")
+            freq_axis = xr.concat(freq_axis_list, dim="frequency", join="outer").sortby(
+                "frequency"
+            )
             self.meta["freq_axis"] = freq_axis
             return self.meta["freq_axis"]
 
@@ -167,6 +169,7 @@ class ProcessingSetXdt:
             "polarization": [],
             "scan_name": [],
             "spw_name": [],
+            "spw_intent": [],
             "field_name": [],
             "source_name": [],
             "line_name": [],
@@ -183,6 +186,7 @@ class ProcessingSetXdt:
             summary_data["name"].append(key)
             summary_data["intents"].append(partition_info["intents"])
             summary_data["spw_name"].append(partition_info["spectral_window_name"])
+            summary_data["spw_intent"].append(partition_info["spectral_window_intent"])
             summary_data["polarization"].append(value.polarization.values)
             summary_data["scan_name"].append(partition_info["scan_name"])
             data_name = value.attrs["data_groups"][data_group]["correlated_data"]
@@ -267,10 +271,10 @@ class ProcessingSetXdt:
         Examples
         --------
         >>> # Select all MSs with intents 'OBSERVE_TARGET#ON_SOURCE' and polarization 'RR' or 'LL'
-        >>> selected_ps = ps.query(intents='OBSERVE_TARGET#ON_SOURCE', polarization=['RR', 'LL'])
+        >>> selected_ps_xdt = ps_xdt.xr_ps.query(intents='OBSERVE_TARGET#ON_SOURCE', polarization=['RR', 'LL'])
 
         >>> # Select all MSs with start_frequency greater than 100 GHz and less than 200 GHz
-        >>> selected_ps = ps.query(query='start_frequency > 100e9 AND end_frequency < 200e9')
+        >>> selected_ps_xdt = ps_xdt.xr_ps.query(query='start_frequency > 100e9 AND end_frequency < 200e9')
         """
 
         if self._xdt.attrs.get("type") not in PS_DATASET_TYPES:
@@ -377,6 +381,7 @@ class ProcessingSetXdt:
                     combined_field_and_source_xds = xr.concat(
                         [combined_field_and_source_xds, field_and_source_xds],
                         dim="field_name",
+                        join="outer",
                     )
 
         if (len(combined_field_and_source_xds.data_vars) > 0) and (
@@ -497,6 +502,7 @@ class ProcessingSetXdt:
                     combined_ephemeris_field_and_source_xds = xr.concat(
                         [combined_ephemeris_field_and_source_xds, field_and_source_xds],
                         dim="time",
+                        join="outer",
                     )
 
         if (len(combined_ephemeris_field_and_source_xds.data_vars) > 0) and (
@@ -719,6 +725,7 @@ class ProcessingSetXdt:
                     dim="antenna_name",
                     data_vars="minimal",
                     coords="minimal",
+                    join="outer",
                 )
 
         # ALMA WVR antenna_xds data has a NaN value for the antenna receptor angle.

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/measurement_set/schema.py

@@ -9,7 +9,7 @@ from xradio.schema.bases import (
 from xradio.schema.typing import Attr, Coord, Coordof, Data, Dataof
 import numpy
 
-MSV4_SCHEMA_VERSION = "4.0.-9988"
+MSV4_SCHEMA_VERSION = "4.0.-9987"
 
 # Dimensions
 Time = Literal["time"]
@@ -674,6 +674,9 @@ class FrequencyArray:
     """ Center frequencies for each channel. """
     spectral_window_name: Attr[str]
     """ Name associated with spectral window. """
+    spectral_window_intent: Attr[str]
+    """ An intent string that identifies the intention of the spectral window, for example
+    continuum, spectral line, etc. See :ref:`spw intents` for possible values. """
     frequency_group_name: Optional[Attr[str]]
     """ Name associated with frequency group - needed for multi-band VLBI fringe-fitting."""
     reference_frequency: Attr[SpectralCoordArray]
@@ -786,7 +789,7 @@ class FlagArray:
             tuple[Time, BaselineId, Frequency, Polarization],
             tuple[Time, AntennaName, Frequency, Polarization],  # SD
         ],
-        bool,
+        Union[bool, numpy.uint8, numpy.uint16, numpy.uint32, numpy.uint64],
     ]
     time: Coordof[TimeCoordArray]
     baseline_id: Optional[Coordof[BaselineArray]]  # Only IF
@@ -795,7 +798,7 @@ class FlagArray:
     polarization: Optional[Coordof[PolarizationArray]] = None
     long_name: Optional[Attr[str]] = "Visibility flags"
 
-    allow_mutiple_versions: Optional[Attr[bool]] = True
+    allow_multiple_versions: Optional[Attr[bool]] = True
 
 
 @xarray_dataarray_schema
@@ -833,7 +836,7 @@ class WeightArray:
     polarization: Optional[Coordof[PolarizationArray]] = None
     long_name: Optional[Attr[str]] = "Visibility weights"
 
-    allow_mutiple_versions: Optional[Attr[bool]] = True
+    allow_multiple_versions: Optional[Attr[bool]] = True
 
 
 # J2000=>fk5 is used most often. icrs is used less often. Both fk5 and icrs are also borrowed from the field center (to fix
@@ -896,7 +899,7 @@ class UvwArray:
     """ To be defined in astropy (see for example https://github.com/astropy/astropy/issues/7766) """
     units: Attr[UnitsMeters] = "m"
 
-    allow_mutiple_versions: Optional[Attr[bool]] = True
+    allow_multiple_versions: Optional[Attr[bool]] = True
 
 
 @xarray_dataarray_schema
@@ -992,7 +995,7 @@ class FrequencyCentroidArray:
 @xarray_dataarray_schema
 class EffectiveChannelWidthArray:
     """
-    Model of frequency related data variables of the main dataset, such as EFFECTIV_CHANNEL_WIDTH.
+    Model of frequency related data variables of the main dataset, such as EFFECTIVE_CHANNEL_WIDTH.
     """
 
     data: Data[
@@ -1321,7 +1324,7 @@ class VisibilityArray:
     long_name: Optional[Attr[str]] = "Visibility values"
     """ Long-form name to use for axis. Should be ``"Visibility values"``"""
     units: Attr[str] = "Jy"
-    allow_mutiple_versions: Optional[Attr[bool]] = True
+    allow_multiple_versions: Optional[Attr[bool]] = True
 
 
 # Info dicts
@@ -1382,9 +1385,22 @@ class ObservationInfoDict:
     """ASDM: Logs of the observation during this execu- tion block."""
     intents: list[str]
     """ An intent string identifies one intention of the scan, such as to calibrate or observe a
-    target. See :ref:`scan intents` for possible values. When converting from MSv2, the list of
-    intents is derived from the OBS_MODE column of MSv2 state table (every comma separated value
-    is taken as an intent). """
+    target. See :ref:`scan intents` for possible intent/subintent values. When converting from MSv2,
+    the list of intents is derived from the OBS_MODE column of MSv2 state table (every comma
+    separated value is taken as an intent).
+    A common convention used in the MSv2 OBS_MODE column is to specify multiple intents separated
+    by commas, each of them giving a main intent and a subintent separated by a '#' character. This
+    is represented in this attribute as a list of "intent#subintent" strings. These are a few
+    example lists:
+    ["CALIBRATE_DELAY#ON_SOURCE" , "CALIBRATE_PHASE#ON_SOURCE", "CALIBRATE_WVR#ON_SOURCE"],
+    ["CALIBRATE_FLUX#ON_SOURCE" , "CALIBRATE_WVR#ON_SOURCE"],
+    ["CALIBRATE_POINTING#ON_SOURCE", "CALIBRATE_WVR#ON_SOURCE", "CALIBRATE_DELAY#ON_SOURCE"],
+    ["CALIBRATE_ATMOSPHERE#AMBIENT", "CALIBRATE_WVR#AMBIENT"],
+    ["CALIBRATE_FOCUS#ON_SOURCE" , "CALIBRATE_WVR#ON_SOURCE"],
+    ["OBSERVE_TARGET#ON_SOURCE"], or ["OBSERVE_TARGE#UNSPECIFIED"].
+    The list of possible intent and subintent names (see :ref:`scan intents`) is derived from the
+    respective ASDM enumerations.
+    """
 
 
 @dict_schema
@@ -1419,7 +1435,8 @@ class DataGroupDict:
 
 @dict_schema
 class DataGroupsDict:
-    """Dictionary of data group dictionaries."""
+    """Dictionary of data group dictionaries. A 'base' data group is mandatory.
+    Additional data groups can be added with different names."""
 
     base: DataGroupDict
 

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/schema/check.py

@@ -382,14 +382,14 @@ def check_data_vars(
 
     issues = SchemaIssues()
     for data_var_schema in data_vars_schema:
-        allow_mutiple_versions = False
+        allow_multiple_versions = False
         for attr in data_var_schema.attributes:
             if hasattr(attr, "name"):
-                if attr.name == "allow_mutiple_versions":
-                    allow_mutiple_versions = attr.default
+                if attr.name == "allow_multiple_versions":
+                    allow_multiple_versions = attr.default
 
         data_vars_names = []
-        if allow_mutiple_versions:
+        if allow_multiple_versions:
             for data_var_name in data_vars:
                 if data_var_schema.name in data_var_name:
                     data_vars_names.append(data_var_name)

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/schema/metamodel.py

@@ -35,7 +35,7 @@ class ValueSchema:
     """
     dict_schema: typing.Optional[DictSchema] = None
     """
-    Dictionary schema, if it is an xarray DataArray
+    Dictionary schema, if it is a dict
     """
     array_schema: typing.Optional[ArraySchema] = None
     """

{xradio-0.0.58 → xradio-0.0.60}/src/xradio/sphinx/schema_table.py

@@ -154,40 +154,46 @@ class SchemaTableDirective(ObjectDescription):
                 self.state.nested_parse(vl, 0, entry)
 
 
-def format_literals(typ):
+def format_literals(literal) -> nodes.line:
 
-    # a | b | c: Recurse and merge
-    if typing.get_origin(typ) == typing.Union:
-        type_args = typing.get_args(typ)
-        options = []
-        for arg in type_args:
-            options += format_literals(arg)
-        return options
+    if isinstance(literal, list) and all([isinstance(item, str) for item in literal]):
+        formatted_literal = [nodes.literal(text=f"'{val}'") for val in literal]
+    else:
+        raise ValueError(f"Must be a list of literal string values: {literal}")
 
-    # Literal['a', 'b', ...]: Wrap into individual "literal" nodes
-    if typing.get_origin(typ) == typing.Literal:
-        return list(map(lambda t: nodes.literal(text=repr(t)), typing.get_args(typ)))
+    # Join the literals with ... , .. , .. , or ...
+    line = nodes.line()
+    for i, lit in enumerate(formatted_literal):
+        if i > 0:
+            if i + 1 >= len(formatted_literal):
+                line += nodes.Text(" or\xa0")
+            else:
+                line += nodes.Text(", ")
+        line += lit
 
-    # list[Literal['a'], Literal['b'], ...]: Format as one literal (compound) value
-    if typing.get_origin(typ) == list:
-        type_args = typing.get_args(typ)
-        if any([typing.get_origin(arg) != typing.Literal for arg in type_args]):
-            raise ValueError(f"List must contain only literals: {typ}")
-        values = [repr(typing.get_args(val)[0]) for val in typing.get_args(typ)]
-        return [nodes.literal(text=f"[{', '.join(values)}]")]
+    return line
+
+
+def format_class_types(state, attr_type) -> nodes.line:
+    line = nodes.line()
+    vl = StringList()
+    vl.append(f":py:class:`~{attr_type}`", "")
+    with switch_source_input(state, vl):
+        state.nested_parse(vl, 0, line)
 
-    raise ValueError(f"Must be either a type or a literal: {typ}")
+    return line
 
 
-def format_attr_model_text(state, attr) -> StringList:
+def format_attr_model_text(state, attr) -> nodes.line:
     """
-    Formats the text for the 'model' column in schema tables (arrays and datasets).
+    For an attribute, formats the text for the 'model' column in schema tables
+    (arrays and datasets).
     Doesn't aim at supporting any literal types or combinations of types in general,
     but the following three ones specifically:
 
-    - Literals (with multiple options (implicit Union of literals))
-    - List of literals (e.g. ["rad","rad"]
-    - Union of list of literals (e.g. ["m","m","m"]/["rad","rad","m"]
+    - String literals (units, frames, measure types, etc.)
+    - Other classes (for example usual built-in types such str, bool or ints,
+      or schema classes: schema dicts and schema arrays)
 
     This is meant to produce readable text listing literals as quoted text and
     their combinations, in schema attributes (particularly quantities and measures).
@@ -196,58 +202,16 @@ def format_attr_model_text(state, attr) -> StringList:
     type name.
     """
 
-    type_args = typing.get_args(attr.typ)
-    is_list_of_literals = typing.get_origin(attr.typ) is list and all(
-        [typing.get_origin(arg) is typing.Literal for arg in type_args]
-    )
-
-    line = nodes.line()
-
-    if not is_list_of_literals:
-        # A type?
-        if isinstance(attr.typ, type):
-            vl = StringList()
-            vl.append(f":py:class:`~{attr.typ.__module__}.{attr.typ.__name__}`", "")
-            with switch_source_input(state, vl):
-                state.nested_parse(vl, 0, line)
-            return line
-
-        if typing.get_origin(attr.typ) == typing.Union:
-            vl = StringList()
-            type_args = typing.get_args(attr.typ)
-            options = []
-            for i, arg in enumerate(type_args):
-                vl.append(f":py:class:`~{arg.__module__}.{arg.__name__}`", "")
-                if i + 1 < len(type_args):
-                    vl.append(" or ", "")
-            with switch_source_input(state, vl):
-                state.nested_parse(vl, 0, line)
-            return line
-
-        # Derived type, e.g. list of types?
-        if typing.get_origin(attr.typ) == list and all(
-            [isinstance(arg, type) for arg in type_args]
-        ):
-            vl = StringList()
-            vl.append("[", "")
-            for i, arg in enumerate(typing.get_args(attr.typ)):
-                if i > 0:
-                    vl.append(", ", "")
-                vl.append(f":py:class:`~{arg.__module__}.{arg.__name__}`", "")
-            vl.append("]", "")
-            with switch_source_input(state, vl):
-                state.nested_parse(vl, 0, line)
-            return line
-
-    # Assume it's a literal of some kind - collect options
-    literals = format_literals(attr.typ)
-    for i, lit in enumerate(literals):
-        if i > 0:
-            if i + 1 >= len(literals):
-                line += nodes.Text(" or\xa0")
-            else:
-                line += nodes.Text(", ")
-        line += lit
+    if getattr(attr, "literal"):
+        line = format_literals(attr.literal)
+    else:
+        if getattr(attr, "dict_schema"):
+            attr_type = attr.dict_schema.schema_name
+        elif getattr(attr, "array_schema"):
+            attr_type = attr.array_schema.schema_name
+        else:
+            attr_type = attr.type
+        line = format_class_types(state, attr_type)
 
     return line
 
@@ -356,7 +320,7 @@ class DictSchemaTableDirective(SchemaTableDirective):
             for attr in schema.attributes:
                 self._add_row(
                     attr.name,
-                    types=[f"{attr.typ.__name__}"],
+                    types=[attr.type],
                     optional=attr.optional,
                     descr=attr.docstring,
                     default=attr.default,

{xradio-0.0.58 → xradio-0.0.60/src/xradio.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: xradio
-Version: 0.0.58
+Version: 0.0.60
 Summary:  Xarray Radio Astronomy Data IO
 Author-email: Jan-Willem Steeb <jsteeb@nrao.edu>, Federico Montesino Pouzols <pouzols@eso.edu>, Dave Mehringer <dmehring@nrao.edu>, Peter Wortmann <peter.wortmann@skao.int>
 License: BSD 3-Clause License
@@ -108,25 +108,32 @@ Xarray Radio Astronomy Data IO is still in development.
 [![Version Status](https://img.shields.io/pypi/v/xradio.svg)](https://pypi.python.org/pypi/xradio/)
 
 # Installing
-It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed:
+XRADIO can be installed in virtual environments via pip. It is recommended to use the conda environment manager from [miniforge](https://github.com/conda-forge/miniforge) to create a clean, self-contained runtime where XRADIO and all its dependencies can be installed, for example:
 ```sh
 conda create --name xradio python=3.12 --no-default-packages
 conda activate xradio
 ```
+> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
+
 XRADIO can now be installed using:
 ```sh
 pip install xradio
 ```
-This will also install the minimal dependencies for XRADIO. To install the minimal dependencies and the interactive components (JupyterLab) use:
+This will also install the minimal dependencies for XRADIO.
+
+Note that if only the minimal dependencies are installed, the functionality to convert MSv2 to MSv4 will not be available.
+This requires installing `python-casacore` (also included in the `all` group, see below), or alternatively the
+`casatools` backend, as explained in the [casatools I/O backend guide](docs/source/measurement_set/guides/backends.md).
+
+To install the minimal dependencies and the interactive components (JupyterLab) use:
 ```sh
 pip install "xradio[interactive]"
 ```
+
 To enable conversion from MSv2 to MSv4 use (this only works for Linux):
 ```sh
 pip install "xradio[python-casacore]"
 ```
-> 📝 On macOS it is required to pre-install `python-casacore` using `conda install -c conda-forge python-casacore`.
-
 To be able to run tests:
 ```sh
 pip install "xradio[test]"
@@ -135,3 +142,10 @@ Multiple-dependencies can be installed using:
 ```sh
 pip install "xradio[interactive,python-casacore,test]"
 ```
+
+To install a more complete set of dependencies:
+```sh
+pip install "xradio[all]"
+```
+This will include the dependencies required to run the interactive Jupyter notebooks, run tests, build documentation,
+and python-casacore to enable MSv2=>MSv4 functionality.