pycmap 0.2.18__tar.gz → 0.2.20__tar.gz

pycmap-0.2.20/PKG-INFO

@@ -0,0 +1,109 @@
+Metadata-Version: 2.4
+Name: pycmap
+Version: 0.2.20
+Summary: Simons CMAP API python client
+Home-page: https://github.com/simonscmap/pycmap
+Author: Mohammad D. Ashkezari
+Author-email: mdehghan@uw.edu
+Keywords: ocean oceanography data visualization satellite model in-situ remote sensing machine learning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: scipy
+Requires-Dist: bokeh
+Requires-Dist: matplotlib
+Requires-Dist: cmocean
+Requires-Dist: tqdm
+Requires-Dist: folium
+Requires-Dist: colorama
+Requires-Dist: plotly
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
+
+[![PyPI version](https://badge.fury.io/py/pycmap.svg)](https://badge.fury.io/py/pycmap)
+[![DOI](https://zenodo.org/badge/199070692.svg)](https://zenodo.org/badge/latestdoi/199070692)
+![Cover](https://github.com/simonscmap/pycmap/raw/master/docs/figures/CMAP.png)
+
+*Mohammad Dehghani Ashkezari <mdehghan@uw.edu>*
+
+
+# pycmap
+Simons CMAP is an open-source data service to retrieve, visualize, and analyze oceanic datasets such as in-situ observations, multi-decade global satellite remote sensing, and model outputs. Pycmap is the python package of Simons CMAP project providing a simple and unified interface to the hosted datasets at Simons CMAP database. It enables the scientists and general public to dive into the vast, and often underutilized, ocean datasets and retrieve customized subsets of these massive datasets without going through the time-consuming process of data collection and preparation.
+
+## Documentation
+See the *table of contents* below for pycmap documentations. 
+<br />The docs are written in jupyter notebook format and each notebook contains one or more example codes. Please download the `/docs` directory and run the example codes locally. Alternatively, you may run the examples on google cloud using the *Colab badge* which appears at the top of each page: 
+<br /><img align="left" src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab" title="Open and Execute in Google Colaboratory">
+
+
+<br /><br />More detailed system documentations can be found at [https://cmap.readthedocs.io/en/latest/](https://cmap.readthedocs.io/en/latest/).
+
+## Website
+Simons CMAP website is under active development at the moment: [https://simonscmap.com](https://simonscmap.com)
+
+**<br />This project is supported by [Simons Foundation](https://www.simonsfoundation.org/).**
+
+
+
+
+## Table of Contents
+
+
+### [Installation](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Installation.ipynb)
+
+### [1. Data Retrieval (API)](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/API.ipynb)
+- [Query](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Query.ipynb)
+- [Catalog](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Catalog.ipynb)
+- [Search Catalog](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/SearchCatalog.ipynb)
+- [Datasets](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Datasets.ipynb)
+- [Datasets With Ancillary](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DatasetsWithAncillary.ipynb)
+- [Dataset MetaData](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DatasetMetaData.ipynb)
+- [Variable MetaData](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/MetaData.ipynb)
+- [Dataset Columns](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Columns.ipynb)
+- [Dataset Head](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Head.ipynb)
+- [Variable Long Name](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/LongName.ipynb)
+- [Variable Unit](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Unit.ipynb)
+- [Variable Resolution](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Resolution.ipynb)
+- [Variable Coverage](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Coverage.ipynb)
+- [Variable Stat](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Stat.ipynb)
+- [If Column Exists](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/HasField.ipynb)
+- [Is Gridded Product](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Grid.ipynb)
+- [Is Climatology Product](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/IsClimatology.ipynb)
+- [List of Cruises](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Cruises.ipynb)
+- [Cruise Details by Name](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseByName.ipynb)
+- [Cruise Spatio-Temporal Bounds](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseBounds.ipynb)
+- [Cruise Trajectory](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseTrajectory.ipynb)
+- [Cruise Variables](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseVariables.ipynb)
+- [Retrieve Dataset](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/RetrieveDataset.ipynb)
+- [Retrieve Dataset With Ancillary](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/RetrieveDatasetWithAncillary.ipynb)
+- [Data Subset: Generic Space-Time Cut](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/SpaceTime.ipynb)
+- [Data Subset: TimeSeries](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/TimeSeries.ipynb)
+- [Data Subset: Depth Profile](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DepthProfile.ipynb)
+- [Compute Climatology](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Climatology.ipynb)
+- [Sampling](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Sampling.ipynb)
+- [Match (colocalize) Cruise Track with Datasets](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/MatchCruise.ipynb)
+
+
+### [2. Data Visualization](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz.ipynb)
+- [Histogram Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_Histogram.ipynb)
+- [TimeSeries Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_TimeSeries.ipynb)
+- [Regional Map, Contour Plot, 3D Surface Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_RegionalMap.ipynb)
+- [Section Map, Section Contour](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_Section.ipynb)
+- [Depth Profile](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_DepthProfile.ipynb)
+- [Cruise Track Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CruiseTrack.ipynb)
+- [Correlation Matrix](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CorrelationMatrix.ipynb)
+- [Correlation Matrix Along Cruise Track](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CruiseCorrelationMatrix.ipynb)
+

{pycmap-0.2.18 → pycmap-0.2.20}/pycmap/__init__.py

@@ -15,6 +15,6 @@ if sys.version_info < (3, 0):
         UserWarning
     )
 
-__version__ = '0.2.18'
+__version__ = '0.2.20'
 # __all__ = []
  

{pycmap-0.2.18 → pycmap-0.2.20}/pycmap/rest.py

@@ -7,8 +7,7 @@ Function: Encapsulates RESTful API logic.
 """
 
 
-import random
-import requests
+import sys, requests, random
 from requests.exceptions import HTTPError
 from urllib.parse import urlencode
 import numpy as np
@@ -237,6 +236,21 @@ class _REST(object):
         self.validate_sp_args(args[0], args[1], args[2], args[3], args[4], args[5], args[6], args[7], args[8], args[9])
         df = self._request(route, method='GET', payload=payload)   
         return df           
+    
+
+    def _validate_table_var(self, table, variable=None):
+        """
+        Check if table and variable exist in the catalog.
+        """
+        if variable:
+            df = self.query(f"exec uspValidate_Table_Variable '{table}', '{variable}'")
+            pot_msg = f"Invalid table ({table}) and/or variable names ({variable})."
+        else:    
+            df = self.query(f"exec uspValidate_Table_Variable '{table}'")
+            pot_msg = f"Invalid table name ({table})."
+        pot_msg += "\nPlease make sure that the dataset is still available in the CMAP catalog (not deprecated)."
+        if len(df) != 1: sys.exit(pot_msg)
+        return len(df) == 1    
 
 
     def get_catalog(self):
@@ -281,11 +295,13 @@ class _REST(object):
 
     def head(self, tableName, rows=5):
         """Returns top records of a data set."""
+        self._validate_table_var(tableName)
         return self.query(f"select top {rows} * from {tableName}")
 
 
     def columns(self, tableName):
         """Returns the list of data set columns."""
+        self._validate_table_var(tableName)
         return list(self.query(f"select top 1 * from {tableName}").columns)
 
 
@@ -299,6 +315,7 @@ class _REST(object):
         if len(df) > 1:
             halt('More than one table found. Please provide a more specific name: ')
             print(df)
+        self._validate_table_var(tableName)    
         return df.iloc[0]['Dataset_ID']            
 
 
@@ -310,6 +327,7 @@ class _REST(object):
         Note that this method does not return the dataset metadata. 
         Use the 'get_dataset_metadata' method to get the dataset metadata.
         """
+        self._validate_table_var(tableName)
         datasetID = self.get_dataset_ID(tableName)
         maxRow = 2000000
         df = self.query("SELECT JSON_stats FROM tblDataset_Stats WHERE Dataset_ID=%d " % datasetID)
@@ -319,7 +337,7 @@ class _REST(object):
             msg = "The requested dataset has %d records.\n" % rows 
             msg += "It is not recommended to retrieve datasets with more than %d rows using this method.\n" % maxRow
             msg += "For large datasets, please use the 'space_time' method and retrieve the data in smaller chunks." 
-            halt(msg)
+            halt(msg)        
         return self.query("SELECT * FROM %s" % tableName)
 
 
@@ -330,6 +348,7 @@ class _REST(object):
         Note that this method does not return the dataset metadata. 
         Use the 'get_dataset_metadata' method to get the dataset metadata.
         """
+        self._validate_table_var(tableName)
         datasetID = self.get_dataset_ID(tableName)
         df = self.query("SELECT JSON_stats FROM tblDataset_Stats WHERE Dataset_ID=%d " % datasetID)
         df = pd.read_json(df['JSON_stats'][0])
@@ -359,6 +378,7 @@ class _REST(object):
 
     def get_dataset_metadata(self, tableName):
         """Returns a dataframe containing the dataset metadata."""
+        self._validate_table_var(tableName)
         return self.query("EXEC uspDatasetMetadata  '%s'" % tableName)
 
 
@@ -367,6 +387,7 @@ class _REST(object):
         Returns a single-row dataframe from tblVariables containing info associated with varName.
         This method is mean to be used internally and will not be exposed at documentations.
         """
+        self._validate_table_var(tableName, varName)
         query = "SELECT * FROM tblVariables WHERE Table_Name='%s' AND Short_Name='%s'" % (tableName, varName)
         return self.query(query)
 
@@ -378,40 +399,45 @@ class _REST(object):
 
     def get_var_long_name(self, tableName, varName):
         """Returns the long name of a given variable."""
+        self._validate_table_var(tableName, varName)
         return self.query("EXEC uspVariableLongName '%s', '%s'" % (tableName, varName)).iloc[0]['Long_Name']
 
 
     def get_unit(self, tableName, varName):
         """Returns the unit for a given variable."""
+        self._validate_table_var(tableName, varName)
         return ' [' + self.query("EXEC uspVariableUnit '%s', '%s'" % (tableName, varName)).iloc[0]['Unit'] + ']' 
 
 
     def get_var_resolution(self, tableName, varName):
         """Returns a single-row dataframe from catalog containing the variable's spatial and temporal resolutions."""
+        self._validate_table_var(tableName, varName)
         return self.query("EXEC uspVariableResolution '%s', '%s'" % (tableName, varName))
 
 
     def get_var_coverage(self, tableName, varName):
         """Returns a single-row dataframe from catalog containing the variable's spatial and temporal coverage."""
+        self._validate_table_var(tableName, varName)
         return self.query("EXEC uspVariableCoverage '%s', '%s'" % (tableName, varName))
 
 
     def get_var_stat(self, tableName, varName):
         """Returns a single-row dataframe from catalog containing the variable's summary statistics."""
+        self._validate_table_var(tableName, varName)
         return self.query("EXEC uspVariableStat '%s', '%s'" % (tableName, varName))
 
 
     def has_field(self, tableName, varName, servers=["rainier"]):
         """Returns a boolean confirming whether a field (varName) exists in a table (data set)."""
-        query = f"select top 1 {varName} from {tableName}"
+        self._validate_table_var(tableName)
+        query = f"select top 1 * from {tableName}"
         df = self.query(query, servers)
-        hasField = False
-        if len(df)>0: hasField = True
-        return hasField
+        return len(df) > 0 and varName in df.columns
 
 
     def is_grid(self, tableName, varName):
         """Returns a boolean indicating whether the variable is a gridded product or has irregular spatial resolution."""
+        self._validate_table_var(tableName, varName)
         grid = True
         query = "SELECT Spatial_Res_ID, RTRIM(LTRIM(Spatial_Resolution)) AS Spatial_Resolution FROM tblVariables "
         query = query + "JOIN tblSpatial_Resolutions ON [tblVariables].Spatial_Res_ID=[tblSpatial_Resolutions].ID "
@@ -429,6 +455,7 @@ class _REST(object):
         """
         Returns True if the table represents a climatological data set.    
         """
+        self._validate_table_var(tableName)
         df = self.query(f"SELECT * FROM tblDatasets d JOIN tblVariables v ON d.ID=v.Dataset_ID WHERE v.Table_Name='{tableName}'", servers)
         if len(df) > 0:
             return True if df.iloc[0]['Climatology'] == 1 else False
@@ -447,6 +474,7 @@ class _REST(object):
         Returns a dataframe containing the associated metadata for a single variable.
         The returned metadata does not include the list of references and articles associated with the variable.  
         """
+        self._validate_table_var(table, variable)
         query = f"{catalog_sql()} WHERE Short_Name='{variable}' AND Table_Name='{table}'"
         return self.query(query)
         
@@ -456,6 +484,7 @@ class _REST(object):
         Returns a dataframe containing the associated metadata.
         The inputs can be string literals (if only one table, and variable is passed) or a list of string literals.
         """
+        self._validate_table_var(table, variable)
         if isinstance(table, str): table = [table]
         if isinstance(variable, str): variable = [variable]
         metadata = pd.DataFrame({})    
@@ -513,6 +542,7 @@ class _REST(object):
 
     def subset(self, spName, table, variable, dt1, dt2, lat1, lat2, lon1, lon2, depth1, depth2, servers):     
         """Returns a subset of data according to space-time constraints."""
+        self._validate_table_var(table, variable)
         query = f"EXEC {spName} '{table}', '{variable}', '{dt1}', '{dt2}', {lat1}, {lat2}, {lon1}, {lon2}, {depth1}, {depth2}"
         return self.query(query, servers)  
 
@@ -522,6 +552,7 @@ class _REST(object):
         Returns a subset of data according to space-time constraints.
         The results are ordered by time, lat, lon, and depth (if exists).
         """
+        self._validate_table_var(table, variable)
         return self.subset('uspSpaceTime', table, variable, dt1, dt2, lat1, lat2, lon1, lon2, depth1, depth2, servers=servers)
 
 
@@ -549,6 +580,7 @@ class _REST(object):
         The results are aggregated by time and ordered by time, lat, lon, and depth (if exists).
         The timeseries data can be binned weekly, monthly, qurterly, or annualy, if interval variable is set (this feature is not applicable to climatological data sets). 
         """
+        self._validate_table_var(table, variable)
         usp = self._interval_to_uspName(interval)
         if usp != 'uspTimeSeries' and self.is_climatology(table, servers):
             print_tqdm(
@@ -563,6 +595,7 @@ class _REST(object):
         Returns a subset of data according to space-time constraints.
         The results are aggregated by depth and ordered by depth.
         """
+        self._validate_table_var(table, variable)
         return self.subset('uspDepthProfile', table, variable, dt1, dt2, lat1, lat2, lon1, lon2, depth1, depth2, servers=servers)
 
 
@@ -571,6 +604,7 @@ class _REST(object):
         Returns a subset of data according to space-time constraints.
         The results are ordered by time, lat, lon, and depth.
         """
+        self._validate_table_var(table, variable)
         return self.subset('uspSectionMap', table, variable, dt1, dt2, lat1, lat2, lon1, lon2, depth1, depth2, servers=servers)
 
 
@@ -600,6 +634,7 @@ class _REST(object):
         For instance, if the dataset is a weekly-averaged product, do not set the `period` to 'dayofyear'.
         The output of this method is a Pandas DataFrame ordered by time, lat, lon, and depth (if exists), respectively.
         """
+        self._validate_table_var(table, variable)
         period = self._climatology_period(period)
         if self.is_climatology(table):
             print_tqdm(
@@ -641,7 +676,9 @@ class _REST(object):
         """
         Takes a cruise name and colocalizes the cruise track with the specified variable(s).
         """
-        df = self.cruise_bounds(cruise)
+        for i in range(len(targetTables)):
+            self._validate_table_var(targetTables[i], targetVars[i])        
+        df = self.cruise_bounds(cruise)     
         return self.match(
                          sourceTable='tblCruise_Trajectory',
                          sourceVar=str(df.iloc[0]['ID']),

{pycmap-0.2.18 → pycmap-0.2.20}/pycmap/sample.py

@@ -15,9 +15,52 @@ from dateutil.parser import parse
 
 
 
-def alias(varName, tableName):
-    """Return an alias for a variable name."""
-    return f"CMAP_{varName}_{tableName}"
+# Friendly alias suffix to use for common SQL aggregation functions.
+# Anything not listed here falls back to the lowercased function name
+# (so e.g. ``"MIN"`` becomes the ``_min`` column suffix).
+AGG_SUFFIX = {
+    "AVG": "avg",
+    "STDEV": "std",
+    "COUNT": "count",
+}
+
+
+def _resolve_aggregations(aggFun):
+    """
+    Normalize a user-supplied list of SQL aggregation function names into
+    a list of ``(suffix, sqlFunc)`` pairs.
+
+    Function names are case-insensitive on input; the SQL form is uppercased
+    and the column-name suffix is the lowercased function name (with a few
+    friendlier short forms — see ``AGG_SUFFIX``).
+    """
+    resolved = []
+    for fn in aggFun:
+        sqlFunc = fn.strip().upper()
+        suffix = AGG_SUFFIX.get(sqlFunc, sqlFunc.lower())
+        resolved.append((suffix, sqlFunc))
+    return resolved
+
+
+def alias(varName, tableName, aggregations):
+    """
+    Return the list of column aliases produced for a single target variable.
+
+    For each aggregation in ``aggregations`` the variable contributes one
+    column to the output dataframe, e.g. with the default ``AVG`` only::
+
+        CMAP_sst_tblSST_AVHRR_OI_NRT_avg
+
+    or with ``["AVG", "STDEV", "COUNT"]``::
+
+        CMAP_sst_tblSST_AVHRR_OI_NRT_avg
+        CMAP_sst_tblSST_AVHRR_OI_NRT_std
+        CMAP_sst_tblSST_AVHRR_OI_NRT_count
+
+    :param list aggregations: list of ``(suffix, sqlFunc)`` pairs as produced
+        by ``_resolve_aggregations``.
+    """
+    return [f"CMAP_{varName}_{tableName}_{suffix}" for suffix, _ in aggregations]
 
 
 def add_target_columns(df, targets):
@@ -30,12 +73,12 @@ def add_target_columns(df, targets):
     return df
     
 
-def add_target_meta(api, targets, servers):
+def add_target_meta(api, targets, aggregations, servers):
     """
     Adds new entries (metadata) to the `targets` dictionary including the 
     temporal coverage of each environmental dataset, if it has depth field, 
     and if it's a climatology dataset.
-    """
+    """   
     for table, env in targets.items():
         df = api.query(f"SELECT MIN([time]) startTime, MAX([time]) endTime FROM {table}", servers)
         if len(df) > 0:
@@ -43,9 +86,10 @@ def add_target_meta(api, targets, servers):
             targets[table]["endTime"] = df.loc[0, "endTime"]
         targets[table]["hasDepth"] = api.has_field(table, "depth", servers)
         targets[table]["isClimatology"] = api.is_climatology(table, servers)
+        targets[table]["aggregations"] = aggregations
         targets[table]["aliases"] = []
         for varName in targets[table]["variables"]:
-            targets[table]["aliases"].append(alias(varName, table))
+            targets[table]["aliases"].extend(alias(varName, table, aggregations))
     return targets
 
 
@@ -87,7 +131,13 @@ def match(df, api, targets, rowIndex, totalRows, replaceWithMonthlyClimatolog, s
             startTime = env["startTime"]
             endTime = env["endTime"]    
             inTimeRange = in_time_window(t, startTime, endTime)
-        selectClause = "SELECT " + ", ".join([f"AVG({v}) {a}" for v, a in zip(variables, aliases)]) + " FROM " + table
+        selectClause = "SELECT " + ", ".join(
+            [
+                f"{sqlFunc}({v}) CMAP_{v}_{table}_{suffix}"
+                for v in variables
+                for suffix, sqlFunc in env["aggregations"]
+            ]
+        ) + " FROM " + table
         timeClause = f" WHERE [time] BETWEEN '{shift_dt(t, -timeTolerance)}' AND '{shift_dt(t, timeTolerance)}' "
         if (not inTimeRange and replaceWithMonthlyClimatolog) or isClimatology: timeClause = f" WHERE [month]={get_month(t)} "
         latClause = f" AND lat BETWEEN {lat-latTolerance} AND {lat+latTolerance} "
@@ -108,10 +158,9 @@ def match(df, api, targets, rowIndex, totalRows, replaceWithMonthlyClimatolog, s
     if 'depth' in df.columns: depth = df.iloc[0]["depth"]
     for table, env in targets.items():
         print(f"\rSampling {table} ... {rowIndex+1} / {totalRows}" + " " * 50, end="", flush=True)
-        # # do the colocalization: if either the target dataset has depth field (it's not sattelite, for example) or 
-        # # the depth of source measurement is less than `MAX_SURFACE_DEPTH`
-        # if env["hasDepth"] or depth <= MAX_SURFACE_DEPTH:
-        if True:       
+        # do the colocalization if either the target dataset has depth field (it's not satellite, for example) or
+        # the depth of source measurement is less than `MAX_SURFACE_DEPTH`
+        if env["hasDepth"] or depth <= MAX_SURFACE_DEPTH: 
             query = construc_query(table, env, t, lat, lon, depth, replaceWithMonthlyClimatolog)
             matchedEnv = api.query(query, servers=servers)
             if len(matchedEnv)>0:
@@ -119,17 +168,17 @@ def match(df, api, targets, rowIndex, totalRows, replaceWithMonthlyClimatolog, s
     return df
 
 
-def Sample(source, targets, replaceWithMonthlyClimatolog, servers=["rossby"]):
+def Sample(source, targets, replaceWithMonthlyClimatolog=False, agg_fun=["AVG"], servers=["rossby"]):
     """
     placeholder for the `Sample` class.
 
-    Samples the targest datasets using the time-location of the source dataset
+    Samples the target datasets using the time-location of the source dataset
     Returns a dataframe containing the original source data and the joined colocalized target variables.
 
     :param dataframe source: a dataframe containing the source datasets (must have time-location columns).
-    :param dictionary targets: dcitionary containing the target table/variables and tolerance parameters.
-    The items in `tolerances` list are: temporal tolerance [days], meridional tolerance [deg], 
-    zonal tolerance [deg], and vertical tolerance [m], repectively.
+    :param dictionary targets: dictionary containing the target table/variables and tolerance parameters.
+    The items in `tolerances` list are: temporal tolerance [days], meridional tolerance [deg],
+    zonal tolerance [deg], and vertical tolerance [m], respectively.
     `targets` example:
 
     targets = {
@@ -143,11 +192,26 @@ def Sample(source, targets, replaceWithMonthlyClimatolog, servers=["rossby"]):
                                     }
             }
 
+    :param bool replaceWithMonthlyClimatolog: if True, when a source point falls outside the target dataset's
+        temporal range, fall back to matching by month (climatological match).
+    :param list agg_fun: SQL aggregation functions applied to each target variable.
+        Each function produces one output column per variable, named
+        ``CMAP_{var}_{table}_{suffix}`` where the suffix is the lowercased
+        function name (with friendlier short forms for the common ones:
+        ``AVG``->``avg``, ``STDEV``->``std``, ``COUNT``->``count``).
+        Case-insensitive. Defaults to ``["AVG"]`` (the historical behavior).
+        Example: ``["AVG", "STDEV", "COUNT"]``.
+    :param list servers: list of CMAP server names to query against.
+
     """
     if len(source) > MAX_SAMPLE_SOURCE: halt(f"Source dataset too large. Maximum allowed number of records is {MAX_SAMPLE_SOURCE}.")
+    aggregations = _resolve_aggregations(agg_fun)
     api = API()       
     print("Gathering metadata .... ")
-    targets = add_target_meta(api, targets, servers)
+    for tableName in targets.keys():
+        for variable in targets[tableName]["variables"]:
+            api._validate_table_var(tableName, variable)
+    targets = add_target_meta(api, targets, aggregations, servers)
     source = add_target_columns(source, targets)
     dfs = [source.loc[i].to_frame().T for i in range(len(source))]
     colocalizedList, columns = [], []
@@ -158,4 +222,4 @@ def Sample(source, targets, replaceWithMonthlyClimatolog, servers=["rossby"]):
             if len(colocalizedList) < 1: columns = list(fo.columns)                                                   
             colocalizedList.append(fo.values.tolist()[0])
     print("\rSampling finished" + " " * 100, end="")
-    return pd.DataFrame(colocalizedList, columns=columns)
+    return pd.DataFrame(colocalizedList, columns=columns)

pycmap-0.2.20/pycmap.egg-info/PKG-INFO

@@ -0,0 +1,109 @@
+Metadata-Version: 2.4
+Name: pycmap
+Version: 0.2.20
+Summary: Simons CMAP API python client
+Home-page: https://github.com/simonscmap/pycmap
+Author: Mohammad D. Ashkezari
+Author-email: mdehghan@uw.edu
+Keywords: ocean oceanography data visualization satellite model in-situ remote sensing machine learning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: scipy
+Requires-Dist: bokeh
+Requires-Dist: matplotlib
+Requires-Dist: cmocean
+Requires-Dist: tqdm
+Requires-Dist: folium
+Requires-Dist: colorama
+Requires-Dist: plotly
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
+
+[![PyPI version](https://badge.fury.io/py/pycmap.svg)](https://badge.fury.io/py/pycmap)
+[![DOI](https://zenodo.org/badge/199070692.svg)](https://zenodo.org/badge/latestdoi/199070692)
+![Cover](https://github.com/simonscmap/pycmap/raw/master/docs/figures/CMAP.png)
+
+*Mohammad Dehghani Ashkezari <mdehghan@uw.edu>*
+
+
+# pycmap
+Simons CMAP is an open-source data service to retrieve, visualize, and analyze oceanic datasets such as in-situ observations, multi-decade global satellite remote sensing, and model outputs. Pycmap is the python package of Simons CMAP project providing a simple and unified interface to the hosted datasets at Simons CMAP database. It enables the scientists and general public to dive into the vast, and often underutilized, ocean datasets and retrieve customized subsets of these massive datasets without going through the time-consuming process of data collection and preparation.
+
+## Documentation
+See the *table of contents* below for pycmap documentations. 
+<br />The docs are written in jupyter notebook format and each notebook contains one or more example codes. Please download the `/docs` directory and run the example codes locally. Alternatively, you may run the examples on google cloud using the *Colab badge* which appears at the top of each page: 
+<br /><img align="left" src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab" title="Open and Execute in Google Colaboratory">
+
+
+<br /><br />More detailed system documentations can be found at [https://cmap.readthedocs.io/en/latest/](https://cmap.readthedocs.io/en/latest/).
+
+## Website
+Simons CMAP website is under active development at the moment: [https://simonscmap.com](https://simonscmap.com)
+
+**<br />This project is supported by [Simons Foundation](https://www.simonsfoundation.org/).**
+
+
+
+
+## Table of Contents
+
+
+### [Installation](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Installation.ipynb)
+
+### [1. Data Retrieval (API)](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/API.ipynb)
+- [Query](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Query.ipynb)
+- [Catalog](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Catalog.ipynb)
+- [Search Catalog](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/SearchCatalog.ipynb)
+- [Datasets](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Datasets.ipynb)
+- [Datasets With Ancillary](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DatasetsWithAncillary.ipynb)
+- [Dataset MetaData](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DatasetMetaData.ipynb)
+- [Variable MetaData](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/MetaData.ipynb)
+- [Dataset Columns](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Columns.ipynb)
+- [Dataset Head](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Head.ipynb)
+- [Variable Long Name](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/LongName.ipynb)
+- [Variable Unit](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Unit.ipynb)
+- [Variable Resolution](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Resolution.ipynb)
+- [Variable Coverage](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Coverage.ipynb)
+- [Variable Stat](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Stat.ipynb)
+- [If Column Exists](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/HasField.ipynb)
+- [Is Gridded Product](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Grid.ipynb)
+- [Is Climatology Product](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/IsClimatology.ipynb)
+- [List of Cruises](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Cruises.ipynb)
+- [Cruise Details by Name](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseByName.ipynb)
+- [Cruise Spatio-Temporal Bounds](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseBounds.ipynb)
+- [Cruise Trajectory](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseTrajectory.ipynb)
+- [Cruise Variables](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/CruiseVariables.ipynb)
+- [Retrieve Dataset](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/RetrieveDataset.ipynb)
+- [Retrieve Dataset With Ancillary](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/RetrieveDatasetWithAncillary.ipynb)
+- [Data Subset: Generic Space-Time Cut](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/SpaceTime.ipynb)
+- [Data Subset: TimeSeries](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/TimeSeries.ipynb)
+- [Data Subset: Depth Profile](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/DepthProfile.ipynb)
+- [Compute Climatology](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Climatology.ipynb)
+- [Sampling](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Sampling.ipynb)
+- [Match (colocalize) Cruise Track with Datasets](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/MatchCruise.ipynb)
+
+
+### [2. Data Visualization](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz.ipynb)
+- [Histogram Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_Histogram.ipynb)
+- [TimeSeries Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_TimeSeries.ipynb)
+- [Regional Map, Contour Plot, 3D Surface Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_RegionalMap.ipynb)
+- [Section Map, Section Contour](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_Section.ipynb)
+- [Depth Profile](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_DepthProfile.ipynb)
+- [Cruise Track Plot](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CruiseTrack.ipynb)
+- [Correlation Matrix](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CorrelationMatrix.ipynb)
+- [Correlation Matrix Along Cruise Track](https://colab.research.google.com/github/simonscmap/pycmap/blob/master/docs/Viz_CruiseCorrelationMatrix.ipynb)
+

pycmap-0.2.20/pycmap.egg-info/SOURCES.txt

@@ -0,0 +1,29 @@
+LICENSE
+README.md
+setup.cfg
+setup.py
+pycmap/__init__.py
+pycmap/annotatedHeatmap.py
+pycmap/baseGraph.py
+pycmap/clean.py
+pycmap/cmap.py
+pycmap/colorMaps.py
+pycmap/common.py
+pycmap/deepC.py
+pycmap/export.py
+pycmap/foliumHeat.py
+pycmap/hist.py
+pycmap/map.py
+pycmap/match.py
+pycmap/rest.py
+pycmap/sample.py
+pycmap/section.py
+pycmap/supervised.py
+pycmap/trend.py
+pycmap/viz.py
+pycmap/w3.py
+pycmap.egg-info/PKG-INFO
+pycmap.egg-info/SOURCES.txt
+pycmap.egg-info/dependency_links.txt
+pycmap.egg-info/requires.txt
+pycmap.egg-info/top_level.txt