cs-service-api 20260531__tar.gz

cs_service_api-20260531/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: cs-service_api
+Version: 20260531
+Summary: ServiceAPI, a base class for APIs which talk to a service, typically a web service via HTTP.
+Keywords: python3
+Author-email: Cameron Simpson <cs@cskk.id.au>
+Description-Content-Type: text/markdown
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
+Requires-Dist: cs.deco>=20260525
+Requires-Dist: cs.fstags>=20260531
+Requires-Dist: cs.logutils>=20250323
+Requires-Dist: cs.pfx>=20250914
+Requires-Dist: cs.resources>=20250915
+Requires-Dist: cs.sqltags>=20260531
+Requires-Dist: cs.tagset>=20260531
+Requires-Dist: cs.upd>=20260526
+Requires-Dist: icontract
+Requires-Dist: requests
+Project-URL: MonoRepo Commits, https://bitbucket.org/cameron_simpson/css/commits/branch/main
+Project-URL: Monorepo Git Mirror, https://github.com/cameron-simpson/css
+Project-URL: Monorepo Hg/Mercurial Mirror, https://hg.sr.ht/~cameron-simpson/css
+Project-URL: Source, https://github.com/cameron-simpson/css/blob/main/lib/python/cs/service_api.py
+
+ServiceAPI, a base class for APIs which talk to a service,
+typically a web service via HTTP.
+
+*Latest release 20260531*:
+* HTTPServiceAPI: add __truediv__(suburl) which does a GET.
+* HTTPServiceAPI: default .API_HOSTNAME comes from the subclass.
+* HTTPServiceAPI.json: catch requests.exceptions.JSONDecodeError also.
+* HTTPServiceAPI: new mode ("response", "json", "data", callable) parameter.
+* HTTPServiceAPI: new check_json optional dict of expected fields and values in JSON responses.
+
+An instance of a `ServiceAPI` embodies some basic features
+that feel common to web based services:
+- a notion of a login
+- local state, an `SQLTags` for data about entities of the service
+- downloads, if that is a thing, with `FSTags` for file annotations
+
+Short summary:
+* `HTTPServiceAPI`: `HTTPServiceAPI` base class for other APIs talking to HTTP services.
+* `RequestsNoAuth`: This is a special purpose subclass of `requests.auth.AuthBase` to apply no authorisation at all. This is for services with their own special purpose authorisation and avoids things like automatic netrc based auth.
+* `ServiceAPI`: `SewrviceAPI` base class for other APIs talking to services.
+
+Module contents:
+- <a name="HTTPServiceAPI"></a>`class HTTPServiceAPI(ServiceAPI)`: `HTTPServiceAPI` base class for other APIs talking to HTTP services.
+
+  Subclasses must define:
+  * `API_BASE`: the base URL of API calls.
+    For example, the `PlayOnAPI` defines this as `f'https://{API_HOSTNAME}/v3/'`.
+
+*`HTTPServiceAPI.get(self, suburl, **kw) -> requests.models.Response`*:
+Call `slef.suburl` with `method="GET"`.
+
+*`HTTPServiceAPI.post(self, suburl, **kw) -> requests.models.Response`*:
+Call `slef.suburl` with `method="POST"`.
+
+*`HTTPServiceAPI.response_as_json(self, rsp: requests.models.Response) -> dict`*:
+Return `rsp.json()`.
+
+*`HTTPServiceAPI.response_as_json_data(self, rsp: requests.models.Response) -> dict`*:
+Return the `"data"` element from a JSON response.
+
+*`HTTPServiceAPI.suburl(self, suburl, *, base_url=None, method='GET', mode=None, check=True, cookies=None, headers=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at 0x111844fe0>, verbose: bool, **rqkw) -> Union[requests.models.Response, dict]`*:
+Request `suburl` from the service, by default using a `GET`.
+The `suburl` must be a URL subpath not commencing with `'/'`.
+
+Return:
+- `mode(Response)` if `mode` is callable
+- the `Response` if `mode=="response"`
+- the `Response.json()` if `mode=="json"`
+- the `Response.json()["data"]` if `mode=="data"`
+
+Keyword parameters:
+* `base_url`: the base request domain, default from `self.API_BASE`
+* `method`: optional request method, default `'GET'`
+* `check`: if true, raise an HTTP error if the response
+  status is not 200; default `True`
+* `cookies`: optional cookie jar, default from `self.cookies`
+* `mode`: optional result mode, default from `self.mode`
+Other keyword parameters are passed to the requests method.
+- <a name="RequestsNoAuth"></a>`class RequestsNoAuth(requests.auth.AuthBase)`: This is a special purpose subclass of `requests.auth.AuthBase`
+  to apply no authorisation at all.
+  This is for services with their own special purpose authorisation
+  and avoids things like automatic netrc based auth.
+- <a name="ServiceAPI"></a>`class ServiceAPI(cs.resources.MultiOpenMixin, cs.sqltags.UsesSQLTags)`: `SewrviceAPI` base class for other APIs talking to services.
+
+*`ServiceAPI.available(self) -> Set[cs.sqltags.SQLTagSet]`*:
+Return a set of the `SQLTagSet` instances representing available
+items at the service, for example purchased books
+available to your login.
+
+*`ServiceAPI.get_login_state(self, do_refresh=False) -> cs.tagset.HasTags`*:
+The login state, a `HasTags`, stored as `login.state.`*login_userid*.
+This performs a login if necessary or if `do_refresh` is true
+(default `False`).
+
+*`ServiceAPI.login(self) -> Mapping`*:
+Do a login: authenticate to the service, return a mapping of related information.
+
+Not all services require this and we expect such subclasses
+to avoid use of login-based methods.
+
+*`ServiceAPI.login_expiry`*:
+Expiry UNIX time for the login state.
+This implementation returns `None`.
+
+*`ServiceAPI.startup_shutdown(self)`*:
+Open/close the `FSTags` and `UsesTagSets`.
+
+# Release Log
+
+
+
+*Release 20260531*:
+* HTTPServiceAPI: add __truediv__(suburl) which does a GET.
+* HTTPServiceAPI: default .API_HOSTNAME comes from the subclass.
+* HTTPServiceAPI.json: catch requests.exceptions.JSONDecodeError also.
+* HTTPServiceAPI: new mode ("response", "json", "data", callable) parameter.
+* HTTPServiceAPI: new check_json optional dict of expected fields and values in JSON responses.
+
+*Release 20241007*:
+HTTPServiceAPI.suburl: support interruption by RunState.cancel.
+
+*Release 20240723*:
+ServiceAPI: acquire the fstags automatically at init.
+
+*Release 20230703*:
+Retry logic for requests.
+
+*Release 20230217*:
+Initial release.

cs_service_api-20260531/pyproject.toml

@@ -0,0 +1,162 @@
+[project]
+name = "cs-service_api"
+description = "ServiceAPI, a base class for APIs which talk to a service, typically a web service via HTTP."
+authors = [
+    { name = "Cameron Simpson", email = "cs@cskk.id.au" },
+]
+keywords = [
+    "python3",
+]
+dependencies = [
+    "cs.deco>=20260525",
+    "cs.fstags>=20260531",
+    "cs.logutils>=20250323",
+    "cs.pfx>=20250914",
+    "cs.resources>=20250915",
+    "cs.sqltags>=20260531",
+    "cs.tagset>=20260531",
+    "cs.upd>=20260526",
+    "icontract",
+    "requests",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
+]
+version = "20260531"
+
+[project.license]
+text = "GNU General Public License v3 or later (GPLv3+)"
+
+[project.urls]
+"Monorepo Hg/Mercurial Mirror" = "https://hg.sr.ht/~cameron-simpson/css"
+"Monorepo Git Mirror" = "https://github.com/cameron-simpson/css"
+"MonoRepo Commits" = "https://bitbucket.org/cameron_simpson/css/commits/branch/main"
+Source = "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/service_api.py"
+
+[project.readme]
+text = """
+ServiceAPI, a base class for APIs which talk to a service,
+typically a web service via HTTP.
+
+*Latest release 20260531*:
+* HTTPServiceAPI: add __truediv__(suburl) which does a GET.
+* HTTPServiceAPI: default .API_HOSTNAME comes from the subclass.
+* HTTPServiceAPI.json: catch requests.exceptions.JSONDecodeError also.
+* HTTPServiceAPI: new mode (\"response\", \"json\", \"data\", callable) parameter.
+* HTTPServiceAPI: new check_json optional dict of expected fields and values in JSON responses.
+
+An instance of a `ServiceAPI` embodies some basic features
+that feel common to web based services:
+- a notion of a login
+- local state, an `SQLTags` for data about entities of the service
+- downloads, if that is a thing, with `FSTags` for file annotations
+
+Short summary:
+* `HTTPServiceAPI`: `HTTPServiceAPI` base class for other APIs talking to HTTP services.
+* `RequestsNoAuth`: This is a special purpose subclass of `requests.auth.AuthBase` to apply no authorisation at all. This is for services with their own special purpose authorisation and avoids things like automatic netrc based auth.
+* `ServiceAPI`: `SewrviceAPI` base class for other APIs talking to services.
+
+Module contents:
+- <a name=\"HTTPServiceAPI\"></a>`class HTTPServiceAPI(ServiceAPI)`: `HTTPServiceAPI` base class for other APIs talking to HTTP services.
+
+  Subclasses must define:
+  * `API_BASE`: the base URL of API calls.
+    For example, the `PlayOnAPI` defines this as `f'https://{API_HOSTNAME}/v3/'`.
+
+*`HTTPServiceAPI.get(self, suburl, **kw) -> requests.models.Response`*:
+Call `slef.suburl` with `method=\"GET\"`.
+
+*`HTTPServiceAPI.post(self, suburl, **kw) -> requests.models.Response`*:
+Call `slef.suburl` with `method=\"POST\"`.
+
+*`HTTPServiceAPI.response_as_json(self, rsp: requests.models.Response) -> dict`*:
+Return `rsp.json()`.
+
+*`HTTPServiceAPI.response_as_json_data(self, rsp: requests.models.Response) -> dict`*:
+Return the `\"data\"` element from a JSON response.
+
+*`HTTPServiceAPI.suburl(self, suburl, *, base_url=None, method='GET', mode=None, check=True, cookies=None, headers=None, runstate: Optional[cs.resources.RunState] = <function uses_runstate.<locals>.<lambda> at 0x111844fe0>, verbose: bool, **rqkw) -> Union[requests.models.Response, dict]`*:
+Request `suburl` from the service, by default using a `GET`.
+The `suburl` must be a URL subpath not commencing with `'/'`.
+
+Return:
+- `mode(Response)` if `mode` is callable
+- the `Response` if `mode==\"response\"`
+- the `Response.json()` if `mode==\"json\"`
+- the `Response.json()[\"data\"]` if `mode==\"data\"`
+
+Keyword parameters:
+* `base_url`: the base request domain, default from `self.API_BASE`
+* `method`: optional request method, default `'GET'`
+* `check`: if true, raise an HTTP error if the response
+  status is not 200; default `True`
+* `cookies`: optional cookie jar, default from `self.cookies`
+* `mode`: optional result mode, default from `self.mode`
+Other keyword parameters are passed to the requests method.
+- <a name=\"RequestsNoAuth\"></a>`class RequestsNoAuth(requests.auth.AuthBase)`: This is a special purpose subclass of `requests.auth.AuthBase`
+  to apply no authorisation at all.
+  This is for services with their own special purpose authorisation
+  and avoids things like automatic netrc based auth.
+- <a name=\"ServiceAPI\"></a>`class ServiceAPI(cs.resources.MultiOpenMixin, cs.sqltags.UsesSQLTags)`: `SewrviceAPI` base class for other APIs talking to services.
+
+*`ServiceAPI.available(self) -> Set[cs.sqltags.SQLTagSet]`*:
+Return a set of the `SQLTagSet` instances representing available
+items at the service, for example purchased books
+available to your login.
+
+*`ServiceAPI.get_login_state(self, do_refresh=False) -> cs.tagset.HasTags`*:
+The login state, a `HasTags`, stored as `login.state.`*login_userid*.
+This performs a login if necessary or if `do_refresh` is true
+(default `False`).
+
+*`ServiceAPI.login(self) -> Mapping`*:
+Do a login: authenticate to the service, return a mapping of related information.
+
+Not all services require this and we expect such subclasses
+to avoid use of login-based methods.
+
+*`ServiceAPI.login_expiry`*:
+Expiry UNIX time for the login state.
+This implementation returns `None`.
+
+*`ServiceAPI.startup_shutdown(self)`*:
+Open/close the `FSTags` and `UsesTagSets`.
+
+# Release Log
+
+
+
+*Release 20260531*:
+* HTTPServiceAPI: add __truediv__(suburl) which does a GET.
+* HTTPServiceAPI: default .API_HOSTNAME comes from the subclass.
+* HTTPServiceAPI.json: catch requests.exceptions.JSONDecodeError also.
+* HTTPServiceAPI: new mode (\"response\", \"json\", \"data\", callable) parameter.
+* HTTPServiceAPI: new check_json optional dict of expected fields and values in JSON responses.
+
+*Release 20241007*:
+HTTPServiceAPI.suburl: support interruption by RunState.cancel.
+
+*Release 20240723*:
+ServiceAPI: acquire the fstags automatically at init.
+
+*Release 20230703*:
+Retry logic for requests.
+
+*Release 20230217*:
+Initial release."""
+content-type = "text/markdown"
+
+[build-system]
+build-backend = "flit_core.buildapi"
+requires = [
+    "flit_core >=3.2,<4",
+]
+
+[tool.flit.module]
+name = "cs.service_api"

cs_service_api-20260531/src/cs/service_api.py

@@ -0,0 +1,289 @@
+#!/usr/bin/env python3
+
+''' ServiceAPI, a base class for APIs which talk to a service,
+    typically a web service via HTTP.
+
+    An instance of a `ServiceAPI` embodies some basic features
+    that feel common to web based services:
+    - a notion of a login
+    - local state, an `SQLTags` for data about entities of the service
+    - downloads, if that is a thing, with `FSTags` for file annotations
+'''
+
+from contextlib import contextmanager
+from functools import cached_property
+from json import JSONDecodeError
+from threading import RLock
+import time
+from typing import Mapping, Set, Union
+
+from icontract import require
+import requests
+from requests import Response
+try:
+  from requests.exceptions import JSONDecodeError as RequestsJSONDecodeError
+except ImportError:
+  RequestsJSONDecodeError = JSONDecodeError
+
+from cs.deco import uses_verbose
+from cs.fstags import FSTags, uses_fstags
+from cs.logutils import warning
+from cs.pfx import pfx_call
+from cs.resources import MultiOpenMixin, RunState, uses_runstate
+from cs.tagset import HasTags, UsesTagSets
+from cs.sqltags import SQLTagSet, UsesSQLTags
+from cs.upd import run_task
+
+__version__ = '20260531'
+
+DISTINFO = {
+    'keywords': ["python3"],
+    'classifiers': [
+        "Development Status :: 3 - Alpha",
+        "Programming Language :: Python",
+        "Programming Language :: Python :: 3",
+    ],
+    'install_requires': [
+        'cs.deco',
+        'cs.fstags',
+        'cs.logutils',
+        'cs.pfx',
+        'cs.resources',
+        'cs.tagset',
+        'cs.sqltags',
+        'cs.upd',
+        'icontract',
+        'requests',
+    ],
+}
+
+class ServiceAPI(MultiOpenMixin, UsesSQLTags):
+  ''' `SewrviceAPI` base class for other APIs talking to services.
+  '''
+
+  API_AUTH_GRACETIME = None
+  API_RETRY_COUNT = 3  # number of request attempts
+  API_RETRY_DELAY = 5  # interval between request retries
+
+  @uses_fstags
+  def __init__(
+      self,
+      *,
+      fstags: FSTags,
+      tagsets: UsesTagSets = None,
+  ):
+    super().__init__(tagsets=tagsets)
+    self.fstags = fstags
+    self._lock = RLock()
+    self.login_state_mapping = None
+
+  @contextmanager
+  def startup_shutdown(self):
+    ''' Open/close the `FSTags` and `UsesTagSets`.
+    '''
+    with self.tagsets:
+      with self.fstags:
+        yield
+
+  def login(self) -> Mapping:
+    ''' Do a login: authenticate to the service, return a mapping of related information.
+
+        Not all services require this and we expect such subclasses
+        to avoid use of login-based methods.
+    '''
+    raise NotImplementedError
+
+  @property
+  def login_expiry(self):
+    ''' Expiry UNIX time for the login state.
+        This implementation returns `None`.
+    '''
+    return None
+
+  def get_login_state(self, do_refresh=False) -> HasTags:
+    ''' The login state, a `HasTags`, stored as `login.state.`*login_userid*.
+        This performs a login if necessary or if `do_refresh` is true
+        (default `False`).
+    '''
+    with self._lock:
+      state = self['login.state', self.login_userid.replace('.', '_')]
+      if do_refresh or (self.API_AUTH_GRACETIME is not None and
+                        time.time() + self.API_AUTH_GRACETIME >= state.expiry):
+        for k, v in self.login().items():
+          if k not in ('id', 'name'):
+            state[k] = v
+    return state
+
+  @cached_property
+  def login_state(self) -> HasTags:
+    ''' The login state, a mapping. Performs a login if necessary.
+    '''
+    return self.get_login_state()
+
+  def available(self) -> Set[SQLTagSet]:
+    ''' Return a set of the `SQLTagSet` instances representing available
+        items at the service, for example purchased books
+        available to your login.
+    '''
+    raise NotImplementedError
+
+class HTTPServiceAPI(ServiceAPI):
+  ''' `HTTPServiceAPI` base class for other APIs talking to HTTP services.
+
+      Subclasses must define:
+      * `API_BASE`: the base URL of API calls.
+        For example, the `PlayOnAPI` defines this as `f'https://{API_HOSTNAME}/v3/'`.
+  '''
+
+  def __init__(
+      self,
+      api_hostname=None,
+      *,
+      default_headers=None,
+      mode='response',
+      check_json=None,
+      **service_api_kw
+  ):
+    if api_hostname is None:
+      api_hostname = type(self).API_HOSTNAME
+    else:
+      self.API_HOSTNAME = api_hostname
+      self.API_BASE = f'https://{api_hostname}/'
+    if default_headers is None:
+      default_headers = {}
+    super().__init__(**service_api_kw)
+    session = self.session = requests.Session()
+    # mapping of method names to requests convenience calls
+    self.REQUESTS_METHOD_CALLS = {
+        'GET': session.get,
+        'POST': session.post,
+        'HEAD': session.head,
+    }
+    self.cookies = session.cookies
+    self.default_headers = default_headers
+    self.mode = mode
+    self.check_json = check_json
+
+  def response_as_json(self, rsp: Response) -> dict:
+    ''' Return `rsp.json()`.
+    '''
+    try:
+      js = rsp.json()
+    except (JSONDecodeError, RequestsJSONDecodeError) as e:
+      warning("response is not JSON: %s\n%r", e, rsp)
+      raise
+    if self.check_json:
+      for field, value in self.check_json.items():
+        js_value = js.get(field)
+        if js_value != value:
+          warning(f'expected {field}={value!r}, got {js_value!r} from {js!r}')
+    return js
+
+  def response_as_json_data(self, rsp: Response) -> dict:
+    ''' Return the `"data"` element from a JSON response.
+    '''
+    js = self.response_as_json(rsp)
+    return js["data"]
+
+  @uses_runstate
+  @uses_verbose
+  @require(lambda suburl: not suburl.startswith('/'))
+  def suburl(
+      self,
+      suburl,
+      *,
+      base_url=None,
+      method='GET',
+      mode=None,
+      check=True,
+      cookies=None,
+      headers=None,
+      runstate: RunState,
+      verbose: bool,
+      **rqkw,
+  ) -> Union[Response, dict]:
+    ''' Request `suburl` from the service, by default using a `GET`.
+        The `suburl` must be a URL subpath not commencing with `'/'`.
+
+        Return:
+        - `mode(Response)` if `mode` is callable
+        - the `Response` if `mode=="response"`
+        - the `Response.json()` if `mode=="json"`
+        - the `Response.json()["data"]` if `mode=="data"`
+
+        Keyword parameters:
+        * `base_url`: the base request domain, default from `self.API_BASE`
+        * `method`: optional request method, default `'GET'`
+        * `check`: if true, raise an HTTP error if the response
+          status is not 200; default `True`
+        * `cookies`: optional cookie jar, default from `self.cookies`
+        * `mode`: optional result mode, default from `self.mode`
+        Other keyword parameters are passed to the requests method.
+    '''
+    rqm = self.REQUESTS_METHOD_CALLS[method]
+    if base_url is None:
+      base_url = self.API_BASE
+    if cookies is None:
+      cookies = self.cookies
+    if mode is None:
+      mode = self.mode
+    url = base_url + suburl
+    rq_headers = {}
+    rq_headers.update(self.default_headers)
+    if headers is not None:
+      rq_headers.update(headers)
+    with run_task(f'{method} {url}', report_print=verbose):
+      for retry in range(self.API_RETRY_COUNT, 0, -1):
+        runstate.raiseif()
+        try:
+          rsp = pfx_call(
+              rqm,
+              url,
+              cookies=cookies,
+              headers=rq_headers,
+              **rqkw,
+          )
+          break
+        except requests.ConnectionError as e:
+          if retry <= 1:
+            # last retry
+            raise
+          warning("%s: %s, retrying in %ds", url, e, self.API_RETRY_DELAY)
+          runstate.sleep(self.API_RETRY_DELAY)
+    if check:
+      rsp.raise_for_status()
+    if callable(mode):
+      return mode(rsp)
+    if mode == 'response':
+      return rsp
+    if mode == 'json':
+      return self.response_as_json(rsp)
+    if mode == 'data':
+      return self.response_as_json_data(rsp)
+    raise ValueError(
+        f'unsupported {mode=}, expected "data", "json", "response" or a callable'
+    )
+
+  def get(self, suburl, **kw) -> Response:
+    ''' Call `slef.suburl` with `method="GET"`.
+    '''
+    return self, suburl(suburl, method='GET', **kw)
+
+  def post(self, suburl, **kw) -> Response:
+    ''' Call `slef.suburl` with `method="POST"`.
+    '''
+    return self.suburl(suburl, method='POST', **kw)
+
+  def __truediv__(self, suburl):
+    return self.suburl(suburl)
+
+# pylint: disable=too-few-public-methods
+class RequestsNoAuth(requests.auth.AuthBase):
+  ''' This is a special purpose subclass of `requests.auth.AuthBase`
+      to apply no authorisation at all.
+      This is for services with their own special purpose authorisation
+      and avoids things like automatic netrc based auth.
+  '''
+
+  def __call__(self, r):
+    return r