libmongocrypt-helper 1.8.0.0.1001 → 1.11.0.0.1001

data/ext/libmongocrypt/libmongocrypt/bindings/python/CHANGELOG.rst

@@ -1,10 +1,70 @@
 Changelog
 =========
 
+Changes in Version 1.10.1
+------------------------
+
+- Bundle libmongocrypt 1.10.1 in release wheels.
+
+Changes in Version 1.10.0
+------------------------
+
+- Add Python async support.
+- Drop support for Python 3.7 and PyPy 3.8. Python >=3.8 or PyPy >=3.9 is now required.
+- Add support for range-based Queryable Encryption with the new "range"
+  algorithm on MongoDB 8.0+. This replaces the experimental "rangePreview" algorithm.
+- Add Secure Software Development Life Cycle automation to release process.
+  GitHub Releases for pymongocrypt now include a Software Bill of Materials, and signature
+  files corresponding to the distribution files released on PyPI.
+
+Changes in Version 1.9.2
+------------------------
+
+- Fix support for building source distributions with setuptools >= 70.
+
+
+Changes in Version 1.9.1
+------------------------
+
+- Fix bug in our release process which blocked uploading 1.9.0.
+
+Changes in Version 1.9.0
+------------------------
+
+- Add support for named KMS providers like "local:name1".
+  This feature requires libmongocrypt >= 1.9.0.
+- Use libmongocrypt native crypto when available which results in 10-50x better performance.
+  On Linux, it is recommended to download the platform specific build and
+  set PYMONGOCRYPT_LIB to the crypto-enabled libmongocrypt.so.
+- Bundle the crypto-enabled libmongocrypt builds in macOS and Windows wheels for better performance.
+- Bundle libmongocrypt 1.9.0 in release wheels.
+
+Changes in Version 1.8.0
+------------------------
+
+- Update from manylinux2010 to manylinux2014 wheels.
+- Bundle libmongocrypt 1.8.4 in release wheels.
+- Add support for manylinux_2_28_aarch64 wheels.
+
+Changes in Version 1.7.0
+------------------------
+
+- Add support for Python 3.12 on MacOS and Linux.
+- Update required cryptography version to >=2.5.
+
+Changes in Version 1.6.1
+------------------------
+
+- Bundle libmongocrypt 1.8.1 in release wheels.
+
 Changes in Version 1.6.0
 ------------------------
 
 - Drop support for Python 2 and Python <3.7. Python >=3.7 is now required.
+- Bundle libmongocrypt 1.8.0 in release wheels.
+- **Remove support for libmongocrypt <=1.8.0, libmongocrypt >=1.8.0
+  is now required.** Note this is only relevant for users that install from
+  source or use the ``PYMONGOCRYPT_LIB`` environment variable.
 
 Changes in Version 1.5.2
 ------------------------

data/ext/libmongocrypt/libmongocrypt/bindings/python/README.rst

@@ -13,7 +13,7 @@ Python wrapper library for libmongocrypt that supports client side encryption
 in drivers. PyMongoCrypt uses `cffi <https://pypi.org/project/cffi/>`_ and
 `cryptography <https://pypi.org/project/cryptography/>`_.
 
-PyMongoCrypt supports Python 3.7+ and PyPy3.7+.
+PyMongoCrypt supports Python 3.8+ and PyPy3.9+.
 
 Support / Feedback
 ==================
@@ -77,7 +77,7 @@ PyMongoCrypt can be installed with `pip <http://pypi.python.org/pypi/pip>`_::
 
   $ python -m pip install pymongocrypt
   $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
-  1.2.1
+  1.9.0
 
 
 PyMongoCrypt ships wheels for macOS, Windows, and manylinux2010 that include
@@ -102,7 +102,10 @@ First, install PyMongoCrypt from source::
   $ git clone git@github.com:mongodb/libmongocrypt.git
   $ python -m pip install ./libmongocrypt/bindings/python
 
-Next, install libmongocrypt.
+Next, install libmongocrypt:
+
+Installing libmongocrypt
+^^^^^^^^^^^^^^^^^^^^^^^^
 
 libmongocrypt is continuously built and published on evergreen.
 The latest tarball containing libmongocrypt built on all supported variants is
@@ -114,38 +117,56 @@ For example::
   $ curl -O https://s3.amazonaws.com/mciuploads/libmongocrypt/all/master/latest/libmongocrypt-all.tar.gz
   $ mkdir libmongocrypt-all && tar xzf libmongocrypt-all.tar.gz -C libmongocrypt-all
   $ ls libmongocrypt-all
-  amazon2             rhel-62-64-bit      rhel72-zseries-test ubuntu1604-arm64
-  debian10            rhel-67-s390x       suse12-64           ubuntu1804-64
-  debian92            rhel-70-64-bit      suse12-s390x        ubuntu1804-arm64
-  linux-64-amazon-ami rhel-71-ppc64el     suse15-64           windows-test
-  macos               rhel-80-64-bit      ubuntu1604
+  amazon2             debian92            rhel-80-64-bit      rhel72-zseries-test ubuntu1804-arm64
+  amazon2-arm64       linux-64-amazon-ami rhel-81-ppc64el     suse12-64           ubuntu2004-64
+  amazon2023          macos               rhel-82-arm64       suse15-64           ubuntu2004-arm64
+  amazon2023-arm64    rhel-62-64-bit      rhel-83-zseries     ubuntu1604          ubuntu2204-64
+  debian10            rhel-70-64-bit      rhel-91-64-bit      ubuntu1604-arm64    ubuntu2204-arm64
+  debian11            rhel-71-ppc64el     rhel-91-arm64       ubuntu1804-64       windows-test
 
 macOS::
 
   $ # Set PYMONGOCRYPT_LIB for macOS:
-  $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/macos/nocrypto/lib/libmongocrypt.dylib
+  $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/macos/lib/libmongocrypt.dylib
   $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
-  1.2.1
+  1.9.0
 
 Windows::
 
   $ # Set PYMONGOCRYPT_LIB for Windows:
-  $ chmod +x $(pwd)/libmongocrypt-all/windows-test/nocrypto/bin/mongocrypt.dll
-  $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/windows-test/nocrypto/bin/mongocrypt.dll
+  $ chmod +x $(pwd)/libmongocrypt-all/windows-test/bin/mongocrypt.dll
+  $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/windows-test/bin/mongocrypt.dll
   $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
-  1.2.1
+  1.9.0
+
+Linux: set the libmongocrypt build for your platform, for example for Ubuntu 22.04 x86_64::
 
-Linux::
+  $ # Set PYMONGOCRYPT_LIB for Ubuntu 22.04 x86_64:
+  $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/ubuntu2204-64/lib/libmongocrypt.so
+  $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
+  1.9.0
+  $ # Check that native crypto is enabled for better performance:
+  $ python -c 'from pymongocrypt.binding import lib;print(lib.mongocrypt_is_crypto_available())'
+  True
+
+Note if your Linux platform is not available, the generic RHEL 6.2 x86_64 "nocrypto" build
+should still be compatible however the "nocrypto" build will result in lower performance
+for encryption and decryption::
 
   $ # Set PYMONGOCRYPT_LIB for RHEL 6.2 x86_64:
   $ export PYMONGOCRYPT_LIB=$(pwd)/libmongocrypt-all/rhel-62-64-bit/nocrypto/lib64/libmongocrypt.so
   $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
-  1.2.1
+  1.9.0
+  $ python -c 'from pymongocrypt.binding import lib;print(lib.mongocrypt_is_crypto_available())'
+  False
+
+Other methods of installation (brew, rpm, yum, apt-get, deb, etc...) are documented here:
+https://www.mongodb.com/docs/manual/core/csfle/reference/libmongocrypt/#linux-installation
 
 Dependencies
 ============
 
-PyMongoCrypt supports Python 3.7+ and PyPy3.7+.
+PyMongoCrypt supports Python 3.8+ and PyPy3.9+.
 
 PyMongoCrypt requires `cffi <https://pypi.org/project/cffi/>`_ and
 `cryptography <https://pypi.org/project/cryptography/>`_.
@@ -163,11 +184,11 @@ installed you will see an error like this:
       from pymongocrypt.binding import libmongocrypt_version, lib
     File "pymongocrypt/binding.py", line 803, in <module>
       lib = ffi.dlopen(os.environ.get('PYMONGOCRYPT_LIB', 'mongocrypt'))
-    File "/.../lib/python3.7/site-packages/cffi/api.py", line 146, in dlopen
+    File "/.../lib/python3.8/site-packages/cffi/api.py", line 146, in dlopen
       lib, function_cache = _make_ffi_library(self, name, flags)
-    File "/.../lib/python3.7/site-packages/cffi/api.py", line 828, in _make_ffi_library
+    File "/.../lib/python3.8/site-packages/cffi/api.py", line 828, in _make_ffi_library
       backendlib = _load_backend_lib(backend, libname, flags)
-    File "/.../lib/python3.7/site-packages/cffi/api.py", line 823, in _load_backend_lib
+    File "/.../lib/python3.8/site-packages/cffi/api.py", line 823, in _load_backend_lib
       raise OSError(msg)
   OSError: ctypes.util.find_library() did not manage to locate a library called 'mongocrypt'
 
@@ -178,7 +199,7 @@ variables, like ``LD_LIBRARY_PATH``. For example::
 
   $ export PYMONGOCRYPT_LIB='/path/to/libmongocrypt.so'
   $ python -c "import pymongocrypt; print(pymongocrypt.libmongocrypt_version())"
-  1.2.1
+  1.9.0
 
 Testing
 =======

data/ext/libmongocrypt/libmongocrypt/bindings/python/RELEASE.rst

@@ -41,7 +41,7 @@ Docker and a Windows machine.
      -REVISION=$(git rev-list -n 1 1.0.0)
      +REVISION=$(git rev-list -n 1 1.0.1)
 
-#. Add a changlog entry for this release in CHANGELOG.rst.
+#. Add a changelog entry for this release in CHANGELOG.rst.
 #. Bump "__version__" in ``pymongocrypt/version.py``.
 #. After merging the PR, clone the repository and check out the commit
    with the version change.
@@ -51,25 +51,9 @@ Docker and a Windows machine.
    $ git tag -a "pymongocrypt-<version>" -m "pymongocrypt-<version"
    $ git push --tags
 
-#. Pushing a tag will trigger a release process in Evergreen which builds
-   wheels for manylinux, macOS, and Windows. Wait for the "release-python-combine"
-   task to complete and then download the "Release Python files all" archive. See:
-   https://evergreen.mongodb.com/waterfall/libmongocrypt?bv_filter=release
-   (requires auth)
-
-   The contents should look like this::
-
-     $ ls path/to/archive
-     pymongocrypt-<version>.tar.gz
-     pymongocrypt-<version>-py2.py3-none-manylinux2010_x86_64.whl
-     pymongocrypt-<version>-py2.py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
-     pymongocrypt-<version>-py2.py3-none-macosx_10_14_x86_64.whl
-     pymongocrypt-<version>-py2.py3-none-macosx_11_0_universal2.whl
-     pymongocrypt-<version>-py2.py3-none-win_amd64.whl
-
-#. Upload all the release packages to PyPI with twine::
-
-     $ python3 -m twine upload dist/*
+#. Pushing a tag will trigger the release process on GitHub Actions that will require a member
+   of the team to authorize the deployment. Navigate to https://github.com/mongodb/libmongocrypt/actions/workflows/release-python.yml
+   and wait for the publish to complete.
 
 #. Create a new PR against the same ticket to update version to a ``.dev0``
    version.
@@ -84,10 +68,10 @@ Manually Creating Wheels
      $ git clone git@github.com:mongodb/libmongocrypt.git
      $ cd libmongocrypt/bindings/python
      $ git checkout "pymongocrypt <release version number>"
-     $ MACOS_TARGET=macos_x86_64 PYTHON=<python37> ./release.sh
+     $ MACOS_TARGET=macos_x86_64 PYTHON=<python38> ./release.sh
      $ PYTHON=<python310> ./release.sh
 
-  Make sure to run using the official binaries for Python 3.7 and 3.10.  You
+  Make sure to run using the official binaries for Python 3.8 and 3.10.  You
   should end up with the same files created by Evergreen (except for the Windows wheel).
 
 #. To build the release package for Windows, launch a windows-64-vsMulti-small
@@ -103,5 +87,3 @@ Manually Creating Wheels
 
      $ ls dist
      pymongocrypt-<version>-py2.py3-none-win_amd64.whl
-
-

data/ext/libmongocrypt/libmongocrypt/bindings/python/build-manylinux-wheel.sh

@@ -1,16 +1,7 @@
 #!/bin/bash -ex
 cd /python
 
-# Compile wheel
-# https://github.com/pypa/manylinux/issues/49
-rm -rf build
-/opt/python/cp37-cp37m/bin/python setup.py bdist_wheel
-
-# Audit wheels and write manylinux tag
-for whl in dist/*.whl; do
-    # Skip already built manylinux wheels.
-    if [[ "$whl" != *"manylinux"* ]]; then
-        auditwheel repair $whl -w dist
-        rm $whl
-    fi
-done
+mkdir /tmp/wheelhouse
+/opt/python/cp38-cp38/bin/python -m build --wheel --outdir /tmp/wheelhouse
+# Audit wheels and repair manylinux tags
+auditwheel repair /tmp/wheelhouse/*.whl -w dist

data/ext/libmongocrypt/libmongocrypt/bindings/python/hatch_build.py

@@ -0,0 +1,36 @@
+"""A custom hatch build hook for pymongo."""
+from __future__ import annotations
+
+import os
+import sys
+from pathlib import Path
+
+from hatchling.builders.hooks.plugin.interface import BuildHookInterface
+
+
+class CustomHook(BuildHookInterface):
+    """The pymongocrypt build hook."""
+
+    def initialize(self, version, build_data):
+        """Initialize the hook."""
+        if self.target_name == "sdist":
+            return
+
+        # Ensure wheel is marked as binary.
+        # On linux, we use auditwheel to set the name.
+        if sys.platform == "darwin":
+            os.environ["MACOSX_DEPLOYMENT_TARGET"] = "11.0"
+            build_data["tag"] = "py3-none-macosx_11_0_universal2"
+            patt = ".dylib"
+        elif os.name == "nt":
+            build_data["tag"] = "py3-none-win_amd64"
+            patt = ".dll"
+        else:
+            patt = ".so"
+
+        here = Path(__file__).parent.resolve()
+        dpath = here / "pymongocrypt"
+        for fpath in dpath.glob(f"*{patt}"):
+            relpath = os.path.relpath(fpath, here)
+            build_data["artifacts"].append(relpath)
+            build_data["force_include"][relpath] = relpath

data/ext/libmongocrypt/libmongocrypt/bindings/python/libmongocrypt-version.txt

@@ -0,0 +1 @@
+1.10.1

data/ext/libmongocrypt/libmongocrypt/bindings/python/pymongocrypt/__init__.py

@@ -12,5 +12,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from pymongocrypt.binding import libmongocrypt_version, lib
-from pymongocrypt.version import __version__
+from pymongocrypt.binding import lib, libmongocrypt_version  # noqa: F401
+from pymongocrypt.version import __version__  # noqa: F401

data/ext/libmongocrypt/libmongocrypt/bindings/python/pymongocrypt/asynchronous/auto_encrypter.py

@@ -0,0 +1,61 @@
+# Copyright 2024-present MongoDB, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from pymongocrypt.asynchronous.state_machine import run_state_machine
+from pymongocrypt.mongocrypt import MongoCrypt
+
+
+class AsyncAutoEncrypter:
+    def __init__(self, callback, mongo_crypt_opts):
+        """Encrypts and decrypts MongoDB commands.
+
+        This class is used by a driver to support automatic encryption and
+        decryption of MongoDB commands.
+
+        :Parameters:
+          - `callback`: A :class:`MongoCryptCallback`.
+          - `mongo_crypt_opts`: A :class:`MongoCryptOptions`.
+        """
+        self.callback = callback
+        self.mongocrypt = MongoCrypt(mongo_crypt_opts, callback)
+
+    async def encrypt(self, database, cmd):
+        """Encrypt a MongoDB command.
+
+        :Parameters:
+          - `database`: The database for this command.
+          - `cmd`: A MongoDB command as BSON.
+
+        :Returns:
+          The encrypted command.
+        """
+        with self.mongocrypt.encryption_context(database, cmd) as ctx:
+            return await run_state_machine(ctx, self.callback)
+
+    async def decrypt(self, response):
+        """Decrypt a MongoDB command response.
+
+        :Parameters:
+          - `response`: A MongoDB command response as BSON.
+
+        :Returns:
+          The decrypted command response.
+        """
+        with self.mongocrypt.decryption_context(response) as ctx:
+            return await run_state_machine(ctx, self.callback)
+
+    async def close(self):
+        """Cleanup resources."""
+        self.mongocrypt.close()
+        await self.callback.close()

data/ext/libmongocrypt/libmongocrypt/bindings/python/pymongocrypt/asynchronous/credentials.py

@@ -0,0 +1,156 @@
+# Copyright 2024-present MongoDB, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+from collections import namedtuple
+from datetime import datetime, timedelta, timezone
+
+try:
+    from pymongo_auth_aws.auth import aws_temp_credentials
+
+    _HAVE_AUTH_AWS = True
+except ImportError:
+    _HAVE_AUTH_AWS = False
+
+import httpx
+
+from pymongocrypt.errors import MongoCryptError
+
+_azure_creds = namedtuple("_azure_creds", ["access_token", "expires_utc"])
+_azure_creds_cache = None
+
+
+async def _get_gcp_credentials():
+    """Get on-demand GCP credentials"""
+    metadata_host = os.getenv("GCE_METADATA_HOST") or "metadata.google.internal"
+    url = (
+        "http://%s/computeMetadata/v1/instance/service-accounts/default/token"
+        % metadata_host
+    )
+
+    headers = {"Metadata-Flavor": "Google"}
+    client = httpx.AsyncClient()
+    try:
+        response = await client.get(url, headers=headers)
+    except Exception as e:
+        msg = "unable to retrieve GCP credentials: %s" % e
+        raise MongoCryptError(msg) from e
+    finally:
+        await client.aclose()
+
+    if response.status_code != 200:
+        msg = f"Unable to retrieve GCP credentials: expected StatusCode 200, got StatusCode: {response.status_code}. Response body:\n{response.content}"
+        raise MongoCryptError(msg)
+    try:
+        data = response.json()
+    except Exception as e:
+        raise MongoCryptError(
+            f"unable to retrieve GCP credentials: error reading response body\n{response.content}"
+        ) from e
+
+    if not data.get("access_token"):
+        msg = (
+            "unable to retrieve GCP credentials: got unexpected empty accessToken from GCP Metadata Server. Response body: %s"
+            % response.content
+        )
+        raise MongoCryptError(msg)
+
+    return {"accessToken": data["access_token"]}
+
+
+async def _get_azure_credentials():
+    """Get on-demand Azure credentials"""
+    global _azure_creds_cache
+    # Credentials are considered expired when: Expiration - now < 1 mins.
+    creds = _azure_creds_cache
+    if creds:
+        if creds.expires_utc - datetime.now(tz=timezone.utc) < timedelta(seconds=60):
+            _azure_creds_cache = None
+        else:
+            return {"accessToken": creds.access_token}
+
+    url = "http://169.254.169.254/metadata/identity/oauth2/token"
+    url += "?api-version=2018-02-01"
+    url += "&resource=https://vault.azure.net"
+    headers = {"Metadata": "true", "Accept": "application/json"}
+    client = httpx.AsyncClient()
+    try:
+        response = await client.get(url, headers=headers)
+    except Exception as e:
+        msg = "Failed to acquire IMDS access token: %s" % e
+        raise MongoCryptError(msg) from e
+    finally:
+        await client.aclose()
+
+    if response.status_code != 200:
+        msg = "Failed to acquire IMDS access token."
+        raise MongoCryptError(msg)
+    try:
+        data = response.json()
+    except Exception as e:
+        raise MongoCryptError("Azure IMDS response must be in JSON format.") from e
+
+    for key in ["access_token", "expires_in"]:
+        if not data.get(key):
+            msg = "Azure IMDS response must contain %s, but was %s."
+            msg = msg % (key, response.content)
+            raise MongoCryptError(msg)
+
+    try:
+        expires_in = int(data["expires_in"])
+    except ValueError as e:
+        raise MongoCryptError(
+            'Azure IMDS response must contain "expires_in" integer, but was %s.'
+            % response.content
+        ) from e
+
+    expires_utc = datetime.now(tz=timezone.utc) + timedelta(seconds=expires_in)
+    _azure_creds_cache = _azure_creds(data["access_token"], expires_utc)
+    return {"accessToken": data["access_token"]}
+
+
+async def _ask_for_kms_credentials(kms_providers):
+    """Get on-demand kms credentials.
+
+    This is a separate function so it can be overridden in unit tests."""
+    global _azure_creds_cache
+    on_demand_aws = "aws" in kms_providers and not len(kms_providers["aws"])
+    on_demand_gcp = "gcp" in kms_providers and not len(kms_providers["gcp"])
+    on_demand_azure = "azure" in kms_providers and not len(kms_providers["azure"])
+
+    if not any([on_demand_aws, on_demand_gcp, on_demand_azure]):
+        return {}
+    creds = {}
+    if on_demand_aws:
+        if not _HAVE_AUTH_AWS:
+            raise RuntimeError(
+                "On-demand AWS credentials require pymongo-auth-aws: "
+                "install with: python -m pip install 'pymongo[aws]'"
+            )
+        aws_creds = aws_temp_credentials()
+        creds_dict = {
+            "accessKeyId": aws_creds.username,
+            "secretAccessKey": aws_creds.password,
+        }
+        if aws_creds.token:
+            creds_dict["sessionToken"] = aws_creds.token
+        creds["aws"] = creds_dict
+    if on_demand_gcp:
+        creds["gcp"] = await _get_gcp_credentials()
+    if on_demand_azure:
+        try:
+            creds["azure"] = await _get_azure_credentials()
+        except Exception:
+            _azure_creds_cache = None
+            raise
+    return creds