sfq 0.0.11__tar.gz → 0.0.13__tar.gz

sfq-0.0.13/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: sfq
+Version: 0.0.13
+Summary: Python wrapper for the Salesforce's Query API.
+Author-email: David Moruzzi <sfq.pypi@dmoruzi.com>
+Keywords: salesforce,salesforce query
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+# sfq (Salesforce Query)
+
+sfq is a lightweight Python wrapper library designed to simplify querying Salesforce, reducing repetitive code for accessing Salesforce data.
+
+For more varied workflows, consider using an alternative like [Simple Salesforce](https://simple-salesforce.readthedocs.io/en/stable/). This library was even referenced on the [Salesforce Developers Blog](https://developer.salesforce.com/blogs/2021/09/how-to-automate-data-extraction-from-salesforce-using-python).
+
+## Features
+
+- Simplified query execution for Salesforce instances.
+- Integration with Salesforce authentication via refresh tokens.
+- Option to interact with Salesforce Tooling API for more advanced queries.
+  
+## Installation
+
+You can install the `sfq` library using `pip`:
+
+```bash
+pip install sfq
+```
+
+## Usage
+
+### Library Querying
+
+```python
+from sfq import SFAuth
+
+# Initialize the SFAuth class with authentication details
+sf = SFAuth(
+    instance_url="https://example-dev-ed.trailblaze.my.salesforce.com",
+    client_id="your-client-id-here",
+    client_secret="your-client-secret-here",
+    refresh_token="your-refresh-token-here"
+)
+
+# Execute a query to fetch account records
+print(sf.query("SELECT Id FROM Account LIMIT 5"))
+
+# Execute a query to fetch Tooling API data
+print(sf.tooling_query("SELECT Id, FullName, Metadata FROM SandboxSettings LIMIT 5"))
+```
+
+### sObject Key Prefixes
+
+```python
+# Key prefix via IDs
+print(sf.get_sobject_prefixes())
+>>> {'0Pp': 'AIApplication', '6S9': 'AIApplicationConfig', '9qd': 'AIInsightAction', '9bq': 'AIInsightFeedback', '0T2': 'AIInsightReason', '9qc': 'AIInsightValue', ...}
+
+# Key prefix via names
+print(sf.get_sobject_prefixes(key_type="name"))
+>>> {'AIApplication': '0Pp', 'AIApplicationConfig': '6S9', 'AIInsightAction': '9qd', 'AIInsightFeedback': '9bq', 'AIInsightReason': '0T2', 'AIInsightValue': '9qc', ...}
+```
+
+### Composite Batch Queries
+
+```python
+multiple_queries = {
+    "Recent Users": """
+        SELECT Id, Name,CreatedDate
+        FROM User
+        ORDER BY CreatedDate DESC
+        LIMIT 10""",
+    "Recent Accounts": "SELECT Id, Name, CreatedDate FROM Account ORDER BY CreatedDate DESC LIMIT 10",
+    "Frozen Users": "SELECT Id, UserId FROM UserLogin WHERE IsFrozen = true",  # If exceeds 2000 records, will paginate
+}
+
+batched_response = sf.cquery(multiple_queries)
+
+for subrequest_identifer, subrequest_response in batched_response.items():
+    print(f'"{subrequest_identifer}" returned {subrequest_response["totalSize"]} records')
+>>> "Recent Users" returned 10 records
+>>> "Recent Accounts" returned 10 records
+>>> "Frozen Users" returned 4082 records
+```
+
+### Static Resources
+
+```python
+page = sf.read_static_resource_id('081aj000009jUMXAA2')
+print(f'Initial resource: {page}')
+>>> Initial resource: <h1>It works!</h1>
+sf.update_static_resource_name('HelloWorld', '<h1>Hello World</h1>')
+page = sf.read_static_resource_name('HelloWorld')
+print(f'Updated resource: {page}')
+>>> Updated resource: <h1>Hello World</h1>
+sf.update_static_resource_id('081aj000009jUMXAA2', '<h1>It works!</h1>')
+```
+
+## How to Obtain Salesforce Tokens
+
+To use the `sfq` library, you'll need a **client ID** and **refresh token**. The easiest way to obtain these is by using the Salesforce CLI:
+
+### Steps to Get Tokens
+
+1. **Install the Salesforce CLI**:  
+   Follow the instructions on the [Salesforce CLI installation page](https://developer.salesforce.com/tools/salesforcecli).
+   
+2. **Authenticate with Salesforce**:  
+   Login to your Salesforce org using the following command:
+   
+   ```bash
+   sf org login web --alias int --instance-url https://corpa--int.sandbox.my.salesforce.com
+   ```
+   
+3. **Display Org Details**:  
+   To get the client ID, client secret, refresh token, and instance URL, run:
+   
+   ```bash
+   sf org display --target-org int --verbose --json
+   ```
+
+   The output will look like this:
+
+   ```json
+   {
+     "status": 0,
+     "result": {
+       "id": "00Daa0000000000000",
+       "apiVersion": "63.0",
+       "accessToken": "00Daa0000000000000!evaU3fYZEWGUrqI5rMtaS8KYbHfeqA7YWzMgKToOB43Jk0kj7LtiWCbJaj4owPFQ7CqpXPAGX1RDCHblfW9t8cNOCNRFng3o",
+       "instanceUrl": "https://example-dev-ed.trailblaze.my.salesforce.com",
+       "username": "user@example.com",
+       "clientId": "PlatformCLI",
+       "connectedStatus": "Connected",
+       "sfdxAuthUrl": "force://PlatformCLI::nwAeSuiRqvRHrkbMmCKvLHasS0vRbh3Cf2RF41WZzmjtThnCwOuDvn9HObcUjKuTExJPqPegIwnLB5aH6GNWYhU@example-dev-ed.trailblaze.my.salesforce.com",
+       "alias": "int"
+     }
+   }
+   ```
+
+4. **Extract and Use the Tokens**:  
+   The `sfdxAuthUrl` is structured as:
+   
+   ```
+   force://<client_id>:<client_secret>:<refresh_token>@<instance_url>
+   ```
+
+   This means with the above output sample, you would use the following information:
+
+   ```python
+   # This is for illustrative purposes; use environment variables instead
+   client_id = "PlatformCLI"
+   client_secret = ""
+   refresh_token = "nwAeSuiRqvRHrkbMmCKvLHasS0vRbh3Cf2RF41WZzmjtThnCwOuDvn9HObcUjKuTExJPqPegIwnLB5aH6GNWYhU"
+   instance_url = "https://example-dev-ed.trailblaze.my.salesforce.com"
+
+   from sfq import SFAuth
+   sf = SFAuth(
+       instance_url=instance_url,
+       client_id=client_id,
+       client_secret=client_secret,
+       refresh_token=refresh_token,
+   )
+
+   ```
+
+## Important Considerations
+
+- **Security**: Safeguard your client_id, client_secret, and refresh_token diligently, as they provide access to your Salesforce environment. Avoid sharing or exposing them in unsecured locations.
+- **Efficient Data Retrieval**: The `query` and `cquery` function automatically handles pagination, simplifying record retrieval across large datasets. It's recommended to use the `LIMIT` clause in queries to control the volume of data returned.
+- **Advanced Tooling Queries**: Utilize the `tooling_query` function to access the Salesforce Tooling API. This option is designed for performing complex operations, enhancing your data management capabilities.
+

sfq-0.0.13/README.md

@@ -0,0 +1,163 @@
+# sfq (Salesforce Query)
+
+sfq is a lightweight Python wrapper library designed to simplify querying Salesforce, reducing repetitive code for accessing Salesforce data.
+
+For more varied workflows, consider using an alternative like [Simple Salesforce](https://simple-salesforce.readthedocs.io/en/stable/). This library was even referenced on the [Salesforce Developers Blog](https://developer.salesforce.com/blogs/2021/09/how-to-automate-data-extraction-from-salesforce-using-python).
+
+## Features
+
+- Simplified query execution for Salesforce instances.
+- Integration with Salesforce authentication via refresh tokens.
+- Option to interact with Salesforce Tooling API for more advanced queries.
+  
+## Installation
+
+You can install the `sfq` library using `pip`:
+
+```bash
+pip install sfq
+```
+
+## Usage
+
+### Library Querying
+
+```python
+from sfq import SFAuth
+
+# Initialize the SFAuth class with authentication details
+sf = SFAuth(
+    instance_url="https://example-dev-ed.trailblaze.my.salesforce.com",
+    client_id="your-client-id-here",
+    client_secret="your-client-secret-here",
+    refresh_token="your-refresh-token-here"
+)
+
+# Execute a query to fetch account records
+print(sf.query("SELECT Id FROM Account LIMIT 5"))
+
+# Execute a query to fetch Tooling API data
+print(sf.tooling_query("SELECT Id, FullName, Metadata FROM SandboxSettings LIMIT 5"))
+```
+
+### sObject Key Prefixes
+
+```python
+# Key prefix via IDs
+print(sf.get_sobject_prefixes())
+>>> {'0Pp': 'AIApplication', '6S9': 'AIApplicationConfig', '9qd': 'AIInsightAction', '9bq': 'AIInsightFeedback', '0T2': 'AIInsightReason', '9qc': 'AIInsightValue', ...}
+
+# Key prefix via names
+print(sf.get_sobject_prefixes(key_type="name"))
+>>> {'AIApplication': '0Pp', 'AIApplicationConfig': '6S9', 'AIInsightAction': '9qd', 'AIInsightFeedback': '9bq', 'AIInsightReason': '0T2', 'AIInsightValue': '9qc', ...}
+```
+
+### Composite Batch Queries
+
+```python
+multiple_queries = {
+    "Recent Users": """
+        SELECT Id, Name,CreatedDate
+        FROM User
+        ORDER BY CreatedDate DESC
+        LIMIT 10""",
+    "Recent Accounts": "SELECT Id, Name, CreatedDate FROM Account ORDER BY CreatedDate DESC LIMIT 10",
+    "Frozen Users": "SELECT Id, UserId FROM UserLogin WHERE IsFrozen = true",  # If exceeds 2000 records, will paginate
+}
+
+batched_response = sf.cquery(multiple_queries)
+
+for subrequest_identifer, subrequest_response in batched_response.items():
+    print(f'"{subrequest_identifer}" returned {subrequest_response["totalSize"]} records')
+>>> "Recent Users" returned 10 records
+>>> "Recent Accounts" returned 10 records
+>>> "Frozen Users" returned 4082 records
+```
+
+### Static Resources
+
+```python
+page = sf.read_static_resource_id('081aj000009jUMXAA2')
+print(f'Initial resource: {page}')
+>>> Initial resource: <h1>It works!</h1>
+sf.update_static_resource_name('HelloWorld', '<h1>Hello World</h1>')
+page = sf.read_static_resource_name('HelloWorld')
+print(f'Updated resource: {page}')
+>>> Updated resource: <h1>Hello World</h1>
+sf.update_static_resource_id('081aj000009jUMXAA2', '<h1>It works!</h1>')
+```
+
+## How to Obtain Salesforce Tokens
+
+To use the `sfq` library, you'll need a **client ID** and **refresh token**. The easiest way to obtain these is by using the Salesforce CLI:
+
+### Steps to Get Tokens
+
+1. **Install the Salesforce CLI**:  
+   Follow the instructions on the [Salesforce CLI installation page](https://developer.salesforce.com/tools/salesforcecli).
+   
+2. **Authenticate with Salesforce**:  
+   Login to your Salesforce org using the following command:
+   
+   ```bash
+   sf org login web --alias int --instance-url https://corpa--int.sandbox.my.salesforce.com
+   ```
+   
+3. **Display Org Details**:  
+   To get the client ID, client secret, refresh token, and instance URL, run:
+   
+   ```bash
+   sf org display --target-org int --verbose --json
+   ```
+
+   The output will look like this:
+
+   ```json
+   {
+     "status": 0,
+     "result": {
+       "id": "00Daa0000000000000",
+       "apiVersion": "63.0",
+       "accessToken": "00Daa0000000000000!evaU3fYZEWGUrqI5rMtaS8KYbHfeqA7YWzMgKToOB43Jk0kj7LtiWCbJaj4owPFQ7CqpXPAGX1RDCHblfW9t8cNOCNRFng3o",
+       "instanceUrl": "https://example-dev-ed.trailblaze.my.salesforce.com",
+       "username": "user@example.com",
+       "clientId": "PlatformCLI",
+       "connectedStatus": "Connected",
+       "sfdxAuthUrl": "force://PlatformCLI::nwAeSuiRqvRHrkbMmCKvLHasS0vRbh3Cf2RF41WZzmjtThnCwOuDvn9HObcUjKuTExJPqPegIwnLB5aH6GNWYhU@example-dev-ed.trailblaze.my.salesforce.com",
+       "alias": "int"
+     }
+   }
+   ```
+
+4. **Extract and Use the Tokens**:  
+   The `sfdxAuthUrl` is structured as:
+   
+   ```
+   force://<client_id>:<client_secret>:<refresh_token>@<instance_url>
+   ```
+
+   This means with the above output sample, you would use the following information:
+
+   ```python
+   # This is for illustrative purposes; use environment variables instead
+   client_id = "PlatformCLI"
+   client_secret = ""
+   refresh_token = "nwAeSuiRqvRHrkbMmCKvLHasS0vRbh3Cf2RF41WZzmjtThnCwOuDvn9HObcUjKuTExJPqPegIwnLB5aH6GNWYhU"
+   instance_url = "https://example-dev-ed.trailblaze.my.salesforce.com"
+
+   from sfq import SFAuth
+   sf = SFAuth(
+       instance_url=instance_url,
+       client_id=client_id,
+       client_secret=client_secret,
+       refresh_token=refresh_token,
+   )
+
+   ```
+
+## Important Considerations
+
+- **Security**: Safeguard your client_id, client_secret, and refresh_token diligently, as they provide access to your Salesforce environment. Avoid sharing or exposing them in unsecured locations.
+- **Efficient Data Retrieval**: The `query` and `cquery` function automatically handles pagination, simplifying record retrieval across large datasets. It's recommended to use the `LIMIT` clause in queries to control the volume of data returned.
+- **Advanced Tooling Queries**: Utilize the `tooling_query` function to access the Salesforce Tooling API. This option is designed for performing complex operations, enhancing your data management capabilities.
+

{sfq-0.0.11 → sfq-0.0.13}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "sfq"
-version = "0.0.11"
+version = "0.0.13"
 description = "Python wrapper for the Salesforce's Query API."
 readme = "README.md"
 authors = [{ name = "David Moruzzi", email = "sfq.pypi@dmoruzi.com" }]

{sfq-0.0.11 → sfq-0.0.13}/src/sfq/__init__.py

@@ -5,16 +5,20 @@ import logging
 import os
 import time
 import warnings
+from collections import OrderedDict
+from concurrent.futures import ThreadPoolExecutor, as_completed
 from queue import Empty, Queue
-from typing import Any, Dict, Optional
+from typing import Any, Dict, Literal, Optional
 from urllib.parse import quote, urlparse
 
 TRACE = 5
 logging.addLevelName(TRACE, "TRACE")
 
+
 class ExperimentalWarning(Warning):
     pass
 
+
 def trace(self: logging.Logger, message: str, *args: Any, **kwargs: Any) -> None:
     """Custom TRACE level logging function with redaction."""
 
@@ -77,13 +81,14 @@ class SFAuth:
         self,
         instance_url: str,
         client_id: str,
-        refresh_token: str,
+        refresh_token: str, # client_secret & refresh_token will swap positions 2025-AUG-1
+        client_secret: str = "_deprecation_warning",  # mandatory after 2025-AUG-1
         api_version: str = "v63.0",
         token_endpoint: str = "/services/oauth2/token",
         access_token: Optional[str] = None,
         token_expiration_time: Optional[float] = None,
         token_lifetime: int = 15 * 60,
-        user_agent: str = "sfq/0.0.11",
+        user_agent: str = "sfq/0.0.13",
         proxy: str = "auto",
     ) -> None:
         """
@@ -92,16 +97,18 @@ class SFAuth:
         :param instance_url: The Salesforce instance URL.
         :param client_id: The client ID for OAuth.
         :param refresh_token: The refresh token for OAuth.
+        :param client_secret: The client secret for OAuth (default is "_deprecation_warning").
         :param api_version: The Salesforce API version (default is "v63.0").
         :param token_endpoint: The token endpoint (default is "/services/oauth2/token").
         :param access_token: The access token for the current session (default is None).
         :param token_expiration_time: The expiration time of the access token (default is None).
         :param token_lifetime: The lifetime of the access token in seconds (default is 15 minutes).
-        :param user_agent: Custom User-Agent string (default is "sfq/0.0.11").
+        :param user_agent: Custom User-Agent string (default is "sfq/0.0.13").
         :param proxy: The proxy configuration, "auto" to use environment (default is "auto").
         """
-        self.instance_url = instance_url
+        self.instance_url = self._format_instance_url(instance_url)
         self.client_id = client_id
+        self.client_secret = client_secret
         self.refresh_token = refresh_token
         self.api_version = api_version
         self.token_endpoint = token_endpoint
@@ -112,6 +119,26 @@ class SFAuth:
         self._auto_configure_proxy(proxy)
         self._high_api_usage_threshold = 80
 
+        if self.client_secret == "_deprecation_warning":
+            warnings.warn(
+                "The 'client_secret' parameter will be mandatory and positional arguments will change after 1 August 2025. "
+                "Please ensure explicit argument assignment and 'client_secret' inclusion when initializing the SFAuth object.",
+                DeprecationWarning,
+                stacklevel=2,
+            )
+
+            logger.debug(
+                "Will be SFAuth(instance_url, client_id, client_secret, refresh_token) starting 1 August 2025... but please just use named arguments.."
+            )
+
+    def _format_instance_url(self, instance_url) -> str:
+        # check if it begins with https://
+        if instance_url.startswith("https://"):
+            return instance_url
+        if instance_url.startswith("http://"):
+            return instance_url.replace("http://", "https://")
+        return f"https://{instance_url}"
+
     def _auto_configure_proxy(self, proxy: str) -> None:
         """
         Automatically configure the proxy based on the environment or provided value.
@@ -124,16 +151,36 @@ class SFAuth:
             self.proxy = proxy
             logger.debug("Using configured proxy: %s", self.proxy)
 
-    def _prepare_payload(self) -> Dict[str, str]:
+    def _prepare_payload(self) -> Dict[str, Optional[str]]:
         """
         Prepare the payload for the token request.
+
+        This method constructs a dictionary containing the necessary parameters
+        for a token request using the refresh token grant type. It includes
+        the client ID, client secret, and refresh token if they are available.
+
+        Returns:
+            Dict[str, Optional[str]]: A dictionary containing the payload for the token request.
         """
-        return {
+        payload = {
             "grant_type": "refresh_token",
             "client_id": self.client_id,
+            "client_secret": self.client_secret,
             "refresh_token": self.refresh_token,
         }
 
+        if self.client_secret == "_deprecation_warning":
+            logger.warning(
+                "The SFQ library is making a breaking change (2025-AUG-1) to require the 'client_secret' parameter to be assigned when initializing the SFAuth object. "
+                "In addition, positional arguments will change. Please ensure explicit argument assignment and 'client_secret' inclusion when initializing the SFAuth object to avoid impact."
+            )
+            payload.pop("client_secret")
+
+        if not self.client_secret:
+            payload.pop("client_secret")
+
+        return payload
+
     def _create_connection(self, netloc: str) -> http.client.HTTPConnection:
         """
         Create a connection using HTTP or HTTPS, with optional proxy support.
@@ -210,27 +257,24 @@ class SFAuth:
         headers_list = [(k, v) for k, v in headers if not v.startswith("BrowserId=")]
         logger.trace("Response headers: %s", headers_list)
         for key, value in headers_list:
-            if key.startswith("Sforce-"):
-                if key == "Sforce-Limit-Info":
-                    current_api_calls = int(value.split("=")[1].split("/")[0])
-                    maximum_api_calls = int(value.split("=")[1].split("/")[1])
-                    usage_percentage = round(
-                        current_api_calls / maximum_api_calls * 100, 2
+            if key == "Sforce-Limit-Info":
+                current_api_calls = int(value.split("=")[1].split("/")[0])
+                maximum_api_calls = int(value.split("=")[1].split("/")[1])
+                usage_percentage = round(current_api_calls / maximum_api_calls * 100, 2)
+                if usage_percentage > self._high_api_usage_threshold:
+                    logger.warning(
+                        "High API usage: %s/%s (%s%%)",
+                        current_api_calls,
+                        maximum_api_calls,
+                        usage_percentage,
+                    )
+                else:
+                    logger.debug(
+                        "API usage: %s/%s (%s%%)",
+                        current_api_calls,
+                        maximum_api_calls,
+                        usage_percentage,
                     )
-                    if usage_percentage > self._high_api_usage_threshold:
-                        logger.warning(
-                            "High API usage: %s/%s (%s%%)",
-                            current_api_calls,
-                            maximum_api_calls,
-                            usage_percentage,
-                        )
-                    else:
-                        logger.debug(
-                            "API usage: %s/%s (%s%%)",
-                            current_api_calls,
-                            maximum_api_calls,
-                            usage_percentage,
-                        )
 
     def _refresh_token_if_needed(self) -> Optional[str]:
         """
@@ -294,6 +338,7 @@ class SFAuth:
         _safe_resource_name = quote(resource_name, safe="")
         query = f"SELECT Id FROM StaticResource WHERE Name = '{_safe_resource_name}'"
         if namespace:
+            namespace = quote(namespace, safe="")
             query += f" AND NamespacePrefix = '{namespace}'"
         query += " LIMIT 1"
         _static_resource_id_response = self.query(query)
@@ -378,6 +423,7 @@ class SFAuth:
         safe_resource_name = quote(resource_name, safe="")
         query = f"SELECT Id FROM StaticResource WHERE Name = '{safe_resource_name}'"
         if namespace:
+            namespace = quote(namespace, safe="")
             query += f" AND NamespacePrefix = '{namespace}'"
         query += " LIMIT 1"
 
@@ -603,13 +649,219 @@ class SFAuth:
         """
         return self.query(query, tooling=True)
 
+    def get_sobject_prefixes(
+        self, key_type: Literal["id", "name"] = "id"
+    ) -> Optional[Dict[str, str]]:
+        """
+        Fetch all key prefixes from the Salesforce instance and map them to sObject names or vice versa.
+
+        :param key_type: The type of key to return. Either 'id' (prefix) or 'name' (sObject).
+        :return: A dictionary mapping key prefixes to sObject names or None on failure.
+        """
+        valid_key_types = {"id", "name"}
+        if key_type not in valid_key_types:
+            logger.error(
+                "Invalid key type: %s, must be one of: %s",
+                key_type,
+                ", ".join(valid_key_types),
+            )
+            return None
+
+        self._refresh_token_if_needed()
+
+        if not self.access_token:
+            logger.error("No access token available for key prefixes.")
+            return None
+
+        endpoint = f"/services/data/{self.api_version}/sobjects/"
+        headers = {
+            "Authorization": f"Bearer {self.access_token}",
+            "User-Agent": self.user_agent,
+            "Accept": "application/json",
+        }
+
+        parsed_url = urlparse(self.instance_url)
+        conn = self._create_connection(parsed_url.netloc)
+        prefixes = {}
+
+        try:
+            logger.trace("Request endpoint: %s", endpoint)
+            logger.trace("Request headers: %s", headers)
+            conn.request("GET", endpoint, headers=headers)
+            response = conn.getresponse()
+            data = response.read().decode("utf-8")
+            self._http_resp_header_logic(response)
+
+            if response.status == 200:
+                logger.debug("Key prefixes API request successful.")
+                logger.trace("Response body: %s", data)
+                for sobject in json.loads(data)["sobjects"]:
+                    key_prefix = sobject.get("keyPrefix")
+                    name = sobject.get("name")
+                    if not key_prefix or not name:
+                        continue
+
+                    if key_type == "id":
+                        prefixes[key_prefix] = name
+                    elif key_type == "name":
+                        prefixes[name] = key_prefix
+
+                logger.debug("Key prefixes: %s", prefixes)
+                return prefixes
+
+            logger.error(
+                "Key prefixes API request failed: %s %s",
+                response.status,
+                response.reason,
+            )
+            logger.debug("Response body: %s", data)
+
+        except Exception as err:
+            logger.exception("Exception during key prefixes API request: %s", err)
+
+        finally:
+            logger.trace("Closing connection...")
+            conn.close()
+
+        return None
+
+    def cquery(
+        self, query_dict: dict[str, str], max_workers: int = 10
+    ) -> Optional[Dict[str, Any]]:
+        """
+        Execute multiple SOQL queries using the Composite Batch API with threading to reduce network overhead.
+        The function returns a dictionary mapping the original keys to their corresponding batch response.
+        The function requires a dictionary of SOQL queries with keys as logical names (referenceId) and values as SOQL queries.
+        Each query (subrequest) is counted as a unique API request against Salesforce governance limits.
+
+        :param query_dict: A dictionary of SOQL queries with keys as logical names and values as SOQL queries.
+        :param max_workers: The maximum number of threads to spawn for concurrent execution (default is 10).
+        :return: Dict mapping the original keys to their corresponding batch response or None on failure.
+        """
+        if not query_dict:
+            logger.warning("No queries to execute.")
+            return None
+
+        self._refresh_token_if_needed()
+
+        if not self.access_token:
+            logger.error("No access token available for query.")
+            return None
+
+        def _execute_batch(queries_batch):
+            endpoint = f"/services/data/{self.api_version}/composite/batch"
+            headers = {
+                "Authorization": f"Bearer {self.access_token}",
+                "User-Agent": self.user_agent,
+                "Accept": "application/json",
+                "Content-Type": "application/json",
+            }
+
+            payload = {
+                "haltOnError": False,
+                "batchRequests": [
+                    {
+                        "method": "GET",
+                        "url": f"/services/data/{self.api_version}/query?q={quote(query)}",
+                    }
+                    for query in queries_batch
+                ],
+            }
+
+            parsed_url = urlparse(self.instance_url)
+            conn = self._create_connection(parsed_url.netloc)
+            batch_results = {}
+
+            try:
+                logger.trace("Request endpoint: %s", endpoint)
+                logger.trace("Request headers: %s", headers)
+                logger.trace("Request payload: %s", json.dumps(payload, indent=2))
+
+                conn.request("POST", endpoint, json.dumps(payload), headers=headers)
+                conn.sock.settimeout(60 * 10)
+                response = conn.getresponse()
+                data = response.read().decode("utf-8")
+                self._http_resp_header_logic(response)
+
+                if response.status == 200:
+                    logger.debug("Composite query successful.")
+                    logger.trace("Composite query full response: %s", data)
+                    results = json.loads(data).get("results", [])
+                    for i, result in enumerate(results):
+                        records = []
+                        if "result" in result and "records" in result["result"]:
+                            records.extend(result["result"]["records"])
+                        # Handle pagination
+                        while not result["result"].get("done", True):
+                            next_url = result["result"].get("nextRecordsUrl")
+                            if next_url:
+                                conn.request("GET", next_url, headers=headers)
+                                response = conn.getresponse()
+                                data = response.read().decode("utf-8")
+                                self._http_resp_header_logic(response)
+                                if response.status == 200:
+                                    next_results = json.loads(data)
+                                    records.extend(next_results.get("records", []))
+                                    result["result"]["done"] = next_results.get("done")
+                                else:
+                                    logger.error(
+                                        "Failed to fetch next records: %s",
+                                        response.reason,
+                                    )
+                                    break
+                            else:
+                                result["result"]["done"] = True
+                        paginated_results = result["result"]
+                        paginated_results["records"] = records
+                        if "nextRecordsUrl" in paginated_results:
+                            del paginated_results["nextRecordsUrl"]
+                        batch_results[keys[i]] = paginated_results
+                        if result.get("statusCode") != 200:
+                            logger.error("Query failed for key %s: %s", keys[i], result)
+                            logger.error(
+                                "Query failed with HTTP status %s (%s)",
+                                result.get("statusCode"),
+                                result.get("statusMessage"),
+                            )
+                            logger.trace("Query response: %s", result)
+                else:
+                    logger.error(
+                        "Composite query failed with HTTP status %s (%s)",
+                        response.status,
+                        response.reason,
+                    )
+                    batch_results[keys[i]] = data
+                    logger.trace("Composite query response: %s", data)
+            except Exception as err:
+                logger.exception("Exception during composite query: %s", err)
+            finally:
+                logger.trace("Closing connection...")
+                conn.close()
+
+            return batch_results
+
+        keys = list(query_dict.keys())
+        results_dict = OrderedDict()
+
+        with ThreadPoolExecutor(max_workers=max_workers) as executor:
+            futures = []
+            for i in range(0, len(keys), 25):
+                batch_keys = keys[i : i + 25]
+                batch_queries = [query_dict[key] for key in batch_keys]
+                futures.append(executor.submit(_execute_batch, batch_queries))
+
+            for future in as_completed(futures):
+                results_dict.update(future.result())
+
+        logger.trace("Composite query results: %s", results_dict)
+        return results_dict
+
     def _reconnect_with_backoff(self, attempt: int) -> None:
         wait_time = min(2**attempt, 60)
         logger.warning(
             f"Reconnecting after failure, backoff {wait_time}s (attempt {attempt})"
         )
         time.sleep(wait_time)
-        self._refresh_token_if_needed()
 
     def _subscribe_topic(
         self,

{sfq-0.0.11 → sfq-0.0.13}/uv.lock

@@ -3,5 +3,5 @@ requires-python = ">=3.9"
 
 [[package]]
 name = "sfq"
-version = "0.0.11"
+version = "0.0.13"
 source = { editable = "." }

sfq-0.0.11/PKG-INFO

@@ -1,133 +0,0 @@
-Metadata-Version: 2.4
-Name: sfq
-Version: 0.0.11
-Summary: Python wrapper for the Salesforce's Query API.
-Author-email: David Moruzzi <sfq.pypi@dmoruzi.com>
-Keywords: salesforce,salesforce query
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-
-# sfq (Salesforce Query)
-
-sfq is a lightweight Python wrapper library designed to simplify querying Salesforce, reducing repetitive code for accessing Salesforce data.
-
-For more varied workflows, consider using an alternative like [Simple Salesforce](https://simple-salesforce.readthedocs.io/en/stable/). This library was even referenced on the [Salesforce Developers Blog](https://developer.salesforce.com/blogs/2021/09/how-to-automate-data-extraction-from-salesforce-using-python).
-
-## Features
-
-- Simplified query execution for Salesforce instances.
-- Integration with Salesforce authentication via refresh tokens.
-- Option to interact with Salesforce Tooling API for more advanced queries.
-  
-## Installation
-
-You can install the `sfq` library using `pip`:
-
-```bash
-pip install sfq
-```
-
-## Usage
-
-### Library Querying
-
-```python
-from sfq import SFAuth
-
-# Initialize the SFAuth class with authentication details
-sf = SFAuth(
-    instance_url="https://example-dev-ed.trailblaze.my.salesforce.com",
-    client_id="PlatformCLI",
-    refresh_token="your-refresh-token-here"
-)
-
-# Execute a query to fetch account records
-print(sf.query("SELECT Id FROM Account LIMIT 5"))
-
-# Execute a query to fetch Tooling API data
-print(sf.query("SELECT Id, FullName, Metadata FROM SandboxSettings LIMIT 5", tooling=True))
-```
-
-### Bash Querying 
-
-You can easily incorporate this into ad-hoc bash scripts or commands:
-
-```bash
-python -c "from sfq import SFAuth; sf = SFAuth(instance_url='$instance_url', client_id='$client_id', refresh_token='$refresh_token'); print(sf.query('$query'))" | jq -r '.records[].Id'
-```
-
-## How to Obtain Salesforce Tokens
-
-To use the `sfq` library, you'll need a **client ID** and **refresh token**. The easiest way to obtain these is by using the Salesforce CLI:
-
-### Steps to Get Tokens
-
-1. **Install the Salesforce CLI**:  
-   Follow the instructions on the [Salesforce CLI installation page](https://developer.salesforce.com/tools/salesforcecli).
-   
-2. **Authenticate with Salesforce**:  
-   Login to your Salesforce org using the following command:
-   
-   ```bash
-   sf org login web --alias int --instance-url https://corpa--int.sandbox.my.salesforce.com
-   ```
-   
-3. **Display Org Details**:  
-   To get the client ID, refresh token, and instance URL, run:
-   
-   ```bash
-   sf org display --target-org int --verbose --json
-   ```
-
-   The output will look like this:
-
-   ```json
-   {
-     "status": 0,
-     "result": {
-       "id": "00Daa0000000000000",
-       "apiVersion": "63.0",
-       "accessToken": "your-access-token-here",
-       "instanceUrl": "https://example-dev-ed.trailblaze.my.salesforce.com",
-       "username": "user@example.com",
-       "clientId": "PlatformCLI",
-       "connectedStatus": "Connected",
-       "sfdxAuthUrl": "force://PlatformCLI::your-refresh-token-here::https://example-dev-ed.trailblaze.my.salesforce.com",
-       "alias": "int"
-     }
-   }
-   ```
-
-4. **Extract and Use the Tokens**:  
-   The `sfdxAuthUrl` is structured as:
-   
-   ```
-   force://<client_id>::<refresh_token>::<instance_url>
-   ```
-
-   You can extract and use the tokens in a bash script like this:
-
-   ```bash
-   query="SELECT Id FROM User WHERE IsActive = true AND Profile.Name = 'System Administrator'"
-
-   sfdxAuthUrl=$(sf org display --target-org int --verbose --json | jq -r '.result.sfdxAuthUrl' | sed 's/force:\/\///')
-   clientId=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '1p')
-   refreshToken=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '2p')
-   instanceUrl=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '3p')
-
-   pip install sfq && python -c "from sfq import SFAuth; sf = SFAuth(instance_url='$instanceUrl', client_id='$clientId', refresh_token='$refreshToken'); print(sf.query('$query'))" | jq -r '.records[].Id'
-   ```
-
-## Important Considerations
-
-- **Security**: Safeguard your refresh token diligently, as it provides access to your Salesforce environment. Avoid sharing or exposing it in unsecured locations.
-- **Efficient Data Retrieval**: The `query` function automatically handles pagination, simplifying record retrieval across large datasets. It's recommended to use the `LIMIT` clause in queries to control the volume of data returned.
-- **Advanced Metadata Queries**: Utilize the `tooling=True` option within the `query` function to access the Salesforce Tooling API. This option is designed for performing complex metadata operations, enhancing your data management capabilities.
-

sfq-0.0.11/README.md

@@ -1,117 +0,0 @@
-# sfq (Salesforce Query)
-
-sfq is a lightweight Python wrapper library designed to simplify querying Salesforce, reducing repetitive code for accessing Salesforce data.
-
-For more varied workflows, consider using an alternative like [Simple Salesforce](https://simple-salesforce.readthedocs.io/en/stable/). This library was even referenced on the [Salesforce Developers Blog](https://developer.salesforce.com/blogs/2021/09/how-to-automate-data-extraction-from-salesforce-using-python).
-
-## Features
-
-- Simplified query execution for Salesforce instances.
-- Integration with Salesforce authentication via refresh tokens.
-- Option to interact with Salesforce Tooling API for more advanced queries.
-  
-## Installation
-
-You can install the `sfq` library using `pip`:
-
-```bash
-pip install sfq
-```
-
-## Usage
-
-### Library Querying
-
-```python
-from sfq import SFAuth
-
-# Initialize the SFAuth class with authentication details
-sf = SFAuth(
-    instance_url="https://example-dev-ed.trailblaze.my.salesforce.com",
-    client_id="PlatformCLI",
-    refresh_token="your-refresh-token-here"
-)
-
-# Execute a query to fetch account records
-print(sf.query("SELECT Id FROM Account LIMIT 5"))
-
-# Execute a query to fetch Tooling API data
-print(sf.query("SELECT Id, FullName, Metadata FROM SandboxSettings LIMIT 5", tooling=True))
-```
-
-### Bash Querying 
-
-You can easily incorporate this into ad-hoc bash scripts or commands:
-
-```bash
-python -c "from sfq import SFAuth; sf = SFAuth(instance_url='$instance_url', client_id='$client_id', refresh_token='$refresh_token'); print(sf.query('$query'))" | jq -r '.records[].Id'
-```
-
-## How to Obtain Salesforce Tokens
-
-To use the `sfq` library, you'll need a **client ID** and **refresh token**. The easiest way to obtain these is by using the Salesforce CLI:
-
-### Steps to Get Tokens
-
-1. **Install the Salesforce CLI**:  
-   Follow the instructions on the [Salesforce CLI installation page](https://developer.salesforce.com/tools/salesforcecli).
-   
-2. **Authenticate with Salesforce**:  
-   Login to your Salesforce org using the following command:
-   
-   ```bash
-   sf org login web --alias int --instance-url https://corpa--int.sandbox.my.salesforce.com
-   ```
-   
-3. **Display Org Details**:  
-   To get the client ID, refresh token, and instance URL, run:
-   
-   ```bash
-   sf org display --target-org int --verbose --json
-   ```
-
-   The output will look like this:
-
-   ```json
-   {
-     "status": 0,
-     "result": {
-       "id": "00Daa0000000000000",
-       "apiVersion": "63.0",
-       "accessToken": "your-access-token-here",
-       "instanceUrl": "https://example-dev-ed.trailblaze.my.salesforce.com",
-       "username": "user@example.com",
-       "clientId": "PlatformCLI",
-       "connectedStatus": "Connected",
-       "sfdxAuthUrl": "force://PlatformCLI::your-refresh-token-here::https://example-dev-ed.trailblaze.my.salesforce.com",
-       "alias": "int"
-     }
-   }
-   ```
-
-4. **Extract and Use the Tokens**:  
-   The `sfdxAuthUrl` is structured as:
-   
-   ```
-   force://<client_id>::<refresh_token>::<instance_url>
-   ```
-
-   You can extract and use the tokens in a bash script like this:
-
-   ```bash
-   query="SELECT Id FROM User WHERE IsActive = true AND Profile.Name = 'System Administrator'"
-
-   sfdxAuthUrl=$(sf org display --target-org int --verbose --json | jq -r '.result.sfdxAuthUrl' | sed 's/force:\/\///')
-   clientId=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '1p')
-   refreshToken=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '2p')
-   instanceUrl=$(echo "$sfdxAuthUrl" | sed 's/::/\n/g' | sed -n '3p')
-
-   pip install sfq && python -c "from sfq import SFAuth; sf = SFAuth(instance_url='$instanceUrl', client_id='$clientId', refresh_token='$refreshToken'); print(sf.query('$query'))" | jq -r '.records[].Id'
-   ```
-
-## Important Considerations
-
-- **Security**: Safeguard your refresh token diligently, as it provides access to your Salesforce environment. Avoid sharing or exposing it in unsecured locations.
-- **Efficient Data Retrieval**: The `query` function automatically handles pagination, simplifying record retrieval across large datasets. It's recommended to use the `LIMIT` clause in queries to control the volume of data returned.
-- **Advanced Metadata Queries**: Utilize the `tooling=True` option within the `query` function to access the Salesforce Tooling API. This option is designed for performing complex metadata operations, enhancing your data management capabilities.
-