dc-python-sdk 1.5.51__tar.gz → 1.6.1__tar.gz

{dc_python_sdk-1.5.51/src/dc_python_sdk.egg-info → dc_python_sdk-1.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dc-python-sdk
-Version: 1.5.51
+Version: 1.6.1
 Summary: Data Connector Python SDK
 Home-page: https://github.com/data-connector/dc-python-sdk
 Author: DataConnector
@@ -14,7 +14,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: fastapi
 Requires-Dist: uvicorn
-Requires-Dist: awslambdaric
+Requires-Dist: awslambdaric==4.0.4
 Requires-Dist: requests
 Requires-Dist: boto3>=1.40.0
 Requires-Dist: openai
@@ -26,6 +26,9 @@ Requires-Dist: numpy>=1.21.6; extra == "test"
 Requires-Dist: pandas>=1.3.5; extra == "test"
 Provides-Extra: ai
 Requires-Dist: openai; extra == "ai"
+Provides-Extra: file
+Requires-Dist: numpy>=1.21.6; extra == "file"
+Requires-Dist: pandas>=1.3.5; extra == "file"
 Dynamic: license-file
 
 # Data Connector Python SDK
@@ -91,6 +94,41 @@ def get_available_objects():
     return objects
 ```
 
+### Selective discovery in the Lambda handler
+
+Authenticate with action `3`, then request only the metadata needed by the current
+screen with action `5`:
+
+```json
+{"action": 5, "sections": ["accounts"], "credentials": {}}
+```
+
+Connectors opt in by defining `get_metadata(self, sections=None)`. Fabric supports
+`accounts`, `schemas`, and `folders`; callers can request more than one section.
+Requested sections retain their usual keys in the metadata dictionary, and static
+capability fields may also be returned. Unrequested dynamic keys are omitted,
+not returned as empty lists. Callers should merge partial metadata into existing
+metadata for the same account, and clear cached metadata when switching accounts.
+
+Action `6` accepts the same optional parameter for `get_objects(self, sections=None)`;
+Fabric supports `tables` and `files`. Use `include_metadata: false` to avoid an
+additional full metadata request when fetching only objects. Object results remain
+a list in the existing response envelope.
+
+Omitting `sections` (or using null) retains full discovery. An empty list requests
+no dynamic sections on an opted-in connector. The SDK validates a list of non-empty
+strings and passes it only to methods with an explicit `sections` keyword; legacy
+connectors still perform full discovery with no arguments. Each opted-in connector
+validates its supported section names. Internal connector TypeErrors are propagated,
+never retried as a legacy call. The SDK does not filter a legacy response or claim
+that a legacy connector avoided fetching unrequested data.
+
+The SDK returns `error_phase: "metadata"` for discovery errors and keeps successful
+authentication status. `AuthenticationError` still invalidates authentication.
+API and UI consumers must use this distinction to show a discovery retry instead
+of requiring reconnection. This contract applies to the Lambda handler; the local
+HTTP server below uses a separate method-based request format.
+
 ### Running the local HTTP server
 
 Install the SDK and start the HTTP server that wraps your connector:
@@ -139,7 +177,7 @@ Our system uses the `dc_sdk` library to gracefully throw errors. This ensures:
 
 ## Error Categories
 
-The SDK provides 21 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
+The SDK provides 25 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
 
 ### Authentication Errors
 - `AuthenticationError` - Invalid credentials, expired tokens
@@ -152,6 +190,8 @@ The SDK provides 21 different error classes organized into logical categories. E
 - `GetFieldsError` - Cannot retrieve field information
 - `BadFieldIDError` - Invalid field ID for object
 - `BadObjectIDError` - Object ID doesn't exist
+- `FieldNotFoundError` - Requested fields are missing from a file or object
+- `ObjectNotFoundError` - File or object could not be found at the source
 
 ### Data Filtering & Mapping Errors
 - `FilterDataTypeError` - Invalid data type for filtering
@@ -160,6 +200,7 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Data Retrieval Errors
 - `DataError` - Generic data retrieval failure
+- `InvalidFileFormatError` - File could not be parsed or has no usable format
 - `APIRequestError` - API returned error status
 - `APITimeoutError` - Request timeout
 - `APIPermissionError` - Insufficient API permissions
@@ -172,6 +213,15 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Implementation Errors
 - `NotImplementedError` - Required method not implemented
+- `FileUtilsDependencyError` - Connector imported `dc_sdk.file_utils` without installing pandas/numpy
+
+File connectors can share parsing helpers with:
+
+```python
+from dc_sdk.file_utils import FileConnectorUtilis
+```
+
+That import is optional. Connectors that use it must also install pandas and numpy (for example `pip install pandas numpy`). Other connectors should keep importing `from dc_sdk import errors` only, so they never load pandas.
 
 For detailed examples and usage patterns, see the [Examples](#examples) section below.
 

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/README.md

@@ -61,6 +61,41 @@ def get_available_objects():
     return objects
 ```
 
+### Selective discovery in the Lambda handler
+
+Authenticate with action `3`, then request only the metadata needed by the current
+screen with action `5`:
+
+```json
+{"action": 5, "sections": ["accounts"], "credentials": {}}
+```
+
+Connectors opt in by defining `get_metadata(self, sections=None)`. Fabric supports
+`accounts`, `schemas`, and `folders`; callers can request more than one section.
+Requested sections retain their usual keys in the metadata dictionary, and static
+capability fields may also be returned. Unrequested dynamic keys are omitted,
+not returned as empty lists. Callers should merge partial metadata into existing
+metadata for the same account, and clear cached metadata when switching accounts.
+
+Action `6` accepts the same optional parameter for `get_objects(self, sections=None)`;
+Fabric supports `tables` and `files`. Use `include_metadata: false` to avoid an
+additional full metadata request when fetching only objects. Object results remain
+a list in the existing response envelope.
+
+Omitting `sections` (or using null) retains full discovery. An empty list requests
+no dynamic sections on an opted-in connector. The SDK validates a list of non-empty
+strings and passes it only to methods with an explicit `sections` keyword; legacy
+connectors still perform full discovery with no arguments. Each opted-in connector
+validates its supported section names. Internal connector TypeErrors are propagated,
+never retried as a legacy call. The SDK does not filter a legacy response or claim
+that a legacy connector avoided fetching unrequested data.
+
+The SDK returns `error_phase: "metadata"` for discovery errors and keeps successful
+authentication status. `AuthenticationError` still invalidates authentication.
+API and UI consumers must use this distinction to show a discovery retry instead
+of requiring reconnection. This contract applies to the Lambda handler; the local
+HTTP server below uses a separate method-based request format.
+
 ### Running the local HTTP server
 
 Install the SDK and start the HTTP server that wraps your connector:
@@ -109,7 +144,7 @@ Our system uses the `dc_sdk` library to gracefully throw errors. This ensures:
 
 ## Error Categories
 
-The SDK provides 21 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
+The SDK provides 25 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
 
 ### Authentication Errors
 - `AuthenticationError` - Invalid credentials, expired tokens
@@ -122,6 +157,8 @@ The SDK provides 21 different error classes organized into logical categories. E
 - `GetFieldsError` - Cannot retrieve field information
 - `BadFieldIDError` - Invalid field ID for object
 - `BadObjectIDError` - Object ID doesn't exist
+- `FieldNotFoundError` - Requested fields are missing from a file or object
+- `ObjectNotFoundError` - File or object could not be found at the source
 
 ### Data Filtering & Mapping Errors
 - `FilterDataTypeError` - Invalid data type for filtering
@@ -130,6 +167,7 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Data Retrieval Errors
 - `DataError` - Generic data retrieval failure
+- `InvalidFileFormatError` - File could not be parsed or has no usable format
 - `APIRequestError` - API returned error status
 - `APITimeoutError` - Request timeout
 - `APIPermissionError` - Insufficient API permissions
@@ -142,6 +180,15 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Implementation Errors
 - `NotImplementedError` - Required method not implemented
+- `FileUtilsDependencyError` - Connector imported `dc_sdk.file_utils` without installing pandas/numpy
+
+File connectors can share parsing helpers with:
+
+```python
+from dc_sdk.file_utils import FileConnectorUtilis
+```
+
+That import is optional. Connectors that use it must also install pandas and numpy (for example `pip install pandas numpy`). Other connectors should keep importing `from dc_sdk import errors` only, so they never load pandas.
 
 For detailed examples and usage patterns, see the [Examples](#examples) section below.
 

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "dc-python-sdk"
-version = "1.5.51"
+version = "1.6.1"
 description = "Data Connector Python SDK"
 readme = "README.md"
 requires-python = ">=3.6"
@@ -19,7 +19,7 @@ classifiers = [
 dependencies = [
     "fastapi",
     "uvicorn",
-    "awslambdaric",
+    "awslambdaric==4.0.4",
     "requests",
     "boto3>=1.40.0",
     "openai",
@@ -38,6 +38,11 @@ ai = [
     "openai",
 ]
 
+file = [
+    "numpy>=1.21.6",
+    "pandas>=1.3.5",
+]
+
 [project.urls]
 "Bug Tracker" = "https://github.com/data-connector/dc-python-sdk/issues"
 
@@ -49,4 +54,4 @@ where = ["src"]
 
 [tool.pytest.ini_options]
 pythonpath = ["src", "tests"]
-testpaths = ["tests"]
+testpaths = ["tests"]

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = dc-python-sdk
-version = 1.5.51
+version = 1.6.1
 author = DataConnector
 author_email = josh@dataconnector.com
 description = A small example package
@@ -22,7 +22,7 @@ python_requires = >=3.6
 install_requires = 
 	fastapi
 	uvicorn
-	awslambdaric
+	awslambdaric==4.0.4
 	pycryptodome
 	requests
 	boto3>=1.40.0

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1/src/dc_python_sdk.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dc-python-sdk
-Version: 1.5.51
+Version: 1.6.1
 Summary: Data Connector Python SDK
 Home-page: https://github.com/data-connector/dc-python-sdk
 Author: DataConnector
@@ -14,7 +14,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: fastapi
 Requires-Dist: uvicorn
-Requires-Dist: awslambdaric
+Requires-Dist: awslambdaric==4.0.4
 Requires-Dist: requests
 Requires-Dist: boto3>=1.40.0
 Requires-Dist: openai
@@ -26,6 +26,9 @@ Requires-Dist: numpy>=1.21.6; extra == "test"
 Requires-Dist: pandas>=1.3.5; extra == "test"
 Provides-Extra: ai
 Requires-Dist: openai; extra == "ai"
+Provides-Extra: file
+Requires-Dist: numpy>=1.21.6; extra == "file"
+Requires-Dist: pandas>=1.3.5; extra == "file"
 Dynamic: license-file
 
 # Data Connector Python SDK
@@ -91,6 +94,41 @@ def get_available_objects():
     return objects
 ```
 
+### Selective discovery in the Lambda handler
+
+Authenticate with action `3`, then request only the metadata needed by the current
+screen with action `5`:
+
+```json
+{"action": 5, "sections": ["accounts"], "credentials": {}}
+```
+
+Connectors opt in by defining `get_metadata(self, sections=None)`. Fabric supports
+`accounts`, `schemas`, and `folders`; callers can request more than one section.
+Requested sections retain their usual keys in the metadata dictionary, and static
+capability fields may also be returned. Unrequested dynamic keys are omitted,
+not returned as empty lists. Callers should merge partial metadata into existing
+metadata for the same account, and clear cached metadata when switching accounts.
+
+Action `6` accepts the same optional parameter for `get_objects(self, sections=None)`;
+Fabric supports `tables` and `files`. Use `include_metadata: false` to avoid an
+additional full metadata request when fetching only objects. Object results remain
+a list in the existing response envelope.
+
+Omitting `sections` (or using null) retains full discovery. An empty list requests
+no dynamic sections on an opted-in connector. The SDK validates a list of non-empty
+strings and passes it only to methods with an explicit `sections` keyword; legacy
+connectors still perform full discovery with no arguments. Each opted-in connector
+validates its supported section names. Internal connector TypeErrors are propagated,
+never retried as a legacy call. The SDK does not filter a legacy response or claim
+that a legacy connector avoided fetching unrequested data.
+
+The SDK returns `error_phase: "metadata"` for discovery errors and keeps successful
+authentication status. `AuthenticationError` still invalidates authentication.
+API and UI consumers must use this distinction to show a discovery retry instead
+of requiring reconnection. This contract applies to the Lambda handler; the local
+HTTP server below uses a separate method-based request format.
+
 ### Running the local HTTP server
 
 Install the SDK and start the HTTP server that wraps your connector:
@@ -139,7 +177,7 @@ Our system uses the `dc_sdk` library to gracefully throw errors. This ensures:
 
 ## Error Categories
 
-The SDK provides 21 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
+The SDK provides 25 different error classes organized into logical categories. Each error is designed to handle specific failure scenarios:
 
 ### Authentication Errors
 - `AuthenticationError` - Invalid credentials, expired tokens
@@ -152,6 +190,8 @@ The SDK provides 21 different error classes organized into logical categories. E
 - `GetFieldsError` - Cannot retrieve field information
 - `BadFieldIDError` - Invalid field ID for object
 - `BadObjectIDError` - Object ID doesn't exist
+- `FieldNotFoundError` - Requested fields are missing from a file or object
+- `ObjectNotFoundError` - File or object could not be found at the source
 
 ### Data Filtering & Mapping Errors
 - `FilterDataTypeError` - Invalid data type for filtering
@@ -160,6 +200,7 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Data Retrieval Errors
 - `DataError` - Generic data retrieval failure
+- `InvalidFileFormatError` - File could not be parsed or has no usable format
 - `APIRequestError` - API returned error status
 - `APITimeoutError` - Request timeout
 - `APIPermissionError` - Insufficient API permissions
@@ -172,6 +213,15 @@ The SDK provides 21 different error classes organized into logical categories. E
 
 ### Implementation Errors
 - `NotImplementedError` - Required method not implemented
+- `FileUtilsDependencyError` - Connector imported `dc_sdk.file_utils` without installing pandas/numpy
+
+File connectors can share parsing helpers with:
+
+```python
+from dc_sdk.file_utils import FileConnectorUtilis
+```
+
+That import is optional. Connectors that use it must also install pandas and numpy (for example `pip install pandas numpy`). Other connectors should keep importing `from dc_sdk import errors` only, so they never load pandas.
 
 For detailed examples and usage patterns, see the [Examples](#examples) section below.
 

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/src/dc_python_sdk.egg-info/SOURCES.txt

@@ -13,7 +13,10 @@ src/dc_sdk/app.py
 src/dc_sdk/cli.py
 src/dc_sdk/data_stream.py
 src/dc_sdk/errors.py
+src/dc_sdk/file_utils.py
 src/dc_sdk/handler.py
+src/dc_sdk/json_safe.py
+src/dc_sdk/row_filters.py
 src/dc_sdk/types.py
 src/dc_sdk/src/__init__.py
 src/dc_sdk/src/ai.py

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/src/dc_python_sdk.egg-info/requires.txt

@@ -1,6 +1,6 @@
 fastapi
 uvicorn
-awslambdaric
+awslambdaric==4.0.4
 requests
 boto3>=1.40.0
 openai
@@ -9,6 +9,10 @@ pycryptodome
 [ai]
 openai
 
+[file]
+numpy>=1.21.6
+pandas>=1.3.5
+
 [test]
 python-dotenv>=0.20.0
 faker>=13.12.0

{dc_python_sdk-1.5.51 → dc_python_sdk-1.6.1}/src/dc_sdk/errors.py

@@ -9,7 +9,9 @@ class Error(Exception):
         self.internal = internal
 
     def __str__(self):
-        return self.error_name + self.message
+        name = self.error_name if self.error_name is not None else type(self).__name__
+        msg = self.message if self.message is not None else ""
+        return f"{name}{msg}"
 
     def to_dict(self):
         return {
@@ -35,6 +37,8 @@ class AuthenticationError(Error):
 
             :param message: An error message - WILL BE VISIBLE TO CLIENT
         """
+        if not message:
+            message = "Authentication failed."
         super().__init__(message, self.error_name, self.internal)
         self.message = message
         self.error_name = "Authentication Error - "
@@ -340,6 +344,54 @@ class NoRowsFoundError(Error):
         self.internal = True
 
 
+class InvalidFileFormatError(Error):
+    """Raised when a file object cannot be parsed or has no usable format."""
+    error_name = "Invalid File Format Error - "
+    internal = False
+
+    def __init__(self, message=None):
+        super().__init__(message, self.error_name, self.internal)
+        self.message = message
+        self.error_name = "Invalid File Format Error - "
+        self.internal = False
+
+
+class FieldNotFoundError(Error):
+    """Raised when requested fields are not present in a file or object."""
+    error_name = "Field Not Found Error - "
+    internal = False
+
+    def __init__(self, message=None):
+        super().__init__(message, self.error_name, self.internal)
+        self.message = message
+        self.error_name = "Field Not Found Error - "
+        self.internal = False
+
+
+class ObjectNotFoundError(Error):
+    """Raised when a file or object id cannot be found at the source."""
+    error_name = "Object Not Found Error - "
+    internal = False
+
+    def __init__(self, message=None):
+        super().__init__(message, self.error_name, self.internal)
+        self.message = message
+        self.error_name = "Object Not Found Error - "
+        self.internal = False
+
+
+class FileUtilsDependencyError(Error):
+    """Raised when a connector imports file utils without pandas/numpy installed."""
+    error_name = "File Utils Dependency Error - "
+    internal = False
+
+    def __init__(self, message=None):
+        super().__init__(message, self.error_name, self.internal)
+        self.message = message
+        self.error_name = "File Utils Dependency Error - "
+        self.internal = False
+
+
 def external_errors():
     error_list = []
     classes = [cls[1] for cls in inspect.getmembers(sys.modules[__name__],
@@ -373,4 +425,8 @@ ERROR_MAPPING = {
     "NotADestinationError": NotADestinationError,
     "NotImplementedError": NotImplementedError,
     "NoRowsFoundError": NoRowsFoundError,
+    "InvalidFileFormatError": InvalidFileFormatError,
+    "FieldNotFoundError": FieldNotFoundError,
+    "ObjectNotFoundError": ObjectNotFoundError,
+    "FileUtilsDependencyError": FileUtilsDependencyError,
 }