pam-python 0.1.0__tar.gz → 0.1.2__tar.gz

pam_python-0.1.2/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.1
+Name: pam-python
+Version: 0.1.2
+Summary: Pam Python Library
+Home-page: https://github.com/heart/pam-python
+Author: Narongrit Kanhanoi
+Author-email: narongrit@pams.ai
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: License :: Other/Proprietary License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Customer Service
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Requires-Dist: setuptools>=70.0.0
+Requires-Dist: Flask>=3.0.2
+Requires-Dist: aiohttp>=3.11.11
+Requires-Dist: pandas>=2.2.3
+Requires-Dist: Faker>=33.1.0
+Requires-Dist: dask>=2024.12.1
+Requires-Dist: dask-expr>=1.1.21
+Requires-Dist: requests>=2.32.3
+
+# pam-python
+
+**pam-python** is a framework designed to simplify the creation of data plugins for PAM Real CDP. These data plugins allow you to extend the capabilities of PAM CDP to process data in a fully customizable way. For example, you can create RFM segments, run analytics such as finding top spenders, and more.
+
+## Features
+
+- Streamlined creation of data plugins for PAM Real CDP.
+- Scaffolding for project and service structures.
+- Built-in templates for rapid development.
+- Command-line utility for managing projects and services.
+- Easy-to-run testing for individual services.
+
+## Getting Started
+
+### Prerequisites
+
+- Python 3.6 or later
+- pip for package management
+
+## Installation
+
+### 1. Create a new folder for your project
+
+```bash
+mkdir my_data_plugin
+cd my_data_plugin
+```
+
+### 2. Set up a Python virtual environment
+
+```bash
+python3 -m venv venv
+source venv/bin/activate
+```
+
+### 3. Install pam-python
+
+```bash
+pip install pam-python
+```
+
+> Once installed, the `pam` command-line utility will be available.
+
+## Usage
+
+### 1. Initialize a New Project
+
+To initialize a new data plugin project, run:
+
+```bash
+pam init
+```
+
+> This command will create the necessary files and structure for your project.
+
+### 2. Add a New Service
+
+Within your project, you can create multiple services. To add a new service, use:
+
+```bash
+pam new service <service_name>
+```
+
+> This will create a folder named `<service_name>` containing the following:
+> Basic template files to help you start developing your service.
+
+### 3. Test a Service
+
+To test a specific service, use:
+
+```bash
+pam test <service_name>
+```
+
+> This command will automatically run all test files (\*.test.py) associated with the specified service.
+
+---
+
+## Example Workflow
+
+### 1. Initialize the project
+
+```bash
+pam init
+```
+
+### 2. Create a new service
+
+```bash
+pam new service RFMAnalysis
+```
+
+### 3.Customize the service by editing the generated templates in RFMAnalysis
+
+### 4. Test the service
+
+```bash
+pam test RFMAnalysis
+```

pam_python-0.1.2/README.md

@@ -0,0 +1,99 @@
+# pam-python
+
+**pam-python** is a framework designed to simplify the creation of data plugins for PAM Real CDP. These data plugins allow you to extend the capabilities of PAM CDP to process data in a fully customizable way. For example, you can create RFM segments, run analytics such as finding top spenders, and more.
+
+## Features
+
+- Streamlined creation of data plugins for PAM Real CDP.
+- Scaffolding for project and service structures.
+- Built-in templates for rapid development.
+- Command-line utility for managing projects and services.
+- Easy-to-run testing for individual services.
+
+## Getting Started
+
+### Prerequisites
+
+- Python 3.6 or later
+- pip for package management
+
+## Installation
+
+### 1. Create a new folder for your project
+
+```bash
+mkdir my_data_plugin
+cd my_data_plugin
+```
+
+### 2. Set up a Python virtual environment
+
+```bash
+python3 -m venv venv
+source venv/bin/activate
+```
+
+### 3. Install pam-python
+
+```bash
+pip install pam-python
+```
+
+> Once installed, the `pam` command-line utility will be available.
+
+## Usage
+
+### 1. Initialize a New Project
+
+To initialize a new data plugin project, run:
+
+```bash
+pam init
+```
+
+> This command will create the necessary files and structure for your project.
+
+### 2. Add a New Service
+
+Within your project, you can create multiple services. To add a new service, use:
+
+```bash
+pam new service <service_name>
+```
+
+> This will create a folder named `<service_name>` containing the following:
+> Basic template files to help you start developing your service.
+
+### 3. Test a Service
+
+To test a specific service, use:
+
+```bash
+pam test <service_name>
+```
+
+> This command will automatically run all test files (\*.test.py) associated with the specified service.
+
+---
+
+## Example Workflow
+
+### 1. Initialize the project
+
+```bash
+pam init
+```
+
+### 2. Create a new service
+
+```bash
+pam new service RFMAnalysis
+```
+
+### 3.Customize the service by editing the generated templates in RFMAnalysis
+
+### 4. Test the service
+
+```bash
+pam test RFMAnalysis
+```

pam_python-0.1.2/pam/api.py

@@ -0,0 +1,53 @@
+from pathlib import Path
+import requests
+from pam.utils import log
+
+
+class API:
+    def __init__(self):
+        self.session = requests.Session()  # Use a session for connection reuse
+
+    def http_post(self, url: str, data: dict) -> requests.Response | None:
+        """
+        Sends an HTTP POST request to the specified URL with the given data as JSON.
+
+        :param url: The URL to send the POST request to.
+        :param data: A dictionary to be used as the JSON body of the POST request.
+        :return: The response from the server, or None if an error occurred.
+        """
+        headers = {'Content-Type': 'application/json'}
+        try:
+            response = self.session.post(
+                url, json=data, timeout=30, headers=headers
+            )
+            response.raise_for_status()
+            return response
+        except requests.RequestException as e:
+            log(f"HTTP POST request failed. URL: {url}, Error: {e}")
+            return None
+
+    def http_upload(self, url: str, file_path: str) -> requests.Response | None:
+        """
+        Uploads a file to the specified URL.
+
+        :param url: The URL to upload the file to.
+        :param file_path: The path to the file to be uploaded.
+        :return: The response from the server, or None if an error occurred.
+        """
+        if not Path(file_path).is_file():
+            log(f"File does not exist: {file_path}")
+            return None
+
+        try:
+            with open(file_path, 'rb') as file:
+                files = {'file': file}
+                response = self.session.post(url, files=files, timeout=300)
+                response.raise_for_status()
+                return response
+        except requests.RequestException as e:
+            log(f"File upload failed. URL: {url}, File: {file_path}, Error: {e}")
+            return None
+
+    def close(self):
+        """Close the session."""
+        self.session.close()

{pam_python-0.1.0 → pam_python-0.1.2}/pam/cli.py

@@ -1,9 +1,10 @@
 import os
 import sys
-import pkg_resources 
+import re
 import shutil
 import glob
 import subprocess
+import pkg_resources 
 
 def main():
     args = sys.argv[1:]
@@ -16,8 +17,8 @@ def main():
     if cmd == "init":
         init_project()
     elif cmd == "new":
-        type = args[1]
-        if type == "service":
+        create_type = args[1]
+        if create_type == "service":
             name = args[2]
             create_service(name)
     elif cmd == "test":
@@ -45,7 +46,7 @@ def test_module(module_name):
     # Run each test file
     for test_file in test_files:
         print(f"\nRunning tests in {test_file}...")
-        result = subprocess.run([sys.executable, test_file], capture_output=True, text=True)
+        result = subprocess.run([sys.executable, test_file], capture_output=True, text=True, check=True)
 
         print("\nTest Output:")
         print(result.stdout)
@@ -54,6 +55,24 @@ def test_module(module_name):
             print(result.stderr)
         print("-" * 40)  # Separator for clarity
 
+
+
+def to_pascal_case(input_string: str) -> str:
+    """
+    Convert a string to PascalCase.
+
+    :param input_string: The string to convert.
+    :return: The string in PascalCase.
+    """
+    # Split the string into words using non-alphanumeric characters as delimiters
+    words = re.split(r'\W+', input_string)
+    
+    # Capitalize each word and join them
+    pascal_case = ''.join(word.capitalize() for word in words if word)
+
+    return pascal_case
+
+
 def cpy(src, dest):
     template_dir = pkg_resources.resource_filename("pam", "templates")
     src_file = os.path.join(template_dir, src)
@@ -61,9 +80,10 @@ def cpy(src, dest):
 
 def replace_template_content(service_name, file_name):
     file_path = os.path.join(service_name, file_name)
-    with open(file_path, 'r+') as file:
+    with open(file_path, 'r+', encoding='utf-8') as file:
         filedata = file.read()
-        updated_data = filedata.replace('#CLASS_NAME#', service_name)
+        updated_data = filedata.replace('#CLASS_NAME#', to_pascal_case(service_name))
+        updated_data = updated_data.replace('#MODULE_NAME#', service_name)
         file.seek(0)  # Move the file pointer to the beginning of the file
         file.write(updated_data)
         file.truncate()  # Remove any leftover content after the replacement
@@ -76,15 +96,14 @@ def create_service(name):
         else:
             print("Cancelled.")
             return
-    
+
     os.mkdir(name)
     open(os.path.join(name, "__init__.py"), 'a', encoding='utf-8').close()
-    cpy("service/ServiceClass.tmpl", os.path.join(name, name+"_service.py") )
+    cpy("service/service_class.tmpl", os.path.join(name, name+"_service.py") )
     cpy("service/service.yaml", os.path.join(name, "service.yaml") )
     cpy("service/functions.tmpl", os.path.join(name, "functions.py") )
     cpy("service/service.test.tmpl", os.path.join(name, name+".test.py") )
 
-    
     replace_template_content(name, name+"_service.py")
     replace_template_content(name, "service.yaml")
     replace_template_content(name, name+".test.py")
@@ -92,16 +111,24 @@ def create_service(name):
     print(f"Service {name} created.")
     print(f"Run `pam test {name}` to run tests for the service.")
 
+
 def init_project():
     cpy("init/main.tmpl", "main.py")
     cpy("docker/Dockerfile", "Dockerfile")
     cpy("buildcmd/pamb", "pamb")
     cpy("buildcmd/pamb-base.sh", "pamb-base.sh")
+    cpy("init/pylintrc.tmpl", ".pylintrc")
+    cpy("init/gitignore.tmpl", ".gitignore")
+    cpy("init/dockerignore.tmpl", ".dockerignore")
+    
     if not os.path.exists("requirements.txt"):
         open("requirements.txt", 'a', encoding='utf-8').close()
     
     with open("requirements.txt", "w", encoding='utf-8') as f:
         subprocess.run(["pip", "freeze"], stdout=f, check=True)
 
+    if not os.path.exists("__init__.py"):
+        open("__init__.py", 'a', encoding='utf-8').close()
+
     print("--- Welcome to PAM ---\n")
-    print("To create a new servive run\n`pam new service <service_name>`\n\n")
+    print("To create a new servive run\n`pam new service <service_name>`\n\n")

pam_python-0.1.2/pam/models/request_command.py

@@ -0,0 +1,160 @@
+"""
+This module defines the RequestCommand class, 
+which wraps an HTTP request to simplify access to its parameters and files.
+
+Classes:
+    RequestCommand: A dataclass that represents an HTTP request command 
+    with various attributes and methods to process the request.
+
+Methods:
+    is_start_command(self): Determine if this command is a start command.
+    is_dataset_command(self): Determine if this command is a dataset command.
+    __safe_str_to_bool(value): Safely convert a string to a boolean.
+    __extract_data_from_request(request: Request): Extract data from a Flask request object 
+        based on its content type.
+    __allowed_file(filename): Check if the file has an allowed extension.
+    __extract_input_file(zip_file, service_name, token): Extract a zip file 
+        into a temporary directory and return the file paths.
+    parse(request, service_name): Create a RequestCommand object from an HTTP request.
+
+"""
+import os
+from pathlib import Path
+from dataclasses import dataclass
+import zipfile
+from flask import Request
+from pam.utils import log
+from pam.temp_file_utils import TempfileUtils
+
+@dataclass
+class RequestCommand:
+    """
+    Wraps an HTTP request to simplify access to its parameters and files.
+    """
+    runtime_parameters: dict
+    token: str
+    cmd: str
+    data_api: str
+    response_api: str
+    is_end: bool
+    next: str
+    input_files: list[str]
+    service_name: str
+
+    def is_start_command(self):
+        """Determine if this command is a start command."""
+        return self.cmd == "start"
+
+    def is_dataset_command(self):
+        """Determine if this command is a dataset command."""
+        return self.cmd == "dataset"
+
+    @staticmethod
+    def __safe_str_to_bool(value):
+        """Safely convert a string to a boolean."""
+        if isinstance(value, bool):
+            return value
+        if isinstance(value, str):
+            value_lower = value.lower()
+            if value_lower in {'true', '1', 'yes'}:
+                return True
+            if value_lower in {'false', '0', 'no'}:
+                return False
+        return False
+
+    @staticmethod
+    def __extract_data_from_request(request: Request):
+        """
+        Extract data from a Flask request object based on its content type.
+        """
+        if request.content_type == 'application/json':
+            return request.get_json() or {}
+        elif request.content_type.startswith('multipart/form-data'):
+            return request.form.to_dict()
+        return {}
+
+    @staticmethod
+    def __allowed_file(filename):
+        """Check if the file has an allowed extension."""
+        return filename.lower().endswith('.zip')
+
+    @staticmethod
+    def __extract_input_file(zip_file, service_name, token):
+        """
+        Extract a zip file into a temporary directory and return the file paths.
+        """
+        if not zip_file.filename:
+            log("No file selected for upload.")
+            return [], "No selected file"
+
+        if not RequestCommand.__allowed_file(zip_file.filename):
+            log("Invalid file type uploaded.")
+            return [], "Invalid file type"
+
+        temp_dir = TempfileUtils.get_temp_file_name(service_name, token, "dataset_")
+        zip_file_path = f"{temp_dir}.zip"
+        zip_file.save(zip_file_path)
+
+        extract_dir = Path(temp_dir)
+        extract_dir.mkdir(parents=True, exist_ok=True)
+
+        try:
+            with zipfile.ZipFile(zip_file_path, 'r') as zip_ref:
+                zip_ref.extractall(extract_dir)
+
+            # Remove unnecessary files or directories
+            for unwanted in ['__MACOSX', '.DS_Store']:
+                unwanted_path = extract_dir / unwanted
+                if unwanted_path.exists():
+                    if unwanted_path.is_dir():
+                        os.rmdir(unwanted_path)
+                    else:
+                        unwanted_path.unlink()
+
+            input_files = sorted(str(file) for file in extract_dir.iterdir() if file.is_file())
+            log(f"Extracted files: {input_files}")
+            return input_files, ""
+
+        except zipfile.BadZipFile:
+            log("Failed to extract zip file.")
+            return [], "Invalid or corrupted zip file"
+
+    @staticmethod
+    def parse(request, service_name):
+        """
+        Create a RequestCommand object from an HTTP request.
+        """
+        params = RequestCommand.__extract_data_from_request(request)
+
+        token = params.pop("token", "")
+        cmd = params.pop("cmd", "")
+        data_api = params.pop("data", "")
+        response_api = params.pop("response", "")
+        is_end = RequestCommand.__safe_str_to_bool(params.pop("is_end", "true"))
+        next_page = params.pop("next", "")
+
+        input_files = []
+        error_message = ""
+
+        if not token:
+            error_message = "The `token` parameter is required."
+        elif cmd == "dataset":
+            zip_file = request.files.get('file')
+            if not zip_file:
+                error_message = "A file must be uploaded for the `dataset` command."
+            else:
+                input_files, error_message = RequestCommand.__extract_input_file(
+                    zip_file, service_name, token
+                )
+
+        return RequestCommand(
+            runtime_parameters=params,
+            token=token,
+            cmd=cmd,
+            data_api=data_api,
+            response_api=response_api,
+            is_end=is_end,
+            next=next_page,
+            input_files=input_files,
+            service_name=service_name,
+        ), error_message