brynq-sdk-meta4 1.0.0__tar.gz

brynq_sdk_meta4-1.0.0/PKG-INFO

@@ -0,0 +1,10 @@
+Metadata-Version: 1.0
+Name: brynq_sdk_meta4
+Version: 1.0.0
+Summary: Meta4 wrapper from BrynQ
+Home-page: UNKNOWN
+Author: BrynQ
+Author-email: support@brynq.com
+License: BrynQ License
+Description: UNKNOWN
+Platform: UNKNOWN

brynq_sdk_meta4-1.0.0/README.md

@@ -0,0 +1,155 @@
+## BrynQ Meta4 SDK
+
+A lightweight toolkit to validate data and generate Meta4-ready CSV files. It validates Pandas DataFrames using Pydantic schemas and writes `.csv` files into the `outputs/` directory. Then upload the files via SFTP.
+
+This SDK consists of two parts:
+- CSV generation: validate and accumulate data in memory, then export to CSV files under `outputs/`.
+- SFTP upload: send the generated CSV files to the target server directories.
+
+### How it works
+
+1. Instantiate the Meta4 client.
+2. Use entity managers (Employees, CostCenters, Jobs) to call `create/update/delete` with DataFrames. Each call validates rows against the entity schema and appends valid rows to that entity’s in-memory `batch_df`. No file open/close happens per call; data is only accumulated.
+3. When ready, call the entity’s `export()` to write all accumulated rows from its `batch_df` into a CSV under `outputs/`.
+4. Call `meta4.upload()` (on the Meta4 client) to send the generated CSV files to the server over SFTP.
+
+- Employee movement codes: `1=Create`, `2=Delete`, `3=Update`
+- Cost center movement codes: `-36=Creation/Modification`, `-37=Deletion`
+- Job movement codes: `-28=Creation`, `-29=Modification/Deletion`
+
+Notes:
+- Date fields expect the `DD/MM/YYYY` format. Schemas will auto-parse strings.
+- DataFrame column names must match schema field names (e.g., `job_id`, `cost_center_id`, `employee_name`). Schemas output by aliases, but input can use field names.
+
+### Design: batch_df per entity
+
+Each entity manager (`employees`, `cost_centers`, `jobs`) maintains its own `batch_df` in memory. Calls to `create`, `update`, and `delete` only validate input data and append valid rows to the corresponding `batch_df`. This avoids repeated reads/writes and keeps I/O minimal. The actual file write happens only when you call the entity’s `export()` method, which flushes the accumulated rows into a CSV under `outputs/`.
+
+### Installation
+
+```bash
+pip install -e .
+```
+
+Or activate your virtual environment and run the same command from the repository root.
+
+### Quick start
+
+```python
+import pandas as pd
+from brynq_sdk_meta4 import Meta4
+
+meta4 = Meta4()  # writes to outputs/
+
+# 1) Cost Centers
+df_cc = pd.DataFrame([
+    {"cost_center_id": "C200", "cost_center_name": "Centro de coste"},
+])
+meta4.cost_centers.create(df_cc)
+meta4.cost_centers.export()  # outputs/cost_center_import.csv
+
+# 2) Jobs
+df_jobs = pd.DataFrame([
+    {"job_id": "100", "job_name": "Puesto 1", "start_date": "01/01/2025", "end_date": "31/12/2025", "cno_subcode": "1120"},
+])
+meta4.jobs.create(df_jobs)
+meta4.jobs.export()  # outputs/job_import.csv
+
+# 3) Employees
+df_emp = pd.DataFrame([
+    {"effective_date": "06/03/2025", "person_id": "M88019", "employee_name": "Carlos", "document_type": "1", "document_number": "16719242J"}
+])
+meta4.employees.create(df_emp)
+meta4.employees.export()  # outputs/employee_import.csv
+```
+
+### CRUD workflow examples
+
+```python
+# Jobs UPDATE/DELETE
+df_update = pd.DataFrame([
+    {"job_id": "100", "job_name": "Puesto 1 actualizado"}
+])
+meta4.jobs.update(df_update)
+
+df_delete = pd.DataFrame([
+    {"job_id": "100"}
+])
+meta4.jobs.delete(df_delete)
+
+# Export (writes all accumulated valid rows)
+meta4.jobs.export()
+```
+
+```python
+# Cost Centers UPDATE/DELETE
+df_cc_update = pd.DataFrame([
+    {"cost_center_id": "C200", "cost_center_name": "Centro actualizado"}
+])
+meta4.cost_centers.update(df_cc_update)
+
+df_cc_delete = pd.DataFrame([
+    {"cost_center_id": "C200"}
+])
+meta4.cost_centers.delete(df_cc_delete)
+meta4.cost_centers.export()
+```
+
+```python
+# Employees UPDATE/DELETE
+df_emp_update = pd.DataFrame([
+    {"effective_date": "06/03/2025", "person_id": "M88077", "ss_number": "2000"}
+])
+meta4.employees.update(df_emp_update)
+
+df_emp_delete = pd.DataFrame([
+    {"effective_date": "06/03/2025", "termination_date": "08/03/2025", "termination_reason": "012", "unemployment_cause": "74", "person_id": "M88087"}
+])
+meta4.employees.delete(df_emp_delete)
+meta4.employees.export()
+```
+
+### Outputs
+
+- `outputs/employee_import.csv`
+- `outputs/cost_center_import.csv`
+- `outputs/job_import.csv`
+
+### Running the tests
+
+See `brynq_sdk_meta4/test_meta.py` for end-to-end examples (create/update/delete/export). Run directly:
+
+```bash
+python brynq_sdk_meta4/test_meta.py
+```
+
+### Upload via SFTP
+
+The client can upload generated CSVs via SFTP. By default, `employee_import.csv` goes to `ENTRADA/EMPLEADOS`, and other files go to `SALIDA`.
+
+```python
+from brynq_sdk_meta4 import Meta4
+meta4 = Meta4()
+
+# After you have exported CSVs
+meta4.upload()  # uploads to '/' + expected folders, then deletes uploaded CSVs locally
+
+# Custom remote base path
+meta4.upload(upload_path="/custom/path")
+```
+
+How it works:
+
+- **Scan output directory**: Looks for all files ending with `.csv` under the local `outputs/` folder (or the custom `output_path` if you set it on the `Meta4` client).
+- **Per-file routing**:
+  - `employee_import.csv` is uploaded to `ENTRADA/EMPLEADOS` on the remote side.
+  - All other CSVs are uploaded to `SALIDA`.
+- **Remote path base**: The optional `upload_path` argument is prepended to the remote folders (e.g., `/custom/path/ENTRADA/EMPLEADOS/employee_import.csv`).
+- **Upload behavior**: Each file is attempted independently. Failures are printed and skipped so the rest can continue.
+- **Return value**: A string summarizing the successfully uploaded remote file paths.
+- **Errors**: If no files are successfully uploaded, an exception is raised.
+
+### Notes & tips
+
+- Replace `NaN` values before validation if needed: `df.replace({pd.NA: None, pd.NaT: None})` (see examples).
+- `export()` writes only the accumulated valid records from the internal `batch_df`. Use `get_batch_df()` to inspect and `clear_batch_df()` to reset when starting a new batch.

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/__init__.py

@@ -0,0 +1 @@
+from .meta4 import Meta4

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/cost_centers.py

@@ -0,0 +1,74 @@
+import pandas as pd
+from .schemas.cost_center import CostCenterSchema
+
+class CostCenters:
+    """
+    Handles all cost center related operations in Meta4
+    """
+    def __init__(self, meta4):
+        self.meta4 = meta4
+        schema_fields = CostCenterSchema.model_fields
+        column_names = [field.alias or name for name, field in schema_fields.items()]
+        self.batch_df = pd.DataFrame(columns=column_names)
+
+    def create(self, df: pd.DataFrame) -> str:
+        """
+        Export cost center data to CSV with validation.
+
+        Args:
+            df (pd.DataFrame): DataFrame containing cost center data to validate and export
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        try:
+            # Set movement_type to ALTA for all records
+            df['movement_type'] = '-36'
+            valid_df = self.meta4.validate(df=df,schema=CostCenterSchema)
+            self.batch_df = pd.concat([self.batch_df, valid_df], ignore_index=True)
+            return {"success": True, "message": "Cost centers created successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to export cost centers: {e}")
+
+    def update(self, df: pd.DataFrame) -> str:
+
+        try:
+            # Set movement_type to MODIFICACION for all records
+            df['movement_type'] = '-36'
+            valid_df = self.meta4.validate(df=df,schema=CostCenterSchema)
+            self.batch_df = pd.concat([self.batch_df, valid_df], ignore_index=True)
+            return {"success": True, "message": "Cost centers updated successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to update cost centers: {e}")
+
+    def delete(self, df: pd.DataFrame) -> str:
+
+        try:
+            # Set movement_type to BAJA for all records
+            df['movement_type'] = '-37'
+            valid_df = self.meta4.validate(df=df,schema=CostCenterSchema)
+            self.batch_df = pd.concat([self.batch_df, valid_df], ignore_index=True)
+            return {"success": True, "message": "Cost centers deleted successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to delete cost centers: {e}")
+
+    def export(self) -> str:
+        """
+        Export cost center data to CSV with validation.
+        """
+        return self.meta4.export(df=self.batch_df, filename="cost_center_import.csv")
+
+    def get_batch_df(self) -> pd.DataFrame:
+        """
+        Get the current batch DataFrame containing all validated cost center records.
+        """
+        return self.batch_df
+
+    def clear_batch_df(self):
+        """
+        Clear the batch DataFrame.
+        """
+        self.batch_df = pd.DataFrame()

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/employees.py

@@ -0,0 +1,121 @@
+import pandas as pd
+from .schemas.employee import EmployeeSchema
+import os
+
+class Employees:
+    """
+    Handles all employee related operations in Meta4
+    """
+    def __init__(self, meta4):
+        self.meta4 = meta4
+        # Initialize batch_df with EmployeeSchema columns
+        schema_fields = EmployeeSchema.model_fields
+        column_names = [field.alias or name for name, field in schema_fields.items()]
+        self.batch_df = pd.DataFrame(columns=column_names)
+
+    def create(self, df: pd.DataFrame) -> str:
+        """
+        Create new employees (ALTA movement type).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing employee data for creation
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        try:
+            # Set movement_type to ALTA for all records
+            df['movement_type'] = '1'
+            validated_df = self.meta4.validate(df=df, schema=EmployeeSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Employees created successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to create employees: {e}")
+
+    def update(self, df: pd.DataFrame) -> str:
+        """
+        Update existing employees (MODIFICACION movement type).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing employee data for update
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        try:
+            # Set movement_type to MODIFICACION for all records
+            df['movement_type'] = '3'
+            validated_df = self.meta4.validate(df=df, schema=EmployeeSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Employees updated successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to update employees: {e}")
+
+    def delete(self, df: pd.DataFrame) -> str:
+        """
+        Delete/terminate employees (BAJA movement type).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing employee data for termination
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        try:
+            # Set movement_type to BAJA for all records
+            df['movement_type'] = '2'
+            validated_df = self.meta4.validate(df=df, schema=EmployeeSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Employees deleted successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to delete employees: {e}")
+
+    def export(self) -> str:
+        """
+        Export employee data to CSV with validation (generic export).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing employee data to validate and export
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        return self.meta4.export(df=self.batch_df, filename="employee_import.csv")
+
+    def get_batch_df(self) -> pd.DataFrame:
+        """
+        Get the current batch DataFrame containing all validated employee records.
+
+        Returns:
+            pd.DataFrame: The batch DataFrame with all validated records
+        """
+        return self.batch_df
+
+    def clear_batch_df(self):
+        """
+        Clear the batch DataFrame.
+        """
+        schema_fields = EmployeeSchema.model_fields
+        column_names = [field.alias or name for name, field in schema_fields.items()]
+        self.batch_df = pd.DataFrame(columns=column_names)

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/jobs.py

@@ -0,0 +1,120 @@
+import pandas as pd
+from .schemas.job import JobSchema
+
+class Jobs:
+    """
+    Handles all job related operations in Meta4
+    """
+    def __init__(self, meta4):
+        self.meta4 = meta4
+        # Initialize batch_df with JobSchema columns
+        schema_fields = JobSchema.model_fields
+        column_names = [field.alias or name for name, field in schema_fields.items()]
+        self.batch_df = pd.DataFrame(columns=column_names)
+
+    def create(self, df: pd.DataFrame) -> str:
+        """
+        Create new jobs (CREATE movement type -28).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing job data for creation
+
+        Returns:
+            str: Success message
+
+        Raises:
+            Exception: If validation fails
+        """
+        try:
+            # Set movement_type to CREATE for all records
+            df['movement_type'] = '-28'
+            validated_df = self.meta4.validate(df=df, schema=JobSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Jobs created successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to create jobs: {e}")
+
+    def update(self, df: pd.DataFrame) -> str:
+        """
+        Update existing jobs (UPDATE movement type -29).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing job data for update
+
+        Returns:
+            str: Success message
+
+        Raises:
+            Exception: If validation fails
+        """
+        try:
+            # Set movement_type to UPDATE for all records
+            df['movement_type'] = '-29'
+            validated_df = self.meta4.validate(df=df, schema=JobSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Jobs updated successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to update jobs: {e}")
+
+    def delete(self, df: pd.DataFrame) -> str:
+        """
+        Delete jobs (DELETE movement type -29).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing job data for deletion
+
+        Returns:
+            str: Success message
+
+        Raises:
+            Exception: If validation fails
+        """
+        try:
+            # Set movement_type to DELETE for all records
+            df['movement_type'] = '-29'
+            validated_df = self.meta4.validate(df=df, schema=JobSchema)
+
+            # Append validated DataFrame to batch_df
+            self.batch_df = pd.concat([self.batch_df, validated_df], ignore_index=True)
+
+            return {"success": True, "message": "Jobs deleted successfully"}
+        except Exception as e:
+            raise Exception(f"Failed to delete jobs: {e}")
+
+    def export(self) -> str:
+        """
+        Export job data to CSV with validation (generic export).
+
+        Args:
+            df (pd.DataFrame): DataFrame containing job data to validate and export
+
+        Returns:
+            str: Path to the generated CSV file
+
+        Raises:
+            Exception: If validation or export fails
+        """
+        return self.meta4.export(df=self.batch_df, filename="job_import.csv")
+
+    def get_batch_df(self) -> pd.DataFrame:
+        """
+        Get the current batch DataFrame containing all validated job records.
+
+        Returns:
+            pd.DataFrame: The batch DataFrame with all validated records
+        """
+        return self.batch_df
+
+    def clear_batch_df(self):
+        """
+        Clear the batch DataFrame.
+        """
+        schema_fields = JobSchema.model_fields
+        column_names = [field.alias or name for name, field in schema_fields.items()]
+        self.batch_df = pd.DataFrame(columns=column_names)

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/meta4.py

@@ -0,0 +1,165 @@
+import os
+import csv
+from typing import List, Optional, Literal
+import pandas as pd
+from brynq_sdk_ftp import SFTP
+from brynq_sdk_brynq import BrynQ
+from brynq_sdk_functions import Functions
+import pydantic
+from .employees import Employees
+from .cost_centers import CostCenters
+from .jobs import Jobs
+
+class Meta4(BrynQ):
+    """
+    Meta4 HR system client for BrynQ integrations.
+    Focuses on schema validation and CSV export functionality.
+    """
+
+    def __init__(self, system_type: Optional[Literal['source', 'target']] = None, output_path:str="outputs", debug=False):
+        """
+        Initialize Meta4 client.
+        """
+        super().__init__()
+        self.debug = debug
+
+        self.output_path = output_path
+        # SFTP client as a composition attribute
+        self.sftp = SFTP()
+        credentials = self.interfaces.credentials.get(system="meta-4", system_type=system_type)
+        credentials = credentials.get('data', credentials)
+
+        self.sftp._set_credentials(credentials)
+
+        # Initialize entity classes
+        self.employees = Employees(self)
+        self.cost_centers = CostCenters(self)
+        self.jobs = Jobs(self)
+
+    def validate(self, df: pd.DataFrame, schema: pydantic.BaseModel) -> pd.DataFrame:
+        """
+        Validate data against schema and return validated DataFrame.
+        """
+        try:
+                data_list = df.to_dict('records')
+
+                valid_data = []
+                invalid_data = []
+                for data_item in data_list:
+                    try:
+                        validated_item = schema(**data_item)
+                        valid_data.append(validated_item.model_dump(by_alias=True, mode="json"))
+                    except Exception as validation_error:
+                        invalid_data.append({
+                            'data': data_item,
+                            'error': str(validation_error)
+                        })
+
+                # Print invalid data count
+                if invalid_data:
+                    print(f" {len(invalid_data)} lines of {schema.__name__} data validation failed:")
+
+                # Convert to DataFrame
+                df = pd.DataFrame(valid_data)
+                return df
+        except Exception as e:
+            raise Exception(f"Failed to validate data: {e}")
+
+    def export(
+      self,
+        df: pd.DataFrame,
+        filename: str
+    ) -> dict:
+        """
+        Validate data against schema and export to CSV file.
+
+        Args:
+            schema_class: Pydantic schema class (e.g., EmployeeSchema, CostCenterSchema, JobSchema)
+            df: DataFrame containing data to validate and export
+
+        Returns:
+            dict: Dictionary containing filepath, valid count, invalid count, and invalid data list
+
+        Raises:
+            ValidationError: If data validation fails
+            Exception: If file writing fails
+        """
+        try:
+            os.makedirs(self.output_path, exist_ok=True)
+
+
+            # Export to CSV
+            df.to_csv(
+                f"{self.output_path}/{filename}",
+                index=False,
+                encoding="utf-8-sig",
+                sep=";",
+                quotechar='"',
+                quoting=csv.QUOTE_MINIMAL
+            )
+            return {
+                'filepath': f"{self.output_path}/{filename}",
+            }
+
+        except Exception as e:
+            raise Exception(f"Failed to export data for {filename}: {e}")
+
+    def upload(self, upload_path: str="/") -> List[str]:
+        """
+        Upload all CSV files from the output directory to the specified remote path.
+
+        This method scans the output directory for CSV files and uploads each one
+        to the remote server via SFTP. Files are uploaded to different directories
+        based on their filename:
+        - employee_import.csv -> ENTRADA folder
+        - Other files -> SALIDA folder
+
+        Args:
+            upload_path (str): Remote directory path where CSV files will be uploaded.
+                             Must be a valid directory path on the remote server.
+
+        Returns:
+            List[str]: List of successfully uploaded remote file paths.
+                      Each path includes the remote directory and filename.
+        Raises:
+            Exception: For any other upload-related errors.
+        """
+        try:
+
+            # Get list of CSV files in output directory
+            csv_files = [f for f in os.listdir(self.output_path) if f.endswith('.csv')]
+            uploaded_files = []
+
+            # Upload each CSV file
+            for csv_file in csv_files:
+                try:
+                    local_filepath = os.path.join(self.output_path, csv_file)
+
+                    # Determine upload directory based on filename
+                    if csv_file == "employee_import.csv":
+                        upload_dir = "ENTRADA/EMPLEADOS"
+                    else:
+                        upload_dir = "SALIDA"
+
+                    remote_filepath = os.path.join(upload_path, upload_dir, csv_file).replace('\\', '/')
+
+                    # Upload file using SFTP attribute
+                    self.sftp.upload_file(
+                        local_filepath=local_filepath,
+                        remote_filepath=remote_filepath
+                    )
+
+                    uploaded_files.append(remote_filepath)
+
+                except Exception as file_error:
+                    print(f"Failed to upload {csv_file}: {file_error}")
+                    # Continue with other files even if one fails
+                    continue
+
+            if not uploaded_files:
+                raise Exception("No files were successfully uploaded")
+
+            return f"The files successfully uploaded: {', '.join(uploaded_files)}"
+
+        except Exception as e:
+            raise Exception(f"Upload failed with unexpected error: {e}")

brynq_sdk_meta4-1.0.0/brynq_sdk_meta4/schemas/__init__.py

@@ -0,0 +1,9 @@
+from .employee import EmployeeSchema
+from .cost_center import CostCenterSchema
+from .job import JobSchema
+
+__all__ = [
+    'EmployeeSchema',
+    'CostCenterSchema',
+    'JobSchema'
+]