jiboia-gpu 0.1.0__tar.gz

jiboia_gpu-0.1.0/LICENSE

@@ -0,0 +1,24 @@
+MIT License
+
+Copyright (c) 2025 Lucas M.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+USE OF THIS SOFTWARE IS AT YOUR OWN RISK. THE AUTHORS PROVIDE NO WARRANTIES,
+AND ARE NOT LIABLE FOR ANY DAMAGES, DIRECT OR INDIRECT, ARISING FROM ITS USE.

jiboia_gpu-0.1.0/PKG-INFO

@@ -0,0 +1,70 @@
+Metadata-Version: 2.4
+Name: jiboia_gpu
+Version: 0.1.0
+Summary: jiboia_gpu is a python package for automatically normalizing DataFrames and optimizing types efficiently.
+Author-email: Lucas Muffato <lucasmuffato.eng@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2025 Lucas M.
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in
+        all copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+        THE SOFTWARE.
+        
+        USE OF THIS SOFTWARE IS AT YOUR OWN RISK. THE AUTHORS PROVIDE NO WARRANTIES,
+        AND ARE NOT LIABLE FOR ANY DAMAGES, DIRECT OR INDIRECT, ARISING FROM ITS USE.
+        
+Project-URL: Homepage, https://github.com/lmuffato/jiboia_gpu
+Project-URL: Bug_Tracker, https://github.com/lmuffato/jiboia_gpu/issues
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: cudf-cu12>=25.8
+Dynamic: license-file
+
+# Jiboia
+
+**jiboia-gpu** is a Python package to **normalize and optimize DataFrames automatically** efficiently using the Nvidia GPU in the RAPIDS ecosystem.
+
+Key features:
+- **String normalization**:
+  - Removes extra spaces.
+  - Strips leading and trailing spaces.
+  - Detects data pollution (e.g., columns that should be numeric but contain strings).
+- **Type conversion**:
+  - Numeric strings and floats ending in `.0` → integers (`int8`, `int16`, `int32`, …).
+  - Converts floats and integers to the most memory-efficient type.
+  - Converts strings in various date formats to `datetime` (`yyyy?mm?dd`, `dd?mm?yyyy`, `yyyymmd`, `dd?mm?yy`).
+  - Converts time strings (`hhmm UTC`, `hh:mm:ss`, `hh:mm:ss.s`) to `timedelta`.
+- **Null standardization** → converts different null representations to `pd.NA`.
+- **Automatic CSV detection**:
+  - Detects delimiter.
+  - Detects encoding.
+- **Memory optimization**:
+  - Provides memory usage information for DataFrames.
+  - Converts columns to the most compact types possible.
+
+---
+
+## Example Usage
+
+```python
+from jiboia-gpu import jiboia_gpu as jb
+
+pd.normalize_category("data_frame_cudf")
+
+```

jiboia_gpu-0.1.0/README.md

@@ -0,0 +1,32 @@
+# Jiboia
+
+**jiboia-gpu** is a Python package to **normalize and optimize DataFrames automatically** efficiently using the Nvidia GPU in the RAPIDS ecosystem.
+
+Key features:
+- **String normalization**:
+  - Removes extra spaces.
+  - Strips leading and trailing spaces.
+  - Detects data pollution (e.g., columns that should be numeric but contain strings).
+- **Type conversion**:
+  - Numeric strings and floats ending in `.0` → integers (`int8`, `int16`, `int32`, …).
+  - Converts floats and integers to the most memory-efficient type.
+  - Converts strings in various date formats to `datetime` (`yyyy?mm?dd`, `dd?mm?yyyy`, `yyyymmd`, `dd?mm?yy`).
+  - Converts time strings (`hhmm UTC`, `hh:mm:ss`, `hh:mm:ss.s`) to `timedelta`.
+- **Null standardization** → converts different null representations to `pd.NA`.
+- **Automatic CSV detection**:
+  - Detects delimiter.
+  - Detects encoding.
+- **Memory optimization**:
+  - Provides memory usage information for DataFrames.
+  - Converts columns to the most compact types possible.
+
+---
+
+## Example Usage
+
+```python
+from jiboia-gpu import jiboia_gpu as jb
+
+pd.normalize_category("data_frame_cudf")
+
+```

jiboia_gpu-0.1.0/jiboia_gpu/__init__.py

@@ -0,0 +1,70 @@
+from .gpu_normalize_string_utils import GPUNormalizeStringUtils
+from .gpu_normalize_null_utils import GPUNormalizeNullUtils
+from .gpu_analysis_utils import GPUAnalysisUtils
+from .gpu_normalize_date_utils import GPUNormalizeDateUtils
+from .gpu_normalize_time_utils import GPUNormalizeTimeUtils
+from .gpu_normalize_datetime_utils import GPUNormalizeDateTimeUtils
+from .gpu_normalize_boolean_utils import GPUNormalizeBooleanUtils
+from .gpu_normalize_numeric_utils import GPUNormalizeNumericUtils
+from .gpu_file_analysis_utils import GPUFileAnalysisUtils
+
+
+class JiboiaGPU:
+    # STRING
+    normalize_str = staticmethod(GPUNormalizeStringUtils.normalize_str)
+    to_category = staticmethod(GPUNormalizeStringUtils.to_category)
+    normalize_category = staticmethod(GPUNormalizeStringUtils.normalize_category)
+    normalize_unique_values = staticmethod(GPUNormalizeStringUtils.normalize_unique_values)
+    
+    # NULL
+    normalize_na = staticmethod(GPUNormalizeNullUtils.normalize_na)
+
+    # BOOL
+    normalize_bool = staticmethod(GPUNormalizeBooleanUtils.normalize_bool)
+    
+    # NUMBER
+    normalize_number = staticmethod(GPUNormalizeNumericUtils.normalize_number)
+    is_probably_number = staticmethod(GPUNormalizeNumericUtils.is_probably_number)
+
+    # DATE
+    normalize_date = staticmethod(GPUNormalizeDateUtils.normalize_date)
+    normalize_datetime = staticmethod(GPUNormalizeDateUtils.normalize_datetime)
+    is_date_column_by_samples = staticmethod(GPUNormalizeDateUtils.is_date_column_by_samples)
+    is_date = staticmethod(GPUNormalizeDateUtils.is_date)
+    to_date_iso = staticmethod(GPUNormalizeDateUtils.to_date_iso)
+    to_datetime = staticmethod(GPUNormalizeDateUtils.to_datetime)
+
+
+    # TIME
+    normalize_time = staticmethod(GPUNormalizeTimeUtils.normalize_time)
+    is_time = staticmethod(GPUNormalizeTimeUtils.is_time)
+
+    # TIMEDELTA
+    normalize_timedelta = staticmethod(GPUNormalizeTimeUtils.normalize_timedelta)
+
+    # DATETIME
+    create_datetime_column_from_date_and_time = staticmethod(GPUNormalizeDateTimeUtils.create_datetime_column_from_date_and_time)
+    create_day_week_column = staticmethod(GPUNormalizeDateTimeUtils.create_day_week_column)
+    create_month_br_category_column = staticmethod(GPUNormalizeDateTimeUtils.create_month_br_category_column)
+    create_day_week_br_category_column = staticmethod(GPUNormalizeDateTimeUtils.create_day_week_br_category_column)
+    create_hour_category_column = staticmethod(GPUNormalizeDateTimeUtils.create_hour_category_column)
+
+    # ANALYSIS
+    drop_columns = staticmethod(GPUAnalysisUtils.drop_columns)
+    cudf_size_info = staticmethod(GPUAnalysisUtils.cudf_size_info)
+    pandasdf_size_info = staticmethod(GPUAnalysisUtils.pandasdf_size_info)
+    get_gpu_memory_info = staticmethod(GPUAnalysisUtils.get_gpu_memory_info)
+    is_vram_use_limit = staticmethod(GPUAnalysisUtils.is_vram_use_limit)
+    get_index_samples = staticmethod(GPUAnalysisUtils.get_index_samples)
+    infer_by_sample = staticmethod(GPUAnalysisUtils.infer_by_sample)
+
+    # FILE
+    get_csv_info = staticmethod(GPUFileAnalysisUtils.get_csv_info)
+    convert_csv_to_utf8 = staticmethod(GPUFileAnalysisUtils.convert_csv_to_utf8)
+    convert_all_csvs_to_utf8 = staticmethod(GPUFileAnalysisUtils.convert_all_csvs_to_utf8)
+    read_csv_files = staticmethod(GPUFileAnalysisUtils.read_csv_files)
+
+# instância única para ser usada como namespace
+jiboia_gpu = JiboiaGPU()
+
+__all__ = ["jiboia_gpu", "JiboiaGPU"]

jiboia_gpu-0.1.0/jiboia_gpu/analysis_utils.py

@@ -0,0 +1,274 @@
+import cudf
+import cupy as cp
+import pandas as pd
+
+
+def print_text_red(text: str) -> str:
+    return f"\033[1;31m{text}\033[0m"
+
+def print_text_yellow(text: str) -> str:
+    return f"\033[1;33m{text}\033[0m"
+
+def print_text_green(text: str) -> str:
+    return f"\033[1;32m{text}\033[0m"
+
+
+def print_job_drop_column_done(
+    columns_to_delete: list[str]
+) -> None:
+    colored_names: list[str] = [
+        print_text_yellow(name) for name in columns_to_delete
+    ]
+    
+    msg: str = ", ".join(colored_names)
+
+    print(
+        print_text_green("Done!"),
+        "column",
+        msg,
+        "was",
+        print_text_red("dropped")
+    )
+
+
+class GPUAnalysisUtils:
+    @staticmethod
+    def drop_columns(
+        current_df: cudf.DataFrame,
+        column_names: list[str],
+    ) -> None:
+
+        columns_to_delete: list = []
+
+        for column_name in column_names:
+
+            if column_name in current_df.columns:
+                columns_to_delete.append(column_name)
+
+
+        if len(columns_to_delete) >= 1:
+
+            current_df.drop(
+                columns=columns_to_delete,
+                inplace=True
+            )
+            print_job_drop_column_done(columns_to_delete)
+
+
+    @staticmethod
+    def cudf_size_info(current_df: cudf.DataFrame, print_info: bool = False) -> None:
+
+        rows: int = current_df.shape[0]
+        columns: int =  current_df.shape[1]
+        vram_size_mb: float = round(current_df.memory_usage(index=True, deep=True).sum() / (1024 * 1024), 2)
+
+        cudf_info: dict[str, any] = {
+            "rows": rows,
+            "columns": columns,
+            "VRAM size Mb": vram_size_mb
+        }
+
+        if print_info:
+            print(
+                print_text_green("Done!"),
+                "rows:",
+                print_text_yellow(rows),
+                "columns:",
+                print_text_yellow(columns),
+                "VRAM size Mb:",
+                print_text_yellow(vram_size_mb),
+            )
+                
+        return cudf_info
+
+
+    @staticmethod
+    def get_gpu_memory_info(device_id: int = 0) -> dict[str, int]:
+        free_bytes, total_bytes = cp.cuda.runtime.memGetInfo()
+        return {
+            "free_mb": round(free_bytes / (1024 * 1024), 2),
+            "total": round(total_bytes / (1024 * 1024), 2),
+            "used_mb":  round((total_bytes - free_bytes) / (1024 * 1024), 2),
+        }
+    
+    @staticmethod
+    def is_vram_use_limit(device_id: int = 0) -> dict[str, int]:
+        free_bytes, total_bytes = cp.cuda.runtime.memGetInfo()
+        vram_percent_in_use: float = round(((total_bytes - free_bytes) / total_bytes) * 100, 1) >= 90
+        
+        if vram_percent_in_use >= 90:
+            return True
+        
+        return False
+    
+
+    @staticmethod
+    def pandasdf_size_info(current_df: pd.DataFrame) -> None:
+        return {
+            "rows": current_df.shape[0],
+            "columns": current_df.shape[1],
+            "RAM MB": round(current_df.memory_usage(index=True, deep=True).sum() / (1024 * 1024), 2)
+        }
+    
+
+    def print_converted_column_type(
+        column_name: str,
+        column_type: str
+    ) -> None:
+        print(
+            "\033[1;32mDone!\033[0m",
+            "column",
+            # f"{column_name}",
+            f"\033[1;33m{column_name}\033[0m",
+            "converted to", f"\033[1;33m{column_type}\033[0m"
+        )
+       
+    # @staticmethod
+    # def get_frequency_by_column(current_df: pd.DataFrame, column: str) -> pd.DataFrame:
+    #     '''
+    #     Gera um current_df com a frequencia de valores de uma coluna
+    #     '''
+    #     current_df_frequency: pd.DataFrame = (
+    #         current_df[column]
+    #         .value_counts()
+    #         .reset_index()
+    #     )
+    
+    #     current_df_frequency.columns = [column, 'frequency']
+    
+    #     current_df_frequency: pd.DataFrame = (
+    #         current_df_frequency
+    #         .sort_values(
+    #             by='frequency',
+    #             ascending=False
+    #         )
+    #     )
+    
+    #     return current_df_frequency
+
+
+    # @staticmethod
+    # def get_unique_values(current_df: pd.DataFrame, column: str) -> pd.DataFrame:
+    #     '''
+    #     Retorna um dataframe com os valores únicos de uma coluna
+    #     '''
+    #     df_unique = pd.DataFrame({
+    #         column: current_df[column].dropna().unique()
+    #     })
+
+    #     df_unique = df_unique.sort_values(
+    #         by=column,
+    #         ascending=True
+    #     ).reset_index(drop=True)
+
+    #     return df_unique
+
+    @staticmethod
+    def get_index_samples(
+        series: cudf.Series,
+        n_parts: int = 10,
+        n_samples: int = 10
+    ) -> list[int]:
+        """
+        Verifica de forma performática na GPU se algum valor nas amostras de uma Series
+        corresponde a um padrão de data.
+
+        Args:
+            s (cudf.Series): A Series de strings que será analisada.
+            n_parts (int): O número de partes em que a Series será dividida para amostragem.
+            n_samples (int): O número de amostras a serem coletadas de cada parte.
+
+        Returns:
+            bool: True se um padrão de data for encontrado nas amostras, False caso contrário.
+        """
+
+        series_size = len(series)
+
+        # print(series.name)
+        # return
+
+        if ((n_parts * n_samples) >= series_size):
+            raise ValueError("The total number of samples requested exceeds or equals the series size. Please provide a smaller value for n_parts or n_samples.")
+        
+        if (series_size // n_parts == 0):
+            raise ValueError("The number of parts is greater than the series size. Please provide a smaller value for n_parts.")
+
+        # Gera todos os índices de amostragem DE UMA VEZ na GPU
+        step_pass = series_size // n_parts
+        
+        # Índices iniciais de cada bloco (ex: 0, 1000, 2000, ...)
+        start_indices = cp.arange(n_parts) * step_pass
+        
+        # Offsets dentro de cada bloco (ex: 0, 1, 2, ... n_samples-1)
+        sample_offsets = cp.arange(n_samples)
+
+        all_indices = (start_indices[:, None] + sample_offsets).flatten()
+        
+        # Garante que os índices não ultrapassem o tamanho da Series
+        all_indices = all_indices[all_indices < series_size]
+
+        return all_indices
+    
+
+    @staticmethod
+    def infer_by_sample(
+        series: cudf.Series,
+        regex_patern: str,
+        n_parts: int = 10,
+        n_samples: int = 10
+    ) -> bool:
+        """
+        Verifica de forma performática na GPU se algum valor nas amostras de uma Series
+        corresponde a um padrão de data.
+
+        Args:
+            s (cudf.Series): A Series de strings que será analisada.
+            n_parts (int): O número de partes em que a Series será dividida para amostragem.
+            n_samples (int): O número de amostras a serem coletadas de cada parte.
+
+        Returns:
+            bool: True se um padrão de data for encontrado nas amostras, False caso contrário.
+        """
+        series_size = len(series)
+
+        if series_size == 0:
+            return False
+
+        # Coluna sem dados
+        if series.notna().sum() == 0:
+            return False
+
+        if series.dtype not in ["object", "string"]:
+            return False
+        
+
+        # print(series.name)
+        # return
+
+        if ((n_parts * n_samples) >= series_size):
+            raise ValueError("The total number of samples requested exceeds or equals the series size. Please provide a smaller value for n_parts or n_samples.")
+        
+        if (series_size // n_parts == 0):
+            raise ValueError("The number of parts is greater than the series size. Please provide a smaller value for n_parts.")
+
+        # Gera todos os índices de amostragem DE UMA VEZ na GPU
+        step_pass = series_size // n_parts
+        
+        # Índices iniciais de cada bloco (ex: 0, 1000, 2000, ...)
+        start_indices = cp.arange(n_parts) * step_pass
+        
+        # Offsets dentro de cada bloco (ex: 0, 1, 2, ... n_samples-1)
+        sample_offsets = cp.arange(n_samples)
+
+        all_indices = (start_indices[:, None] + sample_offsets).flatten()
+        
+        # Garante que os índices não ultrapassem o tamanho da Series
+        all_indices = all_indices[all_indices < series_size]
+
+        # 3. Seleção de todas as amostras em uma única operação
+        samples = series.iloc[all_indices]
+
+        if (samples.str.contains(regex_patern).sum() == samples.notna().sum()):
+            return True
+
+        return False

jiboia_gpu-0.1.0/jiboia_gpu/file_analysis_utils.py

@@ -0,0 +1,179 @@
+import csv
+import chardet
+from pathlib import Path
+
+
+def print_text_red(text: str) -> str:
+    return f"\033[1;31m{text}\033[0m"
+
+def print_text_yellow(text: str) -> str:
+    return f"\033[1;33m{text}\033[0m"
+
+def print_text_green(text: str) -> str:
+    return f"\033[1;32m{text}\033[0m"
+
+
+class GPUFileAnalysisUtils:
+    @staticmethod
+    def get_csv_info(
+        file_path: str,
+        encoding: None|str = None,
+        skiprows: None|int = 0,
+        sample_characters: int = 2048,
+        sample_bytes: int = 100000
+    ) -> dict[str, any]:
+        """
+        Detecta o delimitador e o encoding de um arquivo CSV.
+
+        :param file_path: caminho do arquivo CSV
+        :param encoding: encoding forçado (se None, tenta detectar com chardet)
+        :param skiprows: linhas iniciais a ignorar
+        :param sample_characters: n de caracteres usados para sniffing
+        :param sample_bytes: n de bytes usados para detectar encoding
+        :return: (delimiter, encoding_detectado)
+                :exemplo_de_retorno:
+            {
+                'encoding': 'utf-8',
+                'confidence': 0.99,
+                'language': 'pt',
+                'delimiter': ','
+            }
+        """
+        csv_info: dict[str, any] = {
+            "encoding": encoding
+        }
+
+        # detecta encoding se não informado
+        if encoding is None:
+            with open(file_path, "rb") as f:
+                rawdata = f.read(sample_bytes)
+                enc_info = chardet.detect(rawdata)
+                encoding = enc_info["encoding"] or "utf-8"
+
+            csv_info = {**csv_info, **enc_info}
+
+        # detecta delimitador
+        with open(file_path, "r", encoding=encoding, errors="ignore") as file:
+            for _ in range(skiprows):
+                next(file, None)  # ignora linhas iniciais
+
+            sample = file.read(sample_characters)
+            sniffer = csv.Sniffer()
+            try:
+                dialect = sniffer.sniff(sample)
+                delimiter = dialect.delimiter
+            except csv.Error:
+                delimiter = ";"
+
+            csv_info["delimiter"] =  delimiter
+
+        print(
+            print_text_green("Done!"),
+            "file:",
+            print_text_yellow(file_path),
+            "delimiter:",
+            print_text_yellow(f'"{csv_info["delimiter"]}"'),
+            "encoding:",
+            print_text_yellow(csv_info["encoding"]),
+        )
+        return csv_info
+
+
+    @staticmethod
+    def convert_csv_to_utf8(file_path: str, output_path: str = None) -> str:
+        """
+        Converte um arquivo CSV para UTF-8.
+
+        :param file_path: caminho do arquivo original
+        :param output_path: caminho do arquivo convertido (se None, gera com _utf8.csv)
+        :return: caminho do arquivo convertido em UTF-8
+        """
+        file_path = Path(file_path)
+
+        # se não passar output_path, cria automático
+        if output_path is None:
+            output_path = file_path.with_name(file_path.stem + "_utf8.csv")
+
+        # detecta encoding
+        with open(file_path, "rb") as f:
+            rawdata = f.read(100000)
+            enc_info = chardet.detect(rawdata)
+            source_encoding = enc_info["encoding"] or "latin-1"
+            print(f"Encoding detectado: {source_encoding} (confiança={enc_info['confidence']:.2f})")
+
+        # lê no encoding original e salva em UTF-8
+        with open(file_path, "r", encoding=source_encoding, errors="ignore") as f_in:
+            data = f_in.read()
+
+        with open(output_path, "w", encoding="utf-8") as f_out:
+            f_out.write(data)
+
+        print(f"Arquivo convertido salvo em: {output_path}")
+        return str(output_path)
+
+
+    import chardet
+    from pathlib import Path
+
+
+    @staticmethod
+    def convert_all_csvs_to_utf8(folder_path: str, new_folder: bool = False) -> None:
+        """
+        Converte todos os arquivos CSV em uma pasta para UTF-8.
+        Detecta o encoding de cada arquivo automaticamente.
+        
+        :param folder_path: caminho da pasta contendo os CSVs
+        :param new_folder: se True, cria uma subpasta 'utf8_converted', 
+                           se False, salva os arquivos na mesma pasta com sufixo '_utf8'
+        """
+        folder = Path(folder_path)
+
+        if new_folder:
+            output_folder = folder / "utf8_converted"
+            output_folder.mkdir(exist_ok=True)
+        else:
+            output_folder = folder  # salva na mesma pasta
+
+        for csv_file in folder.glob("*.csv"):
+
+            # Detecta encoding
+            with open(csv_file, "rb") as f:
+                rawdata = f.read(100000)  # lê parte do arquivo
+                result = chardet.detect(rawdata)
+                source_encoding = result["encoding"] or "latin-1"
+                confidence = result["confidence"]
+                # print(f"{csv_file.name}: encoding detectado = {source_encoding} (confiança={confidence:.2f})")
+
+                print(
+                    "file:",
+                    csv_file,
+                    "encoding:",
+                    print_text_yellow(result["encoding"]),
+                    "confidence:",
+                    print_text_yellow(confidence),
+                )
+
+            # Se já estiver em UTF-8, pula
+            if source_encoding.lower() == 'utf-8':
+                continue
+
+            # Lê e converte para UTF-8
+            with open(csv_file, "r", encoding=source_encoding, errors="ignore") as f_in:
+                data = f_in.read()
+
+            if new_folder:
+                output_file = output_folder / csv_file.name
+            else:
+                output_file = output_folder / f"{csv_file.stem}_utf8.csv"
+
+            with open(output_file, "w", encoding="utf-8") as f_out:
+                f_out.write(data)
+
+            # print(f"Arquivo convertido salvo em: {output_file}")
+            print(
+                print_text_green("Done!"),
+                "file:",
+                print_text_yellow(output_file),
+                "converted to",
+                print_text_yellow("utf-8"),
+            )