dataforge-studio 1.0.1__tar.gz

dataforge_studio-1.0.1/PKG-INFO

@@ -0,0 +1,119 @@
+Metadata-Version: 2.4
+Name: dataforge-studio
+Version: 1.0.1
+Summary: Studio visual de preprocesamiento de datos — Universidad Alberto Hurtado
+Author-email: Álvaro Riquelme <alvaroriquelme.14@gmail.com>
+License-Expression: LicenseRef-Proprietary
+Project-URL: Homepage, https://dataforgeUAH.github.io/dataici
+Keywords: data,preprocessing,pandas,visual,pipeline,uah
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Intended Audience :: Education
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: fastapi>=0.110.0
+Requires-Dist: uvicorn[standard]>=0.29.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: openpyxl>=3.1.0
+Requires-Dist: python-multipart>=0.0.9
+Requires-Dist: matplotlib>=3.7.0
+Requires-Dist: numpy>=1.24.0
+
+# DataICI — v0.2
+
+Herramienta visual de preprocesamiento de datos para estudiantes de Ingeniería Civil Industrial.
+
+## Requisitos previos
+- Python 3.9+ → https://python.org
+- Node.js 18+ → https://nodejs.org
+
+---
+
+## Instalación y ejecución
+
+### 1. Backend (FastAPI + pandas)
+
+Abre una terminal en la carpeta `dataici/`:
+
+```bash
+# Windows
+cd backend
+pip install -r requirements.txt
+uvicorn main:app --reload
+
+# Mac
+cd backend
+pip3 install -r requirements.txt
+uvicorn main:app --reload
+```
+
+Backend corriendo en: http://localhost:8000
+
+---
+
+### 2. Frontend (React)
+
+Abre **otra terminal**:
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+App disponible en: http://localhost:5173
+
+---
+
+## Estructura del proyecto
+
+```
+dataici/
+├── backend/
+│   ├── main.py                ← API FastAPI
+│   ├── requirements.txt
+│   └── blocks/                ← un archivo por bloque
+│       ├── load_csv.py
+│       ├── drop_nulls.py
+│       ├── filter_rows.py
+│       ├── groupby.py
+│       └── export_csv.py
+│
+└── frontend/
+    ├── package.json
+    ├── vite.config.js
+    └── src/
+        ├── App.jsx              ← app principal
+        ├── nodes/
+        │   └── BlockNode.jsx    ← nodo del canvas
+        └── panels/
+            ├── Sidebar.jsx      ← bloques disponibles
+            ├── ParamsPanel.jsx  ← parámetros del bloque
+            └── PreviewPanel.jsx ← resultados
+```
+
+---
+
+## Cómo agregar un nuevo bloque
+
+Solo crear `backend/blocks/nuevo_bloque.py`. El frontend lo detecta automáticamente.
+
+```python
+METADATA = {
+    "type": "mi_bloque",
+    "label": "Mi bloque",
+    "category": "Limpieza",   # Entrada / Salida | Limpieza | Análisis
+    "params": [
+        {"key": "columna", "label": "Columna", "type": "text", "default": ""},
+        {"key": "metodo", "label": "Método", "type": "select", "options": ["a", "b"], "default": "a"},
+        {"key": "activo", "label": "Activar", "type": "toggle", "default": False},
+    ]
+}
+
+def run(df, params):
+    col = params.get("columna")
+    df = df.drop(columns=[col])
+    code = [f'df = df.drop(columns=["{col}"])']
+    return df, code
+```

dataforge_studio-1.0.1/README.md

@@ -0,0 +1,97 @@
+# DataICI — v0.2
+
+Herramienta visual de preprocesamiento de datos para estudiantes de Ingeniería Civil Industrial.
+
+## Requisitos previos
+- Python 3.9+ → https://python.org
+- Node.js 18+ → https://nodejs.org
+
+---
+
+## Instalación y ejecución
+
+### 1. Backend (FastAPI + pandas)
+
+Abre una terminal en la carpeta `dataici/`:
+
+```bash
+# Windows
+cd backend
+pip install -r requirements.txt
+uvicorn main:app --reload
+
+# Mac
+cd backend
+pip3 install -r requirements.txt
+uvicorn main:app --reload
+```
+
+Backend corriendo en: http://localhost:8000
+
+---
+
+### 2. Frontend (React)
+
+Abre **otra terminal**:
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+App disponible en: http://localhost:5173
+
+---
+
+## Estructura del proyecto
+
+```
+dataici/
+├── backend/
+│   ├── main.py                ← API FastAPI
+│   ├── requirements.txt
+│   └── blocks/                ← un archivo por bloque
+│       ├── load_csv.py
+│       ├── drop_nulls.py
+│       ├── filter_rows.py
+│       ├── groupby.py
+│       └── export_csv.py
+│
+└── frontend/
+    ├── package.json
+    ├── vite.config.js
+    └── src/
+        ├── App.jsx              ← app principal
+        ├── nodes/
+        │   └── BlockNode.jsx    ← nodo del canvas
+        └── panels/
+            ├── Sidebar.jsx      ← bloques disponibles
+            ├── ParamsPanel.jsx  ← parámetros del bloque
+            └── PreviewPanel.jsx ← resultados
+```
+
+---
+
+## Cómo agregar un nuevo bloque
+
+Solo crear `backend/blocks/nuevo_bloque.py`. El frontend lo detecta automáticamente.
+
+```python
+METADATA = {
+    "type": "mi_bloque",
+    "label": "Mi bloque",
+    "category": "Limpieza",   # Entrada / Salida | Limpieza | Análisis
+    "params": [
+        {"key": "columna", "label": "Columna", "type": "text", "default": ""},
+        {"key": "metodo", "label": "Método", "type": "select", "options": ["a", "b"], "default": "a"},
+        {"key": "activo", "label": "Activar", "type": "toggle", "default": False},
+    ]
+}
+
+def run(df, params):
+    col = params.get("columna")
+    df = df.drop(columns=[col])
+    code = [f'df = df.drop(columns=["{col}"])']
+    return df, code
+```

dataforge_studio-1.0.1/dataforge_studio.egg-info/PKG-INFO

@@ -0,0 +1,119 @@
+Metadata-Version: 2.4
+Name: dataforge-studio
+Version: 1.0.1
+Summary: Studio visual de preprocesamiento de datos — Universidad Alberto Hurtado
+Author-email: Álvaro Riquelme <alvaroriquelme.14@gmail.com>
+License-Expression: LicenseRef-Proprietary
+Project-URL: Homepage, https://dataforgeUAH.github.io/dataici
+Keywords: data,preprocessing,pandas,visual,pipeline,uah
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Intended Audience :: Education
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: fastapi>=0.110.0
+Requires-Dist: uvicorn[standard]>=0.29.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: openpyxl>=3.1.0
+Requires-Dist: python-multipart>=0.0.9
+Requires-Dist: matplotlib>=3.7.0
+Requires-Dist: numpy>=1.24.0
+
+# DataICI — v0.2
+
+Herramienta visual de preprocesamiento de datos para estudiantes de Ingeniería Civil Industrial.
+
+## Requisitos previos
+- Python 3.9+ → https://python.org
+- Node.js 18+ → https://nodejs.org
+
+---
+
+## Instalación y ejecución
+
+### 1. Backend (FastAPI + pandas)
+
+Abre una terminal en la carpeta `dataici/`:
+
+```bash
+# Windows
+cd backend
+pip install -r requirements.txt
+uvicorn main:app --reload
+
+# Mac
+cd backend
+pip3 install -r requirements.txt
+uvicorn main:app --reload
+```
+
+Backend corriendo en: http://localhost:8000
+
+---
+
+### 2. Frontend (React)
+
+Abre **otra terminal**:
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+App disponible en: http://localhost:5173
+
+---
+
+## Estructura del proyecto
+
+```
+dataici/
+├── backend/
+│   ├── main.py                ← API FastAPI
+│   ├── requirements.txt
+│   └── blocks/                ← un archivo por bloque
+│       ├── load_csv.py
+│       ├── drop_nulls.py
+│       ├── filter_rows.py
+│       ├── groupby.py
+│       └── export_csv.py
+│
+└── frontend/
+    ├── package.json
+    ├── vite.config.js
+    └── src/
+        ├── App.jsx              ← app principal
+        ├── nodes/
+        │   └── BlockNode.jsx    ← nodo del canvas
+        └── panels/
+            ├── Sidebar.jsx      ← bloques disponibles
+            ├── ParamsPanel.jsx  ← parámetros del bloque
+            └── PreviewPanel.jsx ← resultados
+```
+
+---
+
+## Cómo agregar un nuevo bloque
+
+Solo crear `backend/blocks/nuevo_bloque.py`. El frontend lo detecta automáticamente.
+
+```python
+METADATA = {
+    "type": "mi_bloque",
+    "label": "Mi bloque",
+    "category": "Limpieza",   # Entrada / Salida | Limpieza | Análisis
+    "params": [
+        {"key": "columna", "label": "Columna", "type": "text", "default": ""},
+        {"key": "metodo", "label": "Método", "type": "select", "options": ["a", "b"], "default": "a"},
+        {"key": "activo", "label": "Activar", "type": "toggle", "default": False},
+    ]
+}
+
+def run(df, params):
+    col = params.get("columna")
+    df = df.drop(columns=[col])
+    code = [f'df = df.drop(columns=["{col}"])']
+    return df, code
+```

dataforge_studio-1.0.1/dataforge_studio.egg-info/SOURCES.txt

@@ -0,0 +1,33 @@
+README.md
+pyproject.toml
+dataforge_studio.egg-info/PKG-INFO
+dataforge_studio.egg-info/SOURCES.txt
+dataforge_studio.egg-info/dependency_links.txt
+dataforge_studio.egg-info/entry_points.txt
+dataforge_studio.egg-info/requires.txt
+dataforge_studio.egg-info/top_level.txt
+dataici/__init__.py
+dataici/charts.py
+dataici/cli.py
+dataici/main.py
+dataici/blocks/__init__.py
+dataici/blocks/aggregate.py
+dataici/blocks/append_column.py
+dataici/blocks/concatenate.py
+dataici/blocks/drop_columns.py
+dataici/blocks/filter_rows.py
+dataici/blocks/handle_missings.py
+dataici/blocks/load_csv.py
+dataici/blocks/read_excel.py
+dataici/blocks/rename_columns.py
+dataici/blocks/reorder_columns.py
+dataici/blocks/replace_values.py
+dataici/blocks/resample.py
+dataici/blocks/sample_rows.py
+dataici/blocks/select_columns.py
+dataici/blocks/set_dtypes.py
+dataici/blocks/set_index.py
+dataici/blocks/write_csv.py
+dataici/static/index.html
+dataici/static/assets/index-CYGnphoW.js
+dataici/static/assets/index-DLK3-mBP.css

dataforge_studio-1.0.1/dataforge_studio.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

dataforge_studio-1.0.1/dataforge_studio.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+dataici = dataici.cli:main

dataforge_studio-1.0.1/dataforge_studio.egg-info/requires.txt

@@ -0,0 +1,7 @@
+fastapi>=0.110.0
+uvicorn[standard]>=0.29.0
+pandas>=2.0.0
+openpyxl>=3.1.0
+python-multipart>=0.0.9
+matplotlib>=3.7.0
+numpy>=1.24.0

dataforge_studio-1.0.1/dataforge_studio.egg-info/top_level.txt

@@ -0,0 +1 @@
+dataici

dataforge_studio-1.0.1/dataici/__init__.py

@@ -0,0 +1,3 @@
+# DataICI — Studio de Preprocesamiento de Datos
+# Universidad Alberto Hurtado
+__version__ = "1.0.0"

dataforge_studio-1.0.1/dataici/blocks/aggregate.py

@@ -0,0 +1,50 @@
+import json
+
+METADATA = {
+    "type":     "aggregate",
+    "label":    "Aggregate",
+    "category": "Resampling",
+    "params": [
+        {"key": "custom",    "label": "Custom Functions",  "type": "text", "default": "false"},
+        {"key": "func",      "label": "General function",  "type": "text", "default": "mean"},
+        {"key": "col_funcs", "label": "Per-column funcs",  "type": "text", "default": "{}"},
+    ]
+}
+
+# pandas resampler does not have a .unique() — map to nunique
+_ALIASES = {"unique": "nunique"}
+
+
+def _is_resampler(obj):
+    try:
+        from pandas.core.resample import DatetimeIndexResampler
+        return isinstance(obj, DatetimeIndexResampler)
+    except ImportError:
+        pass
+    return hasattr(obj, "_selected_obj") and not hasattr(obj, "to_dict")
+
+
+def run(obj, params):
+    custom       = str(params.get("custom",    "false")).strip().lower() == "true"
+    func         = (params.get("func",         "mean") or "mean").strip()
+    col_funcs_raw = params.get("col_funcs", "{}")
+
+    try:
+        col_funcs = json.loads(col_funcs_raw) if col_funcs_raw else {}
+    except Exception:
+        col_funcs = {}
+
+    is_rs  = _is_resampler(obj)
+    prefix = "resampler" if is_rs else "df"
+
+    if custom and col_funcs:
+        # Map aliases per-column
+        mapped = {col: _ALIASES.get(fn, fn) for col, fn in col_funcs.items()}
+        df   = obj.agg(mapped)
+        code = [f"df = {prefix}.agg({json.dumps(mapped)})"]
+    else:
+        actual = _ALIASES.get(func, func)
+        df   = getattr(obj, actual)()
+        code = [f"df = {prefix}.{actual}()"]
+
+    return df, code

dataforge_studio-1.0.1/dataici/blocks/append_column.py

@@ -0,0 +1,18 @@
+import numpy as np
+import pandas as pd
+METADATA = {
+    "type": "append_column",
+    "label": "Append a Column",
+    "category": "Columns",
+    "params": [
+        {"key": "colname", "label": "colname", "type": "text", "default": "new_col"},
+    ]
+}
+
+def run(df, params):
+    colname = params.get("colname", "new_col").strip()
+    if not colname:
+        raise ValueError("Especifica el nombre de la nueva columna.")
+    df[colname] = 'None'
+    code = [f'df["{colname}"] = None']
+    return df, code

dataforge_studio-1.0.1/dataici/blocks/concatenate.py

@@ -0,0 +1,70 @@
+import pandas as pd
+
+METADATA = {
+    "type":        "concatenate",
+    "label":       "Concatenate",
+    "category":    "DataFrame",
+    "multi_input": True,
+    "params": [
+        {"key": "input_count",      "label": "Número de entradas",    "type": "text",   "default": "2"},
+        {"key": "axis",             "label": "Axis",                  "type": "text",   "default": "index"},
+        {"key": "join",             "label": "Join",                  "type": "text",   "default": "outer"},
+        {"key": "change_col_names", "label": "Change column names",   "type": "text",   "default": "None"},
+        {"key": "ignore_index",     "label": "Ignore index",          "type": "text",   "default": "false"},
+        {"key": "sort",             "label": "Sort",                  "type": "text",   "default": "false"},
+    ]
+}
+
+
+def _bool(val):
+    if isinstance(val, bool):
+        return val
+    return str(val).strip().lower() == "true"
+
+
+def run(dfs, params):
+    """dfs: list of DataFrames received from upstream nodes."""
+    axis_raw = params.get("axis", "index")
+    axis     = 0 if axis_raw == "index" else 1
+
+    join          = params.get("join", "outer")
+    change_cols   = params.get("change_col_names", "None")   # None | prefix | suffix
+    ignore_index  = _bool(params.get("ignore_index", False))
+    sort          = _bool(params.get("sort", False))
+
+    if len(dfs) < 2:
+        raise ValueError("Concatenate necesita al menos 2 DataFrames.")
+
+    # ── Rename columns when axis=1 and change_col_names != None ──────────────
+    if axis == 1 and change_cols in ("prefix", "suffix"):
+        renamed = []
+        for i, df in enumerate(dfs):
+            if change_cols == "prefix":
+                df = df.rename(columns=lambda c: f"df{i+1}_{c}")
+            else:
+                df = df.rename(columns=lambda c: f"{c}_df{i+1}")
+            renamed.append(df)
+        dfs = renamed
+
+    # ── Build concat kwargs ───────────────────────────────────────────────────
+    kwargs = {"axis": axis, "join": join, "sort": sort}
+    if axis == 0:
+        kwargs["ignore_index"] = ignore_index
+
+    df_result = pd.concat(dfs, **kwargs)
+
+    # ── Code string ──────────────────────────────────────────────────────────
+    frames_repr = ", ".join(f"df_{i+1}" for i in range(len(dfs)))
+    code_lines  = []
+
+    if axis == 1 and change_cols == "prefix":
+        for i in range(len(dfs)):
+            code_lines.append(f"df_{i+1} = df_{i+1}.rename(columns=lambda c: f'df{i+1}_{{c}}')")
+    elif axis == 1 and change_cols == "suffix":
+        for i in range(len(dfs)):
+            code_lines.append(f"df_{i+1} = df_{i+1}.rename(columns=lambda c: f'{{c}}_df{i+1}')")
+
+    kw_str = ", ".join(f"{k}={repr(v)}" for k, v in kwargs.items())
+    code_lines.append(f"df = pd.concat([{frames_repr}], {kw_str})")
+
+    return df_result, code_lines

dataforge_studio-1.0.1/dataici/blocks/drop_columns.py

@@ -0,0 +1,19 @@
+METADATA = {
+    "type": "drop_columns",
+    "label": "Drop Columns",
+    "category": "Columns",
+    "params": [
+        {"key": "columns", "label": "columns", "type": "text", "default": ""},
+    ]
+}
+
+def run(df, params):
+    cols = [c.strip() for c in params.get("columns", "").split(",") if c.strip()]
+    if not cols:
+        raise ValueError("Selecciona al menos una columna para eliminar.")
+    missing = [c for c in cols if c not in df.columns]
+    if missing:
+        raise ValueError(f"Columnas no encontradas: {missing}")
+    df = df.drop(columns=cols)
+    code = [f"df = df.drop(columns={cols})"]
+    return df, code

dataforge_studio-1.0.1/dataici/blocks/filter_rows.py

@@ -0,0 +1,120 @@
+import json
+import pandas as pd
+
+METADATA = {
+    "type": "filter_rows",
+    "label": "Filter Rows",
+    "category": "DataFrame",
+    "params": [
+        {"key": "conditions", "label": "conditions", "type": "text", "default": "[]"},
+    ]
+}
+
+def run(df, params):
+    raw = params.get("conditions", "[]")
+    try:
+        conditions = json.loads(raw) if isinstance(raw, str) else raw
+    except Exception:
+        raise ValueError("Error al leer las condiciones.")
+
+    if not conditions:
+        raise ValueError("Agrega al menos una condición Where.")
+
+    OP_METHOD = {"==": "eq", "!=": "ne", "<": "lt", "<=": "le", ">": "gt", ">=": "ge"}
+    OP_SYM    = {"and": "&", "or": "|", "xor": "^"}
+
+    masks      = []
+    cond_lines = []
+
+    for i, cond in enumerate(conditions):
+        col     = cond.get("column", "")
+        op      = cond.get("operator", "==")
+        typ     = cond.get("type", "number")
+        val     = str(cond.get("value", "0")).strip()
+        negate  = cond.get("not", False)
+        logical = cond.get("logical", "and")
+
+        if not col or col not in df.columns:
+            raise ValueError(f"Columna '{col}' no encontrada en el DataFrame.")
+
+        s = df[col]
+
+        # ── Build mask ───────────────────────────────────────────────────────
+        if op == "isna":
+            mask      = s.isna()
+            code_expr = f"df['{col}'].isna()"
+
+        elif op == "notna":
+            mask      = s.notna()
+            code_expr = f"df['{col}'].notna()"
+
+        elif op == "isin":
+            items = [v.strip() for v in val.split(",") if v.strip()]
+            if typ == "number":
+                try:
+                    parsed = [float(v) for v in items]
+                except ValueError:
+                    raise ValueError(f"isin numérico: valores inválidos → {items}")
+                code_expr = f"df['{col}'].isin({parsed})"
+            else:
+                parsed    = [v.strip("'\"") for v in items]
+                code_expr = f"df['{col}'].isin({parsed!r})"
+            mask = s.isin(parsed)
+
+        else:
+            method = OP_METHOD.get(op, "eq")
+
+            if typ == "number":
+                try:
+                    parsed = float(val)
+                except ValueError:
+                    raise ValueError(f"Valor numérico inválido: '{val}'")
+                mask      = getattr(s, method)(parsed)
+                code_expr = f"df['{col}'].{method}({parsed})"
+
+            elif typ == "string":
+                parsed    = val.strip("'\"")
+                mask      = getattr(s, method)(parsed)
+                code_expr = f"df['{col}'].{method}('{parsed}')"
+
+            elif typ == "datetime":
+                try:
+                    parsed = pd.Timestamp(val)
+                except Exception:
+                    raise ValueError(f"Fecha inválida: '{val}'")
+                mask      = getattr(s, method)(parsed)
+                code_expr = f"df['{col}'].{method}(pd.Timestamp('{val}'))"
+
+            elif typ == "column":
+                if val not in df.columns:
+                    raise ValueError(f"Columna de comparación '{val}' no existe.")
+                mask      = getattr(s, method)(df[val])
+                code_expr = f"df['{col}'].{method}(df['{val}'])"
+
+            else:
+                raise ValueError(f"Tipo desconocido: '{typ}'")
+
+        if negate:
+            mask      = ~mask
+            code_expr = f"~({code_expr})"
+
+        masks.append({"logical": logical, "mask": mask, "expr": code_expr})
+
+    # ── Combine masks ────────────────────────────────────────────────────────
+    result_mask = masks[0]["mask"]
+    cond_lines  = [f"    ({masks[0]['expr']})"]
+
+    for m in masks[1:]:
+        sym = OP_SYM.get(m["logical"], "&")
+        if m["logical"] == "and":
+            result_mask = result_mask & m["mask"]
+        elif m["logical"] == "or":
+            result_mask = result_mask | m["mask"]
+        elif m["logical"] == "xor":
+            result_mask = result_mask ^ m["mask"]
+        cond_lines.append(f"    {sym} ({m['expr']})")
+
+    df = df[result_mask]
+
+    code = ["cond = (", *cond_lines, ")", "df = df[cond]"]
+    return df, code