data_drain 0.4.0 → 0.5.0

data/docs/glue-jobs-lifecycle.md

@@ -32,16 +32,102 @@ job.default_arguments  # => { "--extra-files" => "s3://..." }
 - Lanza `DataDrain::ConfigurationError` si `job_name` es inválido.
 - Lanza `Aws::Glue::Errors::EntityNotFoundException` si el job no existe.
 
-### `create_job(job_name, role_arn:, script_location:, ...)` → Aws::Glue::Types::Job
+### Subir scripts locales (v0.5.0+)
+
+Desde v0.5.0 la gema puede subir scripts PySpark a S3 automáticamente.
+
+```ruby
+# Opción moderna: script local subido por la gema
+DataDrain::GlueRunner.create_job(
+  "my-job",
+  script_path: "scripts/glue/export.py",  # local
+  script_bucket: "my-bucket",
+  script_folder: "scripts",
+  role_arn: "arn:aws:iam::123:role/GlueRole"
+)
+# → Sube scripts/glue/export.py a s3://my-bucket/scripts/export.py
+# → Crea el job
+```
+
+**Parámetros para upload:**
+- `script_path` (String): ruta local al script Python.
+- `script_bucket` (String): bucket S3 destino. **Requerido si se usa `script_path`.**
+- `script_folder` (String): folder dentro del bucket. Default: `"scripts"`.
+- `script_filename` (String, nil): override del nombre en S3. Default: basename del archivo.
+
+**`script_location` vs `script_path`:**
+- `script_location:` → comportamiento anterior, no hay upload.
+- `script_path:` + `script_bucket:` → la gema sube a S3 primero, luego crea el Job.
+- Si se pasan ambos → `DataDrain::ConfigurationError`.
+- Si no se pasa ninguno → `ArgumentError`.
+
+**Importante:** el upload **sobrescribe** cualquier archivo existente en el mismo path.
+No es idempotente en sentido estricto. Usar `script_filename:` con hash o timestamp
+si necesitás versionado.
+
+### Concurrencia (limitación conocida)
+
+No hay lock distribuido. Si dos procesos llaman `upload_script` con el mismo destino
+simultáneamente, el último `put_object` en llegar a S3 gana. Para scripts PySpark
+esto es típicamente bajo riesgo (scripts son pequeños, rara vez hay writes
+concurrentes al mismo path).
+
+### Permisos IAM mínimos
+
+El IAM role/user que ejecuta `upload_script` necesita:
+
+```json
+{
+  "Effect": "Allow",
+  "Action": ["s3:PutObject"],
+  "Resource": "arn:aws:s3:::my-bucket/scripts/*"
+}
+```
+
+Para usar con `create_job`/`ensure_job` también se necesitan los permisos de Glue
+(ver sección "Permisos Glue" al inicio de este documento) + permiso para que el
+IAM role del Glue Job pueda leer el script:
+
+```json
+{
+  "Effect": "Allow",
+  "Action": ["s3:GetObject"],
+  "Resource": "arn:aws:s3:::my-bucket/scripts/*"
+}
+```
+
+(Este último en el role del Glue Job, no en el role de la aplicación Ruby.)
+
+### API standalone: `upload_script`
+
+Para casos donde solo querés subir (sin crear Job):
+
+```ruby
+s3_path = DataDrain::GlueRunner.upload_script(
+  local_path: "scripts/glue/export.py",
+  bucket: "my-bucket",
+  folder: "scripts"
+)
+# => "s3://my-bucket/scripts/export.py"
+```
+
+Requiere `storage_mode = :s3`.
+
+### `create_job(job_name, role_arn:, ...)` → Aws::Glue::Types::Job
 
 Crea un nuevo job en Glue y retorna el job creado.
 
 **Parámetros requeridos:**
 - `job_name` (String): nombre del job
 - `role_arn` (String): ARN del IAM role de Glue
-- `script_location` (String): path S3 del script Python
+
+**Parámetros de script (mutuamente excluyentes):**
+- `script_location` (String): path S3 del script Python (comportamiento anterior)
+- `script_path` + `script_bucket` (String): upload local a S3 primero (v0.5.0+)
 
 **Parámetros opcionales:**
+- `script_folder` (String): folder S3. Default: `"scripts"`.
+- `script_filename` (String, nil): override del nombre en S3.
 - `command_name` (String): nombre del comando (`"glueetl"`, `"pythonshell"`). Default: `"glueetl"`.
 - `default_arguments` (Hash): argumentos default del job
 - `description` (String): descripción del job
@@ -85,24 +171,28 @@ job = DataDrain::GlueRunner.update_job(
 - Lanza `DataDrain::ConfigurationError` si `job_name` es inválido.
 - Lanza `Aws::Glue::Errors::EntityNotFoundException` si el job no existe.
 
-### `delete_job(job_name)` → nil
+### `delete_job(job_name)` → Boolean
 
-Elimina un job de Glue.
+Elimina un job de Glue. Es idempotente.
 
 ```ruby
 DataDrain::GlueRunner.delete_job("my-job")
-# => nil
+# => true (job existía y fue eliminado)
+
+DataDrain::GlueRunner.delete_job("nonexistent")
+# => false (job no existía)
 ```
 
 - Lanza `DataDrain::ConfigurationError` si `job_name` es inválido.
-- Lanza `Aws::Glue::Errors::EntityNotFoundException` si el job no existe.
+- Lanza otros errores de AWS sin atrapar.
 
-### `ensure_job(job_name, role_arn:, script_location:, ...)` → Aws::Glue::Types::Job
+### `ensure_job(job_name, role_arn:, ...)` → Aws::Glue::Types::Job
 
-Crea o actualiza un job de forma idempotente.
+Crea o actualiza un job de forma idempotente con diffing de configuración.
 
-- Si el job existe → `update_job`
 - Si el job no existe → `create_job`
+- Si el job existe con config diferente → `update_job`
+- Si el job existe con config idéntica → no-op, retorna el job actual (`:unchanged`)
 
 ```ruby
 job = DataDrain::GlueRunner.ensure_job(
@@ -113,6 +203,19 @@ job = DataDrain::GlueRunner.ensure_job(
 )
 ```
 
+También soporta upload de script local (v0.5.0+):
+
+```ruby
+job = DataDrain::GlueRunner.ensure_job(
+  "my-job",
+  script_path: "scripts/glue/export.py",
+  script_bucket: "my-bucket",
+  script_folder: "scripts",
+  role_arn: "arn:aws:iam::123:role/GlueServiceRole",
+  timeout: 1440
+)
+```
+
 - Lanza `DataDrain::ConfigurationError` si `job_name` es inválido.
 - Lanza errores de AWS sin atrapar.
 
@@ -133,17 +236,75 @@ DataDrain::GlueRunner.run_and_wait(
 - Lanza `RuntimeError` si el job falla (`FAILED`, `STOPPED`, `TIMEOUT`).
 - Lanza `DataDrain::Error` si `max_wait_seconds` excede.
 
+## Patrón completo: ensure_job + run_and_wait + PySpark
+
+Workflow end-to-end para archivar y purgar tablas PostgreSQL usando AWS Glue:
+
+```ruby
+# 1. Asegurar que el Glue Job existe con la config deseada (idempotente)
+DataDrain::GlueRunner.ensure_job(
+  "my-export-job",
+  role_arn: "arn:aws:iam::123:role/GlueServiceRole",
+  script_location: "s3://my-bucket/scripts/glue_pyspark_export.py",
+  glue_version: "4.0",
+  worker_type: "G.1X",
+  number_of_workers: 10,
+  timeout: 1440
+)
+
+# 2. Ejecutar el export (delegado a Glue Spark distribuido)
+DataDrain::GlueRunner.run_and_wait(
+  "my-export-job",
+  {
+    "--start_date"   => start_date.to_fs(:db),
+    "--end_date"     => end_date.to_fs(:db),
+    "--s3_bucket"    => bucket,
+    "--s3_folder"    => table,
+    "--db_url"       => "jdbc:postgresql://#{host}:#{port}/#{db}",
+    "--db_user"      => db_user,
+    "--db_password"  => db_password,
+    "--db_table"     => table,
+    "--partition_by" => partition_keys.join(",")
+  },
+  polling_interval: 60,
+  max_wait_seconds: 7200
+)
+
+# 3. Verificar integridad y purgar Postgres (DataDrain solo lee Parquet)
+DataDrain::Engine.new(
+  bucket: bucket,
+  folder_name: table,
+  start_date: start_date,
+  end_date: end_date,
+  table_name: table,
+  partition_keys: partition_keys,
+  skip_export: true  # export ya lo hizo Glue
+).call
+```
+
+### Prerequisites
+
+1. **Subir el script a S3:**
+   ```bash
+   aws s3 cp glue_pyspark_export.py s3://my-bucket/scripts/
+   ```
+
+2. **IAM Role** con permisos para: Glue, S3 (lectura del script + escritura del bucket destino), RDS/Postgres (vía JDBC)
+
+3. **Script PySpark** en `s3://my-bucket/scripts/glue_pyspark_export.py` (ver [ejemplo](../glue_pyspark_example.py))
+
 ## Convenciones de nombres
 
-AWS Glue permite: letras (`a-zA-Z`), números (`0-9`), guiones (`-`). No permite guiones bajos ni espacios.
+AWS Glue permite: letras (`a-zA-Z`), números (`0-9`), guiones (`-`), guiones bajos (`_`). No permite espacios ni caracteres especiales.
 
 ```ruby
 # Válido
 DataDrain::GlueRunner.job_exists?("my-export-job-v2")
+DataDrain::GlueRunner.job_exists?("my_export_job")
 
 # Inválido — lanza ConfigurationError
-DataDrain::GlueRunner.job_exists?("my_export_job")
-# DataDrain::ConfigurationError: job_name 'my_export_job' no es un nombre válido para Glue Job
+DataDrain::GlueRunner.job_exists?("-starts-with-dash")
+# DataDrain::ConfigurationError: job_name '-starts-with-dash' no es un nombre válido para Glue Job
 ```
 
 ## Eventos de telemetría
@@ -151,8 +312,18 @@ DataDrain::GlueRunner.job_exists?("my_export_job")
 | Evento | Nivel | Descripción |
 |--------|-------|-------------|
 | `glue_runner.start` | INFO | Antes de `start_job_run` |
-| `glue_runner.job_exists` | INFO | Job encontrado en `ensure_job` |
+| `glue_runner.job_create` | INFO | Job creado exitosamente |
+| `glue_runner.job_update` | INFO | Job actualizado (incluye `changed_fields`) |
+| `glue_runner.job_delete` | INFO | Job eliminado exitosamente |
+| `glue_runner.job_delete_skipped` | INFO | `delete_job` sobre job inexistente |
+| `glue_runner.job_exists` | INFO | Job encontrado en `ensure_job` (y difiere) |
 | `glue_runner.job_created` | INFO | Job creado en `ensure_job` |
+| `glue_runner.job_unchanged` | INFO | Job existe con config idéntica en `ensure_job` |
+| `glue_runner.job_create_error` | ERROR | Error en `create_job` |
+| `glue_runner.job_update_error` | ERROR | Error en `update_job` |
+| `glue_runner.job_delete_error` | ERROR | Error en `delete_job` |
+| `glue_runner.script_uploaded` | INFO | Script subido a S3 (v0.5.0+) |
+| `glue_runner.script_upload_error` | ERROR | Error al subir script a S3 (v0.5.0+) |
 | `glue_runner.polling` | INFO | Chequeo de estado durante `run_and_wait` |
 | `glue_runner.complete` | INFO | Job terminó `SUCCEEDED` |
 | `glue_runner.failed` | ERROR | Job falló con `FAILED\|STOPPED\|TIMEOUT` |

data/docs/glue_pyspark_example.py

@@ -1,11 +1,29 @@
 """
 Script de AWS Glue (PySpark) compatible con DataDrain::GlueRunner.
 
-Crear el Job en la consola de AWS Glue (Spark 4.0+) y usar este script como base.
-Argumentos requeridos: JOB_NAME, start_date, end_date, s3_bucket, s3_folder,
+Para crear el Glue Job programmatically (en vez de la consola):
+
+    # Opcion moderna: script local subido por la gema (v0.5.0+)
+    DataDrain::GlueRunner.ensure_job(
+      "my-export-job",
+      script_path: "docs/glue_pyspark_example.py",
+      script_bucket: "my-bucket",
+      script_folder: "scripts",
+      role_arn: "arn:aws:iam::123:role/GlueServiceRole",
+      worker_type: "G.1X",
+      number_of_workers: 10,
+      timeout: 1440
+    )
+    # -> Sube este archivo a s3://my-bucket/scripts/glue_pyspark_example.py
+    # -> Crea el Job apuntando a ese path
+
+    # Ejecutar
+    DataDrain::GlueRunner.run_and_wait("my-export-job", { "--start_date" => "2025-01-01", ... })
+
+Argumentos requeridos del job: JOB_NAME, start_date, end_date, s3_bucket, s3_folder,
 db_url, db_user, db_password, db_table, partition_by.
 
-Personalizar la sección de columnas derivadas según las partition_keys de cada tabla.
+Personalizar la seccion de columnas derivadas segun las partition_keys de cada tabla.
 """
 
 import sys
@@ -15,27 +33,38 @@ from awsglue.context import GlueContext
 from awsglue.job import Job
 from pyspark.sql.functions import col, year, month
 
-args = getResolvedOptions(sys.argv, [
-    'JOB_NAME', 'start_date', 'end_date', 's3_bucket', 's3_folder',
-    'db_url', 'db_user', 'db_password', 'db_table', 'partition_by'
-])
+args = getResolvedOptions(
+    sys.argv,
+    [
+        "JOB_NAME",
+        "start_date",
+        "end_date",
+        "s3_bucket",
+        "s3_folder",
+        "db_url",
+        "db_user",
+        "db_password",
+        "db_table",
+        "partition_by",
+    ],
+)
 
 sc = SparkContext()
 glueContext = GlueContext(sc)
 spark = glueContext.spark_session
 job = Job(glueContext)
-job.init(args['JOB_NAME'], args)
+job.init(args["JOB_NAME"], args)
 
 options = {
-    "url": args['db_url'],
-    "dbtable": args['db_table'],
-    "user": args['db_user'],
-    "password": args['db_password'],
+    "url": args["db_url"],
+    "dbtable": args["db_table"],
+    "user": args["db_user"],
+    "password": args["db_password"],
     "sampleQuery": (
         f"SELECT * FROM {args['db_table']} "
         f"WHERE created_at >= '{args['start_date']}' "
         f"AND created_at < '{args['end_date']}'"
-    )
+    ),
 }
 
 df = spark.read.format("jdbc").options(**options).load()
@@ -43,18 +72,19 @@ df = spark.read.format("jdbc").options(**options).load()
 # Agregar columnas derivadas necesarias para las particiones.
 # isp_id ya existe en la tabla fuente — solo agregar las que se calculan.
 # Personalizar esta seccion segun las partition_keys de cada tabla.
-df_final = (
-    df.withColumn("year", year(col("created_at")))
-      .withColumn("month", month(col("created_at")))
+df_final = df.withColumn("year", year(col("created_at"))).withColumn(
+    "month", month(col("created_at"))
 )
 
 output_path = f"s3://{args['s3_bucket']}/{args['s3_folder']}/"
-partitions = args['partition_by'].split(",")
+partitions = args["partition_by"].split(",")
 
-(df_final.write.mode("overwrite")
+(
+    df_final.write.mode("overwrite")
     .partitionBy(*partitions)
     .format("parquet")
     .option("compression", "zstd")
-    .save(output_path))
+    .save(output_path)
+)
 
 job.commit()

data/lib/data_drain/glue_runner.rb

@@ -40,17 +40,29 @@ module DataDrain
       client.get_job(job_name: job_name).job
     end
 
-    def self.create_job(job_name, role_arn:, script_location:, command_name: "glueetl",
-                        default_arguments: {}, description: nil, worker_type: nil, number_of_workers: nil,
-                        timeout: 2880, max_retries: 0, allocated_capacity: nil, glue_version: nil)
+    def self.create_job(job_name, role_arn:, script_location: nil, script_path: nil,
+                        script_bucket: nil, script_folder: "scripts", script_filename: nil,
+                        command_name: "glueetl", default_arguments: {}, description: nil,
+                        worker_type: nil, number_of_workers: nil, timeout: 2880,
+                        max_retries: 0, allocated_capacity: nil, glue_version: nil)
+      @logger = DataDrain.configuration.logger
       DataDrain::Validations.validate_glue_name!(:job_name, job_name)
+
+      final_script_location = resolve_script_location(
+        script_location: script_location,
+        script_path: script_path,
+        script_bucket: script_bucket,
+        script_folder: script_folder,
+        script_filename: script_filename
+      )
+
       opts = {
         name: job_name,
         role: role_arn,
         command: {
           name: command_name,
           python_version: "3",
-          script_location: script_location
+          script_location: final_script_location
         }
       }
       opts[:default_arguments] = default_arguments unless default_arguments.empty?
@@ -63,13 +75,24 @@ module DataDrain
       opts[:glue_version] = glue_version if glue_version
 
       client.create_job(**opts)
+      safe_log(:info, "glue_runner.job_create", {
+                 job: job_name,
+                 glue_version: glue_version,
+                 worker_type: worker_type,
+                 number_of_workers: number_of_workers
+               })
       get_job(job_name)
+    rescue Aws::Glue::Errors::ServiceError => e
+      safe_log(:error, "glue_runner.job_create_error",
+               { job: job_name }.merge(exception_metadata(e)))
+      raise
     end
 
     def self.update_job(job_name, role_arn: nil, command_name: nil, script_location: nil,
                         default_arguments: nil, description: nil, worker_type: nil,
                         number_of_workers: nil, timeout: nil, max_retries: nil, allocated_capacity: nil,
                         glue_version: nil)
+      @logger = DataDrain.configuration.logger
       DataDrain::Validations.validate_glue_name!(:job_name, job_name)
       job_update = {}
       job_update[:role] = role_arn if role_arn
@@ -87,30 +110,77 @@ module DataDrain
       job_update[:glue_version] = glue_version if glue_version
 
       client.update_job(job_name: job_name, job_update: job_update)
+      safe_log(:info, "glue_runner.job_update", {
+                 job: job_name,
+                 changed_fields: job_update.keys.map(&:to_s)
+               })
       get_job(job_name)
+    rescue Aws::Glue::Errors::ServiceError => e
+      safe_log(:error, "glue_runner.job_update_error",
+               { job: job_name }.merge(exception_metadata(e)))
+      raise
     end
 
     def self.delete_job(job_name)
+      @logger = DataDrain.configuration.logger
       DataDrain::Validations.validate_glue_name!(:job_name, job_name)
       client.delete_job(job_name: job_name)
-      nil
+      safe_log(:info, "glue_runner.job_delete", { job: job_name })
+      true
+    rescue Aws::Glue::Errors::EntityNotFoundException
+      safe_log(:info, "glue_runner.job_delete_skipped", { job: job_name, reason: "not_found" })
+      false
+    rescue Aws::Glue::Errors::ServiceError => e
+      safe_log(:error, "glue_runner.job_delete_error",
+               { job: job_name }.merge(exception_metadata(e)))
+      raise
     end
 
-    def self.ensure_job(job_name, role_arn:, script_location:, command_name: "glueetl",
-                        default_arguments: {}, description: nil, worker_type: nil,
-                        number_of_workers: nil, timeout: 2880, max_retries: 0,
-                        allocated_capacity: nil, glue_version: nil)
+    def self.ensure_job(job_name, role_arn:, script_location: nil, script_path: nil,
+                        script_bucket: nil, script_folder: "scripts", script_filename: nil,
+                        command_name: "glueetl", default_arguments: {}, description: nil,
+                        worker_type: nil, number_of_workers: nil, timeout: 2880,
+                        max_retries: 0, allocated_capacity: nil, glue_version: nil)
+      @logger = DataDrain.configuration.logger
+
+      final_script_location = resolve_script_location(
+        script_location: script_location,
+        script_path: script_path,
+        script_bucket: script_bucket,
+        script_folder: script_folder,
+        script_filename: script_filename
+      )
+
       if job_exists?(job_name)
-        safe_log(:info, "glue_runner.job_exists", { job: job_name })
-        update_job(job_name, role_arn: role_arn, command_name: command_name,
-                             script_location: script_location, default_arguments: default_arguments,
-                             description: description, worker_type: worker_type,
-                             number_of_workers: number_of_workers, timeout: timeout,
-                             max_retries: max_retries, allocated_capacity: allocated_capacity,
-                             glue_version: glue_version)
+        current = get_job(job_name)
+        desired = {
+          role: role_arn,
+          command_name: command_name,
+          script_location: final_script_location,
+          default_arguments: default_arguments,
+          description: description,
+          worker_type: worker_type,
+          number_of_workers: number_of_workers,
+          timeout: timeout,
+          max_retries: max_retries,
+          glue_version: glue_version
+        }
+        changed = changed_fields(desired, current)
+        if changed.empty?
+          safe_log(:info, "glue_runner.job_unchanged", { job: job_name })
+          current
+        else
+          safe_log(:info, "glue_runner.job_exists", { job: job_name })
+          update_job(job_name, role_arn: role_arn, command_name: command_name,
+                               script_location: final_script_location, default_arguments: default_arguments,
+                               description: description, worker_type: worker_type,
+                               number_of_workers: number_of_workers, timeout: timeout,
+                               max_retries: max_retries, allocated_capacity: allocated_capacity,
+                               glue_version: glue_version)
+        end
       else
         safe_log(:info, "glue_runner.job_created", { job: job_name })
-        create_job(job_name, role_arn: role_arn, script_location: script_location,
+        create_job(job_name, role_arn: role_arn, script_location: final_script_location,
                              command_name: command_name, default_arguments: default_arguments,
                              description: description, worker_type: worker_type,
                              number_of_workers: number_of_workers, timeout: timeout,
@@ -119,6 +189,72 @@ module DataDrain
       end
     end
 
+    def self.changed_fields(desired, current)
+      changed = []
+      changed << :role if current.role != desired[:role]
+      changed << :command if current.command.name != desired[:command_name] ||
+                             current.command.script_location != desired[:script_location]
+      changed << :default_arguments if current.default_arguments != desired[:default_arguments]
+      changed << :description if current.description != desired[:description]
+      changed << :worker_type if current.worker_type != desired[:worker_type]
+      changed << :number_of_workers if current.number_of_workers != desired[:number_of_workers]
+      changed << :timeout if current.timeout != desired[:timeout]
+      changed << :max_retries if current.max_retries != desired[:max_retries]
+      changed << :glue_version if current.glue_version != desired[:glue_version]
+      changed
+    end
+    private_class_method :changed_fields
+
+    def self.resolve_script_location(script_location:, script_path:, script_bucket:, script_folder:, script_filename:)
+      both_set = script_location && script_path
+      raise DataDrain::ConfigurationError, "provee script_location o script_path, no ambos" if both_set
+
+      return script_location if script_location
+      raise ArgumentError, "script_location o script_path es requerido" unless script_path
+      raise DataDrain::ConfigurationError, "script_path requiere script_bucket" unless script_bucket
+
+      upload_script(
+        local_path: script_path,
+        bucket: script_bucket,
+        folder: script_folder,
+        filename: script_filename
+      )
+    end
+    private_class_method :resolve_script_location
+
+    def self.upload_script(local_path:, bucket:, folder: "scripts", filename: nil)
+      @logger = DataDrain.configuration.logger
+
+      unless File.exist?(local_path)
+        raise DataDrain::ConfigurationError,
+              "Script local '#{local_path}' no existe"
+      end
+
+      actual_filename = filename || File.basename(local_path)
+      s3_key = "#{folder.chomp("/")}/#{actual_filename}"
+      bytes = File.size(local_path)
+
+      adapter = DataDrain::Storage.adapter
+      unless adapter.is_a?(DataDrain::Storage::S3)
+        raise DataDrain::ConfigurationError,
+              "upload_script requiere storage_mode = :s3, actual: #{DataDrain.configuration.storage_mode}"
+      end
+
+      s3_path = adapter.upload_file(local_path, bucket, s3_key, content_type: "text/x-python")
+
+      safe_log(:info, "glue_runner.script_uploaded", {
+                 local_path: local_path,
+                 s3_path: s3_path,
+                 bytes: bytes
+               })
+
+      s3_path
+    rescue Aws::S3::Errors::ServiceError => e
+      safe_log(:error, "glue_runner.script_upload_error",
+               { local_path: local_path, bucket: bucket }.merge(exception_metadata(e)))
+      raise
+    end
+
     def self.run_and_wait(job_name, arguments = {}, polling_interval: 30, max_wait_seconds: nil)
       config = DataDrain.configuration
       config.validate!

data/lib/data_drain/storage/base.rb

@@ -55,6 +55,18 @@ module DataDrain
         raise NotImplementedError, "#{self.class} debe implementar #destroy_partitions"
       end
 
+      # Sube un archivo local al storage.
+      #
+      # @param local_path [String]
+      # @param bucket [String]
+      # @param s3_key [String] key relativo (ej. "scripts/export.py")
+      # @param content_type [String, nil]
+      # @return [String] URI completo del archivo subido
+      # @raise [NotImplementedError]
+      def upload_file(local_path, bucket, s3_key, content_type: nil)
+        raise NotImplementedError, "#{self.class} debe implementar #upload_file"
+      end
+
       protected
 
       # @param bucket [String]

data/lib/data_drain/storage/local.rb

@@ -27,6 +27,19 @@ module DataDrain
         "#{build_path_base(bucket, folder_name, partition_path)}/**/*.parquet"
       end
 
+      # @param local_path [String]
+      # @param bucket [String] Directorio destino
+      # @param s3_key [String] Path relativo dentro del bucket
+      # @param content_type [String, nil] Ignorado en modo local
+      # @return [String] Path absoluto al archivo destino
+      def upload_file(local_path, bucket, s3_key, content_type: nil)
+        _ = content_type
+        dest_path = File.join(bucket, s3_key)
+        FileUtils.mkdir_p(File.dirname(dest_path))
+        FileUtils.cp(local_path, dest_path)
+        dest_path
+      end
+
       # @param bucket [String]
       # @param folder_name [String]
       # @param partition_keys [Array<Symbol>]

data/lib/data_drain/storage/s3.rb

@@ -38,6 +38,23 @@ module DataDrain
         delete_in_batches(client, bucket, objects)
       end
 
+      # @param local_path [String]
+      # @param bucket [String]
+      # @param s3_key [String]
+      # @param content_type [String, nil]
+      # @return [String] "s3://bucket/key"
+      def upload_file(local_path, bucket, s3_key, content_type: nil)
+        client = s3_client
+
+        File.open(local_path, "rb") do |file|
+          params = { bucket: bucket, key: s3_key, body: file }
+          params[:content_type] = content_type if content_type
+          client.put_object(**params)
+        end
+
+        "s3://#{bucket}/#{s3_key}"
+      end
+
       private
 
       # @return [Aws::S3::Client]

data/lib/data_drain/validations.rb

@@ -6,7 +6,7 @@ module DataDrain
     # Regex que valida identificadores SQL (tablas, columnas, etc.).
     # Permite letras, guiones bajos y números (no al inicio).
     IDENTIFIER_REGEX = /\A[a-zA-Z_][a-zA-Z0-9_]*\z/
-    GLUE_NAME_REGEX = /\A[a-zA-Z0-9][a-zA-Z0-9-]*\z/
+    GLUE_NAME_REGEX = /\A(?![_-])[a-zA-Z0-9_-]+\z/
 
     module_function
 
@@ -14,7 +14,7 @@ module DataDrain
       return if GLUE_NAME_REGEX.match?(value.to_s)
 
       raise DataDrain::ConfigurationError,
-            "#{name} '#{value}' no es un nombre válido para Glue Job (usa solo letras, números y guiones)"
+            "#{name} '#{value}' no es un nombre válido para Glue Job (usa solo letras, números, '-' y '_')"
     end
 
     def validate_identifier!(name, value)

data/lib/data_drain/version.rb

@@ -2,5 +2,5 @@
 
 module DataDrain
   # @return [String] versión semver de la gema
-  VERSION = "0.4.0"
+  VERSION = "0.5.0"
 end