data_drain 0.5.1 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +2 -2
- data/data_drain.gemspec +1 -1
- data/lib/data_drain/record.rb +4 -1
- data/lib/data_drain/version.rb +1 -1
- data/skill/SKILL.md +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 414600ce1230908cb1eef7e092ebf9287774ddbe4985286d8aa83995d0e47d4b
|
|
4
|
+
data.tar.gz: d300b31686ccf09320abc070a018566510e3e9a2d8488cb2bc83209dc56a3b21
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f4a7177e6d412995216397de87e9e93806815c395dea206fed75b541f3dafb208a34ea2c1cb35d6226f0dd4d4bf118c9ba13b0e8dc522d73be16ab59487fc7c9
|
|
7
|
+
data.tar.gz: 40dd834ad6af6d0c291b35a4ddbd259ef635400b31f92852d50e9565ff9f14ce880c3d0b4ad04f8a1e924888e92327338356776d065bd07dabec5d450ea0d440
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.5.2] - 2026-04-16
|
|
4
|
+
|
|
5
|
+
### Correcciones
|
|
6
|
+
|
|
7
|
+
- `Record#where()` ahora usa wildcards (`key=*`) para partition keys no especificadas, en lugar de valores vacíos (`key=`). Consistente con `destroy_partitions`. Fixes #1.
|
|
8
|
+
|
|
3
9
|
## [0.5.1] - 2026-04-15
|
|
4
10
|
|
|
5
11
|
### Docs
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# DataDrain
|
|
2
2
|
|
|
3
|
-
[](https://github.com/sequre/data_drain/actions/workflows/main.yml)
|
|
4
4
|
|
|
5
5
|
Micro-framework Ruby para extraer, archivar y purgar datos históricos de PostgreSQL hacia un Data Lake (S3 o disco local) en formato Parquet, usando DuckDB en memoria.
|
|
6
6
|
|
|
@@ -18,7 +18,7 @@ Micro-framework Ruby para extraer, archivar y purgar datos históricos de Postgr
|
|
|
18
18
|
|
|
19
19
|
```ruby
|
|
20
20
|
# Gemfile
|
|
21
|
-
gem 'data_drain', git: 'https://github.com/
|
|
21
|
+
gem 'data_drain', git: 'https://github.com/sequre/data_drain.git', branch: 'main'
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
```bash
|
data/data_drain.gemspec
CHANGED
|
@@ -11,7 +11,7 @@ Gem::Specification.new do |spec|
|
|
|
11
11
|
spec.summary = "Micro-framework para drenar datos de PostgreSQL a Parquet vía DuckDB."
|
|
12
12
|
spec.description = "Extrae datos transaccionales, los archiva en un Data Lake (S3/Local) " \
|
|
13
13
|
"en formato Parquet usando Hive Partitioning, y purga el origen de forma segura."
|
|
14
|
-
spec.homepage = "https://github.com/
|
|
14
|
+
spec.homepage = "https://github.com/sequre/data_drain"
|
|
15
15
|
spec.required_ruby_version = ">= 3.2"
|
|
16
16
|
|
|
17
17
|
spec.files = Dir.chdir(__dir__) do
|
data/lib/data_drain/record.rb
CHANGED
|
@@ -131,7 +131,10 @@ module DataDrain
|
|
|
131
131
|
# @param partitions [Hash]
|
|
132
132
|
# @return [String]
|
|
133
133
|
def build_query_path(partitions)
|
|
134
|
-
partition_path = partition_keys.map
|
|
134
|
+
partition_path = partition_keys.map do |k|
|
|
135
|
+
val = partitions.key?(k.to_sym) ? partitions[k.to_sym] : partitions[k.to_s]
|
|
136
|
+
val.nil? || val.to_s.empty? ? "#{k}=*" : "#{k}=#{val}"
|
|
137
|
+
end.join("/")
|
|
135
138
|
DataDrain::Storage.adapter.build_path(bucket, folder_name, partition_path)
|
|
136
139
|
end
|
|
137
140
|
|
data/lib/data_drain/version.rb
CHANGED
data/skill/SKILL.md
CHANGED
|
@@ -271,7 +271,7 @@ Catálogo completo en [Antipatrones](references/antipatrones.md). Resumen de los
|
|
|
271
271
|
## Referencias
|
|
272
272
|
|
|
273
273
|
- [API Detallada](references/api-detallada.md) — Firmas completas, parámetros, retornos y comportamientos de cada clase pública.
|
|
274
|
-
- [Glue Jobs Lifecycle](https://github.com/
|
|
274
|
+
- [Glue Jobs Lifecycle](https://github.com/sequre/data_drain/blob/main/docs/glue-jobs-lifecycle.md) — Guía completa de gestión de AWS Glue Jobs: crear, actualizar, eliminar, verificar y ejecutar jobs idempotentemente.
|
|
275
275
|
- [Eventos y Telemetría](references/eventos-telemetria.md) — Catálogo completo de eventos KV emitidos por la gema.
|
|
276
276
|
- [Antipatrones](references/antipatrones.md) — Qué NO hacer y alternativas correctas.
|
|
277
277
|
- [Postgres Tuning](references/postgres-tuning.md) — Índices, VACUUM, particionamiento y diagnóstico por tamaño de tabla.
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: data_drain
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.5.
|
|
4
|
+
version: 0.5.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Gabriel
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-04-
|
|
11
|
+
date: 2026-04-16 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activemodel
|
|
@@ -133,7 +133,7 @@ files:
|
|
|
133
133
|
- skill/references/api-detallada.md
|
|
134
134
|
- skill/references/eventos-telemetria.md
|
|
135
135
|
- skill/references/postgres-tuning.md
|
|
136
|
-
homepage: https://github.com/
|
|
136
|
+
homepage: https://github.com/sequre/data_drain
|
|
137
137
|
licenses: []
|
|
138
138
|
metadata: {}
|
|
139
139
|
post_install_message:
|