sibi-dst 0.3.42__py3-none-any.whl → 0.3.44__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. sibi_dst/df_helper/_artifact_updater_multi_wrapper.py +7 -2
  2. sibi_dst/df_helper/_df_helper.py +5 -2
  3. sibi_dst/df_helper/_parquet_artifact.py +33 -3
  4. sibi_dst/df_helper/_parquet_reader.py +5 -1
  5. sibi_dst/df_helper/backends/django/_load_from_db.py +1 -0
  6. sibi_dst/df_helper/backends/parquet/_filter_handler.py +2 -1
  7. sibi_dst/df_helper/backends/parquet/_parquet_options.py +2 -3
  8. sibi_dst/df_helper/backends/sqlalchemy/_db_connection.py +1 -0
  9. sibi_dst/df_helper/backends/sqlalchemy/_io_dask.py +2 -5
  10. sibi_dst/df_helper/core/_filter_handler.py +2 -1
  11. sibi_dst/osmnx_helper/__init__.py +2 -2
  12. sibi_dst/osmnx_helper/v1/basemaps/__init__.py +0 -0
  13. sibi_dst/osmnx_helper/{basemaps → v1/basemaps}/router_plotter.py +85 -30
  14. sibi_dst/osmnx_helper/v2/__init__.py +0 -0
  15. sibi_dst/osmnx_helper/v2/base_osm_map.py +153 -0
  16. sibi_dst/osmnx_helper/v2/basemaps/__init__.py +0 -0
  17. sibi_dst/osmnx_helper/v2/basemaps/utils.py +0 -0
  18. sibi_dst/utils/data_wrapper.py +4 -368
  19. sibi_dst/utils/df_utils.py +7 -0
  20. sibi_dst/utils/log_utils.py +6 -0
  21. sibi_dst/utils/parquet_saver.py +4 -2
  22. sibi_dst/utils/storage_manager.py +14 -7
  23. sibi_dst-0.3.44.dist-info/METADATA +194 -0
  24. {sibi_dst-0.3.42.dist-info → sibi_dst-0.3.44.dist-info}/RECORD +29 -24
  25. sibi_dst-0.3.42.dist-info/METADATA +0 -62
  26. /sibi_dst/osmnx_helper/{basemaps → v1}/__init__.py +0 -0
  27. /sibi_dst/osmnx_helper/{base_osm_map.py → v1/base_osm_map.py} +0 -0
  28. /sibi_dst/osmnx_helper/{basemaps → v1/basemaps}/calendar_html.py +0 -0
  29. /sibi_dst/osmnx_helper/{utils.py → v1/utils.py} +0 -0
  30. {sibi_dst-0.3.42.dist-info → sibi_dst-0.3.44.dist-info}/WHEEL +0 -0
sibi_dst-0.3.44.dist-info/METADATA ADDED
@@ -0,0 +1,194 @@
1
+ Metadata-Version: 2.1
2
+ Name: sibi-dst
3
+ Version: 0.3.44
4
+ Summary: Data Science Toolkit
5
+ Author: Luis Valverde
6
+ Author-email: lvalverdeb@gmail.com
7
+ Requires-Python: >=3.11,<4.0
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Programming Language :: Python :: 3.11
10
+ Classifier: Programming Language :: Python :: 3.12
11
+ Classifier: Programming Language :: Python :: 3.13
12
+ Requires-Dist: apache-airflow-client (>=2.10.0,<3.0.0)
13
+ Requires-Dist: chardet (>=5.2.0,<6.0.0)
14
+ Requires-Dist: charset-normalizer (>=3.4.0,<4.0.0)
15
+ Requires-Dist: clickhouse-connect (>=0.8.7,<0.9.0)
16
+ Requires-Dist: clickhouse-driver (>=0.2.9,<0.3.0)
17
+ Requires-Dist: dask-expr (>=1.1.20,<2.0.0)
18
+ Requires-Dist: dask[complete] (>=2024.11.1,<2025.0.0)
19
+ Requires-Dist: django (>=5.1.4,<6.0.0)
20
+ Requires-Dist: djangorestframework (>=3.15.2,<4.0.0)
21
+ Requires-Dist: folium (>=0.19.4,<0.20.0)
22
+ Requires-Dist: geopandas (>=1.0.1,<2.0.0)
23
+ Requires-Dist: gunicorn (>=23.0.0,<24.0.0)
24
+ Requires-Dist: httpx (>=0.27.2,<0.28.0)
25
+ Requires-Dist: ipython (>=8.29.0,<9.0.0)
26
+ Requires-Dist: jinja2 (>=3.1.4,<4.0.0)
27
+ Requires-Dist: mysqlclient (>=2.2.6,<3.0.0)
28
+ Requires-Dist: nltk (>=3.9.1,<4.0.0)
29
+ Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
30
+ Requires-Dist: osmnx (>=2.0.1,<3.0.0)
31
+ Requires-Dist: pandas (>=2.2.3,<3.0.0)
32
+ Requires-Dist: paramiko (>=3.5.0,<4.0.0)
33
+ Requires-Dist: psutil (>=6.1.0,<7.0.0)
34
+ Requires-Dist: psycopg2 (>=2.9.10,<3.0.0)
35
+ Requires-Dist: pyarrow (>=18.0.0,<19.0.0)
36
+ Requires-Dist: pydantic (>=2.9.2,<3.0.0)
37
+ Requires-Dist: pymysql (>=1.1.1,<2.0.0)
38
+ Requires-Dist: pytest (>=8.3.3,<9.0.0)
39
+ Requires-Dist: pytest-mock (>=3.14.0,<4.0.0)
40
+ Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
41
+ Requires-Dist: s3fs (>=2024.12.0,<2025.0.0)
42
+ Requires-Dist: sqlalchemy (>=2.0.36,<3.0.0)
43
+ Requires-Dist: tornado (>=6.4.1,<7.0.0)
44
+ Requires-Dist: tqdm (>=4.67.0,<5.0.0)
45
+ Requires-Dist: uvicorn (>=0.34.0,<0.35.0)
46
+ Requires-Dist: uvicorn-worker (>=0.3.0,<0.4.0)
47
+ Description-Content-Type: text/markdown
48
+
49
+ # sibi-dst
50
+
51
+ Data Science Toolkit
52
+ ---------------------
53
+ Data Science Toolkit built with Python, Pandas, Dask, OpenStreetMaps, Scikit-Learn, XGBOOST, Django ORM, SQLAlchemy, DjangoRestFramework, FastAPI
54
+
55
+ Major Functionality
56
+ --------------------
57
+ 1) **Build DataCubes, DataSets, and DataObjects** from diverse data sources, including **relational databases, Parquet files, Excel (`.xlsx`), delimited tables (`.csv`, `.tsv`), JSON, and RESTful APIs (`JSON API REST`)**.
58
+ 2) **Comprehensive DataFrame Management** utilities for efficient data handling, transformation, and optimization using **Pandas** and **Dask**.
59
+ 3) **Flexible Data Sharing** with client applications by writing to **Data Warehouses, local filesystems, and cloud storage platforms** such as **Amazon S3, Google Cloud Storage (GCS), and Azure Blob Storage**.
60
+ 4) **Microservices for Data Access** – Build scalable **API-driven services** using **RESTful APIs (`Django REST Framework`, `FastAPI`) and gRPC** for high-performance data exchange.
61
+
62
+ Supported Technologies
63
+ --------------------
64
+ - **Data Processing**: Pandas, Dask
65
+ - **Machine Learning**: Scikit-Learn, XGBoost
66
+ - **Databases & Storage**: SQLAlchemy, Django ORM, Parquet, Amazon S3, GCS, Azure Blob Storage
67
+ - **Mapping & Geospatial Analysis**: OpenStreetMaps, OSMnx, Geopy
68
+ - **API Development**: Django REST Framework, gRPC
69
+
70
+ Installation
71
+ ---------------------
72
+ ```bash
73
+ pip install sibi-dst
74
+ ```
75
+
76
+ Usage
77
+ ---------------------
78
+ ### Loading Data from SQLAlchemy
79
+ ```python
80
+ from sibi_dst.df_helper import DfHelper
81
+ from conf.transforms.fields.crm import customer_fields
82
+ from conf.credentials import replica_db_conf
83
+ from conf.storage import get_fs_instance
84
+
85
+ config = {
86
+ 'backend': 'sqlalchemy',
87
+ 'connection_url': replica_db_conf.get('db_url'),
88
+ 'table': 'crm_clientes_archivo',
89
+ 'field_map': customer_fields,
90
+ 'legacy_filters': True,
91
+ 'fs': get_fs_instance()
92
+ }
93
+
94
+ df_helper = DfHelper(**config)
95
+ result = df_helper.load(id__gte=1)
96
+ ```
97
+
98
+ ### Saving Data to ClickHouse
99
+ ```python
100
+ clk_creds = {
101
+ 'host': '192.168.3.171',
102
+ 'port': 18123,
103
+ 'user': 'username',
104
+ 'database': 'xxxxxxx',
105
+ 'table': 'customer_file',
106
+ 'order_by': 'id'
107
+ }
108
+
109
+ df_helper.save_to_clickhouse(**clk_creds)
110
+ ```
111
+
112
+ ### Saving Data to Parquet
113
+ ```python
114
+ df_helper.save_to_parquet(
115
+ parquet_filename='filename.parquet',
116
+ parquet_storage_path='/path/to/my/files/'
117
+ )
118
+ ```
119
+
120
+ Backends Supported
121
+ ---------------------
122
+ | Backend | Description |
123
+ |--------------|-------------|
124
+ | `sqlalchemy` | Load data from SQL databases using SQLAlchemy. |
125
+ | `django_db` | Load data from Django ORM models. |
126
+ | `parquet` | Load and save data from Parquet files. |
127
+ | `http` | Fetch data from HTTP endpoints. |
128
+ | `osmnx` | Geospatial mapping and routing using OpenStreetMap. |
129
+ | `geopy` | Geolocation services for address lookup and reverse geocoding. |
130
+
131
+ Geospatial Utilities
132
+ ---------------------
133
+ ### **OSMnx Helper (`sibi_dst.osmnx_helper`)
134
+ **
135
+ Provides **OpenStreetMap-based mapping utilities** using `osmnx` and `folium`.
136
+
137
+ #### 🔹 Key Features
138
+ - **BaseOsmMap**: Manages interactive Folium-based maps.
139
+ - **PBFHandler**: Loads `.pbf` (Protocolbuffer Binary Format) files for network graphs.
140
+
141
+ #### Example: Generating an OSM Map
142
+ ```python
143
+ from sibi_dst.osmnx_helper import BaseOsmMap
144
+ osm_map = BaseOsmMap(osmnx_graph=my_graph, df=my_dataframe)
145
+ osm_map.generate_map()
146
+ ```
147
+
148
+ ### **Geopy Helper (`sibi_dst.geopy_helper`)
149
+ **
150
+ Provides **geolocation services** using `Geopy` for forward and reverse geocoding.
151
+
152
+ #### 🔹 Key Features
153
+ - **GeolocationService**: Interfaces with `Nominatim` API for geocoding.
154
+ - **Error Handling**: Manages `GeocoderTimedOut` and `GeocoderServiceError` gracefully.
155
+ - **Singleton Geolocator**: Efficiently reuses a global geolocator instance.
156
+
157
+ #### Example: Reverse Geocoding
158
+ ```python
159
+ from sibi_dst.geopy_helper import GeolocationService
160
+ gs = GeolocationService()
161
+ location = gs.reverse_geocode(lat=9.935, lon=-84.091)
162
+ print(location)
163
+ ```
164
+
165
+ Advanced Features
166
+ ---------------------
167
+ ### Querying with Custom Filters
168
+ Filters can be applied dynamically using Django-style syntax:
169
+ ```python
170
+ result = df_helper.load(date__gte='2023-01-01', status='active')
171
+ ```
172
+
173
+ ### Parallel Processing
174
+ Leverage Dask for parallel execution:
175
+ ```python
176
+ results = df_helper.load_parallel(status='active')
177
+ ```
178
+
179
+ Testing
180
+ ---------------------
181
+ To run unit tests, use:
182
+ ```bash
183
+ pytest tests/
184
+ ```
185
+
186
+ Contributing
187
+ ---------------------
188
+ Contributions are welcome! Please submit pull requests or open issues for discussions.
189
+
190
+ License
191
+ ---------------------
192
+ sibi-dst is licensed under the MIT License.
193
+
194
+
{sibi_dst-0.3.42.dist-info → sibi_dst-0.3.44.dist-info}/RECORD RENAMED
@@ -1,41 +1,46 @@
1
1
  sibi_dst/__init__.py,sha256=CLHfzrFNqklNx5uMKAPtbZfkbBbVYR5qsiMro0RTfmA,252
2
2
  sibi_dst/df_helper/__init__.py,sha256=A-f5cCBy949HHxgiPt0T4MG3qdLAnDpGOpRvP-2dXWc,400
3
- sibi_dst/df_helper/_artifact_updater_multi_wrapper.py,sha256=wstGiqayex4yOa-WzoOnWHY6vsLoh1XJyfS3LppKD3Q,11502
4
- sibi_dst/df_helper/_df_helper.py,sha256=NRiLdHHO45SPwhif5JIQpfj56iC8HcffaRAyT7-TC2w,29585
5
- sibi_dst/df_helper/_parquet_artifact.py,sha256=K9FnKjXDmkqCzYqv5weS9scLHsPGyj0UUUoVzOtWv30,8858
6
- sibi_dst/df_helper/_parquet_reader.py,sha256=HhzhKtV_7qABHJvmpU2CssjNLgQHUB07eF0CqqzmkOs,3654
3
+ sibi_dst/df_helper/_artifact_updater_multi_wrapper.py,sha256=toH2QvNF-CQNJ4Bc8xreytuWr37G0EWz4ciWVdFMVqU,11646
4
+ sibi_dst/df_helper/_df_helper.py,sha256=IS1m9r9U-eJ7EVMBqmITmre6S0JfIl6nJtPIwNI3xKY,29771
5
+ sibi_dst/df_helper/_parquet_artifact.py,sha256=JjRPMfKKaTEX2-SUIPu1nShZB1Bl52GgPdDF_UHwd9A,10146
6
+ sibi_dst/df_helper/_parquet_reader.py,sha256=L6mr2FeKtTeIn37G9EGpvOx8PwMqXb6qnEECqBaiwxo,3954
7
7
  sibi_dst/df_helper/backends/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
8
8
  sibi_dst/df_helper/backends/django/__init__.py,sha256=uWHi-DtQX5re7b2HcqoXUH3_FZWOw1VTmDf552FAkNs,256
9
9
  sibi_dst/df_helper/backends/django/_db_connection.py,sha256=AGbqCnmiX4toMaFPE5ne5h7QCkImjnBKvzGtUD6Ge8Q,3698
10
10
  sibi_dst/df_helper/backends/django/_io_dask.py,sha256=NjvJg6y9qKKCRiNrJL4f_A03iKDKEcjCi7LGbr9DgtM,19555
11
- sibi_dst/df_helper/backends/django/_load_from_db.py,sha256=jQGIeviggHmSfK28Z47nHz3cXs78gHsE8Op7Ov1kFCo,10562
11
+ sibi_dst/df_helper/backends/django/_load_from_db.py,sha256=htG9ec4ix371ClEHQVpx4r3mhBdQaSykeHUCCRhN7L4,10637
12
12
  sibi_dst/df_helper/backends/django/_sql_model_builder.py,sha256=at9J7ecGkZbOOYba85uofe9C-ic4wwOqVgJcHpQNiYQ,21449
13
13
  sibi_dst/df_helper/backends/http/__init__.py,sha256=d1pfgYxbiYg7E0Iw8RbJ7xfqIfJShqqTBQQGU_S6OOo,105
14
14
  sibi_dst/df_helper/backends/http/_http_config.py,sha256=eGPFdqZ5M3Tscqx2P93B6XoBEEzlmdt7yNg7PXUQnNQ,4726
15
15
  sibi_dst/df_helper/backends/parquet/__init__.py,sha256=esWJ9aSuYC26d-T01z9dPrJ1uqJzvdaPNTYRb5qXTlQ,182
16
- sibi_dst/df_helper/backends/parquet/_filter_handler.py,sha256=VrT79Cr1HXu0VLVTuwAYOwSo52p8B1Q5lT3tTIk4uAA,5136
17
- sibi_dst/df_helper/backends/parquet/_parquet_options.py,sha256=RZ5JopJVv9G_j0S9Vw66ohNvNcZHmeBfncMwSsLjM6o,10707
16
+ sibi_dst/df_helper/backends/parquet/_filter_handler.py,sha256=TvDf0RXta7mwJv11GNQttYJsXgFf2XDj4oLIjt4xTzA,5219
17
+ sibi_dst/df_helper/backends/parquet/_parquet_options.py,sha256=TaU5_wG1Y3lQC8DVCItVvMnc6ZJmECLu3avssVEMbaM,10591
18
18
  sibi_dst/df_helper/backends/sqlalchemy/__init__.py,sha256=TuVp8Ce49dCIIxtyrtFGRblarQUl8QGcS-TDZd515IE,348
19
- sibi_dst/df_helper/backends/sqlalchemy/_db_connection.py,sha256=-z3eYZVDKJPbuB_RWbY_zBlLuwvVyY5R5FvnLHp0S8A,2501
19
+ sibi_dst/df_helper/backends/sqlalchemy/_db_connection.py,sha256=Kli83IEg5SFVqkhsK4w45cV6PbZnfdGanfsyiW6Xw00,2502
20
20
  sibi_dst/df_helper/backends/sqlalchemy/_filter_handler.py,sha256=58RCda1Hg_nsuJw-2V36IstsT8O84IQFgsdE7FnqvMk,4655
21
- sibi_dst/df_helper/backends/sqlalchemy/_io_dask.py,sha256=UuAHzZWBADsTwGhwZTJzR66Xdh189OR81C1IITwzls0,5620
21
+ sibi_dst/df_helper/backends/sqlalchemy/_io_dask.py,sha256=BtiRSYA4kFIM-mBCdrwE20vzByfq8_Biv_jPLUCDv58,5466
22
22
  sibi_dst/df_helper/backends/sqlalchemy/_load_from_db.py,sha256=I2Us3RrxHci561yyZYBuUCrLVOhB0F3KBnae78m_ARw,6259
23
23
  sibi_dst/df_helper/backends/sqlalchemy/_sql_model_builder.py,sha256=9oOOGrqAj9yL0FNWR1Fm9PdN7GoFi03ktCOyMjxAKLY,8402
24
24
  sibi_dst/df_helper/core/__init__.py,sha256=o4zDwgVmaijde3oix0ezb6KLxI5QFy-SGUhFTDVFLT4,569
25
25
  sibi_dst/df_helper/core/_defaults.py,sha256=eNpHD2sZxir-2xO0b3_V16ryw8YP_5FfpIKK0HNuiN4,7011
26
- sibi_dst/df_helper/core/_filter_handler.py,sha256=t3uLLJX5hWO_dWKCCz8Dwpc9RZ5PMHBIWkHSELCpFXI,11131
26
+ sibi_dst/df_helper/core/_filter_handler.py,sha256=Pmbzygry2mpkNPVS7DBMulHpAb1yYZNFqUU0bJTWJF0,11214
27
27
  sibi_dst/df_helper/core/_params_config.py,sha256=DYx2drDz3uF-lSPzizPkchhy-kxRrQKE5FQRxcEWsac,6736
28
28
  sibi_dst/df_helper/core/_query_config.py,sha256=Y8LVSyaKuVkrPluRDkQoOwuXHQxner1pFWG3HPfnDHM,441
29
29
  sibi_dst/df_helper/data_cleaner.py,sha256=lkxQoXLvGzXCicFUimnA5nen5qkrO1oxgl_p2Be2o8w,5183
30
30
  sibi_dst/geopy_helper/__init__.py,sha256=Q1RJiUZIOlV0QNNLjxZ_2IZS5LqIe5jRbeQkfD1Vm60,112
31
31
  sibi_dst/geopy_helper/geo_location_service.py,sha256=1ArI980QF_gRw096ZsABHwJt-m55jrfOlB8tPwL1BvY,2959
32
32
  sibi_dst/geopy_helper/utils.py,sha256=Sb7qfSqIyWh-AZ4GBdB9-z5FrQPWtrdtQLLcNjph0yw,3351
33
- sibi_dst/osmnx_helper/__init__.py,sha256=QeAKEeVXZk_qn8o0d3BOoGgv2lzatcI2yBqY3ZqviKI,153
34
- sibi_dst/osmnx_helper/base_osm_map.py,sha256=L7g3VBiayHX41BcCBTOCS0iJOKzp2ZZYcrp8N-mnU90,19392
35
- sibi_dst/osmnx_helper/basemaps/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
36
- sibi_dst/osmnx_helper/basemaps/calendar_html.py,sha256=UArt6FDgoCgoRte45Xo3IHqd-RNzW0YgitgZYfOFasY,4031
37
- sibi_dst/osmnx_helper/basemaps/router_plotter.py,sha256=QznnBGsUwhl8ZITcVNBrQDm-MXAd0jpJGPuyozKyQg0,8537
38
- sibi_dst/osmnx_helper/utils.py,sha256=BzuY8CtYnBAAO8UAr_M7EOk6CP1zcifNLs8pkdFZEFg,20577
33
+ sibi_dst/osmnx_helper/__init__.py,sha256=8061nkUZ-9HzXuGvJvo1e_I9SJXBy1jXAo4rzk-Y-lU,159
34
+ sibi_dst/osmnx_helper/v1/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
35
+ sibi_dst/osmnx_helper/v1/base_osm_map.py,sha256=L7g3VBiayHX41BcCBTOCS0iJOKzp2ZZYcrp8N-mnU90,19392
36
+ sibi_dst/osmnx_helper/v1/basemaps/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
37
+ sibi_dst/osmnx_helper/v1/basemaps/calendar_html.py,sha256=UArt6FDgoCgoRte45Xo3IHqd-RNzW0YgitgZYfOFasY,4031
38
+ sibi_dst/osmnx_helper/v1/basemaps/router_plotter.py,sha256=SWdDz5XGDSHT6Iyr-EIatSNTvGPR3AVDJ5TTcWm0w4g,10947
39
+ sibi_dst/osmnx_helper/v1/utils.py,sha256=BzuY8CtYnBAAO8UAr_M7EOk6CP1zcifNLs8pkdFZEFg,20577
40
+ sibi_dst/osmnx_helper/v2/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
41
+ sibi_dst/osmnx_helper/v2/base_osm_map.py,sha256=YotJR-OcsQyx2fEem6TtIyHF1FCRzoqnWBPbZpLgH1k,5797
42
+ sibi_dst/osmnx_helper/v2/basemaps/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
43
+ sibi_dst/osmnx_helper/v2/basemaps/utils.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
39
44
  sibi_dst/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
40
45
  sibi_dst/tests/test_data_wrapper_class.py,sha256=Nkup5OFH5Cos2fxPaU7g9IEyINJM0uJ5-rOZ-eNtd20,3275
41
46
  sibi_dst/utils/__init__.py,sha256=2uX4IQN02NeQzR1RRK3d7AV51XC54wg4ieOM7yTSfDs,875
@@ -43,15 +48,15 @@ sibi_dst/utils/airflow_manager.py,sha256=-d44EKUZNYJyp4wuNwRvilRQktunArPOB5fZuWd
43
48
  sibi_dst/utils/clickhouse_writer.py,sha256=syXGN9NG1FS8soHuMj6QNRqTRWi-thuYUF-_BWDc_KI,9883
44
49
  sibi_dst/utils/credentials.py,sha256=cHJPPsmVyijqbUQIq7WWPe-lIallA-mI5RAy3YUuRME,1724
45
50
  sibi_dst/utils/data_utils.py,sha256=w07GStEBK6iz6tENTnhngiu1jpRNdo6dTgqP4iAUsTw,10304
46
- sibi_dst/utils/data_wrapper.py,sha256=xA0f4KpHGlAmkR2cZGs2yFuPK4sWttcC9OAU0WUfDo8,29007
51
+ sibi_dst/utils/data_wrapper.py,sha256=Fm-YeFj_utLxN2A91z9Y9q8YMwkAfZNTyYiTbczQ1QY,12069
47
52
  sibi_dst/utils/date_utils.py,sha256=QSsL_O9ZqIdNjhppyiVOjCFLmhJEKH6F2TeiGBZe9m4,18363
48
- sibi_dst/utils/df_utils.py,sha256=OFEtcwVKIilvf9qVf-IfIOHp4jcFAHX5l2IDGudhPZg,10989
53
+ sibi_dst/utils/df_utils.py,sha256=GAX0lthULTmGaDvYzuLmo0op7YKaCM5uot403QpztoM,11278
49
54
  sibi_dst/utils/file_utils.py,sha256=JpsybYj3XvVJisSBeVU6YSaZnYRm4_6YWTI3TLnnY4Y,1257
50
55
  sibi_dst/utils/filepath_generator.py,sha256=volVm0SSlBrtZp1RpTHxyui5rj5asNcVsWEBRY5FOUQ,6673
51
- sibi_dst/utils/log_utils.py,sha256=DtUAcY1aJzlIi9ItWDRz8Yyrz68A908JTm3XEd1HKBo,4484
52
- sibi_dst/utils/parquet_saver.py,sha256=_QkXL3IiC2b4m7sxHpCSeqPwBWxXeiP5sH_WheSMEm4,8042
56
+ sibi_dst/utils/log_utils.py,sha256=eSAbi_jmMpJ8RpycakzT4S4zNkqVZDj3FY8WwnxpdXc,4623
57
+ sibi_dst/utils/parquet_saver.py,sha256=_BFeD0HDXceBaBR0Y48t9MR_fY9G1Y8HrZCkj2X1yJs,8173
53
58
  sibi_dst/utils/phone_formatter.py,sha256=tsVTDamuthFYgy4-5UwmQkPQ-FGTGH7MjZyH8utAkIY,4945
54
- sibi_dst/utils/storage_manager.py,sha256=-zlMrRo_6o6mCd_OHknKqNQl7m0I9VW89grDAUO1V5c,4229
55
- sibi_dst-0.3.42.dist-info/METADATA,sha256=3Zk5DLEHp-R0zAIhHHdeNY-Gbq497P6h8Uah46d_NpU,2564
56
- sibi_dst-0.3.42.dist-info/WHEEL,sha256=Nq82e9rUAnEjt98J6MlVmMCZb-t9cYE2Ir1kpBmnWfs,88
57
- sibi_dst-0.3.42.dist-info/RECORD,,
59
+ sibi_dst/utils/storage_manager.py,sha256=H_itUFJv9nP0BfXYYQDsw4RzB0YWfgVOAHNWAiMpZ_w,4443
60
+ sibi_dst-0.3.44.dist-info/METADATA,sha256=j8kho4OWGcuSb1u78vZBxJSXugeHd_sipJdyLLPz_p0,6514
61
+ sibi_dst-0.3.44.dist-info/WHEEL,sha256=Nq82e9rUAnEjt98J6MlVmMCZb-t9cYE2Ir1kpBmnWfs,88
62
+ sibi_dst-0.3.44.dist-info/RECORD,,
sibi_dst-0.3.42.dist-info/METADATA DELETED
@@ -1,62 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: sibi-dst
3
- Version: 0.3.42
4
- Summary: Data Science Toolkit
5
- Author: Luis Valverde
6
- Author-email: lvalverdeb@gmail.com
7
- Requires-Python: >=3.11,<4.0
8
- Classifier: Programming Language :: Python :: 3
9
- Classifier: Programming Language :: Python :: 3.11
10
- Classifier: Programming Language :: Python :: 3.12
11
- Classifier: Programming Language :: Python :: 3.13
12
- Requires-Dist: apache-airflow-client (>=2.10.0,<3.0.0)
13
- Requires-Dist: chardet (>=5.2.0,<6.0.0)
14
- Requires-Dist: charset-normalizer (>=3.4.0,<4.0.0)
15
- Requires-Dist: clickhouse-connect (>=0.8.7,<0.9.0)
16
- Requires-Dist: clickhouse-driver (>=0.2.9,<0.3.0)
17
- Requires-Dist: dask-expr (>=1.1.20,<2.0.0)
18
- Requires-Dist: dask[complete] (>=2024.11.1,<2025.0.0)
19
- Requires-Dist: django (>=5.1.4,<6.0.0)
20
- Requires-Dist: djangorestframework (>=3.15.2,<4.0.0)
21
- Requires-Dist: folium (>=0.19.4,<0.20.0)
22
- Requires-Dist: geopandas (>=1.0.1,<2.0.0)
23
- Requires-Dist: gunicorn (>=23.0.0,<24.0.0)
24
- Requires-Dist: httpx (>=0.27.2,<0.28.0)
25
- Requires-Dist: ipython (>=8.29.0,<9.0.0)
26
- Requires-Dist: jinja2 (>=3.1.4,<4.0.0)
27
- Requires-Dist: mysqlclient (>=2.2.6,<3.0.0)
28
- Requires-Dist: nltk (>=3.9.1,<4.0.0)
29
- Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
30
- Requires-Dist: osmnx (>=2.0.1,<3.0.0)
31
- Requires-Dist: pandas (>=2.2.3,<3.0.0)
32
- Requires-Dist: paramiko (>=3.5.0,<4.0.0)
33
- Requires-Dist: psutil (>=6.1.0,<7.0.0)
34
- Requires-Dist: psycopg2 (>=2.9.10,<3.0.0)
35
- Requires-Dist: pyarrow (>=18.0.0,<19.0.0)
36
- Requires-Dist: pydantic (>=2.9.2,<3.0.0)
37
- Requires-Dist: pymysql (>=1.1.1,<2.0.0)
38
- Requires-Dist: pytest (>=8.3.3,<9.0.0)
39
- Requires-Dist: pytest-mock (>=3.14.0,<4.0.0)
40
- Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
41
- Requires-Dist: s3fs (>=2024.12.0,<2025.0.0)
42
- Requires-Dist: sqlalchemy (>=2.0.36,<3.0.0)
43
- Requires-Dist: tornado (>=6.4.1,<7.0.0)
44
- Requires-Dist: tqdm (>=4.67.0,<5.0.0)
45
- Requires-Dist: uvicorn (>=0.34.0,<0.35.0)
46
- Requires-Dist: uvicorn-worker (>=0.3.0,<0.4.0)
47
- Description-Content-Type: text/markdown
48
-
49
- # sibi-dst
50
-
51
- Data Science Toolkit
52
- ---------------------
53
- Data Science Toolkit built with Python, Pandas, Dask, OpenStreetMaps, Scikit-Learn, XGBOOST, Django ORM, SQLAlchemy, DjangoRestFrameWork
54
-
55
- Major Functionality
56
- --------------------
57
- 1) Build DataCubes, DataSets and DataObjects from different datasources. These include relational databases, parquet files, xlsx, delimited tables, json, json API REST.
58
- 2) Common dataframe management utilities.
59
- 3) Share Data with client applications by write to Data Warehouses in local filesystems as well as other supported platforms.
60
- 4) Build microservices to communicate/share data via API-REST, gRPC.
61
-
62
-
/sibi_dst/osmnx_helper/{basemaps → v1}/__init__.py RENAMED
File without changes
/sibi_dst/osmnx_helper/{utils.py → v1/utils.py} RENAMED
File without changes