ccfx 1.1.0__tar.gz → 1.1.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
ccfx-1.1.2/PKG-INFO ADDED
@@ -0,0 +1,595 @@
1
+ Metadata-Version: 2.4
2
+ Name: ccfx
3
+ Version: 1.1.2
4
+ Summary: This package simplifies regular common actions for quick prototyping in a user friendly way
5
+ Author-email: Celray James CHAWANDA <celray@chawanda.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/celray/ccfx
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Operating System :: OS Independent
10
+ Requires-Python: >=3.10
11
+ Description-Content-Type: text/markdown
12
+ License-File: LICENSE
13
+ Requires-Dist: netCDF4
14
+ Requires-Dist: yt_dlp
15
+ Requires-Dist: gdal
16
+ Requires-Dist: numpy
17
+ Requires-Dist: shapely
18
+ Requires-Dist: geopandas
19
+ Requires-Dist: pandas
20
+ Requires-Dist: xlsxwriter
21
+ Requires-Dist: pyodbc
22
+ Requires-Dist: sqlalchemy
23
+ Requires-Dist: python-docx
24
+ Requires-Dist: py7zr
25
+ Requires-Dist: mutagen
26
+ Requires-Dist: requests
27
+ Requires-Dist: tqdm
28
+ Requires-Dist: pillow
29
+ Requires-Dist: scipy
30
+ Requires-Dist: rasterio
31
+ Requires-Dist: matplotlib
32
+ Dynamic: license-file
33
+
34
+ # ccfx
35
+
36
+ [![PyPI version](https://badge.fury.io/py/ccfx.svg)](https://badge.fury.io/py/ccfx)
37
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
38
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
39
+ [![GitHub stars](https://img.shields.io/github/stars/celray/ccfx.svg)](https://github.com/celray/ccfx/stargazers)
40
+
41
+ `ccfx` is a comprehensive Python package designed to streamline file and data management, geospatial analysis, NetCDF file processing, database interactions, document generation, and multimedia handling for rapid prototyping and development workflows.
42
+
43
+ ## Table of Contents
44
+
45
+ - [Features](#features)
46
+ - [Installation](#installation)
47
+ - [Quick Start](#quick-start)
48
+ - [Dependencies](#dependencies)
49
+ - [Usage Examples](#usage-examples)
50
+ - [API Reference](#api-reference-selected-functions)
51
+ - [File Management](#file-management-ccfxpy)
52
+ - [Geospatial](#geospatial-ccfxpy)
53
+ - [NetCDF](#netcdf-ccfxpy)
54
+ - [Database](#database-mssqlconnectionpy-sqliteconnectionpy)
55
+ - [Document/Spreadsheet](#documentspreadsheet-wordpy-excelpy)
56
+ - [Multimedia & Web](#multimedia--web-ccfxpy)
57
+ - [Data Analysis & Utilities](#data-analysis--utilities-ccfxpy)
58
+ - [System Requirements](#system-requirements)
59
+ - [Contributing](#contributing)
60
+ - [Changelog](#changelog)
61
+ - [License](#license)
62
+
63
+ ## Features
64
+
65
+ 1. **File Management**:
66
+ * List, delete, move, copy, and count files/directories.
67
+ * Monitor file count over time.
68
+ * Save, load, and manage Python variables via pickle serialization.
69
+ * Compress directories to `.7z` archives.
70
+ * Read/write text files with encoding support.
71
+ * Download files from URLs with resume and multi-connection support.
72
+
73
+ 2. **Geospatial Data Processing**:
74
+ * Read, write, clip (by extent/feature), resample, reproject, merge, and rasterize raster data (GeoTIFF, NetCDF).
75
+ * Read, write, and clip vector data (Shapefile, GeoPackage).
76
+ * Create grids of polygons based on shapefile boundaries.
77
+ * Convert coordinates between Coordinate Reference Systems (CRS).
78
+ * Extract raster values at specific coordinates.
79
+ * Convert point lists to GeoDataFrames.
80
+ * Get vector layer bounds.
81
+
82
+ 3. **NetCDF File Handling**:
83
+ * List variables and dimensions.
84
+ * Export NetCDF variables to GeoTIFF format (single or multiple bands).
85
+ * Calculate sum and average maps from NetCDF data across multiple files.
86
+ * Rename variables using CDO (if available).
87
+
88
+ 4. **Database Connectivity**:
89
+ * **MS SQL Server**: Connect, list databases/tables, read tables (including spatial data into GeoDataFrames), write DataFrames/GeoDataFrames to tables, drop tables.
90
+ * **SQLite**: Connect, create/rename/drop tables, read tables (as dict, specific columns), insert data (rows, dicts, partial dicts), update values, dump tables to CSV.
91
+
92
+ 5. **Document & Spreadsheet Handling**:
93
+ * **Excel**: Create `.xlsx` files, add sheets, write data (including dates), set column widths, add scatter plot charts.
94
+ * **Word**: Create `.docx` files, add headings, paragraphs (with alignment), list items, formatted text (bold/italic), images, page breaks, set margins.
95
+
96
+ 6. **Multimedia & Web**:
97
+ * Read and write MP3 metadata (ID3 tags), including album art.
98
+ * Download videos/audio from YouTube using `yt-dlp`.
99
+
100
+ 7. **Data Analysis & Utilities**:
101
+ * Calculate timeseries statistics (NSE, KGE, PBIAS, LNSE, R2, RMSE, MAE, MSE, MAPE, alpha, beta) with resampling options.
102
+ * Display dynamic progress bars.
103
+ * Check system platform information.
104
+ * Enable or disable warnings programmatically.
105
+ * Set the working directory.
106
+
107
+ ## Installation
108
+
109
+ Install `ccfx` via pip:
110
+ ```bash
111
+ pip install ccfx
112
+ ```
113
+
114
+ > **Note**: GDAL is a core dependency and may require additional system-level installation steps. See [System Requirements](#system-requirements) for details.
115
+
116
+ ## Quick Start
117
+
118
+ Here's a simple example to get you started with `ccfx`:
119
+
120
+ ```python
121
+ import ccfx
122
+
123
+ # File management
124
+ files = ccfx.listFiles("/path/to/directory", ext=".txt")
125
+ print(f"Found {len(files)} text files")
126
+
127
+ # Create and save data to Excel
128
+ excel_doc = ccfx.excel("output.xlsx")
129
+ excel_doc.create()
130
+ excel_doc.addSheet("MyData")
131
+ excel_doc.write("A1", "Hello, ccfx!")
132
+ excel_doc.save()
133
+
134
+ # Work with geospatial data
135
+ bounds = (-180, -90, 180, 90) # Global bounds
136
+ ccfx.clipRasterByExtent("input.tif", "clipped.tif", bounds)
137
+
138
+ # Download and process data
139
+ ccfx.downloadFile("https://example.com/data.zip", "./data.zip")
140
+ ccfx.compressTo7z("./my_folder", "archive.7z")
141
+ ```
142
+
143
+ ## Dependencies
144
+
145
+ `ccfx` relies on the following libraries (automatically installed with pip):
146
+
147
+ ### Core Dependencies
148
+ * **gdal**: For geospatial raster and vector data manipulation.
149
+ * **numpy**: For array processing and numerical operations.
150
+ * **pandas**: For data manipulation and analysis.
151
+ * **geopandas**: Extends pandas to handle geospatial vector data.
152
+ * **shapely**: Provides geometric objects and operations.
153
+ * **netCDF4**: For working with NetCDF files.
154
+ * **rasterio**: Advanced raster I/O and processing.
155
+
156
+ ### Document & Database
157
+ * **xlsxwriter**: For creating and writing Excel `.xlsx` files.
158
+ * **python-docx**: Enables creation and manipulation of Word `.docx` documents.
159
+ * **pyodbc**: Enables connectivity to databases through ODBC (e.g., MS SQL Server).
160
+ * **sqlalchemy**: Provides SQL toolkit and ORM features for database access.
161
+
162
+ ### Utilities & Multimedia
163
+ * **py7zr**: For creating `.7z` archives.
164
+ * **mutagen**: For reading and writing MP3 metadata (ID3 tags).
165
+ * **requests**: For downloading files via HTTP.
166
+ * **tqdm**: For displaying progress bars.
167
+ * **yt-dlp**: For downloading YouTube content.
168
+ * **pillow**: For image processing.
169
+ * **scipy**: For scientific computing.
170
+ * **matplotlib**: For plotting and visualization.
171
+
172
+ ## System Requirements
173
+
174
+ ### Python Version
175
+ - **Python 3.10+** is required
176
+
177
+ ### GDAL Installation
178
+ GDAL can be challenging to install depending on your operating system:
179
+
180
+ #### Windows
181
+ ```bash
182
+ # Using conda (recommended)
183
+ conda install -c conda-forge gdal
184
+
185
+ # Or using pip with pre-compiled binaries
186
+ pip install GDAL
187
+
188
+ # Alternative: If struggling with GDAL installation on Windows
189
+ pip install gdal-installer
190
+ install-gdal
191
+ ```
192
+
193
+ #### macOS
194
+ ```bash
195
+ # Using Homebrew
196
+ brew install gdal
197
+ pip install gdal
198
+
199
+ # Using conda
200
+ conda install -c conda-forge gdal
201
+ ```
202
+
203
+ #### Linux (Ubuntu/Debian)
204
+ ```bash
205
+ # Install system dependencies
206
+ sudo apt-get update
207
+ sudo apt-get install gdal-bin libgdal-dev
208
+
209
+ # Install Python bindings
210
+ pip install gdal
211
+ ```
212
+
213
+ #### Docker
214
+ For a hassle-free setup, consider using the official GDAL Docker images:
215
+ ```bash
216
+ docker pull ghcr.io/osgeo/gdal:ubuntu-small-latest
217
+ ```
218
+
219
+ ## Usage Examples
220
+
221
+ ### File Management
222
+ ```python
223
+ import ccfx
224
+
225
+ # List all Python files in a directory
226
+ python_files = ccfx.listFiles("/path/to/project", ext=".py")
227
+
228
+ # Create a backup directory and copy files
229
+ ccfx.createPath("/backup/location")
230
+ ccfx.copyDirectory("/source/dir", "/backup/location")
231
+
232
+ # Compress a directory
233
+ ccfx.compressTo7z("/data/folder", "backup.7z")
234
+
235
+ # Download a file with resume capability
236
+ ccfx.downloadFile(
237
+ "https://example.com/largefile.zip",
238
+ "downloaded_file.zip",
239
+ exists_action='resume'
240
+ )
241
+ ```
242
+
243
+ ### Geospatial Data Processing
244
+ ```python
245
+ import ccfx
246
+
247
+ # Clip a raster to a specific bounding box
248
+ bounds = (-74.0, 40.7, -73.9, 40.8) # NYC area
249
+ ccfx.clipRasterByExtent("satellite_image.tif", "nyc_clip.tif", bounds)
250
+
251
+ # Resample a raster to different resolution
252
+ ccfx.resampleRaster(
253
+ "high_res.tif",
254
+ "low_res.tif",
255
+ resolution=1000, # 1km resolution
256
+ resamplingMethod='bilinear'
257
+ )
258
+
259
+ # Convert coordinates between projections
260
+ lat, lon = 40.7128, -74.0060 # NYC coordinates
261
+ x, y = ccfx.convertCoordinates(lon, lat, "EPSG:4326", "EPSG:3857")
262
+
263
+ # Extract raster value at specific point
264
+ value = ccfx.extractRasterValue("elevation.tif", 40.7128, -74.0060)
265
+ print(f"Elevation at NYC: {value}")
266
+ ```
267
+
268
+ ### Database Operations
269
+ ```python
270
+ import ccfx
271
+
272
+ # SQLite operations
273
+ db = ccfx.sqliteConnection("my_data.db", connect=True)
274
+ db.createTable("users", ["id INTEGER PRIMARY KEY", "name TEXT", "email TEXT"])
275
+ db.insertDict("users", {"name": "John Doe", "email": "john@example.com"})
276
+ users = db.readTableAsDict("users")
277
+ db.closeConnection()
278
+
279
+ # MS SQL Server operations
280
+ mssql = ccfx.mssql_connection("server", "username", "password", "driver")
281
+ mssql.connect()
282
+ databases = mssql.listDatabases()
283
+ df = mssql.readTable("MyDatabase", "MyTable")
284
+ mssql.close()
285
+ ```
286
+
287
+ ### Document Generation
288
+ ```python
289
+ import ccfx
290
+
291
+ # Create Excel spreadsheet
292
+ excel = ccfx.excel("report.xlsx")
293
+ excel.create()
294
+ excel.addSheet("Sales Data")
295
+ excel.write("A1", "Product")
296
+ excel.write("B1", "Revenue")
297
+ excel.writeColumn("A", ["Product A", "Product B", "Product C"], start_row=2)
298
+ excel.writeColumn("B", [1000, 1500, 800], start_row=2)
299
+ excel.save()
300
+
301
+ # Create Word document
302
+ doc = ccfx.word_document("report.docx")
303
+ doc.addHeading("Monthly Report", level=1)
304
+ doc.addParagraph("This report summarizes our monthly performance.")
305
+ doc.addListItem("Revenue increased by 15%")
306
+ doc.addListItem("Customer satisfaction improved")
307
+ doc.save()
308
+ ```
309
+
310
+ ### NetCDF Processing
311
+ ```python
312
+ import ccfx
313
+
314
+ # List variables in NetCDF file
315
+ variables = ccfx.netcdfVariablesList("climate_data.nc")
316
+ print("Available variables:", variables)
317
+
318
+ # Export NetCDF variable to GeoTIFF
319
+ ccfx.netcdfExportTif(
320
+ "temperature_data.nc",
321
+ "temperature",
322
+ "temp_map.tif",
323
+ band=1
324
+ )
325
+
326
+ # Calculate average from multiple NetCDF files
327
+ nc_files = ["data_2020.nc", "data_2021.nc", "data_2022.nc"]
328
+ avg_map = ccfx.netcdfAverageMap(nc_files, "precipitation")
329
+ ```
330
+
331
+ ### Data Analysis
332
+ ```python
333
+ import ccfx
334
+ import pandas as pd
335
+
336
+ # Calculate timeseries statistics
337
+ observed_data = pd.read_csv("observed.csv")
338
+ simulated_data = pd.read_csv("simulated.csv")
339
+
340
+ # Combine data
341
+ data = pd.merge(observed_data, simulated_data, on='date')
342
+
343
+ # Calculate Nash-Sutcliffe Efficiency
344
+ stats = ccfx.calculateTimeseriesStats(data, observed='obs', simulated='sim')
345
+ print(f"NSE: {stats['NSE']:.3f}")
346
+ print(f"KGE: {stats['KGE']:.3f}")
347
+ print(f"R²: {stats['R2']:.3f}")
348
+ ```
349
+
350
+ ## API Reference (Complete Function List)
351
+
352
+ ### File & Directory Management (`ccfx.py`)
353
+
354
+ **Basic File Operations:**
355
+ * **`listFiles(path: str, ext: Optional[str] = None) -> list`**: Lists files in a directory, optionally filtering by extension.
356
+ * **`listAllFiles(folder: str, extension: str = "*") -> list`**: Recursively lists all files in a folder and its subfolders.
357
+ * **`listFolders(path: str) -> list`**: Lists all folders in a directory.
358
+ * **`listDirectories(path: str) -> list`**: Alias for listFolders.
359
+ * **`deleteFile(filePath: str, v: bool = False) -> bool`**: Deletes a specified file.
360
+ * **`deletePath(path: str, v: bool = False) -> bool`**: Deletes a directory and its contents.
361
+ * **`createPath(pathName: str, v: bool = False) -> str`**: Creates a directory path if it doesn't exist.
362
+ * **`getExtension(filePath: str) -> str`**: Gets the extension of a file.
363
+ * **`getFileBaseName(filePath: str, extension: bool = True) -> str`**: Gets the base name of a file.
364
+
365
+ **File Copying & Moving:**
366
+ * **`copyFile(source: str, destination: str, v: bool = True) -> None`**: Copies a single file.
367
+ * **`copyDirectory(source: str, destination: str, recursive: bool = True, v: bool = True, filter: list = []) -> None`**: Copies a directory's contents.
368
+ * **`copyFolder(source: str, destination: str, v: bool = True) -> None`**: Alias for copyDirectory.
369
+ * **`moveDirectory(srcDir: str, destDir: str, v: bool = False) -> bool`**: Moves all files from source to destination directory.
370
+ * **`moveDirectoryFiles(srcDir: str, destDir: str, v: bool = False) -> bool`**: Moves files and subdirectories from source to destination.
371
+
372
+ **File I/O Operations:**
373
+ * **`readFrom(filename: str, decode_codec: Optional[str] = None, v: bool = False) -> Any`**: Reads ASCII files.
374
+ * **`readFile(filename: str, decode_codec: Optional[str] = None, v: bool = False) -> Any`**: Alias for readFrom.
375
+ * **`writeTo(filename: str, file_text: Any, encode_codec: Optional[str] = None, v: bool = False) -> bool`**: Writes ASCII files.
376
+ * **`writeToFile(filename: str, file_text: Any, encode_codec: Optional[str] = None, v: bool = False) -> bool`**: Alias for writeTo.
377
+ * **`writeFile(filename: str, file_text: Any, encode_codec: Optional[str] = None, v: bool = False) -> bool`**: Alias for writeTo.
378
+
379
+ **Compression & Archives:**
380
+ * **`compressTo7z(input_dir: str, output_file: str, compressionLevel: int = 4, excludeExt: Optional[list] = None, v: bool = False) -> None`**: Compresses a directory into a .7z file.
381
+ * **`uncompress(inputFile: str, outputDir: str, v: bool = False) -> None`**: Extracts various archive formats (.7z, .zip, .tar, etc.).
382
+ * **`uncompressFile(inputFile: str, outputDir: str, v: bool = False) -> None`**: Alias for uncompress.
383
+ * **`unzipFile(inputFile: str, outputDir: str, v: bool = False) -> None`**: Alias for uncompress.
384
+ * **`extractZip(inputFile: str, outputDir: str, v: bool = False) -> None`**: Alias for uncompress.
385
+ * **`extractCompressedFile(inputFile: str, outputDir: str, v: bool = False) -> None`**: Alias for uncompress.
386
+
387
+ **File Monitoring & Statistics:**
388
+ * **`fileCount(path: str = "./", extension: str = ".*", v: bool = True) -> int`**: Gets the number of files in a directory with a specific extension.
389
+ * **`watchFileCount(path: str = "./", extension: str = ".*", interval: float = 0.2, duration = 3, v: bool = True) -> None`**: Monitors file count over time.
390
+
391
+ **Variable Persistence:**
392
+ * **`pythonVariable(filename: str, option: str, variable: Any = None) -> Any`**: Saves ('dump') or loads ('load') Python variables using pickle.
393
+
394
+ **Download Operations:**
395
+ * **`downloadFile(url: str, save_path: str, exists_action: str = 'resume', num_connections: int = 5, v: bool = False) -> None`**: Downloads files with resume and multi-connection support.
396
+ * **`downloadChunk(url: str, start: int, end: int, path: str) -> None`**: Internal function for chunked downloads.
397
+
398
+ ### Geospatial Data Processing (`ccfx.py`)
399
+
400
+ **Raster Operations:**
401
+ * **`clipRasterByExtent(inFile: str, outFile: str, bounds: tuple) -> str`**: Clips a raster using bounding box coordinates.
402
+ * **`clipRasterByVector(inFile: str, outFile: str, vectorFile: str) -> str`**: Clips a raster using a vector file.
403
+ * **`resampleRaster(inFile: str, outFile: str, resolution: float, dstSRS = None, resamplingMethod = 'bilinear', replaceOutput: bool = True, v: bool = True) -> Optional[str]`**: Resamples a raster to a new resolution.
404
+ * **`reprojectRaster(inFile: str, outFile: str, dstProjection: str, resamplingMethod: str = 'mode') -> str`**: Reprojects a raster to a new CRS.
405
+ * **`mergeRasterTiles(tileList: list, outFile: str) -> str`**: Merges multiple raster files into one.
406
+ * **`mergeRasterFiles(tileList: list, outFile: str) -> str`**: Alias for mergeRasterTiles.
407
+ * **`rasterizeRaster(inFile: str, outFile: str, targetField: str, targetResolution: float) -> str`**: Rasterizes a vector layer based on an attribute field.
408
+ * **`rasterizeGDF(gdf: geopandas.GeoDataFrame, valueField: str, outRasterFN: str, resolution: float, isCOG: bool = True, allTouched: bool = False, profileOverrides: dict | None = None) -> str`**: Rasterizes a GeoDataFrame to GeoTIFF/COG.
409
+ * **`tiffWriteArray(array: numpy.ndarray, outputFile: str, geoTransform: tuple = (0, 1, 0, 0, 0, -1), projection: str = 'EPSG:4326', noData: Optional[float] = None, v: bool = False) -> gdal.Dataset`**: Writes a NumPy array to a GeoTIFF file.
410
+
411
+ **Vector Operations:**
412
+ * **`clipVectorByExtent(inFile: str, outFile: str, bounds: tuple) -> str`**: Clips a vector file using bounding box coordinates.
413
+ * **`clipFeatures(inputFeaturePath: str, boundaryFeature: str, outputFeature: str, keepOnlyTypes: Optional[list] = None, v: bool = False) -> geopandas.GeoDataFrame`**: Clips input features by a boundary feature.
414
+ * **`getVectorBounds(grid_gdf: geopandas.GeoDataFrame) -> tuple`**: Gets the bounds of a GeoDataFrame.
415
+
416
+ **Coordinate & Geometry Operations:**
417
+ * **`convertCoordinates(lon: float, lat: float, srcEPSG: str, dstCRS: str) -> tuple`**: Converts coordinates between CRSs.
418
+ * **`extractRasterValue(rasterPath: str, lat: float, lon: float, coordProj: str = 'EPSG:4326') -> Optional[float]`**: Extracts the raster value at a specific point.
419
+ * **`getRasterValue(rasterPath: str, lat: float, lon: float, coordProj: str = 'EPSG:4326') -> Optional[float]`**: Alias for extractRasterValue.
420
+ * **`pointsToGeodataframe(rowList: list, columnNames: list, latIndex: int, lonIndex: int, auth: str = "EPSG", code: str = "4326", outShape: str = "", format: str = "gpkg", v: bool = False, includeLatLon: bool = True) -> geopandas.GeoDataFrame`**: Converts a list of point coordinates to a GeoDataFrame.
421
+ * **`createPointGeometry(coords: list, proj: str = "EPSG:4326") -> geopandas.GeoDataFrame`**: Converts list of coordinate tuples to GeoDataFrame.
422
+ * **`createGrid(topLeft: Optional[list] = None, bottomRight: Optional[list] = None, resolution: Optional[float] = None, inputShape: Optional[str] = None, crs: str = "EPSG:4326", saveVector: Optional[str] = None) -> geopandas.GeoDataFrame`**: Creates a grid of polygons based on shapefile or coordinates.
423
+ * **`createPolygonFromOuterPoints(pointsGdf: geopandas.GeoDataFrame, alpha: float = 1.6, keepHoles: bool = False) -> geopandas.GeoDataFrame`**: Creates concave hull (alpha-shape) from points.
424
+
425
+ ### NetCDF File Handling (`ccfx.py`)
426
+
427
+ * **`netcdfVariablesList(ncFile: str) -> list`**: Lists variables in a NetCDF file.
428
+ * **`netcdfVariableDimensions(ncFile: str, variable: str) -> dict`**: Gets dimensions and their sizes for a NetCDF variable.
429
+ * **`netcdfExportTif(ncFile: str, variable: str, outputFile: Optional[str] = None, band: Optional[int] = None, v: bool = True) -> gdal.Dataset`**: Exports a NetCDF variable (optionally a specific band) to GeoTIFF.
430
+ * **`netcdfAverageMap(ncFiles: list, variable: str, band: int = 1) -> numpy.ndarray`**: Calculates the average map from a variable across multiple NetCDF files.
431
+ * **`netcdfSumMaps(ncFiles: list, variable: str, band: int = 1) -> numpy.ndarray`**: Calculates the sum map from a variable across multiple NetCDF files.
432
+ * **`renameNetCDFvariable(input_file: str, output_file: str, old_var_name: str, new_var_name: str, v: bool = False) -> None`**: Renames a variable in a NetCDF file using CDO.
433
+
434
+ ### Database Connectivity
435
+
436
+ #### MS SQL Server (`mssqlConnection.py`)
437
+ **Class: `mssqlConnection(server, username, password, driver, trust_server_ssl=True)`**
438
+ * **`__init__(server, username, password, driver, trust_server_ssl=True) -> None`**: Initialize connection parameters.
439
+ * **`connect()`**: Establish connection to server.
440
+ * **`listDatabases() -> list`**: List available databases.
441
+ * **`listTables(db_name=None) -> list`**: List tables in database.
442
+ * **`readTable()`**: Read table data (including spatial data into GeoDataFrames).
443
+ * **`connectDB()`**: Connect to specific database.
444
+ * **`dataframeToSql()`**: Write DataFrame/GeoDataFrame to table.
445
+ * **`dropTable()`**: Remove table.
446
+ * **`close()`**: Close connection.
447
+
448
+ #### SQLite (`sqliteConnection.py`)
449
+ **Class: `sqliteConnection(sqlite_database, connect=False)`**
450
+ * **`__init__(sqlite_database, connect=False) -> None`**: Initialize database connection.
451
+ * **`connect(v=True) -> None`**: Establish connection.
452
+ * **`createTable(table_name, initial_field_name, data_type) -> None`**: Create new table.
453
+ * **`renameTable()`**: Rename existing table.
454
+ * **`deleteTable()`**: Remove table.
455
+ * **`readTableAsDict()`**: Read table data as dictionary.
456
+ * **`insertDict()`**: Insert data from dictionary.
457
+ * **`insertRow()`**: Insert single row.
458
+ * **`updateValue(table_name, col_name, new_value, col_where1, val_1, v=False) -> None`**: Update specific values.
459
+ * **`dumpCSV()`**: Export table to CSV.
460
+ * **`commitChanges()`**: Commit transactions.
461
+ * **`closeConnection()`**: Close connection.
462
+
463
+ ### Document & Spreadsheet Generation
464
+
465
+ #### Excel Spreadsheets (`excel.py`)
466
+ **Class: `excel(path)`**
467
+ * **`__init__(path)`**: Initialize Excel document.
468
+ * **`create()`**: Create new workbook.
469
+ * **`addSheet(sheet_name)`**: Add worksheet.
470
+ * **`write(sheet_name, row, column, value)`**: Write data to cells.
471
+ * **`writeDate(sheet_name, row, column, datetime_obj)`**: Write date values.
472
+ * **`setDateFormat(format_string='dd/mm/yyyy')`**: Set date format.
473
+ * **`setColumnWidth(sheet_name, column_names, width=12)`**: Adjust column widths.
474
+ * **`addFigure(sheet_name, x_src_sheet_name, x_start, x_end, y_src_sheet_name, y_start, y_end, position_cell="E2", chart_type='subtype')`**: Insert charts/graphs.
475
+ * **`save()`**: Save workbook.
476
+ * **`open()`**: Open existing file.
477
+
478
+ #### Word Documents (`word.py`)
479
+ **Class: `word_document(path)`**
480
+ * **`__init__(path) -> None`**: Initialize Word document.
481
+ * **`addHeading(heading, level=2)`**: Add document headings.
482
+ * **`addParagraph(text="", alignment='justify')`**: Add text paragraphs.
483
+ * **`addListItem(text="", numbers=False)`**: Add list items.
484
+ * **`addText(text, bold=False, italic=False)`**: Add formatted text.
485
+ * **`addImage(path_to_image, width_=16)`**: Insert images.
486
+ * **`addPageBreak()`**: Insert page breaks.
487
+ * **`setMargins()`**: Configure page margins.
488
+ * **`save()`**: Save document.
489
+
490
+ ### Multimedia & Web (`ccfx.py`)
491
+
492
+ **MP3 Metadata:**
493
+ * **`getMp3Metadata(fn: str, imagePath: Optional[str] = None) -> dict`**: Extracts ID3 metadata from an MP3 file.
494
+ * **`setMp3Metadata(fn: str, metadata: dict, imagePath: Optional[str] = None) -> bool`**: Writes ID3 metadata (including album art) to an MP3 file.
495
+ * **`guessMimeType(imagePath: str) -> str`**: Determines MIME type of image files.
496
+
497
+ **YouTube Downloads:**
498
+ * **`downloadYoutubeVideo(url: str, dstDir: str, audioOnly: bool = False, cookiesFile: Optional[str] = None, dstFileName: Optional[str] = None) -> str`**: Downloads video or audio from a YouTube URL.
499
+ * **`parseYoutubePlaylist(playlistUrl: str) -> list[str]`**: Returns list of video URLs from a YouTube playlist.
500
+ * **`parseYoutubeChannelVideos(channelUrl: str, maxItems: Optional[int] = None) -> list[str]`**: Returns list of video URLs from a YouTube channel.
501
+
502
+ **Image Processing:**
503
+ * **`removeImageColour(inPath: str, outPath: str, colour: tuple = (255, 255, 255), tolerance: int = 30) -> None`**: Removes a specific color from an image.
504
+ * **`makeTransparent(inPath: str, outPath: str, colour: tuple = (255, 255, 255), tolerance: int = 30) -> None`**: Makes pixels in an image transparent.
505
+
506
+ **Video Processing:**
507
+ * **`correctFisheye(inputFile: str, outputFile: str = '', k1: float = -0.1, k2: float = 0.05, cx: float = 0.5, cy: float = 0.5, crf: int = 20) -> str`**: Corrects fisheye distortion in videos.
508
+ * **`correctLens(inputFile: str, outputFile: str = '', k1: float = -0.1, k2: float = 0.05, cx: float = 0.5, cy: float = 0.5, crf: int = 20) -> str`**: Alias for correctFisheye.
509
+
510
+ ### Data Analysis & Statistics (`ccfx.py`)
511
+
512
+ **Timeseries Analysis:**
513
+ * **`calculateTimeseriesStats(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> dict`**: Calculates comprehensive statistics between observed and simulated timeseries.
514
+ * **`getNSE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Nash-Sutcliffe Efficiency.
515
+ * **`getKGE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Kling-Gupta Efficiency.
516
+ * **`getPBIAS(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Percent Bias.
517
+ * **`getLNSE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Log Nash-Sutcliffe Efficiency.
518
+ * **`getR2(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates R-squared.
519
+ * **`getRMSE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Root Mean Square Error.
520
+ * **`getMAE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Mean Absolute Error.
521
+ * **`getMSE(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> float`**: Calculates Mean Square Error.
522
+ * **`getTimeseriesStats(data: pandas.DataFrame, observed: Optional[str] = None, simulated: Optional[str] = None, resample: Optional[str] = None) -> dict`**: Alias for calculateTimeseriesStats.
523
+
524
+ **SWAT+ Integration:**
525
+ * **`readSWATPlusOutputs(filePath: str, column: Optional[str] = None, unit: Optional[int] = None, gis_id: Optional[int] = None, name: Optional[str] = None, coerceNumeric: bool = True) -> Optional[pandas.DataFrame]`**: Reads SWAT+ output files with filtering capabilities.
526
+ * **`runSWATPlus(txtinoutDir: str, finalDir: str, executablePath: str = "swatplus", v: bool = True) -> None`**: Runs SWAT+ model with progress monitoring.
527
+
528
+ ### Utility Functions (`ccfx.py`)
529
+
530
+ **Progress & Display:**
531
+ * **`progressBar(count: int, total: int, message: str = "") -> None`**: Displays a simple console progress bar.
532
+ * **`showProgress(count: int, end: int, message: str, barLength: int = 100) -> None`**: Displays a detailed console progress bar.
533
+ * **`dualProgress(primaryCount: int, primaryEnd: int, secondaryCount: int, secondaryEnd: int, barLength: int = 40, message: str = '') -> None`**: Displays two progress bars simultaneously.
534
+
535
+ **System & Environment:**
536
+ * **`systemPlatform() -> str`**: Gets the system platform.
537
+ * **`setHomeDir(path: str) -> str`**: Sets the working directory to script location.
538
+ * **`ignoreWarnings(ignore: bool = True, v: bool = False) -> None`**: Suppresses or enables Python warnings.
539
+
540
+ **Text & String Processing:**
541
+ * **`formatStringBlock(input_str: str, max_chars: int = 70) -> str`**: Formats a string into a block of text with maximum characters per line.
542
+ * **`formatTimedelta(delta: timedelta) -> str`**: Formats a timedelta duration to readable format.
543
+
544
+ **Mathematical Utilities:**
545
+ * **`isBetween(number: float, a: float, b: float) -> bool`**: Returns True if number is between a and b.
546
+
547
+ **Notifications:**
548
+ * **`alert(message: str, server: str = "http://ntfy.sh", topic: str = "pythonAlerts", attachment: Optional[str] = None, messageTitle: str = "info", priority: int = None, tags: list = [], printIt: bool = True, v: bool = False) -> bool`**: Sends notifications to external servers.
549
+
550
+ ## Contributing
551
+
552
+ Contributions are welcome! Please fork the repository, make your changes, and submit a pull request. Ensure code is well-documented and includes tests where applicable.
553
+
554
+ ### Development Setup
555
+ ```bash
556
+ git clone https://github.com/celray/ccfx.git
557
+ cd ccfx
558
+ pip install -e .
559
+ ```
560
+
561
+ ### Version Management
562
+ To automatically update the version in README.md based on `pyproject.toml`:
563
+ ```bash
564
+ # Make the script executable (if not already)
565
+ chmod +x updateReadmeVersion.py
566
+
567
+ # Run the version update script
568
+ ./updateReadmeVersion.py
569
+ ```
570
+
571
+ You can also integrate this into your build process by adding it to your CI/CD pipeline or as a pre-commit hook.
572
+
573
+ ### Testing
574
+ Please ensure your changes don't break existing functionality and add tests for new features.
575
+
576
+ ## Changelog
577
+
578
+ ### Version 1.1.2 (Current)
579
+ - Enhanced geospatial processing capabilities
580
+ - Improved database connectivity options
581
+ - Added comprehensive multimedia support
582
+ - Better error handling and documentation
583
+
584
+ For detailed release notes, visit the [Releases page](https://github.com/celray/ccfx/releases).
585
+
586
+ ## License
587
+
588
+ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
589
+
590
+ ---
591
+
592
+ **Author**: Celray James CHAWANDA
593
+ **Email**: celray@chawanda.com
594
+ **GitHub**: [@celray](https://github.com/celray)
595
+ **Package Homepage**: [https://github.com/celray/ccfx](https://github.com/celray/ccfx)