cleansheet 1.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 aimenachraff-jpg
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,173 @@
1
+ Metadata-Version: 2.4
2
+ Name: cleansheet
3
+ Version: 1.1.0
4
+ Summary: Clean messy Excel/CSV files offline: logs every change and flags ambiguous values instead of guessing. Arabic, French and English headers, with locale packs for Algeria, Morocco, Tunisia, Egypt and France.
5
+ Author: aimenachraff-jpg
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/aimenachraff-jpg/CleanSheet
8
+ Project-URL: Repository, https://github.com/aimenachraff-jpg/CleanSheet
9
+ Project-URL: Issues, https://github.com/aimenachraff-jpg/CleanSheet/issues
10
+ Keywords: excel,csv,data-cleaning,arabic,openpyxl,algeria,morocco,tunisia,offline
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Environment :: Console
13
+ Classifier: Environment :: Win32 (MS Windows)
14
+ Classifier: Intended Audience :: End Users/Desktop
15
+ Classifier: License :: OSI Approved :: MIT License
16
+ Classifier: Natural Language :: Arabic
17
+ Classifier: Natural Language :: French
18
+ Classifier: Operating System :: OS Independent
19
+ Classifier: Programming Language :: Python :: 3
20
+ Classifier: Topic :: Office/Business :: Financial :: Spreadsheet
21
+ Requires-Python: >=3.8
22
+ Description-Content-Type: text/markdown
23
+ License-File: LICENSE
24
+ Requires-Dist: openpyxl
25
+ Dynamic: license-file
26
+
27
+ # CleanSheet
28
+
29
+ [![tests](https://github.com/aimenachraff-jpg/CleanSheet/actions/workflows/tests.yml/badge.svg)](https://github.com/aimenachraff-jpg/CleanSheet/actions/workflows/tests.yml)
30
+
31
+ Cleans messy Excel and CSV files: extra spaces, inconsistent capitalization, exact duplicates, mixed price and date formats, numbers written as words, Arabic-Indic digits, country phone formats. You get the cleaned file, a log of every change, and a list of the values the tool could not decide on.
32
+
33
+ It is rule-based code, not AI. It never guesses: anything ambiguous is flagged in the output for you to decide. Your original file is never modified, and nothing leaves your computer.
34
+
35
+ Built with real sales data in mind: columns named in French, English or Arabic, files where the table does not start at A1, two-level headers, "Total" rows at the bottom, prices like `1 450 DA`, `1 250 DH` or `دج`, dates like `جانفي 2026` or `غشت 2026`.
36
+
37
+ ![The CleanSheet window: pick a file, check the guessed column types, click Clean](app.png)
38
+
39
+ <!-- Demo GIF (10-15s, messy file -> Clean -> yellow/orange review cells) goes here. -->
40
+
41
+ ## Quick start
42
+
43
+ ```
44
+ # Windows, no Python needed: download the exe from Releases, extract, run CleanSheet.exe
45
+ # Python, any OS:
46
+ pip install cleansheet
47
+ cleansheet file.xlsx # cleans file.xlsx, writes file_cleaned.xlsx
48
+ cleansheet file.xlsx --locale ma
49
+ cleansheet-gui # windowed app
50
+ ```
51
+
52
+ More in [README.ar.md](README.ar.md) (العربية) and [README.fr.md](README.fr.md) (français).
53
+
54
+ ## How it works
55
+
56
+ 1. Open the app and pick a `.xlsx` or `.csv` file.
57
+ 2. The tool finds the table (even below title rows or notes), guesses each column's type from its header, and shows you the guess to confirm or fix.
58
+ 3. Click Clean. You get a new workbook in a `cleaned_output` folder next to your file, with five sheets:
59
+
60
+ | Sheet | What it holds |
61
+ |---|---|
62
+ | Clean | The cleaned data. Yellow = changed with an assumption, check it. Orange = unreadable or missing, original kept. |
63
+ | Review | Every yellow and orange cell with a note. Go through it before you use the file. |
64
+ | Change log | Every change, before and after. |
65
+ | Before-After | First 20 rows side by side. |
66
+ | Summary | Counts, plus the rules that were applied. |
67
+
68
+ ![The Clean sheet: yellow = changed with an assumption, orange = unreadable or missing, original kept](cleaned.png)
69
+
70
+ Rows that look like totals ("Total", "Somme", "المجموع") are not cleaned as data; they are copied to a Skipped rows sheet. You can keep them in with one checkbox.
71
+
72
+ ## Why not just use pandas, OpenRefine or Excel?
73
+
74
+ - **It runs offline and needs no skills.** A shop manager or accountant double-clicks the exe. pandas needs Python, a terminal and a script per file layout.
75
+ - **It never guesses.** pandas and Excel fix-converts silently: `02/09/2026` becomes September 2 without telling you. CleanSheet marks every assumption in yellow and leaves the call to you.
76
+ - **It knows the formats the big tools ignore.** `1 450 DA`, `1.250,50 درهم`, جانفي and غشت month names, `٠١٢٣` digits, 8-digit Tunisian phones. Out of the box, with a change log.
77
+
78
+ ## What it fixes and what it does not
79
+
80
+ Fixed automatically (each one logged):
81
+
82
+ - Extra spaces, upper/lower case of the same name or city
83
+ - Number words like "dix" or "ثلاثة", price formats like `1 450 DA` or `1 250 DH`
84
+ - Clear dates like `3 septembre 2026` or `2026-09-03`; Arabic month names (Maghrebi, standard, Levantine) are understood
85
+ - Phone formats (+213, +212, +216, +20, +33), Arabic-Indic (٠١٢٣) and Persian (۰۱۲۳) digits
86
+ - Exact duplicate rows
87
+ - Typos in headers ("mongtant" → the montant column), two-level headers joined as `Produit1 - Qté`
88
+
89
+ Flagged for you, never decided for you:
90
+
91
+ - Ambiguous dates like `02/09/2026` (day-first or month-first; you pick the default, the tool marks them)
92
+ - Ambiguous numbers like `1,450` (1.45 or 1450)
93
+ - Unreadable or missing values: kept as they were, never filled in
94
+ - Names that differ only by hamza or diacritics (أحمد / احمد): flagged for review, never merged
95
+ - Synonyms across languages ("Thon" and "Tuna"): not merged, not flagged. Real synonyms need a human.
96
+
97
+ ## Locale packs
98
+
99
+ Phone rules, currency words and month names come from a country pack, chosen with `--locale`:
100
+
101
+ | Locale | Country | Phones | Prices read |
102
+ |---|---|---|---|
103
+ | `dz` (default) | Algeria | +213, 9 digits | `1 450 DA`, `دج`, `دينار` |
104
+ | `ma` | Morocco | +212, 9 digits | `1 250 DH`, `درهم`, `MAD` |
105
+ | `tn` | Tunisia | +216, 8 digits, no leading 0 | `12,5 DT`, `د.ت` |
106
+ | `eg` | Egypt | +20, 10 digits | `1,250 LE`, `جنيه` |
107
+ | `fr` | France | +33, 9 digits | `1 250 €` |
108
+ | `generic` | anywhere | length check only | common currency symbols |
109
+
110
+ The default is `dz` (where the project started). See [CONTRIBUTING.md](CONTRIBUTING.md) to add a country; the pack file is a few lines. Sample files: `sample_messy_arabic.xlsx` (Arabic headers, Algerian formats), `sample_messy_moroccan.xlsx` (French headers, Moroccan formats), `sample_messy_sales.xlsx` (English).
111
+
112
+ ## Getting it running
113
+
114
+ The easiest way, no Python needed: download [CleanSheet-1.0.0-windows.zip](https://github.com/aimenachraff-jpg/CleanSheet/releases/latest) from the Releases page, extract it, and run `CleanSheet.exe` from the extracted `CleanSheet` folder (Windows 10 or 11, 64-bit). Windows will show a SmartScreen warning because the exe is not signed; click More info, then Run anyway.
115
+
116
+ From Python:
117
+
118
+ ```
119
+ pip install cleansheet
120
+ ```
121
+
122
+ Command line:
123
+
124
+ ```
125
+ cleansheet client.xlsx --out-dir out/
126
+ # or from source: python clean_sheet.py client.xlsx
127
+ ```
128
+
129
+ Windowed app (pick the file, confirm the column types, click Clean):
130
+
131
+ ```
132
+ cleansheet-gui
133
+ # or from source: python clean_sheet_gui.py
134
+ ```
135
+
136
+ On Windows you can just double-click `start_CleanSheet.bat`.
137
+
138
+ ### Building a standalone .exe
139
+
140
+ Double-click `build_exe.bat` (it downloads PyInstaller once). The result is `dist\CleanSheet\CleanSheet.exe`; keep the whole `CleanSheet` folder together and make a shortcut to the exe. Windows will show a SmartScreen warning for an unsigned exe you built yourself; that is expected.
141
+
142
+ ### Column types
143
+
144
+ `clean_sheet.py` guesses column types from the header and, when the name gives no clue, from the values (phone, date, email, price with DA/دج/DH, plain numbers). To override the guess:
145
+
146
+ ```
147
+ python clean_sheet.py client.xlsx --write-config client.json
148
+ ```
149
+
150
+ Edit `client.json` (types: `id`, `name`, `category`, `qty`, `price`, `date`, `phone`, `email`, `text`, `skip`), then rerun with `--config client.json`. Two example configs are in the repo: `config_sales_demo.json` and `config_client_style.json` (French headers). Other options: `--sheet NAME`, `--header-row N`, `--levels 1`, `--keep-total-rows`, `--also-csv`. A `--config` file always wins over the locale pack.
151
+
152
+ ## Limitations
153
+
154
+ - Reads the first table on the chosen sheet only.
155
+ - No scans or PDFs, no formulas or dashboards, no merging files with different layouts.
156
+ - It does not invent values: what it cannot read stays unreadable, in orange.
157
+ - Tested on synthetic files. If it mishandles your file, please open an issue with a sample.
158
+
159
+ ## Tests
160
+
161
+ ```
162
+ python test_clean_sheet.py
163
+ python test_table_detection.py
164
+ python test_multilevel.py
165
+ python test_gui_logic.py
166
+ python test_locale_packs.py
167
+ ```
168
+
169
+ 57 tests cover the cleaning rules, table and header detection, the GUI logic, and the locale packs.
170
+
171
+ ## License
172
+
173
+ MIT. See [LICENSE](LICENSE).
@@ -0,0 +1,147 @@
1
+ # CleanSheet
2
+
3
+ [![tests](https://github.com/aimenachraff-jpg/CleanSheet/actions/workflows/tests.yml/badge.svg)](https://github.com/aimenachraff-jpg/CleanSheet/actions/workflows/tests.yml)
4
+
5
+ Cleans messy Excel and CSV files: extra spaces, inconsistent capitalization, exact duplicates, mixed price and date formats, numbers written as words, Arabic-Indic digits, country phone formats. You get the cleaned file, a log of every change, and a list of the values the tool could not decide on.
6
+
7
+ It is rule-based code, not AI. It never guesses: anything ambiguous is flagged in the output for you to decide. Your original file is never modified, and nothing leaves your computer.
8
+
9
+ Built with real sales data in mind: columns named in French, English or Arabic, files where the table does not start at A1, two-level headers, "Total" rows at the bottom, prices like `1 450 DA`, `1 250 DH` or `دج`, dates like `جانفي 2026` or `غشت 2026`.
10
+
11
+ ![The CleanSheet window: pick a file, check the guessed column types, click Clean](app.png)
12
+
13
+ <!-- Demo GIF (10-15s, messy file -> Clean -> yellow/orange review cells) goes here. -->
14
+
15
+ ## Quick start
16
+
17
+ ```
18
+ # Windows, no Python needed: download the exe from Releases, extract, run CleanSheet.exe
19
+ # Python, any OS:
20
+ pip install cleansheet
21
+ cleansheet file.xlsx # cleans file.xlsx, writes file_cleaned.xlsx
22
+ cleansheet file.xlsx --locale ma
23
+ cleansheet-gui # windowed app
24
+ ```
25
+
26
+ More in [README.ar.md](README.ar.md) (العربية) and [README.fr.md](README.fr.md) (français).
27
+
28
+ ## How it works
29
+
30
+ 1. Open the app and pick a `.xlsx` or `.csv` file.
31
+ 2. The tool finds the table (even below title rows or notes), guesses each column's type from its header, and shows you the guess to confirm or fix.
32
+ 3. Click Clean. You get a new workbook in a `cleaned_output` folder next to your file, with five sheets:
33
+
34
+ | Sheet | What it holds |
35
+ |---|---|
36
+ | Clean | The cleaned data. Yellow = changed with an assumption, check it. Orange = unreadable or missing, original kept. |
37
+ | Review | Every yellow and orange cell with a note. Go through it before you use the file. |
38
+ | Change log | Every change, before and after. |
39
+ | Before-After | First 20 rows side by side. |
40
+ | Summary | Counts, plus the rules that were applied. |
41
+
42
+ ![The Clean sheet: yellow = changed with an assumption, orange = unreadable or missing, original kept](cleaned.png)
43
+
44
+ Rows that look like totals ("Total", "Somme", "المجموع") are not cleaned as data; they are copied to a Skipped rows sheet. You can keep them in with one checkbox.
45
+
46
+ ## Why not just use pandas, OpenRefine or Excel?
47
+
48
+ - **It runs offline and needs no skills.** A shop manager or accountant double-clicks the exe. pandas needs Python, a terminal and a script per file layout.
49
+ - **It never guesses.** pandas and Excel fix-converts silently: `02/09/2026` becomes September 2 without telling you. CleanSheet marks every assumption in yellow and leaves the call to you.
50
+ - **It knows the formats the big tools ignore.** `1 450 DA`, `1.250,50 درهم`, جانفي and غشت month names, `٠١٢٣` digits, 8-digit Tunisian phones. Out of the box, with a change log.
51
+
52
+ ## What it fixes and what it does not
53
+
54
+ Fixed automatically (each one logged):
55
+
56
+ - Extra spaces, upper/lower case of the same name or city
57
+ - Number words like "dix" or "ثلاثة", price formats like `1 450 DA` or `1 250 DH`
58
+ - Clear dates like `3 septembre 2026` or `2026-09-03`; Arabic month names (Maghrebi, standard, Levantine) are understood
59
+ - Phone formats (+213, +212, +216, +20, +33), Arabic-Indic (٠١٢٣) and Persian (۰۱۲۳) digits
60
+ - Exact duplicate rows
61
+ - Typos in headers ("mongtant" → the montant column), two-level headers joined as `Produit1 - Qté`
62
+
63
+ Flagged for you, never decided for you:
64
+
65
+ - Ambiguous dates like `02/09/2026` (day-first or month-first; you pick the default, the tool marks them)
66
+ - Ambiguous numbers like `1,450` (1.45 or 1450)
67
+ - Unreadable or missing values: kept as they were, never filled in
68
+ - Names that differ only by hamza or diacritics (أحمد / احمد): flagged for review, never merged
69
+ - Synonyms across languages ("Thon" and "Tuna"): not merged, not flagged. Real synonyms need a human.
70
+
71
+ ## Locale packs
72
+
73
+ Phone rules, currency words and month names come from a country pack, chosen with `--locale`:
74
+
75
+ | Locale | Country | Phones | Prices read |
76
+ |---|---|---|---|
77
+ | `dz` (default) | Algeria | +213, 9 digits | `1 450 DA`, `دج`, `دينار` |
78
+ | `ma` | Morocco | +212, 9 digits | `1 250 DH`, `درهم`, `MAD` |
79
+ | `tn` | Tunisia | +216, 8 digits, no leading 0 | `12,5 DT`, `د.ت` |
80
+ | `eg` | Egypt | +20, 10 digits | `1,250 LE`, `جنيه` |
81
+ | `fr` | France | +33, 9 digits | `1 250 €` |
82
+ | `generic` | anywhere | length check only | common currency symbols |
83
+
84
+ The default is `dz` (where the project started). See [CONTRIBUTING.md](CONTRIBUTING.md) to add a country; the pack file is a few lines. Sample files: `sample_messy_arabic.xlsx` (Arabic headers, Algerian formats), `sample_messy_moroccan.xlsx` (French headers, Moroccan formats), `sample_messy_sales.xlsx` (English).
85
+
86
+ ## Getting it running
87
+
88
+ The easiest way, no Python needed: download [CleanSheet-1.0.0-windows.zip](https://github.com/aimenachraff-jpg/CleanSheet/releases/latest) from the Releases page, extract it, and run `CleanSheet.exe` from the extracted `CleanSheet` folder (Windows 10 or 11, 64-bit). Windows will show a SmartScreen warning because the exe is not signed; click More info, then Run anyway.
89
+
90
+ From Python:
91
+
92
+ ```
93
+ pip install cleansheet
94
+ ```
95
+
96
+ Command line:
97
+
98
+ ```
99
+ cleansheet client.xlsx --out-dir out/
100
+ # or from source: python clean_sheet.py client.xlsx
101
+ ```
102
+
103
+ Windowed app (pick the file, confirm the column types, click Clean):
104
+
105
+ ```
106
+ cleansheet-gui
107
+ # or from source: python clean_sheet_gui.py
108
+ ```
109
+
110
+ On Windows you can just double-click `start_CleanSheet.bat`.
111
+
112
+ ### Building a standalone .exe
113
+
114
+ Double-click `build_exe.bat` (it downloads PyInstaller once). The result is `dist\CleanSheet\CleanSheet.exe`; keep the whole `CleanSheet` folder together and make a shortcut to the exe. Windows will show a SmartScreen warning for an unsigned exe you built yourself; that is expected.
115
+
116
+ ### Column types
117
+
118
+ `clean_sheet.py` guesses column types from the header and, when the name gives no clue, from the values (phone, date, email, price with DA/دج/DH, plain numbers). To override the guess:
119
+
120
+ ```
121
+ python clean_sheet.py client.xlsx --write-config client.json
122
+ ```
123
+
124
+ Edit `client.json` (types: `id`, `name`, `category`, `qty`, `price`, `date`, `phone`, `email`, `text`, `skip`), then rerun with `--config client.json`. Two example configs are in the repo: `config_sales_demo.json` and `config_client_style.json` (French headers). Other options: `--sheet NAME`, `--header-row N`, `--levels 1`, `--keep-total-rows`, `--also-csv`. A `--config` file always wins over the locale pack.
125
+
126
+ ## Limitations
127
+
128
+ - Reads the first table on the chosen sheet only.
129
+ - No scans or PDFs, no formulas or dashboards, no merging files with different layouts.
130
+ - It does not invent values: what it cannot read stays unreadable, in orange.
131
+ - Tested on synthetic files. If it mishandles your file, please open an issue with a sample.
132
+
133
+ ## Tests
134
+
135
+ ```
136
+ python test_clean_sheet.py
137
+ python test_table_detection.py
138
+ python test_multilevel.py
139
+ python test_gui_logic.py
140
+ python test_locale_packs.py
141
+ ```
142
+
143
+ 57 tests cover the cleaning rules, table and header detection, the GUI logic, and the locale packs.
144
+
145
+ ## License
146
+
147
+ MIT. See [LICENSE](LICENSE).