excel-list-transform 1.1__tar.gz → 1.3__tar.gz

{excel_list_transform-1.1 → excel_list_transform-1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: excel-list-transform
-Version: 1.1
+Version: 1.3
 Summary: Transform table data with configurable row and column changes.
 Author: Tom Björkholm
 Author-email: Tom Björkholm <klausuler_linnet0q@icloud.com>
@@ -9,6 +9,7 @@ Project-URL: Source code, https://github.com/tom-bjorkholm/excel-list-transform
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
 Classifier: Operating System :: OS Independent
 Classifier: Intended Audience :: Other Audience
 Classifier: Topic :: Office/Business
@@ -16,13 +17,18 @@ Classifier: Typing :: Typed
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
-Requires-Dist: tableio-cfg-json>=0.8
-Requires-Dist: tableio>=1.1
-Requires-Dist: config-as-json>=1.4
-Requires-Dist: versionreporter>=0.4
-Requires-Dist: argcomplete>=3.7.0
+Requires-Dist: tableio-cfg-json>=2.0
+Requires-Dist: tableio>=1.3
+Requires-Dist: config-as-json>=1.8
+Requires-Dist: versionreporter>=0.5
+Requires-Dist: argcomplete>=3.7.2
+Requires-Dist: edit-cfg-json>=1.0.0
+Requires-Dist: edit-cfg-json-tk>=1.0.0
+Provides-Extra: textual
+Requires-Dist: edit-cfg-json-textual>=1.0.0; extra == "textual"
 Dynamic: author
 Dynamic: license-file
+Dynamic: provides-extra
 Dynamic: requires-dist
 Dynamic: requires-python
 
@@ -63,7 +69,8 @@ This small python application:
 
 How this is done is governed by a configuration file. The application can
 create a number of example configuration files with accompanying description
-text files.
+text files, and can open a configuration file in a configuration editor that
+explains every setting.
 
 ## Installing it
 
@@ -84,6 +91,19 @@ pip3 install --upgrade excel-list-transform
 pip install --upgrade excel-list-transform
 ````
 
+### Installing the terminal configuration editor (optional)
+
+The "editcfg" sub-command opens the configuration editor in a window when
+there is a display. To also be able to edit in the terminal (for instance
+over a remote shell without a display), install the optional extra
+"textual":
+
+````sh
+pip3 install --upgrade "excel-list-transform[textual]"
+````
+
+(On Microsoft Windows use `pip` instead of `pip3`.)
+
 ## Running the application
 
 ### Running the application on mac and Linux
@@ -97,6 +117,8 @@ python3 -m excel_list_transform cfg-example -k forms_to_rrs -r by_name -o exampl
 python3 -m excel_list_transform transform -c example.cfg -i input.xlsx -o output.xlsx
 python3 -m excel_list_transform migrate-cfg --help
 python3 -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
+python3 -m excel_list_transform editcfg --help
+python3 -m excel_list_transform editcfg -i example.cfg
 ````
 
 ### Running the application on Microsoft Windows
@@ -110,6 +132,8 @@ python -m excel_list_transform cfg-example -k forms_to_rrs -r by_name -o example
 python -m excel_list_transform transform -c example.cfg -i input.xlsx -o output.xlsx
 python -m excel_list_transform migrate-cfg --help
 python -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
+python -m excel_list_transform editcfg --help
+python -m excel_list_transform editcfg -i example.cfg
 ````
 
 ## Suggested way to get started
@@ -120,6 +144,8 @@ python -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
     description (.txt) files.
  3. Find an example that is close to what you want to achieve.
  4. Modify that configuration file to achieve what you want to achieve.
+    Use the "editcfg" sub-command to modify it in a configuration editor that
+    explains every setting (or modify it in a text editor).
  5. Use the "transform" sub-command to read your data and output it transformed
     or reorganized according to your modified configuration file.
  6. Read the produced output. If necessary go back to step 4 and adjust how the
@@ -139,6 +165,23 @@ text file describing it.
 Read the text file describing the configuration file while looking at the
 configuration file to understand the syntax and the possible options.
 
+### Editing a configuration file
+
+The "editcfg" sub-command opens a configuration file in a configuration
+editor. The editor shows an explanation of every setting, checks the values
+before saving, and keeps a backup (.bak) of the file it overwrites. It opens
+in a window when there is a display, otherwise in the terminal (if the
+optional extra "textual" is installed, see above). Use the "--ui" option to
+choose the editor yourself, and the "-o" option to save to another file than
+the one you opened.
+
+"column_ref" cannot be changed in the editor, as the rest of the file is
+written for it. Generate a new example with the other "-r" value instead.
+
+There is a separate user guide for the editor opened by the "editcfg"
+sub-command. See
+[https://github.com/tom-bjorkholm/edit-cfg-json/blob/master/doc/end_users_guide.md](https://github.com/tom-bjorkholm/edit-cfg-json/blob/master/doc/end_users_guide.md)
+
 ## Performance
 
 This application is written for the moderate amounts of data when
@@ -270,7 +313,8 @@ referenced by number.
 The first operation is to split rows based on column values.
 This operation is configured using the **"s01_split_rows"** record,
 that has an array of splits to be done. Each row split has the
-following keys: **"column"**, **"separators"** and **"not_separators"**.
+following keys: **"column"**, **"separators"**, **"not_separators"**
+and **"empty_parts"**.
 
 The **"column"** keyword is used to identify the column that is
 split into several rows. This is a column number in the case of
@@ -286,11 +330,36 @@ these strings are present in the identified column it is seen
 as the separator between the parts of the column value that go
 into different rows.
 
+Separators must not be empty strings. If more than one
+separator matches at the same position, the longest one is used.
+
 **"not_separators"** take as argument a list of strings. These strings
 are not regarded as separators even if they include the strings
 of one or more separator. (For instance ";" could be a separator,
 but using **"not_separators"** the string "\\;" could be seen as not
 a separator.)
+The column value is scanned from the start. At each position a
+not-separator takes precedence over a separator, so a separator
+that is part of a not-separator never splits the value.
+
+**"empty_parts"** decides what happens to empty parts. A part is the
+text between two separators, or between a separator and the start
+or end of the value. A part is empty if it is "" or only whitespace.
+Empty parts come from a separator at the start or end of the value,
+or from two adjacent separators. **"empty_parts"** can have the values:
+
+- *"KEEP_ALL"*: keep all empty parts. They give rows where the column
+  is empty. "a,,b," split at "," gives "a", "", "b" and "".
+- *"DROP_ALL"*: drop all empty parts. "a,,b," gives "a" and "b".
+- *"DROP_TRAILING"*: drop only the empty parts at the end. "a,,b,"
+  gives "a", "" and "b", and "a,b,," gives "a" and "b". This is
+  useful when a separator is written after every element of a list,
+  also after the last one.
+
+If all parts are dropped (for instance the value "," with *"DROP_ALL"*),
+the row is kept once with the value "" in the column. An empty cell
+is not split. Configuration files without **"empty_parts"** (from older
+versions) get *"KEEP_ALL"*, which is how older versions worked.
 
 ### "s02_merge_rows"
 
@@ -309,6 +378,7 @@ For each column that has the same value for all rows merged, that
 value will be in the merged row. When rows being merged have different
 values for a column, the set of unique values from different rows
 will form the value for that column in the merged row.
+Empty values are left out.
 
 The **"separator"** keyword is used to specify the string that is
 concatenated between values for one column from different rows
@@ -346,10 +416,10 @@ the least confusion is to split columns to the right before
 columns to the left. Named references also helps to avoid
 confusion.)
 
-Each split has the following keys: **"column"**, **"separator"** and
-**"where"**. In the case of number column references there is
-also the key **"store_single"**. In the case of named column references
-there is instead the key **"right_name"**.
+Each split has the following keys: **"column"**, **"separator"**,
+**"where"** and **"end_separators"**. In the case of number column
+references there is also the key **"store_single"**. In the case of
+named column references there is instead the key **"right_name"**.
 
 **"column"** takes as argument the column to be split. This is a
 column number in the case of *"BY_NUMBER"*, and a column name/title
@@ -358,6 +428,7 @@ in the case of *"BY_NAME"*.
 **"separator"** is a string of characters that indicate the position
 where the column shall be split. (For instance a single space
 if splitting between two words.)
+The separator must not be an empty string.
 
 **"where"** can either have the value *"RIGHTMOST"* or the value
 *"LEFTMOST"*. This is used if the separator string is present
@@ -365,6 +436,25 @@ more than once in the column value to split. The split is then
 done at the leftmost or rightmost occurrence according to the
 value of **"where"**.
 
+**"end_separators"** decides what happens to separators at the start
+or end of the value, before the split is done. Without removing them,
+"a,b," split at the rightmost "," gives "a,b" and "", and split at
+the leftmost "," gives "a" and "b,". **"end_separators"** can have the
+values:
+
+- *"KEEP"*: keep the value as it is.
+- *"STRIP_LEADING"*: remove separators at the start of the value.
+- *"STRIP_TRAILING"*: remove separators at the end of the value.
+- *"STRIP_BOTH_ENDS"*: remove separators at both the start and the end
+  of the value.
+
+All consecutive separators at a stripped end are removed, together
+with any whitespace between them. For instance "a,b, ," with
+*"STRIP_TRAILING"* becomes "a,b" before it is split. A value that
+consists only of separators and whitespace becomes "".
+Configuration files without **"end_separators"** (from older versions)
+get *"KEEP"*, which is how older versions worked.
+
 **"right_name"** is only used if column references are *"BY_NAME"*.
 After the split the left column uses the original column name,
 and **"right_name"** is used as the name of the new right column.
@@ -395,6 +485,7 @@ column references are column numbers.
 
 **"separator"** is a string of characters that is inserted between
 the column values being merged.
+Empty values are left out.
 
 ### "s06_place_columns_first"
 
@@ -414,6 +505,8 @@ column number in the case of *"BY_NUMBER"*, and a column name/title
 in the case of *"BY_NAME"*.
 
 **"name"** is the new name/title of the column identified by **"column"**.
+In the case of *"BY_NAME"* the renames are done in the listed order,
+and a rename must not give a column the name of an existing column.
 
 ### "s08_insert_columns"
 
@@ -486,6 +579,8 @@ The value of the **"s10_column_order"** key is a list of column names.
 The columns will be output in this order.
 Columns not mentioned in **"s10_column_order"** will not be output,
 and will thus be removed.
+Each name in **"s10_column_order"** must be the name of an existing
+column (as named after all other steps).
 (For *"BY_NUMBER"* see **"s04_remove_columns"** and **"s06_place_columns_first"**.)
 
 ## Utility for recoding text files
@@ -510,9 +605,10 @@ Source code and tests are available at [https://github.com/tom-bjorkholm/excel-l
 
 ## Test summary
 
-- Test result: 1368 passed, 1 warning in 64s (0:01:04)
+- Test result: 1693 passed in 88s (0:01:28)
 - No flake8 warnings.
 - No mypy errors found.
+- No pylint warnings.
 - No python layout warnings.
-- Built version(s): 1.1
-- Build and test using Python 3.14.6
+- Built version(s): 1.3
+- Build and test using Python 3.15.2

{excel_list_transform-1.1 → excel_list_transform-1.3}/README.md

@@ -45,7 +45,10 @@ This small python application:
   an ODS spreadsheet file, a comma separated values (CSV) file, or
   any of a number of other file formats.
 
-How this is done is governed by a configuration file. The application can create a number of example configuration files with accompanying description text files.
+How this is done is governed by a configuration file. The application can
+create a number of example configuration files with accompanying description
+text files, and can open a configuration file in a configuration editor
+that explains every setting.
 
 ## Cloning
 
@@ -107,9 +110,10 @@ the built and installed application in the virtual environment `./venv`.
 
 ## Test summary
 
-- Test result: 1368 passed, 1 warning in 64s (0:01:04)
+- Test result: 1693 passed in 88s (0:01:28)
 - No flake8 warnings.
 - No mypy errors found.
+- No pylint warnings.
 - No python layout warnings.
-- Built version(s): 1.1
-- Build and test using Python 3.14.6
+- Built version(s): 1.3
+- Build and test using Python 3.15.2

excel_list_transform-1.1/excel_list_transform.egg-info/PKG-INFO → excel_list_transform-1.3/README_pypi.md

@@ -1,31 +1,3 @@
-Metadata-Version: 2.4
-Name: excel-list-transform
-Version: 1.1
-Summary: Transform table data with configurable row and column changes.
-Author: Tom Björkholm
-Author-email: Tom Björkholm <klausuler_linnet0q@icloud.com>
-License-Expression: MIT
-Project-URL: Source code, https://github.com/tom-bjorkholm/excel-list-transform
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Programming Language :: Python :: 3.14
-Classifier: Operating System :: OS Independent
-Classifier: Intended Audience :: Other Audience
-Classifier: Topic :: Office/Business
-Classifier: Typing :: Typed
-Requires-Python: >=3.13
-Description-Content-Type: text/markdown
-License-File: LICENSE.txt
-Requires-Dist: tableio-cfg-json>=0.8
-Requires-Dist: tableio>=1.1
-Requires-Dist: config-as-json>=1.4
-Requires-Dist: versionreporter>=0.4
-Requires-Dist: argcomplete>=3.7.0
-Dynamic: author
-Dynamic: license-file
-Dynamic: requires-dist
-Dynamic: requires-python
-
 # excel-list-transform
 
 ## Background
@@ -63,7 +35,8 @@ This small python application:
 
 How this is done is governed by a configuration file. The application can
 create a number of example configuration files with accompanying description
-text files.
+text files, and can open a configuration file in a configuration editor that
+explains every setting.
 
 ## Installing it
 
@@ -84,6 +57,19 @@ pip3 install --upgrade excel-list-transform
 pip install --upgrade excel-list-transform
 ````
 
+### Installing the terminal configuration editor (optional)
+
+The "editcfg" sub-command opens the configuration editor in a window when
+there is a display. To also be able to edit in the terminal (for instance
+over a remote shell without a display), install the optional extra
+"textual":
+
+````sh
+pip3 install --upgrade "excel-list-transform[textual]"
+````
+
+(On Microsoft Windows use `pip` instead of `pip3`.)
+
 ## Running the application
 
 ### Running the application on mac and Linux
@@ -97,6 +83,8 @@ python3 -m excel_list_transform cfg-example -k forms_to_rrs -r by_name -o exampl
 python3 -m excel_list_transform transform -c example.cfg -i input.xlsx -o output.xlsx
 python3 -m excel_list_transform migrate-cfg --help
 python3 -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
+python3 -m excel_list_transform editcfg --help
+python3 -m excel_list_transform editcfg -i example.cfg
 ````
 
 ### Running the application on Microsoft Windows
@@ -110,6 +98,8 @@ python -m excel_list_transform cfg-example -k forms_to_rrs -r by_name -o example
 python -m excel_list_transform transform -c example.cfg -i input.xlsx -o output.xlsx
 python -m excel_list_transform migrate-cfg --help
 python -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
+python -m excel_list_transform editcfg --help
+python -m excel_list_transform editcfg -i example.cfg
 ````
 
 ## Suggested way to get started
@@ -120,6 +110,8 @@ python -m excel_list_transform migrate-cfg -i old.cfg -o new.cfg
     description (.txt) files.
  3. Find an example that is close to what you want to achieve.
  4. Modify that configuration file to achieve what you want to achieve.
+    Use the "editcfg" sub-command to modify it in a configuration editor that
+    explains every setting (or modify it in a text editor).
  5. Use the "transform" sub-command to read your data and output it transformed
     or reorganized according to your modified configuration file.
  6. Read the produced output. If necessary go back to step 4 and adjust how the
@@ -139,6 +131,23 @@ text file describing it.
 Read the text file describing the configuration file while looking at the
 configuration file to understand the syntax and the possible options.
 
+### Editing a configuration file
+
+The "editcfg" sub-command opens a configuration file in a configuration
+editor. The editor shows an explanation of every setting, checks the values
+before saving, and keeps a backup (.bak) of the file it overwrites. It opens
+in a window when there is a display, otherwise in the terminal (if the
+optional extra "textual" is installed, see above). Use the "--ui" option to
+choose the editor yourself, and the "-o" option to save to another file than
+the one you opened.
+
+"column_ref" cannot be changed in the editor, as the rest of the file is
+written for it. Generate a new example with the other "-r" value instead.
+
+There is a separate user guide for the editor opened by the "editcfg"
+sub-command. See
+[https://github.com/tom-bjorkholm/edit-cfg-json/blob/master/doc/end_users_guide.md](https://github.com/tom-bjorkholm/edit-cfg-json/blob/master/doc/end_users_guide.md)
+
 ## Performance
 
 This application is written for the moderate amounts of data when
@@ -270,7 +279,8 @@ referenced by number.
 The first operation is to split rows based on column values.
 This operation is configured using the **"s01_split_rows"** record,
 that has an array of splits to be done. Each row split has the
-following keys: **"column"**, **"separators"** and **"not_separators"**.
+following keys: **"column"**, **"separators"**, **"not_separators"**
+and **"empty_parts"**.
 
 The **"column"** keyword is used to identify the column that is
 split into several rows. This is a column number in the case of
@@ -286,11 +296,36 @@ these strings are present in the identified column it is seen
 as the separator between the parts of the column value that go
 into different rows.
 
+Separators must not be empty strings. If more than one
+separator matches at the same position, the longest one is used.
+
 **"not_separators"** take as argument a list of strings. These strings
 are not regarded as separators even if they include the strings
 of one or more separator. (For instance ";" could be a separator,
 but using **"not_separators"** the string "\\;" could be seen as not
 a separator.)
+The column value is scanned from the start. At each position a
+not-separator takes precedence over a separator, so a separator
+that is part of a not-separator never splits the value.
+
+**"empty_parts"** decides what happens to empty parts. A part is the
+text between two separators, or between a separator and the start
+or end of the value. A part is empty if it is "" or only whitespace.
+Empty parts come from a separator at the start or end of the value,
+or from two adjacent separators. **"empty_parts"** can have the values:
+
+- *"KEEP_ALL"*: keep all empty parts. They give rows where the column
+  is empty. "a,,b," split at "," gives "a", "", "b" and "".
+- *"DROP_ALL"*: drop all empty parts. "a,,b," gives "a" and "b".
+- *"DROP_TRAILING"*: drop only the empty parts at the end. "a,,b,"
+  gives "a", "" and "b", and "a,b,," gives "a" and "b". This is
+  useful when a separator is written after every element of a list,
+  also after the last one.
+
+If all parts are dropped (for instance the value "," with *"DROP_ALL"*),
+the row is kept once with the value "" in the column. An empty cell
+is not split. Configuration files without **"empty_parts"** (from older
+versions) get *"KEEP_ALL"*, which is how older versions worked.
 
 ### "s02_merge_rows"
 
@@ -309,6 +344,7 @@ For each column that has the same value for all rows merged, that
 value will be in the merged row. When rows being merged have different
 values for a column, the set of unique values from different rows
 will form the value for that column in the merged row.
+Empty values are left out.
 
 The **"separator"** keyword is used to specify the string that is
 concatenated between values for one column from different rows
@@ -346,10 +382,10 @@ the least confusion is to split columns to the right before
 columns to the left. Named references also helps to avoid
 confusion.)
 
-Each split has the following keys: **"column"**, **"separator"** and
-**"where"**. In the case of number column references there is
-also the key **"store_single"**. In the case of named column references
-there is instead the key **"right_name"**.
+Each split has the following keys: **"column"**, **"separator"**,
+**"where"** and **"end_separators"**. In the case of number column
+references there is also the key **"store_single"**. In the case of
+named column references there is instead the key **"right_name"**.
 
 **"column"** takes as argument the column to be split. This is a
 column number in the case of *"BY_NUMBER"*, and a column name/title
@@ -358,6 +394,7 @@ in the case of *"BY_NAME"*.
 **"separator"** is a string of characters that indicate the position
 where the column shall be split. (For instance a single space
 if splitting between two words.)
+The separator must not be an empty string.
 
 **"where"** can either have the value *"RIGHTMOST"* or the value
 *"LEFTMOST"*. This is used if the separator string is present
@@ -365,6 +402,25 @@ more than once in the column value to split. The split is then
 done at the leftmost or rightmost occurrence according to the
 value of **"where"**.
 
+**"end_separators"** decides what happens to separators at the start
+or end of the value, before the split is done. Without removing them,
+"a,b," split at the rightmost "," gives "a,b" and "", and split at
+the leftmost "," gives "a" and "b,". **"end_separators"** can have the
+values:
+
+- *"KEEP"*: keep the value as it is.
+- *"STRIP_LEADING"*: remove separators at the start of the value.
+- *"STRIP_TRAILING"*: remove separators at the end of the value.
+- *"STRIP_BOTH_ENDS"*: remove separators at both the start and the end
+  of the value.
+
+All consecutive separators at a stripped end are removed, together
+with any whitespace between them. For instance "a,b, ," with
+*"STRIP_TRAILING"* becomes "a,b" before it is split. A value that
+consists only of separators and whitespace becomes "".
+Configuration files without **"end_separators"** (from older versions)
+get *"KEEP"*, which is how older versions worked.
+
 **"right_name"** is only used if column references are *"BY_NAME"*.
 After the split the left column uses the original column name,
 and **"right_name"** is used as the name of the new right column.
@@ -395,6 +451,7 @@ column references are column numbers.
 
 **"separator"** is a string of characters that is inserted between
 the column values being merged.
+Empty values are left out.
 
 ### "s06_place_columns_first"
 
@@ -414,6 +471,8 @@ column number in the case of *"BY_NUMBER"*, and a column name/title
 in the case of *"BY_NAME"*.
 
 **"name"** is the new name/title of the column identified by **"column"**.
+In the case of *"BY_NAME"* the renames are done in the listed order,
+and a rename must not give a column the name of an existing column.
 
 ### "s08_insert_columns"
 
@@ -486,6 +545,8 @@ The value of the **"s10_column_order"** key is a list of column names.
 The columns will be output in this order.
 Columns not mentioned in **"s10_column_order"** will not be output,
 and will thus be removed.
+Each name in **"s10_column_order"** must be the name of an existing
+column (as named after all other steps).
 (For *"BY_NUMBER"* see **"s04_remove_columns"** and **"s06_place_columns_first"**.)
 
 ## Utility for recoding text files
@@ -510,9 +571,10 @@ Source code and tests are available at [https://github.com/tom-bjorkholm/excel-l
 
 ## Test summary
 
-- Test result: 1368 passed, 1 warning in 64s (0:01:04)
+- Test result: 1693 passed in 88s (0:01:28)
 - No flake8 warnings.
 - No mypy errors found.
+- No pylint warnings.
 - No python layout warnings.
-- Built version(s): 1.1
-- Build and test using Python 3.14.6
+- Built version(s): 1.3
+- Build and test using Python 3.15.2